Comments (12)
@jinfagang It depends on hardware platform, normally 10%-20% latency increment.
from acon.
没错。p1, p2 和 conv 权重都是可学习参数。
from acon.
你好,我想咨询一下meta-acon里面的r具体作用是啥,可以随便改变默认值吗
from acon.
r是channel放缩系数,可以根据你的需要调整,对准确率影响不大。这是个很常用的减少参数量的技巧,早在2016年的HyperNetworks就已经是常规做法。
这里可以引用原文的解释来回答:“a one-layered hypernetwork would have Nz × Nin × fsize × Nout × fsize learnable
parameters which is usually much bigger than a two-layered hypernetwork does.”
from acon.
@nmaac Did u tested how many speed drop if using MetaAcon compare with normal activation without learnable params? I saw there is no such comparsion in paper but seems will introduce latency increase
from acon.
@nmaac Oh....
from acon.
@jinfagang But ACON is a good choice which has the same speed with Swish, and they have the same speed with ReLU if using hard-sigmoid to implement :)
from acon.
@nmaac You mean ACON-ABC?
from acon.
@jinfagang I suggest ACON-C which improves the performance with a negligible overhead and shows a good accuracy-speed tradeoff.
from acon.
@nmaac I had a question about beta in MetaACON, in paper it mention when beta goes infinite, the loss function will change to the max(x1, x2). However, in MetaACON, beta is generaed by sigmoid function which means that the range of beta is (0, 1).
Is there any reason for this choice?
from acon.
@yxNONG MetaAcon uses a small network to generate beta, in this work we try some network examples which show sigmoid has good performance. More choices and designs of this small network is not the focus of this work but is a promising future direction.
from acon.
@nmaac got it, i will try the ReLU and identities which is much more make sense to me, thanks for your reply!
from acon.
Related Issues (16)
- Parameters of nn.Conv2d in MetaAconC HOT 1
- 请问ACON能用在全连接吗,怎么用 HOT 1
- centernet种使用预训练权重问题 HOT 5
- 速度 HOT 2
- Experimental results on CIFAR-100 HOT 9
- meat-acon
- 请问调用AconC时该传入什么参数? HOT 2
- how to use acon in 3d datasets HOT 2
- 是否可以在yolo5中使用代替silu? 我在替换后效果并不好
- 关于论文中 Fig 4的相关咨询
- Use it in Conv1d HOT 1
- ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 16, 1, 1]) HOT 15
- Hello, I have some question HOT 2
- Use acon in the pre-trained model HOT 3
- 问题请教 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from acon.