Comments (6)
误差反向传播,最后一层的误差就是预测(前向传播的值)减去label真实值,然后反向传播计算每一层的误差
from machinelearning_python.
更新第二层权重theta2需要求costFunction对theta2的导数,costFunction对theta2的导数可以分解为 z3对theta2的导数 与 costFunction对z3的导数 之积,前者是a2,后者不应该只是 h-class_y 吧,我觉得应该是 (h-class_y)乘上h对z3的导数,即sigmoidGradient(z3)。我是初学者,也是自学者,没有进行过专业的学习,很可能犯一些很幼稚的错误,希望你能多多包涵,不吝赐教。
from machinelearning_python.
好久没看都忘记了,你的理解是对的,但是这里最后使用的是交叉熵损失函数,你可以试着推导一下,应该是没错的。
from machinelearning_python.
刚做了个实验,代码里是有一个利用梯度的定义检测梯度的函数的,如果代价函数定义为均方误差(没加正则):
temp1 = (h.reshape(-1,1)-class_y.reshape(-1,1))
temp2 = (temp1**2).sum()
J = 1/(2*m)*temp2
然后修改
delta3[i,:] = (h[i,:]-class_y[i,:])*sigmoidGradient(z3[i,:])
这样输出的梯度是正常的
[[ 5.16628040e-03 5.16628041e-03]
[ -8.31394759e-06 -8.31394784e-06]
[ 5.41669659e-05 5.41669655e-05]
[ 6.68470207e-05 6.68470206e-05]
[ 1.56912539e-03 1.56912539e-03]
[ -1.21626675e-05 -1.21626677e-05]
[ 2.54606233e-05 2.54606235e-05]
[ 3.96755345e-05 3.96755349e-05]
[ -3.47046008e-03 -3.47046009e-03]
[ -4.82780121e-06 -4.82780112e-06]
[ -2.66496109e-05 -2.66496103e-05]
[ -2.39698913e-05 -2.39698907e-05]
[ -5.31940615e-03 -5.31940616e-03]
[ 6.94345692e-06 6.94345726e-06]
[ -5.42630105e-05 -5.42630103e-05]
[ -6.55803162e-05 -6.55803165e-05]
[ -2.27769095e-03 -2.27769095e-03]
[ 1.23327012e-05 1.23327010e-05]
[ -3.19844790e-05 -3.19844792e-05]
[ -4.68952766e-05 -4.68952768e-05]
[ 1.21981949e-01 1.21981949e-01]
[ 6.10153665e-02 6.10153665e-02]
[ 6.10411275e-02 6.10411275e-02]
[ 6.09010195e-02 6.09010195e-02]
[ 6.10584199e-02 6.10584199e-02]
[ 6.09927601e-02 6.09927601e-02]
[ 7.28582367e-02 7.28582368e-02]
[ 3.64005559e-02 3.64005559e-02]
[ 3.63703836e-02 3.63703836e-02]
[ 3.65344619e-02 3.65344619e-02]
[ 3.63501369e-02 3.63501369e-02]
[ 3.64270251e-02 3.64270252e-02]
[ 2.38758286e-02 2.38758286e-02]
[ 1.18930662e-02 1.18930662e-02]
[ 1.20555685e-02 1.20555685e-02]
[ 1.18289556e-02 1.18289556e-02]
[ 1.19627032e-02 1.19627032e-02]
[ 1.20144705e-02 1.20144705e-02]]
(如果我检测梯度函数没写错的话=-=)
from machinelearning_python.
是我惯性思维了,以为这里是欧式距离,你说的是对的。
非常感谢你能写出这么“生撕”机器学习的文章,不用高级库只用numpy实现加深了我不少理解。
提个小tip吧,在算正则化这类的对应项直接相乘的时候,array类型可以直接用’*‘号,相当于matlab的点乘,最后再np.sum,我觉得比较好理解,希望对你有用。
再次感谢
from machinelearning_python.
当时也是刚学完python没多久写的,应该有好多地方不是最简化的。意思意思
from machinelearning_python.
Related Issues (19)
- 这里理论推导应该是负梯度方向啊,为什么这里用的是梯度方向,关键是结果还正确。 HOT 7
- LogisticRegression_scikit-learn.py HOT 1
- readMe文档中目录链接没法跳转。 HOT 1
- 机器学习
- Python-machine learning
- 机器学习
- Pavan
- Website development
- 学习
- 关于nn_params的问题 HOT 2
- Hi sir I want to learn python machine learning for that I should want to learn python first or I want to start machine learning now HOT 5
- 机器学习相关 HOT 1
- 请教如何建立这样的知识体系 HOT 1
- Trading model HOT 3
- 关于nnGradient函数的一个问题 HOT 2
- bug? HOT 1
- Font issue for MacOS HOT 1
- 假负表达式错误,selectThreshold中fn = np.sum((cvPrecision == 1) & (yval == 0)).astype(float) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from machinelearning_python.