• / 19
  • 下载费用:10 金币  

深度学习回传递简介.pptx

关 键 词:
深度学习回传递简介.pptx
资源描述:
Lecture 3: CNN: Back-propagation,boris. ginzburg@intel.com,Agenda,Introduction to gradient-based learning for Convolutional NN Backpropagation for basic layers Softmax Fully Connected layer Pooling ReLU Convolutional layer Implementation of back-propagation for Convolutional layer CIFAR-10 training,Good Links,http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf http://www.iro.umontreal.ca/~pift6266/H10/notes/gradient.html#flowgraph,Gradient based training,Conv. NN is a just cascade of functions: f(x0,w)  y, where x0 is image [28,28], w – network parameters (weights, bias) y – softmax output= probability that x belongs to one of 10 classes 09,,Gradient based training,We want to find parameters W, to minimize an error E (f(x0,w),y0) = -log (f(x0,w)- y0). For this we will do iterative gradient descent: w(t) = w(t-1) – λ * −𝜕𝐸 𝜕𝑤 (t) How do we compute gradient of E wrt weights? Loss function E is cascade of functions. Let’ s go layer by layer, from last layer back, and use the chain rule for gradient of complex functions:𝜕𝐸 𝜕 𝑦 𝑙−1 = 𝜕𝐸 𝜕 𝑦 𝑙 × 𝜕 𝑦 𝑙 (𝑤, 𝑦 𝑙−1 ) 𝜕 𝑦 𝑙−1 𝜕𝐸 𝜕 𝑤 𝑙 = 𝜕𝐸 𝜕 𝑦 𝑙 × 𝜕 𝑦 𝑙 (𝑤, 𝑦 𝑙−1 ) 𝜕 𝑤 𝑙,,LeNet topology,FORWARD,BACKWARD,Data Layer,Convolutional layer [5x5],Convolutional layer [5x5],Pooling [2x2, stride 2],Pooling [2x2, stride 2],Inner Product,ReLUP,Inner Product,Soft Max + LogLoss,Layer:: Backward( ),class Layer { Setup (bottom, top); // initialize layer Forward (bottom, top); //compute : 𝑦 𝑙 =𝑓 𝑤 𝑙 , 𝑦 𝑙−1 Backward( top, bottom); //compute gradient } Backward: We start from gradient 𝜕𝐸 𝜕 𝑦 𝑙 from last layer and 1) propagate gradient back : 𝜕𝐸 𝜕 𝑦 𝑙 → 𝜕𝐸 𝜕 𝑦 𝑙−1 2) compute the gradient of E wrt weights wl: 𝜕𝐸 𝜕 𝑤 𝑙,Softmax with LogLoss Layer,Consider the last layer (softmax with log-loss ): 𝐸=− log 𝑝 𝑦0 =−log⁡( 𝑒 𝑦0 0 9 𝑒 𝑦𝑘 ) = −𝑦0+ log ( 0 9 𝑒 𝑦𝑘 ) For all k=09 , except k0 (right answer) we want to decrease pk:𝜕 𝐸 𝜕 𝑦 𝑘 = 𝑒 𝑦 𝑘 0 9 𝑒 𝑦
展开阅读全文
  微传网所有资源均是用户自行上传分享,仅供网友学习交流,未经上传用户书面授权,请勿作他用。
0条评论

还可以输入200字符

暂无评论,赶快抢占沙发吧。

关于本文
本文标题:深度学习回传递简介.pptx
链接地址:https://www.weizhuannet.com/p-10054656.html
微传网是一个办公文档、学习资料下载的在线文档分享平台!

微传网博客

网站资源均来自网络,如有侵权,请联系客服删除!

 网站客服QQ:80879498  会员QQ群:727456886

copyright@ 2018-2028 微传网络工作室版权所有

     经营许可证编号:冀ICP备18006529号-1 ,公安局备案号:13028102000124

收起
展开