矩陣求導(dǎo)公式.docx

轉(zhuǎn)載矩陣求導(dǎo)公式【轉(zhuǎn)】 (2011-11-15 11:03:34)轉(zhuǎn)載標(biāo)簽：轉(zhuǎn)載原文地址：矩陣求導(dǎo)公式【轉(zhuǎn)】作者：三寅今天推導(dǎo)公式，發(fā)現(xiàn)居然有對矩陣的求導(dǎo)，狂汗-完全不會。不過還好網(wǎng)上有人總結(jié)了。吼吼，趕緊搬過來收藏備份?；竟剑篩 = A * X - DY/DX = AY = X * A - DY/DX = AY = A * X * B - DY/DX = A * BY = A * X * B - DY/DX = B * A1. 矩陣Y對標(biāo)量x求導(dǎo)：相當(dāng)于每個元素求導(dǎo)數(shù)后轉(zhuǎn)置一下，注意MN矩陣求導(dǎo)后變成NM了Y = y(ij) - dY/dx = dy(ji)/dx2. 標(biāo)量y對列向量X求導(dǎo)：注意與上面不同，這次括號內(nèi)是求偏導(dǎo)，不轉(zhuǎn)置，對N1向量求導(dǎo)后還是N1向量y = f(x1,x2,.,xn) - dy/dX = (Dy/Dx1,Dy/Dx2,.,Dy/Dxn)3. 行向量Y對列向量X求導(dǎo)：注意1M向量對N1向量求導(dǎo)后是NM矩陣。將Y的每一列對X求偏導(dǎo)，將各列構(gòu)成一個矩陣。重要結(jié)論：dX/dX = Id(AX)/dX = A4. 列向量Y對行向量X求導(dǎo)：轉(zhuǎn)化為行向量Y對列向量X的導(dǎo)數(shù)，然后轉(zhuǎn)置。注意M1向量對1N向量求導(dǎo)結(jié)果為MN矩陣。dY/dX = (dY/dX)5. 向量積對列向量X求導(dǎo)運算法則：注意與標(biāo)量求導(dǎo)有點不同。d(UV)/dX = (dU/dX)V + U(dV/dX)d(UV)/dX = (dU/dX)V + (dV/dX)U重要結(jié)論：d(XA)/dX = (dX/dX)A + (dA/dX)X = IA + 0X = Ad(AX)/dX = (d(XA)/dX) = (A) = Ad(XAX)/dX = (dX/dX)AX + (d(AX)/dX)X = AX + AX6. 矩陣Y對列向量X求導(dǎo)：將Y對X的每一個分量求偏導(dǎo)，構(gòu)成一個超向量。注意該向量的每一個元素都是一個矩陣。7. 矩陣積對列向量求導(dǎo)法則：d(uV)/dX = (du/dX)V + u(dV/dX)d(UV)/dX = (dU/dX)V + U(dV/dX)重要結(jié)論：d(XA)/dX = (dX/dX)A + X(dA/dX) = IA + X0 = A8. 標(biāo)量y對矩陣X的導(dǎo)數(shù)：類似標(biāo)量y對列向量X的導(dǎo)數(shù)，把y對每個X的元素求偏導(dǎo)，不用轉(zhuǎn)置。dy/dX = Dy/Dx(ij) 重要結(jié)論：y = UXV = u(i)x(ij)v(j) 于是 dy/dX = u(i)v(j) = UVy = UXXU 則 dy/dX = 2XUUy = (XU-V)(XU-V) 則 dy/dX = d(UXXU - 2VXU + VV)/dX = 2XUU - 2VU + 0 = 2(XU-V)U9. 矩陣Y對矩陣X的導(dǎo)數(shù)：將Y的每個元素對X求導(dǎo)，然后排在一起形成超級矩陣。10.乘積的導(dǎo)數(shù)d(f*g)/dx=(df/dx)g+(dg/dx)f結(jié)論d(xAx)=(d(x)/dx)Ax+(d(Ax)/dx)(x)=Ax+Ax（注意：是表示兩次轉(zhuǎn)置）比較詳細(xì)點的如下：/blog/static/145880136201051113615571//wangwen926/blog/item/eb189bf6b0fb702b720eec94.html其他參考：Contents Notation Derivatives of Linear Products Derivatives of Quadratic ProductsNotation d/dx(y)is a vector whose(i)element isdy(i)/dx d/dx(y) is a vector whose(i)element isdy/dx(i) d/dx(yT) is a matrix whose(i,j)element isdy(j)/dx(i) d/dx(Y) is a matrix whose(i,j)element isdy(i,j)/dx d/dX(y) is a matrix whose(i,j)element isdy/dx(i,j)Note that the Hermitian transpose is not used because complex conjugates are not analytic.In the expressions below matrices and vectorsA,B,Cdo not depend onX.Derivatives of Linear Products d/dx(AYB)=A*d/dx(Y) *B d/dx(Ay)=A*d/dx(y) d/dx(xTA)=A d/dx(xT)=I d/dx(xTa)= d/dx(aTx) =a d/dX(aTXb) =abT d/dX(aTXa) =d/dX(aTXTa) =aaT d/dX(aTXTb) =baT d/dx(YZ)=Y*d/dx(Z) +d/dx(Y)* ZDerivatives of Quadratic Products d/dx(Ax+b)TC(Dx+e) =ATC(Dx+e)+DTCT(Ax+b) d/dx(xTCx) = (C+CT)x C: symmetric:d/dx(xTCx) = 2Cx d/dx(xTx) = 2x d/dx(Ax+b)T(Dx+e) =AT(Dx+e)+DT(Ax+b) d/dx(Ax+b)T(Ax+b) = 2AT(Ax+b) C: symmetric:d/dx(Ax+b)TC(Ax+b) = 2ATC(Ax+b) d/dX(aTXTXb) =X(abT+ baT) d/dX(aTXTXa) = 2XaaT d/dX(aTXTCXb) =CTXabT+ CXbaT d/dX(aTXTCXa) =(C + CT)XaaT C:Symmetricd/dX(aTXTCXa) =2CXaaT d/dX(Xa+b)TC(Xa+b) = (C+CT)(Xa+b)aTDerivatives of Cubic Products d/dx(xTAxxT) = (A+AT)xxT+xTAxIDerivatives of Inverses d/dx(Y-1) =-Y-1d/dx(Y)Y-1Derivative of TraceNote: matrix dimensions must result in ann*nargument for tr(). d/dX(tr(X) =I d/dX(tr(Xk) =k(Xk-1)T d/dX(tr(AXk) =SUMr=0:k-1(XrAXk-r-1)T d/dX(tr(AX-1B) =-(X-1BAX-1)T d/dX(tr(AX-1) =d/dX(tr(X-1A) =-X-TATX-T d/dX(tr(ATXBT) =d/dX(tr(BXTA) =AB d/dX(tr(XAT) =d/dX(tr(ATX) =d/dX(tr(XTA) =d/dX(tr(AXT)= A d/dX(tr(AXBXT) =ATXBT+AXB d/dX(tr(XAXT) =X(A+AT) d/dX(tr(XTAX) =XT(A+AT) d/dX(tr(AXTX) =(A+AT)X d/dX(tr(AXBX) =ATXTBT+BTXTAT C:symmetricd/dX(tr(XTCX)-1A) =d/dX(tr(A (XTCX)-1) =-(CX(XTCX)-1)(A+AT)(XTCX)-1 B,C:symmetricd/dX(tr(XTCX)-1(XTBX) =d/dX(tr( (XTBX)(XTCX)-1) =-2(CX(XTCX)-1)XTBX(XTCX)-1+ 2BX(XTCX)-1Derivative of DeterminantNote: matrix dimensions must result in ann*nargument for det(). d/dX(det(X) =d/dX(det(XT) = det(X)*X-T d/dX(det(AXB) = det(AXB)*X-T d/dX(ln(det(AXB) =X-T d/dX(det(Xk) =k*det(Xk)*X-T d/dX(ln(det(Xk) =kX-T Reald/dX(det(XTCX) = det(XTCX)*(C+CT)X(XTCX)-1 C:Real,Symmetricd/dX(det(XTCX) = 2det(XTCX)* CX(XTCX)-1 C:Real,Symmetriccd/dX(ln(det(XTCX) = 2CX(XTCX)-1JacobianIfyis a function ofx, thendyT/dxis the Jacobian matrix ofywith respect tox.Its determinant, |dyT/dx|, is theJacobianofywith respect toxand represents the ratio of the hyper-volumesdyanddx. The Jacobian occurs when changing variables in an integration: Integral(f(y)dy)=Integral(f(y(x) |dyT/dx| dx).Hessian matrixIf f is a function ofxthen the symmetric matrix d2f/dx2=d/dxT(df/dx) is theHessianmatrix of f(x). A value ofxfor which df/dx=0corresponds to a minimum, maximum or saddle point according to whether the Hessian is positive definite, negative definite or indefinite. d2/dx2(aTx) = 0 d2/dx2(Ax+b)TC(Dx+e) =ATCD+DTCTA d2/dx2(xTCx) =C+CT d2/dx2(xTx) = 2I d2/dx2(Ax+b)T(Dx+e) =ATD+DTA d2/dx2(Ax+b)T(Ax+b) = 2ATA C: symmetric:d2/dx2(Ax+b)TC(Ax+b) = 2ATCAhttp:/www.psi.toronto