Epipolar geometry

1、Epipolar geometryCorrespondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x in the second image?Camera geometry (motion): Given a set of corresponding image points xi xi, i=1,n, what are the cameras P and P for the two views?

2、Or what is the geometric transformation between the views?(iii) Scene geometry (structure): Given corresponding image points xi xi and cameras P, P, what is the position of the point X in space?Three questions:The epipolar geometryC,C,x,x and X are coplanarThe epipolar geometryAll points on p projec

3、t on l and lThe epipolar geometryThe camera baseline intersects the image planes at the epipoles e and e.Any plane p conatining the baseline is an epipolar plane. All points on p project on l and l.The epipolar geometryFamily of planes p and lines l and l Intersection in e and eThe epipolar geometry

4、epipoles e,e= intersection of baseline with image plane = projection of projection center in other image= vanishing point of camera motion directionan epipolar plane = plane containing baseline (1-D family)an epipolar line = intersection of epipolar plane with image(always come in corresponding pair

5、s)Example: converging camerasExample: motion parallel with image planeExample: forward motioneeMatrix form of cross product babaaaaaabababababababa 0001213231221311323320)(0)(babbaaGeometric transformation| with 0| with tRMPMpIMMPptRPPCalibrated CameraTTvupvupRptp) 1 , , () 1 ,( with 0)( 0Epp SRRtEE

6、pp with 0Essential matrixUncalibrated Camera 0 pEpscoordinate camerain and toingcorrespond scoordinate pixelin points and pppppMppMp and 1int1int with 01intintMEMFF ppTTFundamental matrixProperties of fundamental and essential matrix Matrix is 3 x 3 Transpose : If F is essential matrix of cameras (P

7、, P). FT is essential matrix of camera (P,P) Epipolar lines: Think of p and p as points in the projective plane then F p is projective line in the right image. That is l=F p l = FT p Epipole: Since for any p the epipolar line l=F p contains the epipole e. Thus (eT F) p=0 for a all p . Thus eT F=0 an

8、d F e =0Fundamental matrix Encodes information of the intrinsic and extrinisic parameters F is of rank 2, since S has rank 2 (R and M and M have full rank) Has 7 degrees of freedom There are 9 elements, but scaling is not significant and det F = 0Essential matrix Encodes information of the extrinisi

9、c parameters only E is of rank 2, since S has rank 2 (and R has full rank) Its two nonzero singular values are equal Has only 5 degrees of freedom, 3 for rotation, 2 for translationScaling ambiguity)( tRPztRPpPzPptRPPTTDepth Z and Z and t can only be recovered up to a scale factorOnly the direction

10、of translation can be obtainedLeast square approach1| constraint under the) Minimize221FFp(piniiWe have a homogeneous system A f =0The least square solution is smallest singular value of A,i.e. the last column of V in SVD of A = U D VTNon-Linear Least Squares ApproachMinimize )()(212iiiiniFppdFppdwi

11、th respect to the coefficients of F Using an appropriate rank 2 parameterizationLocating the epipolesTTFeFeFeFep of nullspace theis ; of nullspace theis 00SVD of F = UDVT.Rectification Image Reprojection reproject image planes onto common plane parallel to line between optical centersRectification R

12、otate the left camera so epipole goes to infinity along the horizontal axis Apply the same rotation to the right camera Rotate the right camera by R Adjust the scale3D Reconstruction Stereo: we know the viewing geometry (extrinsic parameters) and the intrinsic parameters: Find correspondences exploi

13、ting epipolar geometry, then reconstruct Structure from motion (with calibrated cameras): Find correspondences, then estimate extrinsic parameters (rotation and direction of translation), then reconstruct. Uncalibrated cameras: Find correspondences, Compute projection matrices (up to a projective tr

14、ansformation), then reconstruct up to a projective transformation.Reconstruction by triangulationIf cameras are intrinsically and extrinsically calibrated, find P as the midpoint of the common perpendicular to the two raysin space.PTriangulationap ray through C and p, bRp + T ray though C and p expr

15、essed in right coordinate systemTRppcbRpap)(R = ?T = ?lrTlrRTTTRRRPoint reconstructionMXx XMxLinear triangulationXMxMXx 0XMx0MXx 0XmXm0XmXm0XmXmT1T2T2T3T1T3yxyxT2T3T1T3T2T3T1T3mmmmmmmmAyxyx0AX homogeneousLinear combinationof 2 other equations0AX Homogenous system:X is last column of V in the SVD of

16、A= USVT1X geometric error0 x F x subject to ) x ,(x)x (x,T22ddXM x and XMx subject toly equivalentor Geometric errorReconstruct matches in projective frame by minimizing the reprojection error22XM,xMX, xddNon-iterative optimal solutionReconstruction for intrinsically calibrated cameras Compute the e

17、ssential matrix E using normalized points. Select M=I|0 M=R|T then E=TxR Find T and R using SVD of EDecomposition of ERTExE can be computed up to scale factor 222222yxzyzxzyzxyxzxyxzyTxTxTTTTTTTTTTTTTTTTTTTTRRTEE|2)(|T|EETrTT can be computed up to sign(EET is quadratic) Four solutions for the decomp

18、osition,Correct one corresponds to positive depth valuesSVD decomposition of E E = USVT000001-010 Zand 100001010with or WVUWRUWVRUZUTTTTTReconstruction from uncalibrated camerasReconstruction problem:given xixi , compute M,M and XiiiMXx iiXMxfor all iwithout additional information possible only up t

19、o projective ambiguityProjective Reconstruction Theorem Assume we determine matching points xi and xi. Then we can compute a unique Fundamental matrix F. The camera matrices M, M cannot be recovered uniquely Thus the reconstruction (Xi) is not unique There exists a projective transformation H such that112112, 1, 2 HMMHMMHXXiiReconstruction ambiguity: projectiveiiiXHMHMXx P-1 PFrom Projective to Metric Reconstruction Compute homography H such that XEi=HXi for 5 or more control points XEi with known Euclidean position. Then the metric reconstructio