Opencv中Homography的使用

What is Homography?

Two images of a scene are related by a homography under two conditions.

1. The two images are that of a plane (e.g. sheet of paper, credit card etc.).
2. The two images were acquired by rotating the camera about its optical axis. We take such images while generating panoramas.

As mentioned earlier, a homography is nothing but a 3×3 matrix as shown below.

  

Let  be a point in the first image and  be the coordinates of the same physical point in the second image. Then, the Homography  relates them in the following way

  

If we knew the homography, we could apply it to all the pixels of one image to obtain a warped image that is aligned with the second image.

How to find Homography?

Figure 2. Two images of the same 3D plane ( top of the book ) are related by a Homography

If we know 4 or more corresponding points in the two images, we can use the OpenCV function findHomography to find the homography. An example of four corresponding points is shown in the Figure above. The red, green, yellow and orange points are corresponding points.

Internally the function findHomography solves a linear system of equations to find the homography, but in this post we will not go over that math.

Let’s check the usage.

C++

Python

where, points1 and points2 are vectors/arrays of corresponding points, and h is the homography matrix.

该如何使用呢？

利用Homography可以做到这点。 
1.首先获取书本四个顶点的坐标 pts_src 
2.然后我们需要知道书本的宽高比，此书的宽高比是3/4,所以可使输出图像的size 为300*400，就可设其四个点的坐标为（0,0），（299,0），（299,399），（0,399）保存在pts_dst中 
3.通过pts_src和pts_dst 获取homography 
4.对原图应用homography、warpPerspective 得到输出

代码如下：

#include <opencv2/opencv.hpp>

using namespace cv;

using namespace std;

struct userdata{

    Mat im;

    vector<Point2f> points;

};

void mouseDealer(int event, int x, int y, int flags, void* data_ptr)

{

    if  ( event == EVENT_LBUTTONDOWN )

    {

        userdata *data = ((userdata *) data_ptr);

        circle(data->im, Point(x,y),3,Scalar(0,0,255), 5, CV_AA);

        imshow("Image", data->im);

        if (data->points.size() < 4)

        {

            data->points.push_back(Point2f(x,y));

        }

    }

}

void main(char argc,char ** argv)

{

    // Read source image.

    Mat im_src = imread("b1.jpg");

    // Destination image. The aspect ratio of the book is 3/4

    Size size(300,400);

    Mat im_dst = Mat::zeros(size,CV_8UC3);

    // Create a vector of destination points.

    vector<Point2f> pts_dst;

    pts_dst.push_back(Point2f(0,0));

    pts_dst.push_back(Point2f(size.width - 1, 0));

    pts_dst.push_back(Point2f(size.width - 1, size.height -1));

    pts_dst.push_back(Point2f(0, size.height - 1 ));

    // Set data for mouse event

    Mat im_temp = im_src.clone();

    userdata data;

    data.im = im_temp;

    cout << "Click on the four corners of the book -- 顺时针方向" << endl;

    // Show image and wait for 4 clicks. 

    imshow("Image", im_temp);

    // Set the callback function for any mouse event

    setMouseCallback("Image", mouseDealer, &data);

    waitKey(0);

    // Calculate the homography

    Mat h = findHomography(data.points, pts_dst);

    // Warp source image to destination

    warpPerspective(im_src, im_dst, h, size);

    // Show image

    imshow("Image", im_dst);

    waitKey(0);

}