Thursday, July 5, 2012

SURF Source Code (Part-2)

Many of you might have already figured out, how to strip the code. The below is the stripped version. Here, we can just supply the gallery image and probe image, the same way as the previous post as arg1 and arg2. But here, no GUI utilities are being used. Just the output needed is printed to standard output. This drastically reduces the time for the program execution. Also, I have removed the flann method of matching, since any one (findpairs and flannfindpairs - functions) can be used and both the methods given have the same recognition performance.

 * A Demo to OpenCV Implementation of SURF
 * Further Information Refer to "SURF: Speed-Up Robust Feature"
 * Author: Liu Liu
 * Modified by :
 * Modifying Author : Dileep Kumar Kotha

#include <cv.h>
#include <highgui.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>

#include <iostream>
#include <vector>

using namespace std;

static double dis=0;//Dileep:For calculating the distance

IplImage *image = 0;

compareSURFDescriptors( const float* d1, const float* d2, double best, int length )
    double total_cost = 0;
    assert( length % 4 == 0 );
    for( int i = 0; i < length; i += 4 )
        double t0 = d1[i] - d2[i];
        double t1 = d1[i+1] - d2[i+1];
        double t2 = d1[i+2] - d2[i+2];
        double t3 = d1[i+3] - d2[i+3];
        total_cost += t0*t0 + t1*t1 + t2*t2 + t3*t3;
We are sending a total cost, that's slightly greater or smaller than the best 

      if( total_cost > best )
    return total_cost;

naiveNearestNeighbor( const float* vec, int laplacian,
                      const CvSeq* model_keypoints,
                      const CvSeq* model_descriptors )
    int length = (int)(model_descriptors->elem_size/sizeof(float));
    int i, neighbor = -1;
    double d, dist1 = 1e6, dist2 = 1e6;
    CvSeqReader reader, kreader;
    cvStartReadSeq( model_keypoints, &kreader, 0 );
    cvStartReadSeq( model_descriptors, &reader, 0 );

    for( i = 0; i < model_descriptors->total; i++ )
        const CvSURFPoint* kp = (const CvSURFPoint*)kreader.ptr;
        const float* mvec = (const float*)reader.ptr;
     CV_NEXT_SEQ_ELEM( kreader.seq->elem_size, kreader );
        CV_NEXT_SEQ_ELEM( reader.seq->elem_size, reader );
        if( laplacian != kp->laplacian )
        d = compareSURFDescriptors( vec, mvec, dist2, length );

        if( d < dist1 )
            dist2 = dist1;
            dist1 = d;
            neighbor = i;
        else if ( d < dist2 )
            dist2 = d;

/*Dileep:We are finding the distance from every descriptor of probe image to every descriptor of the galley image. Finally in the findpairs function, we divide this distance with the total no. of descriptors to get the average of all the distances
    if ( dist1 < 0.6*dist2 )
        return neighbor;
    return -1;

findPairs( const CvSeq* objectKeypoints, const CvSeq* objectDescriptors,
           const CvSeq* imageKeypoints, const CvSeq* imageDescriptors, vector<int>& ptpairs )
    int i;
    CvSeqReader reader, kreader;
    cvStartReadSeq( objectKeypoints, &kreader );
    cvStartReadSeq( objectDescriptors, &reader );

    for( i = 0; i < objectDescriptors->total; i++ )
        const CvSURFPoint* kp = (const CvSURFPoint*)kreader.ptr;
        const float* descriptor = (const float*)reader.ptr;
        CV_NEXT_SEQ_ELEM( kreader.seq->elem_size, kreader );
        CV_NEXT_SEQ_ELEM( reader.seq->elem_size, reader );
        int nearest_neighbor = naiveNearestNeighbor( descriptor, kp->laplacian, imageKeypoints, imageDescriptors);
//Dileep:For every descriptor, we are trying to find it's nearest neighbour in the probe image
        if( nearest_neighbor >= 0 )

printf("\n%lf\n",(dis/objectDescriptors->total));////Dileep:Here's where I am outputting the distance between the images
/*Dileep: If you are using this for recognition, write this distance to a file along with the name of the image you are matching against. After doing this for several images, you can then sort them in ascending order to find the best possible match - the one with the smallest distance. Here, I am outputting the distance to stdout

int main(int argc, char** argv)
    const char* object_filename = argc == 3 ? argv[1] : "box.png";
    const char* scene_filename = argc == 3 ? argv[2] : "box_in_scene.png";
//Dileep:When you are excuting the object file, please write Command:./objectfile probe_image Gallery_image
Probe_image - This is the image for which you need to find the match
Gallery_image - This is one of the set of images, you use for matching

You keep the same probe image same, repeatedly changing the gallery image and outputting the distance in the format
<Gallery_name distance> into a file
Finally you can sort the distances in ascending order. And the one with the shortest distance - You can output it's name as the best possible match

It may become tedious to continually write the same command multiple times, changing the gallery file name. Try to use shell script with a for loop
    CvMemStorage* storage = cvCreateMemStorage(0);



    IplImage* object = cvLoadImage( object_filename, CV_LOAD_IMAGE_GRAYSCALE );
    IplImage* image = cvLoadImage( scene_filename, CV_LOAD_IMAGE_GRAYSCALE );
    if( !object || !image )
        fprintf( stderr, "Can not load %s and/or %s\n"
            "Usage: find_obj [<object_filename> <scene_filename>]\n",
            object_filename, scene_filename );

    CvSeq *objectKeypoints = 0, *objectDescriptors = 0;
    CvSeq *imageKeypoints = 0, *imageDescriptors = 0;
    int i;
    CvSURFParams params = cvSURFParams(500, 1);

    double tt = (double)cvGetTickCount();
    cvExtractSURF( object, 0, &objectKeypoints, &objectDescriptors, storage, params );
    printf("Object Descriptors: %d\n", objectDescriptors->total);
    cvExtractSURF( image, 0, &imageKeypoints, &imageDescriptors, storage, params );
    printf("Image Descriptors: %d\n", imageDescriptors->total);
    tt = (double)cvGetTickCount() - tt;
    printf( "Extraction time = %gms\n", tt/(cvGetTickFrequency()*1000.));

    vector<int> ptpairs;

    findPairs( objectKeypoints, objectDescriptors, imageKeypoints, imageDescriptors, ptpairs );

    return 0;


Sagar Gauswami said...

Hi Dillep!!!
Thanks for giving code.
It runs successfully on my computer.
My question is how to get location(x and y coordinates) of detected object in above code?
Suppose i i have an image of pen as arg1 and another image which contains pen with some background as arg2.
Then how can i get x and y coordinates of pen in arg2?

Dileep Kumar said...

Getting the coordinates is a bit trickier. You need to dig deep into first two functions. But tell me, why do you need the coordinates?

Sagar Gauswami said...

I want to detect a small bottle. and i want to display coordinates of bottle in image. means i want to tracking of bottle.

Dileep Kumar said...

Ohk, that's not possible, since you have to dig into the SURF built-in functions of opencv. Even though it is possible to dig, it's tedious. Better to use SURF for comparison purposes only.

If you want to detect a bottle and track it's coordinates, I suggest you to construct a haar classifier for bottle detection and use a code, similar to the face detector code (both available in this blog) for the same. In the face detector code, inside the detectfaces function, there is a cvrectangle() used to construct rectangle around the detected face. You can get the coordinates, which are the same as arguements to the cvrectangle().

shahd ewawi said...

I tried to try your code but it keeps giving me an error. I think the error is from the linker. Could you tell me what did you include in the linker?

آرش said...

thanks for your codes,
I got identifier "CvSURFParam" is undefined error when paste your code in VS2010.
I also linked opencv_nonfree249d.lib but problem exist!
Can you help me about it?

Dileep Kumar said...

I am not sure, whether all libraries of openCV are ported to Visual Studio. Please try in linux.

Also, if you still want to stick to windows, library names can be different. So you can do a manual search for the same functions and replace with correct ones.

Niraja Ganpule said...

Hi, I found this post very helpful. I have to use SURF Algorithm for feature extraction using OpenCV 2.4.9 in a certain project. I am aware of the inbuilt SURF function that can help in the same. I am new to OpenCv and not completely comfortable with its advanced concepts.However I have been asked to perform the task without using the inbuilt function which means I have to write the code for every step in the algorithm. Is this possible at all. How do I go about it?

Dileep Kumar said...

Yes, It's possible. Just open the header file related to openCV in the above code and start editing it. Best of luck

Post a Comment