Analytics Development Images and Photos

OpenCV Face Recognition

What is OpenCV?

OpenCV is an open source C++ library. It is used for image processing and computer vision. It was developed by Intel. An exciting thing about OpenCV is that it is free for non-commercial use as well as for commercial purposes. OpenCV is aimed at real image processing, so it’s library contains many built-in functions for this type of work.

Instructions How to install OpenCV are Here

OpenCV Basic Principles

Here are some basic principles we will use for OpenCV programming which include Header files, Namespace and Data Types of Array.

Header Files

Here we will see some important header files for our example programs. If you don’t know which header file to use, include all the header files which are mentioned below. It will just increase your source lines a little more than necessary. There is no penalty for including all header files in your code. Some important header files are shown below:

#include "opencv2/core/core.hpp"
#include "opencv2/flann/miniflann.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/photo/photo.hpp"
#include "opencv2/video/video.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/objdetect/objdetect.hpp"
#include "opencv2/calib3d/calib3d.hpp"
#include "opencv2/ml/ml.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/core/ core_c.h"
#include "opencv2/highgui/highgui_c.h"
#include "opencv2/imgproc/imgproc_c.h"


Function and Classes of OpenCV are declared in the CV Namespace.   After including the header files, you can bring the namespace in scope with the using namespace cv line like this:

#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/imgproc/imgproc_c.h"
#include "opencv2/core/core_c.h"

using namespace cv; 
int main()
    Mat frame = cvQueryFrame( capture );
    imshow( "Video", frame );

Alternatively at the beginning of every OpenCV class, function and data structure in your source code, we have to append the cv:: name space specifier to scope it into the cv:: namespace like this:

#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/imgproc/imgproc_c.h"
#include "opencv2/core/core_c.h”

int main()
    cv::Mat frame = cvQueryFrame( capture );
    cv::imhow( "Video", frame );

Data Types of Array

The number of bits allocated for each element of the array (pixels in an image) and how the value is represented using those bits are defined by Date Types of an Array.

For Single Dimension Array:

  • CV_16U ( use for 16-bit unsigned integer)
  • CV_16S (use for 16 bit signed integer)
  • CV_8U (use for 8-bit unsigned integer)
  • CV_8S (use for 8 bit signed integer)
  • CV_32F (use for 32-bit floating point number)
  • CV_64F (use for 64-bit floating point number)

For Multi-Dimensional Array: For multi dimensional array we can use all of the above data

  • CV_8UC1 (use for single-channel array with 8-bitunsigned integers)
  • CV_8UC2(use for two channel array with 8-bit unsigned integers)
  • CV_8UC3(use for three channel array with 8-bit unsigned integers)
  • CV_8UC4(use for four-channel array with 8-bit unsigned integers)
  • CV_8UC(n)(use for n channel array with 8-bit unsigned integers ) n can be range from 1 to 512.

Some Basic Drawing Examples with Open CV

Now we are moving towards the basic coding of OpenCV. We will draw a line, circle, ellipse and some other shapes using OpenCV

Example of Drawing a Line

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
using namespace cv;
int main( )
    // first we will create a black empty image
    Mat image = Mat::zeros( 400, 400, CV_8UC3 );
    // now we are drawing a line
    line( image, Point( 10, 15 ), Point( 60, 40), 
        Scalar( 120, 240, 0 ),  3, 9 );

waitkey() in the code above tells the program how long to display the image, with 0 being infinite.

Example of Drawing a Circle

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
using namespace cv;

int main( )
    // we are creating a black empty image where we will draw circle
    Mat image = Mat::zeros( 400, 400, CV_8UC3 );
    // now we wre drawing a circle 
    circle( image, Point( 100, 100 ), 32.0, Scalar( 255, 255, 255 ), 2, 7 );

Example of Drawing an Ellipse

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
using namespace cv;
int main( )
    // Creating black empty images where we will draw an ellipse
    Mat image = Mat::zeros( 400, 400, CV_8UC3 );
    // now we are drawing an ellipse 
    ellipse( image, Point(100, 100), Size( 100.0, 160.0 ), 
        135, 0, 360, Scalar( 255, 255, 255 ), 2, 6 );
    ellipse( image, Point(100, 100), Size( 100.0, 160.0 ), 
        135, 0, 360, Scalar(255, 255, 255), 2, 6 );
    ellipse( image, Point( 100, 100 ), Size( 100.0, 160.0 ), 
        135, 0, 360, Scalar(255, 255, 255), 2, 6 );

How to get started with face recognition using the OpenCV library

In the computer vision and biometric fields, Face Recognition is very active. The Face Recognition process involves two main stages first are Face Detection and the second one is Face Recognition. So we will first take a look at the Face Detection process because you can only recognize a face if you can detect it.

Face Detection

Here the face is detected and then using image processing the facial expressions are cleaned up and face recognition is done. You can easily detect a face using its Haar Cascade Face Detector in OpenCV. Here is the code we will use to detect a face:

// Perform face detection on the input image, 
// using the given Haar Cascade.  Returns a 
// rectangle for the detected region in the 
// given image.
CvRect detectFaceInImage(IplImage *inputImg, 
    CvHaarClassifierCascade* cascade)
    // Smallest face size.
    CvSize minFeatureSize = cvSize(20, 20);

    // Only search for 1 face.

    // How detailed should the search be.
    float search_scale_factor = 1.1f;
    IplImage *detectImg;
    IplImage *greyImg = 0;
    CvMemStorage* storage;
    CvRect rc;
    double t;
    CvSeq* rects;
    CvSize size;
    int i, ms, nFaces;
    storage = cvCreateMemStorage(0);
    cvClearMemStorage( storage );

    // If the image is color, use a greyscale copy of the image.
    detectImg = (IplImage*)inputImg;
    if (inputImg->nChannels > 1) {
        size = cvSize(inputImg->width, inputImg->height);
        greyImg = cvCreateImage(size, IPL_DEPTH_8U, 1 );
        cvCvtColor( inputImg, greyImg, CV_BGR2GRAY );
        detectImg = greyImg;	// Use the greyscale image.

    // Detect all the faces in the greyscale image.
    t = (double)cvGetTickCount();
    rects = cvHaarDetectObjects( detectImg, cascade, storage, 
        search_scale_factor, 3, flags, minFeatureSize);
    t = (double)cvGetTickCount() - t;
    ms = cvRound( t / ((double)cvGetTickFrequency() * 1000.0) );
    nFaces = rects->total;
    printf("Face Detection took %d ms and found %d objects\n", ms, nFaces);

    // Get the first detected face (the biggest).
    if (nFaces > 0)
        rc = *(CvRect*)cvGetSeqElem( rects, 0 );
        rc = cvRect(-1,-1,-1,-1); // Couldn't find the face.
    if (greyImg)
        cvReleaseImage( &greyImg );
    cvReleaseMemStorage( &storage );
    cvReleaseHaarClassifierCascade( &cascade );
    return rc;	// Return the biggest face found, or (-1,-1,-1,-1).

Now, whenever you want to find a face in an image you can just call “detectFaceInImage”.

Process Facial Images for Recognition

Now that you have detected the face, the next process we have to do is face recognition. It is very important to apply different image processing techniques to implement face recognition. Some algorithms are light sensitive. If we take an image in the dark, we cannot identify it in the light and vice versa. Some are dependent on the consistency of an image face. If a little bit of change occurs in the face, we cannot recognize it. So it is essential to use good filters for image processing. So here I will tell you how to convert a color image into grayscale and then to use a histogram equation to standardize the brightness of a dark image. Here is the code to convert a color image to grayscale, then apply a histogram for consistent brightness and contrast.

// Either convert the image to greyscale 
// or use the existing greyscale image.
IplImage *imageGrey;
if (imageSrc->nChannels == 3) {
    imageGrey = cvCreateImage( cvGetSize(imageSrc), IPL_DEPTH_8U, 1 );
    // Convert from RGB (actually it is BGR) to Greyscale.
    cvCvtColor( imageSrc, imageGrey, CV_BGR2GRAY );
else {
    // Just use the input image, since it is already Greyscale.
    imageGrey = imageSrc;

// Resize the image to be a consistent size, even if the aspect ratio changes.
IplImage *imageProcessed;
imageProcessed = cvCreateImage(cvSize(width, height), IPL_DEPTH_8U, 1);

// Make the image a fixed size.
// CV_INTER_CUBIC or CV_INTER_LINEAR is good for enlarging, and
// CV_INTER_AREA is good for shrinking / decimation, but bad at enlarging.
cvResize(imageGrey, imageProcessed, CV_INTER_LINEAR);

// Give the image a standard brightness and contrast.
cvEqualizeHist(imageProcessed, imageProcessed);
.....  Use 'imageProcessed' for Face Recognition ....
if (imageGrey)
if (imageProcessed)

For Face Recognition use of Eigenfaces

Now that we have done pre-processing of an image, we can use Eigenfaces for the face recognition. OpenCV has a built-in function of cvEigenDecomposite(), which can be used to perform PCA operations. For this, you need a database of images to know how to recognize each of your faces. For Creating a Database from loaded images, we can use this code:

// Tell PCA to quit when it has enough eigenfaces.
CvTermCriteria calcLimit = cvTermCriteria( CV_TERMCRIT_ITER, nEigens, 1);

// Compute average image, eigenvectors (eigenfaces) and eigenvalues (ratios).
cvCalcEigenObjects(nTrainFaces, (void*)faceImgArr, (void*)eigenVectArr,
    CV_EIGOBJ_NO_CALLBACK, 0, 0, &calcLimit,
    pAvgTrainImg, eigenValMat->data.fl);

// Normalize the matrix of eigenvalues.
cvNormalize(eigenValMat, eigenValMat, 1, 0, CV_L1, 0);

// Project each training image onto the PCA subspace.
CvMat projectedTrainFaceMat = cvCreateMat( nTrainFaces, nEigens, CV_32FC1 );
int offset = projectedTrainFaceMat->step / sizeof(float);
for(int i=0; i<nTrainFaces; i++) { cvEigenDecomposite(faceImgArr[i], nEigens, eigenVectArr, 0, 0, pAvgTrainImg, projectedTrainFaceMat->data.fl + i*offset);

Face Recognition from a Camera in Real Time

Instead of using a file list it is easy to use a camera stream input to face recognition. For this process we will use another built in function in OpenCV, that is cvCreateCameraCapture(). We can use this function for grabbing images from the webcam.

// Grab the next camera frame. Waits until the next frame is ready, and
// provides direct access to it, so do NOT modify or free the returned image!
// Will automatically initialize the camera the first frame.
IplImage* getCameraFrame(CvCapture* &camera)
    IplImage *frame;
    int w, h;

    // If the camera hasn't been initialized, then open it.
    if (!camera) {
        printf("Acessing the camera ...\n");
        camera = cvCreateCameraCapture( 0 );
        if (!camera) {
            printf("Couldn't access the camera.\n");

        // Try to set the camera resolution to 320 x 240.
        cvSetCaptureProperty(camera, CV_CAP_PROP_FRAME_WIDTH, 320);
        cvSetCaptureProperty(camera, CV_CAP_PROP_FRAME_HEIGHT, 240);

        // Get the first frame, to make sure the camera is initialized.
        frame = cvQueryFrame( camera );
        if (frame) {
            w = frame->width;
            h = frame->height;
            printf("Got the camera at %dx%d resolution.\n", w, h);

        // Wait a little, so that the camera can auto-adjust its brightness.
        Sleep(1000); // (in milliseconds)

    // Wait until the next camera frame is ready, then grab it.
    frame = cvQueryFrame( camera );
    if (!frame) {
         printf("Couldn't grab a camera frame.\n");

    return frame;

OpenCV Face Databases

For our face detection process, we need to create a database of which our recognizer tool will recognize faces by reading already stored images in the database. There are many databases which we can use. Some essential databases that we can use as our face databases are:

Yale Face Database
Yale face database contains 165 grayscale images for 15 individuals, and it accepts only Gif format. It contains 11 images for one individual. Each image with different facial expression. Like happy, sad, sleepy, surprised, with glasses, without glasses, wink, left a light, right light, center light and normal.

Yale Face Database B
Yale Face Database B contains 5670 images for ten individuals each under 576 different viewing conditions nine poses x illumination conditions. An image with background illumination was also captured for each in particular pose during the testing of this face database.

CAS-PEAL Face Database
The CAS-PEAL face database has been constructed under the sponsors of National Hi-Tech Program and ISVISION. Major goals to create CAS-PEAL face database were to create a large-scale Chinese face database to provide worldwide FR community. Facilitate that community by providing it with large-scale images with different variations like expressions, posing, lightning, etc.

Image Database of Facial Actions and Expressions
In image database of facial actions and expressions, 24 individuals were tested in the database with 6 to 8 examples of 150 different desired actions. Color Images around 700 were included in the database with each color image matching grey scale image in the neural network.

Sheffield Face Database
Sheffield face database contains 564 images of 20 individuals. Covering each in different poses like happy, sad, etc. each has its directory and directory looks like 1a, 1b, 1c… It accepts PGM formats for a picture. Image size will be 220 x 220px with 256 grayscale images.

UCD Color Face Image Database
The UCD color face image database for face recognition contains two parts. This database was intended for distribution to researchers as it can help them in their biometric research. It can make research easy for them. Its first part contains color pictures with some variations of poses, orientation, expressions of faces, different lighting conditions, etc. the second part of UCD Color Face Image Database contains a manually segmented result for each image in part one of the databases. To acquire these images they used some resources like WWW(world wide web), photo scanners were used for pictures scanning, digital cameras and other face detection databases were used.

M2VTS Multimodal Face Database
M2VTS Multimodal Face Database contains 37 pictures with five pictures of each person. These shots were taken with the interval of 1 week and with the dramatic changes in the face expression in the meantime. During the process of taking shots, persons were asked to rotate their head from 0 to 90 degree and then back to 0 degrees. They were asked to count from 0-9 in their natural languages. Shorts were also taken with and without glasses with moving the head from one angle to another. Later its extended version came with some innovations in it. The Extended M2VTS Multimodal Face Database.

AT&T is formally called as ORL database of faces. For 40 individuals they take ten images for each in different position and different time like happy, sad, glasses, no glasses, dark, lightning, etc

How to use the Face Database

As I mentioned above, we will use AT&T Face Database. It will contain 40 folders, one folder for each, and ten grayscale images of size 92x112px for that individual face. The directory structure will look like this:

--/ subject1 /2.pqm
--/ subject1 /...
--/ subject 2/1.pqm
--/ subject 2/1.pqm

If you want to increase your computation speed, It is recommended to delete folders from subject11 to subject40. You would have to set up a folder for the face you want to detect. I named it subject99 and save ten pictures of any individual whose face you want to detect. Now we are moving towards coding section. We can divide our coding section into different steps for better understanding. Load training images into an array, then perform analysis on those images. After performing analysis, load your sample images. Again perform analysis on those sample images. After that Calculate Euclidian distance from test image to each training image and pick result with the smallest distance.

from sklearn.decomposition import RandomizedPCA
import numpy as nump
import glob
import cv2
import math
import os.path
import string

##here we are getting id from image(filename)##
def ID_from_image(image):
    q = string.split(image, '/')
    return p [1].replace("s", "")
##converting Image to the format which we want##
def prepare_file(image):
    image_color = cv2.imread(image)
    image_gray = cv2.cvtColor(image_color,
    image_gray = cv2.equalizeHist(image_gray)
    return image_gray.flat

##setting image resolution and no of images to store##
IMAGE_RESOLUTION = 92 * 112     

##loading image from folder t_faces##
f = glob.glob('t_faces/*')

 ##creating array##
A = nump.zeros([NUMBER_TRAININGMAGES, IMAGE_RESOLUTION], dtype='int8')
B = []
z= 0
for b, folder in enumerate(f):
    training_images = glob.glob(f + '/*')
    for i, face in enumerate(training_images):
        A[z:] = prepare_file(face_image)
        z = z + 1

##analysis on the images ##
Prin_com_anal = RandomizedPCA(n_components=NUMBER_EIGENFACES, whiten=True).fit(A)
A_ Prin_com_anal = Prin_com_anal.transform(A)

##loading images from folder t_faces##
t_faces = glob.glob('t_faces/*')

##array creation##
A = nump.zeros([len(test_faces), IMAGE_RESOLUTION], dtype='int8')
 for J, face in enumerate(t_faces):
    A[J,:] = prepare_file(faces)
 for i, ref_ Prin_com_anal in enumerate(Prin_com_anal.transform(A)):
    distances = []
    # Calculate euclidian distance from test image to each of the known images and save distances
    for i, test_ Prin_com_anal in enumerate(A_ Prin_com_anal):
        dist = math.sqrt(sum([diff**2 for diff in (ref_ Prin_com_anal - test_ Prin_com_anal)]))
        distances.append((dist, y[i]))
     found_ID = min(distances)[1]
    print "Identified (result: "+ str(found_ID) +" - dist - " + str(min(distances)[0])  + ")"

Now if you put the image in the right folder and have done everything as mentioned above. It will give you the result99 as we named it. Try this yourself.


How to install OpenCV Ubuntu

About the author


A passionate Linux user for personal and professional reasons, always exploring what is new in the world of Linux and sharing with my readers.

Leave a Comment