DEV Community: pranay749254

Simple Face Recognizing System using python and openCV

pranay749254 — Tue, 22 Aug 2017 08:39:08 +0000

Face Recognising System

Face Recognising System is a computer application that is used to identify people from a image or a video footage.It is mainly used in security purposes to get track of who is entering a certain facility or to search someone in a certain place.It may not be as proper as bio metric or iris scanner but it is much easy to implement.This face recognising system works with a database where image of faces of the people are saved.This system runs a couple of algorithms to determine whether the face of the person in camera or footage matches with any image of database.One of those algorithm saves the images of the people in database in grayScale format.Another algorithm searches for a face.Another algorithm compares whether a certain area or size or shape of the face matches with the database.The combine effect of all these algorithms is this system.For making a basic Face Recognising System one of the best way is to use openCV
What is openCV?
openCV is a cross platform open source library written in C++,developed by Intel.openCV is used for Face Recognising System , motion sensor , mobile robotics etc.This library is supported in most of the operating system i.e. Windows,Linux,Mac,openBSD.This library can be used in python , java , perl , ruby , C# etc.

Face Recognition with Python and openCV

Image Source: Wikipedia , url: https://commons.wikimedia.org/wiki/File:Face_detection.jpg

Source Code :

Database for Faces:

import cv2
import numpy as np
import sqlite3

faceDetect=cv2.CascadeClassifier('haarcascade_frontalface_default.xml');
cam=cv2.VideoCapture(0);

def insertOrUpdate(Id,Name,Age,Gen,CR):
    conn=sqlite3.connect("FaceBase.db")
    cmd="SELECT * FROM People WHERE ID="+str(Id)
    cursor=conn.execute(cmd)
    isRecordExist=0
    for row in cursor:
        isRecordExist=1
    if(isRecordExist==1):
        cmd="UPDATE People SET Name="+str(Name)+"WHERE ID="+str(Id)
        cmd2="UPDATE People SET Age="+str(Age)+"WHERE ID="+str(Id)
        cmd3="UPDATE People SET Gender="+str(Gen)+"WHERE ID="+str(Id)
        cmd4="UPDATE People SET CR="+str(CR)+"WHERE ID="+str(Id)
    else:
        cmd="INSERT INTO People(ID,Name,Age,Gender,CR) Values("+str(Id)+","+str(Name)+","+str(Age)+","+str(Gen)+","+str(CR)+")"
        cmd2=""
        cmd3=""
        cmd4=""
    conn.execute(cmd)
    conn.execute(cmd2)
    conn.execute(cmd3)
    conn.execute(cmd4)
    conn.commit()
    conn.close()

Id=raw_input('Enter User Id')
name=raw_input('Enter User Name')
age=raw_input('Enter User Age')
gen=raw_input('Enter User Gender')
cr=raw_input('Enter User Criminal Records')
insertOrUpdate(Id,name,age,gen,cr)
sampleNum=0
while(True):
    ret,img=cam.read();
    gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    faces=faceDetect.detectMultiScale(gray,1.3,5);
    for(x,y,w,h) in faces:
        sampleNum=sampleNum+1;
        cv2.imwrite("dataSet/User."+str(Id)+"."+str(sampleNum)+".jpg",gray[y:y+h,x:x+w])
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
        cv2.waitKey(100);
    cv2.imshow("Face",img);
    cv2.waitKey(1);
    if(sampleNum>20):
        break;
cam.release()
cv2.destroyAllWindows()

Source: https://www.youtube.com/watch?v=1Jz24sVsLE4&list=PLnjEM1fs09cGGjdCLSue8Kw7GmWDhGlMh

This program asks for a person to provide a unique id , name , age , gender and criminal records.Then it collects 20 picture of faces of the person and then converts it to grayScale.It saves all the data of that person in database or updates if existing.Here cv2 is the openCV library and the database use is sqlite3.The cascade classifier haarcascade_frontalface_default.xml is used for face recognising system.

face Trainer:

import os
import cv2
import numpy as np
from PIL import Image

recognizer=cv2.createLBPHFaceRecognizer();
path='dataSet'

def getImagesWithID(path):
    imagepaths=[os.path.join(path,f) for f in os.listdir(path)]
    faces=[]
    IDs=[]
    for imagepath in imagepaths:
        faceImg=Image.open(imagepath).convert('L');
        faceNp=np.array(faceImg,'uint8')
        ID=int(os.path.split(imagepath)[-1].split('.')[1])
        faces.append(faceNp)
        IDs.append(ID)
        cv2.imshow("training",faceNp)
        cv2.waitKey(10)
    return np.array(IDs),faces

IDs,faces=getImagesWithID(path)
recognizer.train(faces,IDs)
recognizer.save('recognizer/trainningData.yml')
cv2.destroyAllWindows()

Source: https://www.youtube.com/watch?v=1Jz24sVsLE4&list=PLnjEM1fs09cGGjdCLSue8Kw7GmWDhGlMh

In this part the program trains the faces and saves the data in a .yml format.The used library os is predefined in python.The other two libraries numpy and pillow have to be installed manually.

face Recognition:

import cv2
import numpy as np
import sqlite3

faceDetect=cv2.CascadeClassifier('haarcascade_frontalface_default.xml');
cam=cv2.VideoCapture(0);
rec=cv2.createLBPHFaceRecognizer();
rec.load("recognizer/trainningData.yml")
font=cv2.cv.InitFont(cv2.cv.CV_FONT_HERSHEY_COMPLEX,0.4,1,0,1)

def getProfile(id):
    conn=sqlite3.connect("FaceBase.db")
    cmd="SELECT * FROM People WHERE ID="+str(id)
    cursor=conn.execute(cmd)
    profile=None
    for row in cursor:
        profile=row
    conn.close()
    return profile

while(True):
    ret,img=cam.read();
    gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    faces=faceDetect.detectMultiScale(gray,1.3,5);
    for(x,y,w,h) in faces:
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
        id,conf=rec.predict(gray[y:y+h,x:x+w])
        profile=getProfile(id)
        if(profile!=None):
            cv2.cv.PutText(cv2.cv.fromarray(img),"Name : "+str(profile[1]),(x,y+h+20),font,(0,255,0));
            cv2.cv.PutText(cv2.cv.fromarray(img),"Age : "+str(profile[2]),(x,y+h+45),font,(0,255,0));
            cv2.cv.PutText(cv2.cv.fromarray(img),"Gender : "+str(profile[3]),(x,y+h+70),font,(0,255,0));
            cv2.cv.PutText(cv2.cv.fromarray(img),"Criminal Records : "+str(profile[4]),(x,y+h+95),font,(0,0,255));
    cv2.imshow("Face",img);
    if(cv2.waitKey(1)==ord('q')):
        break;
cam.release()
cv2.destroyAllWindows()

Source: https://www.youtube.com/watch?v=1Jz24sVsLE4&list=PLnjEM1fs09cGGjdCLSue8Kw7GmWDhGlMh

This part takes photos of faces,converts them to grayScale format,then it compares the photos with the photos from database.When match found it shows all the information on the screen.Without the trainer proper face recognition is not possible.

Output:

My Thoughts

The face recognition system is far away from perfect.This is not nearly as good as a retina scan or so.And the main problem is it also sometimes misidentifies people.It also becomes less predictable to detect a face from a low resolution mage or footage.This thing only works form a certain angle,from a side view of a person the face detection fails most of the time.Even with some facial expression i.e. smile, face recognition may not be proper.

Though it's true that this system has all it's problems , it can be very useful in certain situations.When dealing with crowd , where direct contact with every person is not possible.In this case the more convenient ways like fingerprint or biometric scan is not possible.And another truth is that due to the use of face recognising system in surveillance camera crime in certain places , robbery in banks has been decreased .This technology if properly developed can be a very helpful one for mankind.

Build a simple python web crawler

pranay749254 — Sun, 20 Aug 2017 03:41:55 +0000

What is a Web Crawler?

Web crawler is an internet bot that is used for web indexing in World Wide Web.All types of search engines use web crawler to provide efficient results.Actually it collects all or some specific hyperlinks and HTML content from other websites and preview them in a suitable manner.When there are huge number of links to crawl , even the largest crawler fails.For this reason search engines early 2000 were bad at providing relevant results,but now this process has improved much and proper results are given in an instant

Python Web Crawler

The web crawler here is created in python3.Python is a high level programming language including object-oriented, imperative, functional programming and a large standard library.
For the web crawler two standard library are used - requests and BeautfulSoup4. requests provides a easy way to connect to world wide web and BeautifulSoup4 is used for some particular string operations.

Example Code

import requests
from bs4 import BeautifulSoup
def web(page,WebUrl):
    if(page>0):
        url = WebUrl
        code = requests.get(url)
        plain = code.text
        s = BeautifulSoup(plain, "html.parser")
        for link in s.findAll('a', {'class':'s-access-detail-page'}):
            tet = link.get('title')
            print(tet)
            tet_2 = link.get('href')
            print(tet_2)
web(1,'http://www.amazon.in/s/ref=s9_acss_bw_cts_VodooFS_T4_w?rh=i%3Aelectronics%2Cn%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031%2Cp_98%3A10440597031%2Cp_36%3A1500000-99999999&bbn=1805560031&rw_html_to_wsrp=1&pf_rd_m=A1K21FY43GMZF8&pf_rd_s=merchandised-search-3&pf_rd_r=2EKZMFFDEXJ5HE8RVV6E&pf_rd_t=101&pf_rd_p=c92c2f88-469b-4b56-936e-0e65f92eebac&pf_rd_i=1389432031')

Output:

C:\Python34\python.exe C:/Users/Babuya/PycharmProjects/Youtube/web_cr.py
Apple iPhone 6 (Gold, 32GB)
http://www.amazon.in/Apple-iPhone-6-Gold-32GB/dp/B0725RBY9V
OnePlus 5 (Slate Gray 6GB RAM + 64GB memory)
http://www.amazon.in/OnePlus-Slate-Gray-64GB-memory/dp/B01NAKTR2H
OnePlus 5 (Midnight Black 8GB RAM + 128GB memory)
http://www.amazon.in/OnePlus-Midnight-Black-128GB-memory/dp/B01MXZW51M
Apple iPhone 6 (Space Grey, 32GB)
http://www.amazon.in/Apple-iPhone-Space-Grey-32GB/dp/B01NCN4ICO
OnePlus 5 (Soft Gold, 6GB RAM + 64GB memory)
http://www.amazon.in/OnePlus-Soft-Gold-64GB-memory/dp/B01N1TYZR2
Mi Max 2 (Black, 64 GB)
http://www.amazon.in/Mi-Max-Black-64-GB/dp/B073VLGL5Y
Moto G5 Plus (32GB, Fine Gold)
http://www.amazon.in/Moto-Plus-32GB-Fine-Gold/dp/B071ZZ8N5Y
Apple iPhone SE (Space Grey, 32GB)
http://www.amazon.in/Apple-iPhone-SE-Space-Grey/dp/B071DF166C
Honor 8 Pro (Blue, 6GB RAM + 128GB Memory)
http://www.amazon.in/Honor-Pro-Blue-128GB-Memory/dp/B01N4FMUFH
Apple iPhone 7 (Black, 32GB)
http://www.amazon.in/Apple-iPhone-7-Black-32GB/dp/B01LZKSVRB
BlackBerry KEYone (LIMITED EDITION BLACK)
http://www.amazon.in/BlackBerry-KEYone-LIMITED-EDITION-BLACK/dp/B073ZLLVQ9
Apple iPhone SE (Gold, 32GB)
http://www.amazon.in/Apple-iPhone-SE-Gold-32GB/dp/B071RC52N6
Apple iPhone SE (Rose Gold, 32GB)
http://www.amazon.in/Apple-iPhone-SE-Rose-Gold/dp/B06ZXWWD6R
Apple iPhone 6s (Space Grey, 32GB)
http://www.amazon.in/Apple-iPhone-Space-Grey-32GB/dp/B01LX3A7CC
Samsung Galaxy J7 Max (Gold, 32GB)
http://www.amazon.in/Samsung-Galaxy-J7-Max-Gold/dp/B073PWKTRS
Honor 8 Pro (Black, 6GB RAM + 128GB Memory)
http://www.amazon.in/Honor-Pro-Black-128GB-Memory/dp/B01MQXNY1L
Samsung Galaxy J7 Max (Black, 32GB)
http://www.amazon.in/Samsung-Galaxy-J7-Max-Black/dp/B073PWDMHD
OnePlus 3T (Soft Gold, 6GB RAM + 64GB memory)
http://www.amazon.in/OnePlus-3T-Soft-Gold-memory/dp/B01FM7J3NA
Apple iPhone 6s (Gold, 32GB)
http://www.amazon.in/Apple-iPhone-6s-Gold-32GB/dp/B01M0CJNVL
Apple iPhone 6s (Rose Gold, 32GB)
http://www.amazon.in/Apple-iPhone-Rose-Gold-32GB/dp/B01LXF3SP9
Samsung Galaxy C7 Pro (Navy Blue, 64GB)
http://www.amazon.in/Samsung-Galaxy-Navy-Blue-64GB/dp/B01LXMHNMQ
Samsung J7 Prime 32GB ( Gold ) 4G VoLTE
http://www.amazon.in/Samsung-J7-Prime-32GB-VoLTE/dp/B06Y3HFZBQ
Vivo V5s (Matte Black) with Offers
http://www.amazon.in/Vivo-V5s-Matte-Black-Offers/dp/B071P2FNF2
Vivo V5s (Crown Gold) with Offers
http://www.amazon.in/Vivo-V5s-Crown-Gold-Offers/dp/B071VT6RG2

Here this crawler collects all the product headings and respective links of the products pages from a page of amazon.in . User just need to specify what kind of data or links to be crawled.Though the main use of web crawler is in search engines,this way it can also be used to collect some useful information.
Here all the HTML of the page is fetched using requests in plain text form.Then it is converted into a BeautifulSoup object.From that object all title and href with class s-access-detail-page is accessed.That's all how this basic web crawler works.