IOT

ESP32 CAM Based Face & Eyes Recognition System

In this project, we will design a Face and Eye Recognition System using an ESP32 CAM. This guide will teach you how to use a wireless video streaming method. The ESP32-CAM module, which is a tiny camera module with the ESP32-S chip, was utilized in this project. Apart from the OV2640 camera and many GPIOs for connecting peripherals, it also has a microSD card slot for storing photographs captured by the camera.

We’ll go through the ESP32 CAM’s features, pin descriptions, and how to program it with an FTDI module. Then, using the Arduino IDE, we’ll install the ESP32 CAM Webserver Library and upload the example code. We’ll also install the Python libraries that are required. We’ll go over the Face and Eyes Recognition Python code later.

This is an important tutorial because it will allow you to apply any type of image processing or machine learning on live video without having to write code in the Arduino IDE.

Hardware Required:

  • ESP32-CAM Board-AI-Thinker ESP32 Camera Module
  • FTDI Module-USB-to-TTL Converter Module
  • USB Cable-5V Mini-USB Data Cable
  • Jumper Wires-Female to Female Connectors

ESP32 CAM Module

The ESP32 Based Camera Module was developed by AI-Thinker.  The controller contains a Wi-Fi + Bluetooth/BLE chip and is powered by a 32-bit CPU. It has a 520 KB internal SRAM and an external 4M PSRAM. UART, SPI, I2C, PWM, ADC, and DAC are all supported by its GPIO Pins.

The module is compatible with the OV2640 Camera Module, which has a camera resolution of 1600 x 1200 pixels. A 24-pin gold plated connector links the camera to the ESP32 CAM Board. A 4GB SD Card can be used on the board. The photographs captured are saved on the SD Card.

ESP32-CAM Features 

  • The smallest 802.11b/g/n Wi-Fi BT SoC module.
  • Low power 32-bit CPU, can also serve the application processor.
  • Up to 160MHz clock speed, summary computing power up to 600 DMIPS.
  • Built-in 520 KB SRAM, external 4MPSRAM.
  • Supports UART/SPI/I2C/PWM/ADC/DAC.
  • Support OV2640 and OV7670 cameras, built-in flash lamp.
  • Support image WiFI upload.
  • Supports TF card.
  • Supports multiple sleep modes.
  • Embedded Lwip and FreeRTOS.
  • Supports STA/AP/STA+AP operation mode.
  • Support Smart Config/AirKiss technology.
  • Support for serial port local and remote firmware upgrades (FOTA).

ESP32-CAM FTDI Connection

There is no programmer chip on the PCB. So, any form of USB-to-TTL Module can be used to program this board. FTDI Modules based on the CP2102 or CP2104 chip, or any other chip, are widely accessible.

  • Connect the FTDI Module to the ESP32 CAM Module as shown below.
ESP32 CAM FTDI Module Connection
ESP32-CAMFTDI Programmer
GNDGND
5VVCC
U0RTX
U0TRX
GPIO0GND

Connect the ESP32’s 5V and GND pins to the FTDI Module’s 5V and GND. Connect the Rx to UOT and the Tx to UOR Pin in the same way. The most crucial thing is that you must connect the IO0 and GND pins. The device will now be in programming mode. You can remove it once the programming is completed.

Project PCB Gerber File & PCB Ordering Online

If you don’t want to put the circuit together on a breadboard and instead prefer a PCB. EasyEDA s online Circuit Schematics & PCB Design tool was used to create the PCB Board for the ESP32 CAM Board. The PCB appears as seen below.

ESP32 Cam Board PCB

The Gerber File for the PCB is given below. You can simply download the Gerber File and order the PCB from https://www.nextpcb.com/

Download Gerber File: ESP32-CAM Multipurpose PCB

Now you can visit the NextPCB official website by clicking here: https://www.nextpcb.com/. So you will be directed to the NextPCB website

  • You can now upload the Gerber File to the Website and place an order. The PCB quality is excellent. That is why the majority of people entrust NextPCB with their PCB and PCBA needs.
  • The components can be assembled on the PCB Board.

Installing ESP32CAM Library

Another streaming process will be used instead of the general ESP webserver example. As a result, another ESPCAM library is required. On the ESP32 microcontroller, the esp32cam library provides an object-oriented API for using the OV2640 camera. It’s an esp32-camera library wrapper.

Download the zip library as shown in the image from the following Github Link

After downloading, unzip the library and place it in the Arduino Library folder. To do so, follow the instructions below:

Open Arduino -> Sketch -> Include Library -> Add .ZIP Library… -> Navigate to downloaded zip file -> add

Arduino Source Code/program

The source code/program ESP32 CAM Gesture Controlled Mouse can be found in Library Example. So go to Files -> Examples -> esp32cam -> WifiCam.

  • You need to change your WiFi SSID and Password.
const char* ssid = “*********”;
const char* password = “*********”;
  • Now navigate to the tools section. After that, choose the ESP32 Board. Select ESP32Wroover Module from the list.
  • After that, connect the FTDI Module to your computer and choose the COM Port.
ESP32 CAM Getting Started
  • You can now upload the code. However, the IO0 pin should be shorted to the ground while uploading the code. Remove the shorting Jumper and hit the RESET pin once the code has been uploaded.
  • Similar output must be displayed in the Serial Monitor.
  • Copy the visible IP address; we’ll use it to edit the URL in Python code.

Python Installation & Source Code

In order for the live video stream to appear on our computer, we must develop a Python script that allows us to retrieve the video frames. The first step is to get Python installed. Go to python.org and download Python.

Install Python once downloading is completed.

Install NumPy, and OpenCV,  libraries from the command prompt.

  • type: pip install numpy and press enter. After the installation is done.
  • type: pip install opencv-python and press enter, close the command prompt.

Now open Idle code editor or any other python code editor.

import cv2
import urllib.request
import numpy as np
f_cas= cv2.CascadeClassifier(cv2.data.haarcascades+’haarcascade_frontalface_default.xml’)
eye_cascade=cv2.CascadeClassifier(cv2.data.haarcascades +’haarcascade_eye.xml’)
url=’http://192.168.1.58/cam-lo.jpg’
##”’cam.bmp / cam-lo.jpg /cam-hi.jpg / cam.mjpeg ”’
cv2.namedWindow(“Live Transmission”, cv2.WINDOW_AUTOSIZE)
while True:
img_resp=urllib.request.urlopen(url)
imgnp=np.array(bytearray(img_resp.read()),dtype=np.uint8)
img=cv2.imdecode(imgnp,-1)
gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
face=f_cas.detectMultiScale(gray,scaleFactor=1.1,minNeighbors=5)
for x,y,w,h in face:
cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),3)
roi_gray = gray[y:y+h, x:x+w]
roi_color = img[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(roi_gray)
for (ex,ey,ew,eh) in eyes:
cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
cv2.imshow(“live transmission”,img)
key=cv2.waitKey(5)
if key==ord(‘q’):
break

cv2.destroyAllWindows()

Update the URL variable in the above code with the IP address copied from the Arduino serial monitor. Then save it and execute it.

ESP32 CAM Face Recognition System

Following the execution of the code, a pop-up window titled “Live Transmission” appears, displaying the live video.

ESP32 CAM Face Recognition System
  • If it recognizes eyes, a red-colored rectangular box covering the entire face appears. Then, around the eyes, green-colored boxes appear.
  • Change the URL variable in the python code to url=’http://192.168.1.58/cam-hi.jpg’ for high-resolution live video transmission.
  • Save the code and run it again.

Benefits of Face Recognition System

The main benefit of this technique is that most of our algorithms that include image processing are quite difficult to execute on microcontrollers and require a lot of processing, slowing down the entire system.

Instead, we may use our computers to perform tough and time-consuming activities like image processing or artificial neural networks.

Conclusion

I hope you understand how to design an  ESP32 CAM Based Face & Eyes Recognition System.  We MATHA ELECTRONICS will be back soon with more informative blogs soon.

Leave a Reply

Your email address will not be published.