IOT

Face Recognition Based Attendance System using ESP32 CAM

In an increasingly digital world, protecting confidential information is becoming more difficult day by day. Passwords or pins are not enough to secure our information. To fulfill, this requirement Biometrics is coming into the picture. Biometrics refers to a person’s unique identity. Finger scan, iris scan, and hand scan face recognition are all distinct biometrics. It’s nearly impossible to tamper with biometrically secured or protected equipment.

In this blog, we will discuss how to design a Face Recognition Based Attendance System using ESP32 CAM. We’ll also learn about the ESP32 cam board’s specifications and general hardware.

Proposed system:

The Face Recognition Based Attendance System using the ESP32 CAM Module is discussed in this blog.  This application will be built with OpenCV and Visual Studio. OpenCV is an open-sourced image processing library that is widely used in both industry and research & development. Visual Studio is a Microsoft integrated development environment (IDE) for various forms of software development. It includes completion tools, compilers, and other capabilities to make the software development process easier.

We will learn how to make a Face Recognition Based Attendance system using the ESP32 CAM and Python in this project. The primary heavy application will run on the server, which will be our computer, or a Raspberry Pi can be used as a server. We will not only detect the person in this attendance system, but we will also store the person’s information in a Microsoft Excel file.

The tutorial also covers features, pin descriptions, and how to program the ESP32 Camera Module with the FTDI Module. The Arduino IDE for the ESP32 Camera Module will also be installed. We’ll also update the firmware before moving on to the Face Recognition section. Because the Face Recognition software is built in the Python programming language, we’ll need to install Python and its dependencies.

Introduction to ESP32-Cam:

The ESP32 Based Camera Module was developed by AI-Thinker.  The controller contains a Wi-Fi + Bluetooth/BLE chip and is powered by a 32-bit CPU. It has a 520 KB internal SRAM and an external 4M PSRAM. UART, SPI, I2C, PWM, ADC, and DAC are all supported by its GPIO Pins.

The module is compatible with the OV2640 Camera Module, which has a camera resolution of 1600 x 1200 pixels. A 24-pin gold plated connector links the camera to the ESP32 CAM Board. A 4GB SD Card can be used on the board. The photographs captured are saved on the SD Card.

Because the ESP32-CAM lacks a USB port, you’ll need an FTDI programmer to upload code via the U0R and U0T pins (serial pins).

ESP32-CAM Features:

  • The smallest 802.11b/g/n Wi-Fi BT SoC module
  • Low power 32-bit CPU, can also serve the application processor
  • Up to 160MHz clock speed, summary computing power up to 600 DMIPS
  • Built-in 520 KB SRAM, external 4MPSRAM
  • Supports UART/SPI/I2C/PWM/ADC/DAC
  • Support OV2640 and OV7670 cameras, built-in flash lamp
  • Support image WiFI upload
  • Support TF card
  • Supports multiple sleep modes
  • Embedded Lwip and FreeRTOS
  • Supports STA/AP/STA+AP operation mode
  • Support Smart Config/AirKiss technology
  • Support for serial port local and remote firmware upgrades (FOTA)

ESP32-CAM FTDI Connection

  • There is no programmer chip on the PCB. So, any form of USB-to-TTL Module can be used to program this board. FTDI Modules based on the CP2102 or CP2104 chip, or any other chip, are widely accessible.
  • Connect the FTDI Module to the ESP32 CAM Module as shown below.
ESP32 CAM FTDI Module Connection
ESP32-CAMFTDI Programmer
GNDGND
5VVCC
U0RTX
U0TRX
GPIO0GND

Connect the ESP32’s 5V and GND pins to the FTDI Module’s 5V and GND. Connect the Rx to UOT and the Tx to UOR Pin in the same way. The most crucial thing is that you must connect the IO0 and GND pins. The device will now be in programming mode. You can remove it once the programming is completed.

Project PCB Gerber File & PCB Ordering Online

If you don’t want to put the circuit together on a breadboard and instead prefer a PCB. EasyEDA’s online Circuit Schematics & PCB Design tool was used to create the PCB Board for the ESP32 CAM Board. The PCB appears as seen below.

The Gerber File for the PCB is given below. You can simply download the Gerber File and order the PCB from https://www.nextpcb.com/

Download Gerber File: ESP32-CAM Multipurpose PCB

Now you can visit the NextPCB official website by clicking here: https://www.nextpcb.com/. So you will be directed to the NextPCB website.

  • You can now upload the Gerber File to the Website and place an order. The PCB quality is excellent. That is why the majority of people entrust NextPCB with their PCB and PCBA needs.
  • The components can be assembled on the PCB Board.

Installing ESP32CAM Library

Another streaming process will be used instead of the general ESP webserver example. As a result, another ESPCAM library is required. On the ESP32 microcontroller, the esp32cam library provides an object-oriented API for using the OV2640 camera. It’s an esp32-camera library wrapper.

Download the zip library as shown in the image from the following Github Link

After downloading, unzip the library and place it in the Arduino Library folder. To do so, follow the instructions below:

Open Arduino -> Sketch -> Include Library -> Add .ZIP Library… -> Navigate to downloaded zip file -> add

Source Code/Program for ESP32 CAM Module

Here is a source code for Face Recognition Based Attendance System using ESP32 CAM & OpenCV. Copy the code and paste it in the Arduino IDE.

#include <WebServer.h>
#include <WiFi.h>
#include <esp32cam.h>
const char* WIFI_SSID = “ssid”;
const char* WIFI_PASS = “password”;
WebServer server(80);
static auto loRes = esp32cam::Resolution::find(320, 240);
static auto midRes = esp32cam::Resolution::find(350, 530);
static auto hiRes = esp32cam::Resolution::find(800, 600);
void serveJpg()
{
auto frame = esp32cam::capture();
if (frame == nullptr) {
Serial.println(“CAPTURE FAIL”);
server.send(503, “”, “”);
return;
}
Serial.printf(“CAPTURE OK %dx%d %db\n”, frame->getWidth(), frame->getHeight(),
static_cast<int>(frame->size()));
server.setContentLength(frame->size());
server.send(200, “image/jpeg”);
WiFiClient client = server.client();
frame->writeTo(client);
}
void handleJpgLo()
{
if (!esp32cam::Camera.changeResolution(loRes)) {
Serial.println(“SET-LO-RES FAIL”);
}
serveJpg();
}
void handleJpgHi()
{
if (!esp32cam::Camera.changeResolution(hiRes)) {
Serial.println(“SET-HI-RES FAIL”);
}
serveJpg();
}
void handleJpgMid()
{
if (!esp32cam::Camera.changeResolution(midRes)) {
Serial.println(“SET-MID-RES FAIL”);
}
serveJpg();
}
void setup(){
Serial.begin(115200);
Serial.println();
{
using namespace esp32cam;
Config cfg;
cfg.setPins(pins::AiThinker);
cfg.setResolution(hiRes);
cfg.setBufferCount(2);
cfg.setJpeg(80);
bool ok = Camera.begin(cfg);
Serial.println(ok ? “CAMERA OK” : “CAMERA FAIL”);
}
WiFi.persistent(false);
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASS);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
}
Serial.print(“http://”);
Serial.println(WiFi.localIP());
Serial.println(” /cam-lo.jpg”);
Serial.println(” /cam-hi.jpg”);
Serial.println(” /cam-mid.jpg”);
server.on(“/cam-lo.jpg”, handleJpgLo);
server.on(“/cam-hi.jpg”, handleJpgHi);
server.on(“/cam-mid.jpg”, handleJpgMid);
server.begin();
}
void loop()
{
server.handleClient();
}

You must make a little adjustment to the code before uploading it. Change the SSID and password variables to match the WiFi network you’re using.

Compile the code and upload it to the ESP32 CAM Board. However, you must follow a few steps each time you post.

  • When you push the upload button, make sure the IO0 pin is shorted to the ground.
  • If you notice dots and dashes during uploading, immediately press the reset button.
  • Remove the I01 pin shorting with Ground and push the reset button one more after the code has been uploaded.
  • If the output is still not the Serial monitor, push the reset button once again.

Now you can see a similar output as in the image below.

Here, copy the IP address visible, we will be using it to edit the URL in python code

Installing Visual Studio

We now need to install Visual Studio in order to continue. We’re doing this because we’ll need certain dependencies to install the libraries later in this tutorial. Visit the Visual Studio website to get started. Then, get the most recent community version.

  • Once the software has been downloaded and installed, a welcome installation screen will open; now select the community version.
  • After that, choose Desktop development with C++ from a list of options. Select the boxes on the right-hand side as shown in the image below.
  • After you’ve made your selection, click Install/Modify in the lower right corner. Since it will download large files, this process will take some time.
  • It will prompt you to restart your computer once it has been installed. As a result, that will happen automatically.
  • Now open and extract the downloaded zip folder from the URL below.

Download: ATTENDANCE

  • Now open the command prompt and reach the same directory.
  • In the command prompt write the following command:
pip install -r requirements.txt
  • And then press enter. All required items will be downloaded and installed.
  • In the image folder, we must now add the users who must be detected (this is inside the zip folder that you downloaded).

Python Code for Face Recognition Attendance System

Now copy the code in face_detection.py. We also need to edit it.

import pandas as pd
import cv2
import urllib.request
import numpy as np
import os
from datetime import datetime
import face_recognition

path = r’D:\python\attendace\attendace\image_folder’
url=’http://192.168.231.162/cam-hi.jpg’
##”’cam.bmp / cam-lo.jpg /cam-hi.jpg / cam.mjpeg ”’

if ‘Attendance.csv’ in os.listdir(os.path.join(os.getcwd(),’attendace’)):
print(“there iss..”)
os.remove(“Attendance.csv”)
else:
df=pd.DataFrame(list())
df.to_csv(“Attendance.csv”)


images = []
classNames = []
myList = os.listdir(path)
print(myList)
for cl in myList:
curImg = cv2.imread(f'{path}/{cl}’)
images.append(curImg)
classNames.append(os.path.splitext(cl)[0])
print(classNames)


def findEncodings(images):
encodeList = []
for img in images:
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
encode = face_recognition.face_encodings(img)[0]
encodeList.append(encode)
return encodeList


def markAttendance(name):
with open(“Attendance.csv”, ‘r+’) as f:
myDataList = f.readlines()
nameList = []
for line in myDataList:
entry = line.split(‘,’)
nameList.append(entry[0])
if name not in nameList:
now = datetime.now()
dtString = now.strftime(‘%H:%M:%S’)
f.writelines(f’\n{name},{dtString}’)


encodeListKnown = findEncodings(images)
print(‘Encoding Complete’)

#cap = cv2.VideoCapture(0)

while True:
#success, img = cap.read()
img_resp=urllib.request.urlopen(url)
imgnp=np.array(bytearray(img_resp.read()),dtype=np.uint8)
img=cv2.imdecode(imgnp,-1)
# img = captureScreen()
imgS = cv2.resize(img, (0, 0), None, 0.25, 0.25)
imgS = cv2.cvtColor(imgS, cv2.COLOR_BGR2RGB)

facesCurFrame = face_recognition.face_locations(imgS)
encodesCurFrame = face_recognition.face_encodings(imgS, facesCurFrame)

for encodeFace, faceLoc in zip(encodesCurFrame, facesCurFrame):
matches = face_recognition.compare_faces(encodeListKnown, encodeFace)
faceDis = face_recognition.face_distance(encodeListKnown, encodeFace)
# print(faceDis)
matchIndex = np.argmin(faceDis)

if matches[matchIndex]:
name = classNames[matchIndex].upper()
# print(name)
y1, x2, y2, x1 = faceLoc
y1, x2, y2, x1 = y1 * 4, x2 * 4, y2 * 4, x1 * 4
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.rectangle(img, (x1, y2 – 35), (x2, y2), (0, 255, 0), cv2.FILLED)
cv2.putText(img, name, (x1 + 6, y2 – 6), cv2.FONT_HERSHEY_COMPLEX, 1, (255, 255, 255), 2)
markAttendance(name)

cv2.imshow(‘Webcam’, img)
key=cv2.waitKey(5)
if key==ord(‘q’):
break
cv2.destroyAllWindows()
cv2.imread
  • To begin, we must replace the URL variable in the code with the URL we copied from the Arduino serial monitor earlier.
  • Second, change the path variable in the code to the image folder’s path.
  • Now we are good to go. So run the code & Stand in front of the ESP32 Camera by showing the face directly to Camera.
  • Thus your face has been detected successfully.
  • Whenever the image of restored members is brought in front of the camera, the face is identified and attendance is recorded.
  • To close, press ‘q’.
  • Open Attendace.csv, which is located in the current working directory. Here you will find all of the information on who was detected and when.

Conclusion

I hope all of you had understand how to design an ESP32 CAM Face Recognition Based Attendance System for detecting the students or attendees and also storing their attendance in an excel file. We MATHA ELECTRONICS will be back soon with more interesting blogs.

Leave a Reply

Your email address will not be published.