• Skip to main content
  • Skip to primary sidebar

RNTLab.com

The Ultimate Shortcut to Learn Electronics and Programming with Open Source Hardware and Software

  • Courses
  • Forum
    • Forum
    • Ask Question
  • Shop
  • Account
  • Blog
  • Login

Standalone ESP32 cam face detection

Q&A Forum › Category: ESP32 › Standalone ESP32 cam face detection
0 Vote Up Vote Down
Adam Webb asked 5 years ago

(Totally new to ESP32 but experience Arduino and other programmer.)
I just brought your ESP32-CAM-Projects-eBook in the hope it could show me how to use the ESP32 CAM to activate an IO pin on face detection. (A question that is surprisingly hard to find an answer to online.)
After an initial quick look I’ve decided that it probably can’t do it, as it seems all face detection is done by the browser – ie  the ESP32 can’t do any face detection itself.
Is that a correct conclusion?
(Was hoping to be able to turn the digits on a large digital clock on only when it was looked at.)

7 Answers
0 Vote Up Vote Down
Sara Santos Staff answered 5 years ago

Hi Adam.
In our examples, the face detection runs in the browser.
The ESP32 can do face recognition and detection. However, it is not easy to enroll and save faces to be detected later in a different sketch to activate something. Additionally, face recognition on the ESP32 is a very demanding task.
We have an example to trigger an action when it detects an enrolled face on page 294 (Module 5). However, because the face recognition runs in the web browser, you would need to have a browser opened for it to work – which is not ideal.
What you’re asking for that’s also something that we wanted to do, but we couldn’t find an easy answer yet.
Regards,
Sara

0 Vote Up Vote Down
Adam Webb answered 5 years ago

I don’t require face recognition, only face detection as I don’t care who’s face it is.
I want a solution that is purely ESP32 due to both cost and complexity reasons. (Plus the camera image will never actually be viewed and is irrelevant.)

0 Vote Up Vote Down
Sara Santos Staff answered 5 years ago

Now I better understand what you want to do and I think it is possible with the ESP32-CAM.
However, we don’t have an example for what you’re looking for.
I’ve found this tutorial that might help you setup face detection without the need for a web based application:
techtutorialsx.com/2020/06/13/esp32-camera-face-detection/
Regards,
Sara
 
 

0 Vote Up Vote Down
Adam Webb answered 5 years ago

Thanks Sara, that looks really good.
Will try it out in a few days and let you know how it goes.
Thanks,
Adam.

0 Vote Up Vote Down
Sara Santos Staff answered 5 years ago

Great!
I’ll be waiting.
Regards,
Sara

0 Vote Up Vote Down
Adam Webb answered 5 years ago

It works!
As someone said in a comment, dl_lib_free() doesn’t seem to exist, so I replaced it with free().
Used the flash LED as my ‘face detected’ pin as its easy to see when looking at the camera.
Code:

#include "esp_camera.h"
#include "fd_forward.h"

// CAMERA_MODEL_AI_THINKER
#define PWDN_GPIO_NUM     32
#define RESET_GPIO_NUM    -1
#define XCLK_GPIO_NUM      0
#define SIOD_GPIO_NUM     26
#define SIOC_GPIO_NUM     27
#define Y9_GPIO_NUM       35
#define Y8_GPIO_NUM       34
#define Y7_GPIO_NUM       39
#define Y6_GPIO_NUM       36
#define Y5_GPIO_NUM       21
#define Y4_GPIO_NUM       19
#define Y3_GPIO_NUM       18
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    25
#define HREF_GPIO_NUM     23
#define PCLK_GPIO_NUM     22


#define FACE_DETECT_PIN    4          // Pin for face detect output signal
#define FACE_DETECT_OFF    LOW        // Pin state when no face detected
#define FACE_DETECT_ON     HIGH       // Pin state when face detected
#define FACE_DETECT_TIME   500ul      // Duration of face detect signal, after no face detection


 
mtmn_config_t mtmn_config = {0};

bool initCamera() {
 
  camera_config_t config;
 
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_QVGA;
  config.jpeg_quality = 10;
  config.fb_count = 1;
 
  esp_err_t result = esp_camera_init(&config);
 
  return (result == ESP_OK);
}

 
void setup() {
  Serial.begin(115200);

  Serial.printf("Initializing...\n");

  if (!initCamera()) {
 
    Serial.printf("Failed to initialize camera...\n");
    return;
  }
 
  mtmn_config = mtmn_init_config();

  pinMode(FACE_DETECT_PIN, OUTPUT);
  digitalWrite(FACE_DETECT_PIN, FACE_DETECT_OFF);

  Serial.printf("Ready\n");
}

 
void loop() {
  static unsigned long loopcnt = 0ul;
  static unsigned long FaceDetectMillis = millis();
  static bool FaceDetected = false;
  static unsigned int detections = 0;

  loopcnt++;
//  if (loopcnt % 100 == 0) 
    Serial.printf("Loop %lu\n",loopcnt);
  
  camera_fb_t * frame;
  frame = esp_camera_fb_get();
 
  dl_matrix3du_t *image_matrix = dl_matrix3du_alloc(1, frame->width, frame->height, 3);
  fmt2rgb888(frame->buf, frame->len, frame->format, image_matrix->item);
 
  esp_camera_fb_return(frame);
 
  box_array_t *boxes = face_detect(image_matrix, &mtmn_config);
 
  if (boxes != NULL) {
    detections = detections+1;
    Serial.printf("Faces detected %d times\n", detections);

    digitalWrite(FACE_DETECT_PIN, FACE_DETECT_ON);
    FaceDetectMillis = millis();
    FaceDetected = true;
 
//    dl_lib_free(boxes->score);
//    dl_lib_free(boxes->box);
//    dl_lib_free(boxes->landmark);
//    dl_lib_free(boxes);
    free(boxes->score);
    free(boxes->box);
    free(boxes->landmark);
    free(boxes);
  }
  else if (FaceDetected && (millis() - FaceDetectMillis >= FACE_DETECT_TIME))
  {
    digitalWrite(FACE_DETECT_PIN, FACE_DETECT_OFF);
    FaceDetected = false;
  }
 
  dl_matrix3du_free(image_matrix);
}

 
Many thanks,
 
Adam.

0 Vote Up Vote Down
Sara Santos Staff answered 5 years ago

That’s great. 
Thank you so much for sharing your code.
I’ll mark this issue as resolved. If you need further help, you just need to open a new question in our forum.
Regards,
Sara

Primary Sidebar

Login to Ask or Answer Questions

This Forum is private and it’s only available for members enrolled in our Courses.

Login »

Latest Course Updates

  • [eBook Updated] Learn Raspberry Pi Pico/Pico W with MicroPython eBook – Version 1.2 May 26, 2025
  • [New Edition] Build ESP32-CAM Projects eBook – 2nd Edition April 16, 2025

You must be logged in to view this content.

Contact Support - Refunds - Privacy - Terms - MakerAdvisor.com - Member Login

Copyright © 2013-2025 · RandomNerdTutorials.com · All Rights Reserved

Insert/edit link

Enter the destination URL

Or link to existing content

    No search term specified. Showing recent items. Search or use up and down arrow keys to select an item.