what!?!?!? FPS: 17.00 on 1080p

[josef2600]

if i am not hallucinating right now, i got 17 FPS on config.frame_size = FRAMESIZE_FHD; !!! what? from 7 on 20mhz, to 17 on 15MHz!
need more testing. maybe the sensor is hallucinating! i have to save a pic.

[josef2600]

so far the picture is corrupted. working on it.

[josef2600]

this is the proof of concept. it shows that esp32-s3 can handle at least 17 FPS. so we have to fix the config of the camera.

#include "esp_camera.h"
#include "SD_MMC.h"

#define CAM_BOARD "CAMERA_MODEL_ESP32S3_EYE_FREENOVE"
#define PWDN_GPIO_NUM -1
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 15
#define SIOD_GPIO_NUM 4
#define SIOC_GPIO_NUM 5
#define Y2_GPIO_NUM 11
#define Y3_GPIO_NUM 9
#define Y4_GPIO_NUM 8
#define Y5_GPIO_NUM 10
#define Y6_GPIO_NUM 12
#define Y7_GPIO_NUM 18
#define Y8_GPIO_NUM 17
#define Y9_GPIO_NUM 16
#define VSYNC_GPIO_NUM 6
#define HREF_GPIO_NUM 7
#define PCLK_GPIO_NUM 13

#define SD_MMC_CLK 39 
#define SD_MMC_CMD 38
#define SD_MMC_D0 40
#define SD_MMC_D1 42
#define SD_MMC_D2 1
#define SD_MMC_D3 2

#define MOUNT_POINT "/sdcard"
#define FILE_NAME "/sdcard/capture.avi"
#define sdmmcFreq 40000000
#define formatIfMountFailed false


// stable = 20Mhz - 
// works stable but slow =  10-6-8-9
// unstable all go over = 24-23-22-21
#define camera_xclk_freq_hz 15*1000*1000


camera_config_t config;
FILE* file;

extern void sd_card_initialing();






void setup()
{
  Serial.begin(115200);
  delay(500);
  while (!Serial)
  {
    delay(300);
  }
  
  config.ledc_channel = LEDC_CHANNEL_1;
  config.ledc_timer = LEDC_TIMER_1;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sccb_sda = SIOD_GPIO_NUM;
  config.pin_sccb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  
  config.xclk_freq_hz = camera_xclk_freq_hz;
  
  config.pixel_format = PIXFORMAT_JPEG;  /*!< Format of the pixel data: PIXFORMAT_ + YUV422|GRAYSCALE|RGB565|JPEG  */
  
  //FRAMESIZE_HD = 28fps - FRAMESIZE_XGA = 21fps - FRAMESIZE_P_HD = 7.5fps - FRAMESIZE_FHD = 6.6fps - FRAMESIZE_SVGA = 27.6fps
  //@ 20mhz stable => FRAMESIZE_P_FHD = FPS: 6.74 - FRAMESIZE_HD FPS: 25.42
  config.frame_size = FRAMESIZE_FHD; // FRAMESIZE_P_FHD  FRAMESIZE_HD
  //The quality setting doesn’t impact FPS significantly due to hardware compression.
  config.jpeg_quality = 10;  /*!< Quality of JPEG output. 0-63 lower means higher quality  */ 
  config.fb_location = CAMERA_FB_IN_PSRAM;  //CAMERA_FB_IN_PSRAM  CAMERA_FB_IN_DRAM
  config.grab_mode = CAMERA_GRAB_LATEST;
  
  //This specifies how many frame buffers should be allocated. More frame buffers can help with higher frame rates but use more memory.
  config.fb_count = 4; // no effect over 2 for fps
  
  //sd_card_initialing();
  
  
  esp_err_t err = esp_camera_init(&config);
  delay(300);
  
  if (err != ESP_OK)
  {
    Serial.printf("Camera init failed with error 0x%x\n", err);
    delay(50000);
  }
  
  sensor_t* senn = esp_camera_sensor_get();
  if (senn)
  {
    
    // Stop streaming
    senn->set_reg(senn, 0x3008, 0xFF, 0x42); // Enter software power-down mode
    delay(30);
    
    delay(30);
    
    senn->set_reg(senn, 0x3031, 0xFF, 8);   // Disable the internal regulator
    senn->set_reg(senn, 0x3039, 0xFF, 0x00); // Disable PLL bypass
    senn->set_reg(senn, 0x3034, 0xFF, 0x18); // 10-bit mode - 
    
    senn->set_reg(senn, 0x3035, 0xFF, 0x0F | ((1 & 0x0F) << 4)); // SYSCLK divider Bit[7:4]-  /2
    
    senn->set_reg(senn, 0x3037, 0xFF, (0 & 0x0F)); // PLL pre-divider 1,2,3,4,6,8 - PLL root divider Bypass -  /
    senn->set_reg(senn, 0x3036, 0xFF, 72 & 0xFF); //PLL multiplier Can be any integer for 4~127 and only even integer for 128~252 -  * 72
    
    //senn->set_reg(senn, 0x3108, 0xFF, (2 & 0x3) << 4 | 0x06); // PCLK root divider  -  /2
    senn->set_reg(senn, 0x3108, 0xFF, 16 | 0x06); // PCLK root divider pll_clki/2 - SCLK= pll_clki/4  SCLK2x = pll_clki/2 == /8
    
    senn->set_reg(senn, 0x3824, 0xFF, 1 & 0x1F); // DVP PCLK divider([4:0])  -   
    senn->set_reg(senn, 0x460C, 0xFF, 0x22); // VFIFO CTRL0C  -  DVP PCLK divider control by manual mode
    senn->set_reg(senn, 0x3103, 0xFF, 0x13); // SCCB SYSTEM clock from PLL  -  do not touch
    
    
    /*
    senn->set_reg(senn, 0x3031, 0xFF, 8);   // Disable the internal regulator
    senn->set_reg(senn, 0x3039, 0xFF, 0x00); // Disable PLL bypass
    senn->set_reg(senn, 0x3034, 0xFF, 0x1A); // 10-bit mode
    senn->set_reg(senn, 0x3035, 0xFF, 0x01 | ((3 & 0x0F) << 4)); // SYSCLK divider
    senn->set_reg(senn, 0x3036, 0xFF, 72 & 0xFF); // PLL multiplier
    senn->set_reg(senn, 0x3037, 0xFF, (2 & 0x0F)); // Pre-divider
    senn->set_reg(senn, 0x3108, 0xFF, (2 & 0x3) << 4 | 0x06); // PCLK root divider
    senn->set_reg(senn, 0x3824, 0xFF, 2 & 0x1F); // PCLK divider
    senn->set_reg(senn, 0x460C, 0xFF, 0x22); // Manual PCLK divider
    senn->set_reg(senn, 0x3103, 0xFF, 0x13); // System clock from PLL
    */
    
    delay(300);
    senn->set_reg(senn, 0x3008, 0xFF, 0x02); // Start streaming
    delay(300);
  }
  else
  {
    Serial.println("Failed to get sensor!");
    delay(50000);
  }
  
  Serial.println("setup complete.");
  delay(50);
    size_t free_psram = heap_caps_get_free_size(MALLOC_CAP_SPIRAM);
    // Print the free PSRAM size
    Serial.print("Free PSRAM: ");
    Serial.print(free_psram);
    Serial.println(" bytes");
}

uint32_t frame_count = 0;
uint32_t last_report_time = 0;


uint32_t current_time = 0;




void loop()
{
  
  camera_fb_t* fb = esp_camera_fb_get();  // Captures a frame from the camera.
  if (!fb)
  {
    Serial.println("Failed to capture frame");
    //return;
    delay(1000);
  }
  else
  {
    //fwrite(fb->buf, fb->len, 1, file);
    esp_camera_fb_return(fb);
    ++frame_count;
  }
  current_time = millis();
  
  //fwrite(fb->buf, fb->len, 1, file);
  /*The driver should handle the clock settings and timing automatically, allowing you to achieve 
      the target frame rate if your loop execution time is within the required per-frame time.*/
  
  esp_camera_fb_return(fb);//Releases the frame buffer RAM back to the camera driver.
  
  if (millis() - last_report_time >= 5000) // print every 5sec
  {
    size_t free_psram = heap_caps_get_free_size(MALLOC_CAP_SPIRAM);
    // Print the free PSRAM size
    Serial.print("Free PSRAM: ");
    Serial.print(free_psram);
    Serial.println(" bytes");
    
    
    float fps = frame_count / float ((current_time - last_report_time)/1000);
    Serial.printf("FPS: %.2f\n", fps);
    last_report_time = current_time;
    frame_count = 0;
    fflush(file);
  }
  delay(5);
}

just to say it can. and non the less, with 15MHz clock input. we shouldnt need the 24MHz clock, but then again, who knows? the datasheet is for shit!

[josef2600]

what i understand so far, is that the SYSCLK = 150MHz-200MHz , but as per my tests, it works for under 150mhz, and even for 100mhz. but over 200mhz, it will corrupt the image, clock by clock.
and as my understanding, it is the problem of choosing and setting the Format and frame rate. they are directly related to total FPS.

[josef2600]

if interested, i created an issue here:
espressif/arduino-esp32#10910

[rjsachse]

Great work mate looking forward to this. Wondering if would be better to use external 24M active crystal oscillator

[josef2600]

hi. thanks. that's not the problem. there is no need for 24MHz clock.
it has been few days that i am working with espressif-ide so i have access to the source code and i can change it. right now my problem is the ratio of the sensor. like, in 1080p, it is 16x9 they put this:

//  mw,   mh,  sx,  sy,   ex,   ey, ox, oy,   tx,   ty
{ 2560, 1440,   0, 240, 2623, 1711, 32, 16, 2844, 1488 }

where it means:

1.	mw, mh: Sensor image width and height.
2.	sx, sy: Start X and Y coordinates.
3.	ex, ey: End X and Y coordinates.
4.	ox, oy: Offset X and Y.
5.	tx, ty: Total X and Y size, including blanking intervals.

currently, i don't understand why such a big resolution is cropped for 1080p. or what is the correct numbers.
on the other note, the 30FPS is only possible with 8-bit res of ADC. bot, since we all have boards that wired base on 10bit, then we can have a maximum FPS of around 24.
i am searching and testing.

[josef2600]

for example, lets compare 2 images. first is my setting, check the resolution:

it say 1920x1080.

now the original setting from 1600x1200 :

the resolution say 1600x1200. but it is not! yes the picture is at that, but if we check the window of image, we can see it is way larger than that.
now, either i am crazy, or all the settings are wrong.
what do you think?