Optimizations of the rescaleImage, rgbToGray and _rgb_write methods in motionDetect.cpp

[JBarnaky]

Overall, successful – my goal was to eliminate sluggish float calculations using fixed point, which were slow even on S3, eliminate ESP-heavy division operations and simplify loops overloaded with unnecessary work by making them more predictable for the compiler and cache. The changes were designed as in-place replacements of the original methods without additional dependencies. My ESP was able to be flashed and everything worked more or less responsively – although extensive testing is required, as I may have missed something.

https://github.com/JBarnaky/ESP32-CAM

Structure: 3 .cpp files (+ motionDetect.cpp, which I flashed) with unit tests and simple benchmarks, which I pre-tested in Visual Studio 2022 and on Win 11. The file name includes the name of the file and the refactored method itself, which comes after the commented block with the original implementation in the code (the redundant if checks at the very beginning can be removed). Perhaps my work will be useful to you.

Upd.
Added [motionDetect_checkMotion.cpp]

[s60sc]

Thanks, I'll take a look over the next few days

[JBarnaky]

Added:
motionDetect_checkMotion.cpp
utils_startWifi.cpp
utils_pingTimeout.cpp
avi_buildAviHdr.cpp
mjpeg2sd_processFrame.cpp
mjpeg2sd_closeAvi.cpp

[s60sc]

On S3, rescaleImage() was significantly faster but the other mods had similar performance.
Did you have different results - it may be that the compiler does its own optimisations.

[JBarnaky]

Visual Studio usually does O2 optimization, unfortunately it doesn't know about Arduino IDE.
In fact, I tested this on my old ESP32-CAM (there may be differences on the S3, but fixed-point should be universally faster than float). In places where I replaced floating-point—which worked particularly poorly with doubles—with fixed-point, and where I removed divisions (which work extremely poorly with doubles worse with floats, and to some extent, about 2+x bad, with ints), the performance and responsiveness gains were noticeable in frequently repeated operations with pixels, frames, or buffers (motionDetect.cpp and mjpeg2sd.cpp). In some places, I explicitly let the compiler know what was happening inside the method using standard syntax constructs for more effective compiler optimizations, but this requires disassembling. Elsewhere, I primarily focused on improving cache locality by reducing fragmentation (in some places, you can still explicitly align buffers using "alignas(32)") and simplifying loops by precomputing and reducing the number or complexity of operations—the gains here are smaller, but they're still there due to the reduction in work within the loop. Network recovery logic and WiFi scan in utils.cpp can disrupt real-time capture/recording because it's heavy and blocking.

[JBarnaky]

In less frequently used code sections with floats and without division, to ensure everything is processed in single-precision floating point, you can replace:
float a, b
b = a * 10.0; // double precision
with
float a, b;
b = a * 10.0f; // single precision

[s60sc]

I tend not to use original esp32 now, either using esp32s3 (which I recommend for this app) or esp32c3 for lower performance apps. The S3 has single precision FPU h/w which would reduce performance deficit vs fixed point. Double is still software.
As a lot of users still use original ESP32 I will look further into incorporating your proposals, or you could submit a pull request.

[s60sc]

If you want to do a pull request let me know as I would need to update beforehand with some unpublished changes