Jose/esp32i2s
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor ♻️: Refactored the audio processing and visualization tasks into separate cores, improved CPU usage monitoring, optimized memory usage, managed inter-core communication, and enhanced network functionality.

Browse Source 
- Refactor the audio processing and visualization tasks into separate cores, improve CPU usage monitoring, optimize memory usage, manage inter-core communication, and enhance network functionality.
- This code snippet provides a basic implementation of a piano note detection system using an Arduino. The system includes a setup phase where calibration and initialization are performed, along with serial communication for user interaction. The main loop is empty, as all the work is handled by separate tasks created on different cores.

The `setup()` function sets up the serial connection, initializes the piano note detector, and creates two separate tasks: `audioProcessingTask` and `visualizationTask`. These tasks handle the audio processing and visualization of the detected notes, respectively. The main loop runs in a paused state to allow for task execution on different cores.

The audio processing task (`audioProcessingTask`) reads analog signals from an I2S microphone (C2-C6), processes them using Fourier Transform, and detects note frequencies. It then updates a spectrum visualization and sends the results over the serial interface to the host PC for further analysis.

The visualization task (`visualizationTask`) receives the processed data from the audio processing task, visualizes the spectrum, and sends updates over the serial interface. The main loop in the `loop()` function is empty, as all the work is handled by these tasks.
This commit is contained in:
Jose
2025-04-25 21:13:59 +02:00
parent 5a8dc9c489
commit f500937067
2 changed files with 180 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
README.md
View File

@@ -170,8 +170,16 @@ This project is licensed under the MIT License - see the LICENSE file for detail
## Advanced Configuration
### Task Management
- Audio processing runs on Core 1
- Main loop on Core 0
- Audio processing task on Core 1:
- I2S sample reading
- Audio level tracking
- Note detection and FFT analysis
- Visualization task on Core 0:
- WebSocket communication
- Spectrum visualization
- Serial interface
- Network operations
- Inter-core communication via FreeRTOS queue
- Configurable priorities in `Config.h`
### Audio Pipeline
@@ -191,9 +199,14 @@ This project is licensed under the MIT License - see the LICENSE file for detail
## Performance Optimization
### CPU Usage
- Audio Processing: ~30% on Core 1
- Note Detection: ~20% on Core 1
- Visualization: ~10% on Core 0
- Core 1 (Audio Processing):
- I2S DMA handling: ~15%
- Audio analysis: ~20%
- FFT processing: ~15%
- Core 0 (Visualization):
- WebSocket updates: ~5%
- Visualization: ~5%
- Network handling: ~5%
### Memory Optimization
1. Buffer Size Selection:
@@ -276,3 +289,29 @@ This project is licensed under the MIT License - see the LICENSE file for detail
- `/data`: Additional resources
- `/test`: Unit tests
## Inter-Core Communication
### Queue Management
- FreeRTOS queue for audio data transfer
- 4-slot queue buffer
- Zero-copy data passing
- Non-blocking queue operations
- Automatic overflow protection
### Data Flow
1. Core 1 (Audio Task):
- Processes audio samples
- Performs FFT analysis
- Queues processed data
2. Core 0 (Visualization Task):
- Receives processed data
- Updates visualization
- Handles network communication
### Network Communication
- Asynchronous WebSocket updates
- JSON-formatted spectrum data
- Configurable update rate (50ms default)
- Automatic client cleanup
- Efficient connection management

200
src/main.cpp
View File

@@ -10,6 +10,16 @@
#include "NoteDetector.h"
#include "SpectrumVisualizer.h"
// Function declarations
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len);
void initWebServer();
void handleSerialCommands();
void printNoteInfo(const DetectedNote& note);
void initWiFi();
void audioProcessingTask(void *parameter);
void visualizationTask(void *parameter);
void sendSpectrumData();
// Static instances
static int16_t raw_samples[Config::SAMPLE_BUFFER_SIZE];
static AudioLevelTracker audioLevelTracker;
@@ -24,9 +34,36 @@ static uint32_t lastSpectrumPrintTime = 0;
static uint32_t lastWebUpdateTime = 0;
static bool showSpectrum = false;
// Task handles
TaskHandle_t audioTaskHandle = nullptr;
TaskHandle_t visualizationTaskHandle = nullptr;
// Queue for passing audio data between cores
QueueHandle_t audioQueue;
// Note names for display
const char* noteNames[] = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"};
void sendSpectrumData() {
if (ws.count() > 0 && !noteDetector.isCalibrating()) {
const auto& spectrum = noteDetector.getSpectrum();
String json = "[";
// Calculate bin range for 60-1100 Hz
// At 8kHz sample rate with 1024 FFT size:
// binFreq = index * (8000/1024) = index * 7.8125 Hz
// For 60 Hz: bin ≈ 8
// For 1100 Hz: bin ≈ 141
for (int i = 8; i <= 141; i++) {
if (i > 8) json += ",";
json += String(spectrum[i], 2);
}
json += "]";
ws.textAll(json);
}
}
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len) {
switch (type) {
case WS_EVT_CONNECT:
@@ -118,6 +155,76 @@ void initWiFi() {
Serial.println(WiFi.localIP());
}
// Audio processing task running on Core 1
void audioProcessingTask(void *parameter) {
while (true) {
size_t bytes_read = 0;
readI2SSamples(raw_samples, &bytes_read);
int samples_read = bytes_read / sizeof(int16_t);
// Update level tracking
for (int i = 0; i < samples_read; i++) {
audioLevelTracker.updateMaxLevel(raw_samples[i]);
}
// Only analyze if we have enough signal
int16_t currentMaxLevel = audioLevelTracker.getMaxLevel();
if (currentMaxLevel > Config::NOISE_THRESHOLD) {
// Analyze samples for note detection
noteDetector.analyzeSamples(raw_samples, samples_read);
// Send results to visualization task via queue
if (xQueueSend(audioQueue, &samples_read, 0) != pdTRUE) {
// Queue full, just skip this update
}
}
// Small delay to prevent watchdog trigger
vTaskDelay(1);
}
}
// Visualization and network task running on Core 0
void visualizationTask(void *parameter) {
while (true) {
int samples_read;
// Check if there's new audio data to process
if (xQueueReceive(audioQueue, &samples_read, 0) == pdTRUE) {
uint32_t currentTime = millis();
const auto& detectedNotes = noteDetector.getDetectedNotes();
// Update web clients with spectrum data
if (currentTime - lastWebUpdateTime >= 50) {
sendSpectrumData();
lastWebUpdateTime = currentTime;
}
// Show spectrum if enabled
if (showSpectrum &&
!noteDetector.isCalibrating() &&
currentTime - lastSpectrumPrintTime >= Config::DEBUG_INTERVAL_MS) {
SpectrumVisualizer::visualizeSpectrum(noteDetector.getSpectrum(), Config::FFT_SIZE);
lastSpectrumPrintTime = currentTime;
}
// Print detected notes at specified interval
if (currentTime - lastNotePrintTime >= Config::NOTE_PRINT_INTERVAL_MS) {
if (!detectedNotes.empty() && !noteDetector.isCalibrating()) {
SpectrumVisualizer::visualizeNotes(detectedNotes);
}
lastNotePrintTime = currentTime;
}
}
ws.cleanupClients();
handleSerialCommands();
// Small delay to prevent watchdog trigger
vTaskDelay(1);
}
}
void setup() {
Serial.begin(Config::SERIAL_BAUD_RATE);
while(!Serial) {
@@ -131,73 +238,38 @@ void setup() {
Serial.println("Piano Note Detection Ready (C2-C6)");
Serial.println("Press 'h' for help");
noteDetector.beginCalibration();
}
void sendSpectrumData() {
if (ws.count() > 0 && !noteDetector.isCalibrating()) {
const auto& spectrum = noteDetector.getSpectrum();
String json = "[";
// Calculate bin range for 60-1100 Hz
// At 8kHz sample rate with 1024 FFT size:
// binFreq = index * (8000/1024) = index * 7.8125 Hz
// For 60 Hz: bin ≈ 8
// For 1100 Hz: bin ≈ 141
for (int i = 8; i <= 141; i++) {
if (i > 8) json += ",";
json += String(spectrum[i], 2);
}
json += "]";
ws.textAll(json);
// Create queue for inter-core communication
audioQueue = xQueueCreate(4, sizeof(int));
if (audioQueue == nullptr) {
Serial.println("Failed to create queue!");
return;
}
// Create audio processing task on Core 1
xTaskCreatePinnedToCore(
audioProcessingTask,
"AudioTask",
Config::TASK_STACK_SIZE,
nullptr,
Config::TASK_PRIORITY,
&audioTaskHandle,
1 // Run on Core 1
);
// Create visualization task on Core 0
xTaskCreatePinnedToCore(
visualizationTask,
"VisualTask",
Config::TASK_STACK_SIZE,
nullptr,
1, // Lower priority than audio task
&visualizationTaskHandle,
0 // Run on Core 0
);
}
void loop() {
ws.cleanupClients();
handleSerialCommands();
size_t bytes_read = 0;
readI2SSamples(raw_samples, &bytes_read);
int samples_read = bytes_read / sizeof(int16_t);
// Update level tracking
for (int i = 0; i < samples_read; i++) {
audioLevelTracker.updateMaxLevel(raw_samples[i]);
}
// Only analyze if we have enough signal
int16_t currentMaxLevel = audioLevelTracker.getMaxLevel();
if (currentMaxLevel > Config::NOISE_THRESHOLD) {
// Analyze samples for note detection
noteDetector.analyzeSamples(raw_samples, samples_read);
uint32_t currentTime = millis();
const auto& detectedNotes = noteDetector.getDetectedNotes();
// Update web clients with spectrum data
if (currentTime - lastWebUpdateTime >= 50) { // Update web clients every 50ms
sendSpectrumData();
lastWebUpdateTime = currentTime;
}
// Show spectrum if enabled
if (showSpectrum &&
!noteDetector.isCalibrating() &&
currentTime - lastSpectrumPrintTime >= Config::DEBUG_INTERVAL_MS) {
SpectrumVisualizer::visualizeSpectrum(noteDetector.getSpectrum(), Config::FFT_SIZE);
lastSpectrumPrintTime = currentTime;
}
// Print detected notes at specified interval
if (currentTime - lastNotePrintTime >= Config::NOTE_PRINT_INTERVAL_MS) {
if (!detectedNotes.empty() && !noteDetector.isCalibrating()) {
SpectrumVisualizer::visualizeNotes(detectedNotes);
}
lastNotePrintTime = currentTime;
}
}
// Small delay to prevent WDT reset
delay(1);
// Main loop is now empty as all work is done in tasks
vTaskDelete(nullptr); // Delete the main loop task
}