Compare commits
1 Commits
serial_plo
...
test_bits_
| Author | SHA1 | Date | |
|---|---|---|---|
| 9a3d026176 |
317
README.md
317
README.md
@@ -1,317 +1,2 @@
|
||||
# ESP32 Piano Note Detection System
|
||||
|
||||
A real-time piano note detection system implemented on ESP32 using I2S microphone input. This system can detect musical notes from C2 to C6 with adjustable sensitivity and visualization options.
|
||||
|
||||
## Features
|
||||
|
||||
- Real-time audio processing using I2S microphone
|
||||
- FFT-based frequency analysis
|
||||
- Note detection from C2 (65.41 Hz) to C6 (1046.50 Hz)
|
||||
- Dynamic threshold calibration
|
||||
- Multiple note detection (up to 7 simultaneous notes)
|
||||
- Harmonic filtering
|
||||
- Real-time spectrum visualization
|
||||
- Note timing and duration tracking
|
||||
- Interactive Serial commands for system tuning
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
- ESP32 development board
|
||||
- I2S MEMS microphone (e.g., INMP441, SPH0645)
|
||||
- USB connection for Serial monitoring
|
||||
|
||||
## Pin Configuration
|
||||
|
||||
The system uses the following I2S pins by default (configurable in Config.h):
|
||||
- SCK (Serial Clock): GPIO 8
|
||||
- WS/LRC (Word Select/Left-Right Clock): GPIO 9
|
||||
- SD (Serial Data): GPIO 10
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. Connect the I2S microphone to the ESP32 according to the pin configuration
|
||||
2. Build and flash the project to your ESP32
|
||||
3. Open a Serial monitor at 115200 baud
|
||||
4. Follow the calibration process on first run
|
||||
|
||||
## Serial Commands
|
||||
|
||||
The system can be controlled via Serial commands:
|
||||
|
||||
- `h` - Display help menu
|
||||
- `c` - Start calibration process
|
||||
- `+` - Increase sensitivity (threshold up)
|
||||
- `-` - Decrease sensitivity (threshold down)
|
||||
- `s` - Toggle spectrum visualization
|
||||
|
||||
## Configuration Options
|
||||
|
||||
All system parameters can be adjusted in `Config.h`:
|
||||
|
||||
### Audio Processing
|
||||
- `SAMPLE_RATE`: 8000 Hz (good for frequencies up to 4kHz)
|
||||
- `BITS_PER_SAMPLE`: 16-bit resolution
|
||||
- `SAMPLE_BUFFER_SIZE`: 1024 samples
|
||||
- `FFT_SIZE`: 1024 points
|
||||
|
||||
### Note Detection
|
||||
- `NOTE_FREQ_C2`: 65.41 Hz (lowest detectable note)
|
||||
- `NOTE_FREQ_C6`: 1046.50 Hz (highest detectable note)
|
||||
- `FREQUENCY_TOLERANCE`: 3.0 Hz
|
||||
- `MAX_SIMULTANEOUS_NOTES`: 7
|
||||
- `MIN_NOTE_DURATION_MS`: 50ms
|
||||
- `NOTE_RELEASE_TIME_MS`: 100ms
|
||||
|
||||
### Calibration
|
||||
- `CALIBRATION_DURATION_MS`: 5000ms
|
||||
- `CALIBRATION_PEAK_PERCENTILE`: 0.95 (95th percentile)
|
||||
|
||||
## Visualization
|
||||
|
||||
The system provides two visualization modes:
|
||||
|
||||
1. Note Display:
|
||||
```
|
||||
Current Notes:
|
||||
A4 (440.0 Hz, Magnitude: 2500, Duration: 250ms)
|
||||
E5 (659.3 Hz, Magnitude: 1800, Duration: 150ms)
|
||||
```
|
||||
|
||||
2. Spectrum Display (when enabled):
|
||||
```
|
||||
Frequency Spectrum:
|
||||
0Hz |▄▄▄▄▄
|
||||
100Hz |██████▄
|
||||
200Hz |▄▄▄
|
||||
...
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
1. Start with calibration by pressing 'c' in a quiet environment
|
||||
2. Play notes and observe the detection accuracy
|
||||
3. Use '+' and '-' to adjust sensitivity if needed
|
||||
4. Enable spectrum display with 's' to visualize frequency content
|
||||
5. Adjust `Config.h` parameters if needed for your specific setup
|
||||
|
||||
## Implementation Details
|
||||
|
||||
- Uses FFT for frequency analysis
|
||||
- Implements peak detection with dynamic thresholding
|
||||
- Filters out harmonics to prevent duplicate detections
|
||||
- Tracks note timing and duration
|
||||
- Uses ring buffer for real-time processing
|
||||
- Calibration collects ambient noise profile
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
1. No notes detected:
|
||||
- Check microphone connection
|
||||
- Run calibration
|
||||
- Increase sensitivity with '+'
|
||||
- Verify audio input level in spectrum display
|
||||
|
||||
2. False detections:
|
||||
- Run calibration in a quiet environment
|
||||
- Decrease sensitivity with '-'
|
||||
- Adjust `PEAK_RATIO_THRESHOLD` in Config.h
|
||||
|
||||
3. Missing notes:
|
||||
- Check if notes are within C2-C6 range
|
||||
- Increase `FREQUENCY_TOLERANCE`
|
||||
- Decrease `MIN_MAGNITUDE_THRESHOLD`
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please read the contributing guidelines before submitting pull requests.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License - see the LICENSE file for details.
|
||||
|
||||
## Development Environment Setup
|
||||
|
||||
### Prerequisites
|
||||
- PlatformIO IDE (recommended) or Arduino IDE
|
||||
- ESP32 board support package
|
||||
- Required libraries:
|
||||
- arduino-audio-tools
|
||||
- arduino-audio-driver
|
||||
- WiFiManager
|
||||
- AsyncTCP
|
||||
- ESPAsyncWebServer
|
||||
- arduinoFFT
|
||||
|
||||
### Building with PlatformIO
|
||||
1. Clone the repository
|
||||
2. Open the project in PlatformIO
|
||||
3. Install dependencies:
|
||||
```
|
||||
pio lib install
|
||||
```
|
||||
4. Build and upload:
|
||||
```
|
||||
pio run -t upload
|
||||
```
|
||||
|
||||
## Memory Management
|
||||
|
||||
### Memory Usage
|
||||
- Program Memory: ~800KB
|
||||
- RAM Usage: ~100KB
|
||||
- DMA Buffers: 4 x 512 bytes
|
||||
- FFT Working Buffer: 2048 bytes (1024 samples x 2 bytes)
|
||||
|
||||
### Optimization Tips
|
||||
- Adjust `DMA_BUFFER_COUNT` based on available RAM
|
||||
- Reduce `SAMPLE_BUFFER_SIZE` for lower latency
|
||||
- Use `PSRAM` if available for larger buffer sizes
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Task Management
|
||||
- Audio processing task on Core 1:
|
||||
- I2S sample reading
|
||||
- Audio level tracking
|
||||
- Note detection and FFT analysis
|
||||
- Visualization task on Core 0:
|
||||
- WebSocket communication
|
||||
- Spectrum visualization
|
||||
- Serial interface
|
||||
- Network operations
|
||||
- Inter-core communication via FreeRTOS queue
|
||||
- Configurable priorities in `Config.h`
|
||||
|
||||
### Audio Pipeline
|
||||
1. I2S DMA Input
|
||||
2. Sample Buffer Collection
|
||||
3. FFT Processing
|
||||
4. Peak Detection
|
||||
5. Note Identification
|
||||
6. Output Generation
|
||||
|
||||
### Timing Parameters
|
||||
- Audio Buffer Processing: ~8ms
|
||||
- FFT Computation: ~5ms
|
||||
- Note Detection: ~2ms
|
||||
- Total Latency: ~15-20ms
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### CPU Usage
|
||||
- Core 1 (Audio Processing):
|
||||
- I2S DMA handling: ~15%
|
||||
- Audio analysis: ~20%
|
||||
- FFT processing: ~15%
|
||||
- Core 0 (Visualization):
|
||||
- WebSocket updates: ~5%
|
||||
- Visualization: ~5%
|
||||
- Network handling: ~5%
|
||||
|
||||
### Memory Optimization
|
||||
1. Buffer Size Selection:
|
||||
- Larger buffers: Better frequency resolution
|
||||
- Smaller buffers: Lower latency
|
||||
2. DMA Configuration:
|
||||
- More buffers: Better continuity
|
||||
- Fewer buffers: Lower memory usage
|
||||
|
||||
### Frequency Analysis
|
||||
- FFT Resolution: 7.8125 Hz (8000/1024)
|
||||
- Frequency Bins: 512 (Nyquist limit)
|
||||
- Useful Range: 65.41 Hz to 1046.50 Hz
|
||||
- Window Function: Hamming
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Microphone Specifications
|
||||
- Supply Voltage: 3.3V
|
||||
- Sampling Rate: 8kHz
|
||||
- Bit Depth: 16-bit
|
||||
- SNR: >65dB (typical)
|
||||
|
||||
### Signal Processing
|
||||
1. Pre-processing:
|
||||
- DC offset removal
|
||||
- Windowing function application
|
||||
2. FFT Processing:
|
||||
- 1024-point real FFT
|
||||
- Magnitude calculation
|
||||
3. Post-processing:
|
||||
- Peak detection
|
||||
- Harmonic filtering
|
||||
- Note matching
|
||||
|
||||
### Calibration Process
|
||||
1. Ambient Noise Collection (5 seconds)
|
||||
2. Frequency Bin Analysis
|
||||
3. Threshold Calculation:
|
||||
- Base threshold from 95th percentile
|
||||
- Per-bin noise floor mapping
|
||||
4. Dynamic Adjustment
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Issues
|
||||
1. I2S Communication Errors:
|
||||
- Check pin connections
|
||||
- Verify I2S configuration
|
||||
- Monitor serial output for error codes
|
||||
2. Memory Issues:
|
||||
- Watch heap fragmentation
|
||||
- Monitor stack usage
|
||||
- Check DMA buffer allocation
|
||||
|
||||
### Error Recovery
|
||||
- Automatic I2S reset on communication errors
|
||||
- Dynamic threshold adjustment
|
||||
- Watchdog timer protection
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Core Components
|
||||
1. AudioLevelTracker
|
||||
- Real-time audio level monitoring
|
||||
- Peak detection
|
||||
- Threshold management
|
||||
2. NoteDetector
|
||||
- Frequency analysis
|
||||
- Note identification
|
||||
- Harmonic filtering
|
||||
3. SpectrumVisualizer
|
||||
- Real-time spectrum display
|
||||
- Magnitude scaling
|
||||
- ASCII visualization
|
||||
|
||||
### File Organization
|
||||
- `/src`: Core implementation files
|
||||
- `/include`: Header files and configurations
|
||||
- `/data`: Additional resources
|
||||
- `/test`: Unit tests
|
||||
|
||||
## Inter-Core Communication
|
||||
|
||||
### Queue Management
|
||||
- FreeRTOS queue for audio data transfer
|
||||
- 4-slot queue buffer
|
||||
- Zero-copy data passing
|
||||
- Non-blocking queue operations
|
||||
- Automatic overflow protection
|
||||
|
||||
### Data Flow
|
||||
1. Core 1 (Audio Task):
|
||||
- Processes audio samples
|
||||
- Performs FFT analysis
|
||||
- Queues processed data
|
||||
2. Core 0 (Visualization Task):
|
||||
- Receives processed data
|
||||
- Updates visualization
|
||||
- Handles network communication
|
||||
|
||||
### Network Communication
|
||||
- Asynchronous WebSocket updates
|
||||
- JSON-formatted spectrum data
|
||||
- Configurable update rate (50ms default)
|
||||
- Automatic client cleanup
|
||||
- Efficient connection management
|
||||
# esp32i2s
|
||||
|
||||
|
||||
260
data/index.html
260
data/index.html
@@ -1,260 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>ESP32 Piano Spectrum Analyzer</title>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chartjs-plugin-annotation"></script>
|
||||
<style>
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
margin: 20px;
|
||||
background: #f0f0f0;
|
||||
}
|
||||
.container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
background: white;
|
||||
padding: 20px;
|
||||
border-radius: 10px;
|
||||
box-shadow: 0 0 10px rgba(0,0,0,0.1);
|
||||
}
|
||||
h1 {
|
||||
color: #333;
|
||||
text-align: center;
|
||||
}
|
||||
#spectrum-container {
|
||||
position: relative;
|
||||
height: 400px;
|
||||
margin: 20px 0;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h1>Piano Spectrum Analyzer</h1>
|
||||
<div id="spectrum-container">
|
||||
<canvas id="spectrumChart"></canvas>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
Chart.register('chartjs-plugin-annotation');
|
||||
|
||||
// Piano note frequencies (Hz)
|
||||
const noteFrequencies = {
|
||||
'C2': 65.41, 'C#2': 69.30, 'D2': 73.42, 'D#2': 77.78, 'E2': 82.41, 'F2': 87.31,
|
||||
'F#2': 92.50, 'G2': 98.00, 'G#2': 103.83, 'A2': 110.00, 'A#2': 116.54, 'B2': 123.47,
|
||||
'C3': 130.81, 'C#3': 138.59, 'D3': 146.83, 'D#3': 155.56, 'E3': 164.81, 'F3': 174.61,
|
||||
'F#3': 185.00, 'G3': 196.00, 'G#3': 207.65, 'A3': 220.00, 'A#3': 233.08, 'B3': 246.94,
|
||||
'C4': 261.63, 'C#4': 277.18, 'D4': 293.66, 'D#4': 311.13, 'E4': 329.63, 'F4': 349.23,
|
||||
'F#4': 369.99, 'G4': 392.00, 'G#4': 415.30, 'A4': 440.00, 'A#4': 466.16, 'B4': 493.88,
|
||||
'C5': 523.25, 'C#5': 554.37, 'D5': 587.33, 'D#5': 622.25, 'E5': 659.26, 'F5': 698.46,
|
||||
'F#5': 739.99, 'G5': 783.99, 'G#5': 830.61, 'A5': 880.00, 'A#5': 932.33, 'B5': 987.77,
|
||||
'C6': 1046.50
|
||||
};
|
||||
|
||||
// Create ticks for the x-axis
|
||||
const xAxisTicks = Object.entries(noteFrequencies).filter(([note]) =>
|
||||
note.length === 2 || note === 'C#4' || note === 'F#4' || note === 'A#4'
|
||||
).map(([note, freq]) => ({
|
||||
value: freq,
|
||||
label: note
|
||||
}));
|
||||
|
||||
// Keep track of recent maximum values
|
||||
const maxValueHistory = [];
|
||||
const MAX_HISTORY_LENGTH = 5;
|
||||
|
||||
function updateYAxisScale(newValue) {
|
||||
maxValueHistory.push(newValue);
|
||||
if (maxValueHistory.length > MAX_HISTORY_LENGTH) {
|
||||
maxValueHistory.shift();
|
||||
}
|
||||
return Math.max(...maxValueHistory) * 1.1; // Add 10% margin
|
||||
}
|
||||
|
||||
const ctx = document.getElementById('spectrumChart').getContext('2d');
|
||||
const chart = new Chart(ctx, {
|
||||
type: 'line',
|
||||
data: {
|
||||
// Generate logarithmically spaced frequencies for labels
|
||||
labels: Array.from({length: 134}, (_, i) => {
|
||||
const minFreq = 60;
|
||||
const maxFreq = 1100;
|
||||
return Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
||||
}),
|
||||
datasets: [{
|
||||
label: 'Frequency Spectrum',
|
||||
data: Array(134).fill(0),
|
||||
borderColor: 'rgb(75, 192, 192)',
|
||||
tension: 0.1,
|
||||
fill: true,
|
||||
backgroundColor: 'rgba(75, 192, 192, 0.2)',
|
||||
pointRadius: 0 // Hide points for better performance
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
responsive: true,
|
||||
maintainAspectRatio: false,
|
||||
animation: {
|
||||
duration: 0
|
||||
},
|
||||
parsing: {
|
||||
xAxisKey: 'x',
|
||||
yAxisKey: 'y'
|
||||
},
|
||||
scales: {
|
||||
y: {
|
||||
beginAtZero: true,
|
||||
max: 5000,
|
||||
adapters: {
|
||||
update: function(maxValue) {
|
||||
return updateYAxisScale(maxValue);
|
||||
}
|
||||
},
|
||||
title: {
|
||||
display: true,
|
||||
text: 'Magnitude'
|
||||
}
|
||||
},
|
||||
x: {
|
||||
type: 'logarithmic',
|
||||
position: 'bottom',
|
||||
min: 60,
|
||||
max: 1100,
|
||||
title: {
|
||||
display: true,
|
||||
text: 'Frequency (Hz)'
|
||||
},
|
||||
ticks: {
|
||||
callback: function(value) {
|
||||
// Show C notes and F# notes
|
||||
const entries = Object.entries(noteFrequencies);
|
||||
const closest = entries.reduce((prev, curr) => {
|
||||
return Math.abs(curr[1] - value) < Math.abs(prev[1] - value) ? curr : prev;
|
||||
});
|
||||
|
||||
if ((closest[0].includes('C') || closest[0].includes('F#')) &&
|
||||
Math.abs(closest[1] - value) < 1) {
|
||||
return closest[0];
|
||||
}
|
||||
return '';
|
||||
},
|
||||
sampleSize: 20,
|
||||
autoSkip: false
|
||||
},
|
||||
grid: {
|
||||
color: (ctx) => {
|
||||
const value = ctx.tick.value;
|
||||
// Check if this tick corresponds to a C note
|
||||
if (Object.entries(noteFrequencies)
|
||||
.some(([note, freq]) =>
|
||||
note.startsWith('C') && Math.abs(freq - value) < 1)) {
|
||||
return 'rgba(255, 0, 0, 0.1)';
|
||||
}
|
||||
return 'rgba(0, 0, 0, 0.1)';
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
plugins: {
|
||||
annotation: {
|
||||
annotations: Object.entries(noteFrequencies)
|
||||
.filter(([note]) => note.startsWith('C'))
|
||||
.reduce((acc, [note, freq]) => {
|
||||
acc[note] = {
|
||||
type: 'line',
|
||||
xMin: freq,
|
||||
xMax: freq,
|
||||
borderColor: 'rgba(255, 99, 132, 0.5)',
|
||||
borderWidth: 1,
|
||||
label: {
|
||||
content: note,
|
||||
enabled: true,
|
||||
position: 'top'
|
||||
}
|
||||
};
|
||||
return acc;
|
||||
}, {})
|
||||
},
|
||||
tooltip: {
|
||||
callbacks: {
|
||||
title: function(context) {
|
||||
const freq = context[0].parsed.x;
|
||||
// Find the closest note
|
||||
const closestNote = Object.entries(noteFrequencies)
|
||||
.reduce((closest, [note, noteFreq]) => {
|
||||
return Math.abs(noteFreq - freq) < Math.abs(noteFreq - closest.freq)
|
||||
? { note, freq: noteFreq }
|
||||
: closest;
|
||||
}, { note: '', freq: Infinity });
|
||||
|
||||
return `Frequency: ${freq.toFixed(1)} Hz (Near ${closestNote.note})`;
|
||||
}
|
||||
}
|
||||
},
|
||||
legend: {
|
||||
display: false
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
const wsUrl = `ws://${window.location.hostname}/ws`;
|
||||
const ws = new WebSocket(wsUrl);
|
||||
|
||||
function interpolateSpectrum(spectrum) {
|
||||
const result = [];
|
||||
const minFreq = 60;
|
||||
const maxFreq = 1100;
|
||||
const binWidth = 8000 / 1024; // Hz per bin
|
||||
|
||||
// Generate logarithmically spaced frequencies
|
||||
for (let i = 0; i < 134; i++) {
|
||||
const targetFreq = Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
||||
|
||||
// Find the corresponding linear bin
|
||||
const bin = Math.floor(targetFreq / binWidth);
|
||||
|
||||
if (bin >= 8 && bin < 141) {
|
||||
// Linear interpolation between bins
|
||||
const binFraction = (targetFreq / binWidth) - bin;
|
||||
const value = spectrum[bin - 8] * (1 - binFraction) +
|
||||
(bin - 7 < spectrum.length ? spectrum[bin - 7] : 0) * binFraction;
|
||||
result.push({
|
||||
x: targetFreq,
|
||||
y: value
|
||||
});
|
||||
} else {
|
||||
result.push({
|
||||
x: targetFreq,
|
||||
y: 0
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
ws.onmessage = function(event) {
|
||||
const spectrum = JSON.parse(event.data);
|
||||
const interpolatedData = interpolateSpectrum(spectrum);
|
||||
|
||||
// Update y-axis scale based on new maximum value
|
||||
const maxValue = Math.max(...interpolatedData.map(d => d.y));
|
||||
chart.options.scales.y.max = updateYAxisScale(maxValue);
|
||||
|
||||
// Update chart data
|
||||
chart.data.datasets[0].data = interpolatedData;
|
||||
chart.update('none'); // Use 'none' mode for maximum performance
|
||||
};
|
||||
|
||||
ws.onclose = function() {
|
||||
console.log('WebSocket connection closed');
|
||||
setTimeout(() => {
|
||||
window.location.reload();
|
||||
}, 1000);
|
||||
};
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,24 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <Arduino.h>
|
||||
#include <deque>
|
||||
#include "Config.h"
|
||||
|
||||
class AudioLevelTracker {
|
||||
public:
|
||||
AudioLevelTracker();
|
||||
void updateMaxLevel(int16_t sample); // Changed to int16_t
|
||||
int16_t getMaxLevel() const; // Changed to int16_t
|
||||
void resetMaxLevel();
|
||||
|
||||
private:
|
||||
struct SamplePoint {
|
||||
uint32_t timestamp;
|
||||
int16_t value; // Changed to int16_t
|
||||
};
|
||||
std::deque<SamplePoint> sampleHistory;
|
||||
int16_t maxLevel; // Changed to int16_t
|
||||
|
||||
static const uint32_t HISTORY_DURATION_MS = 3000; // 3 seconds history
|
||||
static const int16_t MAX_RANGE_LIMIT = Config::DEFAULT_RANGE_LIMIT; // Use 16-bit limit directly
|
||||
};
|
||||
@@ -1,90 +0,0 @@
|
||||
#pragma once
|
||||
#include <cstdint>
|
||||
|
||||
namespace Config {
|
||||
|
||||
// Serial Configuration
|
||||
constexpr int SERIAL_BAUD_RATE = 115200; // Serial communication baud rate
|
||||
|
||||
// WiFi Configuration
|
||||
constexpr const char* WIFI_AP_NAME = "ESP32Piano"; // AP name when in configuration mode
|
||||
constexpr const char* WIFI_AP_PASSWORD = "12345678"; // AP password when in configuration mode
|
||||
constexpr uint32_t WIFI_CONFIG_TIMEOUT = 180; // Seconds to wait for WiFi configuration
|
||||
constexpr uint32_t WIFI_CONFIG_PORT = 80; // Web configuration portal port
|
||||
|
||||
// I2S Pin Configuration
|
||||
constexpr int I2S_MIC_SERIAL_CLOCK = 8; // SCK
|
||||
constexpr int I2S_MIC_LEFT_RIGHT_CLOCK = 9; // WS/LRC
|
||||
constexpr int I2S_MIC_SERIAL_DATA = 10; // SD
|
||||
|
||||
// I2S Configuration
|
||||
constexpr int SAMPLE_RATE = 8000; // Hz - good for frequencies up to 4kHz
|
||||
constexpr int BITS_PER_SAMPLE = 16; // Changed to 16-bit
|
||||
constexpr int CHANNELS = 1; // Mono input
|
||||
constexpr int SAMPLE_BUFFER_SIZE = 1024; // Match FFT_SIZE for efficiency
|
||||
|
||||
// DMA Configuration
|
||||
constexpr int DMA_BUFFER_COUNT = 4; // Reduced for lower latency
|
||||
constexpr int DMA_BUFFER_LEN = 512; // Smaller chunks for faster processing
|
||||
|
||||
// Audio Processing
|
||||
constexpr float DC_OFFSET = 0.0f; // DC offset correction
|
||||
constexpr float GAIN = 4.0f; // Increased gain for better note detection
|
||||
constexpr int16_t NOISE_THRESHOLD = 100; // Lower threshold for 16-bit samples
|
||||
constexpr int16_t DEFAULT_RANGE_LIMIT = 32767; // Max value for 16-bit
|
||||
constexpr float DECAY_FACTOR = 0.7f; // Faster decay for quicker note changes
|
||||
|
||||
// FFT Configuration
|
||||
constexpr int FFT_SIZE = 1024; // Must be power of 2, gives ~7.8 Hz resolution at 8kHz
|
||||
constexpr float FREQUENCY_RESOLUTION = static_cast<float>(SAMPLE_RATE) / FFT_SIZE;
|
||||
|
||||
// Piano Note Frequencies (Hz)
|
||||
constexpr float NOTE_FREQ_C2 = 65.41f; // Lowest note we'll detect
|
||||
constexpr float NOTE_FREQ_C3 = 130.81f;
|
||||
constexpr float NOTE_FREQ_C4 = 261.63f; // Middle C
|
||||
constexpr float NOTE_FREQ_C5 = 523.25f;
|
||||
constexpr float NOTE_FREQ_C6 = 1046.50f; // Highest note we'll detect
|
||||
|
||||
// Note Detection Parameters
|
||||
constexpr float FREQUENCY_TOLERANCE = 3.0f; // Hz tolerance, increased for better detection
|
||||
constexpr float MIN_MAGNITUDE_THRESHOLD = 500.0f; // Adjusted for 16-bit samples
|
||||
constexpr int MAX_SIMULTANEOUS_NOTES = 7; // Maximum number of notes to detect simultaneously
|
||||
constexpr float PEAK_RATIO_THRESHOLD = 0.1f; // Peaks must be at least this ratio of the strongest peak
|
||||
|
||||
// Timing and Debug
|
||||
constexpr uint32_t LEVEL_UPDATE_INTERVAL_MS = 50; // Faster updates for better responsiveness
|
||||
constexpr bool ENABLE_DEBUG = true; // Enable debug output
|
||||
constexpr int DEBUG_INTERVAL_MS = 1000; // Debug print interval
|
||||
|
||||
// System Configuration
|
||||
constexpr uint32_t TASK_STACK_SIZE = 4096; // Audio task stack size
|
||||
constexpr uint8_t TASK_PRIORITY = 2; // Increased priority for more consistent timing
|
||||
constexpr uint8_t TASK_CORE = 1; // Run on second core to avoid interference
|
||||
|
||||
// Calibration Parameters
|
||||
constexpr bool CALIBRATION_MODE = false; // Set to true to enable calibration output
|
||||
constexpr int CALIBRATION_DURATION_MS = 5000; // Duration to collect calibration data
|
||||
constexpr float CALIBRATION_PEAK_PERCENTILE = 0.95f; // Use 95th percentile for thresholds
|
||||
|
||||
// Note Detection Timing
|
||||
constexpr int NOTE_PRINT_INTERVAL_MS = 100; // How often to print detected notes
|
||||
constexpr int MIN_NOTE_DURATION_MS = 50; // Minimum duration to consider a note valid
|
||||
constexpr int NOTE_RELEASE_TIME_MS = 100; // Time before considering a note released
|
||||
|
||||
// Serial Commands
|
||||
constexpr char CMD_HELP = 'h'; // Show help
|
||||
constexpr char CMD_CALIBRATE = 'c'; // Start calibration
|
||||
constexpr char CMD_THRESHOLD_UP = '+'; // Increase sensitivity
|
||||
constexpr char CMD_THRESHOLD_DOWN = '-'; // Decrease sensitivity
|
||||
constexpr char CMD_TOGGLE_SPECTRUM = 's'; // Toggle spectrum display
|
||||
constexpr float THRESHOLD_STEP = 0.1f; // Threshold adjustment step
|
||||
|
||||
// Command Response Messages
|
||||
constexpr const char* MSG_HELP =
|
||||
"Commands:\n"
|
||||
"h - Show this help\n"
|
||||
"c - Start calibration\n"
|
||||
"+ - Increase sensitivity\n"
|
||||
"- - Decrease sensitivity\n"
|
||||
"s - Toggle spectrum display\n";
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <Arduino.h>
|
||||
#include <driver/i2s.h>
|
||||
#include "Config.h"
|
||||
|
||||
// I2S setup functions
|
||||
void initI2S();
|
||||
void readI2SSamples(int16_t* samples, size_t* bytesRead);
|
||||
@@ -1,48 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <Arduino.h>
|
||||
#include <arduinoFFT.h>
|
||||
#include <vector>
|
||||
#include <deque>
|
||||
#include "Config.h"
|
||||
|
||||
struct DetectedNote {
|
||||
float frequency;
|
||||
float magnitude;
|
||||
uint8_t noteNumber; // MIDI note number for easy identification
|
||||
uint32_t startTime; // When the note was first detected
|
||||
uint32_t lastSeenTime; // Last time this note was detected
|
||||
};
|
||||
|
||||
class NoteDetector {
|
||||
public:
|
||||
NoteDetector();
|
||||
void analyzeSamples(const int16_t* samples, size_t sampleCount);
|
||||
const std::vector<DetectedNote>& getDetectedNotes() const { return detectedNotes; }
|
||||
const double* getSpectrum() const { return vReal; }
|
||||
void beginCalibration();
|
||||
void endCalibration();
|
||||
bool isCalibrating() const { return calibrationMode; }
|
||||
|
||||
// Threshold adjustment methods
|
||||
void adjustThreshold(float delta);
|
||||
float getCurrentThreshold() const { return dynamicThreshold; }
|
||||
|
||||
private:
|
||||
arduinoFFT FFT;
|
||||
double vReal[Config::FFT_SIZE];
|
||||
double vImag[Config::FFT_SIZE];
|
||||
std::vector<DetectedNote> detectedNotes;
|
||||
std::deque<float> peakMagnitudeHistory; // For calibration
|
||||
|
||||
bool calibrationMode;
|
||||
uint32_t calibrationStartTime;
|
||||
float dynamicThreshold;
|
||||
|
||||
uint8_t freqToNoteNumber(float frequency) const;
|
||||
bool isNoteFrequency(float frequency) const;
|
||||
bool isHarmonic(float freq1, float freq2, float tolerance) const;
|
||||
void findPeaks();
|
||||
void updateNoteTimings();
|
||||
void calculateDynamicThreshold();
|
||||
};
|
||||
@@ -1,27 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <Arduino.h>
|
||||
#include "Config.h"
|
||||
#include "NoteDetector.h"
|
||||
|
||||
class SpectrumVisualizer {
|
||||
public:
|
||||
// Visualization settings
|
||||
static constexpr int DISPLAY_WIDTH = 80; // Characters wide
|
||||
static constexpr int DISPLAY_HEIGHT = 16; // Lines tall
|
||||
static constexpr float DB_MIN = 30.0f; // Minimum dB to show
|
||||
static constexpr float DB_MAX = 90.0f; // Maximum dB to show
|
||||
|
||||
static void visualizeSpectrum(const double* spectrum, int size);
|
||||
static void visualizeNotes(const std::vector<DetectedNote>& notes);
|
||||
static void drawFFTMagnitudes(const double* magnitudes, int size, bool logScale = true);
|
||||
|
||||
private:
|
||||
static constexpr const char* noteNames[] = {
|
||||
"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"
|
||||
};
|
||||
|
||||
static float magnitudeToDb(double magnitude);
|
||||
static int mapToDisplay(float value, float min, float max, int displayMin, int displayMax);
|
||||
static void printBarGraph(float value, float maxValue, int width);
|
||||
};
|
||||
@@ -33,7 +33,4 @@ monitor_filters = esp32_exception_decoder
|
||||
lib_deps =
|
||||
https://github.com/pschatzmann/arduino-audio-tools.git
|
||||
https://github.com/pschatzmann/arduino-audio-driver.git
|
||||
https://github.com/tzapu/WiFiManager.git
|
||||
me-no-dev/AsyncTCP
|
||||
https://github.com/me-no-dev/ESPAsyncWebServer.git
|
||||
kosme/arduinoFFT@^1.6
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
#include "AudioLevelTracker.h"
|
||||
|
||||
AudioLevelTracker::AudioLevelTracker() {
|
||||
resetMaxLevel();
|
||||
}
|
||||
|
||||
void AudioLevelTracker::updateMaxLevel(int16_t sample) {
|
||||
uint32_t currentTime = millis();
|
||||
|
||||
// Remove old samples (older than specified duration)
|
||||
while (!sampleHistory.empty() &&
|
||||
(currentTime - sampleHistory.front().timestamp) > HISTORY_DURATION_MS) {
|
||||
sampleHistory.pop_front();
|
||||
}
|
||||
|
||||
// Add new sample, but cap it at MAX_RANGE_LIMIT
|
||||
int16_t absValue = abs(sample);
|
||||
absValue = min(absValue, MAX_RANGE_LIMIT); // Cap at 16-bit limit
|
||||
SamplePoint newPoint = {currentTime, absValue};
|
||||
sampleHistory.push_back(newPoint);
|
||||
|
||||
// Update maximum
|
||||
maxLevel = 0;
|
||||
for (const auto& point : sampleHistory) {
|
||||
if (point.value > maxLevel) {
|
||||
maxLevel = point.value;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
int16_t AudioLevelTracker::getMaxLevel() const {
|
||||
return maxLevel;
|
||||
}
|
||||
|
||||
void AudioLevelTracker::resetMaxLevel() {
|
||||
maxLevel = 0;
|
||||
sampleHistory.clear();
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
#include "I2SConfig.h"
|
||||
|
||||
void initI2S() {
|
||||
// I2S configuration
|
||||
i2s_config_t i2s_config = {
|
||||
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
|
||||
.sample_rate = Config::SAMPLE_RATE,
|
||||
.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
|
||||
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
|
||||
.communication_format = I2S_COMM_FORMAT_STAND_I2S,
|
||||
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
|
||||
.dma_buf_count = Config::DMA_BUFFER_COUNT,
|
||||
.dma_buf_len = Config::DMA_BUFFER_LEN,
|
||||
.use_apll = false,
|
||||
.tx_desc_auto_clear = false,
|
||||
.fixed_mclk = 0
|
||||
};
|
||||
|
||||
// I2S pin configuration
|
||||
i2s_pin_config_t i2s_mic_pins = {
|
||||
.bck_io_num = Config::I2S_MIC_SERIAL_CLOCK,
|
||||
.ws_io_num = Config::I2S_MIC_LEFT_RIGHT_CLOCK,
|
||||
.data_out_num = I2S_PIN_NO_CHANGE,
|
||||
.data_in_num = Config::I2S_MIC_SERIAL_DATA
|
||||
};
|
||||
|
||||
// Install and configure I2S driver
|
||||
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
|
||||
i2s_set_pin(I2S_NUM_0, &i2s_mic_pins);
|
||||
}
|
||||
|
||||
void readI2SSamples(int16_t* samples, size_t* bytesRead) {
|
||||
i2s_read(I2S_NUM_0, samples, Config::SAMPLE_BUFFER_SIZE * sizeof(int16_t), bytesRead, portMAX_DELAY);
|
||||
}
|
||||
@@ -1,198 +0,0 @@
|
||||
#include "NoteDetector.h"
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
|
||||
NoteDetector::NoteDetector()
|
||||
: FFT(vReal, vImag, Config::FFT_SIZE, Config::SAMPLE_RATE)
|
||||
, calibrationMode(false)
|
||||
, calibrationStartTime(0)
|
||||
, dynamicThreshold(Config::MIN_MAGNITUDE_THRESHOLD) {
|
||||
}
|
||||
|
||||
void NoteDetector::beginCalibration() {
|
||||
calibrationMode = true;
|
||||
calibrationStartTime = millis();
|
||||
peakMagnitudeHistory.clear();
|
||||
Serial.println("Beginning calibration...");
|
||||
}
|
||||
|
||||
void NoteDetector::endCalibration() {
|
||||
calibrationMode = false;
|
||||
calculateDynamicThreshold();
|
||||
Serial.printf("Calibration complete. Dynamic threshold: %.2f\n", dynamicThreshold);
|
||||
}
|
||||
|
||||
void NoteDetector::calculateDynamicThreshold() {
|
||||
if (peakMagnitudeHistory.empty()) {
|
||||
dynamicThreshold = Config::MIN_MAGNITUDE_THRESHOLD;
|
||||
return;
|
||||
}
|
||||
|
||||
// Sort magnitudes to find percentile
|
||||
std::vector<float> sorted(peakMagnitudeHistory.begin(), peakMagnitudeHistory.end());
|
||||
std::sort(sorted.begin(), sorted.end());
|
||||
|
||||
// Calculate threshold at specified percentile
|
||||
size_t idx = static_cast<size_t>(sorted.size() * Config::CALIBRATION_PEAK_PERCENTILE);
|
||||
dynamicThreshold = sorted[idx] * Config::PEAK_RATIO_THRESHOLD;
|
||||
|
||||
// Don't go below minimum threshold
|
||||
dynamicThreshold = (dynamicThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
||||
dynamicThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
||||
}
|
||||
|
||||
void NoteDetector::updateNoteTimings() {
|
||||
uint32_t currentTime = millis();
|
||||
|
||||
// Remove notes that haven't been seen recently
|
||||
detectedNotes.erase(
|
||||
std::remove_if(detectedNotes.begin(), detectedNotes.end(),
|
||||
[currentTime](const DetectedNote& note) {
|
||||
return currentTime - note.lastSeenTime > Config::NOTE_RELEASE_TIME_MS;
|
||||
}),
|
||||
detectedNotes.end()
|
||||
);
|
||||
}
|
||||
|
||||
void NoteDetector::analyzeSamples(const int16_t* samples, size_t sampleCount) {
|
||||
// Copy samples to FFT buffer with gain applied
|
||||
for (size_t i = 0; i < Config::FFT_SIZE; i++) {
|
||||
vReal[i] = i < sampleCount ? samples[i] * Config::GAIN : 0;
|
||||
vImag[i] = 0;
|
||||
}
|
||||
|
||||
// Perform FFT
|
||||
FFT.DCRemoval();
|
||||
FFT.Windowing(FFT_WIN_TYP_HANN, FFT_FORWARD);
|
||||
FFT.Compute(FFT_FORWARD);
|
||||
FFT.ComplexToMagnitude();
|
||||
|
||||
// Find peaks and update timings
|
||||
findPeaks();
|
||||
updateNoteTimings();
|
||||
|
||||
// Handle calibration mode
|
||||
if (calibrationMode) {
|
||||
// Store peak magnitudes during calibration
|
||||
double maxMagnitude = 0;
|
||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||
maxMagnitude = std::max(maxMagnitude, vReal[i]);
|
||||
}
|
||||
peakMagnitudeHistory.push_back(maxMagnitude);
|
||||
|
||||
// Check if calibration period is over
|
||||
if (millis() - calibrationStartTime >= Config::CALIBRATION_DURATION_MS) {
|
||||
endCalibration();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
uint8_t NoteDetector::freqToNoteNumber(float frequency) const {
|
||||
// A4 is 440Hz and MIDI note 69
|
||||
return round(12 * log2(frequency/440.0) + 69);
|
||||
}
|
||||
|
||||
bool NoteDetector::isNoteFrequency(float frequency) const {
|
||||
// Check if frequency is within range of C2 to C6
|
||||
if (frequency < (Config::NOTE_FREQ_C2 - Config::FREQUENCY_TOLERANCE) ||
|
||||
frequency > (Config::NOTE_FREQ_C6 + Config::FREQUENCY_TOLERANCE)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Calculate note number and then expected frequency
|
||||
uint8_t noteNum = freqToNoteNumber(frequency);
|
||||
float expectedFreq = 440.0f * pow(2.0f, (noteNum - 69) / 12.0f);
|
||||
|
||||
// Check if within tolerance
|
||||
return abs(frequency - expectedFreq) <= Config::FREQUENCY_TOLERANCE;
|
||||
}
|
||||
|
||||
bool NoteDetector::isHarmonic(float freq1, float freq2, float tolerance) const {
|
||||
// Check if freq2 is a harmonic of freq1
|
||||
if (freq2 < freq1) {
|
||||
std::swap(freq1, freq2);
|
||||
}
|
||||
|
||||
float ratio = freq2 / freq1;
|
||||
float nearest_harmonic = round(ratio);
|
||||
|
||||
return abs(ratio - nearest_harmonic) <= (tolerance / freq1);
|
||||
}
|
||||
|
||||
void NoteDetector::findPeaks() {
|
||||
std::vector<DetectedNote> candidates;
|
||||
uint32_t currentTime = millis();
|
||||
|
||||
// Find the maximum magnitude
|
||||
double maxMagnitude = 0;
|
||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||
maxMagnitude = (vReal[i] > maxMagnitude) ? vReal[i] : maxMagnitude;
|
||||
}
|
||||
|
||||
// Use either calibrated or relative threshold
|
||||
float relativeThreshold = static_cast<float>(maxMagnitude * Config::PEAK_RATIO_THRESHOLD);
|
||||
float threshold = calibrationMode ? Config::MIN_MAGNITUDE_THRESHOLD :
|
||||
(dynamicThreshold > relativeThreshold ? dynamicThreshold : relativeThreshold);
|
||||
|
||||
// First pass: find all potential peaks
|
||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||
float frequency = i * Config::FREQUENCY_RESOLUTION;
|
||||
|
||||
if (vReal[i] > threshold &&
|
||||
vReal[i] > vReal[i-1] &&
|
||||
vReal[i] > vReal[i+1] &&
|
||||
isNoteFrequency(frequency)) {
|
||||
|
||||
DetectedNote note;
|
||||
note.frequency = frequency;
|
||||
note.magnitude = vReal[i];
|
||||
note.noteNumber = freqToNoteNumber(frequency);
|
||||
note.startTime = currentTime;
|
||||
note.lastSeenTime = currentTime;
|
||||
candidates.push_back(note);
|
||||
}
|
||||
}
|
||||
|
||||
// Sort candidates by magnitude
|
||||
std::sort(candidates.begin(), candidates.end(),
|
||||
[](const DetectedNote& a, const DetectedNote& b) {
|
||||
return a.magnitude > b.magnitude;
|
||||
});
|
||||
|
||||
// Second pass: filter out harmonics and update existing notes
|
||||
for (const auto& candidate : candidates) {
|
||||
// Check if this note is already being tracked
|
||||
auto existingNote = std::find_if(detectedNotes.begin(), detectedNotes.end(),
|
||||
[&candidate](const DetectedNote& note) {
|
||||
return note.noteNumber == candidate.noteNumber;
|
||||
});
|
||||
|
||||
if (existingNote != detectedNotes.end()) {
|
||||
// Update existing note
|
||||
existingNote->magnitude = candidate.magnitude;
|
||||
existingNote->lastSeenTime = currentTime;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check if this is a harmonic of a stronger note
|
||||
bool isHarmonicOfStronger = false;
|
||||
for (const auto& note : detectedNotes) {
|
||||
if (isHarmonic(candidate.frequency, note.frequency, Config::FREQUENCY_TOLERANCE)) {
|
||||
isHarmonicOfStronger = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Add new note if it's not a harmonic and we haven't hit the limit
|
||||
if (!isHarmonicOfStronger && detectedNotes.size() < Config::MAX_SIMULTANEOUS_NOTES) {
|
||||
detectedNotes.push_back(candidate);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void NoteDetector::adjustThreshold(float delta) {
|
||||
float newThreshold = dynamicThreshold + (dynamicThreshold * delta);
|
||||
dynamicThreshold = (newThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
||||
newThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
||||
Serial.printf("Threshold adjusted to: %.1f\n", dynamicThreshold);
|
||||
}
|
||||
@@ -1,82 +0,0 @@
|
||||
#include "SpectrumVisualizer.h"
|
||||
#include <cmath>
|
||||
|
||||
constexpr const char* SpectrumVisualizer::noteNames[];
|
||||
|
||||
float SpectrumVisualizer::magnitudeToDb(double magnitude) {
|
||||
return 20 * log10(magnitude);
|
||||
}
|
||||
|
||||
int SpectrumVisualizer::mapToDisplay(float value, float min, float max, int displayMin, int displayMax) {
|
||||
float normalized = (value - min) / (max - min);
|
||||
normalized = constrain(normalized, 0.0f, 1.0f);
|
||||
return displayMin + (normalized * (displayMax - displayMin));
|
||||
}
|
||||
|
||||
void SpectrumVisualizer::printBarGraph(float value, float maxValue, int width) {
|
||||
int barWidth = mapToDisplay(value, 0, maxValue, 0, width);
|
||||
for (int i = 0; i < width; i++) {
|
||||
Serial.print(i < barWidth ? "█" : " ");
|
||||
}
|
||||
}
|
||||
|
||||
void SpectrumVisualizer::drawFFTMagnitudes(const double* magnitudes, int size, bool logScale) {
|
||||
// Find maximum magnitude for scaling
|
||||
double maxMagnitude = 0;
|
||||
for (int i = 0; i < size/2; i++) {
|
||||
maxMagnitude = max(maxMagnitude, magnitudes[i]);
|
||||
}
|
||||
|
||||
// Print frequency spectrum
|
||||
Serial.println("\nFrequency Spectrum:");
|
||||
Serial.println("Freq (Hz) | Magnitude");
|
||||
Serial.println("-----------|-" + String('-', DISPLAY_WIDTH));
|
||||
|
||||
for (int i = 0; i < size/2; i++) {
|
||||
float freq = i * Config::FREQUENCY_RESOLUTION;
|
||||
if (freq > Config::NOTE_FREQ_C6) break; // Stop at highest note frequency
|
||||
|
||||
float magnitude = magnitudes[i];
|
||||
if (logScale) {
|
||||
magnitude = magnitudeToDb(magnitude);
|
||||
maxMagnitude = magnitudeToDb(maxMagnitude);
|
||||
}
|
||||
|
||||
// Only print if significant magnitude
|
||||
if (magnitude > (logScale ? DB_MIN : 0)) {
|
||||
Serial.printf("%8.1f | ", freq);
|
||||
printBarGraph(magnitude, maxMagnitude, DISPLAY_WIDTH);
|
||||
Serial.println();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void SpectrumVisualizer::visualizeSpectrum(const double* spectrum, int size) {
|
||||
drawFFTMagnitudes(spectrum, size, true);
|
||||
}
|
||||
|
||||
void SpectrumVisualizer::visualizeNotes(const std::vector<DetectedNote>& notes) {
|
||||
if (notes.empty()) return;
|
||||
|
||||
// Find maximum magnitude for scaling
|
||||
float maxMagnitude = 0;
|
||||
for (const auto& note : notes) {
|
||||
maxMagnitude = max(maxMagnitude, note.magnitude);
|
||||
}
|
||||
|
||||
// Print piano keyboard visualization
|
||||
Serial.println("\nDetected Notes:");
|
||||
Serial.println("Note | Freq (Hz) | Level");
|
||||
Serial.println("--------|-----------|" + String('-', DISPLAY_WIDTH));
|
||||
|
||||
for (const auto& note : notes) {
|
||||
int octave = (note.noteNumber / 12) - 1;
|
||||
int noteIndex = note.noteNumber % 12;
|
||||
float dbLevel = magnitudeToDb(note.magnitude);
|
||||
|
||||
Serial.printf("%s%-6d | %8.1f | ",
|
||||
noteNames[noteIndex], octave, note.frequency);
|
||||
printBarGraph(note.magnitude, maxMagnitude, DISPLAY_WIDTH);
|
||||
Serial.printf(" %.1f dB\n", dbLevel);
|
||||
}
|
||||
}
|
||||
338
src/main.cpp
338
src/main.cpp
@@ -1,275 +1,99 @@
|
||||
#include <Arduino.h>
|
||||
#include <WiFi.h>
|
||||
#include <WiFiManager.h>
|
||||
#include <ESPAsyncWebServer.h>
|
||||
#include <AsyncTCP.h>
|
||||
#include <SPIFFS.h>
|
||||
#include "Config.h"
|
||||
#include "I2SConfig.h"
|
||||
#include "AudioLevelTracker.h"
|
||||
#include "NoteDetector.h"
|
||||
#include "SpectrumVisualizer.h"
|
||||
#include "AudioTools.h"
|
||||
// #include "AudioTools/AudioLibs/AudioI2SStream.h"
|
||||
#include "AudioTools/AudioLibs/AudioRealFFT.h" // or AudioKissFFT
|
||||
|
||||
// Function declarations
|
||||
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len);
|
||||
void initWebServer();
|
||||
void handleSerialCommands();
|
||||
void printNoteInfo(const DetectedNote& note);
|
||||
void initWiFi();
|
||||
void audioProcessingTask(void *parameter);
|
||||
void visualizationTask(void *parameter);
|
||||
void sendSpectrumData();
|
||||
I2SStream i2sStream; // I2S input stream for INMP441
|
||||
AudioRealFFT fft; // FFT analyzer
|
||||
FormatConverterStream converter(i2sStream); // Convert 32-bit input to 16-bit output
|
||||
StreamCopy copier(fft, converter); // copy converted data to FFT
|
||||
|
||||
// Static instances
|
||||
static int16_t raw_samples[Config::SAMPLE_BUFFER_SIZE];
|
||||
static AudioLevelTracker audioLevelTracker;
|
||||
static NoteDetector noteDetector;
|
||||
WiFiManager wifiManager;
|
||||
AsyncWebServer server(80);
|
||||
AsyncWebSocket ws("/ws");
|
||||
int channels = 1; // INMP441 is mono
|
||||
int samples_per_second = 11025;
|
||||
int input_bits_per_sample = 32; // INMP441 sends 24-bit data in 32-bit words
|
||||
int fft_bits_per_sample = 16;
|
||||
|
||||
// Timing and state variables
|
||||
static uint32_t lastNotePrintTime = 0;
|
||||
static uint32_t lastSpectrumPrintTime = 0;
|
||||
static uint32_t lastWebUpdateTime = 0;
|
||||
static bool showSpectrum = false;
|
||||
AudioInfo from(samples_per_second, channels, input_bits_per_sample);
|
||||
AudioInfo to(samples_per_second, channels, fft_bits_per_sample);
|
||||
|
||||
// Task handles
|
||||
TaskHandle_t audioTaskHandle = nullptr;
|
||||
TaskHandle_t visualizationTaskHandle = nullptr;
|
||||
const char* solfegeName(uint8_t midiNote) {
|
||||
static const char* solfegeNames[] = {
|
||||
"Do", "Do#", "Re", "Re#", "Mi", "Fa", "Fa#", "Sol", "Sol#", "La", "La#", "Si"
|
||||
};
|
||||
return solfegeNames[midiNote % 12];
|
||||
}
|
||||
|
||||
// Queue for passing audio data between cores
|
||||
QueueHandle_t audioQueue;
|
||||
void fftResult(AudioFFTBase &fft) {
|
||||
float diff;
|
||||
auto result = fft.result();
|
||||
|
||||
// Note names for display
|
||||
const char* noteNames[] = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"};
|
||||
if (result.magnitude > 100) { // avoid noise floor
|
||||
float magnitude_dB = 20.0 * log10(result.magnitude);
|
||||
float freq = result.frequency;
|
||||
|
||||
void sendSpectrumData() {
|
||||
if (ws.count() > 0 && !noteDetector.isCalibrating()) {
|
||||
const auto& spectrum = noteDetector.getSpectrum();
|
||||
String json = "[";
|
||||
|
||||
// Calculate bin range for 60-1100 Hz
|
||||
// At 8kHz sample rate with 1024 FFT size:
|
||||
// binFreq = index * (8000/1024) = index * 7.8125 Hz
|
||||
// For 60 Hz: bin ≈ 8
|
||||
// For 1100 Hz: bin ≈ 141
|
||||
|
||||
for (int i = 8; i <= 141; i++) {
|
||||
if (i > 8) json += ",";
|
||||
json += String(spectrum[i], 2);
|
||||
}
|
||||
json += "]";
|
||||
ws.textAll(json);
|
||||
// MIDI note number
|
||||
int midiNote = round(69 + 12.0 * log2(freq / 440.0));
|
||||
const char* solfege = solfegeName(midiNote);
|
||||
int octave = (midiNote / 12) - 1;
|
||||
|
||||
Serial.print(freq, 0);
|
||||
Serial.print(" Hz | ");
|
||||
|
||||
Serial.print("MIDI ");
|
||||
Serial.print(midiNote);
|
||||
Serial.print(" | ");
|
||||
|
||||
Serial.print("Note: ");
|
||||
Serial.print(result.frequencyAsNote(diff));
|
||||
Serial.print(" | ");
|
||||
|
||||
Serial.print("Solfège: ");
|
||||
Serial.print(solfege);
|
||||
Serial.print(octave);
|
||||
|
||||
Serial.print(" | dB: ");
|
||||
Serial.print(magnitude_dB, 2);
|
||||
|
||||
Serial.print(" | Diff: ");
|
||||
Serial.println(diff, 2);
|
||||
|
||||
Serial.print(" | Amp: ");
|
||||
Serial.println(result.magnitude, 0);
|
||||
}
|
||||
}
|
||||
|
||||
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len) {
|
||||
switch (type) {
|
||||
case WS_EVT_CONNECT:
|
||||
Serial.printf("WebSocket client #%u connected from %s\n", client->id(), client->remoteIP().toString().c_str());
|
||||
break;
|
||||
case WS_EVT_DISCONNECT:
|
||||
Serial.printf("WebSocket client #%u disconnected\n", client->id());
|
||||
break;
|
||||
case WS_EVT_DATA:
|
||||
break;
|
||||
case WS_EVT_ERROR:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
void initWebServer() {
|
||||
if (!SPIFFS.begin(true)) {
|
||||
Serial.println("An error occurred while mounting SPIFFS");
|
||||
return;
|
||||
}
|
||||
|
||||
ws.onEvent(onWebSocketEvent);
|
||||
server.addHandler(&ws);
|
||||
|
||||
// Serve static files from SPIFFS
|
||||
server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");
|
||||
|
||||
// Handle not found
|
||||
server.onNotFound([](AsyncWebServerRequest *request) {
|
||||
request->send(404, "text/plain", "Not found");
|
||||
});
|
||||
|
||||
server.begin();
|
||||
Serial.println("HTTP server started");
|
||||
}
|
||||
|
||||
void handleSerialCommands() {
|
||||
if (Serial.available()) {
|
||||
char cmd = Serial.read();
|
||||
switch (cmd) {
|
||||
case Config::CMD_HELP:
|
||||
Serial.println(Config::MSG_HELP);
|
||||
break;
|
||||
|
||||
case Config::CMD_CALIBRATE:
|
||||
noteDetector.beginCalibration();
|
||||
break;
|
||||
|
||||
case Config::CMD_THRESHOLD_UP:
|
||||
noteDetector.adjustThreshold(Config::THRESHOLD_STEP);
|
||||
break;
|
||||
|
||||
case Config::CMD_THRESHOLD_DOWN:
|
||||
noteDetector.adjustThreshold(-Config::THRESHOLD_STEP);
|
||||
break;
|
||||
|
||||
case Config::CMD_TOGGLE_SPECTRUM:
|
||||
showSpectrum = !showSpectrum;
|
||||
Serial.printf("Spectrum display %s\n", showSpectrum ? "enabled" : "disabled");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void printNoteInfo(const DetectedNote& note) {
|
||||
int octave = (note.noteNumber / 12) - 1;
|
||||
int noteIndex = note.noteNumber % 12;
|
||||
uint32_t duration = millis() - note.startTime;
|
||||
|
||||
Serial.printf("Note: %s%d (%.1f Hz, Magnitude: %.0f, Duration: %ums)\n",
|
||||
noteNames[noteIndex], octave, note.frequency, note.magnitude, duration);
|
||||
}
|
||||
|
||||
void initWiFi() {
|
||||
// Set configuration portal timeout
|
||||
wifiManager.setConfigPortalTimeout(Config::WIFI_CONFIG_TIMEOUT);
|
||||
|
||||
// Set custom portal settings
|
||||
wifiManager.setAPStaticIPConfig(IPAddress(192,168,4,1), IPAddress(192,168,4,1), IPAddress(255,255,255,0));
|
||||
|
||||
// Try to connect to saved WiFi credentials
|
||||
if(!wifiManager.autoConnect(Config::WIFI_AP_NAME, Config::WIFI_AP_PASSWORD)) {
|
||||
Serial.println("Failed to connect and hit timeout");
|
||||
ESP.restart();
|
||||
}
|
||||
|
||||
Serial.println("Successfully connected to WiFi");
|
||||
Serial.print("IP Address: ");
|
||||
Serial.println(WiFi.localIP());
|
||||
}
|
||||
|
||||
// Audio processing task running on Core 1
|
||||
void audioProcessingTask(void *parameter) {
|
||||
while (true) {
|
||||
size_t bytes_read = 0;
|
||||
readI2SSamples(raw_samples, &bytes_read);
|
||||
int samples_read = bytes_read / sizeof(int16_t);
|
||||
|
||||
// Update level tracking
|
||||
for (int i = 0; i < samples_read; i++) {
|
||||
audioLevelTracker.updateMaxLevel(raw_samples[i]);
|
||||
}
|
||||
|
||||
// Only analyze if we have enough signal
|
||||
int16_t currentMaxLevel = audioLevelTracker.getMaxLevel();
|
||||
if (currentMaxLevel > Config::NOISE_THRESHOLD) {
|
||||
// Analyze samples for note detection
|
||||
noteDetector.analyzeSamples(raw_samples, samples_read);
|
||||
|
||||
// Send results to visualization task via queue
|
||||
if (xQueueSend(audioQueue, &samples_read, 0) != pdTRUE) {
|
||||
// Queue full, just skip this update
|
||||
}
|
||||
}
|
||||
|
||||
// Small delay to prevent watchdog trigger
|
||||
vTaskDelay(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Visualization and network task running on Core 0
|
||||
void visualizationTask(void *parameter) {
|
||||
while (true) {
|
||||
int samples_read;
|
||||
|
||||
// Check if there's new audio data to process
|
||||
if (xQueueReceive(audioQueue, &samples_read, 0) == pdTRUE) {
|
||||
uint32_t currentTime = millis();
|
||||
const auto& detectedNotes = noteDetector.getDetectedNotes();
|
||||
|
||||
// Update web clients with spectrum data
|
||||
if (currentTime - lastWebUpdateTime >= 50) {
|
||||
sendSpectrumData();
|
||||
lastWebUpdateTime = currentTime;
|
||||
}
|
||||
|
||||
// Show spectrum if enabled
|
||||
if (showSpectrum &&
|
||||
!noteDetector.isCalibrating() &&
|
||||
currentTime - lastSpectrumPrintTime >= Config::DEBUG_INTERVAL_MS) {
|
||||
SpectrumVisualizer::visualizeSpectrum(noteDetector.getSpectrum(), Config::FFT_SIZE);
|
||||
lastSpectrumPrintTime = currentTime;
|
||||
}
|
||||
|
||||
// Print detected notes at specified interval
|
||||
if (currentTime - lastNotePrintTime >= Config::NOTE_PRINT_INTERVAL_MS) {
|
||||
if (!detectedNotes.empty() && !noteDetector.isCalibrating()) {
|
||||
SpectrumVisualizer::visualizeNotes(detectedNotes);
|
||||
}
|
||||
lastNotePrintTime = currentTime;
|
||||
}
|
||||
}
|
||||
|
||||
ws.cleanupClients();
|
||||
handleSerialCommands();
|
||||
|
||||
// Small delay to prevent watchdog trigger
|
||||
vTaskDelay(1);
|
||||
}
|
||||
}
|
||||
|
||||
void setup() {
|
||||
Serial.begin(Config::SERIAL_BAUD_RATE);
|
||||
while(!Serial) {
|
||||
delay(10);
|
||||
}
|
||||
|
||||
initWiFi();
|
||||
initWebServer();
|
||||
initI2S();
|
||||
|
||||
Serial.println("Piano Note Detection Ready (C2-C6)");
|
||||
Serial.println("Press 'h' for help");
|
||||
noteDetector.beginCalibration();
|
||||
Serial.begin(115200);
|
||||
AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Warning);
|
||||
|
||||
// Create queue for inter-core communication
|
||||
audioQueue = xQueueCreate(4, sizeof(int));
|
||||
if (audioQueue == nullptr) {
|
||||
Serial.println("Failed to create queue!");
|
||||
return;
|
||||
}
|
||||
// Configure I2SStream for INMP441
|
||||
auto cfg = i2sStream.defaultConfig(RX_MODE);
|
||||
cfg.i2s_format = I2S_STD_FORMAT;
|
||||
cfg.bits_per_sample = input_bits_per_sample;
|
||||
cfg.channels = channels;
|
||||
cfg.sample_rate = samples_per_second;
|
||||
cfg.is_master = true;
|
||||
cfg.pin_bck = 8; // SCK
|
||||
cfg.pin_ws = 9; // WS
|
||||
cfg.pin_data = 10; // SD
|
||||
i2sStream.begin(cfg);
|
||||
|
||||
// Create audio processing task on Core 1
|
||||
xTaskCreatePinnedToCore(
|
||||
audioProcessingTask,
|
||||
"AudioTask",
|
||||
Config::TASK_STACK_SIZE,
|
||||
nullptr,
|
||||
Config::TASK_PRIORITY,
|
||||
&audioTaskHandle,
|
||||
1 // Run on Core 1
|
||||
);
|
||||
// Configure FormatConverterStream to convert 32-bit to 16-bit
|
||||
converter.begin(from, to); // Convert to 16-bit
|
||||
|
||||
// Configure FFT
|
||||
auto tcfg = fft.defaultConfig();
|
||||
tcfg.length = 8192; // 186ms @ 11kHz minimun C2 theoretical
|
||||
tcfg.channels = channels;
|
||||
tcfg.sample_rate = samples_per_second;
|
||||
tcfg.bits_per_sample = fft_bits_per_sample; // FFT expects 16-bit data after conversion
|
||||
tcfg.callback = &fftResult;
|
||||
fft.begin(tcfg);
|
||||
|
||||
Serial.println("Setup complete. Listening...");
|
||||
|
||||
// Create visualization task on Core 0
|
||||
xTaskCreatePinnedToCore(
|
||||
visualizationTask,
|
||||
"VisualTask",
|
||||
Config::TASK_STACK_SIZE,
|
||||
nullptr,
|
||||
1, // Lower priority than audio task
|
||||
&visualizationTaskHandle,
|
||||
0 // Run on Core 0
|
||||
);
|
||||
}
|
||||
|
||||
void loop() {
|
||||
// Main loop is now empty as all work is done in tasks
|
||||
vTaskDelete(nullptr); // Delete the main loop task
|
||||
}
|
||||
copier.copy(); // Stream mic data into FFT processor
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user