Compare commits
7 Commits
main
...
serial_plo
| Author | SHA1 | Date | |
|---|---|---|---|
| 4992b75d57 | |||
| f500937067 | |||
| 5a8dc9c489 | |||
| 91b24e0da0 | |||
| c83d04eb23 | |||
| 7331d2fe01 | |||
| 178bfc630a |
317
README.md
317
README.md
@@ -1,2 +1,317 @@
|
|||||||
# esp32i2s
|
# ESP32 Piano Note Detection System
|
||||||
|
|
||||||
|
A real-time piano note detection system implemented on ESP32 using I2S microphone input. This system can detect musical notes from C2 to C6 with adjustable sensitivity and visualization options.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- Real-time audio processing using I2S microphone
|
||||||
|
- FFT-based frequency analysis
|
||||||
|
- Note detection from C2 (65.41 Hz) to C6 (1046.50 Hz)
|
||||||
|
- Dynamic threshold calibration
|
||||||
|
- Multiple note detection (up to 7 simultaneous notes)
|
||||||
|
- Harmonic filtering
|
||||||
|
- Real-time spectrum visualization
|
||||||
|
- Note timing and duration tracking
|
||||||
|
- Interactive Serial commands for system tuning
|
||||||
|
|
||||||
|
## Hardware Requirements
|
||||||
|
|
||||||
|
- ESP32 development board
|
||||||
|
- I2S MEMS microphone (e.g., INMP441, SPH0645)
|
||||||
|
- USB connection for Serial monitoring
|
||||||
|
|
||||||
|
## Pin Configuration
|
||||||
|
|
||||||
|
The system uses the following I2S pins by default (configurable in Config.h):
|
||||||
|
- SCK (Serial Clock): GPIO 8
|
||||||
|
- WS/LRC (Word Select/Left-Right Clock): GPIO 9
|
||||||
|
- SD (Serial Data): GPIO 10
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
1. Connect the I2S microphone to the ESP32 according to the pin configuration
|
||||||
|
2. Build and flash the project to your ESP32
|
||||||
|
3. Open a Serial monitor at 115200 baud
|
||||||
|
4. Follow the calibration process on first run
|
||||||
|
|
||||||
|
## Serial Commands
|
||||||
|
|
||||||
|
The system can be controlled via Serial commands:
|
||||||
|
|
||||||
|
- `h` - Display help menu
|
||||||
|
- `c` - Start calibration process
|
||||||
|
- `+` - Increase sensitivity (threshold up)
|
||||||
|
- `-` - Decrease sensitivity (threshold down)
|
||||||
|
- `s` - Toggle spectrum visualization
|
||||||
|
|
||||||
|
## Configuration Options
|
||||||
|
|
||||||
|
All system parameters can be adjusted in `Config.h`:
|
||||||
|
|
||||||
|
### Audio Processing
|
||||||
|
- `SAMPLE_RATE`: 8000 Hz (good for frequencies up to 4kHz)
|
||||||
|
- `BITS_PER_SAMPLE`: 16-bit resolution
|
||||||
|
- `SAMPLE_BUFFER_SIZE`: 1024 samples
|
||||||
|
- `FFT_SIZE`: 1024 points
|
||||||
|
|
||||||
|
### Note Detection
|
||||||
|
- `NOTE_FREQ_C2`: 65.41 Hz (lowest detectable note)
|
||||||
|
- `NOTE_FREQ_C6`: 1046.50 Hz (highest detectable note)
|
||||||
|
- `FREQUENCY_TOLERANCE`: 3.0 Hz
|
||||||
|
- `MAX_SIMULTANEOUS_NOTES`: 7
|
||||||
|
- `MIN_NOTE_DURATION_MS`: 50ms
|
||||||
|
- `NOTE_RELEASE_TIME_MS`: 100ms
|
||||||
|
|
||||||
|
### Calibration
|
||||||
|
- `CALIBRATION_DURATION_MS`: 5000ms
|
||||||
|
- `CALIBRATION_PEAK_PERCENTILE`: 0.95 (95th percentile)
|
||||||
|
|
||||||
|
## Visualization
|
||||||
|
|
||||||
|
The system provides two visualization modes:
|
||||||
|
|
||||||
|
1. Note Display:
|
||||||
|
```
|
||||||
|
Current Notes:
|
||||||
|
A4 (440.0 Hz, Magnitude: 2500, Duration: 250ms)
|
||||||
|
E5 (659.3 Hz, Magnitude: 1800, Duration: 150ms)
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Spectrum Display (when enabled):
|
||||||
|
```
|
||||||
|
Frequency Spectrum:
|
||||||
|
0Hz |▄▄▄▄▄
|
||||||
|
100Hz |██████▄
|
||||||
|
200Hz |▄▄▄
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Performance Tuning
|
||||||
|
|
||||||
|
1. Start with calibration by pressing 'c' in a quiet environment
|
||||||
|
2. Play notes and observe the detection accuracy
|
||||||
|
3. Use '+' and '-' to adjust sensitivity if needed
|
||||||
|
4. Enable spectrum display with 's' to visualize frequency content
|
||||||
|
5. Adjust `Config.h` parameters if needed for your specific setup
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
- Uses FFT for frequency analysis
|
||||||
|
- Implements peak detection with dynamic thresholding
|
||||||
|
- Filters out harmonics to prevent duplicate detections
|
||||||
|
- Tracks note timing and duration
|
||||||
|
- Uses ring buffer for real-time processing
|
||||||
|
- Calibration collects ambient noise profile
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
1. No notes detected:
|
||||||
|
- Check microphone connection
|
||||||
|
- Run calibration
|
||||||
|
- Increase sensitivity with '+'
|
||||||
|
- Verify audio input level in spectrum display
|
||||||
|
|
||||||
|
2. False detections:
|
||||||
|
- Run calibration in a quiet environment
|
||||||
|
- Decrease sensitivity with '-'
|
||||||
|
- Adjust `PEAK_RATIO_THRESHOLD` in Config.h
|
||||||
|
|
||||||
|
3. Missing notes:
|
||||||
|
- Check if notes are within C2-C6 range
|
||||||
|
- Increase `FREQUENCY_TOLERANCE`
|
||||||
|
- Decrease `MIN_MAGNITUDE_THRESHOLD`
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
Contributions are welcome! Please read the contributing guidelines before submitting pull requests.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is licensed under the MIT License - see the LICENSE file for details.
|
||||||
|
|
||||||
|
## Development Environment Setup
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- PlatformIO IDE (recommended) or Arduino IDE
|
||||||
|
- ESP32 board support package
|
||||||
|
- Required libraries:
|
||||||
|
- arduino-audio-tools
|
||||||
|
- arduino-audio-driver
|
||||||
|
- WiFiManager
|
||||||
|
- AsyncTCP
|
||||||
|
- ESPAsyncWebServer
|
||||||
|
- arduinoFFT
|
||||||
|
|
||||||
|
### Building with PlatformIO
|
||||||
|
1. Clone the repository
|
||||||
|
2. Open the project in PlatformIO
|
||||||
|
3. Install dependencies:
|
||||||
|
```
|
||||||
|
pio lib install
|
||||||
|
```
|
||||||
|
4. Build and upload:
|
||||||
|
```
|
||||||
|
pio run -t upload
|
||||||
|
```
|
||||||
|
|
||||||
|
## Memory Management
|
||||||
|
|
||||||
|
### Memory Usage
|
||||||
|
- Program Memory: ~800KB
|
||||||
|
- RAM Usage: ~100KB
|
||||||
|
- DMA Buffers: 4 x 512 bytes
|
||||||
|
- FFT Working Buffer: 2048 bytes (1024 samples x 2 bytes)
|
||||||
|
|
||||||
|
### Optimization Tips
|
||||||
|
- Adjust `DMA_BUFFER_COUNT` based on available RAM
|
||||||
|
- Reduce `SAMPLE_BUFFER_SIZE` for lower latency
|
||||||
|
- Use `PSRAM` if available for larger buffer sizes
|
||||||
|
|
||||||
|
## Advanced Configuration
|
||||||
|
|
||||||
|
### Task Management
|
||||||
|
- Audio processing task on Core 1:
|
||||||
|
- I2S sample reading
|
||||||
|
- Audio level tracking
|
||||||
|
- Note detection and FFT analysis
|
||||||
|
- Visualization task on Core 0:
|
||||||
|
- WebSocket communication
|
||||||
|
- Spectrum visualization
|
||||||
|
- Serial interface
|
||||||
|
- Network operations
|
||||||
|
- Inter-core communication via FreeRTOS queue
|
||||||
|
- Configurable priorities in `Config.h`
|
||||||
|
|
||||||
|
### Audio Pipeline
|
||||||
|
1. I2S DMA Input
|
||||||
|
2. Sample Buffer Collection
|
||||||
|
3. FFT Processing
|
||||||
|
4. Peak Detection
|
||||||
|
5. Note Identification
|
||||||
|
6. Output Generation
|
||||||
|
|
||||||
|
### Timing Parameters
|
||||||
|
- Audio Buffer Processing: ~8ms
|
||||||
|
- FFT Computation: ~5ms
|
||||||
|
- Note Detection: ~2ms
|
||||||
|
- Total Latency: ~15-20ms
|
||||||
|
|
||||||
|
## Performance Optimization
|
||||||
|
|
||||||
|
### CPU Usage
|
||||||
|
- Core 1 (Audio Processing):
|
||||||
|
- I2S DMA handling: ~15%
|
||||||
|
- Audio analysis: ~20%
|
||||||
|
- FFT processing: ~15%
|
||||||
|
- Core 0 (Visualization):
|
||||||
|
- WebSocket updates: ~5%
|
||||||
|
- Visualization: ~5%
|
||||||
|
- Network handling: ~5%
|
||||||
|
|
||||||
|
### Memory Optimization
|
||||||
|
1. Buffer Size Selection:
|
||||||
|
- Larger buffers: Better frequency resolution
|
||||||
|
- Smaller buffers: Lower latency
|
||||||
|
2. DMA Configuration:
|
||||||
|
- More buffers: Better continuity
|
||||||
|
- Fewer buffers: Lower memory usage
|
||||||
|
|
||||||
|
### Frequency Analysis
|
||||||
|
- FFT Resolution: 7.8125 Hz (8000/1024)
|
||||||
|
- Frequency Bins: 512 (Nyquist limit)
|
||||||
|
- Useful Range: 65.41 Hz to 1046.50 Hz
|
||||||
|
- Window Function: Hamming
|
||||||
|
|
||||||
|
## Technical Details
|
||||||
|
|
||||||
|
### Microphone Specifications
|
||||||
|
- Supply Voltage: 3.3V
|
||||||
|
- Sampling Rate: 8kHz
|
||||||
|
- Bit Depth: 16-bit
|
||||||
|
- SNR: >65dB (typical)
|
||||||
|
|
||||||
|
### Signal Processing
|
||||||
|
1. Pre-processing:
|
||||||
|
- DC offset removal
|
||||||
|
- Windowing function application
|
||||||
|
2. FFT Processing:
|
||||||
|
- 1024-point real FFT
|
||||||
|
- Magnitude calculation
|
||||||
|
3. Post-processing:
|
||||||
|
- Peak detection
|
||||||
|
- Harmonic filtering
|
||||||
|
- Note matching
|
||||||
|
|
||||||
|
### Calibration Process
|
||||||
|
1. Ambient Noise Collection (5 seconds)
|
||||||
|
2. Frequency Bin Analysis
|
||||||
|
3. Threshold Calculation:
|
||||||
|
- Base threshold from 95th percentile
|
||||||
|
- Per-bin noise floor mapping
|
||||||
|
4. Dynamic Adjustment
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
1. I2S Communication Errors:
|
||||||
|
- Check pin connections
|
||||||
|
- Verify I2S configuration
|
||||||
|
- Monitor serial output for error codes
|
||||||
|
2. Memory Issues:
|
||||||
|
- Watch heap fragmentation
|
||||||
|
- Monitor stack usage
|
||||||
|
- Check DMA buffer allocation
|
||||||
|
|
||||||
|
### Error Recovery
|
||||||
|
- Automatic I2S reset on communication errors
|
||||||
|
- Dynamic threshold adjustment
|
||||||
|
- Watchdog timer protection
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
1. AudioLevelTracker
|
||||||
|
- Real-time audio level monitoring
|
||||||
|
- Peak detection
|
||||||
|
- Threshold management
|
||||||
|
2. NoteDetector
|
||||||
|
- Frequency analysis
|
||||||
|
- Note identification
|
||||||
|
- Harmonic filtering
|
||||||
|
3. SpectrumVisualizer
|
||||||
|
- Real-time spectrum display
|
||||||
|
- Magnitude scaling
|
||||||
|
- ASCII visualization
|
||||||
|
|
||||||
|
### File Organization
|
||||||
|
- `/src`: Core implementation files
|
||||||
|
- `/include`: Header files and configurations
|
||||||
|
- `/data`: Additional resources
|
||||||
|
- `/test`: Unit tests
|
||||||
|
|
||||||
|
## Inter-Core Communication
|
||||||
|
|
||||||
|
### Queue Management
|
||||||
|
- FreeRTOS queue for audio data transfer
|
||||||
|
- 4-slot queue buffer
|
||||||
|
- Zero-copy data passing
|
||||||
|
- Non-blocking queue operations
|
||||||
|
- Automatic overflow protection
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
1. Core 1 (Audio Task):
|
||||||
|
- Processes audio samples
|
||||||
|
- Performs FFT analysis
|
||||||
|
- Queues processed data
|
||||||
|
2. Core 0 (Visualization Task):
|
||||||
|
- Receives processed data
|
||||||
|
- Updates visualization
|
||||||
|
- Handles network communication
|
||||||
|
|
||||||
|
### Network Communication
|
||||||
|
- Asynchronous WebSocket updates
|
||||||
|
- JSON-formatted spectrum data
|
||||||
|
- Configurable update rate (50ms default)
|
||||||
|
- Automatic client cleanup
|
||||||
|
- Efficient connection management
|
||||||
|
|
||||||
|
|||||||
260
data/index.html
Normal file
260
data/index.html
Normal file
@@ -0,0 +1,260 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html>
|
||||||
|
<head>
|
||||||
|
<title>ESP32 Piano Spectrum Analyzer</title>
|
||||||
|
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
|
||||||
|
<script src="https://cdn.jsdelivr.net/npm/chartjs-plugin-annotation"></script>
|
||||||
|
<style>
|
||||||
|
body {
|
||||||
|
font-family: Arial, sans-serif;
|
||||||
|
margin: 20px;
|
||||||
|
background: #f0f0f0;
|
||||||
|
}
|
||||||
|
.container {
|
||||||
|
max-width: 1200px;
|
||||||
|
margin: 0 auto;
|
||||||
|
background: white;
|
||||||
|
padding: 20px;
|
||||||
|
border-radius: 10px;
|
||||||
|
box-shadow: 0 0 10px rgba(0,0,0,0.1);
|
||||||
|
}
|
||||||
|
h1 {
|
||||||
|
color: #333;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
#spectrum-container {
|
||||||
|
position: relative;
|
||||||
|
height: 400px;
|
||||||
|
margin: 20px 0;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="container">
|
||||||
|
<h1>Piano Spectrum Analyzer</h1>
|
||||||
|
<div id="spectrum-container">
|
||||||
|
<canvas id="spectrumChart"></canvas>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
Chart.register('chartjs-plugin-annotation');
|
||||||
|
|
||||||
|
// Piano note frequencies (Hz)
|
||||||
|
const noteFrequencies = {
|
||||||
|
'C2': 65.41, 'C#2': 69.30, 'D2': 73.42, 'D#2': 77.78, 'E2': 82.41, 'F2': 87.31,
|
||||||
|
'F#2': 92.50, 'G2': 98.00, 'G#2': 103.83, 'A2': 110.00, 'A#2': 116.54, 'B2': 123.47,
|
||||||
|
'C3': 130.81, 'C#3': 138.59, 'D3': 146.83, 'D#3': 155.56, 'E3': 164.81, 'F3': 174.61,
|
||||||
|
'F#3': 185.00, 'G3': 196.00, 'G#3': 207.65, 'A3': 220.00, 'A#3': 233.08, 'B3': 246.94,
|
||||||
|
'C4': 261.63, 'C#4': 277.18, 'D4': 293.66, 'D#4': 311.13, 'E4': 329.63, 'F4': 349.23,
|
||||||
|
'F#4': 369.99, 'G4': 392.00, 'G#4': 415.30, 'A4': 440.00, 'A#4': 466.16, 'B4': 493.88,
|
||||||
|
'C5': 523.25, 'C#5': 554.37, 'D5': 587.33, 'D#5': 622.25, 'E5': 659.26, 'F5': 698.46,
|
||||||
|
'F#5': 739.99, 'G5': 783.99, 'G#5': 830.61, 'A5': 880.00, 'A#5': 932.33, 'B5': 987.77,
|
||||||
|
'C6': 1046.50
|
||||||
|
};
|
||||||
|
|
||||||
|
// Create ticks for the x-axis
|
||||||
|
const xAxisTicks = Object.entries(noteFrequencies).filter(([note]) =>
|
||||||
|
note.length === 2 || note === 'C#4' || note === 'F#4' || note === 'A#4'
|
||||||
|
).map(([note, freq]) => ({
|
||||||
|
value: freq,
|
||||||
|
label: note
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Keep track of recent maximum values
|
||||||
|
const maxValueHistory = [];
|
||||||
|
const MAX_HISTORY_LENGTH = 5;
|
||||||
|
|
||||||
|
function updateYAxisScale(newValue) {
|
||||||
|
maxValueHistory.push(newValue);
|
||||||
|
if (maxValueHistory.length > MAX_HISTORY_LENGTH) {
|
||||||
|
maxValueHistory.shift();
|
||||||
|
}
|
||||||
|
return Math.max(...maxValueHistory) * 1.1; // Add 10% margin
|
||||||
|
}
|
||||||
|
|
||||||
|
const ctx = document.getElementById('spectrumChart').getContext('2d');
|
||||||
|
const chart = new Chart(ctx, {
|
||||||
|
type: 'line',
|
||||||
|
data: {
|
||||||
|
// Generate logarithmically spaced frequencies for labels
|
||||||
|
labels: Array.from({length: 134}, (_, i) => {
|
||||||
|
const minFreq = 60;
|
||||||
|
const maxFreq = 1100;
|
||||||
|
return Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
||||||
|
}),
|
||||||
|
datasets: [{
|
||||||
|
label: 'Frequency Spectrum',
|
||||||
|
data: Array(134).fill(0),
|
||||||
|
borderColor: 'rgb(75, 192, 192)',
|
||||||
|
tension: 0.1,
|
||||||
|
fill: true,
|
||||||
|
backgroundColor: 'rgba(75, 192, 192, 0.2)',
|
||||||
|
pointRadius: 0 // Hide points for better performance
|
||||||
|
}]
|
||||||
|
},
|
||||||
|
options: {
|
||||||
|
responsive: true,
|
||||||
|
maintainAspectRatio: false,
|
||||||
|
animation: {
|
||||||
|
duration: 0
|
||||||
|
},
|
||||||
|
parsing: {
|
||||||
|
xAxisKey: 'x',
|
||||||
|
yAxisKey: 'y'
|
||||||
|
},
|
||||||
|
scales: {
|
||||||
|
y: {
|
||||||
|
beginAtZero: true,
|
||||||
|
max: 5000,
|
||||||
|
adapters: {
|
||||||
|
update: function(maxValue) {
|
||||||
|
return updateYAxisScale(maxValue);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
title: {
|
||||||
|
display: true,
|
||||||
|
text: 'Magnitude'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
x: {
|
||||||
|
type: 'logarithmic',
|
||||||
|
position: 'bottom',
|
||||||
|
min: 60,
|
||||||
|
max: 1100,
|
||||||
|
title: {
|
||||||
|
display: true,
|
||||||
|
text: 'Frequency (Hz)'
|
||||||
|
},
|
||||||
|
ticks: {
|
||||||
|
callback: function(value) {
|
||||||
|
// Show C notes and F# notes
|
||||||
|
const entries = Object.entries(noteFrequencies);
|
||||||
|
const closest = entries.reduce((prev, curr) => {
|
||||||
|
return Math.abs(curr[1] - value) < Math.abs(prev[1] - value) ? curr : prev;
|
||||||
|
});
|
||||||
|
|
||||||
|
if ((closest[0].includes('C') || closest[0].includes('F#')) &&
|
||||||
|
Math.abs(closest[1] - value) < 1) {
|
||||||
|
return closest[0];
|
||||||
|
}
|
||||||
|
return '';
|
||||||
|
},
|
||||||
|
sampleSize: 20,
|
||||||
|
autoSkip: false
|
||||||
|
},
|
||||||
|
grid: {
|
||||||
|
color: (ctx) => {
|
||||||
|
const value = ctx.tick.value;
|
||||||
|
// Check if this tick corresponds to a C note
|
||||||
|
if (Object.entries(noteFrequencies)
|
||||||
|
.some(([note, freq]) =>
|
||||||
|
note.startsWith('C') && Math.abs(freq - value) < 1)) {
|
||||||
|
return 'rgba(255, 0, 0, 0.1)';
|
||||||
|
}
|
||||||
|
return 'rgba(0, 0, 0, 0.1)';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
plugins: {
|
||||||
|
annotation: {
|
||||||
|
annotations: Object.entries(noteFrequencies)
|
||||||
|
.filter(([note]) => note.startsWith('C'))
|
||||||
|
.reduce((acc, [note, freq]) => {
|
||||||
|
acc[note] = {
|
||||||
|
type: 'line',
|
||||||
|
xMin: freq,
|
||||||
|
xMax: freq,
|
||||||
|
borderColor: 'rgba(255, 99, 132, 0.5)',
|
||||||
|
borderWidth: 1,
|
||||||
|
label: {
|
||||||
|
content: note,
|
||||||
|
enabled: true,
|
||||||
|
position: 'top'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
return acc;
|
||||||
|
}, {})
|
||||||
|
},
|
||||||
|
tooltip: {
|
||||||
|
callbacks: {
|
||||||
|
title: function(context) {
|
||||||
|
const freq = context[0].parsed.x;
|
||||||
|
// Find the closest note
|
||||||
|
const closestNote = Object.entries(noteFrequencies)
|
||||||
|
.reduce((closest, [note, noteFreq]) => {
|
||||||
|
return Math.abs(noteFreq - freq) < Math.abs(noteFreq - closest.freq)
|
||||||
|
? { note, freq: noteFreq }
|
||||||
|
: closest;
|
||||||
|
}, { note: '', freq: Infinity });
|
||||||
|
|
||||||
|
return `Frequency: ${freq.toFixed(1)} Hz (Near ${closestNote.note})`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
legend: {
|
||||||
|
display: false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const wsUrl = `ws://${window.location.hostname}/ws`;
|
||||||
|
const ws = new WebSocket(wsUrl);
|
||||||
|
|
||||||
|
function interpolateSpectrum(spectrum) {
|
||||||
|
const result = [];
|
||||||
|
const minFreq = 60;
|
||||||
|
const maxFreq = 1100;
|
||||||
|
const binWidth = 8000 / 1024; // Hz per bin
|
||||||
|
|
||||||
|
// Generate logarithmically spaced frequencies
|
||||||
|
for (let i = 0; i < 134; i++) {
|
||||||
|
const targetFreq = Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
||||||
|
|
||||||
|
// Find the corresponding linear bin
|
||||||
|
const bin = Math.floor(targetFreq / binWidth);
|
||||||
|
|
||||||
|
if (bin >= 8 && bin < 141) {
|
||||||
|
// Linear interpolation between bins
|
||||||
|
const binFraction = (targetFreq / binWidth) - bin;
|
||||||
|
const value = spectrum[bin - 8] * (1 - binFraction) +
|
||||||
|
(bin - 7 < spectrum.length ? spectrum[bin - 7] : 0) * binFraction;
|
||||||
|
result.push({
|
||||||
|
x: targetFreq,
|
||||||
|
y: value
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
result.push({
|
||||||
|
x: targetFreq,
|
||||||
|
y: 0
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
ws.onmessage = function(event) {
|
||||||
|
const spectrum = JSON.parse(event.data);
|
||||||
|
const interpolatedData = interpolateSpectrum(spectrum);
|
||||||
|
|
||||||
|
// Update y-axis scale based on new maximum value
|
||||||
|
const maxValue = Math.max(...interpolatedData.map(d => d.y));
|
||||||
|
chart.options.scales.y.max = updateYAxisScale(maxValue);
|
||||||
|
|
||||||
|
// Update chart data
|
||||||
|
chart.data.datasets[0].data = interpolatedData;
|
||||||
|
chart.update('none'); // Use 'none' mode for maximum performance
|
||||||
|
};
|
||||||
|
|
||||||
|
ws.onclose = function() {
|
||||||
|
console.log('WebSocket connection closed');
|
||||||
|
setTimeout(() => {
|
||||||
|
window.location.reload();
|
||||||
|
}, 1000);
|
||||||
|
};
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
24
include/AudioLevelTracker.h
Normal file
24
include/AudioLevelTracker.h
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <Arduino.h>
|
||||||
|
#include <deque>
|
||||||
|
#include "Config.h"
|
||||||
|
|
||||||
|
class AudioLevelTracker {
|
||||||
|
public:
|
||||||
|
AudioLevelTracker();
|
||||||
|
void updateMaxLevel(int16_t sample); // Changed to int16_t
|
||||||
|
int16_t getMaxLevel() const; // Changed to int16_t
|
||||||
|
void resetMaxLevel();
|
||||||
|
|
||||||
|
private:
|
||||||
|
struct SamplePoint {
|
||||||
|
uint32_t timestamp;
|
||||||
|
int16_t value; // Changed to int16_t
|
||||||
|
};
|
||||||
|
std::deque<SamplePoint> sampleHistory;
|
||||||
|
int16_t maxLevel; // Changed to int16_t
|
||||||
|
|
||||||
|
static const uint32_t HISTORY_DURATION_MS = 3000; // 3 seconds history
|
||||||
|
static const int16_t MAX_RANGE_LIMIT = Config::DEFAULT_RANGE_LIMIT; // Use 16-bit limit directly
|
||||||
|
};
|
||||||
90
include/Config.h
Normal file
90
include/Config.h
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
#pragma once
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace Config {
|
||||||
|
|
||||||
|
// Serial Configuration
|
||||||
|
constexpr int SERIAL_BAUD_RATE = 115200; // Serial communication baud rate
|
||||||
|
|
||||||
|
// WiFi Configuration
|
||||||
|
constexpr const char* WIFI_AP_NAME = "ESP32Piano"; // AP name when in configuration mode
|
||||||
|
constexpr const char* WIFI_AP_PASSWORD = "12345678"; // AP password when in configuration mode
|
||||||
|
constexpr uint32_t WIFI_CONFIG_TIMEOUT = 180; // Seconds to wait for WiFi configuration
|
||||||
|
constexpr uint32_t WIFI_CONFIG_PORT = 80; // Web configuration portal port
|
||||||
|
|
||||||
|
// I2S Pin Configuration
|
||||||
|
constexpr int I2S_MIC_SERIAL_CLOCK = 8; // SCK
|
||||||
|
constexpr int I2S_MIC_LEFT_RIGHT_CLOCK = 9; // WS/LRC
|
||||||
|
constexpr int I2S_MIC_SERIAL_DATA = 10; // SD
|
||||||
|
|
||||||
|
// I2S Configuration
|
||||||
|
constexpr int SAMPLE_RATE = 8000; // Hz - good for frequencies up to 4kHz
|
||||||
|
constexpr int BITS_PER_SAMPLE = 16; // Changed to 16-bit
|
||||||
|
constexpr int CHANNELS = 1; // Mono input
|
||||||
|
constexpr int SAMPLE_BUFFER_SIZE = 1024; // Match FFT_SIZE for efficiency
|
||||||
|
|
||||||
|
// DMA Configuration
|
||||||
|
constexpr int DMA_BUFFER_COUNT = 4; // Reduced for lower latency
|
||||||
|
constexpr int DMA_BUFFER_LEN = 512; // Smaller chunks for faster processing
|
||||||
|
|
||||||
|
// Audio Processing
|
||||||
|
constexpr float DC_OFFSET = 0.0f; // DC offset correction
|
||||||
|
constexpr float GAIN = 4.0f; // Increased gain for better note detection
|
||||||
|
constexpr int16_t NOISE_THRESHOLD = 100; // Lower threshold for 16-bit samples
|
||||||
|
constexpr int16_t DEFAULT_RANGE_LIMIT = 32767; // Max value for 16-bit
|
||||||
|
constexpr float DECAY_FACTOR = 0.7f; // Faster decay for quicker note changes
|
||||||
|
|
||||||
|
// FFT Configuration
|
||||||
|
constexpr int FFT_SIZE = 1024; // Must be power of 2, gives ~7.8 Hz resolution at 8kHz
|
||||||
|
constexpr float FREQUENCY_RESOLUTION = static_cast<float>(SAMPLE_RATE) / FFT_SIZE;
|
||||||
|
|
||||||
|
// Piano Note Frequencies (Hz)
|
||||||
|
constexpr float NOTE_FREQ_C2 = 65.41f; // Lowest note we'll detect
|
||||||
|
constexpr float NOTE_FREQ_C3 = 130.81f;
|
||||||
|
constexpr float NOTE_FREQ_C4 = 261.63f; // Middle C
|
||||||
|
constexpr float NOTE_FREQ_C5 = 523.25f;
|
||||||
|
constexpr float NOTE_FREQ_C6 = 1046.50f; // Highest note we'll detect
|
||||||
|
|
||||||
|
// Note Detection Parameters
|
||||||
|
constexpr float FREQUENCY_TOLERANCE = 3.0f; // Hz tolerance, increased for better detection
|
||||||
|
constexpr float MIN_MAGNITUDE_THRESHOLD = 500.0f; // Adjusted for 16-bit samples
|
||||||
|
constexpr int MAX_SIMULTANEOUS_NOTES = 7; // Maximum number of notes to detect simultaneously
|
||||||
|
constexpr float PEAK_RATIO_THRESHOLD = 0.1f; // Peaks must be at least this ratio of the strongest peak
|
||||||
|
|
||||||
|
// Timing and Debug
|
||||||
|
constexpr uint32_t LEVEL_UPDATE_INTERVAL_MS = 50; // Faster updates for better responsiveness
|
||||||
|
constexpr bool ENABLE_DEBUG = true; // Enable debug output
|
||||||
|
constexpr int DEBUG_INTERVAL_MS = 1000; // Debug print interval
|
||||||
|
|
||||||
|
// System Configuration
|
||||||
|
constexpr uint32_t TASK_STACK_SIZE = 4096; // Audio task stack size
|
||||||
|
constexpr uint8_t TASK_PRIORITY = 2; // Increased priority for more consistent timing
|
||||||
|
constexpr uint8_t TASK_CORE = 1; // Run on second core to avoid interference
|
||||||
|
|
||||||
|
// Calibration Parameters
|
||||||
|
constexpr bool CALIBRATION_MODE = false; // Set to true to enable calibration output
|
||||||
|
constexpr int CALIBRATION_DURATION_MS = 5000; // Duration to collect calibration data
|
||||||
|
constexpr float CALIBRATION_PEAK_PERCENTILE = 0.95f; // Use 95th percentile for thresholds
|
||||||
|
|
||||||
|
// Note Detection Timing
|
||||||
|
constexpr int NOTE_PRINT_INTERVAL_MS = 100; // How often to print detected notes
|
||||||
|
constexpr int MIN_NOTE_DURATION_MS = 50; // Minimum duration to consider a note valid
|
||||||
|
constexpr int NOTE_RELEASE_TIME_MS = 100; // Time before considering a note released
|
||||||
|
|
||||||
|
// Serial Commands
|
||||||
|
constexpr char CMD_HELP = 'h'; // Show help
|
||||||
|
constexpr char CMD_CALIBRATE = 'c'; // Start calibration
|
||||||
|
constexpr char CMD_THRESHOLD_UP = '+'; // Increase sensitivity
|
||||||
|
constexpr char CMD_THRESHOLD_DOWN = '-'; // Decrease sensitivity
|
||||||
|
constexpr char CMD_TOGGLE_SPECTRUM = 's'; // Toggle spectrum display
|
||||||
|
constexpr float THRESHOLD_STEP = 0.1f; // Threshold adjustment step
|
||||||
|
|
||||||
|
// Command Response Messages
|
||||||
|
constexpr const char* MSG_HELP =
|
||||||
|
"Commands:\n"
|
||||||
|
"h - Show this help\n"
|
||||||
|
"c - Start calibration\n"
|
||||||
|
"+ - Increase sensitivity\n"
|
||||||
|
"- - Decrease sensitivity\n"
|
||||||
|
"s - Toggle spectrum display\n";
|
||||||
|
}
|
||||||
9
include/I2SConfig.h
Normal file
9
include/I2SConfig.h
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <Arduino.h>
|
||||||
|
#include <driver/i2s.h>
|
||||||
|
#include "Config.h"
|
||||||
|
|
||||||
|
// I2S setup functions
|
||||||
|
void initI2S();
|
||||||
|
void readI2SSamples(int16_t* samples, size_t* bytesRead);
|
||||||
48
include/NoteDetector.h
Normal file
48
include/NoteDetector.h
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <Arduino.h>
|
||||||
|
#include <arduinoFFT.h>
|
||||||
|
#include <vector>
|
||||||
|
#include <deque>
|
||||||
|
#include "Config.h"
|
||||||
|
|
||||||
|
struct DetectedNote {
|
||||||
|
float frequency;
|
||||||
|
float magnitude;
|
||||||
|
uint8_t noteNumber; // MIDI note number for easy identification
|
||||||
|
uint32_t startTime; // When the note was first detected
|
||||||
|
uint32_t lastSeenTime; // Last time this note was detected
|
||||||
|
};
|
||||||
|
|
||||||
|
class NoteDetector {
|
||||||
|
public:
|
||||||
|
NoteDetector();
|
||||||
|
void analyzeSamples(const int16_t* samples, size_t sampleCount);
|
||||||
|
const std::vector<DetectedNote>& getDetectedNotes() const { return detectedNotes; }
|
||||||
|
const double* getSpectrum() const { return vReal; }
|
||||||
|
void beginCalibration();
|
||||||
|
void endCalibration();
|
||||||
|
bool isCalibrating() const { return calibrationMode; }
|
||||||
|
|
||||||
|
// Threshold adjustment methods
|
||||||
|
void adjustThreshold(float delta);
|
||||||
|
float getCurrentThreshold() const { return dynamicThreshold; }
|
||||||
|
|
||||||
|
private:
|
||||||
|
arduinoFFT FFT;
|
||||||
|
double vReal[Config::FFT_SIZE];
|
||||||
|
double vImag[Config::FFT_SIZE];
|
||||||
|
std::vector<DetectedNote> detectedNotes;
|
||||||
|
std::deque<float> peakMagnitudeHistory; // For calibration
|
||||||
|
|
||||||
|
bool calibrationMode;
|
||||||
|
uint32_t calibrationStartTime;
|
||||||
|
float dynamicThreshold;
|
||||||
|
|
||||||
|
uint8_t freqToNoteNumber(float frequency) const;
|
||||||
|
bool isNoteFrequency(float frequency) const;
|
||||||
|
bool isHarmonic(float freq1, float freq2, float tolerance) const;
|
||||||
|
void findPeaks();
|
||||||
|
void updateNoteTimings();
|
||||||
|
void calculateDynamicThreshold();
|
||||||
|
};
|
||||||
27
include/SpectrumVisualizer.h
Normal file
27
include/SpectrumVisualizer.h
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <Arduino.h>
|
||||||
|
#include "Config.h"
|
||||||
|
#include "NoteDetector.h"
|
||||||
|
|
||||||
|
class SpectrumVisualizer {
|
||||||
|
public:
|
||||||
|
// Visualization settings
|
||||||
|
static constexpr int DISPLAY_WIDTH = 80; // Characters wide
|
||||||
|
static constexpr int DISPLAY_HEIGHT = 16; // Lines tall
|
||||||
|
static constexpr float DB_MIN = 30.0f; // Minimum dB to show
|
||||||
|
static constexpr float DB_MAX = 90.0f; // Maximum dB to show
|
||||||
|
|
||||||
|
static void visualizeSpectrum(const double* spectrum, int size);
|
||||||
|
static void visualizeNotes(const std::vector<DetectedNote>& notes);
|
||||||
|
static void drawFFTMagnitudes(const double* magnitudes, int size, bool logScale = true);
|
||||||
|
|
||||||
|
private:
|
||||||
|
static constexpr const char* noteNames[] = {
|
||||||
|
"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"
|
||||||
|
};
|
||||||
|
|
||||||
|
static float magnitudeToDb(double magnitude);
|
||||||
|
static int mapToDisplay(float value, float min, float max, int displayMin, int displayMax);
|
||||||
|
static void printBarGraph(float value, float maxValue, int width);
|
||||||
|
};
|
||||||
@@ -36,3 +36,4 @@ lib_deps =
|
|||||||
https://github.com/tzapu/WiFiManager.git
|
https://github.com/tzapu/WiFiManager.git
|
||||||
me-no-dev/AsyncTCP
|
me-no-dev/AsyncTCP
|
||||||
https://github.com/me-no-dev/ESPAsyncWebServer.git
|
https://github.com/me-no-dev/ESPAsyncWebServer.git
|
||||||
|
kosme/arduinoFFT@^1.6
|
||||||
|
|||||||
38
src/AudioLevelTracker.cpp
Normal file
38
src/AudioLevelTracker.cpp
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
#include "AudioLevelTracker.h"
|
||||||
|
|
||||||
|
AudioLevelTracker::AudioLevelTracker() {
|
||||||
|
resetMaxLevel();
|
||||||
|
}
|
||||||
|
|
||||||
|
void AudioLevelTracker::updateMaxLevel(int16_t sample) {
|
||||||
|
uint32_t currentTime = millis();
|
||||||
|
|
||||||
|
// Remove old samples (older than specified duration)
|
||||||
|
while (!sampleHistory.empty() &&
|
||||||
|
(currentTime - sampleHistory.front().timestamp) > HISTORY_DURATION_MS) {
|
||||||
|
sampleHistory.pop_front();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add new sample, but cap it at MAX_RANGE_LIMIT
|
||||||
|
int16_t absValue = abs(sample);
|
||||||
|
absValue = min(absValue, MAX_RANGE_LIMIT); // Cap at 16-bit limit
|
||||||
|
SamplePoint newPoint = {currentTime, absValue};
|
||||||
|
sampleHistory.push_back(newPoint);
|
||||||
|
|
||||||
|
// Update maximum
|
||||||
|
maxLevel = 0;
|
||||||
|
for (const auto& point : sampleHistory) {
|
||||||
|
if (point.value > maxLevel) {
|
||||||
|
maxLevel = point.value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
int16_t AudioLevelTracker::getMaxLevel() const {
|
||||||
|
return maxLevel;
|
||||||
|
}
|
||||||
|
|
||||||
|
void AudioLevelTracker::resetMaxLevel() {
|
||||||
|
maxLevel = 0;
|
||||||
|
sampleHistory.clear();
|
||||||
|
}
|
||||||
34
src/I2SConfig.cpp
Normal file
34
src/I2SConfig.cpp
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
#include "I2SConfig.h"
|
||||||
|
|
||||||
|
void initI2S() {
|
||||||
|
// I2S configuration
|
||||||
|
i2s_config_t i2s_config = {
|
||||||
|
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
|
||||||
|
.sample_rate = Config::SAMPLE_RATE,
|
||||||
|
.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
|
||||||
|
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
|
||||||
|
.communication_format = I2S_COMM_FORMAT_STAND_I2S,
|
||||||
|
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
|
||||||
|
.dma_buf_count = Config::DMA_BUFFER_COUNT,
|
||||||
|
.dma_buf_len = Config::DMA_BUFFER_LEN,
|
||||||
|
.use_apll = false,
|
||||||
|
.tx_desc_auto_clear = false,
|
||||||
|
.fixed_mclk = 0
|
||||||
|
};
|
||||||
|
|
||||||
|
// I2S pin configuration
|
||||||
|
i2s_pin_config_t i2s_mic_pins = {
|
||||||
|
.bck_io_num = Config::I2S_MIC_SERIAL_CLOCK,
|
||||||
|
.ws_io_num = Config::I2S_MIC_LEFT_RIGHT_CLOCK,
|
||||||
|
.data_out_num = I2S_PIN_NO_CHANGE,
|
||||||
|
.data_in_num = Config::I2S_MIC_SERIAL_DATA
|
||||||
|
};
|
||||||
|
|
||||||
|
// Install and configure I2S driver
|
||||||
|
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
|
||||||
|
i2s_set_pin(I2S_NUM_0, &i2s_mic_pins);
|
||||||
|
}
|
||||||
|
|
||||||
|
void readI2SSamples(int16_t* samples, size_t* bytesRead) {
|
||||||
|
i2s_read(I2S_NUM_0, samples, Config::SAMPLE_BUFFER_SIZE * sizeof(int16_t), bytesRead, portMAX_DELAY);
|
||||||
|
}
|
||||||
198
src/NoteDetector.cpp
Normal file
198
src/NoteDetector.cpp
Normal file
@@ -0,0 +1,198 @@
|
|||||||
|
#include "NoteDetector.h"
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cmath>
|
||||||
|
|
||||||
|
NoteDetector::NoteDetector()
|
||||||
|
: FFT(vReal, vImag, Config::FFT_SIZE, Config::SAMPLE_RATE)
|
||||||
|
, calibrationMode(false)
|
||||||
|
, calibrationStartTime(0)
|
||||||
|
, dynamicThreshold(Config::MIN_MAGNITUDE_THRESHOLD) {
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::beginCalibration() {
|
||||||
|
calibrationMode = true;
|
||||||
|
calibrationStartTime = millis();
|
||||||
|
peakMagnitudeHistory.clear();
|
||||||
|
Serial.println("Beginning calibration...");
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::endCalibration() {
|
||||||
|
calibrationMode = false;
|
||||||
|
calculateDynamicThreshold();
|
||||||
|
Serial.printf("Calibration complete. Dynamic threshold: %.2f\n", dynamicThreshold);
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::calculateDynamicThreshold() {
|
||||||
|
if (peakMagnitudeHistory.empty()) {
|
||||||
|
dynamicThreshold = Config::MIN_MAGNITUDE_THRESHOLD;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort magnitudes to find percentile
|
||||||
|
std::vector<float> sorted(peakMagnitudeHistory.begin(), peakMagnitudeHistory.end());
|
||||||
|
std::sort(sorted.begin(), sorted.end());
|
||||||
|
|
||||||
|
// Calculate threshold at specified percentile
|
||||||
|
size_t idx = static_cast<size_t>(sorted.size() * Config::CALIBRATION_PEAK_PERCENTILE);
|
||||||
|
dynamicThreshold = sorted[idx] * Config::PEAK_RATIO_THRESHOLD;
|
||||||
|
|
||||||
|
// Don't go below minimum threshold
|
||||||
|
dynamicThreshold = (dynamicThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
||||||
|
dynamicThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::updateNoteTimings() {
|
||||||
|
uint32_t currentTime = millis();
|
||||||
|
|
||||||
|
// Remove notes that haven't been seen recently
|
||||||
|
detectedNotes.erase(
|
||||||
|
std::remove_if(detectedNotes.begin(), detectedNotes.end(),
|
||||||
|
[currentTime](const DetectedNote& note) {
|
||||||
|
return currentTime - note.lastSeenTime > Config::NOTE_RELEASE_TIME_MS;
|
||||||
|
}),
|
||||||
|
detectedNotes.end()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::analyzeSamples(const int16_t* samples, size_t sampleCount) {
|
||||||
|
// Copy samples to FFT buffer with gain applied
|
||||||
|
for (size_t i = 0; i < Config::FFT_SIZE; i++) {
|
||||||
|
vReal[i] = i < sampleCount ? samples[i] * Config::GAIN : 0;
|
||||||
|
vImag[i] = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Perform FFT
|
||||||
|
FFT.DCRemoval();
|
||||||
|
FFT.Windowing(FFT_WIN_TYP_HANN, FFT_FORWARD);
|
||||||
|
FFT.Compute(FFT_FORWARD);
|
||||||
|
FFT.ComplexToMagnitude();
|
||||||
|
|
||||||
|
// Find peaks and update timings
|
||||||
|
findPeaks();
|
||||||
|
updateNoteTimings();
|
||||||
|
|
||||||
|
// Handle calibration mode
|
||||||
|
if (calibrationMode) {
|
||||||
|
// Store peak magnitudes during calibration
|
||||||
|
double maxMagnitude = 0;
|
||||||
|
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||||
|
maxMagnitude = std::max(maxMagnitude, vReal[i]);
|
||||||
|
}
|
||||||
|
peakMagnitudeHistory.push_back(maxMagnitude);
|
||||||
|
|
||||||
|
// Check if calibration period is over
|
||||||
|
if (millis() - calibrationStartTime >= Config::CALIBRATION_DURATION_MS) {
|
||||||
|
endCalibration();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
uint8_t NoteDetector::freqToNoteNumber(float frequency) const {
|
||||||
|
// A4 is 440Hz and MIDI note 69
|
||||||
|
return round(12 * log2(frequency/440.0) + 69);
|
||||||
|
}
|
||||||
|
|
||||||
|
bool NoteDetector::isNoteFrequency(float frequency) const {
|
||||||
|
// Check if frequency is within range of C2 to C6
|
||||||
|
if (frequency < (Config::NOTE_FREQ_C2 - Config::FREQUENCY_TOLERANCE) ||
|
||||||
|
frequency > (Config::NOTE_FREQ_C6 + Config::FREQUENCY_TOLERANCE)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate note number and then expected frequency
|
||||||
|
uint8_t noteNum = freqToNoteNumber(frequency);
|
||||||
|
float expectedFreq = 440.0f * pow(2.0f, (noteNum - 69) / 12.0f);
|
||||||
|
|
||||||
|
// Check if within tolerance
|
||||||
|
return abs(frequency - expectedFreq) <= Config::FREQUENCY_TOLERANCE;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool NoteDetector::isHarmonic(float freq1, float freq2, float tolerance) const {
|
||||||
|
// Check if freq2 is a harmonic of freq1
|
||||||
|
if (freq2 < freq1) {
|
||||||
|
std::swap(freq1, freq2);
|
||||||
|
}
|
||||||
|
|
||||||
|
float ratio = freq2 / freq1;
|
||||||
|
float nearest_harmonic = round(ratio);
|
||||||
|
|
||||||
|
return abs(ratio - nearest_harmonic) <= (tolerance / freq1);
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::findPeaks() {
|
||||||
|
std::vector<DetectedNote> candidates;
|
||||||
|
uint32_t currentTime = millis();
|
||||||
|
|
||||||
|
// Find the maximum magnitude
|
||||||
|
double maxMagnitude = 0;
|
||||||
|
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||||
|
maxMagnitude = (vReal[i] > maxMagnitude) ? vReal[i] : maxMagnitude;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use either calibrated or relative threshold
|
||||||
|
float relativeThreshold = static_cast<float>(maxMagnitude * Config::PEAK_RATIO_THRESHOLD);
|
||||||
|
float threshold = calibrationMode ? Config::MIN_MAGNITUDE_THRESHOLD :
|
||||||
|
(dynamicThreshold > relativeThreshold ? dynamicThreshold : relativeThreshold);
|
||||||
|
|
||||||
|
// First pass: find all potential peaks
|
||||||
|
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
||||||
|
float frequency = i * Config::FREQUENCY_RESOLUTION;
|
||||||
|
|
||||||
|
if (vReal[i] > threshold &&
|
||||||
|
vReal[i] > vReal[i-1] &&
|
||||||
|
vReal[i] > vReal[i+1] &&
|
||||||
|
isNoteFrequency(frequency)) {
|
||||||
|
|
||||||
|
DetectedNote note;
|
||||||
|
note.frequency = frequency;
|
||||||
|
note.magnitude = vReal[i];
|
||||||
|
note.noteNumber = freqToNoteNumber(frequency);
|
||||||
|
note.startTime = currentTime;
|
||||||
|
note.lastSeenTime = currentTime;
|
||||||
|
candidates.push_back(note);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort candidates by magnitude
|
||||||
|
std::sort(candidates.begin(), candidates.end(),
|
||||||
|
[](const DetectedNote& a, const DetectedNote& b) {
|
||||||
|
return a.magnitude > b.magnitude;
|
||||||
|
});
|
||||||
|
|
||||||
|
// Second pass: filter out harmonics and update existing notes
|
||||||
|
for (const auto& candidate : candidates) {
|
||||||
|
// Check if this note is already being tracked
|
||||||
|
auto existingNote = std::find_if(detectedNotes.begin(), detectedNotes.end(),
|
||||||
|
[&candidate](const DetectedNote& note) {
|
||||||
|
return note.noteNumber == candidate.noteNumber;
|
||||||
|
});
|
||||||
|
|
||||||
|
if (existingNote != detectedNotes.end()) {
|
||||||
|
// Update existing note
|
||||||
|
existingNote->magnitude = candidate.magnitude;
|
||||||
|
existingNote->lastSeenTime = currentTime;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if this is a harmonic of a stronger note
|
||||||
|
bool isHarmonicOfStronger = false;
|
||||||
|
for (const auto& note : detectedNotes) {
|
||||||
|
if (isHarmonic(candidate.frequency, note.frequency, Config::FREQUENCY_TOLERANCE)) {
|
||||||
|
isHarmonicOfStronger = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add new note if it's not a harmonic and we haven't hit the limit
|
||||||
|
if (!isHarmonicOfStronger && detectedNotes.size() < Config::MAX_SIMULTANEOUS_NOTES) {
|
||||||
|
detectedNotes.push_back(candidate);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoteDetector::adjustThreshold(float delta) {
|
||||||
|
float newThreshold = dynamicThreshold + (dynamicThreshold * delta);
|
||||||
|
dynamicThreshold = (newThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
||||||
|
newThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
||||||
|
Serial.printf("Threshold adjusted to: %.1f\n", dynamicThreshold);
|
||||||
|
}
|
||||||
82
src/SpectrumVisualizer.cpp
Normal file
82
src/SpectrumVisualizer.cpp
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
#include "SpectrumVisualizer.h"
|
||||||
|
#include <cmath>
|
||||||
|
|
||||||
|
constexpr const char* SpectrumVisualizer::noteNames[];
|
||||||
|
|
||||||
|
float SpectrumVisualizer::magnitudeToDb(double magnitude) {
|
||||||
|
return 20 * log10(magnitude);
|
||||||
|
}
|
||||||
|
|
||||||
|
int SpectrumVisualizer::mapToDisplay(float value, float min, float max, int displayMin, int displayMax) {
|
||||||
|
float normalized = (value - min) / (max - min);
|
||||||
|
normalized = constrain(normalized, 0.0f, 1.0f);
|
||||||
|
return displayMin + (normalized * (displayMax - displayMin));
|
||||||
|
}
|
||||||
|
|
||||||
|
void SpectrumVisualizer::printBarGraph(float value, float maxValue, int width) {
|
||||||
|
int barWidth = mapToDisplay(value, 0, maxValue, 0, width);
|
||||||
|
for (int i = 0; i < width; i++) {
|
||||||
|
Serial.print(i < barWidth ? "█" : " ");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void SpectrumVisualizer::drawFFTMagnitudes(const double* magnitudes, int size, bool logScale) {
|
||||||
|
// Find maximum magnitude for scaling
|
||||||
|
double maxMagnitude = 0;
|
||||||
|
for (int i = 0; i < size/2; i++) {
|
||||||
|
maxMagnitude = max(maxMagnitude, magnitudes[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print frequency spectrum
|
||||||
|
Serial.println("\nFrequency Spectrum:");
|
||||||
|
Serial.println("Freq (Hz) | Magnitude");
|
||||||
|
Serial.println("-----------|-" + String('-', DISPLAY_WIDTH));
|
||||||
|
|
||||||
|
for (int i = 0; i < size/2; i++) {
|
||||||
|
float freq = i * Config::FREQUENCY_RESOLUTION;
|
||||||
|
if (freq > Config::NOTE_FREQ_C6) break; // Stop at highest note frequency
|
||||||
|
|
||||||
|
float magnitude = magnitudes[i];
|
||||||
|
if (logScale) {
|
||||||
|
magnitude = magnitudeToDb(magnitude);
|
||||||
|
maxMagnitude = magnitudeToDb(maxMagnitude);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only print if significant magnitude
|
||||||
|
if (magnitude > (logScale ? DB_MIN : 0)) {
|
||||||
|
Serial.printf("%8.1f | ", freq);
|
||||||
|
printBarGraph(magnitude, maxMagnitude, DISPLAY_WIDTH);
|
||||||
|
Serial.println();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void SpectrumVisualizer::visualizeSpectrum(const double* spectrum, int size) {
|
||||||
|
drawFFTMagnitudes(spectrum, size, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
void SpectrumVisualizer::visualizeNotes(const std::vector<DetectedNote>& notes) {
|
||||||
|
if (notes.empty()) return;
|
||||||
|
|
||||||
|
// Find maximum magnitude for scaling
|
||||||
|
float maxMagnitude = 0;
|
||||||
|
for (const auto& note : notes) {
|
||||||
|
maxMagnitude = max(maxMagnitude, note.magnitude);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print piano keyboard visualization
|
||||||
|
Serial.println("\nDetected Notes:");
|
||||||
|
Serial.println("Note | Freq (Hz) | Level");
|
||||||
|
Serial.println("--------|-----------|" + String('-', DISPLAY_WIDTH));
|
||||||
|
|
||||||
|
for (const auto& note : notes) {
|
||||||
|
int octave = (note.noteNumber / 12) - 1;
|
||||||
|
int noteIndex = note.noteNumber % 12;
|
||||||
|
float dbLevel = magnitudeToDb(note.magnitude);
|
||||||
|
|
||||||
|
Serial.printf("%s%-6d | %8.1f | ",
|
||||||
|
noteNames[noteIndex], octave, note.frequency);
|
||||||
|
printBarGraph(note.magnitude, maxMagnitude, DISPLAY_WIDTH);
|
||||||
|
Serial.printf(" %.1f dB\n", dbLevel);
|
||||||
|
}
|
||||||
|
}
|
||||||
375
src/main.cpp
375
src/main.cpp
@@ -1,130 +1,275 @@
|
|||||||
#include <Arduino.h>
|
#include <Arduino.h>
|
||||||
#include <driver/i2s.h>
|
#include <WiFi.h>
|
||||||
#include <deque>
|
#include <WiFiManager.h>
|
||||||
|
#include <ESPAsyncWebServer.h>
|
||||||
|
#include <AsyncTCP.h>
|
||||||
|
#include <SPIFFS.h>
|
||||||
|
#include "Config.h"
|
||||||
|
#include "I2SConfig.h"
|
||||||
|
#include "AudioLevelTracker.h"
|
||||||
|
#include "NoteDetector.h"
|
||||||
|
#include "SpectrumVisualizer.h"
|
||||||
|
|
||||||
// you shouldn't need to change these settings
|
// Function declarations
|
||||||
#define SAMPLE_BUFFER_SIZE 512
|
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len);
|
||||||
#define SAMPLE_RATE 8000
|
void initWebServer();
|
||||||
// most microphones will probably default to left channel but you may need to tie the L/R pin low
|
void handleSerialCommands();
|
||||||
#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_LEFT
|
void printNoteInfo(const DetectedNote& note);
|
||||||
// either wire your microphone to the same pins or change these to match your wiring
|
void initWiFi();
|
||||||
#define I2S_MIC_SERIAL_CLOCK 8
|
void audioProcessingTask(void *parameter);
|
||||||
#define I2S_MIC_LEFT_RIGHT_CLOCK 9
|
void visualizationTask(void *parameter);
|
||||||
#define I2S_MIC_SERIAL_DATA 10
|
void sendSpectrumData();
|
||||||
|
|
||||||
// Add sample history tracking for range limiting
|
// Static instances
|
||||||
#define HISTORY_DURATION_MS 1000 // 1 seconds history
|
static int16_t raw_samples[Config::SAMPLE_BUFFER_SIZE];
|
||||||
#define SAMPLES_PER_MS (SAMPLE_RATE / 1000)
|
static AudioLevelTracker audioLevelTracker;
|
||||||
#define HISTORY_SIZE (HISTORY_DURATION_MS * SAMPLES_PER_MS)
|
static NoteDetector noteDetector;
|
||||||
|
WiFiManager wifiManager;
|
||||||
|
AsyncWebServer server(80);
|
||||||
|
AsyncWebSocket ws("/ws");
|
||||||
|
|
||||||
#define MAX_RANGE_LIMIT 150000000 // Maximum allowed range limit
|
// Timing and state variables
|
||||||
#define DEFAULT_RANGE_LIMIT 20000 // Default range limit when no history
|
static uint32_t lastNotePrintTime = 0;
|
||||||
|
static uint32_t lastSpectrumPrintTime = 0;
|
||||||
|
static uint32_t lastWebUpdateTime = 0;
|
||||||
|
static bool showSpectrum = false;
|
||||||
|
|
||||||
class AudioLevelTracker {
|
// Task handles
|
||||||
public:
|
TaskHandle_t audioTaskHandle = nullptr;
|
||||||
AudioLevelTracker() {
|
TaskHandle_t visualizationTaskHandle = nullptr;
|
||||||
resetMaxLevel();
|
|
||||||
}
|
|
||||||
|
|
||||||
void updateMaxLevel(int32_t sample) {
|
// Queue for passing audio data between cores
|
||||||
uint32_t currentTime = millis();
|
QueueHandle_t audioQueue;
|
||||||
|
|
||||||
|
// Note names for display
|
||||||
|
const char* noteNames[] = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"};
|
||||||
|
|
||||||
|
void sendSpectrumData() {
|
||||||
|
if (ws.count() > 0 && !noteDetector.isCalibrating()) {
|
||||||
|
const auto& spectrum = noteDetector.getSpectrum();
|
||||||
|
String json = "[";
|
||||||
|
|
||||||
// Remove old samples (older than 10 seconds)
|
// Calculate bin range for 60-1100 Hz
|
||||||
while (!sampleHistory.empty() &&
|
// At 8kHz sample rate with 1024 FFT size:
|
||||||
(currentTime - sampleHistory.front().timestamp) > HISTORY_DURATION_MS) {
|
// binFreq = index * (8000/1024) = index * 7.8125 Hz
|
||||||
sampleHistory.pop_front();
|
// For 60 Hz: bin ≈ 8
|
||||||
}
|
// For 1100 Hz: bin ≈ 141
|
||||||
|
|
||||||
// Add new sample, but cap it at MAX_RANGE_LIMIT
|
for (int i = 8; i <= 141; i++) {
|
||||||
int32_t absValue = abs(sample);
|
if (i > 8) json += ",";
|
||||||
absValue = min(absValue, MAX_RANGE_LIMIT); // Cap the value
|
json += String(spectrum[i], 2);
|
||||||
SamplePoint newPoint = {currentTime, absValue};
|
|
||||||
sampleHistory.push_back(newPoint);
|
|
||||||
|
|
||||||
// Update maximum
|
|
||||||
maxLevel = 0;
|
|
||||||
for (const auto& point : sampleHistory) {
|
|
||||||
if (point.value > maxLevel) {
|
|
||||||
maxLevel = point.value;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
json += "]";
|
||||||
|
ws.textAll(json);
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t getMaxLevel() const {
|
|
||||||
return maxLevel;
|
|
||||||
}
|
|
||||||
|
|
||||||
void resetMaxLevel() {
|
|
||||||
maxLevel = 0;
|
|
||||||
sampleHistory.clear();
|
|
||||||
}
|
|
||||||
|
|
||||||
private:
|
|
||||||
struct SamplePoint {
|
|
||||||
uint32_t timestamp;
|
|
||||||
int32_t value;
|
|
||||||
};
|
|
||||||
std::deque<SamplePoint> sampleHistory;
|
|
||||||
int32_t maxLevel;
|
|
||||||
};
|
|
||||||
|
|
||||||
AudioLevelTracker audioLevelTracker;
|
|
||||||
|
|
||||||
// don't mess around with this
|
|
||||||
i2s_config_t i2s_config = {
|
|
||||||
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
|
|
||||||
.sample_rate = SAMPLE_RATE,
|
|
||||||
.bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
|
|
||||||
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
|
|
||||||
.communication_format = I2S_COMM_FORMAT_STAND_I2S, // Updated from I2S_COMM_FORMAT_I2S
|
|
||||||
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
|
|
||||||
.dma_buf_count = 4,
|
|
||||||
.dma_buf_len = 1024,
|
|
||||||
.use_apll = false,
|
|
||||||
.tx_desc_auto_clear = false,
|
|
||||||
.fixed_mclk = 0};
|
|
||||||
|
|
||||||
// and don't mess around with this
|
|
||||||
i2s_pin_config_t i2s_mic_pins = {
|
|
||||||
.bck_io_num = I2S_MIC_SERIAL_CLOCK,
|
|
||||||
.ws_io_num = I2S_MIC_LEFT_RIGHT_CLOCK,
|
|
||||||
.data_out_num = I2S_PIN_NO_CHANGE,
|
|
||||||
.data_in_num = I2S_MIC_SERIAL_DATA};
|
|
||||||
|
|
||||||
void setup()
|
|
||||||
{
|
|
||||||
// we need serial output for the plotter
|
|
||||||
Serial.begin(115200);
|
|
||||||
// start up the I2S peripheral
|
|
||||||
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
|
|
||||||
i2s_set_pin(I2S_NUM_0, &i2s_mic_pins);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t raw_samples[SAMPLE_BUFFER_SIZE];
|
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len) {
|
||||||
void loop()
|
switch (type) {
|
||||||
{
|
case WS_EVT_CONNECT:
|
||||||
// read from the I2S device
|
Serial.printf("WebSocket client #%u connected from %s\n", client->id(), client->remoteIP().toString().c_str());
|
||||||
size_t bytes_read = 0;
|
break;
|
||||||
i2s_read(I2S_NUM_0, raw_samples, sizeof(int32_t) * SAMPLE_BUFFER_SIZE, &bytes_read, portMAX_DELAY);
|
case WS_EVT_DISCONNECT:
|
||||||
int samples_read = bytes_read / sizeof(int32_t);
|
Serial.printf("WebSocket client #%u disconnected\n", client->id());
|
||||||
|
break;
|
||||||
// Calculate dynamic range limit based on max level from last 10 seconds
|
case WS_EVT_DATA:
|
||||||
int32_t currentMaxLevel = audioLevelTracker.getMaxLevel();
|
break;
|
||||||
int32_t rangelimit = currentMaxLevel > 0 ? currentMaxLevel : DEFAULT_RANGE_LIMIT; // fallback to default if no history
|
case WS_EVT_ERROR:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// dump the samples out to the serial channel
|
void initWebServer() {
|
||||||
for (int i = 0; i < samples_read; i++)
|
if (!SPIFFS.begin(true)) {
|
||||||
{
|
Serial.println("An error occurred while mounting SPIFFS");
|
||||||
// Update the max level tracker with current sample
|
return;
|
||||||
audioLevelTracker.updateMaxLevel(raw_samples[i]);
|
}
|
||||||
|
|
||||||
// Print range limits for plotter
|
ws.onEvent(onWebSocketEvent);
|
||||||
Serial.print(rangelimit * -1);
|
server.addHandler(&ws);
|
||||||
Serial.print(" ");
|
|
||||||
Serial.print(rangelimit);
|
|
||||||
Serial.print(" ");
|
|
||||||
|
|
||||||
// Print the actual sample
|
// Serve static files from SPIFFS
|
||||||
Serial.printf("%ld\n", raw_samples[i]);
|
server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");
|
||||||
}
|
|
||||||
|
// Handle not found
|
||||||
|
server.onNotFound([](AsyncWebServerRequest *request) {
|
||||||
|
request->send(404, "text/plain", "Not found");
|
||||||
|
});
|
||||||
|
|
||||||
|
server.begin();
|
||||||
|
Serial.println("HTTP server started");
|
||||||
|
}
|
||||||
|
|
||||||
|
void handleSerialCommands() {
|
||||||
|
if (Serial.available()) {
|
||||||
|
char cmd = Serial.read();
|
||||||
|
switch (cmd) {
|
||||||
|
case Config::CMD_HELP:
|
||||||
|
Serial.println(Config::MSG_HELP);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case Config::CMD_CALIBRATE:
|
||||||
|
noteDetector.beginCalibration();
|
||||||
|
break;
|
||||||
|
|
||||||
|
case Config::CMD_THRESHOLD_UP:
|
||||||
|
noteDetector.adjustThreshold(Config::THRESHOLD_STEP);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case Config::CMD_THRESHOLD_DOWN:
|
||||||
|
noteDetector.adjustThreshold(-Config::THRESHOLD_STEP);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case Config::CMD_TOGGLE_SPECTRUM:
|
||||||
|
showSpectrum = !showSpectrum;
|
||||||
|
Serial.printf("Spectrum display %s\n", showSpectrum ? "enabled" : "disabled");
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void printNoteInfo(const DetectedNote& note) {
|
||||||
|
int octave = (note.noteNumber / 12) - 1;
|
||||||
|
int noteIndex = note.noteNumber % 12;
|
||||||
|
uint32_t duration = millis() - note.startTime;
|
||||||
|
|
||||||
|
Serial.printf("Note: %s%d (%.1f Hz, Magnitude: %.0f, Duration: %ums)\n",
|
||||||
|
noteNames[noteIndex], octave, note.frequency, note.magnitude, duration);
|
||||||
|
}
|
||||||
|
|
||||||
|
void initWiFi() {
|
||||||
|
// Set configuration portal timeout
|
||||||
|
wifiManager.setConfigPortalTimeout(Config::WIFI_CONFIG_TIMEOUT);
|
||||||
|
|
||||||
|
// Set custom portal settings
|
||||||
|
wifiManager.setAPStaticIPConfig(IPAddress(192,168,4,1), IPAddress(192,168,4,1), IPAddress(255,255,255,0));
|
||||||
|
|
||||||
|
// Try to connect to saved WiFi credentials
|
||||||
|
if(!wifiManager.autoConnect(Config::WIFI_AP_NAME, Config::WIFI_AP_PASSWORD)) {
|
||||||
|
Serial.println("Failed to connect and hit timeout");
|
||||||
|
ESP.restart();
|
||||||
|
}
|
||||||
|
|
||||||
|
Serial.println("Successfully connected to WiFi");
|
||||||
|
Serial.print("IP Address: ");
|
||||||
|
Serial.println(WiFi.localIP());
|
||||||
|
}
|
||||||
|
|
||||||
|
// Audio processing task running on Core 1
|
||||||
|
void audioProcessingTask(void *parameter) {
|
||||||
|
while (true) {
|
||||||
|
size_t bytes_read = 0;
|
||||||
|
readI2SSamples(raw_samples, &bytes_read);
|
||||||
|
int samples_read = bytes_read / sizeof(int16_t);
|
||||||
|
|
||||||
|
// Update level tracking
|
||||||
|
for (int i = 0; i < samples_read; i++) {
|
||||||
|
audioLevelTracker.updateMaxLevel(raw_samples[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only analyze if we have enough signal
|
||||||
|
int16_t currentMaxLevel = audioLevelTracker.getMaxLevel();
|
||||||
|
if (currentMaxLevel > Config::NOISE_THRESHOLD) {
|
||||||
|
// Analyze samples for note detection
|
||||||
|
noteDetector.analyzeSamples(raw_samples, samples_read);
|
||||||
|
|
||||||
|
// Send results to visualization task via queue
|
||||||
|
if (xQueueSend(audioQueue, &samples_read, 0) != pdTRUE) {
|
||||||
|
// Queue full, just skip this update
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Small delay to prevent watchdog trigger
|
||||||
|
vTaskDelay(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Visualization and network task running on Core 0
|
||||||
|
void visualizationTask(void *parameter) {
|
||||||
|
while (true) {
|
||||||
|
int samples_read;
|
||||||
|
|
||||||
|
// Check if there's new audio data to process
|
||||||
|
if (xQueueReceive(audioQueue, &samples_read, 0) == pdTRUE) {
|
||||||
|
uint32_t currentTime = millis();
|
||||||
|
const auto& detectedNotes = noteDetector.getDetectedNotes();
|
||||||
|
|
||||||
|
// Update web clients with spectrum data
|
||||||
|
if (currentTime - lastWebUpdateTime >= 50) {
|
||||||
|
sendSpectrumData();
|
||||||
|
lastWebUpdateTime = currentTime;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Show spectrum if enabled
|
||||||
|
if (showSpectrum &&
|
||||||
|
!noteDetector.isCalibrating() &&
|
||||||
|
currentTime - lastSpectrumPrintTime >= Config::DEBUG_INTERVAL_MS) {
|
||||||
|
SpectrumVisualizer::visualizeSpectrum(noteDetector.getSpectrum(), Config::FFT_SIZE);
|
||||||
|
lastSpectrumPrintTime = currentTime;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print detected notes at specified interval
|
||||||
|
if (currentTime - lastNotePrintTime >= Config::NOTE_PRINT_INTERVAL_MS) {
|
||||||
|
if (!detectedNotes.empty() && !noteDetector.isCalibrating()) {
|
||||||
|
SpectrumVisualizer::visualizeNotes(detectedNotes);
|
||||||
|
}
|
||||||
|
lastNotePrintTime = currentTime;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ws.cleanupClients();
|
||||||
|
handleSerialCommands();
|
||||||
|
|
||||||
|
// Small delay to prevent watchdog trigger
|
||||||
|
vTaskDelay(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void setup() {
|
||||||
|
Serial.begin(Config::SERIAL_BAUD_RATE);
|
||||||
|
while(!Serial) {
|
||||||
|
delay(10);
|
||||||
|
}
|
||||||
|
|
||||||
|
initWiFi();
|
||||||
|
initWebServer();
|
||||||
|
initI2S();
|
||||||
|
|
||||||
|
Serial.println("Piano Note Detection Ready (C2-C6)");
|
||||||
|
Serial.println("Press 'h' for help");
|
||||||
|
noteDetector.beginCalibration();
|
||||||
|
|
||||||
|
// Create queue for inter-core communication
|
||||||
|
audioQueue = xQueueCreate(4, sizeof(int));
|
||||||
|
if (audioQueue == nullptr) {
|
||||||
|
Serial.println("Failed to create queue!");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create audio processing task on Core 1
|
||||||
|
xTaskCreatePinnedToCore(
|
||||||
|
audioProcessingTask,
|
||||||
|
"AudioTask",
|
||||||
|
Config::TASK_STACK_SIZE,
|
||||||
|
nullptr,
|
||||||
|
Config::TASK_PRIORITY,
|
||||||
|
&audioTaskHandle,
|
||||||
|
1 // Run on Core 1
|
||||||
|
);
|
||||||
|
|
||||||
|
// Create visualization task on Core 0
|
||||||
|
xTaskCreatePinnedToCore(
|
||||||
|
visualizationTask,
|
||||||
|
"VisualTask",
|
||||||
|
Config::TASK_STACK_SIZE,
|
||||||
|
nullptr,
|
||||||
|
1, // Lower priority than audio task
|
||||||
|
&visualizationTaskHandle,
|
||||||
|
0 // Run on Core 0
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
void loop() {
|
||||||
|
// Main loop is now empty as all work is done in tasks
|
||||||
|
vTaskDelete(nullptr); // Delete the main loop task
|
||||||
}
|
}
|
||||||
Reference in New Issue
Block a user