Compare commits
1 Commits
4992b75d57
...
test_bits_
| Author | SHA1 | Date | |
|---|---|---|---|
| 9a3d026176 |
317
README.md
317
README.md
@@ -1,317 +1,2 @@
|
|||||||
# ESP32 Piano Note Detection System
|
# esp32i2s
|
||||||
|
|
||||||
A real-time piano note detection system implemented on ESP32 using I2S microphone input. This system can detect musical notes from C2 to C6 with adjustable sensitivity and visualization options.
|
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
- Real-time audio processing using I2S microphone
|
|
||||||
- FFT-based frequency analysis
|
|
||||||
- Note detection from C2 (65.41 Hz) to C6 (1046.50 Hz)
|
|
||||||
- Dynamic threshold calibration
|
|
||||||
- Multiple note detection (up to 7 simultaneous notes)
|
|
||||||
- Harmonic filtering
|
|
||||||
- Real-time spectrum visualization
|
|
||||||
- Note timing and duration tracking
|
|
||||||
- Interactive Serial commands for system tuning
|
|
||||||
|
|
||||||
## Hardware Requirements
|
|
||||||
|
|
||||||
- ESP32 development board
|
|
||||||
- I2S MEMS microphone (e.g., INMP441, SPH0645)
|
|
||||||
- USB connection for Serial monitoring
|
|
||||||
|
|
||||||
## Pin Configuration
|
|
||||||
|
|
||||||
The system uses the following I2S pins by default (configurable in Config.h):
|
|
||||||
- SCK (Serial Clock): GPIO 8
|
|
||||||
- WS/LRC (Word Select/Left-Right Clock): GPIO 9
|
|
||||||
- SD (Serial Data): GPIO 10
|
|
||||||
|
|
||||||
## Getting Started
|
|
||||||
|
|
||||||
1. Connect the I2S microphone to the ESP32 according to the pin configuration
|
|
||||||
2. Build and flash the project to your ESP32
|
|
||||||
3. Open a Serial monitor at 115200 baud
|
|
||||||
4. Follow the calibration process on first run
|
|
||||||
|
|
||||||
## Serial Commands
|
|
||||||
|
|
||||||
The system can be controlled via Serial commands:
|
|
||||||
|
|
||||||
- `h` - Display help menu
|
|
||||||
- `c` - Start calibration process
|
|
||||||
- `+` - Increase sensitivity (threshold up)
|
|
||||||
- `-` - Decrease sensitivity (threshold down)
|
|
||||||
- `s` - Toggle spectrum visualization
|
|
||||||
|
|
||||||
## Configuration Options
|
|
||||||
|
|
||||||
All system parameters can be adjusted in `Config.h`:
|
|
||||||
|
|
||||||
### Audio Processing
|
|
||||||
- `SAMPLE_RATE`: 8000 Hz (good for frequencies up to 4kHz)
|
|
||||||
- `BITS_PER_SAMPLE`: 16-bit resolution
|
|
||||||
- `SAMPLE_BUFFER_SIZE`: 1024 samples
|
|
||||||
- `FFT_SIZE`: 1024 points
|
|
||||||
|
|
||||||
### Note Detection
|
|
||||||
- `NOTE_FREQ_C2`: 65.41 Hz (lowest detectable note)
|
|
||||||
- `NOTE_FREQ_C6`: 1046.50 Hz (highest detectable note)
|
|
||||||
- `FREQUENCY_TOLERANCE`: 3.0 Hz
|
|
||||||
- `MAX_SIMULTANEOUS_NOTES`: 7
|
|
||||||
- `MIN_NOTE_DURATION_MS`: 50ms
|
|
||||||
- `NOTE_RELEASE_TIME_MS`: 100ms
|
|
||||||
|
|
||||||
### Calibration
|
|
||||||
- `CALIBRATION_DURATION_MS`: 5000ms
|
|
||||||
- `CALIBRATION_PEAK_PERCENTILE`: 0.95 (95th percentile)
|
|
||||||
|
|
||||||
## Visualization
|
|
||||||
|
|
||||||
The system provides two visualization modes:
|
|
||||||
|
|
||||||
1. Note Display:
|
|
||||||
```
|
|
||||||
Current Notes:
|
|
||||||
A4 (440.0 Hz, Magnitude: 2500, Duration: 250ms)
|
|
||||||
E5 (659.3 Hz, Magnitude: 1800, Duration: 150ms)
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Spectrum Display (when enabled):
|
|
||||||
```
|
|
||||||
Frequency Spectrum:
|
|
||||||
0Hz |▄▄▄▄▄
|
|
||||||
100Hz |██████▄
|
|
||||||
200Hz |▄▄▄
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Tuning
|
|
||||||
|
|
||||||
1. Start with calibration by pressing 'c' in a quiet environment
|
|
||||||
2. Play notes and observe the detection accuracy
|
|
||||||
3. Use '+' and '-' to adjust sensitivity if needed
|
|
||||||
4. Enable spectrum display with 's' to visualize frequency content
|
|
||||||
5. Adjust `Config.h` parameters if needed for your specific setup
|
|
||||||
|
|
||||||
## Implementation Details
|
|
||||||
|
|
||||||
- Uses FFT for frequency analysis
|
|
||||||
- Implements peak detection with dynamic thresholding
|
|
||||||
- Filters out harmonics to prevent duplicate detections
|
|
||||||
- Tracks note timing and duration
|
|
||||||
- Uses ring buffer for real-time processing
|
|
||||||
- Calibration collects ambient noise profile
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
1. No notes detected:
|
|
||||||
- Check microphone connection
|
|
||||||
- Run calibration
|
|
||||||
- Increase sensitivity with '+'
|
|
||||||
- Verify audio input level in spectrum display
|
|
||||||
|
|
||||||
2. False detections:
|
|
||||||
- Run calibration in a quiet environment
|
|
||||||
- Decrease sensitivity with '-'
|
|
||||||
- Adjust `PEAK_RATIO_THRESHOLD` in Config.h
|
|
||||||
|
|
||||||
3. Missing notes:
|
|
||||||
- Check if notes are within C2-C6 range
|
|
||||||
- Increase `FREQUENCY_TOLERANCE`
|
|
||||||
- Decrease `MIN_MAGNITUDE_THRESHOLD`
|
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
Contributions are welcome! Please read the contributing guidelines before submitting pull requests.
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is licensed under the MIT License - see the LICENSE file for details.
|
|
||||||
|
|
||||||
## Development Environment Setup
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
- PlatformIO IDE (recommended) or Arduino IDE
|
|
||||||
- ESP32 board support package
|
|
||||||
- Required libraries:
|
|
||||||
- arduino-audio-tools
|
|
||||||
- arduino-audio-driver
|
|
||||||
- WiFiManager
|
|
||||||
- AsyncTCP
|
|
||||||
- ESPAsyncWebServer
|
|
||||||
- arduinoFFT
|
|
||||||
|
|
||||||
### Building with PlatformIO
|
|
||||||
1. Clone the repository
|
|
||||||
2. Open the project in PlatformIO
|
|
||||||
3. Install dependencies:
|
|
||||||
```
|
|
||||||
pio lib install
|
|
||||||
```
|
|
||||||
4. Build and upload:
|
|
||||||
```
|
|
||||||
pio run -t upload
|
|
||||||
```
|
|
||||||
|
|
||||||
## Memory Management
|
|
||||||
|
|
||||||
### Memory Usage
|
|
||||||
- Program Memory: ~800KB
|
|
||||||
- RAM Usage: ~100KB
|
|
||||||
- DMA Buffers: 4 x 512 bytes
|
|
||||||
- FFT Working Buffer: 2048 bytes (1024 samples x 2 bytes)
|
|
||||||
|
|
||||||
### Optimization Tips
|
|
||||||
- Adjust `DMA_BUFFER_COUNT` based on available RAM
|
|
||||||
- Reduce `SAMPLE_BUFFER_SIZE` for lower latency
|
|
||||||
- Use `PSRAM` if available for larger buffer sizes
|
|
||||||
|
|
||||||
## Advanced Configuration
|
|
||||||
|
|
||||||
### Task Management
|
|
||||||
- Audio processing task on Core 1:
|
|
||||||
- I2S sample reading
|
|
||||||
- Audio level tracking
|
|
||||||
- Note detection and FFT analysis
|
|
||||||
- Visualization task on Core 0:
|
|
||||||
- WebSocket communication
|
|
||||||
- Spectrum visualization
|
|
||||||
- Serial interface
|
|
||||||
- Network operations
|
|
||||||
- Inter-core communication via FreeRTOS queue
|
|
||||||
- Configurable priorities in `Config.h`
|
|
||||||
|
|
||||||
### Audio Pipeline
|
|
||||||
1. I2S DMA Input
|
|
||||||
2. Sample Buffer Collection
|
|
||||||
3. FFT Processing
|
|
||||||
4. Peak Detection
|
|
||||||
5. Note Identification
|
|
||||||
6. Output Generation
|
|
||||||
|
|
||||||
### Timing Parameters
|
|
||||||
- Audio Buffer Processing: ~8ms
|
|
||||||
- FFT Computation: ~5ms
|
|
||||||
- Note Detection: ~2ms
|
|
||||||
- Total Latency: ~15-20ms
|
|
||||||
|
|
||||||
## Performance Optimization
|
|
||||||
|
|
||||||
### CPU Usage
|
|
||||||
- Core 1 (Audio Processing):
|
|
||||||
- I2S DMA handling: ~15%
|
|
||||||
- Audio analysis: ~20%
|
|
||||||
- FFT processing: ~15%
|
|
||||||
- Core 0 (Visualization):
|
|
||||||
- WebSocket updates: ~5%
|
|
||||||
- Visualization: ~5%
|
|
||||||
- Network handling: ~5%
|
|
||||||
|
|
||||||
### Memory Optimization
|
|
||||||
1. Buffer Size Selection:
|
|
||||||
- Larger buffers: Better frequency resolution
|
|
||||||
- Smaller buffers: Lower latency
|
|
||||||
2. DMA Configuration:
|
|
||||||
- More buffers: Better continuity
|
|
||||||
- Fewer buffers: Lower memory usage
|
|
||||||
|
|
||||||
### Frequency Analysis
|
|
||||||
- FFT Resolution: 7.8125 Hz (8000/1024)
|
|
||||||
- Frequency Bins: 512 (Nyquist limit)
|
|
||||||
- Useful Range: 65.41 Hz to 1046.50 Hz
|
|
||||||
- Window Function: Hamming
|
|
||||||
|
|
||||||
## Technical Details
|
|
||||||
|
|
||||||
### Microphone Specifications
|
|
||||||
- Supply Voltage: 3.3V
|
|
||||||
- Sampling Rate: 8kHz
|
|
||||||
- Bit Depth: 16-bit
|
|
||||||
- SNR: >65dB (typical)
|
|
||||||
|
|
||||||
### Signal Processing
|
|
||||||
1. Pre-processing:
|
|
||||||
- DC offset removal
|
|
||||||
- Windowing function application
|
|
||||||
2. FFT Processing:
|
|
||||||
- 1024-point real FFT
|
|
||||||
- Magnitude calculation
|
|
||||||
3. Post-processing:
|
|
||||||
- Peak detection
|
|
||||||
- Harmonic filtering
|
|
||||||
- Note matching
|
|
||||||
|
|
||||||
### Calibration Process
|
|
||||||
1. Ambient Noise Collection (5 seconds)
|
|
||||||
2. Frequency Bin Analysis
|
|
||||||
3. Threshold Calculation:
|
|
||||||
- Base threshold from 95th percentile
|
|
||||||
- Per-bin noise floor mapping
|
|
||||||
4. Dynamic Adjustment
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
1. I2S Communication Errors:
|
|
||||||
- Check pin connections
|
|
||||||
- Verify I2S configuration
|
|
||||||
- Monitor serial output for error codes
|
|
||||||
2. Memory Issues:
|
|
||||||
- Watch heap fragmentation
|
|
||||||
- Monitor stack usage
|
|
||||||
- Check DMA buffer allocation
|
|
||||||
|
|
||||||
### Error Recovery
|
|
||||||
- Automatic I2S reset on communication errors
|
|
||||||
- Dynamic threshold adjustment
|
|
||||||
- Watchdog timer protection
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
### Core Components
|
|
||||||
1. AudioLevelTracker
|
|
||||||
- Real-time audio level monitoring
|
|
||||||
- Peak detection
|
|
||||||
- Threshold management
|
|
||||||
2. NoteDetector
|
|
||||||
- Frequency analysis
|
|
||||||
- Note identification
|
|
||||||
- Harmonic filtering
|
|
||||||
3. SpectrumVisualizer
|
|
||||||
- Real-time spectrum display
|
|
||||||
- Magnitude scaling
|
|
||||||
- ASCII visualization
|
|
||||||
|
|
||||||
### File Organization
|
|
||||||
- `/src`: Core implementation files
|
|
||||||
- `/include`: Header files and configurations
|
|
||||||
- `/data`: Additional resources
|
|
||||||
- `/test`: Unit tests
|
|
||||||
|
|
||||||
## Inter-Core Communication
|
|
||||||
|
|
||||||
### Queue Management
|
|
||||||
- FreeRTOS queue for audio data transfer
|
|
||||||
- 4-slot queue buffer
|
|
||||||
- Zero-copy data passing
|
|
||||||
- Non-blocking queue operations
|
|
||||||
- Automatic overflow protection
|
|
||||||
|
|
||||||
### Data Flow
|
|
||||||
1. Core 1 (Audio Task):
|
|
||||||
- Processes audio samples
|
|
||||||
- Performs FFT analysis
|
|
||||||
- Queues processed data
|
|
||||||
2. Core 0 (Visualization Task):
|
|
||||||
- Receives processed data
|
|
||||||
- Updates visualization
|
|
||||||
- Handles network communication
|
|
||||||
|
|
||||||
### Network Communication
|
|
||||||
- Asynchronous WebSocket updates
|
|
||||||
- JSON-formatted spectrum data
|
|
||||||
- Configurable update rate (50ms default)
|
|
||||||
- Automatic client cleanup
|
|
||||||
- Efficient connection management
|
|
||||||
|
|
||||||
|
|||||||
260
data/index.html
260
data/index.html
@@ -1,260 +0,0 @@
|
|||||||
<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<head>
|
|
||||||
<title>ESP32 Piano Spectrum Analyzer</title>
|
|
||||||
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
|
|
||||||
<script src="https://cdn.jsdelivr.net/npm/chartjs-plugin-annotation"></script>
|
|
||||||
<style>
|
|
||||||
body {
|
|
||||||
font-family: Arial, sans-serif;
|
|
||||||
margin: 20px;
|
|
||||||
background: #f0f0f0;
|
|
||||||
}
|
|
||||||
.container {
|
|
||||||
max-width: 1200px;
|
|
||||||
margin: 0 auto;
|
|
||||||
background: white;
|
|
||||||
padding: 20px;
|
|
||||||
border-radius: 10px;
|
|
||||||
box-shadow: 0 0 10px rgba(0,0,0,0.1);
|
|
||||||
}
|
|
||||||
h1 {
|
|
||||||
color: #333;
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
#spectrum-container {
|
|
||||||
position: relative;
|
|
||||||
height: 400px;
|
|
||||||
margin: 20px 0;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="container">
|
|
||||||
<h1>Piano Spectrum Analyzer</h1>
|
|
||||||
<div id="spectrum-container">
|
|
||||||
<canvas id="spectrumChart"></canvas>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
Chart.register('chartjs-plugin-annotation');
|
|
||||||
|
|
||||||
// Piano note frequencies (Hz)
|
|
||||||
const noteFrequencies = {
|
|
||||||
'C2': 65.41, 'C#2': 69.30, 'D2': 73.42, 'D#2': 77.78, 'E2': 82.41, 'F2': 87.31,
|
|
||||||
'F#2': 92.50, 'G2': 98.00, 'G#2': 103.83, 'A2': 110.00, 'A#2': 116.54, 'B2': 123.47,
|
|
||||||
'C3': 130.81, 'C#3': 138.59, 'D3': 146.83, 'D#3': 155.56, 'E3': 164.81, 'F3': 174.61,
|
|
||||||
'F#3': 185.00, 'G3': 196.00, 'G#3': 207.65, 'A3': 220.00, 'A#3': 233.08, 'B3': 246.94,
|
|
||||||
'C4': 261.63, 'C#4': 277.18, 'D4': 293.66, 'D#4': 311.13, 'E4': 329.63, 'F4': 349.23,
|
|
||||||
'F#4': 369.99, 'G4': 392.00, 'G#4': 415.30, 'A4': 440.00, 'A#4': 466.16, 'B4': 493.88,
|
|
||||||
'C5': 523.25, 'C#5': 554.37, 'D5': 587.33, 'D#5': 622.25, 'E5': 659.26, 'F5': 698.46,
|
|
||||||
'F#5': 739.99, 'G5': 783.99, 'G#5': 830.61, 'A5': 880.00, 'A#5': 932.33, 'B5': 987.77,
|
|
||||||
'C6': 1046.50
|
|
||||||
};
|
|
||||||
|
|
||||||
// Create ticks for the x-axis
|
|
||||||
const xAxisTicks = Object.entries(noteFrequencies).filter(([note]) =>
|
|
||||||
note.length === 2 || note === 'C#4' || note === 'F#4' || note === 'A#4'
|
|
||||||
).map(([note, freq]) => ({
|
|
||||||
value: freq,
|
|
||||||
label: note
|
|
||||||
}));
|
|
||||||
|
|
||||||
// Keep track of recent maximum values
|
|
||||||
const maxValueHistory = [];
|
|
||||||
const MAX_HISTORY_LENGTH = 5;
|
|
||||||
|
|
||||||
function updateYAxisScale(newValue) {
|
|
||||||
maxValueHistory.push(newValue);
|
|
||||||
if (maxValueHistory.length > MAX_HISTORY_LENGTH) {
|
|
||||||
maxValueHistory.shift();
|
|
||||||
}
|
|
||||||
return Math.max(...maxValueHistory) * 1.1; // Add 10% margin
|
|
||||||
}
|
|
||||||
|
|
||||||
const ctx = document.getElementById('spectrumChart').getContext('2d');
|
|
||||||
const chart = new Chart(ctx, {
|
|
||||||
type: 'line',
|
|
||||||
data: {
|
|
||||||
// Generate logarithmically spaced frequencies for labels
|
|
||||||
labels: Array.from({length: 134}, (_, i) => {
|
|
||||||
const minFreq = 60;
|
|
||||||
const maxFreq = 1100;
|
|
||||||
return Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
|
||||||
}),
|
|
||||||
datasets: [{
|
|
||||||
label: 'Frequency Spectrum',
|
|
||||||
data: Array(134).fill(0),
|
|
||||||
borderColor: 'rgb(75, 192, 192)',
|
|
||||||
tension: 0.1,
|
|
||||||
fill: true,
|
|
||||||
backgroundColor: 'rgba(75, 192, 192, 0.2)',
|
|
||||||
pointRadius: 0 // Hide points for better performance
|
|
||||||
}]
|
|
||||||
},
|
|
||||||
options: {
|
|
||||||
responsive: true,
|
|
||||||
maintainAspectRatio: false,
|
|
||||||
animation: {
|
|
||||||
duration: 0
|
|
||||||
},
|
|
||||||
parsing: {
|
|
||||||
xAxisKey: 'x',
|
|
||||||
yAxisKey: 'y'
|
|
||||||
},
|
|
||||||
scales: {
|
|
||||||
y: {
|
|
||||||
beginAtZero: true,
|
|
||||||
max: 5000,
|
|
||||||
adapters: {
|
|
||||||
update: function(maxValue) {
|
|
||||||
return updateYAxisScale(maxValue);
|
|
||||||
}
|
|
||||||
},
|
|
||||||
title: {
|
|
||||||
display: true,
|
|
||||||
text: 'Magnitude'
|
|
||||||
}
|
|
||||||
},
|
|
||||||
x: {
|
|
||||||
type: 'logarithmic',
|
|
||||||
position: 'bottom',
|
|
||||||
min: 60,
|
|
||||||
max: 1100,
|
|
||||||
title: {
|
|
||||||
display: true,
|
|
||||||
text: 'Frequency (Hz)'
|
|
||||||
},
|
|
||||||
ticks: {
|
|
||||||
callback: function(value) {
|
|
||||||
// Show C notes and F# notes
|
|
||||||
const entries = Object.entries(noteFrequencies);
|
|
||||||
const closest = entries.reduce((prev, curr) => {
|
|
||||||
return Math.abs(curr[1] - value) < Math.abs(prev[1] - value) ? curr : prev;
|
|
||||||
});
|
|
||||||
|
|
||||||
if ((closest[0].includes('C') || closest[0].includes('F#')) &&
|
|
||||||
Math.abs(closest[1] - value) < 1) {
|
|
||||||
return closest[0];
|
|
||||||
}
|
|
||||||
return '';
|
|
||||||
},
|
|
||||||
sampleSize: 20,
|
|
||||||
autoSkip: false
|
|
||||||
},
|
|
||||||
grid: {
|
|
||||||
color: (ctx) => {
|
|
||||||
const value = ctx.tick.value;
|
|
||||||
// Check if this tick corresponds to a C note
|
|
||||||
if (Object.entries(noteFrequencies)
|
|
||||||
.some(([note, freq]) =>
|
|
||||||
note.startsWith('C') && Math.abs(freq - value) < 1)) {
|
|
||||||
return 'rgba(255, 0, 0, 0.1)';
|
|
||||||
}
|
|
||||||
return 'rgba(0, 0, 0, 0.1)';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
plugins: {
|
|
||||||
annotation: {
|
|
||||||
annotations: Object.entries(noteFrequencies)
|
|
||||||
.filter(([note]) => note.startsWith('C'))
|
|
||||||
.reduce((acc, [note, freq]) => {
|
|
||||||
acc[note] = {
|
|
||||||
type: 'line',
|
|
||||||
xMin: freq,
|
|
||||||
xMax: freq,
|
|
||||||
borderColor: 'rgba(255, 99, 132, 0.5)',
|
|
||||||
borderWidth: 1,
|
|
||||||
label: {
|
|
||||||
content: note,
|
|
||||||
enabled: true,
|
|
||||||
position: 'top'
|
|
||||||
}
|
|
||||||
};
|
|
||||||
return acc;
|
|
||||||
}, {})
|
|
||||||
},
|
|
||||||
tooltip: {
|
|
||||||
callbacks: {
|
|
||||||
title: function(context) {
|
|
||||||
const freq = context[0].parsed.x;
|
|
||||||
// Find the closest note
|
|
||||||
const closestNote = Object.entries(noteFrequencies)
|
|
||||||
.reduce((closest, [note, noteFreq]) => {
|
|
||||||
return Math.abs(noteFreq - freq) < Math.abs(noteFreq - closest.freq)
|
|
||||||
? { note, freq: noteFreq }
|
|
||||||
: closest;
|
|
||||||
}, { note: '', freq: Infinity });
|
|
||||||
|
|
||||||
return `Frequency: ${freq.toFixed(1)} Hz (Near ${closestNote.note})`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
legend: {
|
|
||||||
display: false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
const wsUrl = `ws://${window.location.hostname}/ws`;
|
|
||||||
const ws = new WebSocket(wsUrl);
|
|
||||||
|
|
||||||
function interpolateSpectrum(spectrum) {
|
|
||||||
const result = [];
|
|
||||||
const minFreq = 60;
|
|
||||||
const maxFreq = 1100;
|
|
||||||
const binWidth = 8000 / 1024; // Hz per bin
|
|
||||||
|
|
||||||
// Generate logarithmically spaced frequencies
|
|
||||||
for (let i = 0; i < 134; i++) {
|
|
||||||
const targetFreq = Math.pow(10, Math.log10(minFreq) + (Math.log10(maxFreq) - Math.log10(minFreq)) * i / 133);
|
|
||||||
|
|
||||||
// Find the corresponding linear bin
|
|
||||||
const bin = Math.floor(targetFreq / binWidth);
|
|
||||||
|
|
||||||
if (bin >= 8 && bin < 141) {
|
|
||||||
// Linear interpolation between bins
|
|
||||||
const binFraction = (targetFreq / binWidth) - bin;
|
|
||||||
const value = spectrum[bin - 8] * (1 - binFraction) +
|
|
||||||
(bin - 7 < spectrum.length ? spectrum[bin - 7] : 0) * binFraction;
|
|
||||||
result.push({
|
|
||||||
x: targetFreq,
|
|
||||||
y: value
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
result.push({
|
|
||||||
x: targetFreq,
|
|
||||||
y: 0
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
ws.onmessage = function(event) {
|
|
||||||
const spectrum = JSON.parse(event.data);
|
|
||||||
const interpolatedData = interpolateSpectrum(spectrum);
|
|
||||||
|
|
||||||
// Update y-axis scale based on new maximum value
|
|
||||||
const maxValue = Math.max(...interpolatedData.map(d => d.y));
|
|
||||||
chart.options.scales.y.max = updateYAxisScale(maxValue);
|
|
||||||
|
|
||||||
// Update chart data
|
|
||||||
chart.data.datasets[0].data = interpolatedData;
|
|
||||||
chart.update('none'); // Use 'none' mode for maximum performance
|
|
||||||
};
|
|
||||||
|
|
||||||
ws.onclose = function() {
|
|
||||||
console.log('WebSocket connection closed');
|
|
||||||
setTimeout(() => {
|
|
||||||
window.location.reload();
|
|
||||||
}, 1000);
|
|
||||||
};
|
|
||||||
</script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include <deque>
|
|
||||||
#include "Config.h"
|
|
||||||
|
|
||||||
class AudioLevelTracker {
|
|
||||||
public:
|
|
||||||
AudioLevelTracker();
|
|
||||||
void updateMaxLevel(int16_t sample); // Changed to int16_t
|
|
||||||
int16_t getMaxLevel() const; // Changed to int16_t
|
|
||||||
void resetMaxLevel();
|
|
||||||
|
|
||||||
private:
|
|
||||||
struct SamplePoint {
|
|
||||||
uint32_t timestamp;
|
|
||||||
int16_t value; // Changed to int16_t
|
|
||||||
};
|
|
||||||
std::deque<SamplePoint> sampleHistory;
|
|
||||||
int16_t maxLevel; // Changed to int16_t
|
|
||||||
|
|
||||||
static const uint32_t HISTORY_DURATION_MS = 3000; // 3 seconds history
|
|
||||||
static const int16_t MAX_RANGE_LIMIT = Config::DEFAULT_RANGE_LIMIT; // Use 16-bit limit directly
|
|
||||||
};
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
#include <cstdint>
|
|
||||||
|
|
||||||
namespace Config {
|
|
||||||
|
|
||||||
// Serial Configuration
|
|
||||||
constexpr int SERIAL_BAUD_RATE = 115200; // Serial communication baud rate
|
|
||||||
|
|
||||||
// WiFi Configuration
|
|
||||||
constexpr const char* WIFI_AP_NAME = "ESP32Piano"; // AP name when in configuration mode
|
|
||||||
constexpr const char* WIFI_AP_PASSWORD = "12345678"; // AP password when in configuration mode
|
|
||||||
constexpr uint32_t WIFI_CONFIG_TIMEOUT = 180; // Seconds to wait for WiFi configuration
|
|
||||||
constexpr uint32_t WIFI_CONFIG_PORT = 80; // Web configuration portal port
|
|
||||||
|
|
||||||
// I2S Pin Configuration
|
|
||||||
constexpr int I2S_MIC_SERIAL_CLOCK = 8; // SCK
|
|
||||||
constexpr int I2S_MIC_LEFT_RIGHT_CLOCK = 9; // WS/LRC
|
|
||||||
constexpr int I2S_MIC_SERIAL_DATA = 10; // SD
|
|
||||||
|
|
||||||
// I2S Configuration
|
|
||||||
constexpr int SAMPLE_RATE = 8000; // Hz - good for frequencies up to 4kHz
|
|
||||||
constexpr int BITS_PER_SAMPLE = 16; // Changed to 16-bit
|
|
||||||
constexpr int CHANNELS = 1; // Mono input
|
|
||||||
constexpr int SAMPLE_BUFFER_SIZE = 1024; // Match FFT_SIZE for efficiency
|
|
||||||
|
|
||||||
// DMA Configuration
|
|
||||||
constexpr int DMA_BUFFER_COUNT = 4; // Reduced for lower latency
|
|
||||||
constexpr int DMA_BUFFER_LEN = 512; // Smaller chunks for faster processing
|
|
||||||
|
|
||||||
// Audio Processing
|
|
||||||
constexpr float DC_OFFSET = 0.0f; // DC offset correction
|
|
||||||
constexpr float GAIN = 4.0f; // Increased gain for better note detection
|
|
||||||
constexpr int16_t NOISE_THRESHOLD = 100; // Lower threshold for 16-bit samples
|
|
||||||
constexpr int16_t DEFAULT_RANGE_LIMIT = 32767; // Max value for 16-bit
|
|
||||||
constexpr float DECAY_FACTOR = 0.7f; // Faster decay for quicker note changes
|
|
||||||
|
|
||||||
// FFT Configuration
|
|
||||||
constexpr int FFT_SIZE = 1024; // Must be power of 2, gives ~7.8 Hz resolution at 8kHz
|
|
||||||
constexpr float FREQUENCY_RESOLUTION = static_cast<float>(SAMPLE_RATE) / FFT_SIZE;
|
|
||||||
|
|
||||||
// Piano Note Frequencies (Hz)
|
|
||||||
constexpr float NOTE_FREQ_C2 = 65.41f; // Lowest note we'll detect
|
|
||||||
constexpr float NOTE_FREQ_C3 = 130.81f;
|
|
||||||
constexpr float NOTE_FREQ_C4 = 261.63f; // Middle C
|
|
||||||
constexpr float NOTE_FREQ_C5 = 523.25f;
|
|
||||||
constexpr float NOTE_FREQ_C6 = 1046.50f; // Highest note we'll detect
|
|
||||||
|
|
||||||
// Note Detection Parameters
|
|
||||||
constexpr float FREQUENCY_TOLERANCE = 3.0f; // Hz tolerance, increased for better detection
|
|
||||||
constexpr float MIN_MAGNITUDE_THRESHOLD = 500.0f; // Adjusted for 16-bit samples
|
|
||||||
constexpr int MAX_SIMULTANEOUS_NOTES = 7; // Maximum number of notes to detect simultaneously
|
|
||||||
constexpr float PEAK_RATIO_THRESHOLD = 0.1f; // Peaks must be at least this ratio of the strongest peak
|
|
||||||
|
|
||||||
// Timing and Debug
|
|
||||||
constexpr uint32_t LEVEL_UPDATE_INTERVAL_MS = 50; // Faster updates for better responsiveness
|
|
||||||
constexpr bool ENABLE_DEBUG = true; // Enable debug output
|
|
||||||
constexpr int DEBUG_INTERVAL_MS = 1000; // Debug print interval
|
|
||||||
|
|
||||||
// System Configuration
|
|
||||||
constexpr uint32_t TASK_STACK_SIZE = 4096; // Audio task stack size
|
|
||||||
constexpr uint8_t TASK_PRIORITY = 2; // Increased priority for more consistent timing
|
|
||||||
constexpr uint8_t TASK_CORE = 1; // Run on second core to avoid interference
|
|
||||||
|
|
||||||
// Calibration Parameters
|
|
||||||
constexpr bool CALIBRATION_MODE = false; // Set to true to enable calibration output
|
|
||||||
constexpr int CALIBRATION_DURATION_MS = 5000; // Duration to collect calibration data
|
|
||||||
constexpr float CALIBRATION_PEAK_PERCENTILE = 0.95f; // Use 95th percentile for thresholds
|
|
||||||
|
|
||||||
// Note Detection Timing
|
|
||||||
constexpr int NOTE_PRINT_INTERVAL_MS = 100; // How often to print detected notes
|
|
||||||
constexpr int MIN_NOTE_DURATION_MS = 50; // Minimum duration to consider a note valid
|
|
||||||
constexpr int NOTE_RELEASE_TIME_MS = 100; // Time before considering a note released
|
|
||||||
|
|
||||||
// Serial Commands
|
|
||||||
constexpr char CMD_HELP = 'h'; // Show help
|
|
||||||
constexpr char CMD_CALIBRATE = 'c'; // Start calibration
|
|
||||||
constexpr char CMD_THRESHOLD_UP = '+'; // Increase sensitivity
|
|
||||||
constexpr char CMD_THRESHOLD_DOWN = '-'; // Decrease sensitivity
|
|
||||||
constexpr char CMD_TOGGLE_SPECTRUM = 's'; // Toggle spectrum display
|
|
||||||
constexpr float THRESHOLD_STEP = 0.1f; // Threshold adjustment step
|
|
||||||
|
|
||||||
// Command Response Messages
|
|
||||||
constexpr const char* MSG_HELP =
|
|
||||||
"Commands:\n"
|
|
||||||
"h - Show this help\n"
|
|
||||||
"c - Start calibration\n"
|
|
||||||
"+ - Increase sensitivity\n"
|
|
||||||
"- - Decrease sensitivity\n"
|
|
||||||
"s - Toggle spectrum display\n";
|
|
||||||
}
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include <driver/i2s.h>
|
|
||||||
#include "Config.h"
|
|
||||||
|
|
||||||
// I2S setup functions
|
|
||||||
void initI2S();
|
|
||||||
void readI2SSamples(int16_t* samples, size_t* bytesRead);
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include <arduinoFFT.h>
|
|
||||||
#include <vector>
|
|
||||||
#include <deque>
|
|
||||||
#include "Config.h"
|
|
||||||
|
|
||||||
struct DetectedNote {
|
|
||||||
float frequency;
|
|
||||||
float magnitude;
|
|
||||||
uint8_t noteNumber; // MIDI note number for easy identification
|
|
||||||
uint32_t startTime; // When the note was first detected
|
|
||||||
uint32_t lastSeenTime; // Last time this note was detected
|
|
||||||
};
|
|
||||||
|
|
||||||
class NoteDetector {
|
|
||||||
public:
|
|
||||||
NoteDetector();
|
|
||||||
void analyzeSamples(const int16_t* samples, size_t sampleCount);
|
|
||||||
const std::vector<DetectedNote>& getDetectedNotes() const { return detectedNotes; }
|
|
||||||
const double* getSpectrum() const { return vReal; }
|
|
||||||
void beginCalibration();
|
|
||||||
void endCalibration();
|
|
||||||
bool isCalibrating() const { return calibrationMode; }
|
|
||||||
|
|
||||||
// Threshold adjustment methods
|
|
||||||
void adjustThreshold(float delta);
|
|
||||||
float getCurrentThreshold() const { return dynamicThreshold; }
|
|
||||||
|
|
||||||
private:
|
|
||||||
arduinoFFT FFT;
|
|
||||||
double vReal[Config::FFT_SIZE];
|
|
||||||
double vImag[Config::FFT_SIZE];
|
|
||||||
std::vector<DetectedNote> detectedNotes;
|
|
||||||
std::deque<float> peakMagnitudeHistory; // For calibration
|
|
||||||
|
|
||||||
bool calibrationMode;
|
|
||||||
uint32_t calibrationStartTime;
|
|
||||||
float dynamicThreshold;
|
|
||||||
|
|
||||||
uint8_t freqToNoteNumber(float frequency) const;
|
|
||||||
bool isNoteFrequency(float frequency) const;
|
|
||||||
bool isHarmonic(float freq1, float freq2, float tolerance) const;
|
|
||||||
void findPeaks();
|
|
||||||
void updateNoteTimings();
|
|
||||||
void calculateDynamicThreshold();
|
|
||||||
};
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
#pragma once
|
|
||||||
|
|
||||||
#include <Arduino.h>
|
|
||||||
#include "Config.h"
|
|
||||||
#include "NoteDetector.h"
|
|
||||||
|
|
||||||
class SpectrumVisualizer {
|
|
||||||
public:
|
|
||||||
// Visualization settings
|
|
||||||
static constexpr int DISPLAY_WIDTH = 80; // Characters wide
|
|
||||||
static constexpr int DISPLAY_HEIGHT = 16; // Lines tall
|
|
||||||
static constexpr float DB_MIN = 30.0f; // Minimum dB to show
|
|
||||||
static constexpr float DB_MAX = 90.0f; // Maximum dB to show
|
|
||||||
|
|
||||||
static void visualizeSpectrum(const double* spectrum, int size);
|
|
||||||
static void visualizeNotes(const std::vector<DetectedNote>& notes);
|
|
||||||
static void drawFFTMagnitudes(const double* magnitudes, int size, bool logScale = true);
|
|
||||||
|
|
||||||
private:
|
|
||||||
static constexpr const char* noteNames[] = {
|
|
||||||
"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"
|
|
||||||
};
|
|
||||||
|
|
||||||
static float magnitudeToDb(double magnitude);
|
|
||||||
static int mapToDisplay(float value, float min, float max, int displayMin, int displayMax);
|
|
||||||
static void printBarGraph(float value, float maxValue, int width);
|
|
||||||
};
|
|
||||||
@@ -33,7 +33,4 @@ monitor_filters = esp32_exception_decoder
|
|||||||
lib_deps =
|
lib_deps =
|
||||||
https://github.com/pschatzmann/arduino-audio-tools.git
|
https://github.com/pschatzmann/arduino-audio-tools.git
|
||||||
https://github.com/pschatzmann/arduino-audio-driver.git
|
https://github.com/pschatzmann/arduino-audio-driver.git
|
||||||
https://github.com/tzapu/WiFiManager.git
|
|
||||||
me-no-dev/AsyncTCP
|
|
||||||
https://github.com/me-no-dev/ESPAsyncWebServer.git
|
|
||||||
kosme/arduinoFFT@^1.6
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
#include "AudioLevelTracker.h"
|
|
||||||
|
|
||||||
AudioLevelTracker::AudioLevelTracker() {
|
|
||||||
resetMaxLevel();
|
|
||||||
}
|
|
||||||
|
|
||||||
void AudioLevelTracker::updateMaxLevel(int16_t sample) {
|
|
||||||
uint32_t currentTime = millis();
|
|
||||||
|
|
||||||
// Remove old samples (older than specified duration)
|
|
||||||
while (!sampleHistory.empty() &&
|
|
||||||
(currentTime - sampleHistory.front().timestamp) > HISTORY_DURATION_MS) {
|
|
||||||
sampleHistory.pop_front();
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add new sample, but cap it at MAX_RANGE_LIMIT
|
|
||||||
int16_t absValue = abs(sample);
|
|
||||||
absValue = min(absValue, MAX_RANGE_LIMIT); // Cap at 16-bit limit
|
|
||||||
SamplePoint newPoint = {currentTime, absValue};
|
|
||||||
sampleHistory.push_back(newPoint);
|
|
||||||
|
|
||||||
// Update maximum
|
|
||||||
maxLevel = 0;
|
|
||||||
for (const auto& point : sampleHistory) {
|
|
||||||
if (point.value > maxLevel) {
|
|
||||||
maxLevel = point.value;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
int16_t AudioLevelTracker::getMaxLevel() const {
|
|
||||||
return maxLevel;
|
|
||||||
}
|
|
||||||
|
|
||||||
void AudioLevelTracker::resetMaxLevel() {
|
|
||||||
maxLevel = 0;
|
|
||||||
sampleHistory.clear();
|
|
||||||
}
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
#include "I2SConfig.h"
|
|
||||||
|
|
||||||
void initI2S() {
|
|
||||||
// I2S configuration
|
|
||||||
i2s_config_t i2s_config = {
|
|
||||||
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
|
|
||||||
.sample_rate = Config::SAMPLE_RATE,
|
|
||||||
.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
|
|
||||||
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
|
|
||||||
.communication_format = I2S_COMM_FORMAT_STAND_I2S,
|
|
||||||
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
|
|
||||||
.dma_buf_count = Config::DMA_BUFFER_COUNT,
|
|
||||||
.dma_buf_len = Config::DMA_BUFFER_LEN,
|
|
||||||
.use_apll = false,
|
|
||||||
.tx_desc_auto_clear = false,
|
|
||||||
.fixed_mclk = 0
|
|
||||||
};
|
|
||||||
|
|
||||||
// I2S pin configuration
|
|
||||||
i2s_pin_config_t i2s_mic_pins = {
|
|
||||||
.bck_io_num = Config::I2S_MIC_SERIAL_CLOCK,
|
|
||||||
.ws_io_num = Config::I2S_MIC_LEFT_RIGHT_CLOCK,
|
|
||||||
.data_out_num = I2S_PIN_NO_CHANGE,
|
|
||||||
.data_in_num = Config::I2S_MIC_SERIAL_DATA
|
|
||||||
};
|
|
||||||
|
|
||||||
// Install and configure I2S driver
|
|
||||||
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
|
|
||||||
i2s_set_pin(I2S_NUM_0, &i2s_mic_pins);
|
|
||||||
}
|
|
||||||
|
|
||||||
void readI2SSamples(int16_t* samples, size_t* bytesRead) {
|
|
||||||
i2s_read(I2S_NUM_0, samples, Config::SAMPLE_BUFFER_SIZE * sizeof(int16_t), bytesRead, portMAX_DELAY);
|
|
||||||
}
|
|
||||||
@@ -1,198 +0,0 @@
|
|||||||
#include "NoteDetector.h"
|
|
||||||
#include <algorithm>
|
|
||||||
#include <cmath>
|
|
||||||
|
|
||||||
NoteDetector::NoteDetector()
|
|
||||||
: FFT(vReal, vImag, Config::FFT_SIZE, Config::SAMPLE_RATE)
|
|
||||||
, calibrationMode(false)
|
|
||||||
, calibrationStartTime(0)
|
|
||||||
, dynamicThreshold(Config::MIN_MAGNITUDE_THRESHOLD) {
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::beginCalibration() {
|
|
||||||
calibrationMode = true;
|
|
||||||
calibrationStartTime = millis();
|
|
||||||
peakMagnitudeHistory.clear();
|
|
||||||
Serial.println("Beginning calibration...");
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::endCalibration() {
|
|
||||||
calibrationMode = false;
|
|
||||||
calculateDynamicThreshold();
|
|
||||||
Serial.printf("Calibration complete. Dynamic threshold: %.2f\n", dynamicThreshold);
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::calculateDynamicThreshold() {
|
|
||||||
if (peakMagnitudeHistory.empty()) {
|
|
||||||
dynamicThreshold = Config::MIN_MAGNITUDE_THRESHOLD;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sort magnitudes to find percentile
|
|
||||||
std::vector<float> sorted(peakMagnitudeHistory.begin(), peakMagnitudeHistory.end());
|
|
||||||
std::sort(sorted.begin(), sorted.end());
|
|
||||||
|
|
||||||
// Calculate threshold at specified percentile
|
|
||||||
size_t idx = static_cast<size_t>(sorted.size() * Config::CALIBRATION_PEAK_PERCENTILE);
|
|
||||||
dynamicThreshold = sorted[idx] * Config::PEAK_RATIO_THRESHOLD;
|
|
||||||
|
|
||||||
// Don't go below minimum threshold
|
|
||||||
dynamicThreshold = (dynamicThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
|
||||||
dynamicThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::updateNoteTimings() {
|
|
||||||
uint32_t currentTime = millis();
|
|
||||||
|
|
||||||
// Remove notes that haven't been seen recently
|
|
||||||
detectedNotes.erase(
|
|
||||||
std::remove_if(detectedNotes.begin(), detectedNotes.end(),
|
|
||||||
[currentTime](const DetectedNote& note) {
|
|
||||||
return currentTime - note.lastSeenTime > Config::NOTE_RELEASE_TIME_MS;
|
|
||||||
}),
|
|
||||||
detectedNotes.end()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::analyzeSamples(const int16_t* samples, size_t sampleCount) {
|
|
||||||
// Copy samples to FFT buffer with gain applied
|
|
||||||
for (size_t i = 0; i < Config::FFT_SIZE; i++) {
|
|
||||||
vReal[i] = i < sampleCount ? samples[i] * Config::GAIN : 0;
|
|
||||||
vImag[i] = 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Perform FFT
|
|
||||||
FFT.DCRemoval();
|
|
||||||
FFT.Windowing(FFT_WIN_TYP_HANN, FFT_FORWARD);
|
|
||||||
FFT.Compute(FFT_FORWARD);
|
|
||||||
FFT.ComplexToMagnitude();
|
|
||||||
|
|
||||||
// Find peaks and update timings
|
|
||||||
findPeaks();
|
|
||||||
updateNoteTimings();
|
|
||||||
|
|
||||||
// Handle calibration mode
|
|
||||||
if (calibrationMode) {
|
|
||||||
// Store peak magnitudes during calibration
|
|
||||||
double maxMagnitude = 0;
|
|
||||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
|
||||||
maxMagnitude = std::max(maxMagnitude, vReal[i]);
|
|
||||||
}
|
|
||||||
peakMagnitudeHistory.push_back(maxMagnitude);
|
|
||||||
|
|
||||||
// Check if calibration period is over
|
|
||||||
if (millis() - calibrationStartTime >= Config::CALIBRATION_DURATION_MS) {
|
|
||||||
endCalibration();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
uint8_t NoteDetector::freqToNoteNumber(float frequency) const {
|
|
||||||
// A4 is 440Hz and MIDI note 69
|
|
||||||
return round(12 * log2(frequency/440.0) + 69);
|
|
||||||
}
|
|
||||||
|
|
||||||
bool NoteDetector::isNoteFrequency(float frequency) const {
|
|
||||||
// Check if frequency is within range of C2 to C6
|
|
||||||
if (frequency < (Config::NOTE_FREQ_C2 - Config::FREQUENCY_TOLERANCE) ||
|
|
||||||
frequency > (Config::NOTE_FREQ_C6 + Config::FREQUENCY_TOLERANCE)) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Calculate note number and then expected frequency
|
|
||||||
uint8_t noteNum = freqToNoteNumber(frequency);
|
|
||||||
float expectedFreq = 440.0f * pow(2.0f, (noteNum - 69) / 12.0f);
|
|
||||||
|
|
||||||
// Check if within tolerance
|
|
||||||
return abs(frequency - expectedFreq) <= Config::FREQUENCY_TOLERANCE;
|
|
||||||
}
|
|
||||||
|
|
||||||
bool NoteDetector::isHarmonic(float freq1, float freq2, float tolerance) const {
|
|
||||||
// Check if freq2 is a harmonic of freq1
|
|
||||||
if (freq2 < freq1) {
|
|
||||||
std::swap(freq1, freq2);
|
|
||||||
}
|
|
||||||
|
|
||||||
float ratio = freq2 / freq1;
|
|
||||||
float nearest_harmonic = round(ratio);
|
|
||||||
|
|
||||||
return abs(ratio - nearest_harmonic) <= (tolerance / freq1);
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::findPeaks() {
|
|
||||||
std::vector<DetectedNote> candidates;
|
|
||||||
uint32_t currentTime = millis();
|
|
||||||
|
|
||||||
// Find the maximum magnitude
|
|
||||||
double maxMagnitude = 0;
|
|
||||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
|
||||||
maxMagnitude = (vReal[i] > maxMagnitude) ? vReal[i] : maxMagnitude;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Use either calibrated or relative threshold
|
|
||||||
float relativeThreshold = static_cast<float>(maxMagnitude * Config::PEAK_RATIO_THRESHOLD);
|
|
||||||
float threshold = calibrationMode ? Config::MIN_MAGNITUDE_THRESHOLD :
|
|
||||||
(dynamicThreshold > relativeThreshold ? dynamicThreshold : relativeThreshold);
|
|
||||||
|
|
||||||
// First pass: find all potential peaks
|
|
||||||
for (int i = 2; i < Config::FFT_SIZE/2 - 1; i++) {
|
|
||||||
float frequency = i * Config::FREQUENCY_RESOLUTION;
|
|
||||||
|
|
||||||
if (vReal[i] > threshold &&
|
|
||||||
vReal[i] > vReal[i-1] &&
|
|
||||||
vReal[i] > vReal[i+1] &&
|
|
||||||
isNoteFrequency(frequency)) {
|
|
||||||
|
|
||||||
DetectedNote note;
|
|
||||||
note.frequency = frequency;
|
|
||||||
note.magnitude = vReal[i];
|
|
||||||
note.noteNumber = freqToNoteNumber(frequency);
|
|
||||||
note.startTime = currentTime;
|
|
||||||
note.lastSeenTime = currentTime;
|
|
||||||
candidates.push_back(note);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sort candidates by magnitude
|
|
||||||
std::sort(candidates.begin(), candidates.end(),
|
|
||||||
[](const DetectedNote& a, const DetectedNote& b) {
|
|
||||||
return a.magnitude > b.magnitude;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Second pass: filter out harmonics and update existing notes
|
|
||||||
for (const auto& candidate : candidates) {
|
|
||||||
// Check if this note is already being tracked
|
|
||||||
auto existingNote = std::find_if(detectedNotes.begin(), detectedNotes.end(),
|
|
||||||
[&candidate](const DetectedNote& note) {
|
|
||||||
return note.noteNumber == candidate.noteNumber;
|
|
||||||
});
|
|
||||||
|
|
||||||
if (existingNote != detectedNotes.end()) {
|
|
||||||
// Update existing note
|
|
||||||
existingNote->magnitude = candidate.magnitude;
|
|
||||||
existingNote->lastSeenTime = currentTime;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check if this is a harmonic of a stronger note
|
|
||||||
bool isHarmonicOfStronger = false;
|
|
||||||
for (const auto& note : detectedNotes) {
|
|
||||||
if (isHarmonic(candidate.frequency, note.frequency, Config::FREQUENCY_TOLERANCE)) {
|
|
||||||
isHarmonicOfStronger = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add new note if it's not a harmonic and we haven't hit the limit
|
|
||||||
if (!isHarmonicOfStronger && detectedNotes.size() < Config::MAX_SIMULTANEOUS_NOTES) {
|
|
||||||
detectedNotes.push_back(candidate);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void NoteDetector::adjustThreshold(float delta) {
|
|
||||||
float newThreshold = dynamicThreshold + (dynamicThreshold * delta);
|
|
||||||
dynamicThreshold = (newThreshold > Config::MIN_MAGNITUDE_THRESHOLD) ?
|
|
||||||
newThreshold : Config::MIN_MAGNITUDE_THRESHOLD;
|
|
||||||
Serial.printf("Threshold adjusted to: %.1f\n", dynamicThreshold);
|
|
||||||
}
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
#include "SpectrumVisualizer.h"
|
|
||||||
#include <cmath>
|
|
||||||
|
|
||||||
constexpr const char* SpectrumVisualizer::noteNames[];
|
|
||||||
|
|
||||||
float SpectrumVisualizer::magnitudeToDb(double magnitude) {
|
|
||||||
return 20 * log10(magnitude);
|
|
||||||
}
|
|
||||||
|
|
||||||
int SpectrumVisualizer::mapToDisplay(float value, float min, float max, int displayMin, int displayMax) {
|
|
||||||
float normalized = (value - min) / (max - min);
|
|
||||||
normalized = constrain(normalized, 0.0f, 1.0f);
|
|
||||||
return displayMin + (normalized * (displayMax - displayMin));
|
|
||||||
}
|
|
||||||
|
|
||||||
void SpectrumVisualizer::printBarGraph(float value, float maxValue, int width) {
|
|
||||||
int barWidth = mapToDisplay(value, 0, maxValue, 0, width);
|
|
||||||
for (int i = 0; i < width; i++) {
|
|
||||||
Serial.print(i < barWidth ? "█" : " ");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void SpectrumVisualizer::drawFFTMagnitudes(const double* magnitudes, int size, bool logScale) {
|
|
||||||
// Find maximum magnitude for scaling
|
|
||||||
double maxMagnitude = 0;
|
|
||||||
for (int i = 0; i < size/2; i++) {
|
|
||||||
maxMagnitude = max(maxMagnitude, magnitudes[i]);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Print frequency spectrum
|
|
||||||
Serial.println("\nFrequency Spectrum:");
|
|
||||||
Serial.println("Freq (Hz) | Magnitude");
|
|
||||||
Serial.println("-----------|-" + String('-', DISPLAY_WIDTH));
|
|
||||||
|
|
||||||
for (int i = 0; i < size/2; i++) {
|
|
||||||
float freq = i * Config::FREQUENCY_RESOLUTION;
|
|
||||||
if (freq > Config::NOTE_FREQ_C6) break; // Stop at highest note frequency
|
|
||||||
|
|
||||||
float magnitude = magnitudes[i];
|
|
||||||
if (logScale) {
|
|
||||||
magnitude = magnitudeToDb(magnitude);
|
|
||||||
maxMagnitude = magnitudeToDb(maxMagnitude);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only print if significant magnitude
|
|
||||||
if (magnitude > (logScale ? DB_MIN : 0)) {
|
|
||||||
Serial.printf("%8.1f | ", freq);
|
|
||||||
printBarGraph(magnitude, maxMagnitude, DISPLAY_WIDTH);
|
|
||||||
Serial.println();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void SpectrumVisualizer::visualizeSpectrum(const double* spectrum, int size) {
|
|
||||||
drawFFTMagnitudes(spectrum, size, true);
|
|
||||||
}
|
|
||||||
|
|
||||||
void SpectrumVisualizer::visualizeNotes(const std::vector<DetectedNote>& notes) {
|
|
||||||
if (notes.empty()) return;
|
|
||||||
|
|
||||||
// Find maximum magnitude for scaling
|
|
||||||
float maxMagnitude = 0;
|
|
||||||
for (const auto& note : notes) {
|
|
||||||
maxMagnitude = max(maxMagnitude, note.magnitude);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Print piano keyboard visualization
|
|
||||||
Serial.println("\nDetected Notes:");
|
|
||||||
Serial.println("Note | Freq (Hz) | Level");
|
|
||||||
Serial.println("--------|-----------|" + String('-', DISPLAY_WIDTH));
|
|
||||||
|
|
||||||
for (const auto& note : notes) {
|
|
||||||
int octave = (note.noteNumber / 12) - 1;
|
|
||||||
int noteIndex = note.noteNumber % 12;
|
|
||||||
float dbLevel = magnitudeToDb(note.magnitude);
|
|
||||||
|
|
||||||
Serial.printf("%s%-6d | %8.1f | ",
|
|
||||||
noteNames[noteIndex], octave, note.frequency);
|
|
||||||
printBarGraph(note.magnitude, maxMagnitude, DISPLAY_WIDTH);
|
|
||||||
Serial.printf(" %.1f dB\n", dbLevel);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
338
src/main.cpp
338
src/main.cpp
@@ -1,275 +1,99 @@
|
|||||||
#include <Arduino.h>
|
#include "AudioTools.h"
|
||||||
#include <WiFi.h>
|
// #include "AudioTools/AudioLibs/AudioI2SStream.h"
|
||||||
#include <WiFiManager.h>
|
#include "AudioTools/AudioLibs/AudioRealFFT.h" // or AudioKissFFT
|
||||||
#include <ESPAsyncWebServer.h>
|
|
||||||
#include <AsyncTCP.h>
|
|
||||||
#include <SPIFFS.h>
|
|
||||||
#include "Config.h"
|
|
||||||
#include "I2SConfig.h"
|
|
||||||
#include "AudioLevelTracker.h"
|
|
||||||
#include "NoteDetector.h"
|
|
||||||
#include "SpectrumVisualizer.h"
|
|
||||||
|
|
||||||
// Function declarations
|
I2SStream i2sStream; // I2S input stream for INMP441
|
||||||
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len);
|
AudioRealFFT fft; // FFT analyzer
|
||||||
void initWebServer();
|
FormatConverterStream converter(i2sStream); // Convert 32-bit input to 16-bit output
|
||||||
void handleSerialCommands();
|
StreamCopy copier(fft, converter); // copy converted data to FFT
|
||||||
void printNoteInfo(const DetectedNote& note);
|
|
||||||
void initWiFi();
|
|
||||||
void audioProcessingTask(void *parameter);
|
|
||||||
void visualizationTask(void *parameter);
|
|
||||||
void sendSpectrumData();
|
|
||||||
|
|
||||||
// Static instances
|
int channels = 1; // INMP441 is mono
|
||||||
static int16_t raw_samples[Config::SAMPLE_BUFFER_SIZE];
|
int samples_per_second = 11025;
|
||||||
static AudioLevelTracker audioLevelTracker;
|
int input_bits_per_sample = 32; // INMP441 sends 24-bit data in 32-bit words
|
||||||
static NoteDetector noteDetector;
|
int fft_bits_per_sample = 16;
|
||||||
WiFiManager wifiManager;
|
|
||||||
AsyncWebServer server(80);
|
|
||||||
AsyncWebSocket ws("/ws");
|
|
||||||
|
|
||||||
// Timing and state variables
|
AudioInfo from(samples_per_second, channels, input_bits_per_sample);
|
||||||
static uint32_t lastNotePrintTime = 0;
|
AudioInfo to(samples_per_second, channels, fft_bits_per_sample);
|
||||||
static uint32_t lastSpectrumPrintTime = 0;
|
|
||||||
static uint32_t lastWebUpdateTime = 0;
|
|
||||||
static bool showSpectrum = false;
|
|
||||||
|
|
||||||
// Task handles
|
const char* solfegeName(uint8_t midiNote) {
|
||||||
TaskHandle_t audioTaskHandle = nullptr;
|
static const char* solfegeNames[] = {
|
||||||
TaskHandle_t visualizationTaskHandle = nullptr;
|
"Do", "Do#", "Re", "Re#", "Mi", "Fa", "Fa#", "Sol", "Sol#", "La", "La#", "Si"
|
||||||
|
};
|
||||||
|
return solfegeNames[midiNote % 12];
|
||||||
|
}
|
||||||
|
|
||||||
// Queue for passing audio data between cores
|
void fftResult(AudioFFTBase &fft) {
|
||||||
QueueHandle_t audioQueue;
|
float diff;
|
||||||
|
auto result = fft.result();
|
||||||
|
|
||||||
// Note names for display
|
if (result.magnitude > 100) { // avoid noise floor
|
||||||
const char* noteNames[] = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"};
|
float magnitude_dB = 20.0 * log10(result.magnitude);
|
||||||
|
float freq = result.frequency;
|
||||||
|
|
||||||
void sendSpectrumData() {
|
// MIDI note number
|
||||||
if (ws.count() > 0 && !noteDetector.isCalibrating()) {
|
int midiNote = round(69 + 12.0 * log2(freq / 440.0));
|
||||||
const auto& spectrum = noteDetector.getSpectrum();
|
const char* solfege = solfegeName(midiNote);
|
||||||
String json = "[";
|
int octave = (midiNote / 12) - 1;
|
||||||
|
|
||||||
// Calculate bin range for 60-1100 Hz
|
Serial.print(freq, 0);
|
||||||
// At 8kHz sample rate with 1024 FFT size:
|
Serial.print(" Hz | ");
|
||||||
// binFreq = index * (8000/1024) = index * 7.8125 Hz
|
|
||||||
// For 60 Hz: bin ≈ 8
|
Serial.print("MIDI ");
|
||||||
// For 1100 Hz: bin ≈ 141
|
Serial.print(midiNote);
|
||||||
|
Serial.print(" | ");
|
||||||
for (int i = 8; i <= 141; i++) {
|
|
||||||
if (i > 8) json += ",";
|
Serial.print("Note: ");
|
||||||
json += String(spectrum[i], 2);
|
Serial.print(result.frequencyAsNote(diff));
|
||||||
}
|
Serial.print(" | ");
|
||||||
json += "]";
|
|
||||||
ws.textAll(json);
|
Serial.print("Solfège: ");
|
||||||
|
Serial.print(solfege);
|
||||||
|
Serial.print(octave);
|
||||||
|
|
||||||
|
Serial.print(" | dB: ");
|
||||||
|
Serial.print(magnitude_dB, 2);
|
||||||
|
|
||||||
|
Serial.print(" | Diff: ");
|
||||||
|
Serial.println(diff, 2);
|
||||||
|
|
||||||
|
Serial.print(" | Amp: ");
|
||||||
|
Serial.println(result.magnitude, 0);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
void onWebSocketEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len) {
|
|
||||||
switch (type) {
|
|
||||||
case WS_EVT_CONNECT:
|
|
||||||
Serial.printf("WebSocket client #%u connected from %s\n", client->id(), client->remoteIP().toString().c_str());
|
|
||||||
break;
|
|
||||||
case WS_EVT_DISCONNECT:
|
|
||||||
Serial.printf("WebSocket client #%u disconnected\n", client->id());
|
|
||||||
break;
|
|
||||||
case WS_EVT_DATA:
|
|
||||||
break;
|
|
||||||
case WS_EVT_ERROR:
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void initWebServer() {
|
|
||||||
if (!SPIFFS.begin(true)) {
|
|
||||||
Serial.println("An error occurred while mounting SPIFFS");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
ws.onEvent(onWebSocketEvent);
|
|
||||||
server.addHandler(&ws);
|
|
||||||
|
|
||||||
// Serve static files from SPIFFS
|
|
||||||
server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");
|
|
||||||
|
|
||||||
// Handle not found
|
|
||||||
server.onNotFound([](AsyncWebServerRequest *request) {
|
|
||||||
request->send(404, "text/plain", "Not found");
|
|
||||||
});
|
|
||||||
|
|
||||||
server.begin();
|
|
||||||
Serial.println("HTTP server started");
|
|
||||||
}
|
|
||||||
|
|
||||||
void handleSerialCommands() {
|
|
||||||
if (Serial.available()) {
|
|
||||||
char cmd = Serial.read();
|
|
||||||
switch (cmd) {
|
|
||||||
case Config::CMD_HELP:
|
|
||||||
Serial.println(Config::MSG_HELP);
|
|
||||||
break;
|
|
||||||
|
|
||||||
case Config::CMD_CALIBRATE:
|
|
||||||
noteDetector.beginCalibration();
|
|
||||||
break;
|
|
||||||
|
|
||||||
case Config::CMD_THRESHOLD_UP:
|
|
||||||
noteDetector.adjustThreshold(Config::THRESHOLD_STEP);
|
|
||||||
break;
|
|
||||||
|
|
||||||
case Config::CMD_THRESHOLD_DOWN:
|
|
||||||
noteDetector.adjustThreshold(-Config::THRESHOLD_STEP);
|
|
||||||
break;
|
|
||||||
|
|
||||||
case Config::CMD_TOGGLE_SPECTRUM:
|
|
||||||
showSpectrum = !showSpectrum;
|
|
||||||
Serial.printf("Spectrum display %s\n", showSpectrum ? "enabled" : "disabled");
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void printNoteInfo(const DetectedNote& note) {
|
|
||||||
int octave = (note.noteNumber / 12) - 1;
|
|
||||||
int noteIndex = note.noteNumber % 12;
|
|
||||||
uint32_t duration = millis() - note.startTime;
|
|
||||||
|
|
||||||
Serial.printf("Note: %s%d (%.1f Hz, Magnitude: %.0f, Duration: %ums)\n",
|
|
||||||
noteNames[noteIndex], octave, note.frequency, note.magnitude, duration);
|
|
||||||
}
|
|
||||||
|
|
||||||
void initWiFi() {
|
|
||||||
// Set configuration portal timeout
|
|
||||||
wifiManager.setConfigPortalTimeout(Config::WIFI_CONFIG_TIMEOUT);
|
|
||||||
|
|
||||||
// Set custom portal settings
|
|
||||||
wifiManager.setAPStaticIPConfig(IPAddress(192,168,4,1), IPAddress(192,168,4,1), IPAddress(255,255,255,0));
|
|
||||||
|
|
||||||
// Try to connect to saved WiFi credentials
|
|
||||||
if(!wifiManager.autoConnect(Config::WIFI_AP_NAME, Config::WIFI_AP_PASSWORD)) {
|
|
||||||
Serial.println("Failed to connect and hit timeout");
|
|
||||||
ESP.restart();
|
|
||||||
}
|
|
||||||
|
|
||||||
Serial.println("Successfully connected to WiFi");
|
|
||||||
Serial.print("IP Address: ");
|
|
||||||
Serial.println(WiFi.localIP());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Audio processing task running on Core 1
|
|
||||||
void audioProcessingTask(void *parameter) {
|
|
||||||
while (true) {
|
|
||||||
size_t bytes_read = 0;
|
|
||||||
readI2SSamples(raw_samples, &bytes_read);
|
|
||||||
int samples_read = bytes_read / sizeof(int16_t);
|
|
||||||
|
|
||||||
// Update level tracking
|
|
||||||
for (int i = 0; i < samples_read; i++) {
|
|
||||||
audioLevelTracker.updateMaxLevel(raw_samples[i]);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only analyze if we have enough signal
|
|
||||||
int16_t currentMaxLevel = audioLevelTracker.getMaxLevel();
|
|
||||||
if (currentMaxLevel > Config::NOISE_THRESHOLD) {
|
|
||||||
// Analyze samples for note detection
|
|
||||||
noteDetector.analyzeSamples(raw_samples, samples_read);
|
|
||||||
|
|
||||||
// Send results to visualization task via queue
|
|
||||||
if (xQueueSend(audioQueue, &samples_read, 0) != pdTRUE) {
|
|
||||||
// Queue full, just skip this update
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Small delay to prevent watchdog trigger
|
|
||||||
vTaskDelay(1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Visualization and network task running on Core 0
|
|
||||||
void visualizationTask(void *parameter) {
|
|
||||||
while (true) {
|
|
||||||
int samples_read;
|
|
||||||
|
|
||||||
// Check if there's new audio data to process
|
|
||||||
if (xQueueReceive(audioQueue, &samples_read, 0) == pdTRUE) {
|
|
||||||
uint32_t currentTime = millis();
|
|
||||||
const auto& detectedNotes = noteDetector.getDetectedNotes();
|
|
||||||
|
|
||||||
// Update web clients with spectrum data
|
|
||||||
if (currentTime - lastWebUpdateTime >= 50) {
|
|
||||||
sendSpectrumData();
|
|
||||||
lastWebUpdateTime = currentTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Show spectrum if enabled
|
|
||||||
if (showSpectrum &&
|
|
||||||
!noteDetector.isCalibrating() &&
|
|
||||||
currentTime - lastSpectrumPrintTime >= Config::DEBUG_INTERVAL_MS) {
|
|
||||||
SpectrumVisualizer::visualizeSpectrum(noteDetector.getSpectrum(), Config::FFT_SIZE);
|
|
||||||
lastSpectrumPrintTime = currentTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Print detected notes at specified interval
|
|
||||||
if (currentTime - lastNotePrintTime >= Config::NOTE_PRINT_INTERVAL_MS) {
|
|
||||||
if (!detectedNotes.empty() && !noteDetector.isCalibrating()) {
|
|
||||||
SpectrumVisualizer::visualizeNotes(detectedNotes);
|
|
||||||
}
|
|
||||||
lastNotePrintTime = currentTime;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
ws.cleanupClients();
|
|
||||||
handleSerialCommands();
|
|
||||||
|
|
||||||
// Small delay to prevent watchdog trigger
|
|
||||||
vTaskDelay(1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
void setup() {
|
void setup() {
|
||||||
Serial.begin(Config::SERIAL_BAUD_RATE);
|
Serial.begin(115200);
|
||||||
while(!Serial) {
|
AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Warning);
|
||||||
delay(10);
|
|
||||||
}
|
|
||||||
|
|
||||||
initWiFi();
|
|
||||||
initWebServer();
|
|
||||||
initI2S();
|
|
||||||
|
|
||||||
Serial.println("Piano Note Detection Ready (C2-C6)");
|
|
||||||
Serial.println("Press 'h' for help");
|
|
||||||
noteDetector.beginCalibration();
|
|
||||||
|
|
||||||
// Create queue for inter-core communication
|
// Configure I2SStream for INMP441
|
||||||
audioQueue = xQueueCreate(4, sizeof(int));
|
auto cfg = i2sStream.defaultConfig(RX_MODE);
|
||||||
if (audioQueue == nullptr) {
|
cfg.i2s_format = I2S_STD_FORMAT;
|
||||||
Serial.println("Failed to create queue!");
|
cfg.bits_per_sample = input_bits_per_sample;
|
||||||
return;
|
cfg.channels = channels;
|
||||||
}
|
cfg.sample_rate = samples_per_second;
|
||||||
|
cfg.is_master = true;
|
||||||
|
cfg.pin_bck = 8; // SCK
|
||||||
|
cfg.pin_ws = 9; // WS
|
||||||
|
cfg.pin_data = 10; // SD
|
||||||
|
i2sStream.begin(cfg);
|
||||||
|
|
||||||
// Create audio processing task on Core 1
|
// Configure FormatConverterStream to convert 32-bit to 16-bit
|
||||||
xTaskCreatePinnedToCore(
|
converter.begin(from, to); // Convert to 16-bit
|
||||||
audioProcessingTask,
|
|
||||||
"AudioTask",
|
// Configure FFT
|
||||||
Config::TASK_STACK_SIZE,
|
auto tcfg = fft.defaultConfig();
|
||||||
nullptr,
|
tcfg.length = 8192; // 186ms @ 11kHz minimun C2 theoretical
|
||||||
Config::TASK_PRIORITY,
|
tcfg.channels = channels;
|
||||||
&audioTaskHandle,
|
tcfg.sample_rate = samples_per_second;
|
||||||
1 // Run on Core 1
|
tcfg.bits_per_sample = fft_bits_per_sample; // FFT expects 16-bit data after conversion
|
||||||
);
|
tcfg.callback = &fftResult;
|
||||||
|
fft.begin(tcfg);
|
||||||
|
|
||||||
|
Serial.println("Setup complete. Listening...");
|
||||||
|
|
||||||
// Create visualization task on Core 0
|
|
||||||
xTaskCreatePinnedToCore(
|
|
||||||
visualizationTask,
|
|
||||||
"VisualTask",
|
|
||||||
Config::TASK_STACK_SIZE,
|
|
||||||
nullptr,
|
|
||||||
1, // Lower priority than audio task
|
|
||||||
&visualizationTaskHandle,
|
|
||||||
0 // Run on Core 0
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void loop() {
|
void loop() {
|
||||||
// Main loop is now empty as all work is done in tasks
|
copier.copy(); // Stream mic data into FFT processor
|
||||||
vTaskDelete(nullptr); // Delete the main loop task
|
}
|
||||||
}
|
|
||||||
|
|||||||
Reference in New Issue
Block a user