A Vocoder on the RP2040: From Sine Wave to (Almost) a Talking Robot

The solution was switching to Q16.16 fixed-point arithmetic (32-bit signed integer, 16 integer/16 fractional bits) for everything running in the sample hot path. Only the filter coefficient calculation (sinf/cosf, once per band at startup) stayed in float — there, readability matters more than speed.

Result: 3.56x faster — noticeable, but not enough yet. The M0+ core has a single-cycle 32×32→32-bit multiply, but no native 64-bit multiply, which our q16_mul() needs — the compiler synthesizes that across several instructions.

Instead of tweaking the architecture further, three purely algebraic transformations helped, with zero loss of precision:

Combined with lowering the sample rate to 22.05kHz (our highest band frequency sits well below that, no loss of information), this got me under budget — from 8.6x too slow to a comfortable margin.

The potentiometer and the microphone hang off the same RP2040 ADC (just one hardware mux with a single "currently selected channel" state). Two FreeRTOS tasks switching independently would rip the channel out from under each other mid sample-loop — with no error message, only visible as corrupted audio. Solution: only one task ever touches the ADC; the raw value travels to a second task, which handles the smoothing, via a queue.

Once everything else was working, the potentiometer simply didn’t respond to being turned at all — carrierHz stubbornly stayed at its startup value. Diagnostic values showed: the raw pot reading came through cleanly, but it never reached the task responsible for smoothing it.

The cause was a textbook example of preemptive scheduling: the audio task had not a single genuine FreeRTOS blocking point anywhere in its loop — take_audio_buffer() from pico_audio_i2s is written for bare metal, not FreeRTOS-aware blocking, presumably pure busy-spinning. Under preemptive scheduling, the scheduler only switches to a lower-priority task when the currently running task genuinely blocks — not just when it "has nothing to do right now." Without a real blocking point in the audio task, the pot task never got any CPU time at all, and its own vTaskDelay() never elapsed.

One line of code, but one of the most instructive hours of the entire project.

A sawtooth carrier has a 1/n harmonic rolloff. At a fundamental frequency of, say, 200Hz, the harmonic at 2kHz is already the 10th overtone — with strongly reduced energy. But that’s exactly where (800Hz–3kHz) the formants F2/F3 live, which distinguish vowels from one another. The analysis correctly detected different energy per band, but the carrier had nothing left to shape there. Result: loudness modulates, timbre stays the same — a "modulated hum."

Fix: a narrow pulse train (20% duty cycle) instead of a sawtooth — historically the standard carrier for vocoders, for exactly this reason. A narrow pulse has a much more even, broadband harmonic spectrum.

My reference implementation uses a proper compressor (threshold, ratio, attack/release) instead of a hard clip on the summed output. A hard clip cuts off signal peaks abruptly (harsh/metallic), a compressor squeezes them down gently and preserves more fine detail. In fixed-point, this could be built using the envelope-following infrastructure I already had — just one division per sample for the dynamically changing compression gain, everything else precomputed.

A static noise ratio mixed into the carrier (a fixed percentage of noise blended with the pulse train) can’t serve vowels and consonants well at the same time — vowels need a clean tonal carrier, consonants need noise energy. My first attempt (zero-crossing rate) never had properly calibrated thresholds. The reference implementation showed a proven alternative: split the signal around 1.5kHz into a low and high component, track a smoothed level for each, and use the difference as the decision.

The first calibrated threshold had a reasoning error: I had assumed the high-frequency component had to exceed the low-frequency one in absolute terms. Real measurements showed otherwise: sibilants only reach ~20–26% of the low-band level, never 100% — sibilants are inherently quieter than vowels, acoustically. The threshold needed to be defined as a ratio, not a comparison between two absolute quantities.

After the mic trimmer was already at its (presumed) limit and the signal was still clipping, I added an external voltage divider for extra attenuation. That introduced a new problem: the divider doesn’t just attenuate the AC amplitude, it also shifts the DC operating point along with it — but my code still assumed the original ADC midpoint.

Instead of recalculating the new operating point by hand (which I did at first, and which would have been wrong again the next time a component changed), the code now continuously tracks the DC component itself, via a very slow low-pass filter:

That was the third fix in a row for the same underlying issue — time to solve the problem structurally instead of patching it point by point.

The MAX4466 datasheet supplies a concrete number for the remaining low-pass character: a gain-bandwidth product of 600kHz, with the trimmer’s gain range spanning 25×–125×. At 125x gain, only about 4.8kHz of usable bandwidth remains — a fundamental op-amp trade-off, not a misconfiguration. A comparison test without the voltage divider confirmed it: the band pattern widened noticeably (activity reaching band 7–8 instead of band 3), but not completely — both effects (divider loading and the gain-bandwidth trade-off) appear to be stacking on top of each other.

At this point I’d verified the following as solid, working foundations: CPU budget with room to spare, the priority-starvation fix, a proper compressor, a working two-core split, voiced/unvoiced detection that cleanly separated vowels from sibilants, and a self-calibrating mic input. What remained open was the upper-band energy loss above — and, as it turned out, something else entirely that none of this had touched yet.

In roughly the order I chased them down:

The test that finally nailed the mechanism: pull the DAC’s I2S data line to ground — silencing real audio output — while everything else kept running. The mic side went completely quiet. Reconnect the line, and the runaway came back, specifically at certain carrier pitches. That’s about as clean a confirmation of electrical crosstalk from the DAC output into the mic input as you can get without an oscilloscope.

Rerouting the DAC’s data line away from the mic signal path on the breadboard helped substantially — but not completely; a band near the low end of the pitch range remained troublesome. Narrowing the usable pitch range worked around it, but that’s containment, not a fix, and I said so explicitly at the time.

Growing suspicious of the breadboard itself (after ruling out everything downstream of it), I measured resistance between the ground pins of the Pico, the mic preamp, and the DAC — with the board unpowered, so there was no risk of a short. The result: 3–4Ω, where a clean connection should read close to 0Ω.

That’s the whole story right there. A shared, not-quite-low-enough ground path between a loud output stage and a highly sensitive input stage (the MAX4466 can run at up to 125x gain) turns any current the DAC draws into a small voltage ripple on the ground reference itself — which the mic preamp then dutifully amplifies as if it were signal. It explains the pitch-dependence (different tones draw different current), the self-sustaining nature (louder output → more ground ripple → keeps the gate open → stays loud), and why turning down volume alone never fixed it (same current, same resistor, same ripple, regardless of how "quiet" the signal nominally is).

The fix was almost anticlimactic after two weeks of software-side detours: star grounding — individual, direct wires from a single ground point to each IC, instead of relying on the breadboard’s own rail. Verified over the full original pitch range with no recurrence.

I want to be honest about the shape of this story: several fixes I made along the way (the duty-cycle change, the gate hysteresis, the pitch-range narrowing) were reasonable and stayed in the design — but none of them were the fix, and I said that plainly in my own notes at each step rather than declaring victory prematurely. The actual fix took a five-minute resistance measurement that I could have done in week one.

Six bands (each needing two filters — analysis and synthesis, unlike the talkbox’s single filter) at full 44.1kHz sample rate blew straight through the CPU budget again (avg=6600us against a 5804us budget) — the exact same category of problem as Chapter 2, just triggered by a different parameter. The fix was the same one, too: drop the sample rate to 22.05kHz. With a highest band at 2kHz, Nyquist at 11kHz leaves plenty of headroom.

Partway through, I considered switching from the MAX4466 to the MAX9814 — a mic preamp with a flat 20Hz–20kHz response regardless of gain setting, which would sidestep the gain-bandwidth trade-off from Chapter 6 entirely. The datasheet confirms it: the MAX9814 splits its gain across multiple internal stages instead of one high-gain op-amp, which is exactly why it doesn’t pay the same bandwidth penalty.

But the MAX9814 also has built-in automatic gain control (AGC) — and AGC actively works to flatten out the very loudness dynamics an envelope-follower vocoder depends on as its primary signal. This was the reason I’d chosen the MAX4466 over it at the very start of the project, and revisiting it now didn’t change that trade-off: more bandwidth, but at the risk of fighting the AGC for the thing that actually carries the vocoder’s information. I decided against the swap and instead widened the existing voltage divider (2kΩ/2kΩ → 12kΩ/12kΩ, same attenuation ratio, five times less loading on the op-amp’s output) — confirmed via the same per-band diagnostic that meaningfully more energy now reaches the upper bands than before.

With band-by-band diagnostics back in place, a pattern showed up that held steady across every vowel tested: the lowest band consistently carried roughly 5x the energy of the highest band, regardless of what was being said. That’s not vowel-specific information — it’s just the natural spectral tilt of the human voice source, which rolls off with frequency independent of vowel identity. With that large and constant an imbalance, whatever vowel-dependent shifts existed between the upper bands were getting perceptually swamped by the dominant low band in the summed output.

The fix: a fixed, per-band output gain applied after synthesis, increasing with frequency, to compensate for the natural tilt. This is a different mechanism from an experiment I’d tried and abandoned back in the 12-band project (a uniform envelope boost applied equally to every band) — a uniform multiplier changes nothing about the relative balance between bands, which is exactly why it made no difference back then. A per-band, increasing gain does.

Rather than keep comparing vowels, I isolated a more basic question: does the synthesis side even sound different from band to band, on its own, independent of any voice input? A simple solo switch — mute every band except one — made this directly testable.

The result at Q=5: soloing the lowest band produced near total silence; soloing a band a few steps up produced sound. The analysis side of that low band was picking up real signal; its synthesis side was getting essentially nothing to shape.

The reason is mechanical, not a bug: a periodic pulse-train carrier only has energy at discrete harmonics, spaced by its fundamental frequency, with large gaps in between. At Q=5, a band’s passband is narrow enough that whether any given harmonic happens to fall inside it is close to a coin flip, depending on the exact carrier pitch — and that coin flip has nothing to do with the vowel being analyzed. Turning Q down again (as the earlier Q=2 setting effectively did) "fixes" this by widening every band enough to usually catch something — but then the windows overlap so much that everything picks up the same dominant fundamental, and the bands sound similar regardless of vowel for the opposite reason.

The fix: decouple analysis and synthesis Q. Keep analysis narrow (Q=5, for clean formant separation), but reconfigure each band’s synthesis filter separately with a much wider Q (1.2) — trading frequency selectivity on the output side for reliable coverage, independent of the input side’s precision.

With decoupled Q, both a low and a high band were now clearly audible — but still not clearly different from each other in character. This points at something more fundamental than a filter-width parameter: a purely periodic carrier, no matter how you tune Q, has fundamentally sparse spectral content. There’s no Q setting that resolves the tension between "narrow enough to differentiate" and "dense enough to reliably excite every band" when the excitation signal itself only exists at a handful of discrete frequencies.

The textbook answer for this is the same one I’d already used for consonants back in Chapter 4: mix noise into the carrier. Noise has continuous energy across the whole spectrum, so every band reliably gets real, distinguishable content to shape, independent of carrier pitch. Blending a fixed 40% noise into the carrier (ahead of full voiced/unvoiced switching, which is the natural next refinement) immediately ran into one more constant-Q subtlety: white noise isn’t neutral in a constant-Q filter bank. Since bandwidth scales with center frequency at a fixed Q, a higher band has a wider absolute bandwidth and therefore passes proportionally more flat-spectrum noise energy than a lower one — a highest band came through noticeably louder than the lowest, purely from filter geometry, nothing to do with the voice signal. The fix, matching the well-known technique from analog synthesizers: shape the noise with a simple one-pole low-pass before mixing it in, giving it a "pink" tilt (energy per octave rather than per hertz) that matches how the bands themselves are spaced.

This is where the project stands as I write this: solo-band tests with shaped noise still in progress, aiming for bands that are both audible in comparable proportion and audibly distinct from one another — the actual prerequisite for a vowel comparison to mean anything.