Making The Web Rock

Web Audio

Chris Wilson / @cwilso

Google Chrome Developer Advocate

Why another audio API?

We have <audio> already!

<audio> hides the steps of loading, decoding and playing

<audio controls src="mysound.ogg"></audio>

Sometimes that's the right thing!

Web Audio provides:

1) Precise timing of lots of overlapping sounds

Web Audio provides:

2) An audio pipeline/routing system for effects and filters

Web Audio provides:

3) Hooks to analyze and visualize audio data on the fly


(Analysis, Filtering, Visualization)

What is audio useful for, anyway?

  • Gaming
  • Application UI feedback
  • Musical applications
  • Audio education
  • Audio processing

Building Simple App/Game Audio is easy

  • Load audio files with XHR
  • Tell Web Audio to decode them into buffers
  • Create source node, point at buffer, connect it
  • Call start()!

Loading and Playing a Sound

var myBuffer = null, context = new AudioContext();

function loadDogSound(url) {
    var request = new XMLHttpRequest();"GET", "dogBarking.mp3", true);
    request.responseType = "arraybuffer";
    request.onload = function() {
    context.decodeAudioData( request.response, 
        function(buffer) { myBuffer = buffer;  } ); }

function playSound( buffer ) {
    var sourceNode = audioContext.createBufferSource();
    sourceNode.buffer = myBuffer;
    sourceNode.connect( audioContext.destination );
    sourceNode.start( 0 );

Web Audio API is based on a graph

Web Audio minimizes glitching

Web Audio runs in a separate thread,
so audio and graphics don't compete as much.


You schedule Web Audio events in the future,
and the system takes care of them.

Scheduling Sound Playback

function playEverySecondForTenSeconds( myBuffer ) {
    for (var i=0; i<10; i++) {
        var sourceNode = context.createBufferSource();
        sourceNode.buffer = myBuffer;
        sourceNode.connect( context.destination );
        sourceNode.start( + i );

Scheduling in a complex world

For dynamic rhythms, you need to combine web audio and system timing. See article.


Drum Machine

Scheduling in Web Audio

Not just about start( time )!

ANY AudioParam can be scheduled -
frequency, gain, detune, delayTime...

Scheduling on AudioParams

interface AudioParam {

    attribute value;

    // Parameter automation
    void setValueAtTime( value, time );

    void linearRampToValueAtTime( value, time );

    void exponentialRampToValueAtTime( value, time );

    void setTargetAtTime( target, time, timeConstant );

    void setValueCurveAtTime( values, time, duration );

    void cancelScheduledValues( startTime );


Gain Fade Example

var envelope = context.createGain();
mySoundNode.connect( envelope );
envelope.connect( context.destination );

var now = context.currentTime;
envelope.gain.setValueAtTime( 0, now );
envelope.gain.linearRampToValueAtTime( 1.0, now + 2.0 );
envelope.gain.linearRampToValueAtTime( 0.0, now + 4.0 );


Effects in Web Audio

  • Biquad Filtering - lowpass, hipass, etc.
  • Delays and delay effects
  • Waveform synthesis: oscillators
  • Envelopes
  • Offline processing
  • Compression
  • Convolution
  • Waveshaping
  • Positioning/Panning/Doppler
  • Custom Javascript processing

Audio for Music Applications

Most "musical effects" are more complex than
just a single filter or delay.

AudioParams can also be driven by audio-rate signals -
a chorus effect is just an oscillator changing delayTime!


(you may need headphones for this one, sorry...)

Audio Input Effects

<audio> Integration

Web Audio can also process <audio> streams
(and WebRTC, too!)

Audio Input

Access to audio input devices too!


Building a Synthesizer

Controlling it All

Web MIDI :
Developing standard - Shipping in Chrome, Firefox developing implementation, and I wrote a cross-browser plugin-based polyfill.


This is not cheesy background music!  
That's "Standard MIDI files."

MIDI lets you connect controllers, synthesizers and more to your computer.


So how did I hook up that synthesizer?

Asking for MIDI devices

window.addEventListener('load', function() {   
    onMIDISystemError );

Enumerating MIDI output devices

function onMIDIInit( midi ) {
  for (var input of midiAccess.outputs.values())
    input.send( [0x90, 3, 32] );

Don't forget hot-plugging MIDI devices!

midiAccess.onstatechange = 
    function midiConnectionStateChange( e ) { 

MIDI Message syntax

MIDI has 16 virtual channels, blah blah blah.

MIDI spec:

MIDI note numbers:

Enumerating MIDI input devices

function onMIDIInit( midi ) {
  for (var input of midiAccess.inputs.values())
    input.onmidimessage = midiMessageReceived;

Parsing MIDI messages

function midiMessageReceived( ev ) {
    var cmd =[0] >> 4;
    var channel =[0] & 0xf;
    var noteNumber =[1];
    var velocity = 0;
    if ( > 2)
      velocity =[2];

    // MIDI noteon with velocity=0 is the same as noteoff
    if ( cmd==8 || ((cmd==9)&&(velocity==0)) ) { // noteoff
      noteOff( noteNumber );
    } else if (cmd == 9) { // note on
      noteOn( noteNumber, velocity);
    } else if (cmd == 11) { // controller message
      controller( noteNumber, velocity);
    } else {
      // probably sysex!


Spinning the discs - wubwubwub

The Incredible Thing about Web MIDI

ZERO-FRICTION controller and device access!

Yamaha's Reface series

Peavey's VYPYR series of guitar amps

Web MIDI support

Shipping in Chrome
Works on Android Chrome with USB OTG!
Firefox is actively working on it.

Web Audio Support on Desktop

Chrome, Safari, Firefox, Edge

Web Audio Support on Mobile

Chrome for Android has support (higher latency)
iOS Safari 6.0 has Web Audio (with some caveats)

Future App Opportunities

  • Immersive gaming audio
  • Audio feedback and input in app UX
  • Audio education - explore techniques, share with others
  • Super-low-friction music applications - synthesis to DAW

What's NOT there (yet) for web audio

  • Any kind of plugin/VSTi hooks
  • Multi-interface hooks
    (multi-channel, partly; multi-device, no.)

Next Things for Web Audio & Web MIDI

  • Audio Workers (processing audio in JS)
  • Figuring out device access and selection
  • latency alignment
  • Finalizing v1
  • Low-level access to audio stream and resampling

Other Demos

Web Audio Weekly

If you're not already reading Chris Lowis' Web Audio Weekly, you should.

What I want from you:

  • More developers to explore Web Audio and Web MIDI
  • Tell us what's not there
  • Help us prioritize to make the web platform awesome for audio apps!
  • Explore audio techniques and share with others


+Chris Wilson