Making The Web Rock

Why another audio API?

We have <audio> already!

<audio> hides the steps of loading, decoding and playing

<audio controls src="mysound.ogg"></audio>

Sometimes that's the right thing!

Web Audio provides:

Web Audio provides:

Web Audio provides:

DEMO

What is audio useful for, anyway?

Building Simple App/Game Audio is easy

Loading and Playing a Sound

var myBuffer = null, context = new AudioContext();

function loadDogSound(url) {
    var request = new XMLHttpRequest();
    request.open("GET", "dogBarking.mp3", true);
    request.responseType = "arraybuffer";
    request.onload = function() {
    context.decodeAudioData( request.response, 
        function(buffer) { myBuffer = buffer;  } ); }
    request.send();
}

function playSound( buffer ) {
    var sourceNode = audioContext.createBufferSource();
    sourceNode.buffer = myBuffer;
    sourceNode.connect( audioContext.destination );
    sourceNode.start( 0 );
}

Web Audio API is based on a graph

Web Audio minimizes glitching

Web Audio runs in a separate thread,
so audio and graphics don't compete as much.

You schedule Web Audio events in the future,
and the system takes care of them.

Scheduling Sound Playback

function playEverySecondForTenSeconds( myBuffer ) {
    for (var i=0; i<10; i++) {
        var sourceNode = context.createBufferSource();
        sourceNode.buffer = myBuffer;
        sourceNode.connect( context.destination );
        sourceNode.start( context.now + i );
    }
}

Scheduling in a complex world

DEMO

Scheduling in Web Audio

Scheduling on AudioParams

interface AudioParam {


    attribute value;


    // Parameter automation
    void setValueAtTime( value, time );

    void linearRampToValueAtTime( value, time );

    void exponentialRampToValueAtTime( value, time );


    void setTargetAtTime( target, time, timeConstant );


    void setValueCurveAtTime( values, time, duration );

    void cancelScheduledValues( startTime );


}


Gain Fade Example

var envelope = context.createGain();
mySoundNode.connect( envelope );
envelope.connect( context.destination );

var now = context.currentTime;
envelope.gain.setValueAtTime( 0, now );
envelope.gain.linearRampToValueAtTime( 1.0, now + 2.0 );
envelope.gain.linearRampToValueAtTime( 0.0, now + 4.0 );

mySoundNode.start(0);

Effects in Web Audio

Audio for Music Applications

DEMO

(you may need headphones for this one, sorry...)

<audio> Integration

Audio Input

DEMO

Controlling it All

Web MIDI

CODE

Asking for MIDI devices

window.addEventListener('load', function() {   
  navigator.requestMIDIAccess().then( 
    onMIDIInit, 
    onMIDISystemError );
});

Enumerating MIDI output devices

function onMIDIInit( midi ) {
  for (var input of midiAccess.outputs.values())
    input.send( [0x90, 3, 32] );
}

Don't forget hot-plugging MIDI devices!

midiAccess.onstatechange = 
    function midiConnectionStateChange( e ) { 
        populateMIDIInSelect();
    };

MIDI Message syntax

Enumerating MIDI input devices

function onMIDIInit( midi ) {
  for (var input of midiAccess.inputs.values())
    input.onmidimessage = midiMessageReceived;
}

Parsing MIDI messages

function midiMessageReceived( ev ) {
    var cmd = ev.data[0] >> 4;
    var channel = ev.data[0] & 0xf;
    var noteNumber = ev.data[1];
    var velocity = 0;
    if (ev.data.length > 2)
      velocity = ev.data[2];

    // MIDI noteon with velocity=0 is the same as noteoff
    if ( cmd==8 || ((cmd==9)&&(velocity==0)) ) { // noteoff
      noteOff( noteNumber );
    } else if (cmd == 9) { // note on
      noteOn( noteNumber, velocity);
    } else if (cmd == 11) { // controller message
      controller( noteNumber, velocity);
    } else {
      // probably sysex!
    }
}

DEMO

The Incredible Thing about Web MIDI

Yamaha's Reface series

Peavey's VYPYR series of guitar amps

Web MIDI support

Web Audio Support on Desktop

Web Audio Support on Mobile

Future App Opportunities

What's NOT there (yet) for web audio

• Any kind of plugin/VSTi hooks
• Multi-interface hooks
  (multi-channel, partly; multi-device, no.)

Next Things for Web Audio & Web MIDI

Other Demos

• Immersive, low-friction MIDI: midi.space/
• Farfisa Organ: http://www.farfisaofthefuture.com/
• Soundtrap (DAW): https://www.soundtrap.com/
• Audio Looper: https://github.com/lautr3k/Looper.js
• Yamaha's Web Music Platform OS project: yamaha-webmusic.github.io/webmusicplatform/app/
• Noteflight (notation): Yamaha's NSX1 apps: yamaha-webmusic.github.io
• Synth setupshttp://app.hya.io/
• Drawing a waveform: http://milesofdata.com/waviator/
• Visualization: http://tomasgonzalezvivo.com/soundspirals/

Web Audio Weekly

What I want from you:

• More developers to explore Web Audio and Web MIDI
• Tell us what's not there
• Help us prioritize to make the web platform awesome for audio apps!
• Explore audio techniques and share with others

End