Learning Three.js

or WebGL for Dummies

tQuery WebAudio for More Realistic 3D

This post is about Web Audio API. It is a new HTML5 api which provide great controls on the audio layer. It is much better than the audio tag. It is more suitable for games or music software. As 3D lovers, the one feature that we care about is audio spacialization. This makes Web Audio a complement to webgl. It implements positional sounds, so you get panning, dopler and all. It makes the scene so much more realistic.

The user tends to associate 3D to reality, so efficient 3D is a lot about realism. The brain identifies 2D as artificial while 3D seems immediatly more natural. Anything which appears close to reality increase this effect. Feeding this illusion creates a more immersive experience to the user. Realistic physics is one (stay tuned :). Web Audio audio spacialization is another.

The screencast below is a live coding session using tQuery.WebAudio. If you want, you can experiment with this code in our playground. Just click here and start coding :)

Background on WebAudio API

To learn about WebAudio API is a large subject. It is out of the scope of this introduction. You can refere to many dedicated tutorials tho. The specification itself is clear and very readable.

You can find more comprehensive documentation at html5rocks. It contains several very usefull tutorials to learn Web Audio API. They go from “Getting started with the web audio API” to “mixing positional audio and webgl” or “developing game audio”. If you want, you can find demos in chromium samples.

Now let’s start coding :)

Let’s get Started

First you need to enable WebAudio into your world. Thus you will be able to play sound in it.

1
world.enableWebAudio();

Now that the world is able to do sound, let’s create one

1
var sound = tQuery.createSound();

Now we need to load a sound file and to start playing it. The callback is notified when the file is downloaded. At this point, the sound it ready to play.

1
2
3
sound.load('techno.mp3', function(sound){
    sound.play();
});

If you want to know more, the full API is documented here and here thanks to jsdoc.

Spacialization and Animation

Now that we can play a sound, what else can we do ? One definitively interesting part for webaudio API is the ability to spacialize the sound. The easiest way to do that is to use .follow() function.

1
sound.follow(object3d);

If you want more controls, you can check .updateWithObject3d() or .updateWithMatrix4(). You may want to animate your 3D object based on the sound you hear. For that, a simple function .amplitude() has been written. It is likely a naive implementation but it does the trick, i think.

1
var amplitude = sound.amplitude();

It returns an average of low frequencies of the sound. Note that i dunno if it is the proper term in audio world. Most likely not :) If you know the proper term, please leave a comment.

Want more info ?

You get two examples in the source. The minimal one simply play a sound. The other is more interactive. It is the playground. It allows to play with various parameters. tQuery API documentation is provided via jsdoc. You can find the whole API here. The plan is to do the same for the plugins.

So for tQuery WebAudio plugin, you get API documentation of each class: the main class tQuery.WebAudio and the Sound class. You can even check the Node chain builder doc if you want to go deep and configure your own audio nodes chain :)

Conclusion

Ok so now you can play sounds in your 3D scene. You can make the sound follows a given object3d with realistic sound spatialization. All that in just a few lines. Rather cool if you ask me :) Winamp effects are back !! Later i will likely implement click detection as seen here and environmental effect like ‘cathedral’, ‘telephone’ etc… as seen here.

I love how sound make our 3D scene more realistic. That’s all for today, have fun :)

Comments