- For a simple example. see the “move a cube with your head” demo
- To mess with the code now without any installation, see this jsfiddle example
- for an attempt to make 3d more immersive by using head tracking, see this demo
WebRTC is great!
WebRTC starts to get traction. I love that! We have seen WebRTC and getUserMedia several times in the past: in “Punch a Doom Character in Augmented Reality” post, in “Fun With Live Video in WebGL” post and “Augmented Reality 3D Pong” post. It is already in chrome stable , and will be in firefox real soon. They already talk to each other. Here we don’t need the network part of webrtc. We only need get the webcam video, so getUserMedia is enougth. It is in opera 12 too as you can read here.
auduno is part of Opera team. He wrote it as a demo for opera 12 release which contained getUserMedia. For more info on the library, auduno blogged about internals of his library. You can find details in his blog post. Additionnal info are available in the reference documentation. Some examples are already in three.js, like targets or facekat.
Demo Time !!
As usual we did a plugin for tQuery API to make it easy to use in our environement. One can find 2 examples for it: A educational example where your heads controls a box in 3d. For best result, make sure your face is well and evenly lighted
Let’s Get Started
Ok now lets see how to use this library with
First, we include the
tquery.headtrackr files in your code.
tQuery plugins supports
It makes dependancies much easier to handle.
tquery.headtrackr is no exception, so to include it you can do
1 2 3
Or if you use the good old
<script>, do something like that
headtrackr.js itself, the library which handle the head tracking. Then you just include the plugin itself, and you are done.
Start Tracking Heads
First, you instanciate the object with this simple line.
You can pass various options to
opts is an
with those properties
- opts.width : width of the image containing the face. default to 320px
- opts.height : height of the image containing the face. default to 240px
- opts.headtrackrOpts : options passed directly to headtrackr.js. default to
Default are reasonable, so chances are you dont need to specify anything. To start tracking the head on the webcam, just do the following
It is possible to stop it with
.stop() or to reset it via
Debug View is Cool For User Feedback
If you wish enable the debugView, aka the little visualisation the headtracker result. It gives feedback to the user on what is happening. Thus the user can move his head accordingly or to change lighting of the room.
Face Position Notified thru Events
When a face is found, events are dispatched to notify the detected positions.
1 2 3
event contains normalized coordinates of the detected face.
They use the same axis as WebGL.
If the head is on the center,
event.y will be 0.
And if the head is vertical,
event.angle is 0. More precisely
.y: It is the center position. it varies from [-1,+1], from left to right and bottom to top.
.height: the width and height :) If it is half of whole image, it is equal to 1.
.angle: the Z rotation of the detected head. It is in radian as usual.
.headtrackrEvent: the original facetrackingEvent event from headtrackr.js (see reference )
Head tracking… Kesaco ?
Head tracking is a well known concept. One can find head tracking on ipad. One can find head tracking on wii. They got impressive result using the informations from the wiimote or even the device orientation. With the kinect, they even track the features of the face itself (e.g. mouth, noze, eyes etc…)
In our case, we use the image from the webcam. Unfortunatly face localisation from an image isn’t exactly 100% accurate to say the least :) See here, this is the same demo as the wii one or the ipad one. Yet the result isn’t as convincing. With headtrackr.js and webrtc , we use only the webcam in a uncontrolled environement. So the accuracy is in consequences.
You can improve efficiency by following a few simples advices: Avoid hats or a too crazy haircut. Being bold with a beard doesn’t help :) Make sure your face is well and evenly lighted and you should be fine.
That’s all folks, have fun :)