Learning Three.js

or WebGL for Dummies

Hatsune Miku Dancing in Augmented Reality

This demo shows Hatsune Miku dancing in augmented reality within your browser! This is great and surprisingly easy to do. All that running on your browser, based on web standards. I did it to show it was possible to do AR within the browser. I wanted to share the code to see what you guys can do with it.

Best of all, it even runs on mobile phones that support WebGL and WebRTC. The screenshot on the right is made on a nexus. So we can do augmented reality within the browser on mobile

We can do augmented reality within the browser on currently deployed mobile phones today! There is nothing to wait for.

How is it coded ?

It is mainly a link of 2 parts. First webar extensions for three.js. They bundle what is needed to handle augmented reality with three.js, from the webcam setup to the marker localisation. Second part is the mmd loader which loads the model. It loads the Hatsune Miku model and its animations.

Once you got those 2 parts, things are simple :) you just have to display your model where your marker is. To know the tech details, you will have to dig in the code. I will likely talk more about AR tho.

Now Let’s Do a Bit of History

I did this to show it was possible to do AR within the browser and to share the code to see what you guys can do with it. It was for a AR oriented hackathon in Dublin. It recently landed a new job at Daqri. Part of my job is developer relations, so I get to help people doing AR. I love it :)

All the code to handle Hatsune Miku is from Takahiro Aoyagi. He did a fork on three.js to load mmd formats, the format used to store Hatsune Miku. It may be included in three.js soon. Here is his example for three.js It all started with his mmd viewer in pure webgl. The original contains a LOT of neat features like inverse kinetic, physics for the hair, a toon shader to make the color more cartoonish, a edge shader to enhance the outline of the model like a drawing. He is doing excellent work. Check him out on twitter as @superhoge.

Let’s have a word about Hatsune Miku herself. It is such a nice story. She is a Japanese star but she is purely virtual. She is a humanoid persona which appears as a drawing or as a hologram. There is no human behind, even for the voice! When she sings, what you hear is a voice synthesizer from crypton All that is crazy but true :)

She even does concerts where thousand of fans are go to see her. Actual human fans I mean, not virtual ones. As you can see, Hatsune Miku is no stranger to augmented reality. I love her!

How To Run The Demo ?

You may want to run this demo obviously. Here is how to do it. First you put the demo url in your browser. It will read your webcam using getUserMedia. When it asks for permission, allow it :) Then you need to put this AR marker in front of the camera. You can print it and point the camera toward the paper or you can load the marker web page and put the phone in front of the camera. And now you can see Hatsune Mike Dancing in Augmented Reality.

Now let’s look at it in action in this little screencast.

Conclusion

If you want to experiment with augmented reality and three.js, checkout threex.webar. It is so cool to make augmented reality on the web! I hope you have as much fun as I had doing it :) I can’t wait to see what augmented reality will do on mobile’s browsers.

That’s all folks. Have fun.

LearningThree.js News #8: Stay Tuned With Creative 3D Demos

This is another post for Three.js News. In this eighth edition we are going to travel and dance with cubes, discover Swiss ground with points, and do some commuting with lines. We have a couple of educational viewers, one for asteroids and another, less space related but equally exact, for terrains. We will enjoy turning around with funky vertex and spinning helix. We will discover some beautiful models skillfully displayed in Three.js. We will also be able to measure and forecast happiness.

LearningThree.js News #7: Stay Tuned With Creative 3D Demos

This post is part of the Three.js News series. In this episode we will see some very cool editors, we can share assets and create WebGL experiments with. They are useful, practical and easy to use; they are also a great way to start designing your own 3D models. We will take a look at some impressive demos with cute animals, multiple icons, shaders and crazy skills. All these projects are very fun to play with!

LearningThree.js News #6: Stay Tuned With Creative 3D Demos

This post is our new edition of Three.js News. In this episode we will be amazed by cinematographic experiences and awesome effects. Some of them will even give you goosebumps or make your hairs stand on end. We will explore physics engines, blockbuster movies coming soon, particles, portfolios. We will take a look at everything from a manga universe to a world of change. There is also a show off, a very special mention to a great experience that we found, combining 3d animation, motion capture and projection. These vivid experiences are unforgettable and all too real.

LearningThree.js News #5: Stay Tuned With Creative 3D Demos

This post explores original demos with lots of knowledge. In this episode of Three.js News we will take a tour around 4D dimensions and creative subdivisions. We will also travel over webs of information and have fun with wildlife games. There is a lot of diversity in these projects, but one thing is for sure, they are all developed with the utmost skill and created for the maximum effect.

LearningThree.js News #4: Stay Tuned With Creative 3D Demos

This post introduces another episode of Three.js News. In this edition we explore dynamic and constantly changing experiences. With the featured demos we will travel through time, discover new worlds and parallel universes. We will also get into fun and procedurally generated experiences that never repeat themselves twice; they are different every time, no matter how much you try them out. We will enjoy psychedelic effects and interactive experiments.

See-through Effect for Augmented Reality on Your Phone

This post shows how to read your phone camera and make it appears as transparent, aka to act as a see-through. It may seems unrelated to 3d at first but it is extremely useful in augmented reality. I know it may seem silly :) But this see-throught is the base of Augmented Reality in a phone. This and other AR tech will be the subject of future posts.

It makes your device ‘appears as transparent’ So when the user is looking at her device, she is seeing the reality. You just have to display your augmented part on top and you got augmented reality. A Picture is Worth a Thousand Words :)

Three.js Online Meeting With JoclyGames

Hello everybody.

This is a new Three.js Online Meeting post. The idea is to interview people using Three.js in their projects. I think it is a great way to learn from each other and share our experiences. You know me, I love three.js and I love talking about it. In this video of about 1 hour you are gonna see some interesting people really passionate about games and three.js

For this interview, i talked with Jérôme CHOAIN and Michel Gutierrez , the guys behind Jocly Games. They have over 113 web board games with very cool features as you will see. You can play on and off line with your friends, or just you and your computer. They also have some mobile apps that make their games even more accessible. You can dowload their games from Facebook, Google Play and iTunes. And good news for developers: you can create and embed your own games! It is tons of fun. Be sure to check it out.