Cubelicious - Silverlight 5 + Balder + Physics + SLARToolkit Augmented Reality = Triple Win!

Two months ago I released a new Silverlight 5 sample for my open source Silverlight Augmented Reality Toolkit - SLARToolkit. It uses the new Silverlight 5 hardware accelerated 3D API. You can read more about the new Silverlight 5 XNA 3D API in this blog post.
This post here provides a new demo for SLARToolkit which uses the open source 3D engine Balder by my friend Einar Ingebrigsten. This demo also leverages the open source physics engine JigLibX my buddy Andy Beaulieu ported over to Silverlight. You can try the live sample if you have the Silverlight 5 beta installed or watch a video instead.

I actually spent most of the time on this project a couple of weeks ago. My good Silverlight MVP friend Einar Ingebrigtsen used it in his talk at The Gathering 2011. Now I finally had a bit time to finish the demo, make a video and this blog post.

The SLARToolkit project description from the CodePlex site:

SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. SLARTookit is based on the established NyARToolkit and ARToolkit. 

Live

A webcam and at least the Silverlight 5 beta runtime must be installed to run the sample. It's available here. Alternatively there is also a video of the new sample embedded below.

If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. As an alternative it's also possible to open a marker file on a mobile device and to use the device's screen as marker. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Open the sample in a new page.

Simply press the "Start Cam" Button to start the webcam. The properties of the particle system can  be changed with Sliders. The "Flip x-axis" Checkbox can be used to flip the video (the webcam output is mirror-reversed by default). 

If you click the "Start Cam" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

Video

I've recorded a short video of the new sample with Expression Encoder's Screen Capture feature. Please keep in mind that the screen recording software eats up a lot of resources while recording and that the actual frame rate is even better. The video is also available at YouTube.

Background music is Lullaby by _ghost

This demo shows how the new Silverlight 5 3D API, the Balder engine and the JigLibX physics library can be used to augment the reality with the help of SLARToolkit.

How it works

This sample uses the webcam video stream which fills a Rectangle shape, the video stream is also constantly captured and fed to the SLARToolkit BitmapMarkerDetector to detect the markers. The detection result contains a transformation matrix for each found marker which is then used to apply a global transformation to the cubes and the plane.
I implemented a particle system with a flexible directed emitter which can be controlled through various properties. The particle system is quite generic and can be used for all kinds of particles (3D objects). The particle collision detection and resolving is based on rigid body physics that was implemented with the help of the JigLibX library my Silverlight MVP buddy Andy Beaulieu ported over to Silverlight.
The rendering and the model loading is done by the 3D engine Balder. It's a fantastic open source engine by Einar Ingebrigtsen. You just need to write a couple lines of XAML and you're good to go. This sample only uses a simple cube model, but Balder has built-in model loaders to load complex 3D models and Einar provides a big sample library. He also brought the engine to a few more platforms like Windows Phone 7, OpenGL and has even a neat software rendering fallback. Read his blog post here.
As part of this sample I needed some vector and quaternion methods which were missing in Balder. I contributed those and the generic particle system to the Balder project. Feel free to use the particle system and the other methods in your Balder projects.

Download it, build your app and augment your reality

The open source SLARToolkit library and all samples are hosted at CodePlex. If you have any comments, questions or suggestions don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun with the library and please keep me updated if you use it anywhere so I can put a link on the project site.