Ye Olde Kamera - Silverlight 4 Webcam & Old Movie Shader

Another great and successful year comes to an end and I hope that the next year will get even better. We all don't get younger and we change over the years, but the last Silverlight demo I made for this year allows you take some fake aged memories with your webcam. The Silverlight 4 webcam and microphone support is always fun and even more when it's combined with a pixel shader. This demo uses my Old Movie Pixel Shader to create a neat real time webcam effect that simulates scratches, noise and other effects you might have seen in old movies. I also brushed things up a bit by framing the video output with a filmstrip and using some nice new icons from eggheadcafe.

Live
The Silverlight 4 runtime must be installed to run the demo. It's available for Windows and Mac.

 Open the sample in a new page.

The Webcam capturing could be started and stopped with the camera Button. If you press it for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the new "Webcam / Mic" tab to set them.
Press the disk Button to take a framed snapshot and save it to a JPEG file on your harddisk.
The amount of noise can be changed using the Slider and the movie ToggleButton allows you to disable the Old Movie Shader.

How it works
It's basically the Silverlight 4 CaptureSource class and my Old Movie Pixel Shader attached to the video output. I covered the Silverlight webcam and shader usage in this blog post and the JPEG encoding in this one. The Old Movie Shader was presented here.

Source code
The Visual Studio 2010 solution including the refactored Old Movie Shader is available here.

Final words
Thanks to my kids and especially to my wife for all her patience in 2009. I love her!
I also like to thank all the people and new friends I got to know this year and especially the Silverlight community. I hope you liked my blog posts and enjoyed the Silverlight demos I've made. I have more to come next year.
I wish you all a great new year, all the best in 2010 and peace!

Update 03-20-2010
Updated to the Silverlight 4 release candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

World# - Real Time 3D Augmented Reality with Silverlight

Please read the Update at the end of this post.

In the Silverlight 4 Augmented Reality Proof Of Concept blog post I proved that it's possible to implement Augmented Reality applications with Silverlight 4 and the built-in webcam API. The proof of concept used an image showing a Silverlight logo that was attached to the tracked marker. With this blog post I'm back in this field, but this time with real 3D Augmented Reality in Silverlight!
I use the open source Augmented Reality library NyARToolkitCS again and implemented the necessary interfaces to make it work with Silverlight. The NyARToolkitCS library is a completely managed version of the well known ARToolkit. I also use the open source managed game engine Balder for the rendering of 3D models that augment the reality. Balder is mainly developed by the Silverlight MVP Einar Ingebrigtsen. He is working hard on his 3D game engine and provided quick fixes for some issues I encountered while using Balder and I also had a nice chat with him, so big shout out to Einar.

Live
A webcam and at least the Silverlight 4 runtime must be installed to run the sample. To view the application you need to install the Silverlight 4 runtime. It's available for Windows and Mac. You can also watch a video below.
If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Try it out here.

Simply press the "Start Fun" Button, hold the printed marker in front of the camera and move it around. Use the ComboBox to select a different 3D model. The "Flip x-axis" Checkbox could be used to flip the video (the webcam output is mirror-reversed by default).
If you click the "Start Fun" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.
The Car model was made by Psionic, the Tank model by Ken Beyer. The Teapot is from the Balder samples and I guess it was made by Einar Ingebrigtsen. The rest was made by myself.

Video
This time I used Expression Encoder's Screen Capture feature to record a short video of the demo in action. Please keep in mind that the screen recording software eats up a lot of resources while recording and that the actual frame rate is way better.

Background music is Black Lipstick by 4mat / ATEbit (Matt Simmonds).

How it works
Every time the CompositionTarget.Rendering event is fired, a webcam snapshot is taken asynchronously with the CaptureSource.AsyncCaptureImage method. This snapshot is converted in the right format and passed to the NyARToolkitCS library that outputs a transformation matrix. This matrix is used to transform the 3D model. The Balder viewport has a transparent background and the scene with the 3D objects is laid over the webcam video output. Another layer is an image that uses a WriteableBitmap and the WriteableBitmapEx library to highlight the tracked marker region with a red quad.
Sounds pretty easy, but actually it was not that easy and straight forward. One of the hardest parts was the initialization and to find the right parameters.

Results
I'm quite satisfied with the results and if you consider that all the heavy computation is done by the CPU, the performance is pretty good as well. The demo runs smoothly at 50-60 fps on a dual core machine utilizing its whole power and it should also scale up on more cores.

To be continued
If you expected source code this time I have to disappoint you. The code is not at that level where it should be, but I wanted to push this demo out. I'm not sure if I find enough time this year to work on the code and I don't want that someone tries to use it in the current state. But trust me, I will provide open source functionality to make Augmented Reality in Silverlight in the near future and I normally keep my promises, so stay tuned for more happy days...

Update 02-24-2010
This demo is now part of the open source SLARToolkit library which was announced in this blog post.

Update 03-15-2010
Updated to the Silverlight 4 Release Candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

WriteableBitmapEx - WriteableBitmap extensions now on CodePlex

I finally kept my promise and put the WriteableBitmap extensions up on CodePlex. This step was long overdue, but when the Silverlight 4 beta was released I couldn't resist and had to play with the new webcam API first (though I'm not finished yet).

The WriteableBitmapEx library project description:

The WriteableBitmapEx library is a collection of extension methods for Silverlight's WriteableBitmap. The WriteableBitmap class that was added in Silverlight 3, allows the direct manipulation of a bitmap and could be used to generate fast procedural images by drawing directly to a bitmap. The WriteableBitmap API is very minimalistic and there's only the raw Pixels array for such operations. The WriteableBitmapEx library tries to compensate that with extensions methods that are easy to use like built in methods. The library extends the WriteableBitmap class with elementary and fast (2D drawing) functionality, supporting common shapes like point, line, ellipse, polyline, quad, rectangle, triangle. Conversion methods and functions to combine WriteableBitmaps (Blitting) are part of it too.

Roots
The WriteableBitmapEx library has its origin in several blog posts:

• WriteableBitmap Extension Methods introduced the SetPixel methods.
• Drawing Lines - Silverlight WriteableBitmap Extensions II provided the DrawLine methods.  
• Drawing Shapes - Silverlight WriteableBitmap Extensions III brought the shape functionality (ellipse, polyline, quad, rectangle, triangle).
• In Convert, Encode And Decode Silverlight WriteableBitmap Data I published the byte array conversion methods. 
• Not at least Silverlight MVP Bill Reiss was faster than me on the Blit functions and introduced these in Blitting and Blending with Silverlight’s WriteableBitmap. He was so nice and allowed me to use his code in the WriteableBitmapEx library. 
• Nikola Mihaylov (Nokola) made some optimizations on the DrawLine and DrawRectangle methods and posted them on his own blog or in the comments of my posts. So a release as open source project was really long overdue.
  

Additional work
I recreated the whole Visual Studio 2008 solutions, changed the namespaces, headers, added comments and cleaned up the code base and samples. I also grouped the extension methods into different CS files with a partial class. It is now possible to include just a few methods by using the specific source CS files directly or all extension methods through the built library assembly.

License
The WriteableBitmapEx library is released as open source under the Microsoft Public License (Ms-PL) license. The Ms-PL license allows the use of the library without affecting the license of the user, thus making it also easily usable for commercial projects. On the other hand the weak copyleft characteristic ensures that changes on the source have to be committed back and that the library or parts of it always be free and never become closed source.

Touch it, Buy it, Use it, Break it, Fix it, Trash it, Change it, (Mail) - Upgrade it, ...

The WriteableBitmapEx CodePlex project site lives here. You might also want to see the Issue Tracker for a list of features that will be added in the future or to add your own. If you have any comments, questions, suggestions or want to see your name in the Credits list, don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun and let me know it if you do some cool stuff with it.

Silverlight 4 Augmented Reality Proof Of Concept

Please goto: World# - Real Time 3D Augmented Reality with Silverlight. Such demos are now easily possible with the open source SLARToolkit.

In this short blog post I want to present a demo I've actually made last week. Unfortunately I had no time to work on it since then. This demo is a proof of concept for doing Augmented Reality with Silverlight 4 and the built-in webcam API.
I use the open source Augmented Reality library NyARToolkitCS and implemented the necessary interfaces to make it work with Silverlight. The NyARToolkitCS library is a completely managed version of the well known ARToolkit.

Video
I've recorded a short video with my iPhone 3GS. The video was recorded at night and it's no secret that the iPhone camera is not the best, but I think it's good enough to see how the marker is tracked and the Silverlight logo moves with it.

Background music is "You have something in your scowl" by Henrik José / Bliss.

To be continued
The first results are quite good and the demo runs smoothly at 60-70 fps on a dual core machine utilizing both cores. I will continue my work on this project and provide more details, samples and of course source code.
Stay tuned...