Kodierer [Coder]

Breakdown of Windows Phone App Download Statistics

2012-01-13T20:43:00.003+01:00

Line Graph by Scott Lewis, Noun

In this blog post I want to share the download graph of my quite successful Windows Phone app Pictures Lab and my conclusions. Some of you might have seen a similar graph already on Twitter or on Facebook, but I thought an update and a bit more explanation might probably make sense.

The graph below shows the download statistics of my Pictures Lab app during the last 9 months. The green line represents the daily downloads and the orange are the accumulated downloads. I blurred the numbers at the scales, since I don't release the download number for various reasons. Please note, Pictures Lab offers a trial mode which means users can try the app for free. Therefore the download numbers aren't equal to the number of sold licenses! Although the app has a pretty good conversion rate from trial-to-paid of ~32% worldwide average and in December it even was 97% in Norway! I noticed that all the Scandinavian countries seem to have a great conversion rate in general. Worst seems to be Hong Kong, but that's a story for a different blog post. Just to be clear, Pictures Lab provides me with a solid extra income each month and the trend is positive, but it's not enough to make a living out of that yet.
The absolute download numbers actually don't matter that much for this blog post. The important part of the graph are the text labels I manually added. Those labels mark events I tracked and noted at that time and are usually followed by a download peak.

Pictures Lab download graph from 04-01-2011 to 01-07-2012 (click on the image for the original size).

My conclusions

Steadily releasing updates with new functionality and being featured at the Windows Phone sites has a big impact on the downloads. Keep in mind the major Pictures Lab updates were featured at least at 3-5 of the top Windows Phone sites.
Being featured at the Marketplace also has a significant positive impact on downloads. Of course this depends on how big the local Marketplace is. Unfortunately does the AppHub not provide the information when an app has been featured. There is the Distimo service which tracks that too, but that service needs your live id credentials and I won't give a 3rd party site my live id to store it in their database. Fortunately some of my friends ping me when they see my apps being featured in their local marketplace. I think it's pretty much safe to say that all the graph peaks without a label nearby are related to a Marketplace feature.
The base line increased not only after the 3.0, but also after the peak of the 4.0 update. The 4.0 update brought the multi-language support with 10 languages. The steady download jump after 4.0 is a pretty good indication that localization can help to increase downloads.
The price drop to $0.99 was followed by the highest peak in the graph, but after that it went down pretty quickly. I think the price point is the hardest thing to get right and it largely varies for different kind of apps, so the above Pictures Lab pricing information shouldn't be generalized.
Of course Christmas, the holidays and New Year also resulted in nice peaks. Customers have time to browse the marketplace or just want to fill up their brand new Windows Phones with some apps.

What about the Windows Phone 7.5?
The first Windows Phone 7.5 (previously known as Mango) devices and Nokia hit the market in October and November 2011. This fact also needs to be considered when interpreting the graph. Although from looking at the statistics of my other apps I see that Windows Phone 7.5 didn't have such a huge impact like the updates had. The downloads definitely increased due to the Windows Phone 7.5 launch, but the tracked events align very well with the peaks in the graph and the increase of the base line after.
Let me try to clarify this and my conclusion #3 with the graphs of my successful Helium Voice Free app and my not so successful Benchmark Free app.

Helium Voice Free got a nice update in November, but I didn't inform the Windows Phone sites, so there are no peaks in the graph. You can see that the release of Windows Phone 7.5 slightly increased the downloads and the downward trend turned into an upward trend.

Helium Voice Free graph from 04-01-2011 to 01-07-2012 (click on the image for the original size to see numbers).

In the Benchmark Free download graph there's also this small increase of the average downloads around the Windows Phone 7.5 device launch, but it's not as much as after the updates of Pictures Lab.

Benchmark Free graph from 04-01-2011 to 01-07-2012 (click on the image for the original size).

Considering all that information, I think this means the Windows Phone 7.5 launch of course raised the downloads base line, but the Pictures Lab updates had a bigger impact on the downloads and it's not only the Windows Phone 7.5 launch which raised the average downloads of Pictures Lab.
Keep in mind that such a statistical analysis isn't bullet-proof at all, esp. when there's not all the information available like the date of all marketplace features and more. Please also note not all apps are the same and things that work for Pictures Lab or Helium Voice Free don't necessarily need to be valid for other apps.

Makes sense?
What are your conclusions?

What is your experience?

Thank you!

I would like to thank all the users of the app, the great group of translators who helped me to translate Pictures Lab, the beta testers and not at last all the great Windows Phone sites which keep us informed about all the things happening around Windows Phone and help us developers by informing the world about our apps. Keep it up!

Let me out - Facebook Logout in a Windows Phone App

2012-01-05T21:30:00.000+01:00

A while ago I implemented the Facebook photo endpoint into my Windows Phone Pictures Lab app. The implementation of the login was quite straightforward thanks to OAuth 2.0. Only the logout was way harder than one might expect. This post describes how to logout from Facebook using the Facebook API.

In my Pictures Lab app you can edit photos, make them look awesome and then save or share those with your friends at Twitter or Facebook. The Windows Phone Mango API provides the ShareLinkTask and the ShareStatusTask which can be used by an app to share an URL or text using the social services the user has connected the device to. Unfortunately there's no built-in SharePhotoTask to share a photo using the services the user has already authorized. That's why I had to implement it in a custom way where the user has to authorize again. This blog post by my mate Nick Randolph describes very well how to login to Facebook from a Windows Phone app.

For some situations it might make sense to allow the user to logout from within the app. One might think this can't be hard and in most cases it's pretty easy. Logging out from Twitter is very easy for example. You just have to start the authorization process again. However, logging out from Facebook is way harder since they store a cookie and the WebBrowser control doesn't provide a way to clear the cookies, so just starting the authorization process again doesn't work.
One way to log out from Facebook uses a special Uri that contains a part of the access token which was queried during the app authorization process.

Here's the snippet I use in Pictures Lab to split the access token to get the session key which is then used to build the custom Uri:

public Uri GetLogoutUri(FacebookCredentials credentials)
{
   var sessionkey = ExtractSessionKeyFromAccessToken(credentials.AccessToken);
   var url = String.Format("http://facebook.com/logout.php?app_key={0}&session_key={1}&next={2}", EndpointData.FacebookAppId, sessionkey, EndpointData.FacebookLogoutCallbackUrl);
   return new Uri(url);
}

private static string ExtractSessionKeyFromAccessToken(string accessToken)
{
   if (!String.IsNullOrEmpty(accessToken))
   {
      var parts = accessToken.Split('|');
      if (parts.Length > 2)
      { 
         return parts[1];
      }
   }
   return String.Empty;
}

That logout Uri is then used to navigate the WebBrowser control to it which then correctly triggers the log out process.

Browser.Navigate(FacebookService.GetLogoutUri(EndpointData.Settings.Facebook));

That's it. You just have to know their trick. Hope this helps.

SLARToolkit Samples Updated

2012-01-05T20:25:00.003+01:00

The samples of my open source Windows Phone and Silverlight Augmented Reality Toolkit were updated to the latest version of the WP 7.1 and Silverlight 5 SDKs.
Please note the changed security model in Silverlight 5, which is a big bummer. My Silverlight MVP friend Morten wrote a few true words about it here.
As usual you can find a list of the samples on the project site and also get the code there.

WriteableBitmapEx 1.0 Coming Soon, Test it Now!

2011-12-20T20:37:00.000+01:00

A new version of the WriteableBitmapEx open source library has just been released, but that isn't the last release for this year. No, Andrew Burnett-Thompson and I refactored the library to make it easier portable and we added full WPF support. Andrew did most of the work since he needed the current WriteableBitmapEx library for one of his WPF projects. As a result of the refactoring, WriteableBitmapEx will have maintained support for WPF starting with version 1.0.
I'd have loved to add support for WinRT too, but unfortunately it seems that WinRT only supports streamed reading / writing of the pixel buffer at the moment. I will wait until Microsoft ships the Windows 8 beta early next year and see what they have in there. Many WriteableBitmapEx algorithms need random buffer index access and I don't want to waste my time with massive memory copying now. Who knows what else comes in the beta and it might be better to use a whole different approach for immediate rendering with Windows 8 and WinRT.

All samples were tested with the new version, but due to the massive refactoring more testing is needed. Please test the beta version with your projects and report any bugs you find. You can download the binaries here. Note that this package only contains the WriteableBitmapEx binaries for Silverlight, Windows Phone and WPF. All the samples can be found in the source code repository in the branch "WBX_1.0_BitmapContext". If all goes well this branch will become the trunk in a couple of weeks.

Windows Phone Sites Around the World

2011-12-05T20:52:00.002+01:00

I recently released an update of my Windows Phone app Pictures Lab which brings some nice new features like effect combination, two Christmas and a New Year frame and most importantly 7 new languages. Pictures Lab 4.0 now supports 9 languages: English, German, Japanese, Russian, Dutch, French, Italian, Spanish, Portuguese. This was made possible by a group of awesome native-speaking translators: Alan Mendelevich, Takeshi Miyauchi, David Salazar, Joost van Schaik, Johan Peeters, Paolo Barone, Simone Chiaretta, Pedro Lamas and Quentin Calvez.

I also asked my translators if they know any local Windows Phone news sites / blogs. Below is a list of international and regional Windows Phone news sites we collected. Note, that those are primarily consumer sites read by consumers and not only for developers. This list could be useful if you plan to release a localized version of your app and want to promote it a bit. Most sites have a "Tip Us" or contact form and are mostly happy about new content to write posts about.

Forget this list!
Head over to @ailon's awesome dedicated site: http://windowsphonesites.com

English

German

Russian

Japanese

Dutch

http://wp7.nl

Spanish

Portuguese (Brazilian)

http://windowsphonebrasil.com.br

Italian

French

http://www.monwindowsphone.com

Danish

http://www.windowsphonefan.dk

If you know more sites, please leave a comment and I'll update the list.

Telerik published a nice whitepaper with more resources about app promotion.

WriteableBitmapEx 0.9.8.5 Now Available

2011-10-28T20:36:00.004+02:00

The WriteableBitmapEx open source library has come a long way since I created the CodePlex site in December 2009. A lot of features and the support for new platforms were added in subsequent releases. The package is also available via NuGet since this year.
The new release v0.9.8.5 was just made public. A few new features were added and many small, uncritical issues were fixed (see the list below). The binaries can be downloaded from here or via the NuGet package. Please note that this package only contains the WriteableBitmapEx binaries for Silverlight and Windows Phone. All the samples can be found in the source code repository.

Feature list version 0.9.8.5

Added a Rotate method for arbitrary angles (RotateFree). Provided by montago.
Added Nokola's anti-aliased line drawing implementation.
Updated the Windows Phone project to WP 7.1 Mango.
Added an extension code file for the Windows Phone specific extensions and added SaveToMediaLibrary extensions including support for saveToCameraRoll.
Added an Invert method, which creates an inverted version of the input bitmap. This is useful for WP7 Theme-awareness checks using Application.Current.Resources["PhoneBackgroundBrush"].
Added FromContent method, which provides an easy interface to load a WriteableBitmap from the content of the app.
Added a static overload for the Resize method which takes the pixels array as argument.
Optimized the DrawLine algorithm.
Fixed some issues with DrawRectangle, FillRectangle, DrawEllipse, FillEllipse and DrawPolyline.
Fixed a bug in the bilinear Resize method that appeared when the alpha value is zero.
Fixed other minor issues.

Community Community Community!
Thanks to the community for constantly reporting bugs, suggesting new features and contributing code. That's exactly why open source software is just awesome.

Welcome to Zombieland, the Metro Style Land of WinRT and the Undead

2011-09-17T20:04:00.004+02:00

Silverlight is dead! WPF is dead! .NET is dead! Hey, they didn't talk about SharePoint or SQL Server at the BUILD conference, those must be dead too. Welcome our new Metro-id overlords!
We read such FUD everywhere these days. Actually none of these technologies are dead for the next several years! And you'll probably agree if you know the facts and not the FUD. So here are my thoughts based only on the facts I got from watching BUILD sessions, reading posts that stick to the facts and not listening to people who obviously didn't inform themselves, but nevertheless spread FUD around.

What happened
Microsoft announced the new version of Windows at the BUILD conference and the new Windows 8 runtime called WinRT, which is used to develop Metro style Apps. Metro style Apps are primarily focused on consumers, designed touch first and therefore perfect for modern multi-touch devices like tablets.
The good ol' desktop is still there and the usual Windows applications are now called Desktop Apps.
There's a lot confusion out there at the moment and many think only Metro style Apps are the future. Actually both models can exist side-by-side and I'm sure the mainly used UI will be the classic desktop for the usual business client. Metro is for tablets, maybe later for the phone. Desktop Apps will still work better for the classic business PC scenario in an office.
After trying Windows 8 I don't see myself using Metro style Apps a lot on my PC when I work at the desk. However, I will love to use it when I'm hanging out on my couch with a tablet. The good news is, both models are supported by Windows 8 and therefore can run on one device, including ARM hardware. Awesome!

WinRT

First and foremost, .NET plays a significant role in both development models and is not only used for Desktop Apps. I guess the FUD comes from the architectural picture above, where .NET and Silverlight are only represented by a small box in the lower right corner. Of course Microsoft wants to push the new Metro style Apps, the WinRT model and wants to get web developers into the boat, so they adjusted the graphic and marketing message accordingly.
WinRT is actually a new native COM library, plus some extra infrastructure. Therefore an app developer has not to deal with the COM stuff directly, instead you get a thin / fast projection layer (binding) for each of the supported programming languages. This projection layer is automatically built using WinMD metadata and provides projections for JavaScript, .NET (C# / VB) or C / C++. The UI can be designed with XAML or HTML / CSS in case of JavaScript.
The rendering is completely done using DirectX 11.1 (Direct2D) which results in great performance. In Windows 8 the rendering job is finally fully executed by the right processor. This architecture makes it also possible to implement the complete UI just with DirectX, which will usually be done by games. At the moment it's not possible in the Windows 8 Developer Preview to mix a XAML app with DirectX, which doesn't make much sense since the XAML rendering is done by DirectX. Fortunately all hints that it'll be possible in a later version of Windows 8, maybe in the beta.

The below picture by Doug Seven is a way more accurate picture of the Windows 8 platform architecture.

There are a few good, unbiased posts about WinRT out there, which stick to the facts:
WinRT: An Object Orientated Replacement for Win32
WinRT, the C++ Component Extensions
WinRT demystified
A bad picture is worth a thousand long discussions.
WinRT reference with all namespaces at the unbiased MSDN, where JaveScript is just another language one can use for Metro style Apps.

WinRT and .NET
From what I have seen so far, coding Metro style Apps using C# / .NET seems pretty straightforward if you've done Silverlight, WPF or esp. Windows Phone development before. The BCL used by .NET WinRT is not the full Desktop version of .NET 4.5, it's a reduced set similar to the Silverlight types.
The design of the native WinRT COM library was heavily influenced by .NET. You see it everywhere. Type names, Properties, Events... Even WinMD is the .NET assembly metadata format.
WinRT types map to .NET types and copying of data structures at the boundaries is avoided by the projection layer to get the best performance. Only two types need to be converted using built-in extension methods. The System.IO.Stream can't be mapped to the WinRT stream, so there's an extension method. A Byte[] to WinRT IBuffer conversion is the other extension method.
You can even write your own, custom WinRT components in C# without decorating the classes with ugly COM attributes. Those components are then automatically exposed to the other languages using the generic WinMD metadata. So you can write a component in C++ or in C# and use it from a JavaScript WinRT project. Pretty cool concept if you ask me.
Many WinRT classes look like their origin is .NET / Silverlight and the WinRT WriteableBitmap is also very similar to Silverlight's implementation. So expect a WinRT version of my open source WriteableBitmapEx library in a few weeks when I'm done with the Mango updates for all my Windows Phone apps. After that I will probably port / rewrite some of my WP7 apps for WinRT.

The lifecycle of a WinRT app should be quite familiar for a WP7 developer, since it's similar to the WP 7.5 Mango Fast App Switching. The lifecycle states are: Running > Suspended in RAM > Terminated. Apps get an event for suspending, not for terminate, so the state has to be saved during suspending. Of course an app should only be resurrected from the tombstone state if the app was launched, not when resumed. Apps get 5s for suspending and need to launch within 15s. Unfortunately we don't know the reference machine where those values are measured. Probably the low end configuration.
By the way, the Windows Store will be the better version of the WP7 Marketplace. It will include In-App offers, time limited trials and a very nice dashboard with a lot of analytic capabilities and insights.
In general should a WP7 developer feel most familiar with WinRT. Many concepts made it into WinRT and were improved and actually cleaned up.
My Silverlight MVP friend Morten Nielsen started a blog post series about how to port Silverlight / WPF apps to WinRT.
You should also read this post: WinRT and .NET and watch the ton of great BUILD sessions.

Conclusion

I welcome the new possibilities and great performance we get in Windows 8 with the WinRT. I'm sure Windows 8 will be an awesome tablet OS.
XAML has become a first class citizen and .NET is now installed with Windows 8. Microsoft is also working on a next version of .NET. Version 4.5 of .NET will bring better performance for WPF's ItemsControl for example. Seems not very dead to me. In fact, a Silverlight / WPF / WP7 developer's knowledge of XAML and the Silverlight runtime is more valuable than ever before with WinRT.
Silverlight 5 is still in development and wasn't even released. We don't know if it will be the last version or if Silverlight 6 will see the light of the day. For now I'm quite happy with all the features of Silverlight 5 and we can develop a ton of applications for our customers for the next several years.

One should also not forget how long it will take until Windows 8 is finally out and reached a critical mass. A lot of machines out there still run Windows XP. Windows 7 is way better and I guess the transition to Windows 8 will take even longer. Windows 8 will probably mainly pushed by non-PC multi-touch consumer devices in the near future.

This flowchart by the Telerik guys sums it up pretty nicely.

So get yourself informed and start coding away. Don't give FUD a chance!

Imagine Cup 2011 Worldwide Finals

2011-06-23T22:41:00.004+02:00

The Imagine Cup is one of the world's premier student tech competitions. Students compete in different categories like Software Design, Game Design, Windows Phone 7 and more. The competitions begin each year with local events in over 100 countries. The best of the best will then participate in the worldwide final. The 2011 Imagine Cup Worldwide Finals will be held from 8 - 13 July in New York City, USA.

I'm very honored to take part in the Imagine Cup 2011 Worldwide Finals as a judge in the Windows Phone category. The Windows Phone judges consist of two community members and two Microsoft employees.

People's Choice Award
There's also a People's Choice Award where everyone can vote for their favorite team.

Can you say no if Eva Longoria asks you to vote?

You can find a list of all the Windows Phone finalist teams here. All the teams have built great solutions around the theme for 2011 Imagine a world where technology helps solve the toughest problems.
There's no filter for a category at the People's Choice Award website, that's why I made a list of all the Windows Phone finalists entries:

DREGON by team "Digitron-WP7" from Belgium.
Peekaboo by team "HOMERUN" from Korea.
Hot Potato by team "Zipi Zigi" from Korea.
Recyclocator by team "Code Instincts" from Singapore.
Lifelens by team "The LifeLens Project" from USA.

Please cast your vote.

Cubelicious - Silverlight 5 + Balder + Physics + SLARToolkit Augmented Reality = Triple Win!

2011-06-16T21:45:00.003+02:00

Two months ago I released a new Silverlight 5 sample for my open source Silverlight Augmented Reality Toolkit - SLARToolkit. It uses the new Silverlight 5 hardware accelerated 3D API. You can read more about the new Silverlight 5 XNA 3D API in this blog post.
This post here provides a new demo for SLARToolkit which uses the open source 3D engine Balder by my friend Einar Ingebrigsten. This demo also leverages the open source physics engine JigLibX my buddy Andy Beaulieu ported over to Silverlight. You can try the live sample if you have the Silverlight 5 beta installed or watch a video instead.

I actually spent most of the time on this project a couple of weeks ago. My good Silverlight MVP friend Einar Ingebrigtsen used it in his talk at The Gathering 2011. Now I finally had a bit time to finish the demo, make a video and this blog post.

The SLARToolkit project description from the CodePlex site:

SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. SLARTookit is based on the established NyARToolkit and ARToolkit.

Live

A webcam and at least the Silverlight 5 beta runtime must be installed to run the sample. It's available here. Alternatively there is also a video of the new sample embedded below.

If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. As an alternative it's also possible to open a marker file on a mobile device and to use the device's screen as marker. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Open the sample in a new page.

Simply press the "Start Cam" Button to start the webcam. The properties of the particle system can be changed with Sliders. The "Flip x-axis" Checkbox can be used to flip the video (the webcam output is mirror-reversed by default).

If you click the "Start Cam" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

Video

I've recorded a short video of the new sample with Expression Encoder's Screen Capture feature. Please keep in mind that the screen recording software eats up a lot of resources while recording and that the actual frame rate is even better. The video is also available at YouTube.

Background music is Lullaby by _ghost

This demo shows how the new Silverlight 5 3D API, the Balder engine and the JigLibX physics library can be used to augment the reality with the help of SLARToolkit.

How it works

This sample uses the webcam video stream which fills a Rectangle shape, the video stream is also constantly captured and fed to the SLARToolkit BitmapMarkerDetector to detect the markers. The detection result contains a transformation matrix for each found marker which is then used to apply a global transformation to the cubes and the plane.
I implemented a particle system with a flexible directed emitter which can be controlled through various properties. The particle system is quite generic and can be used for all kinds of particles (3D objects). The particle collision detection and resolving is based on rigid body physics that was implemented with the help of the JigLibX library my Silverlight MVP buddy Andy Beaulieu ported over to Silverlight.
The rendering and the model loading is done by the 3D engine Balder. It's a fantastic open source engine by Einar Ingebrigtsen. You just need to write a couple lines of XAML and you're good to go. This sample only uses a simple cube model, but Balder has built-in model loaders to load complex 3D models and Einar provides a big sample library. He also brought the engine to a few more platforms like Windows Phone 7, OpenGL and has even a neat software rendering fallback. Read his blog post here.
As part of this sample I needed some vector and quaternion methods which were missing in Balder. I contributed those and the generic particle system to the Balder project. Feel free to use the particle system and the other methods in your Balder projects.

Download it, build your app and augment your reality

The open source SLARToolkit library and all samples are hosted at CodePlex. If you have any comments, questions or suggestions don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun with the library and please keep me updated if you use it anywhere so I can put a link on the project site.

Nodo vs Mango - Windows Phone ListBox Performance Improvements in Action

2011-05-27T01:05:00.002+02:00

There were quite a few complaints about the Windows Phone's ListBox scrolling performance in the past. The Windows Phone team obviously heard it and heavily worked on the performance and responsiveness of the whole platform and the ListBox in particular. The touch input processing was loaded off from the UI thread to a new separate thread. Additionally the BitmapImage doesn't load the data on the UI thread anymore. I'm sure lots of other tweaks were implemented to increase the performance and the responsiveness of the platform. I think the Windows Phone team did a very good job!

Showdown

The video below shows a side-by-side comparison of a Nodo device with build 7392 and a prototype device with a Mango prebuild. The Nodo device on the left side is a Samsung Omnia 7, the Mango device on the right side is the ASUS prototype.

For comparison I use the official Twitter app build against 7.0 and in the second half of the video I show the effects preview list of my Pictures Lab app. This effects pivot item only uses a ListBox with a DataTemplate that contains an Image control with a fixed size and a TextBox for each item. So there's no background image loading being performed or any other heavy computing, just a static list with a lot of redrawing.

Background music is Taiga by SMILETRON

As you can see, it's quite a huge difference and even this Mango prebuild runs very smooth on this rather old ASUS hardware. There's still some room left for more improvement, like the rasterizer, but imagine the boost on production devices with the final Mango version.

MANGO, MANGO, MANGO!

I don't have a deal with Microsoft or are paid to blog or tweet this. I'm just exited about all the goodies that are coming with Mango. Good times.

Unity 3D and Silverlight in Harmony with SLARToolkit's Augmented Reality

2011-05-25T21:01:00.002+02:00

If you read this blog regularly, you might have noticed a lot of SLARToolkit posts recently. But I'm not the only one using the open source Silverlight Augmented Reality Toolkit. There's great stuff happening out there and I see quite a few projects, especially in the academic field. You can find a full list of projects (I know of) on the SLARToolkit project site at CodePlex.
I love when people use my open source work, adapt or learn from it. This is what open source is all about for me.

One project recently caught my attention in particular. Ioulian Alexeev made a very amazing action game using the Unity 3D game engine and he combined it with SLARToolkit to control the spaceship with a marker. Ioulian studies Multimedia at the Arteveldehogeschool in Belgium and the game "StarFighter" was his bachelors test.

He recorded this nice video.

Note how he compares the performance of the Flash Augmented Reality Tookit and SLARToolkit:

Performance FLARManager: Not good enough for action games
Performance SLARToolkit: Very fast and accurate marker tracking

Both toolkits use the same marker-based AR algorithms from the ARToolkit.

Live
You can play the game here. If you want to try it yourself you need do download the SLAR and / or L marker, print them and point the camera toward these. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. As an alternative it's also possible to open a marker file on a different device and to use the device's screen as marker.
See the SLARToolkit Markers documentation for more details.

Why is a Y in the Windows Phone Mango Camera API

2011-05-25T20:31:00.003+02:00

The release of the new Mango tools brings Windows Phone development on par with Silverlight 4 and will therefore add many great features to the Windows Phone platform. This means it will also contain the Webcam CaptureSource and VideoSink API from Silverlight 4. Additionally it also introduces the new FileSink class which can be used to record the video stream as MP4 to the Isolated Storage. Most important a new PhotoCamera class with a lot of functionality is present in Mango.
This class is used in the latest SLARToolkit Windows Phone sample and in some other new projects I'm working on.

Don't Reinvent the Wheel
The Silverlight 4 webcam API was explained in this detailed blog post almost a year ago. The techniques and concepts I described there can now also be used with Windows Phone Mango.
The updated MSDN documentation has quite a few articles and samples about the new camera API. My MVP buddy Alex Golesh also has a nice write up about the new Camera API.
This blog post tries to fill the gaps and provides some information especially about the PhotoCamera's YCrCb capture methods.

YCbCr vs ARGB
In the well-known RGB color space the red, green and blue information is stored in separate components which also contains the redundant luminance data for each channel. In the YCbCr color space (or YCrCb) the luminance information is stored in the Y component and the chroma (color) information in the Cb component as blue-difference and in Cr component as red-difference. The RGB-YCbCr conversion can be done with simple addition and multiplication operations. The Y component usually ranges from 0 to 1, Cb and Cr from -0.5 to 0.5.

Y = 0.5, Cr [-0.5, 0.5], Cb[-0.5, 0.5]

Humans are more sensitive to luminance information than to chroma, therefore the resolution of the color information can be reduced and only the luminance needs to be stored in full resolution. Many digital camera sensors use the YCrCb color space and make use of this reduced chroma information.

PhotoCamera
The PhotoCamera class has a lot of useful methods to either capture a full resolution image from the camera or to get a smaller (and faster) preview buffer snapshot. The GetPreviewBufferY and GetPreviewBufferYCrCb methods provide the direct data from the camera without a transformation to 32 bit ARGB. Not only is the alpha channel left out in the YCrCb buffer, also the Cr and Cb color components are stored with reduced resolution. This keeps the buffer size smaller and is way faster, but also makes it a bit trickier when the color components (and brightness) need to be extracted from the byte buffer. Fortunately there's the YCbCrPixelLayout property which contains alls the offsets, strides and other needed information.

Conclusion
The GetPreviewBufferYCrCb method is approximately 4 times faster than the GetPreviewBufferArgb32 method and also takes a smaller buffer, therefore the YCrCb methods are the way to go when only the luminance data is needed or the YCbCr color space can be used for the given scenario. For example many computer vision techniques only need the luminance information for processing.
I like that both color spaces are supported by the API. On mobile devices you need all the performance you can get. I actually helped the Windows Phone camera team with quite a bit feedback to decide about this API design. Very smart people by the way.

Augmented Mango - SLARToolkit for Windows Phone

2011-05-24T22:18:00.005+02:00

The beta of the new Windows Phone Developer Tools was just publicly released. The update with the codename "Mango" comes with many new APIs and will finally contain an API for real-time camera access what a lot of developers have been asking for. The new runtime gives us the needed functionality to implement many cool scenarios. One of these scenarios is Augmented Reality, which leads to my open source Silverlight Augmented Reality Toolkit (SLARToolkit).
This post announces the new Windows Phone version of SLARToolkit and also provides a sample. If you're one of those lucky people with a Mango-enabled device you can download the XAP here or just watch a video instead.

The SLARToolkit project description from the CodePlex site:

SLARToolkit is a flexible marker-based Augmented Reality library for Silverlight and Windows Phone with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource, WriteableBitmap or with the Windows Phone's PhotoCamera. SLARTookit is based on the established NyARToolkit and ARToolkit.

Demo

The sample XAP can be deployed to a Mango-enabled device (tested with build 7629). Alternatively there's also a video of the new sample embedded below.

If you want to try it yourself you need do download the SLAR and / or L marker, print them and point the camera toward these. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. As an alternative it's also possible to open a marker file on a different device and to use the device's screen as marker.
See the SLARToolkit Markers documentation for more details.

Video

I've recorded a short video of the new sample with my Samsung Omnia 7. It's a bit blurry, but it demonstrates how well the sample works even on this quite old ASUS prototype, which's camera pipeline seems a bit slow.
The video is also available at YouTube.

Background music is Melo by Mosaik

This demo shows how the new Windows Phone Mango real-time camera API can be used to augment the reality with the help of the SLARToolkit. This can be nice for educational projects and it's actually no problem to add correctly transformed videos or other content to the demo.
The demo demonstrates just some basic UIElements like a TextBox and an Image control. Mango will also enable the combination of Silverlight and XNA, which means that nice 3D AR games can be developed with the help of the SLARToolkit.

How it works

This sample uses the new PhotoCamera and a timer to constantly get a snapshot of the real-time camera stream. This snapshot is then passed to the SLARToolkit algorithms to get the 3D spatial information of the marker. The computed detection results are used to transform the elements perspectively correct.

protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
   base.OnNavigatedTo(e);

   // Initialize the webcam
   photoCamera = new PhotoCamera();
   photoCamera.Initialized += PhotoCameraInitialized;
   isInitialized = false;

   // Fill the Viewport Rectangle with the VideoBrush
   var vidBrush = new VideoBrush();
   vidBrush.SetSource(photoCamera);
   Viewport.Fill = vidBrush;

   // Start timer
   dispatcherTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(50) };
   dispatcherTimer.Tick += (sender, e1) => Detect();
   dispatcherTimer.Start();
}

The PhotoCamera instance is set up in the OnNavigatedTo event handler of the page and the DispatcherTimer is started. The timer will constantly call the Detect method every 50 milliseconds. Additionally a viewfinder Rectangle is filled with a VideoBrush which in turn has the photoCamera video stream set as source.

void Detect()
{
   if (!isInitialized)
   {
      return;
   }

   // Update buffer size
   var pixelWidth = photoCamera.PreviewBufferResolution.Width;
   var pixelHeight = photoCamera.PreviewBufferResolution.Height;
   if (buffer == null || buffer.Length != pixelWidth * pixelHeight)
   {
      buffer = new byte[pixelWidth * pixelHeight];
   }

   // Grab snapshot
   photoCamera.GetPreviewBufferY(buffer);

   // Detect
   var dr = arDetector.DetectAllMarkers(buffer, pixelWidth, pixelHeight);

   // Calculate the projection matrix
   if (dr.HasResults)
   {
      // Center at origin of the 256x256 controls
      var centerAtOrigin = Matrix3DFactory.CreateTranslation(-128, -128, 0);
            
      // Swap the y-axis and scale down by half
      var scale = Matrix3DFactory.CreateScale(0.5, -0.5, 0.5);

      // Calculate the complete transformation matrix based on the first detection result
      var world = centerAtOrigin * scale * dr[0].Transformation;

      // Viewport transformation
      var viewport = Matrix3DFactory.CreateViewportTransformation(pixelWidth, pixelHeight);

      // Calculate the final transformation matrix by using the camera projection matrix 
      var m = Matrix3DFactory.CreateViewportProjection(world, Matrix3D.Identity, arDetector.Projection, viewport);

      // Apply the final transformation matrix to the controls
      var matrix3DProjection = new Matrix3DProjection { ProjectionMatrix = m };
      Txt.Projection = matrix3DProjection;
      Img.Projection = matrix3DProjection;
   }
}

A snapshot of the current preview buffer is taken in the Detect method using the GetPreviewBufferY method. This method fills up a byte buffer with the luminance data of the current viewfinder frame. This buffer is then passed to the SLARToolkit's MarkerDetector Detect method, which returns the detected marker information. This transformation data is then used to transform the UIElement perspectively correct in 3D.
Read more about the PhotoCamera's YCbCr methods in this blog post.

void PhotoCameraInitialized(object sender, CameraOperationCompletedEventArgs e)
{
   //  Initialize the Detector
   arDetector = new GrayBufferMarkerDetector();

   // Load the marker pattern. It has 16x16 segments and a width of 80 millimeters
   var marker = Marker.LoadFromResource("data/Marker_SLAR_16x16segments_80width.pat", 16, 16, 80);

   // The perspective projection has the near plane at 1 and the far plane at 4000
   arDetector.Initialize(photoCamera.PreviewBufferResolution.Width, photoCamera.PreviewBufferResolution.Height, 1, 4000, marker);

   isInitialized = true;
}

The SLARToolkit's GrayBufferMarkerDetector is created and set up in the PhotoCamera's Initialized event handler. The brand new GrayBufferMarkerDetector uses the byte buffer with luminance data directly without the need of an ARGB 32 bit pixel conversion.

Checkout the source code at CodePlex if you want to see all the details of the sample which were left out for clarity.

Download it, build your app and augment your reality

Have fun with the library and please keep me updated if you use it anywhere so I can put a link on the project site.

WriteableBitmapEx goes NuGet

2011-04-19T00:03:00.002+02:00

The WriteableBitmapEx library is now available as a NuGet package. The package contains the Silverlight and the Windows Phone binaries. The sources and the PDB are also available at SymbolSource.org.
NuGet is a neat package management system for the .Net platform which makes the life of a .Net developer much easier. If you haven't tried NuGet until now, you should definitely give it a try. I'm sure you won't regret it. To install NuGet, open Visual Studio's Tools -> Extension Manager and search the Online Gallery for NuGet. You can then open a project, right click the References and select "Add Library Package Reference".

Silverlight 5 3D + SLARToolkit Augmented Reality = Win!

2011-04-13T21:29:00.000+02:00

It's almost been a year since I wrote a sample for my open source Silverlight Augmented Reality Toolkit - SLARToolkit. The release of the new Silverlight 5 hardware accelerated 3D API was a nice occasion to finally make a new one.
In my last blog post I wrote a summary of all the Silverlight 5 beta features and some notes about the new low-level, XNA 3D API.
This post provides the new demo for SLARToolkit which leverages this fast GPU-based rendering to draw some nice effects with 60 frames per second. You can try the live sample if you have the Silverlight 5 beta installed or watch a video instead.

The SLARToolkit project description from the CodePlex site:

SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. SLARTookit is based on the established NyARToolkit and ARToolkit.

Live

A webcam and at least the Silverlight 5 beta runtime must be installed to run the sample. It's available here. Alternatively there is also a video of the new sample embedded below.

Open the sample in a new page.

Simply press the "Start Fun" Button to start the webcam. The size of the objects can be changed with the "Scale" Sliders. The "Flip x-axis" Checkbox could be used to flip the video (the webcam output is mirror-reversed by default). Click the "Glass" Checkbox to apply a glass effect to the sun model.

If you click the "Start Fun" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

Video

Background music is Neon Aurora by Alpha C

This demo shows how the new Silverlight 5 3D API can be used to augment the reality with the help of SLARToolkit. It also demonstrates how the 3D DrawingSurface can be combined with the webcam video stream and overlaid with ordinary TextBoxes. This can be nice for educational projects and it's actually no problem to add correctly transformed videos or other content to the demo.

How it works

This sample uses the webcam video stream which is used to fill a Rectangle shape, the video stream is also constantly captured and fed to the SLARToolkit BitmapMarkerDetector to detect the markers. The detection result contains a transformation matrix for each found marker which is then used to transform the 3D objects and the TextBoxes.
A couple of vertex and pixel shaders are used to get the desired effects. Techniques like Phong shading, Bump mapping and Refraction mapping (glass) were implemented. The snapshots from the webcam are passed as a texture to the refraction pixel shader to simulate the glass effect.
See the source code if you're interested in the nitty-gritty details. Please note that I wrote an introduction to Pixel Shaders for Coding4Fun a while ago.
The current Silverlight 5 beta doesn't implement RenderTargets, therefore effects like bloom aren't really possible with the limitations of the Shader Model 2. You can add a faded billboard around the sun, but it wouldn't look that nice and when RenderTargets are added we can do way more nice effects in future releases of Silverlight 5.
I also added a new simple anti-jittering functionality to SLARToolkit to prevent the jiggling that mostly occurred due to the varying lightning conditions and noise in the video stream.

Credits
The SolarWind sample by the Silverlight team was used as a base and extended. The sample uses earth textures from the NASA. The pixel shader for the sun uses textures and concepts from an article by Nicolas Menzel. The moon textures are from the Celestia project.

Download it, build your app and augment your reality

XNA for the Web - Silverlight 5

2011-04-13T19:49:00.004+02:00

Many great new things were announced during the second keynote at Microsoft's MIX11 conference. One of it was the Silverlight 5 beta which is available for download here.
The new features are just awesome and many of them were requested by the community. As usual Tim Heuer's great blog post covers all the new goodies in detail and provides videos and source code for most of them. Kudos to the Silverlight team for releasing another great version.
My blog post contains a summary of all the new features the Silverlight 5 beta ships and provides some notes about the new 3D API. The next blog post will bring a new SLARToolkit 3D demo.

Features

XNA 3D Graphics API
Improved Graphics stack – The graphics stack has been re-architected to bring over improvements from WP7, such as Independent Animations.
Hardware video decode for H.264 playback.
Multi-core background JIT support for improved startup performance.
Realtime Sound (low-latency Audio)
Variable Speed Playback (“Trick Play”)
XAML Binding Debugging
Multiple Window Support
Ancestor RelativeSource Binding
Implicit DataTemplates
ClickCount
Binding on Style Setter
Linked Text Containers
Custom Markup Extensions
ComboBox type ahead with text searching.
Full keyboard support in full-screen for trusted in-browser applications, enables richer kiosk and media viewing applications in-browser.
Default filename in SaveFileDialog – Specify a default filename when you launch the SaveFileDialog.
Unrestricted filesystem access – trusted applications can Read write to files in any directory on the filesystem.
More performance optimizations:
XAML Parser performance optimizations.
Network Latency optimizations.
Text layout performance improvements.
Hardware acceleration is enabled in windowless mode with Internet Explorer 9.

Find more details at the Silverlight site and read John Papa's and Pete Brown's blog posts.

XNA for the Web
One of the topmost requested features at the Silverlight Uservoice site was hardware accelerated 3D graphics. The Silverlight team listened closely and added an immediate mode, low-level, XNA 3D API to Silverlight 5. An immediate mode API offers functionality to render graphics directly to the graphics interface without keeping a list of the objects. A retained mode API like WPF 3D on the other hand keeps an internal list of the object graph. An immediate mode API is a great choice since it provides the maximum control over the rendering and mostly results in superior performance compared to retained mode rendering.
The Silverlight 5 3D API doesn't contain all the higher-level classes one might know from the XNA Game Studio. The Content Pipeline, the SpriteBatch class and others were left out. I assume this decision was made to keep the footprint of the Silverlight runtime small. However, the team added all the needed core functionality; features like the SpriteBatch can be build on top of it by the community. My Silverlight MVP friend Bill Reiss is already working on a SpriteBatch version for Silverlight 5.
The new GPU accelerated 3D API gives us Silverlight developers the core XNA functionality with shaders and vertex rendering inside the browser! Shaders aren't easy for most non-graphics programmers, but they give us a huge flexibility and we can build various higher levels on top of the new 3D API.
You can render some pretty impressive things with the new Silverlight 5 3D API and it's possible to draw millions of triangles on the GPU without almost no CPU load.

Here's a sneak peek at the new SLARToolkit demo which also contains a simple real-time glass effect.

Silverlight Integration
The 3D API contains a control called the DrawingSurface, which is basically a rendering surface or a 3D Canvas if you will. The DrawingSurface has the Draw event which is raised for every frame to draw. The event is fired on a render thread which runs in parallel to the UI thread. This is an important concept introduced with Silverlight 5.
Inside a Draw event handler the states and shaders are set and the vertex and index buffers are passed to the GPU for rendering. The vertex and pixel shaders implement the lightning model and define how the vertices are rendered. The current Silverlight 5 3D beta only supports Shader Model 2, but this will probably change in a future release, but even with Shader Model 2 many neat lightning models and effects can be implemented. You will find some nice effects in the new SLARToolkit demo.

Check out the Silverlight 5 3D samples if you want to learn more about the new 3D API. If you're not familiar with 3D graphics concepts or just want to make your life easier, then I strongly recommend to give the 3D engine Balder a try. It's a fantastic open source engine by my good Silverlight MVP friend Einar Ingebrigtsen. You just need to write a couple lines of XAML and you're good to go. Balder has built-in model loaders to load complex 3D models and Einar provides a big sample library. He also brought the engine to a few more platforms like Windows Phone 7, OpenGL and has even a neat software rendering fallback. Read his blog post here.
Another Silverlight MVP friend of mine, Andy Beaulieu made a very cool demo using Balder and a 3D physics engine. He blogged about it and even released the 3D physics engine as open source!

The beta of the 3D API is missing some essential things like RenderTargets at the moment, but it's just a beta and more stuff might be announced at the MIX11 conference. If you would like to know more details about the new Silverlight 5 3D API and the roadmap, then you should definitely check out the MIX11 session Graphics & 3D with Silverlight 5 by the Silverlight program manager Aaron Oneal on Wednesday, April 13 3:30 pm - 4:30 pm PDT. All MIX sessions are recorded and will be available online 24 hours after each session.

Conclusion
No, Silverlight is not dead and it's clear that Microsoft is investing in it. I'm sure the new features and esp. the XNA 3D API will lead to some great applications in the future. The future is bright and we are part of it.

Long Time No See - WriteableBitmapEx 0.9.7.0

2011-02-15T21:32:00.001+01:00

I actually wanted to release the feature complete version 1.0 of the WriteableBitmapEx library after the previous release, but I've been very busy with other projects during the last year. That's why I decided to release version 0.9.7.0 today. It contains many fixes, but also adds some features (see below). This version finally provides an official Windows Phone binary build, although the Windows Phone project was already available shortly after the first CTP was released. The zipped binaries can be downloaded from here. Please note that this package only contains the WriteableBitmapEx binaries for Silverlight and Windows Phone. All the samples can be found in the source code repository.

WPF
There's now also an unmaintained WPF branch of the WriteableBitmapEx library in the source code repository. It was contributed by Szymon Kobalczyk.
One might ask why it's not maintained. I started a poll and it seems that only 7 out of 16 need a WPF version of WriteableBitmapEx. The code is also quite bloated due to conditional compilation flags, which makes it harder to maintain in the future. That's why I decided not to integrate the WPF version. I just don't have enough time to maintain a hardly used version. I would rather point WPF users to Jeremiah Morrill's new project called DirectCanvas. It's in an early stage, but he's a great guy and working hard on it. I'm sure we'll see a great, GPU accelerated 2D drawing library for WPF in the future.

Feature list version 0.9.7.0

Fixed many bugs.
Added the Rotate method which rotates the bitmap in 90° steps clockwise and returns a new rotated WriteableBitmap.
Added a Flip method with support for FlipMode.Vertical and FlipMode.Horizontal.
Added a new Filter extension file with a convolution method and some kernel templates (Gaussian, Sharpen).
Added the GetBrightness method, which returns the brightness / luminance of the pixel at the x, y coordinate as byte.
Added the ColorKeying BlendMode.
Added boundary checks to the Crop() function to avoid OutOfRangeExceptions if the passed parameters are outside the boundaries of the original bitmap.
Optimized the DrawLine algorithm.
Optimized the Resize algorithms (NearestNeighbor is now 10x faster).
Optimized the Clear(Color) method.

Community FTW!
Thanks to the community for constantly reporting bugs and suggesting new features. You rock! That's why I love open source software.

Kostenloser Windows Phone Artikel in der DotNetPro

2011-01-24T20:24:00.001+01:00

I've written a Windows Phone development article for Germany's largest .Net developer magazine. It's written in German.

Für die aktuelle DotNetPro habe ich einen Artikel zu der Entwicklung für Windows Phone 7 geschrieben. Anhand eines praktischen Beispiels werden die einzelnen APIs und Tricks gezeigt um eine Windows Phone App für die einfache Bildmanipulation zu entwickeln.
Als Werbung für das aktuelle Magazin wird der Artikel auch kostenlos online zur Verfügung gestellt. In der Druckausgabe findet man neben meinem Artikel auch andere Windows Phone Artikel. Wer noch tiefer in die Windows Phone Entwicklung einsteigen möchte, dem sei auch das Buch von Patrick Getzmann, Simon Hackfort, und Peter Nowak empfohlen: Entwickeln für Windows Phone 7: Architektur, Frameworks, APIs.

Stay in Touch - How to Contact Your Marketplace Customers

2011-01-17T23:10:00.002+01:00

Photo (CC) by LarimdaME

A while ago I provided some methods and code snippets on how to get feedback from users. Today I show a manual procedure how you can contact some of the customers that reviewed your Windows Phone app. Some might already know this, but not all developers are aware of this Marketplace / Zune functionality, so I though it might be worth sharing.

Most customers are nice people that provide valuable feedback also through the Marketplace review mechanism. Others give you bad reviews and it seems like they might have misunderstood the functionality. And some people just love to rant. Just like in real life. If I feel the urge to contact some of the reviewers, I send a message through the Zune social network and ask for details. Unfortunately, not all review authors are registered there and only ~50% of the people can be found by their username.

Step by Step
1. Go to http://wp7reviews.tomverhoeff.com and enter the Product Id of your app.
This neat little Silverlight app will list all reviews from around the world. I recommend you bookmark the URL with the Product Id.
2. Sort the grid by date, go through the list and copy the username of the reviewer you want to contact.

3. Open the Zune software, go to the Marketplace tab, paste the username into the search field and hit the search button.

4. If the user was found, click on the user's profile.

5. Open the user's profile and send the user a message through Zune ("Send message").

Conclusion
That's it. Not the best solution and you will only reach ~50% of the users, but better than nothing. I would prefer if app developers could comment the reviews.
Please stay professional and don't offend the review authors even if they have written nonsense.

Goodbye 2010 - Hello 2011

2010-12-30T20:51:00.006+01:00

This was an amazing year! Many great things happened and a lot of stuff was released by me. The most important release this year was a personal one. Our third daughter was born in September. Healthy and just plain awesome. She is developing very good and as amazing as her older sisters (photo).
This blog post is just a recap of all the things that happened and I've done in 2010 and some that might happen in 2011.

Community
The whole Silverlight and Windows Phone community is great and it's endless fun to be a part of it. I met a lot of great people online and offline. It's a pleasure to discuss and develop good things in this environment. Thanks a ton!
And don't forget, Silverlight is not dead! Silverlight 5 will be released in 2011 and it will come with a huge bag of new features.
Stay in the 'Light! - like my Silverlight MVP friend Dave Campbell would say.

MVP Award
In April 2010 I was awarded as Silverlight MVP. It is a real honor to be a Silverlight MVP. All the Silverlight MVPs I know are truly outstanding and I can hardly believe I'm considered to be one of them.

Blogging
Yep, I did some blogging this year. Most of it covered Silverlight development and lately also Windows Phone topics. I always try to provide unique content that hasn't been covered anywhere else. I hope you like it and find it useful. Your feedback is always welcome!

Open Source Projects

The one big think this year was the release of the Silverlight Augmented Reality Framework: SLARToolkit. SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. It's open source and hosted at CodePlex.
The development of the library itself took a lot of time, but also the samples (1, 2), the documentation, the support at the forum and via email. It's good to see that some projects use SLARToolkit and the downloads aren't that bad. I'm especially pleased to see that most of the projects are from the academic field. Some projects are listed on the CodePlex site.
I have a lot of things on my idea / todo list I want to add. I hope I'll find a bit time during the next year to implement these.

One part of the SLARToolkit library was extracted and released as separate project. The Matrix3DEx library is a collection of extension and factory methods for Silverlight's Matrix3D struct. The Matrix3DEx library tries to compensate the minimalistic Matrix3D struct with extension and factory methods for common transformation matrices that are easy to use like built in methods. The CodePlex site has all the details and samples.

The WriteableBitmapEx library is another open source project. I actually released it in 2009, but I added a lot of new features during this year and the library got quite well adopted in the community. One reason might be the availability of a Windows Phone version.
The WriteableBitmapEx library is a collection of extension methods for Silverlight's WriteableBitmap. The WriteableBitmap class that was added in Silverlight 3 and which is also available on Windows Phone, allows the direct manipulation of a bitmap and could be used to generate fast procedural images by drawing directly to a bitmap. The WriteableBitmapEx library tries to compensate the minimalistic WriteableBitmap class with extensions methods that are easy to use like built in methods and offer GDI+ like functionality.
There are many different samples available at the CodePlex repository and I also spent a good amount of time with the support at the forum and via email.
The library is still not feature complete for version 1.0 and I will definitely continue my work on it.

Windows Phone Apps
Like many other Silverlight developers I got very exited when Silverlight was announced as the development platform for Windows Phone 7 apps. I started some experiments right away when the first CTP was available at MIX 2010. I also provided the WriteableBitmapEx Windows Phone version a couple of days later.

Every Windows Phone 7 device is required to have at least a 5 megapixel camera with a flashlight. There are also other features which make Windows Phones amazing devices for taking pictures and dealing with pictures. I've been working heavily during the last months to enhance this photo experience with my kind of skills. My Pictures Lab app was the first picture effects application for Windows Phone with unique, high quality effects. If you like to take photos then this app is a perfect addition to your phone’s toolset. Or like msnbc.com wrote: "The app, a Swiss Army knife of photo tweaks". It's the original and the best selling photo app already since the Windows Phone launch. Pictures Lab comes with more than 20 controllable and easy-to-use advanced effects, a dynamic effect preview, crop, rotate and many more features. I just recently added a Twitter sharing function and the Happy New Year effect / frame (see photo above).
The app got very good reviews from various webites, blogs and offline magazines. The well-known site Engadget lists it under The best apps, accessories, and tips. They write: "... a must-have for WP7 devices ... the program provides a set of amazing effects and tweaks for your photos ...".
I will constantly update the app and add more features and effects in 2011.

I also developed a little fun app called Helium Voice. You can record your voice with the app and change the pitch while sliding over a balloon graphic (inhale or exhale helium).

I have a very long list with future app ideas and it's constantly growing. On some days I have the feeling my head explodes with new ideas.
I hope to find some time next year to push some more apps out. Silverlight is fortunately one of the most productive platforms out there.

Articles
I also wrote some articles for Microsoft's Coding4Fun site and a magazine.

It started with the FaceLight article about a simple facial recognition system using Silverlight 4’s webcam. The code for this article ended up on CodePlex, where you can also find a live sample.

The second article explained how to write pixel shaders for the Microsoft Silverlight and WPF platform with HLSL, as well as how to write an extensible Silverlight application with the help of MEF.
The code is also available at CodePlex.

My last two articles for Coding4Fun showed how to write a simple photo effects application for Windows Phone (1, 2). This is actually where Pictures Lab has its origin.

I just recently wrote an article for Germany's largest .Net developer magazine, dotnetpro. The article covers Windows Phone development and is available in the February 2011 issue and also online.

I also reviewed the media / computer graphics chapters of the two best Silverlight books available. The one is Silverlight 4 Unleashed by Laurent Bugnion and the other is Silverlight 4 in Action by Pete Brown. Both are great books and every Silverlight developer should get a copy.

Talks and the Rest
I gave a talk at the .Net Usergroup Dresden about Silverlight 4's media capabilities and presented SLARToolkit. In May I had the pleasure to be interviewed by Carl and Richard from the .Net Rocks talk show. And I worked on some very interesting projects I can't write something about yet. I also spent a good amount of time answering Silverlight questions I got via email, at the official forum or at Stack Overflow.

It was a really busy year and I hope I can keep this high productivity up.

By the way, I'm also a frequent social network user and met great new people there (Twitter, Facebook). It's a pleasure to exchange thoughts and share knowledge with smart people. Thanks for being awesome!

I wish you and your families a happy and successful new year. 2011 will be good!

Sending Windows Phone Screenshots in an Email

2010-12-29T20:25:00.001+01:00

My Silverlight MVP friend Laurent Bugnion blogged about how to take a screenshot from within a Silverlight Windows Phone application. As he wrote this could be helpful for customer feedback and other purposes. To accomplish this, the Silverlight UIElement is rendered to a WriteableBitmap, which can then be saved to the device's Media Library (Pictures Hub). Afterwards it could be send in an email from the Pictures Hub or the email client.
Wouldn't it be even nicer if the user could send an email with the screenshot directly from the app? I show you how.

The Email Task
The only way to send an email from a Windows Phone app is through the EmailComposeTask. This task provides properties for the Subject, Body, CC, Addressee (To) and launches the email client. In the last blog post I showed how I use it to send customer feedback with system information from my apps.
Unfortunately the EmailComposeTask doesn't provide a way to pass an attachment and therefore it's not possible to attach the screenshot to the email. Fortunately there's a way to send binary data as text inside an email. This is where Base64 encoding comes into play.

Embedding the Screenshot
Base64 encoding transforms binary data into ASCII text and thereby makes it possible to send binary data as plain old text. This way we can convert the JPEG screenshot to Base64 text and set it as the Body of the EmailComposeTask. The Body text length is limited and I experimented a bit with different sizes to find the limitation. It seems like the Body text is limited to less than 32k Unicode characters. A JPEG encoded screenshot of a whole Windows Phone page is too large for this (480 x 800 pixels). That's why the screenshot is rendered downscaled if the element is too large and a lower compression quality is used for the JPEG encoding.

private static void SendEmailScreenshot(FrameworkElement element)
{
   // Render the element at the maximum possible size
   ScaleTransform transform = null;
   if (element.ActualWidth * element.ActualHeight > 240 * 400)
   {
      // Calculate a uniform scale with the minimum possible size
      var scaleX = 240.0 / element.ActualWidth;
      var scaleY = 400.0 / element.ActualHeight;
      var scale = scaleX < scaleY ? scaleX : scaleY;
      transform = new ScaleTransform { ScaleX = scale, ScaleY = scale };
   }
   var wb = new WriteableBitmap(element, transform);

   using (var memoryStream = new MemoryStream())
   {
      // Encode the screenshot as JPEG with a quality of 60%
      wb.SaveJpeg(memoryStream, wb.PixelWidth, wb.PixelHeight, 0, 60);
      memoryStream.Seek(0, SeekOrigin.Begin);

      // Convert binary data to Base64 string
      var bytes = memoryStream.ToArray();
      var base64String = Convert.ToBase64String(bytes);

      // Invoke email task
      var emailComposeTask = new EmailComposeTask
                              {
                                 Subject = "Screenshot from my app",
                                 Body = base64String,
                                 To = "foo@bar.com",
                              };
      emailComposeTask.Show();
   }
}

The element is downscaled uniformly if it's larger than a certain size. This is done with a ScaleTransform directly when the Silverlight (vector) element is rendered to a WriteableBitmap. Afterwards the bitmap is encoded as JPEG at a quality of 60%. This low compression quality reduces the size of the JPEG image significantly, but the image is still usable. The binary data is then transformed to Base64 text with the Convert.ToBase64String method. Finally the EmailComposeTask is created, the properties are set and the email client is opened.

Decoding the Screenshot
The email will contain the Base64 encoded image. Base64 encoded data typically looks like this:

/9j/4AAQSkZJRgABAQEBBgEGAAD/2wBDAA0JCgsKCA0LCgsODg0PEyAVExISEyccHhcgLikxMC4pLSwzOko
+MzZGNywtQFdBRkxOUlNSMj5aYVpQYEpRUk//2wBDAQ4ODhMREyYVFSZPNS01T09PT09PT09PT09PT09PT09PT09PT
09PT09PT09PT09PT09PT09PT09PT09PT09PT0//wAARCAFsAPADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAA
AAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM
2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJW
Wl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAw

...

The opposite operation of the ToBase64String method is the Convert.FromBase64String method. So it's easy to write a little tool that extracts the email content, converts it back to JPEG data and saves it as a file for example. Probably it's sufficient to use an online Base64 encoder like this one where the Base64 string can be pasted and the tool will create a binary file which can be downloaded.

The result might look like the below screenshot from the Pictures Lab app. As you can see, the half downscaled version with the 60% quality is usable and even the small text can be read without problems.

Future

There's also room for improvement left. More modern image compression algorithms like they are used in the Hipix or WebP formats can produce better results at lower size. Another way to reduce the size could be a post process step where every odd pixel or line is removed. This technique was described in the Kill Pixel Shader blog post.

Needless to say that the demonstrated, well-known Base64 encoding can be used to send any small binary data inside an email from a Windows Phone app. So it's also possible to send short audio clips, binary serialized data or other data.

Conclusion
For more complex tracking and feedback mechanisms an image upload webservice might be the better way. However there are many scenarios with simple tasks where a webservice isn't feasible and the shown technique can be helpful.

Listen to your Phone - Getting User Feedback with System Information

2010-12-20T23:33:00.003+01:00

Two weeks ago I've blogged about a trick my Windows Phone Pictures Lab app uses to detect if the Zune software is running. Today I have a handful of classes that help to gather system information and provide an easy interface for the email task. No magic at all, but I though it might be helpful for others too.
I use these helpers in Pictures Lab for the user feedback functionality. I got a lot of valuable feedback and great suggestions via this feature.
The user can invoke this functionality on the about page of the app and it's also fired when an unhandled exception occurred and the user wants to send error details.

System Information
It's always useful to include some system, version and device information in such a feedback report. That's why I wrote a SytemInformation class which queries some paramteres and creates a nice, human readable string.

public class SystemInformation
{
   public static string Dump()
   {
      var builder = new StringBuilder();
      builder.AppendLine("***** System Infos *****");
      builder.AppendLine();
      builder.AppendFormat("Time: {0}", DateTime.Now.ToUniversalTime().ToString("r"));
      builder.AppendLine();
      builder.AppendFormat("Culture: {0}", CultureInfo.CurrentCulture);
      builder.AppendLine(); 
      builder.AppendFormat("App Version: {0}", WmAppManifestHelper.GetVersion());
      builder.AppendLine();  
      builder.AppendFormat("Manufacturer: {0}", DevicePropertiesHelper.GetDeviceManufacturer());
      builder.AppendLine(); 
      builder.AppendFormat("Model: {0}", DevicePropertiesHelper.GetDeviceModel());
      builder.AppendLine(); 
      builder.AppendFormat("DeviceHardwareVersion: {0}", DevicePropertiesHelper.GeDeviceHardwareVersion());
      builder.AppendLine(); 
      builder.AppendFormat("DeviceFirmwareVersion: {0}", DevicePropertiesHelper.GeDeviceFirmwareVersion());
      builder.AppendLine(); 
      builder.AppendFormat("OS  Version: {0}", Environment.OSVersion);
      builder.AppendLine(); 
      builder.AppendFormat("CLR Version: {0}", Environment.Version);
      builder.AppendLine(); 
      builder.AppendFormat("Device Type: {0}", Microsoft.Devices.Environment.DeviceType);
      builder.AppendLine(); 
      builder.AppendFormat("Network Type: {0}", NetworkInterface.NetworkInterfaceType);
      builder.AppendLine(); 
      builder.AppendFormat("DeviceTotalMemory: {0:f3} MB", DevicePropertiesHelper.GetTotalMemoryMb());
      builder.AppendLine(); 
      builder.AppendFormat("ApplicationPeakMemoryUsage: {0:f3} MB", DevicePropertiesHelper.GetPeakMemoryUsageMb());
      builder.AppendLine(); 
      builder.AppendFormat("ApplicationCurrentMemoryUsage: {0:f3} MB", DevicePropertiesHelper.GetCurrentMemoryUsageMb());
      builder.AppendLine(); 
      using (var appStorage = IsolatedStorageFile.GetUserStoreForApplication())
      {
         builder.AppendFormat("Iso Storage AvailableFreeSpace: {0:f3} MB", appStorage.AvailableFreeSpace / 1024f / 1024f);
         builder.AppendLine();
      }
      return builder.ToString();
   }
}

As you can see a few interesting properties from the OS, the device and its configuration and memory status are queried. This is done with built-in and custom helper classes.
The WmAppManifestHelper class parses the WmAppManifest.xml file and provides the app version and other properties from this file. The DevicePropertiesHelper wraps the DeviceExtendedProperties class and the available property keys. I added all the keys I need, but not more (YAGNI). However, if you add more property keys to the class, please make sure to post a comment.

Email Service
The EmailService class wraps the EmailComposeTask and adds the system infos to the body of the email.

public class EmailService
{
   private readonly EmailComposeTask emailComposeTask;

   public string Addressee { get; set; }

   public EmailService()
   {      
      Addressee = "foo@bar.com";
      emailComposeTask = new EmailComposeTask();
   }

   public void ShowTask(string subject, string body)
   {
      // Add system infos add the end of the email
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += SystemInformation.Dump();

      // Fill email
      emailComposeTask.To = Addressee;
      emailComposeTask.Subject = subject;
      emailComposeTask.Body = body;

      // Invoke Mail Task
      emailComposeTask.Show();
   }
}

The ShowTask method sets the properties of the EmailCompoaseTask and opens it. The addressee can be defined with the corresponding property.

Please replace the foo@bar.com default value for Addressee with your email address. Think about the poor Foo at bar.com.

Handling the Unhandled
This email functionality is used to enable explicit user feedback when the user taps the 'submit feedback' button. It is also utilized in the UnhandledException event handler of the Application class. The default body of the event handler in the App.xaml.cs file is generated by Visual Studio when a new Windows Phone Application project is created. Just add the email service and set the e.Handled property to true.

private static void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
   var result = MessageBox.Show("An error occured. Do you want to send the error details to the developer?", "Meh", MessageBoxButton.OKCancel);
   if (result == MessageBoxResult.OK)
   {
      new EmailService().ShowTask("App error", e.ExceptionObject.ToString());
   } 
   e.Handled = true;
}

The body of such an error email would then look like this

System.ArgumentException: The parameter is incorrect. 
   at MS.Internal.XcpImports.CheckHResult(UInt32 hr)
   at MS.Internal.XcpImports.SetValue(INativeCoreTypeWrapper obj, DependencyProperty property, DependencyObject doh)
   at MS.Internal.XcpImports.SetValue(INativeCoreTypeWrapper doh, DependencyProperty property, Object obj)
...


***** System Infos *****

Time: 12/11/2010 5:50:02 PM
Culture: en-US
App Version: 1.9.0.0
Manufacturer: SAMSUNG
Modell: SGH-i917
DeviceHardwareVersion: 3.1.0.7
DeviceFirmwareVersion: 2103.10.10.1
OS  Version: Microsoft Windows CE 7.0.7004
CLR Version: 3.7.10218.0
Device Type: Device
Network Type: MobileBroadbandGsm
DeviceTotalMemory: 475.469 MB
ApplicationPeakMemoryUsage: 63.055 MB
ApplicationCurrentMemoryUsage: 41.223 MB
Iso Storage AvailableFreeSpace: 601.641 MB

Conclusion
User feedback is always valuable and every serious Windows Phone developer should at least add a simple functionality like the one presented here.

Source Code
You can download all mentioned classes here. Please comment if you know more useful system infos that should be included.

Issue with the WP7 PictureDecoder and Workaround

2010-12-09T23:59:00.005+01:00

This is fixed in the WP 7.1 SDK / WP 7.5 / Mango!
I noticed a strange behavior of the Windows Phone PictureDecoder DecodeJpeg method while I was working on my Pictures Lab app.
This short post describes the issue I encountered and also provides a workaround.
The built-in DecodeJpeg method decodes a JPEG image stream into a WriteableBitmap. The method has two overloads, where the first only takes the JPEG stream and the second also uses parameters for the maximum width and height of the output. I encountered a strange behavior of the latter method.

Issue
The DecodeJpeg method with 3 parameters swaps the width and the height parameters when a landscape photo should be resized. For example, an image with the original size of 3264 x 2448 should be decoded to a WriteableBitmap and resized to 1024 x 768. But the resulting output will have a size of 768 x 576 cause the method swapped the input parameters while preserving the correct aspect ratio.
Please note that this only happens with landscape pictures, which means the width of the image is greater than the height.

Will it get fixed?
Yes! I contacted Microsoft and they confirmed this issue and said that it will get fixed in a future version of the Windows Phone Silverlight runtime.

Workaround
Isolating the cause of this strange behavior took a bit of time, fortunately the easier was the obvious workaround.

int w = desiredOutputWidth;
int h = desiredOutputHeight;

// Workaround for issue in DecodeJpeg method:
// Swap width and height for landscape pictures.
if (originalWidth > originalHeight)
{
   w ^= h;
   h ^= w;
   w ^= h;
}
var writeableBitmap = PictureDecoder.DecodeJpeg(jpegStream, w, h);

The code tests if the original width is greater than the height and then swaps the output width and height using the good old XOR swap trick.
Alternatively you can avoid the DecodeJpeg overload with the resizing functionality, decode the full sized image and use the WriteableBitmapEx Resize method afterwards. It produces similar results when bilinear interpolation is used, but it's an extra step that costs resources.

The above photo was taken with a HTC Mozart and edited with Pictures Lab (cropped, rotated and 1989 vintage effect).

Windows Phone Unplugged - How to Detect the Zune Software

2010-12-09T22:20:00.006+01:00

I've been blogging about Windows Phone development since the first CTP version and also blogged about the photo and camera API.
This post will cover a little trick my Pictures Lab app uses to detect if the Zune software is running when the device is connected to the PC. This is important because the device's media Hubs can't be accessed when the device is connected and the Zune software is running. The same is true for the media APIs (PhotoChooser, MediaLibrary, ...).
The Pictures Hub and the Zune Hub show a little animation when you try to open these while Zune is running. Unfortunately there's no built-in method to detect this in code.

An app mostly performs the classic IPO Model operations: load, process, save. The load and save steps can fail when Zune is connected. The following two code snippets show how to detect if the Zune software is running and ask the user to close it. I use the PhotoChooser and MediaLibrary here to demonstrate this exemplarily for picture handling, but the same is also valid for other media like audio.

Save fail
If the device is connected to the PC and the Zune software is running, an InvaildOperationException is thrown.
By the way, this blog post covered how to save a WriteableBitmap as JPEG to the device's media library.

private static void Save(WriteableBitmap bitmap)
{
   try
   {
      SaveToMediaLibrary(bitmap, "mySuperCoolPic.jpg");
   }
   catch (InvalidOperationException)
   {
      MessageBox.Show("The Zune software is running and the picture can't be saved. Please close Zune or disconnect the phone.");
   }
}

This is quite straightforward.

Load fail
Unfortunately it's not that easy to detect if the Zune software is running when a picture is opened through the PhotoChooserTask API. (How to use this Task was also covered here.)

Some users contacted me when they tried to open a photo with Pictures Lab and nothing happened. The problem is that the picture viewer just won't open when the PhotoChooserTask.Show method is called and Zune is running on the PC. There's no built-in feedback here. I implemented the following code in the Completed event handler of the PhotoChooserTask to get a better user experience.

private void PhotoProviderTaskCompleted(object sender, PhotoResult e)
{
   try
   {
      if (e != null)
      {
         if (e.TaskResult == TaskResult.OK)
         {
            // Do something with the JPEG stream in e.ChosenPhoto
         }
         else if (e.Error is InvalidOperationException)
         {
            throw e.Error;
         }
         else if (e.TaskResult == TaskResult.Cancel)
         {
            // Try to get an image from the Pictures Hub. 
            // If this fails, the device is connected and Zune is running.
            var pictures = new MediaLibrary().Pictures;
            if(pictures.Count > 0)
            {
               pictures[0].GetThumbnail();
            }
         }
      }
   }
   catch (InvalidOperationException)
   {
      MessageBox.Show("The Zune software is running and the picture can't be saved. Please close Zune or disconnect the phone.");
   }
}

One might wonder why the MediaLibrary is queried when the PhotoResult EventArgs has the Error property. Unfortunately it's not used when Zune is running, only the TaskResult is set to Cancel in this case. This TaskResult value is also used when the user presses the back button while choosing a photo and it's not clear why the TaskResult was set to Cancel.

Fin
That's it.
I hope it helps.
Let me know if you found a better way.

The above photo was taken with a HTC Mozart and edited with Pictures Lab (cropped and Lomo effect).

Tracking Sales Statistics with the Silverlight Analytics Framework for Windows Phone

2010-11-02T19:48:00.015+01:00

The official statistics are now available at the App Hub portal. However, this approach is capable of more than sales tracking.

Are you a Windows Phone developer? Are you also curious how well your app performs in the Windows Phone Marketplace? I am!
Some might say it doesn't matter yet, since there aren't many devices out there, but nevertheless I'd like to know my statistics. Also O2 Germany recently reported they have sold out the first batch of HTC HD7 devices with sales in the 5-figure range. Unfortunately we won't see official statistics from the Marketplace any time soon. An App Hub forum member contacted the Microsoft App Hub support and they told him that sales statistics won't be available before January 2011. Bummer!
Fortunately we are app developers and we can implement our own statistic tracking. On the other hand this means we have to setup and pay some service and implement tracking code. Here is where the Silverlight Analytics Framework and Google Analytics come to the rescue.
This blog post provides a step-by-step guide on how to track your app (sales) statistics with the Silverlight Analystics Framework and Google Analystics. Did I mention it's free and not only free for a limited period?
Let's go...

Setup Google Analytics
If you don't have a Google Analytics account, just sign up here. It's free.

Create an empty HTML page with the minimum tags and upload it to some public webspace. If you don't have webspace available, just create a free Dropbox account and put the page in the public folder.
Go to Google Analytics and use the "Add Website Profile" functionality. Choose "Add a Profile for a new domain". Enter the URL of the dummy website.
The next page setup page will show the necessary Java Script tracking code. Copy the code, paste it into the dummy website and upload the website.
Your website profile is now waiting for activation. Go back to the overview and click the Edit link of your website profile. See the little yellow exclamation mark with the text "Tracking Not Installed" at the upper right corner. Click on "Check Status". The status should change soon and the profile will become active. If not, give Google a couple of minutes and retry it then.

Setup the Silverlight Analytics Framework
Now it's time to submit the tracking events from your app to Google Analytics. This task is pretty easy with the help of the Silverlight Analytics Framework.
The Silverlight Analytics Framework is an extensible web analytics framework mainly developed by the Microsoft evangelist Michael S. Scherotter. It's really nice, designed in a flexible way with the use of modern technologies and open source. Google Analytics is just one of the many supported analytics services.

Download the latest version from the Codeplex site and install it or download the latest source code from the repository and build it yourself. I've chosen the latter since Michael recently implemented a nice feature: The Silverlight Analytics Framework for Windows Phone tracks each page view by default. In the changeset 63020 Michael added an IsPageTracking property which allows to disable the page tracking. This feature is part of the latest release 1.4.8.
Add the following assembly references to your project.
From the Silverlight for Windows Phone Framework: System.Device, System.Windows.Interactivity
From the Silverlight Analytics Framework: Microsoft.WebAnalytics, Microsoft.WebAnalytics.Behaviors, Google.WebAnalytics, System.ComponentModel.Composition, System.ComponentModel.Composition.Initialization

Create a new class in your project:

public class AnalyticsService : IApplicationService
{
   public void StartService(ApplicationServiceContext context)
   {
      // Wire up MEF
      CompositionHost.Initialize(
         new AssemblyCatalog(
            Application.Current.GetType().Assembly),
            new AssemblyCatalog(typeof(Microsoft.WebAnalytics.AnalyticsEvent).Assembly),
            new AssemblyCatalog(typeof(Microsoft.WebAnalytics.Behaviors.TrackAction).Assembly));
   }

   public void StopService() { }
}

Go to the App.xaml file of your project and add the namespace definitions to the first tag:

<Application
    x:Class="MyApp.App"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"    
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

    
    xmlns:local="clr-namespace:MyApp"
    xmlns:ga="clr-namespace:Google.WebAnalytics;assembly=Google.WebAnalytics"
    xmlns:mpc="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"
    xmlns:mwa="clr-namespace:Microsoft.WebAnalytics;assembly=Microsoft.WebAnalytics"
    >

Then add the service references inside the Application.ApplicationLifetimeObjects declaration in your App.xaml file:
```
<Application.ApplicationLifetimeObjects>    
    <local:AnalyticsService />
    <mwa:WebAnalyticsService IsPageTrackingEnabled="False">
        <mwa:WebAnalyticsService.Services>
            <ga:GoogleAnalytics WebPropertyId="UA-XXXXXXX-X" />
        </mwa:WebAnalyticsService.Services>
    </mwa:WebAnalyticsService>
</Application.ApplicationLifetimeObjects>
```
Most important you need to replace the WebPropertyId in the GoogleAnalytics element with the Id of your newly created Google Analytics website profile.
I'm not interested in the page tracking, therefore I've set the IsPageTrackingEnabled property to False. You should also keep in mind that page tracking has an impact on the performance of the app. I recommend to disable it.
Since your app is now using a network connection, you need to add the ID_CAP_NETWORKING capability to your WMAppManifest.xml file.

That was all to get the Silverlight Analytics Framework connected to your Google Analytics profile. It now tracks the Deactivated, Activated and Started events.
You can find the results here:

Please keep in mind that it usually takes 12 - 24 hours until the tracked data shows up in Google Analytics.

Add sales tracking
There's only one puzzle piece left now, the actual sales tracking. I use two Windows Phone APIs for this. First the LicenseInformation.IsTrial method which tells us if the app runs in trial mode or if it was paid. The second API is the DeviceExtendedProperties class with the DeviceUniqueId key which provides a unique 20 byte long hash for the phone. Such an id is needed to count unique installations.

Add a new class to your project for the explicit tracking from code:

public class AnalyticsHelper
{
   // Injected by MEF   
   [Import("Log")]
   public Action<AnalyticsEvent> Log { get; set; }

   public AnalyticsHelper()
   {
      // Inject
      CompositionInitializer.SatisfyImports(this);
   }

   public void Track(string category, string name)
   {
      // Track analytics event
      Log(new AnalyticsEvent { Category = category, Name = name, });
   }
}

Call the Track method when your app launches and provide the necessary information.
Add this code to the Application_Launching event handler in the App.xaml.cs file.

private void Application_Launching(object sender, LaunchingEventArgs e)
{
   var analyticsHelper = new AnalyticsHelper();

   // Get device id
   var value = (byte[])DeviceExtendedProperties.GetValue("DeviceUniqueId");
   var id = Convert.ToBase64String(value);
   
   // Track launch
   analyticsHelper.Track("Launch All", id);

   // Track if paid
   if (!new LicenseInformation().IsTrial())
   {
      analyticsHelper.Track("Launch Paid", id);
   }
}

Gergely Orosz (@Gergely Orosz) gave me the idea to track the statistics only if the phone is connected to Wi-Fi. This is pretty easy to implement, just add an if condition around the tracking: NetworkInterface.NetworkInterfaceType == NetworkInterfaceType.Wireless80211

The app is now using the unique id of the phone, so you have to add the ID_CAP_IDENTITY_DEVICE capability to your WMAppManifest.xml file.

As you can see all app launches are tracked in the category "Launch All" and all paid launches in the "Launch Paid" category. This makes it easy to see all installations and all sales directly. Needless to mention that Launch Trial = Launch All - Launch Paid.
You can check if all works with the Fiddler or the TCPView tool while using the Windows Phone emulator. By the way, the tracking is pretty fast. I measured 20 to 40 ms for each Track call when the phone was connected to Wi-Fi.

The tracked events are best viewed when grouped by categories. The screenshot below shows a test with multiple launches of a phone and the emulator. The total number of events is listed at the bottom right of the table. Note that Unique Events aren't significant here since they only represent unique events for one session.

The id (event name) we supply through the AnalyticsEvent is mapped to the Google Analytics Event Action. You can find all event mappings and more info in the relevant Silverlight Analytics Framework documentation.

Conclusion

The update I submitted to the Marketplace works nicely and I hope to see some meaningful statistics in the next weeks. I think the described method is a good way to track your sales and other statistics until the App Hub team provides the official usage / sales statistics. Actually the provided tracking functionality is even useful when the App Hub team provides official statistics. And it's all free without a limited timeframe.

Silverlight is not dead and not undead either

2010-11-01T20:37:00.003+01:00

No, it's alive and kicking.
You might have heard about the rumors that came up after the PDC conference and Bob Muglia's interview with Mary Jo Foley. Many people flooded Twitter and other sites with speculations that Silverlight is dead for the web. Microsoft's President of the Server and Tools Division Bob Muglia just clarified the situation.

"Make no mistake; we’ll continue to invest in Silverlight and enable developers to build great apps and experiences with it in the future."

You should also read the blog post from the Silverlight program manager Tim Heuer. Without a doubt he puts all his effort into the success of Silverlight. And not to forget the Microsoft Silverlight evangelist John Papa who also proves his commitment to Silverlight here.

Now let's get back to work...

The picture is an old poster from the movie Santo en el Museo de Cera with the famous Mexican lucha libre wrestler El Santo.

Pictures Lab for Windows Phone

2010-10-18T20:47:00.059+02:00

"Windows Phones are going to be absolutely amazing devices for taking pictures and dealing with pictures." - Joe Belfiore at the official Windows Phone 7 launch event in NYC. I totally agree, they are. Just think about that each device needs to have at least a 5 megapixel camera and the little camera button with the "pocket to picture" functionality. I've been working heavily to enhance this photo experience with my kind of skills. This blog post contains information about the Windows Phone app I created and will also provide the references to how it works internally.

Marketplace app description
Pictures Lab is the ultimate picture effects app for all Windows Phone device types.
MSNBC.com: "a Swiss Army knife of photo tweaks".
Engadget.com: "a must-have for WP7 devices ... the program provides a set of amazing effects and tweaks for your photos"
Gizmodo.com : "Pictures Lab offers a ton of effects for your money."

If you like to make beautiful photos, then this app is a perfect addition to your phone’s toolset. Pictures Lab provides you with thousands of different modifications for your photos and the WP 7.5 Mango version has a built-in take photo functionality with face detection auto focus, steady mode and much more. Pictures Lab is the original with high quality filters, features and great user experience. It's fast and responsive and you quickly get your job done with just a few gestures. Often been copied, but never reached.
Microsoft uses Pictures Lab in its presentations as a "delightful app" which "Looks and feels like an integral part of Windows Phone 7". "Uncluttered, polished experience".

The app comes with almost 30 controllable, easy-to-use, but advanced effects like different vintage and hipster-like filters, Tilt Shift (miniature faking), HDR, Lomo, Soften, Auto Adjust, Sharpen, Comic, Bulge, B&W, Sepia and many more. Dynamic previews of the effects are shown in a list, making it possible to easily pick the right one. The high quality borders are also presented in a nice preview list and can be combined with any filter and any aspect ratio. Multiple effects can also be combined.
It is also possible to crop, rotate and flip images. An enhanced picture and the EXIF data can be saved to the phone’s pictures hub or shared directly on Facebook, Twitter and other social networks. The app performs the image processing on the original picture in its original resolution, thus making it also possible to print the image or use it as desktop wallpaper.
Full Pictures Hub Apps integration and support for landscape and portrait orientation. Trial version with complete functionality only save is disabled.

If YouTube works better for you.

Reviews

The well-known site Engadget lists it under The best apps, accessories, and tips. They write: "... a must-have for WP7 devices ... the program provides a set of amazing effects and tweaks for your photos ..."
Gizmodo included Pictures Lab in Windows Phone 7 Essential Apps, Reviewed. They also recorded a video. They write: "Your phone isn't complete until you can take photos that look like they came out of a $5 camera from 1974. Pictures Lab fortunately offers a ton of other effects for your money. $2."
Gizmodo Australia also featured it in the article The Best Windows Phone 7 Apps.
msnbc.com wrote: "The app, a Swiss Army knife of photo tweaks, will be one of my first downloads."
techradar.com wrote that Pictures Lab is the #1 app worth paying for: "Tilt-shift effects, crop and rotate... Pictures Lab combines the fun effects of an iPhone photo app like Hipstamatic with the useful fixes you're mostly likely to need (sharpen and - in the imminent update - crop) in a beautifully designed app that crosses the power of Silverlight with the tools of the Windows Phone 7 interface. Pick a photo, see all the possible effects as thumbnails or swipe up and down to try them one at a time."
msnbc.com listed it in the article: "The first Windows Phone 7 apps you should grab"
MSN UK listed it under the "20 top mobile phone app". And they put my fat nose on the front page. Fortunately two of my girls brought the beauty back to the page.
America's popular WIRED magazine with a circulation of 800,000 featured Pictures Lab in there special edition WIRED App Guide.
UK's popular gadget magazine T3 with a circulation of more than 60,000 featured Pictures Lab in the 02/2011 issue on the page: "Killer Apps"
Pictures Lab was featured in Channel 9's Hot Apps series.
Nokia writes "it’s the closest thing to having Photoshop on your mobile phone."
Nokia included Pictures Lab in Top of the Apps: 10 Hotties.
All About Windows Phone reviewed the app: "Pictures Lab is one of many photo toolboxes for your smartphone, but it's the one that I enjoy using the most." and "Very, very much recommended."
Another very popular UK magazine Stuff wrote a 5 stars review in the March edition. They write that Pictures Lab is better than a well-known iPhone app.
Germany's popular media and design magazine PAGE with a circulation of 20,000 featured Pictures Lab in the 01/2011 issue
Microsoft features Pictures Lab in their presentation slides for Windows Phone under the category delight: "Looks and feels like an integral part of Windows Phone 7". "Uncluttered, polished experience".
Business Insider writes "Pictures Lab is like Instagram for Windows Phone 7. Take pictures with Pictures Lab, apply filters, and share them with friends."
The famous design site Smashing Magazine included Pictures Lab as an example in the article Introduction To Designing For Windows Phone 7 And Metro
Bright Hub lists Pictures Lab as the first app in Top 5 Samsung Focus Apps: "It is considered one of the most popular photo editing programs on the Windows Phone platform, and for good reason. It has an array of sophisticated and beautiful effects ..."
APC (Australian Personal Computer) writes: "WP7’s best picture editing and effects app.Pictures Lab has received a lot of attention as the “go-to” image editing suite for new WP7 users – and for good reason, too."
Mobility Digest wrote: "This app makes me happy. Tons of features, seems simple to use and it’s cheap – that’s a win! It even has a trial mode that includes all effects."
Mobility Digest also posted a walk through: "I didn’t realize it, but when you go to the list of available effects you get to see a preview of the effect which is really great and I have to say, I’m really impressed for how much this app can do."
Mobility Digest listed it in: "The First 25 Windows Phone Apps To Download"
Pocket-Lint wrote in the article Best Windows Phone 7 photography apps: "... the smart way to go is with Pictures Lab..."
maximumpc.com wrote: "Speaking of pictures, for two bucks you can have a great little mobile suite of photo effects, fixes, and basic editing tools. The Windows Phone 7 UI language really shines through in this one."
Mobile Choices listed it as one of the two Windows Phone apps in The 12 best Christmas apps: "...turning lifeless images into photographic masterpieces..."
The blog Noisecast recorded a nice video review: "The program @PicturesLab is the best photo editor for Windows Phone 7, hands down."
1800pocketpc.com wrote: "If you love to add effects to your picture then this is one app video you wouldn’t want to miss."
1800pocketpc.com also wrote a review including a video
wpcentral.com wrote: "... interesting photo effect application."
mobiledownloadblog.com wrote: "There is no doubt that any phone device is incomplete without good picture depiction. A fantastic photograph app known as Picture Lab is now available for Windows Phone 7."
Techland.com has it on the list: "50 Windows Phone 7 Apps to Get You Started"
mobiledownloadblog.com has it on the list: "Samsung Focus: 30 Best WP7 Applications"
The German site phoneseven.de wrote: "Die App sieht wirklich klasse aus und lässt sich allem Anschein nach wirklich gut bedienen. Wer also befürchtet hat, dass Windows® Phone 7 zu wenig Möglichkeiten für gute Apps bietet, der wird mit Pictures Lab sicherlich eines besseren belehrt. Die App wird sicherlich zu meinen ersten Käufen aus dem Marketplace gehören."
The Portuguese site pcdebolso.com also wrote an article: "Pictures Lab a caminho do Windows phone 7"
Dave Campbell wrote a nice review: "All-in-all, I give Pictures Lab 5 stars"
John Papa wrote: "Overall, very nice app, good UI, snappy (which was a surprise to me since it does a lot)."

Feature list

29 advanced effects: Ten different vintage and hipster-like effects, Tilt Shift (miniature faking), HDR, Lomo, Soften, Auto Adjust, Sharpen, Comic, Bulge, B & W, Sepia, Manual, Edge Detection, Night Vision, Rainbow, Mirror, X-Ray, Heat Image, Negative and Bitmap.
Editable effect parameters with ready-to-use default values.
Combination of multiple effects.
11 high quality borders and more seasonal borders.
Share on Facebook, Twitter and other social networks.
The WP 7.5 Mango version has a built-in take photo functionality with face detection auto focus, steady mode, rule of thirds grid. The steady mode requires a device with a gyroscope. The face focus is enabled if the device supports focus at point.
Crop, rotate and flip.
Multi-touch manipulation for Tilt Shift and other effects where multi-touch is meaningful.
The app can save edited pictures with advanced effects applied in full resolution even on a device with 256 MB RAM. Thus making it also possible to print the image or use it as desktop wallpaper.
Full Pictures Hub Extras integration.
The trial version includes the complete functionality and all the effects, only the save functionality is disabled.
Support for landscape and portrait orientation.
Effect list with dynamic previews and caching of the larger previews.
Flicking through the effects on the main page.
Setting for the maximum output size, JPEG quality and many more.
State preservation when the app is deactivated (tombstoning).
Localization for English, German, Japanese, Russian, Dutch, French, Italian, Spanish, Portuguese and Portuguese (Brasil).

Go to
You can follow this link if you have the Zune software installed and your browser supports Zune links. Or just open the web Marketplace.

How it works

Some of the image processing algorithms that are used in the Pictures Lab app would normally be implemented as pixel shaders for Silverlight. The Windows Phone platform doesn't support custom pixel shaders at the moment, therefore all effects are implemented in C# and manipulate the WriteabeBitmap Pixels' in some way. How this works in detail was shown in my Coding4Fun articles I've written months ago. This is actually where Pictures Lab has its roots. The first article described all the necessary basics and two simple effects. The second part of this short Coding4Fun article series shows how to implement the basic tilt shift and vintage effects you can find in the Pictures Lab app. Yes, some parts are open source! Except the secret sauces I gotta keep for me that adds the final touch to the effects. I also wrote blog posts about some parts of the application that are surely interesting for other Windows Phone developers.

The Slider control that is used in the App was originally developed by Nokola and then adapted for Windows Phone and made bindable by Clint Rutkas and myself.

To be continued
The planned feature list for the next versions is already pretty long and I have some real diamonds on there. You can follow @PicturesLab on Twitter to get all the status updates. I'd also like to know what feature you want to see in a future version. Just write a comment.

Windows Phone Memory Constraints

2010-09-20T20:56:00.002+02:00

The final Windows Phone Developer Tools have been released and the launch date is not far away. It's time to finalize the apps.
Windows Phone has some tough hardware requirements, but it's still a mobile platform with limited resources. In this blog post I'll describe how to determine the current memory usage, what the allowed maximum is to get your app approved for the Marketplace and other constraints a developer should be aware of.

Maximum memory usage
The Windows Phone hardware specifications requires that a device has 256 MB RAM or more. The allowed maximum memory an app can use is defined in the Windows Phone 7 Application Certification Requirements at point 5.2.5

The application must not exceed 90 MB of RAM usage. However, on devices that have more than 256 MB of memory, an application can exceed 90 MB of RAM usage.

One might now ask how to get this information at runtime. Fortunately there's the DeviceExtendedProperties class to obtain some device information at runtime. This class defines the GetValue and TryGetValue methods which expect a string key as parameter. The list of available property keys is defined here. Among other interesting properties like the DeviceUniqueId, it's possible to retrieve the DeviceTotalMemory, the ApplicationCurrentMemoryUsage and the ApplicationPeakMemoryUsage. The property keys are interpreted case-sensitive.
The application needs to define the device identity capability in the WMAppManifest.xml file in order to retrieve the DeviceUniqueId property, but it's not needed for the other keys like the ApplicationPeakMemoryUsage.

Here's the code to get the peak memory usage and the available total memory at runtime:

var timer = new DispatcherTimer {Interval = TimeSpan.FromSeconds(2)};
timer.Tick += (s, e) =>
{
   var memuse = (long)DeviceExtendedProperties.GetValue("ApplicationPeakMemoryUsage");
   var maxmem = (long)DeviceExtendedProperties.GetValue("DeviceTotalMemory");
   memuse /= 1024 * 1024;
   maxmem /= 1024 * 1024;
   MyTextBlock.Text = String.Format("Mem usage: {0} / {1} MB", memuse, maxmem);
};
timer.Start();

The timer checks the properties every two seconds and converts the bytes to MB. The values are then shown in a TextBlock. The peak memory is the most interesting since this value must not exceed 90 MB on a device with 256 MB. Also note that an empty application with two grids and a TextBlock already has a peak of 5 - 7 MB in the emulator or on a device.

PhoneApplicationPage.State limitation
The PhoneApplicationPage.State property is typically used to save transient state for tombstoning. The size of the State data is limited to 2 MB for a Page and 4 MB for the whole application. Be warned, if this limit is reached a not very descriptive exception is thrown. The message text is "The client who called IAccessControl::IsAccessPermitted was the trustee provided tot he method".
If more than 2 / 4 MB is needed to save transient data, the IsolatedStorage can be used. The applications IsolatedStorage is only limited by the device's flash storage size. The minimum requirement for Windows Phone is 8 GB. The available free isolated storage in bytes can be retrieved with the AvailableFreeSpace property of the IsolatedStorageFile class.

Rect and Point Extension Methods

2010-08-31T19:10:00.000+02:00

I'm currently working on multiple Windows Phone projects and don't have much time for longer blog posts, but I just wanted to get this out.
For one of the projects I need to calculate some simple vector arithmetics like the Euclidean distance. I also need other properties of the Point and Rect structs that aren't built-in the Silverlight framework. As you might know I'm a huge fan of extension methods for extending built-in functionality (1, 2). That's why I wrote a few extension methods for the Point and Rect structs that might come handy.

Rect

public static class RectExtensions
{   
   // Calculates the center point of this rectangle.
   public static Point Center(this Rect rect)
   {
      return new Point(rect.X + rect.Width * 0.5, rect.Y + rect.Height * 0.5);
   }

   // Tests if this rectangle intersects with the other.
   public static bool IntersectsWith(this Rect rect, Rect other)
   {
      // Test for separating axis
      if (other.Bottom < rect.Top || other.Right < rect.Left 
       || other.Top > rect.Bottom || other.Left > rect.Right)
      {
         return false;
      }
      return true;
   }
}

The usage should be pretty obvious.

Rect r = new Rect(0, 0, 200, 300);
Point center = r.Center();

bool isIntersecting = r.IntersectsWith(new Rect(500, 600, 100, 50));

Point

public static class PointExtensions
{
   // Calculates the distance vector of the points.
   public static Point Distance(this Point p1, Point p2)
   {
      return new Point(p1.X - p2.X, p1.Y - p2.Y);
   }

   // Calculates the length of the vector.
   public static double Length(this Point p)
   {
      return Math.Sqrt(p.X * p.X + p.Y * p.Y);
   }

   // Calculates the square length of the vector.
   public static double LengthSquare(this Point p)
   {
      return p.X * p.X + p.Y * p.Y;
   }
}

The squared length saves some cycles if this information is only needed for comparison.

Point p1 = new Point(200, 300);
Point p2 = new Point(400, 500);
Point distance = p1.Distance(p2);

double length = distance.Length();
double lengthSq = distance.LengthSquare();
length = Math.Sqrt(lengthSq);

Source Code
You can download the two C# files from here.

Coding4Fun - Part 1 of the Windows Phone Picture Effects App and Roadmap

2010-08-10T15:16:00.008+02:00

My new Windows Phone 7 article for Microsoft's Coding4Fun site is live. It's the first part of a short series about the development of a Windows Phone 7 picture effects application called "PicFx". The first article will show how to create the base Windows Phone application and how to implement some basic effects. The source code is licensed under the Ms-PL and can be downloaded from the CodePlex site.
I'm already working on the second part of the series, which will explain how to implement advanced effects (the cool stuff) and other features. After this article I'll add more features and important effects, enhance the existing effects and release it as "Pictures Lab" on the Marketplace.

Video
The video below demonstrates the Coding4Fun PicFx application features and also shows how it can be used. It was recorded with the application running in the emulator.

Book Review - 3D Game Development with Microsoft Silverlight 3: Beginner's Guide

2010-08-02T21:40:00.010+02:00

A couple of weeks ago I got a review copy of the book 3D Game Development with Microsoft Silverlight 3: Beginner's Guide. The book has 452 pages, is written by Gastón C. Hillar and shows how to write games with Silverlight.
Although the title includes "Silverlight 3", the concepts are also valid for newer versions of Silverlight, like the current version 4.
This blog post is a short review of the book. I also have one physical copy of the book to give away. Yeah, a free book, like free beer.

The Content
The book 3D Game Development with Microsoft Silverlight 3: Beginner's Guide starts by explaining what tools are needed for Silverlight game development. Then the first simple Silverlight application is developed.
The second chapter explains what Sprites are, how they can be used for 2D games and frame based animations. Silverlight 2.5D hardware accelerations is also covered and how basic vector transformations work.
The next chapter introduces a Sprite wrapper class and how to simplify the game loop with it. Basic 2D collision detection with an axis-aligned bounding box is covered too. The author also shows in this chapter how to use keyboard input for the game control.
Chapter 4 finally brings 3D game development. It starts with a WPF XAML Browser Application (XBAP) application that draws a 3D model, which was generated with the 3D modelling tool Blender and then exported to a XAML file. After this, the 3D engine Balder is introduced and a simple application is created. To be honest, Balder has come a long way since the book was written and a lot of new features were introduced. Nevertheless, many concepts and the basics are still valid.
The next chapter explains some 3D concepts like Cameras and shows how to use them with Balder and XBAP. The 6th chapter is about input controlling and demonstrates how to use DirectInput with the XBAP application. Chapter 7 explains textures, lights and the usage of these in a game. The next chapter is all about animation.
The 9th chapter introduces the Silverlight physics engine Farseer and chapter 10 shows how to detect collisions and apply basic artificial intelligence. The next chapter adds an asteroid belt to the space game, which is used as an example throughout the book.
In Chapter 12, the author shows how to measure the game progress and how to use Expression Blend to create screens for highscore and other statistics. The next chapter continues with this and also explains the usage of pixel shaders from the WPF Pixel Shader Effects Library. Persisting settings to the IsolatedStorage is covered too. The last chapter demonstrates how to play and control audio and video with the Silverlight MediaElement.

Summary
If you want to get into game development with Silverlight, this is the right book for you. It starts with the basic tools, then shows how to write a simple 2D game, finally how to write a 3D Silverlight game with Balder and how to add a physics system. Another nice thing of this book is that the tools and techniques to create / convert content are also shown step-by-step.
3D Game Development with Microsoft Silverlight 3: Beginner's Guide lives up to its title: "Beginner's Guide". If you are a beginner in the field of Silverlight game development and don't know where to start, buy this book here.

Free Book!
I actually got two physical copies of the book and I like to give one away. If you want the book, just write a comment why you should get this gift. Make sure to include some form of contact information.

Photos, Photos, Photos - How To Save, Load And Iterate Pictures With Windows Phone 7

2010-07-19T19:50:00.003+02:00

The Windows Phone developer tools beta version was recently released and brought some important changes and also new features. For example the emulator is a lot better now and tombstoning was introduced. Tombstoning is an important concept to know when a Launcher or Chooser is being used in an app.
But this blog post is not about tombstoning, this post will show how to load an image from the picture library with the PhotoChooserTask. And most important how to save a picture to the library, which is not as obvious as the usage of the Task API. The last part will show how to get all images from the picture library without using the PhotoChooserTask.

Sample
I've built a Windows Phone 7 app that uses the WriteableBitmapEx library and lets the user draw on the WriteableBitmap surface. The app supports multi touch for drawing and the radius of the pen can be changed with the Slider control. An image from the phone's picture library can be chosen with the folder icon button and the floppy disk button saves the current image to the picture library. When the third button (download icon) is clicked, all pictures from the phone's library are loaded without further user interaction. The trash button clears the draw surface.
The video below demonstrates the features and shows how to use the app.
It would be great if anyone could try this sample app on a real device and give me some feedback. I'm especially interested how the multi touch drawing works.

Background music is Soft Shapes by Planet Boelex.

How it works
The raw touch points are handled through the Touch.FrameReported event. For each multi touch point a circle is being drawn with the WriteableBitmapEx' FillEllipseCentered method and a color map array that provides alternating colors:

private void Draw(IList<TouchPoint> points)
{
   // Check
   if (!isManipulating)
   {
      return;
   }

   // Init some vars
   var bmp = Bitmap;
   var w = bmp.PixelWidth;
   var h = bmp.PixelHeight;
   var r = Radius;

   // Draw
   for (int i = 0; i < points.Count; i++)
   {
      var p = points[i].Position;
      if (p.X < w && p.Y < h)
      {
         bmp.FillEllipseCentered((int)p.X, (int)p.Y, r, r, 
                                 ColorMap[(i + colorBase) % ColorMap.Length]);
      }
   }

   // Show
   Present();
}

Choose a photo
The PhotoChooserTask allows the user to select an image. The code for this task is pretty straight forward with the beta of the developer tools.
A member variable of the PhotoChooserTask is instantiated in the constructor and an event handler for the Completed event is attached.

public MainPage()
{
   // ...

   // Init chooser
   photoChooserTask = new PhotoChooserTask();
   photoChooserTask.Completed += PhotoChooserTaskCompleted;
   
   // ...
}

private void PhotoChooserTaskCompleted(object sender, PhotoResult e)
{
   if (e.TaskResult == TaskResult.OK)
   {
      // Load original image and invalidate bitmap so it gets newly rendered
      var bitmapImage = new BitmapImage();
      bitmapImage.SetSource(e.ChosenPhoto);
      Viewport.Source = bitmapImage;
      bitmap = null;
   }
}

The PhotoChooserTask.Show method is called when the corresponding Application Bar button was clicked:

private void ApplicationBarIconOpenButton_Click(object sender, EventArgs e)
{
   photoChooserTask.Show();
}

The Show method launches the Windows Phone photo app. The current version of the Windows Phone operating system only allows one application to run at the same time and therefore our app gets terminated when the Chooser is started. Here's where the concept of tombstoning comes into play.
After the user selected an image or pressed the back button, the Chooser's Completed event is raised. You might have noticed the black screen in the video that appears after the choose operation. As a result of the app termination, the Visual Studio debugging session is stopped. The Windows Phone 7 emulator detected that the app was started in a debug context and the emulator now waits at the black screen for re-attaching. So just go back to Visual Studio and hit F5 (Start Debugging), then the debugger is being re-attached and the app continues. Of course it's also possible to Start Without Debugging in Visual Studio.
The Chooser's Completed event provides a PhotoResult that contains the TaskResult, the OriginalFileName and the ChosenPhoto as a Stream. If the user hasn't cancelled the operation, the stream is used as the source of a BitmapImage which is then assigned to the WriteableBitmap draw surface (Viewport). The user can now make some nice drawings on the photo.

Save a picture
After the user created his masterpiece he probably wants to save it back to the picture library / photo album. But how could this be done? There's no PhotoSaveTask available in the Silverlight SDK. Fortunately the Windows Phone's XNA MediaLibrary comes to the rescue. Only a reference to the Microsoft.Xna.Framework assembly is needed to use it in our Windows Phone Silverlight application. To make this task a bit easier I wrote some reusable extension methods for the WriteableBitmap. These are located in the file WriteableBitmapMediaLibraryExtensions.cs from the source code download. The signatures look like this:

// Saves the WriteableBitmap encoded as JPEG to the Media library.
// The quality for JPEG encoding has to be in the range 0-100, 
// where 100 is the best quality with the largest size.
void SaveToMediaLibrary(this WriteableBitmap bitmap, string name, int quality);

// Saves the WriteableBitmap encoded as JPEG to the Media library 
// using the best quality of 100.
void SaveToMediaLibrary(this WriteableBitmap bitmap, string name);

The methods use the MediaLibrary's SavePicture method internally. The SavePicture method expects a stream or a byte array as parameter that contains an image encoded in the JPEG format. For the earlier CTP versions I wrote a custom JEPG encoding functionality that used some parts of my Silverlight JPEG encoding blog post and an adapted FJCore version. Fortunately things got a bit easier with the Windows Phone Tools beta release. The Microsoft.Phone assembly now comes with two WriteableBitmap extension methods called LoadJpeg and most important SaveJpeg. The SaveJpeg method expects the targetStream, the width and height of the target, the orientation which is not used at the moment and the quality in a range from 0 to 100. The width and height parameters are useful when a scaled version of the WriteableBitmap should be saved as JPEG.

The current bitmap surface is saved as JPEG when the disk floppy button was clicked:

private void ApplicationBarIconSaveButton_Click(object sender, EventArgs e)
{
   var name = String.Format("MediaLibSample_{0:yyyy-MM-dd_hh-mm-ss-tt}.jpg",
                            DateTime.Now);
   Bitmap.SaveToMediaLibrary(name);
}

Load all pictures
The last thing I'd like to cover in this post is the Pictures property of the MediaLibrary class. This property returns a collection of Picture instances. This class provides all the necessary meta information about the picture and streams of the image and thumbnail data.
I use this in the sample code to create a mosaic of all Windows Phone media library pictures:

private void ApplicationBarIconOpenAllButton_Click(object sender, EventArgs e)
{
   using(var mediaLib = new MediaLibrary())
   {
      // Pictures also includes saved pics, 
      // mediaLib.SavedPictures returns the same collection items
      var allPics = mediaLib.Pictures;

      // Combine the pics to a single WriteableBitmap mosaic
      var bmp = Bitmap;
      bmp.Clear();
      int x = 0;
      int y = 0;
      foreach (var picture in allPics)
      {
         // Load thumbnail stream to WriteableBitmap
         var wb = new WriteableBitmap(0, 0);
         wb.SetSource(picture.GetThumbnail());

         // Blit thumbnail to background bitmap
         var w = wb.PixelWidth;
         var h = wb.PixelHeight;
         bitmap.Blit(new Rect(x, y, w, h), wb, new Rect(0, 0, w, h));
         x += w;

         // Check bounds and move to next row
         if (x >= bitmap.PixelWidth)
         {
            x = 0;
            y += h;
         }
         // Bitmap filled
         if (y >= bitmap.PixelHeight)
         {
            break;
         }
      }
   }

   Present();
}

The GetThumbnail method returns the stream of an image thumbnail sized 99 x 99 pixels. This bitmap is then combined with the surface bitmap by using the WriteableBitmapEx' Blit method.
Beside the Pictures property the MediaLibrary also has the SavedPictures property, but I encountered that the SavedPictures collection returns all pictures in the emulator and not only saved pictures like the name and documentation implies. But as a little bird told me, the SavedPictures property works as expected on a real device.
The MediaLibrary has some more members that might become interesting in the future, esp. when combined with the MediaPlayer class.

Conclusion
This blog post covered the complete workflow of loading, manipulating and saving a picture to the Windows Phone's picture library / photo album. I also showed how to load all images from the library and mentioned some gotchas.

Source code
Download the complete Visual Studio 2010 Windows Phone solution from here.

The Ultimate Gift Card - Two Visual Studio 2010 Ultimate MSDN Subscriptions To Give Away!

2010-07-05T20:03:00.006+02:00

It's over, I picked two people from the comments that deserve the MSDN Subscription cards the most. Thanks!

I recently got three Visual Studio 2010 Ultimate MSDN Subscriptions cards from Microsoft. They are valid for one year, include the whole Ultimate features and can be used for development and testing purposes. Ultimate includes Visual and Expression Studio,Windows, Office, Windows Server, SQL Server, TFS, all other server products, ... and even some Azure benefits are packed in. Only the support and the MSDN magazine are excluded.
The one year Ultimate subscription is worth approximately $12,000! And the best thing, I have two cards left I can give away. What a great gift from Microsoft!

I want to give my two remaining cards to the Windows developer community. If you want one, just write a comment why you should get this great gift and make sure to include some form of contact information. Did I mention that these are full Ultimate subscriptions worth $12,000?
I will prefer people who make significant contributions in the .Net open source world. Of course, your chance is even better if these contributions are for Silverlight or Windows Phone 7.
After July 20th 2010 I'll pick two people that deserve the cards the most.

Here's a photo of the package I got. Yeah, it's real! And free, as in free beer!

Thanks a lot Microsoft!

Interview on .Net Rocks!

2010-06-07T19:25:00.000+02:00

About a month ago Richard Campbell asked me if I'd like to give an interview about SLARToolkit on the .Net Rocks talk show. Of course I agreed right away. .Net Rocks! is the largest .Net developer podcast out there and Carl Franklin and Richard Campbell always have great guests. It's a real honor for me to have been on show #564.
We mainly talked about the Silverlight Augmented Reality Toolkit (SLARToolkit), but also about some of my other projects and articles.
You can download the podcast in various formats from here. I hope you like it. Also make sure you subscribe to .Net Rocks!

Push and Pull - Silverlight Webcam Capturing Details

2010-06-03T21:39:00.003+02:00

Photo (CC) by MShades

It's not a secret that one of my favorite Silverlight 4 features is the webcam support and I already played endless hours with it. There are many blog posts out there demonstrating how to use the webcam and how to take a screenshot with the CaptureImageAsync method. Only a few cover the VideoSink.

This blog post will show how to use the webcam, the CaptureImageAsync method and also how to implement and use the VideoSink. But most important I'll cover what the differences between the CaptureImageAsync and VideoSink are and when to use which.

Silverlight Webcam 101

The Silverlight 4 webcam API is pretty easy to use and just a few lines of code are needed to show a webcam video stream on screen.
Silverlight's CaptureSource class provides the webcam stream that is used as the source of a VideoBrush, which in turn fills a Rectangle with the video feed from the webcam. It's also possible to use any other Shape with a Fill property.
The CaptureDeviceConfiguration class can be used to retrieve a list of all installed video and audio devices on the system. Most of the time it's sufficient to use the GetDefaultVideoCaptureDevice to get the default device. The user can specify the default video and audio devices with the Silverlight configuration; he or she only has to press the right mouse button over the Silverlight application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

Silverlight Webcam / Mic configuration dialog.

The following C# code initializes the webcam (captureSource) in the Loaded event of the page and fills a rectangle (Viewport) with a VideoBrush:

// Member variable (webcam reference)
CaptureSource captureSource;

private void UserControl_Loaded(object sender, RoutedEventArgs e)
{
   // Initialize the webcam
   captureSource = new CaptureSource();
   captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

   // Fill the Viewport Rectangle with the VideoBrush
   var vidBrush = new VideoBrush();
   vidBrush.SetSource(captureSource);
   Viewport.Fill = vidBrush;
}

Now that the webcam is initialized, the streaming can be started. This is done in an event handler of a Button because RequestDeviceAccess has to be called from an user initiated event. Otherwise it would be possible to start the webcam without the user's permission. Of course nobody wants to experience something like what happened to the students with their MacBooks provided by their high school.

Here's the C# code:

private void StartButton_Click(object sender, RoutedEventArgs e)
{
   // Request webcam access and start the capturing
   if (CaptureDeviceConfiguration.RequestDeviceAccess())
   {
      captureSource.Start();
   }
}

This will open a webcam permission dialog asking the user for the device access. This setting is consent and Silverlight can remember if the user has previously allowed that certain application.

Webcam permission dialog

Capturing The Webcam
There are two different approaches to capture the webcam in Silverlight. The CaptureSource's CaptureImageAsync method and CaptureImageCompleted event provide a snapshot on demand and can be considered as a pull-based technology. A custom VideoSink implementation on the other hand constantly gets the raw stream from the webcam and can be considered as a push-based approach.

Pull: CaptureImageAsync Webcam Capture
When the CaptureSource's CaptureImageAsync method is called an asynchronous capturing task is started. After the snapshot is completed, the CaptureImageCompleted event is fired. The event provides a WriteableBitmap as EventArgs.

The following C# code should be added after the captureSource initialization code above:

// Wiring the CaptureImageCompleted event handler
captureSource.CaptureImageCompleted += (s, e) =>
{
   // Do something with the camera snapshot
   // e.Result is a WriteableBitmap
   Process(e.Result);
};

Another button is used to start the asynchronous capturing:

private void SnapshotButton_Click(object sender, RoutedEventArgs e)
{
   // CaptureImageAsync fires the CaptureImageCompleted event
   captureSource.ImageCaptureAsync();
}

Push: VideoSink Webcam Capture
The other capturing approach constantly pushes every frame from the webcam into a VideoSink. The abstract VideoSink class has four methods that have to be implemented in an own subclass in order to use it.

The basic set-up of a custom VideoSink looks like this:

// MyVideoSink is derived from Silverlight's VideoSink
public class MyVideoSink : VideoSink
{
   VideoFormat vidFormat;
   
   // Could be used to initialize a container for the webcam stream data
   protected override void OnCaptureStarted() { }
   
   // Could be used to dispose a container for the webcam stream data
   // or to write a header of a video file format
   protected override void OnCaptureStopped() { }

   // Is called when the VideoFormat was changed
   protected override void OnFormatChange(VideoFormat videoFormat)
   {
      this.vidFormat = videoFormat;
   }

   // Is called every time the webcam provides a complete frame (Push)
   protected override void OnSample(long sampleTime, long frameDuration, 
                                    byte[] sampleData)
   {
      // Process the webcam snapshot 
      // sampleData contains the raw byte stream
      // according to the videoFormat from OnFormatChange
      Process(sampleData, this.vidFormat);
   }
}

The following C# code initializes MyVideoSink with the webcam. It should be added after the captureSource initialization code above:

// Wire the VideoSink and the webcam together
var sink = new MyVideoSink { CaptureSource = captureSource };

The VideoSink's OnCaptureStarted and OnFormatChange are raised after the captureSource.Start() method was called. The OnSample method is constantly called as long as the webcam is activated. The actual interval OnSample will be called is defined in VideoFormat.FramesPerSecond which is provided through the OnFormatChange method. The OnCaptureStopped is raised after the captureSource.Stop() method was called.

Push vs. Pull
Obviously the two approaches have different characteristics.

Pull: CaptureImageAsync

Simple to use
Provides a WriteableBitmap
CaptureImageCompleted is raised on the UI thread (no Dispatcher.BeginInvoke necessary)
Only one PixelFormat (ARGB32)
System resources aren't constantly used
Pull: Samples on demand

Push: Custom VideoSink

Direct usage of the raw byte stream
Less overhead
OnSample is called on a background thread
Automatically called for every frame
Might support more than one PixelFormat in the future
More information like frame number and duration (accurate sample times)
Slightly faster than CaptureImageAsync if every frame is needed
Push: Constant sampling

Please note that the CaptureImageAsync method can also be called periodically. Thereby it's possible to get a snapshot in a defined interval which might be faster than using a VideoSink that fires every 30 or even 60 frames per second (fps).

The following C# code calls CaptureImageAsync every 100 milliseconds, which means every 10 fps a snapshot is taken:

var dispatcherTimer = new DispatcherTimer();
dispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 100); // 10 fps
dispatcherTimer.Tick += (s, e) =>
{
   // Process camera snapshot if started
   if (captureSource.State == CaptureState.Started)
   {
      // CaptureImageAsync fires the CaptureImageCompleted event
      captureSource.CaptureImageAsync();
   }
};
// Start the timer
dispatcherTimer.Start();

Conclusion
Both approaches are helpful for different scenarios. The pull-based CaptureImageAsync method is useful for taking single snapshots, whereas a push-based custom VideoSink can be used for capturing complete sequences and encoding the webcam stream.

Filled To The Bursting Point - WriteableBitmapEx 0.9.5.0

2010-06-01T19:42:00.002+02:00

We're slowly getting to the first feature complete release of the WriteableBitmapEx library. The last version (0.9.0.0) brought the parametric curves, optimizations and other features. This new version 0.9.5.0 focuses on filling routines, transformations, bug fixes and more optimizations.
The new Fill* extension methods are equivalent to the Draw* shape functions. The FillRectangle and FillEllipse methods use specialized implementations to get the best performance. FillPolygon and the other functions use a simple scanline conversion algorithm with the even-odd-rule. The implemented algorithm supports concave and convex shapes.

New features

Fast FillRectangle method that uses Buffer.BlockCopy.
Optimized FillEllipse and FillEllipseCentered functions to draw a filled ellipse / circle.
FillPolygon, FillTriangle and FillQuad methods that use a scanline conversion algorithm.
FillCurve, FillCurveClosed to draw a filled Cardinal spline similar to the GDI+ API.
FillBeziers draws a filled Cubic Beziér spline.
Resize method with support for Bilinear and Nearest neighbor interpolation.
Fast Crop method to cut out a portion of the WriteableBitmap.
GetPixel and GetPixeli methods to get the color as Color struct at a specified x, y coordinate.
FromResource method to load an image from the application's resource only by passing the relative path without the need of the full Pack URI syntax. Example: "Data/flower2.png" instead of "MyAssemblyName;component/Data/flower2.png".
Optimized Clear(Color) method that uses Buffer.BlockCopy.
Renamed BlendMode "AlphaBlend" to "Alpha".
Fixed the clipping of the DrawEllipse methods.
Fixed some missing alpha pre-multiplications.
Other minor tweaks and bug fixes.
Updated the solutions to Microsoft Visual Studio 2010.

The code listing on the project's CodePlex site was updated to demonstrate how to use the new methods.

Live
As usual I also wrote a new sample application that shows the Fill* methods in action. The sample starts with a real-time demo that animates the Cardinal spline's tension of the FillCurveClosed method, plus some random animated filled ellipses. The sample also contains a static page showing some of the possible filled shapes.

Open the sample in a new page.

Go and grab it
The WriteableBitmapEx library is hosted at CodePlex. You can find the new binary release here and the samples in the Subversion repository.

Coding4Fun - Introduction to Silverlight and WPF Pixel Shaders

2010-05-26T19:26:00.003+02:00

My new article for Microsoft's Coding4Fun site is live. It's my second article for Coding4Fun after my first Silverlight Face Detection article. The article explains how to write pixel shaders for Silverlight and WPF, what tools should be used, and how to work with the tools. Furthermore, it shows how to build an extensible Silverlight shader application with the help of the Managed Extensibility Framework (MEF)
The application not only comes with the two shaders that are implemented in the article, it also contains three other shaders I’ve written before. The complete source code is licensed under the Ms-PL and can be downloaded from the CodePlex site.

SLARToolkit Beginner's Guide

2010-05-17T20:29:00.001+02:00

Since I released SLARToolkit I often get similar, basic questions. This is a sign that the SLARToolkit documentation and the samples might be too complex for a beginner. That's why I wrote an easy step by step Beginner's Guide which hopefully closes the gap. The tutorial shows how to build the simplest Silverlight Augmented Reality application possible.

The guide is part of the SLARToolkit documentation at CodePlex. Please read the Beginner's Guide here and let me know your feedback so I can improve it further.

Silverlight 4 Up and Running

2010-04-15T23:02:00.005+02:00

Woohoo! The final Silverlight 4 runtime and the developer tools for Visual Studio 2010 were released today and exactly three years ago the name Silverlight was officially introduced. A lot of great things happened in the Silverlight world since then and we now have a superb RIA platform at hand.

To get started with the Silverlight 4 development you only need Visual Studio 2010 and the Silverlight 4 Tools for Visual Studio. Don't forget, it's all free with the Visual Studio 2010 Express edition! The end-user runtime is available for Windows and Mac. As usual, Tim Heuer wrote a nice blog post about the Silverlight tools.

Past
The APIs haven't changed much since the last release candidate (RC) and exactly one month ago I wrote a summary of all the new features that were added since the Silverlight 4 Beta to the Silverlight 4 RC and the breaking changes. The post also contained some details about the CaptureSource's changed capture usage pattern. So if you used the Silverlight 4 Beta and skipped the RC, this post might be helpful.

Present
I installed the Visual Studio 2010 RTM build on Monday after it was released at Devconnections, the final Silverlight 4 Tools for Visual Studio today and recompiled all my Silverlight 4 samples and open source projects. It all worked like a charm and even on this machine were I had several betas of all the tools installed before. Kudos to the Visual Studio and the Silverlight team for making such great products.
My relevant Silverlight 4 blog posts, the SLARToolkit and the FaceLight CodePlex open source projects are now up to date.

Future
The first beta of the next Silverlight version will most certainly see the light this year. Make sure to vote or suggest new features at the silverlight.mswish.net site. The most important feature suggestion for me is the GPU accelerated 3D support that I suggested a while ago and which was consolidated into this suggestion. If you agree with me, you should vote for it.

Have fun with Silverlight 4!

Microsoft Silverlight MVP Award

2010-04-01T21:38:00.002+02:00

Today, I got an exiting Email from Microsoft:

Congratulations! We are pleased to present you with the 2010 Microsoft® MVP Award! This award is given to exceptional technical community leaders who actively share their high quality, real world expertise with others. We appreciate your outstanding contributions in Silverlight technical communities during the past year.

I hope it wasn't an April Fools' joke. ;-)

It's a real honor for me to get the Silverlight MVP award. All the Silverlight MVPs I know are truly outstanding and I can hardly believe I'm considered to be one of them now. It's even more surreal that I'm the first Silverlight MVP from Germany and the second from the DACH region after the excellent Laurent Bugnion.

I would like to thank Microsoft, the great Silverlight community and especially all that nominated me for the MVP award. It's needless to say that I will continue my work with Silverlight to provide more useful content and I hope I'll be able to satisfy the high expectations.

Thank you all!

Coding4Fun - Silverlight Real Time Face Detection

2010-03-25T20:53:00.006+01:00

A while ago Clint Rutkas (yes, the guy that built the awesome MIX T-Shirt cannon) asked me if I could write an article for Microsoft's Coding4Fun site. I felt honored and of course I agreed. Today I'm proud to announce that my first Coding4Fun article was published.

The article demonstrates how to implement a simple facial recognition system using Silverlight 4’s new webcam feature. The method I've implemented searches for a certain sized skin color region in a webcam snapshot. This technique is not as perfect as a professional computer vision library like OpenCV and the Haar-like features they use, but it runs in real time and works for most webcam scenarios.

The complete source code for the article is available at CodePlex and licensed under the Ms-PL.
If you like the article, I’ll write a follow-up that might cover a more stable (better) segmentation approach, how to optimize the performance further and more.

Update 04-06-2010
The article was translated into Russian and is available at the Russian Coding 4 Fun site.

Update 04-15-2010
Updated the application to the final Silverlight 4 RTW build.

Let it ring - WriteableBitmapEx for Windows Phone

2010-03-17T21:02:00.066+01:00

Actually I never heard a Windows Phone prototype ringing yet, but that's not the point of this blog post.

On Monday, March 15, 2010 Microsoft not only released the Silverlight 4 Release Candidate at MIX, they also made the first CTP of the Windows Phone Developer Tools available. The best news is, Silverlight is THE technology for Windows Phone 7 series development and the SDK doesn't cost $ 100, no, it's completely free!
Of course I downloaded and installed the CTP right away and played around with it. First I was a bit disappointed after I realized that custom pixel shaders are not supported (at the moment) and, correct me if I'm wrong, real time camera access is not built in either, which is essential for a nice SLARToolkit Windows Phone version. BUT, it's only the first CTP and at least we have the WriteableBitmap at hand for some procedural graphics fun.

A while ago I started the WriteableBitmapEx project to make the life a bit easier when working with the WriteableBitmap. And now what would be more natural than porting it to the Windows Phone platform.
So I fired up Visual Studio 2010 for Windows Phone and set up the first real Windows Phone Silverlight project pretty quickly. After I included the original WriteableBitmapEx source code files I started the build. And it just worked, without any errors. Wow, that was an easy port.
Sure, Scott Guthrie mentioned in his keynote that the Windows Phone fully supports Silverlight, but there are actually some differences to the desktop version. Nevertheless the first developer experience was just impressive and WriteableBitmapEx for Windows Phone is actually just another build target and not a real port of the original Silverlight library.

Sample
Unfortunately the current CTP is a bit slow when it comes to real time graphics, but there is hope and the Release Notes say:

"Per frame callbacks (i.e. handling the CompositionTarget.Rendering event) should be used sparingly in this release due to less than optimal performance."

That's why I used the interactive parts of the latest WriteableBitmapEx curve sample to make a Windows Phone sample. All this was done in less than hour and most of the time was spent to arrange the controls and play with the great Windows Phone emulator.
Below is a video that shows the demo running in the emulator. Please note how the phone's back button is used to remove the last control point from the list.

Background music is Loell by mosaik.

How it works
As I mentioned above parts of the code from the latest WriteableBitmapEx curve sample were simply re-used. Furthermore I rearranged the controls for the phone's landscape mode.
The functionality of the phone's back button is implemented by adding an event handler to the Page's BackKeyPress event. The used event handler:

void PhoneApplicationPage_BackKeyPress(object sender, CancelEventArgs e)
{
    // Remove last control point from the list
    if (points != null && points.Count > 0)
    {
        points.RemoveAt(points.Count - 1);
    }

    // Refresh image
    Draw();

    // Back is handled here. Prevent that the phone returns to the home screen
    e.Cancel = true;
}

Source code
From now on you will find a Visual Studio solution and a project file for the Windows Phone version in the WriteableBitmapEx' source code repository. Starting with the next release I will also provide a ready to use Windows Phone build.

Woo Hoo!
Silverlight is becoming more and more a full multi platform technology and all Silverlight developers are now Windows Phone developers too. It's an exciting time to be a .Net developer. Awesome!

Update 03-18-2010
I changed the size of the control points to the appropriate minimum touch target size as required in the Windows Phone UI guidelines. According to it the minimum touch target size has to be 9 mm / 34 pixel and the visual size 60-100% of the touch target size.

The Silverlight 4 Release Candidate

2010-03-15T19:33:00.007+01:00

Scott Guthrie et al. announced many awesome new things during the keynotes at Microsoft's MIX10 conference. One of it was the Silverlight 4 Release Candidate which replaces the beta version and is now available for download. Beside all the great new features that were added, it's now finally possible to use the Visual Studio 2010 RC for Silverlight 4 development. Kudos to the Silverlight team for catching up with Visual Studio 2010 and all the cool new features they implemented.

The final Silverlight 4 RTW version will be available next month (April 2010). Additonaly the first CTP of the Winows Phone SDK was released and Silverlight is THE platform for Windows Phone 7 series development and it's free! Silverlight for Symbian is also now available for Nokia S60 5th Edition devices like the N97. So Silverlight is becoming more and more a full multi platform technology. Awesome!

This blog post focuses on the Silverlight 4 RC and contains a summary of all the new features that were added since the Silverlight 4 Beta to the Silverlight 4 RC and the breaking changes that naturally come along with betas. Plus some details about the CaptureSource's new capture usage pattern.

New Silverlight 4 RC features

Rich Text

RichTextBox rename
Text position and selection APIs
XAML property for serializing text content
XAML clipboard format
FlowDirection support on Runs
"Format then type" support
Thai / Vietnamese / Indic support
UI Automation Text pattern

Networking

UploadProgress support (Client stack)
Caching support (Client stack)
Sockets security restrictions removal (Full Trust)
Sockets policy file retrieval via HTTP
Accept-Language header

Out of Browser (Full Trust)

XAP signing
Silent install and emulation mode
Custom window chrome
Better support for COM Automation
Cancelable shutdown event
Updated security dialogs

Media

Pinned full-screen mode on secondary display
Webcam / Mic configuration preview
More descriptive MediaSourceStream errors
Content & Output protection updates
Updates to H.264 content protection (ClearNAL)
Digital Constraint Token
CGMS-A
Multicast
Graphics card driver validation & revocation

Graphics and Printing

Hardware accelerated perspective transformations
Ability to query page size and printable area
Memory usage and performance improvements

Data

Entity-level validation support of INotifyDataErrorInfo for DataGrid
XPath support for XML

Parser

New architecture enables future innovation
Performance and stability improvements
XmlnsPrefix & XmlnsDefinition attributes
Support setting order-dependent properties

Globalization & Localization

Support for 31 new languages
Arabic, Hebrew and Thai input on Mac
Indic support

Update to DeepZoom code base with hardware acceleration
Support for Private mode browsing
Google Chrome support (Windows)
FrameworkElement.Unloaded event
HTML Hosting accessibility
IsoStore performance improvements
Native hosting performance improvements (e.g. Bing Toolbar)
Consistency with Silverlight for Mobile APIs and Tooling
System.Numerics.dll
Dynamic XAP support (MEF)
Frame / Navigation refresh support

The new webcam and microphone configuration preview is pretty neat. The user can now select the default webcam & microphone device really comfortable and we only need to use CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice() or CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice() to get the selected default video and audio device.

The webcam permission dialog is now consent and can remember if the user has previously allowed that certain application. Tim Heuer's blog post has more details about this handy new feature.

Breaking Changes since the Silverlight 4 beta
The first and most important take away: As usual all Silverlight 4 beta applications have to be updated to the RC and recompiled, otherwise the new RC runtime won't execute them.

Renamed RichTextArea to RichTextBox
Changes to DRM API
TextSelection.SetPropertyValue is now named .ApplyPropertyValue
INotifyDataErrorInfo moved to core (System.Windows.dll)
WebBrowser.ScriptNotify signature change
NotificationWindow.Visible and NotificationWindow.Closed were changed
Webcam and output protection change
COM Interop API changes (name and namespace)
RichTextBox.TextWrapping default value changed
Binding declarations in XAML where the target is a DependencyObject
HtmlBrush renamed to WebBrowserBrush

The official breaking changes document contains all the details and Koen Zwikstra (@kozw) has made some useful diff lists.

Changed usage pattern for AsyncCaptureImage
The biggest difference in the webcam API besides the renaming of some enums, properties and other members is the change of the AsyncCaptureImage method which was renamed to CaptureImageAsync. Additionally the usage pattern of the CaptureImageAsync method was updated to the standard .Net / Silverlight asynchronous event pattern.

Before the Silverlight 4 RC, AsyncCaptureImage was used like this:

// Initialization
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

// Capture image
captureSource.AsyncCaptureImage(bitmap =>
{
   // Process camera snapshot
   Process(bitmap);
});

The new RC CaptureImageAsync usage pattern:

// Initialization
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
// Wiring the CaptureImageCompleted event handler
captureSource.CaptureImageCompleted += (s, e) =>
{
   // Process camera snapshot (e.Result is a WriteableBitmap)
   Process(e.Result);
};

// CaptureImageAsync fires the CaptureImageCompleted event
captureSource.ImageCaptureAsync();

Although the pre-RC pattern was smaller and easier, it's overall more consistent now and I welcome the changes.

Blog.SL4++
I already updated SLARToolkit and the samples and I will continue to update some of my Silverlight 4 beta samples during the next days. Furthermore I will make a Silverlight 4 RC build of the Matrix3DEx sample that should take advantage of the new hardware accelerated perspective transformations.

Poll Results: Do you block Google Analytics tracking in your browser?

2010-03-10T20:33:00.003+01:00

I recently encountered a large difference between the Google Analytics statistics of my website and the server logs. This raised the question in me how reliable the Google Analytics data really is and I asked you to take part in a small survey: Do you block Google Analytics tracking in your browser?. The poll is closed now and here are the results.

The poll got 267 votes total and 68 people voted "Yes, I block Google Analytics." 199 voted for "No, I don't block Google Analytics." This means 25.47% of the participants block Google Analytics in their browser.

You can now use a simple formula to get the real world Google Analytics statistics (at least for my visitors - the best of all). :-)

GAreal = GAreport * 1.25468164794007490636704119850187

Big thanks to all that helped me and took part in the survey!

SLARToolkit - Silverlight Augmented Reality Toolkit

2010-02-24T21:05:00.009+01:00

In my last blog post I've preannounced SLARToolkit: The open source Silverlight Augmented Reality Toolkit. Today I'm proud to announce that SLARToolkit is now available at CodePlex.
The aim of SLARToolkit is to make Augmented Reality with Silverlight as easy as possible so that everyone will be able to make Silverlight Augmented Reality applications. Therefore the SLARToolkit release not only includes the full Balder sample you might know from this blog post, it also comes with a brand new sample that uses Silverlight's 3D projection capabilities through my Matrix3DEx library to project different UIElements on the marker.

The SLARToolkit project description from the CodePlex site:

SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. SLARTookit is based on the established NyARToolkit and ARToolkit. SLARToolkit uses a dual license model and could be used for open or closed source applications under certain conditions. See the License page for details.

Features:

Direct Support for Silverlight's CaptureSource
Flexible through a generic WriteableBitmap detector
Multiple marker detection
Simple black square markers
Custom markers
Real time performance
Easy to use
Documentation
Based on established algorithms and techniques
Uses the Matrix3DEx library

Live
A webcam and at least the Silverlight 4 runtime must be installed to run the sample. It's available for Windows and Mac. Alternatively there is also a new video of the Silverlight projection sample embedded below.
If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Open the sample in a new page.

Simply press the "Start Fun" Button to start the webcam. Use the ComboBox to select a different UIElement that will be mapped on the marker. The size and the orientation can be changed with the "Scale" and "Rotation" Sliders. The "Threshold" Slider controls the threshold for the marker detection. The "Flip x-axis" Checkbox could be used to flip the video (the webcam output is mirror-reversed by default).
If you click the "Start Fun" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

Video
I've recorded a short video of the new sample with Expression Encoder's Screen Capture feature. Please keep in mind that the screen recording software eats up a lot of resources while recording and that the actual frame rate is way better.

Background music is leandi by mosaik.

How it works
This sample uses the SLARToolkit CaptureSourceMarkerDetector to detect the markers in each webcam frame. The detection result contains a transformation matrix for each found marker which is then used to transform the selected UIElement. The tracked marker region is highlighted with a red quad through the WriteableBitmapEx library.

Download it, build your app and augment your reality

The open source SLARToolkit library and the samples are hosted at CodePlex. If you have any comments, questions or suggestions don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun with the library and please keep me updated if you use it anywhere so I can put a link on the project site.

Update 03-15-2010
Updated to the Silverlight 4 Release Candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

SLARToolkit Preannouncement

2010-02-15T20:41:00.013+01:00

SLARToolkit has been released. Please read this blog post.

In my latest blog posts, the comments and on Twitter I hinted at on one of the projects I'm currently working on: SLARToolkit - The Silverlight Augmented Reality Toolkit. My aim for SLARToolkit is to make Augmented Reality with Silverlight as easy as possible, fast and open source. With this everyone will be able to make Silverlight Augmented Reality applications like that one. SLARToolkit is based on NyARToolkit which is a extented port of the famous ARToolkit. I'm almost done with the implementation and in contact with ARToolworks, which are holding the license for ARToolkit, to get a license agreement. I hope I'll be able to release SLARToolkit within the next two or three weeks at CodePlex. Please stay tuned.

Two weeks ago Julien Dollon contacted me and asked if I could help him with an Augmented Reality demo for his Silverlight 4 talk at Microsoft TechDays 2010 in Paris. Since I'm a nice guy, I've built a special SLARToolkit version for him and send it along with the source code of the SLARToolkit Silverlight projection sample. The sample uses my Matrix3DEx library to project UIElements on the markers. Actually the Matrix3DEx library has its roots there.
Julien altered the sample a bit and showed it at MS TechDays 2010 during his Silverlight 4 talk last week. The presentation was recorded and the official video is available, but you can also watch Julien's self recorded video of the session. The SLARToolkit demo starts at 39:02.

And the official screen cast:

Rounder, Faster, Better - WriteableBitmapEx 0.9.0.0

2010-01-28T19:08:00.004+01:00

A lot of things happened since the release of the last WriteableBitmapEx version. I used the time I have to wait for a third party [spoiler] until I could release my Silverlight Augmented Reality framework SLARToolkit [/spoiler] to work on the WriteableBitmapEx project. Beside some optimizations and smaller extensions, the main addition in the new version is the support for parametric curves. I implemented the cubic Beziér spline and the Cardinal spline. The Cardinal spline is not part of Silverlight's and WPF's Geometry nor shape classes, but the DrawCurve method is part of GDI+ and many people appreciate the characteristics of cardinal splines. Fortunately the WriteableBitmapEx library fills this gap now for the WriteableBitmap.

New features

Draw methods for parametric curves: Cubic Beziér splines and the Cardinal splines similar to the GDI+ API.
Clone method to copy the whole WriteableBitmap to a new instance.
Boundary check for the Draw* (Shape) methods. The coordinates are now clipped to the WriteableBitmap's size.
A new, even faster DrawLine method overload that uses the bitmap's pixels int array, width and height directly as parameters. This method should be used if many lines are drawn to the same WriteableBitmap.
Optimization of the byte array conversion methods using the Buffer.BlockCopy method. This improvement was suggested in this comment.
WriteTga method to write a WriteableBitmap as a TGA image to a stream. The original method was provided in this blog post.
Fast path for Blitting using the Buffer.BlockCopy method if the BlendMode is None and the bitmap should not be tinted. The result is a performance boost by factor 3.5 compared to traditional loop iteration. This improvement was suggested in this CodePlex discussion.
Adam Kinney contributed two new blend modes for the Blit method: Multiply and Mask. He used these in his cool torn photo effect.
A new method that takes a function as parameter and iterates over each pixel and sets its color. The method is called ForEach and has two overloads. Usage example:
writeableBmp.ForEach((x, y, color) => Color.FromArgb(color.A, color.R / 2, (byte)(x * y), 100));
The method was suggested in this CodePlex discussion.

The code listing on the project's CodePlex site was updated to show the usage of the new methods.

Live
I also wrote a new sample application that shows the spline methods in action. The sample starts with a demo animation that uses the Cardinal spline DrawCurve method to draw an artificial plant that grows procedurally. The other part of the sample is interactive and allows to draw and manipulate Beziér and Cardinal splines with the mouse.

Open the sample in a new page.

Uncheck the "Growing plant demo" Checkbox to change to the interactive mode where you can add new or move curve control points with the left mouse button. Press the DEL key while a control point is selected to remove it. The used curve type can be changed with the Radio Buttons. A cardinal spline needs at least 3 points and a cubic Beziér 4 points, a start point, the control point 1, the control point 2 and the end point.The Slider controls the tension of the Cardinal spline. The tension defines the shape of the curve and is usually between 0 and 1. A value of 0 would be a straight line.
Uncheck the "Points" Checkbox to hide the control points and press the Clear Button to remove all control points. Click the Save Button to write the WriteableBitmap to a TGA image file.
The other controls should be self-explaining.

Get it while it's hot
The WriteableBitmapEx library is hosted at CodePlex. You can find the new binary release here.

Do you block Google Analytics tracking in your browser?

2010-01-25T20:30:00.019+01:00

Recently I checked the server logs for my website and compared the statistics with the Google Analytics data for the timespan. I expected that the page views and visits won't be the same, but I haven't expected such a large difference I encountered.
Just to make it clear, I don't use Google Analytics for evil, I'm not Google and I'm not interested in advertising or web marketing, I'm just curious what my visitors (you) are looking for and what caught the interest, so I can provide more content in this field.
I know some people that block Analytics tracking in their browser because of the Google octopus, or disabled Java Script at all.

The difference between the Google Analytics statistics and the server logs raised the question in me how reliable the Google Analytics data is when several people block it in their browser. I started to search for some block statistics, but couldn't find any useful information and I also think the actual blocking-rate does crucially depend on the website's audience and used technology. That's why I created a little poll at twtpoll.com and asked my Twitter friends to vote. Big thanks to all that already voted, but not all of you are on Twitter, so I like to ask here too:

Do you block Google Analytics tracking in your browser?

I will publish the results here on my blog after the poll is over on Feb 28, 2010. Thanks for taking your time.

Update 03-10-2010
The results are now available.

Matrix3DEx 1.0 - When PlaneProjection is not enough

2010-01-18T20:13:00.179+01:00

A new year has begun and I started a new open source project at CodePlex. It's called Matrix3DEx and is an extension library for the Silverlight Matrix3D struct. Most of the functionality I implemented in Matrix3DEx was originally required for another open source project I'm currently working on (hint, hint), and as you might know I also like to extend Silverlight's graphics functionality in a reusable manner. That's why I decided to extract the Matrix3D code into a separate open source project, add some more useful methods, document it and write a sample.

The Matrix3DEx project description from the CodePlex site:

The Matrix3DEx library is a collection of extension and factory methods for Silverlight's Matrix3D struct. The Matrix3D struct represents a 4x4 matrix that is used in combination with the Matrix3DProjection to apply more complex semi-3D scenarios to any UIElement than are possible with the simple PlaneProjection. This makes it possible to apply arbitrary model transformation matrices and perspective matrices to Silverlight elements.
The Matrix3D struct is very minimalistic and has only a few members. The Matrix3DEx library tries to compensate that with extension and factory methods for common transformation matrices that are easy to use like built in methods.

Features:

Factory methods

Translation, scaling and rotation around x, y, z or any defined axis
Perspective field of view and orthographic projection
Camera (look-at) with position, target and up vector
Support for left-handed and right-handed coordination systems

Extension methods

Calculation of the matrix' determinant
Matrix transpose
SwapHandedness to change from right-handed to left-handed coordination system and vice versa
Dump of the values row by row into a formatted string

Math helper methods

Angle conversion from degrees to radians and vice versa

Some use cases where Silverlight's semi-3D projection is needed can be implemented with the PlaneProjection, but there are also some scenarios where the PlaneProjection wouldn't work or only with a lot of effort. One example is the usage of a physics library that returns a matrix for an object. Traditional matrix transformations are an elegant alternative for such cases and the Matrix3DEx library has factory and extension methods for all the common matrices.

Live

Open the sample in a new page.

The application loads a bunch of photos asynchronously and randomizes the position vector of each. The sample uses most of the Matrix3DEx features and has some Sliders and CheckBoxes to change the parameters. Uncheck the "Animate" CheckBox to disable the camera movement and click on an Image to select it. The translation, scaling and the rotation matrices of the selected element can be changed with the corresponding Sliders. You can move the camera, change the target and the "Field Of View" with the other Sliders or fix the camera target at the selected element.
The basic functionality of the sample might also be done with the PlaneProjection, but a separate look-at matrix (camera) simplifies the code a lot and makes it easier to read. Other things like a custom field of view are not possible with the PlaneProjection.

How it works
The sample uses a DipatcherTimer to call the Update method every 100 milliseconds.
The core of the Update method:

// Create global transformations
var vw = Viewport.Width;
var vh = Viewport.Height;
var invertYAxis = Matrix3DFactory.CreateScale(1, -1, 1);
var translate   = Matrix3DFactory.CreateTranslation(TranslateX, 
                                                    TranslateY,
                                                    TranslateZ);         
var rotateX     = Matrix3DFactory.CreateRotationX(MathHelper.ToRadians(RotateX));
var rotateY     = Matrix3DFactory.CreateRotationY(MathHelper.ToRadians(RotateY));
var rotateZ     = Matrix3DFactory.CreateRotationZ(MathHelper.ToRadians(RotateZ));
var scale       = Matrix3DFactory.CreateScale(ScaleX, ScaleY, ScaleZ);
var lookAt      = Matrix3DFactory.CreateLookAtLH(CameraX, CameraY, CameraZ,
                                                 CameraLookAtX, 
                                                 CameraLookAtY, 
                                                 CameraLookAtZ);
var viewport    = Matrix3DFactory.CreateViewportTransformation(vw, vh);
var fieldOfView = MathHelper.ToRadians(FieldOfView);
var projection  = Matrix3DFactory.CreatePerspectiveFieldOfViewLH(fieldOfView, 
                                                                 vw / vh,
                                                                 NearPlane,
                                                                 FarPlane);

// Transform all elements
var selectedMatrix = Matrix3D.Identity;
foreach (var elem in this.Elements)
{
   // The UIElement
   var e = elem.Element;

   // Create transformation matrices for UIElement
   var centerAtOrigin = Matrix3DFactory.CreateTranslation(-e.ActualWidth * 0.5,
                                                          -e.ActualHeight * 0.5, 
                                                          0);
   var baseTranslate = Matrix3DFactory.CreateTranslation(elem.PositionX, 
                                                         elem.PositionY,
                                                         elem.PositionZ);

   // Combine the transformation matrices
   var m = Matrix3D.Identity;
   m = m * centerAtOrigin;
   m = m * invertYAxis;

   // Apply addtional world transformations to the seleced element
   if (elem == SelectedElement)
   {
      m = m * scale;
      m = m * rotateX * rotateY * rotateZ;
      m = m * translate;

      // Should the camera target be fixed at the selected element?
      if (ChkLookAtSelected.IsChecked.Value)
      {
         lookAt = Matrix3DFactory.CreateLookAtLH(CameraX, CameraY, CameraZ, 
                                                 elem.PositionX, 
                                                 elem.PositionY, 
                                                 elem.PositionZ);
      }
   }

   // Calculate the final view projection matrix
   m = m * baseTranslate;
   m = Matrix3DFactory.CreateViewportProjection(m, lookAt, projection, viewport);

   // Apply the matrix to the UIElement
   e.Projection = new Matrix3DProjection { ProjectionMatrix = m };
}

First the global transformation matrices like camera projection are created using the left-handed Matrix3DEx factory methods. After that the element local transformations are calculated and the final matrix is applied to the UIElement's Projection property.
See the project site for another simplified code listing.

Go and grab it

The open source Matrix3DEx library is hosted at CodePlex and released under the Microsoft Public License (Ms-PL) license. If you have any comments, questions or suggestions don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun with the library and let me know if it was useful for you.

Goodbye GPL

2010-01-10T20:12:00.002+01:00

In the past I've released the source code that mostly comes along with my blog posts under the GPLv3 license to ensure that changes on the code always stay open and never get closed. But the GPL has a major drawback, it's too viral, which means if someone uses GPL licensed code the new code has to be released under the GPL too. And if someone uses this new code, the new code 2 has to be put under the GPL again and so on. And as the FSF states this even applies when the code is just linked dynamically with a GPL'ed library only except if it uses the GPL linking exception. That's why I wanted to move away from the GPL and searched for some alternatives. Finally I decided to use the Ms-PL license from now on if my work doesn't depend on GPL infected code. I've chosen the Ms-PL because it's approved by the OSI, it's short and easy understandable, the weak copyleft guarantees that changes on the original code have to be committed back and that it stays open and it could be used for commercial products. Jon Galloway (@jongalloway) summarized it nicely in his Twitter reply to me: "Ms-PL is essentially BSD + patent protection." So the Ms-PL is suitable, short, simple and keeps the trolls away.

I have changed all the source code headers and License.txt files and updated the Zip files with the source code you find linked on most blog posts. If you still find a GPL reference, just ignore it and if the Ms-PL is a problem for you, just contact me and I might put the relevant code under a second license.

Ye Olde Kamera - Silverlight 4 Webcam & Old Movie Shader

2009-12-30T19:17:00.008+01:00

Another great and successful year comes to an end and I hope that the next year will get even better. We all don't get younger and we change over the years, but the last Silverlight demo I made for this year allows you take some fake aged memories with your webcam. The Silverlight 4 webcam and microphone support is always fun and even more when it's combined with a pixel shader. This demo uses my Old Movie Pixel Shader to create a neat real time webcam effect that simulates scratches, noise and other effects you might have seen in old movies. I also brushed things up a bit by framing the video output with a filmstrip and using some nice new icons from eggheadcafe.

Live
The Silverlight 4 runtime must be installed to run the demo. It's available for Windows and Mac.

Open the sample in a new page.

The Webcam capturing could be started and stopped with the camera Button. If you press it for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the new "Webcam / Mic" tab to set them.
Press the disk Button to take a framed snapshot and save it to a JPEG file on your harddisk.
The amount of noise can be changed using the Slider and the movie ToggleButton allows you to disable the Old Movie Shader.

How it works
It's basically the Silverlight 4 CaptureSource class and my Old Movie Pixel Shader attached to the video output. I covered the Silverlight webcam and shader usage in this blog post and the JPEG encoding in this one. The Old Movie Shader was presented here.

Source code
The Visual Studio 2010 solution including the refactored Old Movie Shader is available here.

Final words
Thanks to my kids and especially to my wife for all her patience in 2009. I love her!
I also like to thank all the people and new friends I got to know this year and especially the Silverlight community. I hope you liked my blog posts and enjoyed the Silverlight demos I've made. I have more to come next year.
I wish you all a great new year, all the best in 2010 and peace!

Update 03-20-2010
Updated to the Silverlight 4 release candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

World# - Real Time 3D Augmented Reality with Silverlight

2009-12-17T23:48:00.020+01:00

Please read the Update at the end of this post.

In the Silverlight 4 Augmented Reality Proof Of Concept blog post I proved that it's possible to implement Augmented Reality applications with Silverlight 4 and the built-in webcam API. The proof of concept used an image showing a Silverlight logo that was attached to the tracked marker. With this blog post I'm back in this field, but this time with real 3D Augmented Reality in Silverlight!
I use the open source Augmented Reality library NyARToolkitCS again and implemented the necessary interfaces to make it work with Silverlight. The NyARToolkitCS library is a completely managed version of the well known ARToolkit. I also use the open source managed game engine Balder for the rendering of 3D models that augment the reality. Balder is mainly developed by the Silverlight MVP Einar Ingebrigtsen. He is working hard on his 3D game engine and provided quick fixes for some issues I encountered while using Balder and I also had a nice chat with him, so big shout out to Einar.

Live
A webcam and at least the Silverlight 4 runtime must be installed to run the sample. To view the application you need to install the Silverlight 4 runtime. It's available for Windows and Mac. You can also watch a video below.
If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Try it out here.

Simply press the "Start Fun" Button, hold the printed marker in front of the camera and move it around. Use the ComboBox to select a different 3D model. The "Flip x-axis" Checkbox could be used to flip the video (the webcam output is mirror-reversed by default).
If you click the "Start Fun" Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.
The Car model was made by Psionic, the Tank model by Ken Beyer. The Teapot is from the Balder samples and I guess it was made by Einar Ingebrigtsen. The rest was made by myself.

Video
This time I used Expression Encoder's Screen Capture feature to record a short video of the demo in action. Please keep in mind that the screen recording software eats up a lot of resources while recording and that the actual frame rate is way better.

Background music is Black Lipstick by 4mat / ATEbit (Matt Simmonds).

How it works
Every time the CompositionTarget.Rendering event is fired, a webcam snapshot is taken asynchronously with the CaptureSource.AsyncCaptureImage method. This snapshot is converted in the right format and passed to the NyARToolkitCS library that outputs a transformation matrix. This matrix is used to transform the 3D model. The Balder viewport has a transparent background and the scene with the 3D objects is laid over the webcam video output. Another layer is an image that uses a WriteableBitmap and the WriteableBitmapEx library to highlight the tracked marker region with a red quad.
Sounds pretty easy, but actually it was not that easy and straight forward. One of the hardest parts was the initialization and to find the right parameters.

Results
I'm quite satisfied with the results and if you consider that all the heavy computation is done by the CPU, the performance is pretty good as well. The demo runs smoothly at 50-60 fps on a dual core machine utilizing its whole power and it should also scale up on more cores.

To be continued
If you expected source code this time I have to disappoint you. The code is not at that level where it should be, but I wanted to push this demo out. I'm not sure if I find enough time this year to work on the code and I don't want that someone tries to use it in the current state. But trust me, I will provide open source functionality to make Augmented Reality in Silverlight in the near future and I normally keep my promises, so stay tuned for more happy days...

Update 02-24-2010
This demo is now part of the open source SLARToolkit library which was announced in this blog post.

Update 03-15-2010
Updated to the Silverlight 4 Release Candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

WriteableBitmapEx - WriteableBitmap extensions now on CodePlex

2009-12-07T21:10:00.006+01:00

I finally kept my promise and put the WriteableBitmap extensions up on CodePlex. This step was long overdue, but when the Silverlight 4 beta was released I couldn't resist and had to play with the new webcam API first (though I'm not finished yet).

The WriteableBitmapEx library project description:

The WriteableBitmapEx library is a collection of extension methods for Silverlight's WriteableBitmap. The WriteableBitmap class that was added in Silverlight 3, allows the direct manipulation of a bitmap and could be used to generate fast procedural images by drawing directly to a bitmap. The WriteableBitmap API is very minimalistic and there's only the raw Pixels array for such operations. The WriteableBitmapEx library tries to compensate that with extensions methods that are easy to use like built in methods. The library extends the WriteableBitmap class with elementary and fast (2D drawing) functionality, supporting common shapes like point, line, ellipse, polyline, quad, rectangle, triangle. Conversion methods and functions to combine WriteableBitmaps (Blitting) are part of it too.

Roots
The WriteableBitmapEx library has its origin in several blog posts:

WriteableBitmap Extension Methods introduced the SetPixel methods.
Drawing Lines - Silverlight WriteableBitmap Extensions II provided the DrawLine methods.
Drawing Shapes - Silverlight WriteableBitmap Extensions III brought the shape functionality (ellipse, polyline, quad, rectangle, triangle).
In Convert, Encode And Decode Silverlight WriteableBitmap Data I published the byte array conversion methods.
Not at least Silverlight MVP Bill Reiss was faster than me on the Blit functions and introduced these in Blitting and Blending with Silverlight’s WriteableBitmap. He was so nice and allowed me to use his code in the WriteableBitmapEx library.
Nikola Mihaylov (Nokola) made some optimizations on the DrawLine and DrawRectangle methods and posted them on his own blog or in the comments of my posts. So a release as open source project was really long overdue.

Additional work
I recreated the whole Visual Studio 2008 solutions, changed the namespaces, headers, added comments and cleaned up the code base and samples. I also grouped the extension methods into different CS files with a partial class. It is now possible to include just a few methods by using the specific source CS files directly or all extension methods through the built library assembly.

License
The WriteableBitmapEx library is released as open source under the Microsoft Public License (Ms-PL) license. The Ms-PL license allows the use of the library without affecting the license of the user, thus making it also easily usable for commercial projects. On the other hand the weak copyleft characteristic ensures that changes on the source have to be committed back and that the library or parts of it always be free and never become closed source.

Touch it, ~~Buy it~~, Use it, Break it, Fix it, Trash it, Change it, (Mail) - Upgrade it, ...

The WriteableBitmapEx CodePlex project site lives here. You might also want to see the Issue Tracker for a list of features that will be added in the future or to add your own. If you have any comments, questions, suggestions or want to see your name in the Credits list, don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun and let me know it if you do some cool stuff with it.

Silverlight 4 Augmented Reality Proof Of Concept

2009-12-03T23:55:00.026+01:00

Please goto: World# - Real Time 3D Augmented Reality with Silverlight. Such demos are now easily possible with the open source SLARToolkit.

In this short blog post I want to present a demo I've actually made last week. Unfortunately I had no time to work on it since then. This demo is a proof of concept for doing Augmented Reality with Silverlight 4 and the built-in webcam API.
I use the open source Augmented Reality library NyARToolkitCS and implemented the necessary interfaces to make it work with Silverlight. The NyARToolkitCS library is a completely managed version of the well known ARToolkit.

Video
I've recorded a short video with my iPhone 3GS. The video was recorded at night and it's no secret that the iPhone camera is not the best, but I think it's good enough to see how the marker is tracked and the Silverlight logo moves with it.

Background music is "You have something in your scowl" by Henrik José / Bliss.

To be continued
The first results are quite good and the demo runs smoothly at 60-70 fps on a dual core machine utilizing both cores. I will continue my work on this project and provide more details, samples and of course source code.
Stay tuned...

Convert, Encode And Decode Silverlight WriteableBitmap Data

2009-11-30T22:29:00.036+01:00

In the comments of my Silverlight 4 EdgeCam Shots blog post "marcb" asked me how to convert the WriteableBitmap to a byte array to save the snapshot in a database.
I thought the answer might be also useful for others. Furthermore I will provide ready to use code for JPEG encoding and decoding of the WriteableBitmap.

Byte array conversion

Copy WriteableBitmap to ARGB byte array

public static byte[] ToByteArray(this WriteableBitmap bmp)
{
   int[] p = bmp.Pixels;
   int len = p.Length * 4;
   byte[] result = new byte[len]; // ARGB
   Buffer.BlockCopy(p, 0, result, 0, len);
   return result;
}

Copy ARGB byte array into WriteableBitmap

public static void FromByteArray(this WriteableBitmap bmp, byte[] buffer)
{
   Buffer.BlockCopy(buffer, 0, bmp.Pixels, 0, buffer.Length);
}

Usage

// Render UIElement into WriteableBitmap
WriteableBitmap bmp = new WriteableBitmap(UIElement, null);

// Copy WriteableBitmap.Pixels into byte array (format ARGB)
byte[] buffer = bmp.ToByteArray();


// Create a new WriteableBitmap with the size of the original image
WriteableBitmap bmp = new WriteableBitmap(width, height);

// Fill WriteableBitmap from byte array (format ARGB)
bmp.FromByteArray(buffer);

I will include the two methods in my WriteableBitmap extensions that I'm going to put up on Codeplex soon.

JPEG encoding and decoding
If you want to store many images or transport them over a network the needed storage size could quickly become a big problem. For example an image with the size 512 x 512 needs 1 Megabyte storage space and a 1024 x 768 image even 3 MB. A solution could be image compression using JPEG encoding and decoding. To accomplish this I've used the open source FJCore JPEG library which is distributed under the MIT License and works nicely with Silverlight.

Encode WriteableBitmap as JPEG stream

public static void EncodeJpeg(WriteableBitmap bmp, Stream destinationStream)
{
   // Init buffer in FluxJpeg format
   int w = bmp.PixelWidth;
   int h = bmp.PixelHeight;
   int[] p = bmp.Pixels;
   byte[][,] pixelsForJpeg = new byte[3][,]; // RGB colors
   pixelsForJpeg[0] = new byte[w, h];
   pixelsForJpeg[1] = new byte[w, h];
   pixelsForJpeg[2] = new byte[w, h];

   // Copy WriteableBitmap data into buffer for FluxJpeg
   int i = 0;
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++)
      {
         int color = p[i++];
         pixelsForJpeg[0][x, y] = (byte)(color >> 16); // R
         pixelsForJpeg[1][x, y] = (byte)(color >> 8);  // G
         pixelsForJpeg[2][x, y] = (byte)(color);       // B
      }
   }

   // Encode Image as JPEG using the FluxJpeg library
   // and write to destination stream
   ColorModel cm = new ColorModel { colorspace = ColorSpace.RGB };
   FluxJpeg.Core.Image jpegImage = new FluxJpeg.Core.Image(cm, pixelsForJpeg);
   JpegEncoder encoder = new JpegEncoder(jpegImage, 95, destinationStream);
   encoder.Encode();
}

Decode WriteableBitmap from JPEG stream

public static WriteableBitmap DecodeJpeg(Stream sourceStream)
{
   // Decode JPEG from stream
   var decoder = new FluxJpeg.Core.Decoder.JpegDecoder(sourceStream);
   var jpegDecoded = decoder.Decode();
   var img = jpegDecoded.Image;
   img.ChangeColorSpace(ColorSpace.RGB);

   // Init Buffer
   int w = img.Width;
   int h = img.Height;
   var result = new WriteableBitmap(w, h);
   int[] p = result.Pixels;
   byte[][,] pixelsFromJpeg = img.Raster;

   // Copy FluxJpeg buffer into WriteableBitmap
   int i = 0;
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++)
      {
         p[i++] = (0xFF << 24)                    // A
                | (pixelsFromJpeg[0][x, y] << 16) // R
                | (pixelsFromJpeg[1][x, y] << 8)  // G
                |  pixelsFromJpeg[2][x, y];       // B
      }
   }

   return result;
}

Usage

// Save rendered UIElement as JPEG file
private void BtnSaveFile_Click(object sender, RoutedEventArgs e)
{   
   if (saveFileDlg.ShowDialog().Value)
   {
      using (Stream dstStream = saveFileDlg.OpenFile())
      {
         // Render to WriteableBitmap
         WriteableBitmap bmp = new WriteableBitmap(UIElement, null);
         
         // Encode JPEG and write to FileStream
         EncodeJpeg(bmp, dstStream);
      }
   }
}


// Open JPEG file and read into a WriteableBitmap
private void BtnLoadFile_Click(object sender, RoutedEventArgs e)
{   
   if (openFileDialog.ShowDialog().Value)
   {
      using (System.IO.Stream srcStream = openFileDialog.File.OpenRead())
      {         
         // Read JPEG file and decode it
         WriteableBitmap bmp = DecodeJpeg(srcStream);
      }
   }
}

Keep in mind that the standard JPEG format doesn't support alpha values (transparency) and that the compression is lossy. So don't encode and decode images subsequently with JPEG.
It is also possible to use the built-in Silverlight class BitmapSource and its SetSource method to decode an JPEG stream.

public static WriteableBitmap DecodeJpegWithBitmapSource(Stream sourceStream)
{
   // Decode JPEG from stream
   var bitmapSource = new BitmapSource();
   bitmapSource.SetSource(sourceStream);
   return new WriteableBitmap(bitmapSource);
}

Source code
Check out my Codeplex project WriteableBitmapEx for an up to date version of the byte array conversion methods.