Thursday, March 25, 2010

Coding4Fun - Silverlight Real Time Face Detection

A while ago Clint Rutkas (yes, the guy that built the awesome MIX T-Shirt cannon) asked me if I could write an article for Microsoft's Coding4Fun site. I felt honored and of course I agreed. Today I'm proud to announce that my first Coding4Fun article was published.

The article demonstrates how to implement a simple facial recognition system using Silverlight 4’s new webcam feature. The method I've implemented searches for a certain sized skin color region in a webcam snapshot. This technique is not as perfect as a professional computer vision library like OpenCV and the Haar-like features they use, but it runs in real time and works for most webcam scenarios.

The complete source code for the article is available at CodePlex and licensed under the Ms-PL.
If you like the article, I’ll write a follow-up that might cover a more stable (better) segmentation approach, how to optimize the performance further and more.

Update 04-06-2010
The article was translated into Russian and is available at the Russian Coding 4 Fun site.

Update 04-15-2010
Updated the application to the final Silverlight 4 RTW build.

Wednesday, March 17, 2010

Let it ring - WriteableBitmapEx for Windows Phone

Actually I never heard a Windows Phone prototype ringing yet, but that's not the point of this blog post.

On Monday, March 15, 2010 Microsoft not only released the Silverlight 4 Release Candidate at MIX, they also made the first CTP of the Windows Phone Developer Tools available. The best news is, Silverlight is THE technology for Windows Phone 7 series development and the SDK doesn't cost $ 100, no, it's completely free!
 Of course I downloaded and installed the CTP right away and played around with it. First I was a bit disappointed after I realized that custom pixel shaders are not supported (at the moment) and, correct me if I'm wrong, real time camera access is not built in either, which is essential for a nice SLARToolkit Windows Phone version. BUT, it's only the first CTP and at least we have the WriteableBitmap at hand for some procedural graphics fun.

A while ago I started the WriteableBitmapEx project to make the life a bit easier when working with the WriteableBitmap. And now what would be more natural than porting it to the Windows Phone platform.
So I fired up Visual Studio 2010 for Windows Phone and set up the first real Windows Phone Silverlight project pretty quickly. After I included the original WriteableBitmapEx source code files I started the build. And it just worked, without any errors. Wow, that was an easy port.
Sure, Scott Guthrie mentioned in his keynote that the Windows Phone fully supports Silverlight, but there are actually some differences to the desktop version. Nevertheless the first developer experience was just impressive and WriteableBitmapEx for Windows Phone is actually just another build target and not a real port of the original Silverlight library.

Unfortunately the current CTP is a bit slow when it comes to real time graphics, but there is hope and the Release Notes say:
"Per frame callbacks (i.e. handling the CompositionTarget.Rendering event) should be used sparingly in this release due to less than optimal performance."
That's why I used the interactive parts of the latest WriteableBitmapEx curve sample to make a Windows Phone sample. All this was done in less than hour and most of the time was spent to arrange the controls and play with the great Windows Phone emulator.
Below is a video that shows the demo running in the emulator. Please note how the phone's back button is used to remove the last control point from the list.

Background music is Loell by mosaik.

How it works
As I mentioned above parts of the code from the latest WriteableBitmapEx curve sample were simply re-used. Furthermore I rearranged the controls for the phone's landscape mode.
The functionality of the phone's back button is implemented by adding an event handler to the Page's BackKeyPress event. The used event handler:
void PhoneApplicationPage_BackKeyPress(object sender, CancelEventArgs e)
    // Remove last control point from the list
    if (points != null && points.Count > 0)
        points.RemoveAt(points.Count - 1);

    // Refresh image

    // Back is handled here. Prevent that the phone returns to the home screen
    e.Cancel = true;

Source code
From now on you will find a Visual Studio solution and a project file for the Windows Phone version in the WriteableBitmapEx' source code repository. Starting with the next release I will also provide a ready to use Windows Phone build.

Woo Hoo!
Silverlight is becoming more and more a full multi platform technology and all Silverlight developers are now Windows Phone developers too. It's an exciting time to be a .Net developer. Awesome!

Update 03-18-2010
I changed the size of the control points to the appropriate minimum touch target size as required in the Windows Phone UI guidelines. According to it the minimum touch target size has to be 9 mm / 34 pixel and the visual size 60-100% of the touch target size.

Monday, March 15, 2010

The Silverlight 4 Release Candidate

Scott Guthrie et al. announced many awesome new things during the keynotes at Microsoft's MIX10 conference. One of it was the Silverlight 4 Release Candidate which replaces the beta version and is now available for download. Beside all the great new features that were added, it's now finally possible to use the Visual Studio 2010 RC for Silverlight 4 development. Kudos to the Silverlight team for catching up with Visual Studio 2010 and all the cool new features they implemented. 

The final Silverlight 4 RTW version will be available next month (April 2010). Additonaly the first CTP of the Winows Phone SDK was released and Silverlight is THE platform for Windows Phone 7 series development and it's free! Silverlight for Symbian is also now available for Nokia S60 5th Edition devices like the N97. So Silverlight is becoming more and more a full multi platform technology. Awesome!

This blog post focuses on the Silverlight 4 RC and contains a summary of all the new features that were added since the Silverlight 4 Beta to the Silverlight 4 RC and the breaking changes that naturally come along with betas. Plus some details about the CaptureSource's new capture usage pattern.

New Silverlight 4 RC features

Rich Text
  • RichTextBox rename
  • Text position and selection APIs
  • XAML property for serializing text content
  • XAML clipboard format
  • FlowDirection support on Runs
  • "Format then type" support
  • Thai / Vietnamese / Indic support
  • UI Automation Text pattern
  • UploadProgress support (Client stack)
  • Caching support (Client stack)
  • Sockets security restrictions removal (Full Trust)
  • Sockets policy file retrieval via HTTP
  • Accept-Language header
Out of Browser (Full Trust)
  • XAP signing
  • Silent install and emulation mode
  • Custom window chrome
  • Better support for COM Automation
  • Cancelable shutdown event
  • Updated security dialogs
  • Pinned full-screen mode on secondary display
  • Webcam / Mic configuration preview
  • More descriptive MediaSourceStream errors
  • Content & Output protection updates
  • Updates to H.264 content protection (ClearNAL)
  • Digital Constraint Token
  • CGMS-A
  • Multicast
  • Graphics card driver validation & revocation
Graphics and Printing
  • Hardware accelerated perspective transformations
  • Ability to query page size and printable area
  • Memory usage and performance improvements
  • Entity-level validation support of INotifyDataErrorInfo for DataGrid
  • XPath support for XML
  • New architecture enables future innovation
  • Performance and stability improvements
  • XmlnsPrefix & XmlnsDefinition attributes
  • Support setting order-dependent properties
Globalization & Localization
  • Support for 31 new languages
  • Arabic, Hebrew and Thai input on Mac
  • Indic support
  • Update to DeepZoom code base with hardware acceleration
  • Support for Private mode browsing
  • Google Chrome support (Windows)
  • FrameworkElement.Unloaded event
  • HTML Hosting accessibility
  • IsoStore performance improvements
  • Native hosting performance improvements (e.g. Bing Toolbar)
  • Consistency with Silverlight for Mobile APIs and Tooling
  • System.Numerics.dll
  • Dynamic XAP support (MEF)
  • Frame / Navigation refresh support
The new webcam and microphone configuration preview is pretty neat. The user can now select the default webcam & microphone device really comfortable and we only need to use CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice() or CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice() to get the selected default video and audio device.

The webcam permission dialog is now consent and can remember if the user has previously allowed that certain application. Tim Heuer's blog post has more details about this handy new feature.

Breaking Changes since the Silverlight 4 beta
The first and most important take away: As usual all Silverlight 4 beta applications have to be updated to the RC and recompiled, otherwise the new RC runtime won't execute them.
  • Renamed RichTextArea to RichTextBox
  • Changes to DRM API
  • TextSelection.SetPropertyValue is now named .ApplyPropertyValue
  • INotifyDataErrorInfo moved to core (System.Windows.dll)
  • WebBrowser.ScriptNotify signature change
  • NotificationWindow.Visible and NotificationWindow.Closed were changed
  • Webcam and output protection change
  • COM Interop API changes (name and namespace)
  • RichTextBox.TextWrapping default value changed
  • Binding declarations in XAML where the target is a DependencyObject
  • HtmlBrush renamed to WebBrowserBrush

The official breaking changes document contains all the details and Koen Zwikstra (@kozw) has made some useful diff lists.

Changed usage pattern for AsyncCaptureImage
The biggest difference in the webcam API besides the renaming of some enums, properties and other members is the change of the AsyncCaptureImage method which was renamed to CaptureImageAsync. Additionally the usage pattern of the CaptureImageAsync method was updated to the standard .Net / Silverlight asynchronous event pattern.

Before the Silverlight 4 RC, AsyncCaptureImage was used like this:
// Initialization
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

// Capture image
captureSource.AsyncCaptureImage(bitmap =>
   // Process camera snapshot

The new RC CaptureImageAsync usage pattern:
// Initialization
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
// Wiring the CaptureImageCompleted event handler
captureSource.CaptureImageCompleted += (s, e) =>
   // Process camera snapshot (e.Result is a WriteableBitmap)

// CaptureImageAsync fires the CaptureImageCompleted event

Although the pre-RC pattern was smaller and easier, it's overall more consistent now and I welcome the changes.

I already updated SLARToolkit and the samples and I will continue to update some of my Silverlight 4 beta samples during the next days. Furthermore I will make a Silverlight 4 RC build of the Matrix3DEx sample that should take advantage of the new hardware accelerated perspective transformations.

Wednesday, March 10, 2010

Poll Results: Do you block Google Analytics tracking in your browser?

I recently encountered a large difference between the Google Analytics statistics of my website and the server logs. This raised the question in me how reliable the Google Analytics data really is and I asked you to take part in a small survey: Do you block Google Analytics tracking in your browser?. The poll is closed now and here are the results.

The poll got 267 votes total and 68 people voted "Yes, I block Google Analytics." 199 voted for "No, I don't block Google Analytics." This means 25.47% of the participants block Google Analytics in their browser. 

You can now use a simple formula to get the real world Google Analytics statistics (at least for my visitors - the best of all). :-)

GAreal = GAreport * 1.25468164794007490636704119850187

Big thanks to all that helped me and took part in the survey!