Goodbye 2010 - Hello 2011

This was an amazing year! Many great things happened and a lot of stuff was released by me. The most important release this year was a personal one. Our third daughter was born in September. Healthy and just plain awesome. She is developing very good and as amazing as her older sisters (photo).
This blog post is just a recap of all the things that happened and I've done in 2010 and some that might happen in 2011.




Community
The whole Silverlight and Windows Phone community is great and it's endless fun to be a part of it. I met a lot of great people online and offline. It's a pleasure to discuss and develop good things in this environment. Thanks a ton!
And don't forget, Silverlight is not dead! Silverlight 5 will be released in 2011 and it will come with a huge bag of new features.
Stay in the 'Light! - like my Silverlight MVP friend Dave Campbell would say.

MVP Award
In April 2010 I was awarded as Silverlight MVP. It is a real honor to be a Silverlight MVP. All the Silverlight MVPs I know are truly outstanding and I can hardly believe I'm considered to be one of them.

Blogging
Yep, I did some blogging this year. Most of it covered Silverlight development and lately also Windows Phone topics. I always try to provide unique content that hasn't been covered anywhere else. I hope you like it and find it useful. Your feedback is always welcome!

Open Source Projects

The one big think this year was the release of the Silverlight Augmented Reality Framework: SLARToolkit. SLARToolkit is a flexible Augmented Reality library for Silverlight with the aim to make real time Augmented Reality applications with Silverlight as easy and fast as possible. It can be used with Silverlight's Webcam API or with any other CaptureSource or a WriteableBitmap. It's open source and hosted at CodePlex.
The development of the library itself took a lot of time, but also the samples (1, 2), the documentation, the support at the forum and via email. It's good to see that some projects use SLARToolkit and the downloads aren't that bad. I'm especially pleased to see that most of the projects are from the academic field. Some projects are listed on the CodePlex site.
I have a lot of things on my idea / todo list I want to add. I hope I'll find a bit time during the next year to implement these.

One part of the SLARToolkit library was extracted and released as separate project. The Matrix3DEx library is a collection of extension and factory methods for Silverlight's Matrix3D struct. The Matrix3DEx library tries to compensate the minimalistic Matrix3D struct with extension and factory methods for common transformation matrices that are easy to use like built in methods. The CodePlex site has all the details and samples.

The WriteableBitmapEx library is another open source project. I actually released it in 2009, but I added a lot of new features during this year and the library got quite well adopted in the community. One reason might be the availability of a Windows Phone version.
The WriteableBitmapEx library is a collection of extension methods for Silverlight's WriteableBitmap. The WriteableBitmap class that was added in Silverlight 3 and which is also available on Windows Phone, allows the direct manipulation of a bitmap and could be used to generate fast procedural images by drawing directly to a bitmap. The WriteableBitmapEx library tries to compensate the minimalistic WriteableBitmap class with extensions methods that are easy to use like built in methods and offer GDI+ like functionality.
There are many different samples available at the CodePlex repository and I also spent a good amount of time with the support at the forum and via email.
The library is still not feature complete for version 1.0 and I will definitely continue my work on it.

Windows Phone Apps
Like many other Silverlight developers I got very exited when Silverlight was announced as the development  platform for Windows Phone 7 apps. I started some experiments right away when the first CTP was available at MIX 2010. I also provided the WriteableBitmapEx Windows Phone version a couple of days later.

Every Windows Phone 7 device is required to have at least a 5 megapixel camera with a flashlight. There are also other features which make Windows Phones amazing devices for taking pictures and dealing with pictures. I've been working heavily during the last months to enhance this photo experience with my kind of skills. My Pictures Lab app was the first picture effects application for Windows Phone with unique, high quality effects. If you like to take photos then this app is a perfect addition to your phone’s toolset. Or like msnbc.com wrote: "The app, a Swiss Army knife of photo tweaks". It's the original and the best selling photo app already since the Windows Phone launch. Pictures Lab comes with more than 20 controllable and easy-to-use advanced effects, a dynamic effect preview, crop, rotate and many more features. I just recently added a Twitter sharing function and the Happy New Year effect / frame (see photo above).
The app got very good reviews from various webites, blogs and offline magazines. The well-known site Engadget lists it under The best apps, accessories, and tips. They write: "... a must-have for WP7 devices ... the program provides a set of amazing effects and tweaks for your photos ...".
I will constantly update the app and add more features and effects in 2011.

I also developed a little fun app called Helium Voice. You can record your voice with the app and change the pitch while sliding over a balloon graphic (inhale or exhale helium).




I have a very long list with future app ideas and it's constantly growing. On some days I have the feeling my head explodes with new ideas.
I hope to find some time next year to push some more apps out. Silverlight is fortunately one of the most productive platforms out there.

Articles
I also wrote some articles for Microsoft's Coding4Fun site and a magazine.

It started with the FaceLight article about a simple facial recognition system using Silverlight 4’s webcam. The code for this article ended up on CodePlex, where you can also find a live sample.

The second article explained how to write pixel shaders for the Microsoft Silverlight and WPF platform with HLSL, as well as how to write an extensible Silverlight application with the help of MEF.
The code is also available at CodePlex.

My last two articles for Coding4Fun showed how to write a simple photo effects application for Windows Phone (1, 2). This is actually where Pictures Lab has its origin.









I just recently wrote an article for Germany's largest .Net developer magazine, dotnetpro. The article covers Windows Phone development and is available in the February 2011 issue and also online.

I also reviewed the media / computer graphics chapters of the two best Silverlight books available. The one is Silverlight 4 Unleashed by Laurent Bugnion and the other is Silverlight 4 in Action by Pete Brown. Both are great books and every Silverlight developer should get a copy.

Talks and the Rest
I gave a talk at the .Net Usergroup Dresden about Silverlight 4's media capabilities and presented SLARToolkit. In May I had the pleasure to be interviewed by Carl and Richard from the .Net Rocks talk show. And I worked on some very interesting projects I can't write something about yet. I also spent a good amount of time answering Silverlight questions I got via email, at the official forum or at Stack Overflow.

It was a really busy year and I hope I can keep this high productivity up.

By the way, I'm also a frequent social network user and met great new people there (Twitter, Facebook). It's a pleasure to exchange thoughts and share knowledge with smart people. Thanks for being awesome!


I wish you and your families a happy and successful new year. 2011 will be good!

Sending Windows Phone Screenshots in an Email

My Silverlight MVP friend Laurent Bugnion blogged about how to take a screenshot from within a Silverlight Windows Phone application. As he wrote this could be helpful for customer feedback and other purposes. To accomplish this, the Silverlight UIElement is rendered to a WriteableBitmap, which can then be saved to the device's Media Library (Pictures Hub). Afterwards it could be send in an email from the Pictures Hub or the email client.
Wouldn't it be even nicer if the user could send an email with the screenshot directly from the app? I show you how.

The Email Task
The only way to send an email from a Windows Phone app is through the EmailComposeTask. This task provides properties for the Subject, Body, CC, Addressee (To) and launches the email client. In the last blog post I showed how I use it to send customer feedback with system information from my apps.
Unfortunately the EmailComposeTask doesn't provide a way to pass an attachment and therefore it's not possible to attach the screenshot to the email. Fortunately there's a way to send binary data as text inside an email. This is where Base64 encoding comes into play.

Embedding the Screenshot 
Base64 encoding transforms binary data into ASCII text and thereby makes it possible to send binary data as plain old text. This way we can convert the JPEG screenshot to Base64 text and set it as the Body of the EmailComposeTask. The Body text length is limited and I experimented a bit with different sizes to find the limitation. It seems like the Body text is limited to less than 32k Unicode characters. A JPEG encoded screenshot of a whole Windows Phone page is too large for this (480 x 800 pixels). That's why the screenshot is rendered downscaled if the element is too large and a lower compression quality is used for the JPEG encoding.

private static void SendEmailScreenshot(FrameworkElement element)
{
   // Render the element at the maximum possible size
   ScaleTransform transform = null;
   if (element.ActualWidth * element.ActualHeight > 240 * 400)
   {
      // Calculate a uniform scale with the minimum possible size
      var scaleX = 240.0 / element.ActualWidth;
      var scaleY = 400.0 / element.ActualHeight;
      var scale = scaleX < scaleY ? scaleX : scaleY;
      transform = new ScaleTransform { ScaleX = scale, ScaleY = scale };
   }
   var wb = new WriteableBitmap(element, transform);

   using (var memoryStream = new MemoryStream())
   {
      // Encode the screenshot as JPEG with a quality of 60%
      wb.SaveJpeg(memoryStream, wb.PixelWidth, wb.PixelHeight, 0, 60);
      memoryStream.Seek(0, SeekOrigin.Begin);

      // Convert binary data to Base64 string
      var bytes = memoryStream.ToArray();
      var base64String = Convert.ToBase64String(bytes);

      // Invoke email task
      var emailComposeTask = new EmailComposeTask
                              {
                                 Subject = "Screenshot from my app",
                                 Body = base64String,
                                 To = "foo@bar.com",
                              };
      emailComposeTask.Show();
   }
}

The element is downscaled uniformly if it's larger than a certain size. This is done with a ScaleTransform directly when the Silverlight (vector) element is rendered to a WriteableBitmap. Afterwards the bitmap is encoded as JPEG at a quality of 60%. This low compression quality reduces the size of the JPEG image significantly, but the image is still usable. The binary data is then transformed to Base64 text with the Convert.ToBase64String method. Finally the EmailComposeTask is created, the properties are set and the email client is opened.

Decoding the Screenshot
The email will contain the Base64 encoded image. Base64 encoded data typically looks like this:

/9j/4AAQSkZJRgABAQEBBgEGAAD/2wBDAA0JCgsKCA0LCgsODg0PEyAVExISEyccHhcgLikxMC4pLSwzOko
+MzZGNywtQFdBRkxOUlNSMj5aYVpQYEpRUk//2wBDAQ4ODhMREyYVFSZPNS01T09PT09PT09PT09PT09PT09PT09PT
09PT09PT09PT09PT09PT09PT09PT09PT09PT0//wAARCAFsAPADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAA
AAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM
2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJW
Wl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAw

...

The opposite operation of the ToBase64String method is the Convert.FromBase64String method. So it's easy to write a little tool that extracts the email content, converts it back to JPEG data and saves it as a file for example. Probably it's sufficient to use an online Base64 encoder like this one where the Base64 string can be pasted and the tool will create a binary file which can be downloaded.

The result might look like the below screenshot from the Pictures Lab app. As you can see, the half downscaled version with the 60% quality is usable and even the small text can be read without problems.

Future

There's also room for improvement left. More modern image compression algorithms like they are used in the Hipix or WebP formats can produce better results at lower size. Another way to reduce the size could be a post process step where every odd pixel or line is removed. This technique was described in the Kill Pixel Shader blog post.

Needless to say that the demonstrated, well-known Base64 encoding can be used to send any small binary data inside an email from a Windows Phone app. So it's also possible to send short audio clips, binary serialized data or other data.

Conclusion
For more complex tracking and feedback mechanisms an image upload webservice might be the better way. However there are many scenarios with simple tasks where a webservice isn't feasible and the shown technique can be helpful.

Listen to your Phone - Getting User Feedback with System Information

Two weeks ago I've blogged about a trick my Windows Phone Pictures Lab app uses to detect if the Zune software is running. Today I have a handful of classes that help to gather system information and provide an easy interface for the email task. No magic at all, but I though it might be helpful for others too.
I use these helpers in Pictures Lab for the user feedback functionality. I got a lot of valuable feedback and great suggestions via this feature.
The user can invoke this functionality on the about page of the app and it's also fired when an unhandled exception occurred and the user wants to send error details.

System Information
It's always useful to include some system, version and device information in such a feedback report. That's why I wrote a SytemInformation class which queries some paramteres and creates a nice, human readable string.

public class SystemInformation
{
   public static string Dump()
   {
      var builder = new StringBuilder();
      builder.AppendLine("***** System Infos *****");
      builder.AppendLine();
      builder.AppendFormat("Time: {0}", DateTime.Now.ToUniversalTime().ToString("r"));
      builder.AppendLine();
      builder.AppendFormat("Culture: {0}", CultureInfo.CurrentCulture);
      builder.AppendLine(); 
      builder.AppendFormat("App Version: {0}", WmAppManifestHelper.GetVersion());
      builder.AppendLine();  
      builder.AppendFormat("Manufacturer: {0}", DevicePropertiesHelper.GetDeviceManufacturer());
      builder.AppendLine(); 
      builder.AppendFormat("Model: {0}", DevicePropertiesHelper.GetDeviceModel());
      builder.AppendLine(); 
      builder.AppendFormat("DeviceHardwareVersion: {0}", DevicePropertiesHelper.GeDeviceHardwareVersion());
      builder.AppendLine(); 
      builder.AppendFormat("DeviceFirmwareVersion: {0}", DevicePropertiesHelper.GeDeviceFirmwareVersion());
      builder.AppendLine(); 
      builder.AppendFormat("OS  Version: {0}", Environment.OSVersion);
      builder.AppendLine(); 
      builder.AppendFormat("CLR Version: {0}", Environment.Version);
      builder.AppendLine(); 
      builder.AppendFormat("Device Type: {0}", Microsoft.Devices.Environment.DeviceType);
      builder.AppendLine(); 
      builder.AppendFormat("Network Type: {0}", NetworkInterface.NetworkInterfaceType);
      builder.AppendLine(); 
      builder.AppendFormat("DeviceTotalMemory: {0:f3} MB", DevicePropertiesHelper.GetTotalMemoryMb());
      builder.AppendLine(); 
      builder.AppendFormat("ApplicationPeakMemoryUsage: {0:f3} MB", DevicePropertiesHelper.GetPeakMemoryUsageMb());
      builder.AppendLine(); 
      builder.AppendFormat("ApplicationCurrentMemoryUsage: {0:f3} MB", DevicePropertiesHelper.GetCurrentMemoryUsageMb());
      builder.AppendLine(); 
      using (var appStorage = IsolatedStorageFile.GetUserStoreForApplication())
      {
         builder.AppendFormat("Iso Storage AvailableFreeSpace: {0:f3} MB", appStorage.AvailableFreeSpace / 1024f / 1024f);
         builder.AppendLine();
      }
      return builder.ToString();
   }
}

As you can see a few interesting properties from the OS, the device and its configuration and memory status are queried. This is done with built-in and custom helper classes.
The WmAppManifestHelper class parses the WmAppManifest.xml file and provides the app version and other properties from this file. The DevicePropertiesHelper wraps the DeviceExtendedProperties class and the available property keys. I added all the keys I need, but not more (YAGNI). However, if you add more property keys to the class, please make sure to post a comment.


Email Service
The EmailService class wraps the EmailComposeTask and adds the system infos to the body of the email.

public class EmailService
{
   private readonly EmailComposeTask emailComposeTask;

   public string Addressee { get; set; }

   public EmailService()
   {      
      Addressee = "foo@bar.com";
      emailComposeTask = new EmailComposeTask();
   }

   public void ShowTask(string subject, string body)
   {
      // Add system infos add the end of the email
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += Environment.NewLine;
      body += SystemInformation.Dump();

      // Fill email
      emailComposeTask.To = Addressee;
      emailComposeTask.Subject = subject;
      emailComposeTask.Body = body;

      // Invoke Mail Task
      emailComposeTask.Show();
   }
}

The ShowTask method sets the properties of the EmailCompoaseTask and opens it. The addressee can be defined with the corresponding property.


Please replace the foo@bar.com default value for Addressee with your email address. Think about the poor Foo at bar.com.










Handling the Unhandled
This email functionality is used to enable explicit user feedback when the user taps the 'submit feedback' button. It is also utilized in the UnhandledException event handler of the Application class. The default body of the event handler in the App.xaml.cs file is generated by Visual Studio when a new Windows Phone Application project is created. Just add the email service and set the e.Handled property to true.

private static void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
   var result = MessageBox.Show("An error occured. Do you want to send the error details to the developer?", "Meh", MessageBoxButton.OKCancel);
   if (result == MessageBoxResult.OK)
   {
      new EmailService().ShowTask("App error", e.ExceptionObject.ToString());
   } 
   e.Handled = true;
}

The body of such an error email would then look like this

System.ArgumentException: The parameter is incorrect. 
   at MS.Internal.XcpImports.CheckHResult(UInt32 hr)
   at MS.Internal.XcpImports.SetValue(INativeCoreTypeWrapper obj, DependencyProperty property, DependencyObject doh)
   at MS.Internal.XcpImports.SetValue(INativeCoreTypeWrapper doh, DependencyProperty property, Object obj)
...


***** System Infos *****

Time: 12/11/2010 5:50:02 PM
Culture: en-US
App Version: 1.9.0.0
Manufacturer: SAMSUNG
Modell: SGH-i917
DeviceHardwareVersion: 3.1.0.7
DeviceFirmwareVersion: 2103.10.10.1
OS  Version: Microsoft Windows CE 7.0.7004
CLR Version: 3.7.10218.0
Device Type: Device
Network Type: MobileBroadbandGsm
DeviceTotalMemory: 475.469 MB
ApplicationPeakMemoryUsage: 63.055 MB
ApplicationCurrentMemoryUsage: 41.223 MB
Iso Storage AvailableFreeSpace: 601.641 MB

Conclusion
User feedback is always valuable and every serious Windows Phone developer should at least add a simple functionality like the one presented here.

Source Code
You can download all mentioned classes here. Please comment if you know more useful system infos that should be included.

Issue with the WP7 PictureDecoder and Workaround

This is fixed in the WP 7.1 SDK /  WP 7.5 / Mango!
I noticed a strange behavior of the Windows Phone PictureDecoder DecodeJpeg method while I was working on my Pictures Lab app.
This short post describes the issue I encountered and also provides a workaround.
The built-in DecodeJpeg method decodes a JPEG image stream into a WriteableBitmap. The method has two overloads, where the first only takes the JPEG stream and the second also uses parameters for the maximum width and height of the output. I encountered a strange behavior of the latter method.

Issue
The DecodeJpeg method with 3 parameters swaps the width and the height parameters when a landscape photo should be resized. For example, an image with the original size of 3264 x 2448 should be decoded to a WriteableBitmap and resized to 1024 x 768. But the resulting output will have a size of 768 x 576 cause the method swapped the input parameters while preserving the correct aspect ratio.
Please note that this only happens with landscape pictures, which means the width of the image is greater than the height.

Will it get fixed?
Yes! I contacted Microsoft and they confirmed this issue and said that it will get fixed in a future version of the Windows Phone Silverlight runtime.

Workaround
Isolating the cause of this strange behavior took a bit of time, fortunately the easier was the obvious workaround.

int w = desiredOutputWidth;
int h = desiredOutputHeight;

// Workaround for issue in DecodeJpeg method:
// Swap width and height for landscape pictures.
if (originalWidth > originalHeight)
{
   w ^= h;
   h ^= w;
   w ^= h;
}
var writeableBitmap = PictureDecoder.DecodeJpeg(jpegStream, w, h);

The code tests if the original width is greater than the height and then swaps the output width and height using the good old XOR swap trick.
Alternatively you can avoid the DecodeJpeg overload with the resizing functionality, decode the full sized image and use the WriteableBitmapEx Resize method afterwards. It produces similar results when bilinear interpolation is used, but it's an extra step that costs resources.

The above photo was taken with a HTC Mozart and edited with Pictures Lab (cropped, rotated and 1989 vintage effect).

Windows Phone Unplugged - How to Detect the Zune Software

I've been blogging about Windows Phone development since the first CTP version and also blogged about the photo and camera API.
This post will cover a little trick my Pictures Lab app uses to detect if the Zune software is running when the device is connected to the PC. This is important because the device's media Hubs can't be accessed when the device is connected and the Zune software is running. The same is true for the media APIs (PhotoChooser, MediaLibrary, ...).
The Pictures Hub and the Zune Hub show a little animation when you try to open these while Zune is running. Unfortunately there's no built-in method to detect this in code.

An app mostly performs the classic IPO Model operations: load, process, save. The load and save steps can fail when Zune is connected. The following two code snippets show how to detect if the Zune software is running and ask the user to close it. I use the PhotoChooser and MediaLibrary here to demonstrate this exemplarily for picture handling, but the same is also valid for other media like audio.

Save fail
If the device is connected to the PC and the Zune software is running, an InvaildOperationException is thrown.
By the way, this blog post covered how to save a WriteableBitmap as JPEG to the device's media library.

private static void Save(WriteableBitmap bitmap)
{
   try
   {
      SaveToMediaLibrary(bitmap, "mySuperCoolPic.jpg");
   }
   catch (InvalidOperationException)
   {
      MessageBox.Show("The Zune software is running and the picture can't be saved. Please close Zune or disconnect the phone.");
   }
}

This is quite straightforward.


Load fail
Unfortunately it's not that easy to detect if the Zune software is running when a picture is opened through the PhotoChooserTask API. (How to use this Task was also covered here.)

Some users contacted me when they tried to open a photo with Pictures Lab and nothing happened. The problem is that the picture viewer just won't open when the PhotoChooserTask.Show method is called and Zune is running on the PC. There's no built-in feedback here. I implemented the following code in the Completed event handler of the PhotoChooserTask to get a better user experience.

private void PhotoProviderTaskCompleted(object sender, PhotoResult e)
{
   try
   {
      if (e != null)
      {
         if (e.TaskResult == TaskResult.OK)
         {
            // Do something with the JPEG stream in e.ChosenPhoto
         }
         else if (e.Error is InvalidOperationException)
         {
            throw e.Error;
         }
         else if (e.TaskResult == TaskResult.Cancel)
         {
            // Try to get an image from the Pictures Hub. 
            // If this fails, the device is connected and Zune is running.
            var pictures = new MediaLibrary().Pictures;
            if(pictures.Count > 0)
            {
               pictures[0].GetThumbnail();
            }
         }
      }
   }
   catch (InvalidOperationException)
   {
      MessageBox.Show("The Zune software is running and the picture can't be saved. Please close Zune or disconnect the phone.");
   }
}

One might wonder why the MediaLibrary is queried when the PhotoResult EventArgs has the Error property. Unfortunately it's not used when Zune is running, only the TaskResult is set to Cancel in this case. This TaskResult value is also used when the user presses the back button while choosing a photo and it's not clear why the TaskResult was set to Cancel.

Fin
That's it.
I hope it helps.
Let me know if you found a better way.

The above photo was taken with a HTC Mozart and edited with Pictures Lab (cropped and Lomo effect).