Monday, May 7, 2012

One Bitmap to Rule Them All - WriteableBitmapEx for WinRT Metro Style

A couple of weeks ago we added official WPF support to  WriteableBitmapEx. Today I'm happy to announce that WriteableBitmapEx now also officially supports Windows 8 Metro Stlye WinRT .NET XAML. With that WriteableBitmapEx is now available for 4 platforms: WPF, Silverlight, Silverlight for Windows Phone and Metro Style WinRT .NET.
Although Direct2D is the best solution for fast 2D graphics with Windows 8 Metro Style, I think there are scenarios where the WriteableBitmapEx could be helpful, esp. when using C# with XAML. I also know that some devs were waiting for this to port their Windows Phone apps to Windows 8 Metro Style.

WinRT Differences
Unlike the Silverlight WriteableBitmap, the Metro Style WriteableBitmap doesn't provide the pixel data directly. Its IBuffer PixelBuffer property doesn't have an interface to get the color information. Fortunately there are a few C# extension methods available which provide the pixel data as byte array or stream in the BGRA pixel format. Yes, BGRA and not like all the other platforms supported by WriteableBitmapEx as ARGB. The BGRA format is mainly used by Direct2D, which might be the reason for this hidden, but important difference of the Metro Style WriteableBitmap.
The WriteableBitmapEx algorithms are written for the ARGB pixel format. Fortunately I was able to keep the details away from the library user by leveraging the BitmapContext concept we introduced with the WPF support. This approach makes it possible to share almost the same code for all 4 platforms without being cluttered with #if directives all over place.  Actually the most significant WinRT adaptation inside the WriteableBitmapEx methods was done in the FromContent method, which loads an image from the app content and provides it as WriteableBitmap. See this StackOverflow question I answered if you're interested in the details.
Nothing comes for free, but if the BitmapContext is used the right way, the performance hit won't be that much thanks to an internal reference counting WriteableBitmapEx' BitmapContext uses. No worries, you don't have to change all your WriteableBitmapEx calls, just wrap your calls in a simple using(writeableBmp.GetBitmapContext()) and you will only have one buffer conversion instead of one for each draw call.
It's really simple to use:

private void Draw()
{
   // Wrap updates in a GetContext call, to prevent invalidation overhead
   using (writeableBmp.GetBitmapContext())
   {
      writeableBmp.Clear();
      DrawPoints();
      DrawBeziers();
  
   } // Invalidates on exit of using block
}

private void DrawPoints()
{
   foreach (var p in points)
   {
      DrawPoint(p, Colors.Green, PointVisualSizeHalf);
   }
}

private void DrawPoint(ControlPoint p, Color color, int halfSizeOfPoint)
{
   var x1 = p.X - halfSizeOfPoint;
   var y1 = p.Y - halfSizeOfPoint;
   var x2 = p.X + halfSizeOfPoint;
   var y2 = p.Y + halfSizeOfPoint;
   writeableBmp.DrawRectangle(x1, y1, x2, y2, color);
}

private void DrawBeziers()
{
   if (points.Count > 3)
   {
      writeableBmp.DrawBeziers(GetPointArray(), Colors.Yellow);
   }
}

Screenshot WinRT Metro Style sample running in the simulator

All samples were tested with the new version, but due to the refactoring more testing is needed. Please test this version with your projects and report the bugs you encounter. You can download the binaries here. Note that this package only contains the WriteableBitmapEx binaries for Silverlight, Windows Phone, WinRT Metro Style .NET and WPF. All the samples can be found in the source code repository in the branch "WBX_1.0_BitmapContext". If all goes well, this branch will become the trunk and the 1.0 RTM in a few weeks.

WinMD / Windows Runtime Component
There's also a WinMD version available which makes it possible to consume the WriteableBitmapEx library from all the WinRT Metro Style projections, although only C# and C++ XAML make actually sense.
I had to move some parts and leave some functionality like the ForEach out, but it contains 99% of the library's features. Unfortunately the C++ sample I created crashes when the WriteableBitmapExWinMD library is loaded. So for now this WinMD version can be found in a separate branch "WBX_1.0_WinMD" in the source code repository and it won't be part of the trunk and release until it works with the sample. I'm a bit running out of time and don't know where to look for, since it seems all is wired up correctly and compiles fine. If you are a WinMD wizard and have a few minutes, I'd appreciate if you could look into the WinMD version.

13 comments:

  1. Hello Renè, I was trying to use your fantastic library with WInRT to capture live snapshots from a webcam (without passing from isolated storage ;).

    I noticed that there are more bytes than expected, and I have to jump two bytes in FromByteArray in order to display a clear image.

    This is the (working) code:


    ImageEncodingProperties imageProperties = new Windows.Media.MediaProperties.ImageEncodingProperties();
    var devInfoCollection = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(Windows.Devices.Enumeration.DeviceClass.VideoCapture);

    var mediaCaptureMgr = new Windows.Media.Capture.MediaCapture();
    await mediaCaptureMgr.InitializeAsync();

    capturePreview.Source = mediaCaptureMgr;
    await mediaCaptureMgr.StartPreviewAsync();

    imageProperties.Subtype = "BMP";
    imageProperties.Width = 320;
    imageProperties.Height = 240;

    WriteableBitmap w = new WriteableBitmap(320, 240);
    MemoryStream s = new MemoryStream();


    IRandomAccessStream rs = new InMemoryRandomAccessStream();
    await mediaCaptureMgr.CapturePhotoToStreamAsync(imageProperties, rs);

    DataReader rd = new DataReader(rs.GetInputStreamAt(0));
    await rd.LoadAsync((uint)rs.Size);
    byte[] b = new Byte[rs.Size];
    rd.ReadBytes(b);
    Debug.WriteLine(b.Length); // 307254 bytes instread of 307200 (ARGB: 320x340x4) !!

    w.FromByteArray(b, 2, (int)w.PixelBuffer.Length); // => I found that it is good starting from the 3rdbyte, ignoring last 51 (???)

    imgFrame.Source = w; // s;


    Any ideas what are those exceeding bytes...?

    Best,
    David

    ReplyDelete
  2. Hello David,

    You're encoding the bitmap as "BMP" format and there's some header / footer information additionally to the bitmap data.
    Use the FromByteArray() method with data you got from the WriteableBitmap.ToByteArray().

    ReplyDelete
  3. Do you have an example of using FromContent in a WinRT app? I looked at the StackOverflow answer you gave, but that doesn't show calling the method.

    ReplyDelete
  4. Sure. Taken from the Blit sample you can find in the source code repository:

    var particleBmp = await LoadBitmap("///Assets/FlowerBurst.jpg");

    async Task LoadBitmap(string path)
    {
    Uri imageUri = new Uri(BaseUri, path);
    var bmp = await BitmapFactory.New(1, 1).FromContent(imageUri);
    return bmp;
    }

    ReplyDelete
  5. Hi Rene,

    I am sure you might have got this question a lot but I was wondering if WriteableBitmapEx has a feature that I can use to save a Canvas as image(jpg/png) in a Windows 8 app written in C# and XAML. Please let me know.

    ReplyDelete
  6. Indeed.

    Unfortunately there's no way. Not even with DirectX. Only if you render all in DirectX.
    :(

    MS is aware of this and we can only hope a Service Pack will bring this feature.

    ReplyDelete
  7. Please let me know how to convert canvas to image(jpg/png) in Windows 8 app written in C# and XAML. I have one image over another image in canvas.

    ReplyDelete
    Replies
    1. Well, see this blog post which covers that: http://kodierer.blogspot.de/2013/12/how-to-encode-and-save-writeablebitmap.html

      Delete
    2. Hi Rene, I have seen this blog post of yours. But the problem is while i convert the canvas to WritableBitMap to use the function in your blog post, i do not get the exact bitmap. I use Windows.UI.Xaml.Media.Imaging.WriteableBitmap bitmap = await WriteableBitmapRenderExtensions.Render(photoCanvas); This is from WinRTXamlToolkit. This is not working properly. My canvas has two images. one on top of the other. photoCanvas is the name of my canvas. Any suggestions to convert canvas with two images to WritableBitmap?

      Delete
    3. I'm not the author of the WinRTXamlToolkit, so this is the wrong place to ask. Check their CodePlex Discussions.
      But if you read my blog posts, you can see here how to render any UIElement like your photoCanvas into a WB:
      http://kodierer.blogspot.de/2013/12/easy-render-writeablebitmapex-with.html
      And how to encode that WB into a JPEG or PNG:
      http://kodierer.blogspot.de/2013/12/how-to-encode-and-save-writeablebitmap.html

      Delete
    4. I think your logic might be wrong. If you want to merge multiple images and save it, you probably want to manually draw each image on top of each other and then save the resulting bitmap. I know there are methods available that let you render a control to a bitmap, but I don't think it would be efficient if you're developing a high-performance image-processing application.

      Delete
  8. Hello,

    I'm developing an app for Windows 10 (UWP)and I'm looking for a way to draw some text over an image (all in background). I've tried creating all UIElements programmatically but writeableBmp.RenderAsync crashes :( Is it possible to use WritableBitmapEx to achieve that? Many thanks!

    ReplyDelete
    Replies
    1. Not really. You have to create a TextBlock add it to the visual tree and then use RenderToBitmap.

      Delete