Monday, January 18, 2010

Matrix3DEx 1.0 - When PlaneProjection is not enough

A new year has begun and I started a new open source project at CodePlex. It's called Matrix3DEx and is an extension library for the Silverlight Matrix3D struct. Most of the functionality I implemented in Matrix3DEx was originally required for another open source project I'm currently working on (hint, hint), and as you might know I also like to extend Silverlight's graphics functionality in a reusable manner. That's why I decided to extract the Matrix3D code into a separate open source project, add some more useful methods, document it and write a sample.


The Matrix3DEx project description from the CodePlex site:
The Matrix3DEx library is a collection of extension and factory methods for Silverlight's Matrix3D struct. The Matrix3D struct represents a 4x4 matrix that is used in combination with the Matrix3DProjection to apply more complex semi-3D scenarios to any UIElement than are possible with the simple PlaneProjection. This makes it possible to apply arbitrary model transformation matrices and perspective matrices to Silverlight elements.
The Matrix3D struct is very minimalistic and has only a few members. The Matrix3DEx library tries to compensate that with extension and factory methods for common transformation matrices that are easy to use like built in methods.

Features:
  • Factory methods

    • Translation, scaling and rotation around x, y, z or any defined axis
    • Perspective field of view and orthographic projection
    • Camera (look-at) with position, target and up vector
    • Support for left-handed and right-handed coordination systems

  • Extension methods

    • Calculation of the matrix' determinant
    • Matrix transpose
    • SwapHandedness to change from right-handed to left-handed coordination system and vice versa
    • Dump of the values row by row into a formatted string

  • Math helper methods

    • Angle conversion from degrees to radians and vice versa


Some use cases where Silverlight's semi-3D projection is needed can be implemented with the PlaneProjection, but there are also some scenarios where the PlaneProjection wouldn't work or only with a lot of effort. One example is the usage of a physics library that returns a matrix for an object. Traditional matrix transformations are an elegant alternative for such cases and the Matrix3DEx library has factory and extension methods for all the common matrices.

Live
Open the sample in a new page.



The application loads a bunch of photos asynchronously and randomizes the position vector of each. The sample uses most of the Matrix3DEx features and has some Sliders and CheckBoxes to change the parameters. Uncheck the "Animate" CheckBox to disable the camera movement and click on an Image to select it. The translation, scaling and the rotation matrices of the selected element can be changed with the corresponding Sliders. You can move the camera, change the target and the "Field Of View" with the other Sliders or fix the camera target at the selected element.
The basic functionality of the sample might also be done with the PlaneProjection, but a separate look-at matrix (camera) simplifies the code a lot and makes it easier to read. Other things like a custom field of view are not possible with the PlaneProjection.

How it works
The sample uses a DipatcherTimer to call the Update method every 100 milliseconds.
The core of the Update method:
// Create global transformations
var vw = Viewport.Width;
var vh = Viewport.Height;
var invertYAxis = Matrix3DFactory.CreateScale(1, -1, 1);
var translate   = Matrix3DFactory.CreateTranslation(TranslateX, 
                                                    TranslateY,
                                                    TranslateZ);         
var rotateX     = Matrix3DFactory.CreateRotationX(MathHelper.ToRadians(RotateX));
var rotateY     = Matrix3DFactory.CreateRotationY(MathHelper.ToRadians(RotateY));
var rotateZ     = Matrix3DFactory.CreateRotationZ(MathHelper.ToRadians(RotateZ));
var scale       = Matrix3DFactory.CreateScale(ScaleX, ScaleY, ScaleZ);
var lookAt      = Matrix3DFactory.CreateLookAtLH(CameraX, CameraY, CameraZ,
                                                 CameraLookAtX, 
                                                 CameraLookAtY, 
                                                 CameraLookAtZ);
var viewport    = Matrix3DFactory.CreateViewportTransformation(vw, vh);
var fieldOfView = MathHelper.ToRadians(FieldOfView);
var projection  = Matrix3DFactory.CreatePerspectiveFieldOfViewLH(fieldOfView, 
                                                                 vw / vh,
                                                                 NearPlane,
                                                                 FarPlane);

// Transform all elements
var selectedMatrix = Matrix3D.Identity;
foreach (var elem in this.Elements)
{
   // The UIElement
   var e = elem.Element;

   // Create transformation matrices for UIElement
   var centerAtOrigin = Matrix3DFactory.CreateTranslation(-e.ActualWidth * 0.5,
                                                          -e.ActualHeight * 0.5, 
                                                          0);
   var baseTranslate = Matrix3DFactory.CreateTranslation(elem.PositionX, 
                                                         elem.PositionY,
                                                         elem.PositionZ);

   // Combine the transformation matrices
   var m = Matrix3D.Identity;
   m = m * centerAtOrigin;
   m = m * invertYAxis;

   // Apply addtional world transformations to the seleced element
   if (elem == SelectedElement)
   {
      m = m * scale;
      m = m * rotateX * rotateY * rotateZ;
      m = m * translate;

      // Should the camera target be fixed at the selected element?
      if (ChkLookAtSelected.IsChecked.Value)
      {
         lookAt = Matrix3DFactory.CreateLookAtLH(CameraX, CameraY, CameraZ, 
                                                 elem.PositionX, 
                                                 elem.PositionY, 
                                                 elem.PositionZ);
      }
   }

   // Calculate the final view projection matrix
   m = m * baseTranslate;
   m = Matrix3DFactory.CreateViewportProjection(m, lookAt, projection, viewport);

   // Apply the matrix to the UIElement
   e.Projection = new Matrix3DProjection { ProjectionMatrix = m };
}

First the global transformation matrices like camera projection are created using the left-handed Matrix3DEx factory methods. After that the element local transformations are calculated and the final matrix is applied to the UIElement's Projection property.
See the project site for another simplified code listing.

Go and grab it

The open source Matrix3DEx library is hosted at CodePlex and released under the Microsoft Public License (Ms-PL) license. If you have any comments, questions or suggestions don't hesitate and write a comment, use the Issue Tracker on the CodePlex site or contact me via any other media.
Have fun with the library and let me know if it was useful for you.

28 comments:

  1. Very Nice, Great work

    ReplyDelete
  2. Very cool, no need to Google matrix transformations again ;-)

    ReplyDelete
  3. yes, very nice! I've always wanted a simple and straight-forward way to camera/positions/viewport

    ReplyDelete
  4. Could you post project source? I'm kinda new to silverlight. I'm waiting for version 4 coming with database.

    Paul

    ReplyDelete
  5. Hi Paul,

    this is the announcement of the open source project Matrix3DEx which is hosted at CodePlex INCLUDING the sample code of course. Just follow the link above.

    ReplyDelete
  6. I got it now Thank Rene

    I forgot to look at the source code tab. I went to download tab and got Matrix3DEx 1.0.2.0

    I like your project, now i get to play with 3D. Over the past year i only use System.Drawing which is 2D.

    Paul

    ReplyDelete
  7. Is There a way to appy a non-affine transform to two-dimensional image and reneder it on a WriteableBitmap?
    thanks a lot
    Lit

    ReplyDelete
  8. Hi Lit,

    that's possible. You can use the WriteableBitmap constructor or its Render method to apple a 2D (!) transformation before rendering:

    var wb = new WriteableBitmap(myImageControl, myTransform);

    See MSDN for details:
    http://msdn.microsoft.com/en-us/library/dd638675%28v=VS.95%29.aspx

    Or do you want to use a 2.5D perspective transformation?

    ReplyDelete
  9. Yes I want to use a 2.5D perspective transformation. I tested this solution: a lot of images with Matrix3DProjection applied to them, but it's very slow. So I would like to draw a lot of images with 2.5D perspective on a WriteableBitmap: i think this solution has to be faster. is it possible?
    thanks
    Lit

    ReplyDelete
  10. Yes, this should be faster. I guess your images are children of a Canvas or any other container. You can render the container to a WriteableBitmap:

    var wb = new WriteableBitmap(myCanvas, null);

    If Silverlight 4 is an option for you, you should try the improved GPU acceleration. The release candidate introduced HW acceleration for perspective transformations. The final Silverlight 4 build is available since Thursday.
    You just have to enable it and use a BitmpaCache for each Image control. Here's an article that explains how to use the GPU acceleration that was introduced in Silverlight 3 (it's the same for SL 4).

    http://dotnetslackers.com/articles/silverlight/discovering-silverlight3-deep-dive-into-gpu-acceleration.aspx

    ReplyDelete
  11. I've just seen that Silverlight4 introduced HW acceleration for PlaneProjection and then for affine transformations. I need to apply non-affine transformations to my images to obtain a 3D effect (with perspective). I'm using Matrix3DProjection class for it. hopeless :-(

    ReplyDelete
  12. As I wrote, the Silverlight 4 release candidate introduced HW acceleration for perspective transformations. And a perspective transformation is a non-affine transformation.
    See MSDN for details about "Silverlight Hardware Acceleration": http://msdn.microsoft.com/en-us/library/ee309563(v=VS.95).aspx

    Or do you have other information regarding the SL4 HW acceleration?

    ReplyDelete
  13. Ok, perspective transformation is a non-affine transformation (i was wrong), but if you use PlaneProjection or Matrix3DProjection you obtain different effects. I think, but I'm not sure, that Silverlight4 introduced HW acceleration only for for PlaneProjection. I have to use Matrix3DProjection.

    ReplyDelete
  14. I'm not sure about the SL 4 HW acceleration either. I will try to clarify this and come back to you.

    Have you tried the approach I suggested in an earlier comment? (Rendering the Image controls' parent container (Canvas))
    You can render the container to a WriteableBitmap:

    var wb = new WriteableBitmap(myCanvas, null);

    ReplyDelete
  15. do you think rendering the image controls' parent container (Canvas) to a WriteableBitmap is a fast solution?

    ReplyDelete
  16. It depends how often you refresh the WriteableBitmap. If you render it often it's slow, if not it should be fast. The implementation should be straight forward, just try it.

    ReplyDelete
  17. I've just tried GPU acceleration in Silverlight 4: an Image with PlaneProjection or Matrix3DProjection assigned to Projection property is GPU accelerated! :-)

    ReplyDelete
  18. That sounds great. Have you tried it with different browsers? Ho do they behave? And what is the speed up?

    ReplyDelete
  19. No sorry, I've tried it only with IE8. My test has been very simple: one rotating image with CacheMode set to 'BitmapCache'. I've set EnableCacheVisualization to 'true'; my image was in its natural colour!!

    ReplyDelete
  20. Ah, OK. Is it faster? See the frames per second (1st field top left corner).
    You should also enable EnableRedrawRegions = true. If it works you won't see any flickering.

    ReplyDelete
  21. I didn't test it with EnableRedrawRegions = true. So is it possible that my image is cached but it isn't GPU accelerated? :-O

    ReplyDelete
  22. Ok I've just tested it with EnableRedrawRegions = true: no flickering :-)

    ReplyDelete
  23. yes, very nice! Just give us the source code! Please.

    ReplyDelete
  24. its really good work but I am not able to apply it on a simple object...
    as I am new to silverlight so can u help me out??
    I have create a simple project with just a single image on main grid and now I want to test all these effects individually on that Image so how can I do that???
    Please help me....

    ReplyDelete
  25. First I want to say. Great job! The code is very clean. I like it.

    I’m considering using this library to create a 3d panorama whit hotspots.

    Can you help me on my way by giving me some pointers how to use this library for this purpose.

    Tim

    ReplyDelete
  26. Does Matrix3DEx supports winrt?

    ReplyDelete
  27. No WinRT support, but you can easily port it.

    ReplyDelete
  28. Rene, please contact me @ jnixon@microsoft. I would like to talk with about this project asap. I also reached out to you on Linked In.

    ReplyDelete