Friday, November 20, 2009

EdgeCam - Silverlight 4 Webcam & Edge Detection Shader

Only 4 months after Silverlight 3 RTM came out Microsoft released the first beta version of Silverlight 4 on 11-18-2009 at the PDC. The new features are just awesome and many of them were requested by the community. Tim Heuer's great blog post covers all the new goodies in detail and provides videos and source code for most of them.
One of my favorite new features is the webcam and microphone support and I've coded a small demo that uses the new Silverlight 4 CaptureSource class and applied my edge detection pixel shader to it to create a cool real time webcam effect.

Live
To view the application you need to install the Silverlight 4 runtime. It's available for Windows and Mac. You can also watch a video below.



The Webcam capturing could be started and stopped with the Button in the middle. If you press it for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the new "Webcam / Mic" tab to set them.
The threshold of the edge detection can be changed using the Slider and the "Bypass" Checkbox allows you to disable the shader.

Video
I've recorded a short video with my iPhone 3GS that shows the demo running on my Samsung NC 10 Atom Netbook. The built-in webcam is not very good and the video was recorded at night, but I think it's good enough to see how the effect looks.


How it works
The new Silverlight 4 webcam and microphone API is pretty easy to use:
// Init capture devices
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
captureSource.AudioCaptureDevice = CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice();

// Create video brush that preserves the aspect ratio and fill a Rectangle with it
VideoBrush videoBrush = new VideoBrush();
videoBrush.Stretch = Stretch.Uniform;
videoBrush.SetSource(captureSource);
Viewport.Fill = videoBrush;

// Ask user for permission
if (CaptureDeviceConfiguration.AllowedDeviceAccess
|| CaptureDeviceConfiguration.RequestDeviceAccess())
{
   captureSource.Start();
}
The edge detection pixel shader is applied to the "Viewport" Rectangle in the XAML. For a more intuitive behavior I've also attached a negative ScaleTransform to flip the x axis of the Rectangle. Otherwise the webcam output would be mirror-reversed by default.

Source code
The Visual Studio 2010 solution including the edge detection shader is available for download.

Update 11-23-2009
Make sure to read the follow up blog post EdgeCam Shots - Saving Silverlight 4 Webcam Snapshots to JPEG

Update 03-20-2010
Updated to the Silverlight 4 release candidate.

Update 04-15-2010
Updated to the final Silverlight 4 RTW build.

6 comments:

  1. Thanks a lot René. Is there an easy way to use your aplication offline? I need to create a window inside a program but I've no coding skills (I'm working with autoit, a Visual Basic like language).

    Thanks for your contribution.

    ReplyDelete
  2. Hello ! I'm using your solution and i'm getting this error: "Microsoft JScript runtime error: Unhandled Error in Silverlight 2 Application Value does not fall within the expected range. at MS.Internal.XcpImports.CheckHResult(UInt32 hr)at MS.Internal.XcpImports.CaptureGraph_Start(CaptureSource Source)at System.Windows.Media.CaptureSource.Start()at EdgeCam.MainPage.StartStopCapture()at EdgeCam.MainPage.BtnCapture_Click(Object sender, RoutedEventArgs e)at System.Windows.Controls.Primitives.ButtonBase.OnClick()at System.Windows.Controls.Button.OnClick()at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)at System.Windows.Controls.Control.OnMouseLeftButtonUp(Control ctrl, EventArgs e)at MS.Internal.JoltHelper.FireEvent(IntPtr unmanagedObj, IntPtr unmanagedObjArgs, Int32 argsTypeIndex, Int32 actualArgsTypeIndex, String eventName)."

    Can you help me, please? My OS is Windows 7, 32 biti and my webcam is working on other application.
    Thank you for your time !

    ReplyDelete
  3. I have not idea what is captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
    captureSource.AudioCaptureDevice = CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice();

    about!!

    ReplyDelete
  4. Hi Friend,
    You did really a great job. I found your blog very interesting and very informative. I think your blog is great information source & I like your way of writing and explaining the topics. Keep it up. I'm going to follow your blog.

    ReplyDelete
  5. hy i m gettn ur code regardin webcam bt it cant show on my webpage.. i also run ur downloadr file it also nt run ... i run that 2 pages TakingMyPictureTestPage.html and TakingMyPictureTestPage.aspx .. So give me solution as soon as possible..

    or mail me puneetisonfire89@gmail.com

    ReplyDelete