There are many cool effects, which are not already part of the WPF Pixel Shader Effects Library, one could implement. So only the sky is the limit - and the Shader Model 2.0 instruction count limit of course.

Silverlight pixel shaders could be written in HLSL and compiled with the DirectX shader compiler fxc. The produced binary file is then loaded with the Silverlight 3 PixelShader class. Quite easy huh? I must admit that I'm not a complete newbie to shader development. I even wrote some shaders with Shader Model 1.1 in the assembler shading language a few years ago, but haven't done it a while.

For Silverlight 3 I've implemented an edge detection post processing effect. It's a parametric pixel shader, which performs a common image processing technique called convolution.

**Live**

The initial image "Lenna" is a famous test picture for image processing algorithms. Daniel Collin (@daniel_collin) pointed me on the interesting story behind that picture of Lena Söderberg. Although Lena is a pretty lady, you should try another image too.

You can control the threshold of the edge detection with the Slider and disable the shader with the "Bypass" CheckBox. Select one of three preset convolution operators from the ComboBox: Scharr, Prewitt and the well-known Sobel operator. It's also possible to change the first column of the Gx convolution kernel to try your own operator. Actually two 3x3 convolution kernels are used by the algorithm. One for the horizontal (Gx) and another one for the vertical (Gy) direction. The Gy is just a 90° rotation of Gx and the last column of Gx is the inverse of the first column. The middle column is zero. So instead of 18 parameters (3x3x2) only 3 parameters need to be passed to the shader.

**How it works**

I used the Shazzam Tool for the shader development. It's a nice tool to write and test shaders for WPF. It also generates a corresponding class for the pixel shader, which is then used in XAML. Most of the controls take advantage of Silverlight's 3 great (Element) Data binding mechanism.

There's really nothing more to write about the Silverlight application, all the magic happens in the pixel shader:

// Parameters float Threshhold : register(C0); float K00 : register(C1); // Kernel first column top float K01 : register(C2); // Kernel first column middle float K02 : register(C3); // Kernel first column bottom float2 TextureSize : register(C4); // Static Vars static float ThreshholdSq = Threshhold * Threshhold; static float2 TextureSizeInv = 1.0 / TextureSize; static float K20 = -K00; // Kernel last column top static float K21 = -K01; // Kernel last column middle static float K22 = -K02; // Kernel last column bottom sampler2D TexSampler : register(S0); // Shader float4 main(float2 uv : TEXCOORD) : COLOR { // Calculate pixel offsets float2 offX = float2(TextureSizeInv.x, 0); float2 offY = float2(0, TextureSizeInv.y); // Sample texture // Top row float2 texCoord = uv - offY; float4 c00 = tex2D(TexSampler, texCoord - offX); float4 c01 = tex2D(TexSampler, texCoord); float4 c02 = tex2D(TexSampler, texCoord + offX); // Middle row texCoord = uv; float4 c10 = tex2D(TexSampler, texCoord - offX); float4 c12 = tex2D(TexSampler, texCoord + offX); // Bottom row texCoord = uv + offY; float4 c20 = tex2D(TexSampler, texCoord - offX); float4 c21 = tex2D(TexSampler, texCoord); float4 c22 = tex2D(TexSampler, texCoord + offX); // Convolution float4 sx = 0; float4 sy = 0; // Convolute X sx += c00 * K00; sx += c01 * K01; sx += c02 * K02; sx += c20 * K20; sx += c21 * K21; sx += c22 * K22; // Convolute Y sy += c00 * K00; sy += c02 * K20; sy += c10 * K01; sy += c12 * K21; sy += c20 * K02; sy += c22 * K22; // Add and apply Threshold float4 s = sx * sx + sy * sy; float4 edge = 1; edge = 1 - float4( s.r <= ThreshholdSq, s.g <= ThreshholdSq, s.b <= ThreshholdSq, 0); // Alpha is always 1! return edge; }The colors of the current pixel's neighbors are sampled from the image, which are then multiplied with the corresponding kernel value. The results are summed up, the threshold is applied and the new color is returned. I optimized the operation a bit more by calculating some static variables, using the squared threshold and leaving the calculation for the zero kernel column / row out.

**Source code**

The Visual Studio 2008 solution including the pixel shader is available from here.

**Update 12-30-2009**

See this blog post if you want to try the shader in real time with your own webcam.

*Razor photo by Jake Sutton*

WOW!!!

ReplyDeleteYou are my new guru!!!

Thanks for the Silverlight series, I found here info really useful, thanks for sharing your project.

Dr.Luiji

My computer shoe me an error that said something like float2 texCoord = uv - offY;

ReplyDeletefloat4 c00 = tex2D(TexSampler, texCoord - offX);

float4 c01 = tex2D(TexSampler, texCoord);

float4 c02 = tex2D(TexSampler, texCoord + offX);!!!! I do not what is it about!

hello, i think that your post is very good.

ReplyDelete