Thursday, August 20, 2009

Ye Olde Pixels - Silverlight 3 Old Movie Pixel Shader


In my latest WriteableBitmap Performance post I mentioned that the Silverlight 3 pixel shaders run really fast, although they are executed on the CPU and not on the GPU and that nice real-time effects can be done with it. A Silverlight 3 pixel shader could also be applied to any UIElement, thus the MediaElement too. The Silverlight MediaElement is mainly used to play music or video and with an attached shader many cool effects could be realized. As you might know I like nice effects and due to this passion I have coded another pixel shader for Silverlight. This time I have implemented a shader that simulates scratches, noise and other effects you might have seen in old movies.

Live

The intensity of the scratches and the noise is controlled with the Sliders. The shader could be disabled with the "Bypass" checkbox and it's also possible to load a WMV clip with the "Load" Button.
William Moore made some short video clips with After Effects for me and I have uploaded them to my Dropbox. Just copy a URL to the TextBox and press "Load" to try them with the shader:
  • Grass.wmv
  • MooreFamily.wmv

How it works
Here's the pixel shader written in HLSL:
// Parameters
float ScratchAmount : register(C0);
float NoiseAmount : register(C1);
float2 RandomCoord1 : register(C2);
float2 RandomCoord2 : register(C3);
float Frame : register(C4);

// Static
static float ScratchAmountInv = 1.0 / ScratchAmount;

// Sampler
sampler2D TexSampler : register(S0);
sampler2D NoiseSampler : register(S1);

// Shader
float4 main(float2 uv : TEXCOORD) : COLOR
{
    // Sample texture
    float4 color = tex2D(TexSampler, uv);

    // Add Scratch
    float2 sc = Frame * float2(0.001f, 0.4f);
    sc.x = frac(uv.x + sc.x);
    float scratch = tex2D(NoiseSampler, sc).r;
    scratch = 2 * scratch * ScratchAmountInv;
    scratch = 1 - abs(1 - scratch);
    scratch = max(0, scratch);
    color.rgb += scratch.rrr; 

    // Calculate random coord + sample
    float2 rCoord = (uv + RandomCoord1 + RandomCoord2) * 0.33;
    float3 rand = tex2D(NoiseSampler, rCoord);
    // Add noise
    if(NoiseAmount > rand.r)
    {
        color.rgb = 0.1 + rand.b * 0.4;
    }

    // Convert to gray + desaturated Sepia
    float gray = dot(color, float4(0.3, 0.59, 0.11, 0)); 
    color = float4(gray * float3(0.9, 0.8, 0.6) , 1);

    // Calc distance to center
    float2 dist = 0.5 - uv;   
    // Random light fluctuation
    float fluc = RandomCoord2.x * 0.04 - 0.02;
    // Vignette effect
    color.rgb *= (0.4 + fluc - dot(dist, dist))  * 2.8;

    return color;
}

The 1st effect adds some random scratches, which are moving slowly each frame.
Unfortunately the Shader Model 2 doesn't support any random function and the noise() intrinsic could not be used for real-time effects. That's a problem if you want to make pseudorandom-based shaders, but one key element of restricted shader development is pre calculation and texture lookup. The randomness, which is used in the shader, comes from a WriteableBitmap, that is filled with random color in the initialization step of the application. The shader in turn samples random values from that texture.


The 2nd effect adds a bit random noise to the image. If the texture coordinates with the frame counter would only be used, a static, moving pattern would be seen and it wouldn't look noisy at all. So actually a new random noise texture is needed every frame to achieve the desired effect. Generating a random texture with a WriteableBitmap in every frame would be too expensive. Instead of this, a little trick is used and only four random numbers are generated every frame and passed to the shader as parameters. These random seeds are then used as texture coordinates for a random lookup in the random colored texture. Therefore each pixel in every frame samples a different random value. Although it is not highly random, the values are sufficient enough to produce changing noise.


The 3rd step converts the pixel to grayscale and colors it afterwards with a desaturated Sepia tone.










The 4th and last effect computes the distance to the center and multiplies it with the color. This generates the vignette effect as a fadeout to black towards the edge. A random light flickering of an old projector is also simulated.








That's it and it's all about cheating.

Source code
Feel free to download the Visual Studio 2008 solution including the pixel shader from here.

Update 12-29-2009
I refactored the code behind shader class and separated it completly from the calling code (MainPage.xaml.cs). Now the shader class generates a default noise texture and random coordinates itself.
The linked source code was updated.

Update 12-31-2009
See this blog post if you want to try the shader in real time with your own webcam.

6 comments:

  1. Sorry I missed getting the video to you. I had two done but i did not have your email. Way nice effect btw i am going to spend some time dissecting it.

    ReplyDelete
  2. @codenenterp
    Thanks for making this video clips for me. I really appreciate your effort!
    I uploaded your videos to Silverlight Streaming and updated the blog post. The one with the lightning works really well with the shader effect.

    ReplyDelete
  3. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.


    Sara

    http://pianonotes.info

    ReplyDelete
  4. Very nice shader. I'm currently implementing it in my own project, so I have a few questions. What is the range RandomCoord1 and RandomCoord2?
    What exactly is Frame? A frame counter?

    Thanks again for a great shader.

    ReplyDelete
  5. Thanks!

    As I wrote in the post, the RandomCoords are random seeds that are used as lookup coordinates for the random texture:
    "[...] Generating a random texture with a WriteableBitmap in every frame would be too expensive. Instead of this, a little trick is used and only four random numbers are generated every frame and passed to the shader as parameters. These random seeds are then used as texture coordinates for a random lookup in the random colored texture. Therefore each pixel in every frame samples a different random value. Although it is not highly random, the values are sufficient enough to produce changing noise. [...]"

    The Frame parameter is a framecounter and used to animate the scratch scan lines.
    You should checkout the C# source code: Each frame the "Frame" variable is incremented and new random RandomCoords are computed. In the initialization step, the texture with random values is generated. Combinig these produces good pseudo-random values.

    Please inform me if you are using my old movie pixel shader. Will it be online somewhere?

    ReplyDelete
  6. Thanks for the shader. It's been a couple of years later and you might wanna look into https://github.com/ashima/webgl-noise/wiki to optimize this shader.

    ReplyDelete