Thursday, July 4, 2013

Just call us, we'll might call you - How to add Pictures Lab to your App

Windows Phone 8 introduced a nice new feature called File and Uri associations. This feature makes it possible to register an app for certain file extensions and Uri protocols. A couple of apps already support it and Nokia has a nice site which lists a couple.

Pictures Lab recently added support for that as well and uses it to call the app Instagraph. This means with the help of that app Pictures Lab now provides posting / sharing on Instagram. But that's not all, Pictures Lab also supports inbound Uri associations, so you can pass an image to Pictures Lab through the mechanism of an Uri association pretty easily.




How it works
In order to call Pictures Lab from an app this scheme has to be used:

pictureslab:edit?Name=fileNameinthemedialib[&ClientId=ProductIdFromStore]
or
images:edit?Name=fileNameinthemedialib[&ClientId=ProductIdFromStore]
  • fileNameinthemedialib:
    This is the filename of the image in the Media library / Pictures Hub. It does not matter if the file extension like .jpg is provided or not. 
  • ClientId:
    This is an optional parameter. If it's provided then it should be the GUID that also matches the store product Id. It's not used in Pictures Lab now, but might be useful in the future to know the source of the protocol call and one could also use it to build a link like that: http://www.windowsphone.com/s?appid=ClientId

or to just launch Pictures Lab without passing an image:
pictureslab:launch

Sample code:

var photoChooser = new PhotoChooserTask();
photoChooser.Completed += async (sender, e) =>
{
   // When the user selected a photo, pass the filename on to Pictures Lab for editing
   var uri = string.Format("images:edit?Name={0}&Clientid={1}", 
             System.IO.Path.GetFileName(e.OriginalFileName), 
             "9902dac1-c001-440b-add5-f38446b35676");
   // uri = "images:edit?Name=WP_20130527_009&Clientid=9902dac1-c001-440b-add5-f38446b35676" 
   var wasSuccess = await Windows.System.Launcher.LaunchUriAsync(new System.Uri(uri));
};
photoChooser.Show();

Note, it's not necessary to UrlEncode the Uri, the Windows Phone platform handles it automatically. If the user has Pictures Lab not installed and the above Uri is called, the Windows Phone OS is also smart enough to show the user a dialog which asks to search for an app in the Store that can handle the protocol. This will bring up Pictures Lab.

Conclusion
The concept of Uri associations is like the little brother of Windows 8's Share Charm, but it's still quite powerful and gives us app developers a nice way to combine our apps and make it a seamless UX. It's a win for the user and therefore it could be a win-win situation for the involved apps.

Future
For a next version I'm also thinking about a callback scenario. The user story would be: A user opens app A and wants to edit an image, this will open Pictures Lab, when the user is done with editing, Pictures Lab will call app A again. The user is back in app A with the edited image.
This callback mechanism would probably use a Callback parameter in the scheme above where app A can provide the UrlEncoded callback Uri with a placeholder for the edited pictureName. Something like that:

images:edit?Name=MyInputImageName.jpg&Callback=myapp%3Apost%3FName%3D%7B0%7D
The Url decoded part of the Callback: myapp:post?Name={0}
This callback format string will be used in Pictures Lab to call app A and pass the edited picture back.

Makes sense?
Please let me know your thoughts and if that would be useful. Feel free to comment or reach out to me through any channel.

Tuesday, March 19, 2013

Kinect and Leap Motion in Love

Natural User Interfaces are a trend since a few years and the development of new hardware technology for affordable prices helps to push that movement into the consumer market. The Kinect is one example, the soon-to-be-available Leap Motion is another one or the announced MYO device.

I'm one of the lucky people who applied and actually got a Leap Motion developer device before its release. I also have a Kinect for Windows. Both devices use computer vision with various sensors combined with artificial intelligence like machine learning for the object recognition. The Kinect for Windows recognizes skeletons with joints in a distance of  80 - 400 cm or 40 - 300 cm (Near Mode) which means the Kinect is perfect for the mid range distance but not for close PC interaction. The Leap Motion on the other hand is more for the close range and detects hands, fingers and pen-like objects very precisely. That's why I wanted to combine them and build a proof of concept hybrid solution that leverages both Kinect and Leap Motion to get the best of both worlds (close and mid distance).

In the video below I demo my proof of concept which is based on the Kinect for Windows SDK 1.7 and its Interactions and Controls components, plus the Leap Motion developer device and its beta SDK.
By the way, the Kinect for Windows SDK 1.7 is now publicly available and the bits are nothing less than awesome.




In case you are wondering about the infrared interference of the Kinect and Leap Motion. There's some, but I actually didn't notice any serious effect on the recognition capabilities of the Kinect nor the Leap.
Below is a picture of the Leap Motion as seen through the Kinect's infrared stream. The Leap Motion has three light sources but those are not very bright.



Some might say the Leap Motion is a competitor to the Kinect for Windows, I see it more like a nice addition to our developer toolset and the beginning of a story: Kinect ♥ Leap Motion.

Wednesday, December 12, 2012

Robot Work - Reducing the Windows Phone Localization Friction a bit

Windows Phone 8 now supports 3 different screen resolutions which is very nice since there are some great high resolution devices out there and the user has the choice and isn't forced into a one fits all model. However  there's also a downside for Windows Phone developers since now also 3 times the amount of screenshots have to be submitted to the Store. If an app has localized versions this adds up very quickly. For example, Pictures Lab supports 11 languages and the soon-to-released v5 update for WP8 also all 3 resolutions: 11 x 3 x 8 screenshots = 264! That's a crazy amount of mindless click-through work to take all those screenshots, not to mention the Dev Center submission process.
Unfortunately there is no automated way or better support by the tools, so one has to do this ridiculous amount of manual work.  In a future version of the tools I'd really love to see some kind of macro recorder which records certain steps including screenshot actions and can then use the recorded steps to play this for all supported app languages and generate the remaining screenshots automatically.

In this quick post I want to share some practices I used in order to reduce the work at least a bit for the Pictures Lab v5 update:
  1. Only create screenshots for the 720p (720 x 1280) and the WXGA (768 x 1280) resolution. The WVGA (480 x 800) screenshots can be generated from the WXGA images by scaling those down by 1.6. In contrast to upscaling, are the downscaling artifacts hardly visible if a good algorithm is used. I used my favorite image viewer IrfanView and its batch processing feature which can scale down by the longest side, rename by a pattern, convert, etc.



  2. Add some code to easily switch between UI languages in your app so you don't only rely on the system language and need to reboot the emulator to switch languages. An app's UI language can be set globally by using
    System.Threading.Thread.CurrentThread.CurrentUICulture
    For example:
    System.Threading.Thread.CurrentThread.CurrentUICulture = new CultureInfo("de-DE");
    in the App.xaml.cs Launching event.

  3. Use a macro recorder / scripting to save a bit time with the repetitive tasks of taking screenshots only with different UI languages. I used a little free tool called Mouse Recorder Pro. This tool records your mouse and keyboard actions and can then play it back also with up to 40% faster playback speed. Of course this doesn't work for all apps and only when using the emulators, esp. uncontrollable live content would likely not work, but it worked for my scenario quite well and saved me from creating all 264 screenshots manually. In fact I watched the magic ghost mouse do its job most of the time and worked on other stuff.
    A few gotchas when using the Mouse Recorder Pro with the WP emulators:

    1. Don't click on Pivot headers and rather use the swipe gesture to change between Pivot items. The header texts will have different lengths in different languages.

    2. Try to avoid the bounce effect of the ScrollViewer, it would result in random offsets and the Mouse Recorder playback will then have non-matching offsets recorded.

    3. Do a dummy save in the destination folder for the screenshots before each playback so the playback will save the files to the right folder. This will avoid file name collisions which would need manual input.
Of course there's a lot of room for improvements, but maybe the described practices help to reduce the friction that comes with localization at least a bit until a better solution is known.


Thursday, November 1, 2012

WACK ARM

If you are developing Windows 8 Store apps you should really test those on low-powered x86 and ARM devices. The difference to your high-end x64 developer machine can be dramatically, esp. if you use a non-default design.
Testing on ARM is important and Tim Heuer wrote a nice post which shows how to remotely deploy, debug and profile on ARM devices right from Visual Studio 2012. Another nice thing which performs a couple of automated tests is the Windows App Certification Kit (WACK). Unfortunately is the WACK tool for ARM hidden inside the Windows 8 SDK, but here's how to find it:

  1. Download the Windows 8 SDK web setup from here and run it.
  2. Choose to download the SDK files for offline usage.
  3. On the "Select the features you want to download" screen it's enough if you only select the "Windows App Certification Kit" checkbox.
  4. After the download is finished, you will find an "Installers" subfolder in the download target folder.
  5. Copy the "Windows App Certification Kit arm-arm_en-us.msi" from the "Installers" folder to an USB key or SkyDrive.
  6. Open the "Windows App Certification Kit arm-arm_en-us.msi" on your ARM device and install the WACK on it. 
  7. Search for "Windows App Cert Kit" on your ARM device using the Search Charm and run the WACK.

Tuesday, October 30, 2012

Faster! - WriteableBitmapEx for Windows Phone 8 and WinRT Updated

The Windows Phone 8 SDK is now available and with that WriteableBitmapEx was updated too. The performance of the WinRT XAML version was also improved dramatically and an update is highly recommended.
WriteableBitmapEx is available for 4 platforms: Windows Phone 8 and 7, WPF, Silverlight and Windows Store WinRT .NET XAML.
You can download the binaries here or via the NuGet package. The packages contain the WriteableBitmapEx binaries. As usual all samples and the source code can be found in the repository.