Thursday, December 31, 2015

Keep on Climbing - Recap of 2015

It's the last day of 2015 and therefore time for the annual recap of the year. For me it was a great year, on the private side of life as well as the professional life.

I'm still working from home for one of the best interactive agencies in the world with a group of fantastic people. I'm happy that I got back into computer graphics programming this year being able to work in the emerging space of virtual reality with devices like the Oculus Rift, Google Cardboard, Samsung Gear VR and more. Plus the next chapter which is being opened with new innovations in the field of Augmented and Mixed Reality.
I was also able to do a few talks at conferences and honored to speak at the //build developer conference again this year, this time together with my good friend and colleague Laurent.
Oh, and I got promoted to Lead UX Developer as well in 2015.

It was also a great year in my private life. My wife and I had our 10 year wedding anniversary and I love her more every day. Our four daughters are doing fine in (pre-)school and bring us joy every day.
For me personally it was an intense change too when I decided in February 2015 that it's finally time to loose some weight. By now I lost about 35 kilograms (77 lbs) by decreasing the energy input with (unhealthy) food and increasing the energy output with cycling. I likely turned a few more kilograms from fat into muscles through the training as well. I did some sweet bike rides like a nice 113 km tour around Lake Washington and Lake Sammamish in October when I was in the Seattle area and another 100 km+ ride a few days ago climbing into the Erzgebirge (Ore Mountains). In December alone I managed to ride more than 500 km while climbing 5200 meters up. Glad I reached this point considering I had 35 kg more to carry when I started in February.

For the next year I want to reduce my weight just a bit more and I'm also planning to ride my first Mountainbike Cross Country Marathon, the famous Erzgebirge Bike Marathon and some other nice challenges.
In the tech field I hope to be renewed as Windows Platform Dev MVP and being able to work more in the field of VR, AR and MR. I have the feeling it's another UX revolution happening right now. It's an exciting time to be a developer for sure.

Bring it on 2016!

Thursday, December 3, 2015

IdentityMine Post - Client-side caching matters

I was recently part of a project at IdentityMine where worked on a Xamarin application for the Amazon Fire TV Android platform. We faced some challenges which we love to address. One of which was the backend speed; however, this feature was out of our scope. Still, the client application needed to interact with the backend speed. Otherwise, with no data, there is no application.

I wrote a post about the learning experiences for the IdentityMine blog. If you are interested in client side development you might want to read it.

Wednesday, September 23, 2015

We've Come a Long Way Baby - WriteableBitmapEx now on GitHub

The WriteableBitmapEx project is now 6 years old and times have changed since all began. It all started with Silverlight and more XAML flavors were added over time until all XAML flavors were supported including Windows Phone, WPF, WinRT Windows Store XAML, (Windows 10) UWP and still Silverlight.

The version control system and open source services landscape has changed as well and the CodePlex infrastructure has not been too reliable recently. That's why I finally took the time to move the WriteableBitmapEx source code from CodePlex' SVN bridge to Git and make it available on GitHub
Additionally was the open source license changed from Ms-PL to MIT which should simplify the usage as well. See the Codeplex Issue Tracker for a list of features that will be added in the future. Please use the GitHub Issues functionality to add new issues which are not already reported. Of course the latest binaries will continue to be distributed via NuGet.

Friday, May 22, 2015

My Top 5 Wishes for Microsoft Band and Health

You might have seen my previous blog post ZzzZzz - Microsoft Band Sleep Tracking in the Testbed. In that article I compared the Microsoft Band and Health data to a proven scientific method for sleep monitoring. While writing it, a couple of ideas and suggestions came to my mind for the Band in general and some particular for the Sleep tracking feature.

Here's my Top 5:
  1. Pulse Oximeter:
    Oxygen saturation is a key metric for medical analysis. For example the result of sleep apnea can be seen as a drop in oxygen saturation and many other dysfunctions surface through that metric. It's also interesting for fitness and sports in general and some fitness trackers actually provide this data already.
    The Microsoft Band measures the heart rate optically using a photo detector and this is the same principle used for Pulse Oximetry, so the sensor is already there. I think it's just a matter of adding the right algorithm and surfacing the data in Microsoft Health.

  2. Sleep tracking Precise mode:
    I would really love if the Band's sleep tracking had a Precise mode which would take samples more often during a night and was allowed to run down the battery in 8-10h, so I can decide when the precision-battery-tradeoff is applied.

  3. Re-occurring alarms:
    The vibration alarm is awesome but it needs to have a feature so it can be made re-occurring. The Band could just sync with my Windows Phone alarms but should also work when the Band is not connected to the device via Bluetooth.

  4. Smart alarm:
    Although the sleep stages are not 100% reliable it would still be good enough to wake the user up during a certain timeframe when the sleep is not Restful anymore. Lots of fitness trackers provide that feature already which actually have less sensor data.

  5. Notifications:
    Currently the Notification Center and Mail tiles stay at the count until the tiles are opened. This is rather annoying as it is another notification hub which needs to be cleared in addition to the phone. I'm sure it's a technical challenge with the WP APIs but for vNext it would be great if the Mail and Notification Center tile count would also be synced over Bluetooth when those are cleared on the phone.

ZzzZzz - Microsoft Band Sleep Tracking in the Testbed

I'm a perfectionist and hungry for knowledge, so I went to a sleep clinic just to check how well the Microsoft Band sleep tracking is working compared to proven scientific methods.
Unfortunately it's not like that and I actually had to go to the sleep lab since I have some issues there. Anyway, since I had to go to the sleep laboratory in the clinic and I just got my Microsoft Band I took the occasion and also used my Band's sleep tracking feature while being monitored using the lab's Polysomnography sleep study.
After my 2 nights at the clinic I asked the lab staff and physician if I can get some personal printouts of my results. They were all nice people and the printouts provide some good data for comparison with the Microsoft Band and Health data.
A funny anecdote: The lab's senior physician is actually a big Windows Phone enthusiast and even tried Windows 10 for Mobile on his device so we had some good chats, also about HoloLens and how it can help them...

The science of sleeping
Sleep studying is a huge scientific area with a lot of active research and many parts of sleep are still unknown territory. I'm not a sleep research expert but learned a few things recently.
There are a couple of different methods for sleep monitoring / tracking. The gold standard for sleep labs is the Polysomnography (PSG) which uses a bunch of sensor data like EEG, ECG, oxygen saturation, eye movement, breathing sensors, muscle movement, etc. For this 2-3 dozen cables are connected to your body for a PSG sleep study.
Another, more basic method is called Actigraphy which just tracks the movement and maybe a few other parameters so it's only used to get information when a person went to bed and when they got up but it can not provide solid data about sleep stages as those depend mostly on your brain activity. The sleep features of fitness trackers like the Jawbone Up, Fitbit, etc. are basically actigraphs. The folks of Exists wrote a very good blog post about those devices and also link to some research studies about actigraphy and how well it works in comparison to PSG. The Microsoft Band also uses actigraphy plus some additional sensor data like samples of heart frequency. Maybe it's even using the skin temperature sensor as well.

Science basically defines five stages of sleeping: The phase when you are awake (W), the rapid eye movement (REM) phase when you are dreaming and 3 Non-REM phases where NREM1 (N1) is the transition to sleep, NREM2 (N2) is light sleep when the heart rate starts to decrease and NREM3 (N3) defines deep sleep when you are fully relaxed and difficult to awaken.

Enough of the theory...

Analysis of Night 1
The first night at the lab was pretty bad. All those cables and a clinic bed were very distracting so my actual sleep was very short and the phases not at all optimal. Keep this in mind when looking at the graphs. My home sleep data is usually way better.
In the table below I listed the parameters the Band provides and compared those to the Lab data. I omitted the total Duration (TIB), Time to fall asleep and Sleep efficiency since the lab and Band measuring started/ended at different times and those parameters depend on that. The Resting heart rate is also excluded as no clear comparable data is available in the PSG printouts, but looking at the graphs the Band Resting HR is somewhat correct.

Parameter Band Lab PSG Band error [%]
Actual Sleep (TST) [min] 257 319.5 -20%
Light Sleep (N1 + N2 [min] 184 271.6 -32%
Restful Sleep (N3) [min] 73 30.3 141%
Woke up 10 9 11%

As expected does the Band actigraph not provide as accurate data as a PSG and the sleep stages like Light sleep and Restful sleep don't match the actual scientific sleep stages very well.

In the below illustrations I overlaid the Microsoft Band's Health sleep charts (colored bars) with the PSG sleep stage graph (black lines). The black scan of the PSG printout was scaled to match the correct time and size of the colored Band data.

Night 1: Colored Band stages data overlaid with black PSG stages

This overlay is pretty interesting and tells quite a bit. Note the Wake phases match mostly but since the Band sampling only happens every few minutes it does not record higher frequency changes and therefore misses short naps or awake periods.
The REM phase is considered Light sleep by the Band which makes sense. Band's Restful sleep is mostly N3 but also a good portion of N2. I assume the Band only uses the heart frequency to define the Restful stages and since the heart rate starts to decrease in N2 the band labels the Restful stage sometimes in N2 until the body goes into N3. The next two overlays underline this theory.

Night 1: Colored Band stages data overlaid with black PSG heart rate
In the above illustration the PSG heart rate was overlaid on the Band's sleep stages and the heart rate chart. As can be seen the Band's Restful blocks correlate with a longer, lower heart rate period.

The third illustration below has the PSG heart rate on top of the Band's heart rate chart.

Night 1: Band's heart rate data (thick purple line) overlaid with black PSG heart rate

Again, the Band's low sampling frequency for sleep mode does not record higher frequency changes like peeks in heart rate which can be caused by sleep apnea when the body tries to compensate low oxygen saturation. Improving those measurements is one of my top wishes / suggestions for the Microsoft Band / Health service.

Analysis of Night 2
The second night was a bit better and I felt more rested after it. I guess mostly because I was super tired from night 1 and therefore slept better even with all those PSG sensors attached.

Parameter Band Lab PSG Band error [%]
Actual Sleep (TST) [min] 310 298.5 4%
Light Sleep (N1 + N2 [min] 183 199.3 -8%
Restful Sleep (N3) [min] 126 70.1 80%
Woke up 9 8 13%

If the Night 1 and Night 2 data is compared it can be seen that my subjective impression was not wrong. I got more than twice deep sleep (N3) during Night 2.
The Band vs. Lab PSG also seems to have less errors which leads to the conclusion that higher sleep quality produces better Band results. I'm not sure how the Microsoft Health analysis of the Band tracking data actually works but I wouldn't be surprised if it uses Azure's machine learning service and the used training datasets contain mostly healthy and normal sleep patterns.

Night 2: Colored Band stages data overlaid with black PSG stages

Again, the Band labels longer periods of N2 and always N3 as Restful sleep.

Night 2: Colored Band stages data overlaid with black PSG heart rate

Night 2: Band's heart rate data (thick purple line) overlaid with black PSG heart rate

The Microsoft Band sleep tracking feature is a nice gimmick but it's not really surprising it doesn't look great when compared to proven scientific sleep monitoring. The Band is not bad for the total sleep duration data, the actual sleep time and the wake up phases. The different stages like Light and Restful sleep are not terribly wrong either but should not be trusted too much. The analysis of sleep stages requires actual brainwave data which is not provided by the Band sensors. It seems the Band's Restful stage is mostly based on the heart rate and movement.
Of course the Band should not be used to diagnose sleep disorders like sleep apnea due to missing sensors but also the low frequency sampling in sleep mode. Hopefully this can be improved. Still if you feel tired more than usual and the Band / Health sleep patterns are totally off, go see a doctor soon!
In the end the Band is a fitness tracker and not a scientific measurement device. The Band and Microsoft Health's sleep tracking is still a nice feature to get an overall idea about your sleep patterns if you keep the limitations in mind.

Happy tracking!