ARCore, ARKit and HoloLens - How does it compare?

Google released a preview of its Augmented Reality platform ARCore. In this post I provide some thoughts and resources about how it compares to the competition. My earlier post is more about Getting Started.

I'm into 3D and AR development before it was cool and even did an ARToolkit port for Silverlight back in the days. Of course I also had to give ARCore a try.

The release of ARCore was nothing less than an answer to Apple's ARKit, although most of the ARCore algorithms originate from Google's Tango which is being worked on since years. Both provide a similar feature set with motion tracking, horizontal plane detection and ambient light estimation.
I don't own an Apple development environment so I can't compare it directly to ARKit but I tried out some of the experiments myself on a colleague's iPhone a few weeks ago. ARKit's tracking was really stable and I think ARCore is a bit behind but it's hard to say how it compares now without having my own side-by-side comparison. This video though seems to confirm this observation and it's not surprising considering Apple owns the whole stack and can calibrate their sensors and align the software much better. The next generation of their mobile devices will surely even more improve it and Apple can move a bit faster here  but will users care about those slight tracking differences? I doubt it.
There's a great in-depth article comparing Apple's ARKit and Google's ARCore which also covers the tech under the hood: How is ARCore better than ARKit.

Wikitude and Vuforia are two larger third party libraries that provide SLAM-based marker-less tracking for many platforms since quite a while and much before ARKit / ARCore were out. I'm sure they will continue to innovate ahead but I fear both will be become less and less relevant since the licensing cost and third party SDK integration is always a big disadvantage. Even more important can Google and Apple really influence the hardware manufactures and improve the calibration to reduce the tracking errors to its minimum.

I'm a HoloLens developer since day one and actually already before it was publicly available, so I could compare it to Microsoft HoloLens but ARCore/ARKit play in a different league and actually complement each other.
The Microsoft HoloLens is a revolutionary device that might be a bit ahead of its time. The HoloLens shows us how mobile devices will work in the future. The inside-out tracking running on a custom chip is still leading and unseen in any other untethered, mobile device and the NUI input is top notch with voice and gesture input. The stereoscopic 3D see-through displays of an HMD provide a much more immersive UX seeing the world around you with your own eyes than holding up your phone/tablet in an unnatural way to see the world though a monoscopic 2D camera lens on a mobile phone screen.
ARCore/ARKit are taking a different approach and bring AR to the masses on existing hardware and there's a place for all 3. I can easily imagine building cross-platform solutions which target high-end experiences with the HoloLens for a smaller enterprise audience and a counterpart app with Google ARCore / Apple ARKit for the masses. Another benefit of classic mobile AR like ARCore/ARKit is the outdoor support and having easy access to GPS location data which I leveraged in this demo.

The news about HoloLens v2 with the new HPU / AI chip are exciting as it will open amazing use cases. Microsoft is definitely leading the industry with innovation in the AR/MR space and no competitor comes close to its bleeding edge technology. HMDs like the HoloLens are the future of mobile AR/MR but until HMDs will be become cheaper and in forms factors everyone can enjoy, ARCore and ARKit are great additions to the AR/MR landscape providing AR on current gen mobile hardware driving mass adoption and helping to mature the product design challenges.

It's a great time to be a 3D developer!

Below a few more GIFs from my experiments. See all in full length at my YouTube channel.

At the ARCore of Tango - Getting Started

Google recently released a preview of its Augmented Reality platform called ARCore which is based on the amazing R&D that went into Google Tango.

This post provides a collection of useful information I came across and links to the sources of my own ARCore demos.

Google provides a few options for ARCore development depending on what your favorite development environment is: JAVA, Unreal, Web or Unity3D.
There's even a Xamarin port of the JAVA sample available made by my fellow MVP Morten.
The Getting Started with Unity is a great How To I successfully followed.

You can find some videos of my experiments with Google ARCode on my YouTube channel, like the one in the GIF above where I combined Unity Rigid Body physics and added a MeshCollider to ARCore's detected horizontal planes and also used ARCore's ambient light estimation feature.

The source code for my Unity3D ARCore experiments is published on GitHub: ARCoreExperiments.

I own a Samsung Galaxy S8 which is fortunately one of the supported Android devices. If you don't own one of those devices you might still be lucky using this hack.

Google also hosts a nice collection of AR Experiments some of them even include source code.
One of them is Portal Painter by Jane Friedhoff. She also wrote a great blog post how it was built.

Have fun and build some cool stuff!

Below a few more GIFs from my experiments. See all in full length at my YouTube channel.

Summer Silverlight Coding Competition

ComponentArt is hosting the "Summer Silverlight Coding Competition" from June 22nd to Semptember 22nd:

ComponentArt is excited to host the 2009 Summer Silverlight Coding Competition and award $10,000 USD to the author of the best application as selected by our expert panel and the community. Authors of the two runner-up applications will each receive ComponentArt licenses (a $1,299 USD value).

I submitted my Silverlight website and I am very happy that it was published yesterday. The community voting is now open. If you like my Silverlight real-time physics website it would be nice if you vote for it here. Unfortunately you need a ComponentArt.com account to cast your vote, but the registration is quickly done.

Silverlight 2 Real-time Physics Website

It's been 5 years since I developed my previous website in 2004 with Macromedia (Adobe) Flash. A couple of months ago I decided to work on a makeover, which was long overdue. As a media & computer scientist and .Net developer it was a matter of course to give Microsoft's Silverlight 2 a try. I love the ability to write the code in my favorite language C# and having most functionality of the .Net library at hand.

Lightning it up

However, the first Show Stopper was the lack of basic HTML or rich text support in Silverlight 2. I was quite surprised and disappointed that Silverlight Text controls don't provide basic HTML formatting or at least WPF's Inline Hyperlink. I think a lot of people need such a feature. Some guys on Twitter told me that there isn't really a demand for this and it could be done with other techniques, but sometimes performance does matter and an overlayed DIV using Silverlight's windowless mode is just terribly slow.

HTML Link support is essential for this project, because the HTML formatted text is stored in XML files and loaded asynchronously by the ContentPages and then deserialized to POCOs. The controls in turn use Silverlight's great Data Binding mechanism to present the data and load the images.

This XML-based approach gives me the ability to change the content of my site without any recompilation. I also have a fallback solution for non-Silverlight visitors and all that without a redundand information storage. Non-Silverlight visitors could use the POH (Plain Old HTML) version, which is simply the XSL transformation of the underlying XML data. And there's another advantage: If I ever decide to store the data in a database, I just need to change the internal functionality of the data assembly and integrate an ORM framework like NHibernate. The domain models and therefore the DataBinding won't be touched.

My Silverlight website uses the Dependency Injection framework NInject. The whole project, including the physics engine, is developed keeping a contract first design in mind. The usage of the DI pattern and a appropriate framework helps a lot to separate the concerns and produce code with a high maintainability.

Extended Silverlight LinkLabel Control

As I mentioned earlier I was really annoyed by the lack of HTML support in Silverlight and it forced me to extend the Silverlight LinkLabel Control so it supports HTML tags. I was also able to improve the performance of it. TextBlock's performance is still bad, but that's not the fault of the LinkLabel control.

Here is how it works: The original LinkLabel control "parses" the input text and creates a HyperlinkButton for each of the links it founds. The rest of the text is splitted into single words and for each word (!) a TextBlock is created. These many controls are then all added to a WrapPanel, which provides the word wrapping functionality.

I measured the LinkLabel control and encountered that the high number of TextBlocks are killing the performance. Therefore I extended the LinkLabel control by a word grouping functionality. Single words are now grouped to n words and assigned to one TextBlock. This modification reduced the number of TextBlocks and in turn increased the performance significantly. However, a lot of text has still a huge negative impact on the performance. So please Microsoft, implement WPF's Inline Hyperlink into Silverlight. WPF is a desktop technology and has the Inline Hyperlink, but Silverlight as a subset of WPF and RIA platform doesn't come with Inline Hyperlink?! I don't get it.

Soft Body Physics Engine

With my passion for computer graphics and physical simulations it was also a must that my new website simulates real-time physics. That's why I developed a flexible Silverlight soft body physics engine and a simple Renderer. The soft body physic is based on a Newtonian mass-spring system and also implements the Pressure Soft Body Model by Maciej Matyka. The resulting differential equations are solved numerically with a Verlet integration. The engine is flexible designed and could easily be extended. It comes with a constraint system, various types of bodies and force generators, basic collision detection and so on.

The work on the engine is in progress and there's still a lot to be done, but if you want to see it in action, just go to my website click near the balloon and throw it around or grab a corner of the content element. If I find the time, I will continue my work on the engine, hack some demos, release it as open source and blog about it. But I demand high standards of myself and I currently have other projects in the queue, so don't expect a release soon. Especaillay the implemented collision detection is just basic and the development of a proper collision detection system takes some time.

Conclusion

It was a long and tricky task for my first Silverlight and XAML project, but mostly it was a lot of fun and Silverlight 2 is an awesome step in the right direction. I don't regret that decision. Silverlight is definitely now my favorite platform.


Update 08-07-2009

I submitted my website to the Summer Silverlight Coding Competition. Feel free to vote for it here.