Rocketmakers' XR platform recommendations

The technology used to build XR platforms is still in its infancy.  Consequently it can be confusing to get to grips with, difficult to understand and even harder to know where to start.   Whilst it is easy to get carried away with the potential applications for XR technologies, it’s important to know how to build your platform correctly and to take time to select the best tool for the job, especially if you’re just starting out in this area. 

At Rocketmakers we are lucky enough to have an in-house team of XR experts. To help give you a better understanding of the features and capabilities of XR tools available today, we asked them to compare five leading platforms against a core set of features.  

But what exactly is XR?

Before we dive right into our recommendations on the best tools to use for building your next XR platform, it makes sense to first define what XR is. There are three different sub categories of XR, so it is important to first identify what your focus is when deciding which platform to use in line with what you are specifically trying to create.  XR is a blanket term encompassing Virtual, Augmented and Mixed Realities. The simple way to describe each of these is:

Virtual Reality: A totally new environment, separate from the physical world, for example any game/ experience played on a Playstation VR headset or an Occulus quest.

Augmented Reality: Overlaying digital information on top of the physical world, like Google Glass.

Mixed Reality: Adding digital information which is aware of the physical world, such as a virtual ball that can roll off a physical table when viewed through a screen or headset.

These three technologies have their own distinct applications and their own unique potential to change the way we view ‘reality’ in groundbreaking ways.

At Rocketmakers, we have the benefit of a talented team of engineers, skilled across a variety of technologies and programming languages.  So we put them to work, analysing the various tools and approaches that can be used to build these kinds of experiences. We decided to focus on AR/MR and identified these select platforms to monitor and research:

• WebXR
• Unity & XR Foundation
• Unreal
• Vuforia
• Microsoft Hololens
  

Our approach 

Each engineer was allocated one platform to focus on, which enabled us to quickly develop a deep understanding into all five platforms. We then picked a core set of features that were ‘must haves’ and other ‘nice to haves’ to help us compare the tools more easily:

Must haves

• Image marker recognition
  Can the library recognise an image and track a virtual object against it?
• QR code recognition
  Can the library recognise a QR code and track a virtual object against it?
• Plane detection
  Can the library detect a physical plane on which to place objects?
• Extended tracking
  Does the tracking continue when the marker moves out of the device's camera frame?

Nice to haves

• Object recognition
  Can the library recognise and track against a real world 3D object?
• Anchors
  Can you pin an object to an arbitrary point in physical space which gets remembered?
• Lighting Estimation
  Can the library read and mimic real world lighting conditions?
• Occlusion
  Does the library provide a way of physical objects blocking out virtual ones? For instance, a virtual ball under a physical desk wouldn’t be viewable through it.
• Scene Recognition
  Can the library recognise a previous ‘scene’ (physical environment) and re-attach trackable virtual objects to it, such as re-entering a physical room and it will remember where you placed virtual objects?
  
  As is the case with most tools and platforms, there are advantages and limitations to each of them. We have briefly indicated these in the table shown at the end of this blog, but for the purposes of showcasing what we, as a Software Design and Development Agency, believe are the best XR platforms, we have focused on those which we chose to use in house at Rocketmakers: Microsoft Hololens and Unity & XR Foundation.

Microsoft Hololens

The Microsoft Hololens 2 aims to deliver a hands-free mixed reality experience to users. It comes at a high price point of around $3000 so naturally lends itself more to the B2B market. The Hololens is a headset incorporating a set of lenses and a full Windows computer within the device. Through the use of clever optics and sensors, digital information can be overlaid on the user's vision while simultaneously taking into account the physical space. For example, you could place a virtual ball on a physical desk and see both objects whilst wearing the Hololens.

Rocketmakers’ Adam Walker was an early user who strived to push the Hololens to its limits.  He attended Microsoft conferences plus several workshops with other Hololens developers when the product had just been launched. Adam knows first hand that the development has come a long way in recent years.  It is now possible to write for Hololens through Unity, Unreal engine (both IDE’s or Integrated Development Environments) or DirectX (a series of application programming interfaces, or API, that provide low-level access to hardware components like video cards, the sound card, and memory). Rocketmakers platform of choice is Unity and Microsoft's language of choice is C# which Unity uses, so Unity seemed like the sensible choice for us.

The setup is complicated but well-documented. You need to have the following tools installed: 

• Unity, A games development environment
• Visual Studio, an IDE 
• UWP(Universal Windows Platform) & Windows 10 SDK (software development kit).  A set of tools and SDK’s for building applications for Microsoft Windows
• Mixed Reality Toolkit or MRTK, a collection of code and samples to make working with the Hololens in Unity smoother. This includes things like “holographic remoting” which lets you test your apps without having to the deploy them to the device each time 

Tutorials and guidance are very limited, so you do have to have a good level of basic understanding to start with.

To see some examples of how Rocketmakers have utilised Microsoft Hololens, please click here and here to learn more about our work with Team Bath Racing.

Unity & AR Foundation

AR Foundation is Unity’s home-grown AR framework for simplifying cross-platform AR development.  The major advantages are that it’s free and relatively simple to get started with. Unity also offers a paid service on top of AR Foundation called MARS, which is primarily a visual AR authoring tool with some additional functionality around scanning real world objects and environments. MARS offers a free 45 day trial — so it may be worth investigating to see how much it can speed up your AR development.  

The documentation and workflows are comprehensive, with plenty of suggestions for troubleshooting common problems. Unity also has a fairly extensive demo repository which covers most use cases.

AR Foundation works well across both Windows and Mac and when combined with the AR Foundation Remote 2.0 (a paid asset found here ​​https://assetstore.unity.com/packages/tools/utilities/ar-foundation-remote-2-0-201106) it makes for a relatively smooth development experience. You are able to stream the app in development from your computer to your phone in order to test using your phone’s camera and gyroscope. 

There seem to be some issues with image tracking performance, and inconsistencies in plane tracking performance, but these should be possible to address as you gain more experience and expertise with AR Foundation.

Overall AR Foundation is a good, if at times limited option for AR development. It’s quick to get up and running and has some good 3rd party development support in the asset store, plus MARS may well provide you with an even faster way to get an AR app going.

To see how Rocketmakers have utilised Unity, please click here to learn more about our content management system for the 3D realm, Beam. 

Our Conclusions

When choosing the best XR tools for your project, there are two last things we recommend you to consider:

• How comfortable you are working with the technology
• The hardware your target audience is going to use when they consume your content.  

With new hardware constantly being released, Apple's rumoured headset and more software solutions becoming available, consumer uptake is only going to increase. Now is a key time for businesses to be taking this space seriously if they want to be considered at the forefront of technology.

Get in touch to find out more about how Rocketmakers can help you to build your next XR project.

This article was a collaborative effort drawing on the knowledge and experience of Adam Walker, Clare Henning-Marsh, Dave Kendall, Dan Dando and Robbie Howe.