Our resident HoloLens expert Pete Daukintis takes a tour around the HoloLens, and how you can start developing for our holographic platform.
Although I work for Microsoft, I am not on the HoloLens engineering team, and thus won’t speculate about engineering topics. I have, however, deep interest in virtual, mixed and augmented reality, and, by extension, Microsoft HoloLens. Any opinions expressed in this article are mine - Pete Daukintis, Tech evangelist Microsoft UK (twitter LinkedIn Blog)
The whole design of Microsoft HoloLens is centred around it being a fully mobile device with no need to be tethered to another computer; it is a full Windows 10 PC which has an extremely deep understanding of its environment. It is worn on the head and the optics allow viewing holograms mixed with the real world as if they were actually there which it achieves very effectively with detailed, crisp and bright holograms. It knows where the surfaces that make up your environment are and can remember that information enabling the interaction of digital holograms with the real world.
If I ‘throw’ a digital ball it can bounce along my real coffee table, fall off and bounce or roll along the floor. In addition, if I leave holograms in a particular location, say I leave a tiger on the worktop in my kitchen, when I return to my house sometime later the tiger will still be there. HoloLens is a truly unique device which solves a host of traditionally ‘hard’ problems, i.e. inside-out tracking which enables the device to know it’s position and rotation in 3D space (a.k.a six degrees of freedom tracking). This enables the device to have a full understanding of where it is in the world – so holograms stay exactly where you left them and you can view them from any angle or position.
The main user experience paradigms used in the HoloLens are Gaze, Gesture and Voice. Gaze is very intuitive in that the ‘cursor’ position is defined by where the user is looking or more accurately, which way their head is facing.
Then to ‘action’ an item being gazed at a gesture or voice command may be used, for example to select the item I might carry out an air tap or simply say ‘select’. Both the gestures and the voice recognition work impressively well.
2D apps + UWP
Since HoloLens runs Windows 10 it follows that apps built with the Universal Windows Platform will also be supported. They are and are experienced as 2D surfaces projected into the 3D environment. They can be resized, pinned to surfaces in your room and they can even follow in your line of sight as you walk around.
Scenarios / Skype
You can call up your Skype contacts from HoloLens and their Skype video feed will appear as a holographic screen in your view. If your contact is using Skype for Windows they will be able to see what you are seeing and draw holograms into your world.
Both your contact and you have the ability to use inking and draw lines/arrows, etc. really taking remote assistance to a new level. Skype for Windows requires an add-in to enable drawing.
See https://www.microsoft.com/microsoft-hololens/en-us/apps/skype for further details on usage and scenarios.
The most important thing to call out here is that you don’t need a device to get started with understanding the development platform as you can use HoloLens emulator to run and debug your app code. This will allow you to get a head-start before your device arrives or to get a deep insight into what might be possible and what coding and design skills might be required.If you are looking to get started on 2D apps or already have an existing UWP app you want to try then you will need Visual Studio 2015 with the latest update and HoloLens emulator from the links below and you will have enough to build, debug and deploy. You can find documentation and guidance for UWP at Windows Dev Center.
To get started with a 3D app the most popular option is using Unity for which the Holographic Academy tutorials are a great place to start or If you are not familiar at all with Unity, the official Unity website has some great resources to get you started.
You will need the latest Visual Studio 2015 update, HoloLens emulator or a HoloLens device and for development of 3D apps the latest Unity technical preview for HoloLens. Install the tools from here.
The HoloLens emulator install also installs Visual Studio templates for C++ and Direct3D or C# and SharpDX if you prefer lower-level coding but if you are unfamiliar this is a steep learning curve. Here are some resources if that is your preferred route:
- Visual Studio 3D Starter Kit
- DirectX Samples
- DirectX Toolkit
- DirectX Graphics and Gaming Documentation
The HoloLens forums are there if you want to ask questions and find other developers, and here are some other fantastic resources which will help with HoloLens development: