In the last post I talked about how conversations are a powerful way of interacting with a computer. A conversation must have context and memory in order to enable this power. Simple commands and individual queries are good, but not sufficient to enable the UX of the future.
Now let’s look at augmented reality (AR). Since 2014, I’ve had a device that allows for a graphical overlay in the real world, but it was not AR. It many ways it was no different than current audio assistants, it had a set of limited use cases that were ok, but it had no real context. Google Glass is an experiment that was put out in the public long before it had the capabilities to be worth while. Over the years, Google has taken it out of the consumer space, and released an updated pair for the enterprise.
While the ability to overlay information on the real world is helpful, what Glass missed was the context of that world. Both Google and Apple now have improved this functionally in their various AR developer APIs. Anchoring a 3D model to the real world allows your to address use cases like visualization of new furniture in your house. This is a start, but you also need to understand lighting (which has been introduced in ARKit 2.0) and Physics (available in most gaming platforms, e.g. Unity).
Understanding the context of gravity and lighting, allows for interactions in the world that are natural. Natural interaction requires simulations that understand all of this, and more. One thing I’ve not seen to date is the ability to move a virtual 3D object around a room, and see it pass behind other objects in the room. All examples I’ve seen to date have the object bounce forward and backwards as you drag it around.
The other item that is needed is devices that not only show you the world, but allow you to interact. Many device makers require that you have some kind of device you attach to your hand (think of the Oculus). Microsoft’s Hololens 2 seems to have solved this one, but I’ve not had a chance to play with it in person yet.
We are getting there, but putting the conversation in the AR world is the magic that will bring this all together.