How to design for mobile Augmented Reality (AR)
Now is a great time to jump in — join the early majority, learn from and build on these successes. Here’s a summary of best practices for anyone considering creating a mobile AR app for their brand or business.
Whenever a new technology is released innovators feverishly work to discover what it’s good at — how will it enable a new experience for my customers? How can it improve my business? What’s exciting about it? Should we be using it?
This is the most expensive and risky time to build an application — the development environments can be buggy and difficult to use, the developers and designers who understand them are relatively scarce, expensive, and are learning on the job. Most consumers are wary and stay away except for the few that love the new.
How mature is mobile AR?
Augmented Reality is currently somewhere in the middle of its evolution — the development kits and OS-level support from Apple and Google are powerful, but still in their infancy. Many of the new features are only available on top-of-the-line phones and the technology needs ideal conditions to work as promised. AR experiences are still quite fragile — the technology can fail when faced with challenging conditions in the user’s environment.
Despite AR’s limitations we’ve seen early successes with Pokemon Go and Snapchat’s lenses as well as many branded experiments including breakouts here and there. What makes AR so powerful is that we can enhance and alter a user’s perception of their environment by rendering digital content on top of it. This makes it ideal for visualizing products in their natural environment.
For example, Memomi’s augmented reality mirror lets shoppers try on and compare clothing virtually.
Warby Parker’s app lets you try on a pair of glasses at home or wherever you are through the tap of a button.
And of course, AR face filters on Snapchat and Instagram allow us to transform how we look and communicate.
We can also expect to see a revolution in how we navigate through the world. Dent Reality is one of numerous companies working on AR navigation apps.
Let’s dig in
In the near future we will see many more exciting uses as the technology matures. This article will look at how to design for AR with its current limitations. I will primarily speak to working with Apple’s ARKit since it’s the toolkit that my team and I have worked with extensively since its release. At TANK & BEAR we are building an augmented reality game where players go on adventures with their own personal spirit animal and build a magical layer on top of our world. You can sign up to be notified when our beta launches.
Make a good first impression
You should assume that the majority of your customers are still new to AR and you will need to guide them through the experience step by step. The good news is that many experiences follow a standard interaction pattern:
• Turn on the camera.
• Place the content in the environment or on the user.
•Begin interacting with the content.
AR in the user’s environment
The current technology relies on mapping horizontal and vertical planes with the camera and then anchoring content to those surfaces. This process isn’t instantaneous — it takes a couple of seconds for the application to map the environment. While the application is finding planes, provide a graphic or short animation that gives an indication of what’s happening.
If the user is placing the content, provide them with an indication that serves as a target for where the object should appear. This could be a rectangle, circle, dots–something simple often works best.
If your content is more fantastical, feel free to use a design metaphor that fits with your content. Pokemon go uses ferns to show where they want the player to tap and place a Pokemon.
Face AR
In a magic mirror type application like Snapchat filters, you can immediately begin to place content on the user’s body as your application recognizes the bodily features it needs to place the content. The mapping takes less than a second.
The good news is that many of these cumbersome steps will go away as the technology improves and users become more familiar with AR applications.
Keep your experiences short
When you are designing an AR application imagine that the user will be holding up their smartphone or tablet at arm’s length to view the scene that you’ve created for them. This creates a number of challenges. It’s tiring to hold your phone or tablet up for extended periods to interact with content in your environment. Experiences that rely on lengthy session times aren’t very suitable for AR — yet. The session times should be a number of seconds to a maximum of a minute or two. Imagine that users are only likely to use your application for as long as it would take them to snap a great Instagram pic.
The best UI fades away
The primary UI for an augmented reality experience is your camera view. You want to make sure that you don’t clutter the viewing area with unnecessary UI elements. Place most UI elements near the bottom edge of the phone so that the user’s fingers don’t accidentally obscure what they are trying to see while interacting with your UI.
Place text labels layered on a 2D plane where the depth never changes with viewing angle. You can experience this effect on Apple’s Measurement App.
Resist the urge to place text elements and labels in 3D space though it may look cool. Text that is rendered in 3D space can be hard to read since the user’s distance and position from the text changes with respect to their position in space.
That said, there are times when text in 3D space is appropriate. For example, if you’re using text to guide the user along a path or want to simulate signage for a virtual storefront. Just make sure you understand the UX constraints around your use case.
Design for your app breaking
Since AR relies upon the device’s camera to map your environment, many of the same factors that make taking a good photo challenging can also pose difficulty for your application.
ARKit helpfully provides the developer with diagnostic messages about environment mapping failures; I’ve listed some of the most common below:
Not enough texture
Many modern buildings have smooth, reflective floors, and walls that do not provide enough information for your application to successfully place the content.
Solution – guide the user towards using the app on a textured background.
Low light
Low light conditions such as using the application at night or in a poorly lit apartment creates similar challenges.
Solution–guide the user towards using the application in a well lit environment, and provide a fallback solution that can be accomplished on the phone if this such an environment is not available.
Excessive motion
If you’ve tried using an AR app on the subway train or in car, you’ll notice that the content will begin to drift. This is due to the countless vibrations and changing light conditions resulting from the vehicle’s motion.
Solution–let the user know that they need to move slower to use the app or try again when in a more stable environment.
These types of errors will go away in the next couple of years as tracking technology improves, but for now you’ll have deal with them and design accordingly.
Design for errors
The user’s environment
If your application is intended for the user’s environment you have very little control over where and when they will use your app. You can mitigate this somewhat by designing for very specific use cases and by using promotional materials that show your app being used in ideal conditions.
In the end, the only way to reduce errors is to test, test, test. Your team will discover problems early on and can develop strategies to deal with them. Make sure that application is being tested in its intended environment throughout its development and then try contexts that you may not have intended, like a crowded train or in the back of a car.
Make use of the toolkit’s error messages. It’s up to your application to process diagnostic messages and to your experience designers to write clear and concise error messages that will guide your user towards a solution in real time.
When all else fails — provide the user with a fallback experience that allows them to do another thing in your application. For example, in Pokemon Go you can also collect Pokemon without using AR. With a retail app you should always allow users to browse your products the old fashioned way. With new technology you need to have a fallback strategy for when it fails, because it will.
Retail spaces
If your application is intended for a physical retail environment, and you (or someone you work with) are the retail designer, the good news is that you have complete control of the user’s environment. Make sure to provide the environment with ample light and avoid using white walls and reflective surfaces.
Keep your users safe
When you view the world through your smartphone camera, it’s easy to neglect to use your peripheral vision. This can make you oblivious to the world around you, often resulting in collision with other objects, people, or even speeding traffic.
Design your app with the user’s safety in mind:
- Consider how you’re making the user move and whether it could place them in danger.
- Add safety messages to alert them to danger. You’ll notice that Pokemon Go presents the user with a safety message warning each time they open the app.
- Pause the application if you can tell that the user is engaging in reckless behaviors. Since your app has access to device sensors you can figure out how fast the user is moving by tracking their device’s motion.
- If you’re using AR in your retail environment consider how it could impact your customer’s movement patterns in the store.
Conclusion
In this article we have have looked at some of the challenges of designing a mobile AR application and how to address them successfully through careful design. In future articles we’ll look at the new computing platform that’s emerging and scenarios for new products that are still waiting to be built.
Despite the risk and cost of being early, we strongly recommend creating applications right now to learn how AR will impact your business and discover new ways to interact with your customers.
Need help with your product?
Have you been thinking about what AR can do for your company? Are you doing interesting things with machine learning but having trouble articulating your product? Get in touch! I love helping passionate teams build new products.
Swing by and checkout what I’ve helped build in the past — an AI powered primary care app that’s been used by a million patients, an app that donates your money to improve the world and a tamagotchi that feeds off your step count to encourage you to stay fit.
Connect with me on LinkedIn.
Feeling entrepreneurial?
Want to build a new game company that is AR native here in New York? Drop me a line. We’re fundraising for UMI.
