Tell me and I forget. Teach me and I remember. Involve me and I learn.
The digital age has changed the way we communicate and behave. Digitization touches every area of our culture, from ordering a meal, interfacing with our workforce and customers, and to getting a ride to the airport. Digitization has its place in every interaction of ours.
Augmented Reality (AR) is the talk of the town and is one of the hottest technologies that is ripe to make a difference both in the enterprise and consumer world. AR is catalyzing a new information-delivery paradigm, which will have a profound impact on how data is structured, managed, and delivered through varied mediums over the cloud.
Big tech cos. are taking a big Leap into this domain by making large investments in their AR application development frameworks and hardware rendering devices. HoloLens (Microsoft), AR Kit (Apple), AR Core (Google) and AR Foundation (Unity), are the influencers that are driving the AR domain to scale and setting the stage to make an impact.
Augmented Reality (AR) is defined as a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view. In simple terms, AR overlays digital information on real-world elements.
Augmented Reality got its name from a performance support intervention created by Tom Caudell and David Mizell while working at Boeing in the 1990’s. The concept was to “augment the visual field of the user with information necessary in the performance of the current task over a head-mounted apparatus.”
Augmented Reality is the next big thing. This hyper-connectivity tool is key to creating next-generation extended experiences.
With augmented reality, storytelling gains life. The border between the virtual and real world ceases to exist (for a better world). Augmented Reality enables to rapidly and accurately absorb information, make decisions, and execute required tasks quickly and efficiently. In addition to Gaming (Pokemon Go), AR has a wide range of application – Education, Training, Defense, Space Exploration, Healthcare, Industrial Manufacturing, Retail, Real Estate, Architecture and Services sector…all are destined to gain from adopting AR.
The potential of AR is seamless, and organizations are already utilizing this technology in their business to provide brand-new user experiences. Companies are implementing AR to create product demos, interactive & immersive learning, advertising, and providing real-time information to customers.
At Boeing, AR headsets are used by employees to view each step of the complex assembly process of an aircraft directly in their field of view without having to repeatedly refer to a computer dramatically uplifting productivity and quality.
U.S. military uses AR-enabled head-mounted displays to overlay blueprints from a satellite directly onto the soldiers' field of vision.
NFL broadcast networks apply AR to televised football games, which is one of the earlier mainstream implementations of augmented reality. TV viewers can see a yellow “first down” line that spans width of the field in NFL broadcasts. NHL used to have something similar (FoxTrax), a superimposed blue “puck glow” for enhanced tracking but was discontinued (and is now expected to make a comeback).
ThyssenKrupp leverages the Microsoft HoloLens to train maintenance crews on how to repair elevators while viewing AR digital overlays of manuals and guides.
AccuVein is an AR handheld device that allows nurses and doctors to scan a patient’s body to make a vein appear visible.
In the Netherlands, cell phone owners can download an application called Layar that uses the phone's camera and GPS capabilities to gather information about the surrounding area and then shows information about restaurants or other building sites in the area, overlaying this information on the phone's screen.
Augmented Reality (AR) apps can be primarily classified into two broad categories:
Marker-based apps — Marker-based apps are based on image recognition. These apps use predefined markers to trigger the display of AR overlays on top of the image. To see the augmented component, you have to point the camera on a marker's position in your environment. Once the device recognizes the marker, an app overlays the digital data on this marker, and you can see the augmented object. In marker-based apps, the objects (images) are already hard-coded in your app, so they are easier to detect. Most AR apps currently are marker-based and are especially leveraged in advertising industry.
Location-based apps — Location-based apps detect the user's position with the help of a GPS, an accelerometer, or a digital compass and overlay the augmented reality objects on top of real physical environments. Location-based AR apps work without markers. These apps can send notifications and provide AR content to the user based on their location. For example, an app could give recommendations about the best restaurants nearby and provide directions on how to get there. The most famous location-based app is Pokemon Go.
There are multitude of products out there to choose from for developing AR experiences. Let us keep the evaluation of that for a future post. In this blog post, let’s go over the basics of AR application design approach and key design principles that are fundamental to creating any augmented experience in real time.
Predominantly there are two approaches to Design.
Top-Down — This approach is TECHNOLOGY driven and is primarily leaned upon in companies that are driven by technology as a key driver. In the consumer centric world, the downside of this approach is that it generally leads to disappointment.
Bottoms-Up — This is a USER driven approach and is primarily centered around the consumer. The key driver is the unmet and articulated need of the consumer that focuses on customer delight & experience.
Augmented Reality puts human experience over technology. Technology enables to enhance this human experience. At its core, this human-centered approach to innovation looks to align the needs of the people (desirability), the possibilities and potential of technology (feasibility), and finally, the requirements for business success (viability). Augmented Reality is less about technology doing things to people and more about people engaging with the world around them, and having that world enhanced by technology where and when appropriate.
Designing and Developing AR applications demands a collaborative co-creation methodology and is aligned with a Bottoms-Up approach with ‘Users’ as its central and core constituent.
When creating Augmented Reality applications, these are a set of key DESIGN PRINCIPLES to apply when designing layers onto the real world:
Context is Critical — What ...When ...How ...Context is important for designing any technology applications, but it is everything when designing for sixth sense, The Matrix → Augmented Reality. It is not only important to design to engage intended users in their environment, but the application should also be able to observe the unintended users who may be affected by an augmented experience. Location awareness is key to realize AR’s full potential.
Think People — The first step in user-centered design is to define your users – Intended & Unintended Users; Active & Passive Users; Direct & Indirect Users. See and feel their world, identify their needs, communicate your understanding, and prioritize insights. Understanding of people and culture knowledge informed by an empathetic mindset is critical for creating valuable AR experiences.
Focus on Customers — What user problem are we solving? Ask the right questions to help frame the problem correctly...What? How? Why? [Customer] needs to [Customer’s Need] because [Insight]. Generate and evaluate ideas beyond obvious solutions to increase the innovation potential of your solution. But stay focused on meeting the customer need(s) to help them perform their tasks more effectively and efficiently.
Strike a Balance — AR is about mediating reality with technology to build a stage magic. It is important to ensure that the experience delivery uses the right degree of ergonomics & mediation, such as keeping augmentation simple, designing for a narrow depth of field, and prioritizing line of sight. The key objective is to achieve its communication goal without distracting the users.
See the Big Picture — AR is not only restricted to the sense of sight, but it can be applied to other senses such as smell, hearing or touch. During design initialization, chart distinct scenarios of presentation, surrounding environment, expression, and engagement to augment multiple mediums. Quest should be to design, innovate and deliver compelling experiences that captures customers emotions, experiences, and bring them the ideal experience to life.
Build to Think (Prototyping) — Build a rough and rapid prototype(s) to get the ideas and explorations into the physical world. Prototyping is the best way to test the concept, assess feasibility, gather valuable information, and learn quickly. Evaluate and select the right technologies, frameworks & SDKs, to enhance the right senses. Leverage tools such as customer personas and journey maps to validate ideas. Visualize and create 3D models of objects that will be augmented to address user interface issues.
Keep it Simple — Ensure that navigation of interaction opportunities is simple and straightforward (minimalistic design). Define an experience’s inputs and outputs, visual and audio cues, and its fidelity (low or high) at the onset. Be extra cognizant of interaction costs when designing in augmented reality environments, often because those interactions can be literally exhausting. Focus to keep the movement and visual input less cumbersome to avoid multi-tasking.
Safety First — Prioritize line of sight and design for narrow depth of field balancing constraints with safety of the users. Whether the user is navigating by foot or vehicle, only displaying content within a specific range helps the user to effectively parse AR content, prioritize reality, and ensure safety. Remember users’ real-world context & focus on delivering AR experiences that transmit only essential information to the user (without distracting and misleading).
Design for Security — AR data is rich. AR applications and devices can be corrupted by malicious applications to deceive users and create conflicting views of real world by overlaying incorrect data (sensors) or fake interactions. Such ransom attack can trick users into falsely believing that certain objects are or are not present in the real world. Its important to secure the boundaries of the experience enabling multi-factor authentication, tiered data security, data privacy and data encryption practices around input validation, output rendering and sharing of content.
Less is More — Capturing visual attention is key to AR applications. Limit the design to essential elements and omit needless details taking into account the tracking requirements for the experience to be delivered. For example, for certain augmented experiences 3DoF (3 Degrees of Freedom to track rotation) would suffice as opposed to 6DoF (6 Degrees of Freedom to track rotation and translation). Design is not about how it looks, but it’s about how it works.
It’s a Team Sport (Minimum VALUABLE Product) — Putting the designers, business users, solution architects, developers, customers, and testers at the heart of the experience design and creation process is a liberating act. Co-create wonder to draw people in. Focus on the needs vs. wants and create a marketable augmented reality product with the right user experience and feature set. Limit to a specific use case or a subset of a larger user scenario for maximum impact.
Test, Test, Test — Gather and Plan as much information as possible for test scenarios in terms of functional alignment, user interactions and engagement. Real scenario user testing and feedback reveal insights that in-turn helps to redefine the problem.
Stage the Experience — Deliver a WOW experience for your audience, an immersive experience that is worth engaging and sharing. Augmented Reality is a powerful storytelling medium with endless possibilities to change lives.
The Way Forward (Continuous Learning) — Validate product market fit, seek feedback, pivot & iterate (agile execution) leveraging a continuous learning process to collaborate from inception to completion.
We are on the cusp of AR becoming mainstream in the digital world. And it is on us to make sure we do it the right way — augmenting reality to truly enhance human-to-human experiences. As Steve Jobs said:
“You’ve got to start with the customer experience and work backwards to the technology.”
Augmented Reality is here to stay and change the way we interact with the surroundings we live in. Adding digital content to the real-life environment via smart devices can potentially transform every industry, from education and healthcare to manufacturing and gaming.