Mobile devices like iPads or iPhones, which are already built with technologies such as GPS, accelerometers and sensors, are particularly compatible with augmented reality applications and can make the technology more accessible to the average consumer. In recent years, several technology companies have released APIs such as Apple’s ARKit and Google’s Arcore, which facilitate the development of mobile AR applications for Android and iOS.

Though the types of data and sensors a particular AR software might draw on vary, the basic steps of the AR process are:

An AR device receives a stream of video content from a user’s field of vision, sensing the environment and tracking physical objects in view. This might include the collection of data from accelerometers, gyroscopes, GPS or lasers alongside a video stream to track the user’s position and orientation.

AR software scans and processes this environment—this might mean connecting to an object’s digital twin, a 3-D copy of the object stored in the cloud. It might also mean using artificial intelligence (AI) to recognize the physical object. During this process, AR software processes the information it has received, identifying objects and environmental features that can be augmented. This might involve sensors on the physical object sending data to a digital twin, or combining tracking data with other information such as a product’s price or equipment lifecycle data.

Information streamed from the AR software is displayed on the AR device, superimposing computer-generated content onto the user’s field of vision. Digital information is rendered in the correct perspective and orientation, appearing to the user as if the object is physically present. The user follows interaction instructions, sending commands through a touchscreen, with physical gestures, or by voice. These commands are received by the software and sent to the superimposed digital object so it can be manipulated by the user.

Marker-based AR applications overlay digital content onto a physical trigger in a real-world environment. This trigger might be a QR code, an image or another target marker. When a device’s camera detects this marker, it triggers the display of the associated AR experience. As this kind of AR can be accessed at any time from a range of devices, it is the most flexible AR model.

Marker-less AR, by contrast, doesn’t require a specific trigger. This kind of AR relies on device sensors, such as GPS, accelerometers and cameras, to understand and map a user’s environment in real time. By analyzing the user’s physical environment, often by using algorithms and computer vision, these AR systems determine where to place digital content, allowing for a more spontaneous and dynamic experience.

Through AR-enabled apps for mobile devices or in-store displays, shoppers can virtually try on clothing, accessories or cosmetics. They can also visualize home decor in their living spaces—the furniture retailer Ikea has had particular success with its AR application. These immersive experiences blend online and offline shopping, allowing consumers to make more confident purchasing decisions.

AR seamlessly integrates with various shopping channels, including brick-and-mortar stores, ecommerce platforms and mobile apps to deliver consistent and unified shopping experiences across multiple touchpoints. For instance, a customer might use an AR-enabled mobile app to try on makeup before visiting a store to make a final purchase.

AR can enhance the user experience by providing interactive virtual environments such as maps, product locaters and contextual information overlaid onto the customer’s real-world view. Retailers might use AI to guide customers to specific products or promotions within the store or display offers based on a shopper’s location, encouraging exploration within the store.