Software:Commercial augmented reality

Commercial augmented reality (CAR) describes augmented reality (AR) applications that support various B2B (Business-to-Business) and B2C (Business-to-Consumer) commercial activities, particularly for the retail industry. The use of CAR started in 2010 with virtual dressing rooms for E-commerce. For commercial purposes, AR applications are often used to integrate print and video marketing. With an AR-enabled device, such as a smartphone or smart glasses, aiming a camera at a printed material can trigger an AR video version or animation of the promotional and informational material superimposed on the image.

Apart from the primary use of CAR, technological advancements have yielded more commercial applications for retail, B2C and B2B markets operating with physical stores as well as online virtual stores.

History

The history of commercial augmented reality is brief compared to that of augmented reality.

In 2010, virtual dressing rooms were developed for E-commerce retailers to help customers check the look and fit of products such as clothing, undergarments, apparel, fashion products, and accessories. An AR technology was developed in 2012 to market a commemorative coin in Aruba. In 2013, CrowdOptic technology was used to create AR experiences for an annual festival in Toronto, Canada. An AR app Makeup Genius released to try out beauty makeup and styles with the help of handheld devices was released in 2014.

An AR app was launched for the art market in 2015. In 2016, a Wikitude app included an update to provide AR campaign opportunity to businesses. Users can point phone cameras at certain places and get information from websites such as Yelp, Trip Advisor, Twitter, and Facebook. In 2017, Lenovo developed a Tango-enabled smartphone to assist retailers. The Wayfair app enables customers to try a virtual piece of furniture in their home or office before buying.

Technology

CAR technology dates back to the 1960s but grew considerably during the 2000s. According to CAR, technology involves several contemporary technology components. The three major components are the hardware, software, and algorithms of AR.

Hardware for commercial augmented reality

With advancements in computing and allied hardware technologies, AR hardware such as display devices, sensors, input devices, and computing processors have improved over time.

Display hardware components for CAR

Display hardware can be listed in the following ways: Head Mounted Display (HMD) such as a harness or helmet; eyeglasses, Head-Up Display (HUD); contact lenses; virtual retina display (VRD); and Eye Tap. Spatial AR (SAR) enhances real-world objects in spaces without depending on any display device. Those SAR are Shade Lamps, Mobile Projectors, Virtual Tablets, and Smart Projects.

Sensors for CAR

Tracking and networking hardware must work in a seamless combination to bring about the desired level of mobility in CAR systems. The latest smartphones and tablets like mobile devices consist of cameras to act as an optical sensor, accelerometer, and gyroscopes for position tracking, solid-state compass and Global Positioning System (GPS) circuits, as well as sensors, for location detection, Radio-frequency identification (RFID) for radio signal detection, Wi-Fi for networking, and several mobile-based third-party sensors for a myriad of purposes.

Input CAR devices

To bring complete interactivity in AR systems, different input devices are mandatory such as keyboards for textual inputs, speech recognition systems like Siri, Cortana, Google Voice and so on, gloves stylus, pointers, and other body wears with sensors to provide body gesture inputs, eye movement detection sensors, and hardware.

Software and algorithms for commercial augmented reality

AR software should be capable of carrying an image registration process where software is working independently from the camera and camera images, and it drives real-world coordinates to accomplish the AR process. AR software can achieve augmented reality using two-step methods: It detects interest points, fiduciary marker, and optical flows in camera images or videos. Now, it restores the real-world coordinate system from the data collected in the first step. To restore the real-world coordinates data some methods used include: SLAM (Simultaneous Localization and Mapping), structure from Motion methods including-Bundle Adjustment, and mathematical methods like-Projective or Epipolar Geometry, Geometric Algebra, or Rotation representation (with an exponential map, Kalman & particle filters, non-linear optimization, and robust statistics).

Commercial augmented reality programming technology

The aim of ARML (Augmented Reality Markup Language) is defining and interacting with various Augmented Reality scenes. XML and ECMA scripts are parts of ARML. The role of XML is to describe the location as well as the appearance of the virtual objects in AR visualization. The dynamic access to the properties of virtual objects is possible using ECMA scripts binding.

Object model of augmented reality markup language

The model is built on three main concepts. Features: Represents physical objects in AR scene; Virtual Assets: Represents virtual objects in Augmented Reality scene; and Anchor: Define the spatial relationship between a physical and virtual object in AR scene. The Anchors are four different types—Geometries, Tractable, Relative To, and Screen Anchor.

Commercial augmented reality SDKs

Just like other technologies, AR application development kits are available for a rapid development process in the form of Software development kits (SDKs) including: CloudRidAR, Vuforia, AR ToolKit, Catchoom CraftAR, Mobinett AR, Wikitude, Blippar, Layar, Meta, and ARLab.

Applications

Objective of CAR

According to Hemant, the objectives of CAR is to bring virtual objects, which are generated by computers, into the physical world using simulation techniques. Moreover, it permits real-time interactions to benefit various commercial sectors and industries.

Commercial augmented reality simulation process

The optical device that combines the real-world and virtual world experiences is "Combiner" which acts as a platform for commercial Augmented Reality experiences. Thus, the entire CAR process consists of three main phases. The first is recognizing the object or image. The second is object or image tracking in space. The third is superimposing virtual objects on the physical world.

Traditional combiners for CAR process

The traditional combiners cover two implementations: The polarized beam combiners or flat combiners, or the off-axis combiners or curved combiners.

Non-conventional combiners for CAR process

The non-conventional techniques involve diffractive optics as well as Holography. The hologram or waveguide concepts are involved in these non-conventional techniques and technologies. The real theory behind this is the extraction of a collimated image, which is guided by Total Internal Reflection (TIR) through a waveguide pipe. It behaves like a router where a waveguide is transmitting the image to the user's eyes and provides the most sophisticated optics to see through.

User interactions in AR

Most of the AR devices use a touchpad and voice commands to provide user interaction facilities. Smartphones and tablet devices are the excellent candidates to interact with AR applications. Therefore, most of the AR applications in the market are handheld devices based on whether they use traditional or non-traditional AR techniques and technologies.

Commercial augmented reality applications

Augmented reality is gradually changing the scenario of B2B and B2C businesses by providing AR applications. In due course, Hemant has listed several CAR applications in detail.

The AR dressing room application

Fashion and apparel customers buy products after selecting the best fit by trying them on in a Changing room. This can result in lengthy queues waiting a vacant room. Topshop with Kinect has created CAR dressing rooms to overcome the problem to some extent. This technology has even allowed for size estimation in the dressing room. The Gap has followed the trend. The Augmented Reality dressing rooms are equipped with the AR devices, which are in turn helping focus on the targeted dress/product and capture the virtual 3D image of the product/dress. It helps to visualize the dress on the body of the shopper/user.

Product previews application

The in-store retail customer can view a virtual preview of a product packed in a package, and even without opening it at all. An AR app for Lego is an ideal example of this use. Lego displays an animation of a product in an informative manner to the interested children and their parents. Image recognition technology is behind it rather than sticking a code on the box and scanning it.

To accomplish this, Lego has implemented a second-generation Sandy Bridge Intel Processor that can popup 3D animation over the top of the box. Moreover, the animation can move or rotate as the box moves or rotates. This is possible by the recognition of box movements and postures.

The CAR triggered products application

The AR event was triggered automatically by focusing on an Aruba coin with AR hardware. The AR event revealed additional objects and information, which was invisible without the coin.

Makeup CAR application

Shiseido has developed a makeup mirror called TeleBeauty that helps female shoppers to visualize the product performance on their faces well in advance of applying it. The capability of the AR mirror allows it to portray the shopper's image with lipsticks, eyeliners, and blushes with real-time updates.

Beauty style CAR application

The best example is the Burberry Beauty Box AR application. It provides a nail bar application. Shoppers can choose their skin tone with the app and paint different polishes on the bar to check how the polishes look in real life.

Art market CAR application

In 2015, an AR app was developed by Itondo with the aim of visualizing an art piece on different locations on walls before taking it from a gallery. It displayed live previews of a two-dimensional image of the artwork which is capable of scaling on the walls. Moreover, it enables an art gallery to display background previews using pre-saved photos of the different walls provided by the shopper. The app helps the user to visualize the best location for the artwork before they make a purchase.

The color changing CAR application

American Apparel has products in varying types of colors and color combinations. This can make the color selection process daunting. Therefore, it has invented an AR app to help in the selection process without the customer having to wear the actual product. The AR app simulates the same products in available color choices and makes the selection process easier. The AR app provides real-time ratings and reviews uploaded by customers online and tempts online shoppers to visit the bricks-and-mortar stores.

The fitting CAR Application

De Beers is a known entity in jewelry industry. It has released an AR app useful for online shoppers who wish to see jewelry products as if they are wearing them in the real world. The company provides images of products through Forevermark Fitting site the shopper can download and print on paper. Now, the user can use the mobile AR app by focusing a mobile camera on the image of the item. The app displays a virtual simulation of the jewelry products with real-time updates so products move with the user's movements and displays different facets at different angles. Moreover, customers can judge that how the jewelry looks in certain lighting and on different skin tones.

The catalog CAR application

A product catalog for certain items like furniture cannot test in real life for a real environment. Moreover, small 3D images of products are of little use when the user wants to see the furniture product in real life in their home or office. IKEA has launched their AR catalog IKEA Place that helps to visualize the furniture products in real-world spaces like homes or offices. It also helps customers to judge the appropriate size and shape of the furniture be fitting in the actual environment that meets their needs.

The personal shopper CAR application

IBM has released an AR app that helps shoppers to obtain detailed information on a product without touching it or inviting sales assistants to describe it. The CAR personal shopping application is capable of providing highly personalized experiences as well as marketing offers with a personalized touch. All this is possible in real-time if Beacon technology is applied in the store.

The shoe sampler CAR application

The Converse Sampler is an AR app to assist customers to visualize a shoe with real-time updates. A customer needs to focus the camera of their mobile device on their legs after opening the app. The app provides a catalog for the selection of products. Once a selection is made the app begins superimposing products on the real world legs and gives an idea of the fit as well as its look so the customer can purchase the product online with confidence.

Controversy

A controversy was created by Pokémon Go, a game with two technical problems. The tracking and visualization processes handled in the absence of ergonomic, safe, and secure environment. The immersion in the game by players was too deep and resulted in several deaths, which caused some governments like China to ban the game. This unconventional combination of technology may lead to new inventions, but the cost of the hardware, software, and implementation makes it challenging for common commercial production.