While Augmented Reality has been on the scene for quite some time, it has not been since two of the world’s largest companies have announced their plans in this technical space, that AR has recognized such a substantial ramp up of public interest and genuine excitement. On the heels of the Microsoft HoloLens announcement and Google/Qualcomm’s $750 million investment in Magic Leap, followed by an additional $800 million investment in Magic Leap lead by Alibaba, Digi-Capital predicted that Augmented Reality and Virtual Reality will hit $150 Billion by 2020 with $120 Billion of that going to AR and $30 Billion going to VR.
In reviewing the marketplace, it is helpful to start with an understanding of the difference between Augmented Reality, Virtual Reality, Mixed Reality, and Holograms.
Augmented Reality occurs when anything artificial augments (adds to) a real experience. Music, Voice, Graphics, Text, Animation, Video, Pictures and other elements may be used via an electronic device to create an Augmented Reality experience.
Virtual Reality differs from Augmented Reality in that the user’s entire access to their current reality is obscured by the Virtual Reality visuals and audio. Virtual Reality delivers the user to a completely new environment, and blocks their access to any of the elements in the real world so that the illusion is complete.
Mixed Reality occurs in Augmented Reality when the high end systems’ holograms appear in the real world or when the user is “transported” to a Virtual world. Mixed reality also occurs when cameras on the outside of a VR system like the HTC Vive allow the external reality and virtual reality to engage either by confining the user into a space where they will not be accidentally injured, or by mapping elements of the virtual reality on top of the real world experience to make the virtual world experience deepen that much further.
This is happening in virtual reality theme parks where virtual terrain map onto physical terrain and give the user sense of touch and object interaction with items in the real world. It also happens in places like Six Flags Theme Parks where a Gear Virtual Reality headset is worn while riding an actual roller coaster. The virtual reality ride is choreographed to the moves of the roller coaster adding a massively substantial difference to the Virtual Reality experience.
Holograms look just like objects in the real world, except they have no mass and are made entirely of light. Some holograms are 3 dimensional and may be viewed from all angles and distances, others are two dimensional and are intended to be viewed from a fixed perspective.
Per their beautifully executed website for the HoloLens, Microsoft summarizes their product, “Microsoft HoloLens is the first fully untethered holographic computer running Windows 10. It is completely untethered–no wires, phones, or connection to a PC needed. Microsoft HoloLens allows you to place holograms in your physical environment and provides a new way to see your world. The holograms you’ll see with Microsoft HoloLens can appear life-like, and can move, be shaped, and change according to interaction with users or the physical environment in which they are visible.”
The main advantage to the HoloLens is its ability to use holograms to place digital content, such as apps, information, and multi-dimensional videos, in the physical space around you, so you can interact with it in the same ways that you interact with physical objects. Holographic objects seen with Microsoft HoloLens can be placed in physical locations you choose, move according to their own rules, or remain in a specific location regardless of where you are or in which direction you are looking.
The holograms you’ll see with Microsoft HoloLens can appear life-like, and can move, be shaped, and change according to interaction with you or the physical environment in which they are visible. Use gestures to create, shape, and size holograms. Use your gaze to navigate and explore. Use your voice to communicate with your apps. Microsoft HoloLens understands your movements, gaze, and voice, enabling you to interact with content and information naturally. Using holograms, you can place your digital content, such as apps, information, and even multi-dimensional videos, in the physical space around you, so you can interact with it.
Magic Leap remains tight-lipped to date about their platform, but the projections of their product to date appear the most exciting of all of the Augmented Reality products on the immediate horizon. Magic Leap uses a Dynamic Digitized Lightfield Signal™, to generate images indistinguishable from real objects and then places those images seamlessly into the real world.
Magic Leap asks its audience to imagine what experiences you could create if you had this ability. Imagine how this would completely transform how people interact with both the digital and real-worlds. Imagine you being one of the first to help transform the world forever.
Lead by significant investments by Google, Qualcomm and Alibaba, amongst others, Magic Leap is a very significant force to be anticipated in the Augmented Reality and Mixed Reality space. Very little is currently available about Magic Leap other than some videos on their website.
MindMaze has designed a medical virtual reality, augmented reality, and motion capture video game system that immerses the amputee in a virtual environment, where moving the existing arm will move the non-existing arm of the avatar. Neuroscientist and MindMaze founder and CEO Tej Tadi says this “mirroring” tricks the brain into believing the severed limb is actually there, and has proven benefits in phantom pain management.
After testing this technology in Switzerland with amputees at Lausanne University Hospital, MindMaze is going to start working with patients from the U.S. Department of Veterans Affairs through a study at the University of California, San Francisco with Dr. Gary Abram, a neurologist and the director of the Neurorehabilitation Clinic at UCSF Medical Center. In about six months, researchers will have data from this second-quarter VA-associated study, which could open the door for this treatment to be used by additional soldiers. To date, Mindmaze has raised $100 million making it officially the fourth unicorn in the AR/VR arena.
Head Tracking: Six Degrees of Freedom
Integrated Motion Capture: Interact Completely With Augmented and Virtual Realities
Hand and Finger Tracking: High Speed Tracking up to 120 frames per second. High resolution gesture recognition.
Augmented and Virtual Reality: Switch between Augmented and Virtual Reality modes fluously
Revolutionary 3D Camera: Real time operation. Embedded architecture. Stereo and depth sensors
Onboard Embedded System: 3D image processing, Synchronization to external sensors.
Wireless and Multimodal Program: Multipurpose compatibility
Full Body Motion Capture: Multi-user tracking. Finger tracking.
An all-in-one AR system for producing custom apps, using their acquired Layar platform for creating an AR experience which operates off of the massively downloaded Blippar app. Blippar raised $54 million this year and was already valued at $1.5 billion from its previous round in 2015, according to Merel.
Daqri has created The First Wearable Human Machine Interface, which they say has bridged the gap between potential and experience by seamlessly connecting the worker to their environment and providing contextually relevant information that connects the individual to their industry, thereby realizing the true potential of Augmented Reality and 4D technology in the enterprise space.
For the first time, a world-class sensor package has been fused with an intuitive user experience. Driven by DAQRI’s Intellitrack™ and native augmented reality software, users are provided with unprecedented levels of information about the world around them for the most precise display and tracking possible. Daqri went one step further by allowing custom work packages produced in Daqri 4D Studio™ to be published directly into the Daqri Smart Helmet™ for direct access.
Per their site: “While it’s true that your best employee can’t be at every site, every minute of every day, with work packages designed in DAQRI 4D Studio™, their knowledge can. You can even take data from systems, such as MRP or ERP, and ingest that into DAQRI 4D Studio™ for visualization. In fact, all of the acquired information remains available for other sites or even future generations. Built with this in mind, these work packages seamlessly publish to DAQRI SMART HELMET™.”
THE VOID is limitless magic in a cinematic VR experience. At THE VOID you take a literal step into new dimensions and experience worlds without limits. From fighting intergalactic wars on alien planets, to casting spells in the darkest of dungeons, THE VOID presents the fully-realized future of entertainment and more. Their integrated Virtual-Reality technologies and stagecraft allow you to see, move and feel our dimensions of imagination in a completely immersive and realistic way. Their Virtual Entertainment Centers (VECs) are coming to major cities throughout North and South America, Asia, Europe and Australia. More than watching a movie or playing a game … in THE VOID you will live an adventure.
While in THE VOID guests will interact with a variety of key elements in their designed physical stage (VOID Game Pods). Physical sensations will include elevation changes, touching structures & objects, feeling vibrations, air pressure, cold & heat, moisture, simulated liquids and smell. In addition their technology allows them to create the illusion the player is exploring miles of terrain or incredibly tall structures without ever leaving their game pod. The end result is a physical connection to the virtual world and sense of exploration never before possible.
Augmented Pixels produces an Augmented Reality construction framework which leverages a state of the art system for mapping the environment during an AR experience. This allows developers to fuse data from multiple sensor inputs to precisely calculate position.
Simultaneous Localization and Mapping (SLAM) is a computer vision technology
SLAM allows an entity to track its location while simultaneously mapping its surroundings
Cross platform development: iOS, Android, PC, Linux
Dynamic Sensor Input allows: cameras, lasers, sonar, IMU, GPS and more
Aurasma has been an a forerunner in AR for many years, as their augmented reality platform allows customers to integrate their system into the customer’s own app for the creation of outstanding augmented reality experiences that directs customers to points of sale, landing pages, social engagement channels, and more.
Aurasma’s best-in-class web studio for creating, managing, and tracking augmented reality campaigns includes a comprehensive analytics dashboard for views, interactions, clicks, and more.
Solutions include GPS or geolocation to blending contextual information in any situation by way of the applications. The combination of these two separate technologies, enable Augmented Reality applications to go mainstream and bring users a variety of truly immersive experiences.
Their E-Commerce / M-Commerce creates a virtual fitting of eyewear and other garments/accessories to make a try-on and product visualization a reality. They also offer Augmented Reality gaming via Augmented packaging, on street marketing, geolocalized apps, advergaming and interacative consumer products.
Geolocation and its real time display of enhanced maps. Acting as a GPS, Augmented Reality provides classified and suggestive information to the user (restaurants, bars, grocery stores, fashion outlets, etc.)
Holobuilder is an augmented reality program which allows people who need to make virtual floor plans and real estate locations come to virtual access. The process is produced by using 360º cameras and stitching their collected images into a virtual floor plan.
Vuforia is one of the leading Augmented Reality, Virtual Reality and Mixed Reality frameworks. It will be available for use with Microsoft HoloLens this Spring. As with any framework for a technology, the release of features is always in progress. In the latest version of Vuforia 5.5, the new device tracker makes it simple to add a full screen VR experience to your existing AR app. Whether you are using a viewer or doing handheld VR, use it to create apps that track your rotation without the need for a target.
In Unity, you can now use the Mixed Reality Controller to easily move between AR and VR experiences on devices and viewers. Get better calibration results with the improved Vuforia Calibration Assistant on see-through digital eyewear devices such as the ODG R7 and Epson BT-200.
By using your mobile device, AR-Code lets you add a digital dimension made by interactive contents to the real world around you. Download the application, look for the AR-Code Stamp or discover which item is made alive by AR-Code( magazine page, packaging, cinema billboard, retail sign, sticker or even a building), point at the item, download the contents and, in just a few seconds, interact with the range of videos, sounds, games and information that appear in Augmented Reality on your device’s screen.
Maxst is a powerful AR SDK and development company. To augment contents from 2D images in a real time is called NFT (Natural Feature Tracking) and it has 3 important processes. The first process is called ‘Learning’, which enables the ability to change 2D images to trackable images. The second step is called ‘Recognition’ which makes cameras recognize those trackable images in real time. The last process is ‘Tracking’, which enables virtual contents augmented based on 2D images so that the contents remain tracked.
The most important element during the whole process is changing 2D images into trackable images. In general, this process should be done through extracting key points by analyzing pixel information and coding intrinsic descriptor information on the key points.
What differentiates Maxst from others in this process is that they have developed faster and stronger extracting descriptor information.
The all-in-one SDK from Wikitude, an AR Framework system includes vision-based as well as location-based augmented reality for Android, iOS, Smartphone, Tablet, Smart Glasses, Cordova, Titanium and Xamarin platforms. Cloud recognition provides fast and scalable online recognition. The studio is easy to access, no programming skills are required to create a simple AR project on this platform.
Xloudia is a set of Image Recognition + Cloud Technologies for the Mobile industry. Xloudia offers high performance, high scalability and highly reliable plugins for Android, iOS, Windows Mobile and Blackberry smartphones and tablets. Xloudia solves a blocking issue: computer vision is too resource hungry to compare more than a few images on a smartphone. By sending the big job to the cloud’s unlimited servers, Xloudia opens limitless capabilities to image recognition from a smartphone.
Image’s feature points and colors’ RGB are extracted from the smartphone video flow and are sent to Xloudia’s servers for comparison with known patterns in registered images.
The requests are distributed to a neural network of servers. Xloudia reverts with the image ID as registered in Xloudia. When the tracking is activated, the smartphone also receives back the tracking matrix which is used by the 3D visualisation.
The smartphone sends the image ID to an external CMS for getting web contents (prices, images, 3D model,…).
This sophisticated process is typically taking a fraction of a second.
Matterport is a mixed reality product which maps a real environment and makes a virtual version of that environment for view on the web. It provides automated, high-quality reality capture and distribution at the push of a button. Utilizing their custom, proprietary camera, Matterport makes the mapping of any environment as automated as it can possibly be.
Scan any place with the push- button simple Matterport Pro Camera. Scanning is lightning- fast, incredibly easy, and fully automated for consistent results, every time. Upload your scans to the Cloud, where powerful algorithms go to work processing your real-world data into an immersive Matterport Space. In a matter of hours, it’s ready to access from anywhere. Your beautiful, fully immersive Space is easily distributed using our in-browser tools. Embed it in any website just like a video so users can experience on web and mobile. No plug-ins or downloads.
ARTV™ (Augmented Reality Television) is one of the most compelling new technologies that was invented at IllusionQuest Studios. It was initially released as the first Augmented Reality music video featuring global superstar Mel B. Then it was configured for use in augmenting posts to Instagram, Tumblr, Facebook and an IQS proprietary social network. The current uses include apps for augmenting museum and aquarium interactivity, business cards, display ads, broadcast television, internet television, billboards, direct response advertising, pharmaceutical explanations, and much more.
ARCardz™ makes a holographic looking presentation come to life atop of anything. The app sees marker(s) of your choice which trigger video(s) of your choice which look as if they are rising out of your marker.
ARTV Expression Engine™
IllusionQuest Studios’ patent-pending technology simultaneously records a live video while merging that with a prerecorded video to produce a host of exciting visual expressions of the full range of emotions, interests and hobbies.
VRTV – Virtual Reality Television™
IllusionQuest Studios’ patent-pending technology virtual reality television allows the insertion of live or prerecorded people to be inserted seamlessly into a virtual reality environment. In this example, a streaming live dealer deals baccarat to a customer in a virtual reality casino.
Transport, which is in private beta, is an online virtual reality content network where creators will be able to publish their work and users will be able to experience them. An assortment of free and paid content will be available through the Transport app, which will be accessible to owners of any headset, including Google Cardboard, Samsung Gear VR, HTC Vive, Facebook’s Oculus Rift, and Sony’s PlayStation VR.
LIST OF AR SOFTWARE DEVELOPMENT KITS
We recognize that there are many companies whose work did not make this first overview. Please drop us a line, and we will work to try to include it in further blogs and white papers on this topic.