Augmented Reality INTRODUCTION

Augmented Reality Page 1 of 11 by student Odia Edeawe

INTRODUCTION

In course MGD415 in January 2015 we discussed Augmented Reality. AR is discussed in several courses at UTM and it is not our intention to replicate that info here..but, for the students in MGD415 who have not been introduced to the great diversity of how this technology is being applied, it was considered useful to have an online unit that brings together most of the issues into one place. Student Odia Edeawe volunteered to research and write this unit and his helpful contribution is on the following pages – it is naturally expected that this technology will develop very quickly as the number of applications increase so we’ll re-visit this next year and maybe a student in 2016 can do an update. WTG Richardson

Augmented Reality

Augmented Reality or AR, is the blending of computer generated content and real life or a rather a real life view of the world. What does means it super imposes computer generated material like pictures over the user’s POV. Therefore, it enhances our view of the world. With augmented reality, users can interact with both virtual content in real life. The interaction with the virtual content does not interrupt their experience in actual reality as virtual reality does and the users can distinguish between the virtual content and their reality. An example would be in television during sports commentary. The reality is the footage of the game of football, rugby, cricket or what have you and the enhancements are the arrows of the players' movement and the zonal areas marked out that don’t actually appear on the physical field.

The blue line above is an example of augmented reality. It’s a computer generated image that is super imposed with reality which is the field and we can distinguish between both; i.e. we know the blue line isn’t part of the field. History of Augmented Reality: We can trace the history of AR to 1990 with the work and research of Professor Caudell. He was a part of the neural systems project at Boeing, the airplane manufacturing company. The project was focused on inventing new ways for the company to use virtual reality in its engineering processes. Professor Caudell then developed software that was able to display the position if cabling during construction at Boeing plants and led to the removal of the need for complex manuals for the workers. The work led to L.B Rosenberg and his team working on a user directed system for the US air force. At the end of the 1990s, Hirokazu Kato, a professor at

Augmented Reality Page 2 of 11 by student Odia Edeawe

Nara Institute of Science and Technology released an AR toolkit that is a library for building Augmented Reality applications. The library contains information on how to introduce a virtual overlay into the real world. It enabled 3D images to be overlayed in the real world. It allowed for video capture tracking and the combination of virtual objects and 3D graphics in the real world through Windows, Mac and Linux. The ARToolkit allowed for a handheld device with a camera and internet connection to bring Augmented Reality to our world. The smartphone had not been invented yet. Applications of Augmented Reality: Navigation There are different enhanced GPS systems that are using augmented reality to make it easier to move between places. Wikitude Drive which is available in Canada, the United States and Mexico brings the GPS into the 21st century. Using the phone's camera in combination with the GPS, the users can follow the green lines to the selected destination.

Augmented Reality Page 3 of 11 by student Odia Edeawe

Medical Augmented reality is being increasingly used to advance medicine and its applications. Medical students now have the opportunity to use augmented reality to practice surgery. The visualizations on the Augmented Reality devices help the doctors to explain complex medical conditions to their patients. AR can also reduce the risk of surgery because it gives the surgeon an improved sensory perception as the technology can be combined with an MRI or X-ray systems and bring those information into a single screen to the surgeon. There are also advances with Augmented Reality in Neurosurgery too. There are AR applications that have the ability to show the patient’s brain in 3D while in surgery. RETAIL There are a couple of companies that have adopted augmented reality to promote their products and improve customer experience:

Augmented Reality Page 4 of 11 by student Odia Edeawe

Shiseido: The company created a system that has a camera and a screen. The camera takes a picture of the shopper’s face and then they can match the right shade of makeup to their face. De Beers: The company also created a software customers can download at home. The software uses their webcam and allows them to try out the jewelry before they buy it at home.

IKEA: IKEA launched their augmented reality catalogue to enable shoppers to visualise how certain pieces of furniture could look inside their home. Not only that, but the app measures the size of the products against the surrounding room and fixtures to offer a true-to-life size where possible. Video: https://www.youtube.com/watch?v=vDNzTasuYEw The biggest and most popular application of augmented reality might be the Google Glass. The image below explains how the google glass works. The google glass is an augmented reality app because it super imposes virtual content over the layer of our reality.

Augmented Reality Page 5 of 11 by student Odia Edeawe

Augmented Reality Page 6 of 11 by student Odia Edeawe

Tesco, a British grocery chain developed an app compatible with the Google glass that would help shoppers saying commands like “find a product” or “scan a barcode” to the glass and have the glass do those jobs for them. Additionally, the Google glass will allow shoppers add to their shopping carts and also help them view nutritional information of the products of they want to buy right on the glass.

The U.S Military has also developed their own augmented reality system for their soldiers on the ground called the ARC4. The ARC4 allows commanders to send maps and other information to the soldiers’ point of view on the ground. It is attached to their helmets. The device can superimpose satellite information, location of enemies and even mission objectives. The device tracks the location of the soldier and specializes the display to fit the location. Video: https://www.youtube.com/watch?v=zqHjAHWSlsM

Augmented Reality Page 7 of 11 by student Odia Edeawe

Even the Toronto Royal Ontario Museum has used augmented reality to enhance their visitor experience. In 2012, they introduced an iPad app so when visitors came into their dinosaur exhibition they could view what the dinosaur could have looked like if it were alive. The app uses the camera and creates 3D models of the

Augmented Reality Page 8 of 11 by student Odia Edeawe

dinosaurs and skins them and animates them. So the visitor can see how these animals looked and moved before they were extinct.

Video: https://www.youtube.com/watch?v=2ZkZ3kW5CiM The app also lets the visitors to take pictures with these imagined animals. PROBLEMS OF AUGMENTED REALITY Augmented Reality complicates intellectual property law because the technology gathers copyrighted or trademarked material and sometimes transforms them. Many people don’t think that maps for example are copyrighted materials but they in fact are and copyrighted material and if used illegally could spur a lawsuit. In the case of GPS AR technology, making use of maps could potentially lead to a lawsuit. So making use of copyrighted material or manipulating them for AR technology influence laws about how intellectual laws apply to AR. In the United States, their constitutional law does not assume that an expectation of privacy in information transferred to a third party. Augmented Reality Technology AR has the potential to transfer one’s information or observation to another company for analysis and storage. For example, the Google glass technology we talked above about could record videos and information about our surroundings and analyses them like it could tell us the weather and track where we are. The company could violate our privacy by selling our information and our consumer patterns. Also, a lot of these Augmented Reality devices don’t give customers a choice about whether to store on the cloud or locally on the device alone. Sometimes these augmented reality technologies could be wrong and could open the company up for a product liability suit. If the AR device like the Google Glass misidentifies an object that leads a user to have an accident or if it tells the user wrong information that leads to harm then the company may have to pay these consumers for their inconvenience. Augmented Reality has now led to people being assaulted while using them. For example, someone could interfere with the device and cause harm to the user. For example, some of the devices could be hacked into and provide wrong information to the users or induce them with seizures or just place them in a vulnerable position. Ad Age Magazine says that another problem with AR technology is that consumers do not like the state of the present technology. They are not hugely interactive and the graphics are still not what they would like to see. Also, consumers complain about the processing power and battery life of these devices and prefer not to

Augmented Reality Page 9 of 11 by student Odia Edeawe

use them. Also, these devices like the Google Glass aren’t fashionable for consumers to walk around wearing them so they are more likely not to buy them. Another limitation of some of these devices is the high cost. The Google Glasses were sold for $1500 putting them firmly out of the customer’s hands because of the high cost.

VIRTUAL REALITY Virtual reality is a term that is used to describe technology that creates a computer generated environment where the user is totally immersed in it and can manipulate and interact with the objects in the virtual environment and the user cannot tell the difference between the virtual environment and reality. Virtual reality devices are usually manipulated by the user by wearing a head mounted display or glasses which display 3D images that the users can interact with. VR devices can also incorporate sound and videos to heighten the user experience. The biggest development in virtual reality is the Oculus Rift, which was designed and developed by 22-year-old Palmer Luckey, and Facebook acquired it for $2 billion in 2014.

How does the Oculus Rift work? When a user places the Oculus Rift Head mounted display over their eyes, the rift replaces their own vision with digital images. So when the user turns his head, images the viewer sees changes and the user feels like they are inside the game. The body can’t distinguish between reality and the digital images in the rift. The device accomplishes this with a number of different techniques. The head mounted display uses a large screen that takes up 100 degrees of the user’s vision at a time. Each eye looks through different lens at different parts of the screen with creates a stereoscopic 3D effect. Furthermore, the Rift also tracks the movements of the body. So, the Rift has sensors that can detect tilt and orientation of the user. Also, when the Rift has an external camera that monitors the infrared lights on the head mounted display and when these infrared lights moves, the camera takes note of it and makes changes to the images the user sees. Also, the user through the headset can look and lean in all directions inside the virtual world. The oculus rift has also allowed for game controllers to be used in conjunction with the device. So for example, if you are using a mouse or a keyboard or an Xbox controller, the user can control his virtual world using the buttons on these controllers. I came across an interesting video that I think is interesting to see the development of virtual reality. The sex industry is using the oculus rift and other virtual reality and augmented reality devices to advance their business. I provided the video below: https://www.youtube.com/watch?v=FBRSR_LGlOE

Augmented Reality Page 10 of 11 by student Odia Edeawe

Other Developments The NASA Johnson Space Center in Houston is making use of Virtual reality technology to train their astronauts. Their technology models everything an astronaut would encounter in space or in a space station and train them to respond to problems and challenges and perform their job. According to James Tinch, NASA VR Lab manager, "Anything that the crew might see outside on the Space Station we model in here so that when they go outside they feel like they've already been there, because they've experienced it here in the virtual world." Inside the VR HMD, the astronauts in training get a 3D representation of the International Space Station. The NASA virtual reality helmet helps the astronauts practice emergency drills. NASA straps the astronauts to a “SAFER” unit (Simplified Aid For Eva Rescue” and the images that wear in their eyes through their HMD shows them in the middle of a mission and then suddenly there is an emergency where the astronauts needs to return to their station for safety. The astronauts would also feel like they are spinning away from their station. A medical use of VR technology could be psychological therapy. Emory University’s Dr. Barbara Rothbaum and Dr. Larry Hodges of the Georgia Institute of Technology developed the use of virtual environments to treat patients that have phobias and psychological conditions. The virtual environments are used as exposure therapy, which is where the patient is exposed to stimuli that would cause him stress under controlled environments. The doctors report that it is has advantages over real exposure therapy because it is more convenient and the patients are more willing to try this exposure therapy because they know it isn’t actually reality. Ford Motor Company is using the VR technology in the manufacturing of their cars. Ford with their VR technology can test scenarios and designs even before the cars are made. They use the Oculus Rift to guide their developments. With this technology, they can see and experience their designs before they begin production. A worker can wear the oculus rift HMD and can sit in a large room that has 15 feet by 9 feet screen of 4K resolution and experience the car through the virtual world. Ford uses the technology to examine the exterior and interior of a car design and they can observe how an element should look like the dashboard or the wheels or upholstery. The VR technology links right into the company’s Autodesk software on their computers. Virtual Reality History The development of an immersive virtual world began in the mid 1950s when a cinematographer Morton Heilig created a single user console called the Sensorama that had a stereoscopic display, fans, odor emitters, stereo speakers and a chair that moved. He invented a head mounted television that let te user watch television in 3D. In 1961, Philco Corporation created their Head Mounted Device. The HMD had a video screen and a tracking device, which the engineers connected to a closed circuit camera. They wanted the device to be used in dangerous real situations. The users could adjust the camera angles by turning his their heads. In 1966, a computer scientist Ivan Sutherland created an HMD that he connected to a computer. He had wanted to create a device that would let the user look at a virtual world that would appear real to the user. He also wanted the user to interact with objects in the virtual world. The computer created the virtual world that the user can see through his the HMD. Though at this time, the HMD was placed on a suspension system because it was too heavy for the user to carry. The HMD could also track the user’s head movements so that when the user moves the head, the angles change in the display.

Augmented Reality Page 11 of 11 by student Odia Edeawe

LIMITATIONS OF CURRENT TECHNOLOGY Price: Just like the AR devices, virtual reality devices are expensive and sell for more than $300. The Oculus Rift for example sells for $350. Except you are a gaming enthusiast, you aren’t necessarily concerned with spending that amount to buy a virtual reality headset that doesn’t have a lot of applications for the normal citizen outside of gaming. Weight: Another problem limiting the immersion of these technologies in society is the weight. They are heavy and uncomfortable so customers do not feel like buying technology that would weigh heavily on their heads. Graphics: The graphics in VR technology is still a little crude. They haven’t been able to give customers high animation quality or a complete sense that what they see is real except if it’s an actual recording that the user is seeing through his headset.