Dieter Schmalstieg is professor and head of the Institute for Computer Graphics and Vision at Graz University of Technology (TUG), Austria. His current research interests are augmented reality, virtual reality, real-time graphics, 3D user interfaces, and visualization. He received Dipl.-Ing. (1993), Dr. techn. (1997) and Habilitation (2001) degrees from Vienna University of Technology. He is author and coauthor of more than two hundred peer-reviewed scientific publications, associate editor of IEEE Transactions on Visualization and Computer Graphics, member of the editorial advisory board of computers and graphics and of Springer Virtual Reality, member of the steering committee of the IEEE International Symposium on Mixed and Augmented Reality, chair of the EUROGRAPHICS working group on Virtual Environments (1999-2010), advisor of the K-Plus Competence Center for Virtual Reality and Visualization in Vienna, and member of the Austrian Academy of Science. In 2002, he received the START career award presented by the Austrian Science Fund. In 2012, he received the IEEE Virtual Reality Technical Achievement Award for seminal contributions to the field of augmented reality. Since 2008, he is also director of the Christian Doppler Laboratory for Handheld Augmented Reality.

Tobias Höllerer is professor of computer science at the University of California, Santa Barbara, where he leads the Four Eyes Laboratory, conducting research in the four I's of Imaging, Interaction, and Innovative Interfaces. Dr. Höllerer holds a Diplom in informatics from the Technical University of Berlin, as well as an M.S. and Ph.D. in computer science from Columbia University. He is a recipient of the U.S. National Science Foundation's CAREER award for his work on "Anywhere Augmentation." enabling mobile computer users to place annotations in 3D space wherever they go. In 2013, he was named an ACM Distinguished Scientist. Dr. Höllerer is author of more than one hundred fifty peer-reviewed journal and conference publications in the areas of augmented and virtual reality, information visualization, 3D displays and interaction, mobile and wearable computing, and social computing. Several of these publications have been selected for best paper or honorable mention awards at such venues as the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), IEEE Virtual Reality (VR), ACM Virtual Reality Software and Technology, ACM User Interface Software and Technology, ACM MobileHCI, IEEE SocialCom, and IEEE CogSIMA. Dr. Höllerer is an associate editor of IEEE Transactions on Visualization and Computer Graphics. Among his many organizational roles for scientific conferences he served as program chair for IEEE VR 2015, ICAT 2013, IEEE ISMAR 2010 and 2009; as general chair of IEEE ISMAR 2006; and as member of the steering committee of IEEE ISMAR.

Preface xix

Acknowledgments xxv

About the Authors xxvii

Chapter 1: Introduction to Augmented Reality 1

Definition and Scope 3

A Brief History of Augmented Reality 4

Examples 13

Related Fields 28

Summary 31

Chapter 2: Displays 33

Multimodal Displays 34

Visual Perception 39

Requirements and Characteristics 40

Spatial Display Model 56

Visual Displays 58

Summary 84

Chapter 3: Tracking 85

Tracking, Calibration, and Registration 86

Coordinate Systems 87

Characteristics of Tracking Technology 90

Stationary Tracking Systems 96

Mobile Sensors 99

Optical Tracking 105

Sensor Fusion 117

Summary 120

Chapter 4: Computer Vision for Augmented Reality 121

Marker Tracking 123

Multiple-Camera Infrared Tracking 132

Natural Feature Tracking by Detection 138

Incremental Tracking 149

Simultaneous Localization and Mapping 156

Outdoor Tracking 164

Summary 176

Chapter 5: Calibration and Registration 179

Camera Calibration 180

Display Calibration 183

Registration 190

Summary 194

Chapter 6: Visual Coherence 195

Registration 196

Occlusion 199

Photometric Registration 205

Common Illumination 216

Diminished Reality 227

Camera Simulation 231

Stylized Augmented Reality 236

Summary 237

Chapter 7: Situated Visualization 239

Challenges 241

Visualization Registration 245

Annotations and Labeling 248

X-Ray Visualization 254

Spatial Manipulation 260

Information Filtering 265

Summary 270

Chapter 8: Interaction 271

Output Modalities 272

Input Modalities 279

Tangible Interfaces 286

Virtual User Interfaces on Real Surfaces 294

Augmented Paper 295

Multi-view Interfaces 297

Haptic Interaction 304

Multimodal Interaction 304

Conversational Agents 306

Summary 309

Chapter 9: Modeling and Annotation 311

Specifying Geometry 312

Specifying Appearance 317

Semi-automatic Reconstruction 319

Free-Form Modeling 322

Annotation 325

Summary 328

Chapter 10: Authoring 329

Requirements of AR Authoring 330

Elements of Authoring 333

Stand-Alone Authoring Solutions 335

Plug-In Approaches 339

Web Technology 341

Summary 342

Chapter 11: Navigation 345

Foundations of Human Navigation 346

Exploration and Discovery 347

Route Visualization 347

Viewpoint Guidance 350

Multiple Perspectives 354

Summary 360

Chapter 12: Collaboration 361

Properties of Collaboration Systems 362

Co-located Collaboration 364

Remote Collaboration 370

Summary 377

Chapter 13: Software Architectures 379

AR Application Requirements 380

Software Engineering Requirements 382

Distributed Object Systems 385

Dataflow 389

Scene Graphs 395

Developer Support 400

Summary 407

Chapter 14: The Future 409

What May Drive Business Cases 410

An AR Developer’s Wish List 411

Taking AR Outdoors 415

Interfacing with Smart Objects 417

Confluence of Virtual Reality and Augmented Reality 418

Augmented Humans 419

AR as a Dramatic Medium 420

AR as a Social Computing Platform 421

Summary 422

References 423

Index 473