John Hanke is an unassuming man from a tiny Texas town who learned how to code in middle school back in the 80’s, the early days of personal computing. Intrigued by some high school students playing a Star Trek game on an old TRS-80, he quickly learned how to code his own games, even earning himself $250 for submitting his game code to ANALOG Computing magazine. While his love for computers and gaming never waned, he also had a healthy interest in many other subjects and activities and a real love for new and exciting experiences.

After obtaining both a liberal arts education and an MBA, he went on to found multiple gaming and mapping startups, which eventually resulted in his becoming CEO of Keyhole, a geo-spatial data-visualization company that was acquired by Google in 2004. While at Google he worked on the company’s mapping and geo-imaging technology while developing his own pet project company, Niantic.

Hanke loved computers and gaming, but what he really wanted to develop was a game that would encourage players to get out in the world, meeting new people and interacting with their local landscapes. What resulted was a mobile game you might have heard of: Pokémon Go.

Besides being one of the most successful mobile games of all time (6 million downloads in just the US, New Zealand, and Australia, all on the first day of its release), it is by far the most successful augmented reality game to hit digital shelves ever. But that begs the questions: what is augmented reality? How is it different from virtual reality? Is it just for games, or does it have other applications? This article will answer these questions and more, telling you all you need to know about augmented reality and its uses.

What is Augmented Reality?

Augmented reality (AR) is an enhanced version of a physical landscape in one or more sensory modalities in real time. This is a complicated way of saying that AR is a real-time experience of a real place that has been digitally altered in one or more ways. The clearest and most widely-viewed example (though most people might not recognize it as AR) is that yellow first-down line you see in every televised football game. Sports have actually been a leader in augmented reality because overlaying graphics helps spectators clearly understand what is happening in a game or match, where points have been awarded or earned, and what strategies players are using. In a football game, you are seeing a real-life location, in “real time” (maybe it was pre-taped so it’s not live, but it’s still playing out like it did when the game occurred), with helpful digital alterations (the first down line and other graphics) in one or more sensory modalities (in this case, vision).

Though this is by far the most prominent application of AR technology currently, once you understand exactly what is augmented reality, you start to see all of the different situations that would benefit from a little bit of AR. The proliferation of cell phones has really been a driving force in the augmented reality world, not just in the case of Pokémon Go, but also with maps, real estate, and social media, amongst others.

The way augmented reality works on a phone is that the phone’s camera takes in the physical world around you in real time. Artificially intelligent (AI) software recognizes the objects on your screen, allowing you to utilize the options in an application (depending on the app) to manipulate the depicted world in a realistic way. This may be a case of prospectively arranging furniture in a room to see what it would look like without having to actually buy and haul the furniture. Or maybe if you’re traveling in an unfamiliar location, your phone will detect where you are and overlay information about each business or which direction to go to find tourist attractions.

Augmented reality can be used to redesign something without actually changing it or creating a prototype – the AI can just overlay the new design on the old. It’s kind of like what you see in a lot of sci-fi movies (or even with Iron Man in the Avengers movies), where people have glasses, goggles, or electronic eyes that analyze everything in the field of vision and list information related to it. Except that it’s so much more than that as well.

AR versus VR

Now that you understand what is augmented reality, we need to make sure you understand what the difference is between it and virtual reality (VR), another up-and-coming technology that’s attracting major investors. While augmented reality is building on and interacting with a world that already exists, virtual reality is creating an entirely virtual environment. It is the literal opposite of the real world. On a scale of the real world to virtual world, augmented reality falls closer to the real world than it does virtually. And in many ways, this has made it (paradoxically) more difficult to create than virtual reality.

With VR, the creator has complete control over the environment. Everything you experience in virtual reality has been placed there by the creator and has been a deliberate choice. While creating a compelling and believable experience in virtual reality is no mean feat, it pales in comparison to the difficulties encountered by augmented reality programmers.

With AR, it’s not a simple matter of just taking digital objects and pasting them on an image of the real world like stickers. Even the most simplistic digital items are placed with reference to scale, perspective, and object recognition.

Take, for example, IKEA’s augmented reality app for trying out furniture in your home, IKEA Place. If you just take a picture of a room, there’s no telling what the scale is: are you zoomed up close to take a close-up shot of a flower? Or are you zoomed way out to take a photo of the whole room, like for a real estate listing? The IKEA app has to know the size of the room and the scale of the image so that it knows what size to make the virtual furniture that it inserts into your room. It also has to be able to figure out perspective in order to allow you to position the couch exactly how you would place it if it were really there. Without these features, the app would be useless, but there’s an incredible amount of engineering that has to go into these key features and make them applicable to any room anyone ever tries to place furniture in.

Another key example is the Snapchat filters or the iPhone capabilities that allow you to turn your face into an emoji. Though this seems silly and banal, just think of all the engineering that has to go into being able to map out every face that ever was and to apply a filter to it, in 3D. With the augmented reality map for traveling, the technology has to be able to recognize where buildings begin and end before it can go about figuring out what business occupies them and what all the relevant information is. AR creates an entirely new virtual layer of the physical world around us.

The AR company Ubiquity6 is creating an app that opens up the environment to digital editing. People can leave digital notes in physical locations for others to search for and find. Take this a step further and the possibilities are endless: renters can leave virtual notes for subletters about how best to handle the temperamental hot water heater. Teachers can create educational scavenger hunts whose items wouldn’t run the risk of going missing or being damaged. Disaster relief workers can leave notes for later shifts on what areas they’ve worked on or what still needs to be done.

At its core, when understanding what is augmented reality, you’re comprehending that AI has to be programmed and trained to do the things humans do without a second thought. AR is at the forefront of AI development and will be one of the leaders in technological progress. The field can’t move forward without advancing the standard of artificial intelligence. That is why it is such a big deal, and that is why it’s so different from – and more applicable than – virtual reality.

The Future of AR

While virtual reality tends to be more isolating in nature, an intrinsic characteristic of augmented reality is its ability to bring people together and to get them to interact with the outside world more. As John Hanke (aforementioned creator of Pokémon Go) says, “With AR you can paint the physical world with this interactive, two-way interface that can give you information about the status of every physical thing that you’re passing by or interacting with—and let you book it, schedule it, buy it, or learn about it. That’s the ultimate vision that gets me excited.” As AR technology develops and more people come to understand what is augmented reality, its applications and usage will only become more widespread. At the rate it’s moving, it promises to be the dominant technology in just another decade or two. With that in mind, let’s take a look at what the future holds.

The proliferation of augmented reality will make all the devices we currently use (including smartphones, laptops, tablets, etc.) obsolete, the same way the smartphone made digital cameras, MP3 players, landlines, and GPS’s obsolete. Companies are already developing AR tech that, rather than utilizing your cell phone and acting through it, supplants it by projecting a phone or screen onto a nearby surface and reacting when you interact with that projection. Think, again, of sci-fi or Iron Man, where there are floating screens all around you that you can interact with. Except these screens needn’t be actual, physical devices. They can just be projections that you can place wherever you’d like.

In the same way, developers are working on glasses and windows that turn the outside world into a canvas for you to display information, graphics, or images or to manipulate as you please. In this way, AR will alter the very way we interact with and perceive the outside world. People from all different backgrounds and countries will be able to converse naturally as AI takes in what is said, translates it, and either throws up the translated text in your field of vision or plays a translated version in your ear (à la babel fish from Hitchhiker’s Guide to the Galaxy).

As with any game-changing technology, there are, of course, concerns moving forward. There is a very real danger of information overload, over-the-top connectivity, and an inability to unplug. The whole world might become overrun with digital manipulation, or even persistent advertising, culminating in an all-too-real embodiment of many a Black Mirror episode. People who want some space and distance from the technology might find they can’t get it, either from social or work pressure or just because our lives will become too reliant on the tech. Just as augmented reality has enormous potential to drastically change the world for good, it could very easily go the opposite way.

So as we move forward, it is imperative that we stop at every juncture and really ask ourselves: what is augmented reality? What do we want it to be? Where can we let it progress, and where should we check its spread? We need to learn from all of the debate and scandal swirling around the tech giants now (especially Facebook, but they’re all implicated) and place some real restrictions on what AR can become. But once we do that, augmented reality truly has the potential to open up a new world and fundamentally change the way we interact with it.