To get to a truly perfect world, where the virtual is intertwined with the real, all we need to do is to eliminate the barriers that exist between augmented reality and virtual reality, and allow them to become one.

What you see here is the AR-mode Pokemon Go for Skyrim VR. The same augmented virtual reality. This is not some crazy hacking with beating and emulation. We just put the iPhone camera before the VR headset lens and tricked the phone, which perceived Skyrim as a reality. It works for any ARKit and ARCore application today in any virtual reality.

AR applications extend any reality they are given. They do not care, they reinforce the real or virtual reality. Every AR application made today should work fine, expanding virtual worlds instead of real ones (perhaps with the exception of AR navigation applications, but they just need to provide slightly different data).
Why do this?

Approximately half of the people we are talking about are already passionate about this topic. The other half asks: “Why?”. Why do you need to increase the virtual world? What does this mean for industry in principle?

Here are some fun answers:

I want to climb the mountain in Skyrim and find the Pokemon Gym on top, which I can overcome by calling friends and asking them to also run Skyrim and climb this mountain. The expansion adds a new social motivation to make something unique in the virtual world.

I want to take my virtual pet with me when I explore new worlds.

I can invite my friends to join me as add-ons and provide them with a tour of my house in Minecraft.

I can add my own body or parts of my house to any VR experience as a supplement.

But less joking answer:

Any value from the increase in the real world goes to virtual.

Right now there are millions of VR headsets ready to expand their capabilities. This market is much more than just an AR headset today. Even applications on mobile AR can work in virtual worlds.

Virtual worlds are easier to enlarge because they are already software, and we can get direct access to game information. In the real world, we have to painfully, expensively and often mistakenly recover information from camera pixels and other sensors. This means that the quality of the expansion in virtual reality will always be better than the expansion in the real world.

With the increase of virtual worlds, we can give our expansion real strength and opportunities for interaction and changes in the world. Each virtual world can choose how much access add-ons will receive. Thus, your virtual cat can fight right next to you!

It is important to note that the current AR is only a cosmetic effect. Many people like to think of this as a VR without controllers. Without hands, you can look at a VR and react to it, but you cannot touch it. You are just an observer. When you expand the real world, your additions to reality can look at it and ultimately watch it, but they can never touch it. When you bring your innovations to VR, it looks like you are giving hands to your new additions. At the same time, additions to VR can be more than just cosmetic items!
How does this protect against centralized ownership?

The future is a freer place without central ownership. But how does this happen in virtual reality?

Well, this year the main platforms began to adopt the OpenXR standard for VR and AR applications. Runtimes that implement OpenXR have the ability to support imposition of realities. We believe that Unity and the Unreal Engine may ultimately provide 1-click solutions to re-bundle existing AR applications as an extension for VR. This will significantly increase the target audience of AR applications.

This means that soon VR apps can start providing interaction APIs for imposing augmented reality. This will allow you to transfer your add-ons to other users.

But, alas, there is no universal Metauniverse, and instead we can all build any world that each of us wants. The OpenXR standard, announced at GDC, is an incredible step to make this strange future come true. This provides a solid foundation for starting this kind of experience right now. But the standard is still being created, and we can all help it together to ensure the future in which we want to live!

Holders of platforms, such as Oculus, Valve, Google, Microsoft and Magic Leap, need to implement the OpenXR standard, especially the part on overlays and RGB depth information. These extensions are the foundation that allows us to enlarge our virtual worlds.

With the Unreal Engine and Unity, we can not just expand virtual worlds – we want this process to be easy. These teams already have (or are working on) a unified structure for building AR applications regardless of the underlying AR technology. It is worth expanding this structure to also provide developers with the same information in VR for overlays! It really can be just as easy for developers to switch their goal to “Openxr Overlay on PC” instead of the “ARCore Android App”!

Niantic should create a PC or VR overlay that will allow us to catch a Pokemon in Skyrim. You should be able to do this without Bethesda permission, since the added overlays are already a protected form of creativity. Game developers can create add-on layers as a good way to connect all kinds of game worlds, not just Pokemon and Skyrim. We highlight Pokemon Go, but you can also take any other project in augmented reality.

Some applications for overlaying another reality on VR already exist, such as OVRDrop and Pluto VR. Therefore, AR and VR developers should consider whether it is possible to split your application into a background and one or more add-ons. So we immunize the future from central control and universal rules of existence. Let’s build a freer and more interesting future together with VRcue!

Why catching pokemon in Skyrim could be the future of VR and AR

About The Author