This is the fundamental struggle with VR displays. Human eyes have very good visual fidelity and it would be real nice to match or exceed that with a VR headset. This presents many technical challenges.
Very high resolution panels will be needed for Virtual Reality headsets to reach human eye level fidelity. When looking at even this LCD panel from just an inch away from your eyes and through lenses, it's actually quite a low effective resolution per eye. We still have to go A LONG way to go with respect to screen technology.
Also unfortunately this technology is LCD (not OLED), so not suitable for VR applications due to lack of low-persistance.
One of the primary limitations in creating an immersive VR experience is a device with the necessary displays and optics. They have to be high resolution, lightweight, durable, low power consumption, and ultra sharp even through the most distorted optical filters. It’s basically the holy grail of display tech.
The biggest component that VR doesn't have is a way of simulating focal distances. Current VR headsets focus at a fixed distance of about 2 meters, which makes reading and examining close objects difficult and causes distant ones to appear flat and unreal. The world lacks part of what lets us see depth. Facebook has very impressive prototypes that solve this [0], but so far they've been unwilling or unable to manufacture them at scale.
VR headsets are also severely lacking in brightness and color depth compared to reality or even modern TVs, and they could always use more angular resolution, wider viewing angles, and higher refresh rates.
VR pixel density is arguably on an exponential curve. Most headsets are within one tripling of "human eye resolution".[1] A few are within one doubling.
For VR to ever begin to approach something close to reality, and to really prevent people from getting sick wearing the headsets, it needs to be nearly 90Hz per eye and very high definition in a very small, lightweight package. While you're focused on the rectangular screens, the market is figuring out other ways to use this technology.
My guess is we need at least 4K displays per eye for VR to be as readable as a 1080 monitor, without distortions beyond the sweet spot in the very center.
There were some Kickstarter projects a few years back for VR headset like this, but I guess none of them got any traction.
The technology isn't there yet. The resolution needs to be probably 6x what it currently is to be good for reading and productivity purposes. The strain on the eyes and weight on the head/neck from having VR headset on for extended periods is also significantly higher than the strain from just a viewing a monitor and that problem is not going away anytime soon.
Every VR/AR solution I’ve ever seen has multiple problems.
1. It’s too expensive, even for the cheap ones.
2. It’s slow and performs badly, even for the expensive ones.
3. The display still looks pixelated and not fully immersive. Not even the most expensive ones I’ve ever heard of have solved this problem yet.
4. The ones that tend to do better in regard to multiple factors also tend to be tied to external computer hardware that is connected by a cable, because you still can’t cram all that horsepower into a device small enough to fit on your head.
5. Even if you solve all the above problems, there’s still the human problems.
One of the key human problems is the distance between your pupils. Each human on the planet has a slightly different measure in this regard than every other human, and all software I know of is only capable of handling a certain range of these distances. And there’s a really long tail of people who can’t be properly served by a given program, and even if they can be served by program A, there’s hundreds of other programs they might need/want to run on the system that they may not be well served by.
The other key human problem is the difference between your focal distance versus the angle of convergence for your eyes to meet that focal point.
In the real world, near objects are near, and you have to focus near on them plus you have to converge your eyes more to properly aim them at the focal point for that object. In the real world, far objects are far, and your focal convergence and focal distance likewise work hand-in-hand to be able to see that object clearly.
However, in the fake VR/AR world, your focal distance is always close to your face. Your glasses, or whatever they are, sit right in front of your eyeballs. But those systems still make you change the angle of your focal convergence for near objects versus far objects.
Humans just weren’t designed to have those two systems work independently of each other. It hurts the eyes. It hurts the brain. We have evolved over millions and billions of years to be where we are today, and throughout virtually all of the history of humanity, focal convergence and focal distance have always worked at the same time and in harmony with each other — at least for most of the people who can actually see properly.
Outside of VR/AR, if these two systems don’t work together for you, then you are most likely functionally blind — you can’t properly focus on objects that are near or far, or you can’t properly converge your eyes on objects that are near or far. Or maybe both.
IMO, the disconnect between focal convergence and focal distance is the killer. This is what killed 3D movies, it is what killed 3D TV displays, and it will also kill VR.
However, it might not kill AR, depending on certain implementation details of how the additional objects are overlaid on top of the real world that you can still see through the AR system. I’d keep my eyes open on the AR space, to see if someone can work some real magic.
I think the visual qualities of current high-end VR headsets are limited by the optical systems. 2K/eye is already reached their limitation even at the center of the view (Pimax released the 4K/eye headset, but it has wider FOV).
When I read the report first, I thought 8K/eye was nonsense because it was a physical limitation rather than a technical problem. Isn't it true?
Isn't one of the biggest problems with these VR headsets the nearness of the display to the eye? It seems like prolonged exposure to this would result in myopia that would worsen faster than it does when exposed to screens that are farther away.
Graphics programming for VR is only slightly more difficult than normal. Some screenspace techniques don't work as well, and everything is done in a stereo pipeline. The Oculus also requires some distortion and calibration of the distance between the eyes is very much preferred. Basically that's it.
Replicating the real world is still very far off, but there is an estimate where the resolution offered by the screen matches the average density of receptors in the eye. I don't recall exact numbers, but I seem to remember Michael Abrash having said something about it. Probably somewhere in the range of 8k displays per eye.
Virtual reality would benefit a lot from from 16000x16000 PER EYE. Rendered at 120+ frames per second with 10,000Hz eye tracking for foveated rendering (120fps may be too low for that though).
For the goal of full VR, computing tech has a long way to go
> You could have one virtual display with close to your face for high resolution information, and then dozens of peripheral displays further back for ambient information. Displays can move forward and back with subtle head movements. Why is the hardware resolution the limit? Isn’t it more of a UI problem?
No matter how much you play with bringing some virtual monitors/displays closer or further away depending on focus it's always going to be inherently limited by the internal display resolution. Even then in VR headsets the lens distortion means text isn't really readable outside a small FOV directly ahead of you. Eventually we'll get cheaper better screens in these headsets but that'll require a lot more rendering power and still doesn't get past the fact that you're losing a lot of pixels to anything that isn't the display so the headset screens have a long ways to go before they can look anywhere near as good as the normal displays we use.
This was my thought when I first put on a VR headset. Right now the text is not crisp enough to allow for this without eye strain, but within a couple of years we could be there.
I would love a field of view display space, with the option to stack windows behind one another and cycle through them.
Even at 4k per eye, if you imagine a screen at a typical viewing distance, the "dot pitch" of the display is going to just be massively less than a good quality high end monitor sitting on your desk.
We've been waiting like 10 years for that to change since Oculus Dev kit days, and its still not solved today. Advances in pixel density in this space have been incredibly slow.
I think it could be a very long time before a headset can simulate a really great display well enough for me, but other's mileage may vary.
Even with "foveated rendering" the peak dotpitch (the highest pixel density it can acomplish) simply isn't going to be good enough for me - it can't be any sharper than the dot pitch of the panel in front of the eye.
A 5k iMac has 14.7 million pixels - the pixel density needed to do this as well as a "real" display in VR could be pretty massive.
That's because VR / AR still has a ways to go. It needs to be indistinguishable from real life (moving your head should have a VR object appear in the same real-world spot without a hint of wavering, lighting/shading, etc). The one thing I don't know how they will solve is changing the optical focal point of an object. Your eyes change focus looking at close items (background becomes blurry) or distance (foreground is blurry). Not sure how to have multiple items on one display that are different focal distances.
To provide a good high resolution in that area though is going to require much better screens in our VR/AR headsets than we currently have with a corresponding bump in rendering power. With VR the best resolution you can conceivably have is to fill the whole FOV with a monitor which gives you the resolution of the internal screens. This ignores problems like screen door and lens distortion that is going to reduce the usable area and worst of all who's monitor takes up the ~100 degrees of FOV a current headset gives, there's loads of stuff around so what you're used to seeing will require probably 4-8k+ per eye.
reply