I think EyeSight is more important than most people give it credit for :p
A huge problem I've had with demonstrating - and being demonstrated to with - MR headsets is that I can't actually see what the user's looking at. This is much more troublesome with MR and passthrough, where it's very important for external users to be able to see that the user is looking at them.
Yeah, there's still a few quirks that need to be worked out, but I think it's a fundamentally important part of future MR headsets, and Meta agrees [0] (much more rudimentary 2021 prototype at [1]).
This is the fundamental struggle with VR displays. Human eyes have very good visual fidelity and it would be real nice to match or exceed that with a VR headset. This presents many technical challenges.
I see your point -- but at least in the current iteration the HMD doesnt let the user view the actual outside world directly with their eyes (there is no optical pass through). What the user sees with their eyes and focusses on is whatever the HMD presents on the internal 4K displays, one for each eye.
Similarly from perspective of others around, they dont see the real eyes of the user - just a simulation of that. So any "eye-contact" -- even with the best execution of technology with high fidelity and low latency -- will still be something different from real eye-contact. I am no purist but difficult to accept at this point for in-person interactions. We have already gotten used to it for facetime / video calls.
Something like Google Glass might be easier to assimilate, maybe?
VR pass through can get a wider FOV than an AR display but it still is a lot less wide than natural vision. Which was my point.
Latency issues might be subjective but there are clearly going to be population trends. I imagine these will be very similar to latency trends with VR and as such latency will be a significant issue.
Front facing displays are even weirder than someone with a headset on. I’m confused that anyone thinks sticking a view of someone’s eyes on a flat screen is going to be anything other than weird. It’s diving head first into the uncanny valley.
it doesn't really matter how close the screen is to your eyes
Yeah, I didn't mean to suggest there is a continuous relationship between trust and distance from the eyeballs :) The promise of AR and VR is that there is a quantum leap when your entire field of vision (or a very large fraction of it) is enveloped by the display though.
it doesn't have the ability to control your sense of reality
That's the entire premise of the products – the "R" in AR and VR is about inserting a shim between you and reality. That requires a level of trust I wouldn't be fully comfortable giving to myself, let alone other people, and never FB.
Isn't one of the biggest problems with these VR headsets the nearness of the display to the eye? It seems like prolonged exposure to this would result in myopia that would worsen faster than it does when exposed to screens that are farther away.
The biggest component that VR doesn't have is a way of simulating focal distances. Current VR headsets focus at a fixed distance of about 2 meters, which makes reading and examining close objects difficult and causes distant ones to appear flat and unreal. The world lacks part of what lets us see depth. Facebook has very impressive prototypes that solve this [0], but so far they've been unwilling or unable to manufacture them at scale.
VR headsets are also severely lacking in brightness and color depth compared to reality or even modern TVs, and they could always use more angular resolution, wider viewing angles, and higher refresh rates.
If you have to wear physical goggles or glasses, it's not going to happen apart from some very niche areas.
Anyone remember 3D films and TV? Apparently the real killer for that was that people didn't like sitting wearing glasses, and those who have to wear glasses all the time anyway couldn't take part.
So can we expect 4 year old kids to patiently wear an expensive VR headset to chat to granny? I doubt it. People can't even be bothered to turn their webcam on for meetings - they're certainly not going to bother putting on a VR headset and grabbing controllers for a 30 minute status update meeting.
That said, I am intrigued by using a VR headset for a virtual display (rather than monitors). In my limited experience of using VR stuff from 2 or 3 years ago the per-eye resolution sucks though and there is no pupil tracking so I can't imagine it being a great experience, but perhaps that is solved now?
I am interested by the technical aspect. Pass-through is not as obvious than sending the camera feed to the screen.
If the field of view, eye position, focus and latency are not right, the world will feel wrong. If it is way off, you are as good as blind. Close range focus is already a lost cause on current gen headsets. I don't know how tolerable the other factors are and how Oculus dealt with the problem.
If people wanted to wear a phone screen taped over their eyes out in public, the technical limitations wouldn't barely make a dent in that desire. But the vast majority of people want to have real in-person human interactions when they're out in the world. That's what the makers of these devices fail to see. But they can't see that because they're self-described VR enthusiasts. If they talk to non VR enthusiasts (which is everyone else), they'll see that the limiting factor isn't the hardware.
Think of it this way. Most people think it's rude to talk to each other indoors with sunglasses on. That's the slimmest form factor you're going to get, and it still steps on the toes of human connection.
I feel like there's a definitional problem with that for vr: you're still wearing the big black glasses/head gear, so if you can see everyone naturally, you can't actually see their faces
Foveated rendering should be an invisible technology — the image you see should be perceptually indistinguishable from the fully rendered image. How to get there is an interesting engineering problem, but for me that isn’t an interesting domain.
You summed up the article quite good, but I don’t claim it will be great. I just think that gaze as an input device is an underresearched topic. Ken Pfeuffer’s research [0] is beyond awesome, but we need masses of thinkerers to explore the domain more. New wave of VR headsets will enable it.
A huge problem I've had with demonstrating - and being demonstrated to with - MR headsets is that I can't actually see what the user's looking at. This is much more troublesome with MR and passthrough, where it's very important for external users to be able to see that the user is looking at them.
Yeah, there's still a few quirks that need to be worked out, but I think it's a fundamentally important part of future MR headsets, and Meta agrees [0] (much more rudimentary 2021 prototype at [1]).
[0]: https://twitter.com/douglaslanman/status/1732441912043589993
[1]: https://research.facebook.com/blog/2021/8/display-systems-re...
reply