Apple's Headset Said To Feature 14 Cameras Enabling Lifelike Avatars (macrumors.com) 15
Citing a report from The Information's Wayne Ma (paywalled), MacRumors reports Apple's long-rumored AR/VR headset is said to feature 14 cameras that enable lifelike avatars with accurate facial expressions. The company is also working with former design chief Jony Ive on the project. From the report: For starters, one of the headset's marquee features is said to be lifelike avatars with accurate facial expressions captured by 14 cameras: "Other challenges, such as incorporating 14 cameras on the headset, have caused headaches for hardware and algorithm engineers. The cameras include those that will track the user's face to ensure virtual avatars accurately represent their expressions and mouth movements, a marquee feature."
The report adds that Apple's former design chief Jony Ive has remained involved with the headset project as an external consultant to the company: "One person familiar with the matter said Ive's consulting work for Apple since he left includes the headset, adding that he is often brought in to help his former team push through their preferences in areas such as battery, camera placement and ergonomics over those of engineers. Two people said even after Ive left Apple, some employees on the headset project were still required to make the trek from Cupertino to San Francisco, where Ive has a home, to get his approval on changes. Ive has continued to tweak the headset's design. While earlier prototypes had the battery in the headband, he prefers a design that would tether the headset to a battery the user wears, similar to Magic Leap's headset design. It couldn't be learned if this approach will make it into the final design."
The initial version of Apple's headset is said to lack a focus on gaming: "Four people who have worked on the project also criticized its lack of focus on gaming, a category of software that appeals to early adopters, which was important to the success of the iPhone and has been a big priority for Meta's VR group. Those people said Rockwell's group almost never mentioned games in internal presentations about possible uses for the headset. Apple isn't developing game controllers for the device and is aiming to use hand tracking or in combination with a clothespin-like finger clip as inputs for the device, multiple people familiar with the project say." On Thursday, Bloomberg reported that Apple executives previewed the upcoming headset to the company's board last week, "indicating that development of the device has reached an advanced stage."
The report adds that Apple's former design chief Jony Ive has remained involved with the headset project as an external consultant to the company: "One person familiar with the matter said Ive's consulting work for Apple since he left includes the headset, adding that he is often brought in to help his former team push through their preferences in areas such as battery, camera placement and ergonomics over those of engineers. Two people said even after Ive left Apple, some employees on the headset project were still required to make the trek from Cupertino to San Francisco, where Ive has a home, to get his approval on changes. Ive has continued to tweak the headset's design. While earlier prototypes had the battery in the headband, he prefers a design that would tether the headset to a battery the user wears, similar to Magic Leap's headset design. It couldn't be learned if this approach will make it into the final design."
The initial version of Apple's headset is said to lack a focus on gaming: "Four people who have worked on the project also criticized its lack of focus on gaming, a category of software that appeals to early adopters, which was important to the success of the iPhone and has been a big priority for Meta's VR group. Those people said Rockwell's group almost never mentioned games in internal presentations about possible uses for the headset. Apple isn't developing game controllers for the device and is aiming to use hand tracking or in combination with a clothespin-like finger clip as inputs for the device, multiple people familiar with the project say." On Thursday, Bloomberg reported that Apple executives previewed the upcoming headset to the company's board last week, "indicating that development of the device has reached an advanced stage."
Are the cameras aimed at your face? (Score:3)
Re: (Score:2)
Seems like about the only way to 'look' at the user, especially if they are wearing a headset.
I would assume some cameras would still be outward looking, but capture the lower face. I also assume there's probably some inside the headset to capture the eye area too.
14 is a hell of a lot, I can't imagine they would use all of those as outward facing, they just don't need that much material to stitch together an adequate 'AR' representation of the real world.
Is portability the selling feature? (Score:2)
Seems like an inefficient way to get a look at the user, especially if they are wearing a headset
Agreed. If the goal is simply to capture the user's expression and turn that into a moving avatar, then a Kinect-like webcam would be more than enough. Of course, that won't be portable. It seems to me that headsets can perform any of two functions, as a projection system for virtual worlds (classic VR) or as a portable spycam (Google glass), or a fusion of both (Meta's version of the metaverse).
Facial Represenation is nice but.... (Score:1)
Re: Facial Represenation is nice but.... (Score:1)
Re: (Score:2)
Facebook has some experience of this in the Metaverse. They have found that not having legs on your virtual avatar is a good idea, because they are difficult to map onto the environment reasonably. For example you might have some stairs, and the system would need to make your legs climb them, and it would feel weird for the player to see their virtual legs do that while their physical ones remain still.
It also slightly reduces the rampant sexual harassment in the Metaverse, since avatars have nothing down t
Re: Facial Represenation is nice but.... (Score:2)
Legless avatars make no sense, they donâ(TM)t mimic the real world and they eliminate the chance of disabled people to feel a sense of escapism. We donâ(TM)t need legs to work perfectly.
Re: (Score:2)
"battery in the headband" (Score:2)
That does make sense. Whilst it's rare, batteries can catch fire and sometimes explode - I'm not sure I'd want one strapped to my head, near my eyes.
But it also adds to the complication of setup, as does "a clothespin-like finger clip".
It's also a safe bet to assume you'll need an iPhone "for every feature", much like the apple watch.
As for the "gaming" aspect, right now, it
Re: (Score:2)
If the are putting stuff in the headband, it won't take a massive brainwave for them to think of putting an EEG or brain-radar in the headband that can be used to control the headset and monitor you for sleep deprivation and stroke symptoms.
Vertical market applications? (Score:1)