Apple Acquires Startup That Uses AI To Compress Videos (techcrunch.com) 30
Apple has quietly acquired a Mountain View-based startup, WaveOne, that was developing AI algorithms for compressing video. From a report: Apple wouldn't confirm the sale when asked for comment. But WaveOne's website was shut down around January, and several former employees, including one of WaveOne's co-founders, now work within Apple's various machine learning groups. In a LinkedIn post published a month ago, WaveOne's former head of sales and business development, Bob Stankosh, announced the sale. "After almost two years at WaveOne, last week we finalized the sale of the company to Apple," Stankosh wrote. "We started our journey at WaveOne, realizing that machine learning and deep learning video technology could potentially change the world. Apple saw this potential and took the opportunity to add it to their technology portfolio." WaveOne was founded in 2016 by Lubomir Bourdev and Oren Rippel, who set out to take the decades-old paradigm of video codecs and make them AI-powered. Prior to joining the venture, Bourdev was a founding member of Meta's AI research division, and both he and Rippel worked on Meta's computer vision team responsible for content moderation, visual search and feed ranking on Facebook.
Quantity Over Quality? (Score:2)
From my simple understanding, this allows AI to show you "important elements" (text and faces), over "non essential" elements like scenery without requiring new hardware or codecs. Am I right? So, they are finding a way to show you lower quality video when there is a poor connection without sacrificing "important elements".
That's neat, but can I turn it off?
Re: (Score:2)
Whether loss is incurred from quantization, truncation of higher frequency data or several other things, it definitely means loss.
What tends to make one codec show better results in PSnR and SSIM is how well they manage to reduce loss. An excellent example (my daughter and I wrote a paper on it recently), is that a great codec could decrease loss by improving motion search through greater ranges which are availab
Re: (Score:2)
Re:Quantity Over Quality? (Score:4, Interesting)
Feed in a low quality copy of the video. AI generates a high resolution version. Take the difference between the AI version and the real version, and compress it.
The low res version and the compressed difference data will be smaller than compressing the full quality data. Could even be lossless if you want.
Re: (Score:2)
But then you’d have to transmit the low quality version and the difference. That’s like the whole thing.
You could however run the same AI prediction on both ends and then transmit only the difference between the prediction and the next actual frame. That’s what I thought they’d do but it doesn’t seem to be the case.
Re: Quantity Over Quality? (Score:1)
Re: (Score:3)
You've spun it into anti-corporate outrage which is on point for /. but when it comes to compression, "cost savings" and "high quality" are two sides of the same coin.
Codecs devote a lot of time to deciding where to spend bits. Have you ever looked at a scene and noticed compression artifacts floating around some important element? Maybe text on a document, or a figure cloaked in dark backdrop bokeh in a horror. That Game of Thrones episode in the final season that happened entirely at night and was almost
Re: (Score:3)
Attention-based compression is the future (Score:2)
AI that can focus more bits in places we pay more attention and use fewer bits in paces that we don't pay attention is the future of video compression, especially for streaming and doubly so for videoconferencing. I don't care about the details on the leaves of the trees behind the person I'm talking to, I care about their facial expressions and their gestures.
There are several AI video compression efforts out there, from DeepMind's MuZero [bbc.com] to NVIDIA's Maxine [nvidia.com] to smaller players like Deep Render [techcrunch.com] and Orange A [github.com]
Attention-based compression is Marketing Bullshit (Score:2)
AI that can focus more bits in places we pay more attention and use fewer bits in paces that we don't pay attention
Are you using a 28.8k dial-up modem or something? The few pixels you save are meaningless. Attention-based compression is Marketing Bullshit
Re: (Score:2)
Russ Hanneman would disagree.
Re: (Score:3)
The newest codecs ... AV1? have some areas that are open to decisions being decided by AI. Even in h.264 has optimized presets for situations like slide shows or cartoons or black and white film or lower color spaces. Detectors not using AI could change optimization modes but maybe do a smarter job of it. Quality ranking is decided by amount of motion/change already and this can be perfect for AI -- I do not know if anybody has been using existing face detection algorithms (on cameras) in codecs to help ma
Re: (Score:2)
The newest codecs ... AV1? have some areas that are open to decisions being decided by AI.
Even MPEG2 can do that. The way CODECS are specified is in terms of the decoder: the bitstream must decode to the correct values, but you can create a decodable bitstream however you like.
Even with JPEG there are things you can do. In the encoding of a block, you quantize the frequency components. Many of them go to zero. If all the end ones go to zero, there's a special token which indicates "the rest of the block is
Re: Attention-based compression is the future (Score:2)
>> What I would like to see is one which turns video into detailed text descriptions and from that back to video... with really interesting results! (like how they ignore some words) that is, after they get it to do motion and be consistent between frames... although that will be an interesting slow motion as each frame changes drastically.https://hackaday.com/?p=578041
I think the best video compression will soon be AI driven, though probably not using this type of algorithm. Itâ(TM)s still cool
Re: Attention-based compression is the future (Score:2)
Somehow i mangled my reply trying to log in⦠follow the link, that type of âcompressionâ(TM) is attempted at the end of that link. Hereâ(TM)s the primary link, which is what i should have posted anywayâ¦
https://jaspervanloenen.com/bl... [jaspervanloenen.com]
And the training data is from... (Score:1)
AI that can focus more bits in places we pay more attention and use fewer bits in paces that we don't pay attention is the future of video compression
So, where would it get training data to learn where peoples focus would be in a video I wonder?
Interesting that pretty soon they are releasing a headset with integrated eye tracking where one promoted use is watching video...
Re: (Score:2)
Interesting that pretty soon they are releasing a headset with integrated eye tracking where one promoted use is watching video...
By tracking eye movement they could save some data by not displaying image that would be in each eye's blind spot.
Re: (Score:2)
The latencies involved and duration of fixations make this idea a losing proposition. I know it's all the hype these days, but gaze-contingent compression doesn't work unless (a) you can change the compression on the fly from one frame to the next and (b) you have a frame rate above 200 FPS or so. Even at 200 FPS, you can feel the latency. We do experiments that are almost exactly this idea in my lab.
Re: (Score:1)
Interesting data on this, thanks!
Re: (Score:1)
It would be interesting if it used AI to fill in irrelevant details that humans are unlikely to care about while giving us detailed faces and other focal elements. I’m not totally sure they’re doing that though.
Re: (Score:3)
thats the ruling so far so if they use it in making a video then they lose the rights to said video
Actually, it isn't. If the AI makes the creative decisions then the video is not copyrightable, just like a video made by a monkey. However, if a human uses the AI as a tool, making the final creative decisions himself, then he can copyright his work.
As long as the AI is a tool, it is no different from a typewriter. Even though you used a typewriter to write your novel, rather than writing the words by hand, it is still your novel.
Another way to think about it is to imagine that the AI is like a random n
Re: (Score:2)
Like when that dude made a comic book and some editing of the ai produced images had to be done to format it to look how he wanted?
It all depends on how much editing you do. If you do enough, the work is yours.
old old old news (Score:2)
Consider this.
Most video encoding systems support both temporal and spatial scalability. This was a feature present in MPEG-2 and often goes thoroughly under used.
What this feature allows for example is the ability to encode video in a way that efficiently supports multiple resolutions, frame rates (and even color depths) into a single file. So you can encode low, medium, high and master quality into
Re: (Score:2)
This is very interesting since the whole 5G revolution seems to be centered on some silly idea that we'll need a lot of bandwidth to do holographic video
A lot of the focus is on low latency, rather than high throughput per se. It's just that one of the sources of latency is compression, so reducing latency means either toning down or disabling compression. That increases the traffic required, but it's much easier to increase the throughput of a network than to decrease its latency (especially since you're ultimately up against the speed of light, which is slow enough to be a problem by itself without adding more sources of latency on top).
Just remove the boring bits (Score:2)