Apple Researchers Unveil Keyframer, an AI Tool That Animates Still Images Using LLMs (venturebeat.com) 31
Apple researchers have unveiled a new AI tool called "Keyframer," (PDF) which harnesses the power of large language models (LLMs) to animate static images through natural language prompts. From a report: This novel application, detailed in a new research paper published on arxiv.org, represents a giant leap in the integration of artificial intelligence into the creative process -- and it may also hint at what's to come in newer generations of Apple products such as the iPad Pro and Vision Pro. The research paper, titled "Keyframer: Empowering Animation Design using Large Language Models," explores uncharted territory in the application of LLMs to the animation industry, presenting unique challenges such as how to effectively describe motion in natural language.
Imagine this: You're an animator with an idea that you want to explore. You've got static images and a story to tell, but the thought of countless hours bending over an iPad to breathe life into your creations is, well, exhausting. Enter Keyframer. With just a few sentences, those images can begin to dance across the screen, as if they've read your mind. Or rather, as if Apple's large language models (LLMs) have. Keyframer is powered by a large language model (in the study, they use GPT-4) that can generate CSS animation code from a static SVG image and prompt. "Large language models have the potential to impact a wide range of creative domains, but the application of LLMs to animation is under-explored and presents novel challenges such as how users might effectively describe motion in natural language," the researchers explain.
Imagine this: You're an animator with an idea that you want to explore. You've got static images and a story to tell, but the thought of countless hours bending over an iPad to breathe life into your creations is, well, exhausting. Enter Keyframer. With just a few sentences, those images can begin to dance across the screen, as if they've read your mind. Or rather, as if Apple's large language models (LLMs) have. Keyframer is powered by a large language model (in the study, they use GPT-4) that can generate CSS animation code from a static SVG image and prompt. "Large language models have the potential to impact a wide range of creative domains, but the application of LLMs to animation is under-explored and presents novel challenges such as how users might effectively describe motion in natural language," the researchers explain.
reading is the key (Score:2)
i read it as keyfarmer...
Re: (Score:2)
Please leave the poor old man with diminished mental faculties alone.
Maybe some day... (Score:1)
AI researchers will invent something useful, like tools for finding tricky bugs in software. Even more useful would be a simulation that accurately predicted the real world results of proposed laws
Instead, they work on tools that allow people to make more crap that is useless at best, and potentially very harmful if used as a weapon
Re: (Score:3)
AI researchers will invent something useful, like tools for finding tricky bugs in software. Even more useful would be a simulation that accurately predicted the real world results of proposed laws
Instead, they work on tools that allow people to make more crap that is useless at best, and potentially very harmful if used as a weapon
Some of the trickiest bugs in software are not really bugs, but implementation that differs from the intended purpose. To find these, the ML would need to understand what the software should do, which often the people writing the code don't fully understand.
Re: (Score:2)
AI researchers will invent something useful, like tools for finding tricky bugs in software. Even more useful would be a simulation that accurately predicted the real world results of proposed laws
Instead, they work on tools that allow people to make more crap that is useless at best, and potentially very harmful if used as a weapon
Some of the trickiest bugs in software are not really bugs, but implementation that differs from the intended purpose. To find these, the ML would need to understand what the software should do, which often the people writing the code don't fully understand.
Have the LLMs "listen" to the spec meetings where marketing says one thing, accounting says another thing, customer service says another thing, the dev team lower expectations somewhere closer to reality, and then sales comes in to sweep the leg and send everybody back into daydream central. If an LLM can sort that mess out, then maybe the LLM can tell the developers WTF they're supposed to be building.
Re: (Score:2)
AI researchers will invent something useful, like tools for finding tricky bugs in software. Even more useful would be a simulation that accurately predicted the real world results of proposed laws
Instead, they work on tools that allow people to make more crap that is useless at best, and potentially very harmful if used as a weapon
Some of the trickiest bugs in software are not really bugs, but implementation that differs from the intended purpose. To find these, the ML would need to understand what the software should do, which often the people writing the code don't fully understand.
Have the LLMs "listen" to the spec meetings where marketing says one thing, accounting says another thing, customer service says another thing, the dev team lower expectations somewhere closer to reality, and then sales comes in to sweep the leg and send everybody back into daydream central. If an LLM can sort that mess out, then maybe the LLM can tell the developers WTF they're supposed to be building.
Or at least provide a nice concise summary of how what everyone wants all conflicts with what everyone else wants. Though I'm sure even with that, some members of the meetings would still think doing everything at once is possible.
Re: (Score:2)
Are you writing bad science fiction? LLMs can't do any of these things.
Re: (Score:2)
Are you writing bad science fiction? LLMs can't do any of these things.
Isn't pretty much all news on what LLMs will be doing tomorrow pretty much science fiction?
Re: (Score:2)
There's a lot of nonsense about LLMs floating around these days. It wouldn't matter all that much, but those mistaken beliefs lead people to use them in inappropriate ways that have already caused real harm. In an incredibly boring twist, the danger from LLMs wasn't what they could do, but what people believed they could do.
Re: (Score:1)
For that you need domain knowledge, which an LLM does not have.
You could combine it with an expert system, but you then had already most of the software.
They're doing just that (Score:2)
Re: (Score:3)
That's extremely unlikely. These things are far, far, less capable than you think.
AI killed the video star (Score:2)
still image animation using LLM's? (Score:3)
Re: (Score:3)
This is a pretty good problem for an LLM. The description makes it sound more exciting than it is. They've taken an XML description of a bunch of image elements and used an LLM to generate CSS code to animate those elements. LLMs are good at generating simple but tedious code from a description of the desired outcome.
It's not going to make the T1000 reconstitute itself from a puddle of silver goo, but it can make the cartoon moon rise on your animated birthday card web page.
The paper is really a design stud
Re: (Score:2)
Guess when your only tool is a hammer every problem looks like a nail.
I think when progress is being made at the phenomenal rate it is right now, then it's exactly the right course of action to see just how versatile your hammer is.
Sounds similar to Google Lumiere (Score:2)
Re: (Score:1)
Faster shitty powerpoint animations, meh (Score:2)
This is not some first step towards automating 2D character animation, it's a toy with the only claim to fame being from Apple.
"precursor to a new era" (Score:2)
"Keyframer could well be a precursor to a new era where the boundaries between creator and creation become increasingly fluid, guided by the invisible hand of artificial intelligence."
That creeped me out.