
Cringely Predicts Moore's Law Will Continue -- Because of AI (cringely.com) 35
"I predict that Generative Artificial Intelligence is going to go a long way toward keeping Moore's Law in force," writes long-time tech pundit Robert X. Cringely, "and the way this is going to happen says a lot about the chip business, global economics, and Artificial Intelligence, itself."
The current el cheapo AI research frenzy is likely to subside as LLaMA ages into obsolescence and has to be replaced by something more expensive, putting Google, Microsoft and OpenAI back in control. Understand, too, that these big, established companies like the idea of LLMs costing so much to build because that makes it harder for startups to disrupt. It's a form of restraint of trade, though not illegal...
[T]here is an opportunity for vertical LLMs trained on different data — real data from industries like medicine and auto mechanics. Whoever owns this data will own these markets. What will make these models both better and cheaper is they can be built from a LLaMA base because most of that data doesn't have to change over time... Bloomberg has already done this for investment advice using its unique database of historical financial information. With an average of 50 billion nodes, these vertical models will cost only five percent as much to run as OpenAI's one billion node GPT-4...
[I]t ought to be pretty simple to apply AI to chip design, building custom chip design models to iterate into existing simulators and refine new designs that actually have a pretty good chance of being novel.
And who will be the first to leverage this chip AI? China... Look for fabless AI chip startups to spring-up around Chinese universities and for the Chinese Communist Party to put lots of money into this very cost-effective work. Because even if it's used just to slim-down and improve existing designs, that's another generation of chips China might otherwise not have had at all.
[T]here is an opportunity for vertical LLMs trained on different data — real data from industries like medicine and auto mechanics. Whoever owns this data will own these markets. What will make these models both better and cheaper is they can be built from a LLaMA base because most of that data doesn't have to change over time... Bloomberg has already done this for investment advice using its unique database of historical financial information. With an average of 50 billion nodes, these vertical models will cost only five percent as much to run as OpenAI's one billion node GPT-4...
[I]t ought to be pretty simple to apply AI to chip design, building custom chip design models to iterate into existing simulators and refine new designs that actually have a pretty good chance of being novel.
And who will be the first to leverage this chip AI? China... Look for fabless AI chip startups to spring-up around Chinese universities and for the Chinese Communist Party to put lots of money into this very cost-effective work. Because even if it's used just to slim-down and improve existing designs, that's another generation of chips China might otherwise not have had at all.