The Evolution of Large Language Models

The development of large language models (LLMs) has been a significant milestone in the field of artificial intelligence, with transformers being the basis for every major LLM on the market.

According to Justin Dangel, cofounder and CEO of Subquadratic, "The entire AI industry is built on transformers... They are one of the most important innovations in the history of computer science, and they’ve changed the world."

Introduction to Transformers

Transformers were introduced in a 2017 Google research paper, and since then, they have been the foundation of the AI industry, with companies like Anthropic, OpenAI, and Cursor working on LLMs.

The current state of AI is far from an end state for its capabilities, and the frontiers of artificial intelligence are advancing rapidly, with generative AI being a key pillar of the next generation of artificial intelligence.

In 2019, the release of GPT-2 marked a significant milestone in the development of LLMs, followed by the prediction of generative AI as a pillar of next-generation AI in 2020, and the release of the original DALL-E in 2021.

Advances and Limitations

Recent advances in LLMs are workarounds for fundamental flaws in transformer technology, and a new wave of startups is emerging to push the boundaries of LLM technology.

Companies like Google, Microsoft, and Meta are also working on LLMs, and it is expected that they may tighten their grip on the industry, with some startups potentially crashing and burning in the process.

As Bret Little noted, "it feels like these established players are just using the smaller startups to burn through billions, push the tech forward, and then take all the rewards for themselves."

The Next Generation of LLMs

The next generation of large language models is under development at AI startups, with companies like Krea, OpenHome, and Featherless AI working on innovative solutions.

The HF0 Residency program has also been supporting AI startups, with 7 startups pitching their ideas at the Demo Days, which were aired publicly for the first time, and had 608000 subscribers, 188 likes, and 9011 views.

Diego Rodriguez mentioned that his team of 8 people is working on a project that has 25 million users, highlighting the potential of AI startups to reach a large audience.

Future Developments

The development of LLMs is an ongoing process, with new advances and innovations emerging every year, and 2026 is expected to be a significant year for the industry.

As the AI industry continues to evolve, it is likely that we will see new breakthroughs and innovations in the field of LLMs, with both established players and startups working on pushing the boundaries of what is possible.

Why This Matters