Large Language Models (LLMs) have turned out to be one of the biggest tech stories of the decade. There's nothing they can't do when it comes to text: generate and summarize them, write code, respond to us in our own language, and much more.
LLMs are, however, only one piece of the overall picture of artificial intelligence.
As we've all started bumping up against the limitations of using LLMs in isolation, a range of emerging technologies have started coming into play: AI agents, world models, multimodal AI, continual learning, edge AI, and hybrid AI architectures.
So… what do LLMs come next, anyway?
LLMs Might Actually Become Just Another Piece of the Jigsaw Puzzle of AI
Rather than thinking of something replacing LLMs wholesale, it’s much more useful to think of the next wave of AI being a combination of the building blocks:
An ultimate future AI system could integrate many of the following pieces together:
Large Language Models
Long-term memory
AI agents
Multimodal perception
Specialized models
Planning & reasoning
Edge computing
Real-time sensor data
The significant development therefore is less about creating a Bigger Language Model and more about the system architecture.
- AI Agents
LLMs are fundamentally reactive. They take input and produce output.AI agents have the potential to add a layer of proactive autonomy by planning out a set of actions to accomplish a task. For example, breaking down a complex query into a sequence of steps, using an external API, then acting on the result.LLM based prompt\n”input” →\n “output”May develop into →“AI Agent “input” →“task breakdown + external tool use + execution + feedback”The tricky part here becomes ensuring they behave in ways we expect and want, don’t make unexpected and problematic choices, and can handle tasks beyond simple question/answer sets without extensive manual supervision.
- LLM's
World Models LLMs operate based on relationships in data, but truly understanding and predicting how the physical world works is a different type of problem.
World models aim to encode information about an environment and how it evolves. This has obvious implications for things like robotics, autonomous systems, and the simulation/development of virtual spaces and physical entities.
Instead of predicting the next word/token, these models predict how states in an environment will change over time, such as “what happens if I move my arm in this direction while this robot arm is here?”## 3. Multimodal AI Our real world isn't just text.
The reality of perception involves combining information from text, images, video, sound, and sensors. Multimodal AI has set out to create systems that can ingest and process information from diverse data sources simultaneously. Imagine an AI system that can process all of the following:
``text
Text
Image
Video
Audio
Sensor data
`
at the same time and come up with a relevant response or action, particularly when operating beyond chat windows to interact with the physical world.## 4. Edge AI Another major trend is decentralization – pushing AI computation closer to where the data is generated.
Edge AI involves performing AI processing on the device itself rather than sending data back to a cloud for analysis. This offers advantages for applications where low latency is crucial, or in situations with limited internet connectivity. This also opens up the door for intelligent devices that operate more autonomously or work together as networked intelligence.
AIoT The convergence of Artificial Intelligence (AI) and the Internet of Things (IoT) has led to the rise of AIoT.
This trend takes the vast amounts of data generated by connected devices (sensors, machines, consumer gadgets) and leverages AI to make sense of that data, extracting insights that lead to actionable decisions. It is essentially about creating systems that bridge physical objects and intelligent computation:
`text
Sensors
->
Connected Devices
->
Edge Processing
->
AI/ML processing
->
Insights & Actions
`
This is especially important in industrial settings, smart homes, robotics, and the increasingly complex network of physical and digital services.## 6. Specialized and Hybrid Models Perhaps we’ll move away from a “one size fits all” approach.
Instead of a monolithic general-purpose model, the future might involve a mix of specialized models for particular domains, orchestrated by an overarching system. For example, a user might engage with:
`text
A vision model
A planning model
An LLM for user interaction
a specialized sentiment analysis model
``
all working together. The key would be an “orchestrator” layer to decide which component to use and how to integrate their outputs.
So… What Actually Comes After LLMs?
The question of “what comes after LLMs?”
is probably the wrong one. ### The better question to ask might be: “What comes after the standalone LLM?” LLMs will almost certainly remain powerful and useful, but they are more likely to become a component within more sophisticated and expansive AI systems. Agents provide action, multimodal models offer perception, memory provides continuity, world models offer environmental context, edge AI offers local intelligence, and specialized models can handle niche tasks.
Individually and combined, these could give us AI systems that are profoundly more intelligent, adaptable, and aware of the world around them.
The Future is ConvergenceThe next real big leap for AI likely won’t come from a single, transformative technology that supplants LLMs. Instead, it may be the product of convergence. AI agents, multimodal understanding, edge computing, AIoT, robotics, advanced memory capabilities, and domain-specific expertise might merge together into intelligent systems that can perceive their environment, make complex decisions, and take meaningful actions.
Anyone looking to understand the future of technology would be wise to track the interplay and integration across these domains. Aperture Venture Studio is actively engaged in research and innovation across AI, AIoT and other emerging technology trends. Visit our website to learn more: https://aperture.ventures/### One Final ThoughtLLMs have fundamentally shifted how we interface with AI.
The next wave might just as fundamentally shift what AI can interface with, and what AI can do.
The future is likely less about what comes after LLMs, and more about what we build around them.
Top comments (0)