Optimizing NanoGPT: Achieving a New Training Speed Record

Kami Butt — Thu, 15 Oct 2026 19:06:34 +0000

NanoGPT has set a new benchmark in language model training speed, completing a full training section in just 39.914 seconds. This remarkable achievement, which represents a 46.0% reduction in training time compared to the previous record of 73.889 seconds, showcases significant advancements in training methodologies and optimizations.

Overview of the Record

The new record was established by Deven Pietrzak, who documented the achievement in pull request 360. This training session was validated by four independent sources, each of which reproduced the results on their hardware. Pietrzak's method involved extensive testing, with 18 runs all falling under a loss bar probability of less than 1e-9.

Key Optimizations

The speedup in NanoGPT's training can be attributed to several architectural and algorithmic optimizations:

Sampled-softmax Training Loss: This optimization saved over 8 seconds by reducing the number of vocabulary elements scored during training.
Hashed N-gram Table Embeddings: This change provided more capacity for local token patterns, contributing to a time savings of 7.05 seconds.
ANVIL Optimizer: The ANVIL optimizer, which maintains two streams of momentum, accounted for a 5.08-second reduction.
Full-stack FP8 Quantization: This technique contributed to a 4.96-second improvement.
Mixed-width Attention: This optimization saved 3.82 seconds.

Technical Architecture

The record-setting run utilized a larger sparse n-gram table, allowing for more efficient training. Each GPU managed a portion of the bigram and trigram table weights, significantly reducing memory usage. The implementation also featured sampled softmax to limit the number of tokens scored during most of the training process.

Validation and Reproducibility

The final validation cross-entropy achieved was 3.27731, with a standard deviation of 0.00099. The methods and implementation details are publicly accessible, allowing others to reproduce and build upon these results.

For more detailed insights into the optimizations and methodologies behind this record, visit Agentic Affairs.

Managing State in Autonomous AI Agents for Long-Running Tasks

Kami Butt — Thu, 15 Oct 2026 15:16:39 +0000

Managing State in Autonomous AI Agents for Long-Running Tasks

Building autonomous AI agents that operate across multiple steps requires moving beyond simple request-response interactions. When tasks run for extended periods, agents need mechanisms to retain context, track progress, and handle failures without losing work.

The State Problem in Autonomous Agents

A useful architecture separates what the model needs for an immediate interaction from what the application must retain across the entire task. Conversation history alone is insufficient. Robust designs must also account for task progress, tool states, and intermediate files.

Based on layered state models, engineers can evaluate designs across four distinct dimensions:

Conversation State: Which user interactions must remain available as the task continues?
Task State: What information represents completed work versus pending steps?
Tool State: How does the agent track tool outputs and external operations?
File State: Which intermediate artifacts must remain available for subsequent work?

Short-Term vs. Long-Term State

The span of continuity dictates storage choices. Short-term state handles immediate context-window requirements, while long-term state persists across interactions. When handling context-window constraints, separating retained task information from the immediate interaction context is critical.

Handling Failures and Recovery

Persistence across interactions does not automatically guarantee recovery after a crash. Key questions to address in production architectures include:

How does the workflow restore task state and intermediate artifacts after a mid-task crash?
How does the agent verify external operation status after an API timeout?
What idempotency mechanisms prevent unintended duplicate effects upon retry?

Furthermore, production systems must account for hidden failures—where cascading errors result in apparently successful outputs that are actually semantically incorrect.

Production Observability

Traditional monitoring is insufficient for stateful agents. AI-native observability must extend to semantic and quality insights, allowing engineering teams to inspect decision-making logic, verify output quality, and ensure compliance.

For a deeper dive into architectural patterns and considerations, read the full analysis on Agentic Affairs.

Conclusion

Persistent context, structured workspaces, and layered state provide the building blocks for long-running AI tasks. However, ensuring true system resilience requires explicit design decisions regarding restoration, external status reconciliation, and error handling.

DEV Community: Kami Butt

Optimizing NanoGPT: Achieving a New Training Speed Record

Overview of the Record

Key Optimizations

Technical Architecture

Validation and Reproducibility

Managing State in Autonomous AI Agents for Long-Running Tasks

Managing State in Autonomous AI Agents for Long-Running Tasks

The State Problem in Autonomous Agents

Short-Term vs. Long-Term State

Handling Failures and Recovery

Production Observability

Conclusion