DEV Community: miinhho

Does the Model Really Need to Stream JSON?

miinhho — Mon, 06 Jul 2026 13:42:54 +0000

Getting JSON out of LLMs has become fairly routine. You pass a JSON Schema, enable structured output, and if the model is capable enough, the final JSON is usually valid.

But what happens when you enable streaming?

A valid JSON document at the end of generation is not the same as having something your application can safely consume while it's still being generated.

A typical streamed JSON response looks something like this:

{"summary":"Export job failed...

Eventually this becomes a valid JSON object, but at this moment it isn't JSON at all.

In practice, this makes streaming JSON awkward to work with in real applications.

A parse failure isn't really an error—it just means the response hasn't finished yet.
It's difficult to tell which fields are actually complete.
Even if the final output is valid, the intermediate state is broken almost the entire time.

This is simply a property of JSON as a document format. Until the document is complete, there is no stable intermediate object.

Does the model actually need to stream JSON?

Not necessarily.

What most applications want isn't for the model to gradually write out a JSON string. What they really need is structured state that remains usable throughout generation.

Instead of asking the model to stream a JSON document directly, I experimented with having it emit compact slot frames. Each slot ID maps to a JSON path, and incoming values are applied to a server-owned object state.

In other words, the model generates values, while the SDK assembles the JSON object.

For example, even if the model is still generating the summary field, the application can already receive:

{"summary":"Export job f"}

This isn't the final result, but it is a valid, serializable JSON object. It can be rendered immediately in the UI, sent over SSE, or passed to downstream services without special handling.

If a slot fails, only that slot needs to be regenerated. Fields that have already completed don't need to be generated again.

This is the approach behind slot-flight.

Benchmark

I compared two approaches using NVIDIA's openai/gpt-oss-20b endpoint.

slot-flight: the model streams slot frames, and slot-flight assembles the JSON state.
json-schema: uses NVIDIA's native response_format: { type: "json_schema" } streaming.

The results:

slot-flight
final schema ok: 40/40 = 100%
stream JSON stable: 100%
usable run: 40/40 = 100%

json-schema
final schema ok: 39/40 = 97.5%
stream JSON stable: ~2.0%
usable run: 39/40 = 97.5%

The native JSON Schema implementation produced reliable final results: 39 out of 40 responses passed schema validation.

However, only about 2% of streamed chunks represented a valid JSON state. For most of the generation, the client was looking at an incomplete JSON document.

With slot-flight, every intermediate state remained serializable as JSON, including slot-delta events.

A typical event looks like this:

{"type":"slot-delta","slot":"summary","value":"Export job f","state":{"summary":"Export job f"}}

Even though value is still being generated, state is already a complete object that the application can safely consume.

Conclusion

Users often want to see a summary before the entire response is finished. Servers may want to start downstream work as soon as individual fields become available. UIs naturally benefit from objects that fill in incrementally instead of appearing all at once.

In those situations, continuously trying to parse an incomplete JSON prefix isn't a great interface.

I built slot-flight not to answer the question, "Can an LLM produce valid JSON?"

The more interesting question is:

Can an application safely consume structured state before the final JSON document exists?

That's the problem slot-flight is designed to solve.

It also supports stable structured streaming for nested objects and arrays of objects.

https://github.com/miinhho/slot-flight

Bring Spring-Style AOP to NestJS with nestjs-saop

miinhho — Mon, 08 Sep 2025 15:51:46 +0000

Implement powerful Aspect-Oriented Programming in your NestJS applications with complete TypeScript support and familiar Spring-like syntax.

If you've worked with Spring Framework, you've probably enjoyed its powerful AOP (Aspect-Oriented Programming) capabilities. AOP helps you cleanly separate cross-cutting concerns like logging, caching, security, and transaction management from your business logic, making your code more maintainable and readable.

What if you could bring that same elegant AOP experience to your NestJS applications? Enter nestjs-saop – a library that brings Spring-style AOP to the NestJS ecosystem with full TypeScript support.

What is nestjs-saop?

nestjs-saop is a comprehensive AOP library for NestJS that provides:

✅ Complete AOP Advice Types: All 5 Spring-style advice types (Around, Before, After, AfterReturning, AfterThrowing)

✅ Full TypeScript Support: Type safety with generics and interfaces

✅ Seamless NestJS Integration: Works perfectly with the NestJS module system

✅ Decorator Pattern: Clean, familiar decorator-based API

✅ Flexible Configuration: Highly configurable with custom options support

Quick Setup

Install the package:

npm install nestjs-saop

import { AOPModule } from 'nestjs-saop';

@Module({
  imports: [
    AOPModule.forRoot(),
  ],
})
export class AppModule {}

Creating Your First AOP Decorator

Let's create a logging decorator that demonstrates all AOP advice types:

import { AOPDecorator, Aspect } from 'nestjs-saop';

@Aspect()
export class LoggingDecorator extends AOPDecorator {
  around({ method, options }) {
    return (...args: any[]) => {
      console.log('🔄 Around: Before method execution');
      const result = method.apply(this, args);
      console.log('🔄 Around: After method execution');
      return result;
    };
  }

  before({ method, options }) {
    return (...args: any[]) => {
      console.log('▶️ Before: Method called with', args);
    };
  }

  after({ method, options }) {
    return (...args: any[]) => {
      console.log('⏹️ After: Method completed');
    };
  }

  afterReturning({ method, options, result }) {
    return (...args: any[]) => {
      console.log('✅ AfterReturning: Method returned', result);
    };
  }

  afterThrowing({ method, options, error }) {
    return (...args: any[]) => {
      console.error('❌ AfterThrowing: Method threw', error.message);
    };
  }
}

@Module({
  providers: [LoggingDecorator],
})
export class AppModule {}

Real-World Examples

1. Smart Caching Decorator

@Aspect()
export class CachingDecorator extends AOPDecorator {
  private cache = new Map();

  around({ method, options }) {
    return (...args: any[]) => {
      const key = `${method.name}:${JSON.stringify(args)}`;

      // Cache hit
      if (this.cache.has(key)) {
        console.log('💡 Cache hit!');
        return this.cache.get(key);
      }

      // Cache miss - execute method
      console.log('💭 Cache miss, executing method...');
      const result = method.apply(this, args);

      // Store in cache with TTL
      this.cache.set(key, result);
      if (options.ttl) {
        setTimeout(() => this.cache.delete(key), options.ttl);
      }

      return result;
    };
  }
}

// Usage
@Injectable()
export class UserService {
  @CachingDecorator.around({ ttl: 300000 }) // 5 minutes
  async getUserById(id: string): Promise<User> {
    return await this.userRepository.findById(id);
  }
}

2. Performance Monitoring

@Aspect()
export class PerformanceDecorator extends AOPDecorator {
  around({ method, options }) {
    return (...args: any[]) => {
      const start = Date.now();

      try {
        const result = method.apply(this, args);
        const duration = Date.now() - start;

        if (duration > options.threshold) {
          console.warn(`⚠️ Slow method detected: ${method.name} (${duration}ms)`);
        }

        return result;
      } catch (error) {
        const duration = Date.now() - start;
        console.error(`💥 Method failed: ${method.name} (${duration}ms)`);
        throw error;
      }
    };
  }
}

// Usage
@Injectable()
export class DataService {
  @PerformanceDecorator.around({ threshold: 1000 })
  async processLargeDataset(data: any[]): Promise<ProcessedData> {
    // Your expensive operation here
    return await this.complexProcessing(data);
  }
}

3. Error Handling with Retry Logic

@Aspect()
export class RetryDecorator extends AOPDecorator {
  afterThrowing({ method, options, error }) {
    return (...args: any[]) => {
      console.error(`🔄 Method ${method.name} failed:`, error.message);

      if (options.retryCount < options.maxRetries) {
        console.log(`🔄 Retrying... (${options.retryCount + 1}/${options.maxRetries})`);

        // Implement exponential backoff
        const delay = Math.pow(2, options.retryCount) * 1000;
        setTimeout(() => {
          // Retry logic here
          method.apply(this, args);
        }, delay);
      }
    };
  }
}

// Usage
@Injectable()
export class ApiService {
  @RetryDecorator.afterThrowing({ maxRetries: 3, retryCount: 0 })
  async callExternalAPI(endpoint: string): Promise<any> {
    const response = await fetch(endpoint);
    if (!response.ok) {
      throw new Error(`API call failed: ${response.status}`);
    }
    return response.json();
  }
}

TypeScript Power 💪

One of the strongest features of nestjs-saop is its complete TypeScript support. Define custom option types for better developer experience:

interface LoggingOptions {
  level: 'debug' | 'info' | 'warn' | 'error';
  includeTimestamp: boolean;
  logArgs: boolean;
}

@Aspect()
export class TypedLoggingDecorator extends AOPDecorator {
  before({ method, options }: UnitAOPContext<LoggingOptions>) {
    return (...args: any[]) => {
      const timestamp = options.includeTimestamp 
        ? `[${new Date().toISOString()}] ` 
        : '';

      const argsLog = options.logArgs 
        ? ` with args: ${JSON.stringify(args)}` 
        : '';

      console.log(
        `${timestamp}${options.level.toUpperCase()}: ${method.name}${argsLog}`
      );
    };
  }
}

// Usage with full type safety
@Injectable()
export class UserService {
  @TypedLoggingDecorator.before({
    level: 'info',           // ✅ Type-safe
    includeTimestamp: true,  // ✅ Type-safe  
    logArgs: false          // ✅ Type-safe
  })
  async createUser(userData: CreateUserDto): Promise<User> {
    return await this.userRepository.create(userData);
  }
}

Combining Multiple Decorators

You can stack multiple decorators on a single method for powerful combinations:

@Injectable()
export class PaymentService {
  @LoggingDecorator.before({ level: 'info', logArgs: true })
  @PerformanceDecorator.around({ threshold: 2000 })
  @CachingDecorator.around({ ttl: 60000 })
  @RetryDecorator.afterThrowing({ maxRetries: 3 })
  async processPayment(paymentData: PaymentDto): Promise<PaymentResult> {
    // This method gets enhanced with:
    // 1. Logging before execution
    // 2. Performance monitoring 
    // 3. Caching capabilities
    // 4. Automatic retry on failures

    return await this.paymentGateway.process(paymentData);
  }
}

AOP Execution Order

The advice types execute in this specific order:

🔄 Around (before method)
▶️ Before
Method Execution
⏹️ After
✅ AfterReturning OR ❌ AfterThrowing
🔄 Around (after method)

Conclusion

nestjs-saop brings the power and elegance of Spring-style AOP to the NestJS ecosystem. Whether you need logging, caching, performance monitoring, or error handling, this library helps you implement cross-cutting concerns cleanly and efficiently.

The combination of familiar Spring-like syntax, complete TypeScript support, and seamless NestJS integration makes it a perfect choice for building maintainable enterprise applications.

Give it a try in your next NestJS project – your code will thank you! 🚀

📚 Resources:

What's your experience with AOP in backend development? Have you tried implementing cross-cutting concerns in NestJS before? Share your thoughts in the comments below! 💬