DEV Community

César Freitas
César Freitas

Posted on

Compress text to fit an LLM token budget with an API

#api #json #llm #webdev

Compress text to fit an LLM token budget with an API

Large Language Models (LLMs) have revolutionized the way we interact with machines, but they come with a significant limitation: the amount of tokens available for generating text. When working with these models, it's crucial to be mindful of this token budget to avoid errors or unexpected results.

The Problem

When you pass a prompt to an LLM, the model processes the input and generates output based on its training data. However, if your prompt exceeds the allowed number of tokens, the model may not generate any output or produce incorrect results. This can be frustrating, especially when working with text-based models that require careful attention to detail.

The Solution: LLM Dev Utilities API

The LLM Dev Utilities API is a solution designed specifically for developers and indie makers who work with these models. The API provides several features that help you manage your token budget effectively:

  • Repair broken model JSON: If the model's JSON configuration is corrupted or incomplete, the API can repair it automatically.
  • Clean SRT/VTT subtitles: Subtitles are an essential aspect of text-based LLMs. The API ensures these subtitles are clean and error-free.
  • Compress text to token budget: This feature allows you to compress your input text to fit within a specific token budget, ensuring the model receives the correct amount of tokens.

What You Can Expect

When using the LLM Dev Utilities API, you can expect a simple request-response format. Here's what this might look like in plain words:

  • Request: You send a request to the API with your input text and the desired token budget.
  • Response: The API responds with the compressed text or an error message if the input is invalid.

By using this API, you can ensure that your LLM receives the correct amount of tokens for generating text. This feature alone can save you a significant amount of time and frustration when working with these models.

Try It Out

The LLM Dev Utilities API is available on RapidAPI, a platform that offers free tiers for developers. With RapidAPI's free tier, you can try out the API without incurring any costs. Simply create an account, choose the LLM Dev Utilities API, and start experimenting with its features.

Conclusion

Compressing text to fit an LLM token budget is crucial when working with these models. The LLM Dev Utilities API provides a simple solution for this problem, offering features like repairing broken model JSON and cleaning SRT/VTT subtitles. By using this API, you can ensure that your LLM receives the correct amount of tokens for generating text.

If you're looking for an efficient way to manage your token budget, consider trying out the LLM Dev Utilities API on RapidAPI's free tier. With its simple request-response format and free trial available, there's no reason not to give it a shot.

Quick example 

curl -X POST 'https://llm-dev-utilities.p.rapidapi.com/json-repair' \
  -H 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  -H 'X-RapidAPI-Host: llm-dev-utilities.p.rapidapi.com' \
  -H 'Content-Type: application/json' \
  -d '{"text":"{name:\"Ann\", age:30,}"}'
Enter fullscreen mode Exit fullscreen mode

Try it: the LLM Dev Utilities API is live on RapidAPI with a free tier — get your key here.

Top comments (0)