When building AI and LLM-based applications, one of the biggest hidden costs often comes from something simple β the format of your data.
Every {}...
For further actions, you may consider blocking this person and/or reporting abuse
Interesting take. Iβve built a lightweight JSON β TOON Converter for quick benchmarking between both formats.
Sharing here in case others want to experiment:
scalevise.com/json-toon-converter
page not found
scalevise.com/json-toon-converter
Fixed it!
I've created this converter to quickly see the savings, very simple to use:
toon.click/
I've just added support for TOON encoding to my converter utility. mmalcek.github.io/bafi/ as easy as
./bafi -i input.json -t "?{{ toTOON . }}":)Godo article, would appriciate if you can include more quantifying numbers of token optimized for various file format like programming code(.py, .cpp files), CSV and flat text files
This is available in their Github readme, with different token usage of each format for different LLM models.
Created a Dart package for JSON-to-toon conversions. Please give it a look.
Package: pub.dev/packages/toonx
Github: github.com/TheNoumanDev/toonX
Since it's less efficient for nested data, and is only useful for tabular data, I fail to see how this format is better than CSV or TSV.
This is great, thanks a lot!
That's just awesome :)