DEV Community

Wincent Balin
Wincent Balin

Posted on

Closure

#ai

After a pause, this series comes to a conclusion, mostly because of the rapid developments in the area of large language models.

Original intention

At the beginning I intended to create a language model, that would have gotten a prompt "Geschirrabwaschgesetz" (a law about washing dishes) and write me a corresponding law text in German.

I was discouraged from training the original char RNN because of the scary amount of training time with a 110 M training data. Therefore I went with fine-tuning a German GPT-2 (and later the better one; thanks Jo!). The fine-tuning process of such a model is described here or here, for example.

(Un-)expected discovery

I happened to discover that my intended case is covered perfectly by the LLAMA 2 Chat German model (almost, because of a few grammatical errors). This is very likely because of being fine-tuned with the German legal SQuAD dataset, among others.

I do not want to withhold the result from you (produced in LM Studio): Output to "Geschirrabwaschgesetz"

Just look at this beauty! It even defined "Hygiene" in the last subparagraph! And hence this series is concluded.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up