AI Runner is a program that I wrote which allows you to run LLMs and Stable Diffusion art models locally, offline on your own hardware. I have optimized this for lower-end graphics cards (tested on 2080s RTX and 3060 RTX). It has been in development for approximately two years, is free, pure python and released under GPL3.
The latest version, 3.0, has been under development for many months. It is a complete refactor of the application and is nearly ready and will be the first non-alpha release.
AI Runner was created with Pyside6 and Huggingface libraries. In order to provide greater security and privacy I released several libraries which sandbox disc, network and logging operations, each of which has no third-party dependencies.
Previous versions were composed using pyinstaller and released at itch.io but moving forward I will be aiming for Flatpak or Snap on Linux (I have dropped Windows support for now).
If you like my work, you can support me on Patreon - I do this for free and any support is greatly appreciated.
Top comments (0)