FauxPilot: A Locally Hosted Alternative to GitHub Copilot
FauxPilot is an exciting new project that aims to create a locally hosted version of GitHub Copilot. It uses the powerful models provided by SalesForce CodeGen, NVIDIA's Triton Inference Server, and FasterTransformer backend. This project is an attempt to provide an alternative to the cloud-based GitHub Copilot and offer users more control over their data and privacy.
If you are interested in trying FauxPilot, there are a few prerequisites that you need to meet first. You will need Docker, docker-compose version 1.28 or higher, an NVIDIA GPU with Compute Capability of 6.0 or higher, and enough VRAM to run the model you want. Additionally, you will need to install nvidia-docker, curl, and zstd for downloading and unpacking the models.
It is important to note that the VRAM requirements listed by the setup script are for the total VRAM, and if you have multiple GPUs, you can split the model across them. For example, if you have two NVIDIA RTX 3080 GPUs, you should be able to run the 6B model by putting half on each GPU.
FauxPilot is an open-source project that does not provide any support or warranty, but there is some minimal information available on the project's wiki page and a discussion forum where you can ask questions.
To set up a FauxPilot server, run the setup script to choose a model to use. This will download the model from Huggingface/Moyix in GPT-J format and then convert it for use with FasterTransformer. The documentation on the project's website provides detailed instructions on how to set up a FauxPilot server.
Once you have set up the server, you can connect to it using various clients such as the Openai API, Copilot Plugin, or REST API. The documentation also provides instructions on how to set up a client.
FauxPilot is an exciting new project that provides an alternative to the cloud-based GitHub Copilot. It offers users more control over their data and privacy by providing a locally hosted version. The project is open-source, and the documentation provides detailed instructions on how to set up a server and connect to it using various clients. If you are interested in trying FauxPilot, head over to the project's website and get started today!
Top comments (0)