Approaching 2000 GitHub stars, Docker GenAI Stack repository is gaining widespread traction, especially within the data science community. This stack is a collection of pre-configured Docker services designed to streamline the incorporation of large language models (LLMs) and AI workflows into your development setup. It offers a user-friendly solution for handling and executing AI applications, eliminating the necessity for in-depth knowledge of LLMs or containerization.
In October 2023, at the Dockercon event, Docker unveiled the groundbreaking GenAI Stack for the very first time. This collaborative effort brought together the expertise of Docker, Neo4j, LangChain, and Ollama. The primary objective behind this partnership was to forge a pre-configured GenAI stack comprising top-tier technologies. This stack not only boasts seamless integration but also includes sample applications, simplifying the onboarding process for developers and enabling them to dive into their projects effortlessly.
What is GenAI Stack composed of?
This stack comprises a set of Docker containers designed to simplify the process of experimenting with the development and execution of Generative AI (GenAI) applications. Within these containers, a pre-built support agent application is available, catering to data import and response generation use-cases. The key components of the stack include:
1. Ollama
A management tool facilitating the handling of local Large Language Models (LLMs).
2. Neo4j
A database crucial for grounding.
3. GenAI apps based on LangChain
Applications leveraging LangChain as the foundation for Generative AI functionalities.
4. Pre-configured LLMs
Jumpstart your AI projects with preconfigured Large Language Models such as Llama2, GPT-3.5, and GPT-4, embedded within the stack to enhance your development experience.
Why Ollama, Neo4j and LangChain?
The collaboration with LangChain and Ollama stemmed from their specialized knowledge in Large Language Models (LLMs). LangChain serves as a programming and orchestration framework tailored for LLMs, while Ollama excels as a tool specifically designed for the local management and operation of LLMs.
Neo4j's involvement in the collaboration was driven by its proficiency in graph databases and knowledge graphs. Recognizing the potent synergy between graphs and LLMs, Neo4j identified the capability to construct highly accurate and reliable Generative AI (GenAI) applications through this amalgamation. The combined expertise of Ollama, Neo4j, and LangChain thus forms a robust foundation for the GenAI Stack, ensuring a comprehensive and effective solution for developers.
Refer this link if you're interested to setup Docker GenAI Stack on Mac or Linux system
Getting Started
To kick off your journey with the GenAI Stack, follow these simple steps:
Install Docker Desktop for Windows
Ensure you have Docker Desktop installed on your Windows machine. You can download it from the official Docker website.
$ docker version
Client:
Cloud integration: v1.0.35+desktop.5
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:08:44 2023
OS/Arch: windows/amd64
Context: default
Server: Docker Desktop 4.26.1 (131620)
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:08:02 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.25
GitCommit: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Install Git
Git is a fundamental version control tool. If you don't have it installed, grab it from the official Git website to empower collaborative development.
Install WinGet
WinGet is a package manager for Windows that streamlines the installation of software packages. If you haven't installed WinGet yet, do so to enhance your system management capabilities.
With these essential tools in place, you're ready to dive into the world of Generative AI using the Docker GenAI Stack. Let the exploration begin!
Configuration changes in Docker Desktop Settings
To optimize your Docker Desktop experience, follow these configuration changes:
- Navigate to Docker Desktop and select Settings.
- Under the General tab, locate the option: *"Add the .docker.internal names to the host’s /etc/hosts file."
- Enable this option:
- Enabling this setting allows the resolution of *.docker.internal DNS names from both the host machine and your containers.
By incorporating this configuration change, you enhance the communication capabilities between your host and containers, fostering a smoother and more integrated Docker environment. Save the changes and elevate your Docker experience!
Clone the repository
git clone https://github.com/docker/genai-stack
cd genai-stack
Bringing up the Stack
docker compose up -d --build
Results:
Container genai-stack-database-1 Creating
Container genai-stack-pull-model-1 Creating
Container genai-stack-pull-model-1 Created
Container genai-stack-database-1 Created
Container genai-stack-pdf_bot-1 Creating
Container genai-stack-loader-1 Creating
Container genai-stack-bot-1 Creating
Container genai-stack-api-1 Creating
Container genai-stack-pdf_bot-1 Created
Container genai-stack-loader-1 Created
Container genai-stack-api-1 Created
Container genai-stack-front-end-1 Creating
Container genai-stack-bot-1 Created
Container genai-stack-front-end-1 Created
Container genai-stack-pull-model-1 Starting
Container genai-stack-database-1 Starting
Container genai-stack-pull-model-1 Started
Container genai-stack-database-1 Started
Container genai-stack-database-1 Waiting
Container genai-stack-pull-model-1 Waiting
Container genai-stack-database-1 Waiting
Container genai-stack-pull-model-1 Waiting
Container genai-stack-database-1 Waiting
Container genai-stack-pull-model-1 Waiting
Container genai-stack-database-1 Waiting
Container genai-stack-pull-model-1 Waiting
Container genai-stack-pull-model-1 service "pull-model" didn't complete successfully: exit 1
Container genai-stack-pull-model-1 service "pull-model" didn't complete successfully: exit 1
Container genai-stack-pull-model-1 service "pull-model" didn't complete successfully: exit 1
Container genai-stack-pull-model-1 service "pull-model" didn't complete successfully: exit 1
Container genai-stack-database-1 Error
Container genai-stack-database-1 Error
Container genai-stack-database-1 Error
Container genai-stack-database-1 Error
service "pull-model" didn't complete successfully: exit 1
The error message was expected.
Reason - Ollama support for Windows is not yet available.
So, we might need to make changes and replace ollama with GPT-3.5 or GPT-4.0
Here’s the new Docker Compose file:
services:
pull-model:
image: genai-stack/pull-model:latest
build:
context: .
dockerfile: pull_model.Dockerfile
environment:
- LLM=${LLM-gpt-3.5}
networks:
- net
tty: true
database:
image: neo4j:5.11
ports:
- 7687:7687
- 7474:7474
volumes:
- $PWD/data:/data
environment:
- NEO4J_AUTH=${NEO4J_USERNAME-neo4j}/${NEO4J_PASSWORD-password}
- NEO4J_PLUGINS=["apoc"]
- NEO4J_db_tx__log_rotation_retention__policy=false
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"]
interval: 5s
timeout: 3s
retries: 5
networks:
- net
loader:
build:
context: .
dockerfile: loader.Dockerfile
volumes:
- $PWD/embedding_model:/embedding_model
environment:
- NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
- NEO4J_PASSWORD=${NEO4J_PASSWORD-password}
- NEO4J_USERNAME=${NEO4J_USERNAME-neo4j}
- OPENAI_API_KEY=${OPENAI_API_KEY-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL-sentence_transformer}
- LANGCHAIN_ENDPOINT=${LANGCHAIN_ENDPOINT-"https://api.smith.langchain.com"}
- LANGCHAIN_TRACING_V2=${LANGCHAIN_TRACING_V2-false}
- LANGCHAIN_PROJECT=${LANGCHAIN_PROJECT}
- LANGCHAIN_API_KEY=${LANGCHAIN_API_KEY}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
networks:
- net
depends_on:
database:
condition: service_healthy
pull-model:
condition: service_completed_successfully
x-develop:
watch:
- action: rebuild
path: .
ignore:
- bot.py
- pdf_bot.py
- api.py
- front-end/
ports:
- 8081:8080
- 8502:8502
bot:
build:
context: .
dockerfile: bot.Dockerfile
volumes:
- $PWD/embedding_model:/embedding_model
environment:
- NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
- NEO4J_PASSWORD=${NEO4J_PASSWORD-password}
- NEO4J_USERNAME=${NEO4J_USERNAME-neo4j}
- OPENAI_API_KEY=${OPENAI_API_KEY-}
- LLM=${LLM-gpt-3.5}
- EMBEDDING_MODEL=${EMBEDDING_MODEL-sentence_transformer}
- LANGCHAIN_ENDPOINT=${LANGCHAIN_ENDPOINT-"https://api.smith.langchain.com"}
- LANGCHAIN_TRACING_V2=${LANGCHAIN_TRACING_V2-false}
- LANGCHAIN_PROJECT=${LANGCHAIN_PROJECT}
- LANGCHAIN_API_KEY=${LANGCHAIN_API_KEY}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
networks:
- net
depends_on:
database:
condition: service_healthy
pull-model:
condition: service_completed_successfully
x-develop:
watch:
- action: rebuild
path: .
ignore:
- loader.py
- pdf_bot.py
- api.py
- front-end/
ports:
- 8501:8501
pdf_bot:
build:
context: .
dockerfile: pdf_bot.Dockerfile
environment:
- NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
- NEO4J_PASSWORD=${NEO4J_PASSWORD-password}
- NEO4J_USERNAME=${NEO4J_USERNAME-neo4j}
- OPENAI_API_KEY=${OPENAI_API_KEY-}
- LLM=${LLM-gpt-3.5}
- EMBEDDING_MODEL=${EMBEDDING_MODEL-sentence_transformer}
- LANGCHAIN_ENDPOINT=${LANGCHAIN_ENDPOINT-"https://api.smith.langchain.com"}
- LANGCHAIN_TRACING_V2=${LANGCHAIN_TRACING_V2-false}
- LANGCHAIN_PROJECT=${LANGCHAIN_PROJECT}
- LANGCHAIN_API_KEY=${LANGCHAIN_API_KEY}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
networks:
- net
depends_on:
database:
condition: service_healthy
pull-model:
condition: service_completed_successfully
x-develop:
watch:
- action: rebuild
path: .
ignore:
- loader.py
- bot.py
- api.py
- front-end/
ports:
- 8503:8503
api:
build:
context: .
dockerfile: api.Dockerfile
volumes:
- $PWD/embedding_model:/embedding_model
environment:
- NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
- NEO4J_PASSWORD=${NEO4J_PASSWORD-password}
- NEO4J_USERNAME=${NEO4J_USERNAME-neo4j}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- LLM=${LLM-gpt-3.5}
- EMBEDDING_MODEL=${EMBEDDING_MODEL-sentence_transformer}
- LANGCHAIN_ENDPOINT=${LANGCHAIN_ENDPOINT-"https://api.smith.langchain.com"}
- LANGCHAIN_TRACING_V2=${LANGCHAIN_TRACING_V2-false}
- LANGCHAIN_PROJECT=${LANGCHAIN_PROJECT}
- LANGCHAIN_API_KEY=${LANGCHAIN_API_KEY}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
networks:
- net
depends_on:
database:
condition: service_healthy
pull-model:
condition: service_completed_successfully
x-develop:
watch:
- action: rebuild
path: .
ignore:
- loader.py
- bot.py
- pdf_bot.py
- front-end/
ports:
- 8504:8504
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 http://localhost:8504/ || exit 1"]
interval: 5s
timeout: 3s
retries: 5
front-end:
build:
context: .
dockerfile: front-end.Dockerfile
x-develop:
watch:
- action: sync
path: ./front-end
target: /app
ignore:
- ./front-end/node_modules/
- action: rebuild
path: ./front-end/package.json
depends_on:
api:
condition: service_healthy
networks:
- net
ports:
- 8505:8505
networks:
net:
Re-building the GenAI Stack
docker compose up -d --build
Verify the container services
docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
genai-stack-api-1 genai-stack-api "uvicorn api:app --h…" api About an hour ago Up About an hour (healthy) 0.0.0.0:8504->8504/tcp
genai-stack-bot-1 genai-stack-bot "streamlit run bot.p…" bot About an hour ago Up About an hour (healthy) 0.0.0.0:8501->8501/tcp
genai-stack-database-1 neo4j:5.11 "tini -g -- /startup…" database About an hour ago Up About an hour (healthy) 0.0.0.0:7474->7474/tcp, 7473/tcp, 0.0.0.0:7687->7687/tcp
genai-stack-front-end-1 genai-stack-front-end "npm run dev" front-end About an hour ago Up About an hour 0.0.0.0:8505->8505/tcp
genai-stack-loader-1 genai-stack-loader "streamlit run loade…" loader About an hour ago Up About an hour (unhealthy) 0.0.0.0:8502->8502/tcp, 0.0.0.0:8081->8080/tcp
genai-stack-pdf_bot-1 genai-stack-pdf_bot "streamlit run pdf_b…" pdf_bot About an hour ago Up About an hour (healthy) 0.0.0.0:8503->8503/tcp
Top comments (0)