Building an AI application using Python is no longer limited to machine learning researchers. Developers can now integrate Large Language Models (LLMs), build document-aware assistants, and automate multi-step workflows using familiar Python tools.
But there is an important difference between getting a response from an LLM API and building a dependable AI application.
A working application needs proper API design, input validation, error handling, security, observability, and a clear strategy for managing model costs.
In this tutorial, we will build a Python AI application using FastAPI and an LLM API. Then, we will explore how to extend it with Retrieval-Augmented Generation (RAG) and Agentic AI.
If you are exploring a Gen AI & Agentic AI with Python Online Training Institute in Hyderabad or comparing a Gen AI & Agentic AI with Python Online course in Hyderabad, the concepts in this tutorial will help you understand the technologies used in practical AI application development.
What Are We Building?
We will create a simple AI-powered question-answering API.
A user submits a question, the Python backend sends it to an LLM, and the application returns the generated response.
The initial workflow looks like this:
Client
|
v
FastAPI Endpoint
|
v
Request Validation
|
v
LLM API
|
v
Response Processing
|
v
JSON Response
This architecture is intentionally small. We can introduce additional components later when the application requires document retrieval, tool execution, or more complex workflows.
Technologies Used
Python handles the application logic, while FastAPI exposes the functionality through an HTTP endpoint.
Pydantic validates incoming requests and response structures. The OpenAI Python SDK provides model access in this example, although the same architectural principles apply to other LLM providers.
We will also discuss embeddings, vector databases, and agent orchestration frameworks as the application evolves.
You do not need to install every AI framework to get started. Begin with the minimum required dependencies.
Step 1: Create a Python Project
Create a directory and initialize a virtual environment.
mkdir python-ai-app
cd python-ai-app
python -m venv .venv
Activate the environment on Windows:
.venv\Scripts\activate
On Linux or macOS:
source .venv/bin/activate
Install the dependencies:
pip install fastapi "uvicorn[standard]" openai pydantic-settings
Using a virtual environment isolates project dependencies. For collaborative projects, maintain controlled package versions to ensure that development and deployment environments behave consistently.
Step 2: Configure the LLM API
Create a .env file.
OPENAI_API_KEY=your_api_key_here
OPENAI_MODEL=your_supported_model_id
Replace the placeholders with your actual credentials and a model ID supported by your account.
Add .env to .gitignore before committing your project.
.env
.venv/
__pycache__/
For production environments, use a secrets manager rather than storing credentials in source code or deploying a local environment file.
Step 3: Implement the AI Application
Create a file named main.py.
import logging
from fastapi import FastAPI, HTTPException
from openai import AsyncOpenAI, APIError, APITimeoutError
from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings, SettingsConfigDict
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class Settings(BaseSettings):
openai_api_key: str
openai_model: str
model_config = SettingsConfigDict(
env_file=".env",
extra="ignore",
)
settings = Settings()
client = AsyncOpenAI(
api_key=settings.openai_api_key,
timeout=20.0,
max_retries=2,
)
app = FastAPI(title="Python AI Application")
class QuestionRequest(BaseModel):
question: str = Field(
min_length=3,
max_length=2000,
)
class AnswerResponse(BaseModel):
answer: str
@app.post("/ask", response_model=AnswerResponse)
async def ask_question(request: QuestionRequest):
try:
response = await client.responses.create(
model=settings.openai_model,
instructions=(
"You are a technical assistant. "
"Provide clear and accurate answers. "
"State uncertainty when information is insufficient."
),
input=request.question,
max_output_tokens=500,
)
answer = response.output_text.strip()
if not answer:
raise HTTPException(
status_code=502,
detail="The model returned an empty response.",
)
return AnswerResponse(answer=answer)
except APITimeoutError as exc:
logger.warning("Model request timed out")
raise HTTPException(
status_code=504,
detail="The AI service timed out. Please retry.",
) from exc
except APIError as exc:
logger.error(
"Model provider request failed: %s",
type(exc).__name__,
)
raise HTTPException(
status_code=502,
detail="The AI service is temporarily unavailable.",
) from exc
This example uses an asynchronous client, validates incoming data, limits the requested output, and handles common provider failures.
The asynchronous approach is useful when multiple requests are waiting on network operations. It does not make the model itself generate responses faster.
The implementation assumes an installed SDK version that supports the Responses API and a compatible configured model.
One additional production consideration is connection lifecycle management. For larger applications, initialize and close shared clients through FastAPI's lifespan mechanism rather than treating application startup and shutdown as incidental details.
Step 4: Start the Application
Run the development server.
uvicorn main:app --reload
Open the interactive API documentation:
http://127.0.0.1:8000/docs
Find the POST /ask endpoint and submit the following request.
{
"question": "Explain dependency injection in Spring Boot."
}
The application sends the question to the configured model and returns the answer as JSON.
We now have a basic AI-powered REST API.
However, this is still a demonstration. A public deployment also needs authentication, rate limiting, monitoring, automated tests, and appropriate data-handling controls.
Understanding the Application Architecture
The current application contains several logical components.
The API layer validates incoming requests. The model integration layer communicates with the LLM provider. The response layer returns a validated response to the client.
As the project grows, separate these responsibilities into dedicated modules.
python-ai-app/
|
|-- app/
| |-- main.py
| |-- config.py
| |-- schemas.py
| |-- services/
| | |-- ai_service.py
| |-- api/
| | |-- routes.py
| |-- tests/
|
|-- .env
|-- .gitignore
|-- requirements.txt
This separation makes it easier to test business logic independently of HTTP handling and model-provider details.
For example, replacing one LLM provider with another should not require rewriting the entire API layer.
Moving from Basic LLM Calls to RAG
A basic LLM can generate responses, summarize text, and explain concepts. But what happens when the application needs to answer questions using internal company documents?
This is where Retrieval-Augmented Generation becomes useful.
RAG retrieves relevant information from an external knowledge source and supplies it to the model as context.
The workflow looks like this:
Documents
|
v
Text Extraction
|
v
Chunking
|
v
Embedding Generation
|
v
Vector Database
User Question
|
v
Semantic Retrieval
|
v
Relevant Document Chunks
|
v
LLM
|
v
Grounded Answer
For example, an organization might build an assistant that answers employee questions using approved technical documentation, product manuals, or company policies.
Instead of asking the model to rely entirely on its pretrained knowledge, the application provides relevant source material for the current question.
Important RAG Design Decisions
Chunk size affects retrieval quality. Very small chunks may lose important context, while very large chunks may retrieve too much irrelevant information.
Embedding models should be selected according to retrieval quality, supported languages, latency, and cost.
A vector database is not always sufficient by itself. Hybrid retrieval, metadata filtering, and reranking can improve results when the knowledge base contains similar terminology or complex technical documents.
Access control is especially important. Document permissions must be enforced during retrieval rather than relying on the model to avoid revealing unauthorized information.
Use RAG when answers need to reflect a defined and evolving knowledge base. For simple text summarization, adding a retrieval pipeline may create unnecessary complexity.
Understanding Agentic AI with Python
Agentic AI introduces another capability: the application can use a model to select from approved tools and coordinate multiple steps toward a goal.
Consider an IT support assistant.
A user reports that an application is unavailable. The assistant searches the troubleshooting documentation, checks service status through an API, evaluates the result, and prepares a support ticket if necessary.
The workflow becomes:
User Goal
|
v
LLM Decision
|
v
Tool Selection
|
v
Permission Validation
|
v
Tool Execution
|
v
Result Evaluation
|
v
Final Answer or Next Step
This is different from a simple question-answering application because the system can execute approved operations and use their results to determine what happens next.
Frameworks such as LangGraph can help implement stateful workflows, conditional transitions, and human approval steps.
However, a fixed sequence of business operations may be better implemented with ordinary Python functions and explicit control flow. Autonomous planning introduces additional latency, cost, and failure modes.
Building Safer AI Agents
An AI agent should never receive unrestricted access to critical systems.
Tools should be narrowly defined, and every operation should validate its parameters and permissions independently of the model's instructions.
For example, an agent might prepare a purchase order, but approval should remain in the organization's established authorization workflow.
Use execution limits, timeouts, audit records, and idempotency controls for operations that change application state.
Retrieved documents and tool outputs should also be treated as untrusted input. Prompt injection can attempt to manipulate an agent into ignoring its intended task or invoking inappropriate tools.
The correct defense is not simply to add a stronger prompt. Enforce permissions, validate operations, and isolate sensitive capabilities in trusted application code.
Performance Optimization and Cost Management
An AI application's latency includes several components.
Total Latency
=
Input Validation
+
Retrieval
+
Model Inference
+
Post-processing
Network latency, provider queues, and tool execution can add further delays.
Reducing unnecessary context is often a useful optimization. Sending an entire document when only two paragraphs are relevant increases token consumption and may introduce distracting information.
Caching can improve performance when results are stable, but cached responses must respect user permissions and data freshness.
Retries should be bounded. Aggressive retries during a provider outage can increase cost and amplify the failure.
For long-running tasks, background workers and task queues may be more suitable than keeping an HTTP request open while multiple tools execute.
Measure p95 and p99 latency, error rates, token usage, and successful task completion. A lower-cost model may not be economical if it produces substantially more errors or requires extensive human correction.
Testing and Evaluating AI Applications
Traditional unit tests remain important, but model-driven applications also need evaluations that account for variable outputs.
Create a representative dataset containing ordinary questions, ambiguous requests, unsupported questions, and adversarial inputs.
Measure whether responses are factually supported, whether retrieval returns relevant documents, and whether the application correctly handles failures.
For RAG applications, evaluate retrieval quality separately from answer quality. This helps identify whether a failure comes from missing evidence or poor generation.
For agentic applications, test tool selection, argument validation, permission enforcement, maximum iteration limits, and recovery from tool failures.
A successful demonstration is not enough to establish reliability. Repeatable evaluations are necessary whenever prompts, models, retrieval configurations, or application code change.
Real-World Applications
Python AI applications can support customer service, software development, IT operations, analytics, and document processing.
Customer support assistants can retrieve troubleshooting instructions and summarize conversations. Software engineering assistants can explain code, suggest test cases, and search internal documentation.
Data analytics applications can translate natural-language questions into controlled analytical workflows. Generated SQL must be validated and executed with appropriate database permissions.
IT operations assistants can correlate diagnostic information, retrieve runbooks, and recommend remediation steps. Beginning with read-only tools is generally safer than allowing an agent to change production infrastructure.
In every case, the architecture should reflect the business risk, data sensitivity, and cost of an incorrect result.
Learning Gen AI and Agentic AI with Python
Developers who want to work in AI engineering should build practical skills in Python, REST APIs, LLM integration, structured outputs, embeddings, RAG, and agent orchestration.
They should also understand testing, security, performance monitoring, and deployment. These engineering fundamentals are essential when moving from prototypes to reliable applications.
When comparing a Gen AI & Agentic AI with Python Online Training Institute in Hyderabad, evaluate the quality of hands-on projects, instructor experience, code reviews, and coverage of real deployment challenges.
A Gen AI & Agentic AI with Python Online course in Hyderabad is particularly valuable when learners build complete applications instead of only following isolated demonstrations.
A strong portfolio project should show the problem being solved, the architecture, working code, evaluation results, security controls, and known limitations.
Conclusion
Building your first AI application using Python starts with a small, well-defined problem and a working model integration.
FastAPI provides a practical way to expose AI functionality through REST endpoints. RAG helps applications answer questions using external knowledge, while Agentic AI enables controlled multi-step workflows when a task requires tool use and dynamic decisions.
The most reliable applications combine these capabilities with conventional software engineering practices: input validation, security, automated testing, observability, and explicit business rules.
Start with the basic API, measure its behavior, and introduce additional components only when they provide measurable value. This approach builds the practical foundation needed to develop production-oriented Gen AI and Agentic AI applications using Python.
Top comments (0)