Building powerful AI applications with OpenAI's Responses API
Introduction
Artificial Intelligence has revolutionized how we build applications, and OpenAI's APIs are at the forefront of this transformation. Whether you're a seasoned developer or just starting your AI journey, this guide will walk you through everything you need to know to get started with OpenAI's powerful APIs.
In this comprehensive tutorial, we'll explore:
- Setting up your development environment
- Understanding OpenAI's Responses API
- Building practical examples from basic text generation to advanced features
- Working with images, files, web search, and custom functions
- Multi-agent workflows and intelligent routing
- Best practices and real-world applications
By the end of this guide, you'll have a solid foundation to build your own AI-powered applications.
Project Setup
Prerequisites
Before we dive in, make sure you have:
- Node.js installed (version 14 or higher)
- An OpenAI API key (get one from platform.openai.com)
- Basic JavaScript knowledge
Initial Setup
Let's start by creating our project structure:
mkdir openai-api-sandbox
cd openai-api-sandbox
npm init -y
Installing Dependencies
We'll need two main packages:
npm install openai dotenv
-
openai: The official OpenAI JavaScript library -
dotenv: For managing environment variables securely
Project Structure
Here's our final project structure:
openai-api-sandbox/
├── examples/
│ └── 01-basics.js
├── index.js
├── package.json
├── .env
└── README.md
Environment Configuration
Create a .env file in your project root:
OPENAI_API_KEY=your_openai_api_key_here
Important: Never commit your .env file to version control. Add it to your .gitignore:
.env
node_modules/
Package.json Configuration
Update your package.json to use ES modules:
{
"name": "openai-api-sandbox",
"version": "1.0.0",
"type": "module",
"dependencies": {
"dotenv": "^17.2.1",
"openai": "^5.11.0"
}
}
Basic Setup Code
Create index.js as our entry point:
import './examples/01-basics.js';
Understanding OpenAI's Responses API
OpenAI's Responses API is a powerful interface that allows you to interact with various AI models. Unlike the traditional Chat Completions API, the Responses API provides a more flexible structure for handling different types of inputs and outputs.
Key Concepts
-
Model Selection: Choose from various models like
gpt-4o-mini,gpt-4o, etc. - Input Formats: Support for text, images, files, and structured messages
- Instructions: Provide context and behavior guidelines
- Tools: Extend functionality with image generation, web search, and custom functions
- Streaming: Real-time response generation
Example 1: Basic Text Responses
Let's start with the simplest example - generating text responses.
Simple String Input
const basicResponse = async () => {
const response = await openai.responses.create({
model: 'gpt-4o-mini',
instructions: 'You are a helpful assistant that ends every response with a joke.',
input: 'Hi, how are you?'
});
console.log('Response 1:', response.output_text);
}
Expected Output:
Response 1: Hello! I'm doing great, thank you for asking! I'm here and ready to help you with whatever you need.
Why don't scientists trust atoms? Because they make up everything! 😄
Structured Message Input
const response2 = await openai.responses.create({
model: 'gpt-4o-mini',
input: [
{
role: 'developer',
content: 'You are a helpful assistant that ends every response with a joke.'
},
{
role: 'user',
content: 'Hi, how are you?'
}
]
});
Key Differences:
- String input: Quick and simple for basic interactions
- Structured input: More control over conversation flow and context
The developer role acts like a system message, providing instructions that guide the AI's behavior throughout the conversation.
Example 2: Streaming Responses
For longer responses, streaming provides a better user experience by showing content as it's generated.
const streamResponse = async () => {
const stream = await openai.responses.create({
model: 'gpt-4o',
input: 'give a brief of the history of the internet',
stream: true
});
for await (const chunk of stream) {
process.stdout.write(chunk.delta ?? '');
}
}
What happens:
- The response appears word by word in real-time
- Much better user experience for long-form content
- Lower perceived latency
Use cases:
- Chatbots and conversational interfaces
- Content generation tools
- Real-time assistance applications
Example 3: Image Analysis
OpenAI's vision capabilities allow you to analyze and understand images.
const imageAnalysis = async () => {
const imageurl = 'https://upload.wikimedia.org/wikipedia/commons/3/3d/Lionel_Messi_NE_Revolution_Inter_Miami_7.9.25-055.jpg';
const response = await openai.responses.create({
model: 'gpt-4o-mini',
input: [
{
role: 'user',
content: [
{
type: 'input_text',
text: 'What is this image? What is the name of the player in the image? What is the name of the team in the image?'
},
{
type: 'input_image',
image_url: imageurl,
detail: 'low'
}
]
}
]
});
console.log(response.output_text);
}
Expected Output:
This image shows Lionel Messi, one of the greatest soccer players of all time. He's wearing an Inter Miami CF jersey, which is the MLS team he currently plays for. The image appears to be from a match where Inter Miami is playing, and you can see Messi in his characteristic playing style on the field.
Detail Levels:
-
low: Faster processing, good for general analysis -
high: More detailed analysis, better for complex images -
auto: Let OpenAI choose based on the image
Example 4: Image Generation
Generate images directly through the API using AI tools.
const imageGeneration = async () => {
const response = await openai.responses.create({
model: 'gpt-4.1',
input: 'generate a photo of an engineering team sitting and having lunch',
tools: [
{
type: 'image_generation',
quality: 'low'
}
]
});
// Save the image to a file
const imageData = response.output
.filter((output) => output.type === "image_generation_call")
.map((output) => output.result);
if (imageData.length > 0) {
const imageBase64 = imageData[0];
fs.writeFileSync("image.png", Buffer.from(imageBase64, "base64"));
console.log("Image saved as image.png");
}
}
What this does:
- Requests image generation through the tools parameter
- Filters the response to find image generation results
- Converts base64 data to a PNG file
- Saves it locally
Quality Options:
-
low: Faster generation, lower quality -
high: Better quality, slower generation
Example 5: File Analysis
Analyze documents, PDFs, and other files directly.
const fileAnalysis = async () => {
const response = await openai.responses.create({
model: 'gpt-4o-mini',
input: [
{
role: 'user',
content: [
{
type: 'input_text',
text: 'Analyze the following file and give me a brief of the person'
},
{
type: 'input_file',
file_url: 'https://jayeshpadhiar.com/files/Jayesh_Padhiar_Resume.pdf'
}
]
}
],
});
console.log(response.output_text);
}
Supported file types:
- PDFs
- Text files
- Images
- Spreadsheets (CSV, Excel)
- And more
Use cases:
- Resume analysis
- Document summarization
- Content extraction
- Data analysis
Example 6: Web Search Integration
Access real-time information from the web.
const webSearch = async () => {
const response = await openai.responses.create({
model: 'gpt-4o-mini',
input: 'whats the news on supreme court and dogs',
tools: [
{
type: 'web_search_preview',
}
]
});
console.log(response.output_text);
}
What this enables:
- Real-time information retrieval
- Current events and news
- Up-to-date data and statistics
- Fact-checking and verification
Expected Output:
Based on recent web search results, here are some recent news items about the Supreme Court and dogs:
1. There was a recent case involving service dogs and accessibility rights...
2. A Supreme Court justice made comments about pet ownership during a hearing...
3. Legal discussions about animal rights have been ongoing...
[The AI provides current, factual information from web sources]
Example 7: Custom Functions (Advanced)
The most powerful feature - creating custom tools that the AI can use.
const customFunction = async () => {
const tools = [{
type: 'function',
name: 'get_weather',
description: "'Get the latest weather of the given location and return the temperature in celsius and end the response with a joke',"
parameters: {
type: 'object',
properties: {
location: {
type: 'string',
description: 'The location to get the weather for'
}
},
},
}];
let input = [{
role: 'user',
content: 'whats the weather in navi mumbai'
}];
let response = await openai.responses.create({
model: 'gpt-4o-mini',
input,
tools,
tool_choice: 'auto'
});
// Add the AI's response to the conversation
input = input.concat(response.output);
// Get the function call details
const functionCall = response.output.find(output => output.type === 'function_call');
// Your custom function implementation
const getWeather = async (location) => {
// In a real app, you'd call a weather API here
return {
location,
temperature: 25,
weather: 'sunny',
joke: 'Why did the computer scientist get a cold? Because it had too many bugs!'
}
};
const result = await getWeather('navi mumbai');
// Add the function result back to the conversation
input.push({
type: 'function_call_output',
call_id: functionCall.call_id,
output: JSON.stringify(result)
});
// Get the final response
response = await openai.responses.create({
model: 'gpt-4o-mini',
input,
tools,
instructions: 'Give the final response here',
});
console.log(response.output_text);
}
Expected Output:
The weather in Navi Mumbai is currently sunny with a temperature of 25°C. It's a beautiful day!
Why did the computer scientist get a cold? Because it had too many bugs! 😄
Understanding the Function Flow
- Define the function: Specify name, description, and parameters
- AI decides to call it: Based on user input and function description
- Execute your function: Run your custom logic
- Return results: Feed the results back to the AI
- Get final response: AI processes the results and responds to the user
Tool Choice Options
-
auto: AI decides when to use tools -
required: AI must use a tool -
none: AI cannot use tools
Example 8: Multi-Agent Workflows (Advanced)
OpenAI's Agent framework allows you to create sophisticated multi-agent systems where different agents can handle different tasks and hand off to each other.
import {Agent, run} from '@openai/agents'
const agentFlow = async () => {
const spanishAgent = new Agent({
name: 'spanishAgent',
instructions: 'you are an agent that only speaks spanish. start your response with "Hola soy un agente de espanol"',
});
const englishAgent = new Agent({
name: 'englishAgent',
instructions: 'you are an agent that only speaks english. start your response with "Hello I am an english agent"',
});
const germanyAgent = new Agent({
name: 'germanAgent',
instructions: 'you are an agent that only speaks german. start your response with "Hallo ich bin ein deutscher agent"',
});
const langAgent = new Agent({
name: 'langAgent',
instructions: 'you are an agent that takes input from the user and translates it to the language of the agent that is speaking',
handoffs: [spanishAgent, englishAgent, germanyAgent]
});
const result = await run(langAgent, 'was machst du?');
console.log(result.finalOutput);
}
Expected Output:
Hallo ich bin ein deutscher agent
Ich bin ein KI-Assistent, der dir bei verschiedenen Aufgaben helfen kann. Ich kann Fragen beantworten, Texte übersetzen, bei der Problemlösung helfen und vieles mehr. Was kann ich heute für dich tun?
Understanding Multi-Agent Systems
- Agent Definition: Each agent has specific instructions and capabilities
- Language Specialization: Agents are designed for specific languages or tasks
- Handoff Mechanism: The main agent can delegate to specialized agents
- Intelligent Routing: The system automatically determines which agent should handle the request
Key Benefits
- Specialization: Each agent excels at specific tasks
- Scalability: Easy to add new agents for different capabilities
- Maintainability: Clear separation of concerns
- Flexibility: Dynamic routing based on user input
Real-World Applications
- Customer Support: Route queries to language-specific agents
- Content Creation: Different agents for writing, editing, and translation
- Technical Support: Specialized agents for different product areas
- Educational Tools: Subject-specific tutoring agents
Complete Example Code
Here's the complete 01-basics.js file with all examples:
// this example explores the responses api of openai
import dotenv from 'dotenv';
import OpenAI from 'openai';
import fs from 'fs';
dotenv.config();
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY
});
// All the functions we've discussed above...
// [Include all the function code here]
// Run a specific example
basicResponse();
Running Your Examples
To test your setup:
node index.js
Or run specific examples by modifying the function call at the bottom of 01-basics.js.
Best Practices
1. Error Handling
Always wrap API calls in try-catch blocks:
const safeApiCall = async () => {
try {
const response = await openai.responses.create({
model: 'gpt-4o-mini',
input: 'Hello, world!'
});
console.log(response.output_text);
} catch (error) {
console.error('API Error:', error.message);
}
};
2. Rate Limiting
Be mindful of API rate limits:
- Start with smaller, cheaper models for testing
- Implement retry logic with exponential backoff
- Monitor your usage in the OpenAI dashboard
3. Cost Management
- Use
gpt-4o-minifor most tasks (cheaper and faster) - Use
gpt-4oonly when you need the extra capability - Set up billing alerts in your OpenAI account
4. Security
- Never hardcode API keys
- Use environment variables
- Implement proper authentication in production apps
- Validate and sanitize user inputs
Real-World Applications
Customer Support Chatbot
const customerSupport = async (userMessage) => {
return await openai.responses.create({
model: 'gpt-4o-mini',
instructions: 'You are a helpful customer support agent. Be polite, professional, and try to solve problems efficiently.',
input: userMessage
});
};
Content Generation Tool
const generateBlogPost = async (topic, tone) => {
return await openai.responses.create({
model: 'gpt-4o',
instructions: `Write a ${tone} blog post about ${topic}. Include an introduction, main points, and conclusion.`,
input: `Create a comprehensive blog post about: ${topic}`
});
};
Code Review Assistant
const reviewCode = async (codeSnippet) => {
return await openai.responses.create({
model: 'gpt-4o',
instructions: 'You are a senior software engineer. Review this code for bugs, performance issues, and best practices.',
input: `Review this code:\n\n${codeSnippet}`
});
};
Common Challenges and Solutions
1. Token Limits
Problem: Responses get cut off or requests fail due to token limits.
Solution:
- Break large inputs into smaller chunks
- Use summarization for long documents
- Choose appropriate models based on context length needs
2. Inconsistent Outputs
Problem: AI responses vary too much between runs.
Solution:
- Use more specific instructions
- Provide examples in your prompts
- Adjust temperature settings (lower = more consistent)
3. Function Calling Issues
Problem: Custom functions not being called correctly.
Solution:
- Improve function descriptions
- Provide clear parameter definitions
- Test with simple examples first
Next Steps
Now that you've mastered the basics, here are some areas to explore:
1. Advanced Features
- Fine-tuning models for specific use cases
- Working with embeddings for semantic search
- Implementing RAG (Retrieval-Augmented Generation)
2. Production Considerations
- Building robust error handling
- Implementing caching strategies
- Setting up monitoring and logging
3. Integration Ideas
- Build a Slack bot
- Create a web application with React/Vue
- Integrate with databases and external APIs
4. Learning Resources
Conclusion
OpenAI's APIs open up incredible possibilities for building intelligent applications. From simple text generation to complex multimodal interactions, you now have the foundation to create powerful AI-driven solutions.
The examples we've covered demonstrate the versatility of the Responses API:
- Basic text generation and conversation
- Real-time streaming responses
- Image analysis and generation
- File processing and analysis
- Web search integration
- Custom function calling
- Multi-agent workflows and intelligent routing
Remember to start small, experiment frequently, and always keep user experience in mind. The AI landscape is evolving rapidly, and staying curious and experimental will serve you well.
Happy coding! 🚀
Want to see more AI development tutorials? Follow me on Medium for more in-depth guides and real-world projects.
Repository
You can find the complete code for this tutorial on GitHub: openai-api-sandbox
Feel free to fork, experiment, and build upon these examples!
Top comments (0)