Open AI with Vercel: Solution to Gateway Timeouts

Vercel and OpenAI: Not a Perfect Match.

Vercel is great for starting small projects quickly and for free. However, using AI, like OpenAI, with Vercel can cause issues that only show up when you deploy.

Fixing these errors can take up a lot of our precioussss time. Here’s a quick checklist before we start:

• Are enviromental variables set?
• Is database access allowed?
• Does CORS allow origin?

The Solution

To avoid Vercel's Gateway Timeout problems, split the OpenAI API call into two endpoints and one async function:

• Endpoint /request/initiate
• Function processRequestInBackground
• Endpoint /request/status/:hash

Starting a Request

The endpoint /request/initiate begins a new request. It takes a user's message, creates a unique hash, and checks the database for an existing request. If found, it informs the user of the status. If new, it saves it as 'pending' and starts processing in the background.

app.post('/request/initiate', async (req, res) => {
    const { message } = req.body;
    const hash = crypto.createHash('sha256').update(message).digest('hex');

    try {
        const client = await getDbClient();
        const db = client.db('my_db_name');
        const requestsCollection = db.collection('requests');
        const resourcesCollection = db.collection('resources');

        const existingRequest = await requestsCollection.findOne({ hash });
        if (existingRequest) {
            return res.json({ hash, status: existingRequest.status });
        }

        await requestsCollection.insertOne({ hash, message, status: 'pending' });
        res.status(202).json({ hash, status: 'pending' });

        processRequestInBackground(message, hash, requestsCollection, resourcesCollection);

    } catch (error) {
        console.error('DB error on request initiation:', error);
        res.status(500).json({ message: 'Database error during request initiation.' });
    }
});

Background Processing

The processRequestInBackground function handles request processing. It sends the message to the OpenAI API, then updates the database with the result. If successful, it records the content and tokens used. On error, it updates the status and logs the error.

async function processRequestInBackground(message, hash, requestsCollection, resourcesCollection) {
    try {
        const headers = {
            "Content-Type": "application/json",
            "Authorization": `Bearer YOUR_OPEN_AI_API_KEY`
        };

        const payload = {
            model: 'gpt-4o',
            messages: [
                { role: 'system', content: YOUR_GPT_INSTRUCTIONS },
                { role: 'user', content: message }
            ],
            max_tokens: 1000
        };

        const response = await fetch('https://api.openai.com/v1/chat/completions', {
            method: 'POST',
            headers,
            body: JSON.stringify(payload)
        });

        const jsonResponse = await response.json();
        if (!response.ok) {
            throw new Error('API request failed: ' + jsonResponse.error.message);
        }

        const tokens = jsonResponse.usage.total_tokens;
        const content = jsonResponse.choices[0].message.content;

        await resourcesCollection.updateOne({ resource: 'openai' }, { $inc: { tokens } });
        await requestsCollection.updateOne({ hash }, { $set: { status: 'completed', content, tokens } });

    } catch (error) {
        console.error('Error during request processing:', error);
        await requestsCollection.updateOne({ hash }, { $set: { status: 'error', errorDetails: error.message } });
    }
}

Checking Request Status

The endpoint /request/status/:hash checks the status of a specific request. It returns the current status, content, and tokens used. If no record is found, it informs the user.

app.get('/request/status/:hash', async (req, res) => {
    const hash = req.params.hash;

    try {
        const client = await getDbClient();
        const db = client.db('my_db_name');
        const requestsCollection = db.collection('requests');
        const requestRecord = await requestsCollection.findOne({ hash });

        if (!requestRecord) {
            return res.status(404).json({ message: 'Request record not found.' });
        }

        res.json({ status: requestRecord.status, content: requestRecord.content || null, tokens: requestRecord.tokens || 0 });

    } catch (error) {
        console.error('Error retrieving request status:', error);
        res.status(500).json({ message: 'Error processing your request', details: error.message });
    }
});

And That's It!

If you enjoy JavaScript and want to join our open-source community at Webcrumbs, check out our Discord, GitHub, and Waitlist. Follow us for more posts like this one.

See you all soon!

Comments

[Pavel from LocalCloud]

By using Vercel you so complicate the logic because the same task can be implemented with WebSockets and any VPS/cloud/dedicated server much easier and more efficiently.

[Best Codes]

Thank you so much! I've had this issue many times and simply increased my endpoint timeout to 60 seconds. This is more reliable, though.

Awesome job!