How We Cut AI Customer Service Costs by 1000x with BYOK Architecture

When we built EzyshopAI, an AI customer service tool for independent ecommerce sellers, we started with the standard SaaS playbook: platform-managed AI, pay-per-use pricing.

It took us about two weeks to realize this model was broken. Here's why, and how we rebuilt the entire billing architecture around BYOK (Bring Your Own Key).

The problem with per-token pricing

Most AI helpdesk tools charge roughly $1 per AI reply. Sounds reasonable until you do the math.

For a mid-sized seller with 10 storefronts and 500 daily inquiries:

• 500 replies × 30 days = 15,000 replies/month
• 15,000 × $1 = $15,000/month

That's just the AI layer. Add seat fees, store fees, channel fees, and you're looking at $15k–$20k/month.

Meanwhile, the actual cost of calling DeepSeek or OpenAI is roughly $0.001 per reply.

The platform is taking a 1000x markup. Sellers know it. They hate it. And they leave.

---

Why we chose BYOK from day one

BYOK means: the customer brings their own API key. We don't touch the AI billing layer at all.

Benefits:

1. Cost transparency — customers see exactly what they pay OpenAI/DeepSeek
2. No server pressure — we don't proxy all AI requests through our backend
3. No rate limit issues — each customer's API key has its own quota
4. Customer trust — we're not hiding a 1000x markup

The tradeoff: it's more complex to build. Here's how we did it.

Architecture: secure API key storage

The hardest part of BYOK is storing customer API keys securely. We use AES-256 encryption:

const crypto = require('crypto');

const ENCRYPTION_KEY = process.env.API_KEY_ENCRYPTION_KEY; // 32 bytes

function encryptApiKey(plainKey) {
  const iv = crypto.randomBytes(16);
  const cipher = crypto.createCipheriv('aes-256-cbc', ENCRYPTION_KEY, iv);
  let encrypted = cipher.update(plainKey, 'utf8', 'hex');
  encrypted += cipher.final('hex');
  return iv.toString('hex') + ':' + encrypted;
}

function decryptApiKey(encryptedKey) {
  const parts = encryptedKey.split(':');
  const iv = Buffer.from(parts[0], 'hex');
  const encryptedText = parts[1];
  const decipher = crypto.createDecipheriv('aes-256-cbc', ENCRYPTION_KEY, iv);
  let decrypted = decipher.update(encryptedText, 'hex', 'utf8');
  decrypted += decipher.final('utf8');
  return decrypted;
}

Keys are encrypted before hitting the database. We never log them, never expose them to the frontend.

---

Architecture: direct AI call layer

When a customer message comes in:

async function getAIReply(userId, messages) {
  // 1. Fetch encrypted key from DB
  const customer = await Customer.findByPk(userId);
  const encryptedKey = customer.openai_api_key;

  // 2. Decrypt
  const apiKey = decryptApiKey(encryptedKey);

  // 3. Call OpenAI/DeepSeek directly with customer's key
  const response = await axios.post(
    process.env.AI_BASE_URL + '/chat/completions',
    {
      model: customer.preferred_model,
      messages: messages,
      temperature: 0.7,
      max_tokens: 500
    },
    {
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json'
      }
    }
  );

  // 4. Log token usage (for billing dashboard, not for billing)
  await logTokenUsage(userId, response.data.usage);

  return response.data.choices[0].message.content;
}

We don't profit from AI at all. The $15k/month scenario shrinks to $30/month.

---

Handling the edge cases

What if the customer's key runs out of quota?

We added graceful degradation:

try {
  return await callAIWithCustomerKey(userId, messages);
} catch (error) {
  if (error.response?.status === 429) {
    return {
      error: 'API key quota exceeded',
      fallback: 'Please top up your API key or upgrade to platform mode'
    };
  }
  throw error;
}

What if the customer doesn't know how to get an API key?

We keep a "platform mode" as a fallback. New users can start with our complimentary token allowance, then switch to BYOK when they're comfortable.

---

The results

Metric	Before (platform mode)	After (BYOK)
Cost per AI reply	$1.000	$0.001
Monthly cost (500 replies/day)	$15,000	$30
Customer trust complaints	High	Near zero
Server load from AI proxies	Heavy	None

Key takeaway

BYOK isn't just about cost savings. It's about trust. When you stop taking a 1000x markup on AI and let customers pay the provider directly, you build a relationship that lasts.

We also decided not to require Shopify OAuth authorization. Your chat history stays yours, even if your store gets banned. No lock-in, no middleman markup.

That's the SaaS we wanted to build.

About EzyshopAI: AI customer service for independent ecommerce sellers. One-line JS widget, no OAuth required, Telegram instant notifications, BYOK AI. Learn more at ezyshopai.com.

Tags: #saas #ai #nodejs #ecommerce #startup