Chatbot NLP: AYW's Approach to Natural Language Understanding

A deep dive into how AYW processes natural language — from intent recognition to context management. Build chatbots that actually understand your customers.

Published: June 13, 2026 | 12 min read | Part 4 of AYW Dev.to Series

The NLP Challenge in Customer Service

Most chatbots fail at natural language understanding (NLU). They rely on:

• Keyword matching ("order" = order status query)
• Regex patterns (rigid, breaks on variations)
• Rule trees (if this, then that — no flexibility)

Result: Customers have to phrase requests exactly right or the bot fails.

Customer: "I ordered a week ago and it hasn't arrived"
Keyword bot: ❌ No match for "ordered" (expects "order status")
Regex bot: ❌ Pattern doesn't capture "hasn't arrived"
Rule bot: ❌ Goes to "order status" but can't handle "missing" context

AYW's approach: Combine LLM-powered NLU with human-guided intent routing for chatbots that truly understand.

---

AYW's NLP Stack (3 Layers)

Layer 1: Input Preprocessing
         ↓
Layer 2: Intent Recognition (LLM + Human Rules)
         ↓
Layer 3: Context Management (Conversation History)
         ↓
     Bot Generates Response

Let's break down each layer.

---

Layer 1: Input Preprocessing

Before sending to the LLM, we:

1. Sanitize Input (Security First)

import DOMPurify from 'dompurify';

const preprocessInput = (rawInput: string) => {
  // 1. Remove HTML tags (prevent XSS)
  const sanitized = DOMPurify.sanitize(rawInput);

  // 2. Trim whitespace
  const trimmed = sanitized.trim();

  // 3. Check for empty input
  if (!trimmed) {
    throw new Error('Empty message');
  }

  // 4. Limit length (prevent token abuse)
  if (trimmed.length > 2000) {
    return trimmed.substring(0, 2000) + '...';
  }

  return trimmed;
};

2. PII Detection (Privacy Protection)

import { detectPII } from '../utils/piiDetector';

const checkForPII = (message: string) => {
  const piiDetected = detectPII(message);

  if (piiDetected.hasEmail || piiDetected.hasPhone) {
    // Log for audit (don't send PII to OpenAI)
    auditLog('pii_detected', {
      conversationId,
      types: piiDetected
    });

    // Mask PII before sending to LLM
    return maskPII(message);
  }

  return message;
};

// Example mask:
// Input: "My email is [e-mail] and phone is 555-1234"
// Output: "My email is [EMAIL_MASKED] and phone is [PHONE_MASKED]"

Why this matters: GDPR compliance, customer trust, and not sending PII to OpenAI.

---

Layer 2: Intent Recognition (Human-Guided)

Unlike fully autonomous bots that let LLMs "figure out" intent, AYW uses structured intent definitions with LLM assistance.

Intent Schema (Human-Defined)

export interface Intent {
  id: string;
  name: string;
  description: string;
  keywords: string[]; // Fallback for simple matching
  examples: string[]; // Training examples for LLM
  botType: 'support' | 'sales' | 'feedback' | 'knowledge';
  confidenceThreshold: number;
}

const intents: Intent[] = [
  {
    id: 'order_status',
    name: 'Order Status Inquiry',
    description: 'Customer wants to check the status of their order',
    keywords: ['order', 'status', 'shipping', 'tracking', 'where'],
    examples: [
      "Where's my order?",
      "I ordered last week, has it shipped?",
      "Can you track my package?",
      "Order #12345 status please"
    ],
    botType: 'support',
    confidenceThreshold: 70
  },
  {
    id: 'return_request',
    name: 'Return Request',
    description: 'Customer wants to return an item',
    keywords: ['return', 'refund', 'send back', 'defective'],
    examples: [
      "I want to return this laptop",
      "How do I get a refund?",
      "The product is defective, I need to send it back"
    ],
    botType: 'support',
    confidenceThreshold: 75
  },
  // ... more intents
];

LLM-Powered Intent Detection

import OpenAI from 'openai';

const detectIntent = async (message: string, conversationHistory: Message[]) => {
  const prompt = `You are an intent recognition system for a customer service chatbot.

AVAILABLE INTENTS:
${intents.map(i => `- ${i.id}: ${i.description}`).join('\n')}

CONVERSATION HISTORY:
${conversationHistory.map(m => `${m.role}: ${m.content}`).join('\n')}

CURRENT MESSAGE:
${message}

TASK:
Return a JSON object with:
- "intent": The most likely intent ID from the list above
- "confidence": A number between 0 and 1 representing your confidence
- "reasoning": Brief explanation of why you chose this intent

Only return the JSON object, nothing else.`;

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: prompt }],
    temperature: 0.3, // Low temperature for consistent classification
    max_tokens: 150,
    response_format: { type: 'json_object' }
  });

  const result = JSON.parse(completion.choices[0].message.content || '{}');

  return {
    intent: result.intent,
    confidence: result.confidence,
    reasoning: result.reasoning
  };
};

Why This Works Better

Approach	Accuracy	Explainability	Cost
Keyword Matching	40-60%	✅ High	💰 Free
Autonomous LLM	70-80%	❌ Low (black box)	💰💰💰 High
AYW (Structured + LLM)	85-95%	✅ High (reasoning field)	💰💰 Medium

---

Layer 3: Context Management

Understanding a single message isn't enough. Customers reference previous context:

User: "I ordered a laptop last week."
Bot: "I can help with that! Can you provide your order number?"
User: "It was the MacBook Pro."
Bot: ✅ Understands "It" refers to the laptop order

Conversation History Management

interface ConversationContext {
  conversationId: string;
  messages: Message[]; // Last 10 messages
  entities: {
    orderNumbers: string[];
    productNames: string[];
    customerIntent: string;
  };
  sentiment: 'positive' | 'neutral' | 'negative';
}

const buildContext = async (conversationId: string): Promise<ConversationContext> => {
  // Get last 10 messages
  const messages = await prisma.message.findMany({
    where: { conversationId },
    orderBy: { createdAt: 'desc' },
    take: 10
  });

  // Extract entities (order numbers, product names, etc.)
  const entities = extractEntities(messages);

  // Analyze sentiment
  const sentiment = await analyzeSentiment(messages);

  return {
    conversationId,
    messages: messages.reverse(), // Chronological order
    entities,
    sentiment
  };
};

// Example entity extraction:
// Input: "My order #12345 for MacBook Pro hasn't arrived"
// Output: { orderNumbers: ['#12345'], productNames: ['MacBook Pro'] }

Passing Context to Bots

const processMessage = async (message: string, conversationId: string) => {
  // 1. Build context
  const context = await buildContext(conversationId);

  // 2. Detect intent with context
  const { intent, confidence } = await detectIntent(message, context.messages);

  // 3. Route to appropriate bot with context
  const bot = getBotForIntent(intent);

  // 4. Generate response with FULL context
  const response = await bot.generateResponse({
    message,
    context,
    intent,
    confidence
  });

  // 5. Update context with new message
  await updateContext(conversationId, message, response);

  return response;
};

---

Advanced NLP Techniques (AYW Pro Features)

1. Sentiment-Aware Responses

const generateResponse = async (message: string, sentiment: string) => {
  const systemPrompt = `You are a customer service bot.

SENTIMENT DETECTED: ${sentiment}

ADJUST YOUR TONE:
- If NEGATIVE: Be extra empathetic, offer solutions quickly, escalate if needed
- If NEUTRAL: Be professional and helpful
- If POSITIVE: Be warm and appreciative`;

  // ... LLM call with sentiment-aware prompt
};

2. Entity Extraction for Personalization

const personalizeResponse = (response: string, entities: Entities) => {
  let personalized = response;

  // Replace generic references with specific entities
  if (entities.productNames.length > 0) {
    personalized = personalized.replace(
      /that product/gi,
      entities.productNames[0]
    );
  }

  return personalized;
};

3. Multi-Turn Intent Tracking

Sometimes intent evolves over a conversation:

User: "I want to buy a laptop." → Intent: sales_inquiry
Bot: "Great! What's your budget?"
User: "Actually, I need help with my current laptop." → Intent: support_request

AYW tracks intent shifts across turns:

const trackIntentShift = (
  oldIntent: string,
  newIntent: string,
  confidence: number
) => {
  if (oldIntent !== newIntent && confidence > 0.8) {
    auditLog('intent_shift', {
      from: oldIntent,
      to: newIntent,
      confidence
    });
  }
};

---

Measuring NLP Performance

Track these metrics weekly:

Metric	Target	How to Measure
Intent Accuracy	85%+	Sample 100 conversations, manually verify intent detection
Entity Extraction	90%+	Check if order numbers, names correctly captured
Context Retention	95%+	Test follow-up questions ("What about the blue one?")
Sentiment Detection	80%+	Compare AI sentiment vs. human rating

AYW Dashboard: NLP Analytics

Intent Distribution (Last 7 Days):
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
order_status:     ████████████ 35% (1,250 conversations)
return_request:   ████████ 20% (714)
sales_inquiry:     ██████ 15% (535)
faq_general:       █████ 12% (429)
escalated:         ███ 10% (357)
unknown:           █ 8% (286)

Confidence Distribution:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
90-100%: ████████████ 60% (High confidence)
70-89%:  ██████ 25% (Medium confidence)
<70%:    █ 15% (Escalated to human)

---

Common NLP Pitfalls (And How AYW Avoids Them)

1. Over-Reliance on LLMs

Problem: Letting the LLM "figure out" everything (expensive, slow, unpredictable).

AYW Solution: Use LLM for language understanding, not business logic. Human-defined intents + LLM classification.

2. Ignoring Context

Problem: Treating every message as a standalone request.

AYW Solution: Maintain 10-message context window with entity extraction.

3. No Fallback Strategy

Problem: Bot fails silently when confidence is low.

AYW Solution: Confidence threshold (70%) triggers human escalation with FULL context handoff.

---

Try AYW's NLP Stack

Ready to build chatbots that truly understand your customers?

1. Clone the AYW Monorepo

git clone https://github.com/ayw-ai/ayw-monorepo.git
cd ayw-monorepo
npm install

2. Try the NLP Service

import { NLUService } from './services/nluService';

const nlu = new NLUService();

const result = await nlu.processMessage({
  message: "I ordered a week ago and it hasn't arrived",
  conversationHistory: [],
  customerId: 'cust_123'
});

console.log(result);
// {
//   intent: 'order_status',
//   confidence: 0.92,
//   entities: { orderNumbers: [], productNames: [] },
//   sentiment: 'negative',
//   routing: 'support_bot'
// }

3. Join the Beta

ayw.ai/waitlist — Get early access to AYW's NLP-powered chatbots.

---

Resources

• GitHub: github.com/ayw-ai/ayw-monorepo
• Docs: docs.ayw.ai/nlp
• Previous Article: Building Your First AYW Bot
• Next Article: Deploying Interactive Chatbots on Vercel

---

Questions about NLP or chatbot architecture? Drop them in the comments! 👇

Tags: #NLP #AI #Chatbots #TypeScript #Tutorial #OpenAI

Series: AYW Developer Tutorials (Part 4 of 6)