DEV Community: Joshua Tuddenham

Teaching an AI Dog New Tricks (The Gemini Integration)

Joshua Tuddenham — Sat, 23 Nov 2024 13:44:34 +0000

The Art of Prompt Engineering (Or: How I Learned to Stop Worrying and Trust the AI)

First attempt at prompting Gemini:

const prompt = "Plan a nice trip";

Result: "Have you considered going somewhere? Perhaps doing things when you get there? Maybe eating food?"

Right. Maybe I need to be a bit more specific. Turns out "plan a nice trip" is to Gemini what "fetch" is to a golden retriever - technically understood, but interpreted rather loosely.

Take Two: The First Real Attempt

My first serious attempt at structuring the prompts was... well, let's say Wooster had some creative interpretations:

type BasicTripRequest = {
  destination: string;
  duration: number;
  startDate: string;
};

const firstPrompt = `
Plan a ${tripRequest.duration} day trip to ${tripRequest.destination}
starting on ${tripRequest.startDate}.

Please include:

- Daily activities
- Locations
- Approximate costs
- Duration of activities
`;

Result: "Day 1 you could maybe explore downtown? Stay as long as you're having fun! Costs depend on how many treats you buy along the way. Some people spend a little, some spend a lot. Day 2 there's this AMAZING park the locals love..."

Not quite the structured response I was hoping for. Time to try JSON.

Adding More Structure (But Not Enough)

Here was my first JSON attempt:

const secondPrompt = `
  Generate a JSON itinerary for a ${tripRequest.duration}-day trip to ${tripRequest.destination}.
  Each day should have 2-3 activities.

  Format:
  {
    "days": [
      {
        "dayNumber": number,
        "activities": [
          {
            "name": string,
            "location": string,
            "description": string,
            "duration": string,
            "category": string,  // This caused problems later...
            "cost": string
          }
        ]
      }
    ]
  }
`;

This worked better, but the categorization was a mess. I got everything from "Fun stuff" to "Walking around looking at things" as categories. It was like asking Wooster to sort his toys - everything ended up in the "things I can put in my mouth" category.

Take this response for example:

{
  "days": [
    {
      "dayNumber": 1,
      "activities": [
        {
          "name": "Explore the city vibes",
          "location": "wherever the wind takes us",
          "description": "Just soak in the atmosphere, you know?",
          "duration": "as long as we're having fun",
          "category": "Places with good smells",
          "cost": "whatever's in the wallet"
        }
      ]
    }
  ]
}

Time to get serious about types.

The Final Evolution

After much trial and error (and Wooster suggesting "belly rubs" as a valid activity category), I landed on this much more structured approach:

export const createPrompt = (
  days: number,
  location: string,
  startDate: string,
): string => `
  Generate **ONLY** JSON data for a ${days}-day trip to ${location}, starting on ${startDate}.
  Have no more than three activities per day. Exclude arrival and departure logistics.

  Duration must be in format "X hours" or "X.5 hours".
  Price must be in format "$X" where X is a number.

  Activities MUST include:
  - "activityName": string
  - "description": string (20-50 words)
  - "location": string
  - "price": string
  - "duration": string
  - "difficulty": "Easy" | "Moderate" | "Challenging"
  - "category": "Adventure" | "Cultural" | "Nature" | "Food & Drink" | "Shopping" | "Entertainment"
  - "bestTime": "Early Morning" | "Morning" | "Afternoon" | "Evening" | "Night"
  - "bookingRequired": boolean
`;

And later, after realizing I needed geolocation for mapping of activities:

export const destinationPromptTemplate = (destination: string) => `
  Generate **ONLY** a valid JSON object for ${destination}.
  Include:
  - "latitude": number (decimal degrees)
  - "longitude": number (decimal degrees)
  - "destinationName": string
  - "country": string
  - "description": string (50-200 words)
  // ... other structured fields
`;

Finally, I was getting responses that looked like actual travel plans rather than Wooster's diary entries:

{
  "days": [
    {
      "dayNumber": 1,
      "activities": [
        {
          "activityName": "Morning Market Tour",
          "description": "Explore the historic Camden Market, featuring local artisans and food vendors. Perfect for gathering travel supplies (and snacks).",
          "location": "Camden Lock Place, London",
          "price": "$15",
          "duration": "2.5 hours",
          "difficulty": "Easy",
          "category": "Food & Drink",
          "bestTime": "Morning",
          "bookingRequired": false
        },
        {
          "activityName": "Regent's Park Walk",
          "description": "A scenic stroll through one of London's most beautiful parks. Watch for squirrels (Wooster insisted this was important).",
          "location": "Regent's Park, London",
          "price": "$0",
          "duration": "1.5 hours",
          "difficulty": "Easy",
          "category": "Nature",
          "bestTime": "Afternoon",
          "bookingRequired": false
        }
      ]
    }
  ]
}

Still with a hint of Wooster's personality, but now in a format that wouldn't make TypeScript cry.

Handling the Responses

Of course, getting the prompt right was only half the battle. Gemini's responses weren't always perfectly formatted JSON, so I needed some utility functions to clean things up:

export const cleanLLMJsonResponse = (text: string): string => {
  // Step 1: Remove markdown code blocks with any language specification
  const withoutCodeBlocks = text.replace(
    new RegExp('```

(?:json)?\\s*([\\s\\S]*?)\\s*

```', 'g'),
    "$1"
  );

  // Step 2: Remove potential comments
  const withoutComments = withoutCodeBlocks.replace(
    new RegExp('\\/\\*[\\s\\S]*?\\*\\/|\\/\\/.*', 'g'),
    ""
  );

  // Step 3: Detect and replace incomplete URLs with a placeholder
  const withCompleteUrls = withoutComments.replace(
    new RegExp('"website":\\s*"https:([^",}]*)', 'g'),
    '"website": "https://example.com"'
  );

  // Step 4: Trim whitespace
  return withCompleteUrls.trim();
};

// Validate the JSON structure
export const validateJSON = (jsonString: string): void => {
  try {
    const parsed = JSON.parse(jsonString);
    if (typeof parsed !== "object" || parsed === null) {
      throw new Error("Response is not a valid JSON object or array");
    }
  } catch (error) {
    console.error(
      "JSON Validation failed. Invalid JSON string:",
      jsonString.slice(0, 500),
    );
    throw new Error(
      `Invalid JSON response: ${error instanceof Error ? error.message : "Unknown error"}`,
    );
  }
};

// Clean up any non-printable characters
export const cleanJSON = (jsonString: string): string => {
  // Remove control characters (ASCII 0 to 31)
  const cleanedString = jsonString.replace(/[\x00-\x1F]/g, "");

  // Remove non-printable characters
  return cleanedString.replace(/[^\x20-\x7E]/g, "");
};

Why did I need all this? Well, Gemini had some... interesting habits:

Sometimes wrapping responses in markdown code blocks
Occasionally including helpful comments (which broke the JSON)
Returning incomplete URLs
Adding mysterious control characters
And my personal favorite: sneaking in emoji (those non-printable characters had to go)

Using these utilities together:

async function generateTripPlan(tripRequest: TripRequest) {
  try {
    const result = await model.generateContent(createPrompt(/* ... */));
    const text = result.response.text();

    // Clean up the response
    const cleanedResponse = cleanLLMJsonResponse(text);
    const sanitizedJSON = cleanJSON(cleanedResponse);

    // Validate before parsing
    validateJSON(sanitizedJSON);

    return JSON.parse(sanitizedJSON);
  } catch (error) {
    console.error("Failed to generate valid trip plan:", error);
    throw new Error("Failed to generate trip plan");
  }
}

What I Actually Learned

Start with strict types from the beginning
Be explicit about formats
The more specific the prompt constraints, the better
Always sanitize and validate AI responses
Geolocation data should have been there from the start
Rate limiting is important
Regex is your best friend when dealing with AI responses
Never trust an AI to format URLs correctly

Next up: the frontend implementation, or as I like to call it, "Making Wooster Look Presentable for Company" (and this time with proper TypeScript interfaces from day one).

This article was originally published on my blog. Follow me there for more content about AI integration, TypeScript, and full-stack development.

From Fetch Mocks to MSW: A Testing Journey

Joshua Tuddenham — Sat, 23 Nov 2024 13:41:35 +0000

The Catalyst: An Innocent Axios Refactor

It started innocently enough. "I'll just refactor these fetch calls to use Axios," I thought, "What could possibly go wrong?" As it turns out, quite a bit - specifically, all my carefully crafted fetch mocks suddenly becoming about as useful as a chocolate teapot.

Rather than rebuilding all my mocks for Axios, I decided to take this opportunity to modernize my approach. Enter Mock Service Worker (MSW).

The Old Way: Jest Mocks and Fetch

Previously, my tests looked something like this:

const mockFetch = vi.fn();
global.fetch = mockFetch;

describe("API functions", () => {
  beforeEach(() => {
    mockFetch.mockReset();
  });

  test("fetchTrips - should fetch trips successfully", async () => {
    const mockTrips = [{ id: 1, name: "Trip to Paris" }];
    mockFetch.mockResolvedValueOnce({
      ok: true,
      json: async () => mockTrips,
    });

    const trips = await fetchTrips(mockSupabase);
    expect(trips).toEqual(mockTrips);
  });
});

It worked, but it wasn't exactly elegant. Each test required manual mock setup, the mocks were brittle, and they didn't really represent how my API behaved in the real world. I was testing implementation details rather than actual behaviour.

Enter MSW: A Better Way to Mock

Mock Service Worker (MSW) takes a fundamentally different approach to API mocking. Instead of mocking function calls, it intercepts actual network requests at the network level. This is huge for a few reasons:

Runtime Integration: MSW works by intercepting actual HTTP requests, meaning your code runs exactly as it would in production. No more mocking fetch or axios - your actual API calls run unchanged.
API-First Design: Instead of thinking about function mocks, you define mock API endpoints that mirror your real API. This pushes you toward better API design and keeps your tests aligned with your actual endpoints.
Request/Response Fidelity: You get to work with real HTTP concepts - status codes, headers, response bodies - instead of simplified mock objects. This means you can catch more realistic edge cases.

Here's how those same tests look with MSW:

// Your API handler definition
http.get(`${BASE_URL}/trips`, () => {
  return HttpResponse.json([
    { id: "1", location: "Trip 1", days: 5, startDate: "2023-06-01" },
    { id: "2", location: "Trip 2", days: 7, startDate: "2023-07-15" },
  ]);
});

// Your test - notice how much cleaner it is
test("fetchTrips - should fetch trips successfully", async () => {
  const trips = await fetchTrips();
  expect(trips).toEqual([
    { id: "1", location: "Trip 1", days: 5, startDate: "2023-06-01" },
    { id: "2", location: "Trip 2", days: 7, startDate: "2023-07-15" },
  ]);
});

No more manual mock setup for each test - the MSW handler takes care of it all. Plus, these handlers can be reused across many tests, reducing duplication and making your tests more maintainable.

The Setup

Setting up MSW was surprisingly straightforward, which immediately made me suspicious. Nothing in testing is ever this easy...

beforeAll(() => {
  server.listen({ onUnhandledRequest: "bypass" });
});

afterEach(() => {
  server.resetHandlers();
  cleanup();
});

afterAll(() => {
  server.close();
});

Then creating handlers that actually looked like my API:

export const handlers = [
  http.get(`${BASE_URL}/trips`, () => {
    return HttpResponse.json([
      { id: "1", location: "Trip 1", days: 5, startDate: "2023-06-01" },
      { id: "2", location: "Trip 2", days: 7, startDate: "2023-07-15" },
    ]);
  }),
];

The Error Handling Journey

My first attempt at error handling was... well, let's say it was optimistic:

export const errorHandlers = [
  http.get(`${BASE_URL}/trips/999`, () => {
    return new HttpResponse(null, { status: 404 });
  }),
];

The problem? The more general /trips/:id handler was catching everything first. It was like having a catch-all route in your Express app before your specific routes - rookie mistake.

After some head-scratching and test failures, I realized the better approach was handling errors within the routes themselves:

http.get(`${BASE_URL}/trips/:id`, ({ params }) => {
  const { id } = params;

  if (id === '999') {
    return new HttpResponse(null, { status: 404 });
  }

  return HttpResponse.json({
    id,
    location: 'Trip 1',
    days: 5,
    startDate: '2023-06-01',
  });
}),

This pattern emerged: instead of separate error handlers, I could handle both success and error cases in the same place, just like a real API would. It was one of those "aha!" moments where testing actually pushes you toward better design.

Lessons Learned

Mock at the right level: MSW lets you mock the network level rather than the function level, making tests more realistic and robust.
Think in endpoints, not functions: Structuring mocks around API endpoints rather than individual function calls better represents the actual application behavior.
Handle errors where they happen: Instead of separate error handlers, handle errors within the endpoint handlers themselves - just like a real API would.

The End Result

The final setup is more maintainable, more realistic, and actually helpful in catching real issues. Gone are the days of:

mockFetch.mockResolvedValueOnce({ ok: false });

Instead, I have proper API mocks that:

Handle both success and error cases
Use realistic response structures
Can be reused across tests
Actually catch integration issues

What's Next?

Looking forward, I'm excited about:

Simulating network errors more realistically
Using MSW's browser integration for end-to-end testing
Adding response delays to test loading states

Sometimes the best improvements come from being forced to change. What started as a simple Axios refactor ended up leading to a much better testing architecture. And isn't that what refactoring is all about?

This article was originally published on my blog. Follow me there for more content about full-stack development, testing, and API design.

Making Wooster Talk: A Deep Dive into Structured Logging

Joshua Tuddenham — Sat, 23 Nov 2024 13:35:37 +0000

The Problem with console.log

Let's be honest - we've all been there. Your Express backend starts misbehaving, so you do what any self-respecting developer would do:

console.log("HERE1!!!");
console.log("Data:", someData);
console.log("WHY IS THIS UNDEFINED??");

Three months later, you're staring at production logs filled with "HERE1!!!" and wondering what past-you was thinking. It's like leaving passive-aggressive post-it notes for yourself, except less helpful.

Enter Pino: Structured Logging for Grown-ups

After the third time searching through logs for a "HERE!!!" that could have been anywhere, I decided it was time to be a responsible developer and implement proper logging. Enter Pino, because even Wooster needs to keep track of what he's been up to.

Environment-Aware Logging

First revelation: different environments need different logging approaches. Here's how I set up Wooster's logger:

import pino from "pino";

export const logger = pino({
  level:
    process.env.NODE_ENV === "test" ? "error" : process.env.LOG_LEVEL || "info",
  ...(process.env.NODE_ENV === "development"
    ? {
        transport: {
          target: "pino-pretty",
          options: {
            colorize: true,
            translateTime: true,
          },
        },
      }
    : {}),
});

Let's break this down:

Test Environment: Only log errors. Jest output is noisy enough without debug logs.
Development: Pretty, colorized logs that are actually readable by humans.
Production: Raw JSON logs - perfect for log aggregation tools.
Configurable Level: Can be overridden with LOG_LEVEL environment variable.

The Art of Log Levels

Pino gives us proper log levels, and choosing the right one is more art than science:

// Debug: For when you're being extra nosey
logger.debug(
  {
    userId,
    searchParams: req.query,
    timestamp: new Date().toISOString(),
  },
  "User searching for destinations",
);

// Info: The good stuff you actually want to know about
logger.info(
  {
    tripId,
    destination,
    duration,
    userId,
  },
  "Trip created successfully",
);

// Warn: Something's not quite right...
logger.warn(
  {
    userId,
    attemptCount,
    lastAttemptTime: new Date().toISOString(),
  },
  "Multiple failed login attempts detected",
);

// Error: Time to wake up the on-call engineer
logger.error(
  {
    error: {
      message: err.message,
      stack: err.stack,
      code: err.code,
    },
    requestId,
    userId,
  },
  "Failed to generate AI travel plan",
);

Request Context: The Missing Piece

The real power of structured logging comes with context. Here's the middleware I use to track requests:

import { v4 as uuidv4 } from "uuid";
import { performance } from "perf_hooks";

export const requestLogger = (
  req: Request,
  res: Response,
  next: NextFunction,
) => {
  const requestId = uuidv4();
  const startTime = performance.now();

  // Attach to request for use in other middlewares
  req.requestId = requestId;

  logger.info(
    {
      requestId,
      method: req.method,
      url: req.url,
      query: req.query,
      userAgent: req.get("user-agent"),
      ip: req.ip,
      // If you have user context from auth
      userId: req.user?.id,
    },
    "Request started",
  );

  // Log when request completes
  res.on("finish", () => {
    const duration = Math.round(performance.now() - startTime);

    logger.info(
      {
        requestId,
        statusCode: res.statusCode,
        duration,
        // Only log response size for successful requests
        ...(res.statusCode < 400 && {
          responseSize: res.get("Content-Length"),
        }),
      },
      "Request completed",
    );
  });

  next();
};

Practical Examples: Real Logs from Wooster

Here's what my logs actually look like in different scenarios:

Development (using pino-pretty):

[2024-11-15 14:23:45.678] INFO: Request started
requestId: 123e4567-e89b-12d3-a456-426614174000
method: POST
url: /api/trips
userId: auth0|123456
ip: 127.0.0.1

[2024-11-15 14:23:45.890] DEBUG: Generating AI travel plan
requestId: 123e4567-e89b-12d3-a456-426614174000
destination: "Paris"
duration: 7
preferences: ["culture", "food"]

[2024-11-15 14:23:46.123] INFO: Request completed
requestId: 123e4567-e89b-12d3-a456-426614174000
statusCode: 201
duration: 445
responseSize: 1234

Production (JSON format):

{
  "level": 30,
  "time": 1673876625678,
  "pid": 1234,
  "requestId": "123e4567-e89b-12d3-a456-426614174000",
  "method": "POST",
  "url": "/api/trips",
  "userId": "auth0|123456",
  "msg": "Request started"
}

{
  "level": 30,
  "time": 1673876626123,
  "pid": 1234,
  "requestId": "123e4567-e89b-12d3-a456-426614174000",
  "statusCode": 201,
  "duration": 445,
  "responseSize": 1234,
  "msg": "Request completed"
}

Real-World Example: Database Service Logging

When I started building Wooster's destination service, I realized logging needs to tell a story. Here's how I approached it:

export const getOrCreateDestination = async (location: string) => {
  try {
    try {
      const existingDestination = await fetchDestinationDetailsByName(location);
      logger.info({ location }, "Found existing destination");
      return existingDestination;
    } catch (error) {
      // I'll talk about error handling in the next post. I believe that's called 'creating tension'...
      if (!isServiceError(error) || error.code !== "DB_NOT_FOUND") {
        throw error;
      }
    }

    logger.info({ location }, "Creating new destination");
    const destinationData = await generateDestinationData(location);
    return await addDestination(destinationData);
  } catch (error) {
    throw error;
  }
};

export const addDestination = (destinationData: NewDestination) =>
  executeDbOperation(
    async () => {
      const [insertedDestination] = await db
        .insert(destinations)
        .values({
          ...destinationData,
          normalizedName: normalizeDestinationName(
            destinationData.destinationName,
          ),
        })
        .returning();

      logger.info({ destinationData }, "Inserted destination successfully");
      return insertedDestination;
    },
    "Failed to insert destination",
    { context: { destinationData } },
  );

This creates a breadcrumb trail through the application:

[14:23:45.678] INFO: Incoming request
method: POST
url: /api/destinations
location: "Paris"

[14:23:45.789] INFO: Creating new destination
location: "Paris"

[14:23:46.012] INFO: Inserted destination successfully
destinationData: {
destinationName: "Paris",
country: "France",
// ... other fields
}

[14:23:46.123] INFO: Request completed
statusCode: 201
duration: 445ms

Key patterns here:

Log at service boundaries (entering/exiting key functions)
Include relevant business context (location, destination data)
Use consistent message formats
Track operation progress through the stack

The result? When something goes wrong (and it will), you can trace the exact path through your application. Trust me, at 3 AM when your app is failing, you'll thank yourself for these breadcrumbs.

Logging Best Practices I've Learned

Be Consistent with Context

Always include requestId
Add userId when available
Include relevant business context (tripId, destinationId, etc.)

Log Level Selection

Error: Something is broken and needs immediate attention
Warn: Something unexpected that might need investigation
Info: Normal business events you want to track
Debug: Detailed information for development/troubleshooting

Security Considerations

Never log sensitive data (passwords, tokens, etc.)
Be careful with PII (personally identifiable information)
Consider GDPR implications of what you're logging

Performance Impact
- Use appropriate log levels to control output
- Consider log rotation in production
- Be mindful of log size (especially with stack traces)

What I Actually Learned

Structured logging isn't just about pretty output - it's about being able to trace issues across requests
Environment-specific configuration is crucial
Context is king - the more relevant context you add, the easier debugging becomes
Good logging practices save hours of debugging time
Your future self will thank you (mine already has)

What's Next?

Now that I can properly track what's happening in my application, it's time to handle what happens when things go wrong. But that's a story for another post about error handling...

And once we've got error handling sorted, we'll look at how to get these lovely structured logs into a proper monitoring system - because as nice as JSON logs are, they're even better when you can query and visualize them. But one thing at a time...

This article was originally published on my blog. Follow me there for more content about full-stack development, testing, and AI integration.