Drupal x Google Summer of Code - Week 3: Reuse & Previews

WELLLLCOMMEE to Week 3 of this series!

In the last week made quite a few structural changes that will help us. Have created the YAML extractor and parser which can extract the YAML component form any response of the LLM using Symfony to validate it. Made a Utility for this thereby making the code reusable.

Official Issue Tracker: Week 3 Tracker

Let me now give me glimpse for this.

https://www.youtube.com/watch?v=7sH6pv2Vyns

The Output Preview & Validator

Tiny UI Changes

Time to Dive Deep into this

YAML Validator & Extractor

Created a Utility folder under src and created a file named Yaml Processor. The concept of this file was simple, when you get an output from LLM, from whichever Service or Function you pull this from this will extract the YAML and give it to you.

Here it uses this : use Drupal\Component\Serialization\Yaml

This under the hood uses Symfony which helps to validate and extraction. Symfony is the underlying PHP framework that powers modern Drupal ( this is just one of the usecase)

<?php

namespace Drupal\ai_recipe_generator\Utility;

use Drupal\Component\Serialization\Yaml;

/**
 * Utility class to parse and extract YAML from AI responses.
 */
class YamlProcessor {

  /**
   * Cleans a response down to YAML text when possible.
   *
   * @param string $response
   *   The raw model response.
   *
   * @return string
   *   A normalized YAML string or the response with code fences removed.
   */
  public static function normalize(string $response): string {
    $parsed = self::extractAndParse($response);

    if (is_array($parsed)) {
      return trim(Yaml::encode($parsed));
    }

    $cleaned = preg_replace('/```

(?:yaml|yml)?\s*|

```/i', '', $response) ?? $response;
    return trim($cleaned);
  }

  /**
   * Attempts to extract and validate YAML from a given string.
   *
   * @param string $response
   *   The raw string response.
   *
   * @return array|null
   *   The parsed YAML as an array, or null if no valid YAML could be extracted.
   */
  public static function extractAndParse(string $response): ?array {
    // First try parsing the entire response as YAML.
    try {
      $parsed = Yaml::decode($response);

      if (is_array($parsed)) {
        return $parsed;
      }
    }
    catch (\Exception $e) {
      // Not valid YAML, continue with extraction.
    }

    // Extract YAML from markdown code blocks.
    $pattern = '/```

(?:yaml|yml)?\s*(.*?)

```/is';

    if (preg_match($pattern, $response, $matches)) {
      $yaml = trim($matches[1]);

      try {
        $parsed = Yaml::decode($yaml);

        if (is_array($parsed)) {
          return $parsed;
        }
      }
      catch (\Exception $e) {
        return NULL;
      }
    }

    return NULL;
  }

}

Reusable YAML Previewer

If you have seen my previous blog, the older YAML previewers they were not upto the mark. So improved that with having better parsers and lines. What I used for this was a template, using a html.twig format which helped to render properly anywhere, anyhow!

This is slightly difficult to think and then implement (tedious slightly) but since highly reusable, highly useful!

{#
/**
 * @file
 * Template for the fluid YAML previewer component with line numbers.
 */
#}
<div class="ai-yaml-preview-container" {{ attributes }}>
  <div class="ai-yaml-preview-header">
    <span class="ai-yaml-preview-title">{{ title|default('YAML Output Preview') }}</span>

    <button class="ai-yaml-copy-btn" onclick="
      const container = this.closest('.ai-yaml-preview-container');
      const code = container.querySelector('code').innerText;
      navigator.clipboard.writeText(code);
      const originalText = this.innerText;
      this.innerText = 'Copied!';
      setTimeout(() => this.innerText = originalText, 2000);
    ">Copy YAML</button>
  </div>

  <div class="ai-yaml-preview-body">
    {# This empty div will be filled with numbers by the JS below #}
    <div class="ai-yaml-line-numbers" aria-hidden="true"></div>

    <pre class="ai-yaml-preview-content"><code class="language-yaml">{{ yaml_string }}</code></pre>
  </div>
</div>

{# Inline script to generate line numbers based on the code length #}
<script>
  (function() {
    // Find all preview containers on the page
    const containers = document.querySelectorAll('.ai-yaml-preview-container');

    containers.forEach(container => {
      const codeBlock = container.querySelector('code');
      const linesContainer = container.querySelector('.ai-yaml-line-numbers');

      // Only process if it hasn't been filled yet
      if (codeBlock && linesContainer && linesContainer.innerHTML === '') {
        // Split by newline to get total lines
        const lines = codeBlock.innerText.split('\n');

        // Handle the case where the AI leaves a trailing empty line at the end
        const lineCount = lines[lines.length - 1] === '' ? lines.length - 1 : lines.length;

        let numbersHTML = '';
        for (let i = 1; i <= Math.max(1, lineCount); i++) {
          numbersHTML += i + '<br>';
        }
        linesContainer.innerHTML = numbersHTML;
      }
    });
  })();
</script>

Code Reduction

Let's understand the problem, we had 3 files for calling the LLM models & providers. All of them had separate functions. So if I ever wanted to change anything, I would have to change in all the three files : (

So we changed the calling part. for 2 (ArenaService & RecipeGenerationService) of them as they were the same replicas the third one at BenchmarkDashboardService was a different one. Clubbing them was fairly simple, just had to change the function calls.

Started fixing the Pipeline

Did the cspell checking and then the phpcs checking which was initially failing but then started fixing the same. For fixing the cspell introduced a file called: .cspell.json. This needed to have all the words in the document which I wanted to exclude form the checking.

Will write a separate blog on testing!

{
    "version": "0.2",
    "language": "en",
    "words": [
        "ai_recipe_generator",
        "chartjs",
        "crosshairs",
        "dalal",
        "fontawesome",
        "hrishikesh",
        "llm",
        "llms",
        "multichat",
        "onclick",
        "Starshot",
        "tempstore",
        "unconfigured",  
        "xaxis",
        "xmark",
        "yaml",
        "yaxis"
    ],
    "ignorePaths": [
        "vendor/**",
        "node_modules/**",
        "*.min.js",
        "*.min.css"
    ]
}