DEV Community

Cover image for Automated Testing for SCORM E-Learning Packages Using Playwright — A Step-by-Step Guide
Aditya Tiwari
Aditya Tiwari

Posted on

Automated Testing for SCORM E-Learning Packages Using Playwright — A Step-by-Step Guide

#testing #playwright #lms #scorm

Most testing tutorials ignore e-learning completely. Here's how to build a Playwright test suite that validates your SCORM packages actually work across LMS platforms.

Why E-Learning Testing Is Different

If you've ever published a SCORM package to an LMS and watched it silently fail — no completion recorded, quiz scores vanishing, navigation broken — you know the pain. E-learning content doesn't behave like a typical web app. It runs inside an LMS-provided iframe, communicates through a JavaScript API (the SCORM Runtime), and its behavior changes depending on which LMS hosts it.

Manual QA across even 3-4 LMS platforms is slow and error-prone. In this tutorial, I'll walk you through setting up Playwright to automate SCORM package testing — from basic content loading to verifying API calls and completion status.

Prerequisites

Before we start, make sure you have:

  • Node.js 18+ installed
  • Playwright (npm init playwright@latest)
  • A SCORM 1.2 or 2004 package (a .zip file containing your e-learning content)
  • A local LMS for testing — we'll use SCORM Cloud (free tier) or a simple SCORM API shim

Step 1: Set Up a Local SCORM Runtime Shim

Testing SCORM content requires an API that mimics what an LMS provides. Rather than spinning up a full Moodle instance, we'll create a lightweight shim.

Create a file called scorm-api-shim.js:

// scorm-api-shim.js
// Mimics the SCORM 1.2 Runtime API that an LMS would expose

window.API = {
  _data: {},
  _initialized: false,
  _calls: [],

  LMSInitialize: function(param) {
    this._initialized = true;
    this._calls.push({ method: 'LMSInitialize', param, timestamp: Date.now() });
    console.log('[SCORM] LMSInitialize called');
    return "true";
  },

  LMSGetValue: function(key) {
    this._calls.push({ method: 'LMSGetValue', key, timestamp: Date.now() });
    return this._data[key] || "";
  },

  LMSSetValue: function(key, value) {
    this._data[key] = value;
    this._calls.push({ method: 'LMSSetValue', key, value, timestamp: Date.now() });
    console.log(`[SCORM] SetValue: ${key} = ${value}`);
    return "true";
  },

  LMSCommit: function(param) {
    this._calls.push({ method: 'LMSCommit', param, timestamp: Date.now() });
    return "true";
  },

  LMSFinish: function(param) {
    this._calls.push({ method: 'LMSFinish', param, timestamp: Date.now() });
    this._initialized = false;
    return "true";
  },

  LMSGetLastError: function() { return "0"; },
  LMSGetErrorString: function(code) { return "No error"; },
  LMSGetDiagnostic: function(code) { return ""; }
};
Enter fullscreen mode Exit fullscreen mode

This gives us a mock API that logs every SCORM call — which becomes our test assertion layer.

Step 2: Serve the SCORM Package Locally

Unzip your SCORM package and serve it with a simple HTTP server. Create serve-scorm.js:

// serve-scorm.js
const express = require('express');
const path = require('path');
const app = express();

// Serve the SCORM API shim at the parent level (LMS frame)
app.get('/lms', (req, res) => {
  res.send(`
    <!DOCTYPE html>
    <html>
    <head>
      <title>Test LMS</title>
      <script src="/scorm-api-shim.js"></script>
    </head>
    <body>
      <iframe id="content-frame" 
              src="/scorm-content/index.html" 
              width="100%" 
              height="600px">
      </iframe>
    </body>
    </html>
  `);
});

app.use('/scorm-api-shim.js', express.static(path.join(__dirname, 'scorm-api-shim.js')));
app.use('/scorm-content', express.static(path.join(__dirname, 'unzipped-scorm-package')));

app.listen(3000, () => console.log('Test LMS running at http://localhost:3000/lms'));
Enter fullscreen mode Exit fullscreen mode

Run it:

npm install express
node serve-scorm.js
Enter fullscreen mode Exit fullscreen mode

Visit http://localhost:3000/lms — you should see your SCORM content loaded inside the iframe, with API calls logging in the console.

Step 3: Write Your First Playwright Test — Content Loads and Initializes

Create your test file at tests/scorm-basic.spec.js:

// tests/scorm-basic.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Basic Validation', () => {

  test('should load content and call LMSInitialize', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // Wait for the content iframe to load
    const frame = page.frameLocator('#content-frame');
    await frame.locator('body').waitFor({ state: 'visible' });

    // Check that LMSInitialize was called
    const initCalled = await page.evaluate(() => {
      return window.API._calls.some(c => c.method === 'LMSInitialize');
    });

    expect(initCalled).toBe(true);
  });

  test('should set lesson_status to incomplete or browsed on load', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');
    await frame.locator('body').waitFor({ state: 'visible' });

    // Give the content a moment to make its initial API calls
    await page.waitForTimeout(2000);

    const lessonStatus = await page.evaluate(() => {
      return window.API._data['cmi.core.lesson_status'];
    });

    // SCORM content typically sets status to 'incomplete' or 'browsed' on launch
    expect(['incomplete', 'browsed', 'not attempted']).toContain(lessonStatus);
  });

});
Enter fullscreen mode Exit fullscreen mode

Run the test:

npx playwright test tests/scorm-basic.spec.js
Enter fullscreen mode Exit fullscreen mode

Step 4: Test Quiz Interaction Tracking

If your SCORM package has a quiz, you need to verify that interaction data is being recorded correctly. This is where most LMS compatibility bugs live.

// tests/scorm-quiz.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Quiz Interactions', () => {

  test('should record interaction data when answering a quiz question', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');

    // Navigate to the quiz slide (adjust selectors to your content)
    // This will vary based on your authoring tool's output
    await frame.locator('[data-slide="quiz-1"]').click();

    // Select an answer
    await frame.locator('.answer-option').first().click();

    // Submit the answer
    await frame.locator('.submit-btn').click();

    // Verify interaction was recorded via SCORM API
    const interactions = await page.evaluate(() => {
      return window.API._calls.filter(c => 
        c.method === 'LMSSetValue' && 
        c.key.startsWith('cmi.interactions')
      );
    });

    // Should have at least one interaction recorded
    expect(interactions.length).toBeGreaterThan(0);

    // Verify the interaction has a valid type
    const interactionType = interactions.find(i => i.key.includes('.type'));
    expect(['choice', 'true-false', 'fill-in', 'matching', 'sequencing'])
      .toContain(interactionType?.value);
  });

  test('should calculate and commit score after quiz completion', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // ... navigate through quiz and answer all questions ...

    // After completing the quiz, check for score
    const scoreRaw = await page.evaluate(() => {
      return window.API._data['cmi.core.score.raw'];
    });

    const scoreMax = await page.evaluate(() => {
      return window.API._data['cmi.core.score.max'];
    });

    // Score should be a valid number
    expect(Number(scoreRaw)).not.toBeNaN();
    expect(Number(scoreMax)).toBeGreaterThan(0);
    expect(Number(scoreRaw)).toBeLessThanOrEqual(Number(scoreMax));

    // Verify LMSCommit was called after scoring
    const commitCalls = await page.evaluate(() => {
      return window.API._calls.filter(c => c.method === 'LMSCommit');
    });
    expect(commitCalls.length).toBeGreaterThan(0);
  });

});
Enter fullscreen mode Exit fullscreen mode

Step 5: Test Completion and Finish Sequence

The most common SCORM bug: content that never calls LMSFinish or sets lesson_status to completed/passed. This breaks LMS reporting.

// tests/scorm-completion.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Completion Flow', () => {

  test('should set lesson_status to completed/passed after full navigation', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');

    // Navigate through all slides (adjust to your content structure)
    const nextButton = frame.locator('.next-btn, [aria-label="Next"]');

    let hasNext = true;
    while (hasNext) {
      try {
        await nextButton.click({ timeout: 3000 });
        await page.waitForTimeout(500);
      } catch {
        hasNext = false;
      }
    }

    // Check final lesson status
    const finalStatus = await page.evaluate(() => {
      return window.API._data['cmi.core.lesson_status'];
    });

    expect(['completed', 'passed']).toContain(finalStatus);
  });

  test('should call LMSFinish on content exit', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // Navigate through content...
    // Then close or navigate away
    await page.evaluate(() => {
      // Simulate unload event
      window.dispatchEvent(new Event('beforeunload'));
    });

    await page.waitForTimeout(1000);

    const finishCalled = await page.evaluate(() => {
      return window.API._calls.some(c => c.method === 'LMSFinish');
    });

    expect(finishCalled).toBe(true);
  });

});
Enter fullscreen mode Exit fullscreen mode

Step 6: Generate a SCORM API Call Report

The real power of this approach is the ability to dump every SCORM API call for debugging. Add this utility:

// tests/helpers/scorm-report.js
async function generateSCORMReport(page) {
  const calls = await page.evaluate(() => window.API._calls);
  const data = await page.evaluate(() => window.API._data);

  console.log('\n=== SCORM API Call Log ===');
  calls.forEach((call, index) => {
    const time = new Date(call.timestamp).toISOString().split('T')[1];
    if (call.key) {
      console.log(`${index + 1}. [${time}] ${call.method}("${call.key}"${call.value ? ', "' + call.value + '"' : ''})`);
    } else {
      console.log(`${index + 1}. [${time}] ${call.method}()`);
    }
  });

  console.log('\n=== Final SCORM Data Model ===');
  Object.entries(data).forEach(([key, value]) => {
    console.log(`  ${key}: ${value}`);
  });

  return { calls, data };
}

module.exports = { generateSCORMReport };
Enter fullscreen mode Exit fullscreen mode

Use it in your tests:

const { generateSCORMReport } = require('./helpers/scorm-report');

test.afterEach(async ({ page }) => {
  await generateSCORMReport(page);
});
Enter fullscreen mode Exit fullscreen mode

Step 7: Run Across Multiple Browser Contexts (Simulating Different LMS Environments)

Different LMS platforms use different iframe embedding strategies. Test your content across configurations:

// playwright.config.js
module.exports = {
  projects: [
    {
      name: 'chromium-default',
      use: { browserName: 'chromium' },
    },
    {
      name: 'firefox',
      use: { browserName: 'firefox' },
    },
    {
      name: 'webkit-safari',
      use: { browserName: 'webkit' },
    },
  ],
};
Enter fullscreen mode Exit fullscreen mode

Run all:

npx playwright test --reporter=html
Enter fullscreen mode Exit fullscreen mode

This gives you a visual HTML report showing pass/fail across browsers — which maps roughly to how your content will behave across different LMS platforms that use different embedded browser engines.

What This Gets You

With this setup, you can now:

  • Catch SCORM API bugs before publishing — missing LMSInitialize, broken completion triggers, invalid interaction data
  • Debug LMS-specific failures by examining the full API call log
  • Run regression tests every time content is updated or the authoring tool ships a new version
  • Automate cross-browser validation that would take hours to do manually

In our team, this approach cut QA time for SCORM packages by roughly 70% and caught a category of bugs — specifically, race conditions in LMSSetValue calls during quiz scoring — that manual testing had missed for months.

Next Steps

From here, you can extend this framework to:

  • Test SCORM 2004 packages (replace window.API with window.API_1484_11)
  • Test xAPI (Tin Can) statements by intercepting fetch/XMLHttpRequest calls to the LRS endpoint
  • Integrate into CI/CD with GitHub Actions so every content build gets validated automatically
  • Add accessibility checks using @axe-core/playwright alongside SCORM validation

If you're building or testing e-learning content and you've run into weird LMS bugs, I'd love to hear about them — the edge cases in this space are wild.

I'm a senior software engineer with 11 years in e-learning technology. I write about the tools and techniques behind enterprise content authoring. Find me on LinkedIn.

Top comments (0)