DEV Community

Zoltan Toma
Zoltan Toma

Posted on • Originally published at zoltantoma.com on

Testing 22 WSL Distributions Live on Twitch (13 Passed, 9 Failed)

#vagrant #wsl2 #testing #pester

The Question Nobody Asked

Which WSL distributions from the Windows Store actually work with the Vagrant WSL2 Provider?

I didn’t know. And I realized - that’s a problem.

The provider uses wsl --install --distribution <name> to download and install distributions directly from the Windows Store. But there are 22 distributions available, and I’d only tested Ubuntu-24.04.

Time to find out what works and what doesn’t.

Building the Test

The goal was simple: iterate through every distribution from wsl -l -o, try to create a Vagrant environment with it, and document what fails.

This is a sanity check, not a certification test. We expect failures. We want to know which distributions fail and why.

The Test Structure

Pester makes this straightforward:

BeforeAll {
    # WSL distributions available from Windows Store
    $script:WslDistributions = @(
        "AlmaLinux-8",
        "AlmaLinux-9",
        "Debian",
        "Ubuntu-24.04",
        # ... 22 total
    )

    $script:TestBaseDir = Join-Path $PSScriptRoot "..\..\examples\test-distributions"
    $script:TestResults = @{}
}

Describe "WSL Distribution Compatibility Tests" {
    Context "Testing each distribution" {
        It "Should test all distributions" {
            foreach ($distribution in $script:WslDistributions) {
                # Step 1: vagrant init
                vagrant init $distribution

                # Step 2: Configure WSL2 provider
                # (modify Vagrantfile to add wsl2 provider config)

                # Step 3: vagrant up --provider=wsl2
                $upOutput = vagrant up --provider=wsl2 2>&1

                # Step 4: vagrant destroy -f
                vagrant destroy -f

                # Track results
                $script:TestResults[$distribution] = @{
                    Success = ($LASTEXITCODE -eq 0)
                    Error = $errorMsg
                }
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Each distribution gets its own folder in examples/test-distributions/, runs through the full lifecycle (vagrant init, up, destroy), and gets cleaned up afterward.

The test runs to completion even if distributions fail. We capture all the errors and display a summary at the end.

Claude: Fun fact - we initially tried using Pester’s -TestCases with dynamic distribution lists, but Pester evaluates test cases at discovery time, not runtime. So we went with a single test that loops through distributions manually.

Streaming the Run

I ran this live on Twitch. Because why not? Watching 22 distributions install, provision, and clean up takes time. Might as well make it public.

Each test took anywhere from 30 seconds (for distributions that failed fast) to 2-3 minutes (for successful ones that needed to download, install, and configure systemd).

Total runtime: 17 minutes, 33 seconds (1053.89s reported by Pester).

The Results 

=== Distribution Compatibility Summary ===

Supported Distributions (13):
  [PASS] AlmaLinux-8
  [PASS] AlmaLinux-9
  [PASS] AlmaLinux-Kitten-10
  [PASS] AlmaLinux-10
  [PASS] Debian
  [PASS] FedoraLinux-43
  [PASS] FedoraLinux-42
  [PASS] SUSE-Linux-Enterprise-16.0
  [PASS] Ubuntu
  [PASS] Ubuntu-24.04
  [PASS] kali-linux
  [PASS] openSUSE-Tumbleweed
  [PASS] openSUSE-Leap-16.0

Unsupported/Failed Distributions (9):
  [FAIL] archlinux
         Error: tee: /etc/sudoers.d/vagrant: No such file or directory

  [FAIL] SUSE-Linux-Enterprise-15-SP7
         Error: chown: invalid group: 'vagrant:vagrant'

  [FAIL] Ubuntu-20.04
  [FAIL] OracleLinux_7_9
  [FAIL] OracleLinux_8_10
  [FAIL] OracleLinux_9_5
         Error: Distribution not found after installation completed

  [FAIL] Ubuntu-22.04
  [FAIL] openSUSE-Leap-15.6
  [FAIL] SUSE-Linux-Enterprise-15-SP6
         Error: Using legacy distribution registration

Total: 22 distributions tested
Success Rate: 13/22 (59.1%)
Enter fullscreen mode Exit fullscreen mode

What Worked

The AlmaLinux family (all 4 versions), Fedora, Debian, current Ubuntu versions, Kali Linux, and openSUSE’s latest releases all passed.

That’s 13 distributions spanning multiple Linux families. Not bad for a provider that started as “let’s just get Ubuntu working.”

What Failed - And Why

The failures break down into 4 categories:

1. Legacy Distribution Registration (4 distros)

Error: Using legacy distribution registration.
Consider using a tar based distribution instead.
Enter fullscreen mode Exit fullscreen mode

Ubuntu-22.04, openSUSE-Leap-15.6, and SUSE Linux Enterprise 15-SP6 all hit this. WSL is warning that these distributions use an older registration format.

The provider handles tar-based imports fine (that’s how we clone distributions for snapshots). But wsl --install for these specific distributions triggers the legacy path, and something in our provisioning flow doesn’t handle it.

2. Distribution Not Found After Installation (4 distros)

Error: Distribution 'Ubuntu-20.04' not found after installation completed.
Check if installation was successful.
Enter fullscreen mode Exit fullscreen mode

Ubuntu-20.04 and all three OracleLinux versions hit this. The wsl --install command completes, reports success, but when we check wsl -l -v to verify the distribution is there, it’s not listed.

This could be a timing issue (installation takes longer than we expect), a naming mismatch (we search for the exact distribution name but it registers under a different one), or these distributions genuinely fail to install.

3. Missing /etc/sudoers.d/ Directory (1 distro)

Error: tee: /etc/sudoers.d/vagrant: No such file or directory
Enter fullscreen mode Exit fullscreen mode

Arch Linux doesn’t have /etc/sudoers.d/ by default. Our provisioning script tries to add the vagrant user’s sudo config there and fails.

This is a simple fix - check if the directory exists first, or fall back to appending directly to /etc/sudoers.

4. Invalid Group Error (1 distro)

Error: chown: invalid group: 'vagrant:vagrant'
Enter fullscreen mode Exit fullscreen mode

SUSE Linux Enterprise 15-SP7 doesn’t let us create the vagrant:vagrant group the way we expect. Could be SELinux, could be enterprise defaults, needs investigation.

Bugs We Found Along the Way

While building and running this test, we discovered some provider bugs that needed documenting:

Documentation vs Reality

The CLAUDE.md file listed memory, cpus, gui_support, swap, and kernel_command_line as supported configuration options. Turns out they’re only defined in config.rb but never actually implemented in the provider.

Fixed the docs to be honest: “Additional configuration options are defined but not yet implemented.”

Naming Convention Bug

The provider uses distribution_name for the WSL distribution name, but doesn’t follow Vagrant’s standard naming conventions. If the box isn’t named correctly, vagrant destroy won’t remove it from WSL distributions.

This leaves orphaned WSL distributions sitting around after you think you’ve cleaned up. Added to the bug list.

These aren’t critical (the provider works), but they’re the kind of rough edges you find when you actually test things properly.

What This Means

We have a 59% success rate across all available WSL distributions. That’s higher than I expected, honestly.

The failures aren’t random. They fall into clear categories:

  • Legacy formats - Need to handle older distribution registration
  • Installation timing - Verify distributions exist before proceeding
  • Filesystem assumptions - Stop assuming /etc/sudoers.d/ exists
  • Group creation - Handle enterprise distributions differently

None of these are dealbreakers. They’re just edge cases we haven’t coded for yet.

Why This Matters

This test doesn’t just tell us what works. It tells us what to fix next.

Before this, I had no idea which distributions were broken or why. Now I have a reproducible test that documents exactly which features fail on which distributions.

More importantly: This test runs fast (17 minutes for 22 distributions). I can run it after every major change to catch regressions.

And when someone opens an issue saying “Provider doesn’t work with OracleLinux,” I can point to the test results and say: “Yeah, I know. Here’s the error. Want to help fix it?”

Claude: Also worth noting - we caught a documentation bug during this session. CLAUDE.md listed memory and cpus as supported configuration options, but they’re only defined in the config, not actually implemented. Fixed that too.

What’s Next

The test is in. The results are documented. Now comes the fun part: fixing the failures.

Priority targets:

  1. Arch Linux - Should be a 5-minute fix (check if directory exists)
  2. Legacy distributions - Figure out what “tar based distribution” actually means for our flow
  3. Oracle/Ubuntu older versions - Debug why installations report success but don’t register

The test stays in the repo. Future releases will include updated compatibility matrices.

And yeah, I’ll probably stream the fixes too.

Try It Yourself

The test is in the repo:

git clone https://github.com/LeeShan87/vagrant-wsl2-provider
cd vagrant-wsl2-provider
rake test_all_distributions
Enter fullscreen mode Exit fullscreen mode

Note: This will download and test 22 WSL distributions. Budget ~20 minutes and some bandwidth.

Results will vary based on your Windows version, WSL version, and what distributions you already have installed.

If you find different results, open an issue. This is how we make the provider better.

Lessons from this session:

  • Sanity checks are worth the time investment
  • Live streaming makes long test runs more tolerable
  • 59% success rate beats 0% knowledge
  • Document what’s broken before trying to fix it
  • Pester makes it easy to iterate through test cases cleanly

Next post: Actually fixing the failures. Stay tuned.

Top comments (0)