DEV Community

Zoltan Toma
Zoltan Toma

Posted on • Originally published at zoltantoma.com on

Implementing Missing Integration Tests: Provisioners and Docker

#pester #testing #powershell #vagrant

Finishing What We Started

The Vagrant WSL2 provider had a test/integration_old/todos/ folder with unimplemented tests. Two big ones: test_provisioners.ps1 and test_docker.ps1. Both just had TODO comments saying what they should test.

We’d already migrated the working tests to Pester 5.x format. The old homegrown PowerShell scripts worked fine - they just weren’t using a framework. Now we needed to implement these missing tests using the same Pester patterns.

Provisioners: One Monolith, Five Tests

The examples/provisioners/ directory had one massive Vagrantfile testing shell, file, ansible, chef, and salt provisioners all at once. To test them individually, we needed to split them up.

Created a subfolder for each provisioner:

examples/provisioners/
├── shell/
│ └── Vagrantfile
├── file/
│ ├── Vagrantfile
│ └── test-file.txt
├── ansible/
│ ├── Vagrantfile
│ └── test-playbook.yml
├── chef/
│ ├── Vagrantfile
│ └── cookbooks/
└── salt/
    ├── Vagrantfile
    └── states/
Enter fullscreen mode Exit fullscreen mode

Each subfolder has its own self-contained Vagrantfile that demonstrates one provisioner. This makes them usable as examples and testable individually.

Provisioners.Tests.ps1

The Pester test defines separate Describe blocks for each provisioner:

# Provisioners.Tests.ps1
param([switch]$Full)

Describe "Vagrant WSL2 Provider - Shell Provisioner" {
    BeforeAll {
        $script:ExampleDir = Join-Path $PSScriptRoot "..\..\examples\provisioners\shell"
        $script:DistributionName = "vagrant-wsl2-shell-test"
        Push-Location $script:ExampleDir
        vagrant destroy -f 2>$null | Out-Null
    }

    AfterAll {
        vagrant destroy -f 2>$null | Out-Null
        Pop-Location
    }

    Context "When provisioning with shell scripts" {
        It "Should successfully run 'vagrant up --provider=wsl2'" {
            vagrant up --provider=wsl2
            $LASTEXITCODE | Should -Be 0
        }

        It "Should have created shell-test.txt file" {
            $result = vagrant ssh -c "test -f /home/vagrant/shell-test.txt && echo 'exists'" 2>&1
            $result -join "`n" | Should -Match "exists"
        }
    }
}

# Separate Describe blocks for file, ansible, chef, salt...
Enter fullscreen mode Exit fullscreen mode

Quick mode runs shell, file, and ansible (the working provisioners). Full mode adds chef and salt, which have known issues:

Describe "Vagrant WSL2 Provider - Chef Solo Provisioner" -Tag "KnownIssue" -Skip:(-not $Full) {
    # Tests that we expect to fail due to chef symlink issues
}

Describe "Vagrant WSL2 Provider - SaltStack Provisioner" -Tag "KnownIssue" -Skip:(-not $Full) {
    # Tests that we expect to fail due to bootstrap URL changes
}
Enter fullscreen mode Exit fullscreen mode

The -Skip:(-not $Full) means these only run when you pass -Full. Chef and Salt are documented to fail, but the tests are there to track the issues.

Rakefile tasks:

rake test_provisioners # Test shell, file, and ansible
rake test_provisioners_full # Test all including chef and salt
Enter fullscreen mode Exit fullscreen mode

Docker: 22 Distributions, Individual Machines

The Docker test validates that Docker installs and runs on different WSL distributions. The old examples/docker-test/Vagrantfile used a loop that auto-created all VMs:

# Old approach
DISTRIBUTIONS.each do |distro|
  config.vm.define "docker-#{distro}" do |node|
    # auto-created on vagrant up
  end
end
Enter fullscreen mode Exit fullscreen mode

This doesn’t work for selective testing. We refactored to individual machine definitions with autostart: false:

# New approach
config.vm.define "ubuntu2404", autostart: false do |node|
  node.vm.box = "Ubuntu-24.04"
  node.vm.provider "wsl2" do |wsl|
    wsl.distribution_name = "vagrant-docker-ubuntu2404"
    wsl.version = 2
    wsl.systemd = true
  end
  node.vm.provision "ansible_local" do |ansible|
    ansible.playbook = "provisioning/docker-debian.yml"
  end
end

config.vm.define "debian", autostart: false do |node|
  # ...
end

# ... 22 machines total
Enter fullscreen mode Exit fullscreen mode

Now you can run vagrant up ubuntu2404 to test just one, or the Pester test can selectively spin up what it needs.

The test uses the same quick/full pattern:

# Docker.Tests.ps1
param([switch]$Full)

BeforeAll {
    $script:QuickMachines = @(
        @{ Name = "ubuntu2404"; Distro = "vagrant-docker-ubuntu2404" },
        @{ Name = "debian"; Distro = "vagrant-docker-debian" },
        @{ Name = "almalinux8"; Distro = "vagrant-docker-almalinux8" }
    )

    $script:AllMachines = @(
        # All 22 distributions
    )

    $script:TestMachines = if ($Full) {
        Write-Host "Running FULL Docker test (all $($script:AllMachines.Count) distributions)"
        $script:AllMachines
    } else {
        Write-Host "Running QUICK Docker test ($($script:QuickMachines.Count) distributions)"
        $script:QuickMachines
    }
}

Describe "Vagrant WSL2 Provider - Docker Support" {
    Context "When testing Docker installation across distributions" {
        It "Should successfully bring up <Name> with Docker" -ForEach $script:TestMachines {
            vagrant up $Name --provider=wsl2
            $LASTEXITCODE | Should -Be 0
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Rakefile tasks:

rake test_docker # 3 distros
rake test_docker_full # 22 distros
Enter fullscreen mode Exit fullscreen mode

The Parameter Passing Bug

To pass the -Full parameter from Invoke-PesterTests.ps1 to the test scripts, we had broken code:

# This doesn't work - ContainerParameters doesn't exist in Pester 5.x
$config.Run.ContainerParameters = @{ Full = $true }
Enter fullscreen mode Exit fullscreen mode

The fix uses New-PesterContainer:

# Invoke-PesterTests.ps1
if ($Full) {
    $container = New-PesterContainer -Path $TestPath -Data @{ Full = $true }
    $config.Run.Container = $container
} else {
    $config.Run.Path = $TestPath
}
Enter fullscreen mode Exit fullscreen mode

This is the correct way to pass parameters to Pester 5.x test scripts. Works for AllDistributions, Provisioners, and Docker tests.

What’s Still Broken

The Docker test isn’t discovering tests yet:

Discovery found 0 tests in 211ms.
Enter fullscreen mode Exit fullscreen mode

The -ForEach $script:TestMachines parameter binding isn’t working. Something’s wrong with how we’re structuring the test data or the Describe block. That’s next session’s problem - we’re out of context.

What We Shipped

  1. Provisioners.Tests.ps1 - Tests all 5 provisioners (shell, file, ansible, chef, salt)

  2. Refactored provisioner examples - Individual Vagrantfiles for each provisioner

  3. Docker.Tests.ps1 - Tests Docker across distributions

  4. Refactored docker-test Vagrantfile - Individual machine definitions

  5. Fixed parameter passing - Invoke-PesterTests.ps1 now uses New-PesterContainer properly

  6. Updated Rakefile - New test tasks

  7. Updated CLAUDE.md - Documented the new tests and structure

The TODO tests are no longer TODO. They’re implemented - just need to fix the Docker test discovery issue.

Vagrant WSL2 Provider is an MIT-licensed open source project. Check it out on GitHub.

Top comments (0)