I've already written about this in a few places but I've finally done an initial release of PyPyInstaller. This is one of the projects I mentioned in my last distraction post.
Motivation
One of the articles I wanted to write was for managing multiple installations of Python. Unfortunately, I noticed that PyPy seemed to be left out in terms of Windows support. There wasn't a solid option to both add it to PATH and add the appropriate PEP 514 entries to be listed in the Python Launcher. I decided this would be an interesting project to work on as a hobby type deal.
Why Powershell
There are many Windows solutions that run off a command line or GUI executable. The problem is you can't really see what's going on. Even if the source code is available you don't really know it's the code in that executable unless you build it yourself. Making this a Powershell module means everything is a script and the user can easily see every command that's being run.
Another reason is that using commandlets allows for flexibility in how installations are done. For example, there's a command for finding the latest installation which can be piped into the command to install a PyPy version:
> Find-PyPyLatest -PythonSeries "3.9", "3.10" | Install-PyPy
But what if you really need that specific version? You can just run Install-PyPy by itself with the appropriate option:
> Install-PyPy -PythonVersions "3.8.16"
I think it helps strike a balance between just get it done and still allow some more controlled uses.
Code Layout
The layout of the code looks like this:
├─.github
│ └─workflows
├─.vscode
├─src
│ └─PyPyInstaller
│ ├─Functions
│ │ ├─Private
│ │ └─Public
│ └─Wrappers
└─test
└─fixtures
src contains the source of the module itself. Under this is a Functions directory containing the actual ps1 files. This directory is also where code coverage looks to ensure tests are built against everything. Functions are then broken up into Public and Private. Private functions are meant to support the Public versions. It's then the Public versions that are exposed to the user. test contain the Pester tests used to ensure the code works.
Test Test Test...
The is one of the first personal projects where I really went all out on testing. Part of it was because PyPyInstaller touches PATH and registry keys. At the moment the code coverage is at 99% (I still have a big question mark on how to deal with the last %):
Tests completed in 3.35s
Tests Passed: 37, Failed: 0, Skipped: 0 NotRun: 0
Processing code coverage result.
Covered 99.46% / 90%. 185 analyzed Commands in 8 Files.
Pester is what's used for most Powershell testing. While it has some pretty amazing features, it does have issues with .NET class method mocking and sometimes scope can be weird. Here's a simple example:
BeforeAll {
$PackageRoot = "$PSScriptRoot\..\src\PyPyInstaller\"
Remove-Module PyPyInstaller
Import-Module $PackageRoot
. "$PackageRoot\Functions\Private\Utility.ps1"
}
InModuleScope PyPyInstaller {
Describe "Update-PyPyMirror" {
BeforeAll {
$TestRootPath = "$env:temp"
Mock -CommandName Read-PyPyInstallerConfig -MockWith { return @{ RootPath = $TestRootPath } }
Mock -CommandName Out-File -ParameterFilter { $FilePath -eq "$TestRootPath\versions.json" } -MockWith { return $null }
Mock -CommandName Invoke-WebRequest -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -MockWith { return @{ Content = Get-Content "$PSScriptRoot\fixtures\test_versions.json" } }
}
It "Passes Downloads Mirror JSON" {
Update-PyPyMirror
Assert-MockCalled Invoke-WebRequest -Exactly -Times 1 -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -Scope It
Assert-MockCalled Out-File -ParameterFilter { $FilePath -eq "$TestRootPath\versions.json" }
}
}
}
First BeforeAll imports the code that will be tested. Then InModuleScope is used to deal with the scope oddities mentioned. Describe has the function being mentioned, with an additional BeforeAll declaration. This is where the mocking happens in the this particular test case. As an example for one of the mocks:
Mock -CommandName Invoke-WebRequest -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -MockWith { return @{ Content = Get-Content "$PSScriptRoot\fixtures\test_versions.json" } }
This will mock the Invoke-Request command when the -Uri argument is the PyPy mirror versions.json file. Instead of returning that it will return a test version of the file with a specific layout for my tests. When the code is run, it asserts that my mocked version of Invoke-WebRequest was called instead of the actual command:
Assert-MockCalled Invoke-WebRequest -Exactly -Times 1
This makes sure I'm not on the bad side of PyPy mirror infra by hitting their versions file every time I run the test cases! It can also test the registry:
Context "Registry Keys Don't Exist" {
BeforeAll {
New-Item -Path "TestRegistry:\Software\Python" -Force
Mock -CommandName Test-Path -MockWith { return $false }
}
It "Passes Adds Entries To Registry" {
Set-PyPyLauncherEntry -PythonVersion "3.10.12" -PyPyVersion "7.3.12" -InstallPath "C:\PyPy" -RegistryDrive "TestRegistry"
$CoreInfo = Get-ItemProperty "TestRegistry:\Software\Python\PyPyInstaller\3.10"
$CoreInfo.DisplayName | Should -Be "PyPy 3.10.12"
$CoreInfo.SupportUrl | Should -Be "https://github.com/cwgem/pypy-powershell-install"
$CoreInfo.Version | Should -Be "7.3.12"
$CoreInfo.SysVersion | Should -Be "3.10.12"
$CoreInfo.SysArchitecture | Should -Be "64bit"
$InstallPathInfo = Get-ItemProperty "TestRegistry:\Software\Python\PyPyInstaller\3.10\InstallPath"
$InstallPathInfo.'(default)' | Should -Be "C:\PyPy"
$InstallPathInfo.ExecutablePath | Should -Be "C:\PyPy\pypy.exe"
$InstallPathInfo.WindowedExecutablePath | Should -Be "C:\PyPy\pypyw.exe"
}
}
It does so by creating a TestRegistry: entry that can be used in place of the actual registry. This is something I then pass in to the command that deals with the registry as a path, and then assert against the various registry keys.
Interesting Powershell
Of particular interest was how Powershell was easily able to handle pipeline input:
function Find-PyPyLatest {
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline)]
[string[]]
$PythonSeries
)
Here's an example which accepts the value of PythonSeries as a pipeline argument (I can also just pass it in as the -PythonSeries argument):
> "3.9", "3.10" | Find-PyPyLatest
pypy_version : 7.3.12
python_version : 3.9.17
stable : True
latest_pypy : True
date : 2023-06-16
files : {@{filename=pypy3.9-v7.3.12-aarch64.tar.bz2; arch=aarch64; platform=linux; download_url=https://downlo
ads.python.org/pypy/pypy3.9-v7.3.12-aarch64.tar.bz2}, @{filename=pypy3.9-v7.3.12-linux32.tar.bz2; arch
=i686; platform=linux; download_url=https://downloads.python.org/pypy/pypy3.9-v7.3.12-linux32.tar.bz2}
, @{filename=pypy3.9-v7.3.12-linux64.tar.bz2; arch=x64; platform=linux; download_url=https://downloads
.python.org/pypy/pypy3.9-v7.3.12-linux64.tar.bz2}, @{filename=pypy3.9-v7.3.12-macos_x86_64.tar.bz2; ar
ch=x64; platform=darwin; download_url=https://downloads.python.org/pypy/pypy3.9-v7.3.12-macos_x86_64.t
ar.bz2}…}
pypy_version : 7.3.12
python_version : 3.10.12
stable : True
latest_pypy : True
date : 2023-06-16
files : {@{filename=pypy3.10-v7.3.12-aarch64.tar.bz2; arch=aarch64; platform=linux; download_url=https://downl
oads.python.org/pypy/pypy3.10-v7.3.12-aarch64.tar.bz2}, @{filename=pypy3.10-v7.3.12-linux32.tar.bz2; a
rch=i686; platform=linux; download_url=https://downloads.python.org/pypy/pypy3.10-v7.3.12-linux32.tar.
bz2}, @{filename=pypy3.10-v7.3.12-linux64.tar.bz2; arch=x64; platform=linux; download_url=https://down
loads.python.org/pypy/pypy3.10-v7.3.12-linux64.tar.bz2}, @{filename=pypy3.10-v7.3.12-macos_x86_64.tar.
bz2; arch=x64; platform=darwin; download_url=https://downloads.python.org/pypy/pypy3.10-v7.3.12-macos_
x86_64.tar.bz2}…}
Objects can also be accepted via pipeline:
function Install-PyPy {
[CmdletBinding()]
param (
[Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)]
[string[]]
[Alias('python_version')]
$PythonVersions,
This will bind the python_version property of any passed in objects to the PythonVersions argument. It's is quite a blessing from someone who has had to deal with the *NIX version of pipe handling... Another oddity I found was adding to PATH:
function Get-PyPyPathEnvironmentVariable {
return [System.Environment]::GetEnvironmentVariable('Path', 'User')
}
function Set-PyPyPathEnvironmentVariable {
param (
[Parameter()]
[String]
$NewPath
)
[System.Environment]::SetEnvironmentVariable('Path', $NewPath, 'User')
}
These are wrappers around .NET class methods to make Pester play nice as it can't mock those very well (instead it mocks the commands that wrap them). The issue I found was that normal PATH registration in Powershell doesn't fire off the WM_SETTINGCHANGE event. So instead the .NET method is used which does fire off the event. This allows for a simple restart of a terminal session to pick up the changes.
What's Next
This is far from complete and there's still a lot that can be done:
- Code layout consolidation
- Test case consolidation
- List/Remove/Update PyPyInstaller Installations
- Developer documentation
- GitHub workflows for testing and maybe a coverage badge
Can't wait to see where this project leads. Even if it doesn't go places I learned quite a lot about Powershell!
Top comments (0)