Mirroring Composer dependencies with Packeton

Uladzimir Tsykun — Sun, 05 Mar 2023 17:46:34 +0000

Introduce

Packeton is an open-source private Composer repository based on Packagist.org and Satis, distributed under the MIT license. Back in 2018, there was no free alternative to the Private Packagist packagist.com with the necessary functionality, but the Satis functionality is very limited. I like the UI of the Packagist.org thought it would be cool to use it for my own private Packagist, so I forked it to make it available for everyone to reuse. The source code is available on GitHub

Mirror dependencies

Mirroring dependencies can be useful for several scenarios:

Speed-up downloading dependencies
Sharing access to a private Composer repository, such as Magento, with your team and customers.
Preventing Dependency Confusion attacks.

Packeton will create a local copy of a remote repository, allowing you to download dependencies and metadata from the local copy instead of the remote composer repository.

Installation

To get started with Packeton, you'll first need to install the application. You can do this using Docker by creating a docker-compose.yml file with the following configuration:

version: '3.6'

services:
    packeton:
        image: packeton/packeton:latest
        container_name: packeton
        hostname: packeton
        environment:
            ADMIN_USER: admin
            ADMIN_PASSWORD: 123456 # Default password
            ADMIN_EMAIL: admin@example.com
            TRUSTED_PROXIES: 172.16.0.0/12
            # Default SQLite
            # DATABASE_URL: "mysql://app:!ChangeMe!@127.0.0.1:3306/app?serverVersion=8&charset=utf8mb4"
        ports:
            - '127.0.0.1:8081:80'
        volumes:
            - .docker:/data

Run the following command to start the container: docker-compose up -d The application will then be accessible at http://127.0.0.1:8081.

Next, you may want to install an Nginx reverse proxy to make the application available at a custom domain name. Here's an example configuration for this:

server {
    listen *:443 http2;

    server_name packages.example.org;

    ssl_certificate /etc/nginx/ssl/example.org.crt;
    ssl_certificate_key /etc/nginx/ssl/example.org.key;
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4';

    ssl_protocols TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_session_timeout  5m;

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 16k;
    gzip_http_version 1.1;
    gzip_min_length 2048;
    gzip_types text/css application/javascript text/javascript application/json;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:8081/;
    }
}

Now you can login on packages.example.org

Configure Composer proxy

To configure the Composer proxy, you need to add it on configuration level. For example, for a docker installation, this is a file config.yaml in a volume. For source code installation put config file with any name the directory {project dir}/config/packages.

packeton:
    mirrors:
        orocrm:
            url: https://satis.oroinc.com/

            # SSH keys for create ZIP archive from Git source.
            git_ssh_keys:
                git@github.com:oroinc: '/var/www/.ssh/private_key'
#            logo: 'https://example.com/logo.png'
#            http_basic:
#                username: user
#                password: pass

            # Allow public access, default false
#            public_access: true
            # default false
#            sync_lazy: true
#            enable_dist_mirror: false

            # Additional restriction, but you can restrict it in UI
#            available_package_patterns:
#                - 'vend1/*'
#            available_packages:
#                - 'pack1/name1' # but you can restrict it in UI
            # JSON. auth.json to pass composer opts.
#            composer_auth: '{"auth.json..."}'

            # default auto.
            sync_interval: ~
            # Console info message
            info_cmd_message: ~

Now you will need to restart docker container to apply configuration.

The proxy information now will be shown by url https://packages.example.com/proxies/orocrm
Synchronization of metadata is launched by cron, and in order to update the data after creating a proxy, you need to run the update through the console or through UI. If everything happened without errors, then you will see the result of the synchronization as below

To use this proxy you need to add this line to your composer.json.

  "repositories": [
    {
      "type": "composer",
      "url": "https://packages.example.org/mirror/orocrm"
    }
  ]

By default, this resource is available only with aurization, so you need enable auth for composer composer config --global --auth http-basic.packages.example.org {username} {apitoken} Where apitoken you can see on the profile page.

Dependencies Approval.

In Packeton by default dependencies are automatically enabled, when you run the first time composer update. In the example above, I used the repository https://satis.oroinc.com that has third-party dependencies. Example of the application that uses this repo.

The root file packages.json does not contains restrictions by available_package_patterns. Depending on the satis settings, if require-all is enabled, the third party vendor can publish a package under any vendor name. For example symfony/process. So attacker may replace your dependencies, because if a package name is found in this custom package repository, only those versions are loaded at all.

To select specific packages that you use in a project and remove those you don't need, you can turn on strict mode. Go to proxy settings and select the "Strict mode"

Then, using the Mass mirror action, you can enable and approve your packages. Put the composer info output as shown below.

Depending on the User-Agent the metadata is available in two formats for Composer 2 and 1 at once. However, the original repository only provide the metadata for Composer 1.

{
    "providers-lazy-url": "/mirror/orocrm/pkg/%package%.json",
    "mirrors": [
        {
            "dist-url": "/mirror/orocrm/zipball/%package%/%version%/%reference%.%type%",
            "preferred": true
        }
    ],
    "metadata-url": "/mirror/orocrm/p2/%package%.json",
    "available-packages": [
        "oro/platform-enterprise",
        "oro/crm-enterprise",
        "oro/api-doc-bundle",
        "oro/crm-pro-ldap-bundle"
    ]
}

If User-Agent header is match with Composer 1, then all packages will be merge into one snapshot includes for spend up Composer 1 downloads.

When the User-Agent header matches to Composer1, Packeton will merge all packages into one snapshot includes to speed up Composer 1 downloads. Because Composer 1 is not able to perform parallel downloads, so by merging all packages into one snapshot, Packeton can reduce the number of requests Composer 1.

{
    "includes": {
        "include-packeton/all$6150da03358e44ebc3b99713c643e98dc06a221e.json": {
            "sha1": "6150da03358e44ebc3b99713c643e98dc06a221e"
        }
    },
    "mirrors": [
        {
            "dist-url": "/mirror/orocrm/zipball/%package%/%version%/%reference%.%type%",
            "preferred": true
        }
    ]
}