DEV Community

Cover image for Connect 10 formats to one artifact service: Maven, Docker, npm, PyPI, and Helm
Wen Ping
Wen Ping

Posted on Originally published at anguskit.com

Connect 10 formats to one artifact service: Maven, Docker, npm, PyPI, and Helm

#devops #docker #softwaredevelopment #tutorial

A team often runs Java, frontend, Python, .NET, and containers at the same time. Instead of keeping a separate private registry for each stack, manage the artifacts in one AngusRepo service.

This guide covers Maven, Docker, npm, PyPI, NuGet, Helm, Go, APT, YUM, and Raw. For each format you do three things: authenticate, publish one artifact, then pull it back to verify.

A common mix-up: “one artifact service” means one AngusRepo deployment, not one repository that accepts every format. Create a HOSTED repository for each format, such as maven-releases or docker-releases. Repository names and hostnames below are examples. Use the Setup tab on the repository detail page.

AngusRepo artifact workflow

Three minutes of setup

If you use only one stack, read only that section. The first time, verify in the order publish → pull. A successful upload does not prove the client can download.

1. Account, token, and repository

Sign in

Open the AngusRepo console. On a self-hosted instance, use the console URL from your administrator.

Create a token

Go to User Settings → My Tokens and create a token, for example local-macbook. The token is shown only once. Save it immediately. If you lose it, revoke it and create another.

Create a HOSTED repository

Under Repositories, create a HOSTED repository for the format you need, and confirm your account can upload to it.

Copy the setup values

Open the repository Setup tab. Use the repository name, URL, distribution, and component shown there.

Put the login name and token in environment variables for the commands below. export lasts only for the current terminal, so keep running commands in that same window.

export REPO_USER="your-username"
export REPO_TOKEN="your_access_token"
Enter fullscreen mode Exit fullscreen mode

REPO_USER is the login name, not the token name. For Maven, Docker, npm, NuGet, and the other protocols, the token is the password. Every --password, --api-key, and _authToken in this guide takes REPO_TOKEN, not your login password.

Security: A token written into a URL lands in shell history or local config. Use these examples only on a machine you control. On a shared machine or in CI, use a secret store, and remove the temporary config after you verify.

2. Address cheat sheet

Format Pull address or identifier Publish
Maven https://host/maven/<repo>/ Same URL, mvn deploy
Docker host/<repo>/<image>:<tag> Same image name, docker push
npm https://host/npm/<repo>/ npm publish
PyPI /pypi/<repo>/simple/ /pypi/<repo>/legacy/
NuGet /nuget/<repo>/v3/index.json Same source, --api-key is the token
Helm /helm/<repo>/ or oci://host/helm/<repo> HTTP upload or helm push
Go /go/<repo>/, set GOPROXY Console or management API
APT /apt/<repo>/ Upload to pool/<filename>
YUM /yum/<repo>/ POST .../upload
Raw /raw/<repo>/<path> Same path, curl -T

The rest of this guide uses repo.anguskit.com as the host. On a self-hosted instance, replace it with your address.

Maven

Run these commands from a project root that already has a pom.xml. The repository id in settings.xml and pom.xml must match exactly.

Authenticate

Save the following as settings.xml in the project root. If you already have ~/.m2/settings.xml, merge the <server> block into it. Do not overwrite the existing file.

<settings>
  <servers>
    <server>
      <id>angusrepo-maven-releases</id>
      <username>${env.REPO_USER}</username>
      <password>${env.REPO_TOKEN}</password>
    </server>
  </servers>
</settings>
Enter fullscreen mode Exit fullscreen mode

Publish

Add the publish URL to pom.xml, then deploy. A successful run logs BUILD SUCCESS.

<distributionManagement>
  <repository>
    <id>angusrepo-maven-releases</id>
    <url>https://repo.anguskit.com/maven/maven-releases</url>
  </repository>
</distributionManagement>
Enter fullscreen mode Exit fullscreen mode 
mvn -s settings.xml clean deploy
Enter fullscreen mode Exit fullscreen mode

Verify the pull

Add the same repository to another project, then request the groupId, artifactId, and version you just published.

<repositories>
  <repository>
    <id>angusrepo-maven-releases</id>
    <url>https://repo.anguskit.com/maven/maven-releases</url>
  </repository>
</repositories>
Enter fullscreen mode Exit fullscreen mode 
mvn -s settings.xml dependency:get \
  -Dartifact=yourGroupId:yourArtifactId:yourVersion
Enter fullscreen mode Exit fullscreen mode

Docker

Run this from a directory that contains a Dockerfile. The image name is host/repo/image:tag. Do not insert /v2 yourself.

Authenticate 

echo "$REPO_TOKEN" | docker login repo.anguskit.com \
  -u "$REPO_USER" --password-stdin
Enter fullscreen mode Exit fullscreen mode

Continue after you see Login Succeeded.

Publish 

docker build -t myapp:1.0.0 .
docker tag myapp:1.0.0 repo.anguskit.com/docker-releases/myapp:1.0.0
docker push repo.anguskit.com/docker-releases/myapp:1.0.0
Enter fullscreen mode Exit fullscreen mode

Verify the pull 

docker pull repo.anguskit.com/docker-releases/myapp:1.0.0
Enter fullscreen mode Exit fullscreen mode

npm

Run this from a directory that contains package.json. Keep the trailing / on the registry URL, and leave ${NPM_TOKEN} unchanged in .npmrc.

Authenticate

Create or update .npmrc in the project root:

registry=https://repo.anguskit.com/npm/npm-releases/
//repo.anguskit.com/npm/npm-releases/:_authToken=${NPM_TOKEN}
always-auth=true
Enter fullscreen mode Exit fullscreen mode

Then, in the same terminal:

export NPM_TOKEN="$REPO_TOKEN"
Enter fullscreen mode Exit fullscreen mode

Publish

name and version in package.json are the package you publish. If that version already exists, bump version first.

npm publish
Enter fullscreen mode Exit fullscreen mode

Verify the pull 

npm install your-package
Enter fullscreen mode Exit fullscreen mode

For an @scope package, also add this to .npmrc:

@scope:registry=https://repo.anguskit.com/npm/npm-releases/
Enter fullscreen mode Exit fullscreen mode

PyPI

Run this from a directory that contains pyproject.toml. Upload to /legacy/ and install from /simple/. The package name is name in pyproject.toml.

Prepare the environment 

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install build twine
Enter fullscreen mode Exit fullscreen mode

Publish

dist/ is created by the build. You do not create it by hand.

python3 -m build
twine upload \
  --repository-url https://repo.anguskit.com/pypi/pypi-releases/legacy/ \
  -u "$REPO_USER" -p "$REPO_TOKEN" dist/*
Enter fullscreen mode Exit fullscreen mode

Verify the pull 

pip install your-package \
  --index-url "https://${REPO_USER}:${REPO_TOKEN}@repo.anguskit.com/pypi/pypi-releases/simple/"
Enter fullscreen mode Exit fullscreen mode

NuGet

Run this from a directory that contains a .csproj. The NuGet source URL must end with /v3/index.json.

Authenticate

This command stores the credentials in the local NuGet config.

dotnet nuget add source https://repo.anguskit.com/nuget/nuget-releases/v3/index.json \
  --name angusrepo \
  --username "$REPO_USER" \
  --password "$REPO_TOKEN" \
  --store-rest-token
Enter fullscreen mode Exit fullscreen mode

Publish 

dotnet pack -c Release
dotnet nuget push bin/Release/Your.Package.1.0.0.nupkg \
  --source https://repo.anguskit.com/nuget/nuget-releases/v3/index.json \
  --api-key "$REPO_TOKEN"
Enter fullscreen mode Exit fullscreen mode

Use the .nupkg file name that dotnet pack actually printed.

Verify the pull

Run this in another existing project:

dotnet add package Your.Package --version 1.0.0 --source angusrepo
dotnet restore
Enter fullscreen mode Exit fullscreen mode

Helm

Helm speaks HTTP and OCI. Pick the one shown on the repository Setup tab: helm repo add means HTTP, oci:// means OCI. Do not mix the two flows. The current directory should contain a chart with Chart.yaml.

HTTP 

helm repo add angusrepo https://repo.anguskit.com/helm/helm-releases \
  --username "$REPO_USER" --password "$REPO_TOKEN"

helm package ./mychart
curl -u "$REPO_USER:$REPO_TOKEN" \
  -F "chart=@mychart-1.0.0.tgz" \
  "https://repo.anguskit.com/helm/helm-releases/api/charts"

helm repo update
helm install my-release angusrepo/mychart
Enter fullscreen mode Exit fullscreen mode

Replace mychart-1.0.0.tgz with the file name helm package printed.

OCI 

echo "$REPO_TOKEN" | helm registry login repo.anguskit.com \
  -u "$REPO_USER" --password-stdin

helm package ./mychart
helm push mychart-1.0.0.tgz oci://repo.anguskit.com/helm/helm-releases
helm install my-release oci://repo.anguskit.com/helm/helm-releases/mychart --version 1.0.0
Enter fullscreen mode Exit fullscreen mode

Go

go get only downloads modules. Upload from the console or the management API. Do not send the upload to repo.anguskit.com/api. The module path comes from go.mod, and the version must start with v, for example v1.0.0.

Configure the pull

A private module needs GOPROXY and the module domain. Replace GONOSUMDB with your module domain, or checksum verification can fail.

export GOPROXY="https://${REPO_USER}:${REPO_TOKEN}@repo.anguskit.com/go/go-releases"
export GONOSUMDB=example.com
Enter fullscreen mode Exit fullscreen mode

Publish

Run this from the module root. The top directory inside the zip must be module-path@version/ and must contain go.mod. The cloud API base is below. On a self-hosted instance, ask your administrator for the management API base.

export API_BASE="https://bj-c1-prod-apis.anguskit.com/repo"
export MODULE="$(go list -m)"
export VERSION="v1.0.0"
export PREFIX="${MODULE}@${VERSION}"
STAGE="$(mktemp -d)"
mkdir -p "${STAGE}/${PREFIX}"
cp go.mod *.go "${STAGE}/${PREFIX}/"
(cd "${STAGE}" && zip -qr "${OLDPWD}/module-${VERSION}.zip" "${PREFIX}")
rm -rf "${STAGE}"

curl -u "$REPO_USER:$REPO_TOKEN" \
  -F "repositoryName=go-releases" \
  -F "version=${VERSION}" \
  -F "modulePath=${MODULE}" \
  -F "zip=@./module-${VERSION}.zip" \
  "$API_BASE/api/v1/go/upload"
Enter fullscreen mode Exit fullscreen mode

You can also open that Go repository and upload the zip from the upload page.

Verify the pull 

go get example.com/go-demo@v1.0.0
Enter fullscreen mode Exit fullscreen mode

APT

Start from a .deb you have already built. Install by the package name inside the archive, not by the .deb file name. stable and main must match the repository distribution and component. If Setup shows different values, use those.

Publish 

curl -u "$REPO_USER:$REPO_TOKEN" -T ./my-package_1.0.0_amd64.deb \
  "https://repo.anguskit.com/apt/apt-releases/pool/my-package_1.0.0_amd64.deb"
Enter fullscreen mode Exit fullscreen mode

Verify the pull

The index can take a few seconds after upload. Replace my-package with the package name inside the deb:

echo "deb [trusted=yes] https://${REPO_USER}:${REPO_TOKEN}@repo.anguskit.com/apt/apt-releases stable main" \
  | sudo tee /etc/apt/sources.list.d/angusrepo.list
sudo apt-get update && sudo apt-get install my-package
Enter fullscreen mode Exit fullscreen mode

YUM

Start from an .rpm you have already built. Install by the RPM package name, not by the file name.

Publish

The upload must use the form field file:

curl -u "$REPO_USER:$REPO_TOKEN" \
  -F "file=@./my-package-1.0.0-1.noarch.rpm" \
  "https://repo.anguskit.com/yum/yum-releases/upload"
Enter fullscreen mode Exit fullscreen mode

Verify the pull

The command below writes /etc/yum.repos.d/angusrepo.repo. If the package is missing right after upload, wait and try again.

sudo tee /etc/yum.repos.d/angusrepo.repo >/dev/null <<EOF
[angusrepo]
name=AngusRepo YUM
baseurl=https://${REPO_USER}:${REPO_TOKEN}@repo.anguskit.com/yum/yum-releases/
enabled=1
gpgcheck=0
EOF
sudo dnf clean all && sudo dnf install my-package
Enter fullscreen mode Exit fullscreen mode

Raw

A Raw repository stores files that do not belong to a package format. Pick a clear path, such as project/version/filename. Upload and download must use the same URL.

Publish 

curl -u "$REPO_USER:$REPO_TOKEN" -T ./my-app-1.0.0-linux-amd64.tar.gz \
  "https://repo.anguskit.com/raw/raw-releases/my-app/1.0.0/my-app-1.0.0-linux-amd64.tar.gz"
Enter fullscreen mode Exit fullscreen mode

Uploading to the same path again overwrites the file.

Verify the pull

Download from another directory so the file does not collide with the local source:

mkdir -p /tmp/raw-check && cd /tmp/raw-check
curl -u "$REPO_USER:$REPO_TOKEN" -O \
  "https://repo.anguskit.com/raw/raw-releases/my-app/1.0.0/my-app-1.0.0-linux-amd64.tar.gz"
Enter fullscreen mode Exit fullscreen mode

Troubleshooting

What you see Check first
Publish returns 403 The account behind the token has write access to that repository.
Docker push fails The image name must not contain /v2. docker build must succeed locally first.
npm returns 401 or 404 The registry URL and the _authToken path match, including the trailing /. NPM_TOKEN is set.
PyPI upload or install fails Upload goes to /legacy/, install uses /simple/. Run python3 -m build first.
NuGet restore fails The source URL ends with /v3/index.json, and the local credential store has the right token.
Helm returns 401 Do not mix the HTTP and OCI flows. Run helm registry login before an OCI push.
APT or YUM cannot find a package you just uploaded Wait for the index, then refresh the cache. Install by the name inside the package, not the file name.
Go upload fails The top directory in the zip is module-path@version/, and the upload URL is the management API.

How authentication works

Protocol paths such as /maven/, /npm/, and /v2/ use Basic Auth: the username is the login name, and the password is the access token. The management REST API uses a Bearer token where the API reference says so. If authentication fails, check that you did not swap the two.

Originally published at anguskit.com.

Top comments (1)

Collapse
 
suppdevbot profile image
DEV SUPPORTS • 

You need to verify your account.
Enter fullscreen mode Exit fullscreen mode

tr.ee/dev-to