Distributing PyPI Packages using API Tokens in TravisCI

Oscar Cortez — Sun, 11 Aug 2019 18:06:29 +0000

The PyPI package index is one of the tools that makes Python so powerful, with just a simple command, you get access to thousands of cool libraries, ready for you to use in your project.

Writing Python packages and deploying them to PyPI is one way for sharing libraries to the open-source community. For beginners, submitting a package to PyPI is not an easy task, but the community is working hard, for make this process more easier, you can read more about packaging, following the official Python Packaging Guide.

In this post, I'm going to skip some things/steps about packaging, and guide you through the process of using the newest API Tokens feature in PyPI, for your continuous integration and deployment (CI/CD).

For this we'll use the PyPI Test instance, which is a better option for doing initial test in your packages, like: testing if the docs render correctly, or if the classifiers in your setup.py are correct, for example.

Using Travis CI

Travis CI can automatically release your Python packages to PyPI after a successful build. For a minimal configuration, you can use the following code in your .travis.yml file:

deploy:
  provider: pypi
  user: "Your username"
  password: "Your password"

However, as you may notice, this would expose your PyPI password to the world, and that is a bad idea. What Travis recommend is to encrypt the password or save the credentials as Travis Environment Variables, you can read more about it on Travis deployment docs.

Get the PyPI tokens

So, the first thing to do, is go to https://test.pypi.org/ and create an account (if you don't have one) or login, after that, we're gonna need our API Token. How to get it? Go to your PyPI account settings and select "Add API token". When you create an API token, you choose its scope: you can create a token that can be used to upload to all the projects you maintain or own, or you can limit its scope to just one project.

How to use the tokens

API tokens provide an alternative way (instead of username and password) to authenticate when uploading packages to PyPI.

These API tokens can only be used to upload packages to PyPI, and not to log in more generally, since a thief who copies the token won't also gain the ability to delete the project, delete old releases or add/removed collaborators.

As you can see on the following screenshot, I have successfully manually deployed the first version of the demo project on PyPI:

Now we need to enable Travis for this project, so go to https://travis-ci.org/account/repositories and enable it, after that we have to change the following things in our .travis.yml file:

Set the username to __token__.
Set the password to the token value, including the pypi- prefix.

For more extra security, we're going to store the token as an environment variable, go to https://travis-ci.org///settings and now our .travis.yml file will looks like this:

deploy:
  provider: pypi
  user: __token__
  password: $TEST_PYPI_TOKEN
  server: https://test.pypi.org/legacy/
  distributions: "sdist bdist_wheel"
  on:
    branch: staging
    condition: $TRAVIS_PYTHON_VERSION = "3.6"

And now, every time we push code to the staging branch, Travis will trigger an automatic deploy to the PyPI test instance. do a small changes in the codebase, and change the setup.py version number, then push to the staging branch, Travis will trigger a build and if everything works deploy the new version to PyPI.

We can inspect the build logs to see if the deploy process started correctly:

And voilà, on the line 271 we see the deploy process started, if you expand the line you could see the following logs:

That means that our new version was uploaded correctly and now we can see it available in PyPI, automatically deployed with Travis using the new API tokens:

Conclusion

This is a big step forward for the Python Package Index, increasing the security with new multiple features, and reducing all the characteristics that make it hard to work on, for the Python community, thanks to the Packaging Working Group and the grant from the Open Technology Fund.

You can read more about API tokens on the official PSF Blog or help to test this beta feature. If do you want to read the Spanish version of this post, you can find it on my blog. Also the source code is available on GitHub.

And that's it! From here you can automate the process for releasing new package versions, so let's go, publish something to PyPI 🐍.

Thanks to @Darwing1210 and @guadamzjj for reviewing this document.

With ❤️ from 🇳🇮.

DEV Community: Oscar Cortez

Distributing PyPI Packages using API Tokens in TravisCI

Using Travis CI

Get the PyPI tokens

How to use the tokens

Conclusion