DEV Community: Wilfredinni

Python projects with Poetry and VSCode. Part 3

Wilfredinni — Tue, 24 Sep 2019 20:52:30 +0000

In the First Part we:

Started a new project.
Created a Virtual Environment.
Managed dependencies.

In the Second Part:

Added our virtual Environment to VSCode.
Integrated our dev dependencies with the editor:
- Flake8
- Black
- Pytest

And finally, in this third and last part we'll:

Write a sample library.
Build our project with Poetry.
Publish it on PyPI.

Poetry Commands

Here is a table with the commands used in this series as well as their descriptions. For a full list read the Poetry Documentation.

Command	Description
`poetry new [package-name]`	Start a new Python Project.
`poetry init`	Create a pyproject.toml file interactively.
`poetry install`	Install the packages inside the pyproject.toml file.
`poetry add [package-name]`	Add a package to a Virtual Environment.
`poetry add -D [package-name]`	Add a dev package to a Virtual Environment.
`poetry remove [package-name]`	Remove a package from a Virtual Environment.
`poetry remove -D [package-name]`	Remove a dev package from a Virtual Environment.
`poetry update`	Get the latest versions of the dependencies
`poetry shell`	Spawns a shell within the virtual environment.
`poetry build`	builds the source and wheels archives.
`poetry publish`	Publish the package to PyPi.
`poetry publish --build`	Build and publish a package.
`poetry self:update`	Update poetry to the latest stable version.

The Project

You can download the source code from GitHub if you want, but as mentioned earlier, this will be a very simple decorator that the only thing it will do is print to the console how long it takes for a function to run:

from how_long import timer

@timer
def test_function():
    [i for i in range(10000)]

test_function()
# Execution Time: 955 ms.

In the end, the project directory will look, more or less, like this:

how-long
├── how_long
│   ├── how_long.py
│   └── __init__.py
├── how_long.egg-info
│   ├── dependency_links.txt
│   ├── PKG-INFO
│   ├── requires.txt
│   ├── SOURCES.txt
│   └── top_level.txt
├── LICENSE
├── poetry.lock
├── pyproject.toml
├── README.rst
└── tests
    ├── __init__.py
    └── test_how_long.py

Before we start, check for package updates with the poetry update command:

Ok, now add a short description of the project in the README.rst:

how_long
========

Simple Decorator to measure a function execution time.

Example
_______

.. code-block:: python

    from how_long import timer


    @timer
    def some_function():
        return [x for x in range(10_000_000)]

Navigate to how_long/how_long.py:

# how_long.py
from functools import wraps

import pendulum


def timer(function):
    """
    Simple Decorator to measure a function execution time.
    """

    @wraps(function)
    def function_wrapper():
        start = pendulum.now()
        function()
        ellapsed_time = pendulum.now() - start
        print(f"Execution Time: {ellapsed_time.microseconds} ms.")

    return function_wrapper

In how_long/__init__.py:

from .how_long import timer

__version__ = "0.1.1"

And finally, the tests/test_how_long.py file:

from how_long import __version__
from how_long import timer


def test_version():
    assert __version__ == "0.1.1"


def test_wrap():
    @timer
    def wrapped_function():
        return

    assert wrapped_function.__name__ == "wrapped_function"

You can now use poetry install on your terminal to install and prove your package locally. Activate your virtual environment if you haven't and in the Python interactive shell:

>>> from how_long import timer
>>>
>>> @timer
... def test_function():
...     [i for i in range(10000)]
...
>>> test_function()
Execution Time: 705 ms.

Run the tests and if everything is fine, move on.

Building and Publishing

Finally, the time to make this project available to the world has come! First, make sure you have an account on Pypi, and if not, register one. Remember that the package name must be unique, if unsure go and use the search to check it out.

Build

The poetry build command builds the source and wheels archives that will letter be uploaded as the source of the project:

The how_long.egg-info directory will be created.

Publish

This command publishes the package to Pypi and automatically register it before uploading if this is the first time it is submitted:

You can also build and publish your project with $ poetry publish --build.

Enter your credentials and if everything is ok, browse your project and you'll see something like this:

We can now let others know that they can pip install how-long from any machine, anywhere!

Conclusion

I remember the first time I tried to publish a package, and it was a nightmare. I was just starting in Python and I have to spend a "few hours" trying to understand what the setup.py file was and how to use it. In the end, I ended up with several different files: a Makefile, a MANIFEST.in, a requirements.txt and a test_requirements.txt. That's why the words of Sébastien Eustace, the creator of Poetry, made a lot of sense to me:

Packaging and dependency management in Python are rather convoluted and hard to understand for newcomers. Even for seasoned developers it might be cumbersome at times to create all files needed in a Python project: setup.py, requirements.txt, setup.cfg, MANIFEST.in and the newly added Pipfile.

So I wanted a tool that would limit everything to a single configuration file to do: dependency management, packaging and publishing.

It takes inspiration in tools that exist in other languages, like composer (PHP) or cargo (Rust).

And, finally, there is no reliable tool to properly resolve dependencies in Python, so I started poetry to bring an exhaustive dependency resolver to the Python community.

Poetry is by no means perfect but, unlike other tools, it really does what promises.

Python projects with Poetry and VSCode. Part 2

Wilfredinni — Sun, 25 Aug 2019 22:20:17 +0000

In the First Part, we learned what the pyproject.toml file is and how to work with it, used Poetry to start a new project, create a Virtual Environment and to add and remove dependencies. All of that with the following commands:

Command	Description
`poetry new [package-name]`	Start a new Python Project.
`poetry init`	Create a pyproject.toml file interactively.
`poetry install`	Install the packages inside the pyproject.toml file.
`poetry add [package-name]`	Add a package to a Virtual Environment.
`poetry add -D [package-name]`	Add a dev package to a Virtual Environment.
`poetry remove [package-name]`	Remove a package from a Virtual Environment.
`poetry remove -D [package-name]`	Remove a dev package from a Virtual Environment.

In this second part we'll:

Add our virtual Environment to VSCode.
Update our dependencies.
Integrate our dev dependencies with the editor:
- Flake8
- Black
- Pytest

And in the third part we will:

Write a sample library.
Build our project with Poetry.
Publish it on PyPI.

Before we start, make sure you have installed VSCode, added the Python extension and that you have followed and/or understood the First Part of this series.

Setting Up Poetry on VSCode

A few days have passed since the first part, so it may be a good idea to check for new versions of our dependencies. Open your terminal and navigate inside your project folder and type the poetry update command:

Ok, to this day there are no new versions available.

When you create a Virtual Environment with the venv command, VSCode will automatically set it as the default Python Environment for that project. When working With Poetry, the first time we will need to type the following in the terminal and inside the project folder:

$ poetry shell
$ code .

The first command, poetry shell, will spawn us inside our virtual environment, and code . will open the current folder inside VSCode.

Open the how-long folder (or the one with your project name) using the left panel and alongside to __init__.py, create a how-long.py file. In the bottom left corner you'll see the current Python Environment:

Click it and a list of available Environments will display. Choose the one that has the name of your project in it:

Now, let's integrate our dev dependencies, Flake8, Black, and Pytest into Visual Studio Code.

Flake8

Flake8 will provide our projects with linting capabilities. In other words, warning of syntax and style errors, and thanks to VSCode, we will know them as we type.

By default, the Python extension comes with Pylint enabled, which is powerful but complex to configure. To switch to Flake8 make a change to any Python file and save it, in the bottom right corner a popup message will show:

Click on Select Linter and choose Flake8 from the list. Now, VSCode will tell us our syntax and style problems, in green or red depending on its severity, always with a nice description of what is wrong:

Seems like we have two problems: we are missing a blank line at the end of our file (style) and forgot to add quotes to our Hello, World! string (syntax). Fix them and see all warnings disappear.

Black

Black is a code formatter, a tool that will look at our code and automatically format it in compliance with the PEP 8 style guide, the same PEP that uses Flake8 to lint our style errors.

Hold shift + cmd/ctrl + p to open the Command Palette, type Format Document, and press enter. A new popup message will appear:

Select Use Black. Now copy this poorly formatted code into your python file:

for i in range(5):         # this comment has too many spaces
      print(i)  # this line has 6 space indentation.

What an ugly piece of s**... code. Try formatting it again and see how *Black fixes all of them for you!

Another thing we can do is configure VSCode so that every time we save, Black will automatically format our code. Hold cmd/ctrl + , to open the Settings. Make sure you are in the Workspace Settings, search for Format On Save and activate the checkbox:

Lastly, Black defaults to 88 characters per line in contrast with the 80 allowed by Flake8, so to avoid conflicts, open the .vscode folder and add the following at the end of the settings.json file:

{
    ...
    "python.linting.flake8Args": [
        "--max-line-length=88"
    ],
}

Pytest

If you are serious about programming, it is crucial for you to learn how to test your projects. It's an incredibly useful skill that will allow you to write and deliver programs with confidence by reducing the possibility of catastrophic bugs appearing after shipping.

Pytest is a very popular and user-friendly framework for writing tests. We already installed it so we will also integrate it with VSCode.

Open the tests folder and select the test_how_long.py file. Poetry already gives us our first test:

# test_how_long.py
from how_long import __version__


def test_version():
    assert __version__ == '0.1.0'

This test import the __version__ variable from the __init__.py file that is inside the how_long folder and asserts that the current version is 0.1.0. Open the integrated terminal by going to Terminal > New Terminal and type:

$ pytest

The Output will look like this:

Ok, everything is fine. Open your Command Palette with shift + cmd/ctrl + p:

Write unit and select Python: Configure Unit Tests.
Select pytest.
Choose the directory in which the tests are stored, tests in our case.

Three things happened:

A new button appeared at the status bar: Run Tests. This is the same as typing pytest in the terminal. Press it and select Run All Unit Tests. When finished, it will inform you the number of tests that passed and the tests that not:
A new icon at the left bar. If you click on it a panel displaying all the test will appear. Here, you can run each one individually:
Inside the test file, new options will be displayed before every test function: a check icon will appear if is ok, and an x otherwise. It also allows you to run specific tests:

Conclusion

So far we have:

Started a new project.
Created a Virtual Environment with Poetry.
Added, Deleted and Updated dependencies.
Added our Virtual Environment to VSCode.
Configured Flake8 to lint our code as we type.
Choose Black as the formatter of our project.
And Pytest to run our tests in a visual way.

In the third and last part we will:

Write a sample library.
Build our project with Poetry.
Publish it on PyPI.

That's it, Have a nice day!

pythoncheatsheet.org

Wilfredinni — Sat, 20 Jul 2019 19:48:21 +0000

Anyone can forget how to make character classes for a regex, slice a list or do a for loop. This Python cheat sheet tries to provide a basic reference for beginner and advanced developers, lower the entry barrier for newcomers and help veterans refresh the old tricks.

pythoncheatsheet.org

Also, check out the Javascript Cheatsheet.

Python projects with Poetry and VSCode. Part 1

Wilfredinni — Mon, 01 Jul 2019 15:29:06 +0000

Updates:

18-05-2019: Installing Poetry.

A Virtual Environment is an isolated Python installation designed to avoid filling our base one with libraries we might use for only one project. It also allows us to manage multiple versions of the same package in different projects. We could, for example, need Django 2.2 for one and 1.9 in other.

Poetry on the other hand:

poetry is a tool to handle dependency installation as well as building and packaging of Python packages. It only needs one file to do all of that: the new, standardized pyproject.toml.

In other words, poetry uses pyproject.toml to replace setup.py, requirements.txt, setup.cfg, MANIFEST.in and the newly added Pipfile.

In this series of articles, we'll use Poetry to manage our dependencies, build a simple project and, with a single command, publish it on PyPI.

In this first part we will:

Start a new project.
Create a Virtual Environment.
Manage dependencies.

In the Second Part we'll:

Add our virtual Environment to VSCode.
Integrate our dev dependencies with the editor.
- Flake8
- Black
- Pytest

And finally, in a third part we'll:

Write a sample library.
Build our project with Poetry.
Publish it on PyPI.

Installing Poetry

The easiest way is to use pip:

$ pip install poetry

But we will use Poetry own installer to isolate it from the rest of the system by vendorizing its dependencies. This is the recommended way of installing poetry:

$ curl -SSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

If installed this way, you will later be able to update poetry to the latest stable version with the poetry self:update command.

Starting a New Project

We can now start a new Python project by using the poetry new [project_name] command. I will call it how-long and is going to be a very simple library to measure a function execution time:

$ poetry new how-long

Note: For existing projects, you can use the poetry init command and interactively create a pyproject.toml.

The directory how-long is created and inside is a basic project structure:

how-long
├── README.rst
├── how_long
│   └── __init__.py
├── pyproject.toml
└── tests
    ├── __init__.py
    └── test_how_long.py

Note: To be able to publish your project, you need an available name. Use the PyPI search tool for this.

The pyproject.toml File

The pyproject.toml file will manage the details and dependencies of the project:

[tool.poetry]
name = "how-long"
version = "0.1.0"
description = "A simple decorator to measure a function execution time."
authors = ["wilfredinni <carlos.w.montecinos@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^3.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

[tool.poetry]

The details. Adding a license and a Readme might be a good idea:

[tool.poetry]
...
license = "MIT"
readme = "README.rst"

[tool.poetry.dependencies]

First is the Python version. Basically, this project will be compatible with Python 3.7 and up. Also, from now on, every package we install that is meant to be used in production will be listed here.

[tool.poetry.dev-dependencies]

These packages are only for development and will not be included when we publish our project. By default Poetry includes Pytest, so we will use it to test our project later on.

Creating a Virtual Environment

Now, let's create a Virtual Environment and install Pytest with the poetry install command:

$ poetry install

After is done, a new file, poetry.lock will be created.

When Poetry has finished installing, it writes all of the packages and the exact versions of them that it downloaded to the poetry.lock file, locking the project to those specific versions. You should commit the poetry.lock file to your project repo so that all people working on the project are locked to the same versions of dependencies.

Dependency Management

One way to add or remove dependencies is to directly edit pyproject.toml and then run poetry install to apply the changes. We will instead use the add and remove commands to avoid manual modifications.

Adding Dependencies

Let's add two packages to the project, pendulum and coo:

$ poetry add pendulum coo

Open pyproject.toml and poetry.lock and see how they have updated.

Adding Dev Dependencies

These dependencies will be available only during development, Poetry will not include them when building and publishing the project.

We already installed Pytest, but we will also use flake8 for linting and mypy for static typing:

$ poetry add -D flake8 mypy

Now that I think about it, I forgot to add a formatter. We'll go with black:

$ poetry add -D black
[ValueError]
Could not find a matching version of package black

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

This error happens because black is in a pre-release state, so Poetry cannot find any stable version for us. But I really want it so let's install it anyway using the --allow-prereleases flag:

$ poetry add -D black --allow-prereleases

Removing Dependencies

You know what, I changed my mind, this project will use nor coo nor mypy. Start by removing coo, a normal dependency of our project:

$ poetry remove coo

Now mypy which is a dev dependency:

$ poetry remove -D mypy

Conclusion

In this first part, we have started a new project, created a Virtual Environment and added and removed dependencies by using the following commands:

Command	Description
`poetry new [package-name]`	Start a new Python Project.
`poetry init`	Create a pyproject.toml file interactively.
`poetry install`	Install the packages inside the pyproject.toml file.
`poetry add [package-name]`	Add a package to a Virtual Environment.
`poetry add -D [package-name]`	Add a dev package to a Virtual Environment.
`poetry remove [package-name]`	Remove a package from a Virtual Environment.
`poetry remove -D [package-name]`	Remove a dev package from a Virtual Environment.
`poetry self:update`	Update poetry to the latest stable version.

In the Second Part, we will see more Poetry commands, add our Virtual Environment to VSCode and use the dev packages we installed to lint (Flake8), format (Black) and test (Pytest) our code inside the editor. Finally, in a third one, we will write and publish a sample library to PyPI.

Any doubt or suggestion? Please leave a comment.

Python *args and **kwargs Made Easy

Wilfredinni — Tue, 11 Jun 2019 17:20:17 +0000

I do not know about you, but every time I saw some function with *args and **kwargs as parameters, I'd get a little scared. I've even "used" them while doing some backend work with Django without understanding a thing. If you're a self-taught developer like me, I know you've been there too.

A few months ago I decided to stop being lazy and started to research it. To my surprise, they were very easy to grasp when playing with the interpreter but not so much when reading about them. I wrote this post trying to explain args and kwargs the way I would have liked someone explained them to me.

Basics

The first thing you need to know is that *args and **kwargs lets you pass an undefined number of arguments and keywords when calling a function:

def some_function(*args, **kwargs):
    pass

# call some_function with any number of arguments
some_function(arg1, arg2, arg3)

# call some_function with any number of keywords
some_function(key1=arg1, key2=arg2, key3=arg3)

# call both, arguments and keywords
some_function(arg, key1=arg1)

# or none
some_function()

Second, the words args and kwargs are conventions. This means they are not imposed by the interpreter, but considered good practice among the Python community:

# This function would work just fine
def some_function(*arguments, **keywords):
    pass

A note about conventions:

Even if the above function works, don't do it. Conventions are there to help you write readable code for you and anyone that might be interested in your project.

Other conventions include the 4 space indentation, comments, and imports. Reading the PEP 8 -- Style Guide for Python Code is highly recommended.

So, how does Python know that we want our function to accept multiple arguments and/or keywords? Yes, the answers are the * and ** operators.

Now that we have covered the basics, let's work with them 👊.

args

We now know how to pass multiple arguments using *args as a parameter to our functions, but how do we work with them? It's easy: all the arguments are contained within the args variable as a tuple:

def some_function(*args):
    print(f'Arguments passed: {args} as {type(args)}')


some_function('arg1', 'arg2', 'arg3')
# Arguments passed: ('arg1', 'arg2', 'arg3') as <class 'tuple'>

We can iterate over them:

def some_function(*args):
    for a in args:
        print(a)


some_function('arg1', 'arg2', 'arg3')
# arg1
# arg2
# arg3

Access the elements with an index:

def some_function(*args):
    print(args[1])


some_function('arg1', 'arg2', 'arg3')  # arg2

Slice:

def some_function(*args):
    print(args[0:2])


some_function('arg1', 'arg2', 'arg3')
# ('arg1', 'arg2')

Whatever you do with a tuple, you can do it with args.

kwargs

While arguments are stored in the args variable, keywords are within kwargs, but this time as a dictionary where the key is the keyword:

def some_function(**kwargs):
    print(f'keywords: {kwargs} as {type(kwargs)}')


some_function(key1='arg1', key2='arg2', key3='arg3')
# keywords: {'key1': 'arg1', 'key2': 'arg2', 'key3': 'arg3'} as <class 'dict'>

Again, we can do with kwargs the same we would do with any dictionary.

Iterate over:

def some_function(**kwargs):
    for key, value in kwargs.items():
        print(f'{key}: {value}')


some_function(key1='arg1', key2='arg2', key3='arg3')
# key1: arg1
# key2: arg2
# key3: arg3

Use the get() method:

def some_function(key, **kwargs):
    print(kwargs.get(key))


some_function('key3', key1='arg1', key2='arg2', key3='arg3')
# arg3

And a lot more =).

Conclusion

*args and **kwargs may seem scary, but the truth is that they are not that difficult to grasp and have the power to grant your functions with flexibility and readability. If you know about tuples and dictionaries, you are ready to go.

Want to play with args and kwargs? This is an online Jupyter Notebook for you to try.

Also, some examples make use of f-strings, a relatively new way to format strings in Python 3.6+. Here you can read more about it.

Any doubt or suggestion? Please leave a comment and have a nice day!

Python: Who to Follow, What to Read and Where to Ask

Wilfredinni — Tue, 04 Jun 2019 18:15:02 +0000

Updates:

05-21-2019: Blogs, Podcasts.

As a self-taught Python Developer and a newbie myself, I think that one of the most important things when starting is to surround ourselves with all things Python related. We can read Blogs, follow developers on Twitter and try to contribute to their projects on GitHub. We can also subscribe to the different mailing lists that exist, and of course, to the many YouTube channels that put online and free of charge millions of hours of content so we can just keep getting better and better at Python.

In this post, I want to share some of the "assets" that I have collected to this point.

Awesome Python

Awesome Python

A curated list of awesome Python frameworks, libraries, software and resources.

Python LibHunt

Your go-to Python Toolbox.

Blogs

Real Python

The realpython.com tutorial team is known for making the highest-quality Python tutorials available online.

Miguel Grinberg (Here you can find the Flask Mega Tutorial. I used it myself as a starting point to build this site.)

I spent some of my spare time in the last month or so writing a tiny blog engine, and I really like how it came out.

Jahongir Rahmonov

I write about Python, Django, AngularJS and sometimes something non-technical.

simple is better than complex

Stories about Python, Django and Web Development.

Full Stack Python

Now you want to take your initial Python knowledge and make something real, like a web application to show off to friends or sell as a service to customers. That's where Full Stack Python comes in. You have come to the right place to learn everything you need to create, deploy and operate Python-powered applications.

Diajngo Stars

In a blog we tell how to develop and grow sustainable and scalable Fintech, Taxi&Transportation and Booking products. We also answer customers' most common questions and cover useful topics.

The Invent with Python Blog

Learning to program makes you a smarter and more capable human being. Computers are not only the most common tool we encounter in our day-to-day lives, but also the most powerful. Even if your plans don't include becoming a software engineer writing code for the next Mars mission, you can find value in learning to code.

OVERIQ (Here, you can also find coruses on Python, Django, Flask, SQLAlchemy and C)

I'm a simple coder who have a knack with computer languages. My aim is to educate every one with the best of the language knowledge which is not easily available anywhere else in the Internet world.

The Mouse vs the Python

My name is Mike Driscoll. I am a computer programmer by trade and use Python almost exclusively to make my living.

Snarky

Python core developer. Dev lead for the Python extension for VS Code. Tall, snarky Canadian.

Python for the Lab

Python For The Lab is a collection of resources to help you start using Python for developing solutions in your lab.

Armin Ronacher's Thoughts and Writings

My name is Armin Ronacher and I'm a passionate developer interested in web development, graphics, realtime rendering and tons of other stuff. I am one of the founding members of the Pocoo Team and lead developer on a couple of popular Python projects.

The Python Guru

Become a better python developer

Practical Business Python

Taking care of business, one python script at a time.

Cool Python Codes

Code, The Revolution.

All your Base are Belong to Us

Physics. Data Science. General Geekery.

PyDanny

Hi, I'm Daniel Roy Greenfeld, and welcome to my blog. I write about Python, Django, and much more.

Python Tips

Your daily dose of bite sized python tips

PyBites

A Community that Masters Python through Code Challenges

Weekly Mailing Lists

Python Mailing Lists and Newsgroups

Here's an overview of the mail and news resources for python.

Import Python

Insightful Articles, Tutorials, Projects, Videos, Tweets delivered in your inbox.

PyCoders Weekly

A free weekly e-mail newsletter, on Fridays, for those interested in python development and various topics around python.

Python Weekly

A free weekly newsletter featuring curated news, articles, new releases, jobs etc related to Python.

Django Mailing Lists

Subscribe to one of our mailing lists to stay up to date with everything in the Django community

Twitter

Guido van Rossum

Python's creator and emeritus BDFL. Working at Dropbox. Computer History Museum Fellow.

Dan Bader

Full-stack Pythonista & Python Coach. Write Clean + Pythonic code with my free tutorials: https://dbader.org/get-pythonic 🐍Previously: Lead Dev at Mobify.

David Beazley

Author of the Python Essential Reference and Python Cookbook. I tweet deep thoughts about code, bikes, kids, and stuff.

Corey Schafer

Developer and Designer who enjoys woodworking, kayaking, astronomy, and building stuff.

Adrian Holovaty

Web developer, cocreator of @djangoproject, poster of YouTube guitar videos. Currently building @soundslice.

Miguel Grinberg

I write software. And from time to time, I write or speak about writing software.

Harrison Kinsley

I like to build things. Programming is neat. INTJ https://hkinsley.com , https://pythonprogramming.net , http://sentdex.com.

Al Sweigart

Author of programming books. I use Python 4.2.

Michael Kennedy

A Python and MongoDB enthusiast, and an entrepreneur. Host of @TalkPython and @PythonBytes, founder of Talk Python Training. Python Software Foundation Fellow.

Python Software

The Python Software Foundation. For help with Python code, see comp.lang.python.

Python Insider

The Python core development team.

Anaconda

Anaconda is the world’s most popular and trusted #Python #DataScience platform, with over 6M users. We help people change the world.

Project Jupyter

Multi-language interactive computing environments. The future home of the IPython notebook and related projects (same folks, same code, new name, new repos).

IPython Developers

Interactive Python tools. Please, Internet, it is IPython (or ipython), not iPython. Notebook and other language-agnostic parts are moving to @ProjectJupyter.

Django

A high-level Python Web framework that encourages rapid development and clean, pragmatic design. Tweets by Django Software Foundation and @jezdez .

Real Python

Python developer resources: #Python news, tutorials, code examples, snippets, articles, and more!

Daily Python Tip

One #python tip per day. Do you have any? Send it to http://bit.ly/pythontip . Edited by @karlafej, @simecek , @pybites and... you?

Pybites

Python code Challenges, Articles and News.

Python Trending

trending Python repositories on GitHub.

Import Python

Python Newsletter https://clicktracker.in/VkXRR7Po keeps you updated on what's happening in the world of Python.

Full Stack Python

Tweets links to great #Python tutorials. Creating http://fullstackpython.com , http://bit.ly/deploy-python & http://bit.ly/python-entrepreneurs …. Run by @mattmakai 😎

Python Hub

News & links about Python programming.

Get Python

Tweets for Python enthusiasts - by @originalankur .

Python Weekly

Python Weekly is a free weekly newsletter, which features curated news, articles, new releases, tools and libraries, events, jobs etc related to Python.

Python StackOverflow

StackOverflow feed of Python questions matching certain criteria.

Scientific Python

Tweets about SciPy (Scientific Python) and related topics from @JohnDCook.

Pycoders Weekly

Your weekly dose of all things Python! Python tweets by @mgrouchy and @myusuf3.

Boostlog - Python

A blogging platform developers. Here will introduce the Python article posted on #Boostlog.

Python Cheatsheet 🤭

Python cheat sheet and Articles for Beginner and Advanced Developers.

Podcasts

Python Bytes

Python Headlines delivered directly to your earbuds.

Talk Python To Me

A podcast on Python and related technologies.

The Python Podcast._ init _

Each interview explores the motivations, challenges, and experiences behind the software that you use every day.

Test & Code - Software Testing, Developtment, Python

A Podcast about Software Testing, Software Development, and Python

Django Chat

A weekly podcast on the Django Web Framework. Build websites with Python!

Friday Afternoon Deploy: A Developer Podcast

Python and Django, Ruby on Rails, Golang.

YouTube

This list is going to be short, just my favorites. If you want a realy complete one, check The Ultimate List of Python YouTube Channels from Real Python.

Coding Tech

Coding Tech partners with the tech conferences around the world that provided Coding Tech with their EXPLICIT permissions to republish videos on this channel.

Corey Schafer

This channel is focused on creating tutorials and walkthroughs for software developers, programmers, and engineers.

Pretty Printed

I'm Anthony. I make programming videos.

Socratica

Socratica makes high-quality educational videos on math and science. New videos every week!

Sebastiaan Mathôt

A mix of videos about programming in Python and OpenSesame, and psychology-related topics.

sentdex

Python Programming tutorials, going further than just the basics. Learn about machine learning, finance, data analysis, robotics, web development, game development and more.

Questions

Stack Overflow

Each month, over 50 million developers come to Stack Overflow to learn, share their knowledge, and build their careers.

Quora

A place to share knowledge and better understand the world.

Python Community Forums

If you're looking for a forum in your native language, please check out the local user groups page at the Python Wiki.

r/python

News about the dynamic, interpreted, interactive, object-oriented, extensible programming language Python.

r/pythontips

A place to get a quick fix of python tips and tricks to make you a better Pythonista.

r/learnpython

Subreddit for posting content, questions, and asking for general advice about learning the Python programming language.

r/django

Django subreddit

r/djangolearning

A group dedicated to learning Django, a Python web framework.

r/flask

Flask is a Python micro-framework for web development. Flask is easy to get started and a great way to build web sites and web applications.

Conclusion

Just two things: First, we have no excuse. And second, for all the people who try to put quality content online, for the ones that do it in their spare time and for free: subscribe to their channels, give them thumbs up even before you finish to watch their videos (so you don't forget later). If you read an article you liked, share it and disable your ad-blocker for their site. Support Them.

As always, if you think something is missing, leave a comment and I will update the post later.

Have a nice day, and eat what makes you happy!

Python Sets: What, Why and How

Wilfredinni — Sat, 01 Jun 2019 15:40:34 +0000

Python comes equipped with several built-in data types to help us organize our data. These structures include lists, dictionaries, tuples and sets.

From the Python 3 documentation:

A set is an unordered collection with no duplicate elements. Basic uses include membership testing and eliminating duplicate entries. Set objects also support mathematical operations like union, intersection, difference, and symmetric difference.

In this article, we are going to review and see examples of every one of the elements listed in the above definition. Let's start right away and see how we can create them.

Initializing a Set

There are two ways to create a set: one is to provide the built-in function set() with a list of elements, and the other is to use the curly braces {}.

Initializing a set using the set() built-in function:

>>> s1 = set([1, 2, 3])
>>> s1
{1, 2, 3}
>>> type(s1)
<class 'set'>

Initializing a set using curly braces {}

>>> s2 = {3, 4, 5}
>>> s2
{3, 4, 5}
>>> type(s2)
<class 'set'>
>>>

As you can see, both options are valid. The problem comes when what we want is an empty one:

>>> s = {}
>>> type(s)
<class 'dict'>

That's right, we will get a dictionary instead of a set if we use empty curly braces =)

It's a good moment to mention that for the sake of simplicity, all the examples provided in this article will use single digit integers, but sets can have all the hashable data types that Python support. In other words, integers, strings and tuples, but not mutable items like lists or dictionaries:

>>> s = {1, 'coffee', [4, 'python']}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

Now that you know how to create a set and what type of elements it can contain, let's continue and see why we should always have them in our toolkit.

Why You Should Use Them

When writing code, you can do it in more than a single way. Some are considered to be pretty bad, and others, clear, concise and maintainable. Or "pythonic".

From the The Hitchhiker’s Guide to Python:

When a veteran Python developer (a Pythonista) calls portions of code not “Pythonic”, they usually mean that these lines of code do not follow the common guidelines and fail to express its intent in what is considered the best (hear: most readable) way.

Let's start exploring the way that Python sets can help us not just with readability, but also speeding up our programs execution time.

Unordered Collection of Elements

First things first: you can't access a set element using indexes.

>>> s = {1, 2, 3}
>>> s[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'set' object does not support indexing

Or modify them with slices:

>>> s[0:2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'set' object is not subscriptable

BUT, if what we need is to remove duplicates, or do mathematical operations like combining lists (unions), we can, and SHOULD always use Sets.

I have to mention that when iterating over, sets are outperformed by lists, so prefer them if that is what you need. Why? well, this article does not intend to explain the inner workings of sets, but if you are interested, here are a couple of links where you can read about it:

No Duplicate Items

While writing this I cannot stop thinking in all the times I used the for loop and the if statement to check and remove duplicate elements in a list. My face turns red remembering that, more than once, I wrote something like this:

>>> my_list = [1, 2, 3, 2, 3, 4]
>>> no_duplicate_list = []
>>> for item in my_list:
...     if item not in no_duplicate_list:
...             no_duplicate_list.append(item)
...
>>> no_duplicate_list
[1, 2, 3, 4]

Or used a list comprehension:

>>> my_list = [1, 2, 3, 2, 3, 4]
>>> no_duplicate_list = []
>>> [no_duplicate_list.append(item) for item in my_list if item not in no_duplicate_list]
[None, None, None, None]
>>> no_duplicate_list
[1, 2, 3, 4]

But it's ok, nothing of that matters anymore because we now have the sets in our arsenal:

>>> my_list = [1, 2, 3, 2, 3, 4]
>>> no_duplicate_list = list(set(my_list))
>>> no_duplicate_list
[1, 2, 3, 4]
>>>

Now let's use the timeit module and see the excecution time of lists and sets when removing duplicates:

>>> from timeit import timeit
>>> def no_duplicates(list):
...     no_duplicate_list = []
...     [no_duplicate_list.append(item) for item in list if item not in no_duplicate_list]
...     return no_duplicate_list
...
>>> # first, let's see how the list perform:
>>> print(timeit('no_duplicates([1, 2, 3, 1, 7])', globals=globals(), number=1000))
0.0018683355819786227

>>> from timeit import timeit
>>> # and the set:
>>> print(timeit('list(set([1, 2, 3, 1, 2, 3, 4]))', number=1000))
0.0010220493243764395
>>> # faster and cleaner =)

Not only we write fewer lines with sets than with lists comprehensions, we also obtain more readable and performant code.

Note: remember that sets are unordered, so there is no guarantee that when converting them back to a list the order of the elements is going to be preserved.

From the Zen of Python:

Beautiful is better than ugly.

Explicit is better than implicit.

Simple is better than complex.

Flat is better than nested.

Aren't sets just Beautiful, Explicit, Simple and Flat? =)

Membership Tests

Every time we use an if statement to check if an element is, for example, in a list, you are doing a membership test:

my_list = [1, 2, 3]
>>> if 2 in my_list:
...     print('Yes, this is a membership test!')
...
Yes, this is a membership test!

And sets are more performant than lists when doing them:

>>> from timeit import timeit
>>> def in_test(iterable):
...     for i in range(1000):
...             if i in iterable:
...                     pass
...
>>> timeit('in_test(iterable)',
... setup="from __main__ import in_test; iterable = list(range(1000))",
... number=1000)
12.459663048726043

>>> from timeit import timeit
>>> def in_test(iterable):
...     for i in range(1000):
...             if i in iterable:
...                     pass
...
>>> timeit('in_test(iterable)',
... setup="from __main__ import in_test; iterable = set(range(1000))",
... number=1000)
0.12354438152988223
>>>

Note: the above tests come from this StackOverflow thread.

So if you are doing comparisons like this in huge lists, it should speed you a good bit if you convert that list into a set.

How to Use Them

Now that you know what a set is and why you should use them, let's do a quick tour and see how can we modify and operate with them.

Adding Elements

Depending on the number of elements to add, we will have to choose between the add() and update() methods.

add() will add a single element:

>>> s = {1, 2, 3}
>>> s.add(4)
>>> s
{1, 2, 3, 4}

And update() multiple ones:

>>> s = {1, 2, 3}
>>> s.update([2, 3, 4, 5, 6])
>>> s
{1, 2, 3, 4, 5, 6}

Remember, sets remove duplicates.

Removing Elements

If you want to be alerted when your code tries to remove an element that is not in the set, use remove(). Otherwise, discard() provides a good alternative:

>>> s = {1, 2, 3}
>>> s.remove(3)
>>> s
{1, 2}
>>> s.remove(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 3

discard() won't raise any errors:

>>> s = {1, 2, 3}
>>> s.discard(3)
>>> s
{1, 2}
>>> s.discard(3)
>>> # nothing happens!

We can also use pop() to randomly discard an element:

>>> s = {1, 2, 3, 4, 5}
>>> s.pop()  # removes an arbitrary element
1
>>> s
{2, 3, 4, 5}

Or clear() to remove all the values from a set:

>>> s = {1, 2, 3, 4, 5}
>>> s.clear()  # discard all the items
>>> s
set()

union()

union() or | will create a new set that contains all the elements from the sets we provide:

>>> s1 = {1, 2, 3}
>>> s2 = {3, 4, 5}
>>> s1.union(s2)  # or 's1 | s2'
{1, 2, 3, 4, 5}

intersection()

intersection or & will return a set containing only the elements that are common in all of them:

>>> s1 = {1, 2, 3}
>>> s2 = {2, 3, 4}
>>> s3 = {3, 4, 5}
>>> s1.intersection(s2, s3)  # or 's1 & s2 & s3'
{3}

difference()

Using diference() or -, creates a new set with the values that are in "s1" but not in "s2":

>>> s1 = {1, 2, 3}
>>> s2 = {2, 3, 4}
>>> s1.difference(s2)  # or 's1 - s2'
{1}

symmetric_diference()

symetric_difference or ^ will return all the values that are not common between the sets.

>>> s1 = {1, 2, 3}
>>> s2 = {2, 3, 4}
>>> s1.symmetric_difference(s2)  # or 's1 ^ s2'
{1, 4}

Conclusions

I hope that after reading this article you know what a set is, how to manipulate their elements and the operations they can perform. Knowing when to use a set will definitely help you write cleaner code and speed up your programs.

If you have any doubts, please leave a comment and I will gladly try to answer them. Also, don´t forget that if you already understand sets, they have their own place in the Python Cheatsheet, where you can have a quick reference and refresh what you already know.