Python tips: unpack data gracefully

Do you know how to easily assign data to variables from lists of various sizes? How to get a 'head' and 'tail' from a tuple without using clumsy indexes? How to use star expressions? Or simplify your code with a throaway variable? If not, get comfy :)

As you may know, we can assign values from sequences or iterables to variables. In short, a sequence is an ordered collection of items that supports accessing elements by indexing (some_list[0]). An iterable is an object that we can loop over (e.g., set is an iterable: for num in {1, 2, 3}: print(num)).

This assignment process is pretty easy, let's try assigning a list of values concerning some company's CEO:

name, experience, job = ['John Doe', 30, 'CEO']
name, experience, job
('John Doe', 30, 'CEO')

Why not try a tuple?

name, experience, job = ('John Doe', 30, 'CEO')
name, experience, job
('John Doe', 30, 'CEO')

Or a string:

a, b, c, = '123'
a, b, c
('1', '2', '3')

Since the object returned by a range() function produces a sequence, we can easily assign its contents to a bunch of variables:

d, e, f = range(3) 
d, e, f
(0, 1, 2)

If you're not very experienced with this, you may want to try playing with other iterable data structures (say dictionaries, sets).

So, what if you need to process data structures with arbitrary amounts of data? For example, you're expecting a server response with a tuple containing the information you need, but you may get more items than you expected, what then? Trying to unpack the same way will fail:

>>> name, experience, job = ('John Doe', 30, 'CEO', (2012, 1, 24))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 3)

Adjusting your code to any possible mismatch wouldn't benefit your realtime application and can easily result in some ugly code. This is where starred expressions come into play. They're called so because you use a * sign before a variable name like that: *var.

Let's handle our last error:

>>> name, experience, job, *other = ('John Doe', 30, 'CEO', (2012, 1, 24), 100000)
>>> name, experience, job
('John Doe', 30, 'CEO')
>>> other
[(2012, 1, 24), 100000]

Here, we assigned all unnecessary information to a special variable. Python takes all remaining values and stores them in a list (always in a list).

In case you don't need all this other data at all, you can make your code even cleaner and use a special throwaway variable _:

>>> name, experience, job, *_ = ('John Doe', 30, 'CEO', (2012, 1, 24), 100000)

The data will still be assigned to it, but other developers will understand that this information won't be used or processed. By the way, you don't have to use it with starred expressions, it can be just _: a, _, b, _ = [1, 2, 3, 4].

You're welcome to use a starred expression in absolutely any part of your assignment:

>>> first, *middle, last = [1, 2, 3, 4]
>>> first
1
>>> middle
[2, 3]
>>> last
4

Or like that:

>>> *first, middle, last = 1, 2, 3, 4
>>> first
[1, 2]
>>> middle
3
>>> last
4

Have you noticed that we assigned variables to 1, 2, 3, 4? It's not a sequence, why didn't Python trow an exception? This is because we type 1, 2, 3, 4, but Python sees it as a tuple. In the examples at the beginning you may have seen that the interpreter always printed tuples as a result.

Another wonderful thing about star expressions is that they handle the situations where there're no extra values to be stored in them. See how we get an empty list in our John Doe example.

>>> name, experience, job, *other = ('John Doe', 30, 'CEO')
>>> name, experience, job
('John Doe', 30, 'CEO')
>>> other
[]

Now, let's try something more spicy :) Don't forget that we can unpack both sequences and iterables with starred expressions. So, let's unpack an iterator, it should be fun:

>>> seq = range(10)
>>> it = iter(seq)
>>> first, *rest = list(it)
>>> first, rest  # notice that the result is printed as a tuple:
(0, [1, 2, 3, 4, 5, 6, 7, 8, 9])

Can we get even more creative and use a couple of starred expressions in our assignment?

Actually, no. When you use a starred expression, you basically tell Python to assign certain values to certain variables, and store the rest of the data in a special starred variable.

So, if you use more than one *var during a single assignment, Python just won't know what to do. So, for one line of code, use only one star. Otherwise, you'll get an exception.

Bonus. Guess, can we use a starred expression alone? I'll publish the answer in the comments in case you'd like to guess it yourself ;)

So, this is how unpacking with starred expressions works in Python. If you'd like to see more practical examples, read my follow-up post.

Feeling curious to read some other tips? Check out my previous Python tips post about elegant exception handling.

Comments

[Vitaly Shchurov]

About using starred expressions alone. Yes, it's possible. For example, instead of assigning a variable to a list directly, you can do it with a range():

*data, = range(1, 11)

This will produce a list of numbers from 1 to 10. You need to use comma, otherwise you'll get a range object instead of a list. It happens because Python gets comma as a syntax for creating a tuple, not (), so it's a syntactic rule that you cannot create a tuple of one item without using a comma. So, this (1) is an integer for Python, and this (1,) or this 1, is a tuple.