Sundeep

Posted on Jul 25, 2020 • Updated on Jan 24, 2021 • Originally published at learnbyexample.github.io

I know Python basics, what next?

#python #intermediate #resources

What to learn next is an often asked question. Searching for what next on /r/learnpython gives you too many results. Here's some more Q&A and articles on this topic:

Exercises and Projects

I do not have a simple answer to this question either. If you feel comfortable with programming basics and Python syntax, then exercises are a good way to test your knowledge. The resource you used to learn Python will typically have some sort of exercises, so those would be ideal as a first choice. I'd also suggest using the below resources to improve your skills. If you get stuck, reread the material related to those topics, search online, ask for clarifications, etc — in short, make an effort to solve it. It is okay to skip some troublesome problems (and come back to it later if you have the time), but you should be able to solve most of the beginner problems. Maintaining notes will help too, especially for common mistakes.

Exercism, Practicepython, Edabit — these are all beginner friendly and difficulty levels are marked
Codewars, Adventofcode, Projecteuler — more challenging
Checkio, Codingame, Codecombat — gaming based challenges
/r/dailyprogrammer — not active currently, but there's plenty of past challenges with discussions

Once you are comfortable with basics and syntax, the next step is projects. I use a 10-line program that solves a common problem for me — adding body { text-align: justify } to epub files that are not justify aligned. I didn't know that this line would help beforehand, I searched online for a solution and then automated the process of unzipping epub, adding the line and then packing it again. That will likely need you to lookup documentation and go through some stackoverflow Q&A as well. And once you have written the solution and use it regularly, you'll likely encounter corner cases and features to be added. I feel this is a great way to learn and understand programming.

Debugging

Knowing how to debug your programs is crucial and should be ideally taught right from the beginning instead of a chapter at the end of the book. Think Python is an awesome example for such a resource material.

Sites like Pythontutor allow you to visually debug a program — you can execute a program step by step and see the current value of variables. Similar feature is typically provided by IDEs like Pycharm and Thonny. Under the hood, these visualizations are using the pdb module. See also Python debugging with pdb.

Debugging is often a frustrating experience. Taking a break helps (and sometimes I have found the problem in my dreams). Try to reduce the code as much as possible so that you are left with minimal code necessary to reproduce the issue. Talking about the problem to a friend/colleague/inanimate-objects/etc can help too — known as Rubber duck debugging. I have often found the issue while formulating a question to be asked on forums like stackoverflow/reddit because writing down your problem is another way to bring clarity than just having a vague idea in your mind. Here's some more articles on this challenging topic:

Here's an interesting snippet (modified to keep it small) from a collection of interesting bug stories.

A jpeg parser choked whenever the CEO came into the room, because he always had a shirt with a square pattern on it, which triggered some special case of contrast and block boundary algorithms.

Testing

Another crucial aspect in the programming journey is knowing how to write tests. In bigger projects, usually there are separate engineers (often in much larger number than code developers) to test the code. Even in those cases, writing a few sanity test cases yourself can help you develop faster knowing that the changes aren't breaking basic functionality.

There's no single consensus on test methodologies. There is Unit testing, Integration testing, Test-driven development and so on. Often, a combination of these is used. These days, machine learning is also being considered to reduce the testing time, see Testing Firefox more efficiently with machine learning for example.

When I start a project, I usually try to write the programs incrementally. Say I need to iterate over files from a directory. I will make sure that portion is working (usually with print statements), then add another feature — say file reading and test that and so on. This reduces the burden of testing a large program at once at the end. And depending upon the nature of the program, I'll add a few sanity tests at the end. For example, for my command_help project, I copy pasted a few test runs of the program with different options and arguments into a separate file and wrote a program to perform these tests programmatically whenever the source code is modified.

For non-trivial projects, you'll usually end up needing frameworks like built-in module unittest or third-party modules like pytest. Here's some learning resources.

Getting started with testing in Python
calmcode: pytest
obeythetestinggoat — TDD for the Web, with Python, Selenium, Django, JavaScript and pals
Modern Test-Driven Development in Python — TDD guide, has a real world application example

Intermediate Python resources

Official Python docs — Python docs are a treasure trove of information
Calmcode — videos on testing, code style, args kwargs, data science, etc
Practical Python Programming — covers foundational aspects of Python programming with an emphasis on script writing, data manipulation, and program organization
Intermediate Python — covers debugging, generators, decorators, virtual environment, collections, comprehensions, classes, etc
Effective Python — insight into the Pythonic way of writing programs
Fluent Python — takes you through Python’s core language features and libraries, and shows you how to make your code shorter, faster, and more readable at the same time
- Fluent Python, 2nd Edition
Serious Python — deployment, scalability, testing, and more
Practices of the Python Pro — learn to design professional-level, clean, easily maintainable software at scale, includes examples for software development best practices
Pythonprogramming — domain based topics like machine learning, game development, data analysis, web development, etc
Youtube: Corey Schafer — various topics for beginners to advanced users

Algorithms and Design patterns

Problem solving with algorithms and data structures
GitHub: Awesome algorithms
GitHub: Collection of design patterns and idioms
Python design patterns inspired from Design Patterns: Elements of Reusable Object-Oriented Software (also known as Gang of Four book)

Handy cheatsheets

Python Crash Course cheatsheet
Comprehensive Python cheatsheet
Scientific Python cheatsheet
Common beginner errors — use the pdf link

More Python resources

Inspired by this post, I made a Python learning resources repository which is categorized (beginner, intermediate, advanced, domains like web/ML/data science, etc) and includes a handy search feature.

I hope these resources will help you take that crucial next step and continue your Python journey. Happy learning :)