Max Ong Zong Bao

Posted on Jan 22, 2020 • Originally published at maxongzb.com

My Top 5 Awesome Python Projects

#python #beginners #opensource

Introduction

I think there is tons of development for the Python community. That is useful for the development of Python be it your a full stack developer, data scientist or an educator.

I feel these projects are doing a lot to solve problems when you are starting out or just trying things to see how it sticks.

Cookiecutter

If you had not heard about cookiecutter. I suggest that you take a look at starting a new project in Python.

It is a breeze to use cookiecutter with the flexibility to pick your own journey. When building your Python project but with all the project structures that you need without all the worry on how it should be done.

Zappa

If your planning to build your Microservice and you are using AWS. I would suggest you try out Zappa.

It magically takes advantage of Lamdas & API gateway capability. You don't need to learn about the complicated stuff to deploy like Kubernetes or docker.

Because it's serverless which sets up a server within milliseconds to run your API by just installing as a python package and some configuration in AWS and you are done. This applies to Flask and Django while your there do donate to the project to help it stay afloat.

Scrapy

Data Science is a sought after job in the data science world. The unglamorous work is dealing with data that is your secret sauce. From cleaning to getting your hands on data. So that it could provide insights or create data products with it.

Sometimes web platforms do not provide you with APIs access to their data. The next best thing is to do data scrapping by using a library.

I would always go for Scrapy when I'm doing a data scrapping project because it allows you to add in plugins.

Which scrap information with Javascripts capabilities which are founded more often than you think and IP proxy services to scrap data from your website.

MicroPython

If are in the education space and teaching programming to kids. You might have heard of Arduino or Lego Mindstorms or the famous Raspberry Pi.

Well, Microbit had been making waves for kids to introduce them to computer programming due to its affordability.

MicroPython is running on that hardware and is used in schools in the UK and some coding schools in Singapore to help expose kids in the joy of computer programming.

In some videos, MicroPython being used for industrial IoT projects as edge computers.

Pyodide

When you think of Web Assembly, you will think of Rust or Go. In fact unknown to you, Mozilla came out a project called Pyodide.

Why is this useful? Well if you don't know about AI and machine learning technology. It requires expensive hardware to build machine learning models and plus configuration data sciences tools will be a pain for anyone. Which cloud providers are offering it to improve the speed of development for data products and data analysis.

But what if you can build or run machine learning models and use data science tools without the need to configure your own cloud service or get expensive hardware in your own browser?

This essentially opens up a doorway to allow anyone to adopt and use data science tools to build data products or perform data exploration within your own browser.

This could also be flipped to allow delivery & execution of data models to a clients browser by just accessing a URL without installing a plugin.

The additional benefits are that most Python will then be running in a browser instead of just a backend language.

The headache comes with multiple programming languages in a web browser. Which makes debugging much harder, due to the nature of how web assembly binary is built.

Conclusion

This is what it is making me excited about Python. I do hope that there will be more investments for these projects.

I believe these could be useful to attract developers into the Python world. As a gateway for them to become a developer into Microservices, Data Science, IoT or Web Development.

Yes, I know Python is not useful for tons of things and to branch off to solve problems. Armed with the tools to use their fundamentals. They took from Python as a gateway to programming or for a professional level.

If you like this article sign up for my Adventurer's Newsletter for a weekly update in the area of Python, Startup and Web Development.

You can also follow me to get the latest update of my article on Dev

The original post was on My Top 5 Awesome Python Projects - Reading Time: 4 Mins and cover image by Zan on Unsplash

Reference

Top comments (7)

Ali Sherief • Jan 22 '20

One Python project I think is worth mentioning is zignal. It is a simple signal processing library that can manipulate audio wave forms, like the built-in audioop module. It doesn't have many filters but it aims to be a readable guide to understanding signal processing.

Max Ong Zong Bao • Jan 22 '20

Wow that is really a very niche area to use Python with.

rhymes • Jan 22 '20

I was going through FastAPI today and reminded myself I definitely need to try it.

It was nice reading these two pages:

Alternatives, Inspiration and Comparisons: the team explains how their tool compares to others, what they were inspired from and also subtly what they weren't
History, Design and Future: as you all know we definitely do not need another web framework but it's nice to know that we're wrong because there's room for better frameworks embracing open standards

thanks to @citizen428 for reminding me of this project

Max Ong Zong Bao • Jan 22 '20 • Edited

Nice, it looks like its originated from Flask roots but i really like it integrates OpenAPI. Which took me a while to understand, applying it to use as a tool besides Postman.

Nikita Sobolev • Jan 22 '20

I also suggest to check out wemake-python-styleguide. It is the most powerful python linter out there!

wemake-services / wemake-python-styleguide

The strictest and most opinionated python linter ever!

wemake-python-styleguide

Welcome to the strictest and most opinionated python linter ever.

wemake-python-styleguide is actually a flake8 plugin with some other plugins as dependencies.

Quickstart

pip install wemake-python-styleguide

You will also need to create a setup.cfg file with the configuration.

We highly recommend to also use:

flakehell for easy integration into a legacy codebase
nitpick for sharing and validating configuration across multiple projects

Running

flake8 your_module.py

This app is still just good old flake8 And it won't change your existing workflow.

See "Usage" section in the docs for examples and integrations.

We also support Github Actions as first class-citizens Try it out!

What we are about

The ultimate goal of this project is to make all people write exactly the same python code.

flake8	pylint	black	mypy	wemake-python-styleguide
Formats code?	❌	❌	✅	❌	❌
Finds style issues?	🤔	✅	🤔	❌	✅
Finds bugs?	🤔	✅	❌

…

View on GitHub

Juan Carlos • Jan 22 '20 • Edited

Web scraper alternative:

juancarlospaco / faster-than-requests

Faster requests on Python 3

Faster-than-Requests

Library	Speed	Files	LOC	Dependencies	Developers	Multi-Threaded Web Scraper Built-in
PyWGET	`152.39`	1	338	Wget	>17	❎
Requests	`15.58`	>20	2558	>=7	>527	❎
Requests (cached object)	`5.50`	>20	2558	>=7	>527	❎
Urllib	`4.00`	???	1200	0 (std lib)	???	❎
Urllib3	`3.55`	>40	5242	0 (No SSL), >=5 (SSL)	>188	❎
PyCurl	`0.75`	>15	5932	Curl, LibCurl	>50	❎
PyCurl (no SSL)	`0.68`	>15	5932	Curl, LibCurl	>50	❎
Faster_than_requests	`0.45`	1	99	0	1	✔️ 6, One-Liner

Lines Of Code counted using CLOC.
Direct dependencies of the package when ready to run.
Benchmarks run on Docker from Dockerfile on this repo.
Developers counted from the Contributors list of Git.
Speed is IRL time to complete 10000 HTTP local requests.
Stats as of year 2019.
x86_64 64Bit AMD, SSD, Arch Linux.

Use

import faster_than_requests as requests
requests.get("http://httpbin.org/get")                             # GET
requests.post("http://httpbin.org/post

…

View on GitHub

Faster Than Requests with Multi-Thread URL-to-SQLite Web Scraper

Juan Carlos ・ Dec 24 '19 ・ 1 min read

#python

Devang Hingu • Jan 22 '20

wow.. nice information.... i will start research about it

DEV Community