DEV Community

Cover image for requirements.txt Generator [Script]

requirements.txt Generator [Script]

woosal profile image Vusal Ismayilov ・2 min read

Whoever is into Python already knows that for the projects, which are usually shared by the public, on GitHub for instance, mostly include some general files, such as, Procfile, and of which being is the actual requirements.txt file, and this article will be about.
The title already says it all, a default requirements file for everybody, so with one click everybody can get the libraries, which were used in the projects of the others to be easily downloaded and installed with some commands through the terminal as following, for example:

pip install -r requirements.txt (the path to the requirements.txt file)

or for Linux and MacOS
pip3 install -r requirements.txt (the path to the requirements.txt file)

I was very curious about, what if the project is too big, that it takes a lot of time for the users to write down all the libraries one-by-one (considering the cases that virutal python environments are not created by the user, and the project specific libraries can not be extracted seperately) as tens to hundreds of different libraries were used? Or maybe just for the new users or contributors of that project who do not own all the needed/required libraries, is it possible for them to find all the libraries, which were used in the project with only one executable script?

My road starts here, and I have started thinking of an algorithm that would have the following features:

1) Will scan all the files/folders in the given project path

2) Get all the .py extended files and save their names, so the imported custom written python files to the other python files are not cosnidered as seperate libraries

3) Scan all the python files, and only save the lines where only “import” / “from” start the line itself

4) Will not save the same library name more than once if that specific library was used more than once in the same project, however, in different python files
Considering these would be enough points to start the project, and even in the further coming hours, I have noticed there is literally nothing really more needed besides the special cases, which would be better to be configured by the user if needed.



Below, you can find the source code of the script, which, by only specifying the project/folder path, will go through all the folder, then all the files, and extract you the needed library names and write them all down to a newly created requirements.txt file. Thanks for taking your time, if you have any furher additions to the project make sure to contribute!


Discussion (2)

Editor guide
jiayanguo profile image

I think you will need to filter out Python native modules. In all, this is awesome.

woosal profile image
Vusal Ismayilov Author

Thanks for the feedback, @jguo. That is what I have been thinking to add for the next release. For the feedback, again, thanks in advance, it is deeply appreciated.