This article was originally posted on my personal blog
If you ever need to download a website online and its content, this article will let you know how to easily do it. Keep in mind that downloading the website's pages and content means downloading static files like HTML, CSS, Javascript, images, and other assets.
First, we need to download a software called HTTrack. This software will do all the work for you. Go to the Download page and download the installer that works the best for your system. Usually, the website will highlight the installer that's recommended for your system.
After downloading the software and setting it up, you need to run it. It will prompt you to either start a new project (which means start downloading a website for the first time) or resume an old project, which is helpful if an error occurred in a previous time you were trying to download a website or you stopped halfway through. Click Next.
Note: Depending on your system the screenshots will look different, but the same result can be achieved.
Then you'll be prompted to enter Project Name. This is just helpful if you want to reference it later, so name it the website you are trying to download.
I'll be downloading TailwindCSS's website, so I'll name it that. You can also specify the base directory the website will be stored in after downloading. Once you're done, click Next.
In the screen after you need to add the website's URL. You can add multiple URLs.
There are 3 ways to add a URL, either by pasting it in the textbox, or by clicking "Add URL...". This one is helpful if the website you are downloading requires some type of authentication. You can then enter a username and password that should be used for the website. You can also use the "Capture Link" method if the authentication to the website is a bit more complex.
The last way you can add URLs is through a txt file. If you have a .txt file that contains a list of URLs, you can choose to open the URLs from the file.
In the case of our example, I don't need authentication to access TailwindCSS, so I'll just paste the link in the textbox.
You can also set choose options like Proxy, or files to scan like zip files, mp3, or other types, choose the language, add HTTP headers, and much more. To access these options, click on "Set options".
For example, I'll add scanning for zip files as well. To do this, click on "Set options..." then the "Scan Rules" tab:
Once you're done, click Next.
In the last step you can choose to save settings for later or what to do after the download is finished. If you don't need to make any changes just click Finish to start the download.
Now you will see the website download in progress. You can see details like the number of links scanned, files written, errors, and more.
Depending on the size of the website, this might take some time, so you have to wait for it to finish.
Once it's done, go to the path that was chosen in the beginning. You'll see a folder with the project name that we chose. You will see a bunch of folders with different URLs. Go to the URL you chose to download, in our case, it will be tailwindcss.com.
You will find index.html file inside the directory. Open it in your favorite browser and you will find the full website you downloaded!
You can now go through the directories to check the assets and different pages on the website as well.
Top comments (2)
I have search for this before; and there are multiple customizable CLI ways (on SO, IIRC).
But this is one of them.
This is great information to have Shahed. Thank you for posting it!