Built my first web app

Learning and building is fun and that's what i did in the last few months.
I learned flask web framework, SQL, HTML and Bootstrap CSS and built my first web application. The learning journey and the experience of building web application for first is truly cool. I learned something on how websites work and also on back-end and front-end systems.

What I learned?

Firstly, I learned about flask web framework. This is simple and easy to understand micro framework. But it can be used to build a full website. It has many extensions available to achieve that. I started with simple steps like building a bare login page with username, password fields with login button. I made the login to make the login functionality work as expected. It was cool but doesn't look anything fancy like login pages of Facebook, Instagram or any such websites.

Second, I learned SQL language and used SQLite db for my login page that i built. I coded the app to store login credentials in SQLite db. This felt like a little milestone. I developed routes in flask app for login and logout. I made another route for home. The app redirects user to home after login. This is how many website that we use work. The functionality is good but it doesn't look pretty. The HTML pages that i wrote are bare bones. I check the login pages of many websites and took inspiration on how should a modern login page look like. I formed an idea on how my app's login page should look. Now, i need to make it look pretty. Next thing I learned is about CSS. But it is very vast and there are many things that CSS offer to customize the web pages. I chose the easy method of using CSS. That is Bootstrap CSS. I felt this is the right choice for my app. It took me a week to understand on how to use Bootstrap CSS elements and components in my app. I use it to customize my login page. I used Bootstrap cards to make the login page. Now the login page looks modern. This made me feel like Yes!, i am building something exciting.

Now my app has login page, logout page. The home page is empty. The login page should also have an option to create new user accounts and also let users to reset passwords. So i built that and made a fully functional login page for my app. During this i created new HTML pages for new user account creation and password reset. I also created new routes for these in flask app.

The app has functional login system. But I have no idea on what kind of website to build. Checked YouTube on flask projects for some inspiration. I liked corey schafer's flask tutorials. He built a blog style web application. Even though the tutorials are old. But they helped me get some inspiration on building my app. I finally settled on the idea to build an online bookstore app. Just like Amazon in the early days but not much fancy but a functional online bookstore web app.

First thing needed to build an online e-commerce type of web app is data. In my case, i need books data. The app should display the books and their data. I searched online for books data. The data is not clean that is the problem. I checked Kaggle's datasets because they have some good datasets for machine learning competitions. I chose Kaggle's Goodreads books dataset. It is one of the highly rated dataset. The dataset has some rows with not readable characters. So i had to clean them. This took some time.

Why the data needs to be cleaned?
I tried to import the books data to SQLite db. But it failed because of unreadable characters in some rows in the data. After cleaning the dataset, i was able to import the data into SQLite db. It has 12000 to 13000 books data. This is good data for my web app.

In the home page of my app. I want to display the books data. I used jinja template engine to display the books data on HTML page. It displays books title, author, publisher and other book details. But i felt something is lacking. I want book cover images to be displayed and then books details. Just like on Amazon's website where there is product image on left and it's details on right.

The books dataset contains ISBN and ISBN13 numbers. This became handy to search books covers. But it is tedious work to get cover images for 12 - 13 thousand books. I searched for any free api's available. Found out open library offers book covers api for free. I used that api in my flask web app. It displays book cover images for most books but fails for some. As there are no cover images for some of the books.

Every time i check my app, it is functional but not responsive. It takes few seconds for the app to load image from the api and display them. This is not good experience. I tried downloading few images manually and tried displaying them. This is fast compared to getting images from api.

I wrote a python script to download images from covers api. I check the open library website. They clearly mentioned not to scrape book covers images in bulk and offered archives for bulk download. I used that and downloaded cover images for all the books in my books data. The cover images are missing for few hundreds of books but I images for all the remaining books.

App is working great till now. But still not great experience. I learned about web application security issues like hacking user's credentials through HTML editing, login sessions, cookies and csrf(cross site request forgery) etc. I want my app to have good security. So I learned about flask login extension which handles login related session issues and implemented that in my app. Next is flask forms extension which handles forms in HTML using python code. So no need to worry about client side HTML editing or hacking. Flask forms allows you to write or create HTML form in python classes. It also let us handle form validations. Flask also has CSRFProtect from flask_wtf.csrf module. That let's entire app protect from CSRF attacks. Now the web app's security is good. For this to work, need to declare a secret key in flask app.

Next is the database. Till now, I used SQLite. I wanted to use a real database and host my website online. So i chose PostgreSQL. It is old, reliable and open sourced. Learned how to work with PostgreSQL like creating databases, adding tables, importing data from a CSV file to a table and other necessary tasks required for my app. It took few days to integrate PostgreSQL database to my app.

I wanted to host the app online and searched for free providers. I decided to go with Amazon's AWS free tier. Because it is famous and I liked the free tier option. There are many tutorials on how to use it if i ran into some problems. So, I used EC2 instance with Ubuntu and hosted my app. For this to work, i learned something about NGINX which is a reverse proxy server that routes my app's default local host port to an ip-address or custom domain. Got a custom domain and made my app available at the domain address. Flask need to be run with production servers instead of default development server. I used gunicorn server to run my flask web app.

The app is live on the domain now. It displays books details on home page and has a functional login system. That's it. I want to make it a complete online book store. So i wanted users who logged in able to simulate buying books with orders and order history. For this to achieve i need to make changes to database tables. Need to create new tables and form relationships between tables.

Handling this is quite confusing at times because i am dealing with HTML, Python and SQL at the same time. I learned about SQLAlchemy and ORM's at this time. Took a week to learn SQLAlchemy. It was confusing at first because, the online tutorials are describing in different ways like using different syntax's. I went to the source and read the documentation. It's not the easy route but it made me realize that SQLALchemy released new version 2.0 which has many new features and new syntax. Many YouTube tutorials are not updated. They are few who made videos on new version. I got basics of using SQLAlchemy. I implemented it in my app. Faced a lot of errors and bugs while doing it. Took help from ChatGPT to understand the bugs and fixed the code. This process took more than a weeks because i also modularized my mode using Flask's Bluprints. Till now i worked in a single app.py file which made me frustrate while debugging and fixing my own code. Some Reddit groups and other online blogs mentioned about using flask blueprints to organize code. I completely restructured the whole file structure of flask web app just like i did with my first python app TidyBit. Many ways working on this is similar to that. TidyBit has UI and python code for logic so integrating both is achieved easily by writing most of the code in modules. This is a similar experience.

Another hurdle that i faced was with modifying the database tables. If new tables are created or existing ones are modified the app failed to work. As i used SQLAlchemy, I defined classes for new tables in models.py file.
It threw new bugs. I tried to fix them. I decided to delete entire database and create all tables right from the start. It worked but if i wanted to modify a table column. It broke again. So i found Flask Migrations to handle such situations. It is like a Git for Databases that makes versions of the changes and tracks them. This is like a boon. It didn't work initially but I learned to make it work. The problem is if i made changes the changes on my local machines and sync with my GitHub repository and pulled it on EC2 instance. The migrations fail. I learned that migrations made on one machine won't work as expected on another machine. So every time i make migrations i keep this in mind and made proper changes in EC2 instance.

Now my web app has cart, cart items, orders and order items. I made cosmetic changes to the HTML pages by adding Cart Icon, Orders button and a separate page to display order history. Now i think the app is complete. It simulates a fully functional online bookstore app.

Thanks for reading this. If you are interested more on the web app. Please check it: Flask Web Application