DEV Community: Vitalik-Hakim

Building an Online Timetable - Vitalik-Hakim.

Vitalik-Hakim — Sat, 20 Aug 2022 14:30:00 +0000

Summer Break and the Idea

During the summer, probably two weeks before school reopened. I had an idea about making my school timetable accessible from anywhere and everywhere in the school. Just incase my laptop is not working. This however has been easily solved by the school by making everyone's timetable open sourced. So that it will be easy to find your current class on someone's laptop. This is also a major contribution in my project. The open source nature of timetables made it easy for me to do this project.

The problem

Okay so this is not really a major issue in the school because everyone has PDFs of the timetable of others but I wanted to take it to another level by making it available on the web for everyone in the school. This as an alternative to searching through many pages of PDFs to find your Timetable schedule for the current time.

The benefits or addons of this product to the traditional looking through PDF's

Students can find their current period and the next three periods by just searching their name. The results appear in less then a second
Teachers and students can search other students name and find the class they have and find them if they are looking for them.
The Indications of Period Start time and Period End Time will help students realize how late or early they are into the period.
The timetable could always be edited to adjust to changes if a student drops a class or there is a mistake.

Building the project

This project took about two weeks of free time to bring up a working prototype.
I used the previous year timetable to test for functionality and user experience.
I am currently rolling out a beta testing program with this year's timetables. The format of the timetable has changed a little bit, so I had to change some part of the code to fit the new timetable structure.

Coding and Technical Information

I talked a bit about the timetable being in a PDF format.
So indexing something that's in a PDF format will be quite challenging based of the final form of data that you need.
In my case I need a JSON file with all the student as the main key and the other keys being subjects, days , periods(time).

But how do you go from something that is almost un-editable in its raw state to something well structured like a json?
This was a big task actually and I had no plans whatsoever to enter data manually.

I would want to jump ahead of myself and list how the PDF becomes a structured Json.

Process

PDF -> JPG -> Edited JPG -> Excel Sheet -> Edited Excel Sheet -> JSON(Unstructured) -> Structured JSON

PDF - This is real timetable and raw data that is going to be processed. This is my timetable from the last year(MYP5)
JPG - To work on it I first converted the PDF into an image using Python PDFtoImage Library
Edited JPG - Now I use Python CV2. I drew lines on the timetable to cut new boxes for Lunch and Snack time because they become empty spaces.
Excel Sheet - This is next stage which involves converting the image into an excel sheet. This excel sheet format will make it easy for me to remove some words and add some in the timetable
Edited Excel sheet - This sheet is the new cleaned data version of the original converted from the Image. Whitespaces, Empty cells and table labelling has been done to this Excel.
Json(Unstructured) - This is a json file that was made by converting the edited excel sheet into a json format. The tables that were labeled in the edited excel sheet are now keys and values. However, Some tables merged by mistake and uncertainty.
Due to this error I also need to find those instances and replace them with a "&&" token. Just as an identifier for other later programs who would work on the json such as the Backend, API and the Frontend.
Structured JSON - Finally this our master file with all the data that may be needed by the API rest of the web Application.
I will give a sneak peak into how it looks like. This was semester's timetable btw

"ABDUL-HAKIM AREMEYAW": {
            "0": {
                "0": "Free Period",
                "1": "Monday",
                "2": "Tuesday",
                "3": "Wednesday",
                "4": "Thursday",
                "5": "Friday"
            },
            "1": {
                "0": "Period Zero",
                "1": "FORM\nMEETING\nREGISTRATI\nON",
                "2": "FORM\nMEETING\nREGISTRATI\nON",
                "3": "FORM\nMEETING\nREGISTRATI\nON",
                "4": "FORM\nMEETING/\nREGISTRATI\nON",
                "5": "7:15 - 8:00\n\nASSEMBLY"
            },
            "2": {
                "0": "Period One",
                "1": "MC\n\nMATH STD1\n\nMR4",
                "2": "TO\n\nPHY2\n\nPL",
                "3": "CT\n\nENG L/LIT3\n\nER4",
                "4": "AK\n\nFRE4\n\nAVR1",
                "5": "CA\n\nMATH EXT2\n\nMR3"
            },
            "3": {
                "0": "Period Two",
                "1": "Snack Break",
                "2": "Snack Break",
                "3": "Snack Break",
                "4": "Snack Break",
                "5": "Snack Break"
            },
            "4": {
                "0": "Period Three",
                "1": "PH\n\nICT4\n\nMLR1",
                "2": "MATH EXT2\n\nCA MR3",
                "3": "NM\n\nDRA2\n\nER3",
                "4": "ENG LILIT3\n\nCT MR1",
                "5": "Free Period"
            },
            "5": {
                "0": "Period Four",
                "1": "FRE4\n\nAK AVR1",
                "2": "MD\n\nBIO3\n\nBL",
                "3": "MATH STD1\n\nMC MR1",
                "4": "NM\n\nDRA2\n\nMR2",
                "5": "RH\n\nECONS1\n\nHR2"
            },
            "6": {
                "0": "Period Five",
                "1": "Lunch Break",
                "2": "Lunch Break",
                "3": "Lunch Break",
                "4": "Lunch Break",
                "5": "Lunch Break"
            },
            "7": {
                "0": "Period Six",
                "1": "BIO3\n\nBL\n&&\n PHY2\n\nPL",
                "2": "ECONS1\n\nRH HR2\n&&\n CHEM1\n\nCL",
                "3": "SM\n\nTS3\n\nMLR3",
                "4": "DA\n\nVA1\n\nAR",
                "5": "VE\n\nCHEM1\n\nCL"
            }

            "8": {
                "0": "Period Seven",
                "1": "Free Period",
                "2": "MATH SUP2\n\nEAA MR2",
                "3": "DA\n\nVA1\n\nAR",
                "4": "Free Period",
                "5": "14:50 - 15:40\n\nMATH SUP2\n\nEAA MR2"
            },

This is how the final data looks like and this can be queried by our API and then processed to look presentable on our frontend.

Backend, Frontend and API

After the first challenge has been completed and we have JSON data.

Lets now talk about how the web Application was built.

Backend

I built the Backend First because it was exciting.

I used Python's FastAPI framework to build the backend application. I used this because of the Asynchronous nature and how fast it was. And also the UVICORN SERVER. I will be dealing with a lot of requests so I had to decide on a simple but fast framework and I settled on FastAPI.

The backend was simple it would handle GET request with params such as name and time. The time was internally generated by python with a Accra/Africa Time-zone because I will host the app on a US server.

Example when a request like this is made to the server

http://127.0.0.1:8000/load/students This is request to the server will return the list of all the students. This is used by the Frontend for Autocomplete if the user is searching a name.

http://127.0.0.1:8000/search/{name}` A request like this make to the server will return a the current period data and the next three periods for the that matches the records in the json file.

Other requests will be made for special cases.

Frontend

So fortunate for me during the summer I interned at two software companies. One company used Vue.js and Laravel as their tech stack. So I obliged myself to learn them. I already had experience with Laravel but not Vue.js. I had always done React.js. So it was very exciting to try out something new.

I picked up Vue.js very fast. I was even surprised how I did it.
So when I had the idea to do this project. I said why not use Vue.js as the frontend to enhance my skills in the framework.

So I started working on the frontend.

I spent so much time on the CSS and other things I needed to make the frontend work. I had the experience if throwing in some little jQuery to make some magic work too.

The Date and time was also done in the frontend. which was rewarding to do.
I managed to finish that so quickly than I thought.

API

I won't spend time on this because the API is FastAPI and was queried using the normal fetch and return. I will use AXIOS later but I want everything to be simple for now.

Firebase

As a student I don't have any choice but to utilize the Firebase hosting service to host my Frontend Vue.js.

Heroku

Again as a student I just had to host my backend FastAPI on Dyno on Heroku. Which has been quite rewarding and helpful.

Moving forward

I think I will still be working on this to improve and I am also accepting suggestions to make it better because its not fully fledged done and its not open sourced yet. It will be open sourced once it is live for the whole school.

I really learnt a lot by doing this project I got introduced to many practices and many ideas of doing things that also introduced in my code.

There were stumbling blocks at every stage. I just took a break outside to do sports or just cycle, And I would find the solution to the bug. Trust me it works every time! . Some problems seemed like a dead end for my project but I managed to overcome them. I am looking on to building new projects and learning more. This is next major project I have built since the inception of Antimony(CryptoCurrency)

Introduction to my PoW based Cryptocurrency

Vitalik-Hakim — Sat, 19 Feb 2022 18:45:23 +0000

Antimony Project

This is my first post on Dev.to.

I started a programming project in school last year. I planned on researching Blockchain Technology and its applications. I also dug into the theory side too.

I proposed a final project of making a series of videos with detailed explanation of Blockchain and the Genesis of Blockchain the Bitcoin. I found out a lot and also learned a lot from this project. I was tasked to produce a product as an end goal.

I decided to use the detailed videos I made, explaining Blockchain's History and its use. However, I felt that would be a little weak for an end product. Most students in my school had their products as videos and it was a pretty common and effective thing to do. But I wanted otherwise or both.

I therefore thought about making a Blockchain Application entirely just to demonstrate how the concept works. I won't go into the details of what a Blockchain is because I don't want to bore you with the same things you have heard over and over again. Well, A Blockchain is a decentralized ledger of transactions. I believe that if you are reading this, you should know more about a Blockchain more
than me.

So fast forward, I had done all the paperwork of documenting the process of making this product. Which in reality I could have done in a week but It took me months because of debugging, Missing Libraries, Outdated servers and most importantly laziness xD. Seriously, I went on an internet quest to find these libraries. I would break these files down later in this post. But in all I learned how to use Debian, SSH, Ubuntu(Which I already used), Gitian Builder and many others. I really learnt a lot or I feel like I was learning a lot.

So at this point I had made up my mind on what coin/Blockchain I should make. I had a hard time of 5mins coming up with a name for it. I didn't want it to be generic or the usual way we know Blockchains as coins. Example Bitcoin, Litecoin, FeatherCoin ... and so many others that end in coins. I wanted mine to be like Ether or Stellar. No coin suffix. So guess what, I was in a
chemistry class and I thought maybe some elements may have some cool names and be unique like my coin. I initially chose radium☢ (You get it? because it is rare :) ). But I changed it because of the radioactivity and some long history. Long story short I chose Sb(Antimony), because Blockchain is against Fiat or what normal people call money. So "Anti-Money" project was on. I had tasked one of my friends and business partner who was a UI/UX designer to make a logo for me. And the Coding began.

The Technical details

So first of all, you guys don't want me to code a functioning PoW Blockchain from scratch do you?, Even Coblee (Charlie Lee). Forked Bitcoin Project to make Litecoin. [Thanks to Satoshi Nakamoto]. Who am I to start from scratch and risk security and years of hard coding and debugging. So I also forked Litecoin to make Antimony. Easy right? Wrong!. It was a nice journey of learning and debugging in that nice Ubuntu Bionic OS. So first of all, I cloned the GitHub of Litecoin, specifically the 0.8 version. I remove a bit of error that would prevent you from compiling your almighty Coin. Then with a little regex I found and replaced all instances of Litecoin in the code with Antimony!. I changed the Litecoin Port Numbers to my own Port numbers. For both Testnet and main net. Next was cleaning up some things and specifying the necessary details of Antimony. I made the total supply to be 5Million (Anty). Anty is the ticker symbol for my coin. Just like Bitcoin has BTC or Litecoin has LTC.

Replacements and Coding

I went on with more other coding things like replacing the Epic time and also the Timestamp. Like Bitcoin has a headline in its genesis Block. Litecoin one has a headline from the time of the death of Steve Jobs. RIP. I generated a new Merkle root and before that I had gone through a tedious task of replacing the Public key Hex which was separated by colons! ugh!. That part was sheer focus and concentration. I managed to do it though without any errors in both testnet and main net. I made a rookie mistake when changing the peer magic in my coin. I knew it was hexadecimal but I insisted on adding a letter 'g' in it. Imagine and it kept giving me a segmentation fault! Core it bloodily Dumped! I was so frustrated at that time. As any linux user, we know if you have a problem the only way to solve it is by reinstalling a new OS and starting all over again. I don't think this may sound sensible to Mac and Windows users but that's exactly what I did. I installed a new Ubuntu 18.04 or Bionic. I found the problem and solved it anyway.
Difficulty readjustments were done and reward per block was set to 25 ANTY. Everything was going well you think. No, I have a whole battle ahead of me.

Internet file finding Quest

So as I mentioned I made this coin in Ubuntu and I don't think people in y school or my Project Coordinator uses a Linux Distro. So I had to make an Almighty windows executable to share my little Application with the world or more realistically, my school. When I researched on how to do it. I found out I needed to use a new distro for the first time. Which was Debian!. Woohoo! I was getting closer and closer to using Archlinux. So Debian was installed and I couldn't copy and paste freely. What a pain in the foot. That is where an SSH client comes in. I have used Putty before for my Raspberry Pi 4. To connect to SSH, so this wasn't new. But I chose Bitvise SSH client because of the SFTP feature that allows you to download files to the host machine from the distro through an SFTP window. Pretty easy and powerful. So Bitvise was installed. I used a Gitian builder to build the architecture for the windows. At this point I am abstracting the details for better understanding. But Gitian builder is used to Build packages in a secure deterministic fashion inside a VM. Apart from that I needed some files with specific versions to be precise. These files where:

miniupnpc-1.9.20140401.tar.gz
openssl-1.0.1k.tar.gz
db-4.8.30.NC.tar.gz - Berkeley DB
zlib-1.2.8.tar.gz
libpng-1.6.8.tar.gz
qrencode-3.4.3.tar.bz2
boost_1_55_0.tar.bz2
qt-everywhere-opensource-src-4.8.5.tar.gz
boost-mingw-gas-cross-compile-2013-03-03.patch None of them are easy to find either using Wget magic or any other thing. I can bet you that. I took some internet searches and forum scouring to find these files. They are currently in my possession so if you need one of them just send me an email at aremeyaw_a@soshgic.edu.gh. I will be glad to deliver them to you. I don't want anyone to go on such an internet pain quest.

Contribution to Open Source

So I also faced errors even after finding these bad boys. I kept getting a system error but I solved that using help from stack overflow and some Linux forums. Bitcoin.org too was also my close friend. The error was that I haven't started apt-cacher-ng yet. so after solving that I got a new error. With this error it took about three days to find a solution because it was a problem with the code or not with the code but the Ubuntu server location. Ubuntu has moved some archive files from Archive.ubuntu to old-releases.ubuntu. But the gitian builder was still fetching from that place. So as a normal bug solver. I edited the code on GitHub and sent a pull request for DevRandom to review. Guess what MY PULL WAS CORRECT SO HE MERGED ITT!!!! I was soo excited that day that I showed it to all my friends whether they understood or not. I was happy that I had contributed to the software which is literally the backbone of all Altcoins who build through Gitian. I was also happy that I had contributed to the same repository as the names like Gavin Andresen, Luke Dashjr , Hebasto and other prominent developers in the Bitcoin Development community. The link of my two pull request can be found here

I was in heaven that day, my school day was very light. I was a bitcoin superstar but nobody knew sadly :(. Anyways I look forward to making great contributions to open source like this.

When this was fixed I managed to go through the process of making an exe file for windows successfully but it took time.

Here is a screenshot of the Build/out folder

Fast forward the present and future

I managed to run the client on windows and sent myself some Anty coins from Ubuntu to windows and to some of my friends. I didn't mention I hard coded some Ip addresses into the code to make some friends servers or nodes on the network and its working fine. I am yet to submit the product and I have mined at least 30,000 Anty for myself. I have sent a few hundred for my friends. I have also set up a normal Block Explorer using the Iqidius Block explorer. I am thinking of setting up dns Seeds but that will be after the project.
Here is a screenshot of my stash of AntY.

The Official Repository to Antimony is here.
It has the Windows Binary. So make sure you download and connect to a node. Mine some Anty for yourself. Have a good day