DEV Community: Aquib Baig

The current state of hackathons in India

Aquib Baig — Thu, 25 Jun 2020 13:41:54 +0000

Introduction

There have been a lot of technical hackathons over the past few years claiming to generate ideas to solve problems of the modern world and making it a better place. I would not like to present my views on the state of hackathons world wide because I certainly have not experienced those but in my career I have experienced a handful of them in India. I would like to bring them to light and majorly discuss why I am not ready to not attend one ever again. We will discuss major facts and figures in terms of an expectation vs reality graph in order to visualize the details minutely.

A means to churn ideas?

Expectations
Starting off with a rather controversial title, I would want to suggest that this post wont be a rant but a detailed explanation of my experience over the years. Hackathons are a major platform for a group of students to work on a product that targets at solving a modern problem. Solving a issue would essentially prove a team's worth of problem solving and even win them the prize money.

Reality
The hackathon is just a means to find a way for organizers to get the best solution and use them in their use case. I went in a hackathon organized by the government where an idea of online bus service was sold to a company who patented the application.

A waste of work?

Expectations
When you are selected for a big hackathon, and by "big" I mean those which are by the government, state, national or world wide, you normally would expect your solution(if selected) to be an end-product which would make a difference in the real world.

Reality
I wonder how many teams will be able to drive their products to reality in terms of production and marketing after the competition. Mostly, the teams are happy after a prize money and an entry in their resume.

Time constraints are a hoax

Expectations
We go to the competition hoping it's fair at least in terms of time limit. A 24 hour hackathon would essentially expect a team to solve something within 24 hours only.

Reality
Most hackathons expose their themes and problem statements before the competition where the teams come prepared with their products, an unfair means of competition. Also, most use modules already available on the internet.

Judgement

Expectations
Anybody would expect the judges to be focused more on the product than the presentation! Not nitpicking about machine learning in a project, and should be aware of the fact that very few complex models can be trained overnight and most teams will use pre-trained models on the internet using APIs which is pretty easy. The correlation between a solution and the problem is what judges should be looking for.

Reality
In many of the hackathons I've been in, the teams try to make up their product with a good presentation at the end. And if you want to win in a hackathon, use machine learning just to beautify the project which doesn't add value either to the solution or the product.

Conclusion

I am not against any hackathons or competitions but to be honest, this is what I experienced. I would basically say that hackathons are mostly for the experience and community and a win or loss doesn't determine your worth. Thanks for the read.

A notifications system written in symfony, GSoC 2019

Aquib Baig — Fri, 05 Jun 2020 15:12:09 +0000

LibreCores.org lists free and open-source “IP Cores” on the website for the community to view and use. "LibreCore" is such an IP core that is created and distributed in the open-source spirit. LibreCores is a project under FOSSi Foundation that creates open source hardware for the community to use at their discretion.

Introduction

Currently the website extracts the project readme, license and calculates code quality(from issues, commits, trackers, etc. in the repository) and renders them on the project page, along with links to the project homepage and git repository. It also has a classification system for better discoverability of such projects. Also, there is an indexed search using Algolia.

Abstract

A user browsing for “IP cores” on LibreCores may be willing to be informed about events that happen in the website and the community may want to notify their audience with relevant information. Notifications are powerful tools to inform users of project crashes, introduce them to new features & updates, and inform them about new messages. These are system generated by the app. This can be accomplished via emails or text notifications to notify users about various events.

Timeline

In this project, I worked on developing a Notification system for LibreCores. In the existing website, users could add their projects to LibreCores, add classifications and discover other projects efficiently. There was scope for developing user interactivity on the website in the form of system generated notifications,so I worked on an idea to deliver this. The plan was to build a system that will generate a notification message after an event has occurred. As a user, I can click on the bell icon on the navbar and view the list of notifications that I have received and if I’m not logged into the website currently, can receive notifications as an email too. So, my project was divided into the following phases:
1. Improvement of UI Libraries
2. Design and implement a notification system for LibreCores
3. Improvement of code infrastructure
4. Write tests for the implementation

Resources

The proposal for my project can be found here.
The entire Project tracking logs can be found here.
The links to all the issues that I have created/worked upon during my GSoC period is
here.
The links to all the Pull Requests that I have created during my GSoC period are
here.
The corresponding project can be found on GSoC 2019 website.

Coding Period

Improvement of UI Libraries

During the community bonding period, we found out that the current Assetic bundle was going to be deprecated. So, we had to move on to modern frontend asset management tools such as webpack encore. Migrating legacy code to webpack is a very painful task but in the end it's worth it. We had a handful of issues in the frontend such as cache busting, deprecated assetic bundle and by using webpack to handle all our frontend assets, we were able to solve all of them at one go. Also, features such as hot reloading and file watchers came to be very useful. The Pull Requests I made in this period are:
1. Migrate to webpack - #353

Design and implement a notification system for LibreCores

This turned out to be the most vital project goal in my GSoC timeline.

Design of the Notification System

My mentor Philipp Wagner and I had thought of a design beforehand for the notification system but in course of implementation there was
a major problem to synchronise notifications. We won't want UI lag that would be caused in processing the notification itself. It would cause a significant performance drop when the personalized messages need to be built on the spot when a user triggers the notification. An alternative is to generate the message when a user visits the page. However, this would decrease the performance slightly for many users and each notification, and it would be difficult to send the notification to any other destination, like email or with web notifications.

Therefore, the best option is to process the messages in a background process where the heavy lifting of generating the messages do not affect the performance of users. We came up with an architechture enhancement where every notification is processed in the background.

We used RabbitMQ for our use case publish subscribe AMQP based message queue to which we would send notifications to a queue and we created consumers which would pick them up directly from the queue and execute them parallely. It is a message broker that helps to manage the real-time message exchange on web server.

In this phase, I made the following PR's:

[GSoC 2019] RabbitMQ architecture for notification system - #382
[GSoC 2019]Notification System: Mgilet/notification-bundle - #383

Notification Consumers

Rendering notifications to the UI using WebNotification consumer

WebNotificationConsumer will fetch notifications from the queue and process it to render them on the navbar of a specific user on the LibreCores site. The notifications that we intended to share
through this are repocrawler failures for your projects, if your project was added to LibreCores, etc. Apart from notification dropdown which contains unseen notifications, also created a Notification inbox which will contain all of the notifications for a specific user, both seen and unseen.

The PR's I made during this phase are:

[GSoC 2019] Configure Web Notification Consumer - #384
Rendering web notifications to the UI and Notification Inbox - #385

Command to create notifications

Since notifications are completely dependent on user interactions with the website, they are difficult to debug in development. So, created a Symfony console command to create notifications from cli.

php bin/console librecores:send-notification <subject> <message> <type> <recipient>

The Pull Request corresponding to this are:

Command to create Notification - #389

Notification settings

A user can always allow/disallow if he wants to recieve notifications from LibreCores. The option to configure notification settings lies in the user profile.

The Pull Request corresponding to this are:

[GSoC 2019] allow users to configure notification settings - #391

### Sending notifications as emails via Email Consumer
Important community information like details about events and new provisions can be sent directly
over the email to the users. EmailNotification Consumer processes notifications and sends them to
associated users.

The PR's I made during this phase are:

Implement Email Notification Consumer - #395

Writing tests for the implementation

The goal of unit testing is to segregate each part of the program and test that the individual parts are working correctly. It isolates the smallest piece of testable software from the remainder of the code and determines whether it behaves exactly as you expect. I wrote tests for the consumers during this period and
the PR's I made during this phase are:

tests for web notification consumer - #404

Additional Contributions

Apart from the main deliverables above, I also contributed on a few other issues, such as:

Annotations for routing configuration - #301
Custom ranking configuration for projects in algolia - #302
Create a notification service - #401
UI fixes - #304 #403 #414 #418

What tasks were accomplished during this time

Tasks	Planned	Completed
Migrate to webpack	Yes	Yes
RabbitMQ architecture for notification system	Yes	Yes
Implement Web Notification Consumer	Yes	Yes
Rendering notifications to the UI	Yes	Yes
Implement Email Notification Consumer	Yes	Yes
Test out the notification system	Yes	Yes
User Feedback Mechanism	Yes	No
Include user feedbacks in search UI and projects page	Yes	No

My GSoC experience

Overall, GSoC was a really wonderful experience for me. I came to know about a lot of things in Open source development such as:

Importance of small actionable pull requests
Agile workflow
Managing large codebases
Relational database relationships
Message queue systems like RabbitMQ
Object Oriented Concepts and clean code

I would specially like to thank my mentor Philipp Wagner for being extremely patient and helpful towards my mistakes and contributions.

Future scope

This is what I did in my GSoC period but there is plenty of scope for future work. We need to store user generated feedbacks for projects listed in LibreCores and integrate the data to the search UI.

Thank You!!!

Javascript under the hood

Aquib Baig — Fri, 05 Jun 2020 14:58:52 +0000

We tend to think javascript as an on-the-go language for the browsers but deep down it has some dark secrets that need to be unveiled. We will have a thorough understanding of how javascript works and what is happening in the background of the browser when you are writing some piece of javascript code, synchronous or asynchronous.

The Core Of JS

On the core, as we all know javascript is a single-threaded language which means that it can do a single task at a time. It is also synchronous, which means that your code will execute line by line.

console.log("abcd")
console.log("bcdf")

prints out abcd first and bcdf after that. This broadly means that if you have a bunch of statements which take some significant amount of time, such as fetching from backend, while loops, etc. then your code execution is stopped at that phase until the processing occurs and after that only we move to next bunch of code.

console.log("Hello")
let i = 0
while(i < 1000) {
    i++;
    console.log(i)
}
console.log("What's up?")

If you execute this piece of code, you will see that first Hello pops up on the screen, after that, we see numbers from 1 to 999 for about a few seconds and finally, we see What's up on the console.

Why is this a problem?

The problem occurs when we run the code in browsers. While our javascript runtime(v8 as we know it) is busy executing the piece of code, our friendly-neighborhood browser is frozen. Even buttons and links won't work in this case. Ah, that's a big problem! This happens because the DOM is already painted and is not looking for anything to trigger it's re-rendering/ repainting. So, how do we solve that?

Asyncronous Processing with JS

To find a way out of this problem, we use asynchronous processing, which doesn't sequentially execute the code, essentially chopping off the pieces of code which would take a significant amount of time and execting other parts of the code. One simple way is to do a setTimeout() .

console.log("Hello!")
let i=0
setTimeout(() => {
    while(i<1000) {
        i++
        console.log(i)
    }
}, 2000)
console.log("What's up")

When we run this code, we see that at first Hello! pops up on the screen and then What's up!!! That's crazy, isn't it? It seems that our setTimeout() function has magically been removed for other less time-consuming code to run. So, after a few seconds(2 seconds), setTimeout magically appears to run out of nowhere and prints 1 to 999. That is exactly how we wanted our program to behave. Isn't it? The piece of code which takes more time should not block other parts of the program. Aye, therefore we call Javascript as non-blocking. Cool!

Other ways to do asynchronous processing include using Promises or async/await, those are not in the scope of this article, maybe for the next one!

What happens to the runtime in asyncronous processing?

Javascript runtime has no idea about various requests that we make in our browser, such as xhrHttpRequests, setTimeout, etc. v8 just contains the following data structures:

call stack to keep track of where your current position in code is, or which part of the code is running
heap for memory management So, when we make ajax requests and call setTimeouts what is the entity that handles such things? The answer to that is Browsers!!. Browsers come with certain elements such as XML parsers, rendering engine, xhrHttpRequests, setTimeouts, etc. as a part of collective API's called as web API's. So when you write these functions in your code, the web API's present inside the browser handles that.

Synchronous processing

Let's look at the above two examples to understand how things work on the inside:

console.log("Hello") ...... (1)
let i = 0
while(i < 1000) { ...... (2)
    i++;
    console.log(i)
}
console.log("What's up?") ..... (3)

Here first (1) comes up in the v8's call stack and finshes execution and is popped up. Then, (2) comes to the call stack and after a few seconds of processing, it gets popped off. After that, (3) gets into the call stack and gets executed and popped off. This is synchronous.

Asynchronous processing

console.log("Hello!") ...... (1)
let i=0
setTimeout(() => {      ....... (2)
    while(i<1000) {
        i++
        console.log(i)
    }
}, 2000)
console.log("What's up")   ........ (3)

In this asynchronous example, at first (1) is pushed into the stack, gets executed immediately and popped up. Now, when (2) gets inserted to the stack, v8 knows that it doesn't have the web API's to handle this operation, so it pops out (2) from the call stack and starts a timer for the function in a separate area. This timer will start separately and isolated till 2 seconds/2000 milliseconds as inferred from setTimeout function. In the meanwhile in v8's stack (3) gets pushed into, executed immediately and popped off the stack.
After 2 seconds have passed, the timer moves (2) to a temporary queue to prevent these kinds of functions from interacting directly with the call stack which will create mismanagement with the execution flow. Here comes into effect a really important component called as the event loop which checks if the call stack is empty(which is true in this case as (3) has been executed and our call stack is empty) and if the call stack is empty, it pushes the first element in the queue to the call stack for execution. Wow! Read that again.

credits:miro.medium.com

Use case

This is done to prevent all the asynchronous code to be executed after the synchronous to prevent blocking. And it is a very practical approach dealing with writing real-world in-browser javascript.
Another thing is if we use setTimeout(() => {function()}, 0) in the above example in (2), it doesn't execute suddenly after 0 seconds as it should, but it executes after all the synchronous code has finished executing. How?

Remember, the event loop...

First (1) will be pushed into the stack and executed. When (2) will come up, it will be removed from the stack because web API's will handle it and a timer will be set up and immediately it will push this function into the temporary queue(Because here time interval is 0 seconds, not 2000 milliseconds as in the last example). In the meanwhile, as (2) is popped off, (3) has entered in the call stack. So, the event loop will know that (3) is in the stack and stack is not empty, so it won't push (2) from the queue to the stack just yet. After (3) has been executed successfully, then only it will be pushed. So, the output will be the same in each case.

Thanks for the read!!