DEV Community: Andrii Leitsius

Follow GitHub repos the way you like

Andrii Leitsius — Tue, 31 Aug 2021 20:07:39 +0000

Hi, DEV!
I'm looking for GitHub users to help me with feedback.

GitHub notification sucks. It's so easy to be overwhelmed when there is a lot of activity in the repositories.

So I build a newsletter service to be up to date with desired repository activity:
https://ohmycode.cc

Any feedback is welcome!

Satisfy your very first user

Andrii Leitsius — Sat, 03 Apr 2021 09:18:28 +0000

I have good news for you. When you launch something, your first user appears very quickly. And it's you.

First, you solve your pain, and then you solve this pain for others.

But there is also bad news: resolving pain for you may not wholly solve others' pain. Or maybe it's not so painful for them. Pretty harsh, right?

I have been thinking about this for a month since my previous post. When the previous issue was in the email boxes, PoC had already been working and meeting my needs. I received a report every morning.

I've made a lot of useful but not important improvements this month. This time I will skip them and the technical details of their implementation.

Tell me, are you interested in reading low-level implementation details? Please reply to this letter with even one sentence, to let me know. Interested in your opinion.

Among handy things, the name is first.

OhMyCode

It seems easy and friendly to me. And not even overused by other products.

Next, I realized that my pain solvers are quite specific, and it isn't actually a pain for others. Two out of two, yeah?

Why specific? I am interested in very particular data very peculiarly grouped—for example, the total downloads of all releases' assets. And I don't need info about new commits in the repository because only I have the right to push and merge.

But why it isn't a pain for others? I can't find a similar service, so I decided to ask the audience if they would find it helpful. I prepared a post with a link to the waitlist, published it on several subreddits, IndieHackers, and HackerNews.

And didn't get any reply or upvote. It's good that I've already gathered a dozen people interested in the beta; because of such a result, the project could be curtailed.

So I decided to go with something simple so people could try it. But the current version is not suitable at all. Just because it solves only my problems.

Next, I'll try to abstract from them and make a simple MVP that will be useful for more people.

And while I'm working on it, you might find handy SourceKarma. Simple tiny service to discover how people react to your comments on GitHub. I want to do something just that "easy".

How I create GitHub project reporting from scratch

Andrii Leitsius — Fri, 05 Mar 2021 16:41:22 +0000

The previous issue was about the idea and scoping of Proof Of Concept. And now it's time to implement it! Below I will tell you how I came from nothing to this:

One of the most important things for PoС is the speed of implementation, so I decided to use boring technologies that I’m familiar with.

All source code you can find on GitHub here.

Data collection

First of all, I started with a python script that collects data from GitHub.

This source has two interfaces: REST API and GraphQL. The last one is optimized and recommended. It gives you more information with the same rate limit of 5K requests per hour. However, the REST API has a well-documented PyGithub client library, so I used it as a start point.

Data that I needed:

Numbers of downloads, views, stars, and open issues
Issues and Pull Requests that have been updated
Main referrers and views over the past two weeks

Almost all data can be easily collected by several client methods. Only traffic info requires an access token which can be obtained in the account settings.

Data visualization

Now we need to visualize the views on the repository with a line chart.

Firstly, I tried the most popular visualization library matplotlib. But its configuration didn’t seem clear to me, so moved on with other options.

Stopped at altair, its result from the box was quite nice and simple. But I abandoned it as well while implementing the email sending. The reason was that altair saves the rendered image in a specified path. And I would like to avoid disk operations (writing and reading the image in the same process).

Fortunately, I found a chart generator in pandas (a library for data analysis) that uses matplotlib under the hood. Working with it is very simple: create a table, add data, and plot lines.

I redirected the chart output to the buffer using built-in bytesIO, thus avoiding writing then reading from disk.

Data presentation

Regarding the format of the letter, markdown was the first idea I came up with. As it turned out, it was a terrible one.

At the outset, I found out that email can be only html or plain text. Therefore, the markdown must be rendered in html to have an intended look.

For this purpose, I tried markdown library. It's good to create markdown files and very convenient in working with text and tables. However, it renders the content in html without any styles. Hence, the result was far from a pretty looking markdown in editors.

At this point, my inner desire to make things prettily overcame my need to minimize the effort on PoC.

So I found marked.js and learned that tables are an extension to the original markdown. While I was trying to run CLI of this lib in the console, I got the same result: bare html without css.

Next was markdown-it.js. It was getting even harder because this lib didn’t have a CLI. But the demo looked tempting, so I wrote a js script that reads the file, uses markdown-it, and writes the result back to the file. And you know what? The same result. Bare html.

That was a failure.

And at the moment when I found script that gets styles from GitHub for markdown rendering, I stopped.

The first thing I realized was that I don't really need tables. I can display this information in plain text and it looks readable.

The second is that I don't need a markdown at all. I chose it because I liked its simple and nice look. But if I need to send html, why not to generate it directly?

So I went back to boring technology. I took Jinja2 and rendered html from the template. The code has almost halved, and rationality has doubled.

Data sending

The last step is sending it all by email.

As an email service, I took Mailgun. It’s free for three months with 5K emails/month, and then $0.8 for every 1K emails. The interface is as simple as possible: send a request for a specific URL with an authentication token. Used requests for HTTP calls.

To send pictures in the body of the letter (not as an attachment), you need to enclose them as a MIME object. The name of the object is specified in the image tag, and the email client displays it in the body.

The built-in email.mime handles this perfectly.

Well, the script works, so let’s make it run automatically. Following reliable "boring" technologies, I took Heroku. For the current solution, a free instance and Heroku Scheduler are enough to run the script at a certain time.

So,

The Proof Of Concept is completed.

Next, I plan to save the data to show the trends, make the report more informative and prettier.

And I also noticed the strange behavior of issue filtering in GitHub API, which I need to research.

The main motto of this post:

Use boring technologies

This is almost always the right choice.

Shaping idea into Proof Of Concept

Andrii Leitsius — Wed, 24 Feb 2021 21:07:58 +0000

While working on MeetingBar, I noticed that I refresh the statistic pages on GitHub very often. I even wrote a telegram bot that sends me a count of installs and repository stars every morning. So I decided to make a service for analytics and reports for open-source projects.

Let's start from a blank page

When I open my project page, I have two needs: dopamine and useful information.

Dopamine is stored in numbers. Downloads, stars, views. I want to receive this data often and to be able to compare their values with the previous ones.

Useful information lies deeper. In comments on the issues, updates to the pull requests, new project referrers. It already comes in GitHub notifications, but they are generated for each action. So it would be convenient to receive a full report for a certain time instead of viewing a dozen notifications during the day.

In addition to all this, it would be nice to have a project status visualization that can be shared. Thus, every project on GitHub can get the basis to become an open startup without additional effort.

So I need to build a platform that collects&stores data from GitHub, makes dashboards, and sends reports. It looks like a great bunch of work!

Scoping

Let's start with simple Proof Of Concept.

The easiest way to validate an idea is to build a service that periodically collects necessary data and sends an email report. At this point, we will skip the UI part with dashboards and data storing for metrics comparison.

This data will be collected every morning and send to me by email.

The letter will have the following structure:

First, there will be numbers to satisfy my excitement.
Then issues and pull requests that have changed over the past day.
And in the end, there will be traffic.

The traffic section requires a lot of effort with authorization and visualization. But I decided to include it in PoC because it will cover all my needs for opening a GitHub every morning.

What's next?

The implementation story.

Then I'll try to use it for a week. So I'll understand whether it's useful and what needs to be improved.

📌 If you are interested in this idea, contact me. I'll notify you as soon as I plan the beta.

Subscribe to receive the next issue when it's ready!

Intro

Andrii Leitsius — Mon, 22 Feb 2021 22:58:32 +0000

Hi there! I'm leits.

This is my public journal where I think aloud and share what I learned while making stuff (mostly software and around).

Behind my back are seven years in software engineering. During this time, I saw many products from the inside. Launched and scaled a big bunch of them. Made many great and some not the best architectural decisions.

The thing I learned well over that time:

There is no right way to make new stuff

Every time you start doing something new, you move by trial and error. It's quite healthy to master new skills and build your own opinion.

I already have experience in building things in public.

In early 2020, when I had a lot of meetings, I created MeetingBar to be more comfortable with them. I published this project, and now it has 17K+ installations. The public codebase on GitHub gathered 1.8K stars and 20 contributors.

I realized that my experience in building and running projects can be useful. Or just interesting for someone.

Now I'm working on an analytics service for the open-source community. In this place, I'll share my trials and errors on how I make stuff.