DEV Community: Timothy

Building a Responsible AI Solution - Principles into Practice

Timothy — Mon, 10 Jan 2022 15:54:47 +0000

Introduction

This is part 2 of my series of posts on fairness in AI, reflections from a participant in the Monetary Authority of Singapore's (MAS) Global Veritas Challenge. In my first post, I discussed the need to adopt a human-centric approach when dealing with issues of fairness in AI systems. After hours of brainstorming, user interviews and smashing bits of code together, we created VerifyML, an open-source framework to help organisations build fair and responsible models. The solution was one of the three winning submissions of the Veritas challenge!

4 minutes VerifyML pitch

My teammates have written about VerifyML in detail, so if you are interested in giving it a spin, check out the following introductory posts:

In this post, I take a step back and share some of the things I have learned while developing this solution. I start from the user interviews and feedback that we have gathered, before discussing existing tools in the model governance space, and concluding with how we designed a solution that addresses the problem.

User interviews and feedback

The competition gave us many opportunities to engage with a variety of stakeholders who were involved in trying to implement responsible AI workflows in their organisations. While the organisations all have slightly different workflows, there were three recurring pain points that caught our attention:

Friction in exchange of information between teams
Difficulty in synchronizing data science code with docs
Difficulty aligning code with regulatory requirements

Friction in exchange of information between teams

Recognising that the problem is not only one faced by data scientists, but a multi-stakeholder problem that involves product owners, data teams, compliance teams and system engineers shaped our thinking and approach to the problem. Responsible AI is a process, not an outcome, and our solution should be the medium that enables the process to be conducted more efficiently and effectively. The solution also must be the common interface that allows different parties to weigh in throughout the process.

Difficulty in synchronizing data science code with docs

This has always been a nagging problem in our consultation projects so it does not come as a surprise that many data scientists also face challenges synchronizing their code with their documentation. In most projects, documentation is often seen as the least glamorous part of the job and is done only after the project is completed. It is usually hastily written in Microsoft Word and stored in Sharepoint or some internal knowledge base.

This leads to two problems:

The documentation does not capture the process of building the model and subsequent changes.
Documentation is not easily discoverable, searchable or editable by others in the team.

Difficulty aligning code with regulatory requirements

This might be more relevant to the competition that we are participating in, but in sectors where machine learning and AI models will be regulated, it is often quite a stretch to align the process which the model is developed with the regulatory requirements which often comes in a form of a multiple questionnaire checklist.

Data scientists do not have the bandwidth or do not think they should be burdened with trying to align their models with regulatory requirements. They also do not know what metrics that they should be using for a given business problem resulting in either a "spray and pray" approach or simply just omitting such fairness analysis.

Meanwhile, accessors / auditors may not have the technical capabilities to understand all the harms and benefits associated with the model.

Tools and solutions in the model governance space

Having teased out the pain points from our users, we wanted to understand more about the lay of the land and what existing tools and solutions are available. The worse possible outcome of the competition would be to spend a month re-inventing the wheel and creating an inferior version of what is out there in the market. We also did not want our solution to be a compliance exercise where the whole thing becomes just a checklist resulting in teams trying to game the system and subvert the process.

With this in mind, we began our explorations in the model governance space. I use the term here quite liberally to refer to the whole suite of solutions meant to help build and deploy machine learning models into production.

The model governance space could be categorized into the following three areas:

Model building and experimentation
Model versioning, deployment and release
Monitoring and quality control

Here's a brief overview of the three areas. Model building and experimentation tools normally include a large library of algorithms which a data scientist or analyst could easily create machine learning pipelines. It includes popular open-source tools like scikit-learn, as well as commercial solutions like Data Robot and h2o which might have data connectors to existing data marts or databases.

To push a model into production, there are additional concerns which the tools in the versioning, deployment and release space aim to solve. This includes obtaining adequate infrastructure to run the model reliably and facilitating easy model release or rollback. Solutions in the MLOps space includes Kubeflow, Pachyderm and Algorithmia.

While tools in the model experimentation space normally include diagnostic charts on a model's performance, there are also specialised solutions that help ensure that the deployed model continues to perform as they are expected to. This includes the likes of seldon-core, why-labs and fiddler.ai.

This categorisation is relatively simplistic and there certainly are numerous solutions that overlap in capabilities and are positioning themselves as end-to-end machine learning platforms.

The problem space that we are dealing with involves machine learning issues but also compliance and operation processes. Some of the existing solutions support responsible AI development by providing model explainability and fairness analysis, but these explainability or fairness reports tend to be consumed by technical experts i.e. the data scientist. There seems to be a gap for solutions to cater to less technical stakeholders, including internal management teams as well as customers.

Existing solutions in the monitoring and quality control space also served as a great source of inspiration in developing a model governance solution. There are some attempts at placing a regulatory layer on top of these systems, but it strikes me as an odd fit simply because these existing solutions do not cater to non-technical users. Nonetheless, the idea of monitoring the results of a machine learning model in production and providing reports on performance drift or degradation in other aspects of a model is something we adapted as part of our solution.

Designing VerifyML

Having bashed our way around the model governance ecosystem, we needed to distil our knowledge to a minimum viable product. When designing a solution for a competition, especially against other competitors with existing products, I find it very helpful to identify a niche to position ourselves. Essentially, we want it to be a competition of ideas rather than who has the best technical implementation. It's also mandatory to have a visual prototype so that the ideas can be easily communicated and understood.

We decided to carve a niche around model documentation and testing. While tools to build and deploy AI/ML models have flourished and grown, the area of documentation and testing has been mostly neglected. If we could create a solution to improve the documentation and testing process, one that includes the needs and requirements of other stakeholders, we think that it would be a nice complement to the existing model governance ecosystem.

Model documentation

Better documentation is a relatively easily sell - it's something everyone does not really like to do but is required to do. As mentioned previously, teams currently document their findings through various word documents, PowerPoint slides, README.md files or (gulp) Jupyter Notebook.

The diagram above maps out the key phases of a machine learning development lifecycle, along with the various pieces of information that should be extracted from the phases and inserted in a model's documentation.

With VerifyML, we are creating a more structured approach to documentation, one that aligns with the requirements of various stakeholders - product owners, developers and compliance. Documentation is done automatically in a way that the production model and the docs are always in sync. By integrating documentation in a data science workflow, we can automate the creation of boilerplate reports, improve integration with existing tools and make it a collaborative process (rather than a chore).

We explored various solutions in the machine learning space as well as in the neighbouring software development space for inspiration and learning. Some notable ones include Git and Github, Google Model Cardsand IBM's Factsheets. One of the main design decisions we faced was whether to go with a plain-text git-based solution or a structured schema approach. Eventually, we decided to build on top of Google Model Card (structured protobuf schema). This sacrifices immediate readability of the file, but makes it easy to be processed across different systems in a predictable manner.

Model testing

Model testing is a relatively foreign concept - software development best practices have not really trickled down to applied machine learning. Testing exists mostly at the framework or algorithm developing stage, but not when it comes to developing models on top of these algorithms. To some extent, a train / test split could be regarded as a test, that of robustness to overfitting, but running it as part of a Jupyter Notebook obscures the intent and makes it hard to replicate.

Our goal over here is to make these test cases more explicit and promote them to first class data objects that should be maintained, just like test suites in a source code. In engineering speak, model tests are system tests or acceptance tests that ensure that the model meets (and continues to meets) certain prescribed specification. For an AI/ML model, this would include measures of model performance as well as additional considerations like robustness and fairness.

Besides the existing monitoring solution mentioned in the section above, we were also took inspiration from continuous integration and continuous delivery (CI/CD) testing tools like Jenkins and Circle CI, on the engineering front, and existing fairness libraries like Microsoft's Fairlearn and IMB's Fairness 360, on the machine learning side of things.

We decided to create a Python library that provides out of the box test methods for common data science use cases. As a proof of concept, we built the library around binary classification problems. Developing on top of a scikit-learn API proved to be relatively simple and allows us to support most of the existing machine learning Python frameworks. We then focused our efforts on re-framing and re-categorizing the available methods in the fairness literature. Rather than provide more than twenty different metrics which either map to similar concepts or are simply incompatible from a theoretical sense, we let users specify the base metrics that they are interested in comparing and specify an acceptable threshold of deviation.

VerifyML architecture and solution overview

In creating the solution, we felt that the synergies of having documentation and testing in a single solution outweighed the complexity of having to develop both parts. Namely, we would be able to create a solution where test results can be synchronized with the documentation, which acts as a source of truth on all details related to the model. This means that docs are not just static reports to be filled at the end of a model building process, but are evolving documentation that iterates through the conception, development and eventual production of the model.

The above diagram outlines the data flow of our framework. Every model that is created is accompanied by a model card. This acts as the a source of truth, where teams can document qualitative and quantiative aspects of the model, including business objectives, performance analysis and fairness considerations. Teams would be able to iterate on the card from the initial project conception to eventual deployment.

The model card can be easily edited by both business as well as technical users, either through a web form interface or through the Python library. For example, a product owner could fill up the section on the model's objectives through the web form, while a data scientist would update the section on model explainability with the help of the Python library.

The VerifyML python library also makes it easy to test an existing scikit-learn compatible model and ensure that the current model meets certain standards. Naturally, the test results can also be easily appended to the documentation to complete the cycle.

We extend Google Model Card by not only supporting the creation of html reports from the python library, but also creating a separate web app for teams to view the results. There's also a comparison view that allows easy comparison between different model runs or model deployments. Since the competition, we have also created a Github Action which automatically parses the model card in every pull request and checks that the model passes all the required test cases.

Conclusion

Are any of the ideas truly original? I doubt so, but I think we did a good job stitching together various components and making it easy for any company to improve their model documentation and testing processes. Our holistic approach to the problem and the fact that it solves common pain points in the model development lifecycle probably won the votes from the judging panel.

Interested readers should check out the VerifyML website, docs or Github code. Feel free to create a Github issue and drop any suggestions or feedback over there! VerifyML is proudly open-sourced and created by a small tech startup. I would like to think that more companies are going to see responsible AI as a comparative advantage or requirement and having an ecosystem of solutions that are not controlled by the interests of large tech companies would be key in driving the sector forward. I look forward to improving the user experience and integration with more machine learning tools over the next year, as well as sharing more thoughts in the space.

Easy Graph Visualization With Motif

Timothy — Wed, 28 Jul 2021 14:16:22 +0000

Motif is a graph viz web app that simplifies analysis and exploration of rich graph datasets.

Inspired by kepler.gl, it makes visual discovery on graph datasets accessible to analysts, data scientists and managers.

It was created to make analysis and exploration of rich graph datasets with numerous node and edge properties easy. With a web interface, we make it easy to share results and analysis.

In this post, I will highlight the main features of Motif and do a walkthrough showing how we can visualize the Les Misérables dataset!

Feel free to try it out at https://demo.cylynx.io. If you like the project, give us a star at our github repo!

Features

Import options

Motif supports a variety of import options including CSV files (edge lists, or node-and-edge CSVs), JSON files (its native format), Neo4j and it also includes a jupyter notebook integration!

For more information on using Motif with jupyter notebooks, check out our post on pymotif.

Layouts and styling options

A variety of layout options are available including Concentric, Radial, Grid, Sequential, Circular, Force-directed, and fixed coordinates.

You can also customize the colors and size of the nodes, as well as their labels. This includes mapping a particular node attribute to a color which automatically creates a legend! Similar styling options are available for edge properties as well.

Filtering and search options

What's an explorer without filtering and search options? Like
Kepler.gl, we provide a time filtering bar and different filtering options depending on the filtered property’s data type (string, number or date).

Getting started with Motif

Simply head to demo url to get started and try out the various features outlined in the post.

Importing Les Misérables

Let's take a look at how we can visualize the Les Misérables dataset. The Les Misérables dataset is included as a sample data so one can simply click on the "Sample Data" tab highlighted blue in the screenshot below, followed by the Les Misérables box, highlighted in green.

Properties panel and tooltips

We can click on the "Select All" button in the node and edge properties panel to configure the attributes that we would like to see in the tooltips.

Double clicking a node and edge will show the relevant information in a tooltip.

Configuring styles

The toolbar on the left provides a list of panels which comes in handy in data exploration and analysis. This includes a layers panel, a styles panel, a filter panel and a search panel.

Let's switch from the layers panel to the styles panel by clicking on the palette icon 🎨.

Feel free to play around with the numerous styling options available ✨

In the next section, I will show how we can create a pretty looking visualization that looks like this:

Use force-directed layout

Force-directed layout is a good choice and tends to create aesthetically pleasing layouts.

The math behind it is a little complicated and involves simulating the repulsive forces within an electrically charged system. Check out this post for more information.

Within Motif, the logic has been programmed and all that is required is to change the layout dropdown to force-directed!

Add a node label

Next, let's label the nodes with the characters’ names. In our dataset, the names are stored in the "id" field. Click on the 'label' dropdown box within 'node styles' and select the "id" property.

Color the nodes based on the ‘group’ field

The nodes have a "group" attribute that was generated by a clustering algorithm. Let's color the nodes based on their grouping.

To do that, we need to change the 'node color' from 'fixed' to 'legend', followed by choosing the "group" attribute as our variable of choice.

Saving your masterpiece

That's it! You should have successfully replicated the example.

To save or take a screenshot of your graph, head back to the layers panel and click on the camera button to take a screenshot, or the save button to download the file in JSON format.

Share the graph via email or slack, and the recipient can simply drag and drop the graph in the import modal to visualize it as you have created!

Conclusion

I hope this post gives you a good introduction to Motif. Would love to see what you can create with the tool as well! If you have any feedback or comments, feel free to leave them here or at our [Github discussions page].(https://github.com/cylynx/motif.gl/discussions)

Introducing Motif Graph Visualization Platform

Timothy — Tue, 13 Apr 2021 10:08:14 +0000

We just released a demo version of Motif graph visualization platform. Feel free to try out the demo.

We created the tool to help analyst and investigators visualize connected data - mainly around blockchain and financial transactions. I think graph visualization solutions should be more accessible to business users and analyst and took Kepler.gl as an inspiration when creating Motif. The idea is to be backend agnostic and let you use your preferred language/tools while speeding up data discovery and collaboration.

Here are some feature highlights:

Neo4j integration
Multiple layout options
Group edges
Time series filtering
Filter and search panels

There's a jupyter integration on the way as well. Would love to hear your feedback about it and if there's sufficient interest we even plan to open source the graph explorer.

A Comparison of Javascript Graph / Network Visualisation Libraries

Timothy — Thu, 18 Mar 2021 02:55:03 +0000

Graph / Network visualisation is a subcategory within the data visualisation space that has grown in popularity over the past few years. There have been emerging use cases to use graphs to understand IoT, social network or transaction data. The growth in the graph database ecosystem has also resulted in a better understanding of the benefits of a graph data representation.

As part of the development of Motif, our graph intelligence software, we evaluated a range of network visualisation solutions to see which tool best fits our use case. In this article, we document our findings on the state of Javascript graph visualisation libraries. This covers both open source and paid libraries. We hope this will be a useful guide for product owners or developers looking to integrate graph visualisation into their application.

While graph visualisation libraries are often used for diagrammatic drawings such as UML diagrams or mind maps, those areas are out of our focus. Here, we are looking to evaluate libraries for data exploration and investigation.

Why Javascript?

There are many other graph visualisation libraries available including popular desktop applications like Gephi or Cytoscape, and libraries coupled with programming languages like visualisation packages within igraph and networkx. These software libraries are good for an individual analyst but are hard to integrate into larger product offerings.

Javascript with its universal browser support is an extremely attractive solution to build an interactive web application. These libraries could easily be integrated into a larger web application or even published as a desktop electron application. While performance might be poorer than native operating systems capabilities, they are more than good enough for most data visualisation needs.

List of Libraries

Here’s the list of 8 libraries we shortlisted for our evaluation:

D3
Keylines / Regraph
Vis.js
Sigma.js / Graphology
Ogma / Linkurious
G6 / Graphin
Ngraph / Vivagraph
React-force-graph

Evaluation Areas

We identified 3 areas where we are interested in comparing across the different products: performance / rendering engine, algorithms available and out of the box components.

Rendering Engine

The factor that has the greatest impact on performance is the rendering engine which the underlying framework uses. Yworks, a diagrammatic library SDK provider, has a nice comparison of the 3 rendering methods. Using a 2015 macbook, SVG performance tops gives workable performance until it reaches 2k nodes and 2k edges. Canvas performance reaches the limit at 5k nodes and 5k edges while WebGL is usable until 10k nodes and 11k edges.

While WebGL is more performant as it leverages the GPU processing power of a machine, it is also harder to work with, which means there tends to be a tradeoff between performance, ease of use and capabilities.

Algorithms

This refers to the set of available APIs to allow a user to perform a graph operation. This could include layout options, pathfinding algorithms or even network detection algorithms. All these additional functionalities would significantly cut down on the development time required.

Components

Out of the box components that would speed up product development. This could include a time bar filter or a minimap to help a user navigate through the canvas.

Summary Evaluation Matrix

Library	Licence	Rendering Engine	Algorithms	Components
D3	BSD	SVG / Canvas	Low	Low
Keylines	Commercial	Canvas / WebGL	High	Medium
Vis.js	MIT	Canvas	Middle	Low
Sigma.js	MIT	Canvas / WebGL	Middle	Low
Ogma	Commercial	Canvas / WebGL	High	Low
G6	MIT	Canvas	High	High
Ngraph	MIT	WebGL	Middle	Low
React-force-graph	MIT	WebGL	Low	Low

Detailed comparisons available at our original post

Growth Hacking Github - How to Get Github Stars

Timothy — Wed, 10 Mar 2021 14:53:44 +0000

You have written the most impressive package out there, published it as an open-source library on Github but wondering why it is not attracting any interest or getting any Github stars?

Turns out publishing a good project is only one part of the puzzle. Getting stars fall into the realm of digital marketing, something developers are perhaps not as well informed or acquainted with. In this post, I share how acquiring Github stars can be understood through the lens of a marketing conversion funnel. I provide examples along the way, including a case study of how I get a zero star Github repository to 50 stars in 10 days.

What are Github stars ⭐?

In Github, users are able to ⭐ other people's repositories (repos), often to indicate that they like the project and also to bookmark it so they can follow its progress. It is one of the most popular metrics used to evaluate the popularity of a project, though there are other measures such as the number of forks or clones. It's the equivalent of a 👍 on Facebook or a 💖 on Twitter.

As seen in this Github project ranking by stars, top starred projects range from popular web development frameworks like Vue and Bootstrap to data science libraries like Tensorflow. Knowledge sharing repositories like freeCodeCamp and other lists of projects (awesome lists) also tend to rank quite highly.

Why you should promote your Github project?

This might seem obvious, but for the developer community promoting your project is probably regarded as the exception rather than the norm. If you are a developer publishing on Github for internal collaboration purposes or if are just after intrinsic satisfaction that is fine. However, I would argue that for most projects where one is looking for user adoption (a startup or portfolio project), the onus is on the developer to publicise their project and get the attention / stars they deserve.

The alternative is for the project to be consigned to a graveyard of zero-star code repos, buried under the mass of new projects and forgotten for eternity.

One way to think of self-promotion is as a means of making good projects discoverable. If you are going to spend a significant amount of time and effort to improve on the status quo, odds are, there is someone out there with the same problem, looking for a solution. If she discovers your solution, it would save her hours or days of time. This benefits both the developer and potential users.

There are also plenty of existing solutions on Github with potentially a much larger user base. Perhaps your solution is really superior to the existing ones? Or solves a pain point in the existing frameworks? Maybe it is beautifully coded and styled? The truth is, no one is going to know the benefits of your project if you are not going to lay it out in front of them.

Thankfully, there are plenty of channels for you to do so!

Github marketing funnel

As defined in Wikipedia, a marketing funnel is a pathway that illustrates the customer journey towards the purchase of a good or service. How does this relate to Github stars? Substitute customer with developer or potential user and purchase decision with star. To get stars, we need to build awareness and guide the user to star the project (or convert). So let's build a Github marketing funnel.

The funnel lays out the user journey from discovery all the way to staring a project.

At the first stage (awareness), we use social media posts and other channels to build awareness on the project.

At the second stage (interest), we aim to build up sufficient interest within these posts and get the user to click through to view the repository.

At the third stage (desire), the contents within the repository should motivate users to want to use the project (view some demo, clone, fork).

If all goes well the user will click the star button.

3 examples

1. Redux Toolkit

The official, opinionated, batteries-included toolset for efficient Redux development redux toolkit star history

Redux is one of the many state management libraries out there and redux toolkit is a relatively new toolkit to significantly simplify the amount of boilerplate required to implement redux in an application with out of the box good defaults settings.

In a crowded space of state management libraries, and with developers using their favourite solutions out there (many probably using default redux), a significant effort is required to nudge users to check out a new library and try it out in their application.

It probably would have been buried in the sea of newer options if not for the efforts of Mark Erikson, one of the chief maintainers of redux toolkit to continuously popularise the library and answer questions related to it across multiple channels.

Fun fact, I probably came across his answers more than 10 times before it finally clicked and I decided to give it a try. Each of his posts is informative and gives a different spin on the benefits of the library. Now, I can't imagine going back to vanilla redux.

2. Huggingface Transformers

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 huggingface transformers stars

The transformer neural network architecture marked one of the most major advancements in NLP capabilities and outperformed recurrent or convolutional methods that were more popular back in 2017. The Huggingface transformer library is now one of the most popular deep learning NLP packages and widely adopted by data scientist today. It is not the first library that introduced the transformer architecture for NLP, but it provided an easy to use API to prototype and deploy such transformer-based models. Notably, Hugging Face is a for-profit startup going against the tech giants who have similar open-source projects.

What contributed to its success? The ease of access to state of the art methods in NLP analysis definitely plays a part. Equally important, was its push in getting users to adopt its solution. By creating a strong open source community around its product, it was able to get organic traction from initial users who preached the benefits of the library, in turn attracting more users.

Winning over the open-source community definitely put it in the sweet spot for investors and helped it to raise $15m in December 2019 to further develop its commercial platform.

3. Tailwind Nextjs Starter Blog

This is a Next.js, Tailwind CSS blogging starter template. Comes out of the box configured with the latest technologies to make technical writing a breeze.

I started this side project, with the intention of updating my then Hugo blog. Having spent some time on it, I thought it would be a nice experiment to test out some Github SEO ideas into practice.

It's a good test case because unlike the other previous two repos mentioned, I have very little web presence (less than 50 stars across all my projects). I also have a similar repo, namely, my current blog which is essentially the same starter template with more personalised content.

The experiment: Let's publicise one of them (the starter blog template) and not mention anything about my existing blog.

The result: After 10 days of promoting the template, I managed to obtain 50 stars, while the repository that this blog is based on did not receive a single star. The template has also risen organically to more than 100 stars at the time of this post (25 Feb 2021), with no additional efforts on my part to further publicise it.

The rest of the post covers some strategies and learnings inspired by digital SEO articles as applied to the 10 days of promotional activity.

Know your potential

Unless your project is a totally groundbreaking idea with no precedent, one can get a good sense of the maximum potential of stars by referring to other more popular projects in the same category.

If you are developing a new web framework, your maximum potential stars would be the number of stars that React or Vue has (about 180k as at Feb 2021). If you are writing a state management library for React, maybe redux would be a good indicator.

For my tailwind nextjs template, I did not want to restrict it to only Nextjs projects using Tailwind CSS but rather look at other blog templates. Other similar templates in my comparison list include Gatsby Starter Lumen or Hugo Coder which tells me that a realistic upper bound for a personal portfolio template project is about 1.5k stars.

Know your audience and channels

The first step to get stars is to know your audience or the type of people who would be interested in your project, likely to "star" your project or share it. Unlike, content marketing where you will need to explore Google's search console or do keyword research on Google, it is a simpler task on Github - just pick the most well-known repositories that are most similar to yours.

From there, you can see the people who are using the repo, raise issues, or submit pull requests. A Google search of the repository is also useful to get an idea of the possible channels to publicise your project, taking the existing more established project as a point of reference.

In my case, I see popular community blogging platforms like Medium and Dev.to as well as aggregator sites like Jamstack Themes.

Optimise your repository

Once you have identified the outreach channels to publicise your content, you should ensure that your repository contains the necessary materials for success. Otherwise, it is a waste of effort to attract visitors without converting them into stargazers.

There are existing posts out there that discusses this in more detail. Check out this dev.to post or this freecodecamp post.

Here's a checklist of Github project essentials and nice to have, inspired by the above posts on the topic and my own personal experience.

Essentials

Readme.md - The readme file is like the landing page of your project. It's the first thing a user will go through and evaluate whether the project fits what she is looking for.
Add a cover image - As mentioned in one of the reference links: "A good percentage of people will star your project just because it looks good, and not all users will scroll down past the top of your READ-ME, so make the top part as pretty as possible".
Add a licence, preferably open-source. I think people tend to be more generous towards open source licences as opposed to other stricter licences.
Add a GitHub description with a demo link to the project- Seeing is understanding and beats reading through a chunk of text.

Nice to Have

Good docs - It shows the maturity of the project and I think every developer appreciates well maintained, comprehensive documentation.
Social sharing buttons - These could be badges for easy one-click deployment or shields to display the number of pulls from Docker. For an enterprise project, this adds a level of credibility to the project and allows a user to quickly give it a test run.
Respond to GitHub issues - For someone evaluating whether to use the project as part of their work, GitHub issues is always one of the first things I check out. It tells me how actively the project is being maintained and how responsive or helpful the development team is in engaging with feedback. For a larger project with already some traction, managing an open source community will make or break the project.
Add a contributor section in the readme - everyone likes a little appreciation, especially on open source projects. Having a contributor section is a nice touch.

Publicise on various channels and social media

Share your repository across the various sources you have identified. It definitely helps if you have existing credibility or standing on these channels but otherwise posting your project as a showcase or as a means for discussion is a good way of approaching it.

For social media, having the right tags also helps to publicise your project especially if you don't have many followers. Tagging the right accounts on Twitter can help amplify the outreach as these accounts tend to reshare posts. Having the right tags also helps disseminate your content to the relevant audience with similar interest.

Post regularly and across different time zones. With how fast things move on popular forums and Twitter, unless your post gets a lot of traction, it will be buried by other new content. Re-posting the content in various forms is a good way to get different people's interest and posting at different time caters to a range of international audience.

The goal with multiple publicity channels is to try and get sufficient interest and attention such that others would try out your project, clone it, talk about it or share it.

Theory into practice

I applied all these methods while trying to publicise my new blog template. I posted on my Twitter account which started with zero followers but managed to get quite a bit of interested users as the post was re-shared by more popular accounts.

Similarly, posting on Reddit in r/react and r/webdev brought in more users to check out the Github repository despite my account being brand new. Tip: You should set up a social sharing image for your repository before sharing it on these platforms.

A nice benefit is that the project was picked by an editor of a popular newsletter which sent a spike of traffic and stars, helping me achieve 50 stars ahead of my 2 weeks targeted plan.

Analyse the source of traffic

Google Analytics is one of the top tools in a digital marketing toolkit. The Github equivalent of this (with much less detail, but still important information) is Github Insights. It gives you a 14-day history of stars and clones over time as well as insights on visitors are coming from.

This is pretty valuable information that could help to refine a content strategy or help to validate the initial hypothesis on channels and interests.

Here's the clone and visitor trend of my tailwind nextjs repository. The 300 visitor spike in traffic was due to the project being featured in a newsletter.

And here's the corresponding referral traffic statistics.

One thing I noticed is that Github tends to recommend people with similar interests, projects which are trending or are highly starred. That is why it is essential to draw attention outside of Github before Github's recommendation algorithm kicks in, and further recommends more viewers to your repository. This creates a snowball effect.

Compared to the previous referral statistics, Github is now my top referring site bringing in 140+ views every day.

Summary

In this post, I discuss how one could adopt digital marketing best practices to promote an open-source project on Github. I outlined some tips in this article including:

Calculating your star potential
Researching on the target audience and channels
Optimising your repository to convert users
Publicising content
Using Github Insights like Google Analytics

I tried these techniques with a brand new project on social accounts with near-zero social media presence and managed to get 50 Github stars in 10 days (more than 100 currently). This validates the ideas I had in publicising open-source projects and jumpstarting users interest in open-source projects.

I hope to apply it in the near future for a most fascinating project I am working on - and you can bet there will a post about it, so stay tuned! Happy to discuss other strategies to grow a brand new project and learn more about what works for you.

Tailwind Nextjs Starter Blog v0.1.2

Timothy — Sun, 14 Feb 2021 14:14:46 +0000

Tailwind Nextjs Starter Blog V0.1.2 now comes with a project page.

Also fixed a few flex bugs on Safari.

Really enjoying developing on nextjs so far, especially with next/image which makes image optimization pretty straightforward.
The only difficulty is parsing the image dimensions. This is solved over here: https://github.com/timlrx/tailwind-nextjs-starter-blog/blob/master/lib/img-to-jsx.js where I used the image-size library to get the file dimensions.

Feature List
Tailwind 2.0
Near perfect lighthouse score
Preact in production build
Mobile-friendly view
Light and dark theme
MDX - write JSX in markdown documents!
Server-side syntax highlighting with rehype-prism
Math display supported via KaTeX
Automatic image optimization via next/image
Flexible data retrieval with next-mdx-remote
Support for tags - each unique tag will be its own page
Projects page
SEO friendly with RSS feed, sitemaps and more!

Here's how the new projects page looks like:

And here's how the current blog posts looks like:

Let me know if you are using the template / planning to use the template and what other features would you like to see!

Introducing Tailwind Nexjs Starter Blog

Timothy — Sun, 07 Feb 2021 15:22:31 +0000

Welcome to my new blog written with Next.js, Tailwind CSS. I have made the blog available as a starter template so feel free to fork and modify to your needs. Comes out of the box configured with the latest technologies to make technical writing a breeze. Easily configurable and customizable. Perfect as a replacement to existing Jekyll and Hugo individual blogs.

Motivation

I wanted to port my existing blog to Nextjs and Tailwind CSS but there was no easy out of the box template to use so I decided to create one.

It is inspired by Lee Robinson's blog, but focuses only on static site generation. Design is adapted from Tailwindlabs blog.

I wanted it to be nearly as feature-rich as popular blogging templates like beautiful-jekyll and Hugo Academic but with the best of React's ecosystem and current web development's best practices.

Features

Easy styling customization with Tailwind 2.0
Near perfect lighthouse score - Lighthouse report
Lightweight, 43kB first load JS, uses Preact in production build
Mobile-friendly view
Light and dark theme
MDX - write JSX in markdown documents!
Server-side syntax highlighting with rehype-prism
Math display supported via KaTeX
Automatic image optimization via next/image
Flexible data retrieval with next-mdx-remote
Support for tags - each unique tag will be its own page
SEO friendly with RSS feed, sitemaps and more!

Sample posts

Quick Start Guide

Fork Tailwind Nextjs Starter Blog
Rename the project to <yourusername>.github.io
Modify siteMetadata.json and blog posts
Deploy on Vercel

Development

First, run the development server:

npm start
# or
npm run dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the page by modifying pages/index.js. The page auto-updates as you edit the file.

Extend / Customize

data/siteMetadata.json - contains most of the site related information which should be modified for a user's need.

data/logo.svg - replace with your own logo.

data/blog - replace with your own blog posts.

public/static - store assets such as images and favicons.

css/tailwind.css - contains the tailwind stylesheet which can be modified to change the overall look and feel of the site.

components/social-icons - to add other icons, simply copy an svg file from Simple Icons and map them in index.js. Other icons uses heroicons.

components/MDXComponents.js - pass your own JSX code or React component by specifying it over here. You can then call them directly in the .mdx or .md file. By default, a custom link and image component is passed.

layouts - main templates used in pages.

pages - pages to route to. Read the Next.js documentation for more information

Post

Frontmatter

Frontmatter follows Hugo's standards.

Currently 7 fields are supported.

title (required)
date (required)
tags (required, can be empty array)
lastmod (optional)
draft (optional)
summary (optional)
images (optional, if none provided defaults to socialBanner in siteMetadata config)

Here's an example of a post's frontmatter:

---
title: 'Introducing Tailwind Nexjs Starter Blog'
date: '2021-01-12'
lastmod: '2021-01-18'
tags: ['next-js', 'tailwind', 'guide']
draft: false
summary: 'Looking for a performant, out of the box template, with all the best in web technology to support your blogging needs? Checkout the Tailwind Nextjs Starter Blog template.'
images: ['/static/images/canada/mountains.jpg', '/static/images/canada/toronto.jpg']
---

Compose

scripts/compose.js can be used to easily generate a post with pre-filled front matter.

The first argument is the name of the post and the second optional argument is the extension (default to .mdx)

Example code to generate the post called "My First Post" in markdown format

node ./scripts/compose.js "My First Post" .md

This will generate ./data/blog/my-first-post.md with pre-filled front matter.

Deploy

Vercel

The easiest way to deploy the template is to use the Vercel Platform from the creators of Next.js. Check out the Next.js deployment documentation for more details.

Netlify / Github Pages / Firebase etc.

As the template uses next/image for image optimization, additional configurations has to be made to deploy on other popular static hosting websites like Netlify or Github Pages. An alternative image optimization provider such as Imgix, Cloudinary or Akamai has to be used. Alternatively, replace the next/image component with a standard <img> tag. See next/image documentation for more details.

DEV Community: Timothy

Building a Responsible AI Solution - Principles into Practice

Introduction

User interviews and feedback

Friction in exchange of information between teams

Difficulty in synchronizing data science code with docs

Difficulty aligning code with regulatory requirements

Tools and solutions in the model governance space

Designing VerifyML

Model documentation

Model testing

VerifyML architecture and solution overview

Conclusion

Easy Graph Visualization With Motif

Features

Import options

Layouts and styling options

Filtering and search options

Getting started with Motif

Importing Les Misérables

Properties panel and tooltips

Configuring styles

Use force-directed layout

Add a node label

Color the nodes based on the ‘group’ field

Saving your masterpiece

Conclusion

Related content

Introducing Motif Graph Visualization Platform

A Comparison of Javascript Graph / Network Visualisation Libraries

Why Javascript?

List of Libraries

Evaluation Areas

Rendering Engine

Algorithms

Components

Summary Evaluation Matrix

Growth Hacking Github - How to Get Github Stars

What are Github stars ⭐?

Why you should promote your Github project?

Github marketing funnel

3 examples

1. Redux Toolkit

2. Huggingface Transformers

3. Tailwind Nextjs Starter Blog

Know your potential

Know your audience and channels

Optimise your repository

Essentials

Nice to Have

Publicise on various channels and social media

Theory into practice

Analyse the source of traffic

Summary

Tailwind Nextjs Starter Blog v0.1.2

Introducing Tailwind Nexjs Starter Blog

Motivation

Features

Sample posts

Quick Start Guide

Development

Extend / Customize

Post

Frontmatter

Compose

Deploy