DEV Community: Jun Guo

Publish your Scala library to Github Packages for free

Jun Guo — Sat, 26 Sep 2020 00:18:06 +0000

As you may know, Github Packages is a place like npm, which you can host public and private packages. Currently, Github packages supports lots of build tools like npm, gem, mvn, gradle, dotnet (Free for public libraries). However, there is not support for sbt, which is not very friendly to our scala developers. Fortunately, thanks to djspiewak for creating sbt-github-packages plugin. This will allow us to publish our scala library into Github packages. Here is how:

🧧 Publish `sbt` library to Github Packages

We will use sbt-github-packages plugin in this tutorial

1. Generate a personal access token with `package:write` permission

Get your personal access token in settings - developer settings - personal access token

Then goto generate token, please remember this token, we will use it in step 3.

2. Add `sbt-github-packages` plugin into your existing sbt project

I assume you have a library that is ready to publish. Please add this line to your codebase ./project/plugins.sbt

addSbtPlugin("com.codecommit" % "sbt-github-packages" % "0.5.2")

3. Set global git `github.token` to allow the plugin to work

git config --global github.user <github_username>
git config --global github.token <personal_token>

Where:

<personal_token> is the token you just created in step 1. After executing these commands, you should be able to view it at ~/.gitconfig

[github]
    token = <personal_token>
    user = <github_username>
...

4. Get ready to publish your package

Add these these lines to your ./build.sbt

githubOwner := "<github_username>"
githubRepository := "<github_repo_name>"
githubTokenSource := TokenSource.GitConfig("github.token")

after adding them you should re-run the build or restart the IDE (if you're using IntelliJ)

For example, my username is gjuoun and this repo name is github-packages-playground. Fill the fields as yours.

Behind the scene, githubTokenSource := TokenSource.GitConfig("github.token") reads the github token then the plugin is authorized to publish to Github Packages

5. Publish to Github Packages

Run this command to publish your package:

> sbt publish

Then you will see this package tab available in your repo:

All naming conventions are from the variables in ./build.sbt.

The package name is <organization>.<name>_<scalaVersion>, the image indicates package version 0.1.2 which is from <version>.

🧨 Use sbt package from Github Packages

We will use sbt-github-packages plugin in this tutorial
You should have followed step 1-5 above and published your package.

1. Add `sbt-github-packages` plugin to enable sbt to consume the package

Add this line to your ./project/plugins.sbt

addSbtPlugin("com.codecommit" % "sbt-github-packages" % "0.5.2")

2. Add library resolver and dependencies to `./build.sbt`

Add these lines to your codebase ./build.sbt make the plugin to work

githubTokenSource := TokenSource.GitConfig("github.token")

resolvers += Resolver.githubPackages("<github_username>", "<github_repo_name>")

libraryDependencies += "<organization>" %% "<package_name>" % "<version>"

Well, there are many fields we need to match them with your package. You can ignore the <github_repo_name> field, doing so the plugin will resolve all available packages in github account <github_username>.

We'd like to take this package as an example:

Where:

github_username = gjuoun
github_repo_name = github-packages-playground
organization = gjuoun
package_name = hellopackage
version = 0.1.6

3. Compile the code

Now you should able to compile your code with the new plugin

 > sbt compile

4. Use the package

Suppose the package has code:

// https://github.com/gjuoun/github-packages-playground/blob/master/src/main/scala/org/gjuoun/lib/calculator.scala
package org.gjuoun.lib

class Calculator{
  def add(x: Int, y:Int): Int ={
    x+y
  }
}

Importing the package into your code is easy, just follow the package name: org.gjuoun.lib

import org.gjuoun.lib.Calculator


// use it with
  var cal = new Calculator()
  Console.println(cal.add(10, 100))

Congratulations, you've published your scala library to Github packages successfully!

Demo code can be found here

Performance aspect of Deno vs. Node

Jun Guo — Thu, 18 Jun 2020 04:59:35 +0000

Talking about performance is cool

A few days ago, I read Craig Morten's post: What Is The Best Deno Web Framework? It gives an overview of all the Deno frameworks and compares all the aspects of them. If you haven't read about it, I highly encourage you to do so.

From Craig's post, the section "Performance" is fascinating; In his environment, Deno HTTP module gets almost the same result(even more) as Node HTTP module, which amazes me, even though it's just a "Hello Deno!" benchmark test.

But, I knew it's not true for most of the cases. Because deno HTTP module is written in Typescript. Type checking brings pleasure for development, but it costs more resources in compiler time thus has a negative effect on performance.
(Thanks for Oghenovo Usiwoma and Jonathan Beaumont point out in the comment)

But I've tested it before and never got the same result as his. So I would like to see what would happen when I run tests in a more powerful machine.

Environment:
- CPU: i5-9600KF @3.7GHz
- RAM: 16GB DDR4 2133MHz
- OS: Windows 10
- Benchmark tool: autocannon
Benchmark Script:
- "Hello World!" deno server
- "Hello World!" node server

Let's see the result first:

100 concurrent connections

Name	version	AVG req/sec
node.http	12.16.3	47969.2
deno.http	1.0.0	47376
deno.http	1.1.0	46953.7
node.http	14.2.0	44409

Wow! Looks like deno beat node overall! (at least for the version 14.2.0).

But wait, this benchmark is based on 100 concurrent connections by autocannon http://localhost:3000/ -c100, which is too many for a small-to-medium-sized server. So I give it another try: this time, I use -c10, which keeps sending requests in 10 concurrent connections.

10 concurrent connections

Name	version	AVG req/sec
node.http	12.16.3	49926.69
node.http	14.2.0	45345.33
deno.http	1.1.0	34806.79
deno.http	1.0.0	34742.37

This time, node is absolutely the winner. There is a significant difference between the two, node has more than about 10K requests per second than deno.

But, this brings a new question:

Why does the concurrent connection number matter?

A concurrent connection is a connection happening with another connection at the same time.
A user sends a request for an HTML page, then after 5ms
another user sends a request for a css file, which is not considered as a concurrent connection.

Calculate concurrent connection

Let's say, you have 100 active users on an app. You track these users for an hour and find out each of them makes 60 clicks an hour. That's 6000 total requests on the server, and each request takes 2 seconds(which is very slow). That's 12000 seconds of active connections in 3600 seconds(an hour), which results 3.33 concurrent connections (12000/3600 = 3.33).

If the server is powerful enough to allow processing each request within 1 second?

The concurrent number is even fewer, less than 1.7.

How about 1000 active users?

We get 33.3 concurrent connections

Are 100 concurrent connections overkill?

Yes, absolutely! Except you want to build a enterprise-level application that serves 3000+ active users(in above scenario).

Can you please tell me which one performs better? Node or Deno.

With that said, the result with 10 concurrent connections is more meaningful in reality, Node wins for the performance round.

Though deno performs better in the -c100 test, which means deno is doing better than node in a higher traffic network. That's great. From what I've seen, companies would prefer Java or other well known and proven languages to handle a huge amount of traffic over deno, since deno is still young and it's not production-ready yet at this point. However, everyone in deno community is making it happening 🔥.

At last, I also want to mention one point. Performance is not the key of success for a dev community. Look at Python and PHP, they preform much slower than node and deno. What makes them popular? The ecosystem and peoples!

Extra topic

Consider of Google handles over 75,000 queries per second.
Can your server handle this amount of request (>40K) per second?