DEV Community: StevenPG

gRPC vs REST - Simple Performance Test

StevenPG — Sun, 12 Sep 2021 20:20:07 +0000

Overview

As a Java Developer, RESTful APIs are my bread and butter. On most days I spend a majority of my time writing code that either make up or interact with RESTful endpoints, or downstream code triggered by an endpoint.

I'd heard of some rumblings of a new RPC format a few years ago, and I immediately thought back to my days doing some light GWT (Google Web Toolkit) development. With that, I brushed the idea out of researching gRPC and continued with my standard workflows.

However, a few months ago I was given the opportunity to jump into a greenfield Golang project at work. I'd heard plenty of good things about Golang, so I dove right in.

I've since fallen in love with the language, though Java is still where I feel most at home.

However, while using Go I've come across gRPC usage more and more.

So as a part of my continued learning, I developed this simple little test to see how the gRPC vs REST performance debate actually plays out within Java alone.

This isn't an incredibly scientific experiment, but I feel like it at least provides somewhat clear results as to what one might expect when someone at your work starts pitching the new hotness (gRPC in this case) and swears that it will solve X or Y problem and have A or B performance benefits.

I've been that guy before, and it always helps to have relevant and concrete results to show everyone to help sway your point.

So with that out of the way, here's my silly little Java-based gRPC vs Rest performance experiment.

Rest vs gRPC tl;dr

Before we get into the experiment and testing, here's a quick overview.

RESTful APIs use HTTP 1.0/1.1 and include usage of HTTP Verbs (ex. GET/POST/DELETE) to transmit messages from client to server. Most servers will expose endpoints that expect specific headers and verbs and utilize the incoming data to perform operations. Data is commonly in JSON format, which Java code deserializes into Java objects for usage within the application.

gRPC on the other hand, utilizes HTTP/2 and protocol buffers, which can be simplified into binary over HTTP. The protocol buffers describe the length of the components in the binary stream. So rather than delineate data with JSON tokens (such as { , and " ), the data streams as binary and is parsed using known sizes of expected data types. This is then deserialized into Java objects.

Ok, there's an oversimplified overview, onto the test details.

Test Data and layout

The intention of this test was to give some samples of gRPC vs Rest performance, specifically. This is one of the more commonly cited reasons to switch to gRPC, as data is smaller as a binary stream than a json object.

This test was implemented using exactly 2 JVM applications. One is a Spring Boot application that contains RestController annotated classes to handle inputs and also contains a client that uses Spring Webflux's WebClient to make blocking API calls to the RestController.

The second application is a native Java application that uses io.grpc libraries to define a gRPC client and server.

These applications were deployed in two locations and run as runnable jars. Both were deployed on a local desktop as well as an AWS EC2 instance. Details for both are included in the next section.

The test is as follows:

Use randomly generated data in a specific format that is identical in the gRPC and Rest testing
Send increasing numbers of requests in order to see how the processing scales
represent semi-real world performance by making calls from a local system to a datacenter at least 200 miles away.

Here is a sample layout of the Rest JSON template:


{
    "car": {
        "make": "toyota",
        "model": "corolla",
        "trim": "4DR Sedan",
        "color": "gray",
        "year": 2012,
        "mileage": 120000
    },
    "driver": {
        "firstname": "John",
        "lastname": "Smith",
        "driverId": "someId"
    }
}

Here is the proto definition for the gRPC messages


syntax = "proto3";
option java_multiple_files = true;

message Message {

    message Car {
        string make = 1;
        string model = 2;
        string trim = 3;
        string color = 4;
        uint32 year = 5;
        uint32 mileage = 6;
    }

    message Driver {
        string firstname = 1;
        string lastname = 2;
        string driverId = 3;
    }

    Car car = 1;
    Driver driver = 2;
}

These are functionally identical, and would be an example of what a transition from an existing Rest API to a replacement gRPC api might look like.

Test machines

The remote machine where the gRPC and Rest servers were running were t2.small EC2 instance machines. These were generic non-spot, non-dedicated t2.small instances that had all TCP and UDP traffic exposed. With a whitelisting for the local test machine that would be hitting it.

JVM Applications were run locally using the Windows 10 x64 OpenJDK, version 11.0.11. Remote JVMs were run using java-11-amazon-cordetto on a AWS Linux 2 OS.

The local test machine was my own personal development desktop. This system contains a Ryzen 3700x and 64GB of 3200ghz Memory. More than overkill for running the tests described above.

Results

Each test was run (from local to ec2) 10 times and the average taken. This were for sequential requests across Rest and gRPC as seen in the columns below.

Type	1000 (requests)	2000 (requests)	3000 (requests)	4000 (requests)	5000 (requests)
gRPC	14478ms	25239ms	37823ms	53073ms	62624ms
Rest	16451ms	30181ms	43996ms	58221ms	71300ms

There's a couple things we want to review as a part of these results. They are subsectioned below!

How much slower is Rest at each request amount

requests	percent slower
1000	13.6% slower
2000	19.5% slower
3000	16% slower
4000	9% slower
5000	13% slower

We can confidently say that gRPC will always be faster than Rest calls. This aligns with the implementation details around gRPC, specifically binary streams over JSON, binary parsing vs marshaling, etc.

What was the correlation between increase in requests and completion time for Rest and gRPC

Good question! So for each increase in requests, we would expect to see a linear increase in processing time to get through those messages. Ie, twice the messages should take twice as long. Let's see if that was actually the case.

Type	0-1000	1000-2000	2000-3000	3000-4000	4000-5000
gRPC	baseline	74% longer	49% longer	40% longer	17% longer
Rest	baseline	83% longer	45% longer	32% longer	22% longer
Request % Increase	0	100% more	50% more	33% more	25% more

Based on the results above, it's a bit hit or miss. Both Rest and gRPC roughly track the increase. Most of the time, the processing time increases at a faster rate than the request increases, which is good! It means we aren't exponentially taking more and more time to process more requests.

What does any of this tell us?

Outside of this specific test? Not a ton. It's a good example to use as a baseline, and a good simple experiment to compare the performance of the two paradigms.

As one would expect, the repository is available on Github with all source code.

The Lemur Pro - 7 months later

StevenPG — Fri, 28 May 2021 02:44:44 +0000

7 months ago I was faced with a problem I'm sure everyone who writes code in their free time has dealt with. Buying a new laptop that will be used for side projects (and maybe the occasional side work).

Do you get a Mac by default? A massively powerful gaming laptop? An expensive but powerful windows machine? A Surface? A cheapie machine? After all, how much power do you really need!

It's a great problem to have.

I wrote a post about it on dev.to titled "The Lemur Pro - Random Developer's Opinion" because I wanted to write out my thoughts after reading so many other partial opinions across the internet.

So now, 7 months later I've used this laptop for at least 4 hours a day and I wanted to summarize my thoughts again, this time highlighting whether or not I regret making this purchase.

I'll be updating the previous article with a link to this one.

Just to set the stage, this was my situation about 7 months ago. I'd been given a 16" Macbook Pro 6 core for work, had a desktop with an 8 core 16 thread CPU (3700x) and 32GB of memory, and my personal machine was an $800 HP Envy x360 laptop.

It was very apparent that the x360 wasn't able to keep up with the workloads I was trying to run on it and I can't use the Macbook for non-work related projects.

I primarily work with Java professionally, so a Linux-like environment was my default. That doesn't leave a lot of options.

I checked out the XPS Ubuntu line, every variation of Macbook and of course, System76. After extensive research, I decided the best bang for my buck would be the $1100ish dollar Lemur Pro from System 76.

I got the machine in the mail, wrote my "Random Developer's Opinion" piece, and moved on with my life.

Since then, I've spent a ton of time using this machine and there are things that I love about it, and things that I hate.

Diving right in, all the points raised in my first article still stand strong. The Lemur Pro is super light, the keyboard is punchy and still feels great to type on.

I get about 11 hours of battery life while doing moderate usage. Namely web browsing or some small-scale development. Maybe a single IDE window and a single browser window with ~10 tabs. 11 hours is absolutely bonkers, considering my macbook nets me about 3 on a good day and my old x360 got me a cool 5 hours at most.

Additionally, PopOS is a phenomenal operating system. All of the comforts of Ubuntu with a fresh coat of paint. With such a small 14" screen, tile snapping is an excellent way to move windows around.

I installed a resource monitoring tool straight away that displays in the top bar and I can say getting System76's top available NVMe drive makes an insane difference. I've yet to wait on the drive for anything, making software load up that much faster as everything gets pulled off disk.

All of the reviews I recall reading talked about the heat. But as I'll show later what kind of workloads I was running, I never once felt like the laptop was extremely hot sitting on my lap. The fans would kick on as needed and they are relatively quiet, and it would just chug through the processing fine.

Time for the "bad".

This. Thing. Holds. Fingerprints. From reading a review, the chassis is made out of a Magnesium-Aluminum alloy. Whatever it is actually made up of, it very easily smudges and cleaning it lasts until I pick it up again.

After about 2 months, I started to have a peculiar issue. If I closed the machine and put it to sleep while I was running anything at all, upon re-opening the laptop the left click on the touchpad wouldn't work.

I got in contact with System76 and we went back and forth for a few weeks debugging. I sent a bunch of debug logs and reinstalled drivers constantly.

Eventually we agreed that it was a "weird" issue and I reinstalled the OS. This worked for about a week until the issue came back 10 times worse. Any time I walked away for a few minutes my left click and touchpad would just stop working.

Thankfully, I think this was a temporary bug, because I updated PopOS to 20.10 and the issue went away and hasn't come back since.

Along the same vein, occasionally when pressing the power button, the machine would just hang on a gray screen until it was force killed.

I removed docker from my start processes and that seemed to resolve that issue.

8GB of memory is definitely not enough. I recommend purchasing a 32GB stick of laptop memory to install along with the soldered 8GB. Having 40GB of memory gives this laptop so much headroom, it's crazy.

I think a lot of this stems from this being a custom firmware and OS, so nothing I'm heavily faulting System76 for.

The included charger is so short, I ended up picking up a 65 Watt 6 foot long charger that uses usb-c to charge the Lemur with.

Here's some examples of workloads I was able to run with no issues on the Lemur Pro:

12 docker containers running in a local kubernetes
3 IntelliJ IDEA windows and 5 browser windows with 10+ tabs each
Android Studio, Android Emulator and 20+ Chrome tabs
Multiple postgres database instances, with 6 small java apps deployed along-side

and more along those lines above.

Overall, I'm really happy with the machine and it's quirks. Much better than if I'd have gotten a Macbook Air or a similarly small windows laptop.

If you're looking to pull the trigger and you need a middle of the road laptop, and you'd prefer it to be ubuntu, I'd say go for it. System76 support was quick and professional, and the Lemur Pro is pretty competitively priced for the specs.

I don't regret this purchase one bit and if I need a more powerful machine or a new machine for work, I might be checking out the Oryx Pro series from System 76!

Hopefully this follow-up helps anyone else who's looking into making the Lemur Pro their next portable machine!

Logging Request Body with Spring WebClient

StevenPG — Thu, 08 Oct 2020 21:53:10 +0000

A Better Way

Welcome!

Everything below this point is a bit outdated, and I highly recommend checking out this post: https://stevenpg.com/posts/request-body-with-spring-webclient/

In this page, I briefly go over a MUCH better option for logging Spring Webclient in 2024 and beyond, something I've been using for 2+ years.

All of my production applications that were originally logged with the method below have been migrated over.

Either way, the methods listed here still work if you prefer them!

Good luck with your logging journey!

About Spring WebClient

Spring is going Reactive. You only need to take one look at the talks featured at the Spring One 2020 conference to see that Reactive Web and functional programming paradigms in Java and Spring were at the forefront.

One of the interesting issues with doing things in a non-blocking manner is that simple things like logging can sometimes become a little bit more complicated. Because you don't know exactly WHEN the data will be available, you can't really just toss it into a log the same way you'd do it with something like Spring's RestTemplate. (Speaking of RestTemplate, it looks like it's targeted for deprecation! https://docs.spring.io/spring-framework/docs/current/javadoc-api/index.html?org/springframework/web/client/RestTemplate.html)

As per the documentation, we ought to be using org.springframework.web.reactive.client.WebClient for our outbound API calls nowadays, especially because it provides the ability for us to utilize blocking and non-blocking methods.

Now anyone who has used Spring WebClient can attest to the fact that retrieving the request content or response content can sometimes be a little bit difficult, especially if you're looking for a specific format.

There's dozens of unanswered Stack Overflow posts that have the same response, check out Baeldung's article on the subject: https://www.baeldung.com/spring-log-webclient-calls.

Now Baeldung has saved my butt more times than I can count, in side projects AND professionally. However, the article doesn't show much more than basic implementation examples. What's missing is the example output, and sharing the caveats that aren't mentioned in the article.

So without further ado, here's a walk-through of the best method (in my opinion) to do request and response logging (with the HTTP body) in Spring Webclient, with examples, comments and output.

Netty logging is included in Baeldung's post but isn't nearly as granular as the Jetty HTTP client. The very first step is adding the required dependency that will give us access to the underlying HTTP client.

The Code

# Gradle    
implementation group: 'org.eclipse.jetty', name: 'jetty-reactive-httpclient', version: '1.1.4'

# Maven
<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-reactive-httpclient</artifactId>
    <version>1.1.4</version>
</dependency>

Once we have access to the classes we need, there's two components that need to be built.

The first is our enhance method. This method takes a Request and gives a request back, allowing us to intercept and log all of the pieces we care about.

Here's an example enhance method and it's output:

// org.eclipse.jetty.client.api.Request
private Request enhance(Request inboundRequest) {
    StringBuilder log = new StringBuilder();
    // Request Logging
    inboundRequest.onRequestBegin(request ->
            log.append("Request: \n")
            .append("URI: ")
            .append(request.getURI())
            .append("\n")
            .append("Method: ")
            .append(request.getMethod()));
    inboundRequest.onRequestHeaders(request -> {
        log.append("\nHeaders:\n");
        for (HttpField header : request.getHeaders()) {
            log.append("\t\t" + header.getName() + " : " + header.getValue() + "\n");
        }
    });
    inboundRequest.onRequestContent((request, content) ->
            log.append("Body: \n\t")
            .append(content.toString()));
    log.append("\n");

    // Response Logging
    inboundRequest.onResponseBegin(response ->
            log.append("Response:\n")
            .append("Status: ")
            .append(response.getStatus())
            .append("\n"));
    inboundRequest.onResponseHeaders(response -> {
       log.append("Headers:\n");
       for (HttpField header : response.getHeaders()) {
           log.append("\t\t" + header.getName() + " : " + header.getValue() + "\n");
       }
    });
    inboundRequest.onResponseContent(((response, content) -> {
        var bufferAsString = StandardCharsets.UTF_8.decode(content).toString();
        log.append("Response Body:\n" + bufferAsString);
    }));

    // Add actual log invocation
    logger.info("HTTP ->\n");
    inboundRequest.onRequestSuccess(request -> logger.info(log.toString()));
    inboundRequest.onResponseSuccess(response -> logger.info(log.toString()));

    // Return original request
    return inboundRequest;
}

The request object provides plenty of hooks to reach in and grab the data you are looking to log. Interface docs are here -> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/org/eclipse/jetty/client/api/Request.html

To get our enhance method executed during the invocation of our WebClient, we're going to create our own HttpClient and use it in place of the default JettyClientHttpConnector. Here's an example bean that provides the WebClient:

@Bean
public WebClient jettyHttpClient() {
    SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
    HttpClient httpClient = new HttpClient(sslContextFactory) {
        @Override
        public Request newRequest(URI uri) {
            Request request = super.newRequest(uri);
            return enhance(request);
        }
    };
    return WebClient.builder().clientConnector(new JettyClientHttpConnector(httpClient)).build();
}

The Output

Now using the WebClient that we've seeded with our underlying HttpClient, we get the following output:

2020-10-08 15:00:00.000  INFO 2100 --- [   @cafebabe-37] 
c.s.l.examples.JettyWebClient            : 
Request: 
URI: http://httpbin.org/get
Method: GET
Headers:
        Accept-Encoding : gzip
        User-Agent : Jetty/9.4.31.v20200723
        Accept : */*
        Host : httpbin.org
Response:
Status: 200
Headers:
        Date : Thu, 08 Oct 2020 20:24:17 GMT
        Content-Type : application/json
        Content-Length : 297
        Connection : keep-alive
        Server : gunicorn/19.9.0
        Access-Control-Allow-Origin : *
        Access-Control-Allow-Credentials : true
Response Body:
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip", 
    "Host": "httpbin.org", 
    "User-Agent": "Jetty/9.4.31.v20200723", 
    "X-Amzn-Trace-Id": "Root=1-5f7f7571-    157328ac70a3bd900bc1c8bc"
  }, 
  "origin": "12.345.678.91", 
  "url": "http://httpbin.org/get"
}

One of the benefits of using this approach over the Netty approach is that no log changes need to be made for this to show up. It also allows you to save off data from each request if needed.

My goal with this post was to demystify the implementation that gives us the most granular control over the request and response data.

The code above is available in a full working application located in the following repository: https://github.com/StevenPG/DemosAndArticleContent/tree/main/blog/logging-with-webclient

The Lemur Pro - Random Developer's Opinion

StevenPG — Thu, 01 Oct 2020 19:42:40 +0000

While searching for a new laptop for personal use, I came across a dozen reviews that each touched on a specific facet of System 76's new Lemur Pro ultra-book. I never found a single comprehensive review that touched on all of the questions I had and had seen on forums like /r/system76 on Reddit. So in keeping with the mantra "be the change you want to see in the world", here's a review of System 76's Lemur Pro where I attempt to touch on every question I've had and come across online!

I made a new post quickly summarizing my experience of using the Lemur Pro for 7 months here!

The Machine Itself

Firstly, Shipping. I ordered the Lemur Pro off of System76's website on September 22nd. Assembly started the following day and it was shipped 5 days later on the 28th. I paid for 3-day priority shipping and received the machine (signature required) on the 30th, within the 3 days!

The configuration that I'll be testing with today was as follows:

Base Lemur Pro
i5 10210U (1.6 up to 4.2Ghz - 6MB Cache - 4c/8t)
8GB DDR4 @ 2666Mhz
250GB NVMe (Seq Read: 3,500 MB/s, Seq Write: 2,300 MB/s)

I purchased a separate Samsung 32GB DDR4 DIMM to save a few bucks that I'll be putting in soon.

First Impressions and Capabilities

Upon receiving it, I was surprised at how light it was. I mean this thing weighs nothing for a laptop. That alone is a huge plus as this is meant to be a grab-n-go laptop or a lap-laptop on the couch to work away on side projects.

When I booted it up for the first time, I loaded into the Pop! OS screen in about 4 seconds. Compared to the ~30 start-up time of my Windows laptop and ~25 second start-up time of my work-issued 16-inch Macbook Pro this is a dream.

Connecting peripherals using Bluetooth and the hardware ports was easy and I had no problems plugging things in or connecting.

Regarding USB-C docks, the Lemur Pro handles them without issue. I plugged the machine into my work-issued USB-C dock and was able to charge the Lemur (though firmware updates must use the included barrel power charger) through USB-C while displaying to a 1080p 144hz monitor without any issues.

Arrow Keys

One of the call-outs I've seen in a few places about the Lemur Pro is the size of the arrow keys and the proximity of the PgUp and PgDn keys. I don't have an issue with these keys, but I wanted to measure the keys compared to my 16-inch MBP to put the issue to bed.

The Lemur Pro's arrow keys are 1.4cm wide with a .4cm gap between the left-center-right keys. The 16-inch Macbook Pro's arrow keys are 1.6cm wide with a .2cm gap between left-center-right keys.

Therefore the truth of the matter is, the Macbook Pro's arrow keys are 13% larger than the Lemur Pros. To me they feel very similar but to others it seems the opposite.

Physical Build

Keyboard: The keyboard reminds a TON of the Macbook pro keyboard, I don't really notice any flex in the tray as I'm typing, even if it's on my lap.

Screen Twisting: With such a small screen, you CAN twist the screen a little bit on the Lemur which is a bit concerning. Whereas on the much larger Macbook, you basically can't twist the screen at all.

Touchpad: The touchpad is definitely better than most, almost as good as the Macbook but the material feels off. I actually prefer it but it's easy to see how it is inferior to the Macbook.

Keyboard: Keyboard is excellent, very short and easy to type quickly on.

Speaker Audio Quality

This is something that's called out and honestly... Every mention of the Lemur's speakers being bad are correct. The speakers are good to voice and pretty much nothing else. There's absolutely no bass, they sound like speakers from cell phones 5 years ago.

That said, I sorta expected this. An on-the-go ultra-book isn't where I'd look for high-quality speakers. Not like the $2400 Macbook Pro which is commonly used as a desktop replacement as well as a laptop.

Performance

Performance of laptops is sometimes vague and I rarely see a concrete example that isn't a synthetic benchmark. So I've chosen a relatively large open source project to compile. The build has some multi-core tendencies as I've seen on my desktop and should give us stable ground to compare the Lemur Pro to other systems.

Here are the machines that the Lemur will be up against (and itself for reference):

Lemur
    i5 10210U - 1.6/4.2ghz - 4c/8t w/ 8GB Mem
2012 Clevo Gaming Laptop - Running Ubuntu
    i7 3610QM - 2.3/3.3ghz - 4c/8t w/ 16GB Mem
2019 Macbook Pro
    i7 9750H - 2.6/4.5ghz 6c/12t w/ 16GB Mem
HP Envy x360
    Ryzen 7 2700U - 2.2/3.8ghz - 4c/8t w/ 8GB Mem
"Gaming" Desktop
    Ryzen 7 3700x - 8c/16t w/ 32GB Mem

All of these were builds were tested using OpenJDK11.0.5 and Maven 3.6.3 using jenkinsci/jenkins repo on Github, branch stable-2.249. All machines were plugged in to their power source rather than run on battery.

All tests exclude the dependency download, since that's subject to networking conditions at that moment. (I under-estimated how long this build took... Doing multiple tests burned an entire Saturday)

Machine	Average Result	Price	Result Diff	Price Diff to Lemur
Lemur	1h 32m	$1049 (2020)	0	0
Clevo	1h 34m	$1521 (2012)	2%	36%
Macbook	1h 38m	$2399 (2020)	6%	78%
HP Envy	2h 10m	$800	34%	26%
Desktop	1h 2m	$1500 (2018)	38%	35%
Raspberry Pi 4 B+	Crashed	¯\(ツ)/¯	;)	185%

The reference point for these is how long the official Jenkins build takes on ci.jenkins.io where the open source maintainers build and deploy the project. It looks like it takes roughly 2 hours for the project to be build and tested by the open source team.

I'm not 100% sure how the Lemur beat out the Macbook, maybe thermals. But the Boost clocks are similar and both were spinning the fans up and down like crazy.

Just like a synthetic benchmark, this really only proves out a large java compilation taking similar amounts of time.

Thermals

Another thing I see mentioned in a lot of these reviews is how hot the battery area gets during heavy usage. So I pulled out my trusty temperature gun and got to work running those Jenkins compiles and grabbing peak temps. Here are the peak temperatures on the bottom of the case of each laptop tested.

I put each laptop on a glass desk (because glass sucks at spreading heat apparently) and measured every few minutes to get the peak temperature. The Macbook's fans probably turned on the least of the machines listed and was certainly the quietest.

Machine	Measured Peak Temp	CPU Peak Temp as Reported
Macbook Pro	112F	82c
Lemur Pro	120F	97c
Clevo	94F	x
Envy x360	92F	x

Fun fact, the Clevo was blowing 140F air out the back. That was a surprise.

Brightness

This laptop is nearly as bright as the MBP outside. For most screens I've used outside, there's "it works in full sunlight" and "it doesn't". The Lemur Pro definitely has an "it works" rating, unlike the HP Envy x360.

Takeaways

The temperature numbers were surprising and support what others have said about the Lemur Pro becoming VERY hot on your lap. However for my personal use-case, I won't be doing multi-hour builds on a 14inch laptop on the couch so this isn't a concern for me. I didn't notice any high-heat when browsing or doing basic software development with an IDE and browser windows.

In my opinion, this laptop does best what it seems like it should be best at, being a long-lasting battery ultra-book. I've seen complaints about power-saver mode having speed issues and all I can say is this laptop isn't meant to be a compilation machine/desktop replacement.

Comparisons to the Macbook should be taken with a huge grain of salt, since it's over twice the cost. Even the Macbook Air is more expensive with worse specs and performance (though better build quality and probably speakers). As with most things, it's what it most important to you.

I wouldn't buy this for heavy multi-processing workloads, I'd buy a desktop replacement laptop. Gaming on this makes little sense as well, though I've seen others complain about the gaming performance.

In my opinion, this fits the same niche as the Macbook Air, except at a way better value and with a much better OS. I love it and happily use it when I'm away from my desktop. My itch for a Macbook of my own is gone and System76 has earned my trust.

Developer Distributions in the US - Observations

StevenPG — Sun, 21 Jun 2020 13:23:17 +0000

Like many developers, there are plenty of places around the web that I use to keep up to date with current trends in Software. Sometimes I spend some of that time helping new developers and recent college graduates with what one would expect; resumes, job advice, etc.

In some of these forums, there is a distinct West Coast (San Francisco, Silicon Valley, etc) bias, with students and developers in the northeast or midwest concerned with things like salary numbers and total comp when comparing to these west coast pockets.

I set out to see just how many developers are in California and are they in the majority?

State	Developers	% of Total
CALIFORNIA	628414	15.02%
TEXAS	324717	7.76%
NEW YORK	218041	5.21%
VIRGINIA	204699	4.89%
ILLINOIS	186426	4.45%
FLORIDA	181314	4.33%
NEW JERSEY	162977	3.89%
PENNSYLVANIA	152900	3.65%
MARYLAND	147430	3.52%
WASHINGTON	143971	3.44%
MASSACHUSETTS	141755	3.39%
OHIO	136615	3.26%
GEORGIA	129145	3.09%
NORTH CAROLINA	125937	3.01%
MICHIGAN	115359	2.76%
COLORADO	107423	2.57%
MINNESOTA	94211	2.25%
ARIZONA	75685	1.81%
MISSOURI	73757	1.76%
WISCONSIN	72878	1.74%
INDIANA	56862	1.36%
TENNESSEE	54123	1.29%
CONNECTICUT	53543	1.28%
OREGON	52800	1.26%
UTAH	43842	1.05%
ALABAMA	42665	1.02%
SOUTH CAROLINA	42587	1.02%
KANSAS	38170	0.91%
IOWA	36552	0.87%
KENTUCKY	32262	0.77%
OKLAHOMA	27026	0.65%
DISTRICT OF COLUMBIA	26749	0.64%
NEW HAMPSHIRE	26359	0.63%
LOUISIANA	24641	0.59%
ARKANSAS	22878	0.55%
NEVADA	21314	0.51%
NEBRASKA	20755	0.50%
NEW MEXICO	18210	0.44%
IDAHO	16135	0.39%
DELAWARE	15759	0.38%
WEST VIRGINIA	12037	0.29%
MISSISSIPPI	11845	0.28%
RHODE ISLAND	11370	0.27%
HAWAII	10961	0.26%
MAINE	10758	0.26%
VERMONT	6896	0.16%
SOUTH DAKOTA	6709	0.16%
NORTH DAKOTA	6306	0.15%
MONTANA	5848	0.14%
ALASKA	3257	0.08%
WYOMING	2241	0.05%

While California is the largest state for developers, collections of smaller states make up a similar percentage of total developers. We can group some of these together into region-based groups to see a clearer picture.

General Area	Total Percentage of Developers	States Included
Northeast	23.29%	Maine, New York, New Jersey, Vermont, Massachusetts, Rhode Island, Connecticut, New Hampshire, and Pennsylvania, Delaware, Maryland, DC
West Coast	24.76%	Alaska California Colorado Hawaii Idaho Montana Nevada Oregon Utah Washington Wyoming
Midwest	20.18%	Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin
Southeast	21.13%	Alabama, Florida, Georgia, Kentucky, Mississippi, North Carolina, South Carolina, Tennessee, Maryland, Virginia, and West Virginia.
Southwest	10.65%	Arizona New Mexico Texas Oklahoma

The takeaway here is while to some it may feel like there is a majority of developers in California, regionally the distribution is fairly even.

Sources (https://i.pinimg.com/564x/33/e1/97/33e197b4295db9024fbd4d380023f609.jpg, https://dqydj.com/number-of-developers-in-america-and-per-state, https://www.daxx.com/blog/development-trends/number-software-developers-world)

Powershell and Ryzen - a build server

StevenPG — Sat, 30 May 2020 15:36:09 +0000

This being my first article I’ve ever written, I expect it will be awkward, a little dense and potentially hard to read.

But I thought this was an interesting experiment, so I’m putting the information out into the web!

I’m amidst building a companion application for the community edition of SonarQube, a static code analysis tool. After a few light weekends, I finally completed the project and deployed it into my local environment. Feeling successful, I turned my attention to a new project around Kubernetes!

However, as soon as I tried to deploy my application in a large enterprise, with 1500+ projects configured within the SonarQube tool, my application started to break down. I realized I needed to recreate this environment on my home server to be able to development effectively for enterprise size.

The fun thing about SonarQube, is that the simplest way to push a project into it for static code analysis, is by running a build of some code. In my case, this is building using Maven. So I wrote a script that will perform Maven builds in parallel on my development desktop, just to populate the SonarQube instance. This brings us to the purpose of this article.

I figured it would be interesting to see how builds can scale in parallel (at least when triggered within Powershell) on a Windows machine, so I began recording the data. It’s worth noting that these builds do not perform a clean, and part of the build is copying the source directory to a new location.

Here is the script that performs the builds:

param(
[Parameter(Mandatory=$true)][string]$ParentFolder,
[Parameter(Mandatory=$true)][string]$OutputFolder
)
Clear-Host
$ScriptBlock = { param($ParentFolder,$OutputFolder)
    $Guid = [Guid]::newGuid()
    Write-Output "Writing SonarQube project-$Guid from generated folder $OutputFolder/$Guid"
    Copy-Item -Path "$ParentFolder" -Destination "$OutputFolder/$Guid" -Recurse
    & mvn package -q -DskipTests versions:set -D:"newVersion=$Guid" -f "$OutputFolder/$Guid/pom.xml" sonar:sonar -D"sonar.host.url=http://192.168.2.155:19100" -D"sonar.projectKey=project-$Guid"
}
for($i = 0; $i -lt <someNumberOfExecutions>; $i++)
{
    Start-Job $ScriptBlock -ArgumentList $ParentFolder,$OutputFolder
}
Measure-Command {
    While (Get-Job -State "Running") {Start-Sleep 2}
}
Get-Job | Receive-Job
Remove-Job *

The specs of the machines are below:
+------+----------------------------------------+
| Part | Desktop |
+------+----------------------------------------+
| CPU | Ryzen 7 3700X (Resting CPU util = ~5%) |
| Mem | 16GB DDR4 |
| Disk | Samsung NVMe |
+------+----------------------------------------+
+------+----------------------------------------+
| Part | "Server" |
+------+----------------------------------------+
| CPU | i7 3610QM (Resting CPU util = ~1%) |
| Mem | 16GB DDR3 |
| Disk | 7200 RPM |
+------+----------------------------------------+

During any builds (above 2 in parallel), the machine reported itself at 100% CPU utilization. (Around 50 parallel builds is when my machine locked up and things started crashing)
java.nio.file.FileSystemException: C:\Users\Usr.sonar: Insufficient system resources exist to complete the requested service
AND
WARNING: A PSScheduledJob job source adapter threw an exception with the following message: Insufficient system
resources exist to complete the requested service.
And here is the data!

These aren’t the most interesting results, but I think this shows exactly how Powershell is scaling it’s tasks under the hood. While the machine is getting bogged down, the amount of time that builds take to complete is surprisingly linear.

Once builds start to cross over the 35/40ish in parallel threshold, individual builds begin failing. As I mentioned above, when pushing 50/60 builds in parallel, windows begins to struggle to function properly. Not unlike during a stress test!

The takeaway from this experiment is if you’re planning on using a Ryzen 7 3700x in a build server, pay attention to how you’re going to execute the builds. Triggering naive powershell tasks is going to create an invisible boundary that, when crossed, will cause builds to mysteriously start failing without warning.

Edit Notes:
I transitioned this article from my Medium account, and it's pretty old. I'll be the first to admit it's not the most scientific test.