DEV Community: Brian Olore

Peeking at Slack with CDP

Brian Olore — Sun, 11 Apr 2021 12:33:36 +0000

CDP, or Chrome DevTools Protocol, is pretty neat, check this out -

Warning: OSX paths, your OS ~~may~~ will vary

From the Terminal, start Slack with

/Applications/Slack.app/Contents/MacOS/Slack --remote-debugging-port=21212

From the Terminal, start Chrome with

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome http://localhost:21212 --remote-debugging-port 21212

Then click the link that shows the open channel and open DevTools, and switch to the Network tab.

Next, click on the network request that opens the websocket as shown here:

Click on any of the Messages and see the contents!

Note: some are more interesting than others.

This is the first time I opened Slack from the command line, it is also interesting to read the logs as you take various actions within Slack. I’m definitely curious about whatever ublockworkaround.history is!

OpenAPI/Swagger – Tweaking your JSON output

Brian Olore — Mon, 03 Aug 2020 13:37:11 +0000

Cover Image by cwright1 from Pixabay

This week I got my first chance to use the OpenAPI spec, you might have heard it called Swagger. It’s a common way to define APIs. It’s great because it can autogenerate both client and server code, and even documentation on how to use the API that you create.

There is plenty to read about OpenAPI/Swagger, but I want to focus on one particular part that I struggled a bit with and talk through how I reconciled it.

Our application, like most these days, is expected to consume and produce JSON. This is no problem for OpenAPI/Swagger because it allows you to define Media Types as part of the definition. So you can easily return JSON, XML, or anything else.

I needed to return a list of Groups. For this writing, I will simplify the Group such that it only has a “name” and a “description”. So, if you look at the example on the Swagger site you can see how to define an array. It looks like this:

type: array
items:
  type: object
  properties:
    id:
      type: integer

So, for our list of Groups, we’ll tweak this to:

type: array
items:
  type: object
  properties:
    name:
      type: string
    description:
      type: string

This would produce JSON output that looks like the following :

[{
  "name": "MyName",
  "description": "Test description"
}, {
  "name": "Other Group",
  "description": "Test 123"
}]

Not bad!

But then I realized, I don’t want my JSON output to be a top-level Array. I always prefer to return Objects. This makes it easier to modify down the line if I need to add additional fields, like additional Objects/Arrays, etc. An example would be some metadata about the response, such as how long the request took, or how many total Groups there are.

I had trouble finding any rules that say “Don’t return an Array”. The closest thing i could find was this REST site that shows Objects being returned. Since this is what I’ve grown accustomed to, and because this is how Mirage JS returns its data, I figured I was on a good path.

If you have any links or documentation that talks about Best Practices around JSON Arrays, please share in the comments below!

So, what I really want my data to look like is:

{
  "groups": [{
    "name": "MyName",
    "description": "Test description"
  }, {
    "name": "Other Group",
    "description": "Test 123"
  }]
}

To make this happen, we need to tweak our YAML a bit. We’ll introduce an Object that wraps the Array:

type: object
properties:
  groups:
    type: array
    items:
      type: object
      properties:
        name:
          type: string
        description:
          type: string

And there you have it! The JSON that is returned is an Object with a “groups” Array ! Once I change my client code to use response.groups instead of response , everything works as expected.

I hope this has been helpful. I’ve written my own JSON responses in various languages, not really thinking too much about it. I now see the benefit of OpenAPI/Swagger, as it provides a robust, language agnostic, way to describe and define your APIs. This is especially important in a large enterprise. I expect to be doing a lot more with it in the future. Thanks for stopping by!

The post OpenAPI/Swagger – Tweaking your JSON output appeared first on The Brian Olore Story.

Writing Code

Brian Olore — Mon, 01 Jun 2020 01:55:04 +0000

Tap, tap tap. I clack away on my keyboard.
Red. Green. Refactor. Sometimes it's that simple.
But more often, it's not that easy.

Giving up is not an option. I'm so close. I must push through.
Try this, try that, nothing is working.

Add some logging. Looks like what I expected.
Breakpoints? Oh that would be nice, but I never set that up.
Now's not the time, let's put in some more debug messages.

More of the same, this isn't helping.
Get up, walk away, drink some water.
Wait! I think I know what it is, let's take another look.

Yup that's it. How did I miss it for the last hour?
All because I forgot to return from a map() function.

How do you say ~!@#$%

Brian Olore — Mon, 27 Apr 2020 21:01:45 +0000

I had a random thought the other day regarding the characters on the keyboard that us developers use every day. I realized that we probably have very different names and expectations of what each of them are used for, or even call them.

So I did a quick survey of some non-developer friends and family and here's what I came up with:

Character	Me	Other	Others I've heard
~	Tilde	Tildee, Approximately	Squiggle
!	Exclamation	Make a Point	Not, Bang
@	At		Kill
#	Pound	Number, Hashtag
$	Dollar	Money, "String" in Excel
%	Percent		Mod, Modulus
^	Carrot	Accent
&	Ampersand	And
*	Asterisk	Star, "Times" in Excel	Splat
(	Open Paren	Left Paren, Quotation Mark
)	Close Paren	Right Paren, Other end of Question Mark
_	Underscore	Underline	Lodash
+	Plus

I've combined references to "point", "mark" and "sign". They seem to be used interchangeably.

Which did I miss? What do you call these characters, and what do you expect them to do? I'll update the table as you comment

Introducing jaycue – jq in your JS

Brian Olore — Wed, 11 Mar 2020 16:09:42 +0000

Shout out to my son for making the logo! Nice work Joseph!

Previously, I wrote about how useful lodash is when working with JSON in JavaScript. Then we took a look at how to deal with JSON from the command line using the power of jq.

It should come as no surprise where I’m headed here:
I want the power of jq filters in my JavaScript code!

Off we go

Instead of doing what I normally do: googling to see if it existed, then getting depressed that it wasn’t unique, I decided to just do it. I thought it would be a fun way to do a little test-driven development, which I kinda miss doing.

I figured that if I could find a way to write a test and run it against my code, grab the output, run it against jq proper, then compare the results, it’d prove that my code was doing what it should.

I’ve executed shell commands from Node.js before, so it seemed doable.

> const exec = require('child_process').execSync;
undefined
> let command = `echo '{"foo": 123}' | jq ".foo"`
undefined
> exec(command).toString();
'123\n'

Cool. That was easy enough. So, in an attempt to make it more generic for a utility method that I can use in the tests, I should be able to do something like:

> let command = `echo '${json}' | jq "${filter}"`

and pass any JSON string and any filter, then collect up the result, and compare!

My goal is to make a function that would allow me to perform an expectation like:

expect(myjq(jsonObject, filter))
    .toEqual(jq(jsonObject, filter));

Check it out here: https://github.com/olore/jaycue/blob/master/tests/test-helper.js#L22

Now that we can test, let’s code!

Once that was in place, it was a matter of choosing which types of filters I wanted to support first. From the documentation, there are A LOT. I decided to go after the ones I would use most often, and expand from there. In particular, there was a single use case I wanted to solve. But to get there, I had to start with the basics.

The Basic filters could be serviced by lodash.get. For instance, both versionString and .versionString will work with

lodash.get("versionString");

I would just need to chop off the leading period. Something like “name.firstName” would also work with lodash.get.

From there, I started down the list of Basic Filters. Adding in Array indexing and eventually Select filtering, which was the last piece of the puzzle for the use case I had in mind.

Here it is:

{
  "applicants": [
    {
      "identities": [
        {
          "type": null,
          "number": null
        },
        {
          "type": "SSN",
          "number": "987651234"
        }
      ]
    }
  ]
}

We need to get the number whose type is “SSN”, if it exists.

We had code that looked like this:

const ssn = get(data, 'submit.applicants[0].identities', [])
  .reduce((accum, identity) => {
    if (identity.type === 'SSN' && identity.number) {
      accum = identity.number;
    }
    return accum;
  }, '');

Whereas, a jq command like this would work:

cat test.json | jq '.applicants[0].identities[] | select(.type=="SSN") .number'

Now that we had select functionality, the above JavaScript code could be replaced with:

const ssn = jq(data, '.applicants[0].identities[] | select(.type=="SSN") .number');

And just like that, we’re successful!

Wrapping up

Please go install jaycue and let me know what you think:

npm install jaycue

Checkout some more great JSON tools:
node-jq
JSONView (Chrome)
JSONovich (Firefox)

A big shout-out to my 13yo son for making the jaycue logo! It was a true, family effort. Nice work Joseph!

I hope you have found this useful. I’d love to hear about what features of jq you think should be added next. And as always, I’d love to have you contribute to the jaycue project!

The post Introducing jaycue – jq in your JS appeared first on The Brian Olore Story.

jq: The JSON CLI tool

Brian Olore — Fri, 07 Feb 2020 22:44:09 +0000

In Part 1 of Working with JSON, we looked at how lodash makes manipulating JSON from JavaScript easy. Sometimes we’re working with chunks of JSON and it’s easier to deal with it from the shell. Scenarios like this include digging through log output and that (sometimes painful) time when you are first interacting with a new API.

Working with JSON in the shell is quicker and easier than trying to create a temporary file in your project or IDE, then writing some JavaScript code to open that file, parse its contents and scan for whatever it is you are looking for. Those of you that are skilled with the Node.js shell may disagree, but I’m determined to show you something you may not know and hopefully optimize your workflow too!

Hello jq

jq is a CLI (command line interface) for working with JSON. It provides shell friendly features (piping, output redirection, etc) that make it feel as if it’s part of your operating system.

To get started, install jq (on OSX)

brew install jq

Or install it directly from the site

Running the jq command by itself will provide some usage info, but not a ton. You’re going to want to head to the online manual, or the man page (man jq).

Let’s see what `jq` can do!

jq is normally invoked by “piping” JSON data to it (using the | operator). This can be as simple as an echo command. Here we’ll use the simplest filter ., “which copies jq’s input to its output”.

echo '{"name": "Brian"}' | jq '.'
{
  "name": "Brian"
}

By default, jq will pretty print and colorize its output. Neat!

Because it works with pipes, you can also use commands like curl to get an API response, then pass it through jq like so:

curl -s https://swapi.co/api/people/1/ | jq '.'
{
  "name": "Luke Skywalker",
  "height": "172",
  "mass": "77",
  "hair_color": "blond",
  "skin_color": "fair",
  "eye_color": "blue",
  "birth_year": "19BBY",
  "gender": "male",
  "homeworld": "https://swapi.co/api/planets/1/",
  "films": [
    "https://swapi.co/api/films/2/",
    "https://swapi.co/api/films/6/",
    "https://swapi.co/api/films/3/",
    "https://swapi.co/api/films/1/",
    "https://swapi.co/api/films/7/"
  ],
  "species": [
    "https://swapi.co/api/species/1/"
  ],
  "vehicles": [
    "https://swapi.co/api/vehicles/14/",
    "https://swapi.co/api/vehicles/30/"
  ],
  "starships": [
    "https://swapi.co/api/starships/12/",
    "https://swapi.co/api/starships/22/"
  ],
  "created": "2014-12-09T13:50:51.644000Z",
  "edited": "2014-12-20T21:17:56.891000Z",
  "url": "https://swapi.co/api/people/1/"
}

As you can see this can be very useful when you need a quick sanity check of an API response structure, or if you forget if the key name is eyeColor or eye_color.

jq does more than pretty formatting! It let’s you slice & dice the JSON in many ways. For instance, let’s ask jq to return Luke’s eye_color

curl -s https://swapi.co/api/people/1/ | jq '.eye_color'
"blue"

Cool!

Thankfully, similar to lodash.get, if we ask for a path that doesn’t exist, it doesn’t die a horrible death

curl -s https://swapi.co/api/people/ | jq '.does.not.exist'
null

Let’s get more advanced

What if we wanted to get the eye_color of all the people (on the first page of results)? Again, jq provides a syntax similar to lodash.get:

curl -s https://swapi.co/api/people/ | jq '.results[].eye_color'
"blue"
"yellow"
"red"
"yellow"
"brown"
"blue"
"blue"
"red"
"brown"
"blue-gray"

OK, let’s see if we can make it work a little harder. Let’s get the name of all the people (on the first page of results) that have and eye_color of blue?

curl -s https://swapi.co/api/people/ | jq '.results[] | select(.eye_color == "blue") .name'
"Luke Skywalker"
"Owen Lars"
"Beru Whitesun lars"

Sweet! This is where the power of jq really shines. There are many more powerful filters, operators and functions I suggest you explore. Hopefully I was able to whet your appetite as this is just the tip of the iceberg of things you can do with jq.

One more

One last quick tip – this is the one I use for making sense of my JSON log messages all the time. If you are in OSX, you have the ability to access the clipboard from the shell using pbcopy and pbpaste. So when I am working with JSON logs, from an application like Splunk, I copy the relevant JSON structure to the clipboard, then run it through jq like this:

pbpaste | jq '.'

The pretty printing alone is super useful. On top of that, you have all the great filtering commands to coerce an intimidating log message into a more approachable piece of data!

Special thanks to https://swapi.co/ for providing a fun API to play with!

The post jq: The JSON CLI tool appeared first on The Brian Olore Story.

Working with JSON in JavaScript

Brian Olore — Wed, 29 Jan 2020 14:14:54 +0000

As software developers, we spend a lot of time parsing JSON, like, A LOT. There are many tools and libraries available to make our development lives easier. This is the first of a 3 part series where I’ll dig a bit into tools and libraries that I use every day.

Enter lodash

lodash is a phenomenal JavaScript library that provides incredibly useful functions, most of which should exist in the standard library. So what can it do for JSON? Let’s take a look at the get, set and has functions.

First up, lodash.get is the standard for picking through JSON data, or, in fact, any JavaScript object. It allows for easy and safe traversal of nested objects and arrays. get is “safe”, meaning you won’t get the dreaded Cannot read property 'whatever' of undefined that can occur when some nested properties are missing. In this case, get will simply return undefined.

Let’s look at some examples:

const object = {
  "blog": {
    "name": "The Brian Olore Story",
    "articles": [
      {
        "title": "First Post",
        "published": true,
        "tags": ["new", "blog", "post"]
      },
      {
        "title": "Why you should TDD",
        "published": false,
        "tags": ["tdd", "dev"]
      }
    ]
  }
}

// simple path traversal
_.get(object, 'blog.name');
// => 'The Brian Olore Story'

// traverse into arrays
_.get(object, 'blog.articles[1].tags');
// => ["tdd", "dev"]

// path doesn't exist
_.get(object, 'blog.name.rating');
// => undefined

// provide a 3rd parameter to be returned if any part of the path is undefined
_.get(object, 'blog.rating', 100); 
// => 100

Next, let’s take a look at lodash.set. set works the same way as get, but will modify the object passed in. The 3rd parameter is a value to insert/update into the given object. Similar to get, it’s “safe”, meaning: if a portion of the path doesn’t exist, it will be created. Remember: set mutates the object passed in.

// change the blog name
_.set(object, 'blog.name', 'My Story');

// set the rating (that didn't previously exist)
_.set(object, 'blog.rating', 88);

// create a whole new path
_.set(object, 'blog.followers.count', 21);

Finally, while slightly less used and often overlooked, lodash.has returns a boolean if the path provided exists.

// has a name?
_.has(object, 'blog.name');
// => true

// has a rating? (that doesn't exist)
_.has(object, 'blog.rating');
// => false

// has a deeply nested value (that doesn't exist)
_.has(object, 'blog.followers[0].name');
// => false

Another great thing about lodash is that you can choose to bring in all lodash functions, or just individual functions.

const _ = require('lodash'); // all lodash functions (npm install lodash)

const _get = require('lodash.get'); // just the get function (npm install lodash.get)

What tools do you use to traverse the gobs of JSON data we deal with every day?

The post Working with JSON in JavaScript appeared first on The Brian Olore Story.