DEV Community: Skip Everling

A New Era for TypeDB

Skip Everling — Mon, 10 Jul 2023 22:49:53 +0000

When we started building TypeDB, we saw a continuously growing complexity of data in modern applications. Modern programming languages have evolved by introducing powerful abstractions and constructs to express more complex logic – JavaScript is succeeded by TypeScript, Java by Kotlin, C/C++ by Rust, and Haskell emerging as the leader of functional languages. Modern languages all have a stronger and more expressive type system.

However, when we look at the evolution of database technologies since relational algebra and SQL, even though there has been incredible progress in performance and operational capabilities in the past 40 years, databases have yet to provide more powerful abstractions to express more advanced logic. In fact, graph, document, key-value, wide-column, and every other NoSQL database provide lower-level and more primitive data structures for us to express our models and queries. This is a huge problem for future applications, as they will need to work with more complex datasets and interrogate their data more intelligently.

We built TypeDB & TypeQL to provide much more powerful abstractions in the database language, allowing you to tackle the next order of complexity in future applications. For the first time, TypeDB brings to the database the power of a strong type system , the simplicity of a composable language, and the intelligence of symbolic reasoning to infer data.

This year, we’re launching typedb.com to help you understand our paradigm and unique features, and to deliver a brand new and enhanced documentation portal. We are also incredibly excited to invite you to the very first webinar series on TypeDB Fundamentals, happening in just 3 weeks!

We see the problem of data complexity in many modern applications, especially those serving intelligent operations.

In Cybersecurity, Cyber Threat Intelligence and Identity Access Management solutions struggle to keep up with rapid evolution of data and the amount of hidden patterns. In Platform Engineering, Internal Developer Platforms and Continuous Delivery systems struggle with the unknown consequences of change and fragmentation of data. In Knowledge Engineering, Drug Discovery and Risk Management solutions find that their biggest hurdle is connecting independent datasets in a way that makes it easy to derive hidden and implicit connections. Virtual Representations, such as those found in Robotics, Digital Twins, and Gaming, struggle with modeling real-world details in a virtual environment.

This year, we’ll be sharing with you the research we’ve done on applications of TypeDB in these advanced domains, starting with our first webinar series on our new TypeDB Use Cases, happening 4 weeks from now!

TypeDB Cloud Alpha is here!

After a whole year of engineering, we are finally ready to invite you to try out TypeDB Cloud! You can now automatically deploy TypeDB in the US and Europe, on Google Cloud and Amazon AWS. More regions across the globe will be coming soon, as well as the option for Microsoft Azure. At this alpha stage of TypeDB Cloud, we’d love to hear your feedback on how TypeDB Cloud meets your expectations, and you’ll have the opportunity to contribute to our development roadmap!

Join the TypeDB Cloud waitlist!

Finally, TypeDB 2.18 & 2.19 are released with major updates!

We’ve had 2 major releases in the past month: 2.18 and 2.19. We introduced new functionalities to perform arithmetic operations in TypeQL. We’ve significantly improved import/export of schema & data together. We replaced our storage engine from RocksDB to SpeeDB and boosting performance. And finally, we now natively support ARM processors for both Mac and Linux, which means you can run TypeDB on Apple silicon chips as well as Raspberry Pis!

There are a lot more major updates coming in the next few months: a new and highly-performant TypeDB Rust driver – and every other driver wrapping the Rust driver binary, new TypeQL capabilities to fetch all attributes of entities and relations as part of answers, optional query patterns in TypeQL, native JSON response format, and many more! For now, we hope you enjoy the new TypeDB 2.19!

Deploy the new TypeDB

How to Write an OpenAPI Description

Skip Everling — Thu, 16 Jul 2020 02:54:16 +0000

There’s a lot you can do once you have an API description file formatted according to the OpenAPI Specification (or its predecessor spec, Swagger). Using a standardized API description format as a single source of truth allows for better API design & development, as well as the automatic generation and deployment of API reference documentation.

Since API descriptions are plain text format, you can use any text editor to create one. It can be helpful to begin with a template so you don’t have to start from scratch (below there's one you can copy!). Alternatively, if the API you want to document is already in production, we'll explore some other options to generate an API description file.

If you need a review of what OpenAPI and Swagger are, check out ReadMe: How to Use OpenAPI and Swagger for Documentation

OpenAPI Template

To get started with an OpenAPI file, it can be helpful to start with a basic outline that includes the essential syntax. Then you can step through each line and make edits to include the proper details from your API.

You don’t have to start from scratch! Here is a template OpenAPI definition:

openapi: "3.0.0"
info:
  version: 1.0.0 // Version of your API
  title: Your API Title
  description: An API description template
  contact:
    name: Your Name
    email: email@example.com
    url: http://example.com
  license:
    name: License Name
    url: http://example.com/license-url
servers:
  - url: http://api.example.com
paths:
  /your_endpoint:
    get:
      description: |
        Multi-line description
        of your API endpoint
      operationId: yourOperation
      parameters:
        - name: your_param
          in: query
          description: Description of your param
          required: false
          schema:
            type: string
      responses:
        '200':
          description: Your success description

This template doesn’t include complete coverage of all possible OpenAPI fields, but it’s useful as starter code. In most cases you’ll want to add your own response schemas and reusable components. You can dig into the OAS specification itself or see our OpenAPI and Swagger examples below.

Once you have an initial framework for your API described, you can complete coverage for the remainder of your API.

OpenAPI Descriptions are Written in JSON or YAML

When you write your OpenAPI or Swagger file, you can choose from either of two formats: JSON or YAML. Both will use the same data structure (determined by the Swagger or OpenAPI specification), but each will have a different syntax representation. JSON may look familiar to most web developers and it is the most common format used to return actual API results. YAML may also look familiar, as it’s often used in configuration files.

YAML uses whitespace and minimal markup, which can make it more human-readable compared to JSON.

Below is an example OpenAPI 3 YAML description, showing the header and one path/response.

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            format: int32
      responses:
        '200':
          description: A paged array of pets
          headers:
            x-next:
              description: A link to the next page of responses
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"

OAS Examples, Petstore YAML

**JSON **is the more common format for data exchange, but is less human-readable. Here’s the same OpenAPI 3 description fomatted using JSON:

{
   "openapi": "3.0.0",
   "info": {
      "version": "1.0.0",
      "title": "Swagger Petstore",
      "license": {
         "name": "MIT"
      }
   },
   "servers": [
      {
         "url": "http://petstore.swagger.io/v1"
      }
   ],
   "paths": {
      "/pets": {
         "get": {
            "summary": "List all pets",
            "operationId": "listPets",
            "tags": [
               "pets"
            ],
            "parameters": [
               {
                  "name": "limit",
                  "in": "query",
                  "description": "How many items to return at one time (max 100)",
                  "required": false,
                  "schema": {
                     "type": "integer",
                     "format": "int32"
                  }
               }
            ],
            "responses": {
               "200": {
                  "description": "A paged array of pets",
                  "headers": {
                     "x-next": {
                        "description": "A link to the next page of responses",
                        "schema": {
                           "type": "string"
                        }
                     }
                  },
                  "content": {
                     "application/json": {
                        "schema": {
                           "$ref": "#/components/schemas/Pets"
                        }
                     }
                  }
               }
            }
         }
      }
   }
}

OAS Examples, Petstore JSON

Modern programming languages and their respective web frameworks can readily consume JSON objects, and this is a major reason for why many API providers adopt the JSON format. Most web developers are familiar with JSON syntax thanks to its resemblance to Javascript.

Neither YAML or JSON are immune to human error, but editors with linters will catch the most common errors.

You can convert files between JSON and YAML, so feel free to choose whichever format is preferable for you and your team.

How to Write OpenAPI Descriptions With Your Favorite Code Editor

Your current development environment or text editor will include YAML and JSON syntax highlighting and may already include Swagger and OpenAPI syntax support as well. Look for plugins, which can help with syntax suggestions or checking for errors as you write your API description. Alternatively, you can use existing open source or web-based tools.

For example, the VSCode editor has an open source linter plugin to check YAML and JSON files against Swagger and OpenAPI specifications.

Shown above is an example of an in-editor linter program which will raise errors and flag conventions for cleaner code. You can click each error to go to the line where the issue originated. In the screenshot, we accidentally misspelled description, resulting in the linter raising missing required property errors on line 48 with the typo.

With most editors, you can edit either OpenAPI or Swagger files in YAML, with syntax help and built-in linting. When finished, you can programmatically convert YAML to the equivalent JSON.

How to Generate OpenAPI Descriptions From an Existing API

If you already have an API in production, you can benefit from documenting it in an OpenAPI file. To save yourself some time, look for some ways you can generate the descriptions from your code or live traffic.

If your API is built in a common framework, such as Falcon (Python) or Rails (Ruby), your code already has everything needed to create a Swagger or OpenAPI description. Look for an existing project that creates an API definition based on an existing API. For example, zero-rails_openapi gem is a Rails solution and falcon-apispec does the same for Falcon. You can also create API descriptions from code comments.

There could be many reasons it's not possible to reference source code. In that case, you might use a service like Optic to listen to live API traffic. After reviewing live requests and responses, Optic can output an OpenAPI or Swagger file.

Finally, another option is to make small updates to comments in your code. Using the oas Node module, you can generate your definition from inline YAML. This is a ReadMe open source tool where it is used internally to document APIs from code.

Hope this helps you to create and make use of an OpenAPI description for your API!

This is an excerpt from my extended guide on ReadMe: How to Use OpenAPI and Swagger for Documentation

Jamstack Attack! Static Sites, Dynamic APIs, Killer DX

Skip Everling — Mon, 18 May 2020 20:44:23 +0000

What is the Jamstack and why is it on the rise?

What is the Jamstack?

The Jamstack (originally stylized JAMstack) is an approach to building web applications that extends the ideas of “static sites” to emphasize their true dynamic potential. Originally introduced by Matthew Biilmann in 2016, then popularized by Netlify and other companies in 2017 onward, the Jamstack approach continues to grow in popularity thanks to its design for rapid scalability, performance, security, and relatively easy developer experience.

The JAM in Jamstack is an acronym that stands for:

JavaScript
APIs
Markup Image credit: Colby Fayock, What is the JAMstack and How Do I Get Started? (freecodecamp.org)

JavaScript in the client browser handles dynamic interaction using a set of reusable APIs to abstract away server-side (backend) operations. An app often combines the use of many API services, each tailored toward a specific operation set (more on this later!). Markup is the technology used to serve prebuilt websites as static files to end users without delay.

Key facts and features of the Jamstack:

JavaScript web applications
APIs for dynamic services (no reliance on specific servers)
Pre-rendered pages (Markup, including HTML/CSS)
Deployed via CDN

Key advantages:

Highly Scalable
Highly Performant
Highly Secure
Easy to implement

Don’t Call Them Static Sites

“You may have already seen or worked on a Jamstack site! They do not have to include all attributes of JavaScript, APIs, and Markup. They might be built using sites built by hand, or with Jekyll, Hugo, Nuxt, Next, Gatsby, or another static site generator. The thing that they all have in common is that they don’t depend on a web server.” —jamstack.org

Jamstack apps don’t rely on web servers. So how are they able to behave as if they do?

APIs are the secret sauce that allow a Jamstack application to go beyond being a “static site.” Web developers can use relatively simple HTTP calls to endpoints maintained and scaled by other providers, thereby abstracting away all of their server-side concerns. Developers don’t need to worry about the meticulous work that goes into building a reliable and performant backend, and can instead focus on their front end and user experience. They can make a static site remarkably dynamic with extended functionality. Developers are able to take full advantage of best-in-class API solutions offered by the industry.

API services used in Jamstack applications often include:

E-Commerce APIs (e.g. Snipcart)
Content Serving APIs (e.g. Contentful)
Search functionality via third-party services (e.g. Algolia)
Custom serverless functions
Custom data stores and data queries

Developer Experience: Will Your API Jam?

Developers building their site with Jamstack naturally take on a microservices approach to building out their server-side functionality and are eager to make use of existing APIs that fulfill their requirements. In the early prototyping and exploration phases, the search for suitable APIs includes sampling third-party API services.

To get developers to adopt and enjoy your API, you can’t just tell them about features, you need to show them you care about their experience with your documentation. In the case of a bad or missing documentation, the crowd will inevitably seek alternatives that offer a better experience.

If you’re looking for some helpful guidance, Andy Trattner sat down with me in April to write up his 4 Steps to Implementing Stellar Documentation for the ReadMe blog, while API Engineer Jon Ursenbach told us all about How to Make Your API More Enjoyable to Use. Documentation owners will find these articles help them to feel more confident they’ve properly considered their developer experience.

More About the Jamstack

Hungry for more Jamstack knowledge? Here are some sweet resources to help you get started:

awesome-jamstack
- An extensive list of awesome Jamstack resources and APIs, curated on GitHub.
jamstack.org
- Netlify’s Jamstack resource site.
jamstack.training
- Tamas Piros’ free course catalog (itself a Jamstack app!).

Jam On!

The Jamstack development phenomenon shows little sign of slowing down and now you know why: fast, scalable static architecture made dynamic thanks to APIs. With an easy developer experience free of server-side concerns, we’ll continue to see its popularity rise as the age of cloud computing continues. Jam on!

(This article was originally published for ReadMe)