DEV Community: Filipe A. Barroso

Deploy a Dart Shelf server to Google Run

Filipe A. Barroso — Sat, 13 Nov 2021 00:00:00 +0000

Let us learn how to deploy your Dart project to Google Run. I will assume you already have a GCP project running, have gcloud installed, Docker installed, and are familiar with their CLI commands. You will also need a Billing account associated to your GCP Project, and the following GCP services activated in your project:

Container Registry API
Cloud Run Admin API

Cloud Run while has a Free-tier, it is not a Free service, check out the pricing page for more detailed information: https://cloud.google.com/run#section-14

Cloud Run

"Cloud Run: Container to production in seconds."

Cloud Run is a recent Google Cloud Platform product, built on top of KNative published in 2018, which just recently released version 1.0. After 1 year of the release of KNative, Google announced the Cloud Run in which with one container, you can deploy a containerized project, production-ready. It doesn't necessarily to be a Docker container (link).

Note of advice, Docker has a rate limit on downloading images in the free tier, so do consider other repositories for your images to avoid disruptions.

Cloud Run, There are some criteria that your project needs to be under to be a valid project to run in Cloud Run, do check the following website to check if you can run your project: https://cloud.google.com/run/docs/fit-for-run

Dart Shelf

Let me tell you an obvious secret, this section of this blog post will be the only one I need to mention Dart because to deploy to Cloud Run, it doesn't matter how you coded it but what it matters is if it is containerized. This blog post could be using any other programming language to create the server.

I will assume you have a Dart Shelf project ready to deploy, I've shared my notes about Dart Shelf in my previous blog post where I share how to create a simple API, through examples taken from the official resources with some added notes.

To create a container for your Dart project we can use the official Dart Image and the Dockerfile is simple and easy to understand.

FROM google/dart # The official Dart Image

WORKDIR /app # Sets a default folder to keep the project organized

# Do check the note.
ONBUILD ADD pubspec.* /app/
ONBUILD RUN pub get
ONBUILD ADD . /app/
ONBUILD RUN pub get --offline

CMD [] # Makes that is not possible to override parameters while running this image
ENTRYPOINT ["/usr/bin/dart", "bin/server.dart"] # Forces to run the project main server dart file.

Note: Check the reason why I used ONBUILD and why we have duplicated actions here.

Now that we have a Dockerfile and a Dart Shelf project, we can now continue with the deployment to Cloud Run.

How to deploy to Cloud Run

As mentioned, Cloud Run relies on containers to run the project, you can build the project from the source code, deploy through a container image, or even directly from a git repository.

I am going through the process of using a deployed Image to Google's Container Registry. That is done in two steps.

Upload the Container Image to the Container Registry;
Deploy the image to Cloud Run;

1. Uploading a Docker image

Now that we have a container image ready to upload, first, we will need to upload it to the Google Registry. For that, we will first A. tag and then B. submit.

A. Tagging a Docker image to upload to the Google Registry

The first thing you have to do is to tag your image with the prefix gcr.io/<PROJECT_ID>/<IMAGE_NAME>. gcr.io is the registry hosted in the USA for other registers check it here.

Henceforth, our <PROJECT_ID> will be dart_project

To tag an image to be able to upload to the Registry you simple use the following Docker command (more about Docker tags here)

docker tag dart_server:latest gcr.io/dart_project/dart_server

Note how I am tagging our image with the suffix :latest, more about it in the section "Update the project".

B. Deploying the image {#b-dot-deploying-the-image}

With the tag ready you can now upload your image to your Cloud Registry of the project

gcloud builds submit --tag gcr.io/dart_project/dart_server

2. Deploy Image as Google run

We are now ready to deploy our dart server as a Google Run, we only need to define a SERVICE_NAME and deploy it as a Google Run

SERVICE_NAME=dart-server

Note: The name of the services can only include alphanumeric and dashes, hence the name with dash and not the underscore as used so far.

gcloud run deploy $SERVICE_NAME \
  --image=gcr.io/dart_project/dart_server \
  --allow-unauthenticated

Success! Your Cloud Run is now deployed and you can check the URL that the previous command outputs to try your deployed and new server!

Update the project

After the deployment to Cloud Run, it makes sense that you want to update the server with the new code, for that, all you have to do is build your docker image and re-submit your image using the gcloud builds submit since the tag we created is attached to the :latest build of the image.

To make our life easier we could use the following bash script to update our Cloud Run.

SERVICE_NAME=dart-server

gcloud builds submit --tag gcr.io/dart_project/dart_server

gcloud run deploy $SERVICE_NAME \
  --image=gcr.io/dart_project/dart_server \
  --allow-unauthenticated

Conclusion

Cloud Run is incredible to go from code to production, be advised that in this blog post I haven't written about security, managing persistent data, or having a full-blown project running with Cloud Run. Stay tuned for future blogposts to know more.

Meanwhile, if you would like to learn more, there are some Codelabs you can learn from: https://codelabs.developers.google.com/?cat=cloudrun.

Check the Cloud Run official webpage https://cloud.google.com/run.

Shelf — Web Server with Dart

Filipe A. Barroso — Fri, 05 Nov 2021 00:00:00 +0000

This article is mostly a gathering of examples from Shelf and its official add-ons. Where I add my own comments that helped me understand the project and building a web server with Shelf, more about that in the end of the blogpost.

Why Shelf

Shelf, is a Web Server Middleware, not a full blown server framework like Django, it is modular, and you add add-ons for the functionalities you need. This modular structure gives the community an easy way to expand the project.

While other Dart for backend solutions exist. At the time of writing, two of the most promising Dart server frameworks, Aqueduct and Angel, have stopped their development. Relying on the community to grab where they left and continuing the work done. While from Angel came Angel3 and from Aqueduct came Conduit, I prefer to use a more stable solution, stable as maintained by a team with more resources. So, never forget to support the open-source community projects!

Shelf, is maintained by the Dart team itself but still lacks functionalities, again because is not a server framework. If we want Shelf to compete with other server frameworks, such as Django (Python) it is necessary quite a bit of work. However, at the current status, we can create a web server that suits our needs.

These are the official add-ons maintained by the Dart team:

Shelf (core)
Shelf Router
Shelf Router Generator
Shelf Static
Shelf WebSocket
Shelf CORS Headers

Starting a backend Project

I am assuming you have Dart installed on your machine, if not you can follow the instructions at https://dart.dev/get-dart.

To run a server you need to create a new Dart project, I'll be using the terminal and the following command

dart create shelf_core_example

This will create a folder with all the code inside shelf_core_example if you move into the new folder created you will notice the following file structure

 CHANGELOG.md
 analysis_options.yaml
 pubspec.lock
 pubspec.yaml
 README.md
 bin/shelf_core_example.dart  # Same name as the project name created

The file shelf_core_example.dart has a simple main file.

void main(List<String> arguments) {
  print('Hello world!');
}

You can run the existing file, with a simple command:

dart run bin/shelf_core_example.dart

You will be prompted with the following message

$ dart run bin/shelf_core_example.dart
Hello World!

Adding Shelf

package: https://pub.dev/packages/shelf

Now that you have the project created, to use Shelf we simply add a Dart package dependency.

In the project root you can add it by running the following:

dart pub add shelf

Now you are ready to work on your dart server! This command will also be used to add the other Shelf add-ons we will later add.

Shelf Example

import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as shelf_io; // For an easier evocation of its methods

void main() async {
  var handler =
      const Pipeline().addHandler(_echoRequest);

  // Starts the server, in 'localhost' and with port 8080.
  var server = await shelf_io.serve(handler, 'localhost', 8080);

  // From the source documentation
  /**
   * Whether the [HttpServer] should compress the content, if possible.
   *
   * The content can only be compressed when the response is using
   * chunked Transfer-Encoding and the incoming request has `gzip`
   * as an accepted encoding in the Accept-Encoding header.
   *
   * The default value is `false` (compression disabled).
   * To enable, set `autoCompress` to `true`.
   */
  server.autoCompress = true;

  print('Serving at http://${server.address.host}:${server.port}');
}

// Method Handler that replies with the path called.
// The return =Response.ok= is our helper static function to return a 200 code result, but we have many other, such as, Response.found() for 302 redirects or Response.notFound() for 404 not founds and all other codes that you can expect.
Response _echoRequest(Request request) =>
    Response.ok('Request for "${request.url}"');

Permalink to autoCompress source code: autoCompress
Permalink to implementation of other Response static methods

Adding Shelf Add-ons

The core library for Shelf, gives you the main functionalities of a server. The following example shows the barebones of a simplest server you can start running with Shelf.

Shelf Router

package: https://pub.dev/packages/shelf_router

While Shelf (core) lacks the functionality of setting Routes, we use the add-on shelf_router which will give you the router instance, with all the REST methods, such as GET and POST and all other calls that make up REST.

dart pub add shelf_router

Shelf Router Example

server.dart

import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as shelf_io;
import 'service.dart';

/// Similar with the previous example but here we create the routing in our new class 'Service' and we call its handler.
void main() async {
  final service = Service();

  final server = await shelf_io.serve(service.handler, 'localhost', 8080);

  print('Serving at http://${server.address.host}:${server.port}');
}

service.dart

import 'package:shelf/shelf.dart';
import 'package:shelf_router/shelf_router.dart';
import 'api.dart';

/// Class used to set up all the routing for your server
class Service {

  // The handler/middleware that will be used by the server, all the routing for the server will be implemented here.
  Handler get handler {
    final router = Router();

    // GET
    // Replies with the text inserted in the path, example: http://localhost:8080/say-hi/Filipe will prompt "hi Filipe".
    router.get('/say-hi/<name>', (Request request, String name) {
      return Response.ok('hi $name');
    });

    // GET
    // You can use regex to limit the capture of the path arguments, in this case the userId param needs to be only digits. Note how you can also used 'async'
    // With the server running try to open http://localhost:8080/user/132 and http://localhost:8080/user/a123
    router.get('/user/<userId|[0-9]+>', (Request request) async {
      await Future.delayed(Duration(milliseconds: 100));
      return Response.ok('_o/');
    });

    // You can also embed other routers, in this case it will help organize your routers
    // Note: This needs be before the catch 'router.all' that follows
    router.mount('/api/', Api().router);

    // A catch all of the non implemented routing, useful for showing 404 page now found
    // With the server running try to open http://localhost:8080/abc
    router.all('/<ignored|.*>', (Request request) {
      return Response.notFound('Page not found');
    });

    return router;
  }
}

api.dart

import 'package:shelf/shelf.dart';
import 'package:shelf_router/shelf_router.dart';
import 'service.dart'

/// Decoupled endpoints for a easier organisation of all endpoints
class Api {
  Future<Response> _messages(Request request) async {
    return Response.ok('[]');
  }

  Router get router {
    final router = Router();

    router.get('/messages', _messages);
    router.get('/messages/', _messages);
    router.post('/messages/', _messages);
    router.update('/messages/', _messages);
    router.delete('/messages/', _messages);

    router.all('/<ignored|.*>', (Request request) => Response.notFound('null'));
    return router;
  }
}

With the previous code setup you can now run the server in the root of the project simply by executing

dart run bin/server.dart

Shelf Router Generator

package: https://pub.dev/packages/shelf_router_generator

To reduce boilerplate, the add-on shelf_router_generator helps us with the code generation. The add-on gives us annotations that replaces the need to call the 'router' instance.

Shelf router generator leverages build_runner to generate code, check the bullet Dependency for more information.

dart pub add shelf_router_generator

Dependency

package: https://pub.dev/packages/build_runner

To learn on how to use build_runner check check the package URL.

Add the build_runner has dev_dependencies

dart pub add builder_runner

Shelf Router Generator Example

server.dart

import 'package:shelf/shelf.dart';
import 'package:shelf_router/shelf_router.dart';
import 'package:shelf/shelf_io.dart' as shelf_io;
import 'service.dart';

part 'server.g.dart';

void main() async {
  final service = Service();
  final server = await shelf_io.serve(service.handler, 'localhost', 8080);

  print('Serving at http://${server.address.host}:${server.port}');
}

service.dart

class Service {
  @Route.get('/say-hi/<name>')
  Response _hi(Request request, String name) => Response.ok('hi $name');

  @Route.all('/<ignored|.*>')
  Response _notFound(Request request) => Response.notFound('Page not found');

  Handler get handler => _$ServiceRouter(this);
}

You need run once the build_runner and only then the dart run.

Shelf Static

If you need server side processing, to show an html page, you will need to add this add-on so that so you can present the users to

package: https://pub.dev/packages/shelf_static

dart pub add shelf_static

Shelf Static Example

import 'package:shelf_static/shelf_static.dart';
import 'package:shelf/shelf_io.dart' as shelf_io;

void main() async {
  final handler = createStaticHandler('files', defaultDocument: 'index.html');

  final server = await shelf_io.serve(handler, 'localhost', 8080);

  print('Serving at http://${server.address.host}:${server.port}');
}

Shelf WebSocket

package: https://pub.dev/packages/shelf_web_socket/

With Shelf, you don't necessary need to work with HTTP, you also have the possibility to have a socket open through shelf_web_socket might be an add-on that not everyone uses, but it is interesting to know that it has.

dart pub add self_web_socket

Shelf WebSocket Example

import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as shelf_io; // For an easier evocation of its methods
import 'package:shelf_web_socket/shelf_web_socket.dart';

void main() {
  var handler = webSocketHandler((webSocket) {
    webSocket.stream.listen((message) {
      webSocket.sink.add('Echo: $message');
    });
  });

  shelf_io.serve(handler, 'localhost', 8080).then((server) {
    print('Serving at ws://${server.address.host}:${server.port}');
  });
}

To test this server with Web Socket I used the "Simple WebSocket Client" plugin for Chromium
https://chrome.google.com/webstore/detail/simple-websocket-client/pfdhoblngboilpfeibdedpjgfnlcodoo?hl=en

Shelf CORS

package: https://pub.dev/packages/shelf_cors_headers

Last but not least, we have this add-on to help us change the headers to have CORS working locally. Exposes a method called corsHeaders() that returns a Middleware used in the Shelf Pipeline.

Do not confuse this add-on with shelf_cors that is not available for Dart 2.0

dart pub add shelf_cors_headers

Shelf CORS Headers Example

import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as shelf_io;
import 'package:shelf_cors_headers/shelf_cors_headers.dart';

void main() async {
  final overrideHeaders = {
    ACCESS_CONTROL_ALLOW_HEADERS: 'example.com',
    'Content-Type': 'application/json;charset=utf-8',
  };

  final handler = const Pipeline()
    .addMiddleware(
      corsHeaders( // Middleware from 'shelf_cors_headers'
          headers: overrideHeaders,
      ),
    )
    .addHandler(_echoRequest);

  final server = await shelf_io.serve(handler, 'localhost', 8080);

  print('Serving at http://${server.address.host}:${server.port}');
}

Response _echoRequest(Request request) {
  return Response.ok('Request for "${request.url}"');
}

Conclusion

While we wait for other Dart server frameworks to mature, we can always rely on Shelf to implement our project. There is still a lot needed to implement and few add-ons available that extends Shelf (https://pub.dev/packages?q=shelf)

Next article on the topic of Dart Backend will be about integrating Stripe into a server to pay recurrent or by metered API consumption. If I see myself using other Shelf add-on I'll write a new blogpost

Originally published at https://filipebarroso.com/articles/dart_shelf_backend/

Blogging Stack with Org Mode

Filipe A. Barroso — Sun, 03 Oct 2021 15:55:51 +0000

TL;DR

Org Mode - To write the articles, from anywhere
- Ox-hugo to parse the articles into Hugo
Hugo - For the website itself
Firebase - Hosting

Explaining the Stack

I'm lazy, the more hurdles I have to publish a blog post, the harder it is for me to even start writing. To publish something I need to be simple, fast, and easy to write from anywhere, laptop, mobile, or even from handwritten notes.

I use Org mode to write all my notes, professionally or personally. If you are familiar with Markdown, Org Mode is just another Markup Language but on steroids.

Since I've been using Emacs, more precisely Doom Emacs, for a while now together with Org Mode to organize my work and life. To use Org Mode to write articles was the most natural transition, even more, after finding ox-hugo an Emacs package that parses Org Mode files to Markdown, that can be used in Hugo.

Why Org Mode

"Org Mode, your life in plain text"

Org Mode is a superset of Markdown. First and more important, Org Mode is just like any other text file, it can be opened and edited anywhere, just like Markdown. Org Mode is also like Markdown as in it is human-readable but more importantly, can be extended as you need it, check all default features here, you can use it for clocking, tasks management, tables, and much more. Org Mode wins over Markdown since it is extensible as you need it to be, an example is that you can also use it for knowledge keeping, with Org-Roam, just to mention one example.

You can't do all this Markdown so easily but I understand that is unfair since I am using Emacs. However, if you install Emacs solely to use Org Mode, you will be well served, no matter your background or for what you use Org Mode for. Check "Org Mode in the wild" for some examples of uses.

Videos on how to use Org-Mode

Why Hugo

The main reason to use GoHugo is that you create articles in Markdown, in which I can parse them from Org Mode through the use of Ox-Hugo package, there are also plenty of resources, tutorials, and themes for Hugo that makes it easy to build any static website for your need.

Why Firebase

I'll be honest, I picked Firebase given the reason at the beginning of this blog post, it is because I am lazy.
Deploying a static website to Firebase is one of the easiest ways to deploy, and since I am really familiar with Firebase, it was my first choice. Nevertheless, Firebase is the one that is more easily replaceable from this Stack, with the Hugo website being static, you can deploy it to Github Pages, Vercel, Netlify or any other hosting service you prefer.

Steps to publish a new blog post

In an overview the steps I need to publish a new article are the following:

Write article in Org Mode
Parse the article by running a key binding, in Emacs is called key chord, C-c C-e H H
Build Hugo website, for it I only have to run the command $ hugo or $ hugo serve that keeps building the website locally with every new changes
Deploy to Firebase by running $ firebase deploy

In the near future. I am going to add a Makefile so I can run the 3rd and 4th steps in one line. After, I might add a simple CI/CD to simplify even further. But the current setup is so simple that, for now, I don't feel the need.

Conclusion

While the Stack I use for this blog is unorthodox it is the one that fits me the best, and in my honest opinion, the fastest starting from writing to publishing also, and no less important is that everything can be versioned through git and saved in a remote repository.

Any part of the Stack can be replaceable, you can ignore all about Org Mode and write your articles directly in Hugo through Markdown, and you can go all crazy and use Pandoc to convert a plethora of file formats to Markdown if you are crazy enough.

About the hosting, since Hugo builds a static website, you can deploy the website anywhere you prefer.

Hit me up at Twitter let me know if you have any questions or if you prefer to feel free to schedule a 1 on 1 on cal.com.

This blog post was posted originally at filipebarroso.com

Photo by Darwin Vegher on Unsplash