DEV Community: Zaptic

Moving from elm-validate to elm-verify

Michael Jones — Fri, 16 Feb 2018 14:50:06 +0000

At Zaptic, we use Elm for our business administration website to allow customers to configure the processes that they want to carry out. The site contains a number of different forms for adding & editing various entities in our system. All of these forms benefit from client side validation before the data is submitted to our servers.

When we started writing our Elm code, the recommended approach to form validation was the rtfeldman/elm-validate library. For a form backed by a data structure like this:

type alias ProductGroup =
    { name : String
    , mode : Maybe Mode
    , selectedProducts : Dict Int Product

    -- Used for storing validation errors
    , errors : List ( Field, String )
    }

We might have a validation function like this:

validateProductGroup : Validator ( Field, String ) ProductGroup
validateProductGroup =
    Validate.all
        [ .name >> ifBlank ( NameField, "Please provide a name" )
        , .mode >> ifNothing ( ModeField, "Please select a mode" )
        , .selectedProducts >> ifEmptyDict ( ProductsField, "Please select at least one product" )
        ]

This checks that the name field isn't blank, that a mode has been selected and that some products have been selected. The result is a list of errors corresponding to the criteria that aren't met:

validateProductGroup { name = "", mode = Just MultiMode, selectedProducts = Dict.empty }
-- [(NameField, "Please provide a name"), (ProductsField, "Please select at least one product")]

The list can be saved in the model and then the view searches the list for relevant entries when displaying the various input fields in order to display the error state to the user if there is one.

With this in mind, we can create our form to send a SaveProductGroup message when the form is submitted and we can handle this message in our update function with:

        SaveProductGroup ->
            case validateProductGroup model.productGroupData of
                [] ->
                    { model | productGroupData = { productGroupData | errors = Nothing } }
                        ! [ postProductGroup model.productGroupData
                          ]

                results ->
                    { model | productGroupData = { productGroupData | errors = Just results } } ! []

Here, we run the validateProductGroup function and if we get an empty list then we know that there are no errors and we can post our data to the server using postProductGroup that will create a command for us to carry out the necessary HTTP request.

The problem we encounter is that the postProductGroup needs to encode the ProductGroup structure to JSON and when it does that it has to navigate the data that we've given it. We know that the mode value is a Just because we've validated it but the convert to JSON function cannot take any shortcuts because Elm doesn't let us (which is a good thing!) We can try to write the encoding function like:

encodeProductGroup : ProductGroup -> Json.Encode.Value
encodeProductGroup { name, mode, selectedProducts } =
    Json.Encode.object
        [ ( "name", Json.Encode.string name )
        , ( "mode"
          , case mode of
                Just value ->
                    encodeMode value

                Nothing ->
                    -- Help! What do we put here?
          )
        , ( "products", Json.Encode.list <| List.map encodeProduct <| Dict.values selectedProducts )
        ]

As you can see from the comment, it is hard to figure out what to write there. Elm gives us some escape hatches like Debug.crash but that feels wrong. We could not include the mode key if we don't have a value but then we're knowingly allowing a code path that sends incorrect data to the server.

So what do we do? Well, we need to recognise that this function shouldn't be making this decision. It shouldn't have to deal with the Maybe, especially when we have already validated that it has a Just value.

So we create a new type:

type alias ProductGroupUploadData =
    { name : String
    , mode : Mode
    , selectedProducts : List Product
    }

Which is the same data that we're interested in but with the Maybe resolved to an actual value and the Dict.values transformation already applied to selectedProducts. If we change our encodeProductGroup function to expect this type then the implementation becomes trivial:

encodeProductGroup : ProductGroupUploadData -> Json.Encode.Value
encodeProductGroup { name, mode, selectedProducts } =
    Json.Encode.object
        [ ( "name", Json.Encode.string name )
        , ( "mode", encodeMode mode )
        , ( "products", Json.Encode.list <| List.map encodeProduct selectedProducts )
        ]

But how do we convert our ProductGroup to ProductGroupUploadData? This is where the stoeffel/elm-verify library comes in. It allows us to both validate our data and transform it to another structure in the same operation. It does this by using the Result type to allow it to report validation errors, if any are encountered, or the new data structure for us to use. And it does this with a Json.Decode.Pipeline-like interface:

validateProductGroup : ProductGroup -> Result (List ( Field, String )) ProductGroupUploadData
validateProductGroup =
    let
        validateProducts productGroup =
            if Dict.isEmpty productGroup.selectedProducts then
                Err [ ( ProductsField, "Please select at least one product" ) ]
            else
                Ok (Dict.values productGroup.selectedProducts)
    in
    V.validate ProductGroupUploadData
        |> V.verify .name (String.Verify.notBlank ( NameField, "Please provide a name" ))
        |> V.verify .mode (Maybe.Verify.isJust ( ConfigField, "Please select a mode" ))
        |> V.custom validateProducts

Where V is the result of import Verify as V. You can see the "pipeline" approach that might be familiar from Json.Decode.Pipeline. That means we're using ProductGroupUploadData as a constructor and each step of the pipeline is providing an argument to complete the data. We use String.Verify to check that the name isn't blank and Maybe.Verify to check that the mode is specified. Then we use Verify.custom to provide a slight more complex check for the selectedProducts. Verify.custom allows us to write a function that takes the incoming data and returns a Result with either an Err with an array of errors or an Ok with the valid value. We use it to not only check that the dictionary is empty but also extract just the values from the dictionary. We don't have to run Dict.values here, we could also do that in the encodeProductGroup function when generating the JSON but I have a personal preference for the UploadData to match the JSON closely if possible.

With that in place, we can change our SaveProductGroup case in our update function to look like this:

        SaveProductGroup ->
            case validateProductGroup model.productGroupData of
                Ok uploadData ->
                    { model | productGroupData = { productGroupData | errors = Nothing } }
                        ! [ postProductGroup uploadData
                          ]

                Err errors ->
                    { model | productGroupData = { productGroupData | errors = Just errors } } ! []

Which means that the postProductGroup is given a nice ProductGroupUploadData record and no longer has to worry about the Maybe type.

Prior to using the elm-verify library, we used a separate function to convert between the types and we only called postProductGroup if both the validation & the conversion functions were successful. The conversion function always felt a little strange though and switching to elm-verify cleans that up nicely.

Further note: Whilst the elm-verify interface is similar to Json.Decode.Pipeline it isn't quite the same. It has a version of andThen but it doesn't provide some ways to combine operations like Json.Decode.oneOf or helper functions like Json.Decode.map. This is partly to keep the interface simple and partly because it is always manipulating Result values so with a bit of thought you can use helper functions like Result.map quite easily.

I only include this as originally sought to use elm-verify exactly as I would Json.Decode.Pipeline and ended up writing a bunch of unnecessary functions in order to allow me to do just that. I should have spent more time understanding the types in the interface and working with those.

How To: GitLab and Docker Registry

cazgp — Fri, 05 Jan 2018 15:51:59 +0000

How To: GitLab and Docker Registry

I (and one other engineer) spent rather too much time the other afternoon trying to work out how to set up a self-hosted Docker registry on a self-hosted GitLab site.

What we discovered (that the documentation really doesn't explain very well) is that GitLab becomes responsible for running the Docker registry and ensuring that it's accessible on whichever port you configure. That is, all you really need to do is to install Docker and change a couple of things in the GitLab config and everything happens like magic.

The Set-Up

We assume that you have a self-hosted GitLab EE site somewhere, accessible by gitlab.example.com.
We assume that this is on an Ubuntu (16.04) machine.
We also assume that you want to set up a self-hosted Docker registry and that you know what that means.

The Solution

1. Install Docker on your GitLab server

Follow the instructions here to install Docker.

I'll copy out the commands to run to save you from the pain, but do check that link in case you don't know what any of these commands are doing.

$ sudo apt-get update

$ sudo apt-get install \
  apt-transport-https \
  ca-certificates \
  curl \
  software-properties-common

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

$ sudo apt-key fingerprint 0EBFCD88

$ sudo add-apt-repository \
 "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
 $(lsb_release -cs) \
 stable"

$ sudo apt-get update

$ sudo apt-get install docker-ce

Verify that docker installed properly:

$ sudo docker run hello-world

2. Make changes to GitLab config

The container docs say "All you have to do is configure the domain name under which the Container Registry will listen to. Read #container-registry-domain-configuration and pick one of the two options that fits your case."

Click on the link they provide and you reach here which says "There are two ways you can configure the Registry's external domain. Either use the existing GitLab domain where in that case the Registry will have to listen on a port and reuse GitLab's TLS certificate, or use a completely separate domain with a new TLS certificate for that domain."

What we are trying to do is use the existing GitLab domain name because -- how cool is this? -- Docker registry login with GitLab credentials!

So, click the link that takes us here.

... and it says "If the Registry is configured to use the existing GitLab domain, you can expose the Registry on a port so that you can reuse the existing GitLab TLS certificate."

So, to summarise. The docs say "configure the domain name [for] the Container Registry ... If the Registry is configured [with] the existing [domain name]". But they completely fail to tell you how to configure the domain name! This is the part that confused us a lot.

The missing link

We assume that you don't care what port you want to run Docker registry on, and so use 4567. If you do care, change that value.

Edit /etc/gitlab/gitlab.rb.

Find the line which contains registry_external_url and change it to:

registry_external_url 'https://gitlab.example.com:4567'

Enable the registry in nginx:

registry_nginx['enable'] = true
registry_nginx['listen_port'] = 4567

Also copy in the TLS certificate lines:

registry_nginx['ssl_certificate'] = "/path/to/certificate.pem"
registry_nginx['ssl_certificate_key'] = "/path/to/certificate.key"

Save and reconfigure.

gitlab-ctl reconfigure

GitLab will automatically open the ports up and allow you to login to the registry with docker login and your GitLab credentials / 2fa keys (if you use 2fa).

That should be it! Happy GitLabing :)

Right clicks & Elm

Michael Jones — Fri, 08 Dec 2017 14:04:08 +0000

At Zaptic, we allow our customers to create a flow chart describing the business processes that they would like to complete. We draw the flow chart in SVG using a pan & zoom interface to make it easy to look around. We track mouse down, mouse drag & mouse up events to manage this UI.

Unfortunately, in development, the experience was a little annoying as every time we loaded the browser's inspector on an element the mousedown of the right-click would be tracked by the UI but the mouseup would be over the context menu and so would not register. The result? The UI gets left in "pan" mode and attempts to follow your mouse everywhere even though you don't have any mouse buttons pressed any more.

To overcome this, we needed to look into tracking which mouse button was being used so that we could ignore the right-clicks. We are using the elm-lang/mouse package which includes a position decoder that can be used for mousedown events. That decoder only provides the mouse location as pageX and pageY, it doesn't provide any information about which button was pressed so we need to go a little deeper.

If we take a look at the source code for the module we can see that the position decoder is doing the following:

position : Json.Decoder Position
position =
  Json.map2 Position
    (Json.field "pageX" Json.int)
    (Json.field "pageY" Json.int)

It is looking for the pageX and pageY properties, as we suspected, and it expects them to be integers. If we look at the lovely MDN documentation for a mousedown event we can see that it has lots of other properties including a integer button property which contains some indication of which button is being used. We can set up our own type & decoder as:

type alias MouseClick =
    { pageX : Int
    , pageY : Int
    , button : Int
    }


mouseClickDecoder : Json.Decode.Decoder PFM.MouseClick
mouseClickDecoder =
    Json.Decode.map3 PFM.MouseClick
        (Json.Decode.field "pageX" Json.Decode.int)
        (Json.Decode.field "pageY" Json.Decode.int)
        (Json.Decode.field "button" Json.Decode.int)

And adapt our event handler to use it:

type Msg
    = DragStart MouseClick
    | ...

onMouseDown : Attribute Msg
onMouseDown =
    on "mousedown" (Json.Decode.map DragStart mouseClickDecoder)

Then in our update function that is going to handle the DragStart message we can check the value of button and ignore right-clicks. There are some inconsistencies between browsers on how different buttons are represented as integers which you can read about on quirksmode. Fortunately they all align for the 'right-click' which is represented as 2. So we can ignore that:

DragStart click ->
    -- Ignore right-clicks which are given value '2' by browsers
    if click.button /= 2 then
        let
            pos =
                { x = click.pageX, y = click.pageY }
        in
        ({ model | = drag = Just { start = pos, current = pos } }, Cmd.none)
    else
        (model, Cmd.none)

And there we have it! We're free of the right-clicks and can happily jump into the browser dev-tools whenever we like.

If you have any thoughts or advice, please let me know! Always happy to learn.

Update: Checkout Ilias' comment below. It really cleans up the message handling in the update:

-        DragStart click ->
-            -- Ignore right-clicks which are given value '2' by browsers
-            if click.button /= 2 then
-                let
-                    pos =
-                        { x = click.pageX, y = click.pageY }
-                in
-                { model | drag = Just { start = pos, current = pos } } ! []
-            else
-                model ! []
+        DragStart pos ->
+            { model | drag = Just { start = pos, current = pos } } ! []