Connecting local apps to remote servers. An advance Go guide.

This guide will show how I created a system that allows clients (mobiles in this case) to connect to local servers without exposing them to the internet. My exact use case is that I have a website cronusmonitoring.com that brings monitoring and alerting to mobile devices.

A major limitation with my service so far is that if a user wants to connect their prometheus server, they would need to expose it over the internet so Cronus can communicate with it.

This guide will be using the following software I have created

• magicmirror
• socketmanager

Enough rambling. Lets get started.

The proposed design

What do we need to get started

1. Some software that runs on a user's computer that forwards requests to the desired datasource
2. This software should establish a long lasting websocket with the server
3. The client, the Cronus mobile app reaches out to my server over HTTPS. These requests need to be translated and forwarded to the websocket.

Creating magicmirror

Establishing a connection to a websocket on a server

    uRemote, err := url.Parse(remote)
    if err != nil {
        log.Fatalf("Failed to parse remote URL: %v", err)
    }

    dialer := websocket.Dialer{HandshakeTimeout: 45 * time.Second}
    header := make(http.Header)
    if apikey != "" {
        header.Set("Authorization", fmt.Sprintf("Bearer %s", apikey))
    }

    connectionURL := fmt.Sprintf("%s?name=%s", uRemote.String(), name)

    conn, res, err := dialer.Dial(connectionURL, header)
    if res != nil && res.StatusCode != http.StatusSwitchingProtocols {
        msg, err := io.ReadAll(res.Body)
        if err != nil {
            msg = []byte("")
        }
        log.Fatalf("failed to connect to remote host: %v. response code %v with response \n%v", remote, res.Status, string(msg))
    }
    if err != nil {
        log.Errorf("Failed to connect to remote host: %v %v. Retrying...", remote, err)
        return false
    }

    defer conn.Close()

We need to encode and decode requests and responses. I've omitted the code for brevity, But what we are achieving is base64 encoding the following objects.

type EncodedRequest struct {
    Method  string              `json:"method"`
    Uri     string              `json:"uri"`
    Body    []byte              `json:"body"`
    Headers map[string][]string `json:"headers"`
}

type EncodedResponse struct {
    Body       []byte              `json:"body"`
    Headers    map[string][]string `json:"headers"`
    StatusCode int                 `json:"status_code"`
}

Now that we can decode messages into a request object. We can make that request.

// handleMessage takes a base64 encoded message, decodes it, and makes a HTTP request.
func HandleMessage(encoded []byte, local string) (string, error) {

    req, err := messages.DecodeRequest(encoded, local)

    if err != nil {
        return "", err
    }

    // Make the HTTP request using http.Client
    client := &http.Client{}
    response, err := client.Do(req)
    if err != nil {
        return "", fmt.Errorf("error making HTTP request: %v", err)
    }

    resp, err := messages.EncodeResponse(response)
    return resp, err

}

Finally, we can encode the response and send it back to the server.

        _, message, err := conn.ReadMessage()
        if err != nil {
            log.Errorf("Error reading message: %v. Attempting to reconnect...", err)
            return false
        }

        resp, err := HandleMessage(message, local)
        if err != nil {
            log.Errorf("Error handling message: %v", err)
            failedRead++
            continue
        }

        err = conn.WriteMessage(websocket.TextMessage, []byte(resp))
        if err != nil {
            log.Errorf("Error writing message: %v", err)
        }

magicmirror is now created, The link at the top will show the full source code for it.

Updating the Server to forward requests to magicmirror

First of all, we need to handle creating the connection. Using socketmanager we accept inbound connections and then store the connection in socketmanager. Once again i've omitted most of the code.

func WSMirrorHandler(w http.ResponseWriter, r *http.Request, upgrader websocket.Upgrader) {
    sm, err := socketmanager.GetSocketManagerFromContext(r.Context())

        ...

    // what the user calls the connection
    name := r.URL.Query().Get("name")

    conn, _ := upgrader.Upgrade(w, r, nil)

        ... 

    uid := fmt.Sprintf("%s-%s", userid, name)

  // adding the connection to socketmanager
    sm.Add(uid, name)
    sm.SetArb(uid, constants.MIRROR_CONNECTION, conn)
}

Now we need to forward HTTP requests to the connection. I have an endpoint that users query and returns data. In this endpoint we'll update it to forward those requests to magicmirror.

Extracting the connection from socketmanager

func GetSocketConnection(sm *socketmanager.SimpleSocketManager, id string, name string) (*websocket.Conn, error) {
    uid := fmt.Sprintf("%s-%s", id, name)
    arb := sm.GetArb(uid, constants.MIRROR_CONNECTION)
    if arb.Err != nil {
        return nil, arb.Err
    }
    conn := arb.Value.(*websocket.Conn)
    return conn, nil
}

Now we can use the connection to make requests

func FetchMirrorData(sm *socketmanager.SimpleSocketManager, req *http.Request, id string, name string) (*http.Response, error) {
    conn, err := GetSocketConnection(sm, id, name)

    if err != nil {
        return nil, err
    }

    encoded, err := encoder.EncodeRequest(req)
    if err != nil {
        return nil, err
    }

    conn.WriteMessage(1, []byte(encoded))

    _, msg, err := conn.ReadMessage()
    if err != nil {
        return nil, fmt.Errorf("failed to read connection error %v", err)
    }

    resp, err := encoder.DecodeResponse(string(msg))
    if err != nil {
        return nil, fmt.Errorf("failed to read connection message %v", err)
    }

    return resp, nil
}

Finally, for the server, lets add a cleanup function to remove inactive sockets from socketmanager

func MirrorHeartBeat(ctx context.Context, dur time.Duration) {
    for {
        sm, err := socketmanager.GetSocketManagerFromContext(ctx)
        if err != nil {
            log.Errorf("mirror heartbeat failed %v", err)
            time.Sleep(dur)
            continue
        }

        active := GetActiveMirrors(sm)
        for i := range active {

            arb := sm.GetArb(active[i], constants.MIRROR_CONNECTION)
            if arb.Err != nil {
                continue
            }
            conn := arb.Value.(*websocket.Conn)
            err := conn.WriteMessage(websocket.PingMessage, []byte{})
            if err != nil {
                sm.Remove(active[i])
            }
        }
        time.Sleep(dur)
    }
}

The code is done. Time to connect

Establishing a connection from my local computer to cronusmonitoring.com

docker run mattyp123/magicmirror --remote wss://cronusmonitoring.com/mirror --apikey <INSERT> --name localprometheus --local http://192.168.0.78:9090

Now we can select it as a datasource

And then run queries against it

And finally, we can view it on the app

Thanks for reading!