DEV Community: Simon Horlick

Functional Programming Refresher

Simon Horlick — Tue, 17 Mar 2026 07:56:11 +0000

A Magma is a set with a (closed) binary operation.

A Semigroup is a magma where the operation is associative.

class Semigroup a where
  (<>) :: a -> a -> a
  -- ^ read as "append"

Must satisfy:

(x <> y) <> z == x <> (y <> z) -- associativity

Example:

[1,2] <> [3,4] <> [5] == [1,2,3,4,5]

A Monoid is a semigroup with an identity element.

class Semigroup a => Monoid a where
  mempty :: a
  -- ^ identity element of <>

Must satisfy:

mempty <> x == x -- left identity
x <> mempty == x -- right identity
(x <> y) <> z == x <> (y <> z) -- associativity (from Semigroup)

Note: mappend is a historical name for <> and is now deprecated.

Example:

mempty <> [1,2] == [1,2]
singleton 1 <> mempty == singleton 1

A Group is a monoid where every element has an inverse.

class Monoid a => Group a where
  invert :: a -> a
  -- ^ inverse of <>

Must satisfy:

x <> invert x == mempty == invert x <> x

A Functor represents a type that can be mapped over.

class Functor f where
  fmap :: (a -> b) -> f a -> f b
  -- ^ map a function over a structure
  (<$) :: a -> f b -> f a
  -- ^ replace all locations in the input with the same value

Must satisfy:

fmap id == id -- identity
fmap (f . g) == fmap f . fmap g -- composition

Example:

fmap (*2) [1,2,3] == [2,4,6]
fmap negate (Just 2) == Just (-2)

The infix version of fmap is <$>:

(*2) <$> [1,2,3] == [2,4,6]
negate <$> Just 2 == Just (-2)

An Applicative functor supports function application within their context.

class Functor f => Applicative f where
  pure  :: a -> f a
  -- ^ "box" a value into the context
  (<*>) :: f (a -> b) -> f a -> f b  
  -- ^ apply a function inside a context to a value inside a context

Must satisfy:

pure id <*> v = v -- identity

pure (.) <*> u <*> v <*> w = u <*> (v <*> w) -- composition

pure f <*> pure x = pure (f x) -- homomorphism

u <*> pure y = pure ($ y) <*> u -- interchange

Example:

(Just (*2) <*> Just 4) == Just 8
(+) <$> Just 2 <*> Just 3 == Just 5

Applicatives can be traversed with Data.Traversable.

Sequence actions, discarding the value of the first argument:

(*>) :: f a -> f b -> f b

class Applicative m => Monad m where
  (>>=) :: m a -> (  a -> m b) -> m b
  -- ^ sequentially compose two actions, passing any value produced by the
  -- first as an argument to the second
  (>>)  :: m a ->  m b         -> m b
  -- ^ sequencing operator that discards the result of the first action

Must satisfy:

return a >>= k == k a -- left identity
  m >>= return == m -- right identity

m >>= (\x -> k x >>= h) == (m >>= k) >>= h -- associativity

Example:

(Just 4 >>= \x -> Just (x * 2)) == Just 8

A MonoidalMap is a Map variant which uses the value's Monoid instance to accumulate conflicting entries when merging Maps.

If the order of arguments doesn't matter, it's indicative that the operation is Commutative.

Example:

a + b == b + a
[1,2] <> [3,4] /= [3,4] <> [1,2]

Two functions distribute over each other if they can be interchanged without affecting the result: f . g = g . f.

Example:

a * (b + c) == (a * b) + (a * c)

A free construction is one in which the mandatory laws hold, but in which absolutely no other laws hold. The free monoid is a list, and the free commutative monoid is a multiset.

Ten postgres tricks that'll make your colleagues 😯

Simon Horlick — Thu, 02 Oct 2025 05:56:08 +0000

Here are some postgres features I use all the time.

1 - Comment tables and columns

Everyone loves databases that are well commented. If you can easily see the details of the data that's being stored it makes working with it a breeze. Most database tools and downstream consumers will show these comments.

COMMENT ON TABLE public.country_codes IS 'Mapping table for country names and ISO 3166-1 codes.';
COMMENT ON COLUMN public.country_codes.country IS 'Human-readable name of the country.';
COMMENT ON COLUMN public.country_codes.region IS 'Geographic region the country belongs to.';
COMMENT ON COLUMN public.country_codes.iso2 IS 'The ISO 3166-1 alpha-2 code for the country.';
COMMENT ON COLUMN public.country_codes.iana IS 'The IANA assigned top-level domain for the country.';

2 - Automatic timestamps

Have your applications ignore created_at and updated_at fields and hand these over to the database.

ALTER TABLE public.foo ADD COLUMN "created_at" TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW();
ALTER TABLE public.foo ADD COLUMN "updated_at" TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW();

CREATE OR REPLACE FUNCTION trigger_set_updated_at_timestamp()
  RETURNS TRIGGER AS $$
    BEGIN
      NEW.updated_at = NOW();
    RETURN NEW;
    END;
  $$ LANGUAGE plpgsql;

CREATE TRIGGER set_updated_at_timestamp BEFORE UPDATE ON public.foo FOR EACH ROW EXECUTE PROCEDURE trigger_set_updated_at_timestamp();

3 - Audit tables

Want to track changes to a table with minimal effort? Simply create an audit table and have postgres store the changes:

CREATE TABLE X_audit(
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    operation CHAR(1) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
    who TEXT DEFAULT current_user,
    val JSONB NOT NULL
);

CREATE OR REPLACE FUNCTION process_X_audit() RETURNS TRIGGER AS $$
    BEGIN
        --
        -- Create a row in X_audit to reflect the operation performed on X,
        -- making use of the special variable TG_OP to work out the operation.
        --
        IF (TG_OP = 'DELETE') THEN
            INSERT INTO X_audit (operation, val) SELECT 'D', to_jsonb(OLD);
        ELSIF (TG_OP = 'UPDATE') THEN
            INSERT INTO X_audit (operation, val) SELECT 'U', to_jsonb(NEW);
        ELSIF (TG_OP = 'INSERT') THEN
            INSERT INTO X_audit (operation, val) SELECT 'I', to_jsonb(NEW);
        END IF;
        RETURN NULL; -- result is ignored since this is an AFTER trigger
    END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER X_audit_insert
    AFTER INSERT OR DELETE ON X
    FOR EACH ROW
    EXECUTE FUNCTION process_X_audit();

-- Create a separate trigger for updates that checks for changes.
CREATE TRIGGER X_audit_update
    AFTER UPDATE ON X
    FOR EACH ROW
    WHEN (OLD.* IS DISTINCT FROM NEW.*)
    EXECUTE FUNCTION process_X_audit();

4 - Indexable timestamps

Sometimes you need an index on a timestamp, in general this isn't safe as it's locale-dependent. But if we do it in a locale-independent way then we can mark the function as immutable so it is available for indexing.

-- This query *is* immutable, however to_timestamp isn't immutable by default.
-- Re-package it into a function that's marked as immutable so we can use it in
-- indexes.
CREATE OR REPLACE FUNCTION to_timestamp_json(text)
  RETURNS timestamptz AS
$$SELECT to_timestamp($1, 'YYYY-MM-DD"T"HH24:MI:SSTZH:TZM')$$
  LANGUAGE sql IMMUTABLE;

-- Add an index to speed up timestamp queries.
CREATE INDEX idx_payload_date_time ON public.foo (to_timestamp_json(payload->>'date_time'));

5 - Upserts

Postgres natively supports upserts, i.e. insert if new, update if exists.

INSERT INTO books
    ("isbn", "title")
VALUES
    ('9781420953503', 'War and Peace')
ON CONFLICT (isbn) DO UPDATE SET
    title = EXCLUDED.title;

6 - Queues

Believe it or not, you can write a fully functional MPMC queue system only using postgres features.

CREATE TYPE job_status AS ENUM ('pending', 'in_progress', 'done', 'failed');

CREATE TABLE jobs (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    status job_status NOT NULL DEFAULT 'pending',
    payload JSONB,
    visible_at TIMESTAMP WITH TIME ZONE DEFAULT now(),
    retry_count INT DEFAULT 0,
    created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);

Producers simply insert jobs into the queue:

INSERT INTO jobs (payload) VALUES ('{"task_type": "email", "recipient": "user@example.com"}');

Consumers fetch and lock jobs for processing (running the following steps inside a transaction):

UPDATE jobs
SET status = 'in_progress',
    visible_at = now() + INTERVAL '5 minutes', -- Set a visibility timeout
    updated_at = now()
WHERE id IN (
    SELECT id
    FROM jobs
    WHERE status = 'pending' AND visible_at <= now()
    ORDER BY created_at ASC
    FOR UPDATE SKIP LOCKED
    LIMIT 1
)
RETURNING id, payload;

Process the job based on the payload. After processing, update the job status.

UPDATE jobs
SET status = 'done',
    updated_at = now()
WHERE id = <job_id>;

The FOR UPDATE SKIP LOCKED is the magic phrase that ensures jobs are only visible by one consumer at a time.

7 - CHECK constraints

Sometimes database tables are edited directly by humans for a variety of reasons. Perhaps some bad data got in, or nobody got around to building that UI. Either way we can increase the safety of these edits by restricting known bad values. In the example below ISBN is a required text field, but we also check the user doesn't accidentally leave it blank (which would otherwise be allowed).

CREATE TABLE books (
    "isbn" VARCHAR(13) PRIMARY KEY,
    CONSTRAINT isbn_not_empty_ck CHECK (isbn <> ''),

    -- ...
);

Of course, this isn't a replacement for actual validation, especially as check constraints are harder to test and produce worse error messages (but who knows, maybe these can be improved in the future).

8 - Execution plans

Postgres provides great tools to analyse query execution.

book_store=# EXPLAIN (ANALYZE,BUFFERS) select * from books b inner join authors a on b.author_id = a.id;
                                                  QUERY PLAN                                                   
---------------------------------------------------------------------------------------------------------------
 Hash Join  (cost=1.18..20.88 rows=8 width=220) (actual time=0.670..0.679 rows=8 loops=1)
   Hash Cond: (a.id = b.author_id)
   Buffers: shared hit=1 read=1
   ->  Seq Scan on authors a  (cost=0.00..17.00 rows=700 width=88) (actual time=0.281..0.283 rows=7 loops=1)
         Buffers: shared read=1
   ->  Hash  (cost=1.08..1.08 rows=8 width=132) (actual time=0.362..0.362 rows=8 loops=1)
         Buckets: 1024  Batches: 1  Memory Usage: 9kB
         Buffers: shared hit=1
         ->  Seq Scan on books b  (cost=0.00..1.08 rows=8 width=132) (actual time=0.014..0.015 rows=8 loops=1)
               Buffers: shared hit=1
 Planning:
   Buffers: shared hit=105 read=19
 Planning Time: 9.751 ms
 Execution Time: 0.788 ms
(14 rows)

You can paste the output into https://www.pgexplain.dev to get a useful graphical visualisation.

9 - JSONB

Sometimes you just need to dump a bunch of json into a table. We don't always
have control over what other people send us, and sometimes it's the right tool
for the job. That said, postgres really does have great support for json:

CREATE TABLE events (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payload JSONB NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);

Querying is dead simple:

SELECT payload->>'date' AS event_date,
       payload->'details'->>'user_id' AS user_id
FROM events
WHERE payload->>'event_type' = 'user_signup';

You can even index json fields to make them fast.

10 - Lateral joins

Suppose you have two tables:

authors

id	name
1	Jane Austen
2	Mark Twain

books

id	title	author_id
1	Pride & Prejudice	1
2	Emma	1
3	Tom Sawyer	2

You want to get each author’s name and their latest book (by id).

SELECT
  a.name,
  b.title AS latest_book_title
FROM
  authors a
  LEFT OUTER JOIN LATERAL (
    SELECT title
    FROM books
    WHERE books.author_id = a.id
    ORDER BY books.id DESC
    LIMIT 1
  ) b ON TRUE;

The LEFT OUTER JOIN LATERAL (...) b ON TRUE runs the subquery to find that author’s latest book. The LATERAL keyword lets the subquery reference columns (a.id) from the outer query which is useful when you want to fetch one or more related rows per row of the main table.

result

name	latest_book_title
Jane Austen	Emma
Mark Twain	Tom Sawyer

Thoughts on our first production hasura deployment

Simon Horlick — Tue, 10 May 2022 04:10:23 +0000

Hasura is a tool to generate an api directly from your database. The workflow boils down to:

define your database tables and relationships (using normal SQL DDL statements)
spin up hasura
configure permissions rules
hook it into something like auth0 for user management
implement a ui using whatever's hot at the moment

My first thought about hasura is just how boring it is. Everything just.. works? Like, you find yourself needing to add a new feature to your app and with very little effort you find yourself finished with loads of time to spare.

The UI side is also pretty great - I'm using @graphql-codegen/typescript-react-apollo to generate client code for react. You write a graphQL query (this is the hardest part), run the codegen, then it gives you a hook you can use in your component.

Here's an example that draws a table with a bunch of data, including pagination, ordering and a search field that filters by event names. It's what we're using in production:

const EVENTS_QUERY = gql`
  query Events(
    $limit: Int = 10
    $offset: Int = 0
    $order_by: [events_order_by!] = []
    $where: events_bool_exp = {}
  ) {
    events(limit: $limit, offset: $offset, order_by: $order_by, where: $where) {
      date
      eventnumber
      name
      seriesevent {
        id
        seriesid
        series {
          seriesname
        }
      }
    }
    events_aggregate(where: $where) {
      aggregate {
        count
      }
    }
  }
`;

export const DEFAULT_PAGE_SIZE = 10;

export const FieldContainsComparison = (s: string): String_Comparison_Exp => {
  return { _ilike: `%${s}%` };
};

export function EventListContainer(props: { searchText: string }) {
  const [offset, setOffset] = useState(0);
  const [orderBy, setOrderBy] = useState<Events_Order_By>({
    date: Order_By.Desc,
  });

  let filter: Events_Bool_Exp | undefined = undefined;
  if (props.searchText !== "") {
    filter = { name: FieldContainsComparison(props.searchText) };
  }

  const { loading, error, data, previousData, refetch } = useEventsQuery({
    variables: {
      limit: DEFAULT_PAGE_SIZE,
      offset: offset,
      where: filter,
      order_by: orderBy,
    },
  });

  const latest = data ?? previousData;

  if (error) return <div>Error: {error.message}</div>;

  /* Don't attempt to draw the table until after the first set of data has been loaded. */
  if (loading && !latest) return <Loading loading={loading} />;

  return (
    <>
      <Loading loading={loading} />

      <table>
        <thead>
          <tr>
            <td>
              <div>
                Event Number
                <OrderByControls
                  setAsc={() => setOrderBy({ eventnumber: Order_By.Asc })}
                  setDesc={() => setOrderBy({ eventnumber: Order_By.Desc })}
                />
              </div>
            </td>
            <td>
              <div>
                Event Name
                <OrderByControls
                  setAsc={() => setOrderBy({ name: Order_By.Asc })}
                  setDesc={() => setOrderBy({ name: Order_By.Desc })}
                />
              </div>
            </td>
            <td>
              <div>
                Date
                <OrderByControls
                  setAsc={() => setOrderBy({ date: Order_By.Asc })}
                  setDesc={() => setOrderBy({ date: Order_By.Desc })}
                />
              </div>
            </td>
            <td>
              <div>
                Series
                <OrderByControls
                  setAsc={() =>
                    setOrderBy({ seriesevent: { seriesid: Order_By.Asc } })
                  }
                  setDesc={() =>
                    setOrderBy({ seriesevent: { seriesid: Order_By.Desc } })
                  }
                />
              </div>
            </td>
            <td>Action</td>
          </tr>
        </thead>
        <tbody>
          {latest?.events.map((event) => (
            <tr key={event.eventnumber}>
              <td>{event.eventnumber}</td>
              <td>{event.name}</td>
              <td>{event.date}</td>
              <td>{event.seriesevent?.series?.seriesname ?? ""}</td>
              <td>
                <Link to={`/dashboard/events/${event.eventnumber}`}>
                  <img width="20" height="20" src="/edit.svg" />
                </Link>
              </td>
            </tr>
          ))}
        </tbody>
      </table>
      <Pagination
        pageSize={DEFAULT_PAGE_SIZE}
        total={latest?.events_aggregate.aggregate?.count}
        offset={offset}
        setOffset={setOffset}
      />
    </>
  );
}

We went with Auth0 for user management. Figuring out how to create JWTs with the right payloads was definitely not straight forward, but it didn't take that long either. Getting hasura to accept those JWTs was very easy - just copy and paste the JWT secret into an env variable and you're good to go.

One of the screens in our app displays data from a third party REST API. We set up a hasura action to expose the REST endpoint as a graphql query and it pops up in the graphql api definitions like everything else. Pretty neat!

So, what could be improved? I'd say the experience of manipulating data needs some work. If you try to insert a record and something's not right, you'll either get a constraint violation or a permissions error. There's just not enough information to build a proper error message for end users. Another key feature I think is missing is the ability to mark fields as optional or required. Right now the graphql definitions that come out have every field as optional, even though I know many of them will cause an error if omitted. I hope given Hasura's crazy fundraising they'll be able to address these points, but so far I'm really happy with the product.

Setting up nginx+letsencrypt as a reverse proxy

Simon Horlick — Tue, 12 Apr 2022 03:31:38 +0000

This is always made slightly tricky by the fact that nginx requires the ssl certificates to be present in order to start up. This poses a chicken-and-egg problem, in that we require nginx to be serving the letsencrypt ACME challenge in order to retrieve the certificates in order to start nginx. We solve this simply by running the https server separately.

First, create the nginx configurations:

# nginx-ingress-http.conf
events {
}

http {
  include mime.types;

  server {
    listen 80;
    listen [::]:80;

    server_name sg.horlick.me;

    location /.well-known/acme-challenge/ {
      root /var/www/certbot;
    }

    location / {
      return 301 https://$host$request_uri;
    }    
  }
}

# nginx-ingress-https.conf
events {
}

http {
  include mime.types;

  server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name sg.horlick.me;

    ssl_certificate /etc/letsencrypt/live/sg.horlick.me/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/sg.horlick.me/privkey.pem;

    # taken from https://github.com/certbot/certbot/blob/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf
    ssl_session_cache shared:le_nginx_SSL:10m;
    ssl_session_timeout 1440m;
    ssl_session_tickets off;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;

    ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";

    ssl_dhparam /etc/ssl/certs/dhparam.pem;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    location / {
      proxy_pass http://backend:8080/;
      proxy_http_version  1.1;
      proxy_cache_bypass  $http_upgrade;

      proxy_set_header Host              $host;
      proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header X-Real-IP         $remote_addr;
      proxy_set_header X-Forwarded-Host  $host;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Port  $server_port;
    }
  }
}

mkdir -p /etc/nginx
cp nginx-ingress-http.conf /etc/nginx/nginx-ingress-http.conf
cp nginx-ingress-https.conf /etc/nginx/nginx-ingress-https.conf

Create the http server:

# nginx-ingress-http.service
[Unit]
Description=http server that serves letsencrypt ACME challenges and 301 redirects to https 
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop nginx-ingress-http
ExecStartPre=-/usr/bin/docker rm nginx-ingress-http
ExecStartPre=/usr/bin/docker pull nginx
ExecStart=/usr/bin/docker run -i --rm \
  --name nginx-ingress-http \
  --publish=80:80 \
  --volume /etc/letsencrypt:/etc/letsencrypt \
  --volume /var/www/certbot:/var/www/certbot \
  --volume /etc/nginx/nginx-ingress-http.conf:/etc/nginx/nginx.conf:ro \
  nginx

[Install]
WantedBy=multi-user.target

cp nginx-ingress-http.service /etc/systemd/system/nginx-ingress-http.service
systemctl enable nginx-ingress-http.service
systemctl start nginx-ingress-http.service
journalctl -u nginx-ingress-http.service

Now we're listening for ACME challenges, we can run certbot to generate the certificate.

# letsencrypt-generate-cert.service
[Unit]
Description=letsencrypt cert generation oneshot
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/docker run -i --rm \
  --volume=/etc/letsencrypt:/etc/letsencrypt \
  --volume=/var/www/certbot:/var/www/certbot \
  certbot/certbot \
  certonly \
  --webroot \
  --register-unsafely-without-email \
  --agree-tos \
  --webroot-path=/var/www/certbot \
  -d sg.horlick.me

cp letsencrypt-generate-cert.service /etc/systemd/system/letsencrypt-generate-cert.service
systemctl start letsencrypt-generate-cert.service
journalctl -u letsencrypt-generate-cert.service

Create a timer that will auto-renew the certificate before it expires.

# letsencrypt-renew-cert.service
[Unit]
Description=letsencrypt cert update oneshot
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/docker run -i --rm \
  --volume=/etc/letsencrypt:/etc/letsencrypt \
  --volume=/var/www/certbot:/var/www/certbot \
  certbot/certbot \
  renew
ExecStartPost=/usr/bin/docker exec \
  nginx-ingress-https \
  /bin/bash -c "nginx -s reload"

# letsencrypt-renew-cert.timer
[Unit]
Description=letsencrypt oneshot timer
Requires=docker.service

[Timer]
OnCalendar=daily

[Install]
WantedBy=timers.target

cp letsencrypt-renew-cert.service letsencrypt-renew-cert.timer /etc/systemd/system/
systemctl enable letsencrypt-renew-cert.timer
systemctl list-timers --all

Finally we can install and start the https server.

# nginx-ingress-https.service
[Unit]
Description=reverse proxy for api and static asset server
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop nginx-ingress-https
ExecStartPre=-/usr/bin/docker rm nginx-ingress-https
ExecStartPre=/usr/bin/docker pull nginx
ExecStart=/usr/bin/docker run -i --rm \
  --name nginx-ingress-https \
  --publish=443:443 \
  --link=backend \
  --add-host=host.docker.internal:host-gateway \
  --volume /etc/nginx/nginx-ingress-https.conf:/etc/nginx/nginx.conf:ro \
  --volume /etc/letsencrypt:/etc/letsencrypt:ro \
  --volume /etc/ssl/certs/dhparam.pem:/etc/ssl/certs/dhparam.pem:ro \
  nginx

[Install]
WantedBy=multi-user.target

# create the Diffie–Hellman parameters
openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

# create the https server
cp nginx-ingress-https.service /etc/systemd/system/nginx-ingress-https.service
systemctl enable nginx-ingress-https.service
systemctl start nginx-ingress-https.service
journalctl -u nginx-ingress-https.service

Time management for programmers

Simon Horlick — Thu, 13 Jan 2022 02:44:26 +0000

I've recently started reading Four Thousand Weeks: Time Management for Mortals. The essence is (my apologies to the author if I'm butchering it) that you'll always have too much to do, and you'll always be too busy and that all we can reasonably do is to be attentive of this and make wise tradeoffs - and not to try and do everything.

There's a common saying in software development that you should hire lazy developers - the reason being that they'll do whatever they can to reduce the amount of work they have to do. Need to search through hundreds of text documents for email addresses? No way is a lazy developer doing it by hand, they'll come up with a program to do the hard work for them. But there's a fine line: is it still better to write a program to solve a labour intensive task if it takes much longer to write the program than just doing it by hand? Developers have a tendency to create tools to do all sorts of things, and many times they make life easier, but many times they also increase complexity and slow people down.

At what point do time saving features of programming languages begin to take up more time than they save? It's something I've been thinking about constantly throughout my career. On the one hand if your code is straightforward but verbose it can be very quick to read and understand. If you've seen a piece of code a hundred times you can be pretty damn sure that there isn't anything funky going on. The downside of course is that there's just more code and, despite my previous sentence, even simple code will contain bugs at some point.

Smart people have come up with many time saving ways to create software products: all-in-one frameworks such as Ruby on Rails that (as long as you know the framework) allow you to create web applications with lightning speed, code generation tools that take a high level description of your API and spit out the boilerplate code leaving only the implementation to you. One more recent tool that I've come across, Hasura, allows you to expose your database directly as a graphql endpoint. Providing one of the most attractive no-code experiences as almost all of the traditional CRUD "backend" code is handled by the tool.

These are all fantastic tools of course, but what happens when you hire a promising young intern who is proficient at coding, but isn't familiar with all the tricks? Suddenly your time saving solutions become the bottleneck and a more verbose, but straightforward code solution would be faster for them to work with. It could just be that the intern needs to learn all of these tricks before they become productive. My core question, I suppose, is: is it possible to have your cake and eat it? Is it possible to do all the common stuff trivially and at the same time have it in a familiar form (i.e. code in a standard language)?

My intuition is that it's possible, but I haven't seen quite what I'm thinking of here. Leave a comment and let me know what your favourite tools are below!

What makes a great programming language great?

Simon Horlick — Mon, 29 Mar 2021 05:55:27 +0000

A great programming language is one that's loved by developers, has a lot of usage and fosters a healthy community - but what does it take for a language to get there? I'll try and outline what I think are the most important traits of a programming language and why I think this.

Developer experience

Developers no longer just expect a language to "work" - it also needs to provide a decent experience. This is a huge topic in itself, but to me the most important parts are: tooling, documentation, sharing and community.

Over time compilers have become far more user friendly than previous generations. Instead of just spitting out some incomprehensible error message, often unrelated to the problematic code, modern compilers offer hints about what could be wrong with the code and even offer suggestions on how to fix the problem. These days users expect that if they spell a variable name incorrectly that they'll be offered suggestions on variables in the same scope that have similar names, or that there's an extra bracket and precisely where it's missing. When compilers are helpful, users are more likely to stick with a language.

Being productive in a programming language is about more than just writing lines of code though. The majority of time spent programming is reading and discovering code, figuring out how the existing code works and reading its documentation. Discovering and navigating documentation is extremely important for developer productivity. One language that does this well is go: comments can be automatically converted into nice html documentation using the tools bundled with the language and godoc.org provides full access to this documentation. In their IDEs users are often presented with inline documentation as they hover over symbols, or clickable links that take them to the documentation hosted on godoc.org. This really increases productivity and improves the experience, especially for new developers.

Killer apps

Great programming languages have killer apps. The most obvious example of this is javascript. Whatever you may think of the language itself, it's clear that javascript is extremely popular. It doesn't take much investigating to figure out why: the internet is huge, it's everywhere, and web browsers ship with javascript interpreters out-of-the-box. If you want to easily reach billions of people, the most straight forward way is to write a javascript app and deploy it on the internet.

Another, more recent, example is dart - initially conceived as a replacement for javascript in browsers it had very little traction. As it became clear that browser vendors weren't going to ship dart interpreters with their browsers, the number of users stagnated. However, a couple of years later another team within Google decided they needed a language to target mobile devices with their new cross platform framework. The killer app for dart is flutter which provides the extremely attractive sales pitch of being able to write your mobile app once for both Android and iOS. As mobile development is a huge market, and flutter is an attractive framework, this catapults dart into the foreground - onboarding many thousands of new developers.

Community and language governance

Developers typically agree on nothing: bikeshedding is a term for development teams that spend a disproportionate amount of time discussing trivial details. Being able to collectively agree on what the goals of the language are is very important. Part of building a strong community is being very clear about what is in-scope for a language and what is out-of-scope. It's very beneficial for a community to know exactly what to expect from future versions of a language as picking up a new language can be a significant investment. If the goals of the language are made clear up-front and the team building the language sticks to it then that can relieve a lot of uncertainty around what the language will look like and whether it will be appropriate in the future. Often, saying no to new features is more beneficial than saying yes.

When you have a strong community the language ecosystem builds itself - users that love the language begin to promote it which in turn increases the community. This is also important in terms of getting a critical mass to pick up the language - if users feel empowered to create libraries and build the ecosystem then new users are more likely to find libraries that they need and are able and willing to build on that.

Conclusion

Note that I haven't mentioned anything in this article about syntax or semantics - it's not that I don't see these as important, but that in order to gain mass adoption these things are clearly not the main drivers. Take a look at some of the most popular languages: C++, javascript and PHP are all deeply flawed languages with bizarre syntax choices and surprising behaviour - however they're all extremely popular for the reasons I've outlined above.

Do comment if you agree or disagree with anything here - I'd love to discuss it!

Notes on "Erik Meijer: Functional Programming"

Simon Horlick — Thu, 15 Oct 2020 12:18:30 +0000

My notes on an interview of Erik Meijer talking about functional programming: https://www.youtube.com/watch?v=z0N1aZ6SnBk.

It's a super interesting interview. Erik is very articulate and gets straight to the heart of the problem without being overly theoretical. Here are some of my notes on the video.

12:50 - Java is dishonest about it's type signatures. For example int f() doesn't tell you whether f returns the same int each time it is called, or whether it does something in the background. Take for example the following functions:

public class Example {
  int f(int x) {
    return x + 5;
  }

  int g(int x) {
    return x + new Random().nextInt();
  }
}

Without looking at the implementation there's no way to know that g changes each time it's called.

Here's another example that says it returns an int, but actually never returns an int.

  int h(int x) {
    throw new RuntimeException("not an int!");
  }

Why is this problematic? Imagine you're working on the code in the following example. You might notice that there's some duplication going on and want to refactor it.

  int example() {
    return g(5) + g(5);
  }

So you go ahead and refactor out the repeated code and get this:

  int example() {
    int g5 = g(5);
    return g5 + g5;
  }

Great! We no longer compute g(5) twice so the function probably runs twice as fast! Unfortunately it's no longer correct though, because behind the scenes g is accessing global state. Inside the constructor of Random is a call to the OS clock in order to initialise the random number generator, which changes every time it's called.

This is a pretty contrived example, but it illustrates a real problem - it's not safe to refactor without first checking whether the implementation has side effects.

16:50 - FP is programming with mathematical functions. Every time a function is executed with the same inputs you always get the same result.

27:34 Some more examples. Assume we have the current time stored in a variable called Ticks:

static long Ticks // the current time

If we write the following functional-style code, this illustrates the problem before, that Ticks is side-effectful so returning a pair (x,x) is not the same as returning (Ticks, Ticks):

let x = Ticks // not a valid refactoring
in (x, x)

Suppose we do the following instead where we turn x into a lambda and call it twice:

let x = () => Ticks
in (x(), x())

That seems to have solved our problem, except that we could also make the following refactoring and be right back where we started:

let x = () => Ticks
let y = x
in (y, y)

Actually what we need is a way to model the sequential nature of the problem. We need the value of Ticks and then we need the value of Ticks again.

Let's say we have w to represent "the state of the world" and we make x a lambda that takes the state of the world and returns a pair containing Ticks and the state of the world.

let x = w => (Ticks, w)

We can call it like this, feeding in the current state of the world and getting back an updated state:

f(w) {
  (t, w') = x(w)
  (t', w'') = x(w')
  (t, t')
}

What we're trying to acomplish here is a sort-of "threading" through the state of the world w goes into x and comes out as a new state of the world w'. We thread it through again to get w''. Now we've made this sequential dependency explicit so that t and t' really must be different values of Ticks.

You can think about haskell in the following way. Imagine you have the following function:

 A,       W       ------------>   R,     W'

some   state of     function     result  new state
value  the world                         of the world

We can rewrite this as:

A -> W -> R, W'

Bracketing this slightly differently give us a function that takes an argument and returns a function that takes the world and returns a result and the state of the world.

A -> (W -> R, W')

Now we just hide the details of this "state of the world" thing behind a type called IO.

A -> IO R

Conclusion

I've skimmed many details here, and some of it was quite hand-wavy in the video too, but I think it really helps to build and intuition for why Monads exist (sequencing) and why it's a good idea to make computations with side-effects explicit.