DEV Community: Gavin Sykes

For loops in MySQL...kind of

Gavin Sykes — Wed, 06 Dec 2023 15:47:05 +0000

Hi everyone, sorry it's been a while!

I had a bit of a breakthrough today, finally achieving (or at least emulating the behaviour of) a for loop in MySQL, which many sources online have said isn't really possible and it's better to use a scripting language like Python or PHP to do it. That is a feasible solution in most cases, however, when I'm inside an AFTER INSERT trigger in a table where the entries are themselves added by another AFTER INSERT trigger...things get a little tricky there!

So, what's the situation?

In the platform I'm building, we have reviews on sites every so often, with the due date entered by a user when they have that information, and we need alerts to go out when the review is 9 months away, 6 months away, and 3 months away, as well as on the due date.

So our reviews table looks like this:

CREATE TABLE `reviews` (
  `review_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `site_id` int(11) unsigned NOT NULL,
  `review_due_date` datetime NOT NULL,
  PRIMARY KEY (`review_id`),
  KEY `FK_reviews_site_id_idx` (`site_id`),
  CONSTRAINT `FK_reviews_site_id` FOREIGN KEY (`site_id`) REFERENCES `sites` (`site_id`) ON DELETE CASCADE ON UPDATE CASCADE
);

All simple enough, then we have another table with the alerts:

CREATE TABLE `review_alerts` (
  `review_alert_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `review_id` int(11) unsigned NOT NULL,
  `alert_type` enum('due_date','3_month_date','6_month_date','9_month_date') NOT NULL,
  `alert_date` datetime NOT NULL,
  PRIMARY KEY (`review_alert_id`)
);

And now we need a way to track which users have seen and acknowledged the alerts:

CREATE TABLE `review_alert_acknowledgements` (
  `review_alert_acknowledgement_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `review_alert_id` int(11) unsigned NOT NULL,
  `user_id` int(11) unsigned NOT NULL,
  `acknowledged` tinyint(1) NOT NULL DEFAULT 0,
  PRIMARY KEY (`review_alert_acknowledgement_id`),
  UNIQUE KEY `UQ_review_alert_id_user_id` (`review_alert_id`,`user_id`),
  KEY `FK_review_alert_acknowledgements_user_id_idx` (`user_id`),
  CONSTRAINT `FK_review_alert_acknowledgements_review_alert_id` FOREIGN KEY (`review_alert_id`) REFERENCES `review_alerts` (`review_alert_id`) ON DELETE CASCADE ON UPDATE CASCADE,
  CONSTRAINT `FK_review_alert_acknowledgements_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`user_id`) ON DELETE CASCADE ON UPDATE CASCADE
);

So when we create a new review, we need to automatically insert its alert dates into the alerts table, then automatically insert acknowledgement records into the acknowledgements table, for each of those alerts and for each user. A for loop situation if I've ever seen one.

So, how on earth do we even begin to approach this?

Let's begin with the easy part: inserting the alerts when we create a new review, this goes in the AFTER INSERT trigger on reviews:

BEGIN
  INSERT INTO `review_alerts` (`review_id`,`alert_date`,`alert_type`) VALUES (NEW.review_id,NEW.review_due_date,'due_date');
  INSERT INTO `review_alerts` (`review_id`,`alert_date`,`alert_type`) VALUES (NEW.review_id,NEW.review_due_date - INTERVAL 3 MONTH,'3_month_date');
  INSERT INTO `review_alerts` (`review_id`,`alert_date`,`alert_type`) VALUES (NEW.review_id,NEW.review_due_date - INTERVAL 6 MONTH,'6_month_date');
  INSERT INTO `review_alerts` (`review_id`,`alert_date`,`alert_type`) VALUES (NEW.review_id,NEW.review_due_date - INTERVAL 9 MONTH,'9_month_date');
END

And now the tricky part: insert the acknowledgement records for each alert and user when creating the alerts, oh, and did I also mention we only want a subset of users, based on them having the admin role? A lovely little added complication there, I'm sure you'll agree. Let's go into the AFTER INSERT trigger on review_alerts:

BEGIN
  FOR user_id IN -- Oh, wait, that won't work, how about...
  FOR EACH (SELECT user_id FROM users) -- Yeah that won't work either. How on earth do I achieve this?
END

Enter the INSERT SELECT statement.

First of all, can we achieve a table with all the user IDs we want, coupled with just a number? Yes! Let's try:

SELECT
  (SELECT 1),
  user_id
  FROM `users` WHERE user_role = 'admin'; -- The roles are a bit more involved than just a field in the users table, but this is just for demo purposes!

And sure enough, we get

+------------+---------+
| (SELECT 1) | user_id |
+------------+---------+
| 1          | 1       |
| 1          | 2       |
| 1          | 3       |
| 1          | 5       |
| 1          | 8       |
| 1          | 13      |
| 1          | 21      |
+------------+---------+

Success! And now we simply replicate this functionality in our INSERT SELECT statement within the AFTER INSERT trigger.

BEGIN
  INSERT INTO `review_alert_acknowledgements` (`review_alert_id`,`user_id`) SELECT DISTINCT
    (SELECT NEW.review_alert_id),
    users.user_id
    FROM `users`
    WHERE users.role = 'admin';
END

Do you need the DISTINCT? If your role assignment system works as I've reduced it to here for the purposes of this demo, then no. Every user will either be an admin or be something else. In my system as-is however, then it is probable that you'll need it in order to avoid any unique key constraint errors. For example if some of your admins are also superadmins and you want them to be able to do everything an admin can do (because why wouldn't you?)

And that's it! At least for "looping" in an INSERT statement, anyway.

Build a Modern API with Slim 4 - Verify your .env file

Gavin Sykes — Tue, 07 Nov 2023 15:53:49 +0000

Welcome back to this series, it was put on hold some time ago, but I'm now back and ready to go!

In the second part of this series we set up our .env file which contained both environment-specific and secret variables, both of which are reasons to have Git ignore the file.

So, how do I know, then, that when I deploy onto my staging or production servers, that the file contains everything I need? During this project there have been a few times I've made some updates, added an environment variable, then realised I didn't make that same change on the server. This is where a verification script comes in.

Keep in mind that this file is to be committed, because, whilst it contains the names of all our environment variables, it does not contain their values.

I have also opted for something a little bit different and done this in Bash, this allows complete agnosticism as regards programming languages, so you could be writing your API in PHP, Go, C#, JavaScript, it doesn't matter. As long as your server has Bash, this will run (and considering that both Linux and macOS have Bash baked-in, and I'm fairly sure but not certain that you can run Bash scripts in Windows Powershell, you're more likely to have it than not).

So, how do we get started?

The first and most important line is the hashbang or shebang, also known as

#!/bin/bash

This is what makes the script executable just by running ./check_env.sh and telling the system that it needs to use the Bash interpreter to run it.

And now we tell it which environment variables should appear in our file:

#!/bin/bash

required_env_vars=("_ENVIRONMENT" "_PDO_HOST" "_PDO_USERNAME" "_PDO_PASSWORD" "_PDO_NAME" "_ENCRYPTION_CIPHER_METHOD" "_HASHING_COST" "_SMTP_HOST" "_SMTP_USERNAME" "_SMTP_PASSWORD" "_RATE_LIMIT_GET" "_RATE_LIMIT_POST" "_RATE_LIMIT_PUT" "_RATE_LIMIT_DELETE")

And now let's tell it where to find our .env file, this will be in the same directory as our script file so simply

#!/bin/bash

required_env_vars=("_ENVIRONMENT" "_PDO_HOST" "_PDO_USERNAME" "_PDO_PASSWORD" "_PDO_NAME" "_ENCRYPTION_CIPHER_METHOD" "_HASHING_COST" "_SMTP_HOST" "_SMTP_USERNAME" "_SMTP_PASSWORD" "_RATE_LIMIT_GET" "_RATE_LIMIT_POST" "_RATE_LIMIT_PUT" "_RATE_LIMIT_DELETE")

env_file=".env"

However, what if our .env file is somewhere different, like in /shared? Luckily we can check for that as well:

if [ $# -eq 1 ]; then
  env_file="$1"
fi

So, um, what does that do?

This looks for the number of command line arguments($#), and if it equals (-eq) 1, set the env_file variable to the first argument ("$1"), rather than .env. So now if we run ./check_env.sh it will look for our .env file locally, whereas if we run ./check_env.sh /shared/bookstore_api/.env then it will look for it in /shared/bookstore_api/.env instead.

fi being if backwards, just ends the if statement.

Now we need to make sure our .env file actually exists.

if [ ! -f "$env_file" ]; then
  echo "Error: .env file not found."
  exit 1
fi

The if statement here is now looking for the file, signified by the -f flag before "$env_file", with the preceding ! meaning negation i.e. if the file is not found, do this. It will then echo the error message then completely exit the program. But what does that 1 mean?

This is a concept that carries on strong in low-level programming from the very earliest days of computing. Every program ever written anywhere, ever, has to exit or end eventually, and exit codes are how we indicate whether or not it was successful.

A 0 means success - if it helps, consider it to mean "nothing to report", and anything other than 0 means an error. Some programs will use different numbers for different errors, but for the purpose of this short script, exit 1 on an error will do just fine.

#!/bin/bash

# What variables are we looking for?
required_env_vars=("_ENVIRONMENT" "_PDO_HOST" "_PDO_USERNAME" "_PDO_PASSWORD" "_PDO_NAME" "_ENCRYPTION_CIPHER_METHOD" "_HASHING_COST" "_SMTP_HOST" "_SMTP_USERNAME" "_SMTP_PASSWORD" "_RATE_LIMIT_GET" "_RATE_LIMIT_POST" "_RATE_LIMIT_PUT" "_RATE_LIMIT_DELETE")

# Where is our .env file?
env_file=".env"

# Have we been told it's somewhere else?
if [ $# -eq 1 ]; then
  env_file="$1"
fi

# Is it where it should be?
if [ ! -f "$env_file" ]; then
  echo "Error: .env file not found."
  exit 1
fi

Okay so now let's start looking for our variables:

for var in "${required_env_vars[@]}"; do
  grep -q "^$var=" "$env_file"
  if [ $? -ne 0 ]; then
    echo "Error: $var is missing in the $env_file file."
    exit 1
  fi
done

Okay, so...what? What's happening here?

Okay so let's break it down:

for var in "${required_env_vars[@]}"; do
  # Do some stuff
done

So this should be a fairly self-explanatory for loop: for each of the required variables defined above, do some stuff. That [@] is unique to Bash and means that it treats everything in the array as an item, rather than treating the whole array as an item.

  grep -q "^$var=" "$env_file"

The first line inside the loop, this uses grep to search for a line within our .env file that matches ^$var= which means that it contains our variable name, followed immediately by =, and the ^ signifies that the line must start with that. Finally, the -q means quiet, and tells grep not to output anything, just to store its exit code, which will be 0 i.e. success if it finds the line.

  if [ $? -ne 0 ]; then
    echo "Error: $var is missing in the $env_file file."
    exit 1
  fi

That exit code is stored in $? and -ne means not equal, so if grep's exit code is not 0, then echo the error message and, again, exit 1.

Now, finally, if that loop runs fine and we reach the end of it, we can echo a success message and exit 0

echo "All environment variables present and correct in $env_file."

exit 0

Is there anything else we can add?

Well, yes, I am not by any means an expert on Bash scripting, even just writing this article I can see areas for improvement and I'm sure you can as well! There is something that we can add though and that I will demonstrate below, verification of the values.

Obviously, be careful here, you don't want to commit a file that says something like "the database password should match SuperSecretPassword123", because that will negate the entire point of keeping the password in a secret environment file!

What I will demonstrate though is limiting the options for _ENVIRONMENT, which can serve as protection against things like typos.

ENVIRONMENT=$(grep "^_ENVIRONMENT=" "$env_file" | cut -d'=' -f2)
allowed_environments=("development" "staging" "demo" "production")

if ! [[ " ${allowed_environments[@]} " =~ " $ENVIRONMENT " ]]; then
  echo "Error: _ENVIRONMENT must be one of ${allowed_environments[@]}"
  exit 1
fi

So, what's happening here?

We are fetching the provided _ENVIRONMENT value by running grep "^_ENVIRONMENT=" "$env_file" and piping the resulting line into cut -d'=' -f2 meaning "cut this line using = as the delimiter, and return the second part". Then setting the allowed environments as we did the required variables at the start.

Now, we need to make sure that ENVIRONMENT is actually one of our allowed_environments, we take care of this in the if statement, then once again, if needed, we echo our error message and exit 1.

All of this will leave us with a script that looks like the below:

#!/bin/bash

required_env_vars=("_ENVIRONMENT" "_PDO_HOST" "_PDO_USERNAME" "_PDO_PASSWORD" "_PDO_NAME" "_ENCRYPTION_CIPHER_METHOD" "_HASHING_COST" "_SMTP_HOST" "_SMTP_USERNAME" "_SMTP_PASSWORD" "_RATE_LIMIT_GET" "_RATE_LIMIT_POST" "_RATE_LIMIT_PUT" "_RATE_LIMIT_DELETE")

env_file=".env"

if [ $# -eq 1 ]; then
  env_file="$1"
fi

if [ ! -f "$env_file" ]; then
  echo "Error: .env file not found."
  exit 1
fi

for var in "${required_env_vars[@]}"; do
  grep -q "^$var=" "$env_file"
  if [ $? -ne 0 ]; then
    echo "Error: $var is missing in the $env_file file."
    exit 1
  fi
done

ENVIRONMENT=$(grep "^_ENVIRONMENT=" "$env_file" | cut -d'=' -f2)
allowed_environments=("development" "staging" "demo" "production")

if ! [[ " ${allowed_environments[@]} " =~ " $ENVIRONMENT " ]]; then
  echo "Error: _ENVIRONMENT must be one of ${allowed_environments[@]}"
  exit 1
fi

echo "All environment variables present and correct in $env_file."

exit 0

Self-referencing types and generics: what are they and how can I use them in React?

Gavin Sykes — Mon, 30 Oct 2023 10:11:25 +0000

Hi everyone! So I have this React app that I am currently building, and I have a parent component which has a piece of state, and contains a child component which I want to be able to both receive and manage that state. It could be a controlled input component for a form, perhaps.

export function Form() {
  const [username, setUsername] = useState('');
  return (
    <form>
      <ControlledInput value={username} setValue={setUsername} />
    </form>
  )
}

Basically the simplest and most useless form you will ever see, I'm sure you agree!

Now, if we take a look at the props interface for our ControlledInput we will see that nothing looks too out of the ordinary:

interface ControlledInputProps {
  setValue: () => void;
  value: string;
}

Simple enough, right?

Will this work? Yes.

Okay, so what is the problem?

Well, this presents 2 potential problems, mainly surrounding accuracy and maintainability.

Accuracy: if we focus just on the () => void part for a second, we can see that it is not necessarily a state updater, but any function that does not return anything, so if I were to pass it even just setValue={() => {console.log("This will work fine, honest!")}}, it would happily accept that and not tell me that there is an issue.

So, how do we fix this?

Well, React allows us to import 2 types: Dispatch and SetStateAction, both of these seem fairly self-explanatory, and those who have used useReducer with TypeScript in the past will already be familiar with Dispatch. SetStateAction allows you to not have to worry about typing the action yourself, just pass it the type of the state you're updating as a generic so you'll finally end up with Dispatch<SetStateAction<string>>:

interface ControlledInputProps {
  setValue: Dispatch<SetStateAction<string>>;
  value: string;
}

Immediately, even just to the human eye, is that not much more descriptive of what setValue does? This also limits us to only passing in functions that are state updaters for strings. This does mean, though, that you still have to make sure you are passing in the right piece of state with its setter! If you pass value={username} setValue={setPassword} then you will still have a bad time, and as of the time of writing this I am not too sure of a way around that. value={username} setValue={setAge}, on the other hand? You'll get those warnings.

Okay, so what about maintainability?

Let's say we have another component other than an input that takes the same props:

interface ComponentProps {
  setValue: Dispatch<SetStateAction<string>>;
  value: string;
}

But, what if we update the app and now want this to take string | number, or even string | number | null?

interface ComponentProps {
  setValue: Dispatch<SetStateAction<string | number | null>>;
  value: string | number | null;
}

Not only is that starting to look a bit messy and long-winded, but it also means us having to update our types in 2 places at once. I do not care if those 2 places are right on top of each other, we are developers and therefore do not like updating the same thing in 2 places at once!

Enter self-referencing types

interface ComponentProps {
  setValue: Dispatch<SetStateAction<ComponentProps['value']>>;
  value: string | number | null;
}

This now tells the setValue prop that it is a dispatcher for a SetStateAction for whatever type value happens to be.

Now, the eagle-eyed among you will have noticed that ComponentProps['value'] is actually more characters than string | number | null, so why are we making our description even longer? Sure, it explains why the type is how it is, whereas with it set to Dispatch<SetStateAction<string | number | null>> we might have to do some digging to work that out, but is there an even neater way of doing this?

Enter generics

Generics allow us to save so much time when defining our types, because we needn't worry so much about typing things differently if an input happens to be a number instead of a string, or a boolean instead of a number. We simply pass T and let it do its thing.

So in order to make our interface generic we do:

interface ComponentProps<T> {
  setValue: Dispatch<SetStateAction<T>>;
  value: T;
}

Then to make use of this new functionality we can go back to our ControlledInput and declare it as:

function ControlledInput({ setValue, value }: ComponentProps<string>) {}

Which means "hey, I want to have these props, and assign these types to them, based on the fact I'm working with a string".

This then opens the door to a reusable set of props that you can import into any component inside your app, and even extend from there.

export interface ComponentWithStateValueAndSetterProps<T> {
  setValue: Dispatch<SetStateAction<T>>;
  value: T;
}

type ControlledTextInputProps = ComponentWithStateValueAndSetterProps<string>;

export function ControlledTextInput({ setValue, value }: ControlledTextInputProps) {
  return <input type="text" value={value} onChange={e => setValue(e.currentTarget.value)} />
}

Final step

This is all well and good, but now how do I add things like an id or className to my input, TypeScript won't let me!

Well, we can build upon more than 1 base type, depending on whether you prefer interface or type there are a couple of different ways of doing this.

interface ControlledTextInputProps extends HTMLProps<HTMLInputElement>, ComponentWithStateValueAndSetterProps<string>;
// OR
type ControlledTextInputProps = HTMLProps<HTMLInputElement> & ComponentWithStateValueAndSetterProps<string>;

export function ControlledTextInput({ setValue, value, ...inputElementProps }: ControlledTextInputProps) {
  return <input type="text" value={value} onChange={e => setValue(e.currentTarget.value)} {...inputElementProps} />
}

HTMLProps is another import from React that provides all the props that we can add directly to whichever element we pass as a generic, in this case, HTMLInputElement. It makes them all optional so that we can pass in as many or as few as we want and name them exactly the same as if we were adding them to the element itself.

So now the component will take a whole host of additional props, such as className or id and apply them to the input element.

One extra consideration

TypeScript has a whole bunch of native types available, many of which you will probably never actively use. With this in mind, you can restrict T to a limited set of types, if you wanted to restrict it to be either a number or a string you'd do interface ComponentWithStateValueAndSetterProps<T extends string | number>. Be mindful with this approach though, as you may find yourself accidentally disallowing certain types that you will make use of in state, such as arrays or objects.

Before:

interface ComponentProps {
  setValue: () => void;
  value: string;
}

Shorter (hey, I had to find something as a pro!)
Not type-safe
Less descriptive
Not customisable
Reusable if and only if multiple components take the same props
Easily updateable (because of the lack of type-safety)

After:

interface ComponentWithStateValueAndSetterProps<T> {
  setValue: Dispatch<SetStateAction<T>>;
  value: T;
}

Longer (likewise, I had to find something as a con)
Type-safe
More descriptive
Customisable
Easily reusable
Easily updateable (with no loss of type-safety, in fact, would it even need updating at all, due to the generic?)

Always remember that you're not writing code just for today, you're writing code for the person who will be using your application next year, the QA team who will be testing it in 5 years, and the developer who will be maintaining it in 10 years. Keep all of these people happy, because you might just be one of them!

Happy coding!

React Custom Hooks that can take props, but not always, what to do in TypeScript?

Gavin Sykes — Fri, 20 Oct 2023 10:34:54 +0000

Hi everyone, a bit of a shorter article today, but I recently ran into the question of what to do in a React hook if I want it to be able to take props, but not have to?

Let's say we have a hook that fetches blog posts from an API, in its simplest form we have:

import { Post } from "@/contexts/PostContext";

interface usePostsProps {
  postedByUserId?: number;
}

interface usePostsReturn {
  posts: Array<Post>;
  status: 'loading' | 'error' | 'success';
}

export function usePosts(props: usePostsProps): usePostsReturn {
  // session fetching, state setters etc.
  let url = process.env.api_base_url;
  if (props.postedByUserId) {
    const params = new URLSearchParams('posted_by_user_id', props.postedByUserId);
    url = url '?' + params.toString();
  }
  useEffect(() => {
    // Fetching logic here
  },[props, session, etc.]);
  return { posts, status };
}

Now this hook looks perfectly fine and will work fine as well, if you use it on your users/[userId] route as postsByThisUser = usePosts({ postedByUserId: userId }); then it will return that user's posts and you can display them.

Now let's say you also have a /posts route where you want to display all posts, regardless of who posted them. Well, that question mark in the props' type declaration allows us to send it an empty object as props with no issues: posts = usePosts({});.

So you're saying it works fine? Well, what's the problem then? Why are you writing this article?

Well, this approach presents 2 potential problems, one for the developer experience and one bug:

Do you really want to have to send an empty object as props when you know you don't really need to?
In components where you do want to filter down to a user ID, you won't get any TypeScript warnings if you inadvertently submit {} as props.

Okay, so let's adjust the type of our props to get round both these issues.

interface usePostsProps {
  postedByUserId?: number;
}

Now, the difference between interface and type in TypeScript is often little more than personal preference. However, there are some situations where one is better than the other, or one of them even won't work at all. This will turn out to be a situation where interface won't work so we have to use type.

type usePostProps = { // Take note that type also needs an =, whereas interface doesn't allow one
  postedByUserId: number;
}

We also removed the ? from postedByUserId, because if we are passing it props, then they have to contain that property, don't they? But now, I get an error message wherever I call usePosts({}) because, of course, it's missing the user ID, but I don't want to send a user ID, so how do I get round this? Could I maybe use a union type with that type and undefined?

type usePostProps = {
  postedByUserId: number;
} | undefined;

So now I can use usePosts() with no empty object and no - oh, "Expected 1 arguments, but got 0."? So, you're saying I have to pass undefined as a prop now? That isn't what I want at all, and doesn't exactly look great either. What if I try null instead? Hmm, same problem. How do I fix this?

Enter void

void is another keyword that TypeScript uses to signify the presence of nothing. You may already be familiar with () => void as signifying a function that does something, but doesn't return anything. So what happens if we try void instead?

type usePostProps = {
  postedByUserId: number;
} | void;

Now let's try usePosts() and...no error messages! But let's also double-check that it's fine when we do pass props with usePosts({ postedByUserId: userId }), also no error messages, great! But what if I now try with an empty object, as that's what we identified as a possible bug earlier? usePosts({}) now tells us "Argument of type '{}' is not assignable to parameter of type 'usePostsProps'.", which means that now, if we pass any props, it enforces the rule that those props must contain the postedByUserId property.

So now we have gone from a hook that, okay, works fine and gives you the data you want, to one that is properly type-safe and has much less potential to introduce the sort of bug that we could spend half a day looking for, because TypeScript will now actually tell us where the problem is.

Build a Modern API with Slim 4 - Add your First Route

Gavin Sykes — Sun, 05 Mar 2023 12:00:00 +0000

Now that we have our application up and running, we need to make it actually look our for calls on certain routes, as it stands it won't actually do that as we haven't told it what to look out for.

As mentioned way back in the introduction, we will be creating a bookstore API, so let's first of all create a /books endpoint. This all goes in our src/routes folder:

src/routes/books.php

<?php

$app->group('/books', function($app) {
  $app->get('', function($request, $response, $args) {
    // Code will go here
  });
}

And in our index file we add, between creating the app and adding the 3 middlewares:

...
AppFactory::setContainer($container);
$app = AppFactory::create();

require __DIR__ . '/../src/routes/books.php';

$app->addBodyParsingMiddleware();
...

Wait, so, what have we actually done here? Well firstly we've included our books file in our index, as without that it will never even attempt to run it, then in our books file we've added a group called books, inside which we've added a GET endpoint with an empty endpoint, this has enabled the api.bookstore.com/books endpoint. Had we done, say, $app->get('all' ... then it would be api.bookstore.com/books/all.

However, this isn't what our final books file will look like. Ideally we want this file to just handle the routing and add any associated middlewares (JSON validation for example which won't be on every route). We need to make use of our controllers directory for this.

Build a Modern API with Slim 4 - Create your Index File

Gavin Sykes — Sun, 26 Feb 2023 13:00:00 +0000

Now that we have everything set up we can actually start creating our index file which will handle all incoming requests and return the relevant responses.

We first start off by importing the relevant classes from the packages we installed, this is done with the use statement:

<?php
use DI\Container;
use Slim\AppFactory;
// Expect this list to get much, much longer as we progress!

require __DIR__ . '../vendor/autoload.php';
// This script is generated by Composer and brings in the code from all the classes listed above

Now we will create our $container and $app objects, the container is what will handle our dependency injection and the app is what will actually listen out for and handle the requests and responses.

$container = new Container();

AppFactory::setContainer($container);
$app = AppFactory::create();

You may think that we haven't actually done much to the container thus far, and you'd be correct. We will soon be setting different properties on it in order to allow our app to access them.

One major part of any web-based application, be it an API, website or webapp, is middleware; middlewares are what we use to manipulate requests coming into the server before they reach our controllers, as well as responses leaving the server before they reach the user.

The order is which middleware is applied varies in importance, there are some where it doesn't matter where they go in the stack and there are others that have to be below or above certain others. For example if your authentication middleware adds the user_id to the request, then any middleware that relies on that ID must be below it.

For this reason, middleware is applied in what I'll call a layered approach, the outermost middleware is what the request will first encounter, and what its response will last encounter. All midlewares have the capability to manipulate both requests and responses, although I have found it typical to write them in such a way that they only do one or the other.

Slim's diagram of how middleware operates, from the documentation

We will talk about writing our own middlewares later in this series, but for now we will simply add some already built-in to the Slim framework:

$app->addBodyParsingMiddleware(); // This enables us to easily access the contents of, say, a JSON body or form submission
$app->addRoutingMiddleware(); // This lets us access the route of a request and see where it is going and what it is trying to do
$app->addErrorMiddleware(true, true, true); // This handles errors and returns the relevant messages.
// The three parameters above determine whether to display the error details to the user (fine in development, not in production), log the error, and log the details of the error.

The last middleware added is the outermost layer, so in this case we have our app wrapped in the body parsing middleware, wrapped in the routing middleware, wrapped in the error middleware. It is wise to have the error middleware as your outermost layer, and Slim's documentation reflects this.

We now have one final line of this script, the most important of all; actually starting the application. This is done quite simply with $app->run();, leaving us with the below file:

public/index.php

<?php
use DI\Container;
use Slim\AppFactory;

require __DIR__ . '/../vendor/autoload.php';

$container = new Container();

AppFactory::setContainer($container);
$app = AppFactory::create();

$app->addBodyParsingMiddleware();
$app->addRoutingMiddleware();
$app->addErrorMiddleware(true, true, true);

$app->run();

Build a modern API with Slim 4 - Generate your keys

Gavin Sykes — Sun, 19 Feb 2023 13:00:00 +0000

By the end of this piece we will have 3 files in our keys folder:

- keys
  - encryption_key
  - private.key
  - public.key

But what do each of these do? encryption_key is obviously an encryption key but what does it actually encrypt? Well it could encrypt anything but in this case we'll be using it to encrypt data going into the database. Always a good practice if the chances of your database being attacked are more than zero. If your database exists, then those chances are indeed more than 0.

You probably won't need to encrypt everything in the database, but certain info like customer's addresses and phone numbers for example would be best encrypted.

Now on how to generate the key: there are any number of ways to generate one and really there are no rules as to what they are. That said it is always best to use a randomly-generated one, you could just open the encryption_key file and mash your keyboard, but I prefer a more scriptable approach!

Either write a PHP file and run it, or run php -a to open a php shell and run (or write into the file):

file_put_contents('./keys/encryption_key', base64_encode(random_bytes(32)) . PHP_EOL);

The path will depend on where you currently are in the directory of course.

And you should end up with something like /KH6z56IJ9qw74o2Fk4XOJn1M94/WqfaBsW+cDQax6o=.

Now for the private and public keys, what are these for? These aren't anything to do with encrypting data on the database, these are instead for signing and verifying the access tokens given to users of our API.

We will generate them using the RSA algorithm which is known as an asymmetrical algorithm. That means it generates a private key and a public key. The private key signs tokens and the public key verifies that those tokens have been signed with the private one. For this reason, while the public key is safe to share you must NEVER share the private one.

We generate these with 2 commands, these are done in the shell and not PHP (although you can also use shell_exec within PHP but why do that if you don't need to?) First for the private key:

openssl genrsa -out ./keys/private.key 2048

And it should display something like

Generating RSA private key, 2048 bit long modulus
....+++
...+++
e is 65537 (0x10001)

Next for the public key:

openssl rsa -in ./keys/private.key -pubout -out ./keys/public.key

And it should display

writing RSA key

Notice how the public key generator took the private key as an in parameter? That's because these keys always come in pairs, so any change at all to either of them will render the other one useless. We also used the -pubout flag to tell openssl that it is a public key we're generating. I have just generated some disposable ones to show you an idea of how they should look:

private.key

public.key

So now we have all the keys we will need!

Build a modern API with Slim 4 - Set up your filesystem

Gavin Sykes — Sun, 12 Feb 2023 13:00:00 +0000

By now you should have a folder structure that looks a bit like the below:

- vendor
.env
.gitignore
composer.json
composer.lock
README.md - always a good idea to have one of these

You may notice something fairly major missing at this point: where do we actually write the scripts to actually run the API? Well here is where we set that up.

There is any number of differing schools of thought on how best to set up your filesystem for simplicity and ease of development and the world really doesn't need yet another person's opinion on that. So, here's my opinion on that!

At the very minimum, you will need a public directory containing index.php, this is what all requests will be processed through when they come to your API. For now though it needn't contain anything beyond the <?php opener.

- public
  - index.php
- vendor
.env
.gitignore
composer.json
composer.lock
README.md

In theory, this index file could contain everything: routes, controllers, modules, middleware, documentation, all of it. This can certainly be made to work, though I would not at all rate it highly on the ease-of-development scale!

Firstly I would certainly separate out the routes and controllers, in your root directory you can mkdir src/routes src/controllers in order to do that. What I end up with is below:

- aws
- config
- database
- docs
- encryption
- keys
- logging
- public
  - index.php
- src
  - controllers
  - middlewares
  - routes
  - schemas
  - scripts
  - utils
- vendor
.env
.gitignore
composer.json
composer.lock
README.md

Some of the above will be fairly obvious as to what they will do: src is your source code (well, it all is in a sense), logging is where we'll build a logging module unless you install one with Composer. Many options are available but I have built my own so that will be part of this series. If you've guessed that aws means connecting to AWS you'd be correct, do you need to do this? Of course not, but again, I have. Next we'll get possibly the shortest bit out of the way: generating the keys to go into the keys folder, and just what they're for.

Build a Modern API with Slim 4 - Set up your .env file

Gavin Sykes — Sun, 05 Feb 2023 16:00:00 +0000

In the last post you may have spotted mention of a .env file. What is it? What does it do? Where does it sit? How does the application use it?

All of the above will be answered. For now though the very first thing you will want to do, before even creating it, is to add it to your .gitignore. This will contain all of our keys, database login info, and many other bits we don't want anyone to see. So gitignoring it will make sure none of those values are committed.

Make sure that we are in our project root, where our composer.json and composer.lock should be.

echo .env >> .gitignore
touch .env

Now open your newly-created file in your favourite IDE and add some variables.

_ENVIRONMENT="development"
_PDO_HOST="ip.address.or.domainname"
_PDO_USERNAME="user.name"
_PDO_PASSWORD="SuperSecretPassword123"
_PDO_NAME="DatabaseName"
_ENCRYPTION_CIPHER_METHOD="AES-256-CTR"
_HASHING_COST="10"
_SMTP_HOST="smtp.mybookstore.com"
_SMTP_USERNAME="bookstore.ceo"
_SMTP_PASSWORD="ExtraSuperSecretPassword1234"
_RATE_LIMIT_GET="1200"
_RATE_LIMIT_POST="600"
_RATE_LIMIT_PUT="600"
_RATE_LIMIT_DELETE="300"

You'll notice that some of those variables don't necessarily need to be kept secret. In particular, does the SMTP host not get included in every single email our system is going to send? Well, yes, however this file isn't just for secrets, it is also a handy place to store environment-specific variables such as, well, the first line! _ENVIRONMENT=development, staging, production, any others such as demo?

By having this file you can recreate it on each server in each environment and only have to change the relevant variables in one place. The same goes for the rate limits: they're going in our documentation, there's no way they're secret!

Build a Modern API with Slim 4 - Introduction

Gavin Sykes — Sun, 29 Jan 2023 16:00:00 +0000

Gretings! I have decided to put together this series on how to create a modern API based on Slim's latest PHP-based framework, and we will be looking at each individual piece in detail that makes a good API: handling CORS requests from browsers, authentication, database links, encryption, validation, middleware, file uploads and downloads. As well as a couple of bonus "pieces".

This series assumes competency in PHP, your chosen version control system, in my case git; also file management, be this the terminal, VSCode's sidebar or even Windows Explorer, I'll mostly be using the terminal (and a Mac which completely rules out Windows Explorer!).

To use a practical example, all we will do is create a small bookstore API that stores info on books, prices, ISBNs, authors, and can be called to do various things with that info such as selling a book, adding new books, taking on a new author and managing stock.

What packages to install?

slim/slim
The base package that we will be using.
slim/psr7
Slim requires a PSR7 implementation in order to handle request and response objects. There are a few available out there but I find it easier to simply use the one that the people at Slim have written themselves.
vlucas/phpdotenv
Used for extracting environment variables from your .env file.
aws/aws-sdk-php
You found the first bonus piece! Used to connect to your AWS account. This is of course only necessary if you have an AWS account and plan to use any parts of it with your API. If not, then you can skip installing this and also skip its post later in the series.
php-di/php-di
Used to build containers to make dependency injection much simpler.
opis/json-schema
Used to validate JSON that is sent as part of a POST, PUT, or PATCH request.
slim/twig-view
Another package from the people at Slim, this is to handle documentation. Slim is designed for more than just API responses, you can build whole web applications from it which would often involve HTML templates; not really necessary for an API in itself but very useful for documentation.
tuupola/cors-middleware
This is some very handy middleware written by Mika Tuupola to handle CORS requests. Ah, CORS you beautiful infuriating 3xp1371v3, of all the times to rear your head, you decide to do it right at the end of my API building and when I try to get a browser to speak to it rather than mere cURL or Insomnia requests. Do not attempt to write your own CORS middleware unless you have unlimited time left on this earth!

We can install all of these with composer require opis/json-schema php-di/php-di phpmailer/phpmailer slim/psr7 slim/slim slim/twig-view tuupola/cors-middleware vlucas/phpdotenv

After a few minutes you'll find yourself with a composer.json and composer.lock in your project root. These can both be committed, composer.json is the one that can be edited if you need to manipulate package versions or add additional properties (which we will be doing later), but do not touch composer.lock! You can commit these straightaway.

What this will also create is a directory called vendor which we should gitignore as that is all the code of all our installed packages.

Make sure to be in your project root when running the below command, where your composer.json and composer.lock are.

echo /vendor >> .gitignore

Side note - if you're coming from NPM world, you'll notice that all the packages are namespaced, that is true of all packages in Composer. You'll also note the similarity between composer.json and package.json, composer.lock and package-lock.json, and vendor and node_modules.

Creating a setter function in JavaScript Objects

Gavin Sykes — Tue, 17 Nov 2020 09:12:01 +0000

As part of one of my projects I have had a central object called "layout" with it being to do with manipulating SVGs, and as part of that I decided it would be easier to write said layout object such that it has a getter and setter function, so that:

layout.item.points.X = 20;

...becomes:

layout.set('item.points.X',20);

...much more verbose and much more functional, I'm sure you'll agree!

So first, the getter function, well, this couldn't be easier really:

const layout = {
  get: function() {
    return this;
  },
  set: // To be confirmed
  /* All other properties */
}

But now, what about the setter? Well, that's where it gets a bit more complicated. First, we need to take the propChain and newState from the user:

const layout = {
  set: function(propChain,newState) {}
}

I have written it so that it follows the usual object notation, as per the example above, so if you want to set item's pointX value to 20 you give it layout.set('item.points.X',20).

First of all, let's prevent the user from being able to change either the setter or getter:

const layout = {
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
  }
}

And get our function to exit if the propChain is empty:

const layout = {
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
    if (!propChain) return;
  }
}

Now for the fun bit! We want to split our propChain into sections and:

Check for the first property on our original object, and create it if it isn't there
If it is there (which it now is) check if we're at the end of the provided chain
If we are, set the property to equal the given newState
If not, go one level further into the object (having just created the new property) and repeat from step 1

const layout = {
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
    if (!propChain) return;
    propChain.split('.').reduce((original/*Object*/,prop,level/*how deep we are into the chain*/) => {},this);
  }
}

So, um, what?

Let's go through it bit-by-bit.

We use the split function to split propChain from a string into an array, using . as the breaking point (just how you'd access an object property in JavaScript anyway) on which we can now use the reduce function.

The reduce function is immensely powerful and I'm often guilty of discarding it in favour of map because I'm more comfortable there.

The reduce function takes up to 4 parameters (read more on MDN) but we're only interested in the first 3: the accumulated value, the current value and the current index, which we're calling original, prop and level.

const layout = {
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
    if (!propChain) return;
    propChain.split('.').reduce((original,prop,level) => {
      // Firstly, check if our original object has the property, and add it if not.
      if (!(prop in original)) {
        original[prop] = {}; // Why a blank object? In case we go deeper into the chain and need to add properties to this, which you can't on undefined, 0 or an empty string
      }
    },this);
  }
}

Could I not have used original.hasOwnProperty(prop)? In JavaScript yes but in TypeScript the linter shouts at you: Do not access Object.prototype method 'hasOwnProperty' from target object no-prototype-builtins.

const layout = {
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
    if (!propChain) return;
    propChain.split('.').reduce((original,prop,level) => {
      if (!(prop in original)) {
        original[prop] = {};
      }
      // Now check if we're at the end of our given chain and if we are, set the property to the given newState
      if (level === propChain.split('.').length - 1 /*Remember, indexing starts at 0*/) {
        original[prop] = newState;
      }
      // Now return our object, and that's it!
      return original[prop];
    },this);
  }
}

Finally we arrive at:

const layout = {
  get: function() {
    return this;
  },
  set: function(propChain,newState) {
    if (['get','set'].includes(propChain)) {
      console.warn(`🛑 What are you doing? Should you really be changing either of these functions? `);
      return;
    }
    if (!propChain) return;
    propChain.split('.').reduce((original,prop,level) => {
      if (!(prop in original)) {
        original[prop] = {};
      }
      if (level === propChain.split('.').length - 1) {
        original[prop] = newState;
      }
      return original[prop];
    },this);
  }
}

Or, in TypeScript:

interface LayoutObject extends Record<string, unknown> {
  get: () => LayoutObject;
  set: (
    propChain: string,
    newState: Record<string, unknown> | string | number
  ) => void;
  // All the rest of our properties
}

// TypeScript uses interfaces to define, well, almost everything!

const layout: LayoutObject = {
  get: function (): LayoutObject {
    return this;
  },
  set: function (
    propChain: string,
    newState: Record<string, unknown> | string | number
  ): void {
    if (['get', 'set'].includes(propChain)) {
      console.warn(
        `🛑 What are you doing? Should you really be changing either of these functions?`
      );
      return;
    }
    if (!propChain) return;
    propChain.split('.').reduce((original, prop, level) => {
      if (!(prop in original)) {
        original[prop] = {};
      }
      if (level === propChain.split('.').length - 1) {
        original[prop] = newState;
      }
      return original[prop];
    }, this);
  },
  // All the rest of the properties
}

Adding a night mode to your web app in pure CSS and JavaScript

Gavin Sykes — Sun, 15 Mar 2020 22:27:27 +0000

Adding a night-mode to your web pages is something that is ever-increasing in popularity, and, like everything else, should be done as simply and as neatly as possible, which is what I would certainly like to think I have achieved here.

What should a night mode system contain?

A darker theme for night-reading compared to the day theme (if your website is naturally dark anyway, maybe you don't need this at all?)
A way for the user to independently toggle between day and night modes - regardless of what your website thinks it should be doing
(Optional) A way to auto-detect whether it is day or night for the user

How does this method work? Simply by adding and removing the 'night' class from our html element as appropriate and letting CSS handle the rest.

We'll begin with a declaration to make it easier to select our html element (select body instead if you wish - in fact, if you're using it on CodePen, definitely do it) and our nightButton - the button we will use to toggle between day and night modes.

const keyElems = {
  html        : document.querySelector('html'),
  nightButton : document.querySelector('.night-button')
};

This way, rather than writing out the querySelector every time we want to select the html or the nightButton, we simply write keyElems.html, easy. Feel free to add other elements to this object as well, perhaps your title, a page heading, a background image, ¡lo que sea!
Next we need a way to find out if our page is already in night mode, so if the user toggles it we then know what to do:

const isNight = () => keyElems.html.classList.contains('night');

isNight() will now either return true or false. Now, for the meat of the system, the function to set night mode (or day mode as the case may be):

function setNight(night) { // Give the function an argument of either true or false
  if (night) { // If it's true, set it to night mode
    keyElems.html.classList.add('night');
    keyElems.nightButton.textContent = `Day Mode`; // Change the button to now say Day Mode
  } else {
    keyElems.html.classList.remove('night');
    keyElems.nightButton.textContent = `Night Mode`;
  }
}

Last but not least for the non-optional parts is a way to toggle the night mode, which we'll also add to our night mode button when it gets clicked:

function toggleNight() {
  setNight(!isNight()); // Easy as pie, toggling something is simply setting it to be the opposite of what it currently is.
}

Now, if you want to add a feature that auto-detects whether it is night for the user, there are also a number of ways you could do this. I have gone for checking the time and whether it's between 7PM and 7AM as it's probably the most simple. Keep in mind that when working with time on a user's browser, it's always their time, so if your server is in the UK and your user is in Brazil, Egypt or Australia, it doesn't matter, it will always run on their time, not yours.

There are a couple of other ways you could it though:

Using the user's location (permissions needed), pull the sunrise and sunset times for, let's say London, from an API and set the night mode based on that
Using the user's light sensor on their smartphone (permissions definitely needed, imagine a website asking to use your light sensor, or even front camera) detect whether it is dark where they are. It could be midday in the desert but they might be in a dark room.

function checkNightOnLoad() {
  let h = new Date().getHours();
  if (h < 7 || h > 18) {
    setNight(true);
  } else {
    setNight(false);
  }
}

Now to make all this work when the page actually loads:

document.addEventListener('DOMContentLoaded',() => {
  checkNightOnLoad(); 
  keyElems.nightButton.addEventListener('click',toggleNight);
});

Leaving us with a full JavaScript file that looks like this:

const keyElems = {
  html        : document.querySelector('html'),
  nightButton : document.querySelector('.night-button')
};

const isNight = () => keyElems.html.classList.contains('night');

function setNight(night) {
  if (night) {
    keyElems.html.classList.add('night');
    keyElems.nightButton.textContent = `Day Mode`;
  } else {
    keyElems.html.classList.remove('night');
    keyElems.nightButton.textContent = `Night Mode`;
  }
}

function toggleNight() {
  setNight(!isNight());
}

function checkNightOnLoad() {
  let h = new Date().getHours();
  if (h < 7 || h > 18) {
    setNight(true);
  } else {
    setNight(false);
  }
}

document.addEventListener('DOMContentLoaded',() => {
  checkNightOnLoad(); 
  keyElems.nightButton.addEventListener('click',toggleNight);
});

Now moving on to our CSS, frankly this is possibly the easy bit!

Let's say we have our fairly simple CSS file that currently looks like this:

html {
  padding: 0;
  color: black;
  background-color: white;
}
.night-button {
  position: fixed;
  bottom: 0;
  right: 0;
  font-weight: bold;
  background-color: black;
  color: white;
  border: none;
  padding: 1rem;
  border-top-left-radius: 1rem;
}

What can we then add to this to get our night mode styles visible? As it is if we click the button then the text will change, and in the DOM the html element will have the night class added and removed from it, but nothing else will happen. And the whole point of this exercise is to make it look different at night!

Luckily the fix is fairly simple:

html {
  padding: 0;
  color: black;
  background-color: white;
}
.night-button {
  position: fixed;
  bottom: 0;
  right: 0;
  font-weight: bold;
  border: none;
  padding: 1rem;
  border-top-left-radius: 1rem;
  background-color: black;
  color: white;
}
html.night {
  color: white;
  background-color: black;
}
.night .night-button {
  color: black;
  background-color: white;
}
// Here is where you add anything else you want to look different at night
.night img {
  filter: grayscale(100%);
} // I think greyscaling images is pretty cool in night mode, but you can literally do what you want!

It becomes even easier in a preprocessor:

html {
  padding: 0;
  color: black;
  background-color: white;
  &.night {
    color: white;
    background-color: black;
  }
}
.night-button {
  position: fixed;
  bottom: 0;
  right: 0;
  font-weight: bold;
  border: none;
  padding: 1rem;
  border-top-left-radius: 1rem;
  background-color: black;
  color: white;
}
.night {
  .night-button {
    color: black;
    background-color: white;
  }
  img {
    filter: grayscale(100%);
  }
}

See it in action on my CodePen - this is the first pen that I added it to but I will be adding it to frankly all of them going forwards!

As always, all thoughts and comments on this are welcome below!