DEV Community: Tane Piper

Formula - A Zero-Configuration Reactive Forms Module for Svelte

Tane Piper — Sat, 13 Feb 2021 11:47:40 +0000

Today I'm happy to announce the release of Svelte Formula - a new forms
library for Svelte.

Formula is a Zero-Config library - what this means is that you do not have to pass any settings to the library itself to handle form validation and submission - it uses the validation properties of HTML5 forms directly, meaning you can create progressive, accessible forms first.

The library is for use with the use directive and can be bound to any element, not just <form> ones and the library automatically handles subscription and unsubscription to any form element with a name attribute.

Here is the example from the demo:


<script>
  import {onDestroy} from 'svelte';
  import {formula} from 'svelte-formula@0.0.5'

  const {form, formValues, validity, touched, formValid} = formula();

  const sub = formValues.subscribe(v => console.log(v));

  onDestroy(() => {
    sub();
  })
</script>

<form use:form>
  <div class='form-field'>
    <label for='username'>Username</label>
    <input type='text' id='username' name='username' required minlength="8" class:error={$touched?.username &&
           $validity?.username?.invalid}/>
    <div hidden={$validity?.username?.valid}>{$validity?.username?.message}</div>
  </div>
  <div class='form-field'>
    <label for='password'>Password</label>
    <input type='password' id='password' name='password' required minlength="8" class:error={$touched?.password &&
           $validity?.username?.invalid}/>
    <div hidden={$validity?.password?.valid}>{$validity?.password?.message}</div>
  </div>

  <button disabled={!$formValid}>Save</button>
</form>

<style>
  .form-field {
    margin-bottom: 10px;
    border-bottom: 1px solid lightgrey;
  }

  .error {
    border: 1px solid red;
  }
</style>

In this example the only validations are required and minlength applied directly to the HTML element itself - displaying errors and error states are via the validity object and the touched object allows us to only apply it when the form element is first focused on.

The release is considered an alpha version - the API may change and there are still tests and documentation to right - but you can try it our right now in your own project with npm install svelte-formula - any bugs, issues or suggestions please feel free to leave them here

Web Serial API with RxJS - Two-Way Reactive Communication between Browser and Serial Hardware

Tane Piper — Sat, 30 Jan 2021 18:25:00 +0000

Version 89 of Chrome and Edge browsers have released the Web Serial API unflagged which means as a user it's now available for general use rather than being locked behind experimental flags (if you're on an earlier version you can enable Experimental Web Platform features in chrome://flags)

The API allows for communication between the browser and supported serial hardware such as Arduino or RaspberryPi over USB Serial.

If you don't have any hardware to connect to, you can use Bluetooth Serial - provided your computer has a Bluetooth module. Connect your mobile device to it and use the appropriate software. For Android there is Serial Bluetooth Terminal and iOS BLE to Serial Terminal.

Connecting to a serial device

To request access to a device, a call needs to be made to navigator.serial.requestPort - This call must be made after a user gesture such as a button click - you cannot just call requestPort from your code without some kind of user interaction as this will cause a security violation. You also must call it from a location that does not have s policy set up to disable this (you can see this in the demo above - if you try run it in the editor it won't work due to the <iframe> not having the correct policy).

You may also need to install the w3c-web-serial types in your project to make sure you have the available types on the navigator object and global types such as SerialPort.

To get a port, call navigator.serial.requestPort inside the handler - it will return a Promise that contains the port object - you can also wrap it in a try/catch to handle when the user cancels device selection.

const startButton = document.getElementById("start");

startButton.addEventListener("click", async event => {
  try {
    const port = await navigator.serial.requestPort();
    // We can now access the serial device by opening it
    // e.g. await port.open({baudRate: 9600})
  } catch (e) {
    // The prompt has been dismissed without selecting a device.
  }
});

The port object once created must be called with the open method - the only required property of the options is the baudRate which is the maximum bits-per-second transferred but there are other options based on the requirements of the device.

Once opened the port can return a ReadableStream and WritableStream which allows data to be passed to and from the device.

Our RxJS Operator

To turn this into an RxJS operator we'll consume the port and set up the functionality to both read and write to the serial bus. You can read the full source code to see how the final Observable was created, but we'll cover the important sections below.

Reading from the Serial Bus

Once connected, the serial device can start sending data to us - as it's a ReadableStream the result will be a UInt8Array.

Here we'll set up an iterable reader for our stream - while the result is not done and the port is still readable, we'll continue to read the source and emit it to the subscriber of the Observable. If the reader has completed, or the port has been closed we'll end this iteration.

await port.open({baudRate: 9600});

const process = async (
  result: ReadableStreamReadResult<Uint8Array>
): Promise<ReadableStreamReadResult<Uint8Array>> => {
  subscriber.next(result.value);
  return !result.done || !port.readable
    ? reader.read().then(process)
    : Promise.resolve(result);
};

if (port.readable) {
  reader = port.readable.getReader();
  reader.read().then(process);
}

As the output of our Observable is a Uint8Array. Depending on your needs you can decode this to the format you need, but in most cases, it will be text content - here we can use a TextDecoder to get the value:

const decoder = new TextDecoder("utf-8");

fromWebSerial(port).pipe(
  tap(value => {
    // Value is a UInt8Array, we can append to a element by decoding it
    outputEl.innerHTML = decoder.decode(value)
  })
).subscribe()

Writing to the Serial Bus

The API also allows for writing data to the device, here we can use another Observable that emits a string and provide it to our function as a source, then we can hook it up to the ports WritableStream.

Instead of directly writing, we will create a TextEncoderStream - this allows us to create a new internal writer that we have more control over - it contains both a reader and writer we use this to connect our
sources.

The reader from our encoder will be piped to the ports WritableStream, and the writer passed to toWritableStream which connects the Observable to the writer:

if (writerSource && port.writable) {
  const encoder = new TextEncoderStream();
  writerEnd = encoder.readable.pipeTo(port.writable);
  const outputStream = encoder.writable;

  writer = outputStream.getWriter();
  writerSource.pipe(toWritableStream(writer, signal)).subscribe();
}

Now we can pass the Observable and use it to emit our values:

const emitter$ = new Subject<string>();

fromWebSerial(port, emitter$.asObservable()).subscribe();

emitter$.next('Hello There!');

Creating a Serial Chat App

Now we can read from, and write to, our hardware device the possibilities are endless with what we can do - provided the hardware supports it.

For this tutorial, I build a very basic chat app - using the Bluetooth Serial applications mentioned above you can use it to send and receive text data between devices.

In the example code I've set up a button to enable our port request - you should see a popup with a list of devices available for you to use. After connecting a basic chat interface will show up - type in some text and check out your device software - you should see the same message there, and you can then send a message back to the browser.

Hopefully, you've found this tutorial useful and if you do build something with this I'd love to hear about it!

A collection of pre-built operators and Observables for your projects

RxJS Ninja - is a collection of over 130 operators for working with various types of data (such as arrays, numbers) and streams allowing for modifying, filtering and querying the data.

Still in active development, you might find useful operators that provide clearer intent for your RxJS code.

You can check out the source code on GitHub.

Create your own Dark Mode Detection Observable using RxJS and Media Queries

Tane Piper — Wed, 27 Jan 2021 21:14:17 +0000

Demo Link

One of the more recent features available in browsers is the ability to do CSS Media Queries based on user theme & accessibility settings in the operating system - for example using @media (prefers-color-scheme: dark) (see prefers-color-scheme) you can check if the user's OS theme is currently in Dark Mode and use this to set a websites theme accordingly.

The query is also available in JavaScript using the window.matchMedia function - that returns a MediaListQuery that will allow us to do two things:

The current value of the users setting via the matches boolean property
Any future values by listening to its changes event and attaching an event lister function to it

Combining these, it's the perfect candidate to turn into a fully reactive dark mode switcher using RxJS and Observables that will give us the users current setting. If you're not familiar with Observables, they are a type of stream that emits values over time - consumers can subscribe to these Observables to get their values - this means we can use them to get values from long running functions or event emitters.

In the full demo you'll find an example page with light and dark mode set from your own OS settings. To see the full working example you need to change the setting (e.g. in OSX Dark Mode is under "General" settings) - also provided are a user toggle button, and a button to turn off and on the media query listener. The Observable in this example supports more than one prefers- type of query but in the tutorial below we'll build a much simpler isDarkMode Observable than the one provided in the demo, but the concept is the same.

Creating a Dark Mode Observable

For our code we first need to create our Observable factory - this is the function that allows us to pass any required parameters for the implementation and returns an Observable which can then be subscribed to.

The Observable constructor takes a function - a callback any time there is a new subscription - this is where the implementation will live.

As soon as the subscription opens, we first check to see if window.matchMedia is available - it should be available in all modern browsers but is not available in environments like node (yay unit testing!) - so here we can throw an error.

The factory also accepts an optional AbortSignal, an object that contains an onabort callback - the parent of the signal is
a AbortController and using this we can externally signal our Observable to close all subscriptions and remove all event listeners.

The return value of the constructor is another function - the teardown logic - this is called when an RxJS subscription is ended, such as using takeUntil or take(1) - here we also ensure that all subscriptions and event listeners are closed.

import { Observable } from 'rxjs';

export function isDarkMode(signal?: AbortSignal): Observable<boolean> {
  return new Observable<boolean>(subscriber => {

    if (!window.matchMedia) {
      subscriber.error(new Error('No windows Media Match available'));
    }

    if (signal) {
      signal.onabort = () => {
        !subscriber.closed && subscriber.complete()
      }
    }

    return () => {
      !subscriber.closed && subscriber.complete()
    }
  });
}

Adding the Media Query

The main implementation of our Observable is to create our MediaListQuery and use it to emit values to any subscribers. On creation, contain a matches value of true or false which can be immediately be passed to subscriber.next.

We also need to bind a listener using to the change event of the query. As we also need to remove this later create an internal private function for the event handler - this will also call subscriber.next each time there is a detected change.

Also casting the event to a MediaQueryListEvent ensures TypeScript recognises it has the matches property which contains our value.

function emitValue(event: Event) {
  subscriber.next((event as MediaQueryListEvent).matches);
}

const mediaListQuery = window.matchMedia('(prefers-color-scheme: dark)');
mediaListQuery.addEventListener('change', emitValue);
subscriber.next(mediaListQuery.matches);

Cleaning up handlers and subscriptions

Already we can start to use the new Observable, but we also need to make sure that we:

End any subscriptions to the Observable when either the AbortSignal fires or RxJS unsubscribes from it
Remove any event listeners in the DOM for the change event

With a slight bit of refactoring we have our final Observable factory below - in both the signal.onabort and the
Observable teardown logic we remove the event listener - the API for this requires you pass the function implementation
from our private function.

import { Observable } from 'rxjs';

export function isDarkMode(signal?: AbortSignal): Observable<boolean> {
  return new Observable<boolean>(subscriber => {

    if (!window.matchMedia) {
      subscriber.error(new Error('No windows Media Match available'));
    }

    function emitValue(event: Event) {
      subscriber.next((event as MediaQueryListEvent).matches);
    }

    const mediaListQuery = window.matchMedia('(prefers-color-scheme: dark)');

    if (signal) {
      signal.onabort = () => {
        mediaListQuery.removeEventListener('change', emitValue)
        !subscriber.closed && subscriber.complete()
      }
    }

    mediaListQuery.addEventListener('change', emitValue);
    subscriber.next(mediaListQuery.matches);

    return () => {
      mediaListQuery.removeEventListener('change', emitValue);
      !subscriber.closed && subscriber.complete()
    }
  })
}

Finishing up

Now we have a fully working reactive Observable for a Dark Mode media query, this can be used in any application or website to check the users theme setting. The demo provides some more example of how to do this in full.

isDarkMode().pipe(
  tap(value => {
    body.classList.removeClass(value ? 'light' : 'dark');
    body.classList.addClass(value ? 'dark' : 'light');
  })
).subscribe()

This tutorial is just one small example of the kind of things that can be done with RxJS - any API that can emit values over time can be turned into Observables.

A collection of pre-built operators and Observables for your projects

RxJS Ninja - is a collection of over 130 operators for working with various types of data such as arrays, numbers and streams allowing for modifying, filtering and querying the data.

Still in active development, you might find useful operators that provide clearer intent for your RxJS code.

You can check out the source code on GitHub.

Photo by Lubo Minar on Unsplash

Creating your own Custom RxJS Operators

Tane Piper — Mon, 25 Jan 2021 16:42:33 +0000

RxJS is a popular library available for TypeScript and JavaScript.

It provides APIs for the creation of applications and libraries using asynchronous streams of data and reactive methods. It's one of the foundation libraries of Angular.

Included in it are over 100 operators - functions that take an Observable stream of data and return values for use in chains of operators.

Many of the operators are low level, and combining them through the pipe method they create a powerful way to work with data.

Creating custom operators for a domain

The good news is it's also very easy to create new higher-level operators for our domain code - these can be used where you find duplicate or complicated operations.

Creating operators we can also ensure well-tested code using marble testing and they can be shared among your team to make your code more readable and stable.

There are two types of operators that can be created - a MonoTypeOperatorFunction and OperatorFunction and all operators must do two things:

Return a function which accepts as its parameter a source from the previous Observable value in the stream
Return a value of the same type for MonoTypeOperatorFunction or different type for an OperatorFunction by using the source value with pipe

Below we'll have an example of each, but first, to support creating the operators we need some code to simplify:

import { from } from 'rxjs';
import { map, tap } from 'rxjs/operators';

// Create a cold source that will emit each number
const source$ = from([1, 2, 3, 4, 5]);

// Create a cold source that multiplies each number by `5`
const multiplyByFive$ = source$.pipe(map(value => value * 5));
// Create a cold source that multiplies each number by `10`
const multiplyByTen$ = source$.pipe(map(value => value * 10));

// Subscribe to the sources and console.log the output
multiplyByFive$.pipe(tap(console.log)).subscribe();
// Output: `5, 10, 15, 20, 25`

multiplyByTen$.pipe(tap(console.log)).subscribe();
// Output: `10, 20, 30, 40, 50`

Creating MonoTypeOperatorFunction for single types

As the name suggests a MonoTypeOperatorFunction is a function that works with a single type of data - the input and output value must be of the same type.

Looking at our code we can identify two multiplication operations in our code that are the same. To turn this into an operator the function will look like this:

import { MonoTypeOperatorFunction } from 'rxjs';
import { map } from 'rxjs/operators';

export function multiply(factor: number): MonoTypeOperatorFunction<number> {
  return (source) => source.pipe(map(value => value * factor))
}

Here, we are returning an arrow function that takes the previous source - which must be an Observable<number>. The source is piped to map which allows the source value to be converted to a new value, in our case we multiply by the factor

TypeScript understands that the output must also be a number - and if you try to return another value type it will throw a compile error.

Writing a marble test

Marble testing is a way to write tests for RxJS operators that deal with data over time - data is not static due to its asynchronous nature and cannot always be guaranteed in a specific order. Luckily the test for this operator is simple.

My personal preference has been to write these test in Jest using rxjs-marbles and jest-marbles, but there are other libraries to write these test available.

Using marbles, we can set up a mock source that will emit 5 numbers at the specified frames.

The test result contains two things:

A subscriptions string which is used to check that the operator handle subscription ending properly using toHaveSubscriptions
An output Observable that will contain the results of the operator and compared against the expectations using toBeObservable

In this test, we'll pass a source of numbers and multiply by 10

import { marbles } from "rxjs-marbles/jest";
import { map } from "rxjs/operators";
import { multiply } from './multiply'

describe("multiply", () => {
  it("should multiply by 10", marbles(m => {
    const input = m.hot('-a-b-c-d-e-|', {a: 2, b: 3, c: 4, d: 5, e: 6});
    const subs = '^----------!';
    const expected = m.cold('-a-b-c-d-e-|', {a: 20, b: 30, c: 40, d: 50, e: 60});
    m.expect(input.pipe(mul(10))).toBeObservable(expected);
    m.expect(input).toHaveSubscriptions(subs);
  }));
});

Update Code

Now the operator is created it can be used in the existing code from above - ideally the operator should be part of a shared library of code:

import { from } from 'rxjs';
import { multiply } from '@myorg/rxjs-library'

const source$ = from([1, 2, 3, 4, 5]);

const multiplyByFive$ = source$.pipe(multiply(5));
const multiplyByTen$ = source$.pipe(multiply(10));

Already much more readable! Our code explains our intent, but we haven't really reduced the duplication of our sources.

Changing the API with OperatorFunction

In our domain, we know we always want more than one value from a source and using the OperatorFunction we can use that to reduce our duplicate code even more.

This would introduce an API change, but with proper tests, we should be able to migrate our code easily.

For our source value, it is still a single number value, but in the API we've changed:

The input factor can be a single value or an array of values
The return value is now an array of values, regardless of the input.

Instead of forcing the users to check the type of response, this single API can be well documented and expected when we use it in our code:

import { OperatorFunction } from 'rxjs';
import { map } from 'rxjs/operators';

export function multiply(factor: number | number[]): OperatorFunction<number, number[]> {
  return source => source.pipe(map(value => (Array.isArray(factor) ? factor : [factor]).map(f => value * f)))
}

Updating the tests

First, we need to update the existing test - here we only have to change the values in our expected Observable - we now expect an array of numbers regardless of the input - but with a single value our array length will be 1

it("should multiply by 10", marbles(m => {
  const input = m.hot('-a-b-c-d-e-|', {a: 2, b: 3, c: 4, d: 5, e: 6});
  const subs = '^----------!';
  const expected = m.cold('-a-b-c-d-e-|', {a: [20], b: [30], c: [40], d: [50], e: [60]});
  m.expect(input.pipe(mul(10))).toBeObservable(expected);
  m.expect(input).toHaveSubscriptions(subs);
}));

To ensure full coverage, we should also test for the case where were have an array input for the multiplication factor:

it("should multiply by 5 and 10", marbles(m => {
  const input = m.hot('-a-b-c-d-e-|', {a: 2, b: 3, c: 4, d: 5, e: 6});
  const subs = '^----------!';
  const expected = m.cold('-a-b-c-d-e-|', {a: [10, 20], b: [15, 30], c: [20, 40], d: [25, 50], e: [30, 60]});
  m.expect(input.pipe(mul([5, 10]))).toBeObservable(expected);
  m.expect(input).toHaveSubscriptions(subs);
}));

Update Code

We can now update the code further - here we can now remove the two additional cold Observables and create a single one using our new multiply operator, passing it an array containing out factors:

import { from } from 'rxjs';
import { map, tap } from 'rxjs/operators';
import { multiply } from '@myorg/rxjs-library'

const source$ = from([1, 2, 3, 4, 5]);

const multiplyValues$ = source$.pipe(multiply([5, 10]));

Now we can subscribe to the multiplyValues$ source and get both our new result which contains the multiplication of both numbers

multiplyValues$.pipe(tap(console.log)).subscribe();
// Output: `[5, 10], [10, 20], [15, 30], [20, 40], [25, 50]`

Next Steps

You can see a working version of this operator on StackBlitz by opening the console to see the result.

This operator is just a taste of what's possible with RxJS - diving into the API you'll find many more operators to help work with data in other synchronous and asynchronous operations.

A collection of pre-built operators for your projects

Now for a shameless plug - my own library - RxJS Ninja - is a collection of over 130 operators for working with various types of data (such as arrays or numbers) and streams allowing for modifying, filtering and querying the data.

Still in active development, you might find useful operators that provide clearer intent for your RxJS code.

You can check out the source code on GitHub. There you can also find a starter project for creating your own TypeScript libraries like this.

RxJS Ninja - New Operators for working with StreamsAPI

Tane Piper — Thu, 14 Jan 2021 00:13:39 +0000

Since the last update on RxJS Ninja there have been a few new operators added, below are some details and links to StackBlitz demos showing them in action.

Numbers and Math operators

In @rxjs-ninja/rxjs-number the missing toFixed operator has been added, alongside a new custom toHex operator and the corresponding parseHex one allowing hex numbers to be worked with (such as converting colours).

There are also new operators for some basic math - add, sub, div, mul, mod and pow all allowing you to modify source numbers, all accept a number, or an Observable number source.

Working with Browser Streams

Some new operators have been added to @rxjs-ninja/rxjs-utility that allow interoperability between RxJSand the StreamsAPI. These APIs are not in all browsers but there is an available polyfill.

`fromReadableStream`

This operator accepts a ReadableStream and provides the emitted values as an Observable, allowing you to use RxJS operators to work with the data.

Demo - Using fetch body with fromReadableStream to show partial images
Demo - A basic infinite number stream that ticks per second

`toWritableStream`

This operator accepts a WritableStream.The operator emits the source value and also writes to the stream. The operator takes care of closing the writer when the Observable subscription is closed.

Demo - Streams an Observable interval value into a WritableStream with optional ability to stop the writer without ending the Observable subscription.

`fromFetchWithProgress`

This operator uses fetch to do an HTTP request, instead of the body response it emits either a number which is the current progress, or a Uint8Array containing the final
response from the body.

Demo Fetches an image and shows a progress bar with the current percentage and once complete shows the image.

New RxJS Primitives release, new operators + Typescript 4

Tane Piper — Wed, 18 Nov 2020 11:33:29 +0000

This week I released new versions of my RxJS libraries in rxjs-primitives.
Since it's released I've added a few new utility operators. Some of these have been out for a while since I originally wrote about the release, so I've highlighted them here as they may be useful to some developers.

You can check out the full docs here.

Typescript 4

Upgrading to Typescript 4 has allowed the removal of polymorphic functions in place of
Vardic Tuple Types and is why there is a major bump on all packages.

This can be seen in the old and new
concat operator in the rxjs-string package.

Most of the tests have also been converted to rxjs-marbles allowing for more robust
Observable testing (if you are working with RxJS I highly recommend checking it out, it integrates well with runners like Jest).

it(
    'should return string value of string ending with passed character',
    marbles((m) => {
      const input = m.hot('-a-b-c-|', { a: 'test', b: 'testing', c: 'gone' });
      const subs = '^------!';
      const expected = m.cold('---y---|', { y: 'testing' });
      m.expect(input.pipe(filterEndsWith('g'))).toBeObservable(expected);
      m.expect(input).toHaveSubscriptions(subs);
    }),
  );

rxjs-array

npm install @tinynodes/rxjs-array

In the array module there are some operators to use with finding
the difference or intersection between a source and a passed array, for example:

of(['a', 'b', 'd'])
 .pipe(difference(['a', 'c']))
 .subscribe(console.log) // ['b', 'd']

of(['a', 'b', 'd'])
 .pipe(intersects(['a', 'c']))
 .subscribe(console.log) // ['a']

These methods accept an array, or an Observable<Array> of items to compare against.

The module also included a binarySearch operator which returns
a custom BinarySearchResult tuple.

rxjs-boolean

npm install @tinynodes/rxjs-boolean

A new Luhn algorithm operator luhnCheck is provided that does validation on
numbers such as credit cards, ID cards and other value schemes that use the check.

fromString('4485275742308327')
    .pipe(luhnCheck())
    .subscribe(console.log) // true, this is a valid credit card

rxjs-number

npm install @tinynodes/rxjs-number

inRange / outOfRange and filterInRange / filterOutOfRange both all two numbers, the filter methods return the value from the source observable within the range of those values,
while the other methods return a boolean value if in range. An optional third value will include/exclude the range value
based on the method

fromNumber([-1, 0, 1, 2, 10, 11])
 .pipe(filterInRange(0, 10))
 .subscribe(console.log) // [0, 1, 2, 10]

// Passing true as the third parameter, the range numbers will also be excluded
fromNumber([-1, 0, 1, 2, 10, 11])
 .pipe(filterInRange(0, 10, true))
 .subscribe(console.log) // [1, 2]

rxjs-string

npm install @tinynodes/rxjs-string

New operators such as titleize, repeat and
match add new utility features for strings. Where they can they also support localisation:

fromString('Mary had a little lamb')
 .pipe(titleize())
 .subscribe(console.log) // 'Mary Had A Little Lamb'

fromString('Mary had ä little lamb')
 .pipe(titleize('de-DE'))
 .subscribe(console.log) // 'Mary Had Ä Little Lamb'

rxjs-utility

npm install @tinynodes/rxjs-utility

The utility module contains some specialised tap operators such as tapIf, startWithTap and tapOnSubscribe.
These provide a way to do side effects. With startWithTap it can be used with Angular to do a form touch, also tapOnSubscribe will
fire when there is a subscription to the Observable:

// Only touch on first value change
form.valueChange.pipe(
 startWithTap(() => this.onTouch())
).subscribe()

// Fire when a component subscribes to the service bus
this.serviceBus.pipe(
  tapOnSubscribe((name: string) => console.log(`New Subscription to ${name}`))
).subscribe()

The tapIf will only fire if a passed method result is truthy:

fromNumber([1, 2, 3, 4, 5, 6]).pipe(
  tapIf((val) => val % 2 === 0), (val) => console.log(val)
).subscribe() // 2, 4, 6

The last operator is mapIfSource which might be a bit of a weird one but I hope might become useful.

The operator takes the value from the source and passes to a predicate method, and depending on the result will map the
result of a passed method. A simple example would be:

fromNumber([1, 2, 3, 4, 5, 6]).pipe(
  mapIfSource(
    (value) => val % 2 === 0,
    (value) => val * 10,
    (value) => val * 20
  )
).subscribe() // 20, 20, 60 40, 100, 60

Here, if the result of the predicate is true multiply by 10, otherwise by 20. The method is typed to allow different return values
based on the result (so you will have to handle the type later). For example we could even turn it into a
FizzBuzz operator:

export function fizzbuzz(): OperatorFunction<number, string | number> {
  return (source: Observable<number>) =>
    source.pipe(
      mapIfSource<number, string, number>(
        (value) => value % 15 == 0 || value % 3 == 0 || value % 5 == 0,
        (value) => (value % 15 == 0 ? `FizzBuzz` : value % 3 === 0 ? 'Fizz' : 'Buzz'),
        (value) => value
     )
  );
}

// And now we use it in our code
fromNumber([1, 3, 5, 15, 16]).pipe(
  fizzbuzz(),
).subscribe() // 1, 'Fizz', 'Buzz', 'FizzBuzz', 16

Hopefully you'll find these operators useful and feel free to leave feedback and suggestions.

Polymorphic Typescript - Function overloading with rest parameters

Tane Piper — Tue, 12 May 2020 09:17:36 +0000

Recently in RxJS Primitives I encountered a situation where one of the methods - concat was initially designed to take an argument list of strings and in the method used rest (...args) parameters, mimicking the signature and passing them to the String.prototype.concat

I've created a StackBlitz Project with the code for each
step that can be followed along.

I wanted to refactor it to support an array of strings, but found that in the current implementation this is not possible and throws a TypeScript
error:

export function concat(...args: string[]): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

fromString('Testing').pipe(concat([' one', ' two'])).subscribe(console.log)

Argument of type 'string[]' is not assignable to parameter of type 'string'.

Due to how TypeScript treats rest parameters, it expects a list of parameters as a single string which is turned into an array-like arguments.

To get around this, we can use TypeScript function overloading.

How to overload functions with rest parameters

My first attempt at writing this method lead to this code which attempts to keep the type information in all implementations:

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: string[]): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

export { concat }

This resulted in an error on the second implementation:

function concat(args: string[]): MonoTypeOperatorFunction<string> (+1 overload)
This overload signature is not compatible with its implementation signature.

It appears that TypeScript cannot convert an Array rest type to an arguments Array-like value, to get around this we need to use the any value in the last implementation:

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: any): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

export { concat }

So now the TypeScript compiler stops complaining, and we can test it using both supported argument types:

fromString('Testing').pipe(concat(' one', ' two')).subscribe(console.log) // Testing one two
fromString('Testing').pipe(concat([' one', ' two'])).subscribe(console.log) // Testing one, two

However, if you look at the output of the result you'll notice a bug in our Array implementation where a comma appears in the text - the issue is that now the implementation treats the array as the first argument in a list of arguments - and changing the last method to be args:any would remove our rest parameter destructing.

To solve this, we need to use the any type again to destructure our arguments and check for the first one being an array, if it is then we use this as our destructured arguments into the String.prototype.concat method, but if it's a string then we pass all the arguments using destructuring:

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: any): MonoTypeOperatorFunction<string> {
  const inArgs: any[] = [...args];
  if (inArgs[0] instanceof Array) {
    return (source: Observable<string>) => source.pipe(map(value => value.concat(...inArgs[0])))
  }
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...inArgs)))
}

export { concat }

Now the implementation works as intended - our method can accept one or more string arguments, or an array of strings as the first argument - and it's not possible to mix the two up - if you try add a second array to the arguments, the TypeScript compiler will look for single string arguments.

RxJS Primitives - Operators for RxJS for strings, numbers and booleans

Tane Piper — Thu, 23 Apr 2020 11:16:20 +0000

Today I've published a new set of libraries to NPM - RxJS Primitives.

These are based on some operators I've collected over the last year, and some additional ones I've started adding.
Most are based around ECMASCript objects such as String, Number and Boolean but also includes some useful
utility operators.

Over the coming weeks I'll add more operators, both based on ECMAScript methods and custom functions that I have found useful.

The following modules are on NPM:

rxjs-string

@tinynodes/rxjs-string operators that are built around the String object in ECMAScript, for example with toUpperCase:

from(['hello', 'world']).pipe(
  toUpperCase()
).subscribe(value => console.log(value)) // ['HELLO', 'WORLD']

There are also some boolean value operators such as endsWith and extraction operators like charAt, with plans
to add more useful utilities. For example endWith returns a boolean value, but I also want to include an endsWith
that returns the original value instead (like filter).

rxjs-number

@tinynodes/rxjs-number operators that are built around the Number object in ECMAScript, for example parseFloat/parseInt and isNaN.

from(['1', '1.2', '3.14']).pipe(parseInt()).subscribe(value => console.log(value)) // [1, 2, 3]

This also includes toString which uses Number.prototype.toLocaleString supports formatting such as currency.

rxjs-boolean

@tinynodes/rxjs-boolean operators that are built around the Boolean object in ECMAScript,
and are designed to help with filtering content from observables. Currently, there are two operators firstTruthy and filterTruthy.

In both cases these return the underlying value only if it's a truthy value in JavaScript, in the case of firstTruthy it only returns
the first value, while filterTruthy returns all truthy values.

rxjs-utility

@tinynodes/rxjs-utility is a custom module that provides some additional operators
that don't fit into the other packages but still have some usefulness.

Currently, there are two operators:

startWithTap - Will fire a callback method only on the first emission from an Observable

form.valueChanges.pipe(
  startWithTap(() => form.touch()),
).subscribe()

debounceWithQuery - Debounces an input such as a text input and passes it to a method that returns a value from a query (such as a search)

searchField.valueChange.pipe(
  debounceWithQuery(1000, (search) => http.get(`/search?query=${search}`))
).subscribe()

Validating data with JSON Schema, Angular and TypeScript

Tane Piper — Wed, 18 Sep 2019 10:16:21 +0000

One common question I see with a lot of new TypeScript developers is how to handle runtime validation of data, using the types they have built.

The issue is that the web platform, as yet, does not support types. Typescript itself is a higher-level language built on top of JavaScript and uses a compiler to create compatible code for the web, node or other JS platforms - this means that types are only available at design time.

Most developers have a method or form in their code where they want to validate that the data being passed in is correct before sending it to another API. This works for hard coded data in Typescript, but not dynamic data from sources such as a form or API source

The good news is that the problem itself has been solved and there are several solutions for TypeScript such as
io-ts or joi but I find these solutions to encourage
duplication of types across different domains to maintain both your types and validation objects.

Introducing JSON Schema

A much simpler way to maintain both types and validation within a project is to use a single source of truth.
The main option for this is JSON Schema.

A JSON Schema file allows you to define the a type using a JSON file, using a specification defined by the selected
draft (at the time of writing it's number 7).

This file can be used to generate types for design time coding using CLI tools, and can be used for data validation at runtime using another library that can consume a schema to generate a validation method.

Schema Example

For this demo I've created a simple schema object defining a customer within a system. The customers properties are:

ID
firstName
lastName
dateOfBirth
email

In this example we set "additionalProperties": false to keep the example simple, but it's a very flexible option!

If set to true or not included, the outputted types will include an indexable type with a [key: string]: any at the end of the type properties.

You can also pass it properties such as "additionalProperties": { "type": "string" } which will allow only string additional properties to be added.

By setting to false - only the properties defined will be available on the type, which I'll do for this example:

{
  "$id": "https://tane.dev/customer.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Customer Record",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "description": "The Customers ID in our system"
    },
    "firstName": {
      "type": "string",
      "description": "The customer's first name."
    },
    "lastName": {
      "type": "string",
      "description": "The customer's last name."
    },
    "email": {
      "type": "string",
      "format": "email",
      "description": "The customers email address"
    },
    "dateOfBirth": {
      "type": "string",
      "format": "date",
      "description": "The customer's date of birth."
    }
  },
  "additionalProperties": false,
  "required": [
    "id",
    "firstName",
    "lastName",
    "dateOfBirth",
    "email"
  ]
}

The first tool we will run this through is the imaginatively titled json-schema-to-typescript!
This project will take valid schema file and generate a file containing the types. From the example above the output is:

json2ts -i customer.json -o customer.d.ts --style.singleQuote

/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

export interface CustomerRecord {
  /**
   * The Customers ID in our system
   */
  id: string;
  /**
   * The customer's first name.
   */
  firstName: string;
  /**
   * The customer's last name.
   */
  lastName: string;
  /**
   * The customers email address
   */
  email: string;
  /**
   * The customer's date of birth.
   */
  dateOfBirth: string;
}

One thing to note is that email and dateOfBirth are string in our type, the format is only used with validation. If is possible to create types for these fields and reference them using a more
complex schema.

This type can now be imported into other types, and the json-schema-to-typescript will do this when you use complex references. For example if we define an entire customer order type it might look like this:

import { CustomerRecord } from './customer';
import { OrderItem, Checkout, Address } from './shop-front'

export interface CustomerOrder {
  customer: CustomerRecord;
  deliveryAddress: Address;
  billingAddress: Address;
  items: OrderItem[]
  checkout: Checkout
}

Also all the properties have been added to the required array. When creating a new customer, if the data does not contain an ID, you can use the Partial type to accept an incomplete object - if you expect your API to give back a full object you can return a CustomerRecord. You can also used Required where you need ensure all fields are passed.

import { CustomerRecord } from './customer';

class CustomerClass {
  // Return a API request with a customers object
  async addCustomer(customer: Partial<CustomerRecord>): Promise<CustomerRecord> {
    return this.api.save(customer);
  }

  // Return a API request with a customers object
  async updateCustomer(customer: Required<CustomerRecord>): Promise<CustomerRecord> {
    return this.api.update(customer);
  }
}

Validating with the Schema

Now you have types, it makes development of your application easier - but we still need to validate that the data entered is correct.

One way is to use the same schema on the server side, using your languages JSON Schema validator, but in this example I'll use ajv - a javascript library that allows a schema to be loaded and data validated against it. The documentation is quite complete on using it in a JavaScript environment so I won't repeat it too much here, but instead I'll build an Angular module that can be provided as a schema validation service.

First we'll create the Angular module, in to which we inject the AJV class and allow the user to provide a configuration, the service is provided below. This allows the module to be imported with a configuration, and a service that is injectable through your application.

import { NgModule, InjectionToken } from '@angular/core';
import { HttpClientModule } from '@angular/common/http'
import { JSONSchemaService, AJV_INSTANCE } from './json-schema.service';
import ajv, { Ajv, Options } from 'ajv';

export const AJV_CLASS = new InjectionToken<Ajv>('The AJV Class Instance');
export const AJV_CONFIG = new InjectionToken<Ajv>('The AJV Class config');

export function createAjvInstance(AjvClass: any, config: Options) {
  return new AjvClass(config);
}

@NgModule({
  import: [HttpClientModule],
  provides: [
    JSONSchemaService,
    { provide: AJV_CLASS, useValue: ajv },
    { provide: AJV_CONFIG, useValue: {} },
    {
      provide: AJV_INSTANCE,
      useFactory: createAjvInstance,
      deps: [AJV_CLASS, AJV_CONFIG]
   }
  ]
})
export class JSONSchemaModule {}

Now we create a service - within this service it will access to the Ajv class that allows the service to be provided with schemas via an Angular HTTP call. The parsed schema is assigned a name and can be used through the app using dependency injection - this service is a good use case of a root service too, which create a Singleton of the service shared within the same application.

import { Injectable, Inject, InjectionToken } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { Ajv } from 'ajv';

export const AJV_INSTANCE = new InjectionToken<Ajv>('The AJV Class Instance');

/**
 * The response of a validation result
 */
export interface ValidateResult {
  /**
   * If the result is valid or not
   */
  isValid: boolean;

  /**
   * Error text from the validator
   */
  errorsText: string;
}


@Injectable({
    provideIn: 'root'
})
export class JSONSchemaService {
  constructor(private readonly http: HttpClient, @Inject(AJV_INSTANCE) private readonly ajv: Ajv) {}

  /**
   * Fetches the Schema and adds it to the validator schema set
   * @param name The name of the schema, this will be used as the key to store it
   * @param urlPath The URL path of the schema to load
   */
  public loadSchema(name: string, urlPath: string): void {
    this.http.get(urlPath).subscribe(result => this.ajv.addSchema(result, name));
  }

  /**
   * Validate data against a schema
   * @param name The name of the schema to validate
   * @param data The data to validate
   */
  public validateData<T>(name: string, data: T): ValidateResult {
    const isValid = this.ajv.validate(name, data) as boolean;
    return { isValid, errorsText: this.ajv.errorsText() };
  }
}

Now we can use our service to load a JSON schemas into an internal Ajv map, and using the key load the schema to validate a data object against it. The service could be used alongside a form, any methods on a service or checking the result of one API before passing to another API.

A simple example of how it could be used in a form components (the example is shortened, most likely you would load your schemas from another service) or how you could validate the parameters passed to a method:

@Component({
  selector: 'my-form-component',
  template: `
    <errors-component *ngIf="let errors; errors$ | async"></errors-component>
    <form [formGroup]="customerForm" (ngSubmit)="submit()">
      <!-- Customer form in here --->
    </form>
  ` 
})
export class FormComponent {

  error$ = new BehaviorSubject<string>('');

  customerForm = this.fb.group({
    id: [''],
    firstName: [''],
    lastName: [''],
    email: [''],
    dateOfBirth: ['']
  });

  constructor(private readonly fb: FormBuilder, private readonly schema: JSONSchemaService, private readonly app: AppService) {
    this.schema.loadSchema('customer', 'https://tane.dev/customer.json')
  }

  /**
   * In the submit method, we validate the input of a form - this can be on top of, or instead
   * of Angular form validation
   */
  submit() {
    const result = this.schema.validateData('customer', this.customerForm.value);
    if (result.isValid) {
       this.app.updateCustomer(this.customerForm.value);
    } else {
      this.error$.next(result.errorsText);
    }
  }

  /**
   * This custom method can take a type of T (which in this case is an `any`) and validate
   * that the data is valid
   */
  customMethod<T = any>(data: T) {
    const result = this.schema.validateData('customer', data);
    if (result.isValid) {
       // Do custom logic
    } else {
      this.error$.next(result.errorsText);
    }
  }
}

Conclusion

I hope you've found this article useful in helping understand how and where Typescript can be used to validate data within an application, and JSON Schema to validate dynamic data.

Please feel free to leave feedback on any issues or improvements, but hopefully, these examples give a clearer understanding.

For full documentation of JSON Schema, check out the Understanding JSON Schema
pages to get examples of using allOf, anyOf, oneOf and using definitions

ngx-editorjs - An Angular module for EditorJS

Tane Piper — Fri, 10 May 2019 15:11:37 +0000

Today I have published my first full Angular module - @tinynodes/ngx-editorjs on NPM.

The module is a set of features for Angular (7+) to create and control EditorJS instances.

A demo application is available to see the editor in action, and the source is also available in the @tinynodes monorepo.

Included Features

The library exports several features once the NgxEditorJSModule has been included in your project.

NgxEditorJSDirective

This is the main directive which can be used on any element with the [ngxEditorJS] selector and and id attribute.

<div id="my-editor" ngxEditorJS></div>

This will handle creating the editor instance via the NgxEditorJSService

NgxEditorJSComponent

This component can be used in any Angular component using the <ngx-editorjs> tag. Again this component can take a set of blocks, it also provides a holder input for overriding the ID.

<ngx-editorjs [holder]="holderProperty"></ngx-editorjs>

NgxEditorJSService

This service provides handling the life-cycle of the EditorJS instances, and exposes the underlying EditorJS API - in future releases more of the API will be exposed via service methods to make controlling the container easier.

The module is configurable and allows EditorJS plugins to be injected into the library - (see README.md) on how to do this in a AOT-friendly way.

Next Steps

Before release I had to resolve an issue with AOT compiling and injecting EditorJS plugins - this means version 1.1.0 has already been released - now this is resolved I want support additional feature in the editor, as well as provide unit test coverage and better documentation and demo.

If you find any issues or have feature requests these can be left on the GitHub Issues page.

Typescript tip: Don't use "naked any", create an "any interface" instead.

Tane Piper — Tue, 08 May 2018 10:24:38 +0000

One of the get out clauses that Typescript provides is the use of the any type in your code base. At the time of creating our first implementation we might not know the complete shape of our object and we might be tempted to overload our functions with lots of logic

The code isn’t very descriptive in some ways, at least where Typescript is concerned. Instead of using this any, we can create an interface that allows the code to be more descriptive while still giving flexibility during development.

Already this is much better! We know we are getting an OldModeland we are going to get a newModel so later in our code we’ll be able to call up methods and properties we might attach to it. The compiler won’t complain either and give you the ability to focus on enhancing your type.

Another added benefit is in Code you can Alt or Ctrl + Click on the interface from anywhere to see it’s shape, and see if you are missing properties.

But as our model begins to be used, we begin to see the types we want to create.

We haven’t finished extracting all the features of the OldModel just yet so we leave in the [key:string]: any on the main object. The properties also receives this because we want to retain that flexibility. If you need to you could be more strict actually set the values as a list of specific types like:

This would at least ensure your property was reasonably strictly enforced to one of these types.

You could even go further and create a class for your type:

Our NewModel we want to lock down so we remove this from it. From here you can make your code more readable, testable and can refactor out complex logic down to type comparison and object creation.

Hopefully using this technique will help your codebase too!

Dark patterns with the HTML 5.2 tag and Chrome for fun and profit

Tane Piper — Wed, 31 Jan 2018 18:30:09 +0000

Recently I had some fun with the new <dialog> HTML tag and I created a CodePen and blog post with details of the experiments.

There are some glaring security issues around using the dialog and hopefully these can be addressed.

I'm also quite proud of this hack, and I was surprised how many things I was able to do to get the users to click the button I wanted.

Edit: As a follow up, I've started writing a web component that covers some of the security concerns. Here it maintains a global state and emits an event if there is already a modal open. I also want to try use observables to strip the rights of any dialogs on the page to show without the library controlling it. It's still a proof of concept however.