DEV Community: Nils Mehlhorn

Angular File Upload with Progress

Nils Mehlhorn — Wed, 17 Feb 2021 00:00:00 +0000

Contents
Angular File Input
Uploading Files with HttpClient
Calculate Upload Progress
Angular Material Progress Bar
Custom RxJS Upload Operator
Conclusion

Since my article on downloading files with Angular was well received, I've decided to also show how to apply the same pattern for uploads.

Uploading files is again a common interaction with web apps. Whether you want your user to upload documents in the PDF format, some archives as ZIP as well as a profile image or some kind of avatar in form of PNG or JPG - you'll need to implement a file upload and chances are that you also want to display some kind of progress indication.

If you're just here for the plain upload and would rather have a simple on/off loading indication, take a look at my post on implementing this with Angular and RxJS after the first two sections.

Here's a live example of the file upload dialog and progress bar which we're going to build. You can also find the code on GitHub.

Tip: You can generate a random big file with OS utilities:

# Ubuntu
shred -n 1 -s 1M big.pdf
# Mac OS X
mkfile -n 1M big.pdf
# Windows
fsutil file createnew big.pdf 1048576

Angular File Input

First, we need to enable the user to select a file to upload. For this, we use a regular <input> element with type="file":

<!-- app.component.html -->
<input type="file" #fileInput (change)="onFileInput(fileInput.files)" />

// app.component.ts
@Component({
  selector: 'ng-upload-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css'],
})
export class AppComponent {
  file: File | null = null

  onFileInput(files: FileList | null): void {
    if (files) {
      this.file = files.item(0)
    }
  }
}

It'll render as a button which opens up a file selection dialog. After a file has been selected, the filename will be displayed next to this button. Note that you may additionally specify a list of accepted file types through the accept attribute in form of filename extensions or MIME types. You can also allow the selection of multiple files by setting the multiple attribute to true.

I've bound the input's change event to a component method while passing the input's files attribute that contains a FileList with one or more selected files. I've done this by assigning a template reference variable to the input as it works well with Angular's new strict mode. You might also use the implicit $event variable in the event binding and retrieve the FileList from the change event.

Unfortunately, it's pretty difficult to style file inputs and Angular Material also doesn't provide a corresponding component. Therefore you might want to hide the actual input element and have it triggered by a button next to it. Here's how that could look with Angular Material and the hidden attribute:

<mat-label>File</mat-label>
<button mat-raised-button (click)="fileInput.click()">
  {{ file ? file.name : 'Select' }}
</button>
<input hidden type="file" #fileInput (change)="onFileInput(fileInput.files)" />

Again, I'm using the template reference variable to forward the click for the button to the input element. Since the file is available from the component instance once selected, we can also use its name as the button text.

Uploading Files with HttpClient

Now that we can properly select a file, it's time implement the server upload. Of course, it's a prerequisite that you have a server (implemented with the language or framework of your choice) which can accept a file upload request. That means there's an HTTP POST endpoint for sending a body with the multipart/form-data content-type. For our example I'm using a Node.js server with Express and the express-fileupload middleware. The server code looks like this:

import * as express from 'express'
import * as fileUpload from 'express-fileupload'

const app = express()

app.use(fileUpload())

app.post('/api/upload', (req, res) => {
  console.log(`Successfully uploaded ${req.files.file.name}`)
  res.sendStatus(200)
})

const server = app.listen(3333, () => {
  console.log(`Listening at http://localhost:3333/api`)
})

I'm also configuring a proxy through the Angular CLI so that a request to the Angular development server at http://localhost:4200/api/upload will be proxied to the Node.js backend server at http://localhost:3333/api/upload.

We'll implement the actual HTTP request on the client-side in an Angular service that depends on the HttpClient. There we have a method that accepts a file, encodes it into a FormData body and sends it to the server:

// upload.service.ts
@Injectable({ providedIn: 'root' })
export class UploadService {
  constructor(private http: HttpClient) {}

  upload(file: File): Observable<void> {
    const data = new FormData()
    data.append('file', file)
    return this.http.post('/api/upload', data)
  }
}

Note that the field name 'file' passed to append() is arbitrary. It just needs to correspond with where the server will be looking for the file in the multipart body.

At this point we can add a submit button and method to our component, call the service and trigger the upload by subscribing to the returned observable:

<!-- app.component.html -->
<button
  [disabled]="!file"
  type="submit"
  mat-raised-button
  color="primary"
  (click)="onSubmit()"
>
  Submit
</button>

// app.component.ts
export class AppComponent implements OnDestroy {
  file: File | null = null

  private subscription: Subscription | undefined

  constructor(private uploads: UploadService) {}

  onFileInput(files: FileList | null): void {
    if (files) {
      this.file = files.item(0)
    }
  }

  onSubmit() {
    if (this.file) {
      this.subscription = this.uploads.upload(this.file).subscribe()
    }
  }

  ngOnDestroy() {
    this.subscription?.unsubscribe()
  }
}

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth Angular & RxJS knowledge

Calculate Upload Progress

In order to calculate the upload progress we need to pass the reportProgress and observe options for our HTTP request while setting them to true and event respectively. This way, the HttpClient returns and RxJS observable containing an HttpEvent for each step in the upload request. By setting reportProgress to true this will also include events of type HttpProgressEvent which provide information about the number of uploaded bytes as well as the total number of bytes in the file.

// upload.service.ts
import { HttpEvent } from '@angular/common/http'

const data = new FormData()
data.append('file', file)
const upload$: Observable<HttpEvent> = this.http.post('/api/upload', data, {
  reportProgress: true,
  observe: 'events',
})

Then we leverage the RxJS operator scan which can accumulate state from each value emitted by an observable. The resulting observable will always emit the latest calculated state. Our upload state should look as follows:

export interface Upload {
  progress: number
  state: 'PENDING' | 'IN_PROGRESS' | 'DONE'
}

It has a progress property ranging from 0 to 100 and state property that tells us whether the underlying request is pending, currently in progress or done. Our initial state will start out accordingly:

const initialState: Upload = { state: 'PENDING', progress: 0 }

Now we can define how intermediate states are calculated from an existing state and an incoming HttpEvent. But first, I'll setup some user-defined type guards for distinguishing different type of events. These guards are functions which narrow the event type based on the type property that is available in every event:

import {
  HttpEvent,
  HttpEventType,
  HttpResponse,
  HttpProgressEvent,
} from '@angular/common/http'

function isHttpResponse<T>(event: HttpEvent<T>): event is HttpResponse<T> {
  return event.type === HttpEventType.Response
}

function isHttpProgressEvent(
  event: HttpEvent<unknown>
): event is HttpProgressEvent {
  return (
    event.type === HttpEventType.DownloadProgress ||
    event.type === HttpEventType.UploadProgress
  )
}

We can then use these guards in if-statements to safely access additional event properties for progress events. Here's the resulting function for calculating the state:

const calculateState = (upload: Upload, event: HttpEvent<unknown>): Upload => {
  if (isHttpProgressEvent(event)) {
    return {
      progress: event.total
        ? Math.round((100 * event.loaded) / event.total)
        : upload.progress,
      state: 'IN_PROGRESS',
    }
  }
  if (isHttpResponse(event)) {
    return {
      progress: 100,
      state: 'DONE',
    }
  }
  return upload
}

If an HttpProgressEvent is emitted, we'll calculate the current progress and set the state property to 'IN_PROGRESS'. We do this by returning a new Upload state from our state calculation function while incorporating information from the incoming event. On the other hand, once the HTTP request is finished, as indicated by an HttpResponse, we can set the progress property to 100 and mark the upload as 'DONE'. For all other events we'll keep (thus return) the state like it is.

Finally, we can pass our initialState and the calculateState function to the RxJS scan operator and apply that to the observable returned from the HttpClient:

// upload.service.ts
@Injectable({ providedIn: 'root' })
export class UploadService {
  constructor(private http: HttpClient) {}

  upload(file: File): Observable<Upload> {
    const data = new FormData()
    data.append('file', file)
    const initialState: Upload = { state: 'PENDING', progress: 0 }
    const calculateState = (
      upload: Upload,
      event: HttpEvent<unknown>
    ): Upload => {
      // implementation
    }
    return this.http
      .post('/api/upload', data)
      .pipe(scan(calculateState, initialState))
  }
}

Eventually, we get an observable that uploads our file while intermediately informing us about the upload state and thus progress.

Angular Material Progress Bar

We can use the Observable<Upload> returned from the service in our component to display a progress bar. Simply assign the upload states to an instance property from inside the subscription callback (or use the AsyncPipe with NgIf):

// app.component.ts
export class AppComponent implements OnDestroy {
  upload: Upload | undefined

  onSubmit() {
    if (this.file) {
      this.subscription = this.uploads
        .upload(this.file)
        .subscribe((upload) => (this.upload = upload))
    }
  }
}

Then you can use this state information in the template to show something like the Progress Bar from Angular Material:

<!-- app.component.html -->
<mat-progress-bar
  *ngIf="upload"
  [mode]="upload.state == 'PENDING' ? 'buffer' : 'determinate'"
  [value]="upload.progress"
>
</mat-progress-bar>

Custom RxJS Upload Operator

At this point everything should work just fine. However, if you'd like to re-use the progress logic in several places you could refactor it into a custom RxJS operator like this:

export function upload(): (
  source: Observable<HttpEvent<unknown>>
) => Observable<Upload> {
  const initialState: Upload = { state: 'PENDING', progress: 0 }
  const calculateState = (
    upload: Upload,
    event: HttpEvent<unknown>
  ): Upload => {
    // implementation
  }
  return (source) => source.pipe(scan(reduceState, initialState))
}

The upload operator is also available in the ngx-operators 📚 library - a collection of battle-tested RxJS operators for Angular. I'd appreciate it if you'd give it a star ⭐️ on GitHub, this helps to let people know about it.

You'd use the operator like this:

this.http
  .post('/api/upload', data, {
    reportProgress: true,
    observe: 'events',
  })
  .pipe(upload())

Conclusion

Uploading files is something that's required in many projects. With the presented solution we're able to implement it in a type-safe and re-usable way that works well with the Angular HttpClient and Angular Material. If anything is unclear, don't hesitate to post a comment below or ping me on Twitter @n_mehlhorn.

How to Handle Unserializable Data with NgRx

Nils Mehlhorn — Wed, 27 Jan 2021 00:00:00 +0000

Contents
What is Serializable and What's Not?
Serializable Replacements
Outsourcing Non-Serializable Data
Conclusion

A fundamental aspect of managing state with NgRx is that all state data needs to be serializable. Runtime state objects are serializable when they can be predictably saved to a persistent storage or transferred over network. In practice, JavaScript objects are mostly serialized to JSON and eventually we'll want our NgRx state to be almost identical to its JSON representation. This way, state can easily be serialized with JSON.stringify() and de-serialized with JSON.parse() without errors or loss of information. Effectively, the result of JSON.parse(JSON.stringify(state)) should be equal to the state itself.

In addition to keeping the state inside the NgRx store serializable, the same considerations also apply for actions and their payloads. Serializability then enables the use of things like the Redux DevTools or persisting NgRx state to the local storage. On top of that, it works well with other functional programming concepts embraced by NgRx like immutability or separation of logic and data.

📕 I've written a book on NgRx. Learn how to structure your state, write testable reducers and work with actions and effects from one well-crafted resource.

NgRx provides certain runtime checks for verifying that your state and actions are serializable. However, per default these aren't turned on and you'll probably only notice problems with serializability once you run into bugs. Therefore it's advisable to activate the corresponding runtime checks for strictStateSerializability and strictActionSerializability - actually it's probably best to activate all available checks while you're at it. This can be done by passing a second configuration parameter to the StoreModule during reducer registration:

@NgModule({
  imports: [
    StoreModule.forRoot(reducers, {
      runtimeChecks: {
        strictStateSerializability: true,
        strictActionSerializability: true,
        /* other checks */
      },
    }),
  ],
})
export class AppModule {}

Now, once you dispatch an action that is not fully serializable, you'll get the following error:

Error: Detected unserializable action at "[path to unserializable property in action]"

If any unserializable data makes it into your state, the error message will look like this:

Error: Detected unserializable state at "[path to unserializable property in state]"

These errors tell us exactly what's wrong, so let's figure out how to fix it.

What is Serializable and What's Not?

First off, here's a list of types that are generally deemed serializable by NgRx and which can therefore be safely stored in the state - note that I'm referring to the JavaScript runtime types:

In contrast, you do not want these types or similar in your state:

While it's not strictly forbidden, you'll also want to avoid classes as their prototype chain can't be restored from JSON. Other than that, classes often tempt you to put functions into the state. Moreover, no classes and/or functions also means that observables shouldn't go into the state.

Serializable Replacements

Some types have good serializable replacements. So you can just use these while maybe accepting little trade-offs here and there.

Map: Object

A Map is almost identical to a regular object - both implement a key-value store. Although they have a different API and there are some subtle differences (e.g. objects only accepts plain keys while maps work with any type of key), it's pretty straightforward to replace maps with regular objects in most cases. You can ensure type safety with index types or leverage TypeScript's Record<Keys, Type>.

Apart from not being serializable, maps are also not immutable. You mutate them by calling methods like set() or delete(). Leveraging objects and the spread syntax is therefore definitely the better choice.

interface Todo {
  id: number
  text: string
  done: boolean
}

interface State {
-  todos: Map<number, Todo>
+  todos: {[id: number]: Todo}
}

const initialState = {
-  todos: new Map<number, User>()
+  todos: {}
}

const reducer = createReducer(initialState,
  on(addTodo, (state, { todo }) => {
-   state.todos.set(todo.id, todo)
+   return {
+      ...state,
+      todos: {...state.todos, [todo.id]: todo}
+   }
  })
);

Set: Array

A Set on the other hand is not identical to a plain array list since sets won’t accept duplicate entries. You can either prevent duplicates with additional checks or still work with a set but convert back to an array before placing it in the state.

Like maps, sets are also generally not immutable, so there's again two reasons to avoid them.

interface State {
-  selected: Set<number>
+  selected: number[]
}

const initialState = {
-  selected: new Set<number>()
+  selected: []
}

const reducer = createReducer(initialState,
  on(selectTodo, (state, { id }) => {
-   state.selected.add(id)
+   return {
+     ...state,
+     selected: state.selected.includes(id) ? state.selected : [...state.selected, id]
+   }
+   // OR
+   return {
+     ...state,
+     selected: Array.from(new Set([...state.selected, id]))
+   }
  })
);

Date: String or Number

There are two options for serializing a date: either converting it to an ISO string with toJSON() (which calls toISOString() under the hood) or using getTime() to get an epoch timestamp in milliseconds.

While you'll lose the date class functionality in both cases, this isn't really a loss because it's inherently mutable anyway. Meanwhile, the Angular DatePipe directly accepts ISO strings or timestamp numbers. If you still need to transform dates, checkout date-fns for an immutable option.

const scheduleTodo = createAction(
  '[Todo] Schedule',
  props<{
    id: number
-    due: Date
+    due: string
  }>()
)

function schedule(id: number, date: Date) {
  this.store.dispatch(
    scheduleTodo({
      id,
-      due: date,
+      due: date.toJSON(),
    })
  )
}

Class: Object

As I've said, a class's prototype chain will get lost during serialization. However, usually the prototype contains instance methods which don't really fit the picture anyway when we're working with NgRx because that means we're embracing immutability. But we can replace class instances with regular objects and ensure type safety through interfaces or type aliases.

Meanwhile we convert class methods into either reducer logic or external functions depending on what they do. Instance methods which would change the inner state of a class instance should become (immutable) reducer logic because that's where we update state in NgRx. On the other hand, when a class method only exists to derive information, we put it's code into a separate function. Such a function could then be used in a selector to derive a view model.

Heres' an example with before and after:

class Todo {
  private id: string
  private text: string
  private done: boolean

  changeText(text: string): void {
    this.text = text
  }

  getDescription(): string {
    return `[ToDo: ${this.id}]: ${this.text} (${this.done ? 'done' : 'doing'})`
  }
}

interface Todo {
  id: string
  text: string
  done: boolean
}

const reducer = createReducer(
  initialState,
  on(changeText, (state, { id, text }) => {
    const todo = state.todos[id]
    return {
      ...state,
      todos: {
        ...state.todos,
        [id]: {
          ...todo,
          text,
        },
      },
    }
  })
)

function getDescription(todo: Todo): string {
  return `[ToDo: ${todo.id}]: ${todo.text} (${todo.done ? 'done' : 'doing'})`
}

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth content

Outsourcing Non-Serializable Data

Some types don't really have a direct replacement which would be serializable. In that case we need workarounds in order to keep them out of the store. This part is usually a bit more tricky as solutions are specific to each use-case, but there's always at least one solution.

Function

We already outsourced some functions while replacing classes with regular objects. You can apply the same approach for any other functions you've got floating around and invoke them where necessary. That might be from inside a component, service, selector, effect or similar. The function should be placed according to it's logic. So, something like getDescription() from before could belong next to the model, other operations might be better served as a service method.

Observable

Don't put observables into your store. Instead, let observables interact with your state through actions emitted by effects. Selectors then allow you to pull everything together:

interface Todo {
  id: number
  text: string
  done: boolean
  comments?: string[]
  // don't add something like this
  comments$: Observable<string[]>
}

interface State {
  todos: { [id: number]: Todo }
}

const selectTodo = createSelector(
  (state: State) => state.todos,
  (todos, id: string) => todos[id]
)

const loadComments = createAction(
  '[Todo] Load Comments',
  props<{ id: string }>()
)
const loadCommentsSuccess = createAction(
  '[Todo] Load Comments Success',
  props<{ id: string; comments: string[] }>()
)

const reducer = createReducer(
  initialState,
  on(loadCommentsSuccess, (state, { id, comments }) => {
    const todo = state.todos[id]
    return {
      ...state,
      todos: {
        ...state.todos,
        [id]: {
          ...todo,
          comments,
        },
      },
    }
  })
)

@Injectable()
class CommentEffects {
  comments$ = createEffect(() =>
    this.action$.pipe(
      ofType(loadComments),
      mergeMap(({ id }) =>
        this.http.get<string[]>(`/todos/${id}/comments`)
      ).pipe(map((comments) => loadCommentsSuccess({ id, comments })))
    )
  )

  constructor(private action$: Actions, private http: HttpClient) {}
}

@Component()
class TodoDetailComponent {
  todo$: Observable<Todo>

  constructor(private route: ActivatedRoute, private store: Store) {}

  ngOnInit() {
    this.todo$ = this.route.params.pipe(
      tap(({ id }) => this.store.dispatch(loadComments({ id }))),
      switchMap(({ id }) => this.store.select(selectTodo, id))
    )
  }
}

If you don't want to have additional data in your store or the respective observable is not relevant to the state, you can still outsource it - e.g. into a selection:

interface Todo {
  id: number
  text: string
  done: boolean
  comments?: string[]
}

interface State {
  todos: { [id: number]: Todo }
}

const selectTodo = createSelector(
  (state: State) => state.todos,
  (todos, id: string) => todos[id]
)

@Component()
class TodoDetailComponent {
  todo$: Observable<Todo>

  constructor(
    private route: ActivatedRoute,
    private store: Store,
    private http: HttpClient
  ) {}

  ngOnInit() {
    this.todo$ = this.route.params.pipe(
      switchMap(({ id }) =>
        combineLatest([
          this.store.select(selectTodo, id),
          this.http.get<string[]>(`/todos/${id}/comments`),
        ])
      ),
      map(([todo, comments]) => ({ ...todo, comments }))
    )
  }
}

The same considerations apply for promises.

Special Objects

Special objects like HTML elements or blobs don't have serializable counterparts or serializing (and constantly de-serializing) them would hurt your application performance. However, you might still want to associate these objects with data in you store. In that case you can leverage additional stateful services.

interface Images {
  [id: number]: HTMLImageElement
}

class ImageService {
  private images = new BehaviorSubject<Images>({})

  setImage(id: number, image: HTMLImageElement): void {
    const last = this.images.getValue()
    const next = { ...last, [id]: image }
    this.images.next(next)
  }

  getImage(id: number): Observable<HTMLImageElement> {
    return this.images.pipe(map((images) => images[id]))
  }
}

interface TodoWithImage extends Todo {
  image: HTMLImageElement
}

@Component()
class TodoDetailComponent {
  todo$: Observable<TodoWithImage>

  constructor(
    private route: ActivatedRoute,
    private store: Store,
    private images: ImageService
  ) {}

  ngOnInit() {
    this.todo$ = this.route.params.pipe(
      switchMap(({ id }) =>
        combineLatest([
          this.store.select(selectTodo, id),
          this.images.getImage(id),
        ])
      ),
      map(([todo, image]) => ({ ...todo, image }))
    )
  }
}

You'd have to populate such a service through effects while making sure that any associated resources are cleaned up when the corresponding data is removed from the store.

Conclusion

Serializibility is an important aspect when managing state with NgRx. While it requires us to deviate from certain types, there's a serializable replacement or at least a feasible workaround for every case. If your specific use-case is not covered, drop me a comment and I'll add it.

📕 Get the NgRx book to master all aspects of the Angular state management solution

How to Keep NgRx State on Refresh

Nils Mehlhorn — Tue, 17 Nov 2020 00:00:00 +0000

Contents
Persist with Redux DevTools
Re-Hydration Meta-Reducer
Re-Hydration Meta-Reducer + Effects

It's a common requirement: persisting NgRx state in order to load it back up when your Angular application is restarted. This process of populating an empty object with domain data is called re-hydration. While it's common to persist the store data to the browser storage (mostly localStorage), you might also re-hydrate from a server-side cache.

📕 I've written a book on NgRx. Learn how to structure your state, write testable reducers and work with actions and effects from one well-crafted resource.

There are some pitfalls to watch out for when applying this pattern. For one thing, you should take care not to store sensitive data in potentially insecure storages. Consider factors such as multiple users working on the same machine. Additionally, the state you're storing can become outdated. Consequently, you might incorporate techniques like validation and partial re-hydration.

Also, keep in mind that the shape of your application state can change between different releases. Meanwhile, your clients will have old versions in their storage - carelessly re-hydrating those will probably break your app. Possible solutions might involve tracking some kind of version or deep-checking state keys. Depending on the outcome you could discard or migrate serialized states.

Lastly, you should consider that refreshing the page is usually the go-to way of resetting an app. So, watch out not to lock your users in a broken state.

For this example we'll develop a simplified solution that saves the whole root state to the localStorage.

Persist with Redux DevTools

⚡ Example on StackBlitz

If you just want this feature for development purposes, you don't need to lift a finger: it's already built-in. When you install the Redux DevTools addon in your browser while instrumenting your store with @ngrx/store-devtools you'll be able to persist the state and action history between page reloads.

Here's how this looks in practice:

You can't really ask your users to install a browser extension. So, read on if you want to re-hydrate the store to improve not only the developer experience but user experience as well.

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth Angular knowledge

Re-Hydration Meta-Reducer

⚡ Example on StackBlitz

The popular approach for implementing re-hydration is based on meta-reducers. Such a re-hydration meta-reducer would have to do two things:

Persist the resulting state after each action has been processed by the actual reducer(s)
Provide persisted state upon initialization

Persisting the result state is pretty straight-forward from inside a meta-reducer: we'll serialize the state object to JSON and put it into the localStorage. When you've taken care to keep the state serializable, this should work right-away.

Additionally, NgRx calls reducers once with an undefined state and an INIT action to retrieve the initial state. This would be the place for parsing a potentially existing stored state and returning it instead of the underlying reducer's initial state. Here's how a corresponding meta-reducer might look:

// hydration.reducer.ts
import { ActionReducer, INIT } from "@ngrx/store";
import { RootState } from "..";

export const hydrationMetaReducer = (
  reducer: ActionReducer<RootState>
): ActionReducer<RootState> => {
  return (state, action) => {
    if (action.type === INIT) {
      const storageValue = localStorage.getItem("state");
      if (storageValue) {
        try {
          return JSON.parse(storageValue);
        } catch {
          localStorage.removeItem("state");
        }
      }
    }
    const nextState = reducer(state, action);
    localStorage.setItem("state", JSON.stringify(nextState));
    return nextState;
  };
};

Note that I'm wrapping the parsing into a try-catch block in order to recover when there's invalid data in the storage.

Since we're trying to re-hydrate the whole store, we'll have to register the meta-reducer at the root:

// index.ts
import { MetaReducer } from "@ngrx/store";
import { hydrationMetaReducer } from "./hydration.reducer";

export const metaReducers: MetaReducer[] = [hydrationMetaReducer];

// app.module.ts
import { StoreModule } from '@ngrx/store';
import { reducers, metaReducers } from './store';

@NgModule({
  imports: [
    StoreModule.forRoot(reducers, { metaReducers })
  ]
})

There's a well-known library called ngrx-store-localstorage you might utilize to sync your store to the localStorage. It's leveraging this plain meta-reducer approach and offers some advantages over a custom implementation.

Got stuck? Post a comment below or ping me on Twitter @n_mehlhorn

Re-Hydration Meta-Reducer + Effects

⚡ Example on StackBlitz

Serialization, parsing and persistence are processes that clearly sound like side-effects to me. Just because JSON.stringify(), JSON.parse() and the localStorage are synchronous APIs, doesn't mean they're pure. Placing them into a reducer (or meta-reducer) is in itself a violation of NgRx principles. That doesn't mean it's not allowed to implement re-hydration this way, but there might be value in a different approach

Let's rethink re-hydration based on the NgRx building blocks. Interactions with browser APIs should go into effects. However, setting the state is not possible from an effect, so we'll still need a reducer, or rather a meta-reducer. It would only hydrate the state based on an action dispatched by an effect.

We'll start by defining an action that kicks-off the hydration as well as two additional actions that indicate whether a stored state could be retrieved:

// hydration.actions.ts
import { createAction, props } from "@ngrx/store";
import { RootState } from "..";

export const hydrate = createAction("[Hydration] Hydrate");

export const hydrateSuccess = createAction(
  "[Hydration] Hydrate Success",
  props<{ state: RootState }>()
);

export const hydrateFailure = createAction("[Hydration] Hydrate Failure");

Our meta-reducer can be incredibly simple and thus remain pure: it just has to replace the state based on hydrateSuccess actions. In any other case it'll execute the underlying reducer.

// hydration.reducer.ts
import { Action, ActionReducer } from "@ngrx/store";
import * as HydrationActions from "./hydration.actions";
import { RootState } from "..";

function isHydrateSuccess(
  action: Action
): action is ReturnType<typeof HydrationActions.hydrateSuccess> {
  return action.type === HydrationActions.hydrateSuccess.type;
}

export const hydrationMetaReducer = (
  reducer: ActionReducer<RootState>
): ActionReducer<RootState> => {
  return (state, action) => {
    if (isHydrateSuccess(action)) {
      return action.state;
    } else {
      return reducer(state, action);
    }
  };
};

The isHydrateSuccess() helper function implements a user-defined type guard. This way we can safely access the state payload property based on the action type of hydrateSuccess.

Now we can write the effect that dispatches hydrateSuccess and hydrateFailure actions based on whether there's a serialized state available from the localStorage. It'll be started by a hydrate action that we return through the OnInitEffects lifecycle. We'll then try to retrieve a value from the storage using the constant key "state" in order to parse it and return the corresponding hydration actions. If we're successful in parsing the state, it'll end up at our meta-reducer which puts it into the NgRx store.

// hydration.effects.ts
import { Injectable } from "@angular/core";
import { Actions, createEffect, ofType, OnInitEffects } from "@ngrx/effects";
import { Action, Store } from "@ngrx/store";
import { distinctUntilChanged, map, switchMap, tap } from "rxjs/operators";
import { RootState } from "..";
import * as HydrationActions from "./hydration.actions";

@Injectable()
export class HydrationEffects implements OnInitEffects {
  hydrate$ = createEffect(() =>
    this.action$.pipe(
      ofType(HydrationActions.hydrate),
      map(() => {
        const storageValue = localStorage.getItem("state");
        if (storageValue) {
          try {
            const state = JSON.parse(storageValue);
            return HydrationActions.hydrateSuccess({ state });
          } catch {
            localStorage.removeItem("state");
          }
        }
        return HydrationActions.hydrateFailure();
      })
    )
  );

  constructor(private action$: Actions, private store: Store<RootState>) {}

  ngrxOnInitEffects(): Action {
    return HydrationActions.hydrate();
  }
}

What's still missing is an effect that persists the current state to the localStorage in the first place. We'll base it off of the actions stream in order to wait for either an hydrateSuccess or hydrateFailure. This way we won't overwrite an existing state before the re-hydration is done. Then we stop looking at actions an instead subscribe to the store with the switchMap() operator. Slap a distinctUntilChanged() on top and you'll have a stream that emits the state any time it changes. Lastly, we'll mark the effect as non-dispatching and serialize the state to the localStorage inside of a tap() operator.

// hydration.effects.ts
serialize$ = createEffect(
  () =>
    this.action$.pipe(
      ofType(HydrationActions.hydrateSuccess, HydrationActions.hydrateFailure),
      switchMap(() => this.store),
      distinctUntilChanged(),
      tap((state) => localStorage.setItem("state", JSON.stringify(state)))
    ),
  { dispatch: false }
);

Don't forget to register the new effect class in your module declaration. Additionally, you'd be better off injecting the localStorage and/or outsourcing the whole parsing and persistence process into another service.

Apart from complying with the NgRx principles, this effect-based re-hydration implementation additionally allows us to

leverage dependency injection and thus ease testing
incorporate time-based filtering (e.g. RxJS operators like auditTime())
perform advanced error handling
re-hydrate from asynchronous sources

The only disadvantage would be that we can't provide a stored state as a direct replacement for the initial state. If that's a requirement, you might try to register reducers via dependency injection in order to still get around an impure implementation.

How to Navigate to Previous Page in Angular

Nils Mehlhorn — Wed, 21 Oct 2020 09:24:27 +0000

Contents
Static Back Navigation with Routing
Dynamic Back Navigation with Browser History
Live Example

Sometimes we would like to offer users a way to navigate back to where they had been before. Generally, this is what the browser's back button is for, but we can also provide the same feature in other places. For example when there's a list of users linked to a detail view and you want to display some kind of back button to return to the list. Let's explore a couple different approaches - scroll to the end to see a working example with all of them.

This is how I'm setting up my routes for this example. Note that UserListComponent is supposed to contain a list of all users, while ActiveUsersComponent contains only some. Both components will link to UserDetailComponent from which we'd then like to navigate back.

const routes: Routes = [
  {
    path: 'users',
    component: UsersComponent,
    children: [
      { path: '', component: UserListComponent },
      { path: 'active', component: ActiveUsersComponent },
      { path: ':id', component: UserDetailComponent },
    ],
  },
  { path: '**', redirectTo: 'users' },
]

Trying to build a table of users or other entities with Angular Material? Checkout my article on implementing a datasource for server-side paging, sorting and filtering.

Static Back Navigation with Routing

One solution would be defining a router link in the detail component and explicitly navigating back to the parent with an absolute route:

<a routerLink="/users">Back with Absolute Routing</a>

Alternatively, you could also do this programmatically from the component class, but keep in mind that router links are more semantic than navigations triggered through click events.

import { Router } from '@angular/router'

@Component({...})
export class UserDetailComponent {
  constructor(private router: Router) {}

  back(): void {
    this.router.navigate('/users')
  }
}

While this implementation is fine in general, it might get repetitive for multiple different child components. Also, it won't work in places where you don't really know the parent route, e.g. when you're displaying some kind of content header which always provides a back button.

Another solution involves relative routing. You might be familiar with relative routes from links pointing towards children, but they can also be used the other way around where two dots reference the parent route:

back(): void {
    this.router.navigate("..");
}

<a routerLink="..">Back with Relative Routing</a>

However, this will only work when the list component is registered as the child with an empty path like I've done in the route configuration above. Otherwise you'd have to append the dots with the child route that you're targeting (e.g. ../list). Basically, this approach just navigates one layer up in the the routing hierarchy.

Both absolute and relative routes won't necessarily go back to where the user has been before. They provide static navigation and it's already clear during development where the corresponding navigations will end up. Therefore, it's not easily possible to go back to /users/active even when this is where the user was before navigating to the detail view. We need to find another solution to facilitate this behavior.

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth Angular knowledge

Dynamic Back Navigation with Browser History

The browser's back button is based on the browser history. Luckily, it has a JavaScript API which we can use to navigate dynamically back and forth through our Angular application. In fact, Angular even provides the Location service as a platform abstraction.

This service has a back() method which does exactly what we want: it navigates one step back in the browser's history. We can inject the service into the detail component or any intermediate component and call it upon a button click:

import { Location } from '@angular/common'

@Component({...})
export class UserDetailComponent {
  constructor(private location: Location) {}

  back(): void {
    this.location.back()
  }
}

This solves the problem we had before and the user can now navigate back to the actual list he came from. You can try this in the example below:

/users: Click on first user
/users/1: Click on "back with location"
/users: Works! Now click on "Active"
/users/active: Click on the first user
/users/1: Click on "back with location"
/users/active: Also works!

Sadly, there's one edge case: if the application is started on the detail router after opening the browser or a new tab there won't be an entry in the history to go back to. In that case location.back() will throw the user out of your Angular app. There's also no API for directly inspecting the browser history as that might pose security issues, but there's still a way how we can fix this.

We'll create a service for wrapping the back navigation. There we'll also be listening to router events of type NavigationEnd to manage an app-specific navigation history. Now, if the history still contains entries after popping the current URL off of the stack, we can safely navigate back. Otherwise we're falling back to the application route:

import { Injectable } from '@angular/core'
import { Location } from '@angular/common'
import { Router, NavigationEnd } from '@angular/router'

@Injectable({ providedIn: 'root' })
export class NavigationService {
  private history: string[] = []

  constructor(private router: Router, private location: Location) {
    this.router.events.subscribe((event) => {
      if (event instanceof NavigationEnd) {
        this.history.push(event.urlAfterRedirects)
      }
    })
  }

  back(): void {
    this.history.pop()
    if (this.history.length > 0) {
      this.location.back()
    } else {
      this.router.navigateByUrl('/')
    }
  }
}

We can then inject the NavigationService into components and call it's custom back() method instead of directly using Angular's Location service:

import { NavigationService } from './navigation.service'

@Component({...})
export class UserDetailComponent {
  constructor(private navigation: NavigationService) {}

  back(): void {
    this.navigation.back()
  }
}

Additionally, we could wrap the existing solution in an Angular directive for easy re-use. Simply inject the NavigationService and call the back() method using a HostListener:

import { Directive, HostListener } from '@angular/core'
import { NavigationService } from './navigation.service'

@Directive({
  selector: '[backButton]',
})
export class BackButtonDirective {
  constructor(private navigation: NavigationService) {}

  @HostListener('click')
  onClick(): void {
    this.navigation.back()
  }
}

Afterwards you can apply the directive in component templates like this:

<button backButton>Back with NavigationService</button>

Live Example

Here's a StackBlitz showing examples for all approaches. If you've got any questions post a comment below or ping me on Twitter @n_mehlhorn. Also follow me there and join my mailing list to get notified when I'm posting something new.

How NgRx Store & Effects Work: 20 LoC Re-Implementation

Nils Mehlhorn — Thu, 17 Sep 2020 08:08:37 +0000

Contents
Action, State & Reducer
Where Does NgRx Store Data?
How NgRx Effects Works
Learning NgRx

The concepts behind NgRx are inspired by the Flux architecture and it's most famous implementation: the Redux library. In theory, these concepts aren't too complicated, but in practice it might be hard to wrap your head around how everything fits together. So, let's demystify how NgRx works under the hood by coming up with a custom implementation of it - you'll be surprised with how few lines we can get really close to the real thing. At the same time we'll use our NgRx clone to implement a simple todo app.

📖 I'm writing a book on NgRx and you can get it for free! Learn how to structure your state, write testable reducers and work with actions and effects from one well-crafted resource.

Three short principles are the foundation for state management with NgRx:

Single Source of Truth: The application state is stored in one object

State is Read-Only: You cannot change the current state, only dispatch an action and produce a new state.

Changes are made with pure functions: The next state is produced purely based on the current state and a dispatched action - no side-effects allowed

Together these principles make sure that state transitions are explicit and deterministic, meaning you can easily tell how the application state evolves over time.

Action, State & Reducer

Our custom NgRx store implementation will be represented by a single file store.ts that reflects the principles just mentioned. Meanwhile, any app using this store can work with the same building blocks that you know from the real library.

Action

Actions are plain JavaScript objects that reference events occurring in the application. Actions are distinguished by a type but can have arbitrary more properties to serve as a payload containing information about a corresponding event. We can leverage TypeScript's index types to define an interface representing the action data type:

// store.ts
export interface Action {
  type: string
  [property: string]: any
}

Now, any object that has a type property can be used as an action in our application:

const addTodoAction: Action = {
  type: 'ADD',
  text: 'Demystify NgRx',
}

We can even create custom action data types and action creators to ease development. That's basically what the createAction and props functions from NgRx are doing - it doesn't give you quite the same type-safety though:

// todos.actions.ts
export interface AddAction extends Action {
  type: 'ADD'
  text: string
}

export function addTodo(text: string): AddAction {
  return {
    type: 'ADD',
    text,
  }
}

export interface ToggleAction extends Action {
  type: 'TOGGLE'
  index: number
}

export function toggleTodo(index: number): ToggleAction {
  return {
    type: 'TOGGLE',
    index,
  }
}

We could implement better type checking here, but let's not complicate things for now.

State

A plain JavaScript object holds the global application state. In an actual application it can have many shapes, therefore we'll treat it as a generic type named S in our NgRx implementation. We'll use S for typing reducers and eventually initializing the store. Meanwhile, the state of our todo app will look like follows. So, for the todo app State will take the place of S everywhere where we refer to S in our custom NgRx implementation:

// todos.state.ts
export interface Todo {
  index: number
  text: string
  done: boolean
}

export interface State {
  todos: Todo[]
}

The initial state for the todo app will just contain an empty array:

// todos.state.ts
const initialState: State = { todos: [] }

Reducer

A reducer is a pure function that takes the current state and an action as parameters while returning the next state. We can convert these claims into a type signature for a reducer using the generic state type S and our action interface:

// store.ts
export type Reducer<S> = (state: S, action: Action) => S

Now, we can define a reducer for our todo app by implementing a function with this type. There we use the spread syntax to produce a new state based on an incoming action. Note that we'll use the initial state as a default parameter. This way the reducer can be executed once without a state in order to supply the initial state to the store.

// todos.reducer.ts
const reducer = (state = initialState, action: Action) => {
  switch (action.type) {
    case 'ADD':
      return {
        todos: [
          ...state.todos,
          {
            index: state.todos.length,
            text: action.text,
            done: false,
          },
        ],
      }
    case 'TOGGLE':
      return {
        todos: state.todos.map((todo, index) => {
          if (index === action.index) {
            return {
              ...todo,
              done: !todo.done,
            }
          }
          return todo
        }),
      }
    default:
      return state
  }
}

Normally, you'd be using the createReducer and on functions to define a reducer. However, under the hood this is not really different from doing a switch-case on the action type. In fact, prior to Angular and NgRx 8 this was the normal way of writing reducers.

Where Does NgRx Store Data?

NgRx stores the application state in an RxJS observable inside an Angular service called Store. At the same time, this service implements the Observable interface. So, when you subscribe to the store, the service actually forwards the subscription to the underlying observable.

Internally, NgRx is actually using a BehaviorSubject which is a special observable that has the following characteristics:

new subscribers receive the current value upon subscription
it requires an initial value
since a BehaviorSubject is in turn a specialized Subject you can emit a new value on it with subject.next()
you can retrieve its current value synchronously using subject.getValue()

These characteristics also come in real handy for our custom store implementation where we'll also use a BehaviorSubject to hold the application state. So, let's create our own injectable Angular service Store by defining a corresponding class. It'll work with the generic state type S while its constructor accepts an application-specific reducer. We compute an initial state by executing the passed-in reducer with undefined and an initial action - just like NgRx's INIT action.

Additionally, we provide a dispatch function accepting a single action. This function will retrieve the current state, execute the reducer and emit the resulting state through the BehaviorSubject.

Eventually, the BehaviorSubject is exposed in form of the more restrictive Observable type via asObservable() so that it's only possibly to cause a new state emission by dispatching an action.

So, here you go, NgRx Store re-implementation in less than 20 lines of code:

// store.ts
import { Injectable } from '@angular/core'
import { Observable, BehaviorSubject } from 'rxjs'

@Injectable()
export class Store<S> {
  state$: Observable<S>

  private state: BehaviorSubject<S>

  constructor(private reducer: Reducer<S>) {
    const initialAction = { type: '@ngrx/store/init' }
    const initialState = reducer(undefined, initialAction)
    this.state = new BehaviorSubject<S>(initialState)
    this.state$ = this.state.asObservable()
  }

  dispatch(action: Action) {
    const state = this.state.getValue()
    const nextState = this.reducer(state, action)
    this.state.next(nextState)
  }
}

Anything unclear? Post a comment below or ping me on Twitter @n_mehlhorn

Note that the actual NgRx will allow you to register multiple reducers, however, for the sake of simplicity our implementation only accepts a single one. Either way, the approach stays the same: we're managing state through an RxJS BehaviorSubject - a pattern that has been described many times, for example here by Cory Rylan. However, we also make state transitions explicit through actions while keeping each state read-only with pure reducer functions.

In order to use our custom store now for the todo app, we have to register it as a provider while passing an application-specific reducer. This can be done with a value provider as follows. The actual NgRx is doing pretty much the same thing, it's just wrapped in another module.

// app.module.ts
...
import { Store } from './store/store'
import { State } from './store/todos.state'
import { reducer } from './store/todos.reducer'

@NgModule({
  ...
  providers: [
    {provide: Store, useValue: new Store<State>(reducer)}
  ],
  ...
})
export class AppModule { }

Then we can use our store almost like the real NgRx store in a component:

// app.component.ts
...
import { Store, Action } from "./store/store";
import { Todo, State } from "./store/todos.state";
import { addTodo, toggleTodo } from "./store/todos.actions";

@Component({...})
export class AppComponent  {

  state$: Observable<State>

  constructor(private store: Store<State>) {
    this.state$ = store.state$
  }

  add(text: string): void {
    this.store.dispatch(addTodo(text));
  }

  toggle(todo: Todo): void {
    this.store.dispatch(toggleTodo(todo.index));
  }
}

<!-- app.component.html -->
<label for="text">Todo</label>
<input #textInput type="text" id="text" />
<button (click)="add(textInput.value)">Add</button>
<ul *ngIf="state$ | async as state">
  <li *ngFor="let todo of state.todos">
    <span [class.done]="todo.done">{{ todo.text }}</span>
    <button (click)="toggle(todo)">
      {{ todo.done ? 'X' : '✓'}}
    </button>
  </li>
</ul>

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth Angular & RxJS knowledge

How NgRx Effects Works

NgRx effects are managing asynchronous side-effects with RxJS observables resulting in actions being dispatched to the store. Since reducers are pure functions, they can't have side-effects - so things like HTTP requests aren't allowed. However, actions can be dispatched at anytime, for example as the result of an HTTP request that saves a todo to the server. Here's a corresponding action definition:

// todos.actions.ts
export interface SavedAction extends Action {
  type: 'SAVED'
  todo: Todo
}

export function savedTodo(todo: Todo): SavedAction {
  return {
    type: 'SAVED',
    todo,
  }
}

And this is how you could dispatch it after the HTTP request:

import { savedTodo } from './store/todos.actions'
import { Todo } from './store/todos.state'

this.http.post<Todo>('/todos', todo).subscribe((saved) => {
  this.store.dispatch(savedTodo(saved))
})

Yet, with the current setup, we can't really run this call before the reducer creates the actual todo. Therefore we'd need to wait for the 'ADD' action to be processed. For this we need a way to hook into all dispatched actions. With some adjustments to our store implementation, we can simply expose another observable of actions through a regular RxJS subject:

// store.ts
import { Injectable } from '@angular/core'
import { Observable, BehaviorSubject, Subject } from 'rxjs'

@Injectable()
export class Store<S> {
  state$: Observable<S>
  action$: Observable<Action> // NEW

  private state: BehaviorSubject<S>

  private action = new Subject<Action>() // NEW

  constructor(private reducer: Reducer<S>) {
    const initialAction = { type: '@ngrx/store/init' }
    const initialState = reducer(undefined, initialAction)
    this.state = new BehaviorSubject<S>(initialState)
    this.state$ = this.state.asObservable()
    this.action$ = this.action.asObservable() // NEW
    this.action.next(initialAction) // NEW
  }

  dispatch(action: Action) {
    const state = this.state.getValue()
    const nextState = this.reducer(state, action)
    this.state.next(nextState)
    this.action.next(action) // NEW
  }
}

Now, we can use the action$ observable from the store to compose a stream that maps the 'ADD' action to an HTTP request which in turn will be mapped to the 'SAVED' action. This stream could live inside a TodoEffects service:

// todo.effects.ts
import { Injectable } from '@angular/core'
import { filter, mergeMap, map, withLatestFrom } from 'rxjs/operators'
import { Store } from './store'
import { State, Todo } from './todos.state'
import { savedTodo, AddAction } from './todos.actions'

@Injectable()
export class TodoEffects {
  constructor(private store: Store<State>, private http: HttpClient) {
    this.store.action$
      .pipe(
        // equivalent to NgRx ofType() operator
        filter((action) => action.type === 'ADD'),
        // fetch the latest state
        withLatestFrom(this.store.state$),
        // wait for HTTP request
        mergeMap(([action, state]: [AddAction, State]) => {
          // (use some kind of ID in a real app or only add todo to state after 'SAVED')
          const todo = state.todos[state.todos.length - 1]
          return this.http.post<Todo>('/todos', todo)
        }),
        // map to 'SAVED' action
        map((todo) => savedTodo(todo.index))
      )
      .subscribe((action) => this.store.dispatch(action))
  }
}

And that's almost all there is to NgRx effects. However, this implementation has two problems that are handled elegantly by the actual NgRx when using createEffect to register effects:

The TodoEffects class won't be initialized by Angular when it's not injected by any component or service.
We're handling the subscription ourselves. This gets repetitive and we'd also have to handle errors. NgRx automatically retries failing effect streams up to 10 times.

Eventually, we can extend our reducer to handle the 'SAVED' action. Note that I also added a new boolean property saved to the Todo interface. Usually this would rather be some kind of ID. You might also only want to add a todo to the state once it's saved to the server (see Optimistic and Pessimistic UI Rendering Approaches).

// todos.reducer.ts
case "SAVED":
  return {
    todos: state.todos.map((todo, index) => {
      if (index === action.index) {
        return {
          ...todo,
          saved: true
        };
      }
      return todo;
    })
  };

Learning NgRx

While it's fun and a good learning experience to implement NgRx store and effects yourself, you should definitely stick with the official library for real Angular apps. This way you'll get a tested and type-safe implementation with a lot more features.

If you want to learn solid NgRx foundations, you've come to the right place, because I'm writing a book on that and you can get it for free 📖

I'm pouring all my experience into this complete learning resource while allowing you to pay what you want - it's my main goal to help people to gain proper software development skills, so share the link to the book with anyone who might like it.

Either way, hopefully I was able to shed some light on the inner workings of NgRx and thus make the library more approachable for you. Here's a StackBlitz showing the full implementation.

Angular Autosave for Forms, Services and NgRx

Nils Mehlhorn — Mon, 06 Jul 2020 12:53:14 +0000

Saving a user's changes automatically improves user-experience by preventing data loss. Let's see how we can implement autosave behaviors with Angular.

Autosave vs. Caching

I'm referring to autosave as automatically storing a serialization of user input data on the server-side or at least somewhat persistently outside of Angular - e.g. in LocalStorage or IndexedDB.

If you'd just like to cache the state of input fields between in-app navigations, you might already be fine with implementing a custom RouteReuseStrategy in order to re-use component state.

Similarly, you can retain component state by binding to a service that outlives the component. This could either be a custom service based solely on change detection or leveraging RxJS behavior subjects as well as something like NgRx store.

Form Autosave

Let's see how we can actually autosave forms in Angular. Since the framework leverages RxJS we're already in a pretty good situation to reactively save data upon value changes.

When you're using reactive forms, any AbstractControl (e.g. a FormGroup or single FormControl) will expose an observable property valueChanges. Sadly, just like any other form API, this observable is still typed as any despite emitting the value object of your form. Recently, the Angular team announced their work on strongly typed forms, so this might get better soon!

valueChanges: Observable<any>, A multicasting observable that emits an event every time the value of the control changes, in the UI or programmatically -- Angular Documentation

In order to facilitate autosave, you can now easily subscribe to this observable, map the form value to something your server understands, and send off the data.

But not so fast, please don't subscribe manually from inside a subscription callback. This is error-prone and might not yield the result you're looking for. Instead, let's choose the proper RxJS operators for your intended autosave behavior.

@Component({...})
export class MyComponent implements OnInit, OnDestroy {

    form: FormGroup

    private unsubscribe = new Subject<void>()

    constructor(private service: MyService) {}

    ngOnInit() {
        this.form = /* create reactive form */;
        this.form.valueChanges.pipe(
            switchMap(formValue => service.save(formValue)),
            takeUntil(this.unsubscribe)
        ).subscribe(() => console.log('Saved'))
    }

    ngOnDestroy() {
        this.unsubscribe.next()
    }
}

In the snippet above, every change to the form will trigger a save call. Yet, due to the use of switchMap, only the most recent save call will be active at one point in time. Subsequent value changes will cancel prior save calls when these haven't completed yet.

We could replace switchMap with mergeMap and thus have all created autosave requests run simultaneously. Similarly, we might use concatMap to execute the save calls one after another. Another option might be exhaustMap which would ignore value changes until the current save call is done.

Either way, since we're dealing with a long-lived observable (meaning it doesn't just emit one time but indefinitely), we should unsubscribe from the stream once the component encapsulating our form is destroyed. In the snippet above I'm doing this with the takeUntil operator.

What I'd like to do is save only the most recent version of the form while throttling value changes using the debounceTime operator. Debouncing with 500 means that the observable will only start a save call when 500ms passed without further value changes.

this.form.valueChanges.pipe(
    debounceTime(500),
    switchMap(formValue => service.save(formValue)),
    takeUntil(this.unsubscribe)
).subscribe(() => console.log('Saved'))

If you'd like to run a save periodically while the user is constantly inputting data, you can instead use the auditTime or throttleTime operator.

Join my mailing list and follow me on Twitter @n_mehlhorn for more in-depth Angular & RxJS knowledge

Autosave for Subject Services

When you're handling state through any kind of RxJS subject in a service, you can apply the same principle. Just pipe the subject using the operator combination that fits the behavior you want to achieve.

The following service will autosave any setting changes periodically after 1s while they occur thanks to auditTime. The concatMap operator makes sure that none of the save requests are cancelled while keeping them in chronological order.

export interface Settings {
    darkMode: boolean
}

export class SettingsService implements OnDestroy {

    private unsubscribe = new Subject<void>()

    private settings = new BehaviorSubject<Settings>({darkMode: false})

    public settings$ = this.settings.asObservable()

    constructor(private service: MyService) {
        this.settings.pipe(
            auditTime(1000),
            concatMap(settings => service.save(settings)),
            takeUntil(this.unsubscribe)
        ).subscribe(() => console.log('Saved'))
    }

    setDarkMode(darkMode: boolean) {
        this.settings.next({...this.settings.getValue(), darkMode})
    }

    ngOnDestroy() {
        this.unsubscribe.next()
    }
}

NgRx Autosave

When using NgRx, autosave is best implemented as an effect.

Effects are where you handle tasks such as fetching data, long-running tasks that produce multiple events, and other external interactions where your components don't need explicit knowledge of these interactions. -- NgRx Documentation

The specific approach I've chosen for SceneLab is to list all actions that modify the state that should be autosaved. We already did something similar to implement undo-redo with ngrx-wieder.

const STATE_MODIFYING_ACTIONS = [
    addElementSuccess,
    undo,
    redo,
    resizeSelection
    ...
]

Then we can create an effect that listens for any of these actions by initializing the ofType operator with the spread syntax.

autosave$ = createEffect(() => this.actions$.pipe(
    ofType(...STATE_MODIFYING_ACTIONS),
    debounceTime(500),
    map(() => MyActions.save())
))

After being debounced, the effect will create a saving action which we will handle in a separate effect. This allows us to easily trigger a save from other places while properly separating concerns. The actual save effect will eventually look very much how you'd write any asynchronous effect for NgRx. I'm also using withLatestFrom to access the latest state to save.

save$ = createEffect(() => this.actions$.pipe(
    ofType(MyActions.save),
    withLatestFrom(this.store)
    switchMap(([action, state]) => this.service.save(state)),
    map(() => MyActions.saveSuccess())
))

Note that the save call could produce errors which you might want to handle differently than NgRx. By default it will re-subscribe to the effect observable up to 10 times.

Meanwhile, we can also manage a flag in our state indicating to the user that we are currently saving their data.

const myReducer = createReducer(initialState,
    on(...STATE_MODIFYING_ACTIONS, state => {
        return {...state, saved: false}
    }),
    on(MyActions.saveSuccess, state => {
        return {...state, saved: true}
    })
)


@Component({...})
export class MyComponent implements OnInit, OnDestroy {

    saved$ = this.store.select(state => state.saved)

    constructor(private store: Store<State>) {}
}

<p *ngIf="saved$ | async; else saving">saved</p>
<ng-template #saving>
    <p>saving...</p>
</ng-template>

If you want to get the UX here perfectly right and not display 'saving...' before the user has made any changes, you'll also have to manage a pristine flag similar to the one from Angular forms.

Here's how this looks for SceneLab where the indicator is located in the header:

HTTP or WebSocket? LocalStorage?

I've been using service.save(state) as a placeholder for making the HTTP server request that persists data. However, you might be wondering whether HTTP is the right protocol to use for autosave - so am I. From my perspective, there are two aspects to consider:

payload size
request frequency

Since HTTP has a moderate overhead per request, it'd be better fit for lower request frequencies while the payload size can be arbitrarily big. However, you probably want to keep the payload size - just like any serialization efforts - possibly low anyways for good performance.

Websockets, on the other hand, open a connection once in order to send only minimal messages after that. Therefore it'd be better for higher request frequencies with smaller payloads. Websockets are especially useful for pushing data from the server to the client e.g. for a chat application. Though, for autosave, we only need to send client data to the server.

Yet, what are lower and higher request frequencies? I'd argue that with a debounced implementation based on user-changes, the save frequency won't be all that high. Therefore I'd advise you to try out an HTTP-based solution before jumping into a new protocol involving long-lived connections that your servers and proxies need to support - possibly at a certain scale. Make sure though, your server is using HTTP/2 to get the most out of it.

For SceneLab we went with HTTP while the payload size is usually around a few kilobytes. Try it out in the app and see how it feels (you need to be logged-in for autosaving to the server).

As a reference, Google Docs is also sending HTTP POST requests on every single keystroke.

Meanwhile, you might have a use-case where you don't need to send data to the server at all. Maybe it's enough to store the data in LocalStorage or IndexedDB. That's what we're doing in SceneLab when you're using the app without being logged-in. Once you login, it'll allow you to recover a project you've drafted up before committing to a registration.

Angular File Download with Progress

Nils Mehlhorn — Mon, 16 Mar 2020 10:39:55 +0000

Downloading files is a common task for web applications. These files could be some PDF, ZIP or any other binary or text-based file that you want to make accessible to your users. Here's how you can download files in Angular either with a rather simple link or JavaScript-based for more control and progress indication.

The download operator developed in this article is available in the ngx-operators library 📚 - a collection of battle-tested RxJS operators for Angular. I'd appreciate it if you'd give it a star ⭐️ on GitHub, this helps to let people know about it.

Angular Download Link

A simple download link can be easily achieved with plain HTML in Angular. You'll use an anchor tag pointing to the file with the href attribute. The download attribute informs the browser that it shouldn't follow the link but rather download the URL target. You can also specify its value in order to set the name of the file being downloaded.

<a href="/downloads/archive.zip" 
  download="archive.zip">
  archive.zip
</a>

You can bind any of these attributes with Angular in order to set the URL and filename dynamically:

<a [href]="download.url" [download]="download.filename">
  {{ download.filename }}
</a>

Older browsers, like the Internet Explorer, might not recognize the download attribute. For those cases you can open the download in a new browser tab with the target attribute set to _blank. Make sure though to always include rel="noopener noreferrer" when you're using target="_blank" so you're not opening yourself up to security vulnerabilities.

<a [href]="download.url" target="_blank" rel="noopener noreferrer">
  {{ download.filename }}
</a>

If there's no download attribute, the filename for your download will solely depend on the HTTP header Content-Disposition sent by the server that's providing the file. The information from this header might also take precedence even if the download attribute is present.

Recommended Read:

Building a Good Download… Button? - Eric Bailey

A link-based solution conforms well to HTML standards and lets the browser do most of the work. However, if you want more control over the download and would like to display some custom progress indicator you can also download files via Angular's HttpClient.

Download File with HttpClient

A file is best represented as a Blob in the browser:

The Blob object represents a blob, which is a file-like object of immutable, raw data
-- MDN web docs

By specifying the responseType option we can perform a GET request returning a blob representing the downloaded file. Let's assume we've got a designated DownloadService doing just that:

@Injectable({providedIn: 'root'})
export class DownloadService {

  constructor(private http: HttpClient) {}

  download(url: string): Observable<Blob> {
    return this.http.get(url, {
      responseType: 'blob'
    })
  }
}

A component would then be able to call this service, subscribe to the corresponding observable and eventually save the file like this:

@Component({...})
export class MyComponent  {

  constructor(private downloads: DownloadService) {}

  download(): void {
    this.downloads
      .download('/downloads/archive.zip')
      .subscribe(blob => {
        const a = document.createElement('a')
        const objectUrl = URL.createObjectURL(blob)
        a.href = objectUrl
        a.download = 'archive.zip';
        a.click();
        URL.revokeObjectURL(objectUrl);
      })
  }
}

Here, we're creating an anchor tag programmatically when the blob arrives. With URL.createObjectURL we can generate a download link to the blob. Finally, we click() the link like the user would've done with a regular browser download link. After the file is downloaded, we'll discard the blob by revoking the object URL we created.

This approach is pretty verbose though and might not work smoothly for every browser. Therefore I'd advise you to use the popular library FileSaver.js when saving blobs. The saving then becomes a one-liner:

import { saveAs } from 'file-saver';

download() {
    this.downloads
      .download('/downloads/archive.zip')
      .subscribe(blob => saveAs(blob, 'archive.zip'))
}

If you don't like adding a dependency for this and would prefer to use the manual approach shown before, you might as well refactor the code for saving the blob into a separate service. There you probably want to inject document with Angular's built-in injection token DOCUMENT. You can also create a custom injection token for URL - also see below how we'll do this for FileSaver.js.

Calculating the Download Progress

By setting the option observe to events while making an HTTP request, we won't just receive the final response body of the request but also get access to intermediate HTTP events. There are multiple kinds of HTTP events in Angular, all consolidated under the type HttpEvent. We also need to explicitly pass the option reportProgress in order to receive HttpProgressEvents. Our HTTP request will eventually look like follows:

this.http.get(url, {
  reportProgress: true,
  observe: 'events',
  responseType: 'blob'
})

Since we don't just want to forward these events to every component, our service has to do some more work. Otherwise our component would have to deal with HTTP specifics - that's what services are for! Instead let's introduce a data structure representing a download with progress:

export interface Download {
  state: 'PENDING' | 'IN_PROGRESS' | 'DONE'
  progress: number
  content: Blob | null
}

A Download can be in one of three states. Either it hasn't started yet, therefore it's pending. Otherwise it's done or still in progress. We use TypeScript's union types to define the different download states. Additionally, a download has a number indicating the download progress from 1 to 100. Once a download is done, it will contain a Blob as its content - until then this property is not available, therefore null.

Now we want to abstract from specific HTTP events to our newly defined data structure. This way our components can be decoupled from the underlying network protocol. Since we're dealing with multiple events coming in over time, a RxJS operator is well suited here - so let's create one!

The first step for this will be the creation of type guards helping us to distinguish different HTTP events. This way we can access event-specific fields in a type-safe way.

We'll focus on the events HttpResponse and HttpProgressEvents. They both contain the discriminator field type allowing us to easily return a boolean for the type assertion in our guards.

import {HttpEvent, HttpEventType, HttpResponse, HttpProgressEvent} from '@angular/common/http'

function isHttpResponse<T>(event: HttpEvent<T>): event is HttpResponse<T> {
  return event.type === HttpEventType.Response
}

function isHttpProgressEvent(event: HttpEvent<unknown>): event is HttpProgressEvent {
  return event.type === HttpEventType.DownloadProgress 
      || event.type === HttpEventType.UploadProgress
}

The guards can be used with a simple if-statement, however, TypeScript will narrow the event type inside the statement block for us:

const event: HttpEvent<Blob> = ...
console.log(event.loaded) // not accessible, possible compilation error
if (isHttpProgressEvent(event)) {
  console.log(event.loaded) // type is narrowed, property is accessible
}

Based on these guards we can now create our custom operator. It'll leverage scan, an operator that allows us to accumulate state for successive values coming through an observable. It takes up to two arguments: First, we provide an accumulator function which will compute the next Download state from the previous one and the current HttpEvent. Second, we'll pass a seed to scan representing the initial Download state. This seed will represent our download being pending without any progress or content:

{state: 'PENDING', progress: 0, content: null}

Our accumulator will use the previously defined guard to update the Download state over time with information from the HTTP events:

(previous: Download, event: HttpEvent<Blob>): Download => {
  if (isHttpProgressEvent(event)) {
    return {
      progress: event.total
        ? Math.round((100 * event.loaded) / event.total)
        : previous.progress,
      state: 'IN_PROGRESS',
      content: null
    }
  }
  if (isHttpResponse(event)) {
    return {
      progress: 100,
      state: 'DONE',
      content: event.body
    }
  }
  return previous
}

When we encounter a HttpProgressEvent, we calculate the progress based on the number of bytes already loaded and the total bytes. A download is done when we receive a HttpResponse containing the file contents in its body. When receiving any other events than HttpProgressEvent or HttpResponse, we won't alter the download's state and return it as it is. This way, for example, we can keep the information in the progress property while other events that won't allow us to compute the progress can be ignored for now.

Let's finally define our custom operator that's using scan with our accumulator and seed:

export function download(
  saver?: (b: Blob) => void
): (source: Observable<HttpEvent<Blob>>) => Observable<Download> {
  return (source: Observable<HttpEvent<Blob>>) =>
    source.pipe(
      scan((previous: Download, event: HttpEvent<Blob>): Download => {
          if (isHttpProgressEvent(event)) {
            return {
              progress: event.total
                ? Math.round((100 * event.loaded) / event.total)
                : previous.progress,
              state: 'IN_PROGRESS',
              content: null
            }
          }
          if (isHttpResponse(event)) {
            if (saver && event.body) {
              saver(event.body)
            }
            return {
              progress: 100,
              state: 'DONE',
              content: event.body
            }
          }
          return previous
        },
        {state: 'PENDING', progress: 0, content: null}
      )
    )
}

Notice that this download operator accepts an optional parameter saver. Once a HTTP response is received, this function is invoked with the download content from inside the accumulator. This allows us to pass in a strategy for persisting the download to a file without directly coupling the operator to FileSaver.js.

The service code now looks like this when using the custom operator:

import { saveAs } from 'file-saver';
...
download(url: string, filename?: string): Observable<Download> {
    return this.http.get(url, {
      reportProgress: true,
      observe: 'events',
      responseType: 'blob'
    }).pipe(download(blob => saveAs(blob, filename)))
}

Decoupling FileSaver.js

By keeping FileSaver.js out of our custom operator, the resulting code is more maintainable. The download operator can be tested without somehow mocking the saveAs import (see here for corresponding tests). If we apply the same pattern to the service, we'll be able to test it just as easy. So let's do that by creating a custom injection token for saveAs in a file called saver.provider.ts:

import { InjectionToken } from '@angular/core'
import { saveAs } from 'file-saver';

export type Saver = (blob: Blob, filename?: string) => void

export const SAVER = new InjectionToken<Saver>('saver')

export function getSaver(): Saver {
  return saveAs;
}

Then use the token to register a provider in an Angular module:

import {SAVER, getSaver} from './saver.provider'

@NgModule({
  ...
  providers: [
    {provide: SAVER, useFactory: getSaver}
  ]
})
export class AppModule { }

Our service can then get the saving method injected and thus experience more loose coupling:

@Injectable({providedIn: 'root'})
export class DownloadService {

  constructor(
    private http: HttpClient,
    @Inject(SAVER) private save: Saver
  ) {}

  download(url: string, filename?: string): Observable<Download> {
    return this.http.get(url, {
      reportProgress: true,
      observe: 'events',
      responseType: 'blob'
    }).pipe(download(blob => this.save(blob, filename)))
  }
}

Showing a Progress Bar

Let's use the Progress Bar from Angular Material to show how far along our download is. We'll create a component property for binding a download called download$. The component now only has to assign an observable download to this property:

@Component({...})
export class MyComponent  {

  download$: Observable<Download>

  constructor(private downloads: DownloadService) {}

  download(): void {
    this.download$ = this.downloads.download(
        '/downloads/archive.zip', 
        'archive.zip'
    )
  }
}

We can then subscribe to this observable through the AsyncPipe in combination with NgIf. While the download is pending we'll display the progress bar in 'buffer' mode (you may also use 'query'), otherwise the progress is determinate. The bar's value can then easily be applied from Download.

<mat-progress-bar *ngIf="download$ | async as download"
        [mode]="download.state == 'PENDING' ? 'buffer' : 'determinate'" 
        [value]="download.progress">
</mat-progress-bar>

Pro Tip: If you need to map something to more than two values inside a template or rather a ternary statement won't do it for you: map the observable to the type you need or use a custom pipe instead of calling a component function from the template. Both methods are pretty easy to write, more declarative and perform better.

As always, if you've got any questions don't hesitate to leave a comment below or ping me on Twitter @n_mehlhorn. You can also follow me there and join my mailing list to see when new articles are coming up and get smaller tips around Angular and web development in general.

Here's a StackBlitz showing everything in action. The downloaded file is only 3MB, so you might want to enable throttling to see more of the progress bar.

Handling Observables with Structural Directives in Angular

Nils Mehlhorn — Mon, 10 Feb 2020 11:43:40 +0000

Handling observables is a much discussed topic in Angular. There are multiple ways to get reactive values displayed in your template, but sometimes they all just feel a bit clunky. Let's explore which options are available, how they work and how we might improve upon them.

The observe directive developed in this article is available in the ngx-observe library 📚

I'd appreciate it if you'd give it a star ⭐️ on GitHub, this helps to let people know about it

There are two main solutions for handling observables that bring data into a component's view:

Manual Subscription Management
Using the AsyncPipe in combination with NgIf.

Tomas Trajan already wrote a comprehensive article comparing both ways, eventually declaring the second one as the winner.

The Ultimate Answer To The Very Common Angular Question: subscribe() vs | async Pipe

by Tomas Trajan

NgIf and the AsyncPipe are a great fit - but it's not a match made in heaven. It has substantial drawbacks:

falsy values (false, 0, '', null, undefined) emitted by our observable will result in the else-template being displayed - this is because NgIf doesn't know about observables and will plainly evaluated what's passed to it by the AsyncPipe
we can only capture one value with NgIf and thus can't access errors emitted by our observable
the same template reference is used for when the observable is still loading and for when it has errored because both will trigger the else-template of NgIf

Let's find out how the approach works and how we can improve it further.

Deconstructing ngIf and AsyncPipe

Getting reactive data into the view involves defining the observable in our component and binding it by combining the NgIf directive and AsyncPipe through the famous as syntax.

Keep in mind though that you won't be able to use the AsyncPipe when dealing with observables that represent an action - for example when you're updating a user based on a button click.

@Component({
  ...
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class UsersComponent {

    users$: Observable<User[]> = this.users.getAll()

    constructor(private users: UserService) {}
}

<p *ngIf="users$ | async as users; else loading">
  There are {{ users.length }} online.
</p>
<ng-template #loading>
  <p>Loading ...</p>
</ng-template>

Using this method is already a nicely declarative way of handling observables. Let's have a look at its advantages one by one and see how they're working.

No Subscription Management

We don't have to unsubscribe since we've never manually subscribed to the users$ observable. This is all taken care of by the AsyncPipe. Looking at it's code on GitHub you can see how it's subscribing to the passed in observable inside transform() and unsubscribing inside ngOnDestroy() - basically just like we would've done by manually calling subscribe() and unsubscribe() or using the RxJS operator takeUntil, just wrapped into an Angular pipe.

OnPush Change Detection

Once you're using the AsyncPipe you can improve performance by configuring your component to use OnPush as its ChangeDetectionStrategy. This is not magically tied to the AsyncPipe by itself. The pipe rather triggers change detection explicitly once a new observable value is coming through (see lines 140-145 in its code).

Currently there's no official documentation on how the OnPush change detection actually works. I don't like to rely on some third-party blog post for such essential information (and you should neither), so let's look at some code again - or rather tests thereof. There's a designated test suite for OnPush telling us everything we need to know. In this mode change detection is running by itself only in three cases:

when the component's inputs are reassigned
when events occur on the component or one of its children
when the component is "dirty", meaning it's explicitly marked for change detection through a call to markForCheck() on a ChangeDetectorRef (like it's done in the AsyncPipe)

Change detection means that Angular will update the template bindings with the values from your component's class instance. When using the default ChangeDetectionStrategy this is done in a multitude of cases and not just those three mentioned above - this is where the perfomance improvement is coming from when using OnPush.

Updating template bindings often means updating the DOM and that's a relatively costly operation. So when Angular has to do it less often, your application will run more smoothly. On the other hand though you'll have to tell Angular explicitly when changes occur - or rather let the AsyncPipe do it.

Rendering Templates Conditionally

NgIf is what's called a structural directive in Angular - "structural", because it's manipulating the DOM:

Structural directives are responsible for HTML layout. They shape or reshape the DOM's structure, typically by adding, removing, or manipulating elements.

The asterisk (*) in front of the directive's name tells Angular to evaluate the assignment using microsyntax. While that might sound daunting, it's just a short way of calling JavaScript setters on the directive instance. Every keyword in a such a microsyntax expression - like else for NgIf - corresponds to a setter in the directive code. The setter naming obeys a pattern starting with the directive selector followed by the keyword. For else it's set ngIfElse(templateRef: TemplateRef<NgIfContext<T>>|null) like you can see from the official sources in line 187. This setter is accepting a TemplateRef which is a reference to a ng-template tag. In our example above it's labeled with #loading. A structural directive can render template references into the view and conditionally provide a context - more on that soon.

There's also a keyword then which you could use to assign a template for the truthy branch dynamically. Per default though NgIf will use the tag it's assigned to as a template for that (see line 160).

Now anytime the underlying observable emits a new value the AsyncPipe will pass it on to NgIf through our microsyntax expression and trigger re-evaluation inside of it. The directive will subsequently add the else-template while there's no value emitted from the observable (because it's still loading or has errored) or when that value in itself is falsy. The then-template will be added when there's a truthy value emitted by the observable.

The last bit to all of this is the as keyword. As it turns out there's no corresponding setter in the source code of the NgIf directive. That's because it's not specific to NgIf - rather it has to do with the context of a template reference. Such a context is a type that's declaring all variables available while rendering the template. For NgIf this type is NgIfContext<T> and looks like this:

export class NgIfContext<T> {
  public $implicit: T;
  public ngIf: T;
}

The generic type T is referring to the type you're passing into the directive. So when you're binding 'hello' it's going to be string. When you're passing an Observable<string> through an AsyncPipe, the pipe will effectively unwrap the observable and T will again be narrowed to string.

We can get a hold of anything that's in such a template context by declaring a template input variable using the let keyword in the pattern let-<your-var-name>="<context-property>". Here's an example for NgIf:

<ng-template [ngIf]="'hello'" let-a="$implicit" let-b="ngIf" let-c>
  <p>a = {{ a }}</p>
  <p>b = {{ b }}</p>
  <p>c = {{ c }}</p>
</ng-template>
<p *ngIf="'hello' as d">
  d = {{ d }}
</p>

Here's the example in action showing that actually all variables a, b, c and d will been assigned to 'hello'.

The property $implicit in any template context will be assigned to a template input variable that's not referencing a specific context property - in this case c. This is a handy shortcut so you don't have to know the specific context of every directive you're using. It also explains why a and c get the same values.

In the case of NgIf the context property ngIf will also reference the evaluated condition. Therefore b also evaluates to 'hello'. And that's also the basis for the as keyword. More precisely, Angular will create a template input variable based on the literal you put after as and assign to it the context property having the same name as the directive itself. Again, no official documentation on this is available but there are tests for this functionality.

A Structural Directive for Observables

As we've now seen there's no magic to any of these parts - nothing we couldn't implement ourselves. So let's come up with something that's specifically suited for rendering observables in templates and then explore it step-by-step:

import {
  Directive, Input, TemplateRef, ViewContainerRef,
  OnDestroy, OnInit, ChangeDetectorRef
} from '@angular/core'
import { Observable, Subject, AsyncSubject } from "rxjs";
import { takeUntil, concatMapTo } from "rxjs/operators";

export interface ObserveContext<T> {
  $implicit: T;
  observe: T;
}

export interface ErrorContext {
  $implicit: Error;
}

@Directive({
  selector: "[observe]"
})
export class ObserveDirective<T> implements OnDestroy, OnInit {
  private errorRef: TemplateRef<ErrorContext>;
  private beforeRef: TemplateRef<null>;
  private unsubscribe = new Subject<boolean>();
  private init = new AsyncSubject<void>();

  constructor(
    private view: ViewContainerRef,
    private nextRef: TemplateRef<ObserveContext<T>>,
    private changes: ChangeDetectorRef
  ) {}

  @Input()
  set observe(source: Observable<T>) {
    if (!source) {
      return
    }
    this.showBefore()
    this.unsubscribe.next(true);
    this.init.pipe(
      concatMapTo(source),
      takeUntil(this.unsubscribe)
    ).subscribe(value => {
      this.view.clear()
      this.view.createEmbeddedView(this.nextRef, {$implicit: value, observe: value})
      this.changes.markForCheck()
    }, error => {
      if (this.errorRef) {
       this.view.clear()
       this.view.createEmbeddedView(this.errorRef, {$implicit: error})
       this.changes.markForCheck()
      }
    })
  }

  @Input()
  set observeError(ref: TemplateRef<ErrorContext>) {
    this.errorRef = ref;
  }

  @Input()
  set observeBefore(ref: TemplateRef<null>) {
    this.beforeRef = ref;
  }

  ngOnDestroy() {
    this.unsubscribe.next(true)
  }

  ngOnInit() {
    this.showBefore()
    this.init.next()
    this.init.complete()
  }

  private showBefore(): void {
    if (this.beforeRef) {
      this.view.clear()
      this.view.createEmbeddedView(this.beforeRef)
    }
  }
}

Let's also have an example showing it's usage so we can see the connections:

<p *observe="users$ as users; before loadingTemplate; error errorTemplate">
  There are {{ users.length }} online.
</p>
<ng-template #loadingTemplate>
  <p>Loading ...</p>
</ng-template>
<ng-template #errorTemplate let-error>
  <p>{{ error }}</p>
</ng-template>

Starting at with the constructor we can get a handle on the ViewContainerRef. This will allow us to manipulate the DOM by rendering templates in place of our directive.

Angular will also provide us with a reference to the tag's template on which we've put *observe. In our example that's the p tag binding the observables value. We can call it nextRef (as it's for displaying the next observable value) and type its context very similar to how it's done for NgIf. ObserveContext<T> will be typed generically upon the underlying observable and provide its value to an implicit template input variable or through the as keyword (because there's a context property called just like our directive).

We'll also inject a ChangeDetectorRef so that we can make our directive work with OnPush change detection.

The setters observeError and observeBefore are following the microsyntax naming and can be used to pass templates for being displayed before the observable has emitted a value (so basically while loading) and for when the observable has an error.

In the first case we can't provide a meaningful context, that's why the TemplateRef for observeBefore has a generic parameter of null. We'll render this template without a context by calling view.createEmbeddedView() just with the template as you can see in showBefore(). There we'll also make sure to first clear() the view - otherwise we might end up with multiple templates rendered at the same time.

In case of an error we can provide a context containing the actual error in the aforementioned $implicit property. We'll create another type for this specific context called ErrorContext and use it to narrow the corresponding TemplateRef passed into observeError. This eventually allows us to define the template input variable let-error in our example.

The AsyncSubject<void> called init is just an observable wrapper around the OnInit hook. Once we let it complete from inside ngOnInit() it'll always emit upon subscription. This will prevent us from rendering any template too early.

The observe setter is where things get interesting. It's the main setter for our directive and in our example it's receiving the users$ observable. When such a source is passed in, any prior subscriptions are cancelled via this.unsubscribe.next(true) in combination with the takeUntil operator - very similar to how you would cancel subscriptions upon ngOnDestroy() during manual subscription management. We'll than make sure to wait for ngOnInit() by piping off of init and then mapping onto the passed in observable using the RxJS operator concatMapTo. This operator will wait for the previous observable to complete and then listen to the next one.

Eventually we're subscribing to the underlying observable and anytime a new value comes trough we'll update the view by first clearing it and then creating an embedded view based on our template with a context containing the value. Lastly we'll notify the change detector with markForCheck() to support OnPush detection.

When an error occurs we'll do almost the same thing just with the template for displaying errors and only with support for an implicit input variable - provided there is an error template available.

Conclusion

Our new directive is even better suited for handling observables than NgIf and the AsyncPipe:

it can display falsy values
it allows you to define separate templates for loading and errors
it lets you access errors from inside the error template

I've put together an example on StackBlitz showing the directive in action. I think it's even more useful than NgIf combined with the AsyncPipe in certain situations. In any case we've learned a lot about structural directives and change detection giving us a better understanding of the framework.

If you've got any questions don't hesitate to leave a comment below or ping me on Twitter @n_mehlhorn. You can also follow me there and join my mailing list to see when new articles are coming up and get smaller tips around Angular and web development in general.

How to Make Interesting Presentations for Software Projects

Nils Mehlhorn — Mon, 27 Jan 2020 12:14:35 +0000

Originally published on scenelab.io - try our online editor for branding and mockup graphics.

No matter how good your website or app might be, if you cannot communicate its value to your client or employer, your hard work will be overlooked. Here's a guide on how to create interesting presentations for your next software project.

Know Your Presentation Audience

Don't start by just slapping a bunch of slides together. Instead take a moment and think about who you're going to present to. This might be your boss, a potential client or an interested crowd at a conference. Think about the background knowledge your audience will posses or what it might be missing out on. Reflect on their interests. Most of the time people are not attending presentations just because they like hearing you talk. So figure out what they're looking to get out of your presentation. This might be technical knowledge, inspiration as well as strategical or economical insights.

Besides knowing such on-topic aspects about your audience, it could also come in handy to have some general conditions on your mind. This might include the time your audience is willing to spend with you. Will your listeners be relaxed or are they in a hurry and barely able to attend your presentation? You don't want people to miss your key points because they already had to leave.

Define a Goal for Your Presentation

Now that you know the people you'll be talking to, you can figure out what to give them. What's the message you want to get across? What's the goal you're trying to achieve with your presentation? You might want to get support for your next software project. You'd like to get funding for building a website or app. Maybe you're trying to sell some software you've created. Or maybe you want to show the progress of your software project to your employer or client.

Defining a specific goal for presenting your software makes crafting the actual presentation a lot easier. At every step you can ask yourself whether some slide you're going to show or a certain thing you're going to say is beneficial for reaching that goal - if it's not, throw it out.

Defining Goal and Audience for Your Software Project Presentation

Outline A Story

Creating a story doesn't mean you have to write a screenplay. Instead construct a cohesive outline for your presentation. I like doing this by creating some placeholder slides with certain keywords and topics. Make sure that every part of your presentation fits with the next one so that your listeners won't get lost.

I'm not a book author, but just like design has it's principles, storytelling has some as well. Start with a brief setup explaining what your presentation is all about and foreshadow where the story will be going. It's highly subjective whether a story is good. Relatability is a key factor for liking a story - that's why you researched your audience. Now you can carefully select on which parts you'd like to go into details.

All good stories have a twist. It's still a presentation so there might just not be a real twist to your topic. Still, what I mean by that is don't spill all your beans in the first five minutes. Try to intrigue the audience by leading up to something. That could be the demonstration of your software, some exciting feature or the results of a proof-of-concept trial.

Address obstacles you overcame along the way, but don't make yourself look bad by mentioning every mistake you made - that's not an interesting story, is it?

Slide Design

Don't get me wrong, I know that some people are that good at public speaking, their slides don't matter. Other times you might not even need slides and rather just stand there by yourself or with some other props. However, when presenting software projects, having some slides is usually a good idea because it's pretty difficult to convey designs and technical concepts with words only. Having off-putting slides though might still be worse than having none at all. Here are some tips that can go a long way.

Less Text, More Content

You've probably heard some arbitrary upper limit for bullet points to slap on a slide - like no more than four or five. The problem with this rule is how people apply it: they just move their remaining bullet points to the next slide.

You'll get out of this dilemma, if you stop thinking in bullet points. Instead focus on the goal of your presentation and design slides that'll help you reach it. Below you can see some layouts I like to use.

Layouts for Content-focused Slides

Replace text with graphics wherever possible. People are naturally more receptive for graphical representations. An ad-hoc diagram you create directly on the slide can convey much more than plain bullet points. It does so by the means of boundaries, grouping and connections.

Ad-hoc diagram for conveying technical concepts

When doing these kind of diagrams don't try to fit everything in them. Focus again on what kind of understanding you want to achieve. You can also leverage diagram standards like UML or BPMN, but keep in mind that not everyone knows these notations by heart. Some plain boxes and arrows can go a long way and it's better to have a simple diagram that's helping people instead of a complex one that's not.

If you've developed a website or app, or anything with a user interface really, you've probably got a lot to show. You can also include design iterations describing how you got to your final result. However, don't bore people by showing every single interface (or state of the same interface) there is.

Mockup Showcasing User Interface on Tablet

It can also be illustrative to see a mockup of your project in action - like someone holding a phone with your app open or a laptop showing your website. This will display your product in a professional look allowing people to easily picture themselves using it.

You'll find mockup resources and templates on graphics marketplaces which you can then assemble using photo editing software like Photoshop.

If you prefer an easier solution that doesn't require you to search for assets and install software, try SceneLab. Just drop your images onto phone, tablet or laptop screens right in your browser. You can choose from an ever-growing collection of professional design templates and adjust them to your needs with custom colors and texts.

SceneLab drag and drop workflow for iPhone X app user-interface mockup

Go Bold

Don't be afraid to sacrifice a whole slide to one word or number - the impact will be worth it. This could be the number of users you have, the revenue your app made or how many orders your software processed. You might also put a quote from some famous figure to give your statements more weight - though don't overdo it with that, rule of thumb: one quote max per presentation.

When you're showing images you never want to ask during presentation whether everyone can make out what's on it. Either zoom into the parts you're talking about or slice up the image.

Simple and Consistent

Limit yourself to a few font configurations consisting of font-family, font-size, font-style and color. This will give your presentation a clean look and provide consistency. Also, especially when working with little diagrams like shown above, try to work meaning into your color coding by using the same colors for roughly the same things.

Having a limited set of layouts can also be beneficial because it will allow viewers to know faster where they should be looking.

One thing I also sometimes like to do is picking an icons for each specific topic and use those consistently throughout the presentation. This could be a map when you're talking about how you planned things out or a target when it's about goals reached. If you keep to a handful different icons, you'll make these topics easily recognizable allowing you to spice up what would've otherwise been just text.

Templates & Resources

I often see people creating their presentation with a theme based on a popular movie or show. I'd advise you to refrain from doing so as you probably don't want to get into problems with copyright infringement when your slides or a video of your presentation end up on the web. Rather stay with unlicensed resources and give credit where credit is due.

Just because you think you're not very skilled at design doesn't mean your presentation has to look bad - there are templates for that! SlidesCarnival and Slidesgo are great places for free-to-use presentation templates. They list them by color, style and occasion enabling you to always find something that fits your project. They come with inspiring placeholder slides, pre-defined color schemes and a bunch of icons.

Besides presentation templates you might also find these resources helpful:

you can find royalty free stock images on Unsplash, Pixabay or Pexels.
if you're looking for some sweet icons try flaticon.
when you're in need of a stimulating quote go to BrainyQuote.

Live Demos

Showing your app or website in action or doing a live coding session is somewhat the holy grail of software presentation. They can leave a major impact but might seem a bit daunting. The key here is preparation and some practice. Figure out the conditions for your demo and prepare accordingly:

Will you have access to internet? Can your demo work without internet?
What kind of connections are available for the projector? Bring your own adapters if necessary.
Does your demo rely on external services (like some API) and what will you do if those services are not available?

When all falls apart skip the demonstration or, even better, have a pre-recorded video of it ready to show. This way you won't waste your precious presentation time and the audience's attention span on fixing your demonstration.

Also, for a professional impression, do some cleaning up. Clear your desktop of anything that's not related to the presentation. Close every program you don't need as you probably don't want to have delicate mail notifications during showtime. Hide the bookmarks bar of your browser not only to keep those from prying eyes but also to have more room when showing a website or web app.

Practicing Your Presentation

Practicing your presentation will not only make you more confident, you'll also know roughly how long it's going to take. Therefore don't rush while practicing because it's just practice.

You can write down some speaking notes but don't script your presentation to a point where it feels awkward. It's not the end of the world if you say something differently or forget to say it at all - after all a presentation is a live performance.

It can also be really helpful to have a test audience while practicing. This will make transitioning to a real audience much easier and you get valuable feedback early on. If there's no one available, you could still record yourself using your phone.

Conclusion

Following some basic steps will give you a solid foundation for crafting presentations. Knowing your audience, having a clearly defined goal and proper slides might also reduce stage fright, allowing you to have a good time when presenting your next software project. By having a focus on content, not being afraid to go bold and some consistency your slides will help you communicate your message much better. With a professional presentation template your slides are almost guaranteed to be interesting. Throw a carefully prepared live demonstration and some practice into the mix and you're ready!

Improving Website Headers Using Hero Images

Nils Mehlhorn — Mon, 20 Jan 2020 10:54:39 +0000

Originally posted on scenelab.io - try our online editor for branding and mockup graphics.

As we visit a website, our attention is naturally drawn to the most prominent areas because our brains can't process all information at once. Despite the fact that top-to-bottom is the common reading flow, most users probably won't look at the website's navigation first. Instead, what usually catches our interest is the section beneath it, which separates the navigation and logo from the content and often covers a large amount of space. This area is generally referred to as hero area or hero image and can be used by designers or developers for several strategic purposes. As the title suggests, said area is sometimes included under the term 'header'.

You can compare this to a storefront where you might catch a glimpse of the actual contents inside, but what you primarily see are the signs and products displayed on the store window to catch your interest.

Storefront Analogy for Website Headers

You probably identified the example above as a wine store, before reading the full name at the top. Similarly, hero areas are a great way of quickly showing key information you don't want the user to miss.

This article lists different features and variants of hero areas as well as their benefits. These shouldn't be considered standalone, but can be mixed to complement each other.

Prominent Text & Call-to-Action

Starting with the basics, text is the core tool of communicating with the viewer. Hero areas are a great place to incorporate your slogan, summarize your brand or showcase limited offers.

Simple design principles like hierarchy and contrast can be used to make the text stand out and likely to be the first thing that's catching the eye. This can be achieved using vivid background colors or gradients and a larger font size than on the rest of the website.

Simple Hero Area with Color Gradient and Text

While every website has its own intention, having the user stay on the site and keep exploring it is one of the primary objectives. Longer sessions lead to increased chances of returned value, for example through a placed order or other types of conversion.

That's also why text in the hero area is often paired with buttons or links which are often referred to as a call-to-action. Their purpose is to guide the user through the site, usually offering direct shortcuts to the most interesting subpages, products, or topics.

Full-Width Images

Using large photos or illustrations at the top of a website has been a big trend over the last few years. You've probably heard the quote "a picture paints a thousand words", but additionally to carrying information, a picture can also give your website a unique look and overall vibe.

High quality stock photos can help to convey professionalism, even if not directly related to your content or product.

Minimalistic images are especially suitable for this case, as they won't be too distracting and retain readability. For rather complex pictures it might be necessary to add a color overlay (or simply reduce the photos opacity), given that you have text on it.

Hero Area with Photo Background

Isolated Image Elements

By embedding individual image elements into the hero area you can create a very 'clean' and focused presentation of your matter. The previously described effect of professionalism will become even stronger, since it requires more effort and creativity than simply inserting a photo.

Besides that, incorporating smaller images gives you more control over the free space that your text is going to fill in.

Embedding Individual (Flatlay) Images

While arranging individual objects in post-production gives you the most control, there are also approaches of using similar techniques already while taking the photo. So called flatlays are a common trend in photography often used for hero images, where several items are shot from above, usually lying on a non-distracting surface.

There are thousands of photos of this kind available on stock photo sites. If you'd rather have your own, custom arrangement of objects however, you can use SceneLab's library with hundreds of isolated items to compose unique scenes. You don't need Photoshop or any other software, since the editor works right in your browser.

Create Flatlays Inline with SceneLab

If your website is promoting an app, displaying screenshots of it using mockups is a very elegant way of showing the user what he can expect of your product. At the glimpse of an eye, they can identify the platform of your app (e.g. mobile or desktop) and already get familiar with the user interface:

Hero Image App Promo Using a Mockup

Forms & Data Inputs

You could count form elements among call-to-actions, however their specific use case and higher complexity deserves its own chapter.

Including forms in the hero area makes sense if you either have a very high interest in collecting data or your website is heavily reliant on user inputs. The most common use of this is including a newsletter signup, as seen in the last example. For content aggregating and booking sites it's good practice to have a search form easily accessible at the top, because a searching is what a very high percentage of users plan to do in the first place. Limiting the amount of scrolling needed for that massively improves the usability and makes it more likely for people to keep using your website over the competition.

Complex Form Accessible at the Top of the Website

Sliders & Animations

In some occasions, switching the displayed information might be desirable. Stacking multiple hero images on top of each other misses its purpose, since the effect of prominent positioning declines on the way down.

Instead, web designers and developers often embed sliders or image carousels that either change their content automatically or through user interaction. This is a great way of highlighting not just one object, but multiple ones - for example all the featured properties on a real estate site:

Hero Image Slider for Featured Properties

Smooth transitions are much more pleasant for the human eye and also have the advantage that the user becomes more aware of the content changes. It's better to have long pauses instead of short intervals to give even slow readers the chance to gather all information. You can still add bullet points or arrows for the user to navigate through the slides more quickly.

While animating individual parts of the hero area can be playful and fun, keep in mind, that too much movement can easily distract the viewer and doesn't add any purposeful meaning most of the times.

Videos and Streams

Running a video in the background of the hero area makes this section more dynamic and let's you show a lot of content within a short time. However, just like with animations, this comes at the cost of the users distraction or visual overstimulation. That's why only calm videos with subtle movement should be used for such purposes. Moreover, combining this technique with too much text makes it hard for the viewer to recognise which media should have the focus.

Most people consider websites to be something static that can react dynamically to their interaction. Therefore, automatically playing videos should be used with caution since this might be experienced as disruptive. In any case, you should always turn off sound by default - nobody likes to be surprised by sudden audio and having to search for its origin to shut it down.

Interactive Media

In this context, 'interactive' is supposed to mean more than just clicking a button or filling a form. Some website creators came up with really playful and creative ideas to keep the user interested while also adding a lot of recognition value. You could for example update some visuals based on the users scrolling, make game elements controllable via arrow keys or have a characters eyes follow the cursor.

Interactive Day-to-night Scrolling Effect

Although the listed examples add a nice eyecatcher to the site, they don't convey a lot of information. The best application area for such interactive elements would be web design and development agencies, simply to showcase their skills in building awesome websites.

Conclusion

A hero image or area is a large section at the top of a website to display the most relevant information to new visitors and can also provide more visual appeal.

This section stands out due to the prominent position and combination of different media types like image and text. If you're looking for a specific reaction or interaction from the user, hero areas can quickly catch their attention and guide them towards the right direction.
By applying the concepts we explored you'll get your message across and have a great chance to increase conversion.

How Developers Can Design Effectively

Nils Mehlhorn — Wed, 08 Jan 2020 11:29:48 +0000

Originally posted on scenelab.io - try our online editor for branding and mockup graphics.

Building websites or apps requires a range of different skills. Independent of what toolset you choose you'll have to know the basics of HTML, CSS, JavaScript to some extent. Nowdays you might also have frameworks like React, Angular or Vue.js on top. Even if you're doing plain server-side rendering with something like WordPress with a bit of jQuery sprinkled on top, you'd still have two more things to get familiar with.

Managing these technologies well is respectable in and of itself. It's perfectly natural not to excel at everything and therefore you shouldn't feel bad for some lack of designing skills - I'll give you some tips on how to end up with decent user interfaces anyway.

Web Developer Without Design Skills

You definitely can work as a professional web developer without having design skills. In my opinion this shouldn't even be an actual part of the job description. Once you've done some work in the industry though, you might disagree. Especially in enterprise software development there's a notion for giving user experience and design a lower priority. This can lead to some unpleasant surprises as developers are guessing what's best.

In terms of design and usability, consumer-grade websites and apps have long surpassed enterprise software. Nowadays, enterprise users expect the kind of user experience they know from the tools they use everyday - mostly on their phones.

But you might also be missing a designated designer when working in small ventures or when building something just by yourself. In all those environments it pays of to have some basic design skills and a couple tricks up your sleeve. You'll be inevitably more valuable to the project and also increase communication skills with designers because you know where they're coming from.

Design Principles

Principles are a solid foundation for any craft. They provide you with concepts which are applicable to many different domains. You can use them to reflect and validate your creations. This way you have proper arguments when discussing and refining designs rather than just arguing about what "looks better".

You could start by looking at the works of some influential figures. Alan Dix wrote a book called Human Computer Interaction where he structured several principles into the categories Learnability, Flexibility and Robustness. Don Norman wrote The Design of Everyday Things laying down seven principles of usability design.

Although these principles are often highly praised, they're pretty abstract. Instead of trusting what one person came up with when writing a book about design, I like looking at what worked for people designing real user interfaces. That's why I urge you to take a look at design case studies from other companies and the principles they derived from experience and iteration. Here are a couple studies I found interesting:

Another useful set of principles for evaluating your user interface designs are the Gestalt Principles. These psychological principles describe how humans recognize groups and patterns. They'll prevent you from becoming blind to certain errors you might incorporate in your design by helping you understand how new users will approach your interfaces.

Developer Tools For Design

Designing interfaces might seem to call for extensive tool support allowing you to build sophisticated mockups of your website, however, I'd propose the opposite: just use pen & paper or a whiteboard. In my opinion these are the best tools in most of the cases. Their simplicity offers several advantages:

you don't have to learn complex tools
you won't loose yourself in the pointless bits and peaces of complex tools
iteration is fast and without hurt feelings - turn the page or wipe the board and start over

Low-fidelity Paper Prototype

Designing user interfaces with simple measures gives you what's called a low-fidelity prototype. If you're insisting on a digital tool for that, try Wireframe.cc, draw.io or Sketchpad.

If you'd like to have a more refined look you can still transfer your scribbles into more advanced tools, giving you high-fidelity prototypes.

Sometimes though you're design has to be more polished. For those cases I've had some good experiences with balsamiq. It's somewhere between low and high-fidelity - depending on how intensively you'd use it.

UI Prototype made with balsamiq

When you want to go all out and create a high-fidelity prototype, you can refer to tools like Sketch, Figma or Adobe XD. Maybe also Photoshop or Gimp will work for you as they'll enable you to create pixel-perfect designs.

Again, these tools can be challenging to get into. Making them part of your workflow might therefore be harder than starting out with simpler ones.

While you'll definitely get closer to the actual implementation of your website or app, you can also run into problems with high-fidelity prototypes. It's easy to get stuck up on details when discussing such prototypes. Also people might refrain from criticizing fundamental aspects of your design, because they don't want to devalue the effort you put up. Less tech-savvy people also might get the impression that the final product is almost done when you present fully-fledged and possibly interactive prototypes.

High-fidelity UI Prototype

When you're looking for a color palette to apply to your design, try the Adobe Color Wheel or Material Color Tool. Personally I really like Coolors because you can get inspired by existing palettes or play around until you've got something good.

Design Frameworks

By design frameworks (or design systems) I don't mean just mean some CSS theme or a bunch of components you'd use when building your website or app. Although the latter are often based on the former - such as React Material UI is implementing the Google Material Design Framework - a design framework offers principles, styles and best practices for designing interfaces or rather whole user experiences. Some examples include:

You can also find a nice collection of design systems created by different companies in this Design Systems Gallery.

Design systems have several advantages over going at it on a per-interface basis. First, you get a solid set of pre-existing components you can use. This way you don't have to reinvent the wheel for common interactions. Next, you get a set of rules for creating components and interfaces that work well with the existing ones. Moreover your interfaces will be consistent allowing users to navigate seamlessly through your websites.

You might feel restricted by a design framework, yet, that's exactly what it's supposed to do: give your creativity a solid frame to streamline your work and provide coherent results.

If you don't like using someone else's design framework, you can also create your own. Tools like Storybook or Bit come in handy here.

Get Inspired And Iterate

Have a look at the tools you like using and figure out what exactly you like about them. Learning from others is a totally valid approach - as long as you don't straight up copy them. Also, if you need some smaller design cues tailored to developers, have a look at Refactoring UI.

It's important to setup a system that has a solid foundation whilst being rigid enough to adapt to your use-cases. Making errors quickly allows you to iterate faster eventually reducing the risk of fundamental mistakes and allowing you to grow.

Angular Material Pagination Datasource

Nils Mehlhorn — Fri, 03 Jan 2020 13:14:36 +0000

Join my mailing list for in-depth articles and exclusive content on web technologies like Angular and JavaScript.

In the course of this article we're developing a reactive datasource for the Angular Material library that'll be reusable for many different paginated endpoints allowing you to configure search and sorting inputs on a per-instance basis. The final result is available on StackBlitz.

Although there is a bunch of stuff you can do with JavaScript, on many occasions we're using it to fetch and display some data. In Angular, the fetching part is mostly done via HTTP while the displaying part can be performed by a variety of different user-interface components. This could be a table or a list or a tree-like structure or whatever else you might require.

Angular Material offers a couple components that could be used here - such as the table component. The creators even anticipated the need to disconnect data retrieval from data display and are therefore providing us with the concept of a DataSource.

For most real-world applications, providing the table a DataSource instance will be the best way to manage data. The DataSource is meant to serve a place to encapsulate any sorting, filtering, pagination, and data retrieval logic specific to the application. — Angular Material Documentation

Often times the amount of data we'd like to display is too big to be fetched in one batch. You can get around this by slicing your data and delivering it through pagination. Users will then be able to navigate from page to page smoothly. This is something we'll probably need for many different views that display data - it makes sense to encapsulate this behaviour so we don't have to write it over and over again.

Pagination and Sorting Datasource

Let's have a look at a datasource implementation enabling you to sort data and fetch consecutive pages. First, we'll simplify the Material datasource a bit:

import { DataSource } from '@angular/cdk/collections';
import { Observable } from 'rxjs';

export interface SimpleDataSource<T> extends DataSource<T> {
  connect(): Observable<T[]>;
  disconnect(): void;
}

Usually, the methods connect() and disconnect() would accept a CollectionViewer, however, it seems ill-advised to have the component displaying the data also decide which part of the data it's displaying. The official datasource for the Material table is ignoring the parameter as well.

Next we'll define some reusable types for paginated data in a separate file called page.ts.

import { Observable } from 'rxjs';

export interface Sort<T> {
  property: keyof T;
  order: 'asc' | 'desc';
}

export interface PageRequest<T> {
  page: number;
  size: number;
  sort?: Sort<T>;
}

export interface Page<T> {
  content: T[];
  totalElements: number;
  size: number;
  number: number;
}

export type PaginatedEndpoint<T> = (req: PageRequest<T>) => Observable<Page<T>>

The generic parameter T always refers to the type of data we're dealing with - later on in our example it's User.

The Sort<T> type defines a sorting to be applied (aka. send to the server) to the data. This sorting could be created through the headers of a Material table or via selection.

A PageRequest<T> is what we'll eventually pass to a service which in turn will kick off a corresponding HTTP request. This service will then respond with a Page<T> containing the requested data.

A PaginatedEndpoint<T> is a function accepting a PageRequest<T> and returning an RxJS stream aka. observable containing a corresponding Page<T>.

Now we can put these types to use by implementing our paginated datasource as follows:

import { Observable, Subject } from 'rxjs';
import { switchMap, startWith, pluck, share } from 'rxjs/operators';
import { Page, Sort, PaginatedEndpoint } from './page';

export class PaginatedDataSource<T> implements SimpleDataSource<T> {
  private pageNumber = new Subject<number>();
  private sort = new Subject<Sort<T>>();

  public page$: Observable<Page<T>>;

  constructor(
    endpoint: PaginatedEndpoint<T>,
    initialSort: Sort<T>,
    size = 20) {
      this.page$ = this.sort.pipe(
        startWith(initialSort),
        switchMap(sort => this.pageNumber.pipe(
          startWith(0),
          switchMap(page => endpoint({page, sort, size}))
        )),
        share()
      )
  }

  sortBy(sort: Sort<T>): void {
    this.sort.next(sort);
  }

  fetch(page: number): void {
    this.pageNumber.next(page);
  }

  connect(): Observable<T[]> {
    return this.page$.pipe(pluck('content'));
  }

  disconnect(): void {}

}

Let's go through this step-by-step starting at the constructor. It accepts three parameters:

a paginated endpoint which we'll use to fetch pages
an initial sorting to start with
an optional size for the pages to fetch, defaulting to 20 items per page

We initialize the instance property sort with a RxJS subject. By using a subject we can have the sorting change over time based on calls to the class method sortBy(sort: Sort<T>) which just provides our subject with the next value. Another subject pageNumber is also initialized during construction allowing us to tell our datasource to fetch different pages through the method fetch(page: number).

Our datasource will expose a stream of pages through the property page$. We construct this observable stream based on changes to the sorting. The RxJS operator startWith() allows us to easily provide a starting value for the sorting.

Then, anytime the sorting changes we'll switch over to the stream of page numbers by leveraging the switchMap() operator. Now, as long as the sorting doesn't change, we'll just be looking at page numbers starting with the first page of any sorting - again using startWith().

When the datasource is supposed to fetch a different page - triggered by a call to fetch(page: number) - we'll query the paginated endpoint with the required parameters. Eventually this observable now provides data pages to possibly multiple consuming components. Therefore you might use share() to synchronize those subscriptions.

Finally, inside connect() we just provide a stream of lists of items by mapping any page to its contents using the pluck() operator. This method will eventually be called by the Material table or any other component compatible with the DataSource interface. You might be wondering why we don't map our pages directly to just their contents - that's because we need other page properties like size or number which can then be used by a MatPaginator.

The disconnect() method won't have to do anything here - our datasource will close automatically when all consuming components unsubscribe.

Using the Datasource in a Component

Inside a component that is dealing with specific data we can now utilise our datasource with the Material table. We do this by creating a new instance and passing a function that'll forward page requests to a corresponding service. We also pass a default sorting.

The UserService will be responsible for converting the PageRequest<User> to a proper HTTP request that is in line with your server API inside the page() method.

@Component(...)
export class UsersComponent  {
    displayedColumns = ['id', 'name', 'email', 'registration']

    data = new PaginatedDataSource<User>(
      request => this.users.page(request),
      {property: 'username', order: 'desc'}
    )

    constructor(private users: UserService) {}
}

Again, in order to now change the sorting you can call data.sortBy(sort) once the user selects a new sorting.

In your template you'll pass the datasource to the Material table or any other component that can work with this concept. You'll also define a MatPaginator allowing the user to switch pages. The paginator can also easily consume the stream of pages from our datasource through the AsyncPipe and call upon data.fetch(page: number) to get a different page.

<table mat-table [dataSource]="data">
  <ng-container matColumnDef="name">
    <th mat-header-cell *matHeaderCellDef>Username</th>
    <td mat-cell *matCellDef="let user">{{user.username}}</td>
  </ng-container>
  ...
</table>
<mat-paginator *ngIf="data.page$ | async as page"
  [length]="page.totalElements" [pageSize]="page.size"
  [pageIndex]="page.number" [hidePageSize]="true" 
  (page)="data.fetch($event.pageIndex)">
</mat-paginator>

Adding Query Parameters

When there's a lot of data you probably want to assist your users in finding what they're looking for. You might provide a text-based search or structured inputs for filtering the data by a certain property. These query parameters will differ based on the data you're querying. To compensate for this we'll adapt our datasource to work with a generic set of query parameters.

First we'll add a generic parameter Q to the datasource's type representing a query model for some data, ending up with the type PaginatedDataSource<T, Q>.

We'll then add a constructor parameter for an initial query and create a subject property with this.query = new BehaviourSubject<Q>(initalQuery). This type of subject allows us to access it's last value. We use this characteristic to enable partial updates to our query through an instance method:

queryBy(query: Partial<Q>): void {
    const lastQuery = this.query.getValue();
    const nextQuery = {...lastQuery, ...query};
    this.query.next(nextQuery);
}

This method is accepting a partial representation of our query model. We combine this new query with the last one by accessing the BehaviorSubject<Q> and merging both queries via the spread operator. This way old query properties won't be overridden when only one parameter is updated.

Then, instead of just basing our observable stream of pages on the sort subject, we'll combine both changes to sort and query by using the RxJS operator combineLatest(). Both parameter streams are started off with their initial values - sort through startWith(), query through the constructor parameter of BehaviorSubject.

const param$ = combineLatest([
    this.query, 
    this.sort.pipe(startWith(initialSort))
]);
this.page$ = param$.pipe(
    switchMap(([query, sort]) => this.pageNumber.pipe(
      startWith(0),
      switchMap(page => endpoint({page, sort, size}, query))
    )),
    share()
)

Subsequently we'll also pass the query to the pagination endpoint. In order to do this we need to adapt its type like follows:

export type PaginatedEndpoint<T, Q> = (req: PageRequest<T>, query: Q) => Observable<Page<T>>

Now we can update our component to provide some query inputs. First adapt the initialization of PaginatedDataSource<T, Q> with a type for a specific query like UserQuery. Then provide a paginated endpoint that forwards page request and query to UserService. And lastly pass an initial query.

In our example we'll allow users to be searched through text-based input and a date selection for a user's registration date:

interface UserQuery {
  search: string
  registration: Date
}

data = new PaginatedDataSource<User, UserQuery>(
    (request, query) => this.users.page(request, query),
    {property: 'username', order: 'desc'},
    {search: '', registration: undefined}
)

Inside the template we can simply forward input values to the datasource by calling data.queryBy() with a partial query model containing the query parameter:

<mat-form-field>
    <mat-icon matPrefix>search</mat-icon>
    <input #in (input)="data.queryBy({search: in.value})" type="text" matInput placeholder="Search">
</mat-form-field>
<mat-form-field>
    <input (dateChange)="data.queryBy({registration: $event.value})" matInput [matDatepicker]="picker" placeholder="Registration"/>
    <mat-datepicker-toggle matSuffix [for]="picker"></mat-datepicker-toggle>
    <mat-datepicker #picker></mat-datepicker>
</mat-form-field>
<table mat-table [dataSource]="data">
  ...
</table>
...

Now anytime you change the inputs, the displayed page will update accordingly - provided you properly forwarded the query parameters to your servers and handle them there correctly.

Loading Indication

If you like to indicate to the user that you're fetching a page, you can extend the PaginatedDataSource<T, Q> with a corresponding observable property based on a private subject:

private loading = new Subject<boolean>();

public loading$ = this.loading.asObservable();

Then you can either manually update the subject's value before and after calling the PaginatedEndpoint<T, Q> or rather use the operator indicate(indicator: Subject<boolean>) I've introduced in my article about loading indication in Angular. Just attach it to the observable returned by the paginated endpoint and you're good:

this.page$ = param$.pipe(
    switchMap(([query, sort]) => this.pageNumber.pipe(
      startWith(0),
      switchMap(page => this.endpoint({page, sort, size}, query)
        .pipe(indicate(this.loading))
      )
    )),
    share()
)

You can then display a loading indicator like this:

<my-loading-indicator *ngIf="data.loading$ | async"></my-loading-indicator>

Wrapping up

Through clever behaviour parameterization we can reuse a bunch of logic and thus are able to write powerful yet configurable components for displaying any kind of data. Our extension of the Material datasource allows us to perform pagination, sorting and filtering of remote data in just a couple of lines.

Here's the full example on StackBlitz. I've also included a functional version of the datasource omitting the need for classes.