DEV Community: machine-gurning

Async, Await, Future, Completer, in Dart - Succinct guide with examples

machine-gurning — Mon, 30 Jan 2023 14:41:41 +0000

Async

Definition and use

keyword when defining a function that makes that function asynchronous.

e.g.:

void myFunc() async { ... };

When an async function is called, lines will be executed in a synchronous fashion until the keyword await (explained below) is encountered, at which point the interpreter will "wait" until that line "resolves" (described below).
In the meantime, it will treat the function as "paused" (layman's description) and continue to run the next part of the program -- it will go back to the context of where the function is called and run the next line there.

Rationale

If you want a function to be able to "pause" and "wait" for certain lines within it to resolve, because they aren't instantaneously available, the function must be async

Await

Definition and use

Within async functions, the keyword await can "pause" the interpreter on that line until it is "resolved".
Only things that return a Future object (explained below) can be awaited

void parentFunc() async {
  print("parent func start");
  // At the next line, the interpreter will "stop" and "wait"
  // until the thing that is being awaited (Future.delayed, 
  // explained below) is "resolved". 
  await Future.delayed(Duration(seconds: 3), ()=> print("parent func wait is finished")); 
  firstChildFunc();
  print('parent func continuing');
  secondChildFunc();
  print("parent func end");
  }

Rationale

Don't want to block other things from happening while one thing is being resolved, but do want to respect the order of events within a given function.
Therefore, if something is taking time, "pause" there, keep calculating other things below where that function is called, and once that thing is resolved, keep going through that function.

Examples

Here is an exercise in trying to guess the order of print statements. There are more such exercises below. Write down which order you think it will print before scrolling further.

We have

a main func which is called when you run the program
an async parentFunc which is referenced in the main func
the parentFunc has two functions referenced in it. Both of those are await -ed.
each child function is also async, and contains some await clauses

import 'dart:async';

void main() {
  parentFunc();
  print("main func complete");
}

void parentFunc() async {
  print("parent func start");
  await Future.delayed(Duration(seconds: 3), ()=> print("parent func wait is finished"));
  firstChildFunc();
  print('parent func continuing');
  secondChildFunc();
  print("parent func end");
  }

void firstChildFunc() async {
  print("1st child func start");
  await Future.delayed(Duration(seconds: 3), ()=> print("1st child first wait is finished"));
  await Future.delayed(Duration(seconds: 3), ()=> print("1st child second wait is finished"));
  print("1st child func end");
}

void secondChildFunc() async {
  print("2nd child func start");
  await Future.delayed(Duration(seconds: 3), ()=> print("2nd child first wait is finished"));
  print("2nd child func end");
}

.
.
.
.
.
.
Answer:


// parent func start
// main func complete
// parent func wait is finished
// 1st child func start
// parent func continuing
// 2nd func start
// parent func end
// 1st child first wait is finished
// 2nd child first wait is finished
// 2nd func end
// 1st child second wait is finished
// 1st func end

Explanation:

the main function calls the parent function
the parent function has a synchronous print statement, which, as expected, prints right away
then we encounter an await which blocks parentFunc() from executing more lines, but unblocks the main function, hence the main func complete
then we are solely blocked by the parentFunc 's await -- there is nothing remaining to execute in the main func
we enter the firstChildFunc and print a statement immediately, then hit an await. So, as above, the parentFunc keeps running, hence:

// 1st child func start
// parent func continuing
// 2nd func start
// parent func end

and so on..

Future

Definition and use

A future is an object that represents the result of an asynchronous operation. A future object will return a value at a later time.
A future has two states: uncompleted and completed. When a future completes, it has two possibilities:
- Completed with a value
- Failed with an error

Rationale

Give a function the type Future<T> (where T is the type of what is resolved) so that it can be await-ed.
Otherwise you would have to type all the detail of what is being awaited into the async function that has the await keyword on one of its lines
Not hard, just the object type that can be await -end

Examples

Below, the main function is async, therefore we can write await
The function that is being await -ed must be of type Future<String>
That function returns a Future<String> object
Execute it. Nothing for two seconds, then the statement Finally! is printed to the console

void main() async {
  String bigAsk = await slowlyGetTheString();
  print(bigAsk);
}

Future<String> slowlyGetTheString() {
  return Future.delayed(Duration(seconds: 2), () => "Finally!");
}

Completer

Definition and use

An object which can return be returned as a Future.
Use when you want to define for yourself exactly what will be returned as the future.
Initialise as a Completer<T>() (where T is the type that the future will ultimately resolve into
Fill with completer.complete( ... ) (where ... is of type T)
Return as completer.future

Rationale

Have precise control over the type and contents of what is returned from a Future function.

Examples

Say I don't want to return exactly that which was retrieved from an API or a database, but instead I want to augment it slightly before returning it.
The completer lets me pack whatever I like into the Future object and then return that instead.
Below, I call a function which then (pretends to) query an API (firstPartOfPhrase), then uses that string in a larger string, and returns that as the future object, not the original (pretend) query:

void main() async {
  String bigAsk = await slowlyGetTheString();
  print(bigAsk);
}

// Note, this is async only because I am using await, 
// not to do with completer
Future<String> slowlyGetTheString() async {

  // Initialise my completer
  Completer<String> c = Completer<String>();

  // Do something that takes time 
  String firstPartOfPhrase = await Future.delayed(Duration(seconds: 2), () => "Finally!");

  // Assign a string to the completer. Not necessarily the same 
  // string as was received from the line previously 
  c.complete("$firstPartOfPhrase it is finished!");

  // Now that the completer has that information, return completer.future 
  return c.future;
}

Fastai Chapter 4 - The important parts, part 1: Tensors

machine-gurning — Wed, 25 Jan 2023 13:53:13 +0000

Chapter four, "MNIST_BASICS", or "Lesson 4" of the online course, is the most important and apparently most difficult chapter of the fastai course.

In this series of posts post I will cover, in-depth, the most important concepts, and provide a few more examples of each to make them stick. I will include various practice questions throughout; the answers are available at the end of the post. I strongly recommend attempting the questions before looking at the answers.

The book is available online here
The course is accessible here

Full sequence contents:

Tensors and tensor operations
Building and training a simple regression model
Substituting pytorch/fastai components
Building and training a nonlinear model
Answers to practice questions

1. Tensors and tensor operations

Overview

Crucial to all deep learning is the concept of a tensor. Succinctly, a tensor is a generalisation of concepts that we should already be familiar with: vectors and matrices.

A one-dimensional (or "Rank 1") tensor is equivalent to a number
A two-dimensional (or "Rank 2") tensor is equivalent to a 2D matrix of numbers
A three-dimensional (or "Rank 3") tensor is equivalent to a "cube" of numbers, or stacked matrices
and so on..

Nothing else to it, for now

Rationale

As you will see / have seen, neural networks fundamentally comprise large matrices of numbers that have been expertly picked by the computer. Tensors are simply the way in which we perform operations on, store, and query those numbers. The data we feed neural networks must also be represented in tables of numbers. Learning about tensors and tensor operations is crucial for manipulating the data that is fed into the network, as well as investigating what the network itself is doing.

Why use pytorch tensors over numpy arrays and good olde python lists? I quote the author:

The vast majority of methods and operators supported by NumPy on these structures are also supported by PyTorch, but PyTorch tensors have additional capabilities. One major capability is that these structures can live on the GPU, in which case their computation will be optimized for the GPU and can run much faster (given lots of values to work on). In addition, PyTorch can automatically calculate derivatives of these operations, including combinations of operations. As you'll see, it would be impossible to do deep learning in practice without this capability.

Operations

Here is a useful video on basic tensor operations that covers more than you need to know

Initialising a tensor

First ensure pytorch is installed and imported, then make your first tensor:

import torch
torch.tensor(1)

Jeremy has aliased torch.tensor to just tensor, and you can do the same with this line:

tensor = torch.tensor

You can turn python lists and numpy arrays into tensors

tensor([1,2,3])

tensor([[1,2,3],[1,2,3]])

import numpy as np

tensor(np.array([1,2,3]))

Other ways of creating a tensor, not all of which you need to commit to memory, are:

torch.rand(2,3): Creates a rank 2 tensor of random positive numbers
torch.randn(2,3): Creates a rank 2 tensor of random positive and negative numbers
torch.empty(4,4): empty 4x4 rank 2 tensor
torch.zeros(2,3,4): rank three ("cube") tensor of zeros
torch.ones(...): self-explanatory
torch.eye(3,3): Creates an identity matrix of dimensions 3x3 (get it? "eye-dentity")
torch.arange(4): Creates a rank one tensor of the numbers 1 to 4

Parameters

dtype: You can specify data types implicitly or explicitly in the tensor definition.
device: specify the device that the tensor is saved to. Try "cuda" - if it is available, your tensor will be saved to the graphics card.
requires_grad: to be explained below.

tensor([1.]) # implicitly

tensor([1,2,3], dtype = torch.float32, device="cuda") 
# Personally, I get an error

Transforming tensor type

You can call tensor methods to change their type:

x = tensor([0,1,2,3])
x.bool()   # bool : tensor([false, true, true, true])
x.float()  # float32 : tensor([0.0, 1.0, 2.0, 3.0]) 
x.short()  # int16
x.long()   # int64
x.half()   # float16
x.double() # float64


# Typically does what you would expect:
x = tensor([False, True])
x.float() # tensor([0.0, 1.0])

Transforming tensor shape

We will often need to re-shape tensors to make them fit what we're trying to jam them into. In this chapter, we are introduced to these reshaping methods:

torch.stack(): Takes a sequence (list, tuple, whatever) of tensors of the same size and concatenates them "along a new dimension". Imagine having a "stack" of 2D tensors, all the same dimensions, and then combining them to form a "cube" of numbers. The output tensor is of one higher dimension than the inputs. Tensors must be of the same size.

x1 = tensor([1,2])
x2 = tensor([3,4])
x3 = tensor([5,6])

torch.stack([x1,x2,x3])

There is an optional parameter for dimension, dim, which effectively asks you which direction you want the stacking to take place. Consider the scenario of stacking two rank 3 tensors: you can stack them in one of three ways:

torch.cat(): Unlike stack, cat concatenates the tensors along the specified dimension. The output tensor has the same rank as the inputs.

x1 = torch.zeros((2,3))
x2 = torch.ones((2,3))

torch.cat((x1, x2), dim=0) # try dim = 1 and 2

For your information, the following transformations are all considered "views" on an unchanged tensor. When you apply one of the following methods, no new memory is allocated, and the original tensor is simply referenced in a new way. Read more here: https://pytorch.org/docs/stable/tensor_view.html

**tensor().view()**: Returns a new tensor with the same data but a different shape. Having the first parameter as -1 lets pytorch infer that dimension from the other ones. The dimensions entered must be valid, in that the tensor's data must fit the dimensions correctly.
In the chapter, we use view to transform the shape of our 28x28 rank 2 tensor into a rank 1 tensor of 784 long, to then be fed into the neural network. It would not accept a rank 2 tensor, so we flatten the data out.

x = torch.randn(4, 4)
x.size()
# torch.Size([4, 4])

y = x.view(16)
y.size()
# torch.Size([16])

z = x.view(-1, 8)  # the size -1 is inferred from other dimensions
z.size()
# torch.Size([2, 8])

**tensor().squeeze()**: Removes dimensions of size one. Helpful for eliminating "excess" dimensions. Changes a [n, 1] shaped tensor into [n] shape. Optional parameter dim will only squeeze the tensor in that dimension, and ignore other ways it could be squeezed

x1 = torch.zeros((2,1,3))

x1 
# tensor([[[0., 0., 0.]],   # Note that it is 2D data, trapped
#         [[0., 0., 0.]]])  # in a 3D tensor

x1.squeeze()
# tensor([[0., 0., 0.],   
#         [0., 0., 0.]])

**tensor().unsqueeze()**: Does the opposite of squeeze. Adds an extra dimension of size one to the data. Useful for when you need to perform matrix multiplication and need extra dimensions to make it work. dim lets you decide which dimension to insert it along.
In the chapter, we unsqueeze a tensor that represents the image category (train_y) to make sure it has the same dimensionality as train_x. However, if you don't unsqueeze, the rest of the notebook runs without issue, so I am not entirely sure why he did this, other than good practice...

x1 = torch.zeros((2,1,3))

x1.unsqueeze()

Tensor operations

Finally we get to mathematical operations involving tensors.

Standard operations act the same was as they do in linear algebra, where adding, subtracting, multiplying and dividing by a scalar is performed on every element of the tensor:

x = tensor([1,2,3])

x * 2   # tensor([2,4,6])
x / 2
x + 3
# etc

Crucially, matrix multiplication of tensors works using the standard pythong @ symbol.

This will be instrumental to building our neural network. You need not know more than the fact that the following calculations work, and what matrix multiplication is on a conceptual level (3blue1brown has an insightful series of videos on linear algebra)

x = tensor([[1,2],[1,2]]) 
y = tensor([[3,4],[3,4]])

x@y
# tensor([[ 9, 12],
#         [ 9, 12]])

y@x
# tensor([[ 7, 14],
#         [ 7, 14]])

Other operations you will come across are self-explanatory but worth listing:

mean(): Returns a tensor of rank 0 with the value. Inserting a tuple as a parameter constrains the mean to just the dimensions specified in the tuple.
In the chapter, we use this to compare a large stack of 1010 training samples all contained within a rank 3 of shape 1010x28x28 tensor, with an "idealised" image in a rank 2 tensor of shape 28x28. We wish to get back a rank 1 tensor that has one mean value per training sample, so we define the following function.
(-1, -2) specifies that we want to take the mean of the second last and last dimension only, but keep the other dimensions (i.e. the 1010) intact.

def mnist_distance(a,b): 
  return (a-b).abs().mean((-1,-2))

abs(): Self-explanatory
sqrt(): Self-explanatory
item(): returns the actual tensor element value. Only works with rank 0 tensors, i.e. tensors with one element.

Tensor broadcasting

One characteristic of the pytorch tensor which is used frequently during the chapter is that of broadcasting. For many tensor operations, if two tensors are combined in some way, and one tensor has fewer dimensions than the other, pytorch will try to sensibly extend ("broadly cast") the lower dimension or smaller tensor so that it may play nicely with the larger tensor.

A simple example:

tensor([1,2]) * tensor([4])
# tensor([4,8])

An instance of this in the chapter is when we compare a single rank 2 tensor of shape 28x28 with a rank 3 tensor of shape 1010x28x28. We subtract one from the other, and no error occurs because pytorch interprets what we want and artificially 'extends' the first tensor to play nice with the second:

# Function described above
def mnist_distance(a,b): 
  return (a-b).abs().mean((-1,-2)) 

valid_3_dist = mnist_distance(valid_3_tens, mean3)
#

Conclusion

Familiarity with tensors to the depth of this article is satisfactory for having a good time reading this chapter.

Further information on tensors in this in-depth video by pytorch:

https://www.youtube.com/watch?v=r7QDUPb2dCM