DEV Community: derek lawless

Downloading Lambda layer versions using the AWS CLI

derek lawless — Thu, 16 Jun 2022 00:00:00 +0000

The packaged .zip for a Lambda layer can be downloaded using the AWS console (appropriate IAM permissions notwithstanding) - as long as you have access to the host AWS account of course.

If the Lambda layer originates in a different AWS account you don't have console access to, you will need to use the AWS CLI.

Why would you want or need to do this though?

As with any third party dependency it makes sense to examine it rather than simply trusting it to be benign in nature
You want to override the layer's behaviour in a specific way (i.e. by merging a later layer) and need to examine its implementation

Fortunately, the aws lambda CLI provides two useful options - one for downloading a layer version by name, the other by its ARN.

Downloading a Lambda layer version by name

To download version 99 of a Lambda layer called acme-corp-lambda-layer using its name, use the get-layer-version option:

URL=$(aws lambda get-layer-version --layer-name ars:aws:lambda:$AWS_REGION:$AWS_ACME_CORP_ACCOUNT:layer:acme-corp-lambda-layer --version-number:99 --query Content.Location --output text)

curl $URL -o acme-corp-lambda-layer.zip

Downloading a Lambda layer version by ARN

If you want to download version 99 of of the layer using its ARN instead, use get-layer-version-by-arn:

URL=$(aws lambda get-layer-version-by-arn --arn arn:aws:lambda:$AWS_REGION:$AWS_ACME_CORP_ACCOUNT:layer:my-lambda-layer:99 --query Content.Location --output text)

curl $URL -o acme-corp-lambda-layer.zip

Conditionally tagging resources in CloudFormation

derek lawless — Wed, 27 Apr 2022 00:00:00 +0000

While tagging resources in CloudFormation is straightforward, conditionally tagging them is a little non-obvious and requires use of conditions.

Use case: tagging a release version

I wanted the ability to conditionally tag a Secrets Manager secret with a release version when the secret was deployed into production. For lower environments, I didn't care / want to specify a release version.

First, we define a parameter that will contain the release version:

Parameters:
    ReleaseVersion:
        Type: String
        Description: The release version e.g. 1.2.3
        Default: ''

Next, we define a condition to easily check whether a non-default value was provided for the parameter:

Conditions:
    HasNoReleaseVersion:
        !Equals [!Ref ReleaseVersion, '']

Finally, we tie it all together on the resource by conditionally setting the release tag depending on the evaluation of HasNoReleaseVersion:

Resources:
    MySecret:
        Properties:
            Tags:
                - Fn::If:
                    - HasNoReleaseVersion
                    - !Ref AWS::NoValue
                    - Key: release
                    - Value: !Ref ReleaseVersion

The AWS::NoValue pseudo parameter will ensure the tag is not created unless a non-default value for the release version was provided.

The unknown Type in TypeScript

derek lawless — Wed, 21 Apr 2021 00:00:00 +0000

Introduced in TypeScript 3.0, the unknown type exists to provide a type-safe counterpart to the any type. To understand why this was considered a necessary or desirable addition to TypeScript, we need to revisit any and why its use can be problematic.

The `any` type

The any type has been in TypeScript since its initial release and represents all possible JavaScript values (making it a top type of the type system).

any can be problematic because of its flexibility - it allows typing to be circumvented:

// TypeScript

let value: any;

// Any value can be assigned to an 'any' type:
value = true;
value = 0;
value = "Hello world";
value = [];
value = {};
value = null;
value = undefined;
value = Date.now;
value = new Error();
value = Symbol();
// ...

// No check on use:
value * value;
value[1];
value.foo.bar.baz();
new value();
// ...
~~~{% endraw %}

While convenient, {% raw %}`any`{% endraw %} undoubtedly increases the risk of errors or unexpected behaviour sneaking into code simply by negating one of the primary benefits of using TypeScript in the first instance - strong typing.

## The {% raw %}`unknown`{% endraw %} type
The {% raw %}`unknown`{% endraw %} type is less permissive than {% raw %}`any`{% endraw %}. While any value is assignable an {% raw %}`unknown`{% endraw %} can only be assigned to itself (or to{% raw %}`any`{% endraw %}).

Revisiting the previous example:{% raw %}

~~~typescript
// TypeScript

let value: unknown;

// Any value can be assigned to an 'unknown' type:
value = true;
value = 0;
value = "Hello world";
value = [];
value = {};
value = null;
value = undefined;
value = Date.now;
value = new Error();
value = Symbol();
// ...

// An 'unknown' type can be assigned to 'any' or 'unknown':
const toAny: any = value;
const toUnknown: unknown = value;

// Assignments to other types will fail:
const toBoolean: boolean = value;
const toString: string = value;
// ...
~~~{% endraw %}

### Narrowing and asserting
Operations cannot be performed on an {% raw %}`unknown`{% endraw %} type without first [narrowing](https://www.typescriptlang.org/docs/handbook/2/narrowing.html) to a more specific type or asserting:{% raw %}

~~~typescript
// TypeScript

let value: unknown;

// Operations including the following will error:
value[1];
value.foo.bar.baz();
new value();
// ...
~~~{% endraw %}

One method of narrowing is with the {% raw %}`typeof`{% endraw %} operator:{% raw %}

~~~typescript
// TypeScript

const value: unknown = 'hello world';

if (typeof value === 'string') {
    // Narrowed to a string type, can safely call 'toUpperCase()'
    console.log(value.toUpperCase());
}
~~~{% endraw %}

Types can be asserted using the {% raw %}`as`{% endraw %} operator:{% raw %}

~~~typescript
// TypeScript

const value1: unknown = 'hello world';
const value2: unknown = null;

console.log((value1 as string).toUpperCase()); // Ok
console.log((value2 as string).toUpperCase()); // Error
~~~{% endraw %}

🔔 Type assertions should be used with caution as they may result in expected run-time issues.

### Unions and intersections
In a [union](https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#union-types), an {% raw %}`unknown` absorbs everything else (the exception to this rule is a union with `any`).

With an [intersection](https://www.typescriptlang.org/docs/handbook/2/objects.html#intersection-types), the converse applies - that is, other types will subsume the `unknown`.

~~~typescript
// TypeScript

// Unions:
type union1 = unknown | null;
type union2 = unknown | undefined;
type union3 = unknown | boolean;
type union4 = unknown | number;
type union5 = unknown | string;
type union6 = unknown | null | undefined | boolean | number | string;
// ...
type unionN = unknown | any; // any

// Intersections:
type intersection1 = unknown & null;
type intersection2 = unknown & undefined;
type intersection3 = unknown & boolean;
type intersection4 = unknown & number;
type intersection5 = unknown & string;
type intersection6 = unknown & null & string; // never
// ...
type intersectionN = unknown & any; // any
~~~{% endraw %}

## When to use {% raw %}`unknown`{% endraw %}
Wherever possible specify an explicit type and avoid using {% raw %}`any`{% endraw %} or {% raw %}`unknown`{% endraw %}. Explicitly specifying types allows you to take advantage of TypeScript's compile-time checking, provides a more explicit contract (something especially important for library writers to consider), and should result in fewer run-time issues.

Where this is not possible, prefer {% raw %}`unknown` over `any` as its characteristics require more deliberate and thoughtful use.

Nemawashi

derek lawless — Thu, 21 Jan 2021 00:00:00 +0000

Literally translating as "to go around the roots", Nemawashi (根回し) historically described the farming activity of transplanting a tree from one location to another. Before transplanting, the tree's roots are dug around and soil from the new location is introduced. This allows the tree to acclimate to its new environment prior to being moved, increasing its chances of survival.

In modern times, Nemawashi describes a process of laying the foundation for a proposal by first seeking consensus inside and outside the team informally - allowing others to give their opinions and contribute - before formally presenting the proposal. This increases the probability of the proposal being accepted and ultimately executed upon.

Although widespread in Japanese business, Nemawashi is perhaps most recognisable outside Japan as one of the 13 pillars of the Toyota Production System (from which Lean was partly derived).

Nemawashi and software development

Conversation about Nemawashi typically centres around the application of Lean to classical business activities and manufacturing. However, it's also a technique that software development teams can make effective use of.

Getting buy-in

Consider a significant architectural change to an existing software system e.g. moving from a monolithic architecture to a microservices-based one.

An architect can dictate a design to the development team, handing it off to be implemented according to her vision or she can practice Nemawashi.

Nemawashi does not mean the architecture will be compromised as a result of design by committee, quite the opposite. By allowing the development team to examine the proposed new architecture in detail before architecture sign-off and provide their inputs and insights, buy-in amongst team members will be increased.

On a software development project of any complexity having a development team operating beyond the role of reluctant implementor can be a critical factor in maximising the probability of success.

Building consensus

Proposals may be organisationally or politically sensitive for a variety of reasons.

A proposal to move from manual infrastructure management on a project to Infrastructure as Code (IaC) might be viewed with a mix of suspicion, concern, and derision by an existing infrastructure team. Athough there may be an element of self-preservation on the part of the infrastructure team, their views should not be dismissed as entirely unfounded.

Here, Nemawashi can be leveraged to build consensus with the infrastructure team - acclimating them to the proposal through positive engagement, assuaging their concerns, being open to feedback, and so on.

The goal

The goal is to see your proposal succeed.

Nemawashi provides a method for achieving this, recognising that by building consensus and offering inclusivity the art of persuasion is made easier.

Nemawashi is somewhat orthogonal to the model of traditional software team 'leadership', where forcefulness is viewed as a positive attribute and celebrated. (There can be little doubt this is partly because the software development industry is wholly under-representative of society.)

As development teams diversify - bringing new perspectives, approaches, and viewpoints - methods such as Nemawashi may become more prevalent and, I believe, recognised as a key attribute of high-performance software development teams.

Binary Search

derek lawless — Sat, 21 Nov 2020 00:00:00 +0000

Liquid syntax error: 'raw' tag was never closed

Sieve of Eratosthenes

derek lawless — Sat, 07 Nov 2020 00:00:00 +0000

Liquid syntax error: 'raw' tag was never closed

Fallacies of Distributed Computing: 8. The network is homogeneous

derek lawless — Tue, 30 Jun 2020 00:00:00 +0000

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

8. The network is homogeneous

The problem

Networks typically must support a wide variety of connected clients -- from desktop-class machines to mobile and IoT devices. These devices and the applications running on them may not be interoperable and a variety of standards-based and proprietary protocols and data formats may be in use.

The network is not homogenous, it is heterogenous.

Solutions

Interoperability is key.

Where possible, avoid applications that use proprietary protocols in favour of those leveraging standards-based ones. Favour data formats that enable interoperability e.g. JSON or XML over binary formats (fallacy 7 aside) and architectures such as REST or web services, particularly when integrating with third-party services.

Fallacies of Distributed Computing: 7. Transport cost is zero

derek lawless — Mon, 29 Jun 2020 00:00:00 +0000

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

7. Transport cost is zero

The problem

Whereas the second fallacy relates to the time taken to transport data over a network, the seventh fallacy relates to the resources required to do so.

Marshalling

There is a resource (and resultant latency) cost in marshalling data between the application layer (Layer 7 on the OSI Reference Model) and transport layer (Layer 4) in order to send it over the network. While this may be an abstract concern where a system is deployed on own hardware, in a cloud computing context where resource utilisation equals money, it can be a more concrete concern.

Computing time costs money.

Infrastructure

The infrastructure used to set up and run the system will incur initial and ongoing costs -- for on-premises systems these may include the cost of physical hardware such as servers, switches, and routers to the salaries of the network operators who will hopefully manage everything.

For cloud computing, while physical hardware costs will of course not be a factor of concern, you will pay a premium for the resources you utilise.

Solutions

While the cost of infrastructure is mostly outside your control, optimising its use is within your domain. Reducing the time spent marshalling data between Layers 7 and 4 may be significant, depending on the functionality provided by the system. For example, the cost of transport for systems providing audio and video streaming, telephony, or realtime collaboration features can be brought closer to zero by leveraging an optimised data format e.g. protocol buffers vs. opting for JSON or XML.

Fallacies of Distributed Computing: 6. There is one administrator

derek lawless — Thu, 25 Jun 2020 00:00:00 +0000

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

6. There is one administrator

The problem

If your system is self-contained and deployed into environments you fully control and manage there may indeed be only one administrator.

However, it is more likely that any non-trivial system will depend on one or more in-house or third-party services for its operation and be deployed into environments you do not completely control or that differ from each other in any number of ways e.g. operating system and/or application framework versions, security patching policies, UAC, shared resources, firewall rules, etc.

Multiple in-house teams may be involved to deploy and support the system, with each operating outside the processes governing that system's development. Third-party services -- their availability, compatibility, and development cadence -- will typically exist outside your control.

Solutions

Infrastructure as Code

Where possible, automate the provisioning and configuration of the environments into which deployments are made. Infrastructure as Code (or IaC) is a core element of DevOps, with infrastructure modelled in code (or DSL-as-template form) -- versus manually managed -- and are governed by the same quality processes as application source code e.g. revisioned using source control, unit tested, etc.

Ideally, regard IaC as a core competency -- that is, embed DevOps capabilities in your development teams.

Logging and monitoring

Diagnosing issues can be non-trivial at the best of times and particularly complex for distributed systems. To gain better visibility into the behaviour of a distributed system ensure that centralised logging, metrics, and tracing (the Three Pillars of Observability) have been considered as key aspects of the system's design.

Decoupling

Ensuring appopriate decoupling between system components allows for greater resiliency in the event of either planned (system upgrades) or unplanned (service failure) downtime.

Introducing queuing (with appropriate retry policies, exponential backoff, and DLQs) is one method of achieving decoupling.

Queues

derek lawless — Wed, 03 Jun 2020 00:00:00 +0000

Liquid syntax error: 'raw' tag was never closed

Stacks

derek lawless — Thu, 28 May 2020 00:00:00 +0000

Liquid syntax error: 'raw' tag was never closed

Linked Lists, revisited

derek lawless — Tue, 26 May 2020 00:00:00 +0000

Liquid syntax error: 'raw' tag was never closed

DEV Community: derek lawless

Downloading Lambda layer versions using the AWS CLI

Downloading a Lambda layer version by name

Downloading a Lambda layer version by ARN

Conditionally tagging resources in CloudFormation

Use case: tagging a release version

The unknown Type in TypeScript

The any type

Nemawashi

Nemawashi and software development

Getting buy-in

Building consensus

The goal

Binary Search

Sieve of Eratosthenes

Fallacies of Distributed Computing: 8. The network is homogeneous

8. The network is homogeneous

The problem

Solutions

Fallacies of Distributed Computing: 7. Transport cost is zero

7. Transport cost is zero

The problem

Marshalling

Infrastructure

Solutions

Fallacies of Distributed Computing: 6. There is one administrator

6. There is one administrator

The problem

Solutions

Infrastructure as Code

Logging and monitoring

Decoupling

Queues

Stacks

Linked Lists, revisited

The `any` type