Advanced TypeScript: reinventing lodash.get

As a part of a backoffice team in a financial organisation, I have to deal with a lot of complex data structures: customer personal data, transactions, you name it. Sometimes you need to present a value that lies deep inside a data object. To make life simpler, I could use
lodash.get which allows me to access a value by its path, and avoid endless obj.foo && obj.foo.bar conditions (though it's not a case anymore after optional chaining had landed).

What is wrong with this approach?

While _.get works perfectly well in runtime, it comes with a huge drawback when used with TypeScript: in a majority of cases, it cannot infer value type, which could lead to various issues during refactoring.

Let's say a server sends us data with a customer's address stored this way

type Address = {
  postCode: string
  street: [string, string | undefined]
}

type UserInfo = {
  address: Address
  previousAddress?: Address
}

const data: UserInfo = {
  address: {
    postCode: "SW1P 3PA",
    street: ["20 Deans Yd", undefined]
  }
}

And now we want to render it

import { get } from 'lodash'

type Props = {
  user: UserInfo
}
export const Address = ({ user }: Props) => (
  <div>{get(user, 'address.street').filter(Boolean).join(', ')}</div>
)

Later, at some point we would like to refactor this data structure and use slightly different address representation

type Address = {
  postCode: string
  street: {
    line1: string
    line2?: string
  }
}

Since _.get always returns any for path strings, TypeScript will not notice any issues, while code will throw in runtime, because filter method doesn't exist on our new Address object.

Adding types

Since v4.1, which was released in Nov 2020, TypeScript has a feature called Template Literal Types. It allows us to build templates out of literals and other types. Let's see how it could help us.

Parsing dot-separated paths

For the most common scenario, we want TypeScript to correctly infer value type by a given path inside an object. For the above example, we want to know a type for address.street to be able to early notice the issue with an updated data structure. I will also use Conditional Types. If you are not familiar with conditional types, just think of it as a simple ternary operator, that tells you if one type matches another.

First of all, let's check if our path is actually a set of dot separated fields

type IsDotSeparated<T extends string> = T extends `${string}.${string}`
  ? true
  : false

type A = IsDotSeparated<'address.street'> // true
type B = IsDotSeparated<'address'> // false

Looks simple, right? But how could we extract the actual key?
Here comes a magic keyword infer which will help us to get parts of a string

type GetLeft<T extends string> = T extends `${infer Left}.${string}`
  ? Left
  : undefined

type A = GetLeft<'address.street'> // 'address'
type B = GetLeft<'address'> // undefined

And now, it's time to add our object type. Let's start with a simple case

type GetFieldType<Obj, Path> = Path extends `${infer Left}.${string}`
  ? Left extends keyof Obj
    ? Obj[Left]
    : undefined
  : Path extends keyof Obj
    ? Obj[Path]
    : undefined


type A = GetFieldType<UserInfo, 'address.street'> // Address, for now we only taking a left part of a path
type B = GetFieldType<UserInfo, 'address'> // Address
type C = GetFieldType<UserInfo, 'street'> // undefined

First, we are checking if our passed path matches string.string template. If so, we are taking its left part, checking if it exists in the keys of our object, and returning a field type.

If the path didn't match a template, it might be a simple key. For this case, we are doing similar checks and returning field type, or undefined as a fallback.

Adding a recursion

Ok, we got the correct type for a top-level field. But it gives us a little value. Let's improve our utility type and go down the path to the required value.

We are going to:

1. Find a top-level key
2. Get a value by a given key
3. Remove this key from our path
4. Repeat the whole process for our resolved value and the rest of the key until there's no Left.Right match

export type GetFieldType<Obj, Path> =
  Path extends `${infer Left}.${infer Right}`
    ? Left extends keyof Obj
      ? GetFieldType<Obj[Left], Right>
      : undefined
    : Path extends keyof Obj
      ? Obj[Path]
      : undefined

type A = GetFieldType<UserInfo, 'address.street'> // { line1: string; line2?: string | undefined; }
type B = GetFieldType<UserInfo, 'address'> // Address
type C = GetFieldType<UserInfo, 'street'> // undefined

Perfect! Looks like that's exactly what we wanted.

Handling optional properties

Well, there's still a case we need to take into account. UserInfo type has an optional previousAddress field. Let's try to get previousAddress.street type

type A = GetFieldType<UserInfo, 'previousAddress.street'> // undefined

Ouch! But in case previousAddress is set, street will definitely not be undefined.

Let's figure out what happens here. Since previousAddress is optional, its type is Address | undefined (I assume you have strictNullChecks turned on). Obviously, street doesn't exist on undefined, so there is no way to infer a correct type.

We need to improve our GetField. To retrieve a correct type, we need to remove undefined. However, we need to preserve it on the final type, as the field is optional, and the value could indeed be undefined.

We could achieve this with two TypeScript built-in utility types:
Exclude which removes types from a given union, and Extract which extracts types from a given union, or returns never in case there are no matches.

export type GetFieldType<Obj, Path> = Path extends `${infer Left}.${infer Right}`
  ? Left extends keyof Obj
    ? GetFieldType<Exclude<Obj[Left], undefined>, Right> | Extract<Obj[Left], undefined>
    : undefined
  : Path extends keyof Obj
    ? Obj[Path]
    : undefined

// { line1: string; line2?: string | undefined; } | undefined
type A = GetFieldType<UserInfo, 'previousAddress.street'>

When undefined is present in the value type, | Extract<> adds it to the result. Otherwise, Extract returns never which is simply ignored.

And this is it! Now we have a nice utility type that will help to make our code much safer.

Implementing a utility function

Now that we taught TypeScript how to get correct value types, let's add some runtime logic. We want our function to split a dot-separated path into parts, and reduce this list to get the final value. The function itself is really simple.

export function getValue<
  TData,
  TPath extends string,
  TDefault = GetFieldType<TData, TPath>
>(
  data: TData,
  path: TPath,
  defaultValue?: TDefault
): GetFieldType<TData, TPath> | TDefault {
  const value = path
    .split('.')
    .reduce<GetFieldType<TData, TPath>>(
      (value, key) => (value as any)?.[key],
      data as any
    );

  return value !== undefined ? value : (defaultValue as TDefault);
}

We have to add some ugly as any type castings because

1. intermediate values could indeed be of any type;
2. Array.reduce expects the initial value to be of the same type as a result. However, it's not the case here. Also, despite having three generic type parameters, we don't need to provide any types there. As all generics are mapped to function parameters, TypeScript infers these upon the function call from the actual values.

Making component type-safe

Let's revisit our component. In the initial implementation, we used lodash.get which didn't raise an error for a mismatched type. But with our new getValue, TypeScript will immediately start to complain

Adding support for [] notation

_.get supports keys like list[0].foo. Let's implement the same in our type. Again, literal template types will help us to get index keys from square brackets. I will not go step by step this time and instead will post the final type and some comments below.

type GetIndexedField<T, K> = K extends keyof T 
  ? T[K]
  : K extends `${number}`
    ? '0' extends keyof T
      ? undefined
      : number extends keyof T
        ? T[number]
        : undefined
    : undefined

type FieldWithPossiblyUndefined<T, Key> =
  | GetFieldType<Exclude<T, undefined>, Key>
  | Extract<T, undefined>

type IndexedFieldWithPossiblyUndefined<T, Key> =
  | GetIndexedField<Exclude<T, undefined>, Key>
  | Extract<T, undefined>

export type GetFieldType<T, P> = P extends `${infer Left}.${infer Right}`
  ? Left extends keyof T
    ? FieldWithPossiblyUndefined<T[Left], Right>
    : Left extends `${infer FieldKey}[${infer IndexKey}]`
      ? FieldKey extends keyof T
        ? FieldWithPossiblyUndefined<IndexedFieldWithPossiblyUndefined<T[FieldKey], IndexKey>, Right>
        : undefined
      : undefined
  : P extends keyof T
    ? T[P]
    : P extends `${infer FieldKey}[${infer IndexKey}]`
      ? FieldKey extends keyof T
        ? IndexedFieldWithPossiblyUndefined<T[FieldKey], IndexKey>
        : undefined
      : undefined

To retrieve a value from a tuple or array, there's a new GetIndexedField utility type. It returns tuple value by a given key, undefined if the key is out of tuple range, or element type for regular array. '0' extends keyof T condition checks if a value is a tuple, as arrays don't have string keys. If you know a better way to distinguish a tuple and an array, please let me know.

We are using ${infer FieldKey}[${infer IndexKey}] template to parse field[0] parts. Then, using the same Exclude | Extract technique as before, we are retrieving value types respecting optional properties.

Now we need to slightly modify our getValue function. For the sake of simplicity, I will replace .split('.') with .split(/[.[\]]/).filter(Boolean) to support new notation. That's probably not an ideal solution, but more complex parsing is out of the scope of the article.

Here's the final implementation

export function getValue<
  TData,
  TPath extends string,
  TDefault = GetFieldType<TData, TPath>
>(
  data: TData,
  path: TPath,
  defaultValue?: TDefault
): GetFieldType<TData, TPath> | TDefault {
  const value = path
    .split(/[.[\]]/)
    .filter(Boolean)
    .reduce<GetFieldType<TData, TPath>>(
      (value, key) => (value as any)?.[key],
      data as any
    );

  return value !== undefined ? value : (defaultValue as TDefault);
}

Conclusion

Now we not only have a nice utility function that improves code type safety, but also a better understanding of how to apply template literal and conditional types in practice.

I hope the article was helpful. Thank you for reading.

All code is available at this codesandbox

Comments

[QuickMick]

this is f*cking insane. why would you do that. imagine somebody new comes to the project - there is no way to understand this type stuff in a reasonable amount of time.
this is a prime example of why not to use typescript. just create a function that does what you need instead of messing around with these weird types. how would you debug that?

[Braden Napier]

How about adding in support for properly typing the return type so that default value is known uinstead of undefined | default? (Should be Typescript 4.8+ to support {} as "any non null | undefined value")

export function getValue<
  TData,
  TPath extends string,
  TDefault = GetFieldType<TData, TPath>,
>(
  data: TData,
  path: TPath,
  defaultValue?: TDefault,
): GetFieldType<TData, TPath> extends {}
  ? GetFieldType<TData, TPath>
  : TDefault;

[Austin Lee]

Hello, I have used your code with little change to adapter _.get() method and create a pull request github.com/DefinitelyTyped/Definit.... All lodash user can enjoy the feature now, thanks for your working! XDDD

[HelKyle]

Thanks for sharing this great awesome articles!

[Kirill Glazunov]

Thanks a lot @tipsy_dev. It is a great approach. But you missed some cases: string[number][number], [number].string etc.

I did add it here: tsplay.dev/mAjB8W . Check it out please. You can find result types at d.ts tab at the right panel.

[Roger Padilla]

Thanks a lot for sharing this @tipsy_dev !

I'm building a library and this could be really useful there. Do you know if/how to add type-safety in the path of the properties via template literals? I mean, not to return undefined but to make the compiler ensure the path is a valid path.

E.g.

interface User {
  email?: string;
}

interface Group {
  name?: string;
  creator?: User;
}

So if we do:

// compiles
GetFieldType<Group, 'creator.email'>;

// produces compilation error
GetFieldType<Group, 'creator.invalidPath'>;

Is there a way to do this?

[Ran Lottem]

Great read!
To tell arrays apart from tuples - arrays have a length property of type 'number' whereas for a tuple the length property is a specific number. After checking that T is either an array or a tuple (using T extends unknown[], maybe?), it should be possible to tell them apart using number extends T['length']?

[Scott Busche]

Wouldn't it be possible to add this type inference to _.get's typings, why do you have to make a new method?

[Austin Lee]

yeah, I'm agree with you. so I create a pull request to @types/lodash with the code, now you can update to latest version if youo use _.get()

[Gerardo Lima]

I'm not sure using lodash.get should be something I could recommend (and most of lodash at all).

[Cefn Hoile]

This is brilliant and a feature I've needed a few times. Might allow indefinitely deep plucking for cefn.com/lauf/api/modules/_lauf_st...

[Aleksei Tsikov]

Thanks @cefn , I'm really happy the article was helpful 🙌

[Sahan]

This post makes me fall in love with Typescript (again!) Very well done 🎉

[Aleksei Tsikov]

I'm in love with TypeScript for the last four years, and it feels like it's one of the best things that happen to JS community 🤩

[Matan Yadaev]

Wow, mind blowing! I don't use typescript on my day job, but this just makes me feel bad about using regular js.

[Aleksei Tsikov]

You definitely should give it a try! In the end, regular js is a perfectly valid ts code, so the transition might be not that hard as you imagine it 🙂

[Alexey Grinko]

Why use lodash get when the path is static (i.e. you know it in advance)? For the first example just use user?.address?.street, and your code is perfectly readable and type-safe.

If still wanna use get for some reason, add an explicit type cast:

(get(user, 'address.street') as UserInfo['address']['street'])
  .filter(...)

This helps to understand the type from the first glance, especially when you look at the code not in your IDE but somewhere where types can't be inferred easily, e.g. on GitHub. And if type of 'street' property changes, you'll be alerted.

On the other hand, lodash's get is really useful when your path is determined dynamically in runtime, e.g. generated by some algorithm or provided by backend. And that's where inferring type is just impossible without an explicit runtime check.

So why complicate things to solve unrealistic problems?

P.S. Would be cool to see a real-life scenario for using this feature.

[Aleksei Tsikov]

Yes, I absolutely agree with you. For static paths, it's definitely better to prefer regular field access with an optional chaining. However, _.get is still widely used, and some might find it easier to replace a function call or augment lodash typings.

Speaking of type-casting, in the first place, you want to avoid it as much as possible. This thing makes type checking looser, and after all, that's not why we are enabling strict flag, right? In this particular case, you are giving TS a hint that this any is a more specific type. But get anyway returns any, and if for some reason type of address.street changes, TS will not complain. In the end, any could be cast to literally any type.

Regarding dynamic paths, of course, not every case is covered (and GetFieldType should probably return unknown when the field is not found). But this approach still works when you have a union of possible values

type Letter = 'alpha' | 'beta' | 'gamma'

const symbols = {
  alpha: { symbol: 'α' },
  beta: { symbol: 'β' },
}

function getSymbol(letter: Letter) {
  return symbols[letter]?.symbol // TS error: Property 'gamma' does not exist on type
}

function getSymbolWithPath(letter: Letter)/* : string | undefined */ {
  return getValue(symbols, `${letter}.symbol`)
}

And anyway, you could think of this article as an exercise to better understand template literal and conditional types 🙂

As for a real-life scenario, we use it in a slightly different, a bit less dynamic way, to create strongly-typed APIs similar to react-table columns config. But that's a topic for another article 😉

[Tim]

Is there a way to avoid 3 levels of ternary operators? Even 2 levels is hard to read.

[Aleksei Tsikov]

Unfortunately, TS doesn't support conditional operators, so it's not possible to concatenate a few expressions with &&. The only way I know is to extract each ternary operator into a dedicated type, similar to how it is done with FieldWithPossiblyUndefined and IndexedFieldWithPossiblyUndefined. But to my mind, it adds even more boilerplate.

[Aleksei Tsikov]

Thank you! I'm glad you liked it 🙂