File based queries with tanstack start.

Just want to see the code?
https://github.com/IainMcHugh/start-r-query

Having used tanstack start for some months now, it has definitely become my preferred application starter in terms of developer experience. There is something about the end-to-end router type safety that scratches an itch my brain never knew it had. When you extract type safe query parameters from Route.useParams() that you validated as part of the createFileRoute(), something clicks that simply cannot be un-clicked.

What's more, it has provided me with a great ingress into the world of vite as a build system solution. The automatic boilerplate code that gets created when adding a new route file seems like such a quirky feature in some ways but I absolutely love it. How the routerTree.gen.ts is exposed for all those curious enough to dare open it - is a refreshing touch in the modern world of magical meta frameworks that are getting more and more complex.

Also as a massive fan of tanstack query, I am super impressed by the developer experience provided via tanstack start that allows for server side fetching while keeping the familiar useQuery API when in client side react space. Take a look at this boilerplate code provided when creating a new tanstack start app with tanstack query:

// src/routes/posts.$postId.tsx
export const Route = createFileRoute("/posts/$postId")({
  loader: async ({ params: { postId }, context }) => {
    await context.queryClient.ensureQueryData(
      postQueryOptions(postId),
     );
  },
  component: PostComponent,
}); 

function PostComponent() {
  const { postId } = Route.useParams();
  const { data: post } = useSuspenseQuery(postQueryOptions(postId));

What stuck out to me from this snippet when I first came across it is the slight departure from how I was used to using tanstack query. Specifically, I was used to defining a custom hook per query, so something like this:

const usePostById = (postId: string) => {
  return useQuery(...)
}

But a problem arises here when you want to lean into the newer features/hooks provided by the library. For example, we can't easily swap in useSuspenseQuery when the usage of useQuery has been baked into our custom query hook. This is where another seemingly simple yet genius detail of tanstack query's design comes to light:

the query object is the same API across all query related functions.

So what we changed here, is to simply define and expose just the query object, and let the relevant function/hook instantiation be determined at the instance of usage. There is even a helper function to ensure you get intelli-sense for the object parameters:

import { queryOptions } from '@tanstack/react-query'

const postQueryOptions = (postId: string) => queryOptions({
  queryKey: [...],
});

With this subtle change, we can do:

useQuery(postQueryOptions(postId));
useSuspenseQuery(postQueryOptions(postId));
ensureQueryData(postQueryOptions(postId));
// and simple overrides
useQuery({ ...postQueryOptions(postId), initialData: ... });

and this works perfectly because the queryKey is shared 🎉

Great, so where do I put this query object?

Well, similar to custom query hooks, I often find myself asking this question - where should they live? I've seen two approaches here:

1. Co-locate with usage. Put the query object near the code that uses it.

With the caveat that if it starts getting used in a second place within your application, lift up where you define it to a sensible place. Now most of the time I fall into the keep-it-simple/co-location camp but there are reasons here it just seems sub-optimal. Queries are used in features, and would therefor get colocated with features. To me, features are business logic code and should not be optimised for re-usability. I have seen much advice contrary to this so let me explain.

Lets say we have a BlogPostCardcomponent feature, the part of it we should abstract for re-usability is the Card part, and the BlogPost part is the feature implementation. I think of it as the Card being something developers/designers need to enforce as a shared UI/UX standard, and the BlogPost part being the domain of product managers. Products and features constantly change their shape and where they live, and this is where our query logic should reside (NOT in the Card), so it is inevitable to me we will have to constantly revisit moving query logic around as an application changes. And what happens inevitably is:

• Developers are averse to moving logic like this
• query logic has no clear definition point

This leads to honestly, a bit of a mess in application structure. So back to the second approach:

1. Folder structure specific to queries/API stuff

Ultimately I think this is the way to go, but the problem now becomes what folder structure?

Within this I have seen two approaches, although I am sure there are more - one to match the path parameters of the API being called to your folder structure, eg:

// for API calling /api/v2/member-data/posts/post/${postId}
/data
  /api
    /v2
      /member-data
        /posts
          /post
            /$postId.ts // contains fetcher fn, query obj etc

It's not the worst but can lead to some ugly folder structures that can be hard to traverse, and seems brittle (what happens when v2 becomes v3?). I do like that it's easy to find where the relevant code lives provided you have the API path you're looking for, but there's no guarantee the structure will be descriptive of the content being fetched (more often that not it isn't) which is a big negative. This nicely leads to a second approach, define by domain relevant to the frontend - so something like:

data/Posts.ts
data/Users.ts
data/...

and so on. At first glance, this can seem like a clean solution - but when I see the words clean and code in close proximity alarm bells go off in my head. Clean code stops being clean once real problems arise:

• Some APIs refer to posts as blogs, what should our domain be called in this case?
• These 8 different APIs all return slightly different variants of the user, how do we handle this to have a singular clean user?
• When fetching posts by user, should this live with users or posts (and vice-versa)?

I think both of these approaches have their pros and cons, what if there was some best-of-both-worlds solution here?

What about a folder structure based on query keys?

This struck me as a really neat solution for a few reasons:

• It's a solution that decouples where our query code lives from the UI code - a must have for scalable and evolving applications
• We define our own query keys, so we get the cleanliness benefits of approach #2 outlined above (provided we use descriptive patterns for query keys)
• The query key is what we care about! What I mean by this is, after defining a queryObject, our two uses for it practically end up being:
  1. To query the relevant data
  2. To invalidate said query (which requires the query key!)

And this begins to address what is really one of the only pain points I have come across with RQ: query key invalidation for a query that no longer exists. How does this happen? Let's say you have the following setup:

// .../queryKey.tsx
export const queryKeyFactory = {
  someQuery: ['some', 'query']
} as const;

// .../query.tsx
const useSomeQuery = () => {
  return useQuery({
    queryKey: queryKeyFactory.someQuery(),
  })
};

// .../mutation.tsx
const useSomeMutation = () => {
  return useMutation({
    ...,
    onSuccess: () => {
      queryClient.invalidateQueries({
        queryKey: queryKeyFactory.someQuery() 
      });
    },
  })
};

I have seen code where that useSomeQuery is removed but the associated someQuery key is left behind (it shouldn't be but it happens). There is no indication that the onSuccess handler is effectively doing nothing. To me this problem screams out for some rescuing by typescript, and I think together with vite's plugin capabilities we can solve this. Let's go one step further and lean into the pattern used for routing but apply it to configuring queries.

File based queries

Okay, let's go step-by-step. We have an API that we want to make available via RQ, lets define our folder structure based on query keys:

// queries/posts/$postId.ts

At this point (assuming the vite server is up and running), and with a shiny new vite plugin defined here:

export default defineConfig({
 plugins: [
  tanstackStart(),
  viteReact(),
  // new!
  viteDataQueryScaffoldPlugin(),
  ],
});

That kicks in and automatically populates our new ts file with boiler-plate code:

// queries/posts/$postId.ts
import { createDataQuery } from "~/lib/createDataQuery";

export const post$postIdQuery = createDataQuery("posts/$postId")({
  queryFn: ({ options: { postId }}) =>
  fetch(`/api/v2/posts/${postId}`),
});

What happens behind the scenes of createDataQuery is that our queryKey is automatically defined based on the path provided, so ["posts", postId]. It then exposes the query objects as a parameter to the callback function but it attaches the postId as options to the queryFn (which has it's type inferred from the path!). This function exposes two things:

// the query object
posts$postIdQuery.options({ postId })
// the key, ["posts", postId]
posts$postIdQuery.key({ postId });

These are type-safe, and deleting the createDataQuery instance will result in any uses of the key or options above erroring out! Going back to our original boilerplate code would look something like this:

// src/routes/posts.$postId.tsx
export const Route = createFileRoute("/posts/$postId")({
  loader: async ({ params: { postId }, context }) => {
    await context.queryClient.ensureQueryData(
      posts$postIdQuery.options({ postId }),
     );
  },
  component: PostComponent,
}); 

function PostComponent() {
  const { postId } = Route.useParams();
  const { data: post } = useSuspenseQuery(
    posts$postIdQuery.options({ postId }));

I love this solution for the following reasons:

• File-based routing is always going to be more interpretable than programmatic architecture.
• It aligns nicely with the router paradigm
• It removes the question of where should this layer of code live
• File based solutions allow for easy tap in to vite's plugin system with handleHotUpdate, allowing for next level type safety!

So what's next?

I am sharing this approach to get feedback but if there is a world where it could be internalised within tanstack start, that'd be amazing. It doesn't involve any changes internal to tanstack query, but it could give tanstack start another edge on top of other frameworks.

There would be a need for follow up features, for example it's not always as clean as /posts/$postId, how do we handle more complex use cases (eg paginated queries)? I had a stab at this with a separate dependencies parameter:

// queries/posts/$postId.ts
import { createDataQuery } from "~/lib/createDataQuery";

export const post$postIdQuery = createDataQuery("posts/$postId")({
  // define extra dependencies here
  dependencies: ["pageSize", "offset"],
  queryFn: async ({ options: { postId }, deps: { pageSize, offset } }) => {
  fetch(`/api/v2/posts/${postId}?pageSize=${pageSize}&offset=${offset}`),
});

These would then be baked into the query key internally and required when using options/keys.

I also think there could be some use-cases for a queryTree.gen.ts although I'm not 100% what they would be yet! Anyway, if you've got this far - I appreciate the read, check out the very MVP example code here: https://github.com/IainMcHugh/start-r-query