DEV Community

YukiOnishi
YukiOnishi

Posted on

Building a Tech News Aggregator with GraphQL, gRPC and Go Microservices

#graphql #grpc #go #microservices

As an engineer, I read a lot of tech articles every day: Qiita, Zenn and Hatena Bookmark in Japanese, and dev.to and Hashnode in English, plus a long list of company engineering blogs. Opening ten tabs every morning got old, so I built Check Picks, a personal tech news aggregator. It collects articles in one place and lets me bookmark them, sort them into favorite folders, and build my own custom feeds.

It's also a playground for architecture I wanted to try for real: a GraphQL BFF in front of Go gRPC microservices. This post walks through how it's structured, why I made the choices I did, and what I'd change next time.

The whole thing is open source:
👉 https://github.com/YukiOnishi1129/techpicks

The big picture 

%%{init: {"flowchart": {"curve": "basis", "nodeSpacing": 30, "rankSpacing": 55}, "themeVariables": {"fontSize": "16px"}}}%%
flowchart TB
    WEB["Web client<br/>Next.js 15 · Apollo Client · gql.tada"]

    GW["BFF<br/>NestJS · Apollo Server<br/>Supabase auth guard"]

    subgraph Services["Go gRPC microservices · Cloud Run"]
        direction LR
        CONTENT["content-service<br/>articles · feeds · OGP"]
        BOOKMARK["bookmark-service"]
        MYFEED["my-feed-service"]
        FAVORITE["favorite-service"]
    end

    subgraph Batch["batch-service · Cloud Run Jobs"]
        direction LR
        CRAWL["RSS crawlers<br/>company blogs · tech sites"]
        TREND["Trend crawler<br/>Qiita · Zenn · Hatena · dev.to · Hashnode"]
    end

    subgraph Supabase["Supabase"]
        direction LR
        AUTH["Auth"]
        DB[("PostgreSQL")]
    end

    PROTO[["checkpicks-protocol-buffers<br/>Go module + npm package"]]

    WEB == "GraphQL + JWT" ==> GW
    GW == "gRPC" ==> Services
    GW -. "verify token" .-> AUTH
    Services --> DB
    Batch --> DB
    PROTO -. "shared contract" .-> GW
    PROTO -. "shared contract" .-> Services

    classDef client fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e
    classDef bff fill:#ede9fe,stroke:#7c3aed,color:#3b0764
    classDef svc fill:#dcfce7,stroke:#16a34a,color:#14532d
    classDef job fill:#fef3c7,stroke:#d97706,color:#78350f
    classDef data fill:#f8fafc,stroke:#475569,color:#0f172a
    class WEB client
    class GW bff
    class CONTENT,BOOKMARK,MYFEED,FAVORITE svc
    class CRAWL,TREND job
    class AUTH,DB,PROTO data
    style Services fill:#f0fdf4,stroke:#16a34a
    style Batch fill:#fffbeb,stroke:#d97706
    style Supabase fill:#f8fafc,stroke:#475569

The request path is simple: Next.js → GraphQL → NestJS BFF → gRPC → Go services → PostgreSQL. Separately, batch jobs fill the database with articles. All the gRPC contracts come from one shared Protocol Buffers repo.

Everything lives in one monorepo:

Layer Tech Hosting
Web client TypeScript, Next.js 15, Apollo Client, gql.tada, shadcn/ui, Tailwind Cloud Run
Admin panel Next.js, supabase-js, shadcn/ui Cloud Run
BFF NestJS, Apollo Server, @grpc/grpc-js Cloud Run
Microservices Go, gRPC, sqlboiler Cloud Run
Batch Go, sqlboiler Cloud Run Jobs
DB & Auth Supabase (PostgreSQL + Auth) Supabase

The domain is split into content (articles, feeds, OGP, comments), bookmark, my-feed (user-defined feed folders), favorite (favorite article folders), and user.

Why GraphQL and gRPC?

The two protocols do different jobs, and that's the whole point of the setup.

  • GraphQL at the edge. The frontend has screens that need very different slices of the same data. An article card needs the platform, feeds, bookmark state, which favorite folders it's in, and my comment. GraphQL lets each screen ask for exactly that, and the schema is a great contract for frontend work.
  • gRPC between services. Inside the backend I wanted strict, versioned contracts, generated clients in both Go and TypeScript, and cheap service-to-service calls. Protocol Buffers give me all three.

The BFF is the translator between the two worlds: it takes a GraphQL request, fans out to one or more gRPC services, and shapes the result back into the GraphQL types.

One repo for all the contracts

The .proto files live in their own repository, checkpicks-protocol-buffers, which publishes generated code for both languages:

  • a Go module: checkpicks-protocol-buffers/checkpicks-rpc-go
  • an npm package: @checkpicks/checkpicks-rpc-ts

So the Go services and the NestJS BFF import the same contract at a pinned version (currently v1.5.5 on both sides):

import (
    bpb "github.com/YukiOnishi1129/checkpicks-protocol-buffers/checkpicks-rpc-go/grpc/bookmark"
    cpb "github.com/YukiOnishi1129/checkpicks-protocol-buffers/checkpicks-rpc-go/grpc/content"
)
Enter fullscreen mode Exit fullscreen mode 
import { ContentServiceClient } from '@checkpicks/checkpicks-rpc-ts/src/grpc/content/content_grpc_pb';
Enter fullscreen mode Exit fullscreen mode

Changing an API means: edit the proto, release a new version, bump it in each consumer. It's a little ceremony, but the compiler tells me every place that broke, and nothing goes out of sync by accident.

The BFF: NestJS + Apollo Server

The GraphQL schema is written schema-first in .graphql files, grouped by domain (article, bookmark, feed, favorite, myFeed, …). NestJS generates TypeScript classes from them, so resolvers stay type-safe:

type Query {
  articles(articlesInput: ArticlesInput!): ArticleConnection!
  articleOpg(articleUrl: String!): ArticleOGP!
}

type Article implements Node {
  id: ID!
  platform: Platform
  feeds: [Feed!]
  favoriteArticleFolderIds: [String!]!
  title: String!
  articleUrl: String!
  isBookmarked: Boolean!
  comment: ArticleComment
  # ...
}
Enter fullscreen mode Exit fullscreen mode

Lists use Relay-style connections (edges, node, cursor, pageInfo), which pair nicely with infinite scroll on the client.

Authentication at the edge

Auth is handled by Supabase. The client sends the Supabase access token, and a NestJS guard verifies it once, at the BFF:

@Injectable()
export class SupabaseAuthGuard implements CanActivate {
  private supabase = createClient(process.env.SUPABASE_URL, process.env.SUPABASE_ANON_KEY);

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const ctx = GqlExecutionContext.create(context).getContext();
    const req = ctx.req || context.switchToHttp().getRequest();
    const token = req.headers.authorization?.split(' ')[1];
    if (!token) throw new UnauthorizedException('Authorization token not found');

    const { data: { user }, error } = await this.supabase.auth.getUser(token);
    if (error || !user) throw new UnauthorizedException('Invalid or expired token');

    req.user = user;
    return true;
  }
}
Enter fullscreen mode Exit fullscreen mode

After that, the resolver passes user.id down to the gRPC services as an explicit request field. The Go services never deal with JWTs; they trust the BFF and focus on business logic.

@Query(() => ArticleConnection)
@UseGuards(SupabaseAuthGuard)
async articles(@Args('articlesInput') input: ArticlesInput, @Context() context: GraphQLContext) {
  return this.articleService.getArticles(context.req.user.id, input);
}
Enter fullscreen mode Exit fullscreen mode

Calling gRPC from NestJS

Each downstream service gets a small injectable wrapper around its generated client. It uses plaintext locally and TLS in staging/production (Cloud Run gives every service an HTTPS endpoint), with a 10-second deadline:

const grpcCredentials =
  process.env.NODE_ENV !== 'production' && process.env.NODE_ENV !== 'staging'
    ? grpc.credentials.createInsecure()
    : grpc.credentials.createSsl();

@Injectable()
export class GrpcContentClientService implements OnModuleInit {
  private grpcContentService: ContentServiceClient;

  onModuleInit() {
    this.grpcContentService = new ContentServiceClient(grpcUrl, grpcCredentials, { deadline: 10000 });
  }

  getGrpcContentService() {
    return this.grpcContentService;
  }
}
Enter fullscreen mode Exit fullscreen mode

The domain services (ArticleService, BookmarkService, …) build the protobuf request, call the client, and map the response to GraphQL types. Proto3 has no "optional" for scalars by default, so I use the well-known wrapper types (StringValue, Int64Value) to tell "not set" apart from "empty":

const req = new ListArticleRequest();
if (input?.first) req.setLimit(input.first);
if (input?.after) req.setCursor(input.after);
if (input?.languageStatus) req.setLanguageStatus(new Int64Value().setValue(input.languageStatus));
req.setUserId(new StringValue().setValue(userId));
Enter fullscreen mode Exit fullscreen mode

The Go microservices

Every service follows the same layered structure, so jumping between them feels the same:

content-service/
├── cmd/main.go                     # wiring (manual DI)
└── internal/
    ├── interfacess/handler/        # gRPC handlers
    ├── application/usecase/        # business logic
    ├── domain/                     # domain types + repository interfaces
    ├── adapter/
    │   ├── persistence_adapter/    # builds queries for the use cases
    │   └── external_adapter/       # wraps calls to other services
    └── infrastructure/
        ├── persistence/            # sqlboiler-backed repositories
        └── external/               # gRPC clients to other services
Enter fullscreen mode Exit fullscreen mode

There's no DI framework. main.go wires everything by hand, layer by layer, which is wordy but very easy to follow:

// infrastructure layer
aps := persistence.NewArticlePersistence(db)
bex := external.NewBookmarkExternal(bClient)

// adapter layer
apa := persistenceadapter.NewArticlePersistenceAdapter(aps)
bea := externaladapter.NewBookmarkExternalAdapter(bex)

// ...then use cases, then handlers, then grpc.NewServer()
Enter fullscreen mode Exit fullscreen mode

Handlers are deliberately thin: they just delegate to the use case.

func (ch *contentHandler) ListArticle(ctx context.Context, req *cpb.ListArticleRequest) (*cpb.ListArticleResponse, error) {
    return ch.contentUseCase.ListArticle(ctx, req)
}
Enter fullscreen mode Exit fullscreen mode

sqlboiler for data access

I chose sqlboiler because it's database-first: the schema is managed by Supabase migrations, and sqlboiler generates type-safe Go models straight from the real tables. Queries are built with query mods, which stay readable even when they get big:

q := []qm.QueryMod{
    qm.InnerJoin("platforms ON articles.platform_id = platforms.id"),
    qm.LeftOuterJoin("feed_article_relations ON articles.id = feed_article_relations.article_id"),
    qm.Load("FeedArticleRelations.Feed.Category"),
    qm.Where("articles.is_private = ?", false),
    qm.Limit(limit),
}

// cursor-based pagination
if req.GetCursor() != "" {
    q = append(q, qm.Where(
        "articles.published_at < (SELECT published_at FROM articles WHERE id = ?)",
        req.GetCursor(),
    ))
}
Enter fullscreen mode Exit fullscreen mode

Service-to-service calls

Some services call each other over gRPC too. For example, content-service asks bookmark-service and favorite-service about the articles it returns. These calls go through the external / external_adapter layers, so the use cases only depend on an interface and are easy to mock in tests.

Real-database tests with Testcontainers

For repository and use-case tests I didn't want to mock SQL. Each test starts a throwaway PostgreSQL 16 container with testcontainers-go, loads the schema, and runs against a real database:

pgContainer, err := postgres.Run(ctx, "docker.io/postgres:16-alpine",
    postgres.WithInitScripts(scripts...),
    postgres.WithDatabase("postgres"),
    postgres.WithUsername("root"),
    postgres.WithPassword("password"),
    testcontainers.WithWaitStrategy(
        wait.ForLog("database system is ready to accept connections").
            WithOccurrence(2).
            WithStartupTimeout(5*time.Second)),
)
Enter fullscreen mode Exit fullscreen mode

It's slower than mocks, but it catches the bugs that actually happen with SQL, like a wrong join or a missing deleted_at IS NULL.

Batch: where the articles come from

The batch-service is a set of Go commands deployed as Cloud Run Jobs and run on a schedule:

  • article-company-crawler: pulls RSS feeds from company engineering blogs
  • article-site-and-summary-crawler: pulls from tech sites and summary/curation sites
  • trend-article-crawler: collects trending articles and their like/bookmark counts from Qiita, Zenn, Hatena Bookmark, dev.to and Hashnode
  • migrate-seed: seeds platforms and feeds

Each source has its own repository (qiita_repository.go, zenn_repository.go, devCommunity_repository.go, …) behind a common interface, so adding a new platform mostly means writing one more repository. The crawlers also fetch OGP metadata so every card gets a proper thumbnail.

Jobs fit this much better than an always-on service: they start, crawl, write to Postgres, and exit, and I pay only for the time they run.

The frontend: Next.js + Apollo + gql.tada

The web client uses the Next.js App Router with @apollo/experimental-nextjs-app-support, so the same Apollo setup works in Server Components and Client Components.

For type safety I point GraphQL Code Generator directly at the BFF's schema files in the monorepo:

const config: CodegenConfig = {
  schema: "../../bff/apollo-gateway/src/schema/**/*.graphql",
  documents: ["./src/**/*.tsx", "./src/**/*.ts", "!node_modules/"],
  generates: {
    "src/graphql/type.ts": {
      plugins: ["typescript", "typescript-operations", "typescript-react-apollo"],
    },
  },
};
Enter fullscreen mode Exit fullscreen mode

On top of that, gql.tada infers result types from the query string itself and supports fragment colocation, so each component declares the fields it needs right next to the JSX that uses them. If someone changes the schema in the BFF, the frontend fails to type-check in the same PR. That's one of the nicest benefits of a monorepo.

The code is organized by feature (features/articles, features/bookmarks, features/favorites, features/myFeeds, features/trends, …) rather than by technical layer.

Local development & CI/CD

Locally, Supabase CLI runs Postgres and Auth, and Docker Compose starts the BFF and all Go services on the same Docker network. Go services use Air for hot reload. A Makefile wraps the common commands:

make supabase-start   # local Postgres + Auth
make dcb && make dcu  # build & start BFF + services
make migrate          # apply migrations
Enter fullscreen mode Exit fullscreen mode

CI is one GitHub Actions workflow per app, filtered by path, so a change to bookmark-service only runs bookmark checks. Go workflows run golangci-lint, go test (with Testcontainers), and go build.

CD uses a cloudbuild.yaml per service: build the image, push to Artifact Registry, gcloud run deploy. Same pattern for every service, which keeps the pipeline boring in a good way.

What I'd do differently

This was a learning project, and some trade-offs are worth sharing honestly.

1. It's microservices, but with a shared database. All services connect to the same Supabase Postgres. That kept things simple (joins still work, one migration history), but it means the services aren't truly independent: a schema change can affect several of them. For a solo project it was the right call. For a team, I'd move toward a schema per service, or at least strict table ownership.

2. Mapping protobuf ↔ GraphQL by hand is a lot of code. The BFF has plenty of toObject() and field-by-field mapping. Next time I'd look at ts-proto for plain TypeScript interfaces, or Connect for a nicer client API, or generate the mapping code.

3. Watch out for N+1 across services. GraphQL makes it easy to ask for nested data, and every nested field could become another gRPC call. I mostly avoided this by making the gRPC responses "fat" (the content service already returns bookmark and favorite info). In a bigger system I'd add DataLoader-style batching in the BFF.

4. Is this overkill for a personal app? Honestly, yes. A single Next.js app with Supabase would have shipped faster. But the goal was to learn how these pieces fit together in production, with Cloud Run, real auth, real CI/CD and real data, and it did that very well.

Wrapping up

The pattern I'd keep is GraphQL at the edge, gRPC inside, one shared proto repo for contracts, and a monorepo that ties types together end to end. Each part does one job, and the type safety flows from the .proto files and the .graphql schema all the way to the React components.

If you're curious, the full source is here:
👉 https://github.com/YukiOnishi1129/techpicks

Questions or ideas? Leave a comment. I'd especially like to hear how others handle the proto ↔ GraphQL mapping layer. And if you found this useful, a ⭐ on the repo would make my day!

Top comments (0)