DEV Community: ElshadHu

How I Use AST Diffing and LLMs to Keep Docs in Sync with Code

ElshadHu — Sat, 07 Mar 2026 06:05:46 +0000

Oh Here We Go Again: Documentation

Since I started coding, creating projects, and contributing to projects, what I've realized so far is that outdated docs waste more time than missing docs. For that reason, I decided to build a CLI tool called mark-guard which detects Go code changes and updates your markdown docs using AST diffing and LLMs. I'm planning to add support for other languages.

Which One Is More Painful: Creating from Scratch or Modifying the Docs

From my point of view, modifying is harder than creating from scratch, and that's what my tool tries to solve. Its purpose isn't creating docs from scratch but rather modifying them while we modify the project. Currently, I'm using my tool along with my friends. I think it'll really shine once people try it on bigger codebases with lots of legacy docs.

Core Architecture

When I started building this project, my first thought was a full AST tree approach. I had worked on something similar with my friend in wtf-script, but for this tool that was over-engineering it. I did not need to understand every node in the tree. I needed to know what the public API looks like before and after a code change.

After researching, I went with Go's built-in go/parser package. Parse the old version of a file (from git) and the new version (from disk), extract all exported symbols from both, compare them, and feed the structured diff to an LLM that updates the docs.

How Symbol Extraction Works

The entry point is ExtractSymbols


// ExtractSymbols parses Go source code and returns all exported symbols.
func ExtractSymbols(filename, src string) ([]Symbol, error) {
    fset := token.NewFileSet()
    f, err := parser.ParseFile(fset, filename, src,
        parser.SkipObjectResolution|parser.ParseComments)
    if err != nil {
        return nil, fmt.Errorf("parse %s: %w", filename, err)
    }

    var symbols []Symbol
    for _, decl := range f.Decls {
        switch d := decl.(type) {
        case *ast.FuncDecl:
            if sym, ok := extractFunc(fset, d); ok {
                symbols = append(symbols, sym)
            }
        case *ast.GenDecl:
            symbols = append(symbols, extractGenDecl(fset, d)...)
        }
    }
    return symbols, nil
}

token.NewFileSet(): Tracks line numbers and byte offsets for every token. Needed when I print AST nodes back as Go source.

parser.ParseFile(fset, filename, src, flags): Takes raw Go source, returns *ast.File with all top-level declarations (functions, types, constants, variables, imports).

parser.SkipObjectResolution: I don't need identifier resolution. I'm only reading declarations, not analyzing usage. Skipping it makes parsing faster.

parser.ParseComments: I want doc comments so I can include them in the diff context.

Walking Declarations

f.Decls contains every top-level declaration. I handle two types:

*ast.FuncDecl: Functions and methods. I skip unexported ones. For methods, I grab the receiver type and name them Receiver.Method.
*ast.GenDecl: Everything else: structs, interfaces, type aliases, type definitions, consts, vars. We skip imports. Each gets classified by kind and we extract fields, methods, or values accordingly.

Where Comparison Happens

The diffing lives in diff.go. Diff() takes two symbol slices (old and new) and runs three passes:

Pass 1: Added    (in new, not in old)
Pass 2: Removed  (in old, not in new)
Pass 3: Modified (in both, but changed)

I index both slices by name into maps so lookups are fast. For modified symbols, compareSymbols() checks:

Kind change (e.g. struct to interface)
Parameters added, removed, or type changed
Return values changed
Struct fields added or removed
Interface methods changed

// Diff compares old and cur symbol slices and returns a list of changes
func Diff(old, cur []Symbol) []SymbolDiff {
    oldMap := indexByName(old)
    curMap := indexByName(cur)
    var diffs []SymbolDiff

    // Pass 1: added (in cur not in old)
    for name, sym := range curMap {
        if _, exists := oldMap[name]; !exists {
            diffs = append(diffs, SymbolDiff{
                Name:   name,
                Kind:   ChangeAdded,
                Symbol: sym,
            })
        }
    }


    // Pass 2 and 3 follow the same pattern for removed and modified...

    // Then sorting happens ...
    return diffs
}

The result is a sorted []SymbolDiff (added first, then removed, then modified, alphabetical within each group) formatted into either a human-readable summary (FormatDiffSummary) or a compact version for the LLM (FormatDiffSummaryCompact):

// FormatDiffSummaryCompact is a terse version of FormatDiffSummary.
// It omits doc comments and per-field descriptions to reduce LLM input tokens.
func FormatDiffSummaryCompact(diffs []SymbolDiff) string {
    if len(diffs) == 0 {
        return "No changes to exported symbols"
    }
    var sb strings.Builder
    for i := range diffs {
        switch diffs[i].Kind {
        case ChangeAdded:
            fmt.Fprintf(&sb, "+ ADDED   %s: %s\n", diffs[i].Name, diffs[i].Symbol.Signature)
        case ChangeRemoved:
            fmt.Fprintf(&sb, "- REMOVED %s: %s\n", diffs[i].Name, diffs[i].OldSignature)
        case ChangeModified:
            fmt.Fprintf(&sb, "~ CHANGED %s\n  was: %s\n  now: %s\n",
                diffs[i].Name, diffs[i].OldSignature, diffs[i].Symbol.Signature)
        }
    }
    return strings.TrimSpace(sb.String())
}

Making the Prompt Work: Restrictions Over Flexibility

Let me tell you one thing about prompts: you can't know what's gonna happen until you test it. When I first wrote the prompt for this project, the LLM rewrote entire documents instead of making small edits. It didn't work because of lack of detail and lack of experiments.

Then I found this documentation which explains how XML structure makes the prompt more explicit and reduces the misinterpretation. So, I created sanitize.go for wrapping texts with XML structure:

// WrapRole wraps the role in <ROLE>
func WrapRole(role string) string {
    return "<ROLE>\n" + role + "\n</ROLE>"
}

// WrapContext wraps the context in <CONTEXT>
func WrapContext(ctx string) string {
    return "<CONTEXT>\n" + ctx + "\n</CONTEXT>"
}

// same style wrapping functions exist...

then I wrote buildSystemPrompt which combines all these functions:

func buildSystemPrompt() string {
    return strings.Join([]string{
        WrapRole(roleText),
        WrapContext(contextText),
        WrapScale(scaleText),
        WrapRules(rulesText),
        WrapTone(toneText),
        WrapEdgeCases(edgeCasesText),
        WrapExamples(examplesText),
    }, "\n\n")
}

Each section has a specific job:

Role: Tell the LLM what it is (a documentation editor, not a creative writer)
Context: What's happening (code changed, docs need updating)
Scale: How big the project is (so it doesn't over-generate)
Rules: What it can and can't do (no deleting sections, no adding features that don't exist)
Tone: Match the existing doc style
Edge Cases: What to do when nothing needs changing (return empty edits)
Examples: Show it what good output looks like

Maybe it's not the best prompt out there, but out of all my experiments this is what worked. What I realized is that giving the LLM restrictions rather than flexibility made the output more predictable.

Bottlenecks that I didn't think about when Starting:

The first version of the pipeline destroyed my README. It asked the LLM to return the whole file, the response got truncated, and WriteUpdate overwrote the original with a partial copy. I moved to edit-based JSON output where the LLM returns small replace/insert_after/delete operations instead of rewriting files.

// The LLM only emits the exact bytes that need to change, never the full file
type Edit struct {
    File    string `json:"file"`              // relative doc path
    Section string `json:"section,omitempty"` // nearest heading, context only, not used for matching
    Action  string `json:"action"`            // "replace" | "insert_after" | "delete"
    OldText string `json:"old_text"`          // text that must exist in the file
    NewText string `json:"new_text"`          // text to write in its place
}

The second problem was invisible. Parse errors during symbol extraction were silently swallowed, which made every symbol look "added" instead of "modified." The diff was wrong and the LLM was confused. I added warning logs so failures are visible.

The third was trust. The pipeline wrote to disk immediately with no preview. I added --write (default dry-run), content-loss validation that blocks updates losing >50% of a file, and --force for when you know better.

Git Integration: No Extra Dependencies

Anyone using this tool already has git installed. That is why I shell out to git directly with os/exec instead of pulling in go-git (which brings transitive dependencies). All git commands are defined as constants:

const (
    gitCmdDiff     = "diff"
    gitCmdLsFiles  = "ls-files"
    gitCmdShow     = "show"
    gitCmdRevParse = "rev-parse"
    // other commands...
)

Configuration

Everything is driven by .markguard.yaml at the repo root. Two sections: llm for the LLM provider and docs for which files to scan.

llm:
  base_url: "https://generativelanguage.googleapis.com/v1beta/openai"
  api_key_env: "GEMINI_API_KEY"
  model: "gemini-2.5-flash"
docs:
  paths:
    - "README.md"
  exclude:
    - "docs/roadmap.md"
    - "docs/day*.md"
  mappings:
    - docs: ["docs/api.md"]
      code: ["internal/git/", "internal/config/"]

The Go structs map directly to the YAML:

// Config is the top level configuration
type Config struct {
    LLM  LLMConfig  `yaml:"llm"`
    Docs DocsConfig `yaml:"docs"`
}

// LLMConfig holds settings for LLM provider
type LLMConfig struct {
    BaseURL   string `yaml:"base_url"`
    APIKeyEnv string `yaml:"api_key_env"`
    Model     string `yaml:"model"`
}

// DocsConfig holds settings for documentation scanning
type DocsConfig struct {
    Paths    []string           `yaml:"paths"`
    Exclude  []string           `yaml:"exclude"`
    Mappings []model.DocMapping `yaml:"mappings"`
}

If the file doesn't exist, Load() just returns defaults: Gemini as the provider, README.md and docs/ as the paths. You don't need to create a config file to get started. I used yaml.v3 directly instead of Viper because Viper drags in a ton of transitive dependencies for something I can do in 5 lines with yaml.Unmarshal.

The mappings field is what controls token usage. Without it, every doc gets sent to the LLM on every run. With mappings, only docs mapped to the changed code paths get sent. I tested it on the cobra project by changing the overall structure to see how it would handle the docs, and it worked pretty well, only using around 15K tokens even for architectural changes.

What are Next Steps

Right now mark-guard only works with Go, but the architecture is language-agnostic. The parser is isolated in one package. If you wanted to add Python, Rust, or TypeScript support, you'd write a new extractor and plug it in. The rest of the pipeline stays the same. Where I need help right now:

Prompt tuning: The --debug flag prints the full prompt and raw LLM response. I've been running it on projects to see where the output breaks. If you run it on your project and the edits are wrong, that debug output helps me figure out why.
More test coverage: There are open issues for unit tests on edit parsing, doc scanning edge cases, and pipeline integration tests.

If you have ideas, questions, or want to work on something, open an issue. You don't need a full plan, just an idea is enough and we can work on it.

Savior: Low-Level Design

ElshadHu — Fri, 13 Feb 2026 07:04:39 +0000

Grinding Go: Low-Level Design

I went back to the drawing board for interview preparation and to sharpen my problem-solving skills. Software development is in a weird stage. 2 weeks ago, when I saw my friend doing low-level-design, I thought that in current stage it is meaningless. How wrong I was. My friend started asking me about problem solving and critical thinking. I realized that I am killing my skills by only focusing on abstraction nowadays and learning new tools that maybe in a new job I will never use. For that reason, I did research and saw that a pretty good amount of devs are complaining about how inline suggestions on IDE and AI tools kill their problem-solving skills. I watched this video which is related to current interview style by NeetCode. I realized that nothing has changed pretty much and problem-solving skills stay as the strongest skills. That's why, I create a new repository: grinding-go.

Self Reflection

My friend and I have been starting to interview each other for low-level-design and system design. I noticed that my critical thinking has got softer. While making our first interviews, I was mumbling. I wasn't giving details because of my half-formed ideas. So, I planned a strategy for this current stage of my coding skills: Do low-level design barehand and ask LLM to find articles about the design and make it ask socratic follow-up questions for my design in order to sharpen my problem solving skills. Therefore, I went back to the drawing board to do low-level design for problems that are abstracted away and are just big word alerts. I also realized again, algorithms and computational thinking don't go anywhere, they just stay under abstractions in large codebases which if I don't keep my skills up to date with them, I'll struggle a lot.

The Main Resource

I created grinding-go project in Go because I love this language for its simplicity and speed. I started reading the summary of 100 Go Mistakes and coding snippets which are in this repo via this website. I realized as a developer, I need to come back to my basics and make them as strong as possible by doing also boring stuff. After that book, I would definitely say that I stopped treating Go like other languages because it has its own style especially when it comes to error handling and concurrency.

First Technical Challenge: Implement LRU Cache

When it comes to my core structure, I used LRUCache struct which contains LinkedList and mutex for thread-safety and concurrency:

type LRUCache struct {
    mu       sync.RWMutex
    capacity int
    cache    map[int]*Node
    list     *LinkedList
}

Here, the main purpose is to make a quick look-up with map and when we change the position of the least recently used item we use Linkedlist which doesn't take O(n). Also we add a new item into the lru cache if it passes the capacity we need to remove the tail of the linked list which takes O(1) operation.

if len(lru.cache) > lru.capacity {
        tailKey, err := lru.list.removeTail()
                // ...
}

While solving this problem, I forgot to nil out the removed element's pointers from the linkedList. Now in Go this is not a "dangling pointer" like in C/C++ since Go has a garbage collector. But it still matters because the removed node can hold references to other nodes in the list which prevents the GC from freeing memory that should be freed. It can also cause logical bugs if you accidentally traverse through stale references:

// clear dangling pointers
    n.prev = nil
    n.next = nil

Meanwhile handling the neighbours of linked list elements, I noticed that I am making mistakes and losing the references to elements. I wrote everything in one file which reminded me of solving problems in one file rather than abstracting away everything. I first heard this approach from ThePrimeAgen: solve it in one file first, then split later. LRU Cache is everywhere in production anyway. What I don't get is when people tell that solving these problems is dead. It isn't dead. In production and large codebases the same patterns and data structures are being used with over abstraction and if you don't know at low level, it will be more difficult.

Rate Limiter

When I started this problem, I directly thought about Sliding window technique but I already used this technique in different projects and I wanted to do it with Token Bucketing and for that reason I chose it. At the end of the day, we as developers have to challenge ourselves and see how it goes. I already got the basic idea of it and meanwhile solving it I made searches about the main difference between Token Bucketing and Sliding window for checking out the trade-offs again. I watched this video and started implementing it. I implemented a RateLimiter struct which has a composition relationship with TokenBucket:

type RateLimiter struct {
    buckets   map[string]*TokenBucket // maps a user/key to their bucket
    mu        sync.Mutex              // Protects concurrent map access
    rate      float64                 // default rate for new buckets
    maxTokens float64                 //default burst size for new buckets
}

I also wrote in TokenBucket struct a mutex field because it has to handle its own operation concurrently and I implemented Allow method which controls how many requests a client can make during a specific time frame:

func (tb *TokenBucket) Allow() bool {
    tb.mu.Lock()
    defer tb.mu.Unlock()

    now := time.Now()
    tb.lastSeen = now
    newTokens := now.Sub(tb.lastRefill).Seconds() * tb.refillRate
    tb.tokens += newTokens
    if tb.tokens > tb.maxTokens {
        tb.tokens = tb.maxTokens
    }
    tb.lastRefill = now
    if tb.tokens >= 1 {
        tb.tokens--
        return true
    }
    return false
}

After doing that I was happy and thought that I'm done but when it comes to socratic question I saw that I'm not cleaning up idle buckets in RateLimiter. Without cleanup, if you have millions of unique keys hitting your limiter, those buckets just sit in memory forever. So I added a cleanUpLoop that periodically evicts stale entries:

func (rl *RateLimiter) cleanUpLoop(interval, maxIdle time.Duration) {
    ticker := time.NewTicker(interval)

    for range ticker.C {
        rl.mu.Lock()
        for key, tb := range rl.buckets {
            if time.Since(tb.lastSeen) > maxIdle {
                delete(rl.buckets, key)
            }
        }
        rl.mu.Unlock()
    }
}

Since I mentioned the trade-offs between token bucket and sliding window, here is a short comparison:

Token Bucket:

Tokens refill at a fixed rate, each request costs one token
Allows bursts up to bucket size which is good for APIs like payment or webhook endpoints
Low memory, just a counter and a timestamp

Sliding Window:

Tracks requests in a time window and counts them
Smoother and stricter rate control, no big bursts
Higher memory since you need to store timestamps or counters per sub-window

Pub/Sub

As you already know in system design interviews especially when it comes to microservices architecture Pub/Sub is mentioned a lot. Because of its asynchronous nature and substantial workload with rest APIs pub/sub makes sense to use to reduce tight coupling and solve the problem by also keeping the performance good. But, I never thought about its implementation details and when it comes to learning anything, my strategy is to write the code with my naive solution and research about it during building my solution. I watched this video by Hussein Nasser which really helped me a lot. Especially those who want to prioritize concepts over tools, I highly recommend this person's videos.

Types

there are 4 core structs which are:

Topic: this groups related messages
Subscriber: this is the one receives messages.
Broker: knows which subscribers care about which topics and route messages to them
Message: is just the data

Broker plays a middle man role and its methods delegate to Topic methods and return their errors directly. When I first wrote Unsubscribe method, I used Broker's mutex via defer which was a poor decision because the lock was only needed for the topic lookup not for topic-level operations. Holding the broker lock while doing topic operations would block all other broker methods unnecessarily. Then I changed it properly:

func (b *Broker) Unsubscribe(topicName string, sub *Subscriber) error {
    b.mu.Lock()
    t, ok := b.topics[topicName]
    b.mu.Unlock() // release broker lock before touching topic
    if !ok {
        return ErrTopicNotFound
    }
    return t.RemoveSubscriber(sub.id)
}

Also, in the Broadcast method I made sure to pass the subscriber as a parameter to the goroutine. Without this, it becomes a closure bug where all goroutines can end up referencing the last value of the loop variable. Since Go 1.22, the loop variable scoping changed and each iteration gets its own copy. So this specific closure bug is fixed in Go 1.22+. But passing the variable as a parameter is still a good habit, it makes intent clear and keeps your code compatible with older versions. You can read about it in the official Go blog post.

    for _, sub := range t.subscribers {
        // yo I need to  use subscriber as a parameter because in another case it is closure bug
        // Because  by the time goroutine runs, the for loop may have moved and sub points to the last sub
        go func(s *Subscriber) {
            select {
            case s.ch <- msg:
            default:
            }
        }(sub)

In this problem, I made a PR to my own project and made my friend to review it in order to get a different perspective and he gave me feedback. At the end of the day, these are just tools and I should focus more on architecture rather than some fancy words about tools.

Lastly

I will keep this as a series and after each 3 low-level-design, I will come with my naive solutions to these big word alerts. I reminded myself of my engineering skills by coming back to the drawing board, thinking about the problems in low level and understanding trade-offs while implementing it. Nowadays my friend and I are so busy with job-hunting and we are planning to write about our project verdis. In my previous blog, I mentioned that we had our first podcast which looked like we filmed it with a fridge camera but we already did our second podcast and this time is way better :). I added a new technique into my formula besides building in open and contributing also solving the problems with low level design. If you are curious about it and if you see anything in my solutions feel free to give any suggestions and if you know any technical challenges to recommend, feel free to create issues in grinding-go.

Communication Beats AI Slop in Open Source

ElshadHu — Mon, 19 Jan 2026 03:40:31 +0000

Recently, I started issue hunting again and improving my knowledge in different parts of programming such as networking, concurrency and LLM engineering. What I realized in open source is that it might become dead because of AI slop and lack of communication between maintainers and contributors. This blog post gives information about how communication beats AI slop. I'll represent my 2-3 PRs that I did recently and I'll iterate through the process. Nowadays, I contributed to next-ai-draw-io and streamplace meanwhile I will mention how I experienced being a maintainer of an open source project which got popularity thanks to you guys :). That is repo-health which I talked about in my previous blog post.

Communication Matters More Than You Think

Even if I know all tricks in one language and use idiomatic syntax it does not mean anything if I am not capable of coworking with other developers. When I contributed to next-ai-draw-io I wrote under issue that I can tackle it and I got assigned.

Nowadays, I'm sticking to 1-2 projects instead of jumping around other projects. Why? Because spending less time on issues without understanding technical depth and architecture means you just submit shallow PRs that create more issues. I want to actually understand the codebase and architecture by playing around the same projects.

First Attempt at Adding Vertex AI Support

I was trying to activate Vertex AI on the server side via environment variables instead of handling it through the API.

 const vertexProject = process.env.GOOGLE_VERTEX_PROJECT
         const vertexLocation =
                process.env.GOOGLE_VERTEX_LOCATION || "us-central1"

I was handling it in the router without API authentication like some of other models:

 else if (
            provider !== "ollama" &&
            provider !== "edgeone" &&
            provider !== "vertexai" &&
            !apiKey
        )

I documented everything and made a PR. I faced AI review from GitHub Copilot, which was fair because it caught naming conventions and small mistakes. This makes sense for initial review before maintainers spend time on it. I might use this pattern in my own projects only as first review. But I was missing one crucial thing: I was only handling it server-side. Even though my changes were well documented, they weren't suitable for this feature.

Maintainer Told Me to Switch Approach

I got nice feedback from one of the maintainers to make project id and location configurable. So, I switched to this and added extra fields to ValidateRequest interface in route.ts:

interface ValidateRequest {
    provider: string
    apiKey: string
    baseUrl?: string
    modelId: string
    // AWS Bedrock specific
    awsAccessKeyId?: string
    awsSecretAccessKey?: string
    awsRegion?: string
    // Vertex AI specific
    vertexProject?: string
    vertexLocation?: string
}

I also changed the Vertex AI case to allow client-provided project/location OR fallback to environment variables:

case "vertexai": {
    // OR client-provided project/location
    const project =
        vertexProject || process.env.GOOGLE_VERTEX_PROJECT
    const location =
        vertexLocation ||
        process.env.GOOGLE_VERTEX_LOCATION ||
        "us-central1"

    if (!project) {
        return NextResponse.json(
            {
                valid: false,
                error: "Project ID is required (set in Settings or via GOOGLE_VERTEX_PROJECT)",
            },
            { status: 400 },
        )
    }

    const vertex = createVertex({
        project: project,
        location: location,
    })

    model = vertex(modelId)
    break
}

So, maintainer who asked me to do this change told that this PR is solid and ready to merge as is and there was another issue that is not related to that PR. He recalled the creator of the project and he gave me strong feedback with instructions to add vertex-ai via API key instead.

Got Stuck on Authentication

After getting the last feedback, I didn't lose my interest on that PR because of clear instructions and maintainers are always in contact with you and try to help you. For the record, I highly suggest you to read this post which talks about ego olympics how some developers make code reviews in a brutal way. I started reading about express-mode and thinking model. When I started, I got no idea how to do that in that codebase, exactly no idea. I got stuck and wrote under the issue that I am not sure how to handle authentication error because 1/3 times it was failing when I tested API.

Maintainer explained to me how I have to wire up and fix the issue. I needed to pass the API key through headers in the frontend. I added to chat-panel.tsx:

            // Vertex AI credentials (Express Mode)
              ...(config.vertexApiKey && {
                   "x-vertex-api-key": config.vertexApiKey,
                 }),

Then I had to update the model creation logic in route.ts to use the Vertex AI properly:

 export async function POST(req: Request) {
            awsAccessKeyId,
            awsSecretAccessKey,
            awsRegion,
            // Note: Express Mode only needs vertexApiKey
            vertexApiKey,
        } = body

After all the changes my PR got accepted and I am willing to contribute more to this project and learn more. Meanwhile searching about that repo I used my tool repo-health and see how maintainers make fair decisions and pay attention to PRs. Also, I saw PRs that haven't been merged yet (both open and closed ones) via my tool and visited them to understand the patterns that made them unsuitable for merging.

Contributing to streamplace

Software development is not easy and has never been easy. Nowadays, I explored streamplace again which I talked about this project in one of previous blog posts. I added a scroll button to chat message with animation for issue in PR and exactly just one scroll button.

We have to think that when we contribute maintainers don't want the code that isn't documented and solves the problem in a vague way because of not paying attention to details in implementation. Believe me, they can do the vague way of solving within 5 minutes but they respect their software which means also we have to do. I read a blog post which gives info how github graph does not mean anything and I absolutely agree. Making one PR to the critical software and solving one weird bug is more important than anything for me.

Anyway, when it comes to contribution, I implemented animation.

 const flatListRef = useRef<FlatList>(null);

  // Animation for scroll-to-bottom button
  const buttonOpacity = useSharedValue(0);
  const buttonTranslateY = useSharedValue(20);

  useEffect(() => {
    buttonOpacity.value = withTiming(isScrolledUp ? 1 : 0, { duration: 200 });
    buttonTranslateY.value = withTiming(isScrolledUp ? 0 : 20, {
      duration: 200,
    });
  }, [isScrolledUp]);

  const buttonAnimatedStyle = useAnimatedStyle(() => ({
    opacity: buttonOpacity.value,
    transform: [{ translateY: buttonTranslateY.value }],
  }));

  const scrollToBottom = () => {
    flatListRef.current?.scrollToOffset({ offset: 0, animated: true });
  };

Why I chose useRef? Because I needed a reference to the FlatList component that persists across re-renders. If I stored this in state, every state update would cause unnecessary re-renders. useRef gives me a mutable reference that doesn't trigger re-renders when it changes.

Second fix in streamplace

This one could seem boring and easy, but I thought about it and analyzed the project :). I just fixed the issue by making a PR that I added one line of code, but sometimes it is just the way it is and I documented it to be clear even for one line :).

{ overflow: "hidden" }

What I Changed in repo-health

I got nice feedback for repo-health and stars that I was not expecting :). Since my last blog, I tried to fix bugs in that repo and create issues and a discussion to make it more engaging.

Because of the traffic after the blog, I fixed the rate limiting in a temporary way by only allowing one repository analysis per hour when unsigned, but when signed you can search as much as you want. I know it's not ideal right now, but that was required at that time, and I couldn't think of a better option.

I recently integrated ungh, which helps reduce GitHub API requests through intelligent caching and API call optimization. You can read their README to see how it works, but basically it makes fewer requests to GitHub API, which helps with rate limits.

Also, I applied to the GitHub Developer Program and I am now a GitHub Developer Program Member. This gives me access to additional tools and GitHub Actions workflows, which makes development easier.

I removed the dependency section and added setup insights for keeping the project's purpose clear. During the way, I used the Vercel AI SDK:

  const context = buildContext(files, issues);
  const prompt = buildPrompt(context);

  try {
    const response = await getOpenAI().chat.completions.create({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: prompt }],
      response_format: { type: "json_object" },
    });

    const content = response.choices[0].message.content || "{}";
    const result = JSON.parse(content) as AISetupResult;

I added it for the sake of reducing runtime errors and handling data with proper structure instead of sending raw JSON as a prompt with string interpolation. The response_format: { type: "json_object" } forces the AI to return valid JSON, which eliminates parsing errors.

I also added a funding section by analyzing YAML files because I wanted to show it in search results. I created docker-compose to solve the problem related to mitigating the setup process.

Burnout

After several days, I realized I wasn't getting anywhere in repo-health. I immediately stopped adding new features and thinking about problems because I was writing code with anger :(. I realized it was time to switch to contributing to open source projects and starting a new project: verdis. I know that I have a long way to improve repo-health. I am in research hunting and see the tldraw's announcement where they closed their PR section from the community due to untested PRs and lack of technical depth. I have a lot of ideas, but I need a clear mind to convert them into code in that repo.

Also, due to job hunting, life has become hectic for me, but that is the process I have to go through. I need to consume more knowledge, build in the open, contribute, and improve my skills. Btw, my friend and I did our first podcast on Youtube. I know the camera makes it look like we recorded with fridge camera 😂, but we will improve.

Intro to Verdis

My friends and I are planning to create a versioned Redis: a key-value store with multi-version concurrency control (MVCC) and historical data access. In the next blog, I will talk about the books I am reading related to database internals, network programming in Go, concurrency in Go, and package-oriented design and why we are building Verdis.

We discuss while writing code and even argue about variable names and comments :). I also read another post from the same author who wrote about ego olympics in code review. This post talks about not over-engineering like we're building Netflix. I am trying to keep it simple and harmonize the business logic with my ideas.

Lastly

Communication with maintainers beats submitting AI-generated code that doesn't understand the project architecture and not tested code. You can stick to 1-2 projects so you actually learn the technical depth instead of just fixing random issues everywhere. It is better to document your changes even if it's one line. And when you burn out, take a break instead of writing angry code :).

I Built a Tool to Stop Wasting Time on Toxic Open Source Projects

ElshadHu — Mon, 29 Dec 2025 05:10:12 +0000

The Motivation

After contributing to several open source projects, I realized some of them have serious issues. Many maintainers don't provide help when you submit pull requests, and you end up wrestling with automated code reviews just to show one commit on your GitHub profile. So this time, instead of building more personal projects (which I've written about in my blogs like Building My Own HTTP Server in TypeScript and Building a CLI Tool That Made My Life Easier), contributing to random open source repositories, or grinding LeetCode problems, I wanted to create something impactful for the open source community. I decided to build repo-health (live demo) to help contributors choose projects they can successfully contribute to, and learn what works and what doesn't in open source collaboration.

My Tech Stack

Frontend: Next.js 16, React 19, Chakra UI
Backend: tRPC, Octokit, Zod
Data: MySQL (Prisma), Redis

I chose this stack to get familiar with current industry-standard tools and see how they work together in a real application(at the end of the day, they are just tools to build something cool).

First Challenge - What am I building?

When I started, I was just showing some data from GitHub and thinking that this is cool until I show my friends and college students. I realized that even though you spent a lot of time building one feature or solving one big bug it does not really matter if it doesn't solve the real world problem. So, I realized that before writing anything it is better to sit and think why it is needed to write that. So, I challenged myself by putting all my efforts to this product into 2 weeks.

Narrowing the Project Scope

I decided to focus on helping open source contributors with issues I've personally experienced and seen my classmates face in college: toxic communication environments on GitHub and wrestling with automated code reviews without proper guidance from maintainers.

First Feature: Overall Score

My system uses a Hybrid Approach that combines a deterministic formula grounded in industry standards with a qualitative Language Model Judge to account for real-world context.

The Algorithm (0-100 Score)

The base health score is calculated using a custom weighted average that I designed, inspired by standardized CHAOSS Metrics. I tuned the weights myself based on what I believe indicates a healthy modern project:

Score = (0.3 × Activity) + (0.25 × Maintenance) + (0.2 × Community) + (0.25 × Docs)

Activity (30%): Frequency of commits + Recency of updates + Unique authors.
Maintenance (25%): Issue response time + Open issue ratio + Repository age.
Community (20%): Logarithmic scale of Stars & Forks.
Documentation (25%): Existence of README, LICENSE, and CONTRIBUTING files.

The Language Model Adjustment

Standard formulas often misjudge "Feature-Complete" projects as "Dead." To solve this, I added a Judge Layer using a language model.

My Contribution:

I implemented a secondary logic layer where a language model analyzes the repository's purpose (through README content and file structure). I explicitly allow the model to override the algorithmic score by ±20 points if it detects that the metrics are misleading. This was my addition to bridge the gap between raw numbers and real-world context. For the MVP (minimum viable product), I'm currently using GPT-4 Mini due to its cost-effectiveness and fast response times. This allows me to validate the approach before potentially scaling to other models like Claude Sonnet or coming up with stronger ideas (I expect your ideas :)).

Example: A stable utility library with 0 commits in 6 months.
Algorithm: Penalizes it as "Old/Abandoned."
Language Model Judge: Recognizes it as "Completed/Stable" and awards a +20 Stability Bonus.

Implementation Snippet:

// I feed the calculated score into the AI prompt and ask for an adjustment:

prompt += `
  "scoreInsights": {
    "adjustment": {
       "shouldAdjust": true, 
       "amount": 20, // Range: -20 to +20
       "reason": "This is a stable utility library in maintenance mode. Low activity is expected and healthy.", 
       "confidence": "high"
    }
  }
`;

Checking PR Metrics

I built the PR Metrics Analysis to solve the lack of communication in open source. Before contributing, you need to know:

Speed: Is the average merge time hours or months?
Humanity: Are you dealing with real people or just fighting bot reviews?
Growth: Do new contributors actually stick around?

1. Handling Data Efficiently (Backend Concurrency)

To get these stats fast, I couldn't fetch everything one by one. I used Promise.all to fetch Open PRs, Closed PRs, and Template checks in parallel, cutting load time significantly.I dived into this topic in my http server blog post about event loop, here, the longest operation defines the total execution time.

// Efficiently fetching Open PRs, Closed PRs, and Template checks simultaneously
const [openPRs, closedPRs, template] = await Promise.all([
  fetchPRs(octokit, { owner, repo, state: "open" }),
  fetchPRs(octokit, { owner, repo, state: "closed" }),
  checkPRTemplate(octokit, { owner, repo }),
]);

Keeping contributors around matters more than total count. I used a Sankey Diagram to visualize the flow from "First-time" to "Core Team," making it easy to see if contributors stay or leave immediately.

// Visualizing the contributor flow
const data = {
  nodes: [
    { id: "First PR", color: "#58a6ff" },
    { id: "2nd Contribution", color: "#3fb950" },
    { id: "Regular (3-9)", color: "#a371f7" },
    { id: "Core Team (10+)", color: "#f0883e" },
  ],
  links: [
    {
      source: "First PR",
      target: "2nd Contribution",
      value: funnel.secondContribution + funnel.regular + funnel.coreTeam,
    },
    // ... logic to map flows for Regular and Core contributors
  ].filter((link) => link.value > 0),
};

Where I Made the Wrong Choice: The Security Scanner

I initially built a full Secrets Detection to catch exposed API keys, inspired by Gitleaks and TruffleHog.

How it worked:

Regex Pattern Matching: I used ~22 industry-standard patterns to catch known secrets (AWS keys, GitHub tokens, Stripe keys).
Randomness Detection: I implemented mathematics to detect highly random strings that "look" like secrets even if they don't match a pattern.

Why I removed it:

While building a security scanner was a great engineering challenge, I realized it drifted from my core mission. I decided to cut the feature to keep the project focused on community metrics rather than security auditing. Believe me, deleting it was painful, but that is just the way it is. Software Engineering is not about solving hard problems, it is about solving existing real life issues. Thanks to my friends' wake-up calls I opened my eyes and saw how I wasted my time.

Intelligent Issue Analysis

Analyzing Issues is the most effective way to understand a project's activity. I implemented several specific metrics to provide real insight to contributors:

Average Close Time: This measures the project's true speed. A repository with many open issues is still healthy if the average close time is short (e.g., 2 days vs. 6 months). I track both Average and Middle Value to filter out unusual cases (like issues that took 2 years to close).
Hot Issues: To help contributors find active discussions, I use a custom algorithm that prioritizes recent updates (last 48h), high engagement (comments/reactions), and security-related keywords.
Hidden Gems: This highlights "Old" but "High Impact" issues (like ignored feature requests). These are often ideal first contributions because they provide value without the conflict of highly active discussions.
Crackability Score: A calculated difficulty rating (0-100) based on documentation quality, file scope, and testing requirements. This filters complex issue lists into tasks that are feasible for new contributors to complete.

Project Overview & File-Issue Mapping

I have never been a frontend guy, but this journey pushed me to dive in. One problem I wanted to solve was reducing the complexity of exploring a new project. When you land on a repository with 500 files, where do you even start?

My Solution: Language Model-Powered Structure Analysis

I built a system that recursively fetches the entire file tree and feeds the structure to LLM . The LLM then tells you the entry points, key files, and which folders are responsible for which features. This way, the LLM gives you a proper project tour.

// Recursively fetch the entire file tree from GitHub
const { data } = await octokit.git.getTree({
  owner,
  repo,
  tree_sha: "HEAD",
  recursive: "true",  // Fetches entire tree structure at once
});

File-Issue Mapping: Connecting Problems to Code

On top of that, I added File-Issue Mapping. Before reaching the LLM, I use regex to scan all issue descriptions for file paths. If an issue mentions src/components/Button.tsx, I link that issue directly to that file in the overview. This way, a user can click on a file and immediately see if this file has any open issues and whether this is a single-file fix or if the issue affects multiple files.

// Extract file paths mentioned in issue text using regex
const FILE_PATTERN = /[\w\-\/\.]+\.(ts|tsx|js|jsx|py|go|rs|java|cpp|c)/gi;
function extractFilePaths(text: string): string[] {
  const matches = text.match(FILE_PATTERN) || [];
  return [...new Set(matches)]; // Deduplicate
}

Project Tree Visualization (Frontend)

For drawing the project structure, I got inspiration from the repo-visualizer project. On the frontend, I implemented a recursive function that builds a hierarchy from the flat file list. This function traverses each file path, splits it by /, and creates nested parent/child relationships to form a tree.

// Recursively build hierarchy from flat file paths
function buildHierarchy(files: FileNode[], repoName: string, maxDepth = 3): HierarchyNode {
  const root: HierarchyNode = { name: repoName, path: "", children: [] };
  files.forEach((file) => {
    const parts = file.path.split("/");
    let current = root;
    // Traverse and build tree structure
    parts.slice(0, maxDepth + 1).forEach((part, index) => {
      let child = current.children?.find((c) => c.name === part);
      if (!child) {
        child = { name: part, children: [] };
        current.children!.push(child);
      }
      current = child;
    });
  });
  return root;
}

I also added collision reduction by limiting depth and file count. This keeps the visualization readable even for large repositories.

Known Limitation: I tried to add zoom and pan functionality, but the sensitivity was off and it broke normal scrolling (PRs are welcome). I decided to keep the visualization simple and stable rather than ship a broken interactive version. This feature needs more polish in future iterations.

Activity Pattern Detection

As you know, especially during Hacktoberfest, some people make commits and PRs just for the sake of doing it. Spam contributions, bulk deletions, and suspicious patterns are everywhere. To detect these activities, I built a Pattern Detection system based on commit metrics from the GitHub API.

What is a Suspicious Pattern?

A suspicious pattern is any commit activity that looks very different from normal development work. I track several types:

Mass Deletion Detection

"Deletion Rate" measures the ratio of code deleted vs total changes. A commit that deletes 90% of what it touches with 100+ lines removed is suspicious. It could be a cleanup, or it could be vandalism.

// Detect unusual deletion patterns
function detectChurnAnomalies(commits: CommitWithStats[]): PatternAnomaly[] {
  for (const commit of commits) {
    const total = commit.additions + commit.deletions;
    const churnRatio = commit.deletions / total;

    // Flag commits that delete >80% of touched code
    if (churnRatio > 0.8 && commit.deletions > 100) {
      anomalies.push({
        type: "churn",
        severity: churnRatio > 0.9 ? "critical" : "warning",
        description: `Deleted ${Math.round(churnRatio * 100)}% of code (${commit.deletions} lines)`,
      });
    }
  }
}

Rapid-Fire Commits Detection

This catches contribution bombarding. When someone makes 10+ commits in under 10 minutes. Real development doesn't work that way.

// Detect rapid-fire commits (likely spam or farming)
function detectBurstActivity(commits: CommitWithStats[]): PatternAnomaly[] {
  for (let i = 0; i < sorted.length - 4; i++) {
    const windowStart = new Date(sorted[i].date).getTime();
    const windowEnd = new Date(sorted[i + 4].date).getTime();
    const diffMinutes = (windowEnd - windowStart) / (1000 * 60);

    // 5+ commits in under 10 minutes = suspicious
    if (diffMinutes <= 10) {
      anomalies.push({
        type: "velocity",
        severity: count > 10 ? "critical" : "warning",
        description: `Burst: ${count} commits in ${Math.round(diffMinutes)} minutes`,
      });
    }
  }
}

Risk Grades

Grade	Score	Meaning
A	0-10	Normal activity
B	11-30	Minor anomalies
C	31-50	Review recommended
D	51-70	Suspicious
F	71-100	Critical review

Pivot: Checking from Maintainer's Perspective

When I started this project, I was only thinking about contributor's perspective, but reality hit hard. I read blog posts and articles such as :

From these blogs I understood how wild open source can be. Basically, users and companies can ask for features that don't suit the project's goals, or big companies use open source projects without sponsoring them. Also, there's toxicity from users, people who just add their name to the README and feel proud even though that's not a real contribution. So I decided to look at the bigger picture and examine projects from the maintainer's perspective.

Contribution Insights

I built a feature that analyzes rejected PRs and shows why they failed, helping future contributors avoid the same mistakes.

Spam Detection

First, I filter out obvious spam PRs that just add a name to the README:

const SPAM_TITLE_PATTERNS = [
  /add(ed|ing)?\s+(my\s+)?name/i,
  /update(d)?\s+readme/i,
  /hacktoberfest/i,
];

function detectSpam(pr, files): { isSpam: boolean; reason: string } {
  const isReadmeOnly = files.length === 1 && 
    files[0].filename.toLowerCase().includes("readme");

  if (isReadmeOnly && files[0].additions < 5) {
    return { isSpam: true, reason: "Trivial README change" };
  }
  return { isSpam: false, reason: "" };
}

Automated Failure Analysis

For legitimate rejected PRs, I send the code diff and reviewer comments to a language model. It categorizes each failure:

type PitfallAnalysis = {
  prNumber: number;
  mistake: string;
  reviewFeedback: string;
  advice: string;
  category: "tests" | "style" | "scope" | "setup" | "breaking" | "docs";
};

Category	Meaning
`tests`	Missing or broken tests
`style`	Code formatting violations
`scope`	Change too large or out of scope
`setup`	Build/environment issues
`breaking`	Introduced breaking changes
`docs`	Missing documentation

This turns rejected PRs into a learning resource for the community.

Weird Bugs

Cache Security Vulnerability

I wasn't familiar with the stack when I started, and I made a mistake by using the same cache key for both public and private repositories. The cache was making the user experience much faster and smoother, so I was feeling proud of myself until I saw my friend's private project showing up in my account.

Out of curiosity, I asked my friend to tell me their private repo name, and in the fuzzy search and analysis, I saw the project. Boom! Of course, GitHub doesn't allow you to visit someone's private repo, but this was still a security vulnerability. I realized I wasn't creating different cache keys for each user.

The Fix: Token-Based Cache Isolation

I implemented a function that creates a unique hash from the user's access token:

import crypto from "crypto";

export function getTokenHash(token?: string | null): string {
  if (!token) return "public";
  return crypto.createHash("sha256").update(token).digest("hex").slice(0, 8);
}

How It's Used

Every cache key now includes the token hash to isolate private repo data per user:

const tokenHash = getTokenHash(accessToken);
const cacheKey = `repo:info:${owner}:${repo}:${tokenHash}`;

Scenario	Token Hash	Cache Key Example
Public repo	`public`	`repo:info:facebook:react:public`
User A private	`k3m7p2q9`	`repo:info:userA:secret:k3m7p2q9`
User B private	`x4y9z2a5`	`repo:info:userA:secret:x4y9z2a5`

This ensures User B can never see User A's cached private repo data.

Future Improvement: Cache Manager Class

Right now I'm calling getTokenHash() in every service file. I'm planning to create a centralized CacheManager class to handle this consistently.

React Hydration Mismatch

Another issue I faced was the infamous React hydration error. The sign-in button was causing a mismatch between what the server rendered and what the client expected.

The Problem

I was using useSession() but not checking the status properly:

// Before: Not checking status
const { data: session } = useSession();

// This caused hydration mismatch because:
// - Server: session is undefined → renders "Sign In" button
// - Client: session loads → renders user avatar
// React panics because the HTML doesn't match

The Fix

I added a loading state that renders the same placeholder on both server and client:

// After: Checking status properly
const { data: session, status } = useSession();

// In the JSX:
{status === "loading" ? (
  // Loading state - same on server and client
  <Box
    w="100px"
    h="32px"
    bg="#21262d"
    borderRadius="md"
    opacity={0.5}
  />
) : session?.user ? (
  // User is logged in - show avatar
  <UserMenu />
) : (
  // Not logged in - show sign in button
  <SignInButton />
)}

Why This Works

State	Server Renders	Client Renders	Match?
Loading	Placeholder	Placeholder	Yes
Logged In	Placeholder	Avatar	Yes
Logged Out	Placeholder	Sign In	Yes

The important point was here during initial hydration, both server and client render the same placeholder. Only after hydration completes does the client update to show the actual state.

Recommended Resource

I highly recommend reading The Perils of Rehydration. This article helped me understand Next.js server-side rendering and React hydration issues properly.

What I'm planning Further

I realized that in this project I need to think about both contributor's and maintainer's perspectives to create a proper product. So, my future ideas are:

Replace Recent Commits with Significant Commits: Instead of showing all recent commits, I want to highlight commits that made meaningful changes to the project. This means filtering out trivial updates (like typo fixes or formatting changes) and showing commits that added features, fixed bugs, or made architectural improvements.
Check if feature requests are appropriate for project scope: Help maintainers identify feature requests that don't align with the project's goals.
Promote funding platforms: Highlight GitHub Sponsors, Open Collective, and Buy Me a Coffee for open source projects (maintainers deserve it, especially if we use their projects extensively).
Show project lifecycle status: Display whether a project is Active, in Maintenance Mode, Archived, Company-backed, or maintained by a Solo Developer.
Show common setup mistakes: Extract patterns from CONTRIBUTING.md and failed CI builds to help contributors avoid common errors. I'm planning to remove my dependency section because it's not related to this scope, even though I spent a lot of time building and thinking about this feature by relating PRs to dependency vulnerabilities.
Dropdown with 4 levels: First Contributor, Beginner, Expert at Stack, I am Cooked
Add comprehensive testing: Implement unit tests, integration tests, and end-to-end tests to ensure the platform's reliability and make it easier for contributors to add features confidently.

Conclusion

I started this project 2 weeks ago and if you visit my GitHub you can see that this project is being built with pain :). So, again that project does not contain the cleanest code that I have written. I thought of writing this project and ship quickly in order to show you guys, what is my idea and Where I am heading. At the end of the day, code speaks louder than words. Btw, I used LLM for helping me do repetitive tasks that I already know how to write and find the answer from StackOverflow. I wouldn't produce this much of code within short amount of time otherwise. But, ideas, ideas can never be created by LLM, and that is where I need your help and your vision. Also, ideas never come full-formed, most of the time they are ill-formed. Let's make open source safer and better by improving this project together. Right now, this project has many areas that need improvement. But with the help of the open source community, we can build something that saves time for both maintainers and contributors. Lastly, you can follow me if you want on GitHub. If you star the repo-health, that would be amazing. You can contact me on Discord, that is my username: elshad_02838

Contributing to streamplace

ElshadHu — Sat, 13 Dec 2025 21:42:58 +0000

How I found the Project

Nowadays I've been reading and writing Go code regularly, and my Go journey started with A Tour of Go. While building my http-server-ts, I watched this video where Primeagen builds an HTTP server in Go, which showed me how Go handles network programming. After getting confident with the language specifics, I found streamplace. I tackled that issue and successfully made a PR that got merged. I'll give you the details below.

The Issue Description

When the firehose is unavailable, the stream had error-prone code that would crash the loop. After getting familiar with the codebase (without even setting it up, thanks to my C++ background), I commented on the issue saying I'd like to work on it. I mentioned that the issue was in the StartLabelerFirehose() function in labeler_firehose.go, and I was planning to replace the crash with a backoff mechanism. I got nice feedback from the maintainer saying my approach sounded reasonable and that it would be better to add a metric to spmetrics.go. Basically, my conversation with the maintainer reduced the back and forth in my pull request, which I recommend to everybody that communicate beforehand when tackling an issue.

Set Up

Even though the code structure was pretty niche, setting up this project was time-consuming because of handling dependencies. I spent way too much time on this, which I should have done a long time ago if I wanted to contribute regularly. I dealt with sqlite3 version issues when following the documentation.

The Solution

I fixed the issue by adding a streamplace_labeler_firehoses_connected metric in spmetrics.go to track labeler connections:

var LabelerFirehosesConnected = promauto.NewGaugeVec(prometheus.GaugeOpts{
    Name: "streamplace_labeler_firehoses_connected",
    Help: "number of currently connected labeler firehoses",
}, []string{"labeler"})

Then I imported spmetrics.go and changed the log level at the retry threshold from Error to Warn:

log.Warn(ctx, "firehose failed 3 times within a minute, backing off", "err", err)
time.Sleep(5 * time.Second)
retryCount = 0
retryWindow = time.Now()

When fixing this bug, I made a mistake that I corrected before even committing. I was unintentionally creating a race condition by changing cancel():

defer func() {
    spmetrics.LabelerFirehosesConnected.WithLabelValues(did).Set(0)
    cancel()
}()

This was problematic because the defer won't execute until the function returns, but I also had defer cancel() which would be called in the reverse order of declaration. This was creating confusion. When it comes to critical-performance places in the code, it's always better to be skeptical. For that reason, I didn't touch cancel() and it stayed as defer cancel().

Testing and Suggestion from Maintainer

I tested the change locally using Bluesky's official moderation labeler. I made sure that the metric reports 1 while the labeler firehose is connected. I also tested the retry path and saw that the code backs off for 5 seconds after repeated failures instead of crashing.

But I didn't consider more than one labeler firehose by writing:

spmetrics.LabelerFirehosesConnected.WithLabelValues(did).Set(1)
defer spmetrics.LabelerFirehosesConnected.WithLabelValues(did).Set(0)

Via the suggestion from the maintainer, I changed to Inc and Dec instead of Set for connecting more than one labeler firehose:

    spmetrics.LabelerFirehosesConnected.WithLabelValues(did).Inc()
    defer spmetrics.LabelerFirehosesConnected.WithLabelValues(did).Dec()

Eventually my pull request got merged.

Gained Knowledge

For the first time, I dealt with a well-structured Go codebase. I learned about race conditions and reminded myself how important small details are. Also, as I mentioned before, it's better to keep code changes precise and stay in touch with maintainers. I'll keep going and continue contributing to this codebase.

Contributing to Tableau-MCP

ElshadHu — Sat, 13 Dec 2025 17:21:54 +0000

How I Chose the Repository

As I mentioned in my previous blog, I am interested in understanding MCP and how it works. I chose tableau-mcp, I chose this issue to work on. I successfully made a Pull Request that got merged. I plan to continue working on this project and contribute to it regularly.

What Was an Issue

REST API logging was missing the /api/3.26 path segment in logged URLs.I modified logRequest() and logResponse() functions to properly preserve the API version path when constructing URLs for logging.

After my changes I tested it with unit tests and the command below:

node -e "
const baseUrlObj = new URL('https://10ax.online.tableau.com/api/3.26');
const url = new URL(baseUrlObj.origin + baseUrlObj.pathname + '/sites/xxx/datasources');
console.log(url.toString());
"

After confirming the fix worked correctly, I made the Pull Request. However, I forgot to bump the version, which I later committed using npm run version:patch. This is essential because it updates package.json and package-lock.json, keeping other developers aware of version changes.

Suggestions from Maintainer

When I initially wrote the changes, I used the URL constructor to be explicit about types:

const baseUrlObj = new URL(request.baseUrl);
  const url = maskedRequest.url?.startsWith('/')
    ? new URL(baseUrlObj.origin + baseUrlObj.pathname + maskedRequest.url)
    : new URL(maskedRequest.url ?? '', request.baseUrl);

While this was explicit and clean code, the maintainer asked for my opinion on changing it to use the replace method for simplicity. That was reasonable because, APIs can change in the future unpredictably, and a simpler approach would be more maintainable. The suggestion made sense, so I changed it to this format:

 const url = new URL(
    `${maskedRequest.baseUrl.replace(/\/$/, '')}/${maskedRequest.url?.replace(/^\//, '') ?? ''}`,
  );

As you can see, the code is simpler and most of the time in open source simple code is better than thinking about clean code. I applied the same pattern to logResponse() as well.

Gained Knowledge

As a result of this PR, I learned that it's better to discuss ideas with maintainers beforehand to get their insights. Talking to experienced developers provides more valuable learning than any tutorial or book.

Moving forward, I will:

Stop switching between projects and making only small contributions
Deepen my knowledge of the projects I've already discovered

I've already explored many projects, and I don't want to spend too much time setting up new ones and getting familiar with them. Instead, I'll contribute to what I've already discovered.

Release 0.4: My Plan for Contributions to Open Source

ElshadHu — Sat, 13 Dec 2025 05:45:55 +0000

The Plan

For release 0.4, I decided to explore three things I kept hearing about but never really took the time to understand properly. The plan was to dive into Drizzle ORM, start contributing to Go codebases, and figure out what is MCP. I picked three different repositories to work with:

Drizzle ORM

I chose Drizzle because a lot of developers use it for handling database schemas and queries instead of writing raw SQL everywhere. The main reason I got interested because of my curiosity to databases. ORM stands for Object Relational Mapping. Basically it maps your code objects to database tables. Setting up was straightforward. I activated Docker, cloned the repo, and started reading through the codebase. The architecture is pretty lightweight compared to other ORMs I've seen, which is probably why people like it. What I found interesting is how Drizzle lets you define database schemas in TypeScript instead of writing SQL queries scattered throughout your code. I'm planning to make some pull requests to Drizzle in the future. I want to understand how ORMs actually work under the hood. Contributing seems like the best way to learn that.

Streamplace

Streamplace is a project written mainly in Go and TypeScript. It's a platform for live streaming and video processing - basically the infrastructure that handles live video streams. The backend performance-critical parts are in Go, and the more dynamic parts are in TypeScript. I got interested in this project because of how the code is organized. Coming from TypeScript projects where things can get complex because of high abstraction, seeing a well-structured Go codebase is really refreshing to me. I'll write more about the specific contribution in my next blog post.

Tableau-MCP

Tableau-MCP is a project that makes it easier for developers to work with LLMs while building their projects. It provides resources and tools specifically designed for integrating AI models into your development workflow. The project is still in active development, which is actually why I wanted to contribute to it. It's not in the last stages, it's being built right now. I've been exploring the codebase and learning about MCP (Model Context Protocol). It's a protocol for how applications interact with AI models. I'm planning to make contributions to this project. I'll share the details in my next blog post.

Contributing to roslibjs: From Defeat to Merged PR

ElshadHu — Fri, 28 Nov 2025 08:00:42 +0000

While scrolling through GitHub during my free time, I came across roslibjs the standard Robot Operating System JavaScript library. After researching the project, I learned that roslibjs enables web applications to communicate with robots from within a browser. The library uses WebSockets to connect with rosbridge which provides a JSON API to ROS functionality for non-ROS programs, supporting publishing, subscribing, service calls, and other ROS functionality. The core architecture made me interested to work on this issue. This issue was open since June 12, and from the description, it seemed trivial to fix. However, I didn't account for the architectural complexity involved. After communicating with a maintainer about tackling this issue, I dove in. Here's the PR that I made.

The Biggest Mistake: Being Lost in the Codebase

I started by reading the entire documentation, which provided valuable insights but took me far beyond the scope of what I actually needed. I spent three days trying to understand the complete architecture, which turned out to be overkill. Ultimately, I only needed to check the status field in Action.sendGoal instead of the result field which was a much simpler fix than I had anticipated.

Taking a Break to Reset

After exhausting myself by exploring the wrong parts of the codebase, I took a one-week break from the issue. I needed to step back and recharge. But after that week, I pushed myself to restart with a fresh perspective. Because I thought without even trying and doing my best how can I accept the defeat?
With proper research, I found the Rosbridge Protocol and ROS 2 GoalStatus documentation exactly what I needed to solve the problem.

My Initial Approach

I identified the bug in Action.sendGoal, where the method incorrectly relied on the result field as a success/failure indicator. This caused STATUS_CANCELED and other status codes to be handled incorrectly. I updated the sendGoal method in src/core/Action.ts to check message.status against GoalStatus.STATUS_SUCCEEDED as the primary success indicator, routing all other status codes to the failure callback with descriptive error messages.

My Implementation

// Check status code instead of result field
const status = message.status as GoalStatus;

if (status === GoalStatus.STATUS_SUCCEEDED) {
  resultCallback(message.values as TResult);
} else {
  // Handle failures with appropriate error messages
 // with switch case statements
  failedCallback(errorMessage);
}

Functional Review Feedback

A maintainer provided excellent feedback. While the added functionality looked good, they suggested also checking message.result to catch edge cases where it might be false, potentially indicating backend implementation issues. I revised the code to validate both fields:

        if (status === GoalStatus.STATUS_SUCCEEDED && message.result) {
          resultCallback(message.values);
        }

Also, after doing that I did not need to do casting , because TypeScript already knew the type so I did not need to write as TResult so, I removed it.

TypeScript Review and Future-Proofing

Another maintainer told me that it looks good, but also suggested a "future-proofing nit": instead of writing error messages directly in the function, I should create a custom error class that handles template interpolation. This would provide a proper error type for future use.

My Attempt for Error Class


export class GoalError extends Error {
  constructor(status: GoalStatus | number | string, baseError?: unknown) {
    const base = typeof baseError === "string" ? baseError : "";
    let message: string;

    switch (status) {
      case GoalStatus.STATUS_CANCELED:
        message = `Action was canceled${base ? `: ${base}` : ""}`;
        break;
      case GoalStatus.STATUS_ABORTED:
        message = `Action was aborted${base ? `: ${base}` : ""}`;
        break;
      case GoalStatus.STATUS_CANCELING:
        message = `Action is canceling${base ? `: ${base}` : ""}`;
        break;
      case GoalStatus.STATUS_UNKNOWN:
        message = `Action status unknown${base ? `: ${base}` : ""}`;
        break;
      default:
        message = `Action failed with status ${String(status)}${base ? `: ${base}` : ""}`;
    }

    super(message);
    this.name = "GoalError";
    Object.setPrototypeOf(this, GoalError.prototype);
  }
}

My approach to using this error class was somewhat naive. The maintainer helped refine it, and the PR was eventually merged. I felt proud ,because I went from doing nothing to fixing error-prone code in a real open-source project. Later, I examined the maintainer's implementation to understand the cleaner approach:

Maintainer's implementation

class GoalError extends Error {
  override name = "GoalError";
  constructor(status: GoalStatus, errorValue?: string) {
    super(`${makeErrorMessage(status)}${errorValue ? `: ${errorValue}` : ""}`);
  }
}

function makeErrorMessage(status: GoalStatus) {
  switch (status) {
    case GoalStatus.STATUS_CANCELED:
      return `Action was canceled`;
    case GoalStatus.STATUS_ABORTED:
      return `Action was aborted`;
    case GoalStatus.STATUS_CANCELING:
      return `Action is canceling`;
    case GoalStatus.STATUS_UNKNOWN:
      return `Action status unknown`;
    default:
      return `Action failed with status ${String(status)}`;
  }
}

The maintainer extracted the message generation logic into a separate makeErrorMessage function, making the code more modular and easier to test.The template interpolation cleanly combines the base error message with optional additional context. This demonstrated to me even as programming becomes more abstract, clean code principles stay the same.

Test Cases

In most open-source projects, fixing a bug or implementing a feature requires comprehensive test coverage. I needed to create test cases that strictly matched the real protocol.

interface ActionResultMessageBase {
  op: "action_result";
  id: string;
  action: string;
  status: number;
}

Then I split success and failure into two shapes:

interface FailedActionResultMessage extends ActionResultMessageBase {
  result: false;
  values?: string;
}

interface SuccessfulActionResultMessage extends ActionResultMessageBase {
  result: true;
  values: { result: number };
}

type ActionResultMessage =
  | FailedActionResultMessage
  | SuccessfulActionResultMessage;

This pattern is often described as a discriminated union in TypeScript: a union where one field tells you which shape you have. That lets TypeScript narrow types automatically and keeps your code safer.

I also created feedback messages:


interface ActionFeedbackMessage {
  op: "action_feedback";
  id: string;
  action: string;
  values: { current: number };
}

type ActionMessage = ActionResultMessage | ActionFeedbackMessage;

ActionMessage is then a union of result messages and feedback messages. This reminded me that learning is never wasted. Even though I spent the first week diving too deep into the architecture, that exploration helped me design better test types later on.

Writing a Test Case: STATUS_ABORTED Example

Let me walk through how I created the test case for STATUS_ABORTED to show my thought process:

it("should call failedCallback when action is aborted (STATUS_ABORTED)", () => {
      const resultCallback = vi.fn();
      const failedCallback = vi.fn();

      action.sendGoal(
        { target: 10 },
        resultCallback,
        undefined,
        failedCallback,
      );

      const abortedMessage: FailedActionResultMessage = {
        op: "action_result",
        id: "test-id",
        action: "/test_action",
        values: "Action aborted due to error",
        status: GoalStatus.STATUS_ABORTED,
        result: false,
      };

      messageHandler?.(abortedMessage);

      expect(failedCallback).toHaveBeenCalled();
      expect(resultCallback).not.toHaveBeenCalled();
    });

I created a mock message (abortedMessage) that matches the protocol structure for a failed action. The important parts are: status: GoalStatus.STATUS_ABORTED (tells us it failed) and result: false (confirms the failure). Then I just check that the failure callback gets called and the success callback doesn't. Simple as that.

Conclusion

This contribution taught me a lot more than how to fix a bug. I realized if I put effort, care about the code and read the documentation, I can solve most of the bugs. Also, I realized that Senior developers think long-term. That "future-proofing nit" about a dedicated error class showed me how they design for clarity and future changes.Through this single PR, I learned about protocols, WebSockets, error handling, TypeScript type design (including discriminated unions), and cleaner architecture patterns. The roslibjs maintainers are proactive, kind, open to guiding me through the process, I really appreciate them. Most importantly, I went from feeling stuck and intimidated to shipping a real fix in a production library. That feeling is absolutely worth the initial discomfort.

Building My Own HTTP Server in TypeScript

ElshadHu — Thu, 27 Nov 2025 06:04:51 +0000

How I Started

When I started my journey with web development, after some weeks I kept hearing about "requests" and "responses." Wait! What is a request? What is a response? Why does HTTP exist, and why is it called HTTP? I hate pretending to understand something. Therefore, I decided to build my own HTTP server—or at least pretend to :). Eventually, I got to the core idea: HTTP is just Hypertext Transfer Protocol—basically a set of rules for transferring data. But what really mattered was understanding that HTTP is built on top of TCP(Transmission Control Protocol), which I'll explain below. So, I started building my own http-server-ts.

Wake-up Call from my Friend

I asked my friend who had built his own server, and I was genuinely impressed. When I asked him how I could learn all this stuff related to web development, he gave me surprisingly simple advice: "Build your own server, or pretend like you're building one. Eventually, you'll learn." Here's my friend's project that impressed me. I started reading his blog, which turned out to be more beneficial than the two entire web development courses I took in college. People often say "don't reinvent the wheel," but I don't understand why. I think reinventing the wheel is actually the best way to learn, even if I build dumb stuff.

First Challenge: What is TCP?

I know it might sound stupid, but I genuinely didn't know what TCP was, even though I had been creating endpoints and APIs in my web classes, practicing with HTTP libraries and frameworks. So, I decided to search, and thanks to my friend's help, I grasped the main idea. TCP (Transmission Control Protocol) provides reliable communication. HTTP defines the format of messages (requests and responses), while TCP ensures those messages actually arrive correctly.
Basically, it's a 3-way handshake. Imagine a conversation between a client and server:

Client says: "Yo, I want to connect"
Server replies: "Yo, I got your message. Thanks for reaching out."
Client confirms: "Thanks for getting my message. Let's start communicating."

That's it. That's TCP in simple terms. Once this handshake completes, they can exchange data reliably.

Parsing HTTP Requests: Not Rocket Science

I got a lot of hype about parsing HTTP requests, making it sound like some complex computer science problem. After actually implementing it, I realized it's not rocket science at all. You're just reading a string and splitting it into pieces.

Here's what an HTTP request actually looks like on the wire:

GET /api/users?id=123 HTTP/1.1\r\n
Host: localhost:8080\r\n
User-Agent: curl/7.64.1\r\n
Accept: */*\r\n
\r\n

That's it. Just text with \r\n (carriage return + line feed) separating lines, and \r\n\r\n marking the end of headers.

My parsing logic:

static parse(buffer: Buffer): HttpRequest {
    const request = new HttpRequest();
    const rawRequest = buffer.toString('utf-8');
    const lines = rawRequest.split('\r\n');

    // Step 1: Parse the first line "GET /api/users?id=1 HTTP/1.1"
    const requestLineData = RequestLine.parse(lines[0]);
    request.method = requestLineData.method;
    request.path = requestLineData.path;
    request.version = requestLineData.version;

    // Step 2: Parse headers
    // Step 3: Parse body
    // That's it
}

That's literally it. Split on \r\n, grab the first line for the request method/path, grab everything until the blank line for headers, and everything after is the body.

The "complex" part people made sound difficult was just string splitting and basic parsing. Once I saw the actual format, I realized I had been overthinking it.

Building HTTP Responses: Serializing Data Back

If parsing is just reading strings and splitting them, then building responses is just the reverse: taking your data and formatting it back into the HTTP format.

export class ResponseBuilder {
    private statusLine = new StatusLine();
    private headers = new HeaderBuilder();
    private body = "";

    setStatus(code: HttpStatusCode, message?: string): this {
        this.statusLine.set(code, message);
        return this;
    }

    setHeader(name: string, value: string): this {
        this.headers.set(name, value);
        return this;
    }

    setBody(body: string): this {
        this.body = body;
        return this;
    }

    build(): string {
        if (!this.headers.get('Content-Length') && this.body) {
            this.setHeader('Content-Length',
                Buffer.byteLength(this.body).toString());
        }
        return (
            this.statusLine.toString() +    // "HTTP/1.1 200 OK\r\n"
            this.headers.toString() +        // "Content-Type: text/html\r\n..."
            "\r\n" +                         // Blank line
            this.body                        // "<h1>Hello</h1>"
        );
    }
}

The pattern here is called the "Builder Pattern." You chain method calls to configure the response, then call build() to get the final string. It's just a cleaner way to construct something with multiple pieces.

HTTP Keep-Alive: Reusing Connections for Performance

Here's where things got interesting. Initially, my server opened a new TCP connection for every single request:

Request 1: Open connection → Send request → Get response → Close connection
Request 2: Open connection → Send request → Get response → Close connection
Request 3: Open connection → Send request → Get response → Close connection

Each of those "open connection" steps involves the 3-way TCP handshake I mentioned earlier. That's roughly 1.5 ~ times (RTT) of latency, which on a network might be 5ms. For 100 requests, that's 500ms of pure overhead just opening and closing connections.

Then I learned about HTTP Keep-Alive (also called persistent connections). The idea is simple: keep the connection open and reuse it for multiple requests.

Request 1: Open connection → Send request → Get response
Request 2:                   Send request → Get response (same connection!)
Request 3:                   Send request → Get response (same connection!)
           ...after 100 requests or timeout...
           Close connection

Implementation:

export class KeepAliveManager {
    private requestCount = 0;
    private lastActivityTime: number;
    private readonly config: KeepAliveConfig;

    constructor(config: Partial<KeepAliveConfig> = {}) {
        this.config = {
            enabled: config.enabled ?? true,
            timeoutMs: config.timeoutMs ?? 60000,    // 60 seconds
            maxRequests: config.maxRequests ?? 100    // Max 100 requests per connection
        };
        this.lastActivityTime = Date.now();
    }
}

The logic is straightforward. First, track how many requests have been served on this connection. Then, track when the last activity happened. Lastly, keep the connection alive unless we've hit the max requests or the timeout.

I read about this technique in the HTTP/1.1 specification, which helped me understand it properly.

Unexpected Knowledge: JavaScript Event Loop and Callbacks

I thought I understood JavaScript until I started using callbacks for network programming. I had to stop and really learn how the event loop works.

After watching this video, I finally understood that JavaScript execution is divided into parts:

Call Stack: Handles function execution in Last-In-First-Out order
Web APIs: Where asynchronous operations happen
Task Queue: Where completed async operations wait
Event Loop: Moves tasks from the queue to the stack

Here's what happens when you register a callback:

conn.onData((data) => {
    // This callback doesn't execute immediately
    // It waits for data to arrive
});

In the above example, the onData function registers the callback and returns immediately. Then, when data arrives on the socket, Node.js puts the callback in the task queue. The event loop checks: "Is the call stack empty? Yes? Okay, move this callback from the queue to the stack." After that, the callback executes.

The interesting part is that the event loop never ends. By registering callbacks, I'm essentially creating infinite loops. The server keeps running because there are always callbacks waiting for events.

Language Specific: Buffers vs Strings

The dumb way I started:

// String concatenation (slow as hell)
let requestData = "";
conn.onData((data) => {
    requestData += data.toString();  // Creates new string every time!
    // Memory: Copy, copy, copy...
});

Every time you concatenate strings in JavaScript, it creates a brand new string in memory and copies everything over. For a few bytes, who cares? For a bunch of network chunks, this becomes a performance disaster.

The reasonable way:

// Buffer accumulation (fast)
class RequestBuffer {
    private chunks: Buffer[] = [];
    private totalSize: number = 0;

    append(chunk: Buffer): void {
        this.chunks.push(chunk);      // Just store reference!
        this.totalSize += chunk.length;
    }

    toBuffer(): Buffer {
        return Buffer.concat(this.chunks);  // Combine once when needed
    }
}

Instead of copying data around, just store references to the chunks. When you finally need the complete data, combine everything once.

Conclusion

This journey of building an HTTP server in TypeScript has taught me more than any tutorial ever could. I've learned about TCP connections, HTTP protocol structure, event-driven programming, and the internals of how servers actually work. Honestly, networking was the topic I used to actively ignore. Whenever it came up in courses or conversations, I'd zone out or find excuses to skip over it. But diving into this project completely changed my perspective. Now I'm doing my best to learn more about networking, protocols, and everything I once avoided. This project is far from finished, I'll keep improving it as my understanding deepens. If you spot any mistakes, misconceptions, or areas where I'm still confused, please call them out. I'm completely open to criticism because that's how I learn best. Whether it's my understanding of the event loop, my implementation of Keep-Alive, or anything else in this post, I welcome your feedback.

Open Source: My Path to Better Code

ElshadHu — Sun, 23 Nov 2025 09:25:36 +0000

How to Keep Myself Interested via Open Source

Open Source is an adventure. I learn tons of information by checking projects and looking at issues in my free time, which gives me pleasure. In this blog, I'll cover three PRs I made to dive deeper into TypeScript and React semantics:

React Hooks Optimization - samui-wallet | Issue | PR
Date/Time Display Fix - archestra | Issue | PR
JSONata Timeout Validation - node-red-contrib-home-assistant-websocket | Issue | PR

Are hooks Always beneficial ?

Short answer: No. Everything in programming has trade-offs, and hooks are no exception.

The Problem

The 24 words button wasn't working, and the maintainer wanted minimal code changes. I fixed it by removing an unnecessary React hook and changing just one line of code.

The Solution

Before:

   const [wordCount, setWordCount] = useState(12)

After:

  const [strength, setStrength] = useState<MnemonicStrength>(128)
  const wordCount = strength === 128 ? 12 : 24

I just derived it directly , it's a one-line ternary. I realized that there is no need to make things overcomplicated by using useEffect which would add an extra render cycle. Also, useMemo is expensive for one button.

Time Format Challenges

Working on the archestra MCP server project, analyzing the issue was straightforward, but reproducing the bug was time-consuming. I created mock test data to verify the fix. I fixed two issues in the MCP Gateway logs table:

I removed redundant toLocaleDateString() conversion and passed the ISO string directly to formatDate(). I added a TruncatedText component for long server names

The Open Source Reality

After submitting my PR, the maintainer thanked me but had already pushed their own fix, that's just how open source works sometimes.

What I Learned

I reviewed the maintainer's code and realized they used an even simpler solution:

// My approach - thinking beyond, but simpler is better
const dateString = 
  typeof row.original.createdAt === "string"
    ? row.original.createdAt
    : new Date(row.original.createdAt).toISOString();

// Maintainer's approach - cleaner
date: row.original.createdAt,

They simply passed the raw value, trusting that formatDate() handles the conversion internally. I already knew this solution, but sometimes, my empathy with the code and caring too much about code drives me crazy. Remember simplicity is always the key. I regularly study other developers' contributions to learn clean code practices, it's one of the best ways to improve.

Handling JSONata timeout values

I discovered node-red-contrib-home-assistant-websocket, a fascinating project that bridges Node-RED's powerful flow-based programming with Home Assistant's smart home capabilities. I found an issue I could solve, reproduced the bug in the UI, and implemented validation for JSONata timeout values:

    #validateAndConvertTimeout(result: unknown): number {
        const timeout = Number(result);

        if (result === null || isNaN(timeout) || timeout < 0) {
            throw new InputError(
                [
                    'ha-wait-until.error.invalid_timeout',
                    { timeout: String(result) },
                ],
                'ha-wait-until.error.error',
            );
        }

        return timeout;
    }

Firstly I validated if it is undefined which was already being evaluated to isNaN. I fixed that via the suggestion of maintainer, also I changed the error message to String(result) for not making users confused. In open source, if you try your best and show genuine care, maintainers will guide you. What they value most is attention to detail and being specific in your contributions.

Conclusion

Open source is accelerating my learning and growth. I'm not just learning code, I'm learning architecture, clean code principles, communication skills, and new tools. The combination of hands-on problem-solving and collaboration with experienced developers makes every contribution a valuable learning experience.

First Release to My CLI Tool

ElshadHu — Sat, 22 Nov 2025 06:34:43 +0000

The Beginning: "How Hard Could It Be?"

After successfully building my RepositoryContextPackager tool with CMake and getting CI/CD working across Windows, macOS, and Linux, I thought, it is going to be smoother experience. At the end of the day I did my release to my RepoContextPackager, but the path to get there was much harder than expected. Looking back, I should have stuck with purely making Releases on my repo from the start.

The Initial Setup

I started by creating custom overlay ports for vcpkg. The structure seemed straightforward:

ports/
└── repositorycontextpackager/
     ├── portfile.cmake
     └── vcpkg.json

The SHA512

Every time I made a commit and created a new tag, I had to update the REF in portfile.cmake, set SHA512 0 as a placeholder, then run ./vcpkg install repositorycontextpackager --overlay-ports=ports and wait for it to fail so I could copy the real SHA512 from the error message, update the portfile with the correct hash, and try again. I did this countless times. Even after changing the SHA key, vcpkg gave cryptic fetch content errors.

The Dependency Nightmare

My project used:

System libgit2 (via Homebrew/pkg-config)
tomlplusplus
GTest

But vcpkg wanted everything through vcpkg. Mixing them caused linking errors.I also had to use git pull origin main --no-rebase because of direct changes to main. I realized, allowing changes to main branch was really a bad idea, I should not have allowed that. Because even though, code works I will end up destroying git history in the repo.

The Change

After endless hours fighting:

Fetch content errors
Linking errors
SHA512 checksums
Breaking and fixing the build repeatedly

I gave up on vcpkg.I reverted to my simple CMakeLists.txt to make my life easier, I realized, using packager in C++ and handling all these dependencies will not happen in one sitting. I built release binaries for Windows and macOS, and my friend tested the macOS build: "It works perfectly." Everything ran as expected. Also, I originally used install(TARGETS repoctx RUNTIME DESTINATION bin) which explicitly tells CMake that the target is a runtime executable and should go in the bin directory. However, I switched to the simpler install(TARGETS repoctx DESTINATION bin) which lets CMake automatically determine the artifact type (executable, library, etc.) and handle it appropriately. This simpler approach proved more portable across different build systems and worked perfectly when my friend tested it on macOS

Don't Mix Dependency Sources

While doing this project in C++ I came up to the solution that it is better to choose one package and not mix with others.As a result of it, I decided to choose dependency strategy in the early stages of my next projects. Also, I need to document build process properly.

Gained Knowledge

After all of this, I realized that it's better to allocate extra time for tasks such as releases in order to avoid burnout. In projects, especially those related to configuration decisions, choices have to be made carefully at the start.

My Battle with CI/CD - CMake, Windows, and Cross-Platform Pain

ElshadHu — Sat, 15 Nov 2025 03:47:55 +0000

Set Up

First, I created CI.yml to add a CI workflow to my project.I realized that GitHub's recommended CI file wouldn't help much. For that reason, I planned to write my own version supporting Linux, Windows, and macOS. This decision led to significant struggles while making a PR to test my naive CI implementation. At the same time, I added comprehensive tests by creating git_info_test to test git_info module and test CI , basically killing two birds with one stone.

The `vcpkg` Removal Decision

After creating a PR from my add-more-tests branch, CI failed. I checked the error and figured out I needed to remove one line for Windows:

    - name: Setup vcpkg and install libgit2 (Windows)
      if: runner.os == 'Windows'
      run: |
        git clone https://github.com/Microsoft/vcpkg.git
        cd vcpkg
        .\bootstrap-vcpkg.bat
        .\vcpkg install libgit2:x64-windows  # This is redundant!

I realized this manual install was unnecessary because my project uses vcpkg.json with:

{
  "name": "repoctx",
  "version": "0.1.0",
  "dependencies": [
    "libgit2"
  ]
}

When vcpkg sees this file, it automatically installs dependencies,no manual install needed in CI. I corrected my mistake on GitHub's UI, but problems arose afterward. Back on my local branch, I continued working on my tests and made changes to fix Windows-specific issues. Without pulling the remote changes first, I was forced to use git push origin add-more-tests --force-with-lease, which I don't recommend to anyone. It can destroy work. While it's a safer version of git push --force (especially when two people work on the same branch, as it won't allow the force push), it's still bad practice and not a good standard, especially when contributing to open source projects, most of which prohibit force pushing.

Windows Testing Nightmare

Writing tests for Git operations seemed simple, create a temporary repo, run some git commands, verify the output. It worked perfectly on my Mac. Then I pushed to CI and watched Windows fail spectacularly.

My Original Test Code

void createTestGitRepo() {
    system("git init");
    system("git config user.email 'test@gmail.com'");
    system("git config user.name 'Test User'");
    system("git commit -m 'Initial Commit'");
}

Can you see the errors that aren't suitable for Windows? I removed single quotes and spaces. At this moment, I truly understood the benefit of CI workflows, something I'm used to seeing in open source projects, which keeps our master or main branch safer.

The Working Solution


    system("git init");
    system("git config user.email test@gmail.com");
    system("git config user.name TestUser");
    system("git commit -m InitialCommit");

I thought it was time to relax, but CI failed again. I wasn't handling cleanup gracefully for Windows. By passing std::error_code, the function stores the error instead of throwing an exception. I also gave each test its own directory, which prevents test contamination, works around Windows file locking, and is required for reliable cross-platform tests.

Graceful Error Handling and Different Directories for Tests

 void SetUp() override {
        auto testInfo = ::testing::UnitTest::GetInstance()->current_test_info();
        std::string testName = std::string(testInfo->name());

        testRepoPath = std::filesystem::temp_directory_path() / ("test_git_repo_" + testName);

        if(std::filesystem::exists(testRepoPath)) {
            std::error_code ec;
            std::filesystem::remove_all(testRepoPath, ec);
        }

        std::filesystem::create_directories(testRepoPath);
    } 

    void TearDown() override {
        if(std::filesystem::exists(testRepoPath)) {
            std::error_code ec;
            std::filesystem::remove_all(testRepoPath, ec);
        }
    }

Contribution

After all, I made a PR to my groupmate's project. The project structure, modules, and setup were clear, so I didn't struggle much. I kept each test independent to avoid interference. Each test creates its own spy on process.stderr.write to capture the output, ensuring correct messages are written. Finally, it cleans up by restoring the original function.

Usage

    it('handles verbose mode correctly', () => {
        const stderrSpy = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);

        logVerbose('Debug info', true);
        expect(stderrSpy).toHaveBeenCalledWith('Debug info\n');

        stderrSpy.mockClear();
        logVerbose('Hidden info', false);
        expect(stderrSpy).not.toHaveBeenCalled();

        stderrSpy.mockRestore();
    });

Gained Knowledge

I learned the importance of being more precise and paying attention to every detail before making a PR. I already knew this, but CI was a great lesson in precision. I'm also becoming more comfortable writing tests and understanding test semantics.