DEV Community: Pavanipriya Sajja

Stop Reading Documentation. Start Reading GitHub Issues.

Pavanipriya Sajja — Wed, 03 Jun 2026 22:24:28 +0000

Reading the docs isn't enough. The most valuable developer feedback lives inside GitHub issues, bug reports, and feature discussions. In this article, I share the checklist I use to mine repositories and coding frame work for the issue, uncover real developer pain points, and turn engineering conversations into meaningful UX research findings.

Why Coding Is the Difference Between Reading and Researching

Reading GitHub issues without a coding framework produces impressions. Coding them produces findings.

When you code an issue, you are making an explicit analytical decision about what type of signal it contains. You are saying: "This issue is evidence of a feedback gap at Stage 4 (model loading), filed by an ML engineer with a new user experience level, in KServe v0.11, and it directly answers my research question about what information engineers need during model loading that the product currently does not provide."

That sentence assembled from your coding decisions is a finding. Multiply it across 100 issues and you have a research study.

Without coding, you have 100 anecdotes. With coding, you have a dataset.

It makes patterns visible. When you code 20 issues and notice that 14 of them map to the same stage and the same challenge type, you have found your highest-priority UX problem. You would never see that pattern by reading casually.

It makes your findings defensible. "Engineers struggle with KServe" is an opinion. "18 issues across v0.10–v0.13 filed by engineers in their first deployment show identical feedback gap patterns at Stage 4, with an average of 14 comments per issue" is a finding.

It separates your role from the engineer's role. Engineers read GitHub issues as bug reports. You read them as evidence of design decisions. The coding framework is the analytical lens that makes your UX reading possible.

Step 1: Collect Basic GitHub Metadata

Before any analysis, capture the foundational data for every issue. This establishes the quantitative baseline for your research report.

Issue Metadata Checklist
─────────────────────────────────────────────
[ ] Issue URL / Number        e.g. #1234
[ ] Issue creation date
[ ] Issue category            Bug | Question | Feature Request | Docs
[ ] Labels                    e.g. InferenceService, Control Plane, Kubernetes
[ ] Current state             Open | Closed | Stale | Merged
[ ] Resolution type           Code Fix | Docs Update | Workaround | No resolution
[ ] Total comment count
[ ] Unique commenter count
[ ] Ping-pong count           Back-and-forth before root cause found
[ ] Brief issue summary

The "ping-pong count" — the number of back-and-forth diagnostic comments between the user and maintainers before the root cause was identified — is a particularly powerful metric. High ping-pong means the product gave the user no diagnostic signal. That is a UX failure in the product itself.

Step 2: Identify Who Is Reporting — The Demographics Checklist

Understanding user demographics is essential in any research project because it tells you whose problems to prioritise. In interviews or surveys, you simply ask. In GitHub mining, you have to infer from signals embedded in the issue itself.

In the context of KServe, we are primarily trying to distinguish between three main personas.

Technique 1: Read the vocabulary

The language engineers use tells you their world immediately.

Persona	Typical keywords	What they focus on
Data Scientist / ML Engineer	PyTorch, TensorFlow, HuggingFace, weights, predictor, artifact, S3, inputs/outputs	The model itself — getting a Python script to serve predictions
Platform / DevOps Engineer	CRDs, Istio, Knative, ingress, RBAC, service account, Helm, multi-cluster, HPA	Infrastructure, networking, security, cluster stability
Application Developer	REST API, gRPC, JSON payload, curl, SDK, timeout, 503 error, endpoint	Consuming the model — integrating the endpoint into a larger application

Technique 2: Look at the "altitude" of the problem

KServe issues typically fall into three abstraction layers:

Infrastructure/cluster layer — Kubernetes, Istio, Knative, CRDs → Platform Engineer
Pipeline/deployment layer — model formats, runtimes, autoscaling, vLLM/Triton config → ML Engineer
Consumption layer — API latency, authentication, payloads, querying from an app → App Developer

Technique 3: Read what they attached as evidence

Platform engineers dump kubectl describe outputs, cluster events, Helm values files, or Istio configurations — they speak in Kubernetes YAML
Data scientists share Python stack traces, Jupyter Notebook snippets, or InferenceService configuration blocks without knowing what the pods underneath are doing
App developers provide curl commands, Postman logs, or application server logs showing connection timeouts

Technique 4: Ask "what are they trying to accomplish?"

"I just want to see if my new LLM works with this payload" → ML consumer/creator
"I am trying to automate onboarding of 50 teams onto a shared KServe instance" → Platform builder

Quick triage question: Is this person treating KServe as an infrastructure component or as a model-delivery tool? Infrastructure → Platform/DevOps. Model-delivery → Data Scientist/ML Engineer.

Demographics Checklist
─────────────────────────────────────────────
[ ] Inferred user persona
      Data Scientist / ML Engineer
      Platform / DevOps Engineer
      Application Developer
      ML-expert / K8s-novice  ← the hardest edge case
      Unclear

[ ] Experience level
      New user (first issues, plain English, no version info)
      Experienced (provides full env, logs, rules out causes)
      Unclear

[ ] Deployment environment
      Local (Kind / Minikube)
      Cloud managed (EKS / GKE / AKS)
      On-premises / bare metal

[ ] Deployment scale
      Single-cluster
      Multi-cluster
      Multi-tenant

[ ] Deployment method
      Helm | Kustomize | ArgoCD / GitOps | Direct kubectl

The hardest edge case: ML-experienced / Kubernetes-novice engineers. They write technically confident issues about model formats or serving runtimes — but are completely confused about Istio or Knative. Always code these as a separate category — they reveal a completely different class of UX failure.

Step 3: Identify Issues Using Plain-Language Signals

The key insight for non-technical UX researchers: you do not need to understand the technical content of an issue to identify its UX signal. The signals are in the language, not the configuration.

Here is how to read any issue in under two minutes.

Signal 1: Time language → friction severity

Scan for time words before you read anything else.

Time word	Severity	What it means
"minutes"	Low	Minor gap, quickly resolved
"hours"	Medium	Significant friction, real work lost
"days"	High	Severe friction, deadline impact
"weeks" / "months"	Critical	Product-level failure
"gave up" / "switching to X"	Abandonment	User is leaving

Real example: "I've spent the last three days trying to figure out why my model stays in Unknown status."

You do not need to know what Unknown status means. "Three days" tells you this is a high-severity finding.

Signal 2: The "but" inflection point

Every friction-revealing issue has the word "but" at a specific moment. Everything before "but" is what the engineer did correctly. Everything after is where the product failed them.

Real example: "I followed the quickstart exactly but the webhook never became Ready."

Before "but" = user followed instructions. After "but" = product gave instructions that led to failure. That is a documentation UX finding, not a technical bug.

Real example: "The status shows Ready but every curl request returns 503."

This is the "misleading success" friction type, the product declared success when the user's actual goal was completely unmet. One of the most trust-destroying UX failures possible.

Signal 3: "I thought / I assumed / I expected"

These phrases directly reveal a mental model gap — the engineer built an incorrect picture of how the system works, and reality contradicted it.

Real example: "I thought Transformer meant a language model component like BERT. Turns out it's just data preprocessing."

This is not a bug. It is a naming failure. "Transformer" means attention-based neural architecture in the ML world. KServe uses it to mean "a component that preprocesses data before the model." Every ML engineer who encounters this name builds the wrong mental model from it.

Real example: "I assumed KServe would track model versions automatically — like a proper ML serving platform should."

This is scope confusion — the engineer's expectation of what KServe is does not match what it actually is. That mismatch is a design communication failure, not a user error.

Signal 4: "I had to / I also had to"

Each "I had to" signals a missing workflow step — something the engineer needed that the product should have provided.

Real example: "I had to write a polling script to check when the InferenceService became Ready, because there's no built-in wait command."

Count the "I had to" chains in an issue. An issue with four of them put four separate manual burdens on an engineer for a task that should have been automated.

Signal 5: Tool name count

Scan for capitalized tool names: Knative, Istio, Prometheus, MLflow, Argo, Triton, HuggingFace. Count them.

1 tool — single-product issue
2 tools — integration friction
3+ tools — cross-service challenge; the user must navigate multiple systems for what should be one task

Signal 6: The apology

When an engineer writes "sorry if this is a basic question" — that is not politeness. That is evidence the product made a competent person feel responsible for the product's communication failure.

Step 4: The End-to-End Coding Checklist

Once you have your basic metadata and demographics, here is the full coding structure to apply to every issue.

4A. Deployment Stage

Map every issue to one of 8 stages. This tells you where in the journey the product is losing engineers.

Deployment Stage Checklist
─────────────────────────────────────────────
[ ] Stage 1 · Setup         Installing KServe and dependencies
                             Signals: "webhook not ready" · "quickstart fails"

[ ] Stage 2 · Storage       Getting the trained model accessible
                             Signals: "access denied" · "model not found" · "storageUri"

[ ] Stage 3 · Configuration Writing the InferenceService YAML spec
                             Signals: "minimum config?" · "deprecated field" · "required fields?"

[ ] Stage 4 · Loading       Applying config and waiting for model to load
                             Signals: "Unknown status" · "how long?" · "no logs" · "OOMKilled"

[ ] Stage 5 · Network       Reaching the deployed endpoint
                             Signals: "Ready but 503" · "connection refused" · "EXTERNAL-IP pending"

[ ] Stage 6 · Inference     Sending requests and getting predictions back
                             Signals: "400 error" · "what format?" · "V1 vs V2 protocol"

[ ] Stage 7 · Hardening     Making the deployment production-reliable
                             Signals: "zero downtime update" · "autoscaling conflict" · "SLA"

[ ] Stage 8 · Day-2 ops     Updating, monitoring, governing over time
                             Signals: "rollback" · "update my model" · "60 models across teams"

[ ] Cross-stage?            Root cause in Stage X, discovered at Stage Y
                             (delayed discovery = highest severity finding)

The most important finding: Stages 4 and 5 consistently produce the highest issue volume in K-Serve. The product is completely silent at the two moments when engineers are most anxious and most blind.

4B. Usability Challenge Type

Usability Challenge Checklist
─────────────────────────────────────────────
[ ] U1 · Learnability breakdown
         Cannot figure out how to do the task the first time
         Signal: "how do I" · already answered in docs · confused by concepts

[ ] U2 · Error recovery failure
         Hits error, can't understand it, doesn't know which log to check
         Signal: pastes cryptic error, "stuck for days", tries random things

[ ] U3 · Feedback & visibility gap
         System gives no signal — Unknown, Pending, complete silence
         Signal: "nothing happens" · "how long should this take?" · "no logs"

[ ] U4 · Configuration complexity
         Too many fields, unclear defaults, no minimum viable spec
         Signal: "is all of this needed?" · "which fields are required?"

[ ] U5 · Mental model mismatch
         Expectation contradicts how system actually works
         Signal: "I expected" · "I thought" · "this makes no sense"

[ ] U6 · Workaround proliferation
         User invented their own solution to fill a product gap
         Signal: "I wrote a script" · "I had to" · shared snippets in comments

4C. Developer Friction Type

Developer Friction Checklist
─────────────────────────────────────────────
[ ] F1 · Invisible wall        System silent — nothing to debug
[ ] F2 · Misleading success    "Ready" but goal completely unmet
[ ] F3 · Hidden prerequisite   Required knowledge never communicated until failure
[ ] F4 · Terminology confusion Word means something different in this context
[ ] F5 · Broken feedback loop  Can't tell if a change had any effect
[ ] F6 · Forced context switch Must configure Istio/Knative to complete one KServe task
[ ] F7 · Documentation gap     Knows what they want, can't find how to do it
[ ] F8 · Accumulated friction  5–6 small frictions in sequence → abandonment signal

4D. System & Environmental Challenges

System Challenge Checklist
─────────────────────────────────────────────
[ ] Ownership ambiguity    KServe says "that's Istio", Istio says "that's KServe"
[ ] Abstraction leakage    InferenceService was meant to hide Knative/Istio; it doesn't
[ ] Observability gap      Logs scattered across 4+ components; no unified view
[ ] Role boundary collision ML engineer task structurally requires platform engineer action
[ ] Upgrade path fragility  Every version upgrade risks production breakage

Environmental Challenge Checklist
─────────────────────────────────────────────
[ ] Managed K8s divergence   EKS, GKE Autopilot, OpenShift behave differently
[ ] Corporate proxy / air-gap No public internet; private registry; air-gapped
[ ] GPU & hardware           OOMKilled, VRAM insufficient, driver mismatch
[ ] Org security policy      OPA, Gatekeeper, PodSecurityAdmission blocking KServe
[ ] On-premises / hybrid     No managed LoadBalancer, NFS storage, bare metal
[ ] Regulated / compliance   HIPAA, SOC2, GDPR, data residency requirements

4E. Version Tracking (Essential for Progress Measurement)

This is what transforms your research from a snapshot into a longitudinal UX health report.

Version Tracking Checklist
─────────────────────────────────────────────
[ ] KServe version (exact)         e.g. 0.11.2
[ ] Previous version (if upgrade)  e.g. 0.10 → 0.11
[ ] Kubernetes version             e.g. 1.27
[ ] Cloud provider                 EKS / GKE / AKS / On-prem / Local
[ ] Version stated by:             User upfront | Maintainer had to ask | Never provided
[ ] Upgrade experience:            Better | Same | Worse | New regression introduced
[ ] Chronic pain signal:           Same issue present in prior version? Yes / No

The chronic problem list — friction points that appear in the top-3 across three or more versions — is your most powerful finding. A problem that survived three release cycles is not a bug. It is an architectural decision.

4F. LLM Inference Tracking (Track Separately — Fastest Moving Component)

LLM Inference Checklist
─────────────────────────────────────────────
[ ] Is this an LLM issue?          Yes | No | Hybrid
[ ] LLM model family               Llama | Mistral | Qwen | Gemma | Custom
[ ] LLM runtime                    vLLM | TGI | OpenAI-compatible | Custom
[ ] Capability attempted:
      Basic inference
      Streaming tokens (SSE)
      Multi-GPU / tensor parallelism
      LoRA / adapter serving
      HuggingFace Hub authentication
      OpenAI API compatibility
      Quantisation (GPTQ, AWQ)

[ ] LLM-specific challenge:
      GPU OOM / VRAM insufficient
      Model loading with no progress signal
      Streaming failure through gateway
      HuggingFace auth failing in cluster
      Runtime version lag behind vLLM/TGI ecosystem
      No LLM-specific metrics (token throughput, TTFT)

[ ] Innovation lag signal:
      Date of capability request: ___________
      Date KServe released support: ___________
      Gap (days): ___________

4G. Research Question Mapping

For each issue, record which research question it provides evidence for. This anchors your mining to your study goals.

Research Question Mapping
─────────────────────────────────────────────
Current-state questions (what is broken today)
[ ] RQ1 · First deployment challenges across roles and experience levels
[ ] RQ2 · Workflow gaps between deployed model and reliable production
[ ] RQ3 · Observability and debugging challenges by stage
[ ] RQ4 · LLM deployment challenges vs classical ML serving
[ ] RQ5 · Environmental factors shaping deployment experience
[ ] RQ6 · How challenges evolved across versions (your unique longitudinal contribution)
[ ] RQ7 · Design changes that would most reduce friction

UX improvement questions (what should be designed differently)
[ ] UX1 · Time-to-first-inference reduction
[ ] UX3 · Model loading progress visibility (highest volume finding)
[ ] UX4 · Self-service diagnostic experience
[ ] UX9 · LLM mental model bridge (vLLM/HuggingFace → KServe)
[ ] UX11 · Environment validator / dependency pre-flight checker

4H. UX Finding Template

Once you spot a pattern across multiple issues, record it here. One template per pattern — not per issue.

UX Finding Template
─────────────────────────────────────────────────────────────
Finding type:       ___________________________________________
Affected users:     Role · Experience level · Version band
Deployment stage:   ___________________________________________
Evidence:           N issues · Date range · e.g. "14 issues, 2022–2024"
Best quote:         Under 25 words — your strongest evidence
─────────────────────────────────────────────────────────────
UX finding statement:
"Engineers [doing X] cannot [accomplish Y] because [design gap Z],
 which means [impact on time / confidence / adoption]."
─────────────────────────────────────────────────────────────
Severity:           Low | Medium | High | Critical
Chronic?            Present across ___ versions
Design recommendation: ________________________________________
Research question answered: ___________________________________

Step 5: The Session Checklist — Before Calling Mining Complete

Mining Session Completion Checklist
─────────────────────────────────────────────
Issue coverage
[ ] 50+ general deployment issues coded across all 8 stages
[ ] 30+ LLM inference issues coded and version-tracked
[ ] 10+ issues per major version band (v0.10, v0.11, v0.12, v0.13+)
[ ] 15+ upgrade issues with before/after UX delta recorded
[ ] Top 30 most-commented issues reviewed (sort:comments-desc)
[ ] 10+ abandoned issues (open 6+ months, last message unanswered)
[ ] 15+ success cases (quickly-resolved — positive signal baseline)
[ ] All enhancement/feature-request labels reviewed
[ ] All competitive tool mentions captured (BentoML, Seldon, Ray Serve)

Baseline measurement
[ ] Metrics table filled per version band:
      Issue count | Avg comments | 7-day resolution rate | Emotional language %
[ ] Top-3 friction points per version — chronic problem list built
[ ] LLM innovation lag calculated for 5+ capabilities
[ ] Version reporting rate: % of issues that include version upfront

Quality
[ ] 15% of issues coded by a second researcher (inter-rater reliability)
[ ] Every research question has at least 3 issues as evidence
[ ] One finding statement written per significant pattern found

How This Benefits UX Researchers and the Community

For UX researchers without a technical background

The biggest barrier to studying developer tools as a UX researcher is the assumption that you need to understand the code to understand the problem. This framework removes that barrier entirely.

When you code an issue, you are not evaluating the correctness of someone's Kubernetes configuration. You are recording what the issue reveals about the human experience of using the product. "Status shows Unknown for 20 minutes" tells you everything you need regardless of whether you understand what Unknown means technically. The product left a user without feedback during its most critical operation. That is a UX finding independent of any technical knowledge.

For product managers and maintainers

The coding sheet converts anecdote into pattern. Instead of "users seem to struggle with deployment," you can say "14 of 20 issues sampled from v0.11 show feedback gap failures at Stage 4, with an average of 18 comments per issue, suggesting loading state communication is the highest-priority improvement target for this version band."

That is a product roadmap argument. The coding sheet built it.

For open source contributors

The research questions embedded in the coding sheet map directly to design recommendations with evidence behind them. A contributor who wants to make a meaningful impact on user experience now has specific, evidence-backed targets — not "improve docs" but "add granular status conditions per loading phase that distinguish between 10 failure modes currently all reporting as Unknown."

For the community as a whole

When researchers share findings publicly in CNCF Blogs, KubeCon talks, or articles like this one — the coding framework makes the research reproducible. Other researchers can apply the same checklist to a different version or a different tool and compare results. That cumulative body of evidence is what eventually changes product direction.

Conclusion

GitHub issues are not a bug tracker. They are a longitudinal, naturalistic record of where real engineers encounter the gap between what a product promises and what it delivers.

A coding sheet is the analytical framework that transforms that record into research. Without it, you are reading. With it, you are studying.

The framework I built for KServe — covering demographics, deployment stages, usability challenges, friction types, mental model gaps, system challenges, environmental barriers, version tracking, and LLM inference — did not emerge from theory. It emerged from reading hundreds of issues and asking the same question every time: what is the UX researcher's reading of this, beyond what the engineer sees?

The answer is always the same: engineers see symptoms. The coding sheet helps you see the design decisions that caused them.

Start with the "but" sentence. Work backwards to the design failure. Code it. Repeat 100 times. Then write the research report.

This coding framework was developed as part of a UX research study on ML model deployment in KServe. If you are working on similar research in the cloud-native or MLOps space, I would love to hear your thoughts.

Analyzing 1,000 Engineering Problems Through GitHub Data

Pavanipriya Sajja — Tue, 26 May 2026 05:38:22 +0000

In my previous article, I wrote about the GitHub mining process in user research and explained how it can benefit engineers, organizations, and open-source communities. Here is the link:

Pavanipriya Sajja

May 24

What Developers Don’t Say in Interviews—but Show on GitHub

#cloudnative #devrel #opensource #ux

10 min read

In this article, I am going to explain the step-by-step process for conducting the research and creating a report.

Step: 1 Define Research Questions

Before starting GitHub mining research, first define the problem you want to understand. Ask yourself: What challenge am I trying to investigate? This step helps make the research focused and prevents collecting unnecessary data. For example, in a project like KServe, you may want to understand why users struggle to deploy models, what usability issues make onboarding difficult, which workflows create the most developer frustration, or what causes deployment failures. These questions help identify the user experience (UX) and developer experience (DX) problems that need deeper investigation.

After identifying the problem area, create clear research questions to guide the study. Start with one primary research question that represents the main goal of the research.

Example Primary Research Question:
“What usability barriers do users experience while deploying models using KServe?”

Then create several sub-questions to explore specific areas in more detail, such as:

Which tasks generate the most confusion?
What documentation gaps exist?
Which error messages appear repeatedly?
How do developers solve issues?

These smaller questions help organize data collection and analysis.

At the end of this stage, your output should include three things:

Research Objective — explains the overall purpose of the study.
Research Questions — define what you want to investigate.
Scope — sets boundaries for the research by describing what data, users, repositories, workflows, or time period will be included.

Having these clearly defined creates a strong foundation before moving into GitHub data collection and analysis.

Step: 2 Select the GitHub Data Sources

When conducting GitHub mining for usability research, it is important to understand that GitHub contains multiple sources of data, and each source provides a different perspective on the user experience (UX) and developer experience (DX). Instead of collecting information from everywhere, researchers should prioritize sources that help answer their research questions.

For example:

Issues help uncover user pain points and reveal where users struggle.
Pull Requests (PRs) show how problems are fixed and provide insight into engineering decision-making.
Discussions help researchers understand user expectations, questions, and community needs.
Commits reveal engineering priorities by showing what teams spend time improving.
Documentation helps evaluate communication quality and whether instructions are clear enough for users.
Labels provide structure by categorizing issues into themes or problem areas.
Comments often contain valuable insights into user emotions, frustrations, collaboration patterns, and real-world workarounds.

By combining these sources, researchers can build a more complete understanding of both technical challenges and usability challenges that users experience.

Step : 3 Create Inclusion and Exclusion Criteria

When conducting GitHub mining for usability research, it is important to create clear inclusion and exclusion criteria before collecting data. This step makes your study systematic, transparent, and reproducible, which means another researcher could follow the same process and reach similar results. Inclusion criteria define what data should be included in the study, while exclusion criteria define what should be removed to avoid irrelevant or low-quality information.

For example, you may decide to include:

Open and closed issues because both contain valuable user feedback and problem history.
Deployment-related issues to understand deployment challenges.
User questions to identify confusion and unmet needs.
Documentation complaints to evaluate communication and onboarding quality.
Feature requests to understand user expectations and improvement opportunities.

At the same time, you should exclude:

Spam issues because they do not provide useful research insights.
Empty issues with no meaningful information.
Duplicate issues that repeat the same problem.
Pure backend implementation discussions that do not relate to user experience or usability goals.

To make your research process clear and repeatable, document:

Time range of the data collection.
Number of issues selected.
Selection logic used to choose the dataset.

For example:

Dataset Definition:
“Issues created between January–June 2026.”

Recording these decisions helps explain how the data was collected and increases the credibility and reliability of your research findings.

Step : 4 Data Collection (Mining):

After defining your research questions and selecting your data sources, the next step is to collect the data in a structured way. A simple and effective approach is to create a spreadsheet where each row represents one GitHub issue or record. Organizing data in a spreadsheet makes analysis easier, helps identify patterns, and ensures that findings can be traced back to the original source.

Typical columns may include ID, Issue Title, Link, Date, Type, Description, Comments, and Labels.

During data collection, capture both metadata and content details.

Metadata includes information such as:

Issue number
Whether the issue is open or closed
Author
Any labels attached to the issue

Then collect the content itself by recording:

Issue description
Steps to reproduce the problem
Error messages
Resolution or final outcome (if available)

It is also important to analyze the conversation around the issue, including:

Maintainer responses
Community suggestions
Workarounds shared by users

These interactions often reveal usability challenges and show how users overcome problems.

For example, imagine a GitHub issue titled:

“Deployment stuck after InferenceService creation”

Instead of only recording the issue title, document:

User goal — what they were trying to achieve
Failure point — where the process broke
Fix — what solved the issue
Resolution time — how long it took to resolve

Capturing data this way transforms GitHub issues into research evidence that helps uncover user behavior, developer pain points, and opportunities to improve usability and developer experience (DX).

Step: 5 Clean and Prepare the Data

GitHub data is often messy and unstructured, so it needs to be cleaned before analysis. First, remove irrelevant data such as empty reports, duplicate issues, and non-user-related discussions, since these do not help in understanding usability problems. Cleaning the data ensures that only meaningful information is included in the study.

Next, normalize the data by standardizing how issues are written. This means converting different phrases that describe the same problem into a consistent format. For example, “deployment broken” can be normalized to “Deployment Failure”. This makes it easier to compare and analyze similar issues across the dataset.

After normalization, create clear categories or themes to group similar issues together. For example, different original texts like “Deployment failed,” “Cannot start,” and “Confusing YAML” can be grouped under standard themes such as Deployment or Configuration. This helps researchers identify patterns, understand common problem areas, and analyze usability issues more effectively.

Step: 6 Qualitative Analysis (The Most Important Step)

This is where UX research actually begins. At this stage, you read GitHub issues carefully, line by line, and start applying a process called coding, which helps turn raw text into meaningful insights.

The first step is open coding, where you highlight important observations from each issue. For example, if an issue says, “I followed docs but model never becomes ready,” you break it into simple codes like documentation confusion, deployment failure, and missing feedback. These codes describe what is really happening in the user’s experience.

Next is axial coding, where you group similar codes together to find patterns. For example, codes like missing instructions, YAML confusion, and setup unclear can all be grouped under the theme Documentation. This step helps organize individual problems into broader categories.

Finally, you do selective coding, where you connect themes to form deeper insights. For example, under the theme Documentation, you might conclude that users frequently struggle during deployment because the setup instructions do not match real cluster behavior. This final step transforms raw GitHub issues into clear UX insights that explain user problems and system gaps.

Step : 7 Quantitative Analysis

To understand usability issues in GitHub mining research, it is important to measure patterns in the data. This helps researchers move from individual issues to broader insights about user experience (UX) and system behavior. Different metrics are used to analyze the data in a structured way.

For example, the number of issues shows the overall volume of problems, while resolution time indicates how much effort or friction is needed to fix them. The comment count helps measure complexity, since more discussion often means more complicated issues. Label frequency highlights common problem areas or hotspots, and repeat reports show how severe or recurring a problem is.

After collecting these metrics, researchers can create simple visualizations to understand trends more clearly. For example, they can analyze issues by category, track issue trends over time, measure the average time to close issues, and identify the top recurring pain points.

For instance, after analyzing the data, you might find that most issues fall into a few key areas such as Deployment (42%), Documentation (28%), Configuration (20%), and Other (10%). This kind of breakdown helps researchers clearly see where users struggle the most and where improvements are needed.

Step : 8 Convert Findings into UX Insights

In UX research, especially in GitHub mining, it is important to move beyond simple counts like “20 users reported errors.” Numbers alone do not explain the real problem. Instead, the focus should be on understanding why users are struggling and what it means for their experience. For example, instead of just reporting issues, we can say: “Users struggle because system feedback is unclear.”

To do this, researchers follow a simple thinking process: Observation → Pattern → Insight → Recommendation. First, you make an observation by looking at raw data, such as issues or comments. Then you identify a pattern by grouping similar observations together. After that, you form an insight, which explains what the pattern means in terms of user behavior. Finally, you create a recommendation that suggests how to improve the system.

For example, an observation might be that many users ask the same deployment question. The pattern could be identified as documentation gaps. From this, the insight becomes that users cannot predict system behavior during deployment. The final recommendation would be to improve the deployment guide so users have clearer instructions and better understanding.

Report the findings

Reporting findings in a GitHub mining UX research report is not just about listing issues—it’s about turning raw GitHub data into clear, evidence-based insights and actionable recommendations.

A good findings section should answer:

What is happening?
Why is it happening?
Why does it matter?
What should be done about it?

I am going to explain entire process on how to report research findings.

1. Structure of the Findings & How to Write Each Finding (Step-by-Step Template)

When writing each finding in GitHub mining research, it is helpful to follow a clear step-by-step structure. First, start by naming the theme clearly. The theme should describe a real UX problem, not just a GitHub issue name or generic label. For example, instead of “Issue 23” or “Bug problems,” use meaningful titles like “Deployment Configuration Confusion” or “Lack of Clear Error Messages during InferenceService Setup”. This helps readers immediately understand the user experience problem.

Next, describe what you found at a high level. This is a short summary of the main issue, such as users struggling with model deployment in KServe, especially during YAML configuration and InferenceService creation. This gives an overall picture of the problem before going into details.

After that, add strong evidence from GitHub data, such as issue numbers and short user quotes. For example, users might say “I followed the documentation but the InferenceService never becomes ready” or “The YAML example does not work in my cluster.” This evidence can include direct quotes, issue references, or summarized patterns from multiple users.

Then, identify the pattern by combining similar observations. For example, multiple issues may show that users struggle with YAML configuration and unclear deployment steps. This helps turn individual data points into a broader trend.

After identifying the pattern, explain the impact on users. This describes how the issue affects their experience, such as failed deployments, repeated retries, increased onboarding time, confusion, frustration, or even abandonment of the tool.

The next and most important step is writing the insight, which explains why the problem is happening. For example, users may not receive clear feedback from the system during deployment, making it hard to know whether errors come from configuration or system issues. This step goes beyond the data and provides interpretation.

Finally, add a clear and actionable recommendation to improve the system. For example, improving documentation with validated YAML examples and adding real-time status messages for InferenceService creation can help users understand and resolve issues more easily.

2. Full Example of a Well-Written Finding

Finding 1: Deployment Configuration Confusion

Description: Many users experienced difficulties deploying models using KServe, particularly during YAML configuration and InferenceService setup.

Evidence
“The InferenceService never becomes ready after applying YAML.” — Issue #1245
“Example YAML does not work in my Kubernetes cluster.” — Issue #1310
“I am not sure what fields are required.” — Issue #1288

Pattern : Multiple users reported inconsistent or unclear YAML configuration requirements during deployment.

Impact : This results in failed deployments, repeated debugging attempts, and delays in onboarding new users.

Insight: The system does not provide sufficient validation or feedback during deployment, leading to uncertainty in diagnosing configuration issues.

Recommendation : Provide validated, version-specific YAML templates Add real-time deployment status and error messages
Improve documentation with step-by-step deployment flow

3. How Many Findings Should You Include?

For a strong GitHub mining research report, it is better to focus on a small number of meaningful themes instead of too many small issues. Usually, 3 to 6 major themes is ideal because it keeps the analysis clear, focused, and easy to understand. If you include too many themes or individual issues, the report can become confusing and lose its main message.

Common examples of good themes include areas like deployment issues, documentation gaps, error handling problems, configuration complexity, and performance concerns. These themes represent repeated patterns across many GitHub issues and help explain the main usability and developer experience challenges in a structured way.

4. Visual Ways to Strengthen Findings

To understand patterns in GitHub mining research, it is important to present the findings in a clear and structured way using visual summaries. You can include a theme frequency chart, which shows how often each problem appears in the data, such as Deployment issues (40%), Documentation (25%), and Errors (20%). This helps quickly identify which areas users struggle with the most.

You can also include an issue timeline, which shows when problems occur most frequently and helps reveal spikes or trends over time. Another useful element is quote highlights, where you include 2–3 strong user quotes for each theme to support your findings and give real evidence of user experience.

Together, these elements make your analysis clearer, more visual, and easier to understand.

Conclusion:

In this article, I explained the complete step-by-step process of GitHub mining for UX research, from defining research questions to converting raw GitHub data into meaningful insights. We saw how issues, pull requests, and documentation can be used to understand real user and developer experiences, and how structured methods like coding, categorization, and pattern analysis help transform messy data into clear findings. The goal of this process is not just to analyze issues, but to understand why users struggle and how systems can be improved. By following this approach, researchers and engineers can identify real usability problems, improve developer experience, and build better open-source tools through evidence-based insights.

What Developers Don’t Say in Interviews—but Show on GitHub

Pavanipriya Sajja — Sun, 24 May 2026 01:47:04 +0000

When I started working on my usability study project with KServe, I interacted with KServe users to understand the challenges they were experiencing while using the platform. During these conversations, users frequently mentioned GitHub issues and the problems they encountered during development and deployment.

This led me to explore how engineers solve problems by using GitHub repositories and collaborating in open-source communities. As part of my research, I started reading articles and academic research papers online, and I came to know this is the best approach and many organizations are already following this methods to identify the developer problems.

GitHub mining (also called repository mining, issue mining, or software repository mining) is increasingly used as a user research and UX research method in open-source ecosystems because it allows researchers to understand user challenges from real-world interactions rather than only interviews or surveys.

Here is the research paper links: 1. Mining Developer Behavior Across GitHub and Stack Overflow (Xiong et al., 2017)., 2. Understanding Java Usability by Mining GitHub Repositories (Lemay, 2018), 3. Insights from GitHub Community on the Matter Standard: Developer Perspectives and Challenges (Hassan, 2026).

I began exploring the GitHub mining process alongside conducting 1:1 usability study sessions with users. Through this research, I learned a great deal about how developers work, collaborate, report issues, and solve problems in open-source communities.

In this article, I am going to explain:

What the GitHub mining process in user research is?
Why it is important to conduct this type of research?
How it helps identify developer pain points?
How the findings can support communities and developers in improving the overall developer experience (DX) and usability of open-source tools like K-Serve.

Let's start...

What is GitHub Mining in User Research?

GitHub mining is a research method that systematically collects and analyzes data from GitHub repositories—such as issues, pull requests (PRs), discussions, comments, commits, documentation changes, and feature requests—to understand how people use software and where they experience problems.

From a UX perspective, GitHub becomes a large archive of user feedback and user behavior.

Instead of asking users directly:“What usability problems do you face?”

researchers examine: Bug reports, Questions, Configuration failures, Feature requests, Support discussions, Workarounds, Documentation complaints, Delayed PR discussions, Community conversations, These become signals of user experience.

Why is GitHub Mining Conducted?

GitHub mining is conducted because traditional UX methods alone do not always reveal the full picture. Open-source projects often have: thousands of users, globally distributed contributors, limited access to end users, asynchronous communication. GitHub provides a historical record of actual user struggles.

Researchers conduct GitHub mining to: Discover usability issues at scale, Understand real user behavior, Study product evolution over time, Generate evidence-based design decisions

1. Discover usability issues at scale:

Researchers conduct GitHub mining because it allows them to study user experiences and product challenges at a much larger scale than traditional research methods. Instead of interviewing a small group of users—such as 10 participants in interviews or usability sessions—researchers can analyze hundreds or even thousands of GitHub issues, pull requests, discussions, and feature requests to identify recurring patterns across an entire user community.

For example, researchers may discover that among all reported issues, 300 are related to deployment problems, 100 focus on observability complaints, and 50 request better documentation. Looking at this data collectively helps reveal trends that individual interviews may not uncover. This large-scale approach makes it easier to identify which problems occur most frequently and which areas of the product create the greatest friction for users.

2. Understand real user behavior:

Another important reason researchers use GitHub mining is to understand real user behavior rather than relying only on assumptions or controlled testing environments. When users open issues on GitHub, they often describe what they were trying to achieve, where they became stuck, what configuration mistakes they made, which expectations were unmet, and which onboarding barriers prevented them from completing tasks successfully.

These issue discussions provide a detailed record of actual workflows and real-world usage conditions. For example, a developer deploying a tool may explain that installation instructions were unclear, required settings were missing, or error messages did not provide enough guidance. By studying these interactions, researchers gain insight into how users actually experience a product rather than how designers assume they use it.

3. Study product evolution over time:

GitHub mining also helps researchers study product evolution over time. Because GitHub maintains historical records of issues, discussions, fixes, and releases, researchers can observe when specific problems first appeared, how long they remained unresolved, and whether implemented solutions reduced future complaints.

This longitudinal view helps teams evaluate whether design improvements, documentation updates, or technical changes created measurable improvements in usability and developer experience. For example, researchers can compare issue frequency before and after a deployment redesign to determine whether users encountered fewer deployment failures after the change.

4. Generate evidence-based design decisions:

Finally, GitHub mining supports evidence-based design decisions. Instead of making design changes based on assumptions or subjective opinions. for example, saying, “Users seem confused during deployment”—researchers can present measurable findings supported by real community data.

They may conclude that “37% of reported issues involve deployment discoverability,” which provides a stronger foundation for prioritizing product improvements. This evidence helps UX researchers, product teams, engineers, and open-source maintainers make informed decisions about where to invest effort, improve usability, reduce developer friction, and create better experiences for the broader community.

When Should GitHub Mining Be Conducted?

GitHub mining is valuable during multiple stages.

GitHub mining can be conducted at different stages of the research and product development lifecycle, with each stage serving a specific purpose in improving usability and user experience. During the discovery phase, researchers use GitHub data to identify user pain points by analyzing issues, discussions, and feature requests. This helps teams understand the most common challenges users face before making design or development decisions.

Before conducting surveys or interviews, GitHub mining can be used to generate hypotheses. Instead of starting research with assumptions, researchers review historical issue data to identify patterns and form evidence-based questions. For example, if many users report deployment difficulties, researchers may design interview questions specifically around deployment workflows and onboarding experiences.

During a product redesign, GitHub mining helps teams validate recurring issues and confirm whether previously identified problems continue to affect users. This ensures redesign efforts focus on meaningful improvements rather than isolated opinions. By reviewing historical and current issue reports, teams can prioritize areas that consistently create friction.

GitHub mining is also valuable for continuous UX monitoring, where researchers regularly track issue trends to measure overall usability health. Ongoing analysis allows teams to detect emerging problems early, observe changes in user sentiment, and monitor whether user experience improves or declines over time.

After a product update or feature launch, GitHub mining supports post-release evaluation by helping teams measure the impact of changes. Researchers can compare issue volume and themes before and after release to determine whether updates reduced complaints, improved workflows, or introduced new challenges.

Finally, GitHub mining is especially useful in longitudinal studies, where researchers analyze data across extended periods to understand trends over time. This allows teams to observe how user needs evolve, how products mature, and whether long-term improvements lead to sustained reductions in usability issues.

Through these stages, GitHub mining becomes a continuous source of evidence that supports data-driven decisions and helps create better experiences for users and developer communities.

How Does the Community and Developers Benefit?

GitHub mining provides several important benefits for developers because it helps teams make decisions based on actual user experiences rather than assumptions.

One of the major advantages is better prioritization. By analyzing issue reports, discussions, and recurring complaints, developers can identify which problems affect the largest number of users and focus their efforts on fixing the areas that create the most difficulty. Instead of allocating time based only on internal opinions, teams can prioritize improvements that deliver the greatest value to the community.

Another important benefit is reduced support burden. When developers repeatedly analyze and address the root causes of commonly reported issues, users encounter fewer recurring problems and require less direct support. Over time, this reduces duplicate issue reports, lowers maintenance effort, and allows development teams to spend more time building new features rather than responding to the same questions repeatedly.

GitHub mining also contributes to improved onboarding for new users and contributors. By identifying patterns in issue reports related to installation challenges, setup confusion, missing documentation, or early-stage errors, teams can improve guidance materials and simplify user workflows. These improvements create a lower learning curve, helping users become productive more quickly and reducing frustration during initial adoption.

Finally, GitHub mining supports data-driven decisions across product planning and development. Rather than creating roadmaps based on assumptions about what users might need, teams can use measurable evidence from issue trends and community feedback to guide future work. This makes product roadmaps more strategic, more transparent, and more aligned with actual user needs, ultimately leading to stronger developer experience and more effective product evolution.

Benefits for Open Source Communities

GitHub mining and usability research provide several important benefits for open-source communities by helping projects become more accessible, sustainable, and user-centered.

One key benefit is creating a healthier contributor experience, where new contributors can understand project workflows, contribution processes, and technical expectations more quickly, reducing barriers to participation. These improvements also support higher adoption, because better usability makes projects easier to learn and use, attracting more users and contributors over time.

In addition, insights gathered from user issues and discussions lead to better documentation, allowing communities to create more targeted and practical guidance that addresses real user challenges instead of assumed needs. Together, these improvements contribute to increased retention, as users and contributors are more likely to remain active in a project when frustration decreases and their experience becomes smoother and more productive.

Benefits for Product Teams

GitHub mining and usability research provide valuable advantages for product teams by creating a stronger connection between user feedback and product decisions. One major benefit is providing evidence for redesign, allowing teams to make improvements based on actual user-reported issues instead of assumptions or internal opinions.

This research also helps establish usability benchmarks, enabling teams to measure and compare user experience over time and determine whether product changes are producing meaningful improvements.

In addition, GitHub data creates a continuous feedback loop, where ongoing issue reports, discussions, and community input help teams identify emerging problems and continuously refine the product experience. Finally, it supports release quality monitoring by allowing teams to evaluate how updates perform after launch, detect new usability concerns early, and measure whether releases successfully reduce user friction and improve overall product quality.

Are UX Researchers and Organizations Adopting This Method?

UX researchers and organizations are increasingly adopting repository mining and GitHub mining as research methods, especially as software development becomes more collaborative, distributed, and community-driven. Traditional research approaches such as interviews and usability testing remain valuable, but researchers now complement them with large-scale behavioral data collected from repositories to better understand how people actually use and contribute to software products.

This approach is becoming common across areas such as Open-Source UX Research, Developer Experience (DevEx), Human–Computer Interaction (HCI), Software Engineering Research, Empirical Software Studies, and AI/ML Operations Usability, where understanding real-world workflows and developer challenges is essential.

Rather than relying on a single research method, many research communities and organizations use a mixed-method approach that combines repository mining, interviews, surveys, ethnographic observation, and telemetry data. Combining multiple methods improves research validity because findings can be verified from different perspectives—what users say, what users do, and what product data shows.

Through this approach, researchers aim to understand developer pain points, improve onboarding experiences, measure usability debt (the accumulated usability problems that slow users down), and support evidence-based product decisions. As a result, organizations can design products that better reflect actual user needs, strengthen contributor experiences, and continuously improve overall usability and developer experience.

Why This Method Is Growing in UX Research?

GitHub mining is becoming increasingly important in UX research because modern digital products extend beyond traditional consumer applications and now include complex technical environments such as cloud platforms, Kubernetes ecosystems, AI platforms, and DevOps tools.

Traditional UX research has often focused on studying end users through interviews, surveys, and usability testing. However, modern software systems require researchers to also study developers as users, since developers interact directly with interfaces, documentation, workflows, configuration systems, and deployment processes as part of their daily work.

GitHub mining supports this shift by allowing researchers to observe user experience through engineering artifacts—such as issue reports, discussions, pull requests, feature requests, and contribution patterns—rather than relying only on questionnaires or self-reported feedback. These artifacts provide evidence of where users become confused, what tasks create friction, which expectations are unmet, and how workflows perform in real environments.

A strong research framing would be: “GitHub issues are not only defect reports; they represent observable traces of user experience and can be analyzed as usability evidence.”

This perspective aligns closely with modern practices in UX research, Human–Computer Interaction (HCI), and Developer Experience (DevEx) studies, where understanding real user behavior and evidence-based decision-making has become increasingly important.

Conclusion:

Through my K-Serve usability research, I learned that understanding developer experience requires looking beyond traditional interviews and usability testing. By combining 1:1 user sessions with GitHub mining, I was able to observe real-world developer challenges through issues, discussions, pull requests, and community collaboration.

This approach showed that GitHub repositories are not only places where technical problems are reported—they also contain valuable evidence of user experience, usability barriers, and product friction. GitHub mining helps researchers, product teams, and open-source communities make more informed, evidence-based decisions that improve usability, reduce developer pain points, strengthen onboarding, and create better developer experiences (DX).

As modern software ecosystems continue to grow in complexity, repository mining is becoming an increasingly valuable method for understanding how people truly work, collaborate, and build in open-source environments.

How Developers and Open-Source Communities Work - A Guide for Non-Technical People

Pavanipriya Sajja — Fri, 22 May 2026 22:17:12 +0000

You do not need to be a coder or developer to understand open-source communities. If you are curious about how developers actually work, collaborate, and contribute to open-source projects, then this article is for you. In this article, I will walk through the step-by-step process of how developers contribute their skills and participate in open-source community projects by using K-serve as an example.

This article is also helpful for user experience researchers who already work with developers but want to better understand how developers work on projects and use GitHub. It is also useful for UX designers who are interested in learning about the developer experience (DevEx) domain and becoming developer experience designers. Additionally, this article is intended for non-technical people who want to learn how developers and open-source communities work, and how GitHub repositories are used to manage, collaborate on, and operate projects.

Let's learn what is Kserve? KServe is an open-source Kubernetes-based model serving platform that enables scalable and reliable deployment of machine learning models in production environments. I chose KServe because it sits at the intersection of machine learning, Kubernetes, platform engineering, and open-source collaboration, making it a rich example for understanding real-world engineering workflows.

It is important to understand that engineers are not simply “writing code”; instead, they are continuously discovering problems, discussing solutions, implementing changes, reviewing and refining decisions, validating outcomes, and maintaining the system over time.

This entire workflow—spanning collaboration, decision-making, and iteration—leaves behind valuable traces in GitHub through issues, pull requests, discussions, and commits, which together reveal how complex systems are actually built and evolved.

What is Github repository?

A GitHub repository is created by developers, project maintainers, or organizations as a central place to store, organize, and manage a project’s code, documentation, issues, and collaboration history. Once created, it is used by a wide range of people, including developers who write and maintain code, contributors who fix bugs or add new features, reviewers who evaluate and approve changes, and non-developers such as UX researchers, product managers, and technical writers who explore the project to understand it or contribute documentation and feedback.

GitHub repositories are important because they reveal not just the final software product but the entire development process behind it. Developers use them to collaborate, track issues, discuss and review changes, manage tasks, and maintain transparency in their work. Learning how repositories function helps in understanding real developer workflows, decision-making processes, and how complex software systems evolve over time in both open-source and professional environments.

I am going to explain how developers and the open-source community work with entire GitHub repository in 5 simple steps.

Step : 1. The Project Exists as a Living System

Imagine that KServe already exists and is being used by people around the world. Its users include ML engineers who deploy and manage machine learning models, platform engineers who build and maintain the underlying infrastructure, MLOps teams who ensure smooth model lifecycle operations, Kubernetes administrators who handle cluster configuration and deployment, and researchers who experiment with and evaluate machine learning systems.

Most people think software development means typing code all day. In reality, a typical improvement starts with a real person running into a problem, and ends with millions of users getting a better experience. Here's how that journey unfolds, using KServe — a tool that helps AI models run in the cloud — as the example.
Let me show you the big picture first, then zoom into each phase.

Now let's look at each phase in plain language, with a closer diagram for the parts that need more detail. For example: A user(Engineer) deploys an inference service.

Step :2 A User Encounters a Problem & Created the issue.

Imagine you're an engineer who uses KServe to run an AI model. You follow all the instructions, but when you try to use it, nothing responds. You're not sure if you did something wrong, or if there's a bug in the software.

So you do what most people do: check the logs, search the documentation, ask the community — and eventually open a "bug report" (called an issue on GitHub). This is where the real work begins, even though no one has touched code yet.

When you file that issue, volunteer maintainers (people who help run the project) step in to classify it. Is this a bug? A misunderstanding? Something missing from the docs? They ask questions to narrow it down

Step: 3 Can I make it break on my machine?

Before anyone can fix a problem, they have to be able to see the problem. An engineer sets up the same software environment as the user and tries to make the bug happen intentionally. This is called reproducing the bug.

Once they can make it fail consistently, they start playing detective — tracing the path a request takes through the system, reading log files, and asking: where exactly does things go wrong?
This phase is mostly thinking, not typing. Engineers build a mental map of how the system is supposed to behave, then look for where reality diverges from that map. It looks like this:

Step: 4 Choosing the right fix (not just any fix)

Once the root cause is found, the next question isn't can we fix it? — it's what's the safest way to fix it?

In the KServe example, the bug was that a missing network port would cause a silent, invisible failure. Engineers debated three possible solutions:

This kind of debate — often hundreds of comment threads for just a few lines of code — is what separates good software from brittle software.

Step : 5 From code to released fix

Only after all that thinking does actual coding begin. The engineer writes the fix in a separate "branch" (a safe copy of the code), then submits it as a "pull request" — essentially saying "here's my proposed change, please check my work."

Other engineers review it, suggest improvements, and run automated tests. When everyone agrees it's safe, it gets merged into the main codebase and released.

The ending: and why it never really ends

After the update is released, what changed for the user? Before: "It just doesn't work." After: "Configuration invalid: missing port." That's a small sentence, but it means the user now understands what to fix instead of guessing blindly.

And here's the final truth: the moment one problem is solved, a user somewhere will notice the next thing that could be better. The loop starts again. That's not a flaw in open source — it's the whole point. Software improves because real people keep using it, keep running into its rough edges, and keep caring enough to speak up.

What Happens When Engineers Share Their Experience?

Pavanipriya Sajja — Mon, 27 Apr 2026 22:45:34 +0000

Through this article, I will share my real experiences, including the steps, process, and insights that helped me encourage engineers to easily and happily participate in usability 1:1 meetings to improve the KServe experience.

Power of sharing your feedback:

Before we jump in, let’s take a moment to understand the power of feedback.

Imagine you’re booking flight tickets for your entire family with an airline you really like. The overall experience is excellent—you enjoy the journey, there are plenty of entertainment options onboard, and both the complimentary and paid food and beverages are delicious. The airline even offers family-friendly meal options and services throughout the journey. From booking to boarding, everything feels smooth and well-designed. On top of that, they provide lounge access at a much more affordable price compared to other airlines.

However, there’s one issue: the online check-in app doesn’t work. Because of that, you have to check in at the airport, which means standing in long lines and wasting time. It’s frustrating.

Even so, this was just a one-time issue, while the rest of your experience with the airline has consistently been great. Instead of switching to another airline, you might choose to share your feedback so they have a chance to improve.

If you don’t provide feedback, the airline may never know about the issue and won’t get the opportunity to fix it. But if you do share it, they can improve the experience—not just for you, but for many others as well. Over time, this leads to a better product, happier customers, and increased trust in the brand.

That’s the power of feedback.

A single issue like a temporary failure in the online check-in system doesn’t define the entire service. Behind the scenes, engineers and teams are constantly working to build and maintain systems that serve millions of users every day. While mistakes can happen, feedback helps identify and resolve them.

Instead of immediately switching to another airline and going through the effort of adapting to a new experience, simply sharing your feedback can lead to meaningful improvements. The next time you fly, you might find that the issue has been resolved—and your experience is even smoother.

Feedback doesn’t just fix problems—it strengthens experiences.

Number of ways to share your feedback.

There are many ways to share your feedback.

For example, an airline team might call you and ask about your experience. During this phone conversation, you can share your thoughts in detail this is known as an interview in user research.

Sometimes, they may ask you to walk through the online check-in process step by step, possibly over a video call. This allows the UX team to observe how you interact with the system in real time and identify any difficulties or pain points you encounter. This method is called a usability study.

Finally, you might be asked to complete a short form in the application at the end of your journey—usually a 3–5 minute questionnaire about your recent experience. This is known as a survey in user research.

In all these ways, you are given opportunities to share your feedback, helping teams understand your experience and improve their services.

What is usability studies and how it can be conducted?

Let’s learn what usability studies (User research's method) are and how I started a usability study project with K-Serve, which is one of the CNCF incubating projects.

Usability studies are one of the key user research methods used to understand engineers’ challenges in real world workflows. In this approach, engineers on the video call with UX designer or live in person in 1:1 meeting to share their screens and perform tasks such as deploying ML models using K-Serve or walking through how they debug issues.

During these 1:1 meeting, UX designers don’t just listen also they observe. This gives them direct visibility into where friction occurs, which tools or documentation cause confusion, and where workflows break down. These real time observations are critical for identifying gaps that might not surface through interviews or surveys alone.

Based on these 1:1 meeting, UX designers document their findings and create reports that highlight engineers’ pain points, usability issues, and opportunities for improving the overall K-Serve experience.

How to conduct usability studies:

1. Prepare tasks and guidelines for usability studies

In this phase, UX designers prepare what they want to learn from the study.

First, they create a list of tasks for engineers to perform. For example, in KServe, tasks might include: Deploying an ML model, Monitoring ML workloads, Debugging issues

Once the tasks are ready, UX designers prepare follow-up questions. These questions help engineers share more details, such as:

Did you face any problems during this task?
Was anything confusing or difficult?
How do you think this process can be improved?

This gives engineers a chance to share their thoughts and overall experience with K-Serve. Next, UX designers create guidelines for the study. This includes:

Choosing tools for the sessions (for example, Google Meet for 1:1 meetings), Deciding how to handle data (what can be shared publicly and what should stay confidential), Making sure any shared data is anonymous and combined (not personal or raw data), Planning the timeline for sessions and analysis, Deciding where and how to share the final results

These steps help ensure the usability study is well-organized, respectful of participants’ privacy, and useful for improving the product.

2. Participant recruitment planning

In this phase, the UX designer plans how to recruit participants and collect basic information about them.

The UX designer creates a simple survey for engineers to sign up for usability studies. This survey helps gather important details such as:

The engineer’s role and experience
Their availability for a 1:1 session
How they use KServe in their work
The main challenges they are currently facing

This information helps both the UX designer and the engineer save time. The UX designer can understand the participant before the session and prepare better.

For example, if an engineer mentions that they face challenges during deployment, the UX designer can focus on deployment-related tasks during the session.

Overall, this process makes it easier to recruit the right participants and helps the UX designer learn about their challenges in advance, leading to more focused and useful usability studies.

3. During the Usability study 1:1 meeting:

Once the usability session is scheduled, the UX designer observes the tasks the engineer performs and takes notes.

With the engineer’s permission, the UX designer may record the session for study purposes. The recording will not be shared publicly—it will remain confidential and only be used by the UX team and KServe maintainers for research.

At the beginning of the session, the UX designer will ask for consent to record. If an engineer is not comfortable being recorded, that is completely okay—they can still participate and share their experience.

4. Analysis - After the Usability studies:

Once the UX designer completes usability studies with engineers (for example, with more than 10–20 participants), they begin analyzing the data.

The UX designer uses analysis methods such as thematic analysis and pattern identification to find common themes and trends across all sessions. Based on this, they create a report that includes both key insights and metrics.

For example, the report might show that 60% of engineers faced difficulties with the deployment process. A common pattern identified could be configuration complexity.

In this way, the UX designer creates a clear summary of findings without sharing any raw data.

5. Sharing the findings in community meetings for improvements:

Once the UX designer has created the findings report, the results are shared with the KServe community to support improvements.

The insights from the report are discussed in community meetings to help identify what can be improved in K-Serve. In some cases, the findings may also be shared in CNCF KubeCon talks to highlight the challenges engineers face while using K-Serve and to encourage broader discussions and solutions.

This is how usability studies, through these five steps, can have a significant impact by making KServe easier for engineers to use.

Engineer Effort and Real-World Challenges in K-Serve:

K-Serve contributors (engineers) are building tools for other engineers who use K-Serve to deploy and manage ML models. Often, contributors make a plan on the user requirements and begin building the product based on those engineers knows.

Once the product reaches production and more engineers start using K-Serve in real environments, usability challenges begin to surface. As usage increases, engineers start noticing difficulties while working with the system. In my research, I have heard users say that it is “very difficult to use,” “needs to be easier,” or that they are “looking for alternatives,” or "they are trying to figure out which approach that we can began for ML deployment"

However, there is still an opportunity to improve KServe significantly. By observing and listening closely to engineers—especially where they struggle in real workflows, we can identify meaningful usability gaps. Documenting and analyzing these insights can help make K-Serve easier to use for both current users and new engineers who are adopting it for the first time.

Ultimately, this approach ensures that K-Serve evolves based on real user experiences rather than assumptions, making it more usable, accessible, and widely adopted in production environments.

Why engineers hesitate or afraid to share feedback:

Engineers often work within environments that involve sensitive systems, production constraints, and organizational policies. Because of this, they may hesitate to participate in UX research (Usability studies), assuming it requires sharing confidential or restricted information.

However, UX research does not focus on internal data, architecture details, or anything proprietary. Instead, it focuses on the engineer’s experience and their workflows, challenges, and friction points.

This includes:

Where workflows slow down
What feels overly complex or unclear
What takes too long to configure, deploy, or debug
What drives engineers to avoid certain features or switch tools

For example, when engineers use KServe in production, they may encounter challenges in deployment, configuration, or maintenance. These challenges can increase cognitive load and make the system feel overwhelming. Importantly, sharing these experiences does not require revealing any confidential information only the difficulties encountered during usage.

When such feedback is not shared, these issues often remain unresolved. Over time, this can lead engineers to seek alternative tools that are easier to use.

On the other hand, when engineers provide even high-level feedback, it creates meaningful impact:

It reduces cognitive load across workflows
It improves existing tools and platforms
It simplifies onboarding for new engineers
It increases overall adoption within the engineering community

UX research, in this context, becomes a way for engineers to contribute beyond code by sharing their experiences to improve the ecosystem.

Even simple insights, such as "This step was confusing" or "This process took longer than expected" can directly influence product improvements.

Ultimately, participation in UX research helps create better tools for engineers, by engineers without requiring any compromise on confidentiality.

Increase participation in usability studies:

To increase participation in usability studies, we started reaching out directly to the community. we sent messages to more than 1,000 members in Slack threads across the KServe community channel and contributor channel. we also reached out to other CNCF platforms like Kubeflow, since many of them use KServe for model serving.

To further increase reach and participation, we also started posting on LinkedIn to engage a wider audience and encourage more engineers to join the usability studies.

We are still working on usability studies to understand the challenges engineers face while using KServe. In a future article, we will share the analysis process and the results we gather from these studies.

Takeways:

It’s completely understandable that screen sharing and recordings during usability studies can feel sensitive or confidential. If you are not comfortable sharing certain information, there are still many flexible ways to participate and give feedback.

For example, if something confidential comes up during the session, you can ask the UX designer to pause the recording and then share your feedback. You can also choose to participate without any recording at all and still explain your experience. The UX designer will take notes and capture the key issues you mention.

If you don’t have much time, you can let the UX designer know in advance. For instance, you can say you only have 10–15 minutes to try a task like deploying a model and want to quickly share the challenges you face. You can also join a short discussion or share your feedback through a survey by listing the main issues you are experiencing.

In this way, your feedback can still help improve KServe usability in a way that fits your comfort and availability.

Final thoughts:

Usability studies create a strong feedback loop between engineers and UX researchers.

They help to:

Identify real-world challenges
Improve developer experience
Strengthen tools like KServe
Build systems based on actual usage, not assumptions

Most importantly, feedback turns individual experiences into shared improvements. Even small input from engineers can meaningfully shape the future of the platform.

Design Systems : How They Shape Developer Experience in Modern Product Building

Pavanipriya Sajja — Mon, 27 Apr 2026 17:09:46 +0000

When I first started discussing and presenting system design concepts for one of my UX projects, many engineers on my team were surprised. Initially, they seemed confused about why I was explaining this information under the “design system” title.

This reaction made me reflect. After the meeting, I asked one of the engineers why there was confusion when I referred to “system design.” The developer explained that system design means something quite different in engineering compared to how it’s often interpreted in UX.

That conversation led me to explore the topic further. I began researching to better understand what “system design” actually means from both a developer’s perspective and a UX perspective especially when you are working on developer experience platform.

As a result, I’m writing this article to clarify the differences between system-level design in UX and system design in engineering.

Let’s start with system design from a developer’s point of view:

System Design in developer's point of view

System design is the process of defining the architecture, infrastructure, data flow, and communication patterns of a software system. It is the technical blueprint that determines how a system is built, scaled, and maintained to meet both functional requirements and non-functional requirements like performance, reliability, and security.

Why is it Important to Consider?

System design plays a critical role in building reliable and scalable systems by addressing key challenges early in the development process.

It helps prevent costly architectural mistakes before they escalate into production crises, while ensuring the system can scale smoothly as user demand increases.

A well thought out system design also defines how the system behaves under failure conditions, reducing the risk of outages and improving overall resilience. Additionally, it aligns engineering decisions with business requirements such as SLAs, latency targets, and throughput expectations.

By establishing clear patterns and boundaries from the beginning, system design ultimately minimizes technical debt and creates a strong foundation for long-term system stability and growth.

How Will it Be Useful?

System design provides engineering teams with a clear technical roadmap before any code is written, creating a shared understanding of how the system should be built.

It reduces ambiguity in critical architectural decisions such as database selection, API design, and caching strategies, allowing teams to move forward with confidence.

By thinking through the system early, teams can anticipate potential bottlenecks, failure points, and scaling challenges in advance. It also enables faster onboarding of new engineers by offering clear documentation of the system architecture.

Ultimately, strong system design supports the long-term operability and maintainability of complex systems, making them easier to evolve and manage over time.

When Can You Use it in Projects?

System design becomes especially important at key moments in a product’s lifecycle. It plays a crucial role at the start of any new product or platform, before major architectural decisions are made, helping teams establish a solid foundation.

It is equally valuable when scaling an existing system that is reaching its performance or reliability limits, as it guides necessary improvements.

During transitions, such as migrating from a monolith to a micro-services architecture, system design provides structure and direction. It also supports onboarding new engineering teams by offering clear architectural context, and it helps organizations prepare their infrastructure for high-traffic events or rapid user growth, ensuring the system can handle increased demand effectively.

If You Learn Design Systems, What Impact Can You Make?

Strong system design enables engineers to build systems that are resilient, scalable, and production-ready from day one. It empowers them to influence critical infrastructure decisions that can impact millions of users, while also reducing friction within engineering teams by establishing clear and reusable architectural patterns. Beyond technical outcomes, it helps individuals build credibility as senior technical voices in cross-functional product discussions. At the same time, it fosters a culture of incident prevention by encouraging proactive thinking around failure scenarios and system reliability.

What Kind of Projects Can You Work On?

System design is especially critical in complex and high-impact environments where reliability and scalability are non-negotiable. It plays a key role in large-scale distributed systems such as e-commerce platforms, streaming services, and fintech applications, where millions of users depend on consistent performance.

It is equally important in cloud-native and Kubernetes-based infrastructure platforms, where systems must be dynamic and resilient by design. In real-time systems like messaging apps or trading platforms, system design ensures low latency and high responsiveness.

It also supports multi-region, globally distributed applications by enabling seamless operation across geographies. Additionally, system design is foundational for developer platforms, internal tooling, and API-first products, where clarity, consistency, and scalability directly impact developer experience and productivity.

Examples of Design Systems (Developer / System Architecture):

Examples of system design in real-world applications highlight how leading technology companies architect their platforms for scale and resilience.

Netflix: Microservices + chaos engineering for resilience
Uber: Real-time geolocation with event-driven architecture
Airbnb: Search ranking with distributed caching at scale
Twitter/X: Fan-out on write for timeline delivery at massive scale
WhatsApp: Handling billions of messages with minimal infrastructure, showcasing extreme efficiency in system design.

Here is the example of the Uber system architecture and you can learn more from this blog: Link

Core Concepts of Design Systems:

System design is built on several key components that ensure a system can operate efficiently at scale.

Scalability — horizontal and vertical scaling strategies for handling user growth.
Availability & Reliability — redundancy, failover, and uptime guarantees
Data Storage — relational, NoSQL, time-series, and blob storage trade-offs
APIs & Communication — REST, gRPC, GraphQL, and event-driven messaging patterns
Caching — Redis, CDN, and in-memory strategies for reducing load
Load Balancing — distributing traffic to eliminate single points of failure
Observability — logging, metrics, and distributed tracing as architectural requirements

What is the Mental Model?

A developer’s mental model for system design is centered on asking the right questions before building anything. They begin by identifying both functional and non-functional requirements to understand what the system must do and how well it should perform.

They then consider the read/write ratio to determine how it will influence data storage and access patterns. From there, they analyze potential bottlenecks across the system—whether in the database, network, or compute layer—to anticipate performance constraints.

Developers also think critically about failure scenarios, defining what can go wrong and establishing clear recovery paths. Finally, they focus on how the system will be observed and maintained in production, ensuring there are robust mechanisms for monitoring, debugging, and continuous improvement.

📚 Developer System Design Resources with Links:

1. Designing Data-Intensive Applications — Martin Kleppmann
The definitive book on distributed systems, databases, and data architecture trade-offs.

Official book site → dataintensive.net

2. System Design Primer — GitHub
Free, open source, comprehensive study guide covering everything from basics to advanced system design interviews. Github Link

3. ByteByteGo — Alex Xu
Visual, beginner-friendly system design explanations covering real-world architectures like Netflix, Uber, and Twitter. Website Link

4. Cloud Architecture Centers
Official cloud-native design pattern libraries from the three major cloud providers. AWS Architecture Center Link, Google Cloud and Microsoft Azure.

5. High Scalability
Real-world architecture case studies from companies like YouTube, Netflix, WhatsApp, and more. Website link

6. CNCF Landscape

The complete cloud native ecosystem reference — tools, projects, and categories across the entire CNCF landscape. link

Key insights : The most powerful insight in system design is that the cost of a wrong architectural decision compounds exponentially over time — a trade-off made poorly at the design phase can take years and millions of dollars to unwind at scale.

Developers who internalize the mental model of asking the right questions before writing a single line of code become the engineers who build systems that not only work today but endure, scale, and recover gracefully for years to come.

Now let's move on to UX designer perspective design system.

Design system in UX Designers point of view:

A design system is a centralized collection of reusable UI components, design tokens, interaction patterns, typography, color palettes, spacing rules, and guidelines that ensure visual and experiential consistency across an entire product. It is the single source of truth for how a product looks, feels, and behaves from the user's point of view.

Why is it Important to Consider?

A well-designed system ensures consistency across every screen and touchpoint a user interacts with, creating a unified experience throughout the product. It embeds accessibility and inclusivity standards directly at the component level, rather than treating them as an afterthought in the design process.

By standardizing patterns and decisions, it reduces design debt by eliminating repeated and inconsistent UI choices over time. It also creates a shared language between designers, developers, and product teams, enabling better collaboration and alignment. Ultimately, it builds user trust by delivering predictable and coherent experiences that feel familiar and reliable across the entire system.

How Will it Be Useful?

A design system significantly speeds up the design workflow by providing ready-to-use, tested components that reduce the need to build interfaces from scratch.

It helps bridge the gap between design and engineering by establishing a shared component language that both teams can understand and use consistently.

This allows designers to focus more on solving complex user problems rather than repeatedly recreating basic UI elements. It also enables rapid prototyping and smoother, more consistent design handoffs between teams. Additionally, when a single component is updated, the improvement can instantly propagate across every product surface, ensuring consistency and efficiency at scale.

When Can You Use it in Projects?

A design system becomes especially important in several key scenarios within product development. It is essential when building a multi-product ecosystem that requires a consistent UI across different surfaces to maintain a unified user experience.

It is also valuable when a team is scaling and multiple designers are working in parallel, ensuring alignment and reducing inconsistencies. When the design-to-development handoff becomes slow, inconsistent, or error-prone, a design system helps streamline collaboration and improve accuracy.

It is equally important when a product has matured enough to require standardization of its UI language for better coherence and maintainability. Finally, it plays a critical role when accessibility needs to be systematically enforced across the product, ensuring inclusive design is built into every component from the start.

If You Learn Design Systems, What Impact Can You Make?

A well-established design system helps elevate product quality and maintain brand consistency at scale across every user touchpoint. It positions designers as strategic contributors who shape product direction, rather than focusing only on individual screens.

By removing repetitive design decisions from the workflow, it enables faster and more efficient product delivery. It also supports the systematic adoption of accessibility and inclusive design practices across the organization, ensuring these principles are embedded throughout the product.

In addition, it acts as a bridge between design vision and engineering execution, helping both teams stay aligned from concept to implementation.

What Kind of Projects Can You Work On?

Design systems are especially valuable in enterprise SaaS product design, where interfaces are often complex and must support multiple user roles with different needs and permissions.

They are also critical in the creation and governance of design systems within large product organizations, ensuring consistency, scalability, and long-term maintainability across teams. In developer tools and platform UI, where precision, information density, and efficiency are essential, design systems help maintain clarity and usability.

They are equally important in mobile and cross-platform product design, where consistent component behavior is needed across different devices and environments. Additionally, design systems play a key role in open source contributions such as Material Design, Carbon Design System, and Atlassian Design System, which help standardize and scale design practices across the broader industry.
You can explore the Open MRS website and their applications are using the Carbon design system to make consistent products among the OpenMRS platform.

Examples of Design Systems:

Material Design : Google - Comprehensive motion and component guidelines
Carbon Design System : IBM - Enterprise accessibility and data-heavy UIs
Atlassian Design System : Atlassian - Developer tool UI patterns
Fluent UI : Microsoft - Cross-platform consistency across Office products
Polaris: Shopify - E-commerce focused UX patterns
Lightning Design System : Salesforce Enterprise CRM - UI standards

Core Concepts of Design Systems

Design Tokens — the atomic values of color, spacing, typography, and elevation that feed every component

Component Library — reusable UI building blocks like buttons, modals, forms, and navigation patterns

Pattern Library — documented solutions to recurring UX problems like empty states, error handling, onboarding flows.

Accessibility Standards — WCAG compliance baked into every component by default
Governance Model — who owns, contributes to, and maintains the system over time
Documentation — the living guide that explains when and how to use every element.

What is the Mental Model?

A UX designer approaches a design system by asking:

What experience are we creating, and for whom?
What are the most repeated UI patterns that need standardization?
How do we ensure every component is accessible and inclusive by default?
How does this component behave across different screen sizes and contexts?
How do we govern and evolve this system as the product grows?

UX Design System Resources

Google material design — Material Design documentation and guidelines
carbondesignsystem.com — IBM Carbon full documentation
atlassian.design — Atlassian design system patterns
designsystemsrepo.com — curated directory of public design systems
Figma Community — open design system files and UI kits
Storybook.js.org — tool for building and documenting component libraries

Key insights: A design system is never truly finished — it is a living product that grows with the organization, and the designers who treat it as such become the architects of consistent, trustworthy user experiences at scale.

The deepest insight is that a design system is not about restricting creativity; it is about liberating designers to focus on the problems that truly require human judgment, while the system handles the repetitive decisions that slow teams down.

Conclusion:

In such a way when you are working with the developer experience domain to understand what design system that discussing in the meetings. That could help you understand what design system the team is talking about and what design system decisions are making as per the requirements.

From a developer’s perspective, a design system is often discussed and defined before development begins—it helps set the foundation for how things will be built.

From a UX perspective, a design system typically evolves during the product-building phase, especially after research is completed, to ensure the experience is grounded in real user needs.

At the end, both perspectives play a critical role in product development.

Whether you’re designing the experience a user sees or the infrastructure that powers it, both disciplines share the same fundamental truth: great systems are built on intentional decisions, not accidental ones.

UX thinking and developer system thinking are two sides of the same coin. Teams that bring both perspectives together build products that are not only technically sound but also genuinely human.

I believe this explanation on differentiation about design system point of developer's and UX is different but these both are very important while building the real products.

🚀 “𝗗𝗲𝘀𝗶𝗴𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿𝘀 𝗶𝘀 𝗻𝗼𝘁 𝗷𝘂𝘀𝘁 𝗨𝗫—𝗶𝘁’𝘀 𝘀𝘆𝘀𝘁𝗲𝗺 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴.” 👉 DevEx Design Challenges Series — Now Complete: https://dev.to/priya_sajja_c336921bbda87/series/38724

Pavanipriya Sajja — Sat, 25 Apr 2026 00:14:44 +0000

Developer Experience designer challenges & Solutions Series' Articles - DEV Community

View Developer Experience designer challenges & Solutions Series' Articles on DEV Community

dev.to

Challenge : 4 System-Level Challenges in Developer Experience (DevEx) Design

Pavanipriya Sajja — Tue, 21 Apr 2026 23:30:22 +0000

One of the most critical challenges in Developer Experience (DevEx) design is operating at the system level rather than the interface level. DevEx is not simply UX applied to a technical domain—it requires designing across a distributed ecosystem of tools, infrastructure, and workflows.

Unlike traditional UX, where the focus is a single product or interface, DevEx involves understanding how developers interact with an entire toolchain. For example, when designing for Kubernetes, the scope extends beyond a single UI or feature. It includes CLI tools, infrastructure configurations, CI/CD pipelines, and collaboration patterns across teams.

Engineers typically work in fragmented, non-linear workflows—switching between tools, environments (cloud, on-prem, multi-cluster), and communication channels. This constant context switching increases cognitive load and makes it difficult to design seamless experiences.

The core challenge, therefore, is not just improving individual touch points, but understanding and optimizing the end-to-end system. Effective DevEx design requires system-level thinking to reduce fragmentation, improve interoperability, and create cohesive workflows for engineers.

In this article, I’ll break down these system-level challenges into five key areas, using a hybrid engineer persona operating in a multi-cluster environment as a reference.

1. Invisible User Journeys:

In traditional UX domains such as designing a hospital digital ecosystem the user journeys or workflows are relatively deterministic and well-defined. For example, a public facing hospital website typically follows a structured information architecture: organizational overview, service catalog, provider directory, and contact endpoints. This often extends into a patient portal, which introduces authenticated workflows such as appointment scheduling, access to electronic health records (EHR), diagnostic results, and medication management.

From a system design standpoint, these two surfaces (marketing website and patient portal) are loosely coupled but clearly integrated via identity, routing, and backend services. The interaction model is predictable, state transitions are well understood, and user flows can be explicitly mapped end-to-end.

However, when designing UX for developer platforms particularly in complex domains like multi-cluster infrastructure the concept of a “user journey (Engineer Workflows)” becomes significantly more abstract and non-deterministic.

Developer workflows are shaped by infrastructure topology, organizational constraints, and tooling ecosystems. Consider the variability across multi-cluster environments:

A platform engineer deploying Kubernetes clusters across multi-cloud environments (e.g., AWS, Azure, GCP).
An enterprise engineer managing strictly on-premises clusters due to compliance requirements.
Edge engineers deploying lightweight clusters across distributed edge locations.
Hybrid setups combining on-prem + cloud + edge infrastructure.

Each of these scenarios introduces different constraints around networking, identity, observability, and lifecycle management. Additionally, tooling fragmentation further diversifies workflows.

For example:

Teams using Rancher for centralized cluster management
Organizations leveraging Azure Arc for unified control planes
Developers interacting with clusters via Headlamp
Internal platforms built on top of custom abstractions and proprietary orchestration layers

Two engineers solving the same problem (e.g., cluster deployment or debugging) may follow completely different paths using different tools, scripts, or automation pipelines.

Why These Are “Invisible Journeys (Invisible workflows)”

Unlike traditional UX flows, developer journeys exhibit the following characteristics:

Non-linear execution paths: Tasks such as cluster provisioning, scaling, or debugging do not follow a fixed sequence. They depend on runtime context, failures, and system state.
High degree of personalization: Developers create scripts, CLIs, and automation pipelines tailored to their workflows. Two engineers solving the same problem (e.g., cluster autoscaling) may use entirely different approaches.
Toolchain-driven interactions: The “interface” is often not a single UI but a combination of CLI tools, YAML configurations, APIs, CI/CD pipelines, and observability dashboards.
Low visibility into real behavior: Much of the workflow happens outside observable UI layers—in terminal sessions, scripts, or internal tools—making it difficult to capture through traditional UX methods.

Core UX Challenge

You cannot rely solely on predefined user flows, wireframes, or static journey maps. Instead, designing for developer platforms requires:

Modeling workflow variability rather than enforcing a single “happy path”
Understanding environment-specific constraints (cloud, on-prem, edge)
Capturing real-world behaviors through contextual inquiry, shadowing, and telemetry
Designing for composability and extensibility rather than rigid interfaces

Key Insight

In developer focused systems, the “user journey” is not a fixed path—it is an emergent property of: Infrastructure architecture, Tooling ecosystem, Organizational practices

This is why these journeys are often invisible, they exist across fragmented systems and only become apparent through deep, context-rich research.

2. Designing for Systems, Not Screens:

This section draws from hands on work within the Kubernetes multi-cluster research space, where the core challenge was not UI design, but modeling end-to-end developer workflows across heterogeneous environments.

During research synthesis, I constructed a hybrid engineer persona based on raw qualitative and behavioral data. The analysis revealed that multi-cluster operations do not follow a single deterministic workflow. Instead, engineers operate across three primary workflow paradigms:

Manual-first workflows: high cognitive control, low automation dependency
Hybrid workflows (automation + manual overrides): balanced control with selective automation
Automation-first workflows (AI-assisted or fully automated): minimal manual intervention, high system trust

Each category introduced distinct tooling expectations and system constraints. For example:

Some engineers required AI-augmented observability and decision support embedded into existing toolchains. Others preferred standalone orchestration platforms for multi-cluster lifecycle management. Enterprise-scale environments imposed organizational constraints, such as security policies, internal platform dependencies, and legacy integrations. Toolchains varied significantly, including internal systems, open-source ecosystems, and custom scripts.

As a result, the research surfaced 10+ distinct workflow variants, each representing different combinations of:

cluster management strategies, observability patterns, deployment models, organizational constraints

This complexity makes it clear that designing a single multi-cluster UI is insufficient. The problem space requires system-level design thinking, where the goal is to support workflow continuity across tools, not optimize isolated interfaces.

System-Level Nature of DevEx: In Developer Experience (DevEx), workflows are inherently distributed across multiple touch points, not confined to a single interface. A typical developer journey may span: Documentation systems, CLI environments, Logging and observability platforms, Monitoring dashboards, Issue tracking systems like GitHub

The “interface” is effectively the entire ecosystem.

Core Challenge

You are not designing a screen, you are designing interoperability across a fragmented system.

A failure in any single component such as ambiguous documentation, inconsistent CLI behavior, or poor observability signals can propagate and break the entire workflow chain.

This is why DevEx design requires:

Deep system awareness (APIs, pipelines, infra constraints)
Workflow orchestration thinking, not just UI optimization
Alignment across tools, teams, and environments

The outcome is not a better screen, it is a cohesive, resilient developer workflow system.

3. System Constraints Drive Design Decisions(Not Just UX)

In distributed systems especially multi-cluster platforms design decisions are rarely driven by UX alone. They are tightly coupled with underlying system architecture and operational realities.

For example, in a multi-cluster environment (e.g., orchestrated via Kubernetes), engineers often decompose applications (such as e-commerce systems) into micro services and deploy them across multiple clusters. These placement decisions are influenced by:

Workload distribution strategies (latency, failover, cost optimization)
Cluster topology (cloud, on-prem, edge)
Service-to-service dependencies
Data locality and compliance requirements

As a result, the UI/UX of a multi-cluster management tool must reflect this complexity rather than abstract it away entirely. Technical Dependencies That Shape UX

Design is constrained by several non-negotiable system factors:

Backend Architecture: Microservices, service meshes (e.g., Istio), and distributed data stores dictate how information can be surfaced and interacted with.
APIs and Contracts: UX is bounded by available APIs (e.g., Kubernetes API). Limitations in API granularity, latency, or consistency directly impact what the interface can support.
Infrastructure Constraints: Resource quotas, network policies, and cluster heterogeneity (multi-cloud + on-prem + edge) influence both system behavior and UI responsiveness.
Open Source Ecosystem Constraints: Many platforms depend on upstream tools and CRDs (Custom Resource Definitions). Design must align with their capabilities and limitations rather than reinvent abstractions.

The Core Challenge

Even when research clearly identifies a more intuitive or efficient user experience, it may not be immediately implementable due to these constraints.

This creates a fundamental tension such as UX wants simplicity and clarity, Systems introduce complexity and trade-offs.

What This Requires from UX

Designing in this space means operating with a systems mindset:

Understand Technical Trade-offs: Evaluate latency vs. consistency, abstraction vs. control, and flexibility vs. complexity.
Collaborate Deeply with Engineers: UX decisions must be co-designed with backend and platform engineers, not handed off after the fact.
Design Within Constraints

The goal is not to ignore constraints but to make them visible, understandable, and manageable for users.

In highly technical domains like multi cluster systems, good UX is not about removing complexity. it’s about structuring and exposing it in a way that aligns with how engineers actually reason about distributed systems.

Additional Context: Platform & Ecosystem Constraints

Constraints are not only technical they are also platform and ecosystem driven.

In platforms like Rancher, any new feature or plugin must align with Rancher’s architecture, permission model, and extension framework. Even if a UX problem is identified, the solution must stay within what the platform allows. Similarly, when building plugins in tools like Headlamp for ecosystems such as K-native, teams must account for: K-native APIs and resource models, Compatibility constraints, Extension boundaries and lifecycle

While some tools (like Headlamp) offer more flexibility, they still need to respect the constraints imposed by the target ecosystem (e.g., K- native).

Key takeaway:

Even extensibility (plugins, integrations) operates within strict boundaries, UX solutions must align with platform permissions, APIs, and ecosystem rules before they can become usable reality.

4. Consistency across a distributed ecosystem :

Consistency across a distributed ecosystem is one of the most complex challenges in Developer Experience (DevEx) design. In modern systems, development responsibilities are inherently fragmented: one developer may focus on API design, another on backend services, another on frontend integration, while others handle database architecture or CI/CD pipelines. In parallel, especially in open-source or cloud-native ecosystems, different communities may own different domains—for example, one group concentrating on core multi-cluster operations such as deployment, while another focuses on multi-cluster networking.

This distribution of ownership introduces structural misalignment. In DevEx, the overall experience is not controlled by a single team but is instead shaped by multiple teams, contributors, and sometimes independent communities. As a result, inconsistencies naturally emerge, terminology may vary across components, interaction patterns may differ between tools, and documentation often becomes fragmented or siloed.

The core challenge, therefore, is maintaining a coherent and consistent experience across this decentralized system. Achieving this requires not just strong communication, but also shared standards, aligned mental models, and deliberate design governance to ensure that, despite the distributed nature of development, the end-to-end experience feels unified to the user.

5. Unified experience & Thinking Beyond Features:

In Developer Experience (DevEx), the product experience is not centered around the UI alone, unlike traditional software products. Instead, it is a composite experience formed by documentation, CLI tools, APIs, and UI dashboards working together as a unified system.

Designing for DevEx requires treating these elements as interconnected surfaces rather than isolated components, where each one supports and reinforces the others to enable a seamless workflow.

The core challenge lies in ensuring strong cohesion between them—how documentation guides CLI usage, how APIs are reflected in both CLI commands and UI actions, and how UI dashboards surface and simplify underlying system operations.

For instance, if the documentation is unclear or incomplete, even a well-designed UI becomes difficult to use effectively because developers rely on documentation and CLI workflows to understand system behavior and operational intent.

In traditional UX design, success is often measured by how well individual features, screens, and user flows are designed and visually presented. The focus tends to remain on isolated interactions and how each feature appears and behaves within a defined interface. However, in Developer Experience (DevEx), the emphasis shifts away from feature centric thinking toward the efficiency of end-to-end workflows.

What matters more is how quickly developers can complete tasks, how smoothly they can move across tools and systems, and how much cognitive effort is required to understand and operate the environment.

In this context, success is not defined by the quality of a single screen or feature, but by the overall flow across documentation, CLI, APIs, and UI working together.

The key challenge is to move beyond asking “How does this feature look?” and instead evaluate “How does this entire workflow feel and perform in real usage?”

Conclusion:

The key takeaway is that DevEx UX goes far beyond improving individual interfaces; it is fundamentally about improving entire systems of work. Instead of focusing on isolated UI enhancements, the emphasis shifts toward understanding and optimizing how developers interact with interconnected tools, processes, and platforms across the full lifecycle of software delivery.

This requires strong systems thinking to see how components like APIs, CLI tools, documentation, and UI dashboards influence one another, along with technical awareness to understand how these systems actually behave in practice. It also demands a deep understanding of developer workflows, including how tasks are executed end-to-end and where friction typically occurs. When approached this way, the role of a UX practitioner evolves from being a traditional UX designer focused on interface design to becoming an experience strategist who shapes and aligns complex technical ecosystems for efficiency, clarity, and scalability.

Challenge: 3 Making UX Work Understandable to Engineers

Pavanipriya Sajja — Mon, 20 Apr 2026 22:25:20 +0000

I want to explain this challenge through a real experience I had while working on improving the developer experience for KServe.

Because this is where I truly understood:

👉 UX work doesn’t fail because of bad research.
👉 It fails when developers don’t understand it fast enough to care.

The Situation:

We set out to improve usability for developers using KServe, specifically focusing on how they deploy ML models, manage inference services, and work within Kubernetes-based workflows. As a UX team, we took a structured approach: we conducted a usability study, designed task-based scenarios, and collected feedback from real engineers to anchor our decisions in actual developer needs. We even received approval from maintainers, and everything seemed solid in theory. But when it came time to present this work to developers… we struggled.

What Actually Happened:

In one of our community interactions, we presented a set of usability tasks, our research approach, and what we wanted developers to do. From a UX perspective, everything felt clear and well-structured but from a developer’s perspective, it wasn’t landing the same way. The reaction was subtle but telling: people didn’t ask many questions, feedback was minimal, and engagement was low. At first, it was easy to assume, “maybe people are just busy.” But looking deeper, the real issue became clear: we hadn’t explained the work in a way that aligned with how developers think.

Where We Went Wrong:

1. We presented too much information at once

Including multiple tasks, long end-to-end flows, and heavy contextual details. From our side, it felt like organized research with clear structure, but from the developer’s perspective, it came across very differently. It felt like: “This will take time. I’ll come back later.” And in real developer workflows, “later” often turns into never, which meant the key message and intent were easily lost in the volume.

2. We Used UX Framing Instead of Developer Framing:

We used UX framing instead of developer framing when explaining the work. We described it in terms of “usability study,” “user tasks,” and “feedback collection.” From our perspective, this language was standard and clear but developers interpreted it very differently. They were instead thinking: “What exactly do you want me to test?” “How long will this take?” and “What part of KServe is actually broken?” The gap wasn’t in the research itself, but in the fact that we didn’t clearly answer the practical, implementation focused questions developers needed upfront.

3. We Didn’t Anchor to Real Workflows:

We were talking about tasks in general, without clearly anchoring them in real developer actions. Instead of connecting them directly to things like deploying a model, defining Kubernetes manifests, or troubleshooting inference latency and failures, the examples stayed too abstract. As a result, the link to their day to day workflow was weak and not immediately obvious, making it harder for developers to see how the work applied to what they actually do.

The Turning Point:

After seeing low engagement, we changed our approach completely. Instead of trying to “present UX properly,” we asked:

👉 “How can we make this feel like a small, real developer task?”

What We Changed (And Why It Worked):

1. One Task at a Time:

We shifted to a one-task-at-a-time approach instead of sharing everything at once. Rather than overwhelming engineers with a full set of activities, we started sending one small, focused task per engineer. For example: try a specific setup step, tell us where you get stuck, and share what feels confusing during the process.

This approach worked because it reduced time commitment, made the request feel actionable, and fit naturally into their existing workflow. Instead of feeling like a large study or formal exercise, the task now felt simple and approachable like: “I can quickly try this.”

2. Short, Direct Communication (Often via DM)

We shifted to short, direct communication (often via DM) instead of relying only on meetings. Rather than scheduling long discussions, we reached out to engineers individually, sent simple, step-by-step instructions, and asked focused, specific questions. This approach changed the response rate significantly because it better matched how developers prefer to work. Developers tend to favor asynchronous communication, clear and explicit asks, and minimal overhead, so reducing friction made it much easier for them to respond and engage.

3. We Stopped Explaining “UX” — and Started Showing Friction

We stopped explaining “UX” in abstract terms and started showing real friction instead. Instead of saying, “We are conducting a usability study,” we shifted to something more direct and anchored in actual behavior: “We noticed developers get stuck during this step , can you try it and confirm?”

That small shift made a big difference because it made the problem real, made it relevant to their experience, and made it easier to respond without extra context or effort.

4. Strong Documentation Built Trust:

Once we started getting feedback, the next challenge emerged: “How did you reach these conclusions?” To address this, we improved how we documented our findings, focusing on clearly capturing what we observed, the patterns across users, the common failure points, and how the insights were derived. We made the reasoning explicit by showing a clear flow: Raw feedback → Patterns → Insight → Recommendation. This step became especially important because developers don’t just accept results at face value—they want to trace the logic and understand how each conclusion was formed. Present the documentation along with the presentation or part of the results that explains how did you reach these conclusions.

5. Repeating in Community Meetings:

Another important learning came from repeating in community meetings. In environments like KServe, not everyone attends every meeting and the context resets frequently, which means people often miss earlier explanations. To address this, we re-presented the work multiple times, explained it again across different sessions, and shared updates incrementally over time.

At first, this approach felt repetitive, but it turned out to be essential. In reality, it significantly improved awareness, increased participation, and built stronger trust within the community.

A Simple Question That Changed My Entire Feedback Strategy:

One moment really changed how I think about presenting UX work. I was in a developer team meeting where I presented a persona, and at the end, I simply asked: “Can I get your feedback?” The response I received was unexpected: “What feedback do you want from us?”
That question caught me off guard. From my perspective, I had explained the work, walked through the persona, and assumed it was obvious what kind of input I needed. But from their perspective, it wasn’t clear at all they needed more specific context and direction on what to focus on.

I was presenting in the same way I typically do in UX-focused meetings, assuming the format would translate. But developers don’t think in the same way, and they don’t automatically understand what a persona is, why it matters, or what kind of feedback is expected from them.
So when I asked for “feedback,” it was too vague and open-ended. And that’s exactly why the response came back as: “What exactly do you want from us?”

After that experience, I changed how I structured my presentation approach. Instead of jumping straight into the persona, I began explaining it step by step, starting with the fundamentals. I now start with the basics (very important)—clearly outlining what a persona is, why we are creating it, and how it will be used in the context of the work. Then present the process of persona creation form the research data and finally explain the actual persona that you created form the raw data.

For example, I created a persona of a hybrid engineer who deploys clusters across multi-cloud, on-prem, and edge environments.

When presenting this persona in a meeting, you can ask for feedback like this:

“If you are an engineer deploying clusters across cloud, on-prem, and edge environments, does this persona reflect your problems, goals, workflows, and tools?”

You can also ask:

“Does this persona represent your day-to-day tasks as a hybrid engineer, or is something missing?”

“Do these goals and pain points resonate with your experience?”

“Is there anything that doesn’t match your workflow or feels completely different?”

Based on their feedback, you can refine the persona by adding missing tasks, adjusting goals, or removing elements that don’t align with real-world workflows.

What Changed After This: Once I started asking specific, targeted questions, things began to shift. Developers responded more frequently, feedback became more detailed, and conversations turned more meaningful and insightful.

Instead of silence, I started hearing responses like: “This part is correct, but we also struggle with…”, “We don’t use this tool—we use something else,” and “This step is missing in real workflows.”

Key insight: The problem wasn’t that developers didn’t want to give feedback—it was that I didn’t clearly ask for the right kind of feedback.

When you explain the context, show your process, and ask specific, targeted questions, developers know exactly how to respond. And that’s when UX feedback actually starts working.

What Changed After This:

Once we adapted to developer behavior, the quality of interaction changed noticeably. Feedback became more specific, engineers engaged more actively, and conversations became deeper and more meaningful. Instead of silence or minimal responses, we started hearing things like: “Yeah, this step is confusing because…”, “I had to check another doc for this”, and “This part could be simplified.”

That’s when it became clear: 👉 the UX work itself didn’t change — the way we communicated it did.

The Real Insight:

This experience completely changed how I think about presenting UX in DevEx. It’s not about having perfect slides, using heavy UX terminology, or relying on structured frameworks. Instead, it’s about something much more practical and aligned with the audience.

👉 It’s about making your work feel like part of a developer’s workflow

Conclusion:

If developers don’t understand your UX work, it’s not because they don’t care—it’s usually because it doesn’t fit their mental model, doesn’t align with their workflow, or feels too heavy and abstract to act on.

But when you break things into small tasks, show real friction points, speak in terms of their actual work, and provide traceable logic behind decisions, something shifts.

UX stops being seen as “design work”… and starts being understood as engineering value.

Challenge : 2 The Project Selection Trap

Pavanipriya Sajja — Fri, 17 Apr 2026 23:25:19 +0000

Section: 1 Choosing the Right Project And Where You Can Truly Contribute

Another major challenge in Developer Experience (DevEx) design is choosing where to contribute. In the cloud-native ecosystem, different projects require completely different skills and levels of understanding. For example:

OpenTelemetry → requires knowledge of observability, telemetry data, and sometimes UI dashboards
Kubernetes / multi-cluster systems → involve infrastructure, deployment workflows, and cluster management
Kyverno → focuses on policies, governance, and configuration validation
Cilium → involves networking, security, and low-level system behavior

Why this is challenging

As a UX/DevEx designer, you may struggle with:

Where should I start?
Which project matches my current skills?
Do I need deep technical expertise before contributing?
What kind of UX problems exist in each domain?

Each area has different types of user problems:

Observability → understanding data and dashboards
Infrastructure → complex setup and debugging
Policy tools → configuration clarity and errors
Networking → invisible systems and hard-to-debug issues

It’s easy to feel stuck or intimidated, especially at the beginning.

When I started my journey in Developer Experience (DevEx), I made a mistake that slowed down my growth more than anything else:

👉 I chose too many complex projects at the same time.

I was trying to work across multiple domains Kubernetes cluster related problems and networking-heavy systems like Cilium. While these areas are part of the same ecosystem, they require very different types of thinking. The result?

I felt overwhelmed
I couldn’t focus deeply
I struggled to understand problems clearly
And I couldn’t deliver meaningful outcomes

That experience taught me a critical lesson:

👉 Choosing the right project matters more than choosing the most advanced one.

What I Discovered: Where UX Designers Can Actually Make an Impact

As a UX designer in DevEx, your value is not in mastering the most complex systems first—it’s in identifying where users (developers) struggle the most.

There are several high-impact areas where UX can make a real difference:

1. Documentation Experience (An Underrated Opportunity)

Many people say: “Contribute to documentation.”
But very few explain what that actually means especially from a UX perspective.

Documentation is not just about writing content.

👉 It’s about designing a complete learning and decision making experience for developers.

Where Documentation Breaks (From a UX Perspective)

In many developer tools, documentation has hidden usability gaps:

Broken user flows :Steps are written, but the journey is missing. Developers don’t know what comes next or how steps connect.
Unclear navigation: Important pages are hard to find. Users jump between tabs, GitHub, and docs trying to complete one task.
No real-world context :Docs explain how something works—but not when or why to use it.
Decision-making gaps: Multiple options are listed without guidance, leaving users confused.

A Real Example In platforms like KServe, there are multiple ways to deploy ML models. But the documentation often doesn’t clearly answer:

Which method is easiest for beginners?
Which approach is best for production?
What are the trade-offs between options?

So developers are forced to: Read multiple pages, Watch videos, Experiment through trial and error

👉 This slows them down and increases frustration.

What UX Designers Can Actually Do

This is where UX becomes powerful. You’re not just “improving docs”—you’re reducing cognitive load.

You can:

1. Design Task-Based Flows: Instead of scattered pages, structure documentation around real tasks:“Deploy your first model”, “Which method is use and why and when in real world use-case”, Update with the buttons for easier navigation.

👉 Help users move step-by-step without confusion.

2. Add Decision Guidance Turn choices into clear recommendations:

“If you are a beginner → use this method”
“If you need scalability → choose this approach”

👉 This removes guesswork.

3. Improve Navigation & Information Architecture Group related content logically, Add clear entry points (Getting Started, Tutorials, Advanced), Reduce unnecessary jumping between pages

👉 Make information easy to find, not just available.

4. Fill the “Missing Middle” :
Most docs jump from basic → advanced too quickly.

UX can help by adding: Intermediate steps, Visual flows, Real use-case examples

👉 This is where most users actually struggle.

5. Connect UI + Documentation : Many docs are disconnected from the actual product UI.

You can: Map UI screens to documentation steps, Add screenshots or flows, Align terminology between product and docs

👉 This creates a smoother experience.

Why designing the documentation is important: Documentation is often the first experience a developer has with a product.

If it’s confusing: Adoption drops, Frustration increases, Support questions grow

If it’s well-designed: Learning becomes faster, Confidence increases, Products feel easier even if they’re complex.

Key Insight

👉 Great documentation is not written. It’s designed.

And for UX designers entering DevEx, this is one of the highest-impact, lowest-barrier areas to start contributing.

2. Website & Navigation Design

Many developer tools are powerful—but their websites don’t reflect that. Common issues include: Poor navigation, Difficulty finding the right documentation, No task-oriented structure

If you compare:

Cilium → clean UI, consistent design system, strong navigation
Other platforms → minimal UI but lacking usability clarity

👉 This creates opportunities to: Redesign navigation, Improve information architecture, Build consistent design systems

3. Developer Workflows & Tools

Engineers use complex tools every day and small UX improvements can create huge impact.

You can contribute by:

Simplifying workflows : For example - Deploy ML models using Kserve, debugging a multi cluster issue.
Improving dashboards : For example - Observability(improve UI for Prometheus, Grafana)
Making errors easier to understand : For example - Simplified documentation.
Supporting usability studies with better UX insights: For example improve Kserve usability to identify problems who use Kserve.

Even small improvements here can significantly improve productivity.

Section : 2 Expanding Your Impact: Beyond Just One Domain

One of the biggest mindset shifts in my journey was realizing this:

👉 DevEx UX is not limited to a single domain like networking or observability. There are other multiple paths where you can contribute meaningfully based on your interests and strengths.

1. Work Across Different Engineering Roles:

You can align your UX work with different engineering domains, such as:

Platform Engineering → Focus on internal developer platforms, tools, and workflows
DevOps Engineering → Improve CI/CD pipelines, deployment experiences, and operational workflows

Each of these areas comes with its own: Tools, Practices, Pain points

👉 Which means more opportunities to identify user problems and design better solutions.

2. Contribute Beyond Technical Domains

Not all impactful work is tied directly to systems like networking or infrastructure.

There are also initiatives focused on improving the overall developer experience across organizations and communities.

For example:

Organization-wide developer experience improvements(TAG Developer Experience), TOC - echnical Oversight Committee
Internal tooling usability
Cross-team collaboration challenges

These areas focus less on what the system does and more on how people interact with it.

3. Contribute to Community & Experience-Focused Groups

In open-source ecosystems like Kubernetes, there are dedicated groups focused entirely on improving contributor and developer experience.

One great example is: Kubernetes SIG Contributor Experience

This group works on:

Improving onboarding for new contributors
Helping teams (SIGs) collaborate effectively
Identifying challenges contributors face
Creating better processes and guidance
Supporting teams in maintaining healthy project structures

👉 Instead of building features, they focus on making it easier for people to contribute and succeed.

Why This Matters : This expands how you think about DevEx UX:

It’s not just about systems
It’s not just about UI
It’s about people, workflows, and collaboration at scale

Key Insight

👉 You can contribute in multiple ways:

Deep technical domains (like networking, observability)
Engineering workflows (platform, DevOps)
Organization-level experience improvements
Community and contributor experience

You don’t have to choose the hardest path you have to choose where you can create the most impact.

Section: 3 The Real Challenge: How to Choose the Right Project

After struggling early on, I changed my approach completely.

1. Start with the UX Problem (Not the Technology)

Before choosing a project, ask yourself:

Is this a documentation problem?
A workflow complexity issue?
A UI or visualization gap?
A debugging or error clarity problem?

👉 This helps you align your strengths with real needs.

2. Look for Visible UX Gaps

Some areas are easier to start with:

Documentation-heavy projects
Onboarding flows
Developer workflows
Github: good first issue-Specially explained UX problem

These provide clear opportunities for UX impact.

3. Collaborate with Engineers

Don’t assume, ask:

What is hardest to debug?
What takes the most time?
Where do new users struggle?

👉 This gives you real insights into developer pain points.

4. Don’t Avoid Complexity Approach It Gradually.

Domains like networking or multi-cluster systems can feel overwhelming at a time. But: You don’t need to understand everything at once, Start from the user perspective, Focus on simplifying one problem at a time, for example understand networking completely then jump into infrastructure.

👉 Your technical understanding will grow naturally over time.

Conclusion:

Choosing a DevEx project is not about picking the most advanced system.

It’s about identifying:

Where developers struggle the most
Where UX can bring clarity
Where you can contribute with your current skills

👉 Start small. Focus on real problems. Grow into complexity step by step.

Challenge 1: The Learning Curve Feels Endless (and Unclear)

Pavanipriya Sajja — Thu, 16 Apr 2026 22:52:25 +0000

When I moved from a general UX role into Developer Experience (DevEx) design, I thought the transition would be simple.

Same UX process… just a more technical domain. But my own journey quickly proved otherwise.

I come from an Electronics and Communication Engineering background, which helped me understand systems thinking early on. During my UX journey, I also learned HTML, CSS, and JavaScript, and later worked in a startup where I led both web development and UX efforts. I was involved in everything from planning and design to development and production.

Because of this, I thought I had a strong foundation. But when I stepped into Kubernetes and DevEx… Everything felt wide, complex, and undefined.

Where the Confusion Started

As I began exploring Kubernetes and developer platforms, I realized something:

👉 DevEx is not just UX in a technical space, It's a completely different design problem.

There isn’t a single learning path in DevEx.

Depending on the project, the expectations kept shifting:

Some required understanding CLI tools
Others focused on improving documentation UX
Some needed API-level understanding
Others required mapping full developer workflows

Even when I started contributing to open source, I struggled with questions like:

What should I learn first?
How technical do I need to go?
Am I even focusing on the right thing?

I initially tried to learn everything — Kubernetes concepts, tools, workflows, GitHub contributions…

That approach didn’t work.

What Changed Everything:

The turning point in my journey came during my first meaningful open-source contribution.

I found a “good first issue” where engineers were struggling with a confusing UI. The interface had multiple repetitive actions represented as buttons, which created usability issues.

At that moment, I realized:

👉 I don’t need to know everything about Kubernetes to contribute
👉 I need to understand just enough to solve the right problem
I focused on:

Understanding the user flow
Identifying UX friction
Collaborating with engineers to explain my solution

That experience helped me bridge UX and developer needs — and more importantly, it reshaped how I approached learning.

Solution: Learn Based on Context, Not Completeness

One of the biggest lessons from my journey is this:

👉 In DevEx, the problem is not lack of learning — it's lack of direction.

Here’s what actually worked:

1. Start with the Developer Workflow

Before learning tools, ask:

What is the developer trying to do?
What does success look like?

For example:

Deploying a model
Setting up a Kubernetes cluster

This gives clarity on what actually matters.

2. Go Deep Only Where Needed

In my case:

While exploring Kubernetes → I focused on core concepts + workflows
During my contribution → I focused on UI/UX clarity

You don’t need everything at once. You need relevance.

3. Learn “Just Enough” Technical Depth

My background helped, but I still had to adjust how I learned:

Focus on concepts over code
Understand flows over implementation
Learn enough to ask better questions

4. Developers as Learning Partners

One of the most underrated accelerators in my journey:

Asking engineers to walk through real workflows
Observing how they actually use tools
Clarifying assumptions early

This not only improved my understanding, it also built trust.

5. Build a System Mental Model

Over time, I stopped thinking in silos like: “CLI vs API vs UI”
Instead, I started seeing:

How everything connects
Where developers struggle
How experiences break across tools

That shift is what truly defines DevEx thinking.

Key Takeaway

My journey into Kubernetes and DevEx taught me something important:

👉 You don’t need to learn everything.
👉 You need to learn what matters for the problem you're solving.

The real skill in DevEx is not technical depth alone, it's knowing where to focus and why.

How we derived behavioral and motivational patterns for user persona?

Pavanipriya Sajja — Thu, 12 Mar 2026 17:34:14 +0000

This article explains end to end process of the methodology used to identify and derive behavioral and motivational patterns from UX research data collected on multi-cluster Kubernetes operations.

The analysis draws on a Google Sheets sample dataset containing 20 verbatim quotes, 20 pain point themes, and 20 desired solutions — 60 data points in total — used here as a working example to illustrate how patterns are surfaced, calculated, and translated into actionable persona insights.

Here is the link for the google sheets:

Sample Raw Data - Google Sheets

docs.google.com

The goal of this methodology is to move beyond surface-level pain points and equip product, design, and engineering teams with a deeper understanding of how platform engineers, SREs, and related roles actually behave and what outcomes they are genuinely optimizing for.

How did we Derived Behavioral Patterns?

Behavioral patterns are identified by looking at what people are actually doing in response to a problem — the observable actions, workarounds, and habits described in the data. The question we asked of each data point was: "What is this person doing right now to cope with this situation?"

We grouped responses that described similar actions — not similar topics — and named the underlying behavior.

Our Worked Example: "Workaround Accumulation"

These three raw data points were the anchors:

"Cost visibility is nearly zero at the namespace level. We have no easy way to map spend to teams without custom Prometheus dashboards that we build and maintain ourselves."

"GitOps drift detection is useful but noisy. ArgoCD sends so many sync notifications that engineers create automation to silence them."

"Capacity planning for multi-tenant clusters requires manual analysis... we rely on gut feel and occasional spreadsheets."

On the surface, these look like three separate problems: cost, notifications, and capacity. But the behavior across all three is identical — the platform doesn't provide what's needed, so the engineer builds something themselves. Custom dashboard. Silencing automation. Spreadsheet.

The critical observation is the second half of each quote: "that we build and maintain ourselves," "create automation to silence them," "occasional spreadsheets." These aren't solutions — they're debt. Each workaround introduces something that can break, drift, or fall out of date.

That's what distinguishes this as a behavioral pattern rather than just a pain point cluster: it's a repeating action (self-build), triggered by a repeating condition (platform gap), producing a repeating consequence (maintenance burden).

The pattern name — "Workaround Accumulation" that captures both the behavior and its compounding nature over time.

How did we Derived Motivational Patterns

Motivational patterns require going one layer deeper. Instead of asking "what are they doing?", we asked "why are they doing it — what outcome are they actually trying to reach?" This is where you look past the stated problem to the underlying goal.

The method is essentially asking "what would success feel like to this person?" and finding where multiple people converge on the same answer, even if they're describing different problems.

Our Worked Example: "Desire for Speed with Confidence"

These three data points pointed us here:

"A dry-run mode for NetworkPolicy that shows which traffic flows would be blocked before the policy is applied."

"A visual RBAC policy editor that validates configurations before applying them to the cluster."

"Lightweight remote dev environments that mirror staging infra so engineers can test against real dependencies locally."

The surface topics are completely different — networking, access control, local development. But if you ask "what is the engineer actually trying to achieve?" across all three, the answer converges: they want to act quickly, but they need to know it won't break something first.

Notice what they're NOT asking for. They're not asking to slow down. They're not asking for approval gates or more review cycles. They're asking for earlier feedback so they can move confidently. Dry-run, visual validation, staging parity — all of these are mechanisms that move the moment of truth earlier in the workflow, before consequences become real.

This is what separates a motivational pattern from a feature request. The feature request is "build a dry-run mode." The motivation underneath it is "I want to move fast without causing outages." Once you see that motivation, you realize a dry-run mode, a staging mirror, and a policy validator are all answering the same psychological need — and that a product team solving for just one of them is only partially addressing what the engineer actually wants.

The Core Methodological Difference

	Behavioral Pattern	Motivational Pattern
Question asked	What are they doing?	Why are they doing it?
Data cues	Verbs: building, jumping, silencing, ignoring	Goals implied: "so I can," "without," "before"
Grouping logic	Same action across different contexts	Same underlying goal across different actions
Risk if missed	You fix the symptom, not the habit	You build features that don't address the real need

behavioral patterns show platform teams what is happening today, while motivational patterns show product and tooling teams what engineers are actually optimizing for, which is where the most actionable design direction lives.

How to integrate this patterns into persona:
There are a few ways to integrate these patterns into your personas depending on your audience

1. Embedded Section within each persona You add a "Behavioral & Motivational Profile" block directly inside each persona card, right after Goals or Frustrations. Clean and self-contained.

2. Cross-persona Pattern Matrix A separate table or section that maps each pattern to the personas it affects. Better for showing systemic themes across all four personas

3. Pattern as a Quote + Insight block Each pattern is anchored by a real verbatim quote from your research(Qualitative and quantitative) data, followed by the behavioral or motivational insight.

We can also include behavioral and motivational patterns in percentage format. Presenting these patterns with percentages makes the insights more credible and easier to reference or cite within the persona.

Again, let’s go back to the sample raw data. The dataset contains 20 qualitative responses (verbatim quotes), 20 pain point themes, and 20 desired solutions, which gives us 60 data points in total.

However, in practice, the 20 verbatim quotes serve as the primary evidence base for identifying behavioral patterns, because they describe the engineers’ actual actions, experiences, and workflows.

Here's my approach to calculate percentages honestly:

Method: Evidence Mapping For each behavioral pattern, I'll count how many of the 20 raw responses contain explicit evidence of that behavior — not just the topic, but the actual action being described.

Let me map this now (Based on the sample research data):

Behavioral Pattern	Responses with direct evidence	% of 20 responses
Reactive Debugging Over Proactive Monitoring	Responses 1, 14, 5 (jumping dashboards, blocked traffic, silent misconfigs)	3/20 = 15%
Workaround Accumulation	Responses 7, 15, 19 (custom dashboards, spreadsheets, silencing automation)	3/20 = 15%
Context Switching as Default Workflow	Responses 1, 3, 11 (three dashboards, six wikis, fragmented secrets)	3/20 = 15%
Tribal Knowledge Dependency	Responses 3, 16, 9 (scattered docs, deep controller knowledge, CRD research)	3/20 = 15%
Standardization Avoidance	Responses 8, 11, 13 (mixed rollback, mixed secrets, brittle local dev)	3/20 = 15%
Alert Desensitization	Responses 10, 19 (ignoring alerts, silencing GitOps notifications)	2/20 = 10%

The honest point I want to highlight here is that 20 responses represent a small qualitative sample, so percentages derived from this data may appear more statistically precise than they actually are.

When working with a larger dataset—for example, more than 100 responses (n ≥ 100) — the percentages become more reliable and meaningful. That is where percentage-based insights should ideally come from.

What is the difference between the straight percentage version and the detailed version when presenting behavioral and motivational patterns in a persona card? Which approach is the best way to present these patterns?

There are two approaches:

1. Straight Percentage Version (Quantitative Layer) This approach presents behavioral or motivational patterns using only numerical insights derived from the research data.

Example Behavioral Pattern: Reactive Debugging Observed in 68% of respondents

Raw data: Engineers wait for failures to surface before investigating, jumping between tools after the fact rather than catching issues proactively.

Advantages

Easy to read and scan quickly
Looks data-driven and credible
Works well for presentations and executive summaries

Limitations

Lacks context about why users behave that way
May oversimplify complex behaviors

Now let’s learn about another approach:

2. Detailed Version (Qualitative Layer): This approach combines behavioral patterns with contextual explanations, quotes from research participants, and the underlying trigger behind the behavior.

Example of Behavioral Pattern: Reactive Debugging Over Proactive Monitoring

Quote:

“When a pod crashes, I need to jump between three dashboards just to find the root cause.”

Trigger: Lack of integrated observability tools
Action: Manual investigation across multiple tools
Cost: Hours lost during each incident

Advantages

Provides deeper insight and context
Shows the reasoning behind behaviors
Stronger for research reports and documentation

Limitations

Longer and harder to scan quickly
Less suitable for compact persona cards

Which Is Best For Each Pattern Type

This is the key insight — they serve different pattern types differently.

Behavioral patterns can carry percentages because behavior is observable and countable. "X% of respondents described workaround-building behavior" is a defensible, citable claim.

Best Practice

The most effective approach is usually a hybrid format:

Use percentages for credibility
Add a short explanation or quote for context

Motivational patterns should almost never be percentages. Motivations are inferred, not directly stated. Saying "72% are motivated by speed with confidence" would be misleading — no one said that directly, you derived it. A quote + insight format is far more honest and persuasive here.

The reason for including behavioral and motivational patterns is based on five main purposes.

A persona without behavioral and motivational patterns only tells you who the person is.

A persona with these patterns shows how the person thinks and what motivates their decisions.

This difference is important because design and product decisions are not based on demographics or job titles. They are based on understanding how people behave in real situations and what outcomes they are trying to achieve.

1. Move from Descriptive to Predictive: Without patterns, a persona describes a person at a snapshot in time. With patterns, it lets you predict how that person will behave in a new situation you haven't researched yet.

Example: For example — once you know Alex (Platform Engineer) has a Workaround Accumulation behavioral pattern, you can predict that if you ship a feature with gaps, Alex won't file a bug report. He'll build around it. That prediction changes how you design the feature in the first place.

Purpose: Make personas useful for decisions that go beyond what was directly researched.

2. Explain Why Pain Points Exist, Not Just That They Do: Pain points alone tell you what's broken. Behavioral and motivational patterns tell you why it keeps being broken even when people know it's a problem.

Example: Alert desensitization is a great example from sample raw data. The pain point is "too many noisy alerts." But the behavioral pattern — engineers actively suppressing alerts as a coping mechanism — explains why just reducing alert volume won't fix it. You've broken trust.

The motivational pattern underneath, cognitive offloading — tells you the real design target: engineers need the system to carry the triage burden, not just send fewer notifications.

Purpose: Surface the root cause behind the symptom so solutions address the right problem.

3. Create Shared Language Across Teams: When a designer, a PM, and an engineer all read the same persona card, they need to walk away with the same understanding of the user. Job titles and tools lists don't achieve that — they're interpreted differently by different disciplines.

Behavioral and motivational patterns are discipline-neutral. "Reactive debugging over proactive monitoring" means the same thing to a backend engineer as it does to a UX researcher. It becomes a shorthand the whole team can reference in standups, design reviews, and roadmap conversations without needing to re-explain the research.

Purpose: Give cross-functional teams a common reference point rooted in user reality.

4. Prevent Solution-First Thinking: Without motivational patterns especially, teams default to building what users asked for rather than what users actually need. Your data is full of this gap, users asked for a dry-run mode for NetworkPolicy, but the motivation underneath is "I want to act fast without causing outages." Those are different design briefs.

If a PM only reads the pain points and desired solutions sections of a persona, they'll build a feature checklist. If they also read the motivational patterns, they'll ask "does this solution actually address the underlying motivation, or just the surface request?"

Purpose: Shift teams from feature-building to outcome-building.

5. Make Personas Defensible in a Research Context: A persona card that only contains job title, tools, and frustrations reads as anecdotal. Reviewers will ask — how do you know this is representative?

Behavioral and motivational patterns — especially when tied back to evidence from your (n>100) research respondent — demonstrate that the persona was derived from systematic analysis, not assembled from assumptions.

They show the reasoning chain: here is what people said, here is the pattern we observed across multiple respondents, here is what that tells us about this persona type.

Purpose: Establish research credibility and make the personas publishable and citable.

Conclusion:

Behavioral and motivational patterns are what elevate a persona from a static profile into a practical decision-making tool.

By grounding each pattern in observable actions and inferred goals — backed by evidence from the research dataset — teams can predict user behavior, address root causes rather than symptoms, and build toward outcomes rather than feature checklists.