Note: This tool was never officially released. The day I was scheduled to demo it to our wider engineering team, restructuring happened—affecting me. This post is my way of giving the work the light of day it deserves. The tool is live at the link below, built through countless iterations and hard-won lessons about what makes developer tooling actually useful.
The Documentation Paradox
I was drowning in my own documentation. As a technical documentation writer on the Weaviate developer education and experience team, I'd spent months writing comprehensive guides about deploying our vector database on EKS. Configuration guides, best practices, security recommendations, networking tutorials—all meticulously crafted, reviewed, and published. Yet the support tickets kept climbing.
"How do I configure my node groups for Weaviate?"
"What instance types should I use for vector indexing workloads?"
"My Weaviate pods keep getting evicted—what am I doing wrong?"
"Can someone just tell me what YAML I need?"
The pattern was clear: developers weren't struggling because the information didn't exist. They were struggling because assembling that information into a working Weaviate deployment on EKS was overwhelming.
I needed to find a better way.
The Problem: When Complexity Meets Choice
Amazon Elastic Kubernetes Service (EKS) is powerful, but that power comes with complexity—especially when you're deploying a stateful, memory-intensive application like Weaviate. A production-ready EKS cluster for Weaviate requires decisions across multiple dimensions:
- Infrastructure: Node groups, instance types (memory-optimized vs compute-optimized), scaling policies for vector workloads
- Storage: EBS volume types (gp3 vs io2), provisioning for HNSW indices, persistent volume configurations
- Networking: VPC configuration, subnets, security groups, CNI plugins, service mesh considerations
- Security: IAM roles for S3 backups, RBAC policies, pod security standards, encryption at rest
- Observability: Prometheus metrics for query performance, CloudWatch integration, distributed tracing
- Add-ons: CoreDNS, kube-proxy, VPC CNI, EBS CSI driver, cluster autoscaler
And here's where it gets tricky for Weaviate deployments: each decision influences others in ways specific to vector database workloads.
- Choose
r6imemory-optimized instances? You'll need specific EBS volume configurations to match IOPS requirements for HNSW indexing. - Enable multi-tenancy? Your RBAC and network policies need to isolate tenant data.
- Planning for horizontal scaling? You need StatefulSets with headless services and proper pod disruption budgets.
- Using S3 for backups? You'll need IAM roles with specific permissions attached to service accounts via IRSA (IAM Roles for Service Accounts).
I'd written comprehensive documentation covering each topic. We had guides on:
- Selecting instance types based on vector dimensionality and dataset size
- Configuring persistent storage for optimal HNSW performance
- Setting up monitoring for query latency and indexing throughput
- Implementing backup strategies with S3 integration
But developers still needed to:
- Read multiple scattered pages across Weaviate docs and AWS documentation
- Understand how EKS, Kubernetes, and Weaviate-specific requirements interact
- Translate conceptual knowledge into correct YAML manifests
- Size their cluster appropriately for their specific vector workload
- Validate their choices against best practices
- Troubleshoot when combinations didn't work (and they often didn't)
I was asking people to become experts in EKS, Kubernetes storage, vector database performance characteristics, AND AWS networking just to deploy Weaviate. That's unreasonable.
The Inspiration: Learning from Competitors and Our Own Tools
Then I saw something that shifted my thinking. One of our competitors had built a configuration and sizing tool for their vector database. It asked questions about dataset size, query patterns, and performance requirements, then recommended hardware specifications and configuration settings. It wasn't perfect, but users loved it.
Around the same time, I was looking at Weaviate's existing Docker Configurator—a tool that helped users generate docker-compose.yml files for local development. It worked beautifully for getting developers started quickly without drowning them in documentation.
The connection clicked: What if I built something similar for EKS deployments?
This wouldn't just be nice to have—it could be a competitive differentiator. If our competitor could help users size their infrastructure, why couldn't we help them configure an entire production-ready EKS cluster specifically optimized for Weaviate?
The Realization: Documentation Has Limits
After analyzing support patterns and seeing what worked elsewhere, I had an uncomfortable realization: some concepts resist text-based explanation.
When configuration involves:
- Multiple interdependent choices (changing instance type affects storage IOPS, which affects indexing performance, which affects scaling decisions)
- Context-dependent recommendations (a semantic search application has different requirements than a recommendation engine)
- Complex syntax (Kubernetes YAML that must be precisely correct, plus Helm values, plus AWS-specific annotations)
- Validation requirements (certain combinations are invalid or suboptimal for vector workloads)
- Domain-specific expertise (understanding both EKS operations AND vector database performance characteristics)
...traditional documentation struggles. I could write it, but developers couldn't easily apply it.
I needed a different approach.
The Hypothesis: Show, Don't Just Tell
What if instead of explaining all the configuration possibilities, I guided developers through making the right choices for their specific Weaviate deployment?
What if the tool could:
- Ask contextual questions about their vector search use case
- Recommend instance types based on vector dimensionality and dataset size
- Explain options inline, right when decisions are made
- Validate combinations specific to Weaviate workloads in real-time
- Generate correct, tested EKS and Weaviate configuration automatically
This wouldn't replace my documentation—it would integrate with it, creating a complementary experience. Just like our Docker Configurator worked alongside our Docker deployment docs.
Building the EKS Configurator
I built the EKS Configurator as an interactive tool that bridges the gap between documentation and implementation.
Design Principles
I needed this tool to feel natural, not like another form to fill out. Here's what guided my design:
Context-Aware Guidance for Weaviate Workloads
The tool knows Weaviate-specific context that generic EKS docs can't anticipate.
Actionable Output
I generate complete, ready-to-use configurations:
-
eksctlcluster configuration YAML -
storage-classesconfiguration templates for EBS volumes.
No translation layer—just copy, customize if needed, and deploy.
The Technical Stack
I chose Streamlit for rapid development. As a self-taught developer and technical writer, I needed something I could iterate on quickly without wrestling with frontend frameworks. Streamlit's Python-based approach let me focus on the logic and user experience.
The architecture:
Input Layer:
- Streamlit forms and widgets capture user requirements
- Session state management keeps track of choices across pages
- Conditional rendering shows/hides options based on context
Decision Engine:
- Python logic validates combinations and applies AWS best practices
- Weaviate-specific sizing calculations (memory = vectors × dimensions × 4 bytes × safety factor)
- Decision trees map use cases to optimal configurations
Template Generation:
-
eks-cluster-config -
storage-classesfor EBS volumes
The key technical challenge was balancing flexibility with simplicity. I didn't want to expose every possible EKS configuration parameter, but I needed enough options for real-world use cases.
Another challenge: keeping generated YAML valid. I wrote validators that check:
- Required fields are present
- Values match expected types and formats
- Kubernetes resource names follow naming conventions
- Storage sizes are appropriate for instance types
- Memory requests don't exceed node capacity
The tool makes it feel less like filling out a form and more like consulting with a Weaviate deployment expert.
Iteration Through Feedback
The first version was... imperfect. I learned quickly through user feedback and support ticket patterns:
Too Many Options
My first iteration tried to expose every possible EKS configuration option because I wanted to be comprehensive. Users were paralyzed by choice.
I was recreating the documentation problem in tool form.
I learned to provide sensible defaults based on environment type, with the ability to customize when needed. Most users just needed to specify their workload characteristics—the tool could infer the rest.
One-Size-Fits-All
My first templates assumed everyone wanted production-ready configurations with high availability, monitoring, and security hardening. But some users just needed a quick dev environment to test Weaviate features. Others needed proof-of-concepts with minimal costs.
The Impact
Honestly, I didn't expect the impact to be significant, but internally the signals were clear - this tool would be helpful for our community users.
Personal Win:
The tool became a differentiator, not just a helper—exactly what I'd hoped when I first saw our competitor's sizing tool.
The tool and documentation became complementary, each strengthening the other. The tool didn't replace my docs; it made them more accessible and actionable.
Lessons Learned: Tools vs. Docs
Building the EKS Configurator taught me when to build tools instead of (or alongside) documentation. Here's my framework:
Build a Tool When:
✅ Configuration has many interdependencies
Manual validation is error-prone. For Weaviate on EKS, instance type affects memory, which affects storage sizing, which affects IOPS requirements, which affects volume type selection. A tool can validate these combinations automatically.
✅ Context dramatically changes recommendations
A semantic search application has different requirements than a recommendation engine. Static docs can't be sufficiently specific for every use case, but a tool can ask questions and tailor output.
✅ Syntax precision is critical
Kubernetes YAML is unforgiving. One wrong indent, one missing field, one incompatible value—and your deployment fails. Generating correct YAML prevents hours of troubleshooting.
✅ Onboarding friction is measurable
High support ticket volumes and long time-to-first-deployment indicate pain. If you're repeatedly answering the same configuration questions, that's a signal.
✅ Validation can be automated
If you can programmatically determine "this combination won't work" or "this configuration is suboptimal," build that into a tool. Don't make every user learn the hard way.
✅ You have working examples to build from
I had the Docker Configurator as a template and competitor tools as inspiration. Starting from scratch is harder.
Stick with Documentation When:
📝 The concept is fundamentally simple
Don't build a configurator for "How to install kubectl." That's over-engineering.
📝 Flexibility is paramount
Some users need complete control without guardrails. Advanced users might want configurations the tool doesn't support. Always provide an escape hatch to manual configuration.
📝 Maintenance burden would be high
If your configuration options change weekly, a tool might become a maintenance nightmare. I was lucky—EKS best practices are relatively stable.
📝 Understanding is more important than speed
Some topics require deep reading and comprehension. You can't shortcut learning Kubernetes fundamentals with a tool.
📝 The audience is already expert-level
If you're writing for platform engineers who live in Kubernetes daily, they might prefer full control over guided configuration.
The Sweet Spot: Integration
The most powerful approach isn't "tool OR documentation"—it's documentation-integrated tooling:
- Tools provide fast, guided paths to success for beginners
- Documentation provides depth, context, and customization knowledge for those who need it
- Each links to the other contextually
- Users choose their own learning path based on their experience level and timeline
I'm a technical writer who built a tool. That perspective helped me keep documentation at the center—the tool exists to make the docs more actionable, not to replace them.
Looking Forward
The EKS Configurator changed how I think about developer experience.
Short-term:
- GKE and AKS configurators: Users deploy Weaviate on other Kubernetes platforms too
- Terraform output: The most-requested feature from users managing infrastructure as code
- Advanced customization options: More granular control for power users while keeping defaults simple
Long-term:
- Interactive documentation: Embedding configurator-style tools directly in our docs site
- Feedback loops: Using tool usage patterns to identify documentation gaps and unclear concepts
- Multi-tool experiences: Connecting the EKS Configurator with monitoring setup tools, backup configuration tools, etc.
The bigger vision:
I want to create a full deployment journey: sizing tool → infrastructure configurator → application deployment → monitoring setup → backup configuration. Each tool focused on one problem, but all working together seamlessly.
Some think that as a technical writer, our job is to explain things clearly. But I was a developer first, and I know it's to remove friction from understanding and implementation. Sometimes that's documentation. Sometimes that's tooling. Often, it's both working together.
Key Takeaways
Some concepts are better shown through tools than explained in text - Complex, interdependent configurations benefit from interactive guidance
Tools and documentation should be tightly integrated - They're not competitors; they're complementary experiences
Interactive experiences reduce cognitive load for complex topics - Guided decision-making beats exhaustive reference material for onboarding
User feedback should drive tool and documentation evolution - Monitor both support tickets and usage patterns to identify friction
Know when to build vs. when to write - Not every topic needs a tool, but some topics desperately do
Try It Yourself
Explore the EKS Configurator and see how interactive tooling complements documentation. We'd love to hear how you're bridging docs and tools in your organization.
Questions? Feedback? Share your thoughts on building better developer experiences at the intersection of documentation and tooling.
Have you built tools to complement your documentation? What worked, what didn't, and what would you do differently? Let's continue the conversation.
Top comments (0)