As part of the Förderverein AWS Community DACH e.V. I joined the selection committee for talks again this year. Third year in a row (I wrote about the previous editions and my broader community journey in my 2025 year in review). The process is always inspiring, fair, and full of incredibly well-crafted abstracts.
How the evaluation works
Sessionize uses a Comparison Evaluation Mode based on the Elo rating system (from chess). Three sessions are shown at the same time and you rank them relative to each other. You don't assign absolute scores; you just decide "this one is better than that one."
Each comparison produces three "games" (A vs B, B vs C, C vs A) and the algorithm adjusts ratings accordingly. After a first pass through all sessions, the system gets smarter about which triplets to show next, targeting sessions with similar ratings for more precise differentiation.
It's fast, focused and surprisingly fair, because you only ever think about the three in front of you, not all others.
You can also use "Strong opinion" options: Top (this is excellent), Doesn't fit (wrong conference), or Ignore (conflict of interest, can't judge). These signal the algorithm without distorting the ranking.
In my case: I marked 2 as "Doesn't fit" (not because they lacked quality; they were actually interesting but too inspirational for a technical community day, better suited for a different audience).
The other 2 ignored were my own submissions: Vibecoding in Between Meetings and Serverless vs Kubernetes - The Final Showdown.
The full process
The behind-the-scenes post by Philipp Garbe explains the whole pipeline in detail. In short:
- Round 1 (Screening): Board members filter out spam, incomplete proposals, marketing pitches, non-AWS topics, and speakers outside EMEA.
- Round 2 (Content Evaluation): Every association member can participate. This is the Elo-based comparison round. Members must disclose personal connections and skip sessions they can't rate objectively.
- Round 3 (Final Review): The board balances the ranked list: one talk per speaker, topic diversity, speaker diversity, and overall programme coherence.
My stats
| Metric | Value |
|---|---|
| Total sessions | 151 |
| Avg time per session | 87.4 s |
| Total time spent | 3h 37m |
| Doesn't fit | 2 |
| Ignored (my own) | 2 |
The ratings distribution follows a near-perfect Gaussian curve, which is a good sign: it means the method is working. Most sessions cluster around the mean, with a clean spread toward the tails.
Why it's hard
Despite the elegant method, evaluating still requires serious thought. It took me on average 87 seconds per session to read the abstract, check the speaker profile, and make a judgement call.
And, while going through 151 submissions, I can't help feeling uncomfortable. There is so much I don't know, even after 20+ years in the industry. So many deeply technical talks on topics I've barely touched. It's humbling and motivating at the same time.
Beyond the imposter syndrome, some specific challenges:
Balancing topics. More that a quarter of all submissions this year were about AI agents. AgentCore, MCP, agentic SRE, agent security... there might be several excellent ones, but you don't want a monothematic programme. Serverless, security, platform engineering, FinOps were also heavily represented, and I tried to pick the best ( or at least my favourite) among those that were very similar (there were multiple submissions about Lambda Durable Functions, or Cost Optimisations and Souvereign Cloud).
Fighting my own bias. Left to my preferences, the programme would be all Serverless, Kiro, and FinOps. I'd shrug off the deeply technical networking or Kubernetes talks. I have to actively fight that.
The 20% rule. A guideline says at least 20% of the audience should be interested in the topic. This one hurts. Some submissions are extremely technical and well-crafted but incredibly niche. They deserve a stage, just maybe at a more specialised meetup or conference.
Diversity without tokenism. I don't let myself be influenced by renowned speakers; I want to include new faces and give opportunity to underrepresented voices. But I also won't favour a talk just to have a diverse stage. Quality first, then balance.
The abstract vs the speaker dilemma. Sometimes the abstract is meh. The story doesn't tell much, could be just recycled docs. Then I see who the speaker is, someone I know and trust, and I change my mind. I am sure, they will have an incredible story, and provide valuable insights from their hands-on experience.
Is this unfair? Maybe. But the responsibility of choosing the best content is high, and if the abstract doesn't convince and the speaker profile gives no guarantees, I honestly don't see other options.
I sometimes wonder if "blind" reviews would be fairer: only title and abstract in Round 2, then speaker identity in Round 3 for balancing. From what I've seen, the Sessionize comparison mode is the most common approach, but I'd love to hear from other organisers how they handle this.
Please stop with the AI abstracts
Tell your story. Of course use AI to polish your language. But more and more abstracts sound identical: same hyperboles, same structure, same buzzwords ( and same words, that nobody in real life would ever use...)
For many, that signal alone was enough for a downvote.
In this great talk, Einar Høst explains why AI-generated abstracts get rejected from NDC Oslo. That was 2024. Things have only gotten worse.
The mistakes, the pitfalls, all the things the documentation "lied" about, are what make a talk and its abstract compelling.
Learning from someone's real failures helps other devs and teams succeed faster. A catchy title and an honest, specific story beats any amount of AI-polished fluff.
What the community wants to talk about
I categorised most submissions by topic. Here's what the community wanted to talk about:
- AI Agents & Agentic Architecture: 24% - the dominant theme by far ( + 9 submissions about AI/ML and 5 about AI Native DLC)
- Serverless & Lambda: 11% - mature, and evolving. Especially with the announcement from last ReInvent.
- Security & Zero Trust: 10%
- Platform Engineering & IaC: 9% - Terraform, CDK, GitOps, multi-account governance
- Networking: 4% - VPC modernisation, PrivateLink, Gateway API
- FinOps: 4% - tagging, Spot, commitments
- Containers & K8s: 4% - EKS, ECS, DRA
- Data & Databases: 4% - Aurora DSQL, S3 Tables, OpenSearch
- Observability: 3% - OpenTelemetry, alert fatigue
- Sovereign Cloud: 3% - CLOUD Act, European Sovereign Cloud
Agents are clearly the hottest trend. And the more engineers focus on building agentic AI solutions, the more other engineers (cloud and security folks) are right behind them, asking "but what happens when it goes wrong?"
Thank the organisers (and give your feedback)
If you go to a conference, talk to the organisers. Thank them. There is a huge amount of work behind the scenes. I'm involved for roughly 4 hours of evaluation plus a couple of meetings. But finding sponsors, venue, catering, and everything else? That's a full-time job, except it's voluntary, and everyone already has a full-time job.
And please: give your feedback. At the end of each talk there's always a QR code or an app to rate the session. From my experience at several conferences around Europe, less than 30% of attendees take a few seconds to star a session, let alone leave a proper comment. That's a pity, because it's only with attendee feedback that organisers and evaluators can improve and choose the best speakers next year.
Final thought
Overall I'm grateful to be part of this. And if I can do it from the beach, it's not even a big commitment. On a long lazy day I'd likely reach for my phone anyway to doom-scroll LinkedIn or Instagram. This is a much better use of my time.
Results will be out in a couple of weeks. May the best speakers win the spotlight on the stage of AWS Community Day DACH in Berlin on 15th of September.
Top comments (0)