How to measure success in Developer Relations is one of the most discussed (or debated) topics. Having been in Developer Relations for a good number of years I believe it’s important we measure it. Without measuring we don’t know what to improve, what works and what doesn’t work.
Now, I don’t believe there is one way to measure Developer Relations. Every company with a Developer Relations organization or a team will measure its success differently. The same way companies approach success in general, every company has its own unique ways to measure success.
Instead of telling you how to measure success, I’d like to offer a framework. The framework can be adjusted for a company’s specific goal and needs. The framework consists of three parts:
- DevRel Qualified Leads
- Measuring something (tied to business goals)
- Helping developers
DevRel Qualified Leads
The first part is a wonderful framework by Mary Thengvall: DevRel Qualified Leads: Repurposing a Common Business Metric to Prove Value. You can also watch a video of her talk:
Mary’s DevRel Qualified Lead is based on Marketing Qualified Lead. As Developer Advocates are in the community, they meet different people that bring value to different departments within an organization. For example, if you meet someone who has been answering many questions on the forum about your product or has built interesting demo applications, you can connect this person to the Product team so he or she can share feedback or participate in a beta program. This person can also be connected with Marketing for a case study, a guest blog post or a testimonial. This is the list of connections based on Mary’s article:
- Guest content, case study, testimonials → Marketing
- Bugs, interesting solutions → Engineering
- Product feedback, beta testing → Product
- Integrations, partnerships → Business Development
- Hiring → Recruiting/HR
- Potential customers → Sales
This framework shows how many developers we helped by connecting them with marketing, sales, product, engineering or human resources. At the same time we provide value to the company.
As Mary recommends in her article, it’s vital we keep track of these interactions (CRM or any other similar tool). When we have a record of these connections and the value they bring, this is an excellent way to show the value of Developer Relations to the company.
I love this framework from Mary. It’s an excellent approach for many companies, especially for startups, small and medium size companies. Now, although big organizations should leverage this too, I think the bigger the organization the more focus there probably will be on concrete metrics from the leadership such as number of active developers. So that’s part two of the framework.
Measuring something (tied to business goals)
(OK, the title of this is not best but I couldn’t come up with anything else. Let me know if you have another title.)
I believe some companies and especially big enterprises would like to see Developer Relations success more in terms of metrics such as number of active developers or number of projects created. This is partially due to how these companies have been measuring success in the past for other areas of the business — leadership wants to see concrete numbers.
Now, what you measure is not as important as measuring something that ties to your business goals and objectives. I think this is crucial if you want to make Developer Relations succeed at your organization. Tying the program to business goals and measuring the right things — will allow you to show the value of the program to your leadership.
Different companies will measure different things because they have different goals and objectives.
If you have a new product or service that not many developers know about then awareness might be the goal. In the case you can measure how many people attended your meetups or conference talks.
If you have a more established product or service, maybe the number of active developers would be your goal and so you would measure that. You would also need to define what is an active developer – for example, a developer who invoked an API within 30 days.
You can also measure consumption. You can focus on existing customers and measure your service consumption. Or, you can enable new developers that would bring the most consumption.
For other organizations you could measure how many companies have become customers of your platform (on-boarded) instead of direct number of developers. For example, the goal might be to get 100 new companies on-boarded.
Along the same lines, you can measure % of paying customers using the platform. This shows how many existing customers are paying. This doesn’t measure how much they are paying (revenue). This only shows the percentage of paying vs. non-paying customers.
Yet for another organization, ensuring that all questions are answered on the forum can be a measure of success and repeated questions are added to documentation or tutorials (in order to scale).
These are just ideas and there are many more things you can measure. Only you know what makes sense to measure in order to achieve your company goals and objects. You shouldn’t measure something just because another company measures that.
Now, whatever you measure, you want to be cautions and be aware when something can be easily “hacked”. For example, measuring “developers engaged” can be nice and probably easy but also can be easily hacked according to Amir Shevat. First, developers engaged is hard to define. Second, giving a talk to an audience that didn’t find it interesting or valuable is still “engaging developers” but this engagement doesn’t add value Amir explained at SF Developer Relations Meetup where he talked aboutHow to Measure Developer Relations.
Another such metric is a number of developers registered. This metric can also be easily hacked by running a Facebook campaign or similar where a large number of developers would register.
I think it’s still OK to measure these as long as everyone is aware of the pitfalls.
Helping developers
The last part is as important but probably very difficult to measure, if not impossible. This is the part where we help developers, we enable developers to be successful. I understand the first two components I mention here are a part of this. I guess this is where we win the hearts and minds of developers by helping them, making them successful whenever they are. This is where you build trust.
This can be answering a question on Twitter, LinkedIn or a forum. This can be answering an email from someone you met at a meetup or conference. Or maybe you didn’t even meet this person but they wrote down your email. This can be helping a developer to make their very first API call or sharing whether an approach they are trying is the right solution for their particular problem. This is when you create a blog post or record a video when someone asked to help them understand better. This is when you jump on Hangouts/Zoom/Webex and help someone with a questions, problem or a bug.
IBM Developer Advocate Upkar Lidder was asked a question after he presented at a meetup in Portland and jumped on Hangouts to help the developer. Here is a perfect example of this and goes to the heart of Developer Relations.
and here is how the conversation started on Twitter:
Liquid error: internal
If needed, these interactions can also be tracked in whatever tool your company uses but direct value is a lot more difficult, at least right away.
Conclusion
Measuring success in Developer Relations is not easy but I hope this 3-part framework gives you good starting point and foundation. It should help you build, grow and measure you Developer Relations program and help ensure it’s tied to company’s objectives and goals. Because this is a framework, you can focus more on one part of the framework or another. As your company grows and the program grows, you can shift focus onto another parts.
Top comments (0)