DEV Community: David Ryan

Understanding the Software Stack in Quantum Computing

David Ryan — Wed, 15 May 2024 05:57:31 +0000

This article is inspired by the need for more Software Engineers in quantum computing. Not to mention the other Developers, UX Designers, QA Testers, Product Managers and all the rest of the talent that makes it possible to ship a real product to real customers. Especially products in Deep Tech and Frontier Tech like quantum computing.

These products and the teams attempting to create them are subject to a long and challenging journey from "Science to Technology to Engineering to Product". A series of phase shifts that are more about organisational (and community) evolution than just a linear progression through technology readiness.

This evolution doesn't just happen. For those of us working in these teams, we face the challenge of reinventing the organisation as it shifts from academic to technical to something more broadly engaged with the marketplace. Which means sourcing and collaborating with a growing community of talent and continually evolving and growing our own skills.

This was something I touched on in my "Open Source Your Way into a Quantum Computing Career" talk back in 2022 at the Linux Foundation's Open Source Summit. And it's grown even further in the year or so since, with a noticeable industry shift toward "quantum utility" (a term we use to focus on real-world usefulness rather than theoretical supremacy) and some massive projects kicking off. Such as the Australian government's nearly $1B investment in PsiQuantum setting up a commercial quantum computer back in my home town of Brisbane (said with a little homesickness from here in Seattle).

So yes, there is a lot going on. Which makes this a very good time to understand what these quantum systems are actually composed of and where your talent and curiosity fits in. I've included some recommendations at the end for how to get involved. And I should add a quick disclaimer that there's not really one "quantum computer" paradigm. I've abstracted the most common elements of the various systems we work on for educational value, but welcome any challenges or rebuttals as this model evolves over time.

The quantum stack at a glance

In many ways the quantum computing stack matches the pattern of a modern high-performance computing (HPC) stack. And to a lesser degree will be familiar enough to anyone working in the cloud computing space. We go from a high-level user experience down to some form of platform that takes our workload and converts it to something that will then run on the hardware. Simple enough to get our head around.

The nuance is a lot more complicated. Such as understanding that a quantum computer is only really as good as the quantum algorithm being used. All the fancy stuff you've heard about superposition and entanglement really just comes down to a way to reliably run some useful algorithms that, on a hardware level, uses phase and interference to perform the "computation" that spits out a probability of the right answer. Doing this a lot creates a greater likelihood of the right answer. Doing this at all requires a useful algorithm and a reliable system implementation.

Simulation also plays a crucial role. You might see this talked about as having to do with "saving from expensive hardware purchases", but this isn't really the case (and often a clue someone is just using AI to write their quantum clickbait). We rely on simulation to not only help develop new and interesting algorithms, but to explore the various ways to set up a workload. It's also a core part of the workflows many of us are building towards, where a truly hybrid system would use classical computing resources to handle the workload and scheduling, along with acceleration via GPUs (or newer chips like TPUs and LPUs), and effectively push certain workloads to the quantum processing unit (QPU) where a quantum algorithm may be useful to the task at hand. While some, like myself, are focused on integrating quantum computing with existing infrastructure, others are dedicated to building the most powerful standalone quantum system. Hence the wide range of exploration in the industry.

If all you take away from this is that a quantum computer is a specialised system that includes a QPU in addition to the existing compute stack to run specialised quantum algorithms then that's a win. No cats, slits, or spooky hand-waving required.

The quantum stack in detail

The following sections move from the top user layer down through the platform, and ultimately into the hardware layer. While the boundaries between these layers can be blurred in practice, we'll follow a model based on a typical workload or workflow for clarity (and sanity).

1. Quantum programming languages and developer tools

At the highest level of the quantum system is the human punching away at the keyboard. Quantum programming languages provide the high level of abstraction required for exploring quantum algorithms and creating programs in a manageable form. The experience of working with these languages is expanded by Software Development Kits (SDKs) that offer the libraries and tools required to develop quantum software.

There is some blurring of the distinction between SDKs and frameworks and Integrated Development Environments (IDEs). This is shaped by the diverse approaches of quantum vendors and the integration of platforms and product verticals tailored to specific end-users. A researcher wanting full local access and pulse level control will differ from an enterprise team developing hybrid workloads, which will differ from a fintech startup building on top of a cloud-based quantum platform. This pattern is familiar in enterprise or cloud-based projects, but it will evolve with nuances as the commercial value of quantum systems becomes more apparent and influences product design. Meanwhile, the most prevalent SDKs and their associated programming languages are as follows.

IBM Quantum and (Python-based) Qiskit
Microsoft Quantum Development Kit (QDK) and Q#
Google Cirq and Python
Amazon Braket and Python
Intel Quantum SDK for Python, and C/C++
Rigetti Forest and (Python-based) PyQuil

2. Quantum algorithms and applications

Moving down the stack we come to the algorithms at the heart of any desired quantum workload. As the various competing approaches to creating quantum computers improve, so do the opportunities for real-world applications. A range of software libraries and packages are being built towards specific functional areas of use (such as IBM's Qiskit Machine Learning for quantum machine learning or Google's OpenFermion for quantum chemistry), and the existing libraries of known quantum algorithms are being extended and optimised by researchers and commercial vendors (such as Stephen Jordan's Quantum Algorithm Zoo and Classiq's library).

Some quantum algorithms have achieved near-celebrity status. Others are quantum adaptations of classical algorithms or serve as building blocks for larger workloads. There are even some quantum algorithms that are functionally useless in the real world (insert a joke about physicists here if you dare), but are important examples of quantum advantage. For a deeper dive, see my feature on quantum algorithms, but here are some notable examples.

Shor's Algorithm, known for its potential to break encryption, is proposed to factor large numbers exponentially faster than classical algorithms.
Grover's Search Algorithm offers a starting point for accelerating unstructured data searches.
Deutsch-Jozsa Algorithm isn't technically useful in and of itself, but was an early example of showing quantum advantage over classical methods.
Quantum Fourier Transform (QFT) is the quantum version of the Fast Fourier Transform at the heart of many powerful algorithms.
Variational Quantum Eigensolver (VQE) is a hybrid algorithm being explored for near-term applications in quantum chemistry, materials simulation, and optimization problems.

3. Quantum simulators and emulators

Quantum simulators are the software tools used to replicate the behaviour of quantum systems on classical computers. They form an essential part of our workflow developing algorithms and optimising potential workloads (especially with simulators that feature the same set of gates or other elements of the specific hardware). The role of simulators has evolved as the industry itself evolves from pure academic research to potential commercial utility. The accuracy of the simulation for specific quantum hardware has improved to the point of modelling the system's unique noise and errors. Note that the following examples are subject to change or deprecate as vendors (looking at you Qiskit 1.0) iterate or streamline their product ranges as the industry matures.

4. Quantum cloud platforms

Before we move further down the stack we need to make a small side note on the topic of quantum cloud platforms. In the current era there are a handful of major vendors with a few hardware systems in operation. Each faces the same question of whether to attempt to sell hardware units directly, host on their own campus, sell access via the internet, or some combination of the above. And then add private interconnects, public cloud vendors, sovereign capability and research labs to the mix. It's not a given that the cloud platform model will prove to be the defining economic model for quantum computing, although it occupies the greatest mindshare from outside of the sector given the patterns of cloud computing that came prior.

Having said all of that, pay attention also to the companies who choose not to provision their systems for cloud platform access. At Quantum Brilliance my focus was on highly-parallelised edge-compute clusters using the diamond NV-centre approach that enables small form factor and room-temperature QPUs (with the first deployed at the Pawsey Supercomputing Centre in Australia). In speaking with other quantum startups, the use cases for all forms of fixed or mobile deployment seem to apply, and there's a lot of interesting (and often undisclosed) work being done away from the web. Among those accessible online, here are some to watch.

5. Quantum compilers and circuit optimisation

The role of the quantum compiler is to translate the high-level quantum programs into the low-level instructions to be executed on quantum hardware. While the specifics are beyond this article's scope, the process involves gate decomposition (to match the abstract gates to the physical qubits), mapping and scheduling (to match the logical qubits of the algorithm to the physical qubits) and details specific to the vendor and their particular system (such as fidelity, error rates and connectivity).

To simplify this example stack, we will roll into this level the various forms of quantum circuit optimization that apply techniques to minimize the number of quantum gates, depth, or other resource elements without changing the underlying function. This can occur before compilation, as a part of it during the compilation process, or later on as part of a fine-tuning for the hardware process. For clarity, let's group it here within our workflow. Here's some examples to be aware of.

6. Quantum error correction software

The role of quantum error correction in the current era of "noisy" quantum systems is especially important. To the point that there are companies specialising in this layer of the stack. The need for these companies and the wider error correction effort is due to the fragile nature of quantum systems. While superconducting quantum computers dominate popular imagination, error correction is vital across all methods (trapped ion, photonic, NV-centre, etc.).

Regardless of qubit generation method, challenges arise in preparation, workload execution, and measurement. Decoherence affects all methods, along with gate errors, measurement errors, and individual qubit quality. Quantum error correction is understandably complex, but can incorporate techniques such as system redundancy (spreading the quantum information across multiple qubits), syndrome management (using ancillary qubits to detect errors without disturbing the encoded information), and profiling the performance of individual or clustered qubits over time. While these will still be important if the era of truly fault-tolerant quantum computing is achievable, they are an exciting topic of research in the present Noisy Intermediate-Scale Quantum (NISQ) era we find ourselves in. Major vendors and examples include the following.

7. Quantum control systems

The quantum control system is responsible for managing and controlling quantum hardware operations. It handles tasks at the hardware level such as calibration, pulse shaping, and qubit control. Given the different forms of quantum computing being developed, and the range of implementations of monolithic or networked quantum systems, we can consider the term "quantum control system" to be inclusive rather than specific. Likewise there is no hard and fast definition of a "quantum operating system".

This will likely change over time as the phase of development moves from the "Science to Technology" phases and into "Engineering to Product". Especially where closer alignment with existing product patterns and the terminology of the users and/or marketplace are desirable. For the most part, the control system will be an internal resource (something between an OS and firmware) but the following vendors and products are notable.

8. Quantum hardware

The final layer in our stack is the quantum hardware itself. It's useful to remember that there is no single or right way to generate and work with qubits. Nor is there a clear market leading approach. Each method or approach has its own challenges, and there may prove to be multiple approaches with benefits for specific scenarios.

The hardware layer is what many think of when they hear the term "quantum computing", much in the same way that the early tube and valve computing devices defined the terminology and language by sheer force of mechanical interaction. Switches and punch cards gave way to instructions and programs over time, in turn being extended by memory and storage, and then connected by networks and servers.

These patterns are present in the research and development of quantum devices. As we can see from the above, the layers of an example quantum stack provide opportunity for existing expertise at each layer to provide new and interesting ideas. Hopefully in new ways we might never have considered, given the nuance that each underlying technology and layer in the stack provides.

TL;DR?

The work being done by quantum physicists, electrical and electronic engineers, fabricators and all kinds of manipulators of atoms is not only supported but enabled by those of us who wrangle the bits and (zetta)bytes. It's a great time to get involved.

A good start for new learners is Q-CTRL's Black Opal, Delft university's high level MOOC on edX, and the inevitable path into IBM's Qiskit. For those of you from the C# side of things, this Microsoft introduction has a quick look at their approach with Q# and how it's rolled into the Azure ecosystem. And the Classiq algorithm library is worth bookmarking.

I'll do a dedicated article on learning quantum for software engineers soon, as this is a topic that I get emailed about nearly weekly since my Open Source Summit talk (and the rolling layoffs of engineering talent that has occurred since). While any industry shift is hard for the people and families affected by the colder edge of the economics, it can in turn be an opportunity to unlock the talent (and the career paths) that would otherwise be locked away. So if you're considering further exploration, please do so, and drop me a line if there's anything I can do to help. And be sure to sign up to the Product In Deep newsletter where we dig into the strategy and craft that goes into shipping real products in the Deep Tech and Frontier Technology realms.

Hey, Product Managers! We need to share our career journeys too.

David Ryan — Sun, 18 Jun 2023 06:54:55 +0000

The official line is that "product management is an exciting and ever-evolving field that attracts professionals from diverse backgrounds". But the truth is that every single meme about nobody being able to say what a Product Manager does is... sometimes too real.

What makes the role even harder is that those on the outside struggle to see a way in, and those on the inside often feel alone, or like impostors. We've all been there. One of the most valuable resources at our disposal is often overlooked, and that is, the collective wisdom and experiences of product managers at every level.

Sharing our career journeys, regardless of our level of expertise, can have a profound impact on our growth and development as product managers. So let's explore the importance of product managers sharing their career journeys and how it can benefit individuals at all stages of their professional lives.

Inspiring Newcomers

When we're just starting out in product management, it can be intimidating to navigate the intricacies of the field. Hearing the stories of experienced product managers can provide invaluable inspiration and guidance. Not just the glory stories, but the normal lives and thoughts and routes to the role. By sharing our career journey, we can help each other understand the challenges we may face and the skills we need to develop. These stories can serve as a beacon of hope, encouraging us to persevere through the initial hurdles and embrace the learning opportunities along the way.

Learning from Mistakes

Product management is not without its share of setbacks and failures. They can hurt. A lot. But mistakes can be powerful teaching moments that contribute to personal and professional growth. By openly discussing our own missteps and the lessons learned, we create a safe space for others to learn from our experiences. This transparency fosters a culture of continuous improvement and encourages others to share their own failures, promoting a supportive and collaborative community of product managers. TLDR? Let's do more of that.

Expanding Networks

Networking is a crucial aspect of any profession, and product management is no exception. A little transparency about our unique career journey goes a long way. And by doing so we can connect with other professionals who can offer insights, advice, and potential collaboration opportunities. As we share your experiences, we open doors for meaningful connections and expand our network within the product management community. These connections can lead to mentorship opportunities, job referrals, and collaborations that enhance our professional growth.

Developing Self-Awareness

Reflecting on our own career journey and articulating it to others requires a deep level of self-awareness. By sharing our experiences, we gain a clearer understanding of our own strengths, weaknesses, and areas for improvement. This self-reflection helps us identify patterns and strategies that have led to success or failure in our careers. Not to mention that receiving feedback and engaging in conversations with others who have faced similar challenges can provide fresh perspectives and insights, further enhancing our self-awareness and personal growth.

Encouraging Diversity and Inclusion

The product management field, like many others, benefits greatly from diverse perspectives and experiences. By sharing our career journeys, we contribute to building a more inclusive and diverse community of product managers. And the wider technology ecosystem around us. Hearing stories from professionals with diverse backgrounds, cultures, and perspectives broadens the collective knowledge and promotes a more comprehensive understanding of product management. It encourages aspiring product managers from underrepresented groups to pursue their dreams, knowing that others have paved the way before them.

Fostering a Culture of Knowledge Sharing

Product management requires a continuous learning mindset. Sharing our career journey instils a sense of responsibility for the collective growth of the community. Which might sound a little dramatic. But consider when you openly share your successes, failures, and insights, you inspire others to do the same. This culture of knowledge-sharing elevates the entire product management profession and helps everyone stay informed about emerging trends, best practices, and innovative approaches.

This isn't a comprehensive list but covers the major levers we have at our professional and personal disposal. Put simply, sharing our product management journey is important whatever our level of expertise. By inspiring newcomers, learning from mistakes, expanding networks, developing self-awareness, encouraging diversity and inclusion, and fostering a culture of knowledge sharing, we contribute to our own growth and the growth of the product management community as a whole. And just maybe have a greater sense of purpose while doing it.

This is what happened to open source in 2018 (and beyond)

David Ryan — Thu, 07 Mar 2019 14:26:52 +0000

Looking at the recent events in open source give us some interesting clues for the future. Especially as 2018 was one of the most important years for open source since its foundation.

More than a few of the peak moments are new shoots that suggest open source will grow in interesting and unpredictable ways. While also extending (and replenishing) its roots as a stable form of both enterprise and community innovation.

These are also the topics that pop up over and over again. At the start of 2019 I wrote that I was going to begin researching the future of open source. In the interviews conducted in the two months since I keep hearing the same themes.

Everyone seems deeply concerned with at least one of these major streams. So I thought it would be worthwhile to dig into the top ten themes that keep emerging — and ask what the major events of open source in 2018 might mean for the year (and years) ahead. Let’s go!

Open source companies acquired in record amounts

Salesforce announced in March of 2018 that it was acquiring Mulesoft for $6.5B. Then it was Adobe’s turn to announce it was acquiring Magento for $1.68B. And then SUSE got sold off to EQT for $2.5B. That’s quite a few billions right there.

If that wasn’t enough to get the mechanical keyboards clacking on Hacker News, along came the IBM to announce a $34B acquisition of Red Hat. In and around those larger numbers are acquisitions and acquihires of comparatively smaller open source companies. The news of CoreOS’s acquisition in early 2018 already feels like a millennia ago.

All of this activity creates a positive sentiment for open source — if not outright pricing the market with a premium in the process. Much like Red Hat’s record IPO kicked off way back in 1999, a rising tide floats all boats. Although we might just ignore the years 2000 to 2002 that followed.

Significant IPOs make significantly more capital available

Speaking of IPOs, there were some notable activity there too. Red Hat has held the record of largest open source IPO for nearly 20 years — with a not insignificant $3.4B. Right up until Pivotal’s IPO in 2018 set a new high of $3.9B.

What’s interesting here is that this is a whole other era. Red Hat’s cofounder Bob Young once explained to me the hysteria around their IPO back in the day. Sure it was a great company with solid numbers — but it was the first open source brand to raise venture capital and IPO. They were a symbol of an entire new vertical opening up. It’s not hard to imagine that mix of positive sentiment from within the open source world mixing with the sense of discovering new frontiers (and greed) from without. The market responded accordingly.

More capital is ostensibly a good thing. Free software being written in free time is great. But having a steady wage already aligned with the open source mission is almost immeasurably important to the overall ecosystem.

Likewise increased capital means greater ability to acquire necessary technology. While Linux gets the historic mindshare for open source, it’s impossible to imagine the open source landscape of the last decade without without middleware, virtualisation, or storage. Some of these IP investments might not quite be the revolution as intended (hello Big Data) but others such as containerisation are rewriting the rulebooks in realtime.

Github acquired by Microsoft

The acquisition of open source companies in 2018 was a big deal. And so was the acquisition of the platform that open source communities are increasingly relying on. We saw three expected things happen when Microsoft announced a $7.5B acquisition of GitHub.

The first was the immediate knee jerk from parts of the community that were concerned that evil Microsoft was getting its hands on the home of so much of open source. This is a valid concern but it didn’t really happen.

This is probably for reasons that have a lot to do with Microsoft bought them in the first place — GitHub isn’t just a repository but a workflow and a community. As we say in the SaaS ecosystem it’s a very sticky product. And with all those developers stuck fast, Microsoft swooped in to announce its first nefarious move after the acquisition was… to make private repositories free.

Next came the questions and hot takes about what Microsoft owning GitHub would actually mean. Some great insights but something that will pan out over time.

And finally came the investment announcements from GitLab (following the opportunity of an acquisition pricing and energising the competitive market). Which is interesting because GitLab isn’t just a competing code repository but a product aimed at the entire DevOps lifecycle. And open source. Watch this space.

The search for sustainable business models gains momentum

Aside from GitLab’s raise we see other notable venture moves. Fresh from raising $15M in May for its open source management subscription service, Tidelift announced another $25M.

This is some of the few bits of good news in the topic of sustainability in open source. An issue that Nadia Eghbal called “the internet’s biggest blind spot” back in 2016. Nadia talks articulately on the topic, and her “lemonade stand” is a handy guide to financial support for open source.

Nadia’s work stands out due to her experience across multiple points of view, ranging from VC to a part of GitHub for a time. My own view back at the SustainOSS summit, is summed up by my project submission in the report. Others like Tidelift are taking an enterprise approach to community resourcing.

Which isn’t as common as one might think. Xavier Damman of Open Collective wrote a great piece explaining the personal and brand incentives for supporting a project. And of course there’s no shortage of well-meaning tip jar platforms, patronage and bug bounties. All better than nothing.

But the example of Tidelift is one that echoes the rise of open source in the enterprise itself. If it wasn’t for Red Hat’s ability to elevate Linux into a trusted enterprise relationship… would the industry be where it is today? Is Red Hat’s example of $3B revenue from the enterprise realm a better goal than rattling three billion tip jars?

Whatever the ultimate mix of methods that maintains the open source world, there’s little question that the venture capital product line benefits from it (but doesn’t support it), and that the answer will likely be to actively engage in the market versus seeking handouts from it. The search continues.

Linus steps back from Linux

Open source is not always a very nice place to work. Mailing lists and even conferences can be frustrating and exhausting things to encounter. Dedicated contributors even from within the culture can be burned out and turned off — let alone curious or casual explorers. Especially those from outside of the genesis cultural norms.

And a large part of this comes from the first wave of open source and free software identities that forged its image in an era (and as an act) of counter-culture. Such an emerging culture requires the forging of an identity. And such a culture built around a dogmatic battle for righteousness usually does so and against a clear villain.

And yet here we are in an era where the war is over, open source has won, and it’s just maybe a little less okay to be a jerk to one another.

Something that Linus Torvalds admitted when in September of 2018 he announced he was going to “take time off and get some assistance on how to

understand people’s emotions and respond appropriately”.

As one might imagine there were feelings about this. But the world didn’t stop and ultimately…

Linux explores a new code of conduct

In the aftermath of the announcement from Torvalds emerged a new Code of Conduct. The reactions were somewhere in the range of behaviour expected. We live in interesting political and social times and the conversation reflects those extremes.

It’s a topic that deserves more careful nuance than a summary list on a blog post. But one inseparable from prevailing politics, and perhaps political fatigue, in a manner that will be influencing and affecting the open source culture for some time to come.

Redis attempting a more convenient “open source”

At face value the issue seems simple enough. Cloud vendors being accused of profiting from open source projects as they fill out their product offerings and battle for dominance. Amazon in particular.

Retold with less dramatic emphasis, it is understandable if companies based on an open source development model get annoyed when the likes of Amazon use their codebase without either paying or contributing. Especially when the increasing monopoly in the cloud space is putting pressure on those companies running the upstream communities.

Under that lens it would make sense to make changes to the open source license to stop the freeloading. Which is what Redis did in 2018. Kind of. The move wasn’t particularly popular among open source supporters. Not at all.

Another perspective offered was that this feeling has always existed with open source. Others point out that licenses applied to specifically block out a cloud vendor is just one commercial company using an open source community against another. And that the problem isn’t AWS itself — it’s open source revenue models.

Especially considering that Redis just closed a $60M Series E funding round — with the likes of Goldman Sachs Private Capital Investing and Bain Capital Ventures participating. Not organisations necessarily known for passionate open source advocacy.

All of these perspectives have an interesting point of view. The nuance of this issue makes for a fascinating case study of the open source ecosystem itself. Such as…

Open source doubling down on being open source

The interesting thing about companies like Redis adjusting their position on open source licensing isn’t just the ongoing tide of conversation around it. It’s the serious work of self-reflection at the core of open source itself.

In this case the Affirmation of the Open Source Definition released by the Open Source Initiative. It’s a fascinating read. If you’ve ever wondered what the point of difference between free software and open source is, there’s your primer. Specifically on the topics of “trust” and “commerce”.

The interesting ripples in this issue are a growing awareness that open source is evolving. There will come a time where the current licenses are updated to match. But as the OSI indicates, the very existence of open source as a commercially successful enabler of collaborative creation relies on trust and the ability to satisfy personal goals within the greater scope of the whole.

It’s not that cloud vendors are freeloaders (although they’re arguably contributing more than anyone else right now). It’s just that freeloading isn’t a bug of open source — it’s actually a feature. Although one spread unevenly.

Microsoft open sources its patents (sort of)

The headlines about Microsoft open sourcing 60,000 of its patents would be pretty confusing for someone waking up from a coma. Microsoft giving it all to open source? This Microsoft being run by this guy?

Thankfully things have changed at Microsoft (and not just the share price). But not so much that they’ve given away patents. The nuance takes a little more digging to understand exactly what happened.

The implications of Microsoft joining the Open Invention Network means that Microsoft is making a public (and legal) agreement “not to use any of its 60,000 patents against other OIN participants for their use or distribution of ‘Linux System’ technologies”.

The interesting thing about Microsoft is that it actually is “listening to customers and developers and is firmly committed to Linux and other open source programs”. That’s Corporate Vice President and Chief IP Counsel Erich Andersen as quoted from the press releasing announcing the news.

And that’s a Microsoft embracing open source as it chases Amazon’s cloud leadership, refocusing after decades of singing and dancing outside of the actual major innovation shifts, and even convincingly getting behind startups. Not in the “oh wow free credits” way most programs run — but in a focus on co-selling.

Microsoft might have been the ideal villain for the open source community to find its strength and voice fighting against. But now it looks like they’re taking an impressive leadership role.

Open source no longer controlled by the open source community

Software ate the world. And open source is the backbone is the entire software development experience. The definition of open source might have only recently celebrated its 20th birthday, but it’s long since grown up and moved out of home.

In a future post I will dive into this topic further. And acknowledge the greatest difficulty here is knowing that the need for wider discussion about the future of open source is something that will antagonise some of the open source community itself. Especially now that open source is for… everyone.

Books such as Jim Whitehurst’s wonderful The Open Organisation showcases how open source’s origins as a community model can be strategically applied as an organisational framework. Then there’s CapitalOne’s open source culture as defined by policies and programs. Or Data61, the digital arm of Australia’s CSIRO, releasing guidance on open banking. Still in Australia we see the open government partnership releasing open data, which was valued in 2014 at around AUD $24B.

These may seem like efforts drifting away from the core definition and origin or open source. But all are orbiting the same point of gravity in terms of structuring collaboration and contribution at scale. And all serve as examples of the applicability of the lessons of open source going far beyond the origins of the culture.

So what happens next?

That’s an excellent question. What do you think?

I’m currently researching the future of open source and would love to hear your thoughts or comments… 🙌✏️💻🧨

Corilla hosts your developer documentation (for free and forever)

David Ryan — Tue, 03 Apr 2018 04:22:31 +0000

Hosting your developer documentation just got easier. Today we’re announcing that all Corilla users have access to forever free hosting of unlimited documentation. And that includes the free tiers too 🙌

Let’s get back to solving user problems

Why we’re making this change is surprisingly simple. It’s needed. Like many developers and technical writers I spent years living under the delusion of “just spinning it up myself”. This was originally a necessity — we’re not shy in voicing our frustration that the major technical writing products on the market are stagnant to the point of irrelevance.

I shudder to think of how many project hours my teams in the past have burned setting up (let alone maintaining) publishing workflows. We justify this of course because we like to tinker — especially those of us with design and developer backgrounds. But the hard questions that need to be asked are:

Is spending time building documentation and publishing tools really helping our users solve their problems?

Could we be speaking to users instead? Getting feedback from the support teams? Working through the user journeys with the UX team?

The answer is often uncomfortable but simple. It’s time to get back to writing.

Escape the hidden costs of in-house

The legacy of the in-house era was shattered for me when I started to think about the cost and complexity of internal productions. The rise of the SaaS model brought a certain painful reality: if we’re spending anything more than one hour a month on building and maintaining our documentation toolchain, we’re actually costing the company more money than using a dedicated SaaS product.

And one of the best things about using a SaaS product is that it keeps on getting better while you keep paying the same price. Something we make easy to track here at Corilla — especially with our transparent public roadmap and release notes.

Extending our free hosting policy to companies and projects on the free tier makes sense — especially after moving our knowledge base into open beta and a full pricing update. You get the world’s best collaborative content management tool under active development and we get the chance to show you enough value that you will grow your company with us through the various pricing tiers.

Get started hosting on Corilla

Are you a SaaS company looking to publish your user documentation? A small team wanting an internal knowledge base built around collaboration. We’ve got you.

A solo project needing robust documentation? A startup needing to get a head-start on managing all that content and information with pre-configured templates? A team needing help doing something with that legacy collection of content? Yeah we’ve got you too.

So here we go. Get started on Corilla today.

Breaking into Product Management with Open Source

David Ryan — Sat, 21 Oct 2017 07:22:42 +0000

Photo by Finn Hackshaw on Unsplash

“How did you break into product management?”

That’s a question that a product manager quickly gets used to being asked. As the role has evolved from a marketing function to one of delivery, so has the appeal.

If the in-person interest is on the rise, then the digital world is ablaze. There are an overwhelming range of Slack channels, Facebook groups, LinkedIn Groups and training courses to jump into, engage with, and learn from. Product is 🔥. Managing it even hotter.

What is surprising given this trend is how few people are taking advantage of what is possibly the most under-utilised path to practical product management experience.

Open source.

Start your product career in open source

I‘ve written elsewhere about the power of open source as a way to break into careers like technical writing. [Documentation is regularly described](http://yellowpencils.co/] as the single greatest unsolved issue of modern developer culture — and something we’re on a mission to solve at Corilla.

But product management? How about that?

Open source is more than code

I know what you’re thinking. Isn’t open source just for developers? Isn’t it just a kind of code on GitHub? Something to do with a penguin and angry people with beards?

Not anymore.

What began as a movement encouraging the open sharing of source code has evolved into a powerful product model with a beautifully diverse community. Here’s the cheat sheet for what changed in the favour of product teams:

Open source won and is now realising it probably needs to make money instead of begging for it.
Companies like Red Hat proved that open source projects can be viable open source products.
The incredible advances in consumer technology have become a powerful motivation for enterprise design innovation.
The open source community refined a range of distributed collaborative models that have turn out to be useful for all kinds of things.

What does this mean to product managers?

Startups and enterprises have realised that the open source way gives them a market advantage. Even something as simple as Twitter releasing Bootstrap showcases the impact a project can have on an ecosystem.

Back on the product side we see Red Hat sailing through the two billion dollar revenue mark, boosted by a focus on cloud services beyond the origins of “that Linux company”. But for every Red Hat, other great technical teams like RethinkDB have crashed and burned. The sector is valid — but only for those with a valid product-market fit.

Now at the same time that open source is coming out of the basement to embrace product culture or die — oh no — the average consumer device suddenly becomes a magical everything machine. And it looks amazing! Dammit. Who could help close the gap between an era of developer-centric software and the expectations of customers spoiled by the runaway success of Apple’s design-first aesthetic?

It… it sounds a lot like an opportunity for product leadership, doesn’t it? But what’s really in it for you?

Open source as opportunity

The immediate benefit of open source is being able to gain experience in something by actively doing it. And usually with the encouragement and support of a team that itself benefits from any level of meaningful contribution you can muster.

The long tail benefit of open source is probably the most significant. Everything you contribute to an open source project becomes a powerful public record of your development as a product manager.

Evidence of your ability to work with teams, willingness to take the initiative and general development within the role is more what will land you that breakthrough role. Far more than any vague reference to a previous corporate job or training course (aka “your resume is less important than products shipped”).

Open source as community

It’s also worth repeating that open source is also as much about the community as the technology. Most of which are always in need of additional support across all possible functions. It’s not just software development that turns an enthusiastic project into an actual product, but roles like marketing, documentation, design, UX and… product management.

Two open source paths into product management

There are a number of avenues for those starting out in product management to engage with the open source community. Here’s two that sit on either side of the ecosystem.

The enterprise route

At one end of the scale are the larger and enterprise-backed projects and products. The advantage here is that the community is relatively well structured and following some form of process. They are resourced by talented contributors and employees and contain some form of onboarding process.

The advantage of getting involved with open source enterprise projects or products is that you will have the opportunity to learn a defined process. Ideally with the mentoring of experienced product managers who work for well-known companies. Keep in mind that many open source companies tend to hire directly from the communities that gather around their ecosystem.

A contrarian takeaway from this route is that you will also observe the inefficiencies of scale and the challenges of upstream and downstream product alignment. Like any form of leadership the value of a manager comes from not just being able to prescriptively follow a process, but being able to get 💩 done when 💩 goes down.

The startup route

If you think an open source enterprise project needs you, wait until you meet an open source startups. Hi 👋.

The advantage of engaging at this level is that you will have more direct involvement and more rapid learning cycles with often intensely talented teams.

Like any startup you will be challenged with a general lack of process. This is perfect for an ambitious product manager to showcase their direct influence on the success of any given initiative. The direct challenges will give real-world application (or inspiration) for applying any other learning avenues you’re undertaking. Startups are an engine room of just-in-time learning.

If enterprise is where you learn and network with a consistent pace, the startup is where you prove your ability to move the dial in a meaningful way. And if you’re anything like me, you will already be thinking…

My spirit guide.

Finding open source projects as a product manager

This is where things are easier than you probably expect. If you’re reading articles like this then you already know who cares enough about the topic to dedicate time to it. Reach out in the comments or find the authors on Twitter or directly by email.

Speaking of Twitter — use that too.

David Ryan

@davedri

What are some #opensource projects that a product manager could contribute to?

00:11 AM - 18 Oct 2017

33

In terms of discovery mechanisms there are a few options. AngelList has a directory of open source companies and a search for “open source” on CrunchBase returns over 6000 results.

Going to the source (pardon the pun) on GitHub can be useful. Recent updates to GitHub Explore have been excellent and a major stop forward for what has moved beyond a code repository to an important community destination in itself.

Another area for strategic discovery is to reach out to developer advocates. The entire devrel ecosystem is built atop technically savvy people with a passion for community engagement.

And finally, there’s probably few communities more versed in open source and the opportunities within it than opensource.com team. Tell them I sent you.

You got this far (so keep going)

If there’s any single point to repeat it’s that open source needs exactly this kind of motivation and product focus to survive and thrive. If you choose this little known path, you come in at what is likely a tipping point for the industry. And that’s both as a wonderful way to advance your product management career as well as a chance for you to bring your ideas and perspectives into the ecosystem realising it can’t just keep begging for support — it needs to earn it with market-defining products.

My own journey isn’t far from this experience—the story of Corilla is well documented these days but is built atop this exact path. A bunch of technical writers building their own content product internally. And then spinning out to build it properly this time around. What could go wrong, right?

If there’s anything I can do to help nudge you towards your own opportunities in similar context please let me know. Whatever you are drawn towards just please get started. If this is a mission you’re on — make it happen. The sooner you do the sooner you get to make something amazing for the world.

Thanks for reading. This post originally appeared on Minimum Viable Paragraph.

The limits of the MVP (and the importance of kung-fu)

David Ryan — Thu, 05 Oct 2017 21:34:46 +0000

Source: Patrick Perkins via Unsplash

Have you seen this kind of conversation before? Chances are you’re going to nod with a certain pained familiarity.

“I recently worked on a project where the term MVP was constantly bandied around by both the client and some of the team… when what we were delivering was nothing of the sort. Might this article be true?”

The article in question was Rik Higham’s interesting piece “The MVP is dead. Long live the RAT”, and the quote above from a comment on the the Sydney Startups group on Facebook.

Specifically the linked article argues some interesting points that the term “MVP is used so much it’s lost its original meaning.” It’s easy to have empathy for Higham’s frustration here, as the lean startup model and concept of a minimum viable product are ubiquitous. And with ubiquity comes increasing interpretation and a growing cultural footprint.

To Higham’s credit he creates another lens through which to approach product and experience design. His RAT model is one that I’ve enjoyed discussing in my own communities and has some great ideas.

But here’s a secret.

The models don’t matter as much as two other specific things you learn as the typical product management career moves through the junior phases of prescriptive application and into more complex and nuanced releases.

Models matter less than an understanding of the history and cultural signals surrounding them.
Models are not immutable and are affected by the cultural signals of the intended organisation applying them.

The problem with naming something

The challenge here is semantic. If not entirely semiotic. But don’t throw out those post-it notes yet.

Terms like “MVP” are a concept with a practical example, but quickly gain other cultural signals. And within those signals they are weighed down by assumptions, by given examples, and by association with whoever is transmitting that knowledge.

We see this in the opening quote.

…the term MVP was constantly bandied around by both the client and some of the team… when what we were delivering was nothing of the sort.

The perceived misuse of a term has led to a total rejection of what was considered in recent years to be a revolutionary shift in product development methodology.

It’s no wonder that smart and experienced product managers (in this case Higham) are reinterpreting the prevailing model through their own lens, while also distancing their theories from the rising noise around the previous model.

I’ve done this myself on a project in 2016, hijacking elements of Startup Weeken, Google’s Design Sprint, and IDEO’s Design Thinking. I called it “Immutable Design Theory”, and used that lens to both lead an innovation project and affect cultural change within an organisation. Experience designers are crafty like that.

Why? Because this is what humans do with literally everything in our human experience.

Understanding this comes naturally through a product manager’s career, where repeat application of one and then another generation of product model as a prescriptive mission starts to highlight that — okay — there’s more to it than a step-by-step recipe here.

The same goes for the sub-sequent journey through other models until an eventual product nirvana is reached. Which brings us to…

Product management as kung-fu

“What do you mean you haven’t update the roadmap for this week’s standup?”

One of the most famous cultural icons in martial arts is none other than Bruce Lee. Originally trained with Yip Man in the Wing Chun style, Lee is perhaps most widely known for his move towards a style of no style at all. Something he called Jeet Kune Do.

You can see where this analogy is going already. If Wing Chun was the equivalent of the MVP model, Lee is the product manager growing frustrated at the limitations of the style.

His journey through subsequent styles to form what Lee called “the art of fighting without fighting” is legendary. But perhaps is easy to overlook something — the fact that Lee’s dedication to the exploration of Kung-Fu’s application (and therefore limitations) was one of discipline and mastery.

“Be like water my (product managing) friend”

We can learn a lot from looking at product management through the lenses of semiotics and kung fu. As ridiculous as that is to both read and to type.

In either case the lesson is to gain a deep understanding of not only the style or model in question, but how it is viewed in the communities and ecosystems around it. The ability of that style will depend greatly on the context of its application.

As Lee says, “be like water my friend”. Nothing is static. Not even product management and development styles.

Thanks for reading. This post originally appeared on Minimum Viable Paragraph.

On building your project (and letting people enjoy things)

David Ryan — Thu, 31 Aug 2017 22:43:45 +0000

You know that thing you want to build?

But there’s already a thing doing something like that? And someone is bound to get up in your face and tell you all about it?

If that sounds familiar then here’s two quick tweets and a hot take for you.

Recently Pieter Levels made something called Hoodmaps. You might have seen it on Product Hunt. It’s a fun project that lets people overlay Google Maps with other information. In this case, funny references to the neighbourhood demographics of major cities.

Here’s the map for San Francisco. And yeah it’s pretty accurate. Although I don’t agree with the annotations for Mountain View, where I’m currently writing this, for obvious reasons 😂.

Tell it like it is — multi-coloured map.

It’s pretty cool huh? Yeah I think so too. But because this is the internet let’s brace ourselves for the inevitable polar response in 3, 2, 1…

// Detect dark theme var iframe = document.getElementById('tweet-903311742373122049-286'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=903311742373122049&theme=dark" }

I don’t want to unfairly ping Matthew here because it’s actually pretty cool that he cared to reach out and engage with Pieter on this.

Like any founder building any product at all I find myself on Pieter’s side of the conversation at least a few times a week. The minute you ship anything at all you’ve got no end of people telling you it already exists in the world. Regardless of your domain expertise or particular vision.

But like any member of the global startup ecosystem (which probably includes you if you’re reading this) I’ve also had to learn how to channel my pattern recognition for similar projects/products into questions that founders can answer. It’s human nature after all and really healthy to acknowledge that we all do this instinctively.

The awesome thing is that giving it five minutes and letting the founder explain their vision can often be enlightening — another reminder not to scream “Simpsons did it” every time something looks familiar.

// Detect dark theme var iframe = document.getElementById('tweet-903353533914906632-876'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=903353533914906632&theme=dark" }

Pieter doesn’t seemed bothered here and is having some fun in reply. Let’s remember that the internet can be a fun place if we don’t take things too personally (AKA put down your pitchfork for a minute 🔥🔫).

This simple exchange highlights a few points that I’d like to mark up in highlighter and draw a few circles around. In no particular order…

Build any damn passion project you want to

The world is an amazing place full of amazing people doing amazing things because they want to. Be one of those people (they tend to be successful).

When it comes to passion projects there’s really only one person that matters. And that’s you.

Pieter’s a bit of an outlier in the passion project category as he’s certainly been prolific over the last few years. Which is why it’s even more odd to call him out in particular on something both fun and personal. And as others point out, has already gained a certain amount of cultural traction.

Passion projects are immutably awesome. The more you do the more you learn. And the more you share the more the world levels up with you. Imagine if someone would have told Linus to not bother creating Linux as “just a hobby”? After all, Minix already existed, right?

Learn to ask for clarification like a real human

Pretty much everything invented has a bunch of other things just like it. Just like birds. Have you seen how many breeds of bird there are? Are there any ornithologists snarking “kookaburras are interesting and all, but eagles have been doing a better version of flight for almost two millennia”?

No. Because that would be rididculous.

My tip is that if you encounter a founder working on a project or product that seems similar to something else you’ve seen, you’ve got a chance to learn some nuance about that space. Try this…

“I’m not an expert in that field, but I’m familiar with [NAME_OF_THING]. What’s your take on this?”

Let people enjoy things

As someone who cofounded a documentation tool for software teams I can certainly relate. I have to declare my obvious bias here.

And I can ensure you that it’s okay to ask these questions. If you’re interested I’ll tell you all about our unique focus at Corilla, the backstory of our experience as an internal startup at Red Hat and even how much I (genuinely) love the handful of similar companies exploring this space as well. That’s a blog post for another time.

But for now it really just comes down to some sage advice that Adam Ellis so perfectly summed up in pixel form…

Extract from Adam Ellis.

Thanks for reading. This post originally appeared on Minimum Viable Paragraph.