The reality of being a software developer today makes integrating data and services a primary concern. If you’ve just completed a coding or computer science course, when you start working on a real-world project, you’ll likely find yourself dealing with APIs. Unfortunately, in education contexts, we sometimes create a false impression of what being a developer involves on a daily basis - APIs are a prime example of this. In this post, we’ll touch on some of the issues you may face when you start to work on real applications, and provide tips for navigating the API ecosystem.
API (disambiguation)
First, a clarification: the term API is used to refer to lots of different things, and its emphasis has shifted a great deal over the years. If you’ve learned anything about Object Oriented development, you may be familiar with APIs as code components that have well-defined interfaces, allowing “customer” code to access the features and functionalities within a system. These APIs usually involve objects and methods / functions. This is not the type of API we’re going to talk about in this article. 😅
In this article, we’re dealing with the kind of API you use to manage access to data and services over the web. If you’ve come across the terms HTTP API or REST API, that’s what we’re talking about. With this type of API, client apps make requests to endpoints to carry out operations on data. An example would be a database for a shop that has a web app and a mobile app - with an API managing access to the data from both apps.
Even more confusingly, you’ll also see the expression “Web API,” used to refer to both types of API. At client side, Web APIs normally refer to browser technology that you can interact with using JavaScript (e.g. the DOM). At server side, a Web API would be the type we’re talking about here. 😭
APIs can support a range of operations, to retrieve, insert, update, and delete data (and more generally to provide services). In most cases, API requests need to be authenticated, with various authentication models to choose from. APIs sometimes provide support resources to aid integration, such as documentation, specifications, and definitions (each of which satisfies slightly different criteria 😩).
Consumer and publisher
The chances are, if you work as a software developer, you’ll consume APIs, and depending on the kind of development you do, you might also build and publish APIs. By utilizing data and services from external sources, development teams can create complex products more quickly and efficiently. If someone has made it their business to be really good at messaging for example, it makes sense to plug into their service (which will hopefully continually improve) and focus more of your own efforts on the unique parts of the system you’re working on.
You’ll also potentially work with microservices, which are related to APIs - but where microservices are about dividing up the services in an application, APIs are more about how you access the services as a developer. Okay, let’s leave this semantic rabbit hole and move on to more practical concerns. 🎉
The problem
In education, we usually have pretty rigid constraints, like learning outcomes, that shape the student experience. This lets us hit on the desired objectives and help learners acquire a specific set of tech skills. The downside is that it can make learning experiences quite unlike the reality of acquiring and applying new coding skills in the wild.
This isn’t just a problem for new coders coming from bootcamps and CS courses, it’s a general issue in education for developer software tools, however we’re slowly getting better at creating more realistic learning experiences:
Why this is important
Integrating services is one of the defining characteristics of developing software today - it’s not just an activity, it can determine the entire approach to a product. The field of Developer Relations (DevRel) has emerged and grown enormously over the past few years for this reason.
Where in the past a typical software developer might have built apps for end users who were not developers themselves, there’s a strong chance now, that at least some of your users will be other developers looking to integrate whatever projects they’re working on with your own - often via APIs, but also via SDKs, extensions, open source initiatives, and so on.
All of this means that navigating a network of dependencies and connections is going to be a vital part of the job for most developers (not to mention other members of product and engineering teams, such as QA engineers, and even PMs to some extent).
OK, so what is the reality?
So, what is working with APIs going to involve, that you might not have encountered before working in the software industry? Please feel free to share your own horror stories too… 😉
Lots of integration
At a granular level, you might find yourself building an API or consuming one - but in fact, if you work on any kind of substantial software system you’ll probably be integrating with multiple external services. Making the pieces all play nicely with one another can be quite a task - and a frustrating one, since you only really have immediate control (or even influence) over the parts of your implementation that are being developed within your team.
In theory all of this integration should be powered by automation, with the data completely interoperable between services, but sadly this is not often the case.
Reality is messy
As soon as you start working with external services, whether from a developer tool exposed by another organization, or even just the data brought by a client you might be building an application for, you’ll find the situation tends to get messy. (If you work for a larger organization this might even happen with internal APIs. 🙊)
As developers, we love automating things, especially repetitive tasks, but when you get stuck into real data from business contexts, you’ll often find yourself wrestling with an unpredictable muddle of information that makes automation extremely challenging.
If there is documentation available - and often there isn’t - you’ll frequently find that it’s incomplete or inaccurate.
This was a particular issue for an ex-employer of mine, where we used the OpenAPI specification to automate integration with client data, but more often than not, there were discrepancies between what the spec indicated and how the API requests actually behaved. From the perspective of software that consumed rather than exposed APIs, it was fascinating to see the rapid growth of DevRel fueled in part by the need to address these problems in developer engagement and adoption.
When it comes to working with clients, add to these challenges the fact that you might not even be dealing with the person who built the API in the first place - it could be someone who has inherited it, or is integrating it from a third party source, and their understanding / control over it may be minimal.
Sidebar: this is a helpful analogy
It’s worth pointing out here that this messy data issue is analogous to another missing ingredient from many coding learning experiences - the reality of integrating, not with data, but with people. Many of the greatest challenges you’ll face working as a developer will relate to how you engage with your team, community, clients, partners, and so on. Operating in a business context with competing / shifting priorities means delivering your work in an ever-changing, often very dynamic environment.
Just like we pretend not to google everything during engineering job interviews 🙄, in most developer education resources, we behave as though we're operating in a vacuum. The truth is that you’ll need to integrate your tasks within a system that has all sorts of parameters outside your control.
Tips and tools
The good news is that devs successfully deal with this stuff all the time - we wouldn’t have many of the current apps we use on a daily basis otherwise. With any luck you’ll be on a team with other people who have more experience wading through integration difficulties. Watching how a senior developer deals with dependency hell is invaluable if you get the opportunity.
However, as a developer you will inevitably spend lots of time working on your own trying to troubleshoot problems - so here are some tips and tools for defeating any API development and integration demons that trip you up.
Trust no one
Only joking, but a bit of healthy skepticism is in order when dealing with APIs. You can’t place too much faith in documentation (even the best teams struggle to keep these resources 100% up to date). When your app depends on data from another source, testing becomes increasingly important.
In some cases you might even be working with a spec or definition that has been automatically output from a backend system, in which case you might think it’s likely to match the actual behavior of the API - sadly not always. That unexpected null value coming back might derail your application.
Thankfully there are some great dev tools that can help with various parts of the API development, testing, and publishing process:
- Postman, most associated with testing but supports various workflows for different parts of the API development lifecycle, including collaboration and documentation.
- The Swagger toolset which includes multiple utilities, such as the ability to start from a spec and codegen stubs in a range of languages for both API server implementations and client library builds.
- Stoplight, particularly if your API development project is specification-oriented, e.g. around OpenAPI.
There are loads of API tools out there, these are just the ones I've used most - please share yours in the comments!
It’s actually worth spending a bit of time investing in your learning around API development and testing tools as you would when you e.g. learn a new language or framework - these are skills you’ll find yourself calling on again and again. In terms of upcoming proficiencies, it’s likely worth looking into GraphQL, as its popularity seems to be growing.
A note on standards
As implied earlier, another confusion point awaits you if you deal with API documentation, specifications, and definitions - all of which are related, but perform slightly different functions. They all focus on indicating what an API does, but differ in the relative extent to which the info is being conveyed to humans or machines.
- Documentation is intended for human consumption and essentially comprises your API user guidance.
- You can think of a specification as a contract, operating as a kind of shared description of how an API will behave. The OpenAPI spec is a commonly used standard for describing APIs in this way, in either JSON or YAML format.
- Definitions also describe an API, but are intended specifically for machine processing.
However, although there have been a lot of efforts to advance standardization around APIs, there is no real consensus to speak of (at this stage anyway). Add to this the fact that authoring specifications etc is not necessarily a trivial task, and you can see the benefit of API tools that are flexible and that support differing documentation types / formats.
Your users might be other developers
If you’re exposing an API or developer service, you’ll find yourself in the position of treating other developers as your users. This means that you’ll be in the world of DX (Developer Experience), which comes with its own set of challenges that are not reserved for front-end or UX, as they come into play from the API design and structuring phase forward.
Note that even if you’re developing an API for internal use within your team or company, it pays to consider DX (much like code comments, you might be glad you thought of DX when you attempt to use a service you developed a while ago and your memory is not so clear). 🤓
DX is a whole field in itself, but since dev education is my thing, here are a few quick takeaways:
- Developer backgrounds vary wildly - there is no such thing as a normal skillset that can be assumed.
- Developer context is also extensively varied - from people working in enterprises to freelancers. You may have a huge range of projects integrating with your API.
- With developer tools, community becomes an important component, and onboarding needs to be tailored to developers. Again, these are whole areas of specialism in their own right, hence DevRel, but if you happen to be involved in building developer portals or learning resources, here are some key points:
- Support “quick start” pathways letting new users try requests quickly without having to e.g. sign up and wait for an API key.
- If your API supports lots of different use cases, create pathways to help people from different backgrounds and with different needs get started.
- Try to provide full walkthroughs for sample implementations using your API - a common failing is only including reference documentation that explains endpoints but doesn’t convey how the pieces fit together in practice, so context is often missing.
- It might seem obvious to you but make sure it’s clear what your API does. 😲 The same goes for explaining any domain-specific terms.
- Capture feedback and include contribution pathways for your API users - being able to have an impact is usually important to developers.
- Although it’s essential to be developer-oriented, bear in mind that other stakeholders might be engaging with your resources - you might come across business language relevant to APIs, like “digital transformation.” With dev products the developer might be the user, but someone else in an org might be the buyer.
Conclusion
What we’ve touched on in this post is really just the tip of the iceberg when it comes to how integral APIs are to contemporary software development, but hopefully it’s given you a heads-up in terms of what to expect. The good thing is that these integrations let you maximize on your own development efforts to create innovative applications.
With integration such a key element in the developer landscape, it’s likely that we’ll also see more tools and products dedicated to facilitating those integrations, not to mention a continued increase in companies paying attention to developer experience. On that note, if you find yourself interested in doing something other than writing code all day every day, you might want to consider a career in DevRel yourself! 🚀
Top comments (1)
Thanks