Loosely coupled Python code with Dependency Injection
Software has to be flexible in order to respond to change. Dependency injection is a technique for managing the dependencies between software components. It is a design pattern that uses inversion of control to provide the ability to swap out components for testing or other purposes without changing any other part of the system.
The main idea behind dependency injection is to pass the responsibility for providing services from constructor functions or direct instantiation methods onto other objects. In other words, an object is given its dependencies instead of creating them. This way, if we want to add new features, we don’t have to change the existing code but instead, pass new dependencies into it.
Creating Abstractions with Interfaces
There are three steps in writing loosely coupled Python code with dependency injection. The first step is to identify what dependencies the code needs, the second step is to create interfaces for each dependency, and the third step is to pass them into the dependent object through its constructor or method parameters.
Recently, I implemented functionality that required a byte encoder and decoder to manipulate business data (I’ve simplified its implementation for readability purposes). I abstracted out their responsibilities to interfaces. Python doesn’t have an interface keyword like Golang, but you can replicate interface functionality with the ABC library.
Focusing on the IPacketEncoder interface, a concrete encoder class must implement it. The initial business requirements required all types to be encoded and strings to be padded with their length. Luckily, Python provides the struct library to do most of the hard work.
Implementing Dependency Injection
The encode_name function is a simple implementation for creating an encoded payload with the specified name. It takes an IPacketEncoder object and a string as parameters.
The two key implementation details are using the IPacketEncoder interface instead of the PaddedPacketEncoder concrete class and encoder is an argument rather than being initialized inside the function. This is dependency injection. Because IPacketEncoder is passed in, it can be easily extended, changed or provided with a stub, thus keeping the system loosely coupled.
A few weeks later, there was a new requirement for an encoder to handle null terminated encodings. Luckily, due to the existing implementation, it can be easily implemented.
NullTerminatedPacketEncoder essentially adds NULL_BYTE after each encoded string. Going back to the encode_name function, an instance of NullTerminatedPacketEncoder is a valid encoder argument since NullTerminatedPacketEncoder implements the IPacketEncoder interface similarly to PaddedPacketEncoder.
If dependency injection wasn’t used here, it could result in code that is tightly coupled and difficult to change, for example:
If a new encoder is required, the encode_name function will have to be changed almost entirely along with its tests, extending delivery times. What’s worse, it could result in hidden side effects, especially if the original developer is no longer available.
To conclude, I’ll provide a less simplistic example using this technique. SocketService is responsible for decoding an operation code, calling its relevant handler and returning its result. All of its argument types are interface abstractions, including its constructor. The CLI is responsible for determining which encoder to use, i.e. NullTerminated or Padded, and passing its instance down to the SocketService class.
As projects continue to grow, its recommended to utilise a dependency injection framework to “inject” these dependencies automatically, such as Dependency Injector.
Hopefully, this demonstrates how to write loosely coupled Python code with dependency injection. Many Python developers dislike dependency injection because they feel it's not “pythonic”, but I would argue that its benefits can’t be discredited, especially when it's present in so many other programming languages.
Thanks for reading 😊. If you would like to connect with me, you can find me on Linkedin or Github.
Top comments (3)
Thank you for this Shane. I have the ABC library before: I wanted the abstract feature for methods. I have not thought of using its as "interface" as you have explained -- very articulated, easy to follow too. It makes sense when compare to what I used to do in Delphi. I don't think interfaces make development time any longer or complex, a well thought out interface can save us a lot of pain in the long run.
Well detailed and documented, thanks for posting this!
The one objection that i have with dependency injection (and interfaces in general) in Python, is that it makes the code just a bit more complex and larger.
This may not look like a big deal, but in practical day-to-day sprints of Data Engineer teams, that gives you hours to deliver a component, its harder to think and implement a well crafted interface.
And since most of those teams don't have programmers with more extensive knowledge in software architecture (that i assume is the case for software engineering teams), they tend to think that interfaces are overkill.
Because of this, i always wonder with myself when should i go for interfaces and extensibility over fast deliveries that solves the problem. The answers are not always clear.
Hey Renato,
Thanks for your detailed response :).
I agree that interfaces add additional complexity, but once a developer understands their benefits, it's easier to utilize them. Interfaces should only add a little time to a deliverable since they are just abstractions; for example, the implemented fast solution would technically already have its abstractions in place; its public methods.
Pull requests are a great place to educate team members that lack software architecture skills.
Some Python developers prefer monkey-patching, but I personally dislike this technique since I've only seen it in Python and find it "hacky".
If the deadline is extremely tight, I usually add a story to the backlog to come back and refactor.