Neat Python API

I've seen many Python interfaces in my career (as in API, not UI).
You can quickly spot an ugly one by its size. Below are some recipes on how to make a neat one!

Interfaces in Python

An "interface" is a nebulous concept that might be initially hard to grasp.
It is a blueprint of interactions with an object (reading a few answers here is not going to hurt).

Do you need interfaces in Python at all? There are several good posts about it:

• Chelsea Troy
• Stack Overflow (please read several answers!)
• Diego Barba

I’m going to assume that you read all that and decided to go with interfaces.
I think Protocols are the best way to create interfaces in Python. This is what we are going to use here.

Here, we are going to discuss what makes a good interface.

An interface has to be small

You’re designing a Zoo simulator and made an interface for "a generic animal":

class Animal(Protocol):
   def current_weight(self) -> float: ...
   def price(self) -> float: ...
   def sleep(self, hours): ...
   def eat(self, food): ...
   def draw(self, context): ...
   def look_for_food(self): ...
   def hunger_level(self) -> float: ...
   def is_asleep(self) -> bool: ...
   def is_awake(self) -> bool: ...
   def current_position() -> Tuple[float, float, float]: ...
   def current_orientation() -> RotationMatrix: ...
   def current_transform3d() -> HomogeneousMatrix: ...

You should be alarmed when you see an interface with more than 4-5 methods.
Chances are that it was not well-designed, and the project it is used in will be overly convoluted:

• this interface will be used in completely unrelated parts of the software. It will couple them together
• there will be extensive use of class hierarchies
• interface implementations are going to be overly complicated
• it will be hard to document and confusing to use.

You might think: “Clearly, animals do tons of different things, so it's justified to have many methods.”
But you can come up with ten more methods to add to a “generic animal,” and there is never going to be an end to it.
The art is to model a complicated thing with a set of small decoupled interfaces.

What are the recipes for reducing an interface's size?

Semantic overlap

Sometimes there could be some obvious duplication methods (e.g., get_weights vs. current_weight).
But more often you'll find methods that somewhat overlap in semantics.
After a bit of thought, it's possible to remove some of them.
In the example above:

   def current_position() -> Tuple[float, float, float]:
       pass

   def current_orientation() -> RotationMatrix:
       pass

   def current_transform3d() -> HomogeneousMatrix:
       pass

The user extract position and orientation from the HomogeneousMatrix returned by current_transform3d.
The other two getters could be replaced by utility functions.

A similar situation is probably happening with:

   def is_asleep(self) -> bool:
       pass

   def is_awake(self) -> bool:
       pass

Is it true that animal.is_asleep() == not animal.is_awake()? If yes, then you better remove one of them.

Decoupling interfaces

After removing duplicates, you should try to split the interface into independent parts.
Most likely, you don’t use all Animal features in every part of your software:

1. there could be a function that handles the drawing of animals that uses only current_transform3d and draw:

def draw_entity(animal: Animal):
    t = animal.current_transform3d()
    c = some_drawing_context(t)
    animal.draw(c)

2. there could be an animal life cycle algorithm:

def life_management(animal: Animal):
    eating_logic(
      animal.hunger_level(), animal.look_for_food(), 
      animal.eat())
    sleeping_logic(animal.is_awake(), animal.sleep())

3. and a Zoo management system

def purchase_decision(animal: Animal, budget: float) -> bool:
   w = animal.current_weight()
   p = animal.price()
   decide_to_buy(p, w, budget)

The “generic animal” is a collection of quite different interfaces that could be disentangled:

class VisualEntity(Protocol):
   def current_transform3d() -> HomogeneousMatrix: ...
   def draw(self, context): ...

class BehavingAgent:
   def sleep(self, hours): ...
   def eat(self, food): ...
   def look_for_food(self): ...
   def hunger_level(self) -> float: ...
   def is_awake(self) -> bool: ...

class ZooAsset:
   def current_weight(self) -> float: ...
   def price(self) -> float: ...

Now you can see the advantages of many small interfaces over a single big one:

• You to decouple parts of your codebase and company workflows. A rendering team will not depend on Zoo management developers.
• No need to implement unnecessary methods for slight variations of “Animals.” Changing a price algorithm does not lead to a new species!
• You would be able to reuse small interfaces and utilities. You might find that the draw_entity and purchase_decision work for your Plant object.

Now you can see the advantages of many small interfaces over a single big one.

• They decouple parts of your codebase (while the large ones have a high chance of coupling unrelated parts). This in turn decouples company workflows (e.g. teams don't break each other's stuff too much)
• You don't have to write many implementations when instantiating a short interface. This ripples through your design because now you more easily avoid base classes and class hierarchies. Which in turn promotes composition over inheritance (there are many resources on why it's a good idea, e.g. see here
• You would be able to reuse small interfaces in more places than large ones. This in turn makes the total surface area of APIs in your codebase smaller, easier to learn with simpler onboarding

Recursive semantics

Next, you should be alarmed if one function calls another in the same interface.
Continuing with the Zoo example:

class BehavingAnimal(Protocol):
   def lifecycle(self):
       """ What happens during 24 hours period """

   def eat(self, food):
       """ What happens when the animal consumes food """

   def sleep(self, hours):
       """ What happens when the animal sleeps """

It is unlikely that an animal will spend a day without eating and sleeping.
So there is a high chance that eat and sleep will be called from the lifecycle implementation.

Such an interface couples together two levels of abstraction.
These are two interfaces in one, similar to the previous section.
A "lifecycle" part is used in some global application context, but eat and sleep are probably used only locally inside it.
Let's split them apart:

class ElementaryAnimal(Protocol):
   def eat(self, food): ...
   def sleep(self, hours): ...

class LifecycleManagement(Protocol):
   def lifecycle(self, animal: ElementaryAnimal): ...

Notice that now lifecycle takes ElementaryAnimal as an argument.
This clearly states that a lifecycle depends on something that can eat and sleep.

Too generic

Yet another thing to avoid is an overly-generic interface:

class Creature(Protocol):
   def act(self, *args, **kwargs) -> object:
       """ Do anything you want """

 def perform_action(creature: Creature):
     custom_args = ...
     creature.act(*custom_args)

An interface is like a contract between two parties.
But a contract that says “do whatever” is as good as no contract at all.
It is too easy to make such interfaces in an optional/dynamic-typed language like Python, especially with its wildcard argument features.

Unfortunately, such interfaces come up in some situations, but we can summarize it as “passing a black box around.”
Passing “blackboxes” is typically worse than explicit components.
But if you have to do it, it’s better to acknowledge its "blackness" by passing
a functor and moving to a more functional design (here is an in-depth tutorial):

def perform_action(actor: Callable[..., Any]):
    custom_args = ...
    actor(*custom_args)

Constructors in an interface

Related bonus topic: have you noticed that __init__ is typically not a part of any interface?
Why not? It seems like it is just another method you can have.

It goes back to C++, where constructors are not virtual.
Bjarne Stroustrup gives an answer on why it is so:

“A virtual call is a mechanism to get work done given partial information.
To create an object, you need complete information. Consequently, a "call to a constructor" cannot be virtual.”

I like high-level reasoning more: a constructor is a function for making objects. It belongs to the “object-makers” realm.
It just can’t belong to an already-made object: you shouldn't make an object out of itself and I'm not a fan of cloning).

A constructor could be viewed as a method of a factory interface
(similar ideas can be found here).
In other words, a constructor and an object interface are yet another example of two coupled levels of abstraction.

Conclusion

To summarize, a good interface should:

• be small and concise
• avoid semantic duplication
• avoid coupling several parts in one
• avoid mixing multiple abstraction levels
• be specific and not too generic
• not include __init__

At the same time, real life has many exceptions and complications.
But those must be justified exceptions, not excuses to keep messy interfaces around.
Thank you for reading!

You can find me on LinkedIn or Twitter.

Originally published at https://sinavski.com.