Python "magic methods" allow for some very fun and powerful code.
Magic methods refers to all those special methods in Python classes that start and end with __
.
The documentation for Special method names says:
A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names.
For instance the __init__
method is called when a new instance of a class is instantiated. __str__
is a method that returns a string representation of the class. As stated in the docs, these are not methods that generally get invoked directly on the class. For example, you don't call my_object.__str__()
but __str__
would be called by print(my_object)
.
In this post, I'm going to demonstrate using the __getattr__
and __call__
magic methods to build a dynamic API client.
Let's say you're building a client for an API with endpoints like:
- GET https://some-api.net/user
- GET https://some-api.net/user/permissions
- POST https://some-api.net/article
- GET https://some-api.net/article/
There are a lot of ways to build a client for this. One might look like:
import logging
import requests
class APIClient:
BASE_URL = "https://some-api.net"
def make_request(method, url, params=None, data=None):
response = requests.request(
method,
f"{self.BASE_URL}/{url}"
params=params,
json=data
)
response_body = {}
try:
response_body = response.json()
except ValueError:
log.warning("Unexpected Response '%s' from '%s'",
response.content, response.url)
return response_body
def get_user(self):
return self.make_request("GET", "user")
def get_user_permissions(self):
return self.make_request("GET", "user/permissions")
def create_article(self, data):
return self.make_request("POST", "article", data=data)
def get_article(self, id):
return self.make_request("GET", f"article/{id}")
We would then use the client:
client = APIClient()
user = client.get_user()
permissions = client.get_user_permissions()
new_article = client.create_article({"some": "data"})
existing_article = client.get_article(123)
And this is fine.
But what if the API is changing often? Or if there are dozens of endpoints and you don't know which you need to call? If later on you discover you need to call GET https://some-api.net/user/roles
or GET https://some-api.net/some_other_thing?some_param=foo
, you'll need to go back to you client and add matching methods.
It would be nice to have things a bit more dynamic, and one way to do that is with magic methods. Specifically, I'll be using:
-
__getattr__
which is "Called when the default attribute access fails with an AttributeError" (see the docs). This means that if you callmy_object.some_missing_attr
, then__getattr__
will get invoked with"some_missing_attr"
as thename
parameter. -
__call__
which is "Called when the instance is "called" as a function" (see the docs), eg.my_object()
Using these, we can build something like:
import logging
import requests
log = logging.getLogger(__name__)
class APIClient:
BASE_URL = "https://some-api.net"
def __init__(self, base_url=None):
self.base_url = base_url or BASE_URL
def __getattr__(self, name):
return APIClient(self._endpoint(name))
def _endpoint(self, endpoint=None):
if endpoint is None:
return self.base_url
return "/".join([self.base_url, endpoint])
def __call__(self, *args, **kwargs):
method = kwargs.pop("method", "GET")
object_id = kwargs.pop("id"))
data = kwargs.pop("data", None)
response = requests.request(
method,
self._endpoint(object_id),
params=kwargs,
json=data
)
response_body = {}
try:
response_body = response.json()
except ValueError:
log.warning("Unexpected Response '%s' from '%s'",
response.content, response.url)
return response_body
This allows us to call the API like so:
client = APIClient()
user = client.user()
permissions = client.user.permissions()
new_article = client.article(method="POST", data={"some": "data"})
existing_article = client.article(id=123)
How does this work?
Well in the case of client.user.permissions()
, .user
is an attribute and since the client doesn't have that attribute, it calls __getattr__
with name="user"
, which then returns an APIClient
instance with the name appended to the endpoint URL. The same happens when .permissions
is invoked on the APIClient
instance that was returned by .user
, in turn giving us another APIClient
instance, now with a path of https://some-api.net/user/permissions
. Finally this APIClient
instance is called by ()
, which invokes the __call__
method. This method makes the actually HTTP call to the constructed URL based in any parameteres passed in, but defaults to a GET
request.
In the case of the article calls, client.article(id=123)
works much the same way, but the APIClient
is called with an id parameter. This id gets appended to the url by the same internal _endpoint()
method that __getattr__
uses, which results in a GET call to https://some-api.net/article/<id>
.
For client.article(method="POST", data={"some": "data"})
, we override the method and add a data payload to the POST.
As you can see, the client itself can handle a large variety of API calls without any significant changes, allowing the utilizing application a lot of flexibility.
In the example of the newly required calls, GET https://some-api.net/user/roles
and GET https://some-api.net/some_other_thing?some_param=foo
, no changes to the client are needed:
roles = client.user.roles()
thing = client.some_other_things(some_param="foo")
Top comments (0)