One of our clients was a mid-sized U.S. financial services firm that administered around $15 billion in client assets for registered investment advisers, broker-dealers and institutional managers. Its operations staff used a Java-based client servicing portal to update things like standing instructions and account details, and those updates went through MuleSoft to OData services on SAP Gateway. In development and test, every write worked. In production, somewhere between 10 and 15 percent of them failed, apparently at random, with HTTP 403 and the message "CSRF token validation failed". I'll come back to what caused it in the section on CSRF tokens.
Sooner or later, most integration developers get a ticket that says "pull X from SAP" or "write Y back to SAP", and quite often nobody on the team has ever logged into an SAP system. There is good news and bad news. The good news is that S/4HANA exposes most of what you need as OData services over plain HTTPS, so you can work with them from Python using requests. The bad news is that SAP's OData comes with a handful of conventions that look strange to anyone outside the SAP world, and each one tends to cost a day of confusion the first time you run into it.
This is the guide I wish I could hand to every developer in that position. I'll cover finding the API, authentication (briefly), the CSRF token step that every write needs, optimistic locking with ETags, paging, the differences between OData V2 and V4 that cause real bugs, $batch, and the cases where OData is the wrong interface altogether. One caveat before we start. What's available to you depends on the SAP release, on whether the customer runs S/4HANA Cloud or on-premise, and on what they have actually activated. So always check against the customer's own system.
Find the API before you write any code
SAP publishes its released APIs on the SAP Business Accelerator Hub, at api.sap.com. (It used to be called the API Business Hub, and you'll still see that name around.) For S/4HANA, search for the business object you care about. The business partner API, API_BUSINESS_PARTNER, is a good one to learn on. A business partner is SAP's general record for a customer, a supplier, or anyone else the company deals with. Each API page lists the entity sets, their properties, the operations they support and the EDMX metadata, and most S/4HANA Cloud APIs come with a sandbox you can call with a personal API key. To get the key, sign in and choose "Show API Key" on the API page.
Use the sandbox to learn how the payloads are shaped before you go anywhere near a customer system:
import os
import requests
SANDBOX = "https://sandbox.api.sap.com/s4hanacloud/sap/opu/odata/sap/API_BUSINESS_PARTNER"
resp = requests.get(
f"{SANDBOX}/A_BusinessPartner",
headers={"APIKey": os.environ["SAP_HUB_API_KEY"], "Accept": "application/json"},
params={
"$select": "BusinessPartner,BusinessPartnerFullName,BusinessPartnerCategory",
"$filter": "BusinessPartnerCategory eq '2'",
"$top": "5",
},
timeout=30,
)
resp.raise_for_status()
for bp in resp.json()["d"]["results"]:
print(bp["BusinessPartner"], bp["BusinessPartnerFullName"])
Notice the d and results wrapper around the data. That comes from the OData V2 JSON format, and I'll come back to it. The sandbox holds read-only sample data, so it's good for learning what payloads look like, but you can't test writes against it.
While you're on the Hub, check one more thing: whether the API is marked as released. Released APIs are the ones SAP commits to keeping stable across upgrades, which matters a great deal on a program that will live through several of them. If someone offers you a custom service built on a Z function module instead – in SAP, anything whose name starts with Z or Y was built by the customer, not by SAP – you can still use it. Just know that you're taking on the upgrade risk yourself.
Authentication, briefly
I'll keep this section short, because the right answer is decided by the customer's Basis and security teams. (Basis is what SAP shops call the team that runs the system itself – roughly, SAP's version of infrastructure and operations.) On S/4HANA Cloud, inbound API access is set up through a communication arrangement, which ties a communication scenario – the set of APIs – to a communication system and a communication user. That user can authenticate with basic credentials or a client certificate, and OAuth 2.0 is available too. On-premise systems usually expose OData through SAP Gateway, with basic authentication, X.509 client certificates, or a SAML or OAuth setup, depending on what the landscape already uses.
From the Python side, all of these end up as a requests.Session carrying an auth tuple, a client certificate or a bearer token. Ask for a technical user scoped to the APIs you need and nothing more. Also ask which SAP client you should target (the sap-client number). On-premise systems often host several clients, and the wrong one will return data that looks perfectly plausible and is still wrong.
The CSRF token step for writes
This is the first thing that trips everyone up. SAP Gateway protects every modifying request – POST, PUT, PATCH, MERGE, DELETE, and $batch, which is always a POST – with a CSRF token. To get one, you send a non-modifying request with the header x-csrf-token: Fetch. The response carries the token in an x-csrf-token header, along with some session cookies. You send the token back on your writes, and – this is the part people miss – you must send the cookies with it. A token that arrives without its session cookies gets a 403, with an x-csrf-token: Required header in the response. That combination is the most common cause of "but I sent the token!" bug reports.
The names of those session cookies vary, and SAP warns that they can change, so don't go picking out a cookie by name. Let one requests.Session hold the whole cookie jar for you:
import requests
class SapODataClient:
def __init__(self, service_url: str, auth, sap_client: str | None = None):
self.base = service_url.rstrip("/")
self.s = requests.Session()
self.s.auth = auth
self.s.headers["Accept"] = "application/json"
if sap_client:
self.s.params = {"sap-client": sap_client}
self._token: str | None = None
def _fetch_token(self) -> str:
r = self.s.get(f"{self.base}/", headers={"x-csrf-token": "Fetch"}, timeout=30)
r.raise_for_status()
token = r.headers.get("x-csrf-token")
if not token:
raise RuntimeError("Service did not return a CSRF token")
self._token = token
return token
def write(self, method: str, path: str, **kwargs) -> requests.Response:
"""Send a modifying request, refetching the token once if it expired."""
for attempt in (1, 2):
token = self._token or self._fetch_token()
headers = {**kwargs.pop("headers", {}), "x-csrf-token": token}
r = self.s.request(method, f"{self.base}/{path.lstrip('/')}",
headers=headers, timeout=60, **kwargs)
if (r.status_code == 403
and r.headers.get("x-csrf-token", "").lower() == "required"
and attempt == 1):
self._token = None # token or session expired; fetch a fresh pair
kwargs["headers"] = {k: v for k, v in headers.items() if k != "x-csrf-token"}
continue
return r
return r
The retry happens once, on purpose. If a token you've just fetched is rejected as well, something else is wrong – usually a proxy that strips cookies, or a load balancer without session affinity – and looping won't fix it.
That was exactly the problem at the financial services client. The first suggestion was to fetch a fresh token before every write. I wasn't convinced that would fix anything, so we turned on detailed HTTP logging in MuleSoft – with credentials and account data masked – and followed successful and failed requests through the application, the API gateway, the load balancer and SAP Gateway. In the failed requests, the token was there on the POST, but the SAP_SESSIONID_<SID>_<client> cookie that had come back with it was missing. A CSRF token belongs to a server-side session, and without its session cookie, SAP rejects it.
The cookie was being removed by an F5 BIG-IP in front of SAP Gateway. An earlier security-hardening change had added an iRule to keep unnecessary application cookies away from external clients, and it matched SAP's session cookies too broadly. The failures only looked random because some MuleSoft flows reached SAP by an internal route that bypassed that virtual server. Once we grouped the failures by network route, the pattern was obvious, and we proved it by sending the same token fetch, with the same service account, both ways. Through the internal route, the cookie came back. Through the F5, it didn't.
The network and security teams changed the F5 rule to preserve SAP session cookies on the trusted integration path, and we made MuleSoft keep the cookies from the token fetch and send them back with the write. (In Python, the requests.Session in the client above does that for you, which is one reason it's there.) We didn't switch off CSRF validation in SAP. That would have made the error go away by weakening a real security control. We also added an integration test that fetches a token, immediately makes a permitted update to a test record, and checks that the session cookie survived along with the token. When I debug one of these failures now, I don't start by assuming the token is bad. I look at the token, the cookies, the session identity and the network route together.
ETags and If-Match for updates
Many S/4HANA APIs use optimistic concurrency, which sounds more complicated than it is. When you read an entity, the service gives you an ETag – think of it as a version stamp on the record. You'll find it in the ETag response header when you read a single entity, and in __metadata.etag inside V2 payloads. When you update the entity, you send the ETag back in an If-Match header. If the record has changed since you read it, you get 412 Precondition Failed. If you send no If-Match at all, on an entity that requires one, you get 428 Precondition Required.
You'll find advice online to send If-Match: * to make the 428 go away. Please don't. It switches off the concurrency check, and your write will quietly overwrite someone else's change, which is the exact thing the ETag is there to prevent. Read first, write with the real ETag, and when you get a 412, re-read and decide again:
def update_search_term(client: SapODataClient, bp_id: str, term: str) -> None:
path = f"A_BusinessPartner('{bp_id}')"
for _ in range(3):
r = client.s.get(f"{client.base}/{path}", timeout=30)
r.raise_for_status()
etag = r.headers.get("ETag") or r.json()["d"]["__metadata"].get("etag")
if not etag:
raise RuntimeError(f"No ETag returned for {path}")
w = client.write("PATCH", path, json={"SearchTerm1": term},
headers={"If-Match": etag, "Content-Type": "application/json"})
if w.status_code == 412:
continue # someone else changed it; re-read and try again
w.raise_for_status() # PATCH normally returns 204 No Content
return
raise RuntimeError(f"{path} kept changing underneath us; giving up")
Whether "re-read and retry" is the right response depends on the field. For a search term, it's fine. For a credit limit or a price, a 412 should usually go to a person, who can decide what the value ought to be.
Filtering, selecting and paging
Always use $select. S/4HANA entities are wide (a business partner carries a great many fields), and fetching all of them to read three wastes the server's time and yours. Use $filter so that SAP does the filtering, instead of pulling everything into Python and filtering there. For paging, use $top and $skip, or better still, follow the server's own next-page link when it sends one.
The paging details differ between versions, which is mildly annoying. In V2, a collection arrives as {"d": {"results": [...]}}, a server-driven next page is in d.__next, and getting a total count needs $inlinecount=allpages. In V4, the collection is in value, the next page is in @odata.nextLink, and the count option is $count=true. A small generator can hide both:
from collections.abc import Iterator
from urllib.parse import urljoin
def iterate(client: SapODataClient, entity_set: str, params: dict,
page_size: int = 500) -> Iterator[dict]:
url = f"{client.base}/{entity_set}"
query = {**params, "$top": str(page_size), "$skip": "0"}
server_paging = False
while url:
r = client.s.get(url, params=query, timeout=60)
r.raise_for_status()
body = r.json()
if "d" in body: # OData V2
rows = body["d"]["results"]
next_link = body["d"].get("__next")
else: # OData V4
rows = body["value"]
next_link = body.get("@odata.nextLink")
yield from rows
if next_link: # server-driven paging
url, query, server_paging = urljoin(r.url, next_link), None, True
elif not server_paging and len(rows) == page_size: # client-driven paging
query["$skip"] = str(int(query["$skip"]) + page_size)
else:
url = None
If you page with $skip, pass an $orderby on the key in params. Without a stable order, rows can move between pages while you're reading, and you'll skip or duplicate records with no error to tell you it happened.
V2 versus V4: the differences that actually bite
Most S/4HANA APIs on the Hub are still V2, though a growing number of V4 services are being built on SAP's newer programming model. The V4 services live under a different URL path (/sap/opu/odata4/... instead of /sap/opu/odata/...), so you can't switch versions by changing a header. Beyond the envelope and the paging, three differences cause real bugs.
Numbers come first. In V2 JSON, Edm.Decimal and Edm.Int64 are serialized as strings, so an amount arrives as "1250.00". Code that does arithmetic on it either fails or, worse, concatenates. In V4 they are JSON numbers by default, which Python's json module turns into floats – and a float is the wrong type for money. Either ask for strings, by sending Accept: application/json;IEEE754Compatible=true, or parse with r.json(parse_float=Decimal).
Dates are next. V2 represents Edm.DateTime as "/Date(1704067200000)/" – milliseconds since 1970, wrapped in a rather odd little string – and filter literals look like datetime'2024-01-01T00:00:00'. V4 uses ordinary ISO 8601 strings in payloads and plain literals in filters. Write a helper that converts the V2 form into a timezone-aware datetime once, and keep it.
Then there are partial updates. V2 historically used the non-standard MERGE method, and V4 uses PATCH. SAP Gateway accepts PATCH on V2 services too, which is why the example above uses it. But some older client libraries still send MERGE with an X-HTTP-Method override, and a proxy in the middle may not let that through.
$batch: fewer round trips, one transaction per change set
$batch lets you send several operations in a single HTTP request, as a multipart/mixed body. Inside it, a change set groups modifying operations that SAP processes as a unit. A failed change set doesn't roll back the other change sets in the same batch, and the outer response is typically 202 Accepted even when operations inside it failed, so you have to parse the status of each part yourself. To make things more interesting, when a change set fails, SAP Gateway typically sends back a single error response for the whole change set, instead of one per operation, so count the statuses before you try to match them to your requests. And the batch POST needs the CSRF token, like any other write.
import json
import re
import uuid
def batch_patch(client: SapODataClient, updates: list[tuple[str, str, dict]]) -> list[int]:
"""updates: (entity_path, etag, fields). Returns the status of each operation."""
batch, cs = f"batch_{uuid.uuid4().hex}", f"changeset_{uuid.uuid4().hex}"
parts = []
for path, etag, fields in updates:
parts.append(
f"--{cs}\r\nContent-Type: application/http\r\n"
f"Content-Transfer-Encoding: binary\r\n\r\n"
f"PATCH {path} HTTP/1.1\r\nContent-Type: application/json\r\n"
f"If-Match: {etag}\r\n\r\n{json.dumps(fields)}\r\n"
)
body = (f"--{batch}\r\nContent-Type: multipart/mixed; boundary={cs}\r\n\r\n"
+ "".join(parts) + f"--{cs}--\r\n--{batch}--\r\n")
r = client.write("POST", "$batch", data=body.encode(),
headers={"Content-Type": f"multipart/mixed; boundary={batch}"})
r.raise_for_status()
statuses = [int(s) for s in re.findall(r"^HTTP/1\.1 (\d{3})", r.text, re.MULTILINE)]
if len(statuses) != len(updates): # a failed change set comes back as one error response
raise RuntimeError(f"change set rejected as a whole: {statuses}")
return statuses
Two cautions. Whether a change set with several operations really commits as one unit depends on how the service was implemented, so check the API's documentation before you rely on it. And on S/4HANA Cloud, deep PATCH and PUT aren't supported, and SAP points to $batch as the way to update several entities.
When OData is the wrong tool
OData is the right surface for queries, and for record-level transactions where the caller needs an answer now. To be honest, though, it's a poor fit for a fair amount of integration work. Extracting millions of rows through $skip paging is slow and loads the application server; that job belongs to SAP's extraction and replication tooling. Asynchronous business documents exchanged with partners – orders, deliveries, invoices – are still a natural fit for IDocs, as long as someone owns the monitoring of failed statuses. And where there's no released API for an operation, particularly on older ECC systems, a BAPI called over RFC and wrapped behind your own service is often the honest answer. I wrote a longer comparison of when to use IDoc, BAPI, OData or Integration Suite, including the questions that usually settle the choice.
There's one more mistake, and it's an easy one to make: polling an OData service every few minutes to see what has changed. If you need to react to something happening in SAP, S/4HANA business events are the better mechanism, where the customer has them enabled. You use OData afterward, to read the full record the event points to. Polling looks simpler, right up until the entity set grows and the poll's $filter on a change date becomes the most expensive query in the system.
None of this is exotic once you've seen it. Keep the session and its cookies together, respect the ETag, select only what you need, and know which OData version you're talking to. Most of the surprises left after that come from how the customer has configured their system – so make friends with their Basis team early.
I'm Jayant Chaudhary, founder of eProxim. We build the integration layer between SAP and the systems around it, working alongside the implementation partners who run SAP programs.
Top comments (0)