DEV Community

Cover image for How to Parse XML with Python Requests: Beginner's Guide
Shakhzhakhan Maxudbek
Shakhzhakhan Maxudbek

Posted on Edited on Originally published at args.tech

How to Parse XML with Python Requests: Beginner's Guide

#python #xml #requests #parsing

Introduction: What is XML?

XML stands for Extensible Markup Language. It is a very common format used for storing, structuring, and transporting data between different systems. Unlike HTML, which is designed to display data in a browser, XML is designed purely to carry data.

The best part about XML is its flexibility: you are not limited to predefined tags. You can create any tag names that make sense for your specific project. It is widely used in RSS feeds, sitemaps, API responses, and configuration files.

Here is a simple example of an XML file. For this tutorial, we will use a custom file containing a basic server inventory:

<?xml version="1.0" encoding="UTF-8"?>
<inventory>
    <server>
        <hostname>web-node-01</hostname>
        <os>Ubuntu 24.04</os>
        <ram>16GB</ram>
        <status>Active</status>
    </server>
    <server>
        <hostname>db-node-01</hostname>
        <os>Debian 12</os>
        <ram>64GB</ram>
        <status>Active</status>
    </server>
    <server>
        <hostname>cache-node-01</hostname>
        <os>Alpine Linux</os>
        <ram>8GB</ram>
        <status>Maintenance</status>
    </server>
    <server>
        <hostname>backup-node-01</hostname>
        <os>Ubuntu 24.04</os>
        <ram>32GB</ram>
        <status>Offline</status>
    </server>
</inventory>
Enter fullscreen mode Exit fullscreen mode

How is this structured? Think of an XML file as a family tree.

  • In our example, <inventory> is the root element (the trunk of the tree). It wraps everything else.
  • Inside the root, we have multiple <server> elements. These are the child nodes (the branches).
  • Each <server> contains specific data points: <hostname>, <os>, <ram>, and <status>.

Our goal now is to download this data using Python and extract exactly the information we need.

Setting Up the Workspace

Before we write any code, we need to prepare our working environment. We will use a Python virtual environment. A virtual environment isolates your project's dependencies from your main system. This is a best practice because it ensures that installing packages for this specific project won't accidentally break anything else on your computer.

First, make sure you have the virtual environment package installed (instructions for Debian/Ubuntu):

sudo apt update && sudo apt install python3-venv -y
Enter fullscreen mode Exit fullscreen mode

Next, create a new folder for your project and navigate into it:

mkdir my_project && cd my_project
Enter fullscreen mode Exit fullscreen mode

Now, create a Python virtual environment named env inside this folder:

python3 -m venv env
Enter fullscreen mode Exit fullscreen mode

Activate the virtual environment. You will notice that your command prompt changes to show (env) at the beginning, meaning the environment is active:

source env/bin/activate
Enter fullscreen mode Exit fullscreen mode

Finally, we need to install the Requests library. This is a powerful, user-friendly third-party library that makes sending HTTP requests (like downloading files from a web server) incredibly easy in Python:

pip install requests
Enter fullscreen mode Exit fullscreen mode

With our workspace ready and the requests library installed, we can start coding!

Fetching Data and Parsing the Tree

Create a file named main.py in your project folder. The first thing we need to do is download the XML data and parse it. Python has a built-in module called xml.etree.ElementTree that is perfect for this.

Open main.py and insert the following code:

import requests
import xml.etree.ElementTree as ET

# Step 1: Download the XML file
url = 'https://args.tech/media/uploads/3112c7f5-8f48-4cca-994d-9b3f326f0cf4.xml'
response = requests.get(url)

# Step 2: Parse the raw content into an XML tree
root = ET.fromstring(response.content)

# Step 3: Iterate through all elements to see the structure
for item in root.iter('*'):
    print(item.tag)
Enter fullscreen mode Exit fullscreen mode

How does this work?

  • requests.get(url) reaches out to the server and downloads our file. The raw data is stored in response.content.
  • ET.fromstring() takes those raw bytes and transforms them into a structured "tree" object that Python can understand. The variable root now represents our <inventory> tag.
  • Finally, root.iter('*') acts like a scanner. It goes through absolutely every element inside the tree, no matter how deep, and item.tag prints the name of the tag.

Run your script in the terminal:

python main.py
Enter fullscreen mode Exit fullscreen mode

The output will show you the exact structure of your XML document:

inventory
server
hostname
os
ram
status
server
hostname
os
ram
status
...
Enter fullscreen mode Exit fullscreen mode

This is great for exploring unknown XML files, but usually, we want to extract the actual data, not just the tag names. We'll do that in the next step.

Extracting Specific Data with f-strings

Now that we know the structure of our XML file, let's extract the actual values (like the hostname and RAM) instead of just printing tag names.

Open main.py and replace the previous code with this updated version:

import requests
import xml.etree.ElementTree as ET

url = 'https://args.tech/media/uploads/3112c7f5-8f48-4cca-994d-9b3f326f0cf4.xml'
response = requests.get(url)
root = ET.fromstring(response.content)

# Loop only through the <server> blocks
for item in root.iterfind('server'):
    # Extract the text from specific child tags
    hostname = item.findtext('hostname')
    os_name = item.findtext('os')
    ram = item.findtext('ram')
    status = item.findtext('status')

    # Print the data using a modern Python f-string
    print(f'Server: {hostname} | OS: {os_name} | RAM: {ram} | Status: {status}')
Enter fullscreen mode Exit fullscreen mode

What changed here?

  • iterfind('server'): Instead of scanning every single tag in the document, we tell Python to only loop through the <server> blocks.
  • findtext('tag_name'): This is a very handy method. It searches inside the current block for a specific tag (like hostname) and immediately returns the text inside it.
  • f'...' (f-strings): Notice the f before the string in the print() function. This is a modern Python feature that allows you to inject variables directly into a string by wrapping them in curly braces {}. It makes your code much easier to read compared to older string formatting methods.

Run the script again:

python main.py
Enter fullscreen mode Exit fullscreen mode

You will receive a clean, readable list of your servers:

Server: web-node-01 | OS: Ubuntu 24.04 | RAM: 16GB | Status: Active
Server: db-node-01 | OS: Debian 12 | RAM: 64GB | Status: Active
Server: cache-node-01 | OS: Alpine Linux | RAM: 8GB | Status: Maintenance
Server: backup-node-01 | OS: Ubuntu 24.04 | RAM: 32GB | Status: Offline
Enter fullscreen mode Exit fullscreen mode

Bonus: Basic Error Handling

When working with network requests, things don't always go according to plan. A website might be down, or the URL might be wrong. If we try to parse an empty or error response as XML, our Python script will crash.

To prevent this, it is a good practice to check the HTTP status code before parsing. A status code of 200 means "OK" (success). Let's add a simple check to our code:

import requests
import xml.etree.ElementTree as ET

url = 'https://args.tech/media/uploads/3112c7f5-8f48-4cca-994d-9b3f326f0cf4.xml'
response = requests.get(url)

# Proceed only if the request was successful
if response.status_code == 200:
    root = ET.fromstring(response.content)

    for item in root.iterfind('server'):
        hostname = item.findtext('hostname')
        status = item.findtext('status')
        print(f'Server: {hostname} is {status}')
else:
    # Print an error message if something went wrong
    print(f'Failed to fetch data. HTTP Status Code: {response.status_code}')
Enter fullscreen mode Exit fullscreen mode

Now your script is much more robust and ready for real-world scenarios!

Conclusion

In this tutorial, you learned how to download structured data from the web using the requests library and parse it using Python's built-in xml.etree.ElementTree module. You also learned how to navigate the XML tree, extract specific text values, and format your output using modern f-strings.

You can view or download the XML file used in this guide here: demo.xml

Top comments (2)

Collapse
 
mohammadraziei profile image
Mohammad Raziei •

As I am the maintainer of the pygixml package, I invite you to try it. It is up to 14 times faster than ElementTree of the built-in Python XML parser, and is the lightest one on PyPI.

Collapse
 
xinitd profile image
Shakhzhakhan Maxudbek •

I'll definitely try it. Thank you!