This guide outlines the steps to set up a Python-based system for collecting, storing, and analyzing real-time weather data. We’ll pull data from a weather API, store it in a structured format, and visualize trends, making this an adaptable solution for fields impacted by weather, such as agriculture, tourism, and event planning.
Step 1: Import Libraries
We’ll begin by importing essential libraries for data collection, storage, scheduling, and visualization:
python
import requests # For API data retrieval
import pandas as pd # For data storage and manipulation
import time # For scheduling data retrieval
import matplotlib.pyplot as plt # For data visualization
from datetime import datetime # For timestamping each data entry
Step 2: Configure the Weather API
Set up the connection to the weather API. This example uses weatherapi.com, but it can be adapted to other weather APIs by updating the endpoint and request parameters.
API Key: Replace 'your_api_key' with your API key from the provider.
City: Adjust the city variable to monitor different locations.
python
api_key = 'your_api_key'
city = 'Nairobi'
url = f"http://api.weatherapi.com/v1/current.json?key={api_key}&q={city}&aqi=no"
Step 3: Function for Data Retrieval
This function queries the API, extracts the weather information, and structures it into a dictionary. Each query returns the city name, temperature, humidity, wind speed, and a timestamp.
python
def fetch_weather_data():
response = requests.get(url)
data = response.json()
weather_info = {
'city': data['location']['name'],
'temperature': data['current']['temp_c'],
'humidity': data['current']['humidity'],
'wind_speed': data['current']['wind_kph'],
'timestamp': datetime.now()
}
return weather_info
Step 4: Automate Data Collection
Using a while loop, the script repeatedly fetches data every 10 minutes (600 seconds) and appends it to a DataFrame. The script will run continuously, logging new data with each interval.
python
weather_data = pd.DataFrame(columns=['city', 'temperature', 'humidity', 'wind_speed', 'timestamp'])
try:
while True:
# Fetch and append new data to the DataFrame
new_data = fetch_weather_data()
weather_data = weather_data.append(new_data, ignore_index=True)
# Display the latest data entries
print(weather_data.tail())
# Wait for the next data retrieval interval (10 minutes)
time.sleep(600)
except KeyboardInterrupt:
print("Data collection stopped.")
Step 5: Data Visualization
The following function generates a line plot displaying temperature, humidity, and wind speed changes over time. This step provides a visual representation of weather patterns.
python
import matplotlib.dates as mdates
def plot_weather_data(df):
plt.figure(figsize=(10, 5))
plt.plot(df['timestamp'], df['temperature'], label='Temperature (°C)')
plt.plot(df['timestamp'], df['humidity'], label='Humidity (%)')
plt.plot(df['timestamp'], df['wind_speed'], label='Wind Speed (kph)')
# Format x-axis for timestamp readability
plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d %H:%M'))
plt.gca().xaxis.set_major_locator(mdates.HourLocator(interval=1))
plt.gcf().autofmt_xdate()
plt.title(f'Real-time Weather Data for {city}')
plt.xlabel('Time')
plt.ylabel('Measurements')
plt.legend()
plt.show()
Call this plot function every 30 minutes for updates
while True:
plot_weather_data(weather_data)
time.sleep(1800)
Step 6: Data Storage for Historical Analysis
To store collected data, export it to a CSV file. This allows for historical weather analysis and trend evaluation.
python
Save data to a CSV file for long-term storage and analysis
weather_data.to_csv('weather_data.csv', index=False)
Optional: Setting Up Forecasting Capabilities
With an extensive dataset over days or weeks, consider implementing predictive models like ARIMA or Facebook’s Prophet to identify weather trends or create forecasts. This would involve additional libraries and configuration but can significantly enhance the project’s utility.
This setup provides a foundational framework for real-time weather monitoring and analysis, with opportunities for further customization to suit domain-specific requirements.
Top comments (0)