DEV Community: La Tasha "L." Pollard

An NBA Player Data Lake automation script, leveraging AWS S3, Glue, and Athena.

La Tasha "L." Pollard — Sat, 08 Feb 2025 22:21:25 +0000

Introduction

This is the Week 1 Day 3 Project of the 30-Day DevOps challenge I am participating in. Learn more about the challenge and it's creators.

Project Overview

In today's project, we are implementing a data lake to ingest, store, and manage the high volume, complex data that we will retrieve from our API call to the NBA API.

What's a Data Lake?

A data lake is kind of like a database in the sense that we store large amount of data in both. The difference is that, whereas a database is structured (tables, schemas, relationships), a data lake can handle large volumes of all of kinds data regardless of format(structured, semi-structured, unstructured), and your data doesn't need to organized first.

We can think of a database like a organized filling cabinet: when we add to it, we add the file to the correct location based on the schema of the cabinet.

Whereas a data lake would be more like a storage warehouse where we can add any kind of data. We don't have to organize it and put it in the correct location, just dump it on in there.

But what good for us is a whole bunch of unprocessed, unorganized data? So in our project we are gathering all this data, storing it in a data lake in an S3 bucket, and then using other AWS services (e.g. Glue and Athena) to make the data queryable.

Key Features

Fetches NBA player data using SportsDataIO's NBA API.
Stores raw and processed data in AWS S3 buckets.
Uses AWS Glue Crawlers to infer schema from raw data in S3.
Stores metadata in AWS Glue Data Catalog for structured querying.
Allows SQL-based querying of structured data stored in S3.
Implements IAM roles and policies to control access to S3, Glue, and Athena.

Technical Architecture

Technologies

Cloud Provider: AWS
Core Services: S3, Glue, Athena
External API: NBA Game API (SportsData.io)
Programming Language: Python 3.x
IAM Security: Least privilege policies

NBA API

We are making a request to the NBA API, this time for data about NBA players. This is how I coded the request:

nba_endpoint = f"https://api.sportsdata.io/v3/nba/scores/json/Players?key={api_key}"
try:
    response = requests.get(nba_endpoint)
    response.raise_for_status()  
    print("Fetched NBA data successfully.")
    return response.json()  
except Exception as e:
    print(f"Error fetching NBA data: {e}")
    return []

The response looks something like this:

[
  {
    "PlayerID": 20000441,
    "SportsDataID": "",
    "Status": "Active",
    "TeamID": 29,
    "Team": "PHO",
    "Jersey": 3,
    "PositionCategory": "G",
    "Position": "SG",
    "FirstName": "Bradley",
    "LastName": "Beal",
    "Height": 76,
    "Weight": 207,
    "BirthDate": "1993-06-28T00:00:00",
    "BirthCity": "St. Louis",
    "BirthState": "MO",
    "BirthCountry": "USA",
    "HighSchool": null,
    "College": "Florida",
    "Salary": 50203930,
    "PhotoUrl": "https://s3-us-west-2.amazonaws.com/static.fantasydata.com/headshots/nba/low-res/0.png",
    "Experience": 12,
    "SportRadarPlayerID": "ff461754-ad20-4eeb-af02-2b46cc980b24",
    "RotoworldPlayerID": 1966,
    "RotoWirePlayerID": 3303,
    "FantasyAlarmPlayerID": 200464,
    "StatsPlayerID": 606912,
    "SportsDirectPlayerID": 750970,
    "XmlTeamPlayerID": 3395,
    "InjuryStatus": "Scrambled",
    "InjuryBodyPart": "Scrambled",
    "InjuryStartDate": "2025-02-06T00:00:00",
    "InjuryNotes": "Scrambled",
    "FanDuelPlayerID": 15595,
    "DraftKingsPlayerID": 606912,
    "YahooPlayerID": 5009,
    "FanDuelName": "Bradley Beal",
    "DraftKingsName": "Bradley Beal",
    "YahooName": "Bradley Beal",
    "DepthChartPosition": "SG",
    "DepthChartOrder": 6,
    "GlobalTeamID": 20000029,
    "FantasyDraftName": "Bradley Beal",
    "FantasyDraftPlayerID": 606912,
    "UsaTodayPlayerID": 8315651,
    "UsaTodayHeadshotUrl": "http://cdn.usatsimg.com/api/download/?imageID=24445236",
    "UsaTodayHeadshotNoBackgroundUrl": "http://cdn.usatsimg.com/api/download/?imageID=24445234",
    "UsaTodayHeadshotUpdated": "2024-10-09T14:17:10",
    "UsaTodayHeadshotNoBackgroundUpdated": "2024-10-09T14:17:04",
    "NbaDotComPlayerID": 203078
  },
...
]

That's like 50 lines of code for one player! ..Googles the number of current NBA players, multiples times 50.. Yea, this is good introductory example of a large data set.

S3

AWS S3 - Simple Storage Solution - will be our storage layer. The raw data retrieved from the API call will be stored in our S3 bucket. The processed data (after being processed by Glue) is also stored in an S3 bucket.

Glue

AWS Glue is a serverless data integration service that includes tools for ETL (Extract, Transform, Load) jobs and a Data Catalog for metadata management. We will use Glue Crawler to process the raw data we are storing in our S3 bucket and Glue Data Catalog to store and manage metadata about the structured data. This will make the data accessible for querying using AWS Athena.

Athena

AWS Athena is a serverless querying service used to analyze data. We will use it to run SQL queries on the data in our S3 bucket after its been processed by AWS Glue.

A helpful analogy:

S3 = The Library
- Where all the raw and processed data is stored.
Glue Data Catalog = The Library's Catalog
- A system that organizes and records information about the books/data stored in S3.
Athena = A Reader
- Who looks at the catalog (Glue Data Catalog) to find and understand the structure of the books (data) before reading them directly from the library (S3))

Python Script

The creation of these resources could be done manually through the AWS Console, however this project automates this process by preparing python code, utilizing the boto3 library. The script will be copy-and-pasted into the AWS CloudShell terminal and executed to provision the necessary AWS resources, including fetching data from the API, creating the S3 bucket, configuring the Glue Crawler, updating the Glue Data Catalog, and setting up Athena for querying the stored data.

Project Structure

nba-data-lake/
├── .env                             # holds environment variables
├── policies/        
│   └── IAM_role.json                # json for policy permissions
├── src/
│   ├── setup_nba_data_lake.py       # main script
│   ├── delete_nba_data_lake.py      # script to delete resources
├── README.md                        # documentation
└── requirements.txt                 # dependencies

Set Up Instructions

Prerequisites

AWS Account with the following permissions:
- S3: CreateBucket, PutObject, DeleteBucket, ListBucket
- Glue: CreateDatabase, CreateTable, DeleteDatabase, DeleteTable
- Athena: StartQueryExecution, GetQueryResults
NBA API key

1. Clone the Repo

git clone <url>

2. Go to the AWS Console, and click the CloudShell icon to open the terminal.

3. Type `nano setup_nba_data_lake.py` into the shell console and press enter.

4. Copy the code from our repo file: `src/setup_data_lake.py` to the shell console.

Update the api_key variable to your actual key
Update the nba_endpoint variable.
Update the bucket_name variable.
Update the region variable (if necessary).
Hit crtl + x to exit, then y to save.
Hit enter to confirm the name.

5. Type `python setup_nba_data_lake.py` and run the code.

6. Manually check for the resources.

Go to S3 and you should see the new bucket with 3 objects inside of it.
Click on raw-data and open the file inside of it.

7. Query the data with Athena.

Go to Athena and paste the sample query

SELECT FirstName, LastName, Position, Team
FROM nba_players
WHERE Position = 'PG';

Click run and you should see output under "Query Results"

8. Delete Resources

Navigate to the CloudShell console.
Type nano delete_aws_resources
Copy-and-paste the content from the file src/delete_aws_resources into the console.
Update the bucket_name variable to match the bucket created.
Hit crtl + x to exit, then y to save.
Hit enter to confirm the name
Type python delete_aws_resources into the console and hit enter.
Manually confirm resources have been deleted.

Today I learned...

What data lakes are and how they differ from databases
How to create a python script to automate the provisioning of AWS resources
What the services AWS Glue and AWS Athena are
How to use Glue Crawler and Glue Data Catalog to process data
How to use Athena to query data

Future Enhancements

I am writing this after reaching the initial goal of this challenge: to automate the creation of a data lake using AWS services.

If I have the capacity I will enhance this app by integrating a data visualization dashboard that connects to AWS Athena. I could also add logic to include NFL player data.

And if you've made it this far, thanks for reading! Feel free to checkout my GitHub. I'd love to connect on LinkedIn.

The VSCode one looks awesome, I’m going to try it!

La Tasha "L." Pollard — Fri, 07 Feb 2025 02:00:24 +0000

10 Creative & Open Source Portfolio Templates

Justin3go ・ Feb 5

#webdev #opensource #portfolio #github

A Cloud-based NBA Game Day Alert System using Python, AWS Lambda, SNS, and EventBridge.

La Tasha "L." Pollard — Wed, 05 Feb 2025 18:14:17 +0000

Introduction

This is the Week 1 Day 2 project of the 30-Day DevOps challenge that I am participating in. Learn more about the challenge and its creators, s/o to them, here.

Project Overview

This project creates a cloud based NBA game day alert system leveraging Python, AWS Lambda, Amazon EventBridge, Amazon SNS, and an NBA API. NBA fans can subscribe to this service to receive real-time notifications of game scores via email or text message.

Key Features

Fetches NBA game scores using SportsDataIO's NBA API.
Sends score updates to subscribers via email or text message using Amazon SNS.
Automates sending of notifications using Amazon EventBridge.
Follows principle of least privilege for IAM roles.

Technologies

Cloud Provider: AWS
Core Services: SNS, Lambda, EventBridge
External API: NBA Game API (SportsData.io)
Programming Language: Python 3.x
IAM Security: Least privilege policies

NBA API

(The more I interact with, research, and write about APIs, the better I understand them. Have you heard the notion that you only really know something if you can explain in the simplest of terms? Well, I'm gradually feeling more confident in my ability to explain APIs and the HTTP request-response cycle and I am going to attempt to do so in a forthcoming post that I'll have linked here once its up.)

In this project, we make a request to the NBA API for data about games for the current day. The request looks something like this:

api_url = f"https://api.sportsdata.io/v3/nba/scores/json/GamesByDate/{todays_date}?key={your_api_key}"
try: 
  response = requests.get(api_url)
  response.raise_for_status()
  data = response.json()
  print (json.dumps(data, indent=4))
except Exception as e: 
  print (f"An error occurred: {e}")

With the call wrapped in a try-except block, the NBA API responds to our request with either an error message or with the data I requested. The data comes through in a format that is hard to decipher so we decipher it by using methods, e.g. .json() and json.dumps().

The formatted response looks something like this:

[
    {
        "GameID": 21678,
        "Season": 2025,
        "SeasonType": 1,
        "Status": "Scheduled",
        "Day": "2025-02-05T00:00:00",
        "DateTime": "2025-02-05T19:00:00",
        "AwayTeam": "MIL",
        "HomeTeam": "CHA",
        "AwayTeamID": 15,
        "HomeTeamID": 2,
        "StadiumID": 2,
        "Channel": "FDSS",
        "Attendance": null,
        "AwayTeamScore": null,
        "HomeTeamScore": null,
        ...
    },
   ... 
]

Note that each game's information is held in a dictionary. So I can extract what I want from that decoded data using the .get() method. For example, I can extract the home team with a line like:

home_team = game.get("HomeTeam", "Unknown")

This creates a variable home_team and sets it to the value associated with the key "HomeTeam" and if that key doesn't exist, home_team is set to "Unknown".

Thanks NBA API!

Event driven architecture

Today I learned... about event driven architecture. This project is an example of event driven architecture as we set up a cron job schedule in EventBridge that invokes the code we house in Lambda. The code in Lambda makes the API call, transforms the data, and sends it to SNS which then sends out the notifications. The event, the schedule we set in EventBridge, automatically kicks off the rest of the workflow.

EventBridge

EventBridge helps route events from different AWS services and custom applications. In this case, we use EventBridge to define a scheduled rule that triggers our Lambda function at specific intervals. This eliminates the need for manual execution and ensures the workflow runs consistently.

Lambda

Lambda runs the code we write to fetch data from the NBA API, process it, and send it out, in this case to an SNS topic.

SNS

SNS, Simple Notification Service, is a pub-sub service that sends notifications to its subscribers. People can subscribe to a topic and receive notifications by text and/or email.

Project Structure

game-day-notifications/
├── .env.                            # holds environment variables
├── policies/        
│   └── gd_sns_policy.json           # json for the sns policy
├── src/
│   ├── game_day_notifs.py           # main logic
├── README.md                        # documentation
└── requirements.txt                 # dependencies

Set Up Instructions

Follow these steps to create the workflow for this app.

Clone the Repo

git clone https://github.com/LilLoveBytes/game-day-notifications.git
cd game-day-notifications

Create an SNS Topic

Open the AWS management console.
Navigate to the SNS service.
Click Create Topic and select Standard as the topic type
Name the topic (e.g. GameDay) and note the ARN.
Click create topic.

Add Subscriptions to the SNS Topic

After creating the topic, click on the topic name from the list.
Navigate to the Subscriptions tab and click Create subscription
Select a Protocol:
- Email: Choose Email and enter a valid email address
- SMS: Choose SMS and enter a valid phone number in international format (e.g. +1-213-555-5555)
Click Create Subscription.
Confirm the subscription
- Email: Check the inbox of this account and confirm the subscription by clicking the confirmation link.
- SMS: Subscription will be active after creation, no need to further confirm.

Create the SNS Publish Policy

Open the IAM service in the AWS Management Console.
Navigate to Policies -> Create Policy
Click JSON and paste the JSON policy from the gd_sns_policy.json file in the repo
Update the value of the "Resource" key to the ARN noted earlier.
Click Next: Tags (you can skip adding tags).
Click Next: Review
Enter a name for the policy
Review and click Create Policy.

Create an IAM Role for Lambda

Open the IAM service in the AWS Management Console.
Click Roles -> Create Role.
Select AWS Service and choose Lambda.
Attach the following policies
- SNS Publish Policy (created in the previous step).
- Lambda Basic Execution Role (AWSLambdaBasicExecutionRole) (an AWS managed policy).
Click Next: Tags (you can skip adding tags).
Click Next: Review
Enter a name for the role
Review and click Create Role.
Copy and save the ARN of the role for use in the Lambda function.

Deploy the Lambda Function

Open the AWS Management Console and navigate to the Lambda service.
Click Create Function.
Select Author from Scratch.
Enter a function name
Choose Python 3.x as the runtime.
Assign the IAM role created earlier to the function.
Under the Function Code section:
- Copy the content of the src/game_day_notifs.py file from the repository.
- Paste it into the inline code editor.
Under the Environment Variables section, add the following:
- NBA_API_KEY: your NBA API key.
- SNS_TOPIC_ARN: the ARN of the SNS topic created earlier.
Click Create Function

Set up Automation with EventBridge

Navigate to the EventBridge service in the AWS Management Console.
Go to Rules -> Create Rule.
Select Rule Type: Schedule.
Select Continue in EventBridge Scheduler.
Name your schedule and select Recurring Schedule
Set the cron schedule for when you want updates
Under Targets, select the Lambda function you just created
Select Create a new role and hit next.

Test the System

Open the Lambda function in the AWS Management Console.
Create a test event to simulate execution.
Run the function and check CloudWatch Logs for errors.
Verify that notifications are sent to the subscribed users.

Conclusions

Today I learned...

what event driven architecture is
what AWS Lambda, SNS, and EventBridge are.
how to design a notification system with SNS and Lambda.
how to use AWS services like EventBridge and Lambda to build automated workflows.
how to secure AWS services with least privilege IAM policies.
how to automate workflows using EventBridge.
how to format a cron expression to define scheduled execution times.
how to integrate an external API into a cloud-based workflow.

Future Enhancements

I am writing this after reaching the initial goal of this challenge: to create a simple notification system that fetches NBA game data from an API, formats the data, and sends it to subscribers.

If I have the capacity I will enhance this app by creating a webpage/ user interface for user interaction and data visualization. I could also add logic to include NFL game data notifications.

And if you've made it this far, thanks for reading! Feel free to checkout my Github. I'd love to connect on Linkedin.

Hello, world! (Can you tell I'm new here?)

La Tasha "L." Pollard — Tue, 04 Feb 2025 22:22:30 +0000

Hi, hello there! I'm new here (this is one of my first post, excitement). Instead of over analyzing every single word -- and ultimately feeling like I'll never find the perfect combination of words-- I am going to treat this as what it is, a blog. I'm here to document my learning, but the educator in me hopes this aids in your learning process in some way too.

About Me

I'm a cough-cough year old, career changing, aspiring Software/DevOps Engineer. I come from a background in education and social justice work. I completed a coding bootcamp with Ada Developers Academy in 2024 and worked briefly for a FAANG+ company before being affected by company wide layoffs -- I was impacted by layoffs five months into my first full time role, sad face. Since then I've been taking my time to up skill, build out my portfolio, and practice DSAs (sadder face).

Why this blog?

"Have you ever played have you ever?" - Kendrick Lamar

Have you ever read a book or watched a show and walked away feeling like it was one of the best things ever, and then maybe a month or so later, a friend starts said media from the beginning, wants to converse with you about it, begins to tells you things and you're like... "Oh yea.. that did happen!" ?

That's analogous to my journey in tech: I finished a coding bootcamp, completed a Software Engineering internship with a FAANG+ company, was hired full time by the company... But when it came to reflectively talking about the work I was doing, I felt like Spongebob when he couldn't express what he learned in boating school...

I found myself in interviews and other conversations drawing a blank on what it is I've actually done. I was learning enough to complete assigned tasks but weeks later I was left with only the gist of what I had done. And when it came time to talk in depth about said tasks, like in an interviewwwwugh, the details were even further removed from my memory.

Ayo, why am I not remembering everything I've ever done, scratches head?? It's partially because our memories don't work like that and partially because I wasn't documenting my process enough.

Now, I make notes about everything I do, so later down the line, when it's not fresh in memory, I can review my notes and be reminded, "Oh yea... I did do... all of that".

(When I say document, I just mean utilizing some note-taking method: summarize my task, make bullet points highlighting the action plan, screenshot error messages, note what did and didn't resolve the issues, etc..)

Note-taking, documenting, and reviewing were underrated, but now are essential, intentional parts of my learning process. Translating my notes/documentation into a blog helps all these new concepts sink in.

So I'm creating this blog as part of my learning process and to document my learning journey. It's a way for me to process each new thing I learn, each project I work on, document it, and later review and reflect as necessary.

Maybe this blog will showcase my understanding of these concepts to potential employers. Maybe it'll prompt discussions that will help my understanding and thought processes evolve. Or maybe this is just a sounding board and I am posting to the void.

Either way, I'll be here :)

And if you've read this far along, thanks for sticking around! My name's La Tasha and you can find me on Linkedin and Github. Let's connect!

A weather data collection system using AWS S3 and OpenWeather API.

La Tasha "L." Pollard — Mon, 03 Feb 2025 23:39:57 +0000

Introduction

Me:

Hi, there! I'm new here (this is my first post, excitement). Instead of over analyzing every single word -- and ultimately feeling like I'll never find the perfect combination of words-- I am going to treat this as what it is, a blog. I'm here to document my learning, but the educator in me hopes this aids in your learning process in some way too.

I have experience in Full Stack development and have gotten my toes wet with some DevOps tools during my internship at Microsoft. To expound upon that and gain more familiarity with cloud computing, I recently joined a 30-Day DevOps coding challenge. Our Week 1 Day 1 challenge is to create a weather data collection system using AWS S3 and OpenWeather API. This blog will cover the steps I took to create this app.

Project Overview

The weather dashboard app is a python application that utilizes the OpenWeather API to fetch real-time weather data for multiple cities, displays that data in a readable format in the terminal, and automates storage of the data to an AWS S3 bucket. Storing our real-time weather insights in scalable cloud storage buckets makes the data more accessible and persistent.

Key Features

Fetches real time weather data for specified locations using OpenWeather API.
Displays temperature, humidity, and weather conditions.
Automatically stores weather data in AWS S3.
Uses environment variables to securely manage API keys.
Timestamps all data for historical tracking.

OpenWeather API

In order to dynamically fetch data or populate a web page or web app, I need to make an API call to get the data I need from other systems or services.

Since my project needs to dynamically provide feedback about the weather, I used OpenWeather API which retrieves real-time weather data, forecasts, and other climate-related information for a specified location.

Here's what it looks like to make a request:

base_url = "https://api.openweathermap.org/data/2.5/weather"
        params = {
            "q": "city-name",
            "appid": "your-api-key",
            "units": "imperial"
        }
        try:
            response = requests.get(base_url, params=params)
            response.raise_for_status()
            return response.json()
        except requests.exceptions.RequestExceptions as e:
            print(f'Error fetching weather data: {e}')
            return None

I wrap my request is in a try-except block for error handling. If for any reason my request fails, rather than the program simply exiting (or doing some other not-detailed action), I'll get a response with details about the failure that will aid in debugging.

Data Storage with AWS S3

This project was my first time using AWS. So in order to get started I had to set up my AWS account (using the free tier), install the AWS CLI, configure the CLI to use SSO (single sign on which is a means of authentication), learn what Amazon S3 is, and learn how to use boto3, a python library, to create and save files to an S3 bucket, in a python script. Whew.

Briefly, S3 is Amazons cloud storage, short for Simple Storage Service.. we can save data things (files, images, etc.) there. SSO - Single Sign On - is a means of authentication for accessing AWS accounts. Boto3 is the python library we use to interact with AWS services and manage resources, like S3, using python code.

To use boto3 to create and save files to an S3 bucket looks something like this:

import boto3
import json

s3_client = boto3.Session(
            profile_name='pro-name', region_name='your-region').client('s3')

s3_client.create_bucket(Bucket='bucket_name', CreateBucketConfiguration={
                    'LocationConstraint': 'your-region'})


s3_client.put_object(
                Bucket='bucket_name',
                Key=file_name,
            Body=json.dumps(weather_data),
                ContentType='application/json'
            )

Project Structure

weather-dashboard/
  src/
    __init__.py            # initiates python 
    weather_dashboard.py   # Defines functions/logic
  .env                     # Holds environment variables
  requirements.txt         # Dependencies

Set up Instructions

These are the steps one could take to run my version of the app.

1. Clone the Repository

git clone https://github.com/LilLoveBytes/weather-dashboard.git
cd weather-dashboard

2. Install Dependencies

pip install -r requirements.txt

3. Configure Environment Variable

OPENWEATHER_API_KEY=your_api_key
AWS_BUCKET_NAME=your_bucket_name

4. Configure AWS Credentials

Make sure you have AWS CLI installed and run:

aws configure

5. Run the application

python src/weather_dashboard.py

Conclusion

insert spongebob meme

What I learned today is...

how to set up an AWS account using the free tier
what amazon S3 is
how to use boto3/python to manage S3 resources
how to make API calls to OpenWeather API
how to set up AWS CLI to use SSO

Further Enhancements

I am writing this after reaching the initial goal of this challenge: to create a simple app that fetches weather data, displays it to the terminal, and stores it to cloud storage.

If I have the capacity I will enhance this app by creating a webpage/ user interface for user interaction and data visualization. I will also use AWS Lambda to schedule automated data collection.

Let's Connect

You can view this project on my Github.
Also feel free to connect with me on LinkedIn.

DEV Community: La Tasha "L." Pollard

An NBA Player Data Lake automation script, leveraging AWS S3, Glue, and Athena.

Introduction

Project Overview

What's a Data Lake?

Key Features

Technical Architecture

Technologies

NBA API

S3

Glue

Athena

A helpful analogy:

Python Script

Project Structure

Set Up Instructions

Prerequisites

1. Clone the Repo

2. Go to the AWS Console, and click the CloudShell icon to open the terminal.

3. Type nano setup_nba_data_lake.py into the shell console and press enter.

4. Copy the code from our repo file: src/setup_data_lake.py to the shell console.

5. Type python setup_nba_data_lake.py and run the code.

6. Manually check for the resources.

7. Query the data with Athena.

8. Delete Resources

Today I learned...

Future Enhancements

The VSCode one looks awesome, I’m going to try it!

10 Creative & Open Source Portfolio Templates

Justin3go ・ Feb 5

A Cloud-based NBA Game Day Alert System using Python, AWS Lambda, SNS, and EventBridge.

Introduction

Project Overview

Key Features

Technologies

NBA API

Event driven architecture

EventBridge

Lambda

SNS

Project Structure

Set Up Instructions

Clone the Repo

Create an SNS Topic

Add Subscriptions to the SNS Topic

Create the SNS Publish Policy

Create an IAM Role for Lambda

Deploy the Lambda Function

Set up Automation with EventBridge

Test the System

Conclusions

Future Enhancements

Hello, world! (Can you tell I'm new here?)

About Me

Why this blog?

A weather data collection system using AWS S3 and OpenWeather API.

Introduction

Me:

Project Overview

Key Features

OpenWeather API

Data Storage with AWS S3

Project Structure

Set up Instructions

1. Clone the Repository

2. Install Dependencies

3. Configure Environment Variable

4. Configure AWS Credentials

5. Run the application

Conclusion

Further Enhancements

Let's Connect

3. Type `nano setup_nba_data_lake.py` into the shell console and press enter.

4. Copy the code from our repo file: `src/setup_data_lake.py` to the shell console.

5. Type `python setup_nba_data_lake.py` and run the code.