Explore how countries produce energy for MongoDB Atlas Hackathon

Overview of My Submission

The climate change is an important topic we should all care to protect our planet. So, I wanted to create a simple web application to explore how countries produce energy, if they rely on renewable energy or fossil fuels.

To visit my web application click here! Similarly to the TV show Our Planet, I wanted to call my app Our Energy because it is a resource we all should be care of!

Below you can see the homepage:

You can look the amount of MegaWatts of energy produced by a country. To find the countries in the Mongo Database I used Atlas Search service. Below you can see the values for my country, Italy:

Moreover, from the homepage you can explore the top 5 countries producing energy using a particular renewable source. You can choose among solar, wind, waves and tidal, and hydro. Below there is a screenshot for the wind energy:

Data preparation

The original dataset was in .csv format. Firstly I removed many columns that were not necessary to my scope to reduce the size. Then I converted it to a JSON file which is easy to be uploaded to MongoDB Atlas.

Connecting to Atlas Cloud is as easy as connecting to any local database thank you to this official guide. To upload the dataset onto Mongo Cloud I used literally one function: insert_many(). You can explore the full code here.

Atlas Search

To search data by country I had to match the input string with the country_long field of the dataset. I used the Atlas UI to create an index with static mapping for this field. You can refer to this guide to create one.

Aggregation pipeline

The dataset consists of many power plants located in many countries. To retrieve a summary information for a certain country I exploited the power aggregation pipeline feature of MongoDB. Look at this code snippet that I use to find the 5 countries producing the biggest quantity of energy from a particular fuel (oil, gas, wind, solar, etc..., renewable energy sources are considered fuels as well):

result = collection.aggregate([
      {
         '$match': {
            'primary_fuel': fuel
         }
      },
      {
         # Accumulate capacity by fuel type
         '$group':
         {
            '_id': '$country_long',
            'totCapacity': {
               '$sum': {'$toDecimal': '$capacity_mw'}
            }
         }
      },
      {
         '$sort': {'totCapacity': -1}
      },
      {
         '$limit': 5
      }
   ])

A pipeline is made of many stages that performs different operations to extract the data we want from the database. The output of a stage is the input of the next stage.

1. The first step find all the entries whose field 'primary_fuel' matches the string fuel:

{
    '$match': {
        'primary_fuel': fuel
    }
},

1. This step groups the output of the previous step by country (using the field country_long, the country name) and accumulate the energy production of a power plant. Up to now, we know the amount of wind energy produced by each country.

{
     '$group':
     {
        '_id': '$country_long',
        'totCapacity': {
           '$sum': {'$toDecimal': '$capacity_mw'}
        }
     }
},

1. We sort the output in descending order

{
    '$sort': {'totCapacity': -1}
},    

1. Eventually, we limit the output to only 5 results

{
     '$limit': 5
}

And in the web app we can see the Top 5 countries producing that kind of energy. For instance, if you want to discover who are the 5 biggest producer of energy from waves and tydal are:

Conclusion

It was exciting to produce a simple web application from scratch to production using Python and MongoDB.

If you want to contribute to my project or propose some improvements, visit the GitHub repository.

Submission Category:

Choose Your Own Adventure

GitHub repository

andregri / OurEnergy

Climate Care is Flask application made for the MongoDB Atlas Hackathon 2021

🌍 OurEnergy

OurEnergy is a web application to explore some meaningful data about the Global Power Plant Dataset. You can explore either how each country is producing energy or little charts comparing the top 5 countries that are distinguishing for producing green energy 🌱

Live Demo

🔧 Tech Stack

• Python Flask
• MongoDB Atlas
• Heroku

🔌 Run Locally

Preparation of the dataset on MongoDB

Environment variables:

export MONGODB_URI='<your connection string to MongoDB'
export GPPDB_PATH='path/to/dataset/csvfile' # Do not write the extension '.csv'

For example my GPPDB_PATH env variable for the file ./data/global_power_plant_database.csv is:

export GPPDB_PATH='./data/global_power_plant_database'

Finally you can run two scripts to convert the dataset to JSON and to upload it to MongoDB Atlas:

python db_preparation/convert.py
python db_preparation/upload.py --cloud

Run Flask app

FLASK_ENV=development FLASK_DEBUG=true FLASK_APP=app flask run

Deploy to production

Create a .env file with the following variables:

export MONGODB_URI='<your

…

View on GitHub

Resources

• Global Power Plant Database

Comments

[Andrea Grillo]

Thank you! Indeed, my goal was to keep the design simple with no external frameworks. It was the hardest part for me because I almost never design web ui 😅

[MariaZentsova]

Really great and unusual design, and the topic is very relevant 🔥!

[Andrea Grillo]

I appreciate it! I'd like to correlate this information with other data such as the global temperature change over the years

[Benoît Durand]

Good job !