DEV Community ๐Ÿ‘ฉโ€๐Ÿ’ป๐Ÿ‘จโ€๐Ÿ’ป

Andrea Grillo
Andrea Grillo

Posted on • Updated on

Explore how countries produce energy for MongoDB Atlas Hackathon

Overview of My Submission

The climate change is an important topic we should all care to protect our planet. So, I wanted to create a simple web application to explore how countries produce energy, if they rely on renewable energy or fossil fuels.

To visit my web application click here! Similarly to the TV show Our Planet, I wanted to call my app Our Energy because it is a resource we all should be care of!

Below you can see the homepage:

homepage

You can look the amount of MegaWatts of energy produced by a country. To find the countries in the Mongo Database I used Atlas Search service. Below you can see the values for my country, Italy:

italy energy production

Moreover, from the homepage you can explore the top 5 countries producing energy using a particular renewable source. You can choose among solar, wind, waves and tidal, and hydro. Below there is a screenshot for the wind energy:

top 5 wind

Data preparation

The original dataset was in .csv format. Firstly I removed many columns that were not necessary to my scope to reduce the size. Then I converted it to a JSON file which is easy to be uploaded to MongoDB Atlas.

Connecting to Atlas Cloud is as easy as connecting to any local database thank you to this official guide. To upload the dataset onto Mongo Cloud I used literally one function: insert_many(). You can explore the full code here.

Atlas Search

To search data by country I had to match the input string with the country_long field of the dataset. I used the Atlas UI to create an index with static mapping for this field. You can refer to this guide to create one.

Aggregation pipeline

The dataset consists of many power plants located in many countries. To retrieve a summary information for a certain country I exploited the power aggregation pipeline feature of MongoDB. Look at this code snippet that I use to find the 5 countries producing the biggest quantity of energy from a particular fuel (oil, gas, wind, solar, etc..., renewable energy sources are considered fuels as well):

result = collection.aggregate([
      {
         '$match': {
            'primary_fuel': fuel
         }
      },
      {
         # Accumulate capacity by fuel type
         '$group':
         {
            '_id': '$country_long',
            'totCapacity': {
               '$sum': {'$toDecimal': '$capacity_mw'}
            }
         }
      },
      {
         '$sort': {'totCapacity': -1}
      },
      {
         '$limit': 5
      }
   ])
Enter fullscreen mode Exit fullscreen mode

A pipeline is made of many stages that performs different operations to extract the data we want from the database. The output of a stage is the input of the next stage.

  1. The first step find all the entries whose field 'primary_fuel' matches the string fuel:
{
    '$match': {
        'primary_fuel': fuel
    }
},
Enter fullscreen mode Exit fullscreen mode
  1. This step groups the output of the previous step by country (using the field country_long, the country name) and accumulate the energy production of a power plant. Up to now, we know the amount of wind energy produced by each country.
{
     '$group':
     {
        '_id': '$country_long',
        'totCapacity': {
           '$sum': {'$toDecimal': '$capacity_mw'}
        }
     }
},
Enter fullscreen mode Exit fullscreen mode
  1. We sort the output in descending order
{
    '$sort': {'totCapacity': -1}
},    
Enter fullscreen mode Exit fullscreen mode
  1. Eventually, we limit the output to only 5 results
{
     '$limit': 5
}
Enter fullscreen mode Exit fullscreen mode

And in the web app we can see the Top 5 countries producing that kind of energy. For instance, if you want to discover who are the 5 biggest producer of energy from waves and tydal are:

top 5 wave

Conclusion

It was exciting to produce a simple web application from scratch to production using Python and MongoDB.

If you want to contribute to my project or propose some improvements, visit the GitHub repository.

Submission Category:

Choose Your Own Adventure

GitHub repository

GitHub logo andregri / OurEnergy

Climate Care is Flask application made for the MongoDB Atlas Hackathon 2021

๐ŸŒ OurEnergy

OurEnergy is a web application to explore some meaningful data about the Global Power Plant Dataset. You can explore either how each country is producing energy or little charts comparing the top 5 countries that are distinguishing for producing green energy ๐ŸŒฑ

Live Demo

OurEnergy Homepage

๐Ÿ”ง Tech Stack

  • Python Flask
  • MongoDB Atlas
  • Heroku

๐Ÿ”Œ Run Locally

Preparation of the dataset on MongoDB

Environment variables:

export MONGODB_URI='<your connection string to MongoDB'
export GPPDB_PATH='path/to/dataset/csvfile' # Do not write the extension '.csv'
Enter fullscreen mode Exit fullscreen mode

For example my GPPDB_PATH env variable for the file ./data/global_power_plant_database.csv is:

export GPPDB_PATH='./data/global_power_plant_database'
Enter fullscreen mode Exit fullscreen mode

Finally you can run two scripts to convert the dataset to JSON and to upload it to MongoDB Atlas:

python db_preparation/convert.py
python db_preparation/upload.py --cloud
Enter fullscreen mode Exit fullscreen mode

Run Flask app

FLASK_ENV=development FLASK_DEBUG=true FLASK_APP=app flask run

Deploy to production

Create a .env file with the following variables:

export MONGODB_URI='<your
โ€ฆ
Enter fullscreen mode Exit fullscreen mode

Resources

Top comments (6)

Collapse
 
andreagrillo96 profile image
Andrea Grillo

Thank you! Indeed, my goal was to keep the design simple with no external frameworks. It was the hardest part for me because I almost never design web ui ๐Ÿ˜…

Collapse
 
mariazentsova profile image
MariaZentsova

Really great and unusual design, and the topic is very relevant ๐Ÿ”ฅ!

Thread Thread
 
andreagrillo96 profile image
Andrea Grillo

I appreciate it! I'd like to correlate this information with other data such as the global temperature change over the years

Collapse
 
duranbe profile image
Benoรฎt Durand

Good job !

We are hiring! Do you want to be our Senior Platform Engineer? Are you capable of chipping in across sysadmin, ops, and site reliability work, while supporting the open source stack that runs DEV and other communities?

This role might just be for you!

Apply now