DEV Community: Ananyashree

Mid-internship project report

Ananyashree — Thu, 09 Feb 2023 11:39:51 +0000

Hello and welcome to the fourth blog for Outreachy. This blog is a mid-point report of the project. It has been in the drafts for too long thanks to my perception that it is not good enough. I guess it is finally time to talk about the project without the hindrance of the thought about the quality of conveyance.

1. What was my original project timeline?

The original timeline at the starting of the internship was to create DVC pipelines and implement CML action for GCBM Carpathians. It was divided among time to learn about MLOps, followed by a few couple of weeks each for DVC pipelines and CML workflow implementation.

2. What have I accomplished in the first half of the internship?

In the first half, I focused on creating a DVC pipeline for GCBM Carpathians. It currently consists of 6 stages which were put into the pipeline through DVC. Each stage in the dvc.yaml file, consists of the command to be executed for that stage, the working directory where the command is to be executed, dependencies for the stage and the outputs. The outputs of each stage are tracked by DVC and hence, are not by git (unless used with --force).
To execute the DVC pipeline in a particular order, one can provide the output of one stage as the input for the next stage in the pipeline. I used log files from each stage as its output. Hence, with the command dvc repro, the user can now recreate or reproduce the pipeline and execute all the stages in a Standalone GCBM model. With the help of DVC, one can create a pipeline where one can monitor how different the stages vary if the dependencies change. One can also track datasets and results with DVC. This helps us to keep bulky datasets or results out of our git repository while only keeping a lightweight metadata in git. Besides pushing the outputs of a stage, one can also directly upload the same to a remote storage to be tracked by DVC. The six stages of the DVC pipeline are shown below:

3. What project goal took longer than expected?

It took me a few days to get all the dependencies right so that GCBM Carpathians simulation was running, along with latest DVC. There were many conflicts which had to be resolved while working with python 3.8+. Eventually, after some testing, I settled for installing dependencies through the wheel files originally released with GCBM Carpathians.

4. What would you do differently if you were starting the project over?

I would probably not do anything differently. I had very limited knowledge as I started the project. It took me time and effort to experiment and learn the skills required for the project. If I were to start the project again, with the same limited knowledge, I would probably go down the same path. The only difference would be to communicate with my mentors more as I was highly unsure of my capabilities in the beginning.

5. Plan for the second half of the internship?

The plan for the next half remains the same. I am currently working on a CML report for GCBM Carpathians which will help plot figures and provide metrics with different configurations. To create reports through CML action, GCBM Carpathians requires a Postprocessing step that summarizes the results. I have added this Postprocessing analysis to the same.

Think about your audience

Ananyashree — Thu, 12 Jan 2023 17:21:21 +0000

Hello and welcome to the third blog for Outreachy. We are almost done with half of the internship already! Times flies by fast doesn’t it. After being with moja global for over a month, I am in a great position to compile the gist of moja global as a community along with talking about my project. Let’s get started!

1. What does moja global do?

As a community, moja global’s mission is to develop open-source monitoring, reporting and verifying (MRV) software for forestry, agriculture and other land use (AFOLU) sectors. We aim to provide an accurate and affordable estimate of greenhouse gas emissions and its removal from AFOLU sectors.

The next question comes - why do we need to do so? You see, the AFOLU sector accounts for about 25% of global greenhouse gas emissions. When we talk about climate change and achieving net neutrality, it is important to reduce emissions and improve sinks in AFOLU sectors.

To put it in brief:

"Moja global came into being, by building a community of experts, scientists and developers, to create the first open-source source software that is affordable, accurate and fully customizable to a user’s needs and available data, whether the user is a country, a region, a local government, a company or a project planner."

2. What is my project?

To understand my project, let me first briefly explain the software developed by moja global.

FLINT (Full Lands Integration Tool) is moja global’s flagship software. It is an open-sourced tool which provides countries the ability to build and run advanced MRV system quickly and efficiently and GCBM (generic Carbon Budget Model) is an open-sourced model that operates on the FLINT platform. GCBM assesses and reports the cumulative effects of anthropogenic and natural disturbances on our forests.

Now for my project. Simply put, the objective of the project is to adopt the reproducibility and resilience of MLOps in GCBM models through DVC and CML. A lot of new words at once, eh? Yeah, I was in a similar space at the beginning of the internship. While we will talk about these terms in detail in following blogs, let me define a few, briefly, here.

A. MLOps (Machine Learning Operations)

The formal definition of MLOps states it as the extension of the DevOps methodology to include Machine Learning and Data Science assets as first-class citizens within the DevOps ecology. In other words, MLOps provide us with capabilities to deploy, monitor, manage and govern ML models in a production environment.

B. DVC (Data Version Control)

DVC is a tool developed by iterative.ai that helps manage data and ML models making them shareable and reproducible. DVC helps us use remote storage for data and models. Hence, we do not need to keep large amounts of data and gigantic models in our git repositories. When we wish to work with or modify these files, we can pull them from the storage through DVC. Moreover, DVC can manage pipelines for ML projects to make the experiments reproducible for say, all members. These lightweight pipelines list names of stages, their dependencies, their outputs, etc and are created using dependency graphs.

Now you have a general idea of what the project is about, including a high-level view of how DVC can be used for ML projects. This is exactly what I have done as the first task for my project! I used DVC for keeping the large output files in a remote storage, for easier sharing of files, along with creating a pipeline for the numerous stages in GCBM models to make the experiments reproducible.

3. What makes me most excited to work on this project?

One of the major reasons I was excited to work with moja global is their objective. I had been looking for opportunities to work on a project or with an organisation that primarily works towards climate change. Moja global provides me with the same. On top of that, the project itself helps me learn how to generate business value out of ML models. So, it is an exciting opportunity towards gaining a better understanding of a career in this field!

Outreachy - Introduce Yourself

Ananyashree — Sat, 10 Dec 2022 11:10:40 +0000

Hey there! 🌻✨

I’m Ananyashree. I am a graduate in Computer Science & Engineering from India and am currently an intern with Moja Global for Outreachy December 2022 cohort. This internship is an exciting opportunity for me to explore, build skills and push myself to learn!

Starting off with my interests. My interests lie in Artificial Intelligence, Computer Vision and the potential use of same for tackling Climate Change. Also, I like hiking, long walks, reading, photography and astronomy.

I would describe myself with three core values: Continuous Learning, Hard Work and Respect

Learning never really ends. And it’s really fun! While curiosity and continuous learning have some obvious benefits in my professional life, personally, they broaden my mindset and help me become a better person. For me, learning makes life fulfilling. Moreover, the excitement to know about so many new things drives me to learn continuously.

Next, I have always valued hard work. It has helped in difficult times and got me going when I thought I would never make it through a seemingly impossible challenge. You see, hard work never goes to waste. You either get what you wanted or you gain a ton of experience & learn. Either way, you emerge victorious!

Finally, being respectful goes a long way. In both professional and personal bonds, it builds strong foundations and sense of community. Being respectful not only helps me build a safe place for others but also for myself. While not perfect, I strive to be better at it every day.

Being an Outreachy intern with moja global is an exciting opportunity for me. Through their goal, I get insights into what can be done for societies and climate change. Not only will this internship help me build confidence but also aid in gaining experience in this field. I look forward to learning and growing with this amazing community!