Problem Solving In A Team

Nicholas — Wed, 23 Jun 2021 19:54:59 +0000

Over this past month, I have had the pleasure to work along side a team of talented web developers and data scientists to polish and add features to an existing application that will be used by a non-profit organization called Human Rights First. The application will be used by asylum lawyers and human rights activists to determine biases that judges may have against certain groups of people who are seeking asylum to the United States.

The organization that this application is going to be used for is doing some admirable work. The application itself is very cool as well. But the thing that has peaked my interest this past month has been working on a cross-functional team to solve problems that actually have an impact to the project as a whole.

Discovering The Problem

In the first week of working on this project, my entire team was working on understanding the codebase that we had inherited. After all, if you don't understand how the application works, then you'll have a pretty tough time making meaningful contributions to it. In this process of discovery, one thing that the team realized was that PDF uploads were not functioning correctly within the application.

PDF uploads are a crucial part of the application; A user needs to have the ability to upload a case document to the app, the app should then upload that PDF to an AWS s3 bucket, after the PDF has been placed in the bucket, the backend of the application sends a request to our data science API with the UUID of the document that had just been uploaded, from there, the data science side runs a scraper on the PDF and pulls out useful information such as: judges, case outcome, case origin city/state, etc. The data science API then sends the scraped data back to the application's backend and it is stored in our database. PDF uploads are the lifeline of the application, they are the only way that the application collects data for visualization and table display.

Gathering Information and Diagnosing

After the team had realized that there was an issue with the document upload system, I decided to take on this problem. I quickly realized that I was a bit out of my depth. I had never worked with document uploads in an application and I had almost zero understanding of how the system worked.

The benefit of working in a large cross-functional team is that every team member has different skillsets and different experience under their belts.

I knew that one of my teammates was good with backend programming so I went to them to pair up and gain a better understanding of the issue at hand. After gaining understanding of how the document upload system is supposed to work, we began diagnosing the problem.

The first thing that we had noticed was that upon request to the data science API, the application returned a 500 error code. This led us to investigate the data science API. We got together with some of the data scientists on our team and gave them a run down of the information we had gathered so far. With the help from a few people from a few areas of the team, we had gathered that the data science application had a memory leak. The data science team came up with a fix for this within a few days, but there were more issues to come.

The biggest issue that we had ended up running into had to do with request timeouts and the time it takes for the data science PDF scraper to run. The application's backend is hosted on Heroku, which limits the time that a request can be open to thirty seconds, but the PDF scraper, which we requested to run from the data science API, took at least forty seconds to run. There is an obvious issue here, but I'll point it out anyways; the data science PDF scraper took longer than the set request timeout. This was causing the application to run into a 503 error every time a user tried to upload a case. Heroku has a few articles about H12 errors that closely relate to this issue.

Creating a Solution

Fixing this issue came down to two solid options. One of my teammates had suggested that we implement a queue that would serve as an intermediary between the backend of the application and the data science scraper. Essentially, the queue would prevent the data science application from running too many operations at once. The second option was to not wait for the data science endpoint to resolve/return anything, so that way the request cannot time out.

If we were to not wait for the API call to resolve, then we would need to store the scraped data on the data science side of the application. If we decided to implement the queue, it would take more than a few days, of which we did not have.

What we ended up deciding on was the temporary solution of not waiting for the API to resolve, and having the data science application store the information into the database directly.

As I mentioned, the solution we decided upon was a temporary one; we would need to leave detailed documentation and a solid plan for the developers that inherit this project to follow to implement the long term solution: the queue.

The Aftermath

Even though the issue at hand was only partially solved, I still learned many valuable lessons along the way. I spent hours learning about document uploads, HTTP errors, and most important of all: teamwork.

Working on a large team sometimes has its challenges; some team members might not be the best at speaking in large groups, team members from the data science team might have a tough time understanding concepts presented from the web team and vice versa, but I think that these challenges can be looked at as benefits.

Large teams bring together many different personalities from many different backgrounds, so everyone has something meaningful to contribute to the team. If I had not been working on a team to solve the issue I described, I definitely would not have been able to debug it, and I for sure would not have been able to come up with the solutions on my own.

This past month has taught me that being a developer is more than just sitting down at a computer and typing away for hours on end. Most applications are built on teams, and in order to be an effective contributor to a team, you need to know more than just how to program. The biggest thing you need to be successful in a large team is communication skills. Without communication, your team is dead in the water.

Working in a team also allows for many opportunities for personal growth. Every member of the team brings a new perspective to the table when it comes to every issue, so there is a lot of information that you can learn from. Teamwork also brings a lot of opportunity for many individuals to give you some direct feedback that you can try and implement into your professional or even personal life.

To wrap things up; working on an application that is already pretty established can be a daunting task at first, but if you have a great team to help you through it and teach you things along the way, some of the frustration can be mitigated. So if you ever have the chance to work on a large development team, do it!

I'm Stuck

Nicholas — Tue, 08 Dec 2020 01:01:59 +0000

Right now I am twenty years old. I have been trying to teach myself development (web development, then application development, and now I am back to web development) for about 4 years. When I was in high school I got my first gaming PC and I discovered that PCs are multi-purpose tools. I discovered HTML and was surprised that it really is that easy to create a static web page. I was amazed at making words bold with the <strong> tag and making words big with the <h1> tag. In my senior year in high school, I took a 'web design' course. The web design course was for some reason taught by the school's business and marketing instructor. Most of the assignments I finished in a day when other kids took a week to complete them. At times I even felt like I knew more than the instructor. After this course, I thought I was a web development genius. Then I started looking online at things people had made and things people could do, I started looking at job postings and learned that I knew almost nothing.

By the time I was seventeen or so I had purchased a udemy course called the complete web developer bootcamp taught by Colt Steele. I started going through all of the modules and then for some reason I stopped.

By the time I was eighteen I got a job at an Amazon fulfillment center (a really bad place to work in my opinion). When working at Amazon they had internal wiki pages that people could make for themselves and I made a few and realized that that is what I want to do as a job. Web development. I needed to get back into it. I quit my job at the fulfillment center and enrolled in community college.

The first semester I took Intro to computer programming where we used Python, I took a class where we learned Oracle databases and SQL, and I took Computer Science I: C++. I easily got high marks in all of my classes and I did so for the rest of my semesters as well. The problem is that I didn't feel as if I was learning anything and I still don't.

I started to go back to the udemy course I mentioned earlier and felt that I was learning more valuable skills there than in school.

I have completed most of my requirements to graduate (all except four classes), but I still feel like I don't know what I need in order to get a development job.

Currently, I am still trying to learn development. I have been hung up on CSS lately and am working on my own framework called NimbleUI. I really do hope something comes of this because I am stuck and I feel like I have been going nowhere for the past two years.