I just finished my largest and most involved experience working on a development team, and I’m here to reflect on my takeaways. This experience consisted of a month-long internship, working on an already established software project, while I had previously only worked on teams where we built the project from the ground up, so many of the conventions around development process and communication were new to me. The project my team inherited was a data collation and exploration tool for asylum court case outcomes, which we were building on behalf of Human Rights First, an international human rights organization. They noticed that many asylum case outcomes were highly dependent on the legal opinions of the judge hearing the case, and that in cases where multiple potential avenues of legal argument were possible, some framings did better with many judges than others did.
The goal of this product was to scrape this case information from files uploaded by authorized attorneys in an automated process, and have it available to explore and filter to other lawyers and legal researchers, so they could be better informed on the big-picture statistics of what cases had been approved, denied, or appealed. I requested and fulfilled the role of a back-end developer on this project, as I had less experience in that role on the projects I’d worked on previously and wanted to broaden my experience. Other members of the team consisted of 12 other interns, in front-end, back-end, data science, UX, or project management roles, as well as several supporting mentors.
Our Initial Goals
I addressed two major issues during my time on the project, one planned, and one discovered in process: first, my team had been asked to create a third role with designated permissions within the application. There were already pre-existing ‘user’ and ‘admin’ roles that had been implemented by the previous team, but our clients wanted a third ‘moderator’ role, which could perform every function a user could, as well as approve or deny case uploads like an admin, but could not manage users themselves or site functionality and changes. While implementing the necessary back-end changes to add this feature, I discovered a major security vulnerability in how the application verified a user’s role and subsequent permissions, discussed the possibilities for how to fix it, and implemented the plan we agreed was best.
At first, when approaching the task of adding a new role to the ones already within the app, I discussed the database schema with the rest of my team. As we inherited it, the database stored role information as a column within the table for user profiles, as a string. This was not ideal because it could allow for typos to be entered, and user permissions to break as a result, so we decided to update the database to include a roles column, with a primary key integer and a role name, and have each profile contain a column that pointed to a role id as a foreign key. I began working on this directly with another back-end developer and a project manager over a video call, and it turned out to be much more difficult than we expected.
At first, we attempted my idea of simply adding an additional migration using Knex.js, a library which allowed us to build and query our Postgres database inside a Node framework. This required saving the existing role information from the database, deleting that column, creating a new table and column, and repopulating it with foreign keys from the new table. Unfortunately, we kept running into different errors with deleting or adding columns to the existing profiles table, and at one point had to delete the local database altogether, starting over with the preexisting Knex migration and seed files. After being unable to find a solution to these errors, one of my teammates tried a different approach, starting over on our goal. Eventually we gave up on modifying the database with additional files, and decided to rewrite the original migrations and seeds, adding in one for the roles table as well, and running functions to remove and rebuild the database entirely before seeding it with new sample data. We had chosen not to try this approach at first because it is risky in a production environment, where important data already exists in the database that could easily be lost in the process, but we decided there was no risk with only sample data that could be repopulated with a single npm command. This approach worked, much to all of our relief! We then quickly built new models and API endpoints for the new roles table in Express.js.
Security Vulnerability and Redirecting Focus
While digging through the repositories to find out how user permissions were implemented, I realized it seemed like they weren’t referred to at all in the back-end codebase, and it also didn’t seem like our identity management provider, Okta, was even storing role information. Confused, I started looking through the browser development tools as I navigated through the local live instance of the app. I found that there were three things being saved to the browser local storage, two JWTs related to Okta authentication, and one ‘role’ variable. Curious, I logged in as one of our test accounts designated with user permissions. By editing the string stored under ‘role’ from ‘user’ to ‘admin’, and continuing to navigate through the application, I was able to unlock special pages that should have only been accessible to admins, and to add, delete, and change data without any sort of verification.
Immediately on realizing this, and verifying that the wrongfully edited data was in fact getting saved into the database, I notified the rest of my team that it seemed like a huge issue for the security of the application, and that it should be a high priority issue to be fixed. While it seemed unlikely that any authenticated users would be malicious enough to try vandalizing the data, all it could take was one compromised account by one motivated attacker to completely wipe out the entire app’s information. In our team discussion, we decided it would be excessively complicated to change how the front-end was handling role information, that it had the potential for breaking parts of the app that already worked, and that our best course of action was to patch the back-end, verifying on each endpoint request that a user had the permissions required to access it. I volunteered to write middleware to implement this solution.
Two Issues, One Solution
The actual middleware function ended up being very simple, once I realized a call to the database was already being made by another middleware function attached to every endpoint, which simply verified that any user was logged in and authenticated with Okta at all. This function queried the database for the profile of the authenticated user, and attached that information to the request headers before continuing the request. My function ended up taking an array of role ids which designated the roles who were allowed access to the endpoint, checking each of those ids against the role of the authenticated user, and allowing or forbidding access as a result. After testing the function, I had to add in some special cases to allow access for users to read or update their own information or their own uploaded cases. I chose to write the function to handle an array of roles, rather than a specific role id, in order to simultaneously implement the new moderator role in the backend. The function could still take an array with only one role id, as for the endpoints that only admins should access, but it could also take two or more ids, as for endpoints that should be accessible to admins and moderators. This flexibility in how the function worked would also allow easy changes in the future if more roles were added to the application.
After the middleware function was added to all the appropriate endpoints, I tested the vulnerability to see if it was patched. While I could still change my role in local storage and gain access to pages for admin-only tools, none of the data would load within them, and no new data could be changed or added! Even sending a direct request to the endpoint was denied without a valid token from a user with the required roles, and I considered this a success. In addition, all of the back-end setup for the moderator role was complete, and only a bit of work needed to be done on the front-end to conditionally display these tools in a manner that already existed for admins. Moving forward, the next team might have issues if they decided to strengthen the patch on the front-end as well, if they decided to prevent the admin tool pages from displaying at all to unauthorized users. As far as I can tell, that would require a separate JWT to be created upon login on the back-end, and sent to the front-end local storage, which would allow a user’s role information to be stored and queried on the front-end in a more secure and less editable fashion. This might be complicated and increase the overhead of the app’s runtime, especially on login, and would require cross-collaboration between segments of the new team.
My Takeaways
I grew a lot more as a developer and professional than I expected to throughout this experience. While I had worked on teams before, it had always been more ad-hoc, without explicitly declared roles on the team, for periods of only about a week, and never on an established project. I’d also never experienced meeting with clients who were investing in actually using the product, and in having it ship as soon as possible. These weekly meetings, as well as the level of communication I needed to have with my team over Zoom and Slack, helped me realize the importance of communication on a team, why things can take so long to get done in a larger production environment, and how easy it was for one or more people on the team to get lost without reconnecting over our shared goals and priorities. The feedback I got from my team, praising my technical skills and my attention to detail, helped me feel more confident in my abilities and contributions, both on this project and future ones I plan to work on. The last month has made me realize how much I care about working on something that matters which will help people, how much more motivating it is and how I would like to look for these kinds of projects in my future career. Working in a back-end role, which I had little experience with in previous projects, and collaborating with data scientists and front-end developers has broadened my skills and given me a better understanding of how different parts of a team need to work together to accomplish shared goals. I hope that the development in my technical and professional skills will help me achieve my future goals, and that I can continue to learn and grow in these areas throughout the years to come.
Latest comments (1)
Very nice story. Did the project consider using database roles and policies to resolve authentication and authorization? Your first task them would be to add policies (declarative) and that's it! I'm interested because scale might be a reason, but until now, I've never seen scale where database roles for end-users don't work.