DEV Community: Kelly Hopkins

Ditch the Spreadsheets: Secure and Scale Your Nonprofit Data with AWS

Kelly Hopkins — Tue, 11 Mar 2025 03:49:18 +0000

Nonprofits run on data. Donor information, program outcomes, volunteer hours, and financial records – it's all crucial for driving your mission. Yet, many small organizations struggle with data scattered across countless Excel spreadsheets, personal computers, and disparate systems. This not only creates security risks but also hinders collaboration and limits your ability to gain valuable insights. AWS allows nonprofits to streamline their operations, enhance security, and leverage their data for more informed decision-making.

Imagine the frustration of compiling a grant report, a task that should be straightforward, but instead involves aggregating data from dozens of disparate spreadsheets, each with its own formatting and potential errors. Or consider the challenge of tracking donor engagement across multiple, disconnected files, making it difficult to get a holistic view of donor relationships and identify opportunities for deeper engagement. The time wasted on this manual data wrangling – time spent copying, pasting, and reconciling data – detracts significantly from your core mission. It pulls staff away from vital work, delays reporting, and increases the risk of inaccuracies that can impact decision-making and stakeholder trust. Amazon Web Services (AWS) offers a solution to these challenges.

Common Nonprofit Data Workflows

Choosing the Right AWS Storage Service

AWS offers various storage services tailored to different data needs. To help nonprofits effectively manage their diverse information, AWS provides a range of storage solutions, each designed with specific data characteristics and use cases in mind. Understanding these options is crucial for selecting the most appropriate service for your organization's data.

Amazon S3 (Simple Storage Service)

Think of S3 as a highly secure and scalable online storage solution. It's an ideal choice for nonprofits needing to store a wide variety of unstructured data, including documents, images, videos, and large datasets. For instance, S3 can be used to store donor contact lists in CSV format, program evaluation reports in PDF, or multimedia content from fundraising campaigns. Its scalability and security make it a versatile option for managing diverse digital assets.

Accessing Data Stored in S3

The AWS Management Console provides a user-friendly web interface for accessing your data in S3. You can easily navigate to your S3 bucket, view the files and folders within, and download or upload files as needed. This method is well-suited for occasional access to your data or for managing individual files.

However, the true power of S3 lies in its ability to be accessed programmatically. Interacting with S3 buckets programmatically, through the use of AWS SDKs, unlocks many opportunities for automating and streamlining data workflows. This allows nonprofits to efficiently process and manage large volumes of data, perform repetitive tasks automatically, and integrate S3 seamlessly into their applications, significantly improving operational efficiency and reducing manual effort.

Amazon EBS (Elastic Block Store):

Amazon EBS provides block storage volumes for use with EC2 instances. Think of these as virtual hard drives that you can attach to your virtual servers in the cloud. Unlike S3, which is object storage, EBS provides persistent block storage, which is essential for running operating systems, databases, and applications that require fast, consistent disk access.

EBS (Elastic Block Storage):

Amazon EBS provides block storage volumes for use with EC2 instances, functioning as virtual hard drives for cloud-based virtual servers. This persistent block storage is crucial for running operating systems, databases, and applications requiring fast, consistent disk access. For nonprofits, this is particularly useful for storing data related to mission-critical applications, ensuring high availability and performance. It also simplifies disaster recovery planning with easy-to-manage snapshots and enables scalable storage solutions that can grow with the organization's needs.

Amazon RDS (Relational Database Service)

RDS provides managed relational databases like MySQL, PostgreSQL, and SQL Server. This service is particularly well-suited for handling structured data, which is organized in tables with rows and columns, and requires complex queries and relationships. This type of data is commonly found in donor databases, program management systems, and financial records. For instance, a donor database might store information such as donor names, contact details, donation amounts, and the dates of those donations. Because this data is structured, you can use SQL (Structured Query Language) to perform sophisticated searches, generate reports, and analyze relationships between different data points.

Data Warehousing on Amazon Redshift

Redshift is a powerful data warehousing service designed for large-scale data analysis. It's ideal for aggregating and analyzing data from multiple sources to gain deeper insights into your programs and operations. By consolidating data from various systems, Redshift enables nonprofits to identify trends that might otherwise be hidden, measure the impact of their programs with greater accuracy, and optimize their resource allocation to maximize the effectiveness of their efforts. For example, by combining donor data with program data, you can analyze which fundraising efforts are most effective in attracting donors who are likely to support specific programs. Similarly, by integrating financial data, you can track how program costs correlate with outcomes, allowing for better resource management and strategic planning.

For example, you could use Redshift to combine donor data, program data, and financial data to identify trends, measure impact, and optimize resource allocation.

Choosing the Right Data Storage Solution for Your Nonprofit

Nonprofits have diverse data storage needs, and selecting the appropriate solution is crucial for efficiency, security, and cost-effectiveness. Here's a breakdown to help you make the right choice:

Data Type and Structure:
- Unstructured Data: If your data consists of files like documents, images, and videos, Amazon S3 is the most suitable option.
- Structured Data: For data organized in tables, such as donor databases or financial records, Amazon RDS is the way to go.
- Large-Scale Analytics: If you need to analyze massive datasets to gain insights, Amazon Redshift is the best choice.
Access Patterns:
- If you need to access data frequently and with low latency, Amazon EBS is ideal.
- If you are accessing data occasionally, then Amazon S3 is a good choice.
Scalability and Performance: Consider how your data needs might evolve. S3 and Redshift offer excellent scalability.
Cost: Evaluate the pricing models of each service. S3 offers cost-effective storage for large amounts of data, while RDS and Redshift have different pricing structures based on usage and compute resources.
Security and Compliance: Ensure the storage solution meets your organization's security standards

Taking the First Step

Moving from scattered spreadsheets to a secure and scalable AWS environment is a significant step, but the benefits are substantial. Start by identifying your most critical data and exploring the AWS services that best align with your needs. With AWS, you can transform your data into a strategic asset that fuels your mission.

End-to-End Survey Data Analysis on AWS

Kelly Hopkins — Tue, 11 Mar 2025 03:48:05 +0000

This week we're continuing our series on how nonprofits can leverage AWS analytics tools to drive more community value from their data. This post will tackle the common data problem of analyzing survey data and generating deep insights and effective data visualizations at a fraction of the cost of other leading analytics platforms. You will learn how to use AWS tools like QuickSight, S3, Athena, and Glue to build out your organization's data stack and quickly start driving value with your data.

Nonprofits and Survey Data

Nonprofit organizations frequently rely on surveys to gather crucial information and feedback from their target populations. These surveys help them understand community needs, measure program effectiveness, evaluate satisfaction with services, and inform strategic decision-making. Whether it's assessing the impact of a community workshop, gauging opinions on a new initiative, or collecting demographic data, surveys are a cornerstone of nonprofit operations.

However, as nonprofits grow, managing the increasing volume of survey data in spreadsheets becomes challenging. Spreadsheets can be difficult to scale, analyze, and collaborate on, and they often lack adequate security, posing risks of data loss or unauthorized access. AWS addresses these issues with a suite of services for scalable, secure, and cost-effective data storage and analysis, enabling nonprofits to maximize the value of their survey data.

Amazon QuickSight: Powerful Dashboards and Reporting at Any Size

Amazon QuickSight is a powerful, cloud-based business intelligence (BI) service that enables organizations of all sizes to easily create and share interactive dashboards and reports. It provides a user-friendly interface for visualizing data, performing ad-hoc analysis, and gaining actionable insights.

For nonprofits, QuickSight offers a compelling alternative to traditional BI tools like Tableau and Power BI. While these tools are feature-rich, their pricing can be prohibitive for small to medium-sized nonprofits with limited budgets. QuickSight's pricing model is designed to be more cost-effective, with a pay-per-session structure that aligns well with the often fluctuating data analysis needs of nonprofits. This makes it an attractive option for organizations looking to maximize their resources and minimize overhead costs.

QuickSight integrates seamlessly with other AWS services, allowing nonprofits to securely store their data in services like Amazon S3 and analyze it directly within QuickSight. This eliminates the need to transfer data between different platforms, reducing the risk of data breaches and ensuring data integrity. In contrast, the free tiers of Tableau and Power BI often require users to publish their dashboards publicly, which may not be suitable for nonprofits handling sensitive community data. By leveraging AWS, nonprofits can maintain greater control over their data and ensure that it is protected by robust security measures.

Creating a Dashboard in QuickSight

In the following example, our nonprofit just delivered a series of workshops for local community leaders, and after each session, a post-workshop survey was sent out to participants. The survey platform that they used didn't offer the types of visualizations they wanted to analyze the survey responses, so the decide to create a custom dashboard in Quicksight for their reporting.

Uploading survey data directly to Quicksight

To start prototyping our dashboard, we will begin by uploading our data directly to Quicksight as a dataset. Our dataset contains a list of responses from the post-workshop survey that contains basic information, ratings, and feedback from the participants. You can follow along with this post by accessing the data here.

From the "Datasets" tab on the QuickSight homepage, select the "New dataset" button.

There are a number of connectors and services offered to make data available for use in Quicksight, which we will explore later on in this post. For now, select the "Upload a file" tile and select the survey data file

After clicking through to the next screen, select the "Visualize" button to open the visualization editor. Keep the standard settings and select
"Create" on the new sheet page:

For our first chart, we're going to create a histogram that generates the distribution of the workshop ratings. Similar to other BI platforms like Tableau, you can create the histogram by selecting it from the chart types and dragging the workshop_rating column onto the parameters pane. After giving our chart a title, we've create the first component for our final dashboard:

After adding a few more visuals, including a distribution of the instructor rating, distribution of settings attended, and the breakdown of participant job titles, we now have a basic report of our survey data:

Now that we've explored our survey data and selected the final visuals we want to display, we can publish our analysis to make it a QuickSight Dashboard (select all of the default options):

Final Dashboard: