Speaker: Raagini Lalwani, Aditya Kurella @ AWS FSI Meetup Q1/2023
Introduction by Ragini Lalwani:
Leads public cloud onboarding and migration at Goldman Sachs.
Wishes everyone a Happy Women's Day and Holi.
Has worked at Goldman Sachs for almost 15 years, starting as an intern.
Has experience with various cloud platforms: private, hybrid, and public.
Discussion on Goldman Sachs' approach to ensuring rapid and secure cloud use by developers in a regulated industry.
Introduction by Aditya:
Leads finance planning engineering teams and is the lead architect for FP&A at Goldman Sachs.
Also has close to 15 years with the firm, starting with web app development, now focused on enterprise systems.
Discussion on the firm's unique phase of cloud onboarding and the transition from on-prem to cloud.
Will talk about onboarding experience with public cloud, secure tooling, acceleration of cloud onboarding, and key highlights and learnings.
Overview of Goldman Sachs:
A leading financial institution offering a broad range of services including investment banking, securities, investment management, and consumer banking.
Serves a large and diversified client base including corporations, financial institutions, governments, and individuals.
Founded in 1869, headquartered in New York, with offices across major financial centers globally.
Focus on accelerating Goldman Sachs' cloud journey with a security-first mindset.
Fast Track Section Overview:
- Emphasizes a customer-centric mindset using Amazon's "working backwards" process.
Problem Statement from the Customer's Perspective (Application Developer):
New developers need to understand the firm's security approach, posture, and modules.
Deployment requires passing through security reviews, which can be manual and time-consuming.
Security reviews are required not only at the initial deployment but also for subsequent changes, leading to a potentially slow and iterative process.
Developers may receive alerts after deployment indicating incorrect configurations, highlighting the need for preventive controls rather than detective controls.
Developers, skilled in application development, may not have the expertise in cloud security, creating a dichotomy between development and security requirements.
Tenets for Addressing the Problem Statement:
Implement guardrails to ensure secure-by-design deployments, automating the security baseline.
Establish a clear shared responsibility model delineating the roles of the cloud service provider, Goldman Sachs Cloud teams, and application developers.
Utilize cloud-native technologies to minimize the need for learning firm-specific tooling, allowing developers to leverage existing knowledge of AWS CDK or Kubernetes.
Aim for self-service capabilities to reduce the need for ticket raising or manual reviews, enabling more developers to access the cloud without hurdles.
Introduction to Fast Track:
Fast Track is the solution developed in response to the identified problem statement.
It involves a technical deep dive into the implementation details.
The goal is to provide a secure, efficient, and self-service-oriented cloud onboarding experience for developers.
Fast Track Overview:
A fully managed service enabling Goldman Sachs developers to rapidly develop and deploy software to AWS.
Enforces the firm's security, regulatory, and compliance requirements through guardrails.
Enhances the security posture by programmatically and proactively enforcing security policies.
Key Features of Fast Track:
Handles account creation and pipeline management behind the scenes.
Users interact with Fast Track via CLI or API to create, delete, or share pipelines.
Pipeline creation triggers an account provisioning workflow, creating a service account for AWS resources and a pipeline account for management.
Users push code changes, which run through a deployment pipeline and are checked against guardrails.
Guardrails enforce the firm's baseline security policy, ensuring resources meet security requirements before deployment.
Simplified User Journey:
User creates a Fast Track pipeline and associates it with a GitLab repo.
Writes infrastructure as code using AWS CDK.
Pipeline synthesizes CDK code into CloudFormation templates.
Guardrails run against these templates, and the policy engine determines if the deployment can proceed ("go") or not ("no go").
Guardrails Details:
Guardrails follow a default-deny policy.
Written using Open Policy Agent (OPA).
Run against standard CloudFormation templates.
Example guardrail: Ensures that malicious actors cannot access unencrypted objects.
Threat Modeling and Control Definition:
Uses NIST 800-53 guidelines to identify threats for specific technologies.
ISO 27001 is an international standard for information security management systems, while NIST SP 800-53 is a U.S. government standard for security and privacy controls. 2. ISO 27001 focuses on the management of information security, while NIST SP 800-53 focuses on the technical security controls.
Example threat: Malicious actors can access an unencrypted object.
Control definition: Encrypt the object.
Controls are defined and used as OPA policies, reviewed, and unit tested by the security team.
Collaboration and Release of Controls:
Collaboration between the Cloud enablement guardrail development team and security teams.
Controls are released as guardrails, creating a secure environment but potentially posing hurdles for developers.
Accelerators:
Built to work with guardrails, allowing developers to quickly deploy resources that comply with all guardrails.
Example: Fast Track S3 bucket that works with all 10 guardrails blocking S3.
Sample Code and Starter Projects:
- Provided to help users bootstrap their cloud journey.
Fast Track Launch and Production Use:
- Fast Track has been launched and is available, with several applications in production and many more underway.
Organizational Alignment and Model:
Goldman Sachs recognized the need for a first-class developer experience without compromising security.
Fast Track was deemed a flagship project, with Cloud enablement leading the development in collaboration with technology risk teams.
Guardrail development involved a mix of developers, cloud architects, and security engineers.
A guardrail contribution model ensures that specific technology teams (e.g., storage engineering) review and contribute to relevant guardrails.
Response to Question on Organizational Structure:
Acknowledges the typical siloed model in large enterprises.
Emphasizes the need for a collaborative approach to balance developer experience and security.
Highlights the importance of a mandate from management to foster cross-team collaboration.
Notes the constraint that not all Cloud enablement team members are experts in every cloud service (e.g., S3).
The mix of skills required for guardrail development and the contribution model for specialized reviews.
Clarification on Ownership and Responsibilities in the Cloud:
In the cloud, the application team owns all their cloud resources, including computers and databases, which are deployed into their VPC.
The database team is responsible for working with Cloud enablement to establish access control, policy management, and audit-related pieces.
On-premises, the database team owns the databases, but in the cloud, the customer (application team) owns the databases running in AWS.
Summary of Organizational Model:
A central Cloud team establishes the guardrails framework and is the authority around cloud development.
The CIO, CTO, and Tech Risk Office sponsor new guardrail development and policies around cloud onboarding and governance.
Associated teams (DB teams, IAM teams, etc.) work with the central Cloud team to onboard their services and policies.
Application development teams implement their apps using Fast Track and may request additional services, which are then worked on with related teams.
Collaborative Model:
- Multiple teams play their part in the cloud onboarding process, ensuring a collaborative approach to cloud development and security.
Hands-On Approach for the First Application:
The initial application on Fast Track required a more hands-on approach as the team was learning about the customer journey.
Future applications aim to leverage existing guardrails, allowing customers to use the service independently.
Investment in tutorials and documentation to support self-service usage.
Platform Team's Role:
The platform team provides guidance and builds security controls (guardrails) for resources like Lambda, DynamoDB, and RDS.
Developers are free to choose the resources they need, with the platform team helping to build the necessary security controls.
Accelerators are provided to make it easier for developers to deploy resources safely.
Guardrails in Development vs. Production:
The same guardrails are applied in both development (sandbox) and production environments.
Guardrails are in blocking mode in all environments to maintain a consistent security posture.
Fine-grained control allows for specific guardrail bypasses on a per-pipeline level, subject to tech risk review.
Technical Implementation of Guardrails:
Guardrails are checked via an API that the pipeline interacts with.
The pipeline synthesizes infrastructure as code (e.g., AWS CDK) into CloudFormation templates.
These templates are then checked against the guardrails and run using Open Policy Agent (OPA).
The policy engine determines if the deployment can proceed ("go") or not ("no go").
Technical Implementation of Guardrails (Continued)
Additionally, an IDE plugin is available to provide developers with feedback sooner, without the need to deploy the pipeline.
Developers can see the feedback as they develop in their IDE, helping them adhere to security standards early in the development process.
Question on NIST Implementation and Threat Database:
The project started with NIST 800-53 as a baseline.
The firm's cyber defense and Intel team keeps the team informed about new threats.
Decisions on managing new threats are made based on the vulnerability level (critical or not) and integrated into the software development lifecycle (SDLC).
Detective controls complement the preventative controls provided by the guardrails.
Introduction to FP&A (Finance Planning and Analysis):
FP&A is responsible for firm-wide business planning, cost allocations, and driving financial insights.
FP&A is pursuing a cloud-first strategy, aiming to move 100% of on-prem and new applications to the cloud by the end of 2024.
Reasons for adopting the cloud include monthly compute usage peaks and troughs, limitations of current on-prem solutions (installed and run on an organization's own physical hardware and servers, located within their own facilities), and the desire to reduce engineering time spent on maintenance.
FP&A has aggressively decoupled legacy on-prem infrastructure, with approximately 70% completed, and has built new applications with cloud considerations in mind over the past three years.
FP&A's Cloud Strategy and Adoption:
FP&A has been building microservice-friendly, cloud-ready architectures and heavily investing in Infrastructure as Code (IaC) even on-prem.
The adoption of the cloud is driven by the low cost of compute, increased productivity, and reduced build and maintenance cycles.
Fast Track, with sponsorship from engineering leaders, became the strategic platform for the firm.
First Application: FBA Central:
A simple gateway application was chosen as the first app to learn AWS and Fast Track.
Developed in three to four months, went live with an MVP in September-October.
Representative microservice app with Spring Boot APIs, deployed using ECS Fargate and DynamoDB, with a React.js UI.
Subsequent Applications:
Built more complex applications with increased microservices, on-prem connectivity, and workflow solutions.
Deployed the first app in a dedicated FP&A-specific VPC, a first for the firm.
Despite increased application complexity, Fast Track enabled increased productivity and decreased delivery times.
Teams became more skilled over time, further reducing timelines.
Summary:
- FP&A's cloud-first strategy, combined with Fast Track, has led to successful onboarding of applications with increasing complexity, improved productivity, and reduced delivery times.
FP&A's Cloud Journey and Future Focus:
Deployed three cloud-native apps with eight plus services.
Learned to build software, undergo cloud onboarding, tech risk reviews, architecture bar raising, and SDLC processes.
Trained about half of FP&A engineers.
Key focus for 2023: high-usage apps and data-intensive calculations, processing about 6 billion records daily.
Will explore serverless offerings (Lambda, Step Functions) and data processing/analytics services (Glue, Athena, S3, Snowflake).
Aim to build FP&A-centric accelerators using CDK constructs to further accelerate development.
Key Decisions and Success Factors:
Formation of a core team of four developers from different regions to train scrum teams, accelerating development.
Choice of Fast Track for secure cloud onboarding, dealing with sensitive data.
Fast Track's guardrails are tech risk-approved and continuously executed at every pipeline, providing a secure and up-to-date profile.
Fast Track's CDK accelerators and common patterns helped improve developer productivity.
Deployment and Learnings:
Deployed the first simple app, FBA Central, using AWS components like DynamoDB, ECS Fargate, and ALB.
Underwent rigorous tech risk reviews and architecture bar raising.
Learned about gold live mechanisms, environment creation, observability, logging, and using CloudWatch and PagerDuty.
Decommissioned the on-prem app, becoming fully cloud-native for FP&A.
Cloud Learnings:
Cloud provides a broader and more sophisticated set of options to closely match application compute consumption needs.
Encourages organizations starting their cloud journey to adapt the playbook and learn from FP&A's experience.
Importance of Design Choices:
Engineers must understand the consequences of their design choices, which impact operational costs and maintenance.
Serverless options offer lower costs, easy scaling, and low maintenance.
Infrastructure as Code (IaC) allows for faster innovation and toggling between different compute and cost profiles.
Benefits of IaC:
Enables faster innovation and optimization for cloud costs.
Allows for different costs and compute profiles based on architecture choices.
Business Continuity Planning (BCP) is more straightforward with cloud strategies compared to on-prem.
Increased Developer Productivity and Cost Optimization:
Developer productivity increased with the ability to manage costs on a daily basis.
The maturing ecosystem and Fast Track have facilitated the onboarding of applications.
Resources for Further Learning:
- Goldman Sachs Engineering developer blog for insights on cloud and related builds.
Clarification on Fast Track and Open Source:
Fast Track itself has not been open-sourced.
Goldman Sachs has an open-source team and has open-sourced various projects.
The guardrails for Fast Track were initially built in partnership with AWS and have been shared back with AWS, who are using them as part of CFN Card.
Goldman Sachs does not consider their guardrails a competitive advantage and is open to sharing them.
SkyPath: Cloud to On-Prem Integration:
SkyPath is a connectivity solution that allows applications in different trust zones to connect.
It addresses the challenge of connecting on-prem and cloud applications in a hybrid environment.
SkyPath leverages a Service Identity Framework (SIF) to establish a unique identity for each endpoint before onboarding to SkyPath.
This ensures that only endpoints with a clearly defined identity can connect, addressing potential identity challenges in the integration.
Service Identity Framework (SIF):
SIF requires every endpoint to have a unique identity defined before it can onboard to SkyPath.
This acts as a pseudo-cloud identity for on-prem services, ensuring secure and controlled access.
Establishing a service identity is a prerequisite for enabling access to on-prem APIs via SkyPath.
Conclusion:
- The discussion highlights the importance of secure and controlled integration between on-prem and cloud environments, facilitated by solutions like SkyPath and the Service Identity Framework.
Top comments (0)