Week 2, Day 1 - Data Governance and Security
Month 1: Foundations of Data Engineering
Week 2: Core Concepts
Monday
Before we get into it
I need to say this out loud: I am genuinely relieved to be sitting down and taking notes instead of staring at a terminal today.
Last week was a full week of installing things and immediately breaking them. WSL refused to start twice. Postgres wouldn't accept connections until we figured out that pg_hba.conf was still set to localhost only. Then there was the fun afternoon of changing permissions on database users because a query kept failing with a permission denied error that made no sense until someone pointed out the user had never been granted rights on the schema, only the database. By Friday my hands knew sudo systemctl restart postgresql better than they knew my own phone number.
So walking in today and being told "grab your notebook, no terminal needed" felt like a genuine reward. This is the first theoretical day of the program, and the topic is data governance and security. After a week of learning how databases actually get built and secured on a machine, it makes sense that today is about the rules and thinking behind why we secure them that way in the first place.
What we're covering today
- What data governance actually is, and why it is not just a compliance buzzword
- The core pillars of governance: quality, cataloguing, lineage, stewardship
- Data security principles: the CIA triad, defense in depth, zero trust, privacy by design
- How data gets classified and managed across its lifecycle
- The big compliance frameworks: GDPR, HIPAA, PCI-DSS, and Kenya's PDPA
- Access control models and the principle of least privilege
- How governance, security, and compliance actually connect to each other
1. What is data governance, really?
Data governance is the set of rules, policies, and processes that decide how data is collected, stored, used, and protected across an organization.
Put simply, it answers questions like:
- Who owns this data?
- Who is allowed to see it?
- How long do we keep it?
- How do we know it is accurate?
- Where did it come from?
- What happens if it gets misused or lost?
Why this matters to me as a future data engineer
Last week I was the one creating database users and deciding what privileges they got. That is access control in miniature. Multiply that by an entire company, with hundreds of tables, dozens of pipelines, and multiple teams, and you start to see why governance is not optional. If data goes missing or the wrong person sees something sensitive, there is a good chance a pipeline someone built is where it happened. That someone is eventually going to be me.
2. The four pillars of data governance
| Pillar | What it means |
|---|---|
| Data quality | Making sure data is accurate, complete, consistent, and current |
| Data cataloguing | Keeping a record of what data exists, where it lives, and what it means |
| Data lineage | Tracking where data came from and how it changed as it moved |
| Data stewardship | Assigning real people who are responsible for maintaining data standards |
A quick way I'm remembering this: quality is "is the data good," cataloguing is "do we know it exists," lineage is "do we know its history," and stewardship is "does someone actually own it." Without all four, a company ends up with three versions of the same customer and no idea which one is correct.
3. Data security: the technical backbone of governance
If governance is the rulebook, security is what actually enforces it. Today we broke security down into a few core principles.
The CIA triad
- Confidentiality: only the right people and systems can see the data. This is enforced through access controls, authentication, and encryption.
- Integrity: data stays accurate and unaltered unless someone with permission changes it. Checksums, audit trails, and digital signatures help here.
- Availability: data is there when it is needed. This is where backups, disaster recovery, and uptime planning come in.
Defense in depth
This is the idea that you never rely on a single layer of protection. Instead you stack them:
- Physical security (secure data centers, restricted facility access)
- Network security (firewalls, VPNs, network segmentation)
- Application security (secure coding, input validation, authentication)
- Data security (encryption, classification, access permissions)
If one layer fails, the next one is still standing. It reminded me a lot of last week's pg_hba.conf and postgresql.conf split: one file controls how the server behaves, the other controls who is allowed through the door. Neither one alone is enough.
Zero trust
The mindset here is "never trust, always verify." No user or system gets automatic trust just because it is inside the network. Every request gets checked, every time. This usually shows up as multi-factor authentication, conditional access policies, and continuous monitoring rather than a single login at the start of the day.
Privacy by design
Instead of bolting privacy on at the end, it gets built in from the start. The idea is that privacy should be the default setting, not something a user has to dig through menus to switch on.
4. Encryption: protecting data at rest and in transit
| State | Meaning | Example |
|---|---|---|
| At rest | Data sitting in storage | Rows in a Postgres table, files in a bucket |
| In transit | Data moving between systems | An API response flowing into a pipeline |
All the major cloud platforms encrypt by default, but it is still on the data engineer to verify it and manage the keys correctly. That's where a Key Management Service (KMS) comes in: it is the system that creates, stores, and controls the encryption keys, basically the safe that holds the vault combination.
5. Access control and the principle of least privilege
Access control decides who can do what with which data. The golden rule is the principle of least privilege: give every person and system only the minimum access they need to do their job, nothing more.
This one actually landed for me because of last week. When we created database users and had to explicitly GRANT privileges on a schema before anything worked, that was least privilege in action. Nobody gets access by default. You earn it, deliberately, one grant at a time.
Common access control models
| Model | How it works | Example |
|---|---|---|
| RBAC (Role-Based) | Access is tied to a role | All analysts get read-only access |
| ABAC (Attribute-Based) | Access depends on attributes like department or location | Only the Nairobi team can see Nairobi sales data |
| DAC (Discretionary) | The data owner decides who gets access | A team lead grants access to their own folder |
6. Data classification and the data lifecycle
Not all data carries the same risk, so it gets classified based on sensitivity:
- Public: no harm if disclosed
- Internal: minor harm if disclosed
- Confidential: significant harm if disclosed
- Restricted: severe harm if disclosed
There is also classification by regulation, like Personally Identifiable Information (PII), Protected Health Information (PHI), and payment card data.
Data also moves through a lifecycle, and governance applies differently at each stage:
- Creation and collection: define why the data is being collected and apply classification early
- Processing and use: enforce access controls based on that classification
- Storage and retention: keep it only as long as necessary, with appropriate security
- Sharing and disposal: assess risk before sharing, and dispose of it securely once it is no longer needed
7. Compliance frameworks
Compliance is what makes governance and security legally enforceable. A few frameworks came up repeatedly today.
GDPR (General Data Protection Regulation)
- Applies to anyone processing the personal data of EU citizens, regardless of where the company is based
- Effective since May 2018
- Grants rights like access, correction, erasure ("right to be forgotten"), and data portability
- Breaches must be reported within 72 hours
- Fines can reach 20 million euros or 4 percent of global annual revenue, whichever is higher
HIPAA (Health Insurance Portability and Accountability Act)
- US-only, and only applies to Protected Health Information (PHI)
- Built around three rules: Privacy, Security, and Breach Notification
- Requires administrative, physical, and technical safeguards
- Penalties can reach into the millions per year depending on severity
PCI-DSS
Global standard for anyone handling credit card data. It sets requirements around encryption, access control, and regular security testing.
PDPA (Kenya's Data Protection Act, 2019)
Since we are training in Kenya, this one is directly relevant. It mirrors a lot of GDPR's principles and is enforced by the Office of the Data Protection Commissioner (ODPC). Any system handling Kenyan citizens' personal data has to comply with it, which means it will realistically apply to a lot of what we build in this program.
A quick way to keep GDPR and HIPAA straight
GDPR is the speed limit that applies everywhere you drive with an EU citizen in the car. HIPAA is the special rule that only applies inside the hospital parking lot. GDPR is broad and industry-agnostic. HIPAA is narrow and healthcare-specific.
8. How governance, security, and compliance connect
Data Governance
-> defines the rules and policies for the data
Data Security
-> implements the technical controls that enforce those rules
Compliance
-> confirms those rules and controls meet legal requirements
Governance is the rulebook. Security is the locks and guards. Compliance is the inspector checking the locks actually meet the law.
9. Key terms from today
| Term | Meaning |
|---|---|
| Data governance | Rules and processes controlling how data is managed and used |
| Data lineage | A record of where data came from and how it changed |
| Data catalogue | A directory of what data exists and what it means |
| CIA triad | Confidentiality, Integrity, Availability |
| Defense in depth | Layered security controls across physical, network, application, and data layers |
| Zero trust | Never trust automatically, always verify every request |
| KMS | Key Management Service, used to manage encryption keys |
| Principle of least privilege | Give the minimum access needed, nothing more |
| RBAC | Role-Based Access Control |
| GDPR | EU regulation protecting personal data of EU citizens |
| HIPAA | US regulation protecting patient health data |
| PDPA | Kenya's Data Protection Act, 2019 |
| PCI-DSS | Security standard for payment card data |
| Audit log | A tamper-proof record of who accessed or changed data, and when |
10. What I'm taking away from today
Last week gave me the hands, this week is starting to give me the reasons. Every GRANT, every edit to pg_hba.conf, every password I set for a database user last week was actually a small act of governance and security without me fully realizing it at the time. Today just put labels on things I was already doing instinctively.
Homework to myself tonight: go back through last week's terminal history and map each command to a concept from today. CREATE USER and GRANT are access control. Editing postgresql.conf and pg_hba.conf is defense in depth. Setting a password is confidentiality. It is all the same picture, just seen from two different angles this time.
See you in the next class.
Notes by [Your Name] | Month 1, Foundations of Data Engineering | Week 2, Day 1
Top comments (0)