CompTIA SecAI+ CY0-001 | Domain 1.0: Basic AI Concepts Related to Cybersecurity
"Explain the importance of data security in relation to AI."
🧠 What's in This Section?
Data is the fuel that AI runs on. No matter how good the car is, put bad fuel in it and it either breaks down or ends up in the wrong place. AI models are no different. Bad data means a bad model. This section covers four main topics:
Data Processing — how data gets prepped and secured before it ever reaches the AI
Data Types — how AI works with different kinds of data
Watermarking — how you trace where an AI output came from
RAG (Retrieval-Augmented Generation) — the safe way to expand an AI's knowledge base
🔷 PART 1: Data Processing
AI models live and die by one principle: garbage in, garbage out. If the data you feed the model is dirty, wrong, incomplete or manipulated, the outputs won't be trustworthy either. Data processing is the whole job of cleaning, validating and securing raw data before it gets fed into a model.
Security matters at every step of this. If an attacker tampers with any stage of the data (data poisoning), the model can start making completely wrong calls.
1. Data Cleansing
What it does: Scrubs out the errors, inconsistencies, missing values, duplicates and noise in raw data.
Why it's critical: Uncleaned data teaches the model the wrong patterns. Say you're training a malware detection model, but some harmless files in your dataset got mislabeled as "malicious." The model starts flagging clean files as threats too, and now you've got a false positive explosion on your hands.
The cleansing steps:
Dedup: Removing repeated records. Duplicate data makes the model over-weight certain patterns.
Handling missing data: Filling in blank fields (imputation), deleting them or flagging them. If a log entry is missing its timestamp, for example, the reliability of that record is questionable.
Format standardization: Fixing things like date formats (DD/MM/YYYY vs MM/DD/YYYY), IP address formats and inconsistent capitalization.
Outlier detection: Spotting and dealing with values that make no sense, like a negative packet size in network traffic data.
Noise reduction: Clearing out random errors or meaningless data points from the set.
Security angle:
An attacker can deliberately inject corrupted data into your training set (data poisoning). Data cleansing is the first line of defense against that kind of manipulation.
Sensitive data (PII, credentials) should get deleted or anonymized during cleansing, not slip through unnoticed.
The cleansing process itself needs to be logged and auditable. What got deleted or changed, and why?
🍕 Real-World Example: A healthcare company is training a disease detection model. Some patients show up twice in the dataset, listed once as "John Smith" and once as "J. Smith." If those duplicates don't get cleaned up, the model over-weights this patient's data and develops a bias toward certain demographic groups. Cleansing merges the duplicates so the model learns in a more balanced way.
2. Data Verification
What it does: Checks that the data is correct, consistent and in the format you expect. It comes one step after cleansing. Instead of asking "is this clean?" it asks "is this right?"
Cleansing vs. verification:
Cleansing: "Is this data dirty? Clean it."
Verification: "Is this data correct? Check it and confirm it."
Verification methods:
Cross-referencing: Comparing data against more than one trusted source. For instance, verifying the IP addresses in a threat intelligence feed against several other sources.
Schema validation: Checking whether the data fits the expected structure, format and data types. Does the JSON log data contain the expected fields? Are the data types right?
Range checking: Making sure values fall within sensible bounds. Is the port number between 0 and 65535? Is the timestamp not set in the future?
Consistency checking: Catching contradictions inside the dataset. Is a user flagged as both "active" and "deleted"?
Security angle:
Unverified data leads the model to make bad decisions.
Attackers can inject data that looks legitimate but is actually wrong. Verification is what catches that manipulation.
The accuracy of the labels in your training data matters most of all. One piece of malware mislabeled as "harmless" can cause the model to miss every threat of that type.
🍕 Real-World Example: A security firm is training a phishing URL detection model. They discover that 3% of the URLs they pulled from threat intelligence feeds were mislabeled. Some legit sites tagged as "phishing," some phishing sites tagged as "legit." By cross-referencing multiple feeds against each other, they catch the errors. After the fix, the model's precision climbs 12%.
3. Data Lineage
What it does: Tracks and documents every transformation, movement and operation a piece of data goes through, from its origin all the way to its final use. Think of it as the data's life story.
Analogy: It's like farm-to-table traceability for a food product. Which field did the tomato grow in? Which plant processed it? Which truck carried it? Which store sold it? Data lineage gives you that same traceability for data.
The components of data lineage:
Origin → Collection → Transformation → Storage → Use → Archive/Delete
↑ ↑
└─── Every step documented: who did what, and when? ─────┘
Source tracking: Where did the data come from? Which system, API, sensor or user?
Transformation tracking: What was done to the data? Normalization, filtering, merging?
Access tracking: Who accessed or modified this data, and when?
Version tracking: Which version of the data was used?
Security angle:
When you find bias or an error in an AI model, data lineage is what lets you trace the problem back to its source.
Compliance requirements (GDPR, KVKK) make documenting your data processing mandatory.
If you suspect a data poisoning attack, lineage is critical for figuring out exactly where the data got manipulated.
It's a must for model auditing and accountability. The answer to "why did this model make this decision?" starts with the data.
🍕 Real-World Example: A bank's credit scoring AI is systematically giving low scores to a particular ethnic group. The regulator opens an investigation. Thanks to data lineage, the team discovers that a data integration three years back had tagged applications from certain neighborhoods with the wrong risk code. The problem was in the data, not the model. Without lineage, tracking down the source of that error would have taken months.
4. Data Integrity
What it does: Guarantees that data stays accurate, consistent, complete and untampered with throughout its entire lifecycle.
How it differs from verification: Verification is the act of checking whether data is correct. Integrity is the principle of keeping data correct across its whole lifecycle. Verification is a point-in-time check. Integrity is a continuous guarantee.
The dimensions of data integrity:
Accuracy: Does the data correctly reflect the real world?
Consistency: Does the data contradict itself across different systems?
Completeness: Are all the required fields filled in?
Timeliness: Is the data current, or has it gone stale?
Validity: Does the data follow the defined rules and formats?
Ways to maintain data integrity:
Hashing: Computing the hash of the data to detect whether anything changed. Algorithms like SHA-256 give your data files a "fingerprint."
Digital signatures: Verifying both the source and the integrity of the data.
Access controls: Preventing unauthorized changes.
Audit trails: Keeping a record of every operation performed on the data.
Checksums: Detecting corruption during data transfer.
Security angle:
Integrity checks are critical for catching data poisoning attacks.
If the integrity of your training data is compromised, the model becomes untrustworthy.
Incoming data needs integrity checks at inference time too. Manipulated input means wrong output.
🍕 Real-World Example: A security company refreshes its network traffic analysis model with new data every week. An attacker slips into the company's data collection pipeline and injects their own C2 (Command & Control) traffic into the training data, labeled as "normal traffic." With no hash-based integrity check in place, nobody notices. The model learns the attacker's traffic as "normal" and can no longer detect that traffic type. Integrity checks (hash comparison, source verification) could have blocked this attack right at the start.
5. Data Provenance
What it does: Documents where data came from, who created it and when, and how trustworthy its original source is.
How it differs from data lineage: These two get mixed up constantly, and you'll need to know the difference on the exam:
| Concept | Focus | The question it answers |
|---|---|---|
| Data Provenance | The data's ORIGIN | "Where did this data come from? Who made it? Is it trustworthy?" |
| Data Lineage | The data's JOURNEY | "What stages did this data pass through? How was it transformed?" |
Provenance asks about the origin. Lineage tracks the journey. Provenance focuses on the starting point. Lineage covers the whole process.
Analogy: Picture a diamond. Provenance says "this diamond was mined from the X mine in South Africa, certified conflict-free." Lineage says "cut at the mine, then processed in Antwerp, graded in London, put up for sale in Istanbul."
The components of data provenance:
Source identity: Who or what system produced the data?
Creation time: When was it created?
Collection method: How was it gathered? Sensor? API? Manual entry?
Trust level: How reliable is the source?
License and legal status: Do we even have the right to use this data?
Security angle:
Data from untrustworthy sources can manipulate your model.
In supply chain attacks, verifying the source of your data is critical.
For compliance, you need documentation showing the data is legal to use.
The provenance of the datasets used in model training directly affects how trustworthy the model is.
🍕 Real-World Example: A security firm is buying a new threat intelligence feed. They look into its provenance: the data was collected from dark web honeypots, the source company has a 10-year track record and the data is processed on ISO 27001 certified infrastructure. Another feed is built from data sourced from "anonymous contributors," with murky provenance. They use the first feed for model training and reject the second one. A source with unclear provenance could be a channel where an attacker is deliberately spreading disinformation.
6. Data Augmentation
What it does: Grows the size and variety of a dataset by generating new, synthetic samples from the existing data.
Why you need it: AI models are hungry for data. But sometimes there just isn't enough, especially when you're dealing with rare events. In security, real zero-day attack samples are extremely scarce. Data augmentation closes that gap by producing new variations from the samples you do have.
Augmentation techniques:
For text: Synonym replacement, changing sentence structure, back-translation. Rewriting a phishing email with different wording, for example.
For network traffic: Slightly shifting packet timings, adding variation to payload sizes, randomizing IP addresses.
For images: Rotation, cropping, color shifts, adding noise. Producing variations of screenshots for malware analysis.
Synthetic data generation: Using GANs or other generative models to produce entirely new samples.
Security angle:
Done badly, augmentation can actually hurt the model's real-world performance. Unrealistic synthetic data teaches the wrong patterns.
Augmented data needs to be distinguishable from the original (flag it in the metadata).
Watch the privacy risk when you augment sensitive data. Can the augmented data be reverse-engineered back into the original sensitive data?
🍕 Real-World Example: A SOC team is building an insider threat detection model. The trouble is, real insider threat cases are rare. Just 8 cases over two years, nowhere near enough to train a model. With data augmentation, they take the patterns from those 8 cases (after-hours access, large file downloads, unusual USB activity) and generate 500 synthetic cases. Now the model can detect insider threats with 78% accuracy.
7. Data Balancing
What it does: Fixes the imbalance between classes in a dataset.
What's the problem? Security data has a serious imbalance problem. Normal traffic can run to millions of records while attack traffic might be just a few hundred. A model trained on that imbalance leans hard toward the majority class (normal traffic) and misses the minority class (attacks).
An example of imbalance:
Normal traffic: 998,000 records (99.8%)
Attack traffic: 2,000 records ( 0.2%)
A model trained on this data can hit 99.8% accuracy by predicting "normal" for everything, and still catch zero attacks.
Data balancing techniques:
Oversampling: Boosting the minority class by copying its samples or generating synthetic ones (with techniques like SMOTE). You scale up the underrepresented class.
Undersampling: Cutting down the majority class by randomly dropping samples from it. You scale down the overrepresented class.
Hybrid: A mix of both. Bump up the minority a little, trim the majority a little.
Class weighting: Instead of changing the data, you put more weight on the minority class in the model's loss function. You're basically telling the model that missing an attack is 100 times worse than mislabeling normal traffic.
Security angle:
Security models trained without balancing miss real attacks (low recall).
Too much oversampling can lead to overfitting, where the model memorizes the synthetic samples.
Pick your balancing strategy based on the use case. Do you want high recall (never miss an attack) or high precision (no false alarms)?
🍕 Real-World Example: An e-commerce company is building a fraud detection model. The dataset has 10 million normal transactions and 500 fraudulent ones. Trained without balancing, the model never catches any fraud. After generating synthetic fraud samples with SMOTE and applying class weighting, the model catches 89% of fraud while keeping the false positive rate at an acceptable level.
🔷 PART 2: Data Types
AI models work with different kinds of data, and each type comes with its own security challenges and processing methods. You'll need to know these three data types and the differences between them for the exam.
1. Structured Data
What it means: Orderly data that follows a predefined schema and format. It lives in tables made of rows and columns.
Analogy: Picture an Excel sheet. Every column has a header (name, age, IP address) and every row is a record. The data is orderly, searchable and queryable.
Examples:
Database tables (SQL)
Firewall logs (timestamp, source IP, destination IP, port, action)
CSV files
Structured event records in a SIEM
User authentication logs
Why it's an advantage in security:
Easy to query and analyze
Usable directly by ML models with minimal preprocessing
Statistical methods for anomaly detection apply easily
Security challenges:
Exposed to attacks like SQL injection
Sensitive data (PII) sits out in the open, so it needs masking and encryption
Storing and protecting high-volume structured data gets expensive
🍕 Real-World Example: A SIEM takes in thousands of structured log records a minute. Every record has the same format:
timestamp | source_ip | dest_ip | port | protocol | action. Because the structure is so orderly, the ML model can easily spot patterns like "more than 100 connection attempts to different ports from the same IP in the last 5 minutes." That's a sign of a port scan.
2. Semi-structured Data
What it means: Data that doesn't follow a rigid schema but still has some organization to it. It's organized with tags or keys, but each record can have different fields.
Analogy: Picture a stack of business cards. Every card has a name and phone number, but some have an email and some don't. Some list a company, others a fax number. There's a structure, just not a rigid one.
Examples:
JSON files
XML files
YAML config files
Email (headers are structured, the body is unstructured)
HTML pages
NoSQL databases (like MongoDB)
A JSON example, a security event:
{
"event_id": "SEC-2024-001",
"timestamp": "2024-01-15T03:22:11Z",
"source_ip": "185.234.xx.xx",
"alert_type": "brute_force",
"details": {
"attempts": 500,
"target_user": "admin",
"geo_location": "Russia"
},
"notes": "Repeated login attempts detected during night hours"
}
Why it's an advantage in security:
Flexible structure that can hold data of different formats from different sources
APIs usually return data as JSON/XML, things like threat intelligence feeds and cloud logs
Human-readable
Security challenges:
Inconsistent structure makes automated analysis harder, since some records may be missing fields
Vulnerable to XML/JSON injection attacks
Without schema validation, malicious data can sneak in
Security holes can crop up during parsing
3. Unstructured Data
What it means: Free-form data with no predefined format or organization. Roughly 80 to 90% of all the data in the world is unstructured.
Analogy: A drawer full of papers, photos, post-it notes and voice memo tapes all thrown in together. It all holds data, but none of it is in an orderly structure.
Examples:
Free text (email bodies, chat messages, forum posts)
Images and video (security camera footage, CAPTCHAs)
Audio files (recorded phone calls)
PDF documents
Social media posts
Dark web forum content
Malware binaries
How it's used in security:
Analyzing phishing email content with NLP
Scraping dark web forums to pull out new threat intel
Malware binary analysis
Deepfake detection (video/audio)
Detecting social engineering attacks
Security challenges:
The hardest data type to process and analyze, since it needs AI/ML
Hidden sensitive info can live inside it, and detecting that automatically is hard
Requires large-volume storage and processing
It's more complex for DLP (Data Loss Prevention) systems to scan unstructured data
Comparing the three data types:
| Feature | Structured | Semi-structured | Unstructured |
|---|---|---|---|
| Format | Rigid schema (table) | Flexible schema (tagged) | No schema (free-form) |
| Storage | SQL databases | NoSQL, files | File systems, object storage |
| Search | SQL queries | XPath, JSONPath | Full-text search, NLP |
| AI fit | Used directly | Needs parsing | Needs heavy preprocessing |
| Volume | ~10-20% | ~5-10% | ~80-90% |
| Example | Firewall logs | JSON threat feed | Email content |
🍕 Real-World Example: A threat intelligence team has to merge three kinds of data from different sources: Structured (SIEM logs with IP, port, timestamp), Semi-structured (JSON reports from the VirusTotal API, where each report can have different fields) and Unstructured (free-text messages scraped from dark web forums). Merging the three types means building a separate preprocessing pipeline for each. NLP analyzes the unstructured text, a JSON parser breaks down the semi-structured data and SQL queries pull the structured data. Then it all gets combined for correlation in the threat intelligence platform.
🔷 PART 3: Watermarking
What Is Watermarking?
What it does: Adds an invisible or visible mark to AI-generated content (text, images, audio, video) so the content's origin can be traced and verified.
Analogy: Think of the watermark inside paper money. You can't see it at a glance, but hold it up to the light and it appears. It proves the bill is real and helps catch counterfeiting attempts. AI watermarking works on the same logic.
Why you need it: Telling the content generative AI produces (deepfakes, synthetic text, fake voices) apart from real content is getting harder and harder. Watermarking is one of the main solutions to this problem.
Types of watermarking:
Text Watermarking
Hiding statistical patterns in the text the model generates
Leaving a hidden "signature" in specific word choices, sentence structures or token distributions
Imperceptible to a human, but detectable algorithmically
Image Watermarking
Hiding a signature by making invisible changes to the pixels
Frequency changes the human eye can't pick up
It's important that it stays robust against cropping, compression and resizing
Model Watermarking
Adding a watermark to the model itself, to prove ownership if the model gets stolen (model theft)
Adding special "trigger" inputs to the model, where only that model responds to those inputs in a specific way
Why it matters in security:
Deepfake detection: Detecting fake video/audio/images produced by AI
Fighting disinformation: Marking AI-generated news text and tracing its source
IP protection: Proof of ownership against model theft
Compliance: Regulations like the EU AI Act are starting to require AI-generated content to be marked
Forensic analysis: Verifying how trustworthy AI-generated evidence is in a security incident
The challenges of watermarking:
Attackers may try to strip the watermark out (watermark removal attacks)
The watermark shouldn't degrade the quality of the content
It has to hold up against different compression methods and transformations
False positives, where content with no watermark gets wrongly detected as "watermarked"
🍕 Real-World Example: During an election season, a fake AI-generated audio clip starts spreading, making it sound like a politician said things they never said. If the clip was produced with a watermarked AI system, watermark analysis can pull out the detail that "this recording was generated by model X on date Y." But if the attacker used a model with no watermark, detection gets much harder. That's exactly why you need both watermarking and watermark-independent deepfake detection techniques.
🔷 PART 4: Retrieval-Augmented Generation (RAG)
What Is RAG?
What it does: Lets an LLM produce answers by pulling real-time information from outside knowledge sources (databases, documents, the web), beyond its own training data.
Why you need it: LLMs have two big problems:
Knowledge cutoff: The model doesn't know anything from after its training date. A model trained in 2023 doesn't know about the vulnerabilities found in 2024.
Hallucination: The model can confidently produce wrong information even on a topic it knows nothing about.
RAG solves both. It makes the model consult current, trustworthy sources before it generates an answer.
How RAG works (step by step):
User question → Convert the query to an embedding → Find the most relevant
documents in the vector DB → Hand those documents to the LLM as context →
The LLM generates its answer using that context
The flow in more detail:
Indexing: Knowledge sources (documents, wiki pages, threat intelligence reports) get split into chunks, converted to embeddings and loaded into a vector database.
Retrieval: The user's question gets converted to an embedding. That embedding is used to find the most similar document chunks in the vector database.
Augmentation: The relevant document chunks that were found get handed to the LLM as context, along with the user's question.
Generation: The LLM generates an answer using both its own knowledge and the context it was given.
How RAG is used in security:
Letting SOC analysts ask questions grounded in the current CVE (Common Vulnerabilities and Exposures) database
Building an AI assistant that knows the organization's internal security policies
Querying threat intelligence reports in real time
Quickly surfacing the relevant procedures during incident response
🍕 Real-World Example: A SOC team spots a new attack pattern on the night shift. The analyst asks a RAG-powered AI assistant: "Which APT group is this attack signature linked to?" The RAG system scans the organization's threat intelligence database, the last 30 days of CVE records and the MITRE ATT&CK framework, then answers with the most current info available. The LLM on its own might not have had any of this. But thanks to RAG, it produced an answer that's current and correct.
Vector Storage
What it does: Specialized databases that store embeddings (vectors) and can run fast similarity searches across them.
How it differs from a traditional database: In a SQL database you run exact-match searches, like SELECT * WHERE name = 'John'. In a vector database you run similarity searches instead, like "find the 5 document chunks most similar to this question."
Analogy: A traditional database is like a library catalog where you have to know the book's exact title. A vector database is like telling a librarian "I want books about computer security, but specifically about network attacks." The librarian brings you the books that are closest in meaning.
How it works:
Text/data gets converted into numerical vectors with an embedding model (a 1536-dimensional vector, for example)
Those vectors get saved into the vector database
At search time, the system computes the similarity between the query's vector and every vector in the database (using cosine similarity, euclidean distance and the like)
The documents with the most similar vectors get returned
Popular vector databases: Pinecone, Weaviate, Milvus, Chroma, pgvector (a PostgreSQL extension)
Security angle:
Data-at-rest security: Even though the embeddings in a vector database don't let you perfectly reconstruct the original data, they can still leak information about sensitive content (inversion attacks). Encryption is a must.
Access control: Different users' access to different document sets needs to be controlled. A SOC analyst probably doesn't need access to every executive report.
Data poisoning: If wrong or malicious documents get deliberately injected into the vector database, the RAG system starts returning wrong information.
Query injection: A user's query can be manipulated to gain access to documents they normally couldn't reach.
Data leakage: The RAG system can include information in its answer that the user wasn't supposed to see, so you need proper access control and output filtering.
🍕 Real-World Example: A company loads its HR documents and security policies into the same vector database. An employee asks the AI assistant "what's the salary policy?" and the RAG system retrieves not just the general policy but also a confidential document with manager salary details, and folds it into the answer. The reason: access control wasn't applied at the vector database level. That's a serious data leak. The fix is metadata-based filtering. Store each document's security level as metadata in the vector database and filter by the user's clearance level.
Embeddings
What it does: Converts text, images or other data types into fixed-size numerical vectors. These vectors capture the semantic meaning of the data.
Analogy: Think of a GPS coordinate. The word "Istanbul" maps to the point (41.0, 29.0) on a world map. In the same way, the phrases "network intrusion" and "network breach" map to nearby points in vector space, because they're close in meaning.
How it works technically:
An embedding model turns a word or sentence into a vector with hundreds or thousands of dimensions:
"phishing attack" → [0.23, -0.45, 0.78, 0.12, ..., -0.34] (1536 dimensions)
"ataque de phishing" → [0.21, -0.43, 0.76, 0.14, ..., -0.32] (1536 dimensions)
"chocolate cake" → [0.89, 0.12, -0.67, 0.55, ..., 0.91] (1536 dimensions)
Notice how the first two vectors land very close to each other while the third is way off. The first two mean roughly the same thing (one's English, one's Spanish), and the third is a completely different topic. Embeddings are language-independent, which is why "phishing attack" and its Spanish equivalent end up as neighbors.
Measuring similarity: The similarity between two vectors is usually measured with cosine similarity. The closer the value is to 1, the more semantically similar the two pieces of text are:
cosine_similarity("phishing attack", "ataque de phishing") ≈ 0.95 (very similar)
cosine_similarity("phishing attack", "chocolate cake") ≈ 0.08 (nothing alike)
How embeddings are used in security:
Finding similar attack patterns: Convert a new attack log into an embedding and find similar past attacks in the vector database
Threat intelligence correlation: Semantically matching threat reports that come in from different sources
Semantic search: A search for "ransomware attack procedure" also surfacing the "ransomware incident response playbook" document
Anomaly detection: Building embeddings of normal system behavior and detecting the abnormal ones
Malware family classification: Comparing the embeddings of malware behaviors to decide whether they belong to the same family
Security angle:
Inversion attacks: It may be possible to partially reverse-engineer the original text out of an embedding. Keep this risk in mind when you convert sensitive data into embeddings.
Embedding poisoning: By manipulating the embeddings of documents with malicious content, an attacker can make a RAG system return wrong results.
Model dependency: When the embedding model changes, all your embeddings have to be recomputed, and inconsistencies can creep in during that process.
Privacy concerns: Embeddings should be treated as just as sensitive as the original data, because they preserve its semantic meaning.
🍕 Real-World Example: A threat intelligence platform wants to analyze APT reports written in different languages (English, Russian, Chinese). They convert each report into an embedding. Because embeddings work independent of language, a Russian-language APT28 report and an English-language Fancy Bear report come out very close to each other in vector space, since they're about the same group. The platform can automatically correlate reports across different languages.
🔷 QUICK-REFERENCE TABLES
Data Processing Concepts
| Concept | What it does | Security connection |
|---|---|---|
| Data Cleansing | Clears out errors and noise | Catches poisoned data firsthand |
| Data Verification | Checks that data is correct | Detects bad labels and manipulation |
| Data Lineage | Tracks the data's journey | Lets you trace a problem to its source |
| Data Integrity | Guarantees data isn't corrupted | Detects tampering via hash/signature |
| Data Provenance | Documents the data's origin | Weeds out untrustworthy sources |
| Data Augmentation | Grows the dataset with synthetic samples | Multiplies rare attack samples |
| Data Balancing | Fixes class imbalance | Boosts recall in attack detection |
Comparing Data Types
| Structured | Semi-structured | Unstructured | |
|---|---|---|---|
| Structure | Rigid schema | Flexible schema | No schema |
| Example | SQL table, CSV | JSON, XML | Email, PDF, video |
| AI fit | High | Medium | Low (needs preprocessing) |
| Volume | 10-20% | 5-10% | 80-90% |
RAG Components
| Component | Its role | Security risk |
|---|---|---|
| Embedding | Turns data into a vector | Inversion attack, privacy leakage |
| Vector Storage | Stores and searches vectors | Data poisoning, unauthorized access |
| Retrieval | Fetches the relevant documents | Query injection, data leakage |
| Generation | The LLM produces an answer | Hallucination, prompt injection |
🔷 EXAM TIPS
💡 Tip 1: Know the data lineage vs. data provenance difference cold. Provenance = the ORIGIN (where did it come from?). Lineage = the JOURNEY (what did it pass through?). The exam may give you questions that make you choose between the two.
💡 Tip 2: On data balancing questions, know the pros and cons of oversampling and undersampling. Oversampling carries an overfitting risk. Undersampling leads to information loss.
💡 Tip 3: On RAG questions, know that RAG reduces hallucination but doesn't eliminate it completely. The model can still misinterpret the context it was given.
💡 Tip 4: Watermarking isn't just for images. Text, audio, video and even the model itself can be watermarked. The exam may have questions that test whether you know the difference between "model watermarking" and "content watermarking."
💡 Tip 5: Know the differences between structured, semi-structured and unstructured data with clear examples. Don't forget that JSON is semi-structured especially. It looks structured but it has a flexible schema.
💡 Tip 6: Know that embeddings carry a security risk. Embeddings are not "anonymous" or "safe." They can leak information about the original data. That's why embeddings need to be protected like sensitive data too.
🔷 BONUS: Concepts Not in the Objectives, but Worth Knowing
Data Pipeline Security
The whole process of data collection → processing → storage → model training is called the "data pipeline." Every point in this pipeline is an attack surface. The security of the pipeline directly affects the security of the model. If you know the CI/CD concept, you can think of the data pipeline as the "CI/CD for data."
Data Drift
This is when real-world data changes over time after the model has been trained. A malware detection model got trained on 2023 data, but by 2025 the attack patterns are different. The model is now out of date. That's data drift. Monitoring for it keeps the model current.
Feature Engineering
The process of extracting the most meaningful features from raw data so the model can learn. For example, pulling features like "requests per hour," "number of unique destination IPs" and "average packet size" out of network traffic logs. Good feature engineering improves model performance dramatically.
Tokenization (in the NLP context)
Splitting text into the smallest units a model can process (tokens). "Cybersecurity" might become ["Cyber", "security"] or ["Cy", "ber", "security"]. Token limits determine the context window size in RAG systems, and therefore how much information you can supply. This ties into the "token limits" topic in Section 2.2.
Chunking (in the RAG context)
Splitting large documents into smaller pieces for a RAG system. If the chunk size is too big, you pull in irrelevant info too (low precision). Too small, and you lose context (low recall). Finding the optimal chunk size directly affects how well your RAG system performs.
Drilling the material: Reading is one thing, recall is another. I built BREACH // PROTOCOL, a roguelite-style question app (spaced repetition, active recall and an exam sim mode) to actually drill this stuff. It's free and open source. → https://github.com/Furkan-Taskin/breach-protocol
More sections dropping in this series. Follow along if you're on the same grind.
Top comments (0)