Azure data-engineering interviews usually open with "explain your project", and in most projects Azure Data Factory (ADF) is the orchestrator, so it gets the most follow-up questions. Interviewers move fast from definitions to "what happens when it fails". Here are 10 questions that come up again and again, with model answers. Limits and behaviours were checked against Microsoft Learn in November 2026; Azure changes them, so confirm before you quote one in production.
1. What is the difference between a linked service, a dataset and an integration runtime?
Level: Easy. What the interviewer is testing: whether you separate connection, data shape and compute.
A linked service is how to connect: the endpoint plus authentication, ideally a managed identity or a Key Vault reference. A dataset is which data inside that store: a table, a folder, a file format. The integration runtime (IR) is where the work runs. One linked service can back many datasets.
2. Which integration runtime would you use, and when?
Level: Medium. What the interviewer is testing: choosing compute from where the data lives.
| IR | Runs | Use it when |
|---|---|---|
| Azure IR | Data flows, data movement, activity dispatch | Cloud sources and sinks; with a managed virtual network, private endpoints too |
| Self-hosted IR | Data movement, activity dispatch | On-premises or private-network sources, or connectors that need your own driver |
| Azure-SSIS IR | SSIS packages | Lifting existing SSIS packages to Azure |
Details that impress: the self-hosted IR runs on Windows, only makes outbound connections, and supports up to four nodes for high availability. Mapping data flows cannot run on it.
3. What are the Lookup activity's limits?
Level: Medium. What the interviewer is testing: whether you know the limits that break real pipelines.
Lookup returns at most 5,000 rows (extra rows are dropped without an error), its output must stay under 4 MB or the activity fails, and it times out after 24 hours. The documented workaround is two levels: an outer pipeline pages through ranges and an inner pipeline looks up one chunk at a time.
4. What are the rules for ForEach?
Level: Medium. What the interviewer is testing: concurrency and its traps.
ForEach runs in parallel by default. batchCount caps concurrency (default 20, maximum 50) and isSequential makes it one at a time. You cannot nest a ForEach inside another ForEach; call a child pipeline with Execute Pipeline instead. Do not use Set Variable inside a parallel ForEach, because pipeline variables are shared by the whole run and parallel iterations overwrite each other.
5. When would you use a tumbling window trigger instead of a schedule trigger?
Level: Medium. What the interviewer is testing: matching the trigger to the shape of the work.
When each run processes one specific time slice that must be tracked. A tumbling window trigger keeps state per window, passes the window start and end to the pipeline, creates backfill runs (oldest first) when its start time is in the past, supports retries and dependencies (including on the previous window of itself), and limits parallel windows with maxConcurrency (1 to 50). It references exactly one pipeline. A schedule trigger just fires at times.
6. Activity A fails, and its "Upon Failure" handler succeeds. Did the pipeline succeed?
Level: Medium. What the interviewer is testing: how ADF decides a pipeline's status.
It depends on how the handler is wired. ADF evaluates the leaf activities (if a leaf was skipped, it looks at its parent):
- Only a failure path (Try-Catch): the pipeline reports Succeeded.
- Both a success path and a failure path (Do-If-Else): the success branch is skipped and its parent failed, so the pipeline reports Failed, even though the handler succeeded.
If a caught failure should still show red in monitoring, end the catch path with a Fail activity.
7. How do you implement a watermark-based incremental load?
Level: Medium. What the interviewer is testing: the most common incremental pattern and its edge cases.
Keep a watermark table with the last loaded value per source table. Each run: look up the old watermark, look up the new high-water mark, copy rows between the two, and update the watermark only after the copy succeeds, so reruns are safe.
WITH wm AS (SELECT last_value FROM etl_watermark WHERE table_name = 'src_orders'),
new_wm AS (SELECT MAX(modified_at) AS v FROM src_orders)
SELECT o.order_id, o.status, o.modified_at
FROM src_orders o, wm, new_wm
WHERE o.modified_at > wm.last_value
AND o.modified_at <= new_wm.v
ORDER BY o.modified_at;
Output (run on the practice dataset below in SQLite):
| order_id | status | modified_at |
|---|---|---|
| 3 | refunded | 2026-11-02 10:00:00 |
| 4 | paid | 2026-11-02 11:30:00 |
| 5 | pending | 2026-11-02 23:59:59 |
| 6 | paid | 2026-11-03 07:45:00 |
Order 3 was placed on 1 November but refunded on 2 November, so it is picked up again. That is why the watermark must be a modified timestamp, not the business date. A watermark cannot see deletes; for those you need change tracking or CDC.
8. How would you copy 300 tables without building 300 pipelines?
Level: Hard. What the interviewer is testing: whether your design scales.
Use a control table (schema, table, load type, watermark column and value, target path, active flag). A Lookup reads the active rows, a ForEach iterates in parallel with a batchCount the source can handle, and each iteration calls a child pipeline with parameterised datasets that copies one table, writes it to raw/<system>/<table>/load_date=..., updates its watermark and writes an audit row. One table failing must not stop the others. Adding a table means inserting a row, with no deployment.
9. On ADLS Gen2, how do Azure RBAC and ACLs work together?
Level: Medium. What the interviewer is testing: why an ACL sometimes "doesn't work".
Azure checks role assignments first (including any conditions). If a role grants access, ACLs are not evaluated at all, so you cannot use an ACL to take away access a role already gives. Only when no role grants access do ACLs decide. To read a file through ACLs you need Read on the file and Execute on every folder above it. Requests authorised with Shared Key (or an account or service SAS) bypass RBAC and ACLs entirely, which is why many teams disable Shared Key on lake accounts. Assign ACLs to Microsoft Entra security groups, not individuals.
10. Why enable the hierarchical namespace, and what do you give up?
Level: Medium. What the interviewer is testing: whether you check feature support before you design backup and DR.
The hierarchical namespace makes directories real: renames and deletes of a folder are single atomic operations, which makes Spark and Delta commits faster and safer, and you get POSIX-style ACLs. The upgrade from a plain Blob account is one-way. On a hierarchical-namespace account, blob versioning, change feed, point-in-time restore and object replication are not supported, while soft delete, lifecycle management and access tiers are. So recovery plans rely on soft delete, Delta time travel and copies to a second account.
Practice dataset
Show the practice dataset (SQL)
CREATE TABLE src_orders (order_id INTEGER PRIMARY KEY, customer_id INTEGER, order_date TEXT, amount REAL, status TEXT, modified_at TEXT);
INSERT INTO src_orders VALUES (1,10,'2026-11-01',1200,'paid','2026-11-01 09:20:00'),(2,11,'2026-11-01',450,'paid','2026-11-01 18:45:00'),(3,10,'2026-11-01',300,'refunded','2026-11-02 10:00:00'),(4,12,'2026-11-02',990,'paid','2026-11-02 11:30:00'),(5,13,'2026-11-02',150,'pending','2026-11-02 23:59:59'),(6,11,'2026-11-03',700,'paid','2026-11-03 07:45:00');
CREATE TABLE etl_watermark (table_name TEXT PRIMARY KEY, last_value TEXT);
INSERT INTO etl_watermark VALUES ('src_orders','2026-11-01 23:59:59');
How to practise these
Build the smallest version of each answer: one Copy pipeline, one ForEach over three tables, one Try-Catch path, and check the pipeline status yourself. Then break something on purpose (a wrong path, an expired permission, a duplicate file) and explain out loud what you would check first. That is exactly what the scenario round tests.
Generated with AI by ATS Data Labs. The SQL output above comes from running the query in SQLite. Limits and behaviours were checked against Microsoft Learn in November 2026; Azure changes them, so confirm details in the current docs before relying on them.
Top comments (0)