DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

S3 Burst on ONTAP Files — Collect via S3 API, Consume via FlexCache NFS/SMB

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 10 Aug 2026 01:06:17 +0000

Introduction

You have an S3 ingestion pipeline. But the consumers only see an NFS mount. Sound familiar?

You want to deliver driving logs to a HiL test bench, stage design jobs for an EDA toolchain, push rendering assets to production nodes, or feed genomic sequencer output to an HPC cluster. In each case the writer speaks S3 API in the cloud while the reader speaks NFS/SMB on physical hardware. Without a way to connect these directly, you end up adding a sync job in between — and with it comes latency, cost, and one more thing to monitor.

This structure spans industries: automotive, semiconductor, media/VFX, oil and gas, life sciences, manufacturing, remote work, and IoT (see the full table in When This Pattern Fits below).

This post introduces a pattern that removes that sync job. Write to an Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP) S3 Access Point, and the data appears on a FlexCache NFS mount. No copy job, no configuration between the two. I measured all four directions to see how fast the visibility propagates.

Here's the conclusion up front:

S3 PutObject → FlexCache NFS read visibility: p50 8 ms. No copy job, no configuration
FlexCache overhead is +5 ms (vs. reading origin directly) — nearly transparent for same-region VPC peering
Reverse direction (NFS write → S3 AP read) is p50 44 ms, dominated by S3 API overhead. Not the main path in this design
FlexCache duality (NAS bucket S3 reads on FlexCache volumes) works, but only after -is-s3-enabled true is set on the cache volume in advanced privilege. My first attempt returned AccessDenied because that step was missing

Repository: Yoshiki0705/s3-burst-on-ontap-files

The Architecture

[S3 Client] --PutObject--> [S3 Access Point] --> [Origin Volume]
                                                       |
                                                  FlexCache
                                                       |
                                                [NFS/SMB Client]

Layer	What	Protocol
Collect (write)	S3 Access Point on FSx for ONTAP	S3 API
Source of truth	Origin volume	—
Distribution	FlexCache	Cluster/SVM peering
Consume (read)	Cache volumes	NFS/SMB only

The S3 Access Point attaches to the origin only. Cache volumes serve NFS/SMB. This single decision simplifies the design considerably:

One write path. The origin is authoritative; writes always go through the S3 AP
No S3 implementation differences pushed to the edge
Cache side requires only FlexCache + NFS/SMB — fewer platform constraints to hit

Test Environment

Item	Value
Date	2026-08-09
ONTAP version	9.18.1P3D1 (both clusters)
Configuration	SINGLE_AZ_1, 128 MBps × 2 clusters
Connectivity	VPC peering (same region, same account)
Mount	NFSv3, `actimeo=0` (client cache disabled)
Object size	64 bytes
Concurrency	1
Method	boto3 persistent session, same host, 30 iterations

Why NFS First

The initial measurements used NFS. Three reasons:

Client cache control. NFS actimeo=0 fully disables kernel attribute caching, isolating storage-side visibility latency. SMB oplocks/leases let the client cache independently, making it harder to measure the same thing at the same granularity.
S3 Access Point requirement. The FSx for ONTAP S3 AP attaches to UNIX security-style volumes. SMB access on a UNIX volume is possible (via name-mapping + permission mapping), but NFS is the simplest consumption path for this architecture.
Primary use-case distribution. HiL test benches, rendering farms, and IoT analysis appliances are predominantly Linux + NFS. When the read side is SMB-dominant (Windows workstations in a production studio), the origin security style changes to NTFS — a different design path (see First Decisions).

SMB Shows Identical Results

I ran a separate verification with SMB (mount -t cifs, cache=none) on the same architecture:

Protocol	Mount method	p50	p90	max	n
SMB	`mount -t cifs`, `cache=none`	7 ms	8 ms	9 ms	30
NFS	`mount -t nfs`, `actimeo=0`	7 ms	8 ms	15 ms	30

With persistent mounts, SMB and NFS have identical visibility latency. No protocol difference.

Note: smbclient (establishing a new session per request) showed p50 43 ms. This is SMB session setup overhead — the same structural problem as the AWS CLI cold-start that inflated our first NFS→S3 measurement to 873 ms. Production environments with persistent connections won't see this.

SMB Considerations

FlexCache serves both NFS and SMB. This architecture doesn't exclude SMB — but note:

With a UNIX security-style origin, SMB clients receive UNIX permission-based access control, not NTFS ACLs
For SMB-primary workloads, NTFS security style is more natural, but S3 AP attaches to UNIX-style volumes. This is a genuine tradeoff for "collect via S3, consume via SMB" scenarios
mixed security style is available in the API but officially not recommended by AWS — it's labeled "advanced users only." Permission type is determined by the last client that wrote, making troubleshooting difficult. This architecture doesn't use it
SMB requires Active Directory (FSx for ONTAP does not support workgroup mode)

Recommendation for this architecture: Origin uses UNIX security style (S3 AP requirement). SMB readers use name-mapping and accept UNIX permission-based access. If your workload requires NTFS ACL granularity for Windows clients, this architecture isn't the best fit.

actimeo=0 measures the minimum visibility latency by disabling client-side caching. Production deployments should use appropriate actimeo values for their workload — the default (~60 s) means subsequent reads hit kernel cache at ~0.05 ms, but changes aren't visible during that window.

Results — All Four Directions

#	Direction	p50	p90	p99	max
1	S3 AP PutObject → FlexCache NFS read	8 ms	9 ms	19 ms	19 ms
2	S3 AP PutObject → Origin NFS read (direct)	3 ms	5 ms	8 ms	8 ms
3	NFS write (Origin) → FlexCache NFS read	6 ms	7 ms	25 ms	25 ms
4	NFS write (Origin) → S3 AP GetObject	44 ms	49 ms	328 ms	328 ms

What the Numbers Say

Direction 1 is the main path. S3 write, FlexCache NFS read: p50 8 ms.

The +5 ms gap between directions 1 and 2 is FlexCache overhead. For same-region VPC peering, FlexCache is nearly transparent.

Direction 3 is faster than 1 (6 ms < 8 ms). NFS writes commit directly to the origin — no S3 API overhead — and FlexCache propagation alone is faster than the full S3-to-cache path.

Direction 4 (reverse) is slowest at 44 ms. S3 read-side processing dominates. This design keeps reads on NFS/SMB, so direction 4 isn't the main path.

NFS Client Cache Effect

Condition	p50
`actimeo=0` (cache disabled)	7 ms
`actimeo=60` (subsequent access)	0.05 ms (kernel cache hit)

With defaults, reads within 60 seconds of the last attribute check hit the kernel cache. The tradeoff: you won't see changes during that window. Tune per workload.

FlexCache Duality — It Works (With One Extra Step)

ONTAP 9.14.1 introduced NAS buckets on FlexCache volumes (FlexCache duality) — S3 reads from the cache side. I tested this on FSx for ONTAP 9.18.1P3D1.

Result: Works After Enabling `-is-s3-enabled true`

My initial test returned AccessDenied and I concluded it didn't work. After additional investigation, I found missing configuration step: S3 access must be explicitly enabled on the FlexCache volume:

set -privilege advanced
flexcache config modify -vserver <svm> -volume <fcache_vol> -is-s3-enabled true

After applying this setting:

Operation	Before (missing setting)	After (-is-s3-enabled true)
HeadBucket	✅	✅
ListObjectsV2	❌ AccessDenied	✅ KeyCount=1
GetObject	❌ AccessDenied	✅ Content verified

The fsxadmin role on FSx for ONTAP has access to advanced privilege commands including flexcache config modify. Documented at NetApp: Enable FlexCache duality.

Design Implication

FlexCache duality works, which means S3 reads from the cache side are possible. However, this architecture still recommends NFS/SMB on the cache side because:

ONTAP native S3 (NAS buckets) and AWS-managed S3 Access Points are different mechanisms
NAS buckets are read-only (no PutObject)
Requires additional configuration (advanced privilege + S3 user management)
No IAM integration or access point policy governance like S3 AP provides

FlexCache duality becomes an option when "S3 reads at the cache site" is a hard requirement. For AWS service integration (Lambda, Bedrock, etc.), the origin-side S3 AP remains the better fit.

When This Pattern Fits

"Collect via S3 API in the cloud, consume via NFS/SMB at the edge" — this structure exists across industries.

Industry	Collect side	Consume side	Example
Automotive (AV/ADAS)	Driving logs and sensor data ingested to S3	HiL test benches replay via NFS	AWS + NetApp: Hybrid Cloud HiL
Semiconductor (EDA)	Design job I/O staged via S3	Toolchains (Synopsys, Cadence) run on NFS	EDA Scale with FSx for ONTAP
Media and VFX	Rendering assets collected to S3	Artist workstations mount SMB/NFS	FlexCache: distributed product development
Oil and Gas	Seismic survey data uploaded to S3	Interpretation workstations mount NFS	VDI for Subsurface O&G
Life Sciences	Genome sequencer output stored in S3	Bioinformatics HPC processes via NFS	Sequencer → S3 → NFS pipeline
Manufacturing / QA	Inspection camera images collected to S3	Line-side inspection software reads via NFS	Image → judgment → archive flow
Remote Work	Central design data updated via S3	Remote-site WorkSpaces access FlexCache NFS/SMB	FlexCache in AWS WorkSpaces
IoT / Edge	Sensor data streamed to S3	On-site analysis appliances read via NFS	Factory gateway → cloud → shopfloor

Common structure: Few write sites (often one), multiple read sites. Writes are bursty; reads touch only what's needed.

When It Doesn't Fit

Need full S3 semantics (versioning, event notifications, lifecycle): Use S3 natively
Object names aren't NAS-friendly (flat namespace, millions of keys with no directory separators): Performance degrades as root directory grows
Consumers need S3 reads at the cache site: Not supported on FlexCache today
Need write-back from the cache: This pattern keeps cache read-centric
Need conditional writes (If-None-Match): Returns 501 NotImplemented. Handle exclusion at the application layer

S3 Access Point Design Notes

A few things to know when using the S3 AP as the collect layer.

S3 AP ≠ Amazon S3

The FSx for ONTAP S3 AP supports a subset of S3 operations. GetObject, PutObject, ListObjectsV2, HeadObject, DeleteObject, and MultipartUpload work. The following do not:

S3 Event Notifications (use FPolicy + EventBridge instead)
Lifecycle rules (use FabricPool)
Versioning (use ONTAP Snapshots)
Conditional writes If-None-Match (returns 501)
S3 Select, SSE-S3/KMS, Cross-AP Copy

Throughput is shared with NFS/SMB

S3 AP, NFS, and SMB all consume the same FSx for ONTAP provisioned throughput. In this architecture, origin and cache are separate clusters so this rarely matters — but if NFS clients also access the origin directly, account for the shared bandwidth. Rule of thumb: 128 MBps supports 2–5 concurrent S3 requests, 512 MBps supports 10–20.

Directory design matters

S3 PutObject keys map directly to directory structure. Beyond 100K files per directory, ListObjectsV2 slows down. Partition by date, tenant, or hash prefix.

This directly affects NFS usability. If you flat-dump millions of objects without hierarchy, FlexCache NFS clients will struggle with ls and find. Design your S3 keys with "how does this look when I ls on NFS?" in mind.

Key recommendations:

Date-based partitioning (year=YYYY/month=MM/day=DD/) to keep directories under 10K files
Separate ingest volumes from consumption volumes; apply FlexCache only to consumption
On the NFS side, use manifest files or path generation instead of directory traversal (find)

For full details, see the S3 AP Design Guide in the repository.

What It Costs — Egress First, Requests Second

I framed this section around S3 request pricing when I first wrote it. That was the wrong axis, and the modelling I did afterwards says so plainly. Corrected below.

The cost that hurts when your readers sit outside AWS is data transfer. Egress is charged on bytes leaving the Region, so reading the same file ten times moves ten times the bytes and pays ten times over. A cache removes the multiplier: it carries the working set once and every later read is served locally over NFS or SMB, never becoming an S3 request at all.

A worked case — 20 TiB dataset, 2 TiB monthly working set, 4 MiB objects, each file read 30 times, internet egress from ap-northeast-1:

Option	Monthly
Read S3 directly from on premises	$6,211 — egress alone is $5,693 (92%)
Copy the whole dataset down with DataSync	$2,847
FSx for ONTAP + FlexCache	$1,333

Storage and requests are rounding errors next to transfer here.

The read count decides it

Reads per file per month	Direct	This architecture	Ratio
1	$746	$1,333	0.6x
5	$1,680	$1,333	1.3x
10	$2,593	$1,333	1.9x
30	$6,211	$1,333	4.7x
100	$18,451	$1,333	13.8x

At one read per file the direct path wins — the same bytes move either way and there is no reason to carry a file system's fixed cost. The crossover is between five and ten. Where it falls is a property of your workload, not of the products.

Requests matter too, but only below a certain object size

I had assumed S3 GET charges were a co-equal problem. At 4 MiB objects they are not: the entire 5.2 million reads in the ten-read case cost $1.94, about a thousandth of the transfer bill. Through an access point it is $0.31, and through FlexCache $0.16 — the reads are NFS and SMB, so they generate no S3 requests, and only the origin-side capacity pool fetch on cache fill remains.

Where the assumption does hold is small objects. Holding bytes read constant and varying object size:

Object size	Reads/month	GET charges	Share of that option's transfer bill
8 KiB	2.68 B	$993	48%
64 KiB	336 M	$124	6%
256 KiB	84 M	$31	1.5%
4 MiB	5.2 M	$1.94	0.1%

So both charges need designing for, and they respond to opposite remedies:

Transfer falls by carrying fewer bytes — cache the working set, move the readers into AWS, or drop the unit rate with Direct Connect ($0.041/GB against $0.114 for the first 10 TB of internet egress). Making objects bigger does not help; the byte count is unchanged.
Requests fall by making fewer calls — batch at collection time so files are larger, and serve reads over a file protocol so they never become S3 calls. Negotiating transfer rates does not help when the money is on the request side.

Batch too far and you hit the collection side: a single PutObject caps at 5 GiB and a whole object at 50 GiB, and the 50 GiB check happens at CompleteMultipartUpload — after the entire payload has been transferred and paid for. Batching past the consumer's read unit also sends bytes nobody reads. The repository's design guide sets the consumer's read unit as the reference for how far to go, with the measurement and monitoring steps.

If the readers can move, move them

Worth stating because it outweighs every storage choice. In-Region transfer is free, so putting the consumers in AWS deletes the entire egress line — $2,079 at ten reads on this workload. Reading S3 directly from EC2 comes to $514/month; S3 Files costs $515, ninety-four cents more, because 4 MiB objects sit above its size threshold and never reach its high-performance storage; FSx for ONTAP in the same Region is $1,053, twice the direct path, since the transfer gap that justified it is gone.

S3 Files cannot serve this architecture's consumers. It speaks NFSv4.1 and NFSv4.2 only — no NFSv3, no SMB — so an appliance pinned to v3 and any Windows stage are out, and the documented compute targets are EC2, Lambda, EKS and ECS. It belongs in the picture as the option you get after migrating the readers, not as a substitute for a cache.

This architecture is for the cases where the readers cannot move: equipment on site, proximity to whatever is being measured, capital already spent on it.

On the distribution side

A FlexCache cache volume cannot be tiered — an origin with FabricPool tiering can be cached, but the cache itself never tiers, so it sits entirely on SSD. That is affordable because it is sparse: NetApp's guidance is at least 10% of origin, which is also the create default. At 10% the distribution side runs 2.7x to 6.1x below a full second copy, with the copy given its capacity pool discount. Size it like a copy and it inverts — at 100% it costs more than the copy, precisely because it cannot tier.

Cache volumes are writable, which I understated earlier. Write-around is the default and withholds the client acknowledgement until the origin has committed; write-back, from ONTAP 9.15.1, commits at the cache and propagates asynchronously. The default being synchronous with respect to the origin matters for freshness: there is no window where a cache-side write is missing from the canonical copy.

One assumption drives more of these numbers than any other, so treat it carefully. Background storage efficiency does not run on data once it has been tiered — only savings applied while the block was on SSD are preserved, and a block tiered before efficiency ran keeps none (AWS, NetApp). Expecting better than 40% on a tiering-enabled volume is optimistic. The model takes the SSD rate from AWS's published per-workload figures, assumes the pool tier retains half of it, and carries a sensitivity table — because the assumption only ever flatters this architecture: ONTAP deduplication does not reduce an S3 storage bill.

Full breakdown, every rate with its effective date, the workload models and the cross-layer pitfalls: FinOps cost structure and the S3 AP Design Guide (Japanese). Cost tables are generated from a model in the repository, so a price change moves one declaration rather than fifty typed totals.

Deploy

CloudFormation and Terraform templates:

GitHub: Yoshiki0705/s3-burst-on-ontap-files

Conclusion

"Collect via S3, consume via NFS" — one volume, no copy jobs. FSx for ONTAP S3 Access Point + FlexCache. Main path p50 8 ms. FlexCache adds ~5 ms for same-region — nearly transparent.

This verification was done entirely on AWS (FSx for ONTAP to FSx for ONTAP over VPC peering), but the cache side isn't limited to AWS. On-premises NetApp AFF/FAS, ONTAP Select, Cloud Volumes ONTAP, Azure NetApp Files, Google Cloud NetApp Volumes cache volumes— any ONTAP-based environment can potentially serve as a FlexCache target for this same architecture. The repository's Portability and Support Matrix pages track what's confirmed and what's next. Cross-platform verification is on the roadmap.

FlexCache duality (S3 reads on the cache side) does work, once S3 access is enabled on the cache volume itself with -is-s3-enabled true in advanced privilege. The earlier AccessDenied was a missing setting, not a platform limitation. The architecture still keeps cache-side access to NFS/SMB, because ONTAP native S3 and the AWS-managed access point are separate mechanisms and the NAS bucket is a read-only view without IAM integration.

All test resources torn down. Numbers are from a specific test environment and vary by workload and configuration.

Embedding AI Agents into a File Portal — From AgentCore MCP to Multi-Agent Teams (Part 3)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:21:40 +0000

Let an AI agent work the NAS, with a human approving — Amazon Bedrock AgentCore and MCP (part 3 of 3)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1 I built the file portal foundation on the S3 Access Points of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP), and in Part 2 I embedded storage operations (ARP/AI incident response, Tamperproof Snapshots, regulatory retention management). At that point the UI was sufficient for anyone who already knew which button to press — but a different kind of friction was still there. "Where did I put last week's simulation results?" "Summarize the key points of this PDF in three lines." The intent is perfectly clear; the cost of translating it into system operations was what stayed high.

So I integrated AI agents into the portal. Express your intent in natural language, and the agent finds the files, reads them, analyzes the content, and proposes storage management operations when needed. Because storage management includes irreversible operations, the execution decision stays with a human through a Human-in-the-Loop (HITL) flow.

Here's the conclusion up front:

Combining the Amazon Bedrock Converse API with MCP tools lets you explore and analyze files on an S3 AP in natural language
Connecting a Bedrock Knowledge Base directly to an S3 AP means you can reach a file through semantic search without knowing its name
Wrapping destructive operations in a HITL approval modal keeps agent autonomy and safety in the same design
Multi-agent collaboration pays off only for tasks with several phases — "discover → analyze → judge"

In this article, I'll cover the 3-mode AgentChat design, file access via MCP tools, the HITL flow, and the implementation patterns for multi-agent collaboration.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Repository: solutions/amplify-portal/

Terminology note: "MCP" in this article refers to Model Context Protocol — a standard protocol that allows AI models to invoke external tools (file operations, database queries, etc.). AgentCore Gateway acts as the MCP server, exposing Lambda-based tools to the AI model.

If You're Already Using ChatGPT / Copilot / Claude for Document Analysis

When using general-purpose AI assistants (ChatGPT, GitHub Copilot, Claude, etc.) for file analysis, the typical flow is "upload file → AI analyzes → returns results." The portal agent takes a different approach.

Aspect	ChatGPT / Copilot (upload-based)	Portal Agent (NAS-connected)
File access	Manual upload (one file at a time)	Direct access to files on NAS (via S3 AP)
Scope	Only uploaded files	Cross-volume search across entire storage
Storage operations	Not possible	Proposes SnapLock enable, user block, etc. (with HITL approval)
ONTAP-specific features	Not supported	ARP/AI status checks, Snapshot operations, export-policy changes
Data movement	File content sent to external service	File content processed within same-region Bedrock (VPC-contained possible)
Destructive operation safety	N/A	HITL modal for human approve/reject

This portal does not attempt to replace general-purpose AI assistants. It adds NAS-connected file intelligence — file discovery, content understanding, and storage operation proposals. For general Q&A, ChatGPT and Copilot remain well-suited tools for those tasks.

Verification status: how far each feature in this article has actually been confirmed is recorded in verification results, split four ways (live E2E / live read / tests only / DemoMode). Agent and team execution sits in the "tests only" category: handler and component tests pass, but it has not been driven from a browser against a real system.

What I Built

The portal moved from "browse files and check operational state" to "ask in natural language, and the agent finds files, understands content, and proposes storage operations." Here's what was added:

AgentChat: Multi-tool AI chat with 3 switchable modes
SemanticSearch: Vector search via Bedrock Knowledge Base — resolves "where did I put that file?" in natural language
Agent Directory / Creator: Define custom agents and share them with your team
Agent Teams: Run multiple agents in collaboration
ActionApproval (HITL): Insert a human approval flow before destructive operations
Multimodal input: Drag and drop an image into the chat for file analysis

Why Embed AI Agents in the Portal

The admin features from Part 2 work well for people who already know "what to operate and how." But what I actually hear in practice sounds more like:

"Where did I put last week's simulation results?"
"Can you summarize what caused the error in this log file?"
"I want to run legal review on the contracts folder, but I don't know the steps"

These aren't cases of "not knowing how to operate the system." They're cases of "I know what I want, but translating that intent into system operations is tedious." This applies equally to non-technical team members:

"Where's the attachment for last month's expense report?"
"Summarize this PDF in 3 lines"
"Compile a file list from this folder for my weekly report"

The motivation for embedding AI agents was to eliminate this translation cost entirely. Express your intent in natural language, and the agent finds files, reads them, analyzes content, and proposes admin operations if needed. Users only need to think about "what they want to accomplish."

However, giving an agent unrestricted authority in a storage management context is dangerous. Enabling SnapLock is irreversible. Modifying an export-policy immediately cuts off user access. So I combine the Human-in-the-Loop (HITL) pattern: "the agent proposes, but humans approve execution."

AgentChat — 3-Mode Integrated Chat

A persistent AI chat in the portal's right panel, fully replacing the previous "Bedrock Q&A" (single-file question-answering).

3 Modes

Mode	Icon	Purpose
🧠 Knowledge	KB	Cross-volume semantic search. Answers "where's that file?"
📁 File Agent	agent	File read/list/analyze. Accesses S3 AP via MCP tools
🤖 Multi-Agent	multi	Multiple agents collaborate. File discovery → content analysis → operation proposal in sequence

Modes switch via pill buttons below the header. Welcome screen task cards change per mode, showing "what you can do in this mode."

Architecture

Browser (React)
    ↓ AppSync Query (agentQuery)
Lambda: Agent Orchestrator (outside VPC)
    ├── mode=kb   → Bedrock KB RetrieveAndGenerate
    ├── mode=agent → Bedrock Converse + MCP Tools
    └── mode=multi → Multi-agent collaboration
                         ↓ MCP Client
                    AgentCore Gateway
                         ↓ Lambda Invoke
                    MCP Tool Lambda (list_files / read_file / search_files)
                         ↓ S3 API
                    FSx for ONTAP S3 Access Point

Light theme shown. A dark theme version is available.

The VPC split principle applies here too. The Agent Orchestrator Lambda is placed outside VPC, accessing Bedrock API and S3 AP (Internet-origin). Admin operations calling ONTAP REST API are handled by a separate VPC-internal Lambda.

Tool Execution Visibility

When the agent invokes tools, a timeline appears in the chat:

🔧 Tool Calls (3)                          2.1s
├── ✅ 🔍 file-explorer: list_files         {"prefix": "/engineering/"}
├── ✅ 📄 file-explorer: read_file          {"key": "/engineering/thermal-spec-v3.pdf"}
└── ✅ 🧠 knowledge-analyst: analyze_file   {"key": "...", "question": "..."}

Each tool's input/output examples:

Tool	Input Example	Output Example
`list_files`	`{"prefix": "/engineering/", "max_keys": 20}`	File path list + size + last modified
`read_file`	`{"key": "/engineering/thermal-spec-v3.pdf"}`	File content (text-extracted)
`search_files`	`{"query": "thermal limit exceeded"}`	Matched files + relevant snippets

Displayed in <details> — expanded by default when 3 or fewer tool calls, collapsed when 4+. This balances Nielsen's "visibility of system status" with Wroblewski's "progressive disclosure."

Chat History Persistence

Conversation sessions auto-save to DynamoDB (2-second debounce). Past sessions can be recalled from the history panel to continue questioning.

// Auto-save: 2 seconds after message change → DynamoDB
useEffect(() => {
  if (messages.length === 0 || !currentSessionId) return;
  const timer = setTimeout(() => saveCurrentSession(), 2000);
  return () => clearTimeout(timer);
}, [messages, currentSessionId]);

The rationale: forcing an explicit "Save" button interrupts the natural flow of chat. Close the browser, and next time you open it, you can pick up from "where we left off yesterday."

Multimodal Input

Drag-and-drop images into the chat for Bedrock Vision analysis:

Circuit diagram image → "Where's the bottleneck in this circuit?"
Screenshot → "What's causing this error screen?"
Graph image → "Explain this performance degradation trend"

Supports JPEG/PNG/GIF/WebP under 5MB. Base64-encoded and sent to Bedrock Converse API.

SemanticSearch — Vector Search

Keyword search on file names has limits. When you want to find "the spec document about thermal design limits," you might find thermal-spec-v3.pdf by name, but TC-2024-0089.pdf? No chance.

Keyword / Semantic Mode Toggle

Mode toggle pills in the search bar:

🔍 [Search files...                        ] [🔍]
   [📂 Keyword]  [🧠 Semantic]

Keyword mode: S3 AP ListObjectsV2 + prefix filter. Auto-searches with 500ms debounce after 2+ characters
Semantic mode: Bedrock Knowledge Base RetrieveAndGenerate API. Vectorizes natural language queries for similarity search

Bedrock Knowledge Base Integration

FSx for ONTAP's S3 AP can be specified directly as a Knowledge Base data source (AWS official tutorial).

FSx for ONTAP Volume
    ↓ S3 AP (Data Source)
Bedrock Knowledge Base
    ↓ Embeddings (Titan V2)
OpenSearch Serverless (Vector Store)
    ↓ RetrieveAndGenerate
Results: related chunks + source file path + relevance score

Light theme shown. A dark theme version is available.

Results include scores (percentage display) and snippets. Clicking a result navigates to the file in the All Files view.

Semantic Search Input Examples

When KB is unconfigured, help text guides users:

💡 Semantic search examples:
• "Files with anomalies in last month's sales reports"
• "Test results exceeding thermal design specifications"
• "Description of the approval process for contract renewal"

Constraints

KB sync is asynchronous (minutes to tens of minutes lag after file changes)
Large file environments incur OpenSearch Serverless indexing costs
PDF/Office text extraction accuracy depends on file quality
If bedrockKbId is empty in portal-config.ts, semantic mode shows a "KB not configured" error

KB sync detection: Data source sync status can be checked via the bedrock-agent:GetIngestionJob API. Periodic sync can be scheduled in the Bedrock console. If immediate search reflection after file changes is required, consider an EventBridge + StartIngestionJob API configuration for on-demand sync triggers.

Agent Directory — Agent Catalog

A UI for discovering "what agents are available."

Preset Agents

Agent	Icon	Specialty
file-explorer	📁	File operations (list, read, search)
knowledge-analyst	🧠	Vector search + document analysis
safety-controller	🛡️	ARP status checks + containment action proposals
compliance-auditor	📋	Retention period and SnapLock configuration audits
ops-advisor	⚙️	EMS event analysis + operational recommendations

Custom Agent Creation (Agent Creator)

When presets aren't enough, define your own agents:

Agent Creation Wizard:
1. Set name and icon
2. Write system prompt
3. Select tools to use (checkboxes)
4. Test execution
5. Sharing settings (personal only / team-shared)

Created agents appear in the Agent Directory, available to team members (when shared). This lets you codify patterns like "this department frequently does this kind of analysis" as reusable agents.

Agent Teams — Multi-Agent Collaboration

For complex tasks that a single agent can't handle alone, multiple agents collaborate.

When to Use Multi-Agent vs Single Agent

Scenario	Recommended	Reason
"Summarize this PDF"	Single agent	Completes with one tool (read_file + analysis)
"Analyze all files in engineering/ and report security issues"	Multi-agent	Requires file discovery → content analysis → security judgment
"Audit files changed last month for compliance violations"	Multi-agent	File identification → content reading → regulatory assessment → report generation

Decision criteria: Multi-agent is effective when the task has multiple phases ("discover → analyze → judge") requiring different expertise at each phase. For simple tasks, multi-agent adds cost and latency without benefit — use single agent instead.

Example Team Configuration

Team: "Security Audit Team"
├── 🔍 file-explorer (Collaborator): Identify target files
├── 🛡️ safety-controller (Supervisor): Verify ARP/security state
└── 📋 compliance-auditor (Reviewer): Final compliance judgment

Roles

Role	Responsibility
Supervisor	Manages overall task progress. Directs other agents
Collaborator	Performs work based on instructions (file ops, analysis, etc.)
Reviewer	Reviews work results. Quality checks and approve/reject

Team Creation Wizard

Enter team name and description
Add agents from Agent Directory (minimum 2)
Assign roles to each agent
Sharing settings

Team execution flow:

User: "Analyze all simulation results in the engineering/ folder"
    ↓
Supervisor (safety-controller):
    → Instructs file-explorer: "Get file listing for engineering/"
    ↓
Collaborator (file-explorer):
    → list_files → read_file (multiple)
    → Result: 12 files found, 8 are simulation results
    ↓
Supervisor:
    → Instructs knowledge-analyst: "Create summaries for each file"
    ↓
Collaborator (knowledge-analyst):
    → Analyzes each file, generates summaries
    ↓
Reviewer (compliance-auditor):
    → "Verify no PHI data is included"
    → Result: 2 files may contain personal information → recommends guardrail application
    ↓
Final response to user:
    "Analyzed 8 of 12 files. Summaries below.
     Note: 2 files were filtered due to potential personal information."

Light theme shown. A dark theme version is available.

Each agent leverages its specialty while the Supervisor coordinates the overall flow. From the user's perspective, a single chat message completes a complex task.

ActionApproval — Human-in-the-Loop

When the AI agent proposes a destructive or irreversible operation, a modal requesting human approval appears before execution.

Why HITL Matters

In a storage management context, some operations are irreversible or have immediate impact. Enabling SnapLock can never be undone. A SnapMirror break severs the replication relationship. If an AI agent autonomously executes these, recovery may be impossible. The HITL pattern ensures the agent "proposes" but the final execution decision rests with a human.

Which Operations Require Approval

Operations flagged with isDestructive: true or isReversible: false require approval:

Operation	Icon	Irreversible?	Why Approval is Required
File deletion	🗑️	△ (recoverable from snapshot)	Prevent unintended bulk deletion
SnapLock enable	🔐	◎ (completely irreversible)	Once enabled, cannot be disabled
SnapMirror break	🔗	◎ (relationship must be re-established)	Replication severance requires manual recovery
User block	🚫	○ (reversible but immediate impact)	Risk of business disruption
IP block	🚫	○	Risk of service disruption
Retention period change	⏱️	△ (can only extend, never shorten)	Shortening is not possible
Threat containment	🛡️	○	May affect multiple users

How the Approval Flow Works

1. Agent proposes a destructive operation
2. UI displays modal (showing action, target, reason)
3. User reviews the details
4. [Approve] → operation executes / [Reject] → operation cancelled, agent notified

Example: SnapMirror Break Approval Flow

User: "Execute failover to the DR site"
    ↓
Agent: Determines SnapMirror break is required
    ↓
┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  SnapMirror break                │
│ Target:  vol_production → vol_dr         │
│ Reason:  DR failover execution           │
│                                          │
│ 🔴 This operation severs the replication │
│    relationship. Re-sync requires manual │
│    intervention.                         │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

UI Design

┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  User block                      │
│ Target:  DOMAIN\suspicious_user          │
│ Reason:  ARP detected abnormal write     │
│          patterns                        │
│                                          │
│ ⚠️ This operation will immediately cut   │
│    off the target user's SMB access      │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

Safe-side default: Reject button has autoFocus. Reflexively pressing Enter falls to the safe side. Clicking outside the modal also triggers "reject."

Irreversibility warning: Operations with isReversible: false show a red warning bar, ensuring users visually recognize the severity of the operation.

Bedrock Guardrails — Response Filtering

Bedrock Guardrails are applied to all AgentChat responses. While the PHI guardrail (Part 2) blocks based on path patterns, Guardrails filter response content itself.

Applied Checks

Check	Target	Behavior
PII detection	Input/Output	Masking (`*--**`)
Harmful content	Output	Block + substitute message
Topic denial	Input	Refuse response to specific topics
Grounding check	Output	Suppress ungrounded answers

When Guardrails are applied, a 🛡️ badge appears on the message. Users see "some information was filtered" with filter reason details available via tooltip.

Model, Cost, and Fallback

Model Configuration

Setting	Default	Changeable	Notes
Model	`amazon.nova-lite-v1:0`	○	Lowest cost. Use Claude 3.5 Haiku for quality
Max tokens	4096 (output)	○	Increase for long analyses
History sent	Last 10 messages	○	Balance with token consumption
Cost per request	~$0.001–$0.01	—	For Nova Lite. Larger context from file reads increases cost

Model is configurable via bedrockModelId in portal-config.ts. Recommended: Nova Lite for testing, Claude 3.5 Haiku for production.

Cost note: Token cost per conversation depends on tool call count and file sizes. A typical "file search + summarize" conversation costs ~$0.005–$0.02 with Nova Lite. To control context window size, adjust maxHistoryMessages in portal-config.ts (default: 10). Reducing history lowers cost but loses conversation context in longer sessions.

Data residency note: When using Bedrock in ap-northeast-1 (Tokyo), input data processing completes within the same region. Deploying FSx for ONTAP, the portal, and Bedrock in the same region ensures file contents never leave the region. Important for financial institutions and healthcare organizations with data residency requirements.

Security note: File contents are included in Bedrock API request bodies, but a VPC-contained configuration is possible. Bedrock does not use request data for model training (AWS Bedrock Data Protection). With VPC endpoint routing, data never leaves the VPC.

Bedrock KB Configuration Parameters

Setting	Recommended	Rationale
Embedding Model	Titan Embeddings V2	Japanese support, cost-efficient
Chunk Size	512 tokens	Appropriate granularity for file-level search
Chunk Overlap	64 tokens	Prevents context loss at chunk boundaries
Top K	5	Balance of precision and cost
Vector Store	OpenSearch Serverless	Auto-provisioned with Managed KB

These are starting-point recommendations. Adjust based on average file size and content characteristics.

Chunk Size tradeoff: Smaller chunks (256 tokens) improve recall but fragment context, reducing precision. Larger chunks (1024 tokens) provide richer context but allow irrelevant content to mix in. 512 tokens balances well for "file-level search." For technical documents (long sections), consider 768–1024.

Fallback Behavior

When Bedrock API is unavailable (regional outage, throttling, etc.), the agent chat displays "Cannot connect. Please use conventional search" and falls back to keyword search. The design ensures "agent unavailable ≠ portal unavailable."

Chat History Retention Policy

Set TTL on DynamoDB conversation history. Default: auto-deleted after 90 days. If auditing "who asked what" is required, separately retain CloudTrail API call logs.

Right to deletion: If a user requests deletion of their chat history, administrators can manually delete the relevant DynamoDB sessions. For GDPR/privacy law environments, implementing a self-service "Clear history" button is recommended.

Design Tradeoffs

Decision	Benefit	Tradeoff
AppSync Query (not Subscription)	Simple implementation, single Cold Start	No streaming response (waits for full completion)
Agent Orchestrator outside VPC	Short Cold Start (no ENI)	Cannot call ONTAP REST API (delegates to VPC Lambda)
DynamoDB session storage	History shared across browsers	Cost (write capacity), TTL management needed
HITL via modal	Reliably blocks destructive ops	UX interruption, reduced agent autonomy
Mode pill toggle (3 modes)	Optimal agent for each purpose	Learning cost of mode concept
Bedrock Guardrails always-on	Structural PII leak prevention	Latency increase (hundreds of ms)
Multi-agent (Agent Teams)	Handles complex tasks	Harder debugging, increased cost

On Streaming Responses

The current implementation uses AppSync Query (synchronous), so agent responses display only after full generation completes. For long responses (file analysis, etc.), users wait several seconds.

Token-level streaming via AppSync Subscription (WebSocket) is technically possible, but implementation complexity (reconnection handling, partial response parsing, error management) increases substantially. I've deferred this as a next step for when latency becomes a problem.

Security Model (AI Agent Specific)

Layer	Implementation	Intent
Authorization	Cognito Groups (authenticated)	All authenticated users can use chat
Tool permissions	Agent autonomously executes read-only tools	Write operations require HITL
Input filter	Bedrock Guardrails (input)	Prompt injection countermeasure
Output filter	Bedrock Guardrails (output)	PII masking
PHI path	isPhiPath() check	Don't pass `/dicom/` etc. file contents to agent
Cost control	Only last 10 messages sent as history	Token consumption cap
Audit	CloudTrail + DynamoDB session logs	Track "who asked what"

Monitoring

Recommended operational metrics:

Metric	Source	Alert Threshold
Agent Orchestrator error rate	Lambda CloudWatch Metrics	> 5% warrants investigation
Response latency (p99)	Lambda Duration	> 10s check model or tool side
Bedrock Throttling	Bedrock CloudWatch Metrics	ThrottledCount > 0 consider provisioned throughput
DynamoDB write throttling	DynamoDB ConsumedWriteCapacityUnits	On-demand auto-scales, typically no issue

Key design decision: The agent can propose storage-admin operations, but execution is authorized against the user's Cognito Group. If a regular user asks the agent "enable SnapLock," it fails with an authorization error. The agent explains "you don't have permission for this operation."

Compliance note: When acting on AI responses about regulated data, the final judgment responsibility lies with the human who approved the operation. Agent responses are "assistive information" — not substitutes for legal or regulatory judgment. Audit trail via CloudTrail + DynamoDB session logs records "who asked what, and what was approved."

Frequently Asked Questions

The following addresses common questions from different perspectives.

Cost note: "What's the token cost per conversation? How do I control context window size?" — With Nova Lite, ~$0.005–$0.02/conversation (varies by tool call count). Adjust maxHistoryMessages to balance cost vs. context retention.

Security note: "What data leaves the VPC? Can Bedrock see file contents?" — File contents are included in Bedrock API requests, but Bedrock does not use data for model training. With VPC endpoint configuration, data stays within the VPC. S3 AP access in Internet-origin configuration also completes within the same region.

Operations note: "How do I add a new MCP tool? What's the deploy process?" — New tools are implemented as Lambda functions and registered as targets in AgentCore Gateway. Deploy via sam deploy or Amplify CI/CD pipeline. See AgentCore MCP Tools Reference for details.

End-user note: "Can I just ask 'find my expense report from last month' and it works?" — Yes. With Knowledge mode and semantic search enabled, you can search by intent without knowing file names. File Agent mode can also traverse folder structures to locate files.

Scaling note: "How does this work with multiple Knowledge Bases across departments?" — KB data sources can be specified per-volume via S3 AP, so you can create separate KBs per department volume. The agent switches target KB via bedrockKbId configuration.

Front-end note: "How do I customize the chat UI? Can I add my own components?" — The chat UI is implemented as a React component (AgentChat.tsx), supporting style customization and custom message renderers. Tool call result displays are also extensible via the ToolCallTimeline component.

Staged Adoption Steps

All AI agent features support DemoMode. They work without FSx for ONTAP for evaluation.

Step 1: No KB, No Agent (File Search Only)

Leave bedrockKbId empty in portal-config.ts. Only keyword search available.

Step 2: Add Bedrock Knowledge Base

Register FSx for ONTAP S3 AP as KB data source
Set bedrockKbId in portal-config.ts
Semantic search mode becomes active

Step 3: Enable AgentChat (File Agent Mode)

Deploy MCP tool Lambda. list_files, read_file, search_files become available.

Step 4: Multi-Agent Mode and HITL

Add admin operation tools (block_user, enable_snaplock, etc.). HITL modal activates automatically.

Step 5: Agent Teams and Custom Agents

Define team-specific analysis patterns as agents. Share via Agent Directory.

Resources

Summary and Next Steps

I integrated AI agents into the "file portal + storage operations" foundation built in Parts 1 and 2:

Feature	What became possible
AgentChat (3 modes)	Complete file operations, analysis, and admin ops in natural language
SemanticSearch	Solve "where did I put that file?" with vector search
Agent Directory/Creator	Codify team-specific analysis patterns as agents
Agent Teams	Collaboratively process complex tasks with multiple agents
HITL (ActionApproval)	Execute destructive operations only after human approval
Bedrock Guardrails	Structurally prevent PII leaks and harmful responses

This three-part series documented the portal's evolution:

Part 1 covered the file portal foundation (S3 AP + Amplify Gen2 / Nextcloud)
Part 2 added storage operations (ARP/AI, Tamperproof, regulatory retention)
Part 3 integrated AI agents (natural language operations, semantic search, multi-agent)

Through the progression of "browse files" → "manage storage" → "talk to AI," the portal reached a point where daily operations and data utilization complete in the browser — without opening ONTAP System Manager or the CLI.

All code is published in the GitHub repository.

For next steps, I'd suggest starting with keyword search in DemoMode, then adding a Bedrock Knowledge Base for semantic search, and layering on MCP tools and HITL after that. When you move on to a production connection, see the PoC → Production Guide.

I hope this post helps someone out there.

See you next time.

Embedding Storage Operations into a File Portal — From ARP/AI Incident Response to Regulatory Retention Management (Part 2)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:20:41 +0000

Run detection through containment from the screen — ARP/AI, SnapLock, audit logs (part 2 of 3)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1, I looked at putting a file portal on top of the S3 Access Point of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP S3 AP, or S3 AP), comparing Amplify Gen2 and Nextcloud as two approaches. The portal at that point stopped at "browse the files" — the moment you wanted to know something about the storage layer, you had to go back to the ONTAP CLI or the REST API.

Once people started using it, the first two things that came back were "I want to check whether ARP has detected anything myself" and "is this snapshot actually locked?" Both requests came from team members outside storage administration, and the sticking point was the same: you cannot check any of it without a VPN connection to the management LIF. So I added ONTAP operations features to the Amplify Gen2 portal, up to the point where health checks and incident first response can be done entirely in a browser.

Here's the conclusion up front:

ONTAP System Manager-equivalent admin operations can be executed from a browser through AppSync + Lambda
Separating storage-admin from regular users with Cognito Groups gives you a safe split between read-only viewing and change operations
Managing ransomware response as four states — Detected → Contained → Investigating → Resolved — removes the hesitation from first response
Turning regulatory retention periods (FISC 7 years / SOX 5 years / HIPAA 6 years) into presets prevents day-count mistakes

In this article, I'll cover the design of the storage operations features embedded in the portal, the ONTAP REST API implementation behind them, and the points I was careful about when handling irreversible operations.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Relationship to existing tools: This portal makes the same storage management operations previously available only through the ONTAP CLI or REST API accessible from a browser with Cognito authentication. It is not a replacement for those — it's an additional layer that opens the same operations to people who should not hold cluster-administrator credentials.

Verification status: how far each feature in this article has actually been confirmed is recorded in verification results, split four ways (live E2E / live read / tests only / DemoMode). Features made reachable after this article first went out — agent and team execution, folder watch, file rename/trash/restore, upload links, the Glue catalog, document analysis — sit in the "tests only" category: handler and component tests pass, but none has been driven from a browser against a real system.

The features added are as follows:

Storage Dashboard: a 4-card health overview
ARP/AI Incident Lifecycle: managed as four states — Detected → Contained → Investigating → Resolved
S3 Object Lock Configuration UI: select a bucket, specify a retention mode, apply
PHI Guardrail: structurally blocks AI processing for paths such as /dicom/
EMS Events: real-time ONTAP alert display
Regulatory Retention Presets: FISC 7-year / SOX 5-year / HIPAA 6-year
Welcome Modal: first-time user onboarding
Audit Log: file access trail via CloudTrail, viewable in the UI
FlexClone Restore: one-click recovery from snapshots
Athena Query: SQL analysis on NAS data

Scope: this article is an implementation record of adding "operations features usable by members outside storage administration" to the file portal built in Part 1. Part 3 covers AI agent integration.

Repository: solutions/amplify-portal/

Why Embed Admin Operations in a Portal

For FSx for ONTAP, the management interfaces you can reach without routing through an additional third-party SaaS are three: the AWS Management Console / FSx API, the ONTAP CLI over SSH, and the ONTAP REST API. ONTAP System Manager is not among them — System Manager support for FSx for ONTAP is delivered through the vendor's SaaS console, and that SaaS covers FSx for ONTAP only in its SaaS-connected mode (sources in management interfaces).

So learning anything about the ONTAP side means SSH to the ONTAP CLI or a REST call, both of which assume cluster-administrator credentials. When team members outside storage administration (security staff, compliance officers, data protection teams) want to "check ARP status" or "verify whether a snapshot is locked," the way to let them is to hand over an SSH key.

Motivation for embedding admin operations:

Security staff should be able to instantly check ransomware detection state and execute containment actions
Compliance officers should be able to verify retention settings and audit logs themselves
Operations teams should be able to check ONTAP alerts without opening another tool

Cognito Groups (storage-admin / authenticated) separate authorization — regular users are read-only, only admins can execute changes.

What You Used to Do vs What You Can Do Now

Operation	Previous Method	Portal Method
Check ARP threats and contain	CLI: `security anti-ransomware volume show` → block by hand	Dashboard detection → one-click containment
WORM-lock a snapshot	CLI: `volume snapshot modify-retention` or direct REST API call	Lock panel → select preset → apply
DR failover with SnapMirror	CLI: `snapmirror break` / `snapmirror resync`	Status list → click action button

Storage Dashboard — Admin Landing Page

The first thing admins see after login is a 4-card health dashboard. This follows ONTAP System Manager's "dashboard first" pattern.

┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ 💾 12       │  │ 🛡️ 8         │  │ 🔐 5        │  │ 📊 3.2x 　   │
│ Volumes     │  │ARP Protected│  │ Locked Snaps│  │ Efficiency  │
│ Avg: 62%    │  │✅ No threats│  │ Tamperproof │  │ 69% saved   │
└─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘

The 4 Cards

Card	Shows	Navigates to
Volume Capacity	Volume count + average utilization	Volume Manager
ARP/AI Protection	Protected volumes + threat count	ARP Admin
Locked Snapshots	Locked snapshot count	Snapshot Admin
Storage Efficiency	Dedup/compression ratio + savings	Efficiency Panel

Implementation

Four APIs fetched in parallel with Promise.allSettled. If one fails (e.g., ARP not configured), remaining cards still render.

const [volResp, arpResp, snapResp, effResp] = await Promise.allSettled([
  adminQuery({ action: "listVolumes", params: JSON.stringify({}) }),
  adminQuery({ action: "listArpVolumes", params: JSON.stringify({}) }),
  protectionQuery({ action: "listSnapshots", params: JSON.stringify({ maxResults: 50 }) }),
  adminQuery({ action: "getEfficiencyStats", params: JSON.stringify({}) }),
]);

Why Promise.allSettled over Promise.all: with Promise.all, one failure rejects everything. On a storage dashboard, "ARP not configured" is a normal state, not an error.

ARP/AI Incident Lifecycle

After ONTAP Autonomous Ransomware Protection (ARP/AI) detects a threat, the response flow is managed across 4 states.

State Transitions

🔴 Detected
  → 🟠 Contained
    → 🟡 Investigating
      → 🟢 Resolved

Light theme shown. A dark theme version is available.

What's Recorded at Each State

State	Timestamp	Additional Data
Detected	detectedAt	—
Contained	containedAt	blockedUsers[], blockedIps[], snapshotName
Investigating	—	notes (free text)
Resolved	resolvedAt	—

Containment Actions

Actions executable directly from the portal:

Action	ONTAP REST API	Effect
Block SMB User	name-mapping deny (win→unix)	Immediately denies target user's SMB access
Block NFS IP	export-policy deny rule	Immediately denies NFS access from target IP
Full Containment	snapshot + block + disconnect	Preserves snapshot + blocks all + disconnects sessions
Unblock	delete name-mapping/export-policy rules	Recovery after investigation

This implements the equivalent of NetApp DII Storage Workload Security blocking directly via ONTAP REST API.

Implementation

// useIncidentState hook — persists per-volume in localStorage
export type IncidentState = "none" | "detected" | "contained" | "investigating" | "resolved";

export function useIncidentState(volumeName: string) {
  const [incident, setIncident] = useState<IncidentRecord>(() => loadIncident(volumeName));

  const markContained = useCallback((snapshot?: string, users?: string[], ips?: string[]) => {
    transition("contained", { snapshotName: snapshot, blockedUsers: users || [], blockedIps: ips || [] });
  }, [transition]);

  // ...
}

The UI displays a state badge with the next action button changing dynamically:

[🔴 Detected] → [Execute Containment] button
[🟠 Contained] → [Start Investigation] button
[🟡 Investigating] → [Resolve] button
[🟢 Resolved] → (complete)

Current limitation: Incident state is stored in localStorage and not shared across browsers. There is also a risk of losing incident state if the browser crashes. DynamoDB persistence is recommended for production use so that multiple members see the same state.

Incident response note: After executing containment actions, integrate with your existing incident runbooks (PagerDuty, OpsGenie, etc.). Portal containment is a first-response action and should be positioned as part of your full incident management workflow.

Security note: Executing containment actions (SMB block, NFS IP block) requires membership in the storage-admin group. Unblocking requires the same privilege, so even if a regular user's browser session is compromised, the attacker cannot lift a block. Compromise of a storage-admin account itself is guarded by Cognito MFA.

Multi-tenant note: When multiple teams (tenants) share the same portal, ensure tenant A cannot view tenant B's ARP alerts by combining Cognito Groups with volume-level access control. In the current implementation, all storage-admin group members can view ARP status for all volumes. If tenant isolation is required, consider deploying separate portal instances per SVM.

PHI Guardrail — Structurally Blocking AI Processing

Prevents the "accidentally sent PHI data to an external AI" scenario at the UI level in HIPAA environments.

How It Works

Files in paths containing /dicom/, /phi/, /pii/, /hipaa/, or /protected-health/ have their AI processing button automatically disabled.

const isPhiPath = (path: string): boolean => {
  const lower = path.toLowerCase();
  return /\/(dicom|phi|pii|hipaa|protected-health)[\/-]/.test(`/${lower}`) ||
         lower.startsWith("dicom/") || lower.startsWith("phi/") || lower.startsWith("pii/");
};

UI display:

Normal folder: [⚡ Run AI Processing]     ← clickable
PHI folder:    [🚫 PHI — AI Blocked]      ← disabled, not clickable

Design Intent

Guardrail, not detect-and-respond: rather than "processed it, then discovered it was PHI," this prevents processing from starting at all
Cannot be bypassed even by admins: the button itself is disabled regardless of permissions
Depends on folder naming convention: path-pattern based, so it presupposes organizational folder structure rules

Important limitation: This guardrail blocks based on folder path only — it does NOT scan file contents. A file like /contracts/sensitive-patient.pdf placed outside PHI paths will not be blocked. For content-based PHI detection, combine with Bedrock Guardrails (response filtering) described in Part 3. Path guardrail + content filtering gives you ideal 2-layer defense.

HIPAA note: Using this portal in a HIPAA environment presupposes a Business Associate Agreement (BAA) with AWS. Cognito, Lambda, S3, Step Functions, and Bedrock are all HIPAA eligible services. The PHI guardrail is an additional UI-level defense — both the BAA and technical controls are required.

S3 Object Lock — WORM Protection for Output Buckets

AI processing results and compliance reports are stored in standard S3 buckets outside FSx for ONTAP. The portal provides a UI to configure S3 Object Lock on these output buckets.

3-Tab Lock Panel

Tab	Target	Purpose
ONTAP SnapLock	Volumes	WORM protection for NAS data (shared across NFS/SMB/S3 AP)
S3 Object Lock	S3 Buckets	WORM protection for AI processing results
Tamperproof Snapshot	Snapshots	Tamper prevention for specific snapshots

S3 Object Lock Configuration Flow

1. Fetch bucket list → select from dropdown
2. Choose mode:
   - Governance: Authorized users can override (recommended for AI output)
   - Compliance: No one, including root, can delete until retention expires (regulatory archives)
3. Specify retention days
4. [Apply] sets Object Lock Configuration

When to Use SnapLock vs S3 Object Lock

Characteristic	ONTAP SnapLock	S3 Object Lock
Target	Files on NAS volumes	Objects in S3 buckets
Access protocols	NFS/SMB/S3 AP	S3 API
Use case	Source data WORM protection	AI results/reports protection
Regulatory	SEC 17a-4, FISC, HIPAA, NARA	SEC 17a-4, HIPAA

Combining both achieves "source data AND processing results both tamper-proof."

Compliance framework note: The AWS services used by this portal (Cognito, Lambda, S3, Step Functions, Bedrock, AppSync) are in scope for ISMAP and SOC 2. FedRAMP environments for US government agencies require deployment in AWS GovCloud regions. Confirm compliance requirements with your legal and compliance teams — this portal is a technical implementation pattern and does not provide compliance judgments.

Regulatory Retention Period Presets

The Tamperproof Snapshot lock form offers retention periods as dropdown selections.

Preset List

Preset	Retention Days	Regulatory Basis
30 days	30	Short-term validation
90 days	90	Quarterly
1 year	365	Annual
SOX/J-SOX	1,825 (5 years)	Securities Exchange Act — financial records retention
HIPAA	2,192 (6 years)	Medical records minimum retention
FISC	2,557 (7 years)	Center for Financial Industry Information Systems standard, Chapter 9

Compliance note: FISC/SOX/HIPAA presets are provided for technical convenience. Whether the regulatory mapping and retention periods apply to your organization's specific situation should be validated with your legal and compliance teams. Regulatory interpretation varies by industry, business scope, and jurisdiction.

Why Presets Are Needed

SnapLock retention settings are irreversible. Once a snapshot is locked, the retention period cannot be shortened.

Prevents "how many days is 5 years?" calculation errors
Each preset shows a tooltip with regulatory basis
Minimizes risk of incorrect selection (can extend but never shorten)

<option value={1825} title="SOX/J-SOX: Financial records must be retained for 5 years">
  1,825 days — SOX/J-SOX (5 years)
</option>
<option value={2192} title="HIPAA: Medical records require minimum 6-year retention">
  2,192 days — HIPAA (6 years)
</option>
<option value={2557} title="FISC: Financial institution data requires 7-year retention">
  2,557 days — FISC (7 years)
</option>

EMS Events — ONTAP Alert Viewer

Retrieves events from ONTAP's Event Management System (EMS) with severity alert/error/emergency and displays them in the admin panel.

Backend Implementation

def _get_ems_events(http, headers, event):
    """ONTAP REST: GET /api/support/ems/events"""
    max_records = min(event.get("maxRecords", 20), 50)
    severity_filter = event.get("severity", "alert,error,emergency")

    query = f"/support/ems/events?max_records={max_records}"
    query += f"&severity={severity_filter}"
    query += "&order_by=time desc"
    query += "&fields=time,severity,message.name,message.text,node.name"

    data = _ontap_request(http, headers, "GET", query)
    events = [
        {
            "time": e.get("time", ""),
            "severity": e.get("severity", ""),
            "messageName": e.get("message", {}).get("name", ""),
            "messageText": e.get("message", {}).get("text", ""),
            "node": e.get("node", {}).get("name", ""),
        }
        for e in data.get("records", [])
    ]
    return {"events": events, "count": len(events)}

Example Events

Severity	Message Name	Meaning
emergency	ha.takeover.byPartner	HA partner takeover
alert	raid.disk.predictiveFailure	Disk predictive failure
error	scsiblade.san.netLIFDown	SAN LIF down
alert	arw.volume.attack	ARP detected ransomware attack

Storage administrators can check recent alerts directly in the portal without opening an ONTAP CLI session.

Refresh frequency note: EMS events are fetched on demand when the dashboard is displayed (not by periodic polling). If you need real-time alerting, push EMS events via EventBridge + SNS instead (see fsxn-observability-integrations). The portal is for "checking recent events"; for "detecting in real time," consider pairing it with a separate observability stack.

Welcome Modal — First-Time Onboarding

First-time users see a 3-step guided tour.

3 Steps

📂 File Browsing — Browse and search NAS files from your browser
⚡ AI Processing — Select files and trigger AI/ML workflows
🔒 Data Protection — Snapshots, WORM locks, ransomware protection

Implementation

export function WelcomeModal() {
  const [dismissed, setDismissed] = useState(() => {
    try { return localStorage.getItem("portal-welcome-dismissed") === "true"; }
    catch { return false; }
  });

  if (dismissed) return null;

  // 3-step carousel with dot navigation
  // "Don't show again" checkbox → localStorage persistence
}

Design Intent

Standard onboarding pattern in SaaS products
Often omitted in internal tools, but highly effective for conveying "what this portal can do" in 10 seconds
localStorage state management, works without backend changes

Resource Management — 20-Panel Card-Grid Admin

ONTAP admin operations organized into 5 categories × 20 panels.

Category Structure

Category	Panels
🗄️ Storage	Volumes / 🧬 FlexClone / ⚡ FlexCache / Qtrees / Quotas / Efficiency
🔐 Access Control	Export Policies / SMB Shares / 👤 Local Users / 🔀 Name Mapping / QoS
🛡️ Data Protection	ARP/AI / Snapshots / SnapLock / 📡 FPolicy / 🦠 Vscan / 🪞 SnapMirror
🖥️ Cluster	🔗 Peering / 🖥️ Cluster Information
🤖 Services	AI Settings

Each panel displays as a card grid, clicking navigates to the detail view. Follows System Manager's card-based navigation.

The Cluster category covers the part the AWS Management Console does not expose. Cluster peering and SVM peering had to be done from the ONTAP CLI or by hand-writing REST calls, which is a real operational cost for an AWS-centric team that otherwise never leaves the console.

New Panel Highlights

Vscan — "Zero to Configured" Setup Guidance

Vscan (antivirus scanning) has the highest setup barrier of any ONTAP feature — it requires external Windows/Linux servers and antivirus vendor licenses. When unconfigured, the Vscan panel displays a 5-step guided wizard:

Vendor selection (6-vendor comparison table with license page links)
NetApp Antivirus Connector download
EC2 deployment (architecture diagram + AWS Blog/GitHub samples)
ONTAP CLI commands (scanner-pool / policy / enable)
Verification in this panel

Even someone starting from "what is Vscan?" can follow the setup path directly from the portal.

One correction from the first version of this panel: the guidance only rendered while Vscan was disabled, so it disappeared the moment Vscan came up — exactly when you still want the scanner-side steps and the interoperability matrix link. It now stays reachable from a toolbar toggle in both states.

FlexClone — Instant Zero-Copy Clones

Create and split volume clones from the UI for ransomware recovery, forensics, or dev/test. Metadata-only copy completes in seconds with near-zero additional capacity.

SnapMirror — Replication Lifecycle Management

Monitor DR and cross-region replication relationships in the browser and execute actions (sync, break, resync, quiesce, resume, delete). Click a relationship to expand the last 10 transfer records (size, duration, success/failure). Lag times exceeding your RPO target display in red so they are noticed immediately.

DR planning note: SnapMirror Break/Resync operations are executable from the browser, but understand the RPO impact before clicking Break. After a break, the replication relationship is severed and resync requires delta transfer. For production environments, pair the confirmation dialog (implemented) with your team's DR runbook procedures.

FlexCache — Remote Read Cache CRUD

Create FlexCache volumes (async, with prepopulate path support), view the list (origin→cache arrow display), and delete (automated 3-step: unmount→offline→delete). When unconfigured, guidance is displayed with a datalist for selecting origin volumes.

Async polling note: FlexCache creation uses async polling to detect completion. If the user closes the browser during creation, the ONTAP-side operation continues. The created volume appears on the next list refresh — no data loss or inconsistent state results.

Cluster and SVM Peering — the gap the console leaves

Peering is a two-sided operation, and the authentication passphrase has to travel between the two clusters. The panel drives one side at a time.

The prerequisite that trips people up is the intercluster LIF, so it gets its own tab rather than being left as a footnote. Both clusters need at least one in up state, and TCP 11104, 11105 plus ICMP have to be allowed between their addresses — the intercluster LIF addresses, not the management LIF.

The flow is: check the LIFs, create the peer on one cluster with Generate passphrase, then enter that passphrase under Accept on the other. The generated value is shown once — ONTAP returns it in the creation response only, so the panel surfaces it in a banner you have to dismiss deliberately. Lose it and the peer has to be deleted and recreated.

SVM peers come after the cluster peer reaches available, and are accepted on the remote side without a passphrase. An SVM-level SnapMirror additionally needs the source subtype default and the destination dp_destination.

Cluster Information — nodes, licences, LIFs, protocols, DNS, jobs

Four tabs: overview (cluster name and version, nodes, licences), interfaces (LIF list with enable/disable), services (NFS, SMB and S3 state with enable/disable, plus the SVM DNS domains and servers), and jobs.

The overview tab is quieter than that list suggests. On the cluster I tested (ONTAP 9.17.1P7D1) both /cluster/nodes and /cluster/licensing/licenses returned zero records with no error, because AWS manages the cluster rather than the tenant. The first version of the panel rendered that as a bare "No nodes", which reads as a failure, so it now states that an empty list is expected here. The cluster name and ONTAP version come from /cluster and are still populated.

The jobs tab matters more than it looks. FlexCache creation, FlexClone split, SnapMirror transfers and peering all run as asynchronous ONTAP jobs, so this is where their progress and, more usefully, their failure reasons show up.

DNS note: an AD-joined SVM resolves its domain controllers through the servers set here. A wrong value breaks SMB, and on an AD-joined SVM it also makes S3 Access Point data operations fail with AccessDenied — while HeadBucket keeps succeeding, which sends you looking at IAM instead of the file system layer.

Destructive operations are gated in both layers

SnapMirror break, resync and delete, Vscan and FPolicy policy deletion, peer deletion, and disabling a LIF or a protocol all show an inline confirmation row before anything is sent.

The confirmation is not only in the UI. The Lambda refuses the same actions unless confirm=true is present, so a direct call that bypasses the browser is rejected identically. That split matters: a UI-only guard is a suggestion, not a control.

Getting this wrong is instructive. In the first cut, the Vscan and FPolicy delete buttons sent no confirm flag and had no confirmation row, and the handler did not check for one either — while the documentation already claimed the operation was confirm-gated. The button appeared to work and the docs looked right. The fix was to make all three agree, and to pin the contract in tests: for every confirm-gated action, assert it refuses without confirm and succeeds with exactly the parameters the UI sends. The second half of that assertion is the one that would have caught it.

Security Model

Layer	Implementation
Authentication	Cognito User Pool + MFA (TOTP/SMS)
Authorization (API)	Cognito Groups (`storage-admin` / `authenticated`)
Authorization (Files)	S3 AP + UNIX/Windows file system identity
Transport encryption	HTTPS (AppSync) + TLS 1.2 (ONTAP REST API)
Secrets management	Secrets Manager (ONTAP credentials)
Audit	CloudTrail S3 data events
WORM	SnapLock (Compliance/Enterprise) + S3 Object Lock
Ransomware protection	ONTAP ARP/AI + portal containment actions
PHI protection	Path-based guardrail (AI processing block)

Only users in the storage-admin group can execute admin operations (volume creation, SnapLock configuration, ARP containment, etc.). Regular users are limited to file browsing and AI processing.

Enterprise SSO note: Cognito User Pool supports SAML 2.0 and OIDC federation. To integrate with an existing IdP such as Okta, Azure AD, or Google Workspace, see Cognito's Adding SAML identity providers. Group mapping lets you map IdP groups to the Cognito storage-admin group.

Infrastructure protection note: Amplify Hosting is served via Amazon CloudFront, so AWS Shield Standard (DDoS protection) is automatically applied. If additional WAF rules are needed, attach AWS WAF to the CloudFront distribution.

Usage Scenarios

Scenario 1: Ransomware Response

1. Storage Dashboard shows 🚨 ARP Threats: 1
2. Navigate to ARP/AI panel → view threat details
3. Click [Contain] → SMB user blocked + snapshot taken
4. State transitions to 🟠 Contained
5. After forensic analysis, [Investigation Complete] → [Resolve]
6. Unblock to resume normal operations

Scenario 2: Compliance Audit

1. Lock panel → SnapLock tab to review Compliance volume list
2. Tamperproof tab to apply FISC 7-year lock to snapshots
3. S3 Object Lock tab to verify output bucket retention settings
4. Audit Trail to review operation history

Scenario 3: Daily Monitoring

1. Storage Dashboard — glance at 4 cards (30 seconds)
2. If any volume exceeds 85% capacity, drill into Volume Manager
3. Check EMS Events for recent alerts
4. No issues → done

Design Trade-offs

Design Decision	Benefit	Trade-off
localStorage (Incident State)	Simple deployment	Not shared across browsers; incident state can be lost if the browser crashes. DynamoDB migration recommended for production
PHI path regex	Simple, immediate	Requires folder naming convention compliance
Generic Dispatch (10 endpoints)	Avoids CFn 1MB limit	IAM policy granularity is coarser
VPC split	Cold start optimization	Two types of Lambda required
Promise.allSettled (Dashboard)	Resilient to partial failures	Failed cards show 0
Regulatory presets (hardcoded)	Ready to use immediately	Code change needed if regulations update

Rollback Procedure

Recovery paths if a portal update causes problems:

Situation	Rollback method
Frontend UI issue	One-click revert to the previous build from the Amplify Hosting console
Lambda function issue	`git revert` + `git push` triggers automatic redeploy (Amplify Gen2)
ONTAP configuration change issue	Restore directly via ONTAP REST API (export-policy, name-mapping, etc.)
Cognito configuration change	CDK stack rollback is not available. Revert manually to the previous state

Because frontend and backend are both under Git management, git revert + git push rolls back immediately. Some ONTAP-side changes (such as enabling SnapLock) are irreversible, so verification before execution matters.

8-Language Support (i18n)

All admin features included, the entire portal supports 8 languages:

Code	Language
`ja`	日本語
`en`	English
`ko`	한국어
`zh-CN`	简体中文
`zh-TW`	繁體中文
`fr`	Français
`de`	Deutsch
`es`	Español

Technical terms (ONTAP, SnapLock, FlexClone, S3 AP, ARP/AI, WORM, FISC, SOX, HIPAA) are not translated. Browser navigator.language auto-detection with instant switching via the header language picker.

Audit Log — Checking "Who Accessed What and When" in the UI

The most frequent request from compliance officers was "I want to check the file access trail myself." Grepping CloudTrail logs from the CLI is not realistic for anyone outside the security team.

The portal's Audit tab runs filtered queries:

Filter	Example
File path	Show only accesses under `/contracts/`
Event type	READ / WRITE / ALL
Time range	2026-07-01 – 2026-07-28

Architecture

AppSync Query → Lambda → Athena SQL → CloudTrail S3 Data Event logs

Light theme shown. A dark theme version is available.

CloudTrail S3 data events are registered as a Glue table and queried with Athena. Results display in table form, and access denials (AccessDenied) are highlighted in red.

Prerequisites

Enable data events for the S3 AP ARN on a CloudTrail Trail
Create the Athena table via Glue Crawler or a manual CREATE TABLE
Set ATHENA_AUDIT_DATABASE, ATHENA_AUDIT_TABLE, and ATHENA_AUDIT_OUTPUT as Lambda environment variables

If these are unconfigured, the UI displays "configuration required" guidance with the setup steps. The design treats this as onboarding rather than an error.

Data Retention

Query results are stored in the Athena workgroup output location (S3). Retention of the original CloudTrail logs is controlled by the S3 bucket lifecycle policy on the Trail's bucket. Set retention periods such as FISC 7 years or HIPAA 6 years according to your audit requirements.

Cost note: If you store audit trails in DynamoDB, storage costs grow in large environments. Set a retention policy (TTL) and archive old records to S3 Glacier. CloudTrail log retention is controlled separately via the Trail's lifecycle policy.

FlexClone Restore — Instant Recovery from Snapshots

When recovering from ransomware damage or accidental deletion, the requirement is "I want to go back to the snapshot from before the infection." Previously this meant creating a FlexClone from the CLI and attaching an S3 AP as a sequence of manual operations.

In the portal, from the 📸 button on the Files tab:

1. Enter the snapshot name (e.g., daily.2026-07-18_0010)
2. Click [Clone & Attach]
3. Step Functions automatically runs FlexClone creation → S3 AP attach
4. FlexClone information appears on the Results tab

FlexClone Status Display

🔄 FlexClone Volume
  Volume:  clone-uc6-20260718-abc123
  Parent:  vol_data
  Status:  🟢 online
  Created: 2026-07-18 15:00:00
  Size:    128 MB
  S3 AP:   clone-uc6-abc123-s3alias

Use Cases

Scenario	Purpose
Ransomware recovery	Immediate access to a pre-infection snapshot
Forensics	Isolate a point-in-time copy for investigation
Testing	Validate against a clone without touching production data

FlexClone is a metadata-only copy, so it completes in seconds with near-zero additional capacity. Differential capacity is consumed only as writes occur.

Accessibility note: Confirmation dialogs for FlexCache deletion and volume deletion are displayed inline. For screen reader users, confirmation messages are announced via role="alertdialog" and aria-describedby. Keyboard navigation (Tab/Enter/Escape) is fully supported.

Throughput note: A FlexClone volume shares the throughput budget with its parent volume. If clones are used actively in parallel in production, account for the performance impact on the parent volume. For read-oriented uses such as forensics this is typically not a concern.

Athena Query — SQL Analytics on NAS Data

When files on FSx for ONTAP are cataloged via Glue Crawler, you can run SQL queries using Athena. The portal's "Analytics" tab lets you enter a database name and SQL, and execute directly.

There is no need to open the AWS Athena console separately — everything completes inside the portal.

UI Layout

┌──────────────────────────────────────────────────┐
│ SQL Query (Athena)      [default        ]        │
├──────────────────────────────────────────────────┤
│ 💡 How to use: catalog files on FSx for ONTAP    │
│ with Glue Crawler, then analyze them here with   │
│ SQL.                                             │
│ ▸ 📝 View query examples (collapsible)           │
├──────────────────────────────────────────────────┤
│ SHOW TABLES IN default                           │
│                                                  │
│ [Run query]                                      │
└──────────────────────────────────────────────────┘

Usage Flow

First time: run SHOW TABLES IN default to discover available tables
Once you know table names: execute specific SELECT queries
Results: displayed in table form

Practical Query Examples

-- Find large files (capacity planning)
SELECT key, size FROM default.s3_objects
WHERE size > 1000000000
ORDER BY size DESC LIMIT 20

-- Total size of a specific folder (project sizing)
SELECT SUM(size) as total_bytes
FROM default.s3_objects
WHERE key LIKE 'engineering/%'

-- Files modified in last 7 days (change tracking)
SELECT key, last_modified
FROM default.s3_objects
WHERE last_modified > current_date - interval '7' day

The portal UI includes an expandable "View query examples" section, so even users unfamiliar with SQL can get started by copy-pasting.

Not just "browse the files" but "ask questions of the data" — directly from the portal.

File Formats Glue Crawler Recognizes

Category	Formats
Structured data	CSV, TSV, JSON, Parquet, ORC, Avro
Logs	Apache/NGINX logs, CloudTrail JSON
Documents	— (text extraction requires Textract or Bedrock)

CAD/EDA binary files cannot be cataloged by Glue Crawler, but file metadata (size, last modified, path) is still retrievable.

Athena Cost Optimization Tips

Athena bills on the volume of data scanned ($5/TB). Ways to keep costs down:

Date partitions (year=/month=/day=) reduce scan volume when a WHERE clause narrows the time range
With columnar formats such as Parquet/ORC, specifying only the columns you need instead of SELECT * reduces cost
Make it a habit to always add a LIMIT clause to exploratory queries
Setting a query limit on the Athena workgroup (for example 10 GB/day) caps unexpected scan volume

Cost Awareness

Rough additional cost if every feature in this article is enabled:

Resource	Estimate	Notes
VPC Lambda (Admin)	~$5/month	Depends on invocation frequency. Negligible for daily checks
CloudTrail S3 Data Events	~$10–$50/month	Proportional to file access volume
Athena queries	$5/TB scanned	Audit queries scan little; partitions reduce it further
Secrets Manager	$0.40/secret/month	One ONTAP credential
DemoMode	$0	No FSx for ONTAP needed. Runs on S3 alone

The cost of FSx for ONTAP itself (~$194/month at 128 MBps) was covered in Part 1. The portal's incremental cost depends on usage frequency, but evaluating in DemoMode adds nothing.

Deployment Time Estimates

Step	Time	Prerequisite
DemoMode deploy	~15 min	Amplify CLI installed
Add VPC + ONTAP connectivity	~30 min	FSx for ONTAP file system running
Audit Log configuration	~20 min	CloudTrail + Glue Crawler
Full feature deploy	~60 min	All of the above

What to Expect After Deployment

Once the DemoMode deploy completes, the portal is reachable at https://<branch>.amplifyapp.com. Cognito sign-up → log in → the file list shows test data from the S3 bucket
Once VPC connectivity is added, the Storage Dashboard shows actual volume counts and utilization, and protected volumes appear in the ARP panel
Once all features are deployed, the Audit Log tab can run CloudTrail queries and the Lock panel shows SnapLock/Object Lock settings

Environment Variables and Configuration Parameters

Parameter	Description	Example Value
`DEMO_MODE`	Run without FSx for ONTAP	`true` (PoC) / `false` (production)
`ONTAP_SECRET_ARN`	Secrets Manager ARN for ONTAP credentials	`arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:fsxn-admin-XXXXXX`
`ATHENA_AUDIT_DATABASE`	Glue database name for audit queries	`cloudtrail_logs`
`ATHENA_AUDIT_TABLE`	Athena table name for audit queries	`s3_data_events`
`ATHENA_AUDIT_OUTPUT`	S3 location for Athena query results	`s3://my-audit-results/athena-output/`

PoC → Production Checklist

After evaluating in DemoMode, use this checklist when moving to production connectivity:

Category	Verification
Network	VPC subnet contains the FSx for ONTAP ENIs; Security Group allows TCP 443 (management LIF). Note: NFS (2049) / SMB (445) are for data access and are not needed for the portal's admin operations
Auth	Cognito User Pool MFA enabled; `storage-admin` group members confirmed
Secrets	ONTAP credentials registered in Secrets Manager
Audit	CloudTrail Trail with S3 data events enabled for the S3 AP ARN
Backup	`portal-config.ts` values in Git (secrets excluded)
Cost	CloudTrail data event estimate confirmed (proportional to access volume)
SSO	(Optional) SAML/OIDC federation configured if using an enterprise IdP

Role-Based Documentation (8 Languages × 3 Guides)

Alongside the feature additions, role-specific documentation was built. Based on usability principles (visibility of system status, task-oriented structure, ease of error recovery), 3 guides are provided in 8 languages:

Guide	Target Role	Key Content
User Guide	End users	Sign in, file operations, AI processing, FAQ
Compliance Guide	Security/Compliance officers	ARP verification, WORM checks, audit trail, PHI guardrail validation, regulatory mapping (FISC/HIPAA/SOX/NIST/ISO)
Quick Reference	All roles	1-page cheat sheet (navigation, tasks by role, status indicators, troubleshooting)

Supported languages: 日本語, English, 한국어, 简体中文, 繁體中文, Français, Deutsch, Español

Design choices for the compliance officer guide:

All tasks executable without storage-admin privileges (read-only access is sufficient)
Each task includes "Evidence for auditors" instructions
Explicit "What you cannot do" escalation table showing who to contact

FlexCache / SnapMirror Management

Direct ONTAP REST API Operations from the Browser

FlexCache, FlexClone, SnapMirror, Vscan, FPolicy, cluster and SVM peering, and the cluster services are all managed through the browser UI — 110 actions in total. A VPC-deployed Lambda connects to the FSx for ONTAP management endpoint via HTTPS, calling ONTAP REST API with credentials from Secrets Manager.

Browser → AppSync (Cognito auth) → Lambda (in VPC) → ONTAP REST API
                                                       ↓
                                                Secrets Manager
                                                (fsxadmin credentials)

Light theme shown. A dark theme version is available.

FlexCache Create UI

Select the origin volume from a dropdown (datalist), specify cache name, size, and prepopulate paths, then create. The experience mirrors ONTAP System Manager — accessible from any browser with Cognito authentication.

SnapMirror Lifecycle Management

The replication relationship list shows state badges (✅ Snapmirrored / 🔴 Broken-off / 🔄 Transferring / ⏸️ Paused) with context-sensitive action buttons. DR failover (Break → Resync) is fully accessible from the browser. Lag times exceeding your RPO target display in red with a ⚠️ RPO warning.

FPolicy — Event Definitions and Notification Policies

FPolicy is where file access notification is configured: an event definition
names the protocol and the operations to watch, and a policy subscribes to those
events and points at an engine. The panel exposes all of it — create, enable,
disable, delete — across three tabs for policies, events and connections.

Note the Delete button on the enabled policy. ONTAP refuses to delete a policy
while it is enabled, so the panel disables the control rather than letting the
call fail. Disable first, then delete. The same ordering applies through the
REST API, and it is the kind of constraint that is easier to encode in the UI
than to explain in a runbook.

Where the Portal Is Self-Contained, and Where It Needs Something Else

The containment actions in this portal are a port of ontap_response.py from a
separate repository, fsxn-observability-integrations
— the module docstring records it. The ONTAP mechanisms are identical:
name-mapping deny, export-policy deny rule, protective snapshot, CIFS session
disconnect. What differs is the trigger, and the layers around it.

Self-contained in the portal

Capability	Requires
Review ARP/AI state, attack probability and suspect files	The portal reaching the ONTAP REST API
Block and unblock an SMB user or an NFS client IP	`storage-admin` group
Create a protective snapshot; lock a snapshot (WORM)	Same
Disconnect CIFS sessions	An AD-joined SVM
List active blocks and lift them individually	—
Block expiry and automatic lifting (24 hours by default)	The block ledger (DynamoDB) being reachable
Apply the same block across several SVMs (`svms` or `allSvms`)	—
Audit file access that went through the S3 Access Point	CloudTrail data events + Athena
FlexClone a snapshot to browse it; diff two generations	—

Every one of these starts when a person clicks. Nothing in the portal
contains a threat unattended — expiry is the one exception, and it only ever
works in the direction of ending a lockout.

Needs something outside the portal

Goal	What it takes
Contain without waiting for a human	An SNS topic plus a response Lambda
Be told about a detection instead of finding it	EMS → webhook → SIEM or an observability platform
Cut NFS off without the client cache window	A VPC NACL deny rule (network layer)
Judge a recovery point before restoring from it	A verification workflow (FlexClone + isolated scan)
Detect anomalies against a per-user ML baseline	A SIEM with anomaly detection, or a dedicated storage security product
Trace file access that arrived over NFS or SMB directly	An ONTAP audit log / FPolicy delivery pipeline

The audit boundary is the one most likely to be misread. The portal's audit trail
reads CloudTrail S3 data events for the S3 Access Point. Access that arrived over
NFS or SMB does not appear there at all — that needs ONTAP's own audit log or
FPolicy events. The two are complementary, not substitutes.

The delivery pipeline is still yours to build, but there is now somewhere for it
to deliver to. Once FPolicy (or Transfer Family) publishes to EventBridge, a
bridge Lambda writes what arrives into a notification table and a Folder
Watch section shows the events under the prefixes you are watching. It is
enabled by an admin toggle and off by default: until an admin states that a
publisher exists, turning it on would only ever show an inbox that cannot fill.
Filtering applies the Cognito group path boundary first and each user’s own
watches second. A watch is your own record, so you may register /, but that
cannot reveal anything outside the group boundary.

So the portal is the hands of incident response: it puts the ONTAP
containment actions in a browser, behind Cognito groups, with confirmation and an
audit trail. If you need something watching around the clock and moving those
hands for you, detection and response belong in a pipeline. Conversely, if you
already detect in a SIEM and only lack a way to stop it at the storage layer, an
SNS-triggered response Lambda fits that shape better than a portal button.

One click was too few

Writing this section is what exposed the gap worth fixing. A block removes a
principal's data access across the whole SVM. The first version executed on a
single click, with no confirmation and no confirm parameter in the payload —
the only guards were the Cognito group and a protected-accounts list. At the time
there was also no expiry, so a block stayed until a person lifted it.

There are now two gates. The UI asks, with wording specific to the action,
because the consequences differ: an NFS block is subject to client-side attribute
caching for up to 60 seconds, an SMB block is not, and disconnecting sessions on
its own does not stop the next login. The Lambda then refuses any call that
arrives without confirm: true, in front of the ONTAP client construction — a
dialog in a browser is a suggestion, and anything calling AppSync directly skips
it.

Session disconnect became its own action in the same pass. It existed in the
handler but only ever ran as step 4 of full containment, so an operator who had
already blocked a user had no way to cut the sessions that survived the block.
Order matters: block first, then disconnect. Disconnecting before the block is in
place just invites a successful reconnect.

Lessons Learned

Async Operations with `return_timeout=0`

FlexCache creation is an asynchronous ONTAP job taking 30-120 seconds. By specifying return_timeout=0, ONTAP returns immediately with 202 Accepted + job UUID, preventing Lambda timeout. The UI auto-refreshes the list at 10s/30s/60s intervals.

One Wrong Field Name Empties the Whole List

The SnapMirror list was requesting fields=...,last_transfer_type,last_transfer_size. last_transfer_size is not a field on /snapmirror/relationships, and ONTAP rejects the entire request when a single field name is unknown:

The value "last_transfer_size" is invalid for field "fields" (<field,...>)

So every relationship vanished — not with a visible error, but as an empty list. What made this survive the test suite is worth noting: the mock ONTAP in the unit tests returns records regardless of which fields are requested, so a test that asserts on the mapped response passes either way. The fix was to assert on the outgoing query string instead. Per-transfer byte counts were already available from /snapmirror/relationships/{uuid}/transfers, so nothing was lost by dropping the field.

Automated 3-Step FlexCache Deletion

Mounted FlexCache volumes cannot be deleted directly. The portal automates:

Remove junction path (unmount)
Set volume offline
Delete FlexCache (async)

Credential Synchronization

When changing the fsxadmin password, update both the FSx for ONTAP API and Secrets Manager simultaneously. Updating only one side causes authentication failures that can trigger ONTAP's account lockout mechanism. Recovery: aws fsx update-file-system password reset → Secrets Manager sync.

FSx for ONTAP Specifics

Item	FSx for ONTAP	On-prem ONTAP
Aggregate specification	Not needed (auto-selected)	Required
SVM creation	AWS API only	CLI/REST available
Intra-cluster FlexCache	Supported (no peering needed)	Supported
fsxadmin password reset	`aws fsx update-file-system`	`security login password`

Coexistence with Existing Tools

This portal is not a replacement for the ONTAP CLI or REST API. It makes a subset of those operations accessible from a browser UI with Cognito authentication — an additional management layer, not a substitute.

Scenario	Portal's Role
Using cloud file-sharing SaaS	AI processing and audit for large-scale NAS data only
Running self-hosted file sharing	Add S3 AP as External Storage for management operations
Using hybrid file services	Add data protection and ARP/AI visibility layer
Operating daily through the ONTAP CLI / REST API	Provide visibility to non-storage-admin team members

The portal provides a management layer for NAS-specific capabilities (Snapshot, FlexClone, FlexCache, SnapMirror, ARP/AI) accessible from the browser. Adoption can be incremental — management operations only, AI processing only, or the full suite — depending on existing workflows.

PoC → Production Migration Flow

After evaluating in DemoMode (no FSx for ONTAP required), migrate to production connectivity in three phases:

┌────────────────┐     ┌────────────────┐     ┌────────────────┐
│ Phase 1: PoC   │     │ Phase 2: VPC   │     │ Phase 3: Prod  │
│ (~15 min)      │     │   Connectivity │     │   Hardening    │
│                │     │ (~30 min)      │     │ (~60 min)      │
│ DemoMode=true  │ ──→ │ ONTAP mgmt LIF │ ──→ │ Least-privilege│
│ S3 bucket      │     │ Secrets Mgr    │     │ MFA required   │
│ Auth: Cognito  │     │ VPC Endpoint   │     │ WAF added      │
│                │     │                │     │ Audit enabled  │
└────────────────┘     └────────────────┘     └────────────────┘

Light theme shown. A dark theme version is available.

Phase	Time	Added Cost	What You Get	What Changes	What Stays the Same
1 (PoC)	~15 min	$0	UI/UX evaluation, AI processing test	Nothing (fresh deploy)	—
2 (VPC)	~30 min	Lambda VPC ~$5/mo	ONTAP admin operations, dashboard	Lambda VPC config added, Secrets Manager registered	Frontend UI, Cognito settings, S3 bucket config
3 (Prod)	~60 min	CloudTrail ~$10–50/mo	Audit, WAF, MFA, least-privilege IAM	IAM policy tightening, WAF attachment, MFA enforcement	Application code, ONTAP connection settings

Post-Deployment Verification

Expected state after each phase completes:

After Phase 1: you can log in to the portal, and DemoMode file listing and AI processing tests work. The Storage Dashboard still shows "not connected"
After Phase 2: the Storage Dashboard shows actual volume information, and ONTAP data appears in the ARP panel and EMS Events
After Phase 3: login is impossible without MFA, CloudTrail records all operations, and WAF applies rate limiting

Irreversible operations warning: The following operations cannot be undone once executed. Verify your organization's policies before enabling them in Phase 2+:

SnapLock Compliance enablement: Volume SnapLock type cannot be changed after creation

Tamperproof Snapshot retention period: Once set, the period cannot be shortened (only extended)

S3 Object Lock Compliance mode: No one, including root, can delete objects until retention expires

For detailed migration steps, see the PoC → Production Guide.

Authorization Model (PoC → Production)

Item	PoC	Production
AppSync auth	`allow.authenticated()`	`allow.groups(["storage-admin"])`
IAM resource scope	`"*"`	Specific ARN patterns
generateClient	`authMode: "userPool"` required	Same

In Amplify Gen2 with multiple auth providers, generateClient() must explicitly specify authMode: "userPool" to ensure the Cognito token is sent to AppSync. This is a recommended pattern per official documentation.

Resources

Summary and Next Steps

Building on the file portal from Part 1, ONTAP management operations and data protection features were embedded:

Feature	What it enables
Storage Dashboard	Health at a glance immediately after login
Incident Lifecycle	Ransomware response with state management
PHI Guardrail	Structurally prevents AI processing of regulated data
S3 Object Lock	Tamper prevention for AI results, configurable from UI
EMS Events	ONTAP alerts without CLI
Retention Presets	Just pick the regulation, get the right retention
Audit Log	"Who accessed what and when" — self-service for compliance
FlexClone Restore	Recover from snapshot in seconds
FlexCache CRUD	Create/delete read cache volumes from browser
SnapMirror Lifecycle	Sync, break, resync, quiesce — DR from a browser
Athena Query	Ask SQL questions against NAS data
Welcome Modal	First-time users productive in 10 seconds

From a portal that "just browses files" to one that "completes storage operations from the web." Daily monitoring and incident first response now finish in the browser without opening an ONTAP CLI session.

All code is available in the GitHub repository.

As a next step, after running DemoMode using the instructions in Part 1, try adding the VPC configuration to enable ONTAP connectivity. That gets you to a state where every feature in this article is usable. Migration details are collected in the PoC → Production Guide.

In Part 3, I'll write about embedding AI agents into this portal — completing file operations, analysis, and admin operations in natural language.

I hope this post helps someone out there.

See you next time.

Adding a File Portal to FSx for ONTAP S3 Access Points — Choosing Between Amplify Gen2 and Nextcloud (Part 1)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:16:29 +0000

Reach the NAS from a browser and from AI, without stopping existing mounts — choosing between Amplify Gen2 and Nextcloud (part 1 of 3)

A Japanese version of this article is available: 日本語版

Introduction

I've been asked more than a few times how to open files sitting on a NAS in a browser and run AI over them. The data lives on NFS/SMB volumes and runs to hundreds of terabytes — between the sheer volume, the latency, and data-residency rules, copying it out to an external service isn't realistic. Requiring an NFS mount instead puts the data out of reach for anyone who isn't an engineer. Closing that gap felt worth doing, so I built a verification environment for it.

What I used is Amazon FSx for NetApp ONTAP S3 Access Points (hereafter FSx for ONTAP S3 AP, or S3 AP — an access point that exposes NAS volumes for read/write through the S3 API). Because it reaches the data in place, you can add a browser UI and AI processing while leaving existing NFS/SMB workflows exactly as they are. I implemented the portal with two approaches: Amplify Gen2 as an AI-processing dashboard, and Nextcloud as a file-sharing UI.

Here's the conclusion up front:

S3 AP gives you browser access to NAS volumes with no data migration
Amplify Gen2 and Nextcloud aren't mutually exclusive — both can attach to the same volume at the same time
The deciding axis: Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
With DemoMode you can bring the portal up in under 30 minutes without FSx for ONTAP, and try file browsing and AI processing

In this article, I'll walk through the design of the two approaches and how to choose between them, the DemoMode startup steps, and the places where I got stuck during implementation.

Here's the overall picture first.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Figure 1: Overall architecture — two frontends reaching the same volume through one S3 Access Point

The part worth noticing is that neither frontend moves any data. The S3 Access Point reads the existing NFS/SMB volume in place, so the same files stay available to existing clients at the same time.

Approach	Role it plays
Amplify Gen2	AI processing dashboard (file selection → AI trigger → result display)
Nextcloud	File sharing UI (browsing, sync, sharing links)

Pick based on what you want to do. Both can connect to the same volume simultaneously, so you can start incrementally.

License: MIT License. Commercial use, modification, and redistribution permitted.

Repository: Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

Prerequisites and Positioning

Audience: Anyone with unstructured data on NAS who wants to protect and leverage that data. On-premises NAS, cloud NAS (FSx for ONTAP, EFS, Azure NetApp Files, etc.) — the platform doesn't matter.

File data on NAS shares several common challenges:

Challenge	Context
Data grows but stays underutilized	Files accumulate year over year — ownership unclear, labels missing, search difficult (CSA study)
Ransomware target	Shared drives are primary targets for encryption attacks. Without detection/stop mechanisms, damage spreads
Want AI but can't move data	Data volume, latency requirements, and data residency regulations make copying to external services impractical
Limited remote access	VPN + mount is required — browser-based access needs additional infrastructure
Audit & compliance overhead	Access logs exist but answering "who accessed what when" in a UI requires a separate analytics layer

This project implements approaches to these challenges using FSx for ONTAP S3 Access Points. FSx for ONTAP was chosen because it combines NFS/SMB/S3 multi-protocol access with integrated data protection: Snapshots, SnapLock (WORM), FlexClone, and ARP/AI (autonomous ransomware detection).

How you use the portal depends on your environment:

Your Environment	How to Use This Portal
Considering migration from on-prem NAS	Build FSx for ONTAP + S3 AP for browser access, AI processing, and data protection in one step
Already using FSx for ONTAP	Enable S3 AP on existing volumes — all portal features become available immediately
NAS + SaaS side by side	Keep SaaS as-is. Add AI processing, audit, and protection for NAS data
Using another cloud NAS	Connect via SnapMirror or DataSync to FSx for ONTAP, then add capabilities incrementally

What this portal provides:

Browser access to NAS files (no VPN)
AI analysis, classification, semantic search (Bedrock integration)
ONTAP-specific data protection (Snapshot, SnapLock, FlexClone, ARP/AI)
Audit trail visibility (CloudTrail + Athena)
Sharing links (Presigned URLs), QR code access

How it differs from SaaS file sharing:

No real-time co-authoring (like Google Docs simultaneous editing)
No desktop sync client (NFS/SMB mount serves this purpose)
No native mobile app (responsive web UI as alternative)

Note: Authentication is independent via Cognito (SAML/OIDC federation supported). No impact on existing SaaS contracts or licenses.

Current Setup	What this portal adds
Box for contracts	Tamperproof Snapshot on NAS originals (FISC 7-year retention)
SharePoint for design docs & reports	AI anomaly detection and classification on NAS files (one-click trigger)
Google Drive for team files	"Who accessed what when" audit trail (CloudTrail + Athena)
Dropbox for project file sync	Ransomware detection + automated containment (ARP/AI)
NFS/SMB direct access only	Browser-based file access for non-technical users + AI analysis

Industry-specific examples:

Financial services: "Hundreds of TB of trading logs and risk calculation results on NFS. Daily documents in SharePoint, but we need anomaly detection on logs and 7-year audit trail retention"
Manufacturing: "500TB of CAD files on NFS, daily document sharing via SharePoint. Add only AI quality inspection on CAD files through the portal"
Healthcare: "DICOM images stored on NAS, clinical data in a dedicated system. Need AI diagnostic support access to images with HIPAA-compliant retention management"
Media & entertainment: "Multiple PB of video assets (ProRes/MXF). Daily editing via NFS mount from Avid/Premiere, but need AI auto-tagging of content management metadata"
Legal: "Tens of thousands of contract PDFs accumulated on NAS. Need AI-powered content search, automated classification, and deadline visibility"
Research: "Hundreds of TB of genomics analysis and simulation results. Researchers want to search and preview results from a browser without using CLI"

Adding Web Experiences to NFS/SMB File Servers

NFS/SMB file servers have clear strengths: high throughput, low latency, multi-protocol support, and deep integration with workstation tools. These qualities make them the backbone of many organizations' workflows. Meanwhile, some of the web-based experiences that SaaS file sharing provides — browser access, AI integration, searchable audit trails — require additional tooling when working with file servers.

This portal adds those web experiences to existing file server environments without changing anything about how the servers operate:

Experience Added	Traditional Approach	How This Portal Delivers It
Browser access (no VPN)	VPN + mount configuration	S3 AP + Cognito auth (Internet-origin)
Natural language file search	grep or find on file names/paths	Bedrock Knowledge Base semantic search
Sharing links (expiring, password-protected)	Manual smbclient or email attachment	Presigned URL time-limited links + QR codes
Version history visibility	Snapshots exist but require CLI/API	Snapshot list in UI + FlexClone one-click restore
Audit trail in a UI	Parse auditd or FPolicy logs	CloudTrail + Athena self-service query
AI analysis & classification	Requires building a separate pipeline	Bedrock + Step Functions one-click trigger
Ransomware visibility	ONTAP ARP runs but checking needs the ONTAP CLI / REST API	ARP/AI dashboard in browser
Mobile / tablet access	Requires dedicated apps or VPN	Responsive web UI + QR code access
Compliance reporting	Run SnapLock CLI commands and compile manually	Lock panel with FISC/SOX/HIPAA presets + status overview

No data movement required. S3 AP reads and writes data directly on the NAS volume. Edit via NFS/SMB → browse via S3 AP in browser → AI results written back to the same volume. All the same data.

Note: Existing NFS/SMB workflows are unaffected. NFS mounts from CAD tools, SMB access from Avid, cron job file processing — all continue working exactly as before. S3 AP is an additional access path to the same volume.

Note: Authentication is independent from existing SaaS (Cognito or LDAP/SAML). No impact on existing SaaS contracts or licenses.

How to Choose

Selection Flow (Q1-Q3)

Q1. Do users need to operate files from a browser?
  → No: EventBridge Scheduler (scheduled batch) is sufficient
  → Yes: Go to Q2

Q2. Primary need: "file sharing & sync"? Or "AI/ML processing launch & result viewing"?
  → File sharing focused: Nextcloud (or Storage Browser for S3)
  → AI processing focused: Amplify Gen2 portal
  → Both: Use together (covered in this article)

Q3. Are users technical? Or non-engineers?
  → Technical: Any of the GUI portals above
  → Non-engineers: Add Amazon Quick (MCP, natural language operation)

Comparison

What you want to do	Suited approach	Trade-off
Trigger AI/ML processing from UI, see results in real time	Amplify Gen2	Custom code required, React knowledge assumed
Browse, upload, sync, and share files via web	Nextcloud	AI integration needs Webhook development
Zero-custom-code file operation UI	Storage Browser for S3	Limited features (browse/DL/UL only)
Natural language file search & analysis	Amazon Quick + AgentCore MCP	Preview stage, region availability limited
No frontend needed, batch is fine	EventBridge Scheduler	No interactive operation

How to choose: Amplify Gen2, Nextcloud, Storage Browser for S3, and Amazon Quick each suit a different context. The trade-offs above are listed symmetrically, including the constraints of the option I reached for, and the article records what actually worked on technical grounds. Read it as material for choosing against your own requirements.

Cost Estimates

Approach	Monthly Cost	Includes
Amplify Gen2 portal	~$18-46	Cognito/AppSync/Lambda (within Free Tier) + AI pay-per-use
Nextcloud (EC2)	~$70-110	EC2 + RDS + ALB. Local Docker = $0
Storage Browser for S3	~$5	Amplify Hosting only
Quick + AgentCore MCP	~$5-15	Lambda execution only
EventBridge direct	~$1-5	Lambda + Step Functions execution only

Cost note: FSx for ONTAP infrastructure (~$194/month, 128 MBps configuration) is common across all approaches. The portal itself adds only tens of dollars per month. At 100 users, Cognito/AppSync/Lambda costs remain within pay-per-use bounds (serverless). At 1000 users, FSx for ONTAP throughput scaling (128→256 MBps, +~$194/month) is the primary scaling cost.

Approach 1: Amplify Gen2 — AI Processing Dashboard

What is Amplify Gen2 (Key Points for NAS Users)

AWS Amplify Gen2 is a full-stack development framework that lets you define both frontend (UI) and backend (auth, API, database, storage) in TypeScript and deploy to AWS.

What this means for NAS/file server users:

Aspect	What Amplify Gen2 Handles
No infrastructure management	No servers to provision, patch, or scale. Everything runs serverless (Lambda, AppSync, Cognito)
Built-in authentication	Cognito provides email login, MFA, and enterprise SSO (SAML/OIDC) out of the box. No custom auth system to build
Direct AWS service integration	Connect to S3 Access Points, Bedrock (AI), Step Functions (workflows), Athena (analytics) with minimal code. No need to design IAM from scratch
Git-based deployment	`git push` triggers automatic build and deploy. No separate CI/CD pipeline required
Per-developer sandboxes	Isolated environments auto-created per developer. Team members work in parallel without conflicts
CDK extensibility	When standard features aren't enough, add any AWS resource (VPC Lambda, DynamoDB, etc.) via AWS CDK

In short: when you want "browser UI + AI processing + authentication on NAS data," Amplify Gen2 delivers this without building or maintaining web servers. You write UI in React; AWS managed services handle the backend.

Architecture

┌───────────────────────────────────────────────────┐
│  Amplify Gen2                                     │
│  ┌──────────┐  ┌───────────────────────────────┐  │
│  │ Cognito  │  │ AppSync (Generic Dispatch)    │  │
│  │ + MFA    │  │  fileQuery → ListFiles λ      │  │
│  │ + Groups │  │  adminQuery → ResourceMgmt λ  │  │
│  └──────────┘  │  arpQuery → ArpResponse λ     │  │
│                │  protectionQuery → Snapshot λ │  │
│                └──────────────┬────────────────┘  │
└───────────────────────────────┼───────────────────┘
                                │
          ┌─────────────────────┼────────────────────┐
          │                     │                    │
          ▼                     ▼                    ▼
┌──────────────────┐  ┌─────────────────┐  ┌────────────────┐
│ S3 Access Point  │  │ ONTAP REST API  │  │ Step Functions │
│ (Internet-origin)│  │ (inside VPC)    │  │ (AI/ML jobs)   │
└──────────────────┘  └─────────────────┘  └────────────────┘

Light theme shown. A dark theme version is available.

Figure 2: Amplify Gen2 architecture — Lambda outside the VPC reads and writes the volume through the S3 Access Point

Keeping Lambda outside the VPC and letting it call the S3 Access Point directly is the crux of this design. Here's why.

Key Design Decisions

VPC split architecture:

Non-VPC Lambda: Reads/writes files via S3 AP (Internet-origin). Fast cold starts.
VPC Lambda: Calls ONTAP REST API (management LIF). Must be inside VPC.
Never combine both in one Lambda (Internet-origin S3 AP unreachable via S3 Gateway VPC Endpoint).

Network note: VPC Lambda handles admin operations only. Required VPC Endpoints: Secrets Manager and CloudWatch Logs. S3 AP data access uses non-VPC Lambda, so NAT Gateway is not needed.

Generic Dispatch pattern: 155 operations consolidated into 10 generic endpoints (action + params: AWSJSON) to avoid CloudFormation 1MB limit.

Extensibility note: To add new operations, simply add an action entry to the Lambda handler's dispatch map. No AppSync schema or CloudFormation changes needed. See the PSTK Action Catalog for the full action list.

UI Layout (Sidebar)

┌───────────────┬─────────────────────────────────────┐
│ Browse        │                                     │
│  📂 All Files │  [Main Content Area]                │
│  ⭐ Favorites │                                     │
│  🕐 Recent    │                                     │
│  🔔 Watch     │                                     │
│  📤 Upload    │                                     │
│               │                                     │
│ AI & Proc.    │                                     │
│  ⚡ AI Proc.  │                                     │
│  🤖 AI Chat   │                                     │
│  🔍 Search    │                                     │
│  📋 History   │                                     │
│  📊 Analytics │                                     │
│  🗂️ Agent Dir │                                     │
│               │                                     │
│ Data Prot.    │  + Right Panel: AI Assistant        │
│  📸 Snapshots │  (appears on file selection)        │
│  🔒 Lock      │                                     │
│  🛡️ ARP/AI    │                                     │
│               │                                     │
│ Admin         │                                     │
│  🔧 Resources │                                     │
│  🔄 Version   │                                     │
│  🔍 Audit     │                                     │
└───────────────┴─────────────────────────────────────┘

4 groups × 17 sections. Same sidebar pattern as Google Drive and Box.

Key Features

File selection → AI processing trigger → real-time result display (5s polling)
Bedrock Q&A, Rekognition image analysis, Athena SQL, Textract OCR
FlexClone snapshot restore (one-click from UI)
Job execution history (DynamoDB, per-user tracking)
8-language i18n (JA/EN/KO/ZH-CN/ZH-TW/FR/DE/ES)
Keyboard navigation + ARIA labels + Dark/Light mode
Responsive (hamburger menu at 768px, tablet-ready)

Audit note: All operations are recorded in CloudTrail. Audit log retention is controlled via CloudTrail settings (default 90 days, extendable to 7 years for compliance). Self-service Athena queries available from the portal's Audit section.

Security note: Blast radius if a Cognito token is compromised — tokens are scoped by Cognito Groups via AppSync resolver authorization; access is limited to resources within the user's group. Default token expiry: 1 hour. MFA-required setting mitigates leakage risk.

Gotchas

AppSync APPSYNC_JS runtime: new Date() not available → use util.time.nowISO8601()
Data Source must be in the same CDK stack as AppSync API ("Data source not found" otherwise)
Bedrock Nova Lite: requires converse() API (invoke_model + inputText is Titan-only)
Presigned URLs: need explicit SigV4 + regional endpoint

Approach 2: Nextcloud — File Sharing UI

What is Nextcloud (Key Points for NAS Users)

Nextcloud is a self-hosted, open-source file sharing and collaboration platform. It provides Google Drive/Dropbox-like user experience on infrastructure you control.

What this means for NAS/file server users:

Aspect	What Nextcloud Handles
Data sovereignty	File data stays on your servers. No need to entrust data to external SaaS
Direct NAS connection	External Storage feature mounts S3 AP / NFS / SMB / WebDAV as additional folders. No data copy needed
Familiar experience	Desktop sync client (Windows/Mac/Linux), mobile apps, sharing links — SaaS-equivalent UX
No per-user licensing	AGPL-3.0, free to use. No per-seat pricing
Auth integration	Supports LDAP, SAML, OIDC. Connects to existing Active Directory

When you configure FSx for ONTAP's S3 AP as Nextcloud's External Storage backend, ONTAP volume files appear in Nextcloud's folder listing. Files saved via NFS/SMB are immediately visible in the browser and syncable to desktop clients.

Architecture

# Nextcloud's External Storage App mounts S3 AP as S3-compatible storage
Nextcloud (Docker) → External Storage App (S3 backend) → S3 AP → FSx for ONTAP

Light theme shown. A dark theme version is available.

Figure 3: Nextcloud architecture — the External Storage App mounts the S3 Access Point as S3-compatible storage

Pass the S3 AP alias as the bucket name, and ONTAP volume contents appear in Nextcloud's file listing.

Setup (~5 minutes)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or bucket name (e.g., "my-volume-ap-s3alias")
export S3_BUCKET=<YOUR_S3AP_ALIAS>

# Configure External Storage to mount ONTAP volume via S3 AP
make configure-s3

# Verify connection (success = file listing returned)
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from your ONTAP volume (or S3 bucket) via S3 AP.

Key Features

Desktop sync client (Windows/Mac/Linux)
Sharing links (password-protected, time-limited)
Comments & annotations
WebDAV access
LDAP/SAML authentication

Trade-offs

Strengths	Considerations
Mature NAS file sharing tool	AI/ML integration needs Webhook development
Desktop sync + mobile apps	EC2/RDS operational cost (production config)
Free under AGPL-3.0 (Docker as-is)	Source disclosure obligation if server code is modified

Gotchas

use_path_style=true required (S3 AP alias only works with path-style addressing)
occ files_external:config must be run once per parameter ("Too many arguments" otherwise)
Empty credentials cause IMDS (169.254.169.254) fallback timeout in Docker

Cost

Local Docker: $0. Production (EC2 + RDS + ALB): ~$70-110/month.

Coexistence Pattern: Amplify + Nextcloud

They are not mutually exclusive — both can access the same FSx for ONTAP volume simultaneously.

Feature Comparison

Feature	Nextcloud	Amplify Gen2
File browsing	✅	✅
File DL/UL	✅	✅ (Presigned URL, 50GB object limit)
Desktop sync	✅	△ (Use NFS/SMB instead)
Sharing links (time-limited)	✅	✅ (Presigned URL)
AI/ML processing workflows	⚠️ Webhook needed	✅
File Q&A (Bedrock)	❌	✅
FlexClone restore	❌	✅
ONTAP admin operations	❌	✅

Expected Usage Flow

1. Team uses Nextcloud daily for file browsing & sharing
   (same data visible as NFS/SMB users)

2. When AI processing is needed on a specific folder,
   launch it from the Amplify portal

3. Result files are written back to the same volume,
   instantly visible from Nextcloud and NFS

Light theme shown. A dark theme version is available.

Figure 4: Running both — AI processing and file management share one S3 Access Point

Authentication is independent (Nextcloud: LDAP/SAML, Amplify: Cognito). For gradual adoption, set up Nextcloud first, then add Amplify when processing needs arise.

Data Consistency

ONTAP guarantees strong consistency across multi-protocol access. A PutObject via S3 AP is immediately visible from NFS/SMB after write completion. The reverse is also true. Whichever portal writes, all see consistent data.

Storage operations note: Your existing procedures (volume creation, snapshot management, and so on) remain unchanged. This portal is an end-user-facing AI processing and file access layer; administrators carry on with the AWS Management Console / FSx API and the ONTAP CLI / REST API. Those three are the management interfaces reachable for FSx for ONTAP — ONTAP System Manager is not among them (management interfaces).

Comparison with Alternatives

Multiple approaches exist for "accessing NAS data from a browser." Each has trade-offs:

Approach	Suited for	Trade-off
This portal (S3 AP + Amplify/Nextcloud)	AI processing + admin ops without data movement	Custom code required, hours of initial setup
DataSync + S3 + any portal	Leveraging S3-native ecosystem	Data copy, sync lag, storage cost duplication
Transfer Family SFTP	Existing SFTP client compatibility	No web UI, AI integration needs extra development
EFS + Lambda	Serverless simplicity	No ONTAP-specific features (SnapLock, FlexClone, ARP)
Full SaaS (Box/SharePoint etc.)	Zero infrastructure, immediate availability	NAS data copy required, limited data protection features

The S3 AP characteristic is "access data via S3 API without moving it." No data copy means no consistency issues or double storage costs. However, it's not "sign up and use immediately" like SaaS.

How to choose: DataSync + S3, Transfer Family SFTP, EFS + Lambda, and SaaS file sharing each suit a different context. The table above also carries the constraints on the FSx for ONTAP S3 AP side (custom code required, a few hours to stand up). Read it as material for choosing against your own requirements.

Try It (~30 Minutes)

Prerequisites

Tool	Version	Check Command	Time
Node.js	20.x or later	`node --version`	0 min if installed
npm	10.x or later	`npm --version`	Bundled with Node.js
AWS CLI	2.x	`aws --version`	~2 min
AWS Account	—	`aws sts get-caller-identity`	0 min if authenticated
Docker	24.x or later	`docker --version`	Nextcloud only

FSx for ONTAP is NOT required for DemoMode. You can verify portal behavior with a regular S3 bucket.

Fastest Path (~5 min, copy-paste 4 lines)

If Node.js and AWS credentials are ready:

# Clone repository and navigate to portal directory
git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git && cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/amplify-portal

# Install dependencies (~2 min)
npm install

# Copy config (DemoMode default — no edits needed)
cp amplify/portal-config.example.ts amplify/portal-config.ts

# Start sandbox (first run also creates Cognito users, ~3 min)
npx ampx sandbox

What to expect: Terminal shows ✅ Deployment complete, then http://localhost:5173 opens the portal. You can browse files and access AI processing screens from the sidebar.

portal-config.ts Key Parameters

// portal-config.ts — parameter explanations
export const portalConfig = {
  // S3 AP alias (empty string = DemoMode, uses regular S3 bucket)
  // Production example: "my-ontap-vol-ap-s3alias"
  s3ApAlias: "",

  // VPC ID (empty = no VPC Lambda = DemoMode)
  // Production example: "vpc-0123456789abcdef0"
  vpcId: "",

  // Cognito group-to-S3AP mapping (different access per group)
  // Example: analysts group gets analysis AP only, admins get full access
  groupApMapping: {
    // "analysts": "analysis-vol-ap-s3alias",
    // "admins": "admin-vol-ap-s3alias"
  },
};

CI/CD note: Amplify Gen2 auto-deploys on git push. git push origin main triggers Amplify Hosting build/deploy. No separate CI/CD pipeline needed.

Nextcloud Setup (~5 min)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or S3 bucket name
export S3_BUCKET=<YOUR_BUCKET_OR_AP_ALIAS>

# Configure External Storage
make configure-s3

# Verify connection
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from the ONTAP volume (or S3 bucket) via S3 AP.

About DemoMode

Leave VPC settings empty in portal-config.ts to experience the portal UI and AI processing flow without FSx for ONTAP.

Feature	DemoMode	Production
Sidebar UI / navigation	✅	✅
File browsing	⚠️ (S3 bucket)	✅ (S3 AP)
AI processing	⚠️ (needs Step Functions)	✅
ONTAP admin panels	⚠️ (shows "connection required")	✅

For production connectivity, see PoC → Production Guide.

Role-Based Documentation (8 Languages)

Guide	Audience	Content
User Guide	End users	Sign in → file ops → AI → FAQ
Compliance Guide	Security/Compliance officers	ARP verification → WORM checks → audit trail
Quick Reference	All roles	1-page operations cheat sheet

Demo note: Minimum 30-minute demo setup is "4-line fastest path → DemoMode launch → file browsing → AI processing screen." No FSx for ONTAP pre-build required.

Operational Considerations

File-Level Access Control

S3 AP access is governed by ONTAP's UNIX permissions (UID/GID) or Windows ACLs. The File System Identity specified when creating the S3 AP determines "who" the access runs as.

# Example: separate team access with multiple S3 APs
S3 AP "portal-readonly"  → UID 1001 (read-only)
S3 AP "portal-analyst"   → UID 1002 (analysis team, specific dirs only)
S3 AP "portal-admin"     → UID 0    (admin, full access)

Throughput Sharing

S3 AP access shares the same FSx for ONTAP throughput budget as NFS/SMB. Normal portal usage (directory listing + individual reads) is not an issue, but if heavy parallel reads coexist with NFS workloads, monitor CloudWatch ThroughputUtilization.

Options if throughput is insufficient:

Scale up throughput capacity (128→256/512/1024/2048 MBps)
Set QoS policy to cap portal volume throughput

Multi-Account Environments

This portal is designed for single-account use. AWS Organizations considerations:

S3 AP: Cross-account access requires AP resource policy
Cognito: Multi-account user consolidation recommended via SAML federation
Tenant isolation: groupApMapping assigns different S3 APs per group

About Presigned URLs

AWS documentation lists Presign as "Not supported," but they work in practice (confirmed with AWS Support). Presigning is client-side SigV4 signature calculation, and since GetObject is Supported, there's no structural way to block it. However, production reliance is at your own risk since documentation says "Not supported." Set short expiry (5 minutes or less recommended).

Scaling Characteristics

Component	Scaling	Downtime
Lambda / AppSync / Cognito / DynamoDB	Automatic (serverless)	None
FSx for ONTAP	Manual throughput/storage expansion	Minutes during throughput change

Portal components are all serverless — the only scaling bottleneck is FSx for ONTAP throughput.

Data engineering note: File metadata is available via CloudTrail + S3 AP access logs. Queryable with Athena, but building a dedicated metadata catalog (Glue Data Catalog etc.) requires separate design.

Series Structure (3 Parts)

Part	Theme	Key Content
Part 1 (this article)	Portal foundation	Amplify Gen2 vs Nextcloud, coexistence patterns, DemoMode
Part 2	Storage operations	ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore
Part 3	AI agent integration	AgentChat, semantic search, multi-agent teams, HITL

FAQ

Q: I'm not using FSx for ONTAP yet. Can I try this?
Yes. DemoMode uses a regular S3 bucket to demonstrate the portal's UI and AI processing flow. FSx for ONTAP can be added later.

Q: Do I need to replace my existing Box/SharePoint?
No. This portal is an additional layer for AI processing, data protection, and audit on NAS data.

Q: How much does it cost?
DemoMode is free (within AWS Free Tier). Production portal add-on: ~$5–50/month. Main cost is FSx for ONTAP infrastructure (~$194/month minimum).

Q: What about security?
Cognito MFA + Groups role separation, HTTPS, Secrets Manager. Enterprise SSO (SAML/OIDC) supported via Cognito federation. Part 2 covers the security model in detail.

Q: Can it handle large files (multiple GB)?
S3 AP upload has a 50GB object size limit (a single PutObject is capped at 5GB, so Storage Browser switches to multipart above that). For files larger than 50GB, place them on the volume via NFS/SMB and use the portal for browsing/processing only.

Q: Does this affect my existing operational procedures?
No. This portal is an end-user-facing additional layer. Operating through the ONTAP CLI / REST API and the AWS Management Console continues exactly as before.

Verified Environment

Component	Version
Node.js	20.18.x
React	18.3.x
Amplify Gen2	1.x (as of 2026-07)
Python (Lambda)	3.12 (ARM64)
FSx for ONTAP	ONTAP 9.17.1
Region	ap-northeast-1
Verified	2026-07

Resources

GitHub: FSx for ONTAP S3 AP Serverless Patterns — This repository (MIT License)
User Guide (8 languages) — End-user daily operations guide
Compliance Guide (8 languages) — ARP/WORM/audit trail verification procedures
PoC → Production Guide — DemoMode to production connectivity migration
Scaling Guide — Capacity planning and throughput design
Nextcloud External Storage Setup — Nextcloud + S3 AP detailed configuration
S3 AP Compatibility Notes — API support status and known constraints
DemoMode Guide — Running without FSx for ONTAP
AWS: FSx for ONTAP S3 Access Points — Official documentation

Summary and Next Steps

Point	Conclusion
Data migration	Not needed — S3 AP reads the same volume NFS/SMB already uses
Portal choice	Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
Running both	Both can attach to the same volume at once. Authentication stays independent
How to try it	DemoMode takes about 30 minutes and needs no FSx for ONTAP
Existing workflows	NFS/SMB mounts and ONTAP CLI / REST API operations continue unchanged

For next steps, start with the 4-line fastest path in DemoMode and spend a few minutes in the sidebar UI and the AI processing screen. Once you can see which approach fits, the PoC → Production Guide covers moving to a production connection.

Part 2 covers storage operations (ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore), and Part 3 covers AI agent integration (AgentChat, semantic search, multi-agent teams, HITL).

I hope this post helps someone out there.

See you next time.

KNFSD File Cache FSx for ONTAP: NFS Read Speedup + S3 Access Points Dual-Path

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Thu, 23 Jul 2026 12:59:50 +0000

Introduction

In July 2026, AWS released KNFSD File Cache as Preview — an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems.

This is the first time an NFS read-acceleration layer has shipped as an official AWS solution. Working daily with FSx for ONTAP NFS performance for EDA and VFX workloads, my reaction was immediate: spin up a verification environment and see what happens.

What I tested was a Dual-Path architecture: KNFSD caching NFS reads from the same FSx for ONTAP volume that S3 Access Points expose to serverless Lambda processing. The question: can compute fleets read at local-NVMe speed while Lambda runs AI/ML workloads on the same data, without bandwidth contention?

Here's what I found:

Dual-Path E2E test passed: A file written via S3 AP was immediately readable through the KNFSD NFS cache
Cache effect: first read 55ms → subsequent reads 2ms (28x improvement), proxy cache serves 422-619 MB/s
NFSv4.1 is mandatory: NFSv3 re-export causes Stale file handle on file creation (kernel-level filehandle size constraint)
FSID_MODE=local + SQLite on FSx for ONTAP: FSID persistence at zero additional cost

This post covers every pitfall I hit during deployment, and why this architecture still makes sense despite those issues.

When to Consider This — and When Not To

This architecture suits workloads that:

Need NFS read bandwidth beyond FSx for ONTAP's provisioned throughput (EDA DRC/LVS, VFX rendering, genomics)
Want both NFS (high-speed reads) and S3 API (Lambda AI/ML post-processing) on the same data
Use Spot instances for burst compute and need the cache to stay warm through reclamation
Unify multiple NFS sources (FSx for ONTAP + on-premises NAS + other clouds) into one cache layer

Consider alternatives when:

FSx for ONTAP throughput scaling alone covers your bandwidth needs (steady-state workloads)
Write caching is needed (FlexCache offers configurable write-back/write-around)
You want fully managed operations with no EC2 to run (FlexCache or Amazon File Cache)
Production SLA is non-negotiable (KNFSD is Preview — wait for GA)

On vendor neutrality: This post evaluates KNFSD File Cache on technical merits only. FlexCache, Amazon File Cache, EFS, and other options each suit different contexts. The goal here is to document what actually works, not to rank solutions.

What is KNFSD File Cache?

KNFSD File Cache is an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems. NFS re-export means mounting a remote NFS share and publishing it as a new NFS export to downstream clients — the proxy sits transparently between source and consumers. Wētā FX (Avatar: The Way of Water) and ILM used the predecessor project in production.

NFS Source (FSx for ONTAP)
     │ NFS mount (NFSv4.1, fsc)
     ▼
KNFSD Proxy (EC2, NVMe L2 cache)
     │ NFS re-export (NFSv4.1)
     ▼
Compute Fleet (Spot × hundreds)

L1 cache: Linux Page Cache (RAM — 5-9 GB/s on 16 GB in this test)
L2 cache: FS-Cache (local NVMe — survives reboot, 106 MB/s measured)
Writes: write-through (immediately forwarded to source)
Scaling: ASG + DNS round-robin or NLB
Prerequisite: source must be an NFS v3/4.x server (parallel filesystems like Lustre/GPFS/WekaFS are not supported)

The key difference from FlexCache: KNFSD is a protocol-level transparent proxy that doesn't require the source to be ONTAP. It can unify FSx for OpenZFS, on-premises NetApp/Pure/Dell/DDN (A3I), and other-cloud NFS servers (via Direct Connect / Cloud Interconnect) into a single cache layer.

Test Environment

Component	Details
Region	ap-northeast-1
Source	FSx for ONTAP (128 MBps, Single-AZ, ONTAP 9.17.1)
KNFSD Proxy	m6gd.xlarge (arm64 Graviton, 16 GB RAM, 237 GB NVMe)
Kernel	7.1.3-knfsd (Packer custom build)
FSID Mode	local (SQLite on FSx for ONTAP NFS mount)
NFS Version	NFSv4.1 (source → proxy → client)
Test Client	t4g.micro (AL2023, 916 MB RAM)
S3 AP	Internet-origin, UNIX identity (root)

Cost note: KNFSD-specific incremental resources only (Packer Spot build ~$0.30 + m6gd.xlarge 1hr ~$0.29). Prerequisites: a running FSx for ONTAP (~$194/mo minimum) + VPC infrastructure. If you already operate FSx for ONTAP, the incremental evaluation cost is effectively < $1.

Dual-Path E2E Test Results (Verified)

Architecture

[Lambda / CLI]                    [Compute Fleet]
     │                                 │
     │ S3 PutObject                    │ NFS read (NFSv4.1)
     ▼                                 ▼
┌──────────────────┐         ┌──────────────────────┐
│ S3 Access Point  │         │ KNFSD File Cache     │
│ (Internet-origin)│         │ (m6gd.xlarge, NVMe)  │
└────────┬─────────┘         └──────────┬───────────┘
         │                              │
         │  ← same volume →             │ NFS mount (NFSv4.1)
         ▼                              ▼
┌──────────────────────────────────────────────────┐
│ FSx for ONTAP Volume (/vol1, UNIX security style)│
└──────────────────────────────────────────────────┘

Results

Test	Result	Details
S3 AP write → KNFSD NFS read	✅	Immediate reflection, content match
NFS write → S3 AP read (MD5)	✅	`c092ef65...` match both directions
S3 AP batch 50 files → NFS bulk read	✅	50 files in 107ms (2.1ms/file)
S3 AP multipart 50MB → NFS read	✅	MD5 verified
NFSv4.1 write via KNFSD	✅	Write-through reflects immediately
fsidd + SQLite persistence	✅	SQLite created on FSx for ONTAP
Proxy restart → client continues	✅	NFSv4.1 grace period, no remount needed

# Actual test log
$ aws s3 cp test.txt s3://arn:aws:s3:ap-northeast-1:XXXX:accesspoint/knfsd-dualpath-test/s3ap-e2e-1784803253.txt
# → upload: success

$ cat /mnt/knfsd/vol1/s3ap-e2e-1784803253.txt  # NFSv4.1 via KNFSD
dual-path-e2e-test-1784803253
# → read success ✅

Performance Numbers

Test environment note: These numbers are from m6gd.xlarge (237 GB NVMe, single drive). Production-recommended instances (im4gn.16xlarge: 30 TB NVMe RAID, i3en.24xlarge: 60 TB) are expected to deliver significantly higher L2 NVMe throughput (estimated 2-8 GB/s sequential).

3-Layer Cache Hierarchy

Layer	Throughput	Use Case
L1: Proxy RAM (page cache)	5.0-9.1 GB/s	Hot data, repeated reads
L2: NVMe (FS-Cache)	106 MB/s	Warm-up shortcut after proxy restart
Source: FSx for ONTAP (NFS fetch)	18-19 MB/s	Cache miss, initial fetch

Important note on source performance: The 18-19 MB/s figure reflects test-time NFS mount constraints (NFSv3 single-stream, rsize=64K) — not FSx for ONTAP's throughput limit (128 MBps). Switching the source mount to NFSv4.1 + rsize=1M is expected to significantly improve cache miss performance (proposed upstream in Issue #42).

L2 NVMe is 5.6x faster than source fetch. After a proxy reboot with empty RAM, data is served from NVMe cache immediately.

Throughput

Operation	Throughput
Sequential read (proxy L1 → client)	422-619 MB/s
Large write (write-through)	157-218 MB/s
S3 AP multipart upload (50 MB)	36.6 MiB/s

Latency

Operation	Cold	Cached	Improvement
10 MB file read	55 ms	2 ms	28x
4 KB small file	1.75 ms	1.5 ms	14%
S3 AP batch → NFS	—	2.1 ms/file	—

Working Set > Client RAM

With client RAM at 916 MB and a 500 MB dataset, reading after drop_caches still achieves 422-428 MB/s — served from the KNFSD proxy's 16 GB RAM cache. This is the core value proposition: small compute nodes can access datasets larger than their local memory at proxy-cache speed.

nconnect Caveat

nconnect	100 MB cold read
1 (default)	619 MB/s
16	184 MB/s

On t4g.micro (5 Gbps NIC), nconnect=16 is counterproductive. Only effective on 100 Gbps instances (c5n.18xlarge, hpc7g).

Cache Coherency

S3 AP write visibility through KNFSD depends on NFS attribute cache timeouts:

Parameter	Default	Effect
`acregmin`	3s	Minimum file attribute cache
`acdirmax`	60s	Maximum directory attribute cache

In our tests, S3 AP-written files were immediately visible through KNFSD (new files bypass directory attribute cache). For frequently-overwritten files, set acregmax=5-10 to reduce stale-data windows.

Every Pitfall We Hit (Documenting All of Them)

NFSv3 Breaks Writes (Most Important Finding)

This consumed the most debugging time. When mounting the KNFSD proxy via NFSv3, existing file reads succeed but new file creation returns Stale file handle.

The cause is documented in the Linux Kernel NFS re-export documentation:

reexport server's filehandle for the reexported object will be X+22 bytes

FSx for ONTAP filehandles are ~40+ bytes. After +22 bytes = ~62+ bytes. New file creation (which adds subdirectory inode information) pushes past NFSv3's 64-byte limit.

NFSv4.1 supports 128 bytes — ample headroom. Our Terraform validates against nfs_version="3" with a clear error message linking to the upstream issue.

Filed upstream as Issue #40.

Other Pitfalls

Problem	Cause	Fix
proxy-startup.sh immediate failure	No EC2 API access (no Public IP)	`assign_public_ip = true`
SSM `GetParametersByPath` denied	IAM missing path-level ARN	Include both `.../*` and `...` (Issue #41)
"No storage devices found"	`CACHEFILESD_DISK_TYPE=local_nvme`	Correct value: `local-nvme` (hyphen)
`FSID_MODE=static` breaks all writes	fsidd disabled, filehandle unresolvable	Switch to `FSID_MODE=local`
S3 AP create BadRequest	`--type One_Zone` is for OpenZFS	Use `--type ONTAP`
`exportfs: No fsid found`	fsidd first-time allocation warning	No operational impact (ignore)

FSID_MODE=local: SQLite on FSx for ONTAP

KNFSD's NFS re-export requires stable FSID management. FSID_MODE=static (fsidd disabled) is fundamentally broken — all write operations fail with Stale file handle.

The fix: use the Linux kernel's fsidd daemon with SQLite backend, placing the database file on the FSx for ONTAP NFS mount:

/srv/nfs/vol1/.knfsd/fsids.sqlite  ← on FSx for ONTAP (persistent, 99.99% SLA)

Aspect	local (SQLite on FSx)	external (RDS)
Additional cost	$0	$15-45/mo
Persistence	✅ (FSx for ONTAP SLA)	✅ (RDS SLA)
Multi-node	△ (single-node only)	✅
Complexity	Low	Medium-High
Best for	PoC / single-node	Production multi-node

FlexCache and KNFSD: Different Tools for Different Contexts

Criterion	FlexCache	KNFSD
Source must be ONTAP	✅ Native integration	○ Any NFS-compatible filer
Multiple source unification	Not possible	✅
Write-back/write-around	✅ (configurable)	△ (write-through only)
Burst read scaling	△ (within FSx throughput)	✅ (NVMe cache scales with ASG)
Managed operations	✅	△ (EC2 operations)
Protocol	NFSv3/4.x	NFSv4.1 (required for re-export)
Data protection integration	✅ (SnapMirror/SnapLock)	None (pass-through)
Node elasticity	△ (manual)	✅ (Auto Scaling)

FlexCache note: FlexCache also doesn't consume origin bandwidth on cache hits (same structure as KNFSD). However, in FSx for ONTAP's managed environment, auto-scaling FlexCache node count is not possible. For burst read bandwidth exceeding FSx max throughput, KNFSD offers more flexibility.

Amazon File Cache / FSx for Lustre: Amazon File Cache is a managed Lustre-compatible cache requiring the Lustre kernel client module on every node. KNFSD uses standard NFS clients (built into Linux/macOS), requiring no additional modules. However, Lustre's parallel I/O bandwidth is inherently higher than NFS re-export — if raw bandwidth is the priority and you can deploy Lustre clients on all nodes, File Cache / FSx for Lustre is the better fit.

Cost break-even guideline: KNFSD becomes economical when you need burst bandwidth beyond FSx for ONTAP's maximum throughput tier (4,096 MBps at ~$2,000/mo). For steady workloads where FSx throughput scaling suffices, increasing FSx throughput is simpler.

Target Workloads

Industry	Scenario	KNFSD Role	S3 AP Role
Semiconductor EDA	DRC/LVS burst	Tech file repeated reads	Verification result AI classification
VFX	Rendering	Texture/asset Fanout	Render quality AI check
Genomics	Reference genome + dbSNP	Hundreds of nodes on same reference	Variant AI classification
Financial Risk	Monte Carlo VaR	Market data sub-ms delivery	Anomaly scenario AI detection
Automotive CAE	Structural sim	Shared mesh data reads	Result comparison AI
Weather	NWP ensemble	GFS initial condition cache	Extreme weather AI detection

Step-by-Step Deployment

1. Build AMI (~25 min)

# Find your subnet (same AZ as FSx for ONTAP recommended)
aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-YOUR-VPC" \
  --query 'Subnets[].{Id:SubnetId,AZ:AvailabilityZone}' --output table

git clone https://github.com/awslabs/knfsd-file-cache.git /tmp/knfsd-file-cache
cd /tmp/knfsd-file-cache/image
packer build -var 'REGION=ap-northeast-1' -var 'ARCH=["arm64"]' \
  -var 'ASSOCIATE_PUBLIC_IP_ADDRESS=true' -var 'SUBNET=subnet-YOUR-SUBNET' .
# → Note the AMI ID (ami-0xxxxxxxxxx)

The AMI is reusable across deploy/destroy cycles. Rebuilds are only needed for kernel updates (~monthly). Normal deploys just pass the existing AMI ID to Terraform.

2. Terraform Deploy (~3 min)

cd infrastructure/knfsd-file-cache/terraform
cp terraform.tfvars.example terraform.tfvars
# Edit: vpc_id, subnet_ids, knfsd_ami_id, source_mounts

./scripts/preflight-check.sh   # Validates environment before deploy
terraform init && terraform apply

3. Mount from Client

sudo mkdir -p /mnt/knfsd
sudo mount -t nfs -o vers=4.1 <KNFSD_IP>:/vol1 /mnt/knfsd
echo test > /mnt/knfsd/hello.txt && cat /mnt/knfsd/hello.txt

⚠️ Always specify vers=4.1. Using vers=3 will cause Stale file handle on file creation.

4. Verify Cache Effect

dd if=/dev/urandom of=/mnt/knfsd/bench.dat bs=1M count=10; sync
sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
time cat /mnt/knfsd/bench.dat > /dev/null   # ~55ms (first read — fetched from source)
time cat /mnt/knfsd/bench.dat > /dev/null   # ~2ms  (subsequent — served from cache, 28x)

5. Dual-Path Test (if you have an S3 AP)

# Write via S3 AP
aws s3 cp test.txt "s3://arn:aws:s3:REGION:ACCOUNT:accesspoint/AP-NAME/file.txt"

# Read via KNFSD NFS (immediate reflection)
cat /mnt/knfsd/file.txt

Security note: The above test uses root (UID 0) as the S3 AP FileSystemIdentity. This is for testing only. In production, use a dedicated non-root user following least-privilege principles.

6. Cleanup

terraform destroy
aws ec2 deregister-image --image-id ami-xxx

Upstream Feedback

Five issues filed during this verification:

#	Summary
#38	Verification report + minimal example for existing source filers
#39	Private subnet support (graceful degradation without EC2 API)
#40	NFSv3 re-export filehandle overflow documentation
#41	IAM policy missing path-level ARN (bug fix)
#42	NFSv4.1 source mount support (end-to-end v4.1)

Filing upstream issues for things you find broken (or under-documented) is one of the most impactful things you can do with a Preview-stage OSS project. It ensures the maintainers have real-world data to prioritize GA fixes.

Summary and Next Steps

Dual-Path E2E test passed — S3 AP write → KNFSD NFS read works.

Three key findings:

NFSv4.1 mandatory (NFSv3 filehandle size limit breaks writes)
FSID_MODE=local + SQLite on FSx for ONTAP provides FSID persistence at $0
S3 AP writes reflect immediately through KNFSD NFS

What's next:

Wait for GA (SLA terms, support scope confirmation)
Multi-node (ASG + NLB) verification
EDA/VFX-specific file access pattern benchmarks
im4gn.16xlarge (30 TB NVMe) L2 bandwidth at production scale

KNFSD File Cache is Preview, but it clearly functions as a practical read-acceleration layer for burst compute workloads.

Small-scale starting point: m6gd.xlarge × 1 running 24/7 = ~$215/mo. Daytime-only (10h) = ~$65/mo.

Resources

KNFSD File Cache GitHub
AWS Launch Blog
Dual-Path Architecture Guide (JA) — 7 industry deep dives + cost estimates + observability
Terraform + Demo Guide
FSID Backend Options — SQLite/RDS/Aurora comparison
Linux Kernel NFS Re-export Docs
Verification Issues #38-#42

Preview notice: KNFSD File Cache is in Preview as of July 2026. The code is OSS (Apache 2.0) but AWS Preview Service Terms apply. Evaluate for non-production or supplementary use until GA. AWS Support coverage may not extend to KNFSD-specific issues — GitHub Issues is the primary channel. NVMe instance store cache data is protected by AES-256 hardware encryption and auto-erased on instance termination.

Mackerel's Log Feature Just Opened in Beta — Here's What It Takes to Wire It Into an OTLP Pipeline

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 20 Jul 2026 08:05:24 +0000

TL;DR

Mackerel — Hatena's Japan-origin observability platform — opened its log feature as public beta on July 16, 2026. In response, this repository added Mackerel integration for shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events via the existing Lambda/OTel Collector pipeline.

The OTel Collector path worked immediately — just add an exporter config entry. But the direct-send path (Lambda → vendor OTLP endpoint, no Collector) exposed two compatibility gaps:

Mackerel requires a custom Mackerel-Api-Key header, not Bearer/Basic
Mackerel's OTLP endpoint only accepts Protobuf, rejecting OTLP/JSON outright

This article covers the generic auth header primitive and limited Protobuf encoder added to close those gaps. Both delivery paths are confirmed E2E against a live Mackerel organization, but since the log feature is still in beta, production deployments should use the Collector path and should not treat Mackerel as the sole audit trail destination.

Scope: This article covers shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events to Mackerel via the existing Lambda/OTel Collector integration. It does not cover Collector high availability, Lambda retry/DLQ design, NAT Gateway routing, or log-loss recovery — see the base article (Part 5) and repository operations docs for those topics.

FSx for ONTAP
├─ File access audit (EVTX/XML)
├─ EMS events
└─ FPolicy events
         │
         ▼
Lambda: builds OTLP Logs Data Model
         │
         ├──▶ OTel Collector ──▶ Mackerel [recommended]
         │
         └──▶ Direct to Mackerel [verification path]
              - Mackerel-Api-Key header
              - Accept: */*
              - OTLP/Protobuf required

GitHub: Yoshiki0705/fsxn-observability-integrations
New files: integrations/mackerel/, integrations/otel-collector/otel-collector-config-mackerel.yaml
Modified: integrations/otel-collector/lambda/{handler,ems_handler,fpolicy_handler}.py

This is a standalone entry in the Serverless Observability for FSx for ONTAP series — it's not a continuation of the incident-response article, and doesn't assume you've read it. It builds on Part 5 (the vendor-neutral OTel Collector pattern), since that's exactly the mechanism a new backend plugs into.

Why Mackerel, and Why Now

This series has already shipped FSx for ONTAP audit logs to 9 observability platforms — Datadog, Splunk, Grafana Cloud, Honeycomb, and others — all confirmed E2E against real accounts. A few readers, mostly from Japan, have asked why Mackerel wasn't on that list. Fair question, and one I'd been wanting to answer for a while.

The short answer: it wasn't this repository's gap to close. Mackerel simply didn't have a log-sending feature yet. It's offered OpenTelemetry-based tracing (APM) for a while, but logs were a gap in Mackerel's own product.

There's another direct trigger behind this article. At SRE NEXT 2026, I had a chance to casually catch up with some longtime acquaintances on the Mackerel team and trade notes on Observability for Amazon FSx for NetApp ONTAP. That particular conversation didn't lead anywhere specific on its own, but the conference as a whole gave me useful ideas and motivation for infrastructure-focused development, which fed directly into picking this integration up.

That changed on July 16, 2026, when Mackerel opened its log feature as a public beta. A feature I'd been waiting on finally showed up, which is the actual reason this article exists.

That said, "just opened in beta" is worth taking seriously. The feature itself is new and explicitly non-GA — a genuinely different situation from adding an eighth or ninth already-mature vendor without much thought. This article walks through what actually broke while wiring it up, and why it still sits in a separate section instead of being quietly folded into the main vendor table.

Vendor-neutral note: this article evaluates Mackerel's OTLP log ingestion on its technical merits — the same way this repository's vendor-comparison.md treats Datadog, Grafana, Splunk, and the other 9. No platform in this comparison is framed as superior to another; each fits a different context, and the goal here is documenting what's actually supported, not ranking who's "better."

When this integration is a good fit:

You already use Mackerel for monitoring/APM
You want to integrate FSx for ONTAP events into your existing incident investigation workflow
You use an OTel Collector as a shared telemetry router
You can evaluate a beta feature in a non-production or supplementary capacity

When to consider other options for now:

Regulatory/audit requirements demand guaranteed data retention
This would be your sole security log destination
Log loss is unacceptable and formal support terms are required
Strict data residency or contract-path conditions apply

What Mackerel's Beta Actually Supports

Before writing any integration code, the first step was reading Mackerel's own documentation rather than assuming logs would work like traces. As of July 2026, the log-sending method documented by Mackerel's official help is OpenTelemetry-based. This article uses OTLP/HTTP accordingly. Some things turned out to be shared, some didn't:

	Mackerel Tracing (APM)	Mackerel Logs (beta, 2026-07-16)
Protocol	OTLP/HTTP	OTLP/HTTP (same)
Endpoint	`https://otlp-vaxila.mackerelio.com`	Same endpoint
Auth	`Mackerel-Api-Key` header (Write scope)	Same header, same scope
Required extra header	`Accept: /` (documented as required by Mackerel; the reason isn't publicly documented)	Same requirement
Grouping key	—	`service.namespace` + `service.name` resource attributes
Retention	(APM's own terms)	30 days planned at GA; beta operates under the same window but with no guarantee
Collector batch config	—	~3.5MB (`sending_queue.batch.max_size: 3500000` bytes). This is a Collector-side setting matching Mackerel's official config example, not an absolute API max request size

The practical upshot: if you already have an OTel Collector sending traces to Mackerel, adding logs is a second exporter pipeline entry, not a new integration pattern.

Mackerel's Logs feature top page. The beta notice (no data retention guarantee, unscheduled maintenance possible) is explicitly displayed.

The Architecture Question: Collector or Direct-Send?

The Collector path is recommended for most deployments. Direct-send is an option for minimal setups without a Collector, or for verifying protocol compatibility. Mackerel's own documentation also recommends placing a Collector in between for filtering and masking.

Criteria	Recommended Path
Production use, masking, multiple destinations, retry control	Collector
Minimize components in a small environment	Direct-send
Just want to try Mackerel integration first	Local Collector test
Handling security audit logs	Collector (filter sensitive attributes)

This repository's OTel Collector integration already ships FSx for ONTAP audit/EMS/FPolicy logs to Datadog, Grafana Cloud, and Honeycomb simultaneously from one Lambda codebase — the Lambda builds a backend-neutral OTLP payload, and the Collector's exporters config decides where it goes. Adding Mackerel as a fourth backend to that pattern needed zero Lambda changes:

# otel-collector-config-mackerel.yaml
# NOTE: This repo's verified Collector version (0.152.0) uses otlp_http. Older
# versions may require otlphttp (no underscore). Check your version's docs.
exporters:
  otlp_http/mackerel:
    endpoint: https://otlp-vaxila.mackerelio.com
    headers:
      Accept: "*/*"
      Mackerel-Api-Key: ${env:MACKEREL_APIKEY}
    sending_queue:
      batch:
        max_size: 3500000
        sizer: bytes
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [otlp_http/mackerel]

That's the whole integration for the Collector-mediated path. But this repository also supports a direct-send path — Lambda posts straight to a vendor's OTLP endpoint, skipping the Collector entirely — used today for Grafana Cloud via AUTH_MODE=basic. Checking whether that path also worked for Mackerel is where things got more interesting.

The Gap: A Header Mackerel Needs That the Direct-Send Path Couldn't Send

"Add an exporter and you're done" turned out not to cover every path. The direct-send auth logic in handler.py, ems_handler.py, and fpolicy_handler.py only supported two modes:

if AUTH_MODE == "basic":
    encoded = base64.b64encode(token.encode("utf-8")).decode("utf-8")
    auth_headers = {"Authorization": f"Basic {encoded}"}
else:
    auth_headers = {"Authorization": f"Bearer {token}"}

Mackerel's auth is neither. It's a bare custom header, Mackerel-Api-Key: <token>, with no Bearer or Basic wrapping. At that point, direct-send to Mackerel without a Collector simply wasn't possible.

The fix is a generic primitive rather than a Mackerel-specific branch:

AUTH_MODE = os.environ.get("AUTH_MODE", "bearer")  # "bearer", "basic", or "header"
AUTH_HEADER_NAME = os.environ.get("AUTH_HEADER_NAME", "Authorization")
EXTRA_HEADERS_JSON = os.environ.get("EXTRA_HEADERS_JSON", "")

# ...
elif AUTH_MODE == "header":
    auth_headers = {AUTH_HEADER_NAME: token}
# ...
if EXTRA_HEADERS_JSON:
    extra_headers = json.loads(EXTRA_HEADERS_JSON)
    auth_headers = {**(auth_headers or {}), **extra_headers}

AUTH_MODE=header sends the secret verbatim under any header name you specify. Mackerel happens to be the first consumer, but nothing about the option itself references Mackerel. EXTRA_HEADERS_JSON covers Mackerel's required Accept: */*, which isn't authentication at all — just a static header their backend depends on.

EXTRA_HEADERS_JSON constraint: This is for static, non-secret headers like Accept only. Do not specify auth headers (Authorization, Mackerel-Api-Key, etc.), Content-Type, Content-Length, or Host — these are controlled by AUTH_MODE/OTLP_CONTENT_TYPE settings or set automatically by the HTTP library.

The same fix went into ems_handler.py and fpolicy_handler.py too. The CloudFormation template passes these env vars to all three Lambda functions, so patching only one would have left the other two silently ignoring AUTH_MODE=header if someone set it.

12 new unit tests cover this across the three handlers (4 each in handler.py, ems_handler.py, and fpolicy_handler.py): custom-header auth, EXTRA_HEADERS_JSON merging, the case where no API key secret is configured but extra headers still apply, and invalid EXTRA_HEADERS_JSON producing a startup warning with extra headers disabled. cfn-lint and gitleaks run clean on the modified template and new files.

The Second Gap: Auth Was Fixed, But the Payload Format Wasn't

Fixing auth wasn't the finish line, though. The unit tests mock the HTTP layer entirely, so they passed without catching what a real API key call turned up next:

OTLP endpoint error 400: {"code":400,"message":"json is not supported yet"}

_send_otlp_payload always sends OTLP/JSON with Content-Type: application/json, but Mackerel's OTLP endpoint accepts Protobuf only and rejects JSON outright. The Collector-mediated path never surfaced this: the Collector decodes received OTLP data into its internal data model, then re-serializes via the otlphttp exporter whose default encoding is Protobuf (proto). So even though the Lambda sends OTLP/JSON to the Collector, the Collector sends OTLP/Protobuf to Mackerel. The direct-send Lambda code simply never had a Protobuf encoder.

This project's policy is to include no additional Python dependency packages in the Lambda deployment package. Only boto3 and urllib3 — both available in the Lambda Python runtime environment — are used; protobuf/opentelemetry-proto are not bundled. Adding those packages for one vendor felt disproportionate, so the fix is a small hand-rolled encoder (otlp_protobuf.py). This encoder is not a general-purpose OTLP implementation. It covers only the limited data structures this repository's FSx for ONTAP log builders produce: timeUnixNano, severityNumber, severityText, body.stringValue, attributes (stringValue only), resource attributes (stringValue only), and InstrumentationScope. It does not support the full OTLP Logs field set or all AnyValue types, and is not intended for reuse outside this project. Field numbers came straight from the official OTLP proto definitions (Apache License 2.0); the encoder's byte output was cross-checked against the official opentelemetry-proto generated Python classes in a throwaway virtualenv before it touched a real account. We monitor official proto definition updates and maintain compatibility tests against the generated classes.

OTLP_CONTENT_TYPE = os.environ.get("OTLP_CONTENT_TYPE", "json")  # "json" or "protobuf"

if content_type == "protobuf":
    headers = {"Content-Type": "application/x-protobuf"}
    body = encode_logs_data(payload)
else:
    headers = {"Content-Type": "application/json"}
    body = json.dumps(payload).encode("utf-8")

After adding OTLP_CONTENT_TYPE=protobuf, calling handler.py's actual build_otlp_payload and _send_otlp_payload functions — the same code the Lambda runs, not a reimplementation — against the real Mackerel API key succeeded. For the two sample records sent (ReadData/Success and Delete/Access Denied), this repository's generated audit attributes — operation type, result, file path, SVM name, user information, event timestamp — were confirmed searchable on Mackerel's log search UI.

Unit tests verify auth headers, Content-Type branching, and encoding output. However, they cannot detect vendor-side acceptance behavior, rate limits, network path issues, or search-indexing delays — those require E2E tests against the real endpoint, which we run separately. 20 more unit tests were added: 3 per handler for the new content-type path, plus 11 in a dedicated test_otlp_protobuf.py, bringing the OTel Collector integration's test suite to 110 passing tests.

Verifying one delivery path for a vendor doesn't verify another. Auth headers, payload wire format, and network reachability can each fail independently, and a unit-test suite that mocks the HTTP layer won't catch a payload-format mismatch that only a real vendor endpoint enforces.

Why This Still Isn't in the "9 Vendors, All Verified" List

Every other vendor in this repository has a checkmark for a reason: an actual payload was sent to a live account and confirmed to arrive. Mackerel now has that checkmark too. Sample FSx audit log payloads showed up in Mackerel's log search UI with this repository's generated audit attributes confirmed searchable, through both the Collector-mediated path and the direct-send path once OTLP_CONTENT_TYPE=protobuf was in place. This repo's README.md reflects that:

✅ E2E verified (open beta) — Confirmed end-to-end against a live account, but the backend platform's own feature is itself in open beta (no data retention guarantee, unscheduled maintenance possible) — see the linked integration README for beta constraints.

Verification Detail	Value
E2E verified	2026-07-18
Collector version	0.152.0 (OpenTelemetry Collector Contrib)
Region	ap-northeast-1
Repository commit	`main` branch HEAD at time of publication

E2E evidence: Searching by the fsxn-audit service shows 4 records — ReadData/Success, WriteData/Success, Open/Failure, Delete/Access Denied at WARN/INFO severity levels.

Log detail: Operation type, result, SVM name, file path, user info, client IP, and timestamp are all retained as searchable attributes.

So the code-side gate is cleared. There's a second reason the integration still doesn't sit in the main table, though, and it's not about this repository's code at all — it's about what Mackerel itself has published about its own feature:

No data-retention guarantee during the beta period
Unscheduled maintenance is possible
GA is planned for fall 2026, but the exact date isn't fixed yet

Cost note: As of July 2026, the beta period is free. GA pricing is planned as ingest-volume-based billing. Pricing details, measurement units, and free-tier terms may change after this article's publication — check Mackerel's official pricing page for current terms.

Folding this integration quietly into the "supported vendors" table with the same checkmark as the other 9 would let someone deploying it for security alerting (this repository's automated incident-response module, for instance) reasonably assume the same confidence level as Datadog or Splunk. That assumption would be wrong, not because of anything this repo's code does, but because the platform underneath it is explicitly pre-GA. So the integration stays in a separate "Emerging / Beta Vendors Under Evaluation" section of vendor-comparison.md, marked as E2E-verified-but-beta, until Mackerel's own log feature reaches GA.

One thing worth flagging for anyone reproducing this kind of verification: during investigation we also inspected GraphQL requests used internally by the browser UI, but we have not confirmed these are public/stable APIs. This repository's permanent E2E verification does not depend on them — final confirmation uses the official UI's log search screen combined with the Collector's own send metrics (otelcol_exporter_sent_log_records). Internal implementation details on Mackerel's side may change without notice.

Security Considerations

FSx for ONTAP audit logs may contain usernames, file paths, share names, and client IP addresses — content that may constitute sensitive or personally-identifiable information depending on your organization's policies. Before sending production data to any external observability service:

Confirm your organization's data classification, storage location, access control, and retention requirements
Use OTel Collector filter/transform processors to remove or mask sensitive attributes as needed (see this repository's PII redaction cookbook)
Do not use beta-period Mackerel as the sole storage destination for audit trails

On EXTRA_HEADERS_JSON specifically:

Never put secret values in it — API keys belong in Secrets Manager
Never put API keys directly in CloudFormation parameters or Lambda environment variables
Never log full header contents at runtime

Troubleshooting: 4-Layer Isolation

When issues occur, isolate across these four layers:

Layer	Target	What to Check
1. Generation	Lambda	OTLP record count, required attributes, timestamp, severity, resource attributes. If enabling debug logging, avoid outputting sensitive info like file paths or usernames
2. Transmission	Lambda / Collector exporter	HTTP response code, timeouts, retry results (400, 401, 403, 429, 5xx)
3. Acceptance	Collector metrics / OTLP response	`sent`/`failed` log records, partial success, drops
4. Consumption	Mackerel UI	service attribute match, search time range, filter conditions, indexing delay

Distinguishing Collector/HTTP transmission failures (layers 2-3) from post-acceptance search/display issues (layer 4) is critical. If Mackerel rejects with HTTP 400, that's a layer 2-3 problem, not layer 4. See the troubleshooting table in integrations/mackerel/docs/ for detailed guidance.

Recommended Adoption Steps

For evaluating this beta integration in stages:

E2E verification with sample logs using a local Collector (test-local-mackerel.sh)
Send non-production FSx for ONTAP audit logs in limited scope
Evaluate send volume, data loss, searchability, and sensitive-information handling
In production, implement dual-send to your existing log platform (CloudWatch Logs, S3, existing vendor)
During the beta period, do not use Mackerel as your sole audit trail destination
Re-evaluate for primary-path promotion once data retention terms, support conditions, and pricing are finalized at GA

What's Actually Usable Today

Both paths below are now confirmed working end-to-end:

# Collector-mediated (recommended, zero Lambda changes)
cp integrations/otel-collector/.env.mackerel.example .env.mackerel
# edit .env.mackerel with a Write-scoped Mackerel API key
bash integrations/otel-collector/scripts/test-local-mackerel.sh

# Direct-send (skips the Collector; needs AUTH_MODE=header AND OtlpContentType=protobuf)
aws cloudformation deploy \
  --template-file integrations/otel-collector/template.yaml \
  --stack-name fsxn-otel-integration \
  --parameter-overrides \
    OtlpEndpoint=https://otlp-vaxila.mackerelio.com \
    AuthMode=header \
    AuthHeaderName=Mackerel-Api-Key \
    ExtraHeadersJson='{"Accept":"*/*"}' \
    OtlpContentType=protobuf \
    ApiKeySecretArn=<secret-arn> \
    ... \
  --capabilities CAPABILITY_NAMED_IAM

OtlpContentType=protobuf is not optional for Mackerel — without it, the send fails with the JSON-rejection error described above, even with correct auth.

Full setup guides (bilingual) are in integrations/mackerel/docs/, including a troubleshooting table that specifically tells you to isolate Collector-side failures from Mackerel-side rejections before debugging the wrong layer.

Wrapping Up, and What's Next

So that's where Mackerel's log beta stands today. Getting there took two fixes that aren't visible when only the Collector-mediated path is tested — a custom auth header format, and a JSON-vs-Protobuf mismatch — and neither one surfaced until real credentials hit the direct-send path. A green unit-test suite doesn't tell you that.

The AUTH_MODE=header, EXTRA_HEADERS_JSON, and OTLP_CONTENT_TYPE=protobuf options added here are not Mackerel-specific code — they're reusable primitives for any future OTLP/HTTP backend that requires a custom auth header or Protobuf-only ingestion.

A few things to watch going forward:

Watch for Mackerel's GA announcement for its log feature (data retention terms, pricing) — this article and the linked docs will be updated, not silently left stale, once that happens.
Once GA lands with confirmed data retention conditions, support terms, and pricing, move Mackerel from "Emerging / Beta Vendors" into the main "Supported Vendors" comparison table.
If you're running Mackerel already, the setup guide's sample OTLP payload is a good starting point to validate your own organization's setup independently.

Mackerel's log feature still has some growing to do before GA, and I'm looking forward to seeing where it lands. Hope this was useful to someone wiring up the same thing.

Resources

Automated Access Blocking for FSx for ONTAP — From Ransomware Detection to Storage-Layer Deny

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 12 Jul 2026 12:11:54 +0000

Introduction

Problem: On Amazon FSx for NetApp ONTAP — a file system running on AWS — how do you automatically stop ransomware from spreading after detection (via ARP/AI, SIEM, or CloudWatch alarm)?
Solution: A Lambda function calls ONTAP REST API to immediately block the compromised user/IP at the storage layer and create a protective snapshot — the same containment-phase actions DII Storage Workload Security performs, but triggerable from any detection source, not just DII's own.
Scope: This is storage-layer access blocking plus evidence preservation. Host isolation, malware removal, credential rotation, and blocking lateral movement to other systems (eradication/recovery phases) are out of scope and still require human judgment.
Mechanism: SMB user blocking via name-mapping + session disconnect (immediate), NFS IP blocking via export-policy rules + NACL deny rules (immediate, network-layer) — all through REST API and AWS VPC APIs.
Trigger: Any source that can publish to SNS (CloudWatch Alarm, Datadog Monitor, Elastic SIEM, manual CLI).
Deploy: One CloudFormation stack. Tested with 52 unit tests (36 core + 16 network-layer). CLI helper for manual operations.
Cost: ~$0.51/month (Lambda + SNS + Secrets Manager). Add ~$14/month if VPC Endpoints are needed.
Timing: Under 2 minutes end-to-end (verified). Worst-case with Lambda cold start + VPC ENI attach: under 3 minutes.
Where this fits: In NIST CSF 2.0 terms, this module is a Respond-function tool (RS.MI mitigation + RS.AN analysis groundwork). It doesn't touch Govern, or the ML side of Detect — see the dedicated section below. (Companion modules in this repo now cover parts of Recover and Identify too — also covered below.)

GitHub: Yoshiki0705/fsxn-observability-integrations
Template: shared/templates/automated-response.yaml
Module: shared/python/ontap_response.py

This is Part 18 of the Serverless Observability for FSx for ONTAP series. It builds on the detection capabilities from Part 17 (CloudWatch Log Alarm) and Part 3 (ARP + Datadog).

The Gap Between Detection and Response

The previous articles in this series covered detection:

Part 3: ARP detects ransomware → EMS event → alert in ~30 seconds
Part 17: CloudWatch Log Alarm detects suspicious admin operations → alert in ~90 seconds

But an alert is just information. The real question is: between when detection fires and when a human completes their response, what should the system do automatically?

In a ransomware scenario, every second counts. If ARP detects file encryption and you get a Slack notification, but the SOC analyst is in a meeting, the attacker keeps encrypting files for 15-30 minutes before anyone reacts.

Where This Idea Came From

We have previously recommended dedicated storage security products — such as NetApp DII (Data Infrastructure Insights, formerly Cloud Insights) Storage Workload Security — to close this gap. DII's built-in per-user ML baselines detect anomalies and automatically block users or IPs at the storage layer, using the same ONTAP mechanisms (name-mapping, export-policy rules) this module uses. If you already run DII in your on-premises NetApp ONTAP environment, it remains a valid choice on AWS as well.

The friction shows up in a specific situation this series keeps running into: detection that originates outside DII. Part 3's ARP → Datadog pipeline, Part 17's CloudWatch Log Alarm, or any third-party SIEM monitor produces an alert. Based on DII's publicly available documentation, its response actions are designed around DII's own detection rather than an arbitrary SNS message from your existing observability stack — we didn't find a documented path to trigger a DII block from an external alert, and building one was outside the scope of this AWS-native series. There's also a practical entry-cost question: DII is a capacity-based licensed product with its own SaaS data plane, which is a different investment than "add a Lambda to a stack you already have."

So the question became: "Can the same containment-phase actions — blocking storage access, taking a snapshot, disconnecting sessions — be triggered by any event that can publish to an SNS topic, using only ONTAP REST API and AWS-native services?" That's what shared/python/ontap_response.py and this article implement. It's not a replacement for DII's ML detection — see the comparison table further down this article, and the fuller comparison plus FAQ ("Does this replace DII Storage Workload Security entirely?") in the Automated Response Guide on GitHub — it's a way to get the same storage-layer response when your detection already lives somewhere else.

Architecture

+--------------------------------------------------------------+
| Detection (existing - any source)                            |
|                                                              |
|  CloudWatch Log Alarm --+                                    |
|  ARP EMS -> Monitor ----+                                    |
|  FPolicy -> SIEM -------+-- SNS Trigger Topic                |
|  Manual CLI ------------+                                    |
|                                                              |
+--------------------------------------------------------------+
| Response (NEW - this article)                                |
|                                                              |
|  SNS -> Lambda (VPC) -> ONTAP REST API                       |
|                           |                                  |
|                           +- Block SMB user (name-mapping)   |
|                           +- Block NFS IP (export-policy)    |
|                           +- Block NFS IP (NACL — immediate) |
|                           +- Create snapshot (evidence)      |
|                           +- Disconnect CIFS sessions        |
|                           +- Notify (SNS -> email/PagerDuty) |
|                                                              |
+--------------------------------------------------------------+

The key design choice: detection and response are decoupled via SNS. Any system that can publish a JSON message to an SNS topic can trigger a storage-layer block. This means you're not locked into a single detection product.

How ONTAP User Blocking Works

SMB: Name-Mapping Deny

ONTAP resolves Windows users to UNIX users via name-mapping rules. By mapping a Windows user to an empty UNIX identity (" "), the SID-to-UNIX translation fails and all file operations are denied.

Before: CORP\jdoe → maps to → jdoe (UNIX) → access granted
After:  CORP\jdoe → maps to → " " (empty)  → access DENIED

This is an SVM-wide block — it affects every volume and share within the SVM.

Operational safety note: The deny mapping is inserted at position 1 by default, meaning it's evaluated before any existing name-mappings. If your SVM has existing name-mappings for service accounts or automation, use a higher index (the module uses index 1; customize via ontap_response.py if needed). Always test with health_check and a non-production user first. To immediately undo an accidental block: ./automated-response-cli.sh unblock-smb --domain CORP --user jdoe

ONTAP REST API call:

POST /api/name-services/name-mappings
{
  "direction": "win_unix",
  "index": 1,
  "pattern": "CORP\\jdoe",
  "replacement": " ",
  "svm": {"name": "svm-prod-01"}
}

NFS: Export-Policy Deny Rule

For NFS clients, blocking is done via export-policy rules with ro_rule: never and rw_rule: never:

POST /api/protocols/nfs/export-policies/{policy_id}/rules
{
  "clients": [{"match": "fsxn_auto_response,10.0.5.99"}],
  "ro_rule": ["never"],
  "rw_rule": ["never"],
  "superuser": ["never"],
  "protocols": ["any"],
  "index": 999
}

The fsxn_auto_response marker in the client match makes these rules identifiable and easy to clean up. Use a high index (e.g., 999) to avoid conflicts with existing rules.

Scope note: Both SMB and NFS blocks are SVM-wide — they affect ALL volumes and shares within the target SVM, not just the volume specified in the contain_* action. In multi-tenant SVMs (multiple applications sharing one SVM), this is the intended blast radius for containment. If you need volume-level granularity instead, use create_snapshot (per-volume) and coordinate with your network team for host-level isolation.

NFS: Why Export-Policy Alone Isn't Instant (and How We Fixed It)

The export-policy rule above takes effect on the server side immediately — but Linux NFS clients cache access decisions for up to 60 seconds (actimeo default). During that window, the attacker's existing mount can still read/write despite the ONTAP-layer deny.

Unlike SMB (where disconnect_smb_sessions forces re-authentication), NFS has no equivalent "kill this client's connection" API. (NFSv4 has lease management internally, but ONTAP does not expose a REST API endpoint for forced lease revocation of a specific client.)

Solution: Network-layer blocking via VPC NACL deny rules.

When FsxSubnetId is configured, contain_nfs_threat automatically applies both layers:

Layer	Mechanism	Effect	Bypass?
ONTAP	Export-policy deny rule	Persistent, survives NACL removal	Client cache (up to 60s)
Network	NACL deny rule (rule 50-99)	Immediate (packet-level)	Cannot be bypassed by client

# Deploy with network-layer NFS blocking enabled:
aws cloudformation deploy \
  --template-file shared/templates/automated-response.yaml \
  --stack-name fsxn-automated-response \
  --parameter-overrides \
    ... \
    FsxSubnetId=<subnet-where-fsx-enis-reside> \
  --capabilities CAPABILITY_NAMED_IAM

The NACL rule blocks ALL traffic from the attacker IP at the VPC level — no client-side cache, no retry window. The export-policy rule persists as the long-term block after the NACL is eventually removed during investigation.

Blast radius note: By default, the NACL rule blocks ALL protocols from the IP (not just NFS). If the attacker IP is also used for monitoring or management, set "block_all_ports": false in the SNS message to block only NFS port 2049.

Why NACL and not Security Group? Security Groups are allow-only — you cannot add a deny rule for a specific IP. NACLs support explicit deny and are evaluated before Security Groups, making them the correct mechanism for emergency IP blocking.

VPC Endpoint requirement: The Lambda calls the EC2 API (CreateNetworkAclEntry) to manage NACL rules. If your VPC has a NAT Gateway, this works automatically. If not, add an EC2 Interface VPC Endpoint (com.amazonaws.<region>.ec2) or set CreateVpcEndpoints=true and add EC2 manually. Without network access to the EC2 API, the NACL block will timeout while the ONTAP export-policy block still succeeds.

Session Disconnect

Blocking prevents new operations, but existing CIFS sessions can continue. The composite action also disconnects active sessions:

DELETE /api/protocols/cifs/sessions/{svm_uuid}/{identifier}/{connection_id}

Safety Features

Input Validation

The module validates all inputs before calling ONTAP:

Username injection prevention (blocks ;, |, &, `, $, newlines)
Protected account denylist (fsxadmin, administrator, vsadmin, system)
IP address format validation (valid IPv4 octets 0-255)

Snapshot Storm Prevention

A configurable cooldown (default: 15 minutes) prevents creating multiple snapshots during a sustained attack:

# If a snapshot with our prefix was created <15 min ago → skip
result = client.create_snapshot(
    svm_name="svm-prod",
    volume_name="vol_data",
    cooldown_minutes=15,  # Prevents storm
)
# result["status"] == "skipped" if cooldown active

Marker-Based Cleanup

All response rules include a consistent marker (fsxn_auto_response), making it straightforward to find and remove all automated blocks:

# Find all our rules
export-policy rule show -clientmatch *fsxn_auto_response*
vserver name-mapping show -direction win-unix -replacement " "

Evidence-handling note: The protective snapshot this module creates is evidence, and evidence needs a chain of custody, not just a timestamp. Trigger source, exact UTC time, and post-action API confirmation are already captured in CloudWatch Logs — but pre-action state (what the name-mapping or export-policy looked like before the block) isn't currently queried and logged, and the SNS trigger message itself isn't hashed. If this snapshot might ever need to hold up in an investigation, not just an ops postmortem, review the chain-of-custody gap table in the Security Addendum before treating "snapshot created" as "evidence preserved."

Time-Limited Blocks

Persistent blocks are a real operational risk — a false positive can lock out a legitimate user indefinitely if nobody notices. The companion automated-response-ttl.yaml stack deploys an EventBridge Scheduler that periodically checks for and auto-removes blocks older than a configurable TTL, so a missed alert doesn't turn into a standing lockout. Deploy it alongside the main stack.

Deployment

One Stack Deploy

aws cloudformation deploy \
  --template-file shared/templates/automated-response.yaml \
  --stack-name fsxn-automated-response \
  --parameter-overrides \
    OntapMgmtIp=<management-ip> \
    OntapCredentialsSecretArn=<secret-arn> \
    VpcId=<vpc-id> \
    SubnetIds=<subnet-1>,<subnet-2> \
    SecurityGroupId=<sg-id> \
    DefaultSvmName=svm-prod-01 \
    SharedPythonLayerArn=<layer-arn> \
    NotificationEmail=security-team@example.com \
  --capabilities CAPABILITY_NAMED_IAM

New parameter (from verification): SharedPythonLayerArn — provide the ARN of the fsxn-shared-python Lambda Layer containing ontap_response.py. Without this, the handler fails with ModuleNotFoundError. Build and publish the layer first:
bash shared/python/build-layer.sh
aws lambda publish-layer-version --layer-name fsxn-shared-python \
  --zip-file fileb://shared/python/dist/fsxn-shared-python-layer.zip \
  --compatible-runtimes python3.12 --query 'LayerVersionArn' --output text

The stack creates:

SNS Trigger Topic (entry point)
SNS Notification Topic (results)
Lambda function (VPC, Python 3.12)
DLQ with encryption
CloudWatch Log Group (365-day retention)
DLQ depth alarm
VPC Endpoints for Secrets Manager + SNS (if CreateVpcEndpoints=true)

DLQ alarm = urgent: A message in the DLQ means a containment action failed. Unlike most DLQs where retry-later is fine, here it means the attacker may still have access. Treat DLQ depth > 0 as a P1 incident requiring immediate human investigation. See the DLQ Replay Runbook for recovery steps.

Critical: VPC Endpoints Required. Lambda in a VPC cannot reach AWS APIs (Secrets Manager, SNS) without Interface VPC Endpoints or a NAT Gateway. The stack creates these by default (CreateVpcEndpoints=true). If your VPC already has them, set it to false. Without this, Lambda will timeout on every invocation.

Cost note: If your VPC already has a NAT Gateway, set CreateVpcEndpoints=false — the Lambda can reach AWS APIs through NAT without the additional ~$14/month VPC Endpoint cost. The $0.51/month figure is infrastructure cost only; operational cost (tuning detection rules, investigating false positives, running quarterly drills) is the larger ongoing investment.

The CloudFormation template embeds the Lambda code inline for a quick first deployment. For a GitOps-friendly setup, package ontap_response.py as a Lambda Layer instead using build-layer.sh and version it independently of the stack.

Comprehensive deployment guidance: For VPC Endpoint conflict avoidance, parameter file templates, and pre-deployment environment validation, see the Deployment Guide. It includes a pre-flight check script that detects existing VPC Endpoints before you deploy — the single most common source of deployment rollbacks in this project.

Connect Detection Sources

Subscribe the trigger topic to your detection:

Source	How to Connect
CloudWatch Log Alarm	Set alarm action to trigger topic ARN
Datadog Monitor	Use `@sns-<topic>` in notification
Elastic SIEM	Use SNS action in Kibana alert
Manual	`aws sns publish --topic-arn <arn> --message '<json>'`

Alert-tuning note: Whatever you wire to the trigger topic will fire on every match, and every fire generates a notification. A detection rule tuned for "catch everything" rather than "catch what matters" turns this pipeline into another source of alert fatigue — the Reliability note in the Automated Response Guide already flags the self-inflicted-DoS risk of an over-triggering rule; the analyst-facing version of that same risk is that a noisy rule buries the one block that actually mattered in a pile of routine ones. Tune the upstream detection rule's specificity before wiring it here, not after the first pager rotation complains.

CLI Helper

For manual operations and testing, use the included CLI wrapper:

export RESPONSE_TOPIC_ARN="arn:aws:sns:ap-northeast-1:123456789012:fsxn-automated-response-trigger"
export DEFAULT_SVM="svm-prod-01"

# Full containment (snapshot + block + disconnect)
./shared/scripts/automated-response-cli.sh contain-smb \
  --domain CORP --user jdoe --volume vol_data \
  --reason "ARP detection - arw.volume.state alert"

# Block a single IP
./shared/scripts/automated-response-cli.sh block-nfs \
  --ip 10.0.5.99 --reason "Mass deletion from FPolicy"

# Unblock after investigation
./shared/scripts/automated-response-cli.sh unblock-smb \
  --domain CORP --user jdoe

# Dry-run test (no publish)
./shared/scripts/automated-response-cli.sh test

If the same compromised identity or IP has access across multiple SVMs (e.g., a domain user with shares on both production and DR), use automated-response-multi-svm-cli.sh instead — it wraps the single-SVM CLI, fans out the same action across a comma-separated SVM list, and reports per-SVM success/failure:

./shared/scripts/automated-response-multi-svm-cli.sh contain-smb \
  --svms "svm-prod-01,svm-prod-02,svm-dr-01" \
  --domain CORP --user jdoe --volume vol_data \
  --reason "ARP detection - multi-SVM block"

Composite Actions

Individual actions (block_smb_user, block_nfs_ip, create_snapshot, disconnect) can be combined into a single message:

Action	Steps	Use Case
`contain_smb_threat`	Snapshot → Block SMB user → Disconnect sessions	Compromised AD user detected
`contain_nfs_threat`	Snapshot → Block NFS IP	Suspicious NFS client activity
`contain_multiprotocol_threat`	Snapshot → Block SMB user → Block NFS IP → Disconnect sessions	Volume accessed via both SMB and NFS — a single-protocol block lets the attacker switch protocols and continue

The contain_* naming maps to the Containment phase in NIST SP 800-61's incident handling model. This module only covers storage-layer access blocking and evidence preservation, though. Isolating the compromised host, removing malware, rotating credentials, and blocking lateral movement to other systems (eradication/recovery) remain out of scope and still require a human (or another IR tool) to handle.

Comparison with DII Storage Workload Security

Both approaches use the same underlying ONTAP mechanisms. The difference is where detection intelligence lives:

Aspect	DII Storage Workload Security	This Approach
Blocking mechanism	ONTAP REST API	ONTAP REST API (identical)
Detection	Built-in ML (per-user baselines)	Your choice of SIEM/observability
Context scope	Storage only	Storage + network + application
Data residency	SaaS (vendor cloud)	Your AWS VPC
Integration	Limited export	Native (detection originates from your tools)
Cost model	Capacity-based license	~$0.51/month (pay-per-use)
Setup time	Days (agent + collector + AD config)	30 minutes (CloudFormation)

Neither approach is "better" — they suit different contexts. If you already have a SIEM with anomaly detection (Datadog ML, Elastic ML Jobs, Splunk MLTK), or if your alert originates outside DII, this approach extends your existing investment to cover storage-layer access blocking. If you already run DII and your detection lives there too, its built-in ML and turnkey containment remain a solid choice — see the Automated Response Guide for the fuller writeup.

The table above compares mechanisms. The next section places both mechanisms inside a wider risk-management frame, because "which tool blocks the user" and "is your organization's ransomware posture actually sound" are different questions.

Where This Fits in Cyber Resilience: NIST CSF 2.0

It's tempting to read a 65-second block-and-snapshot flow as "ransomware handled." It isn't — it's one function, in one framework, among six. NIST CSF 2.0 organizes an organization's entire cybersecurity risk-management program into Govern, Identify, Protect, Detect, Respond, Recover, and NIST's dedicated ransomware profile (NIST IR 8374r1) maps ransomware-specific outcomes onto those six functions. Both AWS (Ransomware Risk Management on AWS Using the NIST CSF) and NetApp (Fortify your cybersecurity defenses with NIST framework) publish their own mappings for the same reason this section exists: to be explicit about which function a given tool covers, and which functions remain the organization's job regardless of tooling.

Placed on that map, this module and DII Storage Workload Security occupy the same narrow slice:

CSF 2.0 Function	What It Covers	Where This Module (or DII) Sits
Govern	Risk strategy, roles, policy, board oversight	Out of scope for both — this is an organizational responsibility no storage-layer tool automates
Identify	Asset/data inventory, classification	Out of scope for this module, but a companion module in the same repo — a Content-Level PII Classification Scanner using Amazon Comprehend — covers text/structured-data content discovery. Office/PDF content extraction is not implemented; DII's parent product line offers broader data classification separately
Protect	Safeguards that reduce likelihood/impact	Partially covered — ONTAP Snapshot/SnapLock, export-policy, name-mapping are the underlying safeguards both approaches call
Detect	Finding anomalies via monitoring	Not this module's job — it responds to a detection, it doesn't produce one (see Part 3 and Part 17 for the Detect side)
Respond	Mitigation, analysis, reporting on a detected incident	✅ This is what this module does — block, snapshot, disconnect, notify
Recover	Restoring systems and coordinating with stakeholders	The snapshot this module creates is Respond-phase evidence, not a verified recovery point on its own. A companion Verified-Clean Recovery Point Guide closes part of that gap — see below — though the restore itself and stakeholder-level coordination remain manual

Governance-reporting note: If you're reporting this capability up to a risk committee or board, resist framing it as "ransomware is covered." It's more accurate — and more useful for risk-register purposes — to say "we have automated Respond-phase containment; Govern-phase maturity, and the ML side of Detect, are tracked separately." CloudFormation-as-code and CloudWatch Logs give you the audit evidence (who deployed what, when a block fired and why) that a Govern program consumes as input, but they aren't a substitute for having the program itself.

Privacy note: The Content-Level PII Classification Scanner referenced in the Identify row above is worth a second look if your org is tracking data-minimization obligations. Its findings deliberately record only entity type, count, and confidence per file (EMAIL, SSN, CREDIT_DEBIT_NUMBER, and so on via Amazon Comprehend) — never the matched text itself — so a "this file has 3 SSN-pattern matches" report doesn't itself become a new PII exposure. That's useful for prioritizing where to apply DLP controls, but it isn't a substitute for a DPO's formal data classification review, and it currently can't see inside Office documents or PDFs (no text extraction yet) — plan around that blind spot if your regulated data lives mostly in those formats.

A point worth being precise about on the Recover side, since it's easy to overstate: a snapshot existing is a Protect-phase artifact, not proof that recovery works.

Recovery-sufficiency note: "We took a snapshot" and "we have a verified-clean recovery point" are two different maturity levels. Elastio's mapping of ransomware recovery to CSF 2.0 makes the same point industry-wide: RC.RP (Incident Recovery Plan Execution) is only credible once someone has actually restored from the recovery point and confirmed it's free of compromise — not merely confirmed the snapshot job succeeded. This module alone only delivers the first maturity level (the snapshot). A companion Verified-Clean Recovery Point Guide in this repo now automates a version of the second: it clones the candidate snapshot (FlexClone), scans the clone through an isolated S3 Access Point for ransomware-associated file extensions, and records a pass/fail verdict — without ever touching the production volume.

What "Verified Recovery" Looks Like in Practice

After the containment above creates its protective snapshot, you can verify it before restoring:

Candidate Snapshot
     │
     ▼ FlexClone (copy-on-write, instant, never modifies source)
     │
     ▼ Isolated S3 Access Point (VPC-scoped)
     │
     ▼ Scan for ransomware-associated extensions (.encrypted, .locked, etc.)
     │
     ▼ Verdict: clean / suspicious / error
     │
     ▼ Auto-cleanup (clone + access point deleted)

This is a single Step Functions workflow (5 Lambda functions). Deploy it and run a verification:

# Deploy the verification workflow
aws cloudformation deploy \
  --template-file shared/templates/restore-verification.yaml \
  --stack-name fsxn-restore-verification \
  --parameter-overrides \
    VpcId=<vpc-id> SubnetIds=<subnet-1>,<subnet-2> \
    SecurityGroupId=<sg-id> OntapMgmtIp=<mgmt-ip> \
    OntapCredentialsSecretArn=<secret-arn> \
  --capabilities CAPABILITY_NAMED_IAM

# Verify a specific snapshot before restoring
aws stepfunctions start-execution \
  --state-machine-arn <from stack outputs> \
  --input '{"svm_name":"svm-prod","volume_name":"vol_data","snapshot_name":"incident_response_20260712_005307"}'

The workflow completes in 2-5 minutes (depending on clone creation and access point attachment time). The verdict appears in the execution output and is logged to CloudWatch for audit.

What it doesn't do: This is a fast extension-pattern pre-filter — it flags a volume dominated by renamed/encrypted files. It is not a deep content inspection, and it does not exercise an actual end-to-end restore. Treat a "clean" verdict as a necessary first gate before committing to a restore, and still run periodic full restore drills into an isolated environment separately.

Restore-testing note: Be precise about what that verdict actually certifies, because it's easy to over-claim from the runbook name alone. The scan is a fast extension-pattern pre-filter — it flags a volume dominated by .encrypted/.locked-style renamed files — not a deep content inspection, and it does not exercise an actual restore end-to-end. Treat a "clean" verdict as a necessary first gate before a human commits to a restore, and still run periodic full restore-and-verify drills into an isolated environment separately. Don't let an automated pre-filter quietly become your only recovery test.

This module's protective snapshot, in other words, is Respond-phase evidence preservation, and the verification workflow above is a fast pre-filter toward Recover-phase confidence — neither is a substitute for a tested, human-verified restore. A fuller function-by-function breakdown — including where this repository's other components (ARP, EMS detection, Forensics dashboards, the recovery verification and PII scanning modules) land on Identify/Protect/Detect/Recover — lives in the Cyber Resilience Capability Map.

End-to-End Flow: ARP Detection → Auto-Block

Here's the full flow from ransomware detection to an automated storage-layer block. This flow has been verified end-to-end on ONTAP 9.17.1P7D1 (FSx for ONTAP, ap-northeast-1). Key timings from actual execution:

ARP/AI detection: triggered within minutes of ransomware-like file operations (no learning period — ARP/AI in ONTAP 9.16.1+ is immediately active)
contain_smb_threat Lambda execution: 1.8 seconds (snapshot + block + session disconnect in a single invocation)
SMB access denial confirmed after re-authentication

1. User "jdoe" starts encrypting files on vol_data
   (zip with password → delete original → new extension)
         │
2. ONTAP ARP/AI detects anomaly (entropy + extension changes)
   No learning period required (ONTAP 9.16.1+ / ARP/AI)
         │  Detection within minutes
         ▼
3. EMS event: callhome.arw.activity.seen (severity: alert)
   ONTAP auto-creates Anti_ransomware_attack_backup snapshot
         │
4. EMS Webhook → API Gateway → Lambda → Observability Platform
         │  ~30 seconds
         ▼
5. Observability Monitor fires (ARP alert detected)
         │
6. Monitor → SNS publish (contain_smb_threat)
         │  ~5 seconds
         ▼
7. Response Lambda executes (1.8s measured):
   a) Creates incident_response_20260712_005307 snapshot
   b) Blocks CORP\jdoe via name-mapping (deny on re-authentication)
   c) Attempts CIFS session disconnect
   d) Sends notification to security team
         │
8. Total time: detection + routing (~35s) + response (1.8s) ≈ under 2 minutes

SOC correlation tip: Always include incident_id and detection_source in your SNS message so that the storage-layer block can be correlated with the upstream detection in your SIEM timeline:
{"action":"contain_smb_threat","svm_name":"svm-prod","domain":"CORP",
 "username":"jdoe","volume_name":"vol_data",
 "reason":"ARP alert","incident_id":"INC-2026-0712-001",
 "detection_source":"datadog-monitor-arp"}
How immediate blocking actually works: The contain_smb_threat composite action achieves effective immediate cutoff through a two-step mechanism:

name-mapping deny — blocks all future authentication attempts (evaluated at session setup time, not per-I/O)

disconnect_smb_sessions — forcefully terminates the user's existing active sessions via DELETE /protocols/cifs/sessions/{svm}/{id}/{conn}

When the existing session is terminated, the Windows SMB client automatically attempts to re-establish the connection — at which point the name-mapping deny takes effect and access is refused. The net result: the user is cut off within seconds of the Lambda executing, not at the next natural session expiry.

If the disconnect call returns HTTP 404 (session already gone — e.g., the encrypting process finished before the Lambda fired), that's expected and logged but not treated as an error. The name-mapping deny still prevents any future reconnection.

ARP/AI verification note (ONTAP 9.16.1+): The CLI command security anti-ransomware volume attack simulate referenced in some ONTAP documentation does NOT exist in ONTAP 9.17.1. To trigger ARP/AI detection for testing, perform actual ransomware-like file operations: encrypt files with a password, delete originals, and add a new file extension. ARP/AI detects the high-entropy + deletion + never-seen-extension pattern. Detection threshold: 5+ distinct new extensions within 48 hours. See the ONTAP REST API Quick Reference for details.

What Blocking Looks Like: Visual Evidence

The following was captured during E2E verification on ONTAP 9.17.1P7D1 (July 2026).

NFS: Before and After

Before — user can list and read files normally:

$ ls /mnt/fsxn/
hr-salary.txt  project-spec.txt  reports/

$ cat /mnt/fsxn/hr-salary.txt
Confidential HR Record - Employee Salary Data

After — export-policy deny rule applied (immediate, same-subnet):

$ ls /mnt/fsxn/
ls: cannot access '/mnt/fsxn/': Permission denied

$ cat /mnt/fsxn/hr-salary.txt
cat: /mnt/fsxn/hr-salary.txt: Permission denied

SMB: Before and After (AD-joined SVM)

Before — domain user DEMO\testuser can access CIFS share:

PS> Get-ChildItem X:\
Mode   Length Name
----   ------ ----
d-----        reports
-a---- 46     hr-salary.txt
-a---- 43     project-spec.txt

PS> Get-Content X:\hr-salary.txt
Confidential HR Record - Employee Salary Data

After — block_smb_user executed (name-mapping → nobody, volume perms 750):

PS> Test-Path X:\
False
# ACCESS DENIED — drive not accessible after block

Step Functions: Recovery Point Verification Workflow

Step Functions console: All states succeeded (green). Verified-clean recovery point workflow completed end-to-end — waiting for FSx API to discover the ONTAP-created FlexClone. This is the ONTAP→FSx sync delay documented by AWS ("may take up to several minutes").

Detection & Forensics: Datadog Dashboard

Datadog Forensics Dashboard: Incident timeline (Detection → Response → Verification), user activity tracking, affected volumes, automated response actions log, ARP alert feed, and recovery verification results — all from a single pane.

Datadog Log Explorer: Real-time ARP detection events (callhome.arw.activity.seen, callhome.arw.volume.state) with structured attributes — entropy score, affected file count, new extensions detected.

File Access Audit Trail: Who (CORP\jdoe) accessed what (/vol_data/finance/budget-2026.docx) with what operation (WriteData|Delete) — the same forensic visibility DII Storage Workload Security provides.

Scrolled right: Client IP (198.51.100.99) visible alongside the structured JSON content — enabling "who accessed what from where" forensic queries.

Vendor-neutral note: The screenshots above use Datadog as a representative example, but this repository delivers the same logs to all 9 supported observability vendors (Datadog, Grafana, Splunk, Elastic, New Relic, Dynatrace, Sumo Logic, Honeycomb, OTel Collector). Deployable forensics dashboard definitions (user timeline, file access audit trail, affected volumes, client IP analysis) are provided for Datadog (JSON), Grafana (JSON), Splunk (SPL), and Elastic (KQL Saved Searches). See the per-vendor reference.

Testing

The module has 52 unit tests covering:

Category	Tests	What's Verified
SMB blocking	3	Success, SVM not found, API error
SMB unblocking	2	Success, not found
NFS blocking	2	Success, policy not found
NFS unblocking	2	Success, not found
Snapshots	5	Success, cooldown active/expired/disabled, volume not found
Session disconnect	3	By user, no sessions, missing params
Composite actions	3	Full sequence, partial failure, NFS containment
Input validation	7	Protected accounts, injection prevention, IP validation
Configurable protected accounts	1	`PROTECTED_ACCOUNTS_EXTRA` env var
List blocks	2	With results, empty
Health check	3	Success, SVM not found, unreachable
Error handling	3	Error attributes, timeout, malformed response
NACL blocking	16	Block IP, unblock, list active, find NACL, rule number allocation, marker validation

python3 -m pytest shared/python/tests/test_ontap_response.py shared/python/tests/test_network_block.py -v
# 52 passed in 0.06s

Verify Your Deployment

After deploying to your environment, run these checks within the first 30 days:

Day 1: health_check action succeeds (ONTAP reachable, credentials valid)
Day 1: Test create_snapshot on a non-production volume (confirm snapshot appears)
Week 1: Run a full contain_smb_threat drill against a test user on a test SVM
Week 1: Verify TTL cleanup removes the test block after configured minutes
Day 30: Confirm DLQ is empty (no silent failures), CloudWatch dashboard shows expected invocation count

If any of these fail, check the Troubleshooting section in Prerequisites.

EMS Webhook Integration: Payload Format

When connecting ARP EMS events to your detection pipeline, note the actual ONTAP EMS webhook payload format (verified on ONTAP 9.17.1P7D1):

{
  "message-name": "callhome.arw.activity.seen",
  "message-severity": "alert",
  "message-timestamp": "2026-07-12T00:42:06+00:00",
  "parameters": {
    "vserver-name": "svm-prod",
    "volume-name": "vol_data"
  }
}

Caution: Field names use hyphens (message-name), not camelCase (messageName) or snake_case (message_name). This matters for SIEM parsing rules and Lambda handler field validation.

The EMS event for ARP attack detection is callhome.arw.activity.seen (severity: alert), not arw.volume.state (which is a state-change notification for ARP enable/disable). The callhome.* prefix indicates an ONTAP AutoSupport-class event (internal telemetry classification, not external data transmission). Configure your detection rules to match the correct event name.

Resources

GitHub: ontap_response.py
GitHub: automated-response.yaml
GitHub: Automated Response Guide
GitHub: ONTAP REST API Quick Reference — Common pitfalls (svm.uuid duplicate, allow_privileged_access, ARP/AI behaviors), verified on ONTAP 9.17.1P7D1
GitHub: EMS Detection Capabilities — 30+ events, Push vs Pull delivery, latency comparison
GitHub: Cyber Resilience Capability Map — full NIST CSF 2.0 function mapping with FSx for ONTAP implementation and alternative implementation paths
GitHub: Verified-Clean Recovery Point Guide — FlexClone + isolated scan to verify a snapshot before restoring (CSF 2.0 RC.RP)
GitHub: Content-Level PII Classification Scanner — Amazon Comprehend-based PII discovery for file contents (CSF 2.0 Identify)
NetApp: ONTAP REST API Reference
AWS: FSx for ONTAP ARP
NIST Cybersecurity Framework (CSF) 2.0
NIST IR 8374r1 — Ransomware Risk Management: A CSF 2.0 Community Profile

Alerting on FSx for ONTAP Audit Logs — No Metric Filter Required, with CloudWatch Log Alarms

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Fri, 03 Jul 2026 16:15:43 +0000

TL;DR

Once FSx for ONTAP audit logs land in CloudWatch Logs, a common next question is: "How do I get alerted when someone touches a specific file?" Until recently you had to build a metric filter first. As of July 1, 2026, CloudWatch Log Alarms let you create an alarm directly from a Logs Insights query — no metric filter step.

CloudWatch Log Alarm (GA 2026-07-01): alarm straight from a log query. No metric filter.
How it works: a Logs Insights query matches events, count(*) turns them into a number, and the alarm fires when the count crosses a threshold.
5 detection presets: sensitive-path access / failed access / bulk delete / privileged-user activity / custom.
One-command deploy: bash shared/scripts/deploy-log-alarm.sh (working in ~15 minutes).
Cost: ~$6.6/month (100 MB/day). Slightly higher than the metric-filter approach, but you get log lines in the notification and retroactive queries.
E2E verified: CloudFormation deploy through state transition, in the Tokyo region.

GitHub: Yoshiki0705/fsxn-observability-integrations

This is Part 17 of the Serverless Observability for FSx for ONTAP series. It builds directly on the admin-audit pipeline from Part 14.

Two AWS Launches That Make This Work

This article sits on top of two recent CloudWatch launches. If you're arriving from outside this series, read these first — together they remove the EC2 syslog server and the metric-filter step that used to stand between ONTAP logs and an alert.

1. Managed syslog ingestion (June 2026) — how the logs get in

ONTAP emits its admin audit trail (and EMS events) as syslog. Historically, capturing that meant running an EC2 syslog server. In June 2026, CloudWatch Logs added managed syslog ingestion: you send syslog (RFC 5424 / RFC 3164 / Cisco FTD/ASA) over a VPC endpoint straight into a log group — no agent, no EC2.

For FSx for ONTAP, that means pointing ONTAP log-forwarding at the syslog VPC endpoint and having the admin audit log arrive in CloudWatch Logs directly.

What's New: Amazon CloudWatch Logs supports managed syslog ingestion
Setup guide: Setting up syslog ingestion — VPC endpoint, log group, resource policy, and the three transport options (TCP+TLS 6514 encrypted, TCP 1514 plaintext, UDP 514 best-effort)

This is the piece the previous article (Part 14 / Syslog VPCE setup) already wired up. If you followed it, your admin audit log is in CloudWatch Logs and this article's alarm has something to query.

2. CloudWatch Log Alarms (July 2026) — how you alert on them

The second launch is the subject of this article: alarms defined by a Logs Insights query rather than a metric.

What's New: Amazon CloudWatch supports creating alarms from log queries
Docs: Alarming on logs

Put together: ONTAP → managed syslog ingestion → CloudWatch Logs → Log Alarm → SNS. No EC2, no metric filter, no forwarding Lambda.

What Changed

Alerting on log content is a basic monitoring need. On CloudWatch it used to take a detour: create a metric filter, generate a custom metric from it, then attach an alarm to that metric — three steps. From "I want to detect this" to an alert actually firing was 15–30 minutes of setup.

Log Alarms collapse the middle steps. You give a Logs Insights query a threshold, and the alarm reads straight from the logs.

Aspect	Before (metric filter)	Log Alarm (NEW)
Setup steps	3 (filter → metric → alarm)	1 (Log Alarm only)
Query flexibility	Filter pattern syntax only	Full Logs Insights syntax
Log lines in notification	No	Yes (up to 50)
Retroactive to existing logs	No	Yes
IaC	2 resources	1 resource (`AWS::CloudWatch::LogAlarm`)

The part I appreciate most: the notification can include the matched log lines. When the alert lands, you already see who touched which file and what they did — you can form a first read before opening the console.

FSx for ONTAP Logs × Log Alarm

There are three FSx for ONTAP log/event families, and it's worth separating them because only some are a natural fit for Log Alarms.

Log type	What it records	Delivery path	Log Alarm target
Admin audit log	ONTAP management ops (CLI / REST API)	Managed syslog ingestion → CloudWatch Logs	✅ this article
File access audit log	NAS file/folder ops (NFS / SMB)	FSx for ONTAP S3 AP → Lambda	Separate pipeline
EMS events	ONTAP system events (capacity / HA / ARP)	Managed syslog ingestion → CloudWatch Logs, or EMS Webhook → Lambda	✅ when syslog-delivered

Admin audit log is the "who did what as an administrator" record — logins (success/failure), security login account and role changes, volume create/offline/delete, vserver config changes, and privileged operations like system node systemshell or set -privilege diagnostic.

Two ONTAP settings, don't conflate them: security audit decides what gets recorded (e.g., security audit modify -cliget on -httpget on -ontapiget on to capture read/GET operations — off by default, so without it your sensitive-file-access query on admin ops comes up empty), while cluster log-forwarding decides where it's sent. The exact commands are in the Syslog VPCE setup guide. Note cluster log-forwarding supports multiple destinations, so you can add CloudWatch alongside an existing on-prem SIEM without cutting over.

File access audit log is the "which user did what to which file" record, enabled via vserver audit create and emitted in Windows Security Event format (EVTX / XML). It's the right source for sensitive-folder access and mass-delete detection — but to use it with a Log Alarm you'd need a separate pipeline to land EVTX/XML into CloudWatch Logs. In this project, that content flows via the FSx for ONTAP S3 Access Point → Lambda path to each vendor instead.

⚠️ Critical scoping (read before you trust a preset name): the presets in this article run against the admin audit log (/syslog/fsxn-admin-audit). That log contains management-plane operations — it does not see end-user file I/O over NFS/SMB. So on this log group:

bulk-delete-operations detects admin-plane destructive ops (Snapshot delete, volume offline/delete) — not ransomware encrypting user files over SMB.

sensitive-file-access matches a path only when an admin command references it — not when a user opens that file.

For user-file ransomware/mass-delete/sensitive-access detection, use ONTAP ARP (Part 3) and FPolicy / file-access audit (Part 4). The same preset works against a file-access-audit log group if you land that data in CloudWatch Logs — but on /syslog/fsxn-admin-audit it only sees the admin plane.

EMS events (system events)

The third source is EMS (Event Management System) — ONTAP's internal system-event notifications. Where audit logs say "who did it", EMS says "what happened to the system", across seven severities (emergency → debug). Representative events this project normalizes:

Event	Severity	Meaning
`arw.volume.state` / `arw.vserver.state`	alert	ARP (Autonomous Ransomware Protection) state transition
`monitor.volume.full` / `wafl.vol.full`	alert	Volume space exhaustion
`wafl.quota.hardlimit.exceeded`	error	Quota hard-limit exceeded
`cf.fsm.takeoverStarted`	alert	HA takeover started
`net.linkDown`	alert	Network link down

EMS has two routes. This project's EMS Webhook path (HTTPS → API Gateway → Lambda) normalizes and ships events to any vendor/OTLP. Alternatively, ONTAP 9.x can forward EMS over syslog (event notification destination create -syslog ...) to the same CloudWatch Logs syslog VPC endpoint you built for the admin audit log — no separate EC2 syslog server — and then EMS becomes a Log Alarm target directly. If your need is "alert immediately on volume-full or an ARP state change", syslog-to-CloudWatch + a Log Alarm is the shortest path.

Architecture

FSx for ONTAP (ONTAP log-forwarding)
    │ Syslog TCP (TLS 6514)
    ▼
CloudWatch Logs managed syslog ingestion (VPC endpoint)
    │  → /syslog/fsxn-admin-audit
    │ Scheduled Query (5 min)
    ▼
CloudWatch Log Alarm
    │ count(*) > threshold → ALARM
    ▼
SNS → Email / Slack / PagerDuty (with log lines)

The actual log format

Writing a Log Alarm query is easier if you know what the line looks like. A real admin-audit line in CloudWatch Logs:

<190>Jul  2 03:17:37 FsxId...-02: [kern_audit:info:6392]
...:: FsxId...:ssh :: <source-ip>:unknown ::
FsxId...:fsx-control-plane:admin ::
system node systemshell -node * -command "top -d 1 -s 1"
:: Success

It reads like an incantation, but everything is there: when, which protocol (ssh/http), from where (source IP), who (user), what (command), and the outcome (Success/Failure). A query usually just needs one of those.

Why "Alert on a String" Actually Means "Count Then Compare"

Worth recalling: a CloudWatch alarm compares a number against a threshold. It is not natively a "fire when this string appears" trigger.

So how does a Log Alarm alert on content? It turns the string into a number first. A Logs Insights query narrows to matching events, count(*) converts them to a count, and the alarm fires when that count crosses the threshold. Read it as string → count → threshold and it clicks.

For example, to detect access to a confidential folder:

fields @timestamp, @message
| filter @message like /\/vol\/data\/confidential/

Set the aggregation to count(*) and the threshold to > 0, and a single access within the 5-minute window flips the alarm to ALARM. "Tell me the moment anyone touches the confidential folder" — done, with just that.

Deploy

The repo ships a deploy script, so a few environment variables bring up the whole set of resources.

Deploy script (recommended)

# Sensitive-file access detection (auto-creates the SNS topic)
DETECTION_TYPE=sensitive-file-access \
TARGET_PATTERN="/vol/data/confidential" \
CREATE_SNS_TOPIC=true \
SNS_TOPIC_NAME=fsxn-security-alerts \
  bash shared/scripts/deploy-log-alarm.sh

The script creates the SNS topic, deploys the CloudFormation stack, and prints the alarm name and console URL.

CloudFormation template

# shared/templates/cloudwatch-log-alarm.yaml (excerpt)
Resources:
  SensitiveFileAccessAlarm:
    Type: AWS::CloudWatch::LogAlarm
    Properties:
      AlarmName: fsxn-sensitive-file-access
      ComparisonOperator: GreaterThanThreshold
      Threshold: 0
      QueryResultsToEvaluate: 3
      QueryResultsToAlarm: 1
      TreatMissingData: notBreaching
      ScheduledQueryConfiguration:
        QueryString: |
          fields @timestamp, @message
          | filter @message like /\/vol\/data\/confidential/
        LogGroupIdentifiers:
          - /syslog/fsxn-admin-audit
        ScheduledQueryRoleARN: !GetAtt ScheduledQueryRole.Arn
        AggregationExpression: "count(*)"
        ScheduleConfiguration:
          ScheduleExpression: "rate(5 minutes)"
          StartTimeOffset: 300
      AlarmActions:
        - !Ref AlarmSNSTopic
      ActionLogLineCount: 5
      ActionLogLineRoleArn: !GetAtt LogLineRole.Arn

On QueryResultsToEvaluate / QueryResultsToAlarm (M-of-N): this is the flapping control — 3 / 1 fires fast but can flap on a single spike; 3 / 2 smooths transient blips at ~one extra interval of latency. Tune this before the threshold. Per-use-case recommendations are in the setup guide.

Detection presets

The template ships five presets for common patterns. Switch DetectionType and each gets an appropriate query and threshold.

DetectionType	Use case	Default threshold
`sensitive-file-access`	Access to a specific path	> 0
`failed-access-attempts`	Authentication/authorization failures	> 10
`bulk-delete-operations`	Mass deletion (ransomware signal)	> 50
`specific-user-activity`	Privileged-user monitoring	> 0
`custom`	Any Logs Insights query	your choice

⚠️ Regulated environments — one thing to get right first: ActionLogLineCount puts the matched log lines (usernames, file paths, client IPs — potentially PHI) into the SNS notification, which leaves the CloudWatch boundary. For healthcare/finance/public-sector data, default to ActionLogLineCount: 0 and let responders pivot into Logs Insights for the detail. This is a detection mechanism, not a compliance attestation — classify your fields and confirm APPI/FISC/ISMAP/HIPAA scope with your compliance team first. Full guidance, the regulated default, the alert-audit-trail requirements, and multi-account (StackSets) rollout are in the setup guide.

Use Cases

1. Admin-plane destructive-op detection (defense in depth)

To be precise about scope: on the admin audit log this preset catches management-plane destructive operations — a burst of Snapshot deletes, volume offline, volume delete — the kind of action an attacker with stolen admin credentials (or a mistaken operator) would take to remove recovery points. It does not see user-file encryption over SMB; that's ONTAP ARP's job at the storage layer (Part 3), with FPolicy (Part 4) for per-file operations. Layer all three: ARP for encryption, FPolicy for file ops, and this Log Alarm for admin-plane tampering (e.g., someone deleting the Snapshots you'd restore from).

DETECTION_TYPE=bulk-delete-operations \
ALARM_THRESHOLD=50 \
QUERY_RESULTS_TO_ALARM=2 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

Detection layer	Catches	Method	Latency
Storage layer (ARP)	User-file encryption	ML-based entropy analysis	Real-time
File ops (FPolicy)	Per-file create/write/delete/rename over NFS/SMB	Protocol-level intercept	~6 s (validated, Part 4)
Admin plane (Log Alarm)	Snapshot/volume destructive ops by admins	Count-based threshold on admin audit log	~5 min

Three different vantage points on the same attack: ARP sees the encryption, FPolicy sees the file operations, and this Log Alarm sees an admin deleting the Snapshots you'd recover from. Layering them means whatever slips past one is more likely caught by another.

In ATT&CK terms, that admin-plane view is T1490 Inhibit System Recovery — an attacker deleting your restore points so you can't roll back the encryption ARP detects (T1486). Two techniques, two controls: detect the Snapshot deletion here, and prevent it with SnapLock (WORM Snapshots that can't be deleted before expiry) so your recovery points survive the attempt. The full ATT&CK mapping, tamper-resistance guidance (who can delete the alarm and how to guard it), and a one-slide coverage map are in the setup guide.

Ops note (baseline first): Scheduled bulk operations — nightly backups, batch ETL, archive cleanups — can legitimately exceed a "50 deletes / 5 min" threshold and page on-call for nothing. Baseline your normal delete volume for a few days without an alarm action, then set the threshold above your routine peak (the exact baseline query is in the setup guide). This mirrors the ARP learning-period caveat from Part 3.

2. Compliance: notify on regulated-data access

For finance or healthcare, where "any touch of this data must be recorded and notified", a simple per-path detection works.

DETECTION_TYPE=sensitive-file-access \
TARGET_PATTERN="/vol/finance/" \
ALARM_THRESHOLD=0 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

Alert-fatigue note: A > 0 threshold on an actively-used path pages on every access and quickly becomes noise. Reserve > 0 for genuinely restricted paths (break-glass directories, quarantined data). For paths with legitimate regular access, prefer a rate threshold (e.g., access from an unexpected principal, or volume above a baseline) and route to a ticket/Slack channel rather than a pager.

3. Privileged-user monitoring

Keeping a record of every admin-account action uses the same mechanism.

DETECTION_TYPE=specific-user-activity \
TARGET_PATTERN="fsxadmin" \
ALARM_THRESHOLD=0 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

E2E Validation (Tokyo Region)

Because it's a brand-new feature, I was skeptical it would behave as documented. So I deployed the template in a real Tokyo-region environment and drove the alarm through a state transition.

Item	Result
CloudFormation deploy	✅ Success
IAM role auto-creation	✅ ScheduledQueryRole + LogLineRole
Scheduled Query execution	✅ INSUFFICIENT_DATA → OK transition confirmed
Console display	✅ Shown as "Log alarm" type
Logs Insights query	✅ Matched (12 hits / 3,482 records scanned for `/volume/`; 472 hits for `ssh`)

In the console it shows up as a new "Log alarm" type, distinct from the metric alarms you already have. In the screenshot below, look at the Type column — the new alarm is labeled "Log alarm" rather than "Metric alarm".

Opening the alarm shows the Log Alarm detail page. Note the query configuration (the Logs Insights query string, the target log group /syslog/fsxn-admin-audit, and the 5-minute schedule) and the two IAM roles CloudFormation created automatically — ScheduledQueryRole (runs the query) and LogLineRole (attaches matched log lines to the notification).

Running the query in Logs Insights hits real audit data. The bar chart at the top shows the match count per interval, and the results table below lists the matched log lines — here the /volume/ filter returned 12 matches across 3,482 records scanned.

The alarm itself stays OK while there's no matching access — the screenshot shows the state after the initial INSUFFICIENT_DATA → OK transition, with an empty history graph because nothing has crossed the threshold yet. With threshold > 0, it flips to ALARM the moment a single access to the sensitive path appears.

Audit trail of the alert itself: for compliance you also need evidence that the alarm fired and who was notified. Alarm state history is retained 90 days (fixed); for multi-year evidence, route CloudWatch Alarm state-change events (EventBridge) to S3, and rely on CloudTrail for the "who configured this detection" record. Details and retention table: setup guide.

Gotchas

Gotcha	Workaround
AWS CLI not yet supported (no `put-log-alarm`)	Use CloudFormation
cfn-lint E3006 (resource type not yet in the spec)	Suppress per-resource (not a blanket disable); exact `Metadata` snippet in the setup guide
First evaluation takes 5–10 min	Just wait
Notification includes log lines (PII/PHI risk)	Set `ActionLogLineCount=0` in regulated environments (see the callout above)
Multi-node log streams (`FsxId...-01/-02`)	Query the whole log group, don't pin `@logStream`, or you miss half the traffic on HA takeover

The "AWS CLI doesn't have put-log-alarm yet" is a just-after-GA reality — for now CloudFormation or the console are the only creation paths. Platform/CoE note: drift detection won't cover a resource type the CLI can't yet describe, so treat CloudFormation as the single source of truth until the API surface completes.

Cost

Rough estimate for one alarm at a 5-minute cadence.

Logs/day	Log Alarm	Metric filter
100 MB	~$6.6/month	~$3/month
500 MB	~$33/month	~$3/month
1 GB	~$66/month	~$3/month

How the cost scales: the table above is per alarm. Each alarm runs its own Scheduled Query over the same log group, so cost is alarms × cadence × scan size — ten alarms on one log group is ~10×, not a flat add-on. Narrow queries with filter/limit and consolidate related detections to keep it bounded. Full breakdown in the setup guide.

Honestly, Log Alarms cost more than the metric-filter approach — the Scheduled Query scans logs on each run. But the difference is recoverable elsewhere: log lines in the notification cut investigation time, it applies retroactively to existing logs, and setup is a single step.

When the premium is worth it (a line you can put in a proposal): choose Log Alarms when the alert content matters for triage (you want the matched lines in the page), when the detection query needs full Logs Insights syntax a metric filter can't express, or when you need to apply it retroactively to existing logs. Stick with metric filters for high-volume, well-understood, purely numeric signals where a few dollars × many alarms adds up. For most teams the crossover is engineer time: if the log-line context saves even one 15-minute console dig per incident, the premium pays for itself.

If volume makes cost a concern, dropping the cadence to 15 minutes cuts it to a third, and adding limit to the query bounds the scan.

When to Use Which

Log Alarms aren't meant to replace every kind of monitoring. Depending on the need, metric filters or the OTel Collector fit better. Among this project's three delivery paths, the Log Alarm is the simplest entry point.

Path	When	Extra infra
Log Alarm (this article)	Simple threshold alerts	None (self-contained in CloudWatch)
Lambda → vendor	Dashboards, SIEM	Forwarding Lambda
OTel Collector	Multi-backend, PII redaction	Forwarding Lambda + Collector (ECS Fargate)

Concretely, the OTel Collector path's "extra infra" is two layers: three forwarding Lambdas (audit-log shipper, EMS handler, FPolicy handler) sending OTLP/HTTP, plus a resident otel/opentelemetry-collector-contrib task on ECS Fargate (with NAT Gateway for egress, Cloud Map for task IP resolution, ALB/autoscaling under load). Fan-out to Grafana/Honeycomb/Datadog and PII redaction are then a single change in the Collector's config YAML. The Log Alarm, by contrast, is the "everything stays inside CloudWatch" minimal option.

A Log Alarm is a first alert, not a full investigation tool. Let it handle "notice it first", and hand off the deep dive to vendor tooling like Datadog or Splunk. That division of labor is the realistic one.

For ONTAP operators, the setup guide covers the storage-side specifics: what security audit captures vs what log-forwarding sends, keeping an existing SIEM alongside CloudWatch (multiple destinations), ONTAP EMS native email/SNMP as an alternative to pushing to AWS, a dead-man's-switch heartbeat alarm for when syslog delivery stops, and DR-region deployment — ONTAP operational notes.

Cleanup

# Delete all Log Alarm stacks
bash shared/scripts/cleanup-log-alarm.sh --all -y

What's Next

CloudWatch Log Alarms aren't a flashy feature. But "turn what you noticed in the logs straight into an alert" lowers the bar for setting up monitoring — you're done with one query before you'd have finished building a metric filter, staring at a metric, and wiring an alarm. It pairs well with FSx for ONTAP audit logs, answering the "I just want to notice it first" needs of storage security without extra infrastructure. There's some just-after-GA roughness (the AWS CLI hasn't caught up), but CloudFormation works today, verified in a real environment.

Upcoming in the project:

Phase 4: Terraform module equivalents
Phase 4: CDK construct library
PagerDuty escalation for CloudWatch alarms — see pagerduty-escalation-guide

See the full ROADMAP.

Resources

GitHub: github.com/Yoshiki0705/fsxn-observability-integrations
Template: cloudwatch-log-alarm.yaml
Deploy script: deploy-log-alarm.sh
Setup guide (EN): cloudwatch-log-alarm.md
Runbook: log-alarm-triggered.md
Syslog VPCE setup: syslog-vpce-setup-guide.md

AWS references

What's New — CloudWatch supports creating alarms from log queries (2026-07)
What's New — CloudWatch Logs supports managed syslog ingestion (2026-06)
Docs — Alarming on logs
Docs — Setting up syslog ingestion

Series Navigation

Part 1: Why Your FSx for ONTAP Logs Deserve Better
Part 3: Event-Driven Ransomware Detection with ONTAP ARP + Datadog
Part 5: Escape Vendor Lock-in with OTel Collector
Part 13: 9 Vendors, One Architecture
Part 14: Can You Use System Manager with FSx for ONTAP?
Part 17: Alerting on Audit Logs with CloudWatch Log Alarms (this post)

If you deploy this, I'd love to hear how it went — drop a comment or open a GitHub issue.

GitHub: github.com/Yoshiki0705/fsxn-observability-integrations

42 Patterns, Category Architecture, and HA LifeKeeper Monitoring — FSx for ONTAP S3 Access Points, Phase 18

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 21 Jun 2026 16:54:22 +0000

TL;DR

Phase 18 restructures the entire repository from 41 flat directories into a categorized solutions/ hierarchy, adds HA LifeKeeper Monitoring as a new pattern category, introduces 5 category-specific architecture diagrams, and establishes modern Python project infrastructure. The repository now contains 42 deployable patterns organized for discoverability at scale.

Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

Why Restructure?

With 41 pattern directories at the repository root, navigating the project had become unwieldy. New contributors could not quickly find patterns by domain, the README required extensive scrolling, and adding new categories (HA, GenAI) had no clear placement convention.

Before (flat):

legal-compliance/  financial-idp/  semiconductor-eda/  sap-erp-adjacent/
flexcache-anycast-dr/  genai-kb-selfservice-curation/  event-driven-fpolicy/
... (41 directories at root)

After (categorized):

solutions/
├── industry/           # 28 UC patterns (UC1-UC28)
├── flexcache/          # 7 FlexCache/FlexClone patterns
├── genai/              # 2 GenAI patterns (UC29-UC30)
├── sap/                # SAP/ERP pattern
├── ha/                 # HA monitoring (new)
├── event-driven/       # 2 FPolicy event-driven patterns
└── edge/               # CDN/edge delivery

The Move: git mv with History Preservation

All 41 directories were moved using git mv, preserving full commit history. Key renames include:

Old Path	New Path	Reason
`sap-erp-adjacent/`	`solutions/sap/erp-adjacent/`	Category grouping
`dynamic-flexcache-render-workflow/`	`solutions/flexcache/dynamic-render-workflow/`	Shorter name
`genai-kb-selfservice-curation/`	`solutions/genai/kb-selfservice-curation/`	Strip prefix
`devops-flexclone-cicd/`	`solutions/flexcache/devops-cicd/`	Strip prefix
`content-edge-delivery/`	`solutions/edge/content-delivery/`	Category grouping

Use git log --follow <file> to trace history across the move.

HA LifeKeeper Monitoring — New Pattern

SIOS LifeKeeper is a Linux/Windows HA clustering solution that can be used on Amazon EC2 for application-aware failover scenarios. With FSx for ONTAP Multi-AZ as shared storage (NFS/iSCSI, depending on OS and configuration), this pattern focuses on observing LifeKeeper logs without putting monitoring agents on the HA nodes.

The new HA pattern (solutions/ha/lifekeeper-monitoring/) provides non-intrusive log analysis:

graph TB
    subgraph "HA Cluster"
        LK1[LifeKeeper Node 1<br/>Active]
        LK2[LifeKeeper Node 2<br/>Standby]
    end

    subgraph "Shared Storage"
        FSXN[FSx for ONTAP Multi-AZ]
        S3AP[S3 Access Point<br/>Read-only log access]
    end

    subgraph "Analysis Pipeline"
        SFN[Step Functions]
        DISC[Discovery Lambda<br/>Log classification]
        PROC[Processing Lambda<br/>Bedrock Root Cause Analysis]
        RPT[Report Lambda<br/>Health score + alerts]
    end

    LK1 -->|Log write| FSXN
    FSXN --> S3AP -->|Non-intrusive read| DISC
    SFN --> DISC --> PROC --> RPT
    PROC -->|Nova Pro| BEDROCK[Amazon Bedrock]
    RPT --> SNS[SNS Alert]

Key Design Decisions

Non-intrusive to HA nodes: No monitoring agent is installed on HA nodes. The S3 AP read path avoids host-level changes, while still consuming FSx/S3 API throughput like any other read workload.
Human-in-the-loop: AI analysis is advisory only. LifeKeeper's own health checks handle failover decisions.
Health Scoring: 0-100 score with deductions for failover events, comm path latency, and resource state anomalies.
Root Cause Analysis: Bedrock Nova Pro analyzes state transitions (ISP→OSF, ISS→ISP) to identify likely causes.

Implementation note: This pattern observes LifeKeeper logs and produces advisory analysis. It does not replace LifeKeeper cluster design, quorum/witness configuration, split-brain prevention, protocol-specific recovery kit setup, or application-level failover testing.

Example Operational Metrics

These are suggested evaluation metrics for future validation. Phase 18 verifies the DemoMode pipeline, not real-cluster failover triage time.

Category	Metric	Demo Target
Operations	Time from workflow start to structured triage report	< 10 min
Technical	Log discovery completeness	All configured LifeKeeper log paths in scope
Quality	False positive alert rate	< 5% (requires real-cluster validation)
Cost	Monthly monitoring cost	< $15 (5-min polling, 1 cluster)

Deploy (DemoMode)

DemoMode=true deploys without FSx for ONTAP — uses a regular S3 bucket with sample LifeKeeper logs:

cd solutions/ha/lifekeeper-monitoring
sam build && sam deploy --guided \
  --parameter-overrides DemoMode=true \
    S3AccessPointAlias=your-demo-bucket \
    OutputBucketName=your-output-bucket

DemoMode verification confirms: SAM template deploys successfully, Step Functions workflow executes all states, Discovery Lambda classifies sample logs correctly, and Processing Lambda generates a health score report. Bedrock analysis produces structured, advisory observations based on log patterns; failover decisions remain outside the AI workflow.

Verified in ap-northeast-1 on 2026-06-21. The sample log set intentionally contains failure-like events, resulting in a demo health score of 40/100. The score is not a benchmark for LifeKeeper or FSx for ONTAP; it demonstrates that the scoring pipeline detects and reports anomalies from sample logs.

Coming in Phase 19: Full E2E verification with a real SIOS LifeKeeper HA cluster (AWS Marketplace + FSx for ONTAP Multi-AZ) including live failover testing, actual state transition detection, and Bedrock RCA quality assessment against real-world failure scenarios.

Category Architecture Diagrams

Phase 18 adds 5 mermaid architecture diagrams to README.md (both JA and EN), each in a collapsible <details> block:

Category	Key Components
🏭 FlexCache	ONTAP REST API → HealthCheck → RouteDecision → DynamoDB routing → Create/Cleanup lifecycle
🤖 GenAI	FPolicy → SQS → EventBridge → Bedrock KB → RetrieveAndGenerate / Agentic tools
🛡️ HA	S3 AP non-intrusive read → Bedrock RCA → Health score → SNS alerts
⚡ Event-Driven	FPolicy Engine → ECS Fargate TCP → SQS → EventBridge rule routing
🌐 Edge/CDN	3 delivery modes (ORIGIN_PULL, OAC, PUBLISH_PUSH) → vendor-neutral CDN

All diagrams include accTitle and accDescr for screen reader accessibility.

Project Infrastructure Improvements

pyproject.toml (PEP 621)

Modern Python project metadata with unified tool configuration:

[tool.ruff]
target-version = "py312"
line-length = 120

[tool.pytest.ini_options]
addopts = "-v --tb=short --import-mode=importlib"

[tool.coverage.report]
fail_under = 80

Dependency Pinning

# requirements.txt — exact versions for reproducibility
boto3==1.43.29
urllib3==2.7.0
jsonschema==4.17.3

# requirements-dev.txt
pytest==9.1.0
hypothesis==6.155.2
moto==5.2.2
ruff==0.15.17
cfn-lint==1.51.4

.cfnlintrc

Project-wide cfn-lint configuration that discovers all templates under solutions/:

templates:
  - "solutions/**/template.yaml"
  - "solutions/**/template-deploy.yaml"
ignore_checks:
  - W3002  # Local CodeUri (sam build handles upload)
  - W1031  # Fn::Sub false positive with Secrets Manager ARNs
regions:
  - ap-northeast-1

Additional Tooling

File	Purpose
`.gitattributes`	Consistent line endings, language-specific diff drivers
`.github/PULL_REQUEST_TEMPLATE.md`	Project-specific PR checklist
`solutions/README.md`	Category navigation index
`CHANGELOG.md`	Keep a Changelog format, all releases
`CONTRIBUTING.md`	"Adding a New Pattern" section with category guide

CI/CD Changes

The CI pipeline was split to avoid pytest importlib mode namespace collisions when multiple patterns with identically-named handler.py files are collected together:

# Before: single pytest invocation (collision risk)
pytest shared/tests/ solutions/**/tests/ --cov=shared

# After: isolated invocations
- name: Run shared tests with coverage
  run: pytest shared/tests/ --cov=shared
- name: Run pattern tests
  run: pytest solutions/industry/*/tests/ solutions/flexcache/*/tests/ ...

Also added persist-credentials: false to all actions/checkout steps (zizmor security hardening).

Multi-Perspective Review

The restructuring and HA monitoring pattern were reviewed from partner delivery, storage architecture, HA operations, security, CI/CD, accessibility, and contributor onboarding perspectives. The review resulted in wording changes around neutrality, operational caveats, connector validation, HA safety boundaries, and repository discoverability.

What This Means for You

Role	Impact
Partner / SI / delivery team	Pattern selection is now intuitive by category — pick `solutions/industry/` for industry PoCs, `solutions/flexcache/` for distributed workloads
New contributor	`CONTRIBUTING.md` now includes "Adding a New Pattern" with required checklist and category selection guide
Existing repository users	`git log --follow <file>` still works. `sam build` and `sam deploy` are unchanged (run from the pattern directory)
CI/CD	Zero changes needed in your `samconfig.toml` or deployment scripts — CodeUri is relative to template.yaml

Infrastructure cost: Zero. This is a repository organization change; it does not deploy AWS resources by itself.

What's Next

Phase 16-17 blog articles (GenAI patterns) — ready to publish
dev.to series updated for directory restructuring (Phase 13 link fixed)
Next pattern candidates under evaluation

Getting Started

git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

# Browse patterns by category
ls solutions/

# Deploy HA LifeKeeper monitoring (DemoMode)
cd solutions/ha/lifekeeper-monitoring
sam build && sam deploy --guided --parameter-overrides DemoMode=true

# Run tests
make test-quick PYTHON=.venv/bin/python

Yoshiki Fujiwara

Amazon Quick Agentic Workspace Powered by FSx for ONTAP S3 Access Points — Phase 17

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 21 Jun 2026 15:54:23 +0000

TL;DR

UC30 bridges the gap between file-based business data and AI-powered actions. Business users maintain structured and unstructured data on an FSx for ONTAP SMB share, while Amazon Quick (Index / Sight / Flows) consumes it through S3 Access Points and a serverless Action API — providing search, BI, and governed action workflows from a single workspace.

Where UC29 focuses on "self-service knowledge ingestion into Bedrock KB," UC30 focuses on unifying search, analytics, and action execution behind Quick Suite's agentic interface.

New in this release: generate_brief_with_web action augments internal context with real-time web search results via AgentCore Web Search Tool (GA June 2026), enabling briefs that combine primary internal data with current public context.

Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns (see solutions/genai/quick-agentic-workspace/ and samconfig.toml.example)

Amazon Quick × S3 AP Data Mapping

Quick Feature	Role	S3 AP Data	Implementation
Quick Index / Research	Unstructured file search	`index/<role>/` (md/pdf)	S3 AP as data source
Quick Sight	Structured BI & visualization	`analytics/<role>/` (csv)	Glue/Athena (Athena Query Lambda)
Quick Flows	Action automation	`flows/<role>/` (json)	Action API (API Gateway + Lambda + Bedrock)
Quick Flows + Web	Web-augmented briefs	`flows/<role>/` + web	Action API + AgentCore Web Search (opt-in)

Seven roles (sales / marketing / finance / IT / operations / legal / developers) share the same AI-dedicated volume — reusable from UC29.

Design note: FSx for ONTAP S3 Access Points are useful as an integration boundary, but they do not remove the need to validate each consuming service connector. The access path combines S3/IAM policy evaluation with file-system-level identity authorization.

Architecture

Amazon Quick provides a unified workspace — search (Index), BI (Sight), and action automation (Flows) — powered by FSx for ONTAP data via S3 Access Points.

Windows Explorer (drag & drop into quick-workspace/ SMB share)
  ├── index/<role>/ → Quick Index (unstructured search)
  ├── analytics/<role>/ → Glue/Athena → Quick Sight (BI)
  └── flows/<role>/ → Action API → Quick Flows (actions)

Action API (6 actions):
  API Gateway (IAM auth / SigV4)
  → Lambda (per-action authorization + HITL gate)
  → generate_brief           → Bedrock Converse (internal context only)
  → generate_brief_with_web  → Bedrock Converse + AgentCore Web Search (hybrid)
  → create_action_item       → SNS notification
  → request_approval         → DynamoDB (HITL entry)
  → approve                  → DynamoDB (admin only)
  → execute_approved         → DynamoDB check + execution (enforced HITL)

Hybrid RAG Flow (generate_brief_with_web)

Quick Flows request: {"action": "generate_brief_with_web", "params": {...}}
  ├─→ [1] Internal context (from params.context — FSx for ONTAP file content)
  ├─→ [2] AgentCore Web Search (us-east-1, MCP protocol)
  │       query from params.web_query or params.title
  │       → Amazon web index → snippets + URLs + titles + dates
  └─→ [3] Bedrock Converse (ap-northeast-1)
           system prompt: internal = primary, web = supplementary, untrusted
        → Unified brief with [Internal: ...] + [Web: title](URL) citations

Security Design

Authentication + Per-Action Authorization

The Action API uses IAM authentication (SigV4). The handler extracts the authenticated caller identity (requestContext.identity) — not self-declared body fields — and performs per-action authorization:

ACTION_AUTH_MODE=open (default/demo): No enforcement; audit fields still bound to authenticated caller

For production, use ACTION_AUTH_MODE=enforce and explicitly define AUTHORIZED_PRINCIPALS and ADMIN_PRINCIPALS.

ACTION_AUTH_MODE=enforce (production):

Read-only actions (generate_brief, generate_brief_with_web): always allowed

Mutating actions: caller must match AUTHORIZED_PRINCIPALS

Admin actions (approve): caller must match ADMIN_PRINCIPALS

Mismatch → 403 Forbidden

Enforced Human-in-the-Loop (HITL)

High-risk operations are gated by a DynamoDB approval store:

request_approval → persists record as pending_approval (enforced=true)
approve → admin transitions to approved (ConditionExpression prevents race)
execute_approved → only executes if record is approved; otherwise 409

Verified live: execute pre-approval → 409, post-approval → 200, re-execute → 409 (no replay).

Security note: Approval records have a 7-day TTL (DynamoDB Time-to-Live). Stale pending approvals auto-expire, preventing indefinite accumulation of unreviewed requests. Expired records cannot be approved or executed.

Additional Controls

Prompt injection defense: Both generate_brief and generate_brief_with_web treat context as untrusted data with explicit delimiter boundaries (<internal_context>, <web_search_results>)
Web query safety: Only params.web_query or params.title is sent to Web Search — never internal document content
Raw SQL disabled by default: ALLOW_RAW_SQL=false; role-level data boundaries enforced via Lake Formation (LF-TBAC) in production
Results bucket hardening: PublicAccessBlock + TLS-only + 30-day lifecycle
API throttling: Rate/burst limits against denial-of-wallet
Web Search citation obligation: Source URLs + titles are always included in responses (Acceptable Use Policy compliance)

Web-Augmented Brief Generation (opt-in)

GA at AWS Summit NYC 2026 (June 17, 2026). Powered by AgentCore Web Search Tool.

The Problem

Business briefs based solely on internal documents lack current market context. A sales brief about a product launch needs both the internal product spec and awareness of relevant public announcements published recently. A legal compliance brief needs both the internal policy document and the latest regulatory guidance.

The Solution

A new action generate_brief_with_web combines internal context with real-time web search results. The internal context remains the primary source; web results are supplemental, cited, and treated as untrusted input.

Usage

{
  "action": "generate_brief_with_web",
  "params": {
    "title": "Q3 Data Protection Regulatory Update",
    "context": "Internal operations follow FISC safety standards...",
    "web_query": "data protection regulation 2026 Japan financial services"
  }
}

Response

{
  "status": "completed",
  "action": "generate_brief_with_web",
  "title": "Q3 Data Protection Regulatory Update",
  "brief": "Based on internal FISC compliance documentation... Additionally, [Web: FISC 2026 Revision Summary](https://example.com/fisc) published on 2026-06-10 introduces...",
  "web_citations": [
    {"source": "https://example.com/fisc", "title": "FISC 2026 Revision Summary", "publishedDate": "2026-06-10"}
  ],
  "web_search_enabled": true,
  "guardrail_applied": true
}

Design Properties

Property	Detail
Internal context priority	Internal documents are the primary source; web supplements
Graceful degradation	Web Search failure → behaves like `generate_brief` (internal only)
Citation separation	Internal sources and web sources are visually distinct in the brief
Query safety	Only the `web_query` (or `title`) is sent externally — never internal content
Cross-region	Gateway in us-east-1 (Web Search Tool constraint); adds ~100-200ms
Authorization	Read-only action (same tier as `generate_brief`)
Prompt injection defense	Web results wrapped in `<web_search_results>` as untrusted data

Activation

sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Without these parameters, generate_brief_with_web still works but produces internal-only briefs (graceful degradation).

Verification Findings

Lake Formation + Athena

Athena queries running against Glue tables backed by S3 AP data — the foundation for Quick Sight analytics.

The UC30 CloudFormation stack with all resources (API Gateway, Lambda, DynamoDB ApprovalsTable, Athena WorkGroup) deployed.

The test account had Lake Formation governing the Data Catalog. The Athena Query Lambda's execution role required Lake Formation permission grants (DESCRIBE on DB, SELECT/DESCRIBE on tables) in addition to IAM. Production deployments should design LF-TBAC for role-based data visibility.

Amazon Quick × FSx for ONTAP S3 AP Integration Boundary

Amazon Quick's S3 KB connector accepts the S3 AP alias but authorization fails due to FSx for ONTAP's dual-layer auth — leading to the recommendation below.

Quick provides multiple data integration paths — for FSx for ONTAP data, Bedrock KB (UC29) or Athena-mediated access is the validated route in this repository.

Amazon Quick's S3 knowledge base connector accepts an FSx for ONTAP S3 AP alias as a "valid URL," but connection verification fails with "You do not have permissions to access the S3 bucket." This is not a missing IAM permission that can be simply added — it is a structural authorization path mismatch. Three factors block the standard connector path:

ARN format mismatch — FSx for ONTAP S3 APs use arn:aws:s3:{region}:{account}:accesspoint/{name} (not the arn:aws:s3:::{bucket} format). The connector's internal IAM evaluation likely targets the alias as a bucket name, which does not match IAM policy evaluation for the AP ARN.
AP resource policy rejects the principal — Adding Quick's data access role to the S3 AP resource policy returns MalformedPolicy: Invalid principal in policy, indicating a principal constraint on FSx for ONTAP S3 AP policies.
Layer 2: filesystem identity — Even if IAM layers were resolved, the ONTAP file-system identity (UNIX UID or AD user) associated with the S3 AP must independently permit read access to the target files.

For readers attempting this path: Use CloudTrail to capture the exact AccessDenied event and identify Quick's calling principal ARN. If the MalformedPolicy constraint can be resolved (potentially by configuring the S3 AP with an AD-based identity rather than UNIX root), the direct path may become viable. As of June 2026, Bedrock Knowledge Base → FSx for ONTAP S3 AP is the confirmed working route for RAG ingestion.

Evidence-based implementation guidance:

FSx for ONTAP → RAG: Bedrock KB (UC29) is the validated path in this repository
Quick Index: Stage to a standard S3 bucket for predictable connector behavior
Quick Sight (BI): Athena-mediated access works (QuickSight role needs Athena/Glue/LF/results-bucket permissions)
Direct Quick → FSx for ONTAP S3 AP (standard connector path): validated — does not work (tested with UNIX root identity; MalformedPolicy on AP resource policy). AD-based S3 AP identity configuration is tracked as a future hypothesis

Glue Tables

analytics/<role>/ CSVs are pointed to by Glue tables (sales_pipeline / it_incidents) created via Athena DDL. LOCATION uses S3 AP alias format: s3://<alias>/quick-workspace/analytics/<role>/. For scale, migrate to Parquet + partitioning to reduce Athena scanned costs.

Data Classification

Output	Classification	Rationale
Action API response (generate_brief)	INTERNAL	Source-derived summary; no external disclosure
Action API response (generate_brief_with_web)	INTERNAL	Contains internal citations; web portion is PUBLIC
Action API response (create/approve/execute)	INTERNAL	Business operation records
Athena query results (results bucket)	INTERNAL	Encrypted + 30-day lifecycle + TLS
DynamoDB ApprovalsTable	INTERNAL	Approval state metadata
SNS notifications	INTERNAL	Action summaries only; no file content
Web Search results (raw)	PUBLIC	External public information

Extend shared/data_classification.py for regulated workloads (CUI / FISC / HIPAA).

Cost

Component	Monthly estimate	Notes
Amazon Quick	Per-user/plan billing	Separate; unsubscribe when done
Athena	Scanned-data pricing	Reduce with Parquet
Lambda / API Gateway	Serverless pay-per-use	< $10 for moderate usage
Bedrock LLM (briefs)	Usage-based	Usage-based; verify the current model price
DynamoDB (approvals)	Pay-per-request	Minimal for approval records
AWS Budgets alarm	Free (SNS delivery cost only)	Created when `NotificationEmail` set
AgentCore Web Search (opt-in)	Per-query pricing (see AgentCore pricing)	Gateway invocation pricing
Cross-region transfer (opt-in)	< $0.02	us-east-1 ↔ ap-northeast-1

Teardown / rebuild: one-command idempotent scripts (scripts/teardown-uc29-uc30.sh / scripts/rebuild-uc29-kb.py)

Getting Started

git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/genai/quick-agentic-workspace

# Install dependencies
pip install -r requirements.txt        # or: uv pip install -r requirements.txt

cat samconfig.toml.example  # Review parameters

sam build && sam deploy --guided

# DemoMode=true runs without FSx for ONTAP (regular S3 bucket)

# Optional: Enable Web Search hybrid RAG
sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Governance Note

This article is technical architecture guidance, not legal, compliance, or regulatory advice. Amazon Quick features, pricing, regional availability, and connector behavior are subject to change — verify with official documentation and your own account settings. S3 AP data source boundaries are at volume/prefix granularity. For per-user visibility control, use Quick's document-level ACL or Custom Permission-Aware RAG. Web Search Tool usage requires compliance with the Acceptable Use Policy (source citations must be retained and displayed in end-user output).

Yoshiki Fujiwara

Amazon Bedrock Knowledge Bases + FSx for ONTAP S3 Access Points: Self-Service AI Curation via Windows Drag & Drop — Phase 16

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 21 Jun 2026 15:19:58 +0000

TL;DR

"Put files in this Windows folder and your AI assistant can use them after the next governed sync" — UC29 reduces the handoff friction around enterprise AI knowledge updates. Business users maintain Amazon Bedrock Knowledge Base content through Windows Explorer drag & drop on an FSx for ONTAP SMB share. No S3 console, no ETL, no copy. Data remains in the file system as the operational source; the S3 Access Point provides a governed access path for ingestion without creating a separate object copy.

Three maturity stages + one opt-in enhancement:

Scenario A (manual): User places files → triggers sync from console/CLI
Scenario B (scheduled): EventBridge Scheduler + Step Functions poll every 15 minutes
Scenario C (event-driven): FPolicy detects file placement → real-time KB sync
Hybrid RAG (opt-in): Internal KB answers augmented with real-time web search via AgentCore Web Search Tool (GA June 2026)

Vector store: Amazon S3 Vectors — a managed, cost-aware vector store option for Bedrock Knowledge Bases, with metadata filtering available for permission-aware retrieval designs.

Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns (see solutions/genai/kb-selfservice-curation/ and samconfig.toml.example for deployment parameters)

Why This Pattern?

Challenge	Traditional approach	This pattern
Knowledge updates blocked on IT	Ticket → manual ETL	Business user drags & drops
Dual management (NAS + S3 copy)	Source drifts from S3 replica	S3 AP reads the single source directly
Forgotten re-ingestion	Manual and easy to forget	Automatic (Scenario B/C)
Specialist skills required	ETL / S3 / Bedrock expertise	Familiar Windows folder operations
Answers limited to internal docs	No current external context	Hybrid RAG: internal + web search (opt-in)
Vector store cost	OpenSearch Serverless ~$175/month minimum	S3 Vectors: cost-aware pay-per-use profile

Architecture

Users drag files into an AI-dedicated NTFS volume (SMB share, ACL-separated by department) on FSx for ONTAP. The same volume is exposed via S3 Access Point as a Bedrock Knowledge Base data source:

Users see a familiar Windows folder structure — each department has its own folder with NTFS ACL separation.

Drag & drop a product spec into the sales folder — that is the only content-maintenance step for the user.

Windows Explorer (drag & drop)
  → FSx for ONTAP volume (ai_knowledge/)
  → S3 Access Point (read-only, internet-origin)
  → Bedrock Knowledge Base Data Source (inclusionPrefixes: ai-knowledge/)
  → StartIngestionJob → S3 Vectors (vector store) updated

Seven business roles (sales / marketing / finance / IT / operations / legal / developers) share the volume with NTFS ACL-based permission separation.

Design note: S3 Access Points for FSx for ONTAP let S3-compatible AWS services access file data without copying it to an S3 bucket. Access is authorized twice: first through AWS/IAM policies on the access point path, and then through the file-system identity and permissions associated with the access point.

Hybrid RAG Flow (opt-in)

When EnableWebSearch=true, the Query Lambda augments internal KB answers with real-time web information:

User question
  ├─→ [1] Bedrock KB RetrieveAndGenerate (internal docs via S3 AP → S3 Vectors)
  ├─→ [2] AgentCore Web Search (us-east-1, cross-region MCP call)
  │       Amazon-operated web index (tens of billions of docs, continuously updated)
  └─→ [3] Bedrock Converse → unified answer with dual citations
            [Internal: product-spec.pdf] + [Web: Market Report 2026](https://...)

Scenario B: Scheduled Automation (Step Functions)

The most common production deployment. EventBridge Scheduler triggers a Step Functions workflow every 15 minutes:

EventBridge Scheduler (rate(15 minutes))
  └─→ Step Functions State Machine
       ├─→ DetectAndStartIngestion Lambda
       │     • ListObjectsV2 via S3 AP → compare with last-known state
       │     • If changes detected → StartIngestionJob
       │     • If no changes → skip (cost-zero run)
       ├─→ Wait (30s) → CheckIngestionStatus Lambda (poll loop)
       └─→ NotifySuccess / NotifyFailure → SNS

Key design choices:

Differential detection: Compares current S3 AP listing against prior state; only triggers ingestion when files actually changed
Idempotent: Re-running the same schedule with no file changes is a no-op (no wasted Bedrock ingestion cost)
Observable: Each Step Functions execution is visible in the console with per-state timing and error details
Configurable interval: The ScheduleExpression parameter accepts any EventBridge rate/cron expression

This is the required safety net for Scenario C (see below) — it catches any files missed during the lost-update window.

Scenario C: FPolicy Event-Driven Real-Time Sync

Scenario B's 15-minute polling is replaced by FPolicy real-time detection:

Windows/NFS file operation
  → FPolicy instant detection (CREATE/WRITE/DELETE/RENAME)
  → FPolicy Server → SQS → Bridge Lambda → EventBridge custom bus
  → EventBridge Rule (file_path prefix = ai_knowledge)
  → KB Trigger Lambda (debounce) → StartIngestionJob
  → Bedrock KB → reflected in tens of seconds to minutes

The FPolicy → SQS → EventBridge front-end reuses the existing solutions/event-driven/fpolicy pattern infrastructure. UC29 adds only an EventBridge rule and the KB Trigger Lambda.

Lost-Update Window (Critical)

The EventBridge rule routes only FPolicy events matching the ai_knowledge volume path to the KB Trigger Lambda.

Bedrock Ingestion performs a full-source scan at job start time. Files added during a running job are not included in that execution. Scenario C alone does not guarantee zero missed files.

Mandatory: Always pair Scenario C with Scenario B (periodic reconcile sync) as a safety net. The KB Trigger Lambda skips when a job is already in progress (debounce + ConflictException handling + reserved concurrency = 2).

Namespace Pitfall

FPolicy reports ONTAP volume-path namespace (ai_knowledge/..., underscore). The KB S3 ingestion prefix (ai-knowledge/, hyphen) is a different namespace. Initial implementation confused the two, causing false-skip. The EventBridge rule and Lambda secondary filter now use a dedicated FPOLICY_PATH_FILTER parameter for the volume-path namespace.

Hybrid RAG: Internal KB + Web Search (opt-in)

GA at AWS Summit NYC 2026 (June 17, 2026). Powered by AgentCore Web Search Tool.

Enterprise knowledge from FSx for ONTAP is treated as the primary internal source, while public web context is supplemental and untrusted. For questions that benefit from current external context — regulatory updates, market trends, public market information — the Query Lambda can optionally augment answers with real-time web search results.

How It Works

Internal KB retrieval (always): Bedrock KB searches S3 Vectors for relevant chunks from FSx for ONTAP documents
Web search (opt-in): AgentCore Gateway invokes Amazon's purpose-built web index via MCP protocol
Unified answer: Bedrock Converse merges both contexts, with internal documents as primary source

Key Design Decisions

Decision	Rationale
Opt-in (`EnableWebSearch=false` default)	Most enterprise QA needs internal data only
Graceful degradation	Web Search failure → internal-only answer (no error surfaced to user)
Cross-region (us-east-1 Gateway)	Web Search Tool is us-east-1 only; adds ~100-200ms latency
Query safety	Only user's question text is sent to Web Search — never internal document content
Citation separation	`[Internal: filename]` vs `[Web: title](URL)` — users see exactly which source informed each claim
Prompt injection defense	Web results wrapped in `<web_search_results>` with explicit "untrusted data" instruction
Acceptable Use compliance	Source URLs and titles are always displayed (Web Search Tool TOS requirement)

Deployment

sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Example Response

{
  "status": "completed",
  "query": "What are the latest FISC guidelines for cloud data protection?",
  "answer": "Based on internal documentation, our current FISC compliance posture covers... Additionally, [Web: FISC 2026 Guidelines Update](https://example.com/fisc-2026) published last month introduces...",
  "citations": [{"source": "s3://.../legal/compliance/fisc-overview.pdf"}],
  "web_citations": [{"source": "https://example.com/fisc-2026", "title": "FISC 2026 Guidelines Update", "type": "web"}],
  "web_search_enabled": true
}

Verification Highlights

Windows-Identity S3 Access Point with Dedicated AD

The Windows EC2 domain-joined to AWS Managed Microsoft AD, with the FSx SMB share mapped — proving the literal drag & drop experience works end to end.

To demonstrate the literal Windows drag & drop experience, we built a dedicated AWS Managed Microsoft AD + domain-joined Windows EC2 + AD-joined SVM:

AD-joined SVM OU: AWS Managed AD's OU=Computers lacks delegation rights → use the domain-name OU (OU=<domain>,DC=...)
CIFS share creation: Executes against the filesystem management LIF, not the SVM LIF
Windows-identity S3 AP: Works correctly with a running dedicated AD; files dropped in Explorer are readable via S3 AP

Deletion Lifecycle

The Bedrock KB data source connected to the FSx for ONTAP S3 AP alias. Click "Sync" for manual ingestion, or let Scenario B/C automate it.

Scenario B's Step Functions workflow: detect changes → start ingestion → poll status → notify on completion.

"User deletes a file → AI forgets it" verified end-to-end: file deletion → next sync → numberOfDocumentsDeleted=1 → re-query returns "no information found". Powered by dataDeletionPolicy=DELETE. For urgent revocation between syncs, call the Ingestion API directly.

Performance Considerations

Shared bandwidth: S3 AP reads share the FSx throughput capacity (128/256/512 MBps) with NFS/SMB workloads. Scenario B's 15-minute interval and Scenario C's reserved concurrency (2) throttle ingestion flow
Bulk re-index: For full re-ingestion (e.g., embedding model change), use a FlexClone volume as the Ingestion target — zero impact on production I/O, consistent point-in-time read
Tiering: Frequently accessed AI knowledge should remain on the SSD tier. Capacity Pool retrieval latency affects GetObject time during ingestion
Web Search latency: Cross-region call to us-east-1 adds ~100-200ms. Total hybrid query latency depends on KB size, model, and network conditions (KB retrieve + Web Search + Converse generation)

Access Control — Three Layers

S3 AP boundaries are volume/prefix-level. For per-user visibility:

Search narrowing = Bedrock KB metadata filters (this UC; not AWS authorization)
Document-level ACL = Amazon Quick S3 Knowledge Base (UC30; user/group-level)
Chunk-level permission filter = Custom Permission-Aware RAG (FC3; AD SID/NTFS ACL for regulated industries)

Web Search results are public information — no ACL filtering needed. However, the unified answer that combines internal + web sources is subject to the same access control as internal-only answers (the internal citations remain permission-scoped).

Vector Store: Why S3 Vectors

This pattern uses Amazon S3 Vectors as the Bedrock KB vector store. OpenSearch Serverless remains a valid option when its operational and latency profile fits the workload better.

Criterion	OpenSearch Serverless	S3 Vectors
Minimum monthly cost	~$175 (2 OCU)	Pay-per-use only
Cost at scale	OCU-based	Cost savings for large vector datasets (see AWS documentation)
Metadata filtering	Supported	Supported (department, owner, role)
Permission-Aware RAG compatibility	Supported	Compatible with metadata-filtered retrieval designs; authorization enforced by application layer
Infrastructure management	Managed but OCU scaling required	Managed vector operations
Scale	Millions of vectors	2 billion vectors per index
Query latency	Sub-100ms	Sub-100ms

For this project — 28 industry patterns + PoC-to-production lifecycle — S3 Vectors' pay-per-use model is the right fit. We evaluated Bedrock Managed Knowledge Base (GA June 2026, AWS Summit NYC) but chose Custom KB + S3 Vectors for cost control, ACL metadata flexibility, and FSx for ONTAP lifecycle integration (see ADR: docs/investigations/managed-kb-vs-custom-kb-s3vectors.md).

Data Classification

Output	Classification	Rationale
KB vectors + metadata	INTERNAL	Inherits source file classification
Ingestion job status / SNS	INTERNAL	Operational metadata only
CloudWatch Metrics / Logs	INTERNAL	Aggregate metrics, no file content
Web Search results	PUBLIC	External public information
Hybrid answer (internal + web)	INTERNAL	Contains internal document citations

For regulated workloads (CUI / FISC / HIPAA), extend shared/data_classification.py labels. If retention-period requirements apply, use dataDeletionPolicy=RETAIN and design a separate purge procedure.

Cost

Component	Monthly estimate	Notes
Lambda (sync + query)	< $5	Serverless pay-per-use
S3 API (ListObjects, GetObject)	< $1	S3 AP reads
EventBridge Scheduler	< $1	15-min interval
Bedrock KB Ingestion	Usage-based	Per-document embedding
S3 Vectors	Usage-based	Compare with OpenSearch Serverless for your query volume, latency, and operations requirements
Bedrock LLM (query)	Usage-based	Nova Pro: $0.0008/1K input tokens
FPolicy Server (Scenario C)	~$35	ECS Fargate (set desiredCount=0 when idle)
AgentCore Web Search (opt-in)	Per-query pricing (see AgentCore pricing)	Gateway invocation pricing
Cross-region transfer (opt-in)	< $0.02	us-east-1 ↔ ap-northeast-1

Getting Started

git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/genai/kb-selfservice-curation

# Install dependencies (shared modules used by Lambda handlers)
pip install -r requirements.txt        # or: uv pip install -r requirements.txt

# Review parameters
cat samconfig.toml.example

# Build and deploy (requires configured AWS credentials + FSx for ONTAP S3 AP)
sam build && sam deploy --guided

# DemoMode=true runs without FSx for ONTAP (regular S3 bucket)

# Optional: Enable Web Search hybrid RAG
sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Governance Note

This article is technical architecture guidance, not legal, compliance, or regulatory advice. Pricing, regional availability, and benchmark numbers are time-sensitive; verify them against current AWS documentation before production use. S3 AP data source boundaries are at volume/prefix granularity — for per-user visibility control, consider Custom Permission-Aware RAG. If retention-period requirements (NARA / FISC) apply, use dataDeletionPolicy=RETAIN and design purge procedures separately. Web Search Tool usage requires compliance with the Acceptable Use Policy (source citations must be displayed).

Yoshiki Fujiwara

From Logs to Detection: Building a File Security Pipeline in Datadog for FSx for ONTAP

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 14 Jun 2026 16:44:19 +0000

TL;DR

For security teams: Four threshold monitors + one ML anomaly monitor + four Cloud SIEM detection rules catch mass file deletion, data exfiltration, permission tampering, and unusual geography in under 5 minutes. EventID codes become human-readable operation names. All created via Datadog API — no manual clicking required.

For engineers: The Log Pipeline (6 processors including GeoIP enrichment) transforms raw ONTAP XML events into searchable, alertable fields. Five Saved Views cover the most common investigation patterns. Total AWS cost remains ~$1.50/month.

For architects: This pattern demonstrates the full detection-to-response lifecycle — from raw file system audit events through enrichment, categorization, alerting, cross-service correlation, and automated remediation — entirely serverless, entirely as code.

FSx for ONTAP → S3 Access Point → Lambda → Datadog Logs API v2
                                              │
                                              ▼
                                   ┌──────────────────────┐
                                   │ Log Pipeline (6)     │
                                   │  • Category Processor│
                                   │  • Status Remapper   │
                                   │  • Date Remapper     │
                                   │  • Attribute Remapper│
                                   │  • GeoIP Enrichment  │
                                   └──────────┬───────────┘
                                              │
                          ┌───────────────────┼───────────────────┐
                          ▼                   ▼                   ▼
                ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
                │ Log Monitors (4)│ │ Cloud SIEM (4)  │ │ Log Archive     │
                │ + Anomaly (1)   │ │ Security Signals│ │ → S3 → Glacier  │
                └────────┬────────┘ └────────┬────────┘ └─────────────────┘
                         │                   │
                         ▼                   ▼
                ┌──────────────────────────────────────┐
                │ Workflow → Slack + Case + Lambda     │
                │          (ONTAP Snapshot remediation)│
                └──────────────────────────────────────┘

Reading guide: Sections 1-9 cover the core pipeline (30 min to deploy). Sections 10+ cover advanced SOC integration (additional 15 min). Jump to Deployment if you want to start immediately.

Business value: This pipeline reduces mean-time-to-detect (MTTD) for file-based threats from "hours/days" (batch log review) to under 5 minutes (real-time alerting). For regulated environments, it provides continuous compliance evidence with automated archival — replacing manual quarterly reviews with always-on monitoring.

This is Part 16 of Serverless Observability for FSx for ONTAP.

Why Post-Ingestion Processing Matters

In Part 1, we shipped raw audit events to Datadog. That's necessary but insufficient. Raw events look like this:

{
  "event_type": "4660",
  "user": "CORP\\contractor-ext-03",
  "path": "/share/engineering/designs/prototype-v3.dwg",
  "result": "Audit Failure"
}

Questions that raw logs can't answer quickly:

What is EventID 4660? (Answer: Object Delete)
Should this alarm? (Answer: depends on volume and context)
Who should investigate? (Answer: Storage team + SOC)
What's the user's baseline? (Answer: need faceted historical view)

This article builds the processing layers that transform raw data into actionable security intelligence.

Architecture

Why EventBridge Scheduler? FSx for ONTAP S3 Access Points do not support S3 Event Notifications or EventBridge object-level events. Lambda polls on a schedule and uses checkpointing to process only new files. XML format is chosen for Lambda-native parsing without binary dependencies (vs EVTX which requires Windows-specific libraries).

┌──────────────────────────────────────────────────────────────────┐
│ Lambda Handler (Python 3.12)                                     │
│  • Parse XML → Normalize → HEC-compatible JSON                   │
│  • POST to Datadog Logs API v2                                   │
│  • Fields: event_type, user, path, client_ip, svm, result        │
└──────────────────────────────────────────────────────────────────┘
        │  HTTP 202
        ▼
┌──────────────────────────────────────────────────────────────────┐
│ Datadog Log Pipeline: "FSx for ONTAP Audit Logs"                 │
│ Filter: source:fsxn                                              │
│                                                                  │
│ 1. Category Processor → @operation_name                          │
│    4663→Object Access, 4660→Object Delete, 4656→Handle Request   │
│                                                                  │
│ 2. Status Remapper → log severity from @result                   │
│    "Audit Success" → info, "Audit Failure" → error               │
│                                                                  │
│ 3. Date Remapper → @timestamp as official log time               │
│                                                                  │
│ 4. Attribute Remapper → @user→usr.id, @client_ip→network.client  │
└──────────────────────────────────────────────────────────────────┘
        │
        ▼
┌──────────────────────────────────────────────────────────────────┐
│ Datadog Security Monitors (3)                                    │
│                                                                  │
│ • [FSxN] Mass File Deletion    → >50 deletes/5min per user       │
│ • [FSxN] Abnormal Access Volume → >1000 accesses/1h per user     │
│ • [FSxN] Access Failure Spike  → >10 failures/15min per user     │
└──────────────────────────────────────────────────────────────────┘

The Log Pipeline

Why a Pipeline?

Without a pipeline, every query requires remembering that 4660 means "delete" and 4663 means "access." With a pipeline, you search @operation_name:Object Delete — and Datadog handles the translation at ingest time.

Creating the Pipeline via API

import json, urllib3, boto3

sm = boto3.client('secretsmanager', region_name='ap-northeast-1')
api_key = sm.get_secret_value(SecretId='fsxn-datadog-api-key')['SecretString']
app_key = sm.get_secret_value(SecretId='datadog/fsxn-app-key')['SecretString']

http = urllib3.PoolManager()

pipeline = {
    "name": "FSx for ONTAP Audit Logs",
    "is_enabled": True,
    "filter": {"query": "source:fsxn"},
    "processors": [
        {
            "type": "category-processor",
            "name": "EventID to Operation Name",
            "is_enabled": True,
            "target": "operation_name",
            "categories": [
                {"filter": {"query": "@event_type:4663"}, "name": "Object Access"},
                {"filter": {"query": "@event_type:4656"}, "name": "Handle Request"},
                {"filter": {"query": "@event_type:4660"}, "name": "Object Delete"},
                {"filter": {"query": "@event_type:4670"}, "name": "Permission Change"},
                {"filter": {"query": "@event_type:4658"}, "name": "Handle Close"},
                {"filter": {"query": "@event_type:5140"}, "name": "Share Access"},
                {"filter": {"query": "@event_type:5145"}, "name": "Share Check"},
                {"filter": {"query": "@event_type:4624"}, "name": "Logon"},
                {"filter": {"query": "@event_type:4634"}, "name": "Logoff"},
            ]
        },
        {
            "type": "status-remapper",
            "name": "Map result to log status",
            "is_enabled": True,
            "sources": ["result"]
        },
        {
            "type": "date-remapper",
            "name": "Use event timestamp",
            "is_enabled": True,
            "sources": ["timestamp"]
        },
        {
            "type": "attribute-remapper",
            "name": "Map user to usr.id",
            "is_enabled": True,
            "sources": ["user"],
            "source_type": "attribute",
            "target": "usr.id",
            "target_type": "attribute",
            "preserve_source": True,
            "override_on_conflict": False
        },
        {
            "type": "attribute-remapper",
            "name": "Map client_ip to network.client.ip",
            "is_enabled": True,
            "sources": ["client_ip"],
            "source_type": "attribute",
            "target": "network.client.ip",
            "target_type": "attribute",
            "preserve_source": True,
            "override_on_conflict": False
        }
    ]
}

resp = http.request("POST",
    "https://api.ap1.datadoghq.com/api/v1/logs/config/pipelines",
    body=json.dumps(pipeline).encode("utf-8"),
    headers={
        "Content-Type": "application/json",
        "DD-API-KEY": api_key,
        "DD-APPLICATION-KEY": app_key
    })
print(f"HTTP {resp.status}: Pipeline ID = {json.loads(resp.data)['id']}")

Site note: Replace api.ap1.datadoghq.com with your Datadog site's API endpoint (e.g., api.datadoghq.com for US1, api.datadoghq.eu for EU1).

EventID Mapping Table

EventID	Operation Name	MITRE ATT&CK	Description
4663	Object Access	T1005 (Data from Local System)	File read/write operation
4656	Handle Request	T1005	File handle opened
4660	Object Delete	T1485 (Data Destruction)	File deleted
4670	Permission Change	T1222 (File Permissions Modification)	ACL/permission modified
4658	Handle Close	—	File handle closed (low signal)
5140	Share Access	T1021.002 (SMB/Windows Admin)	SMB share connected
5145	Share Check	T1135 (Network Share Discovery)	Share permission checked
4624	Logon	T1078 (Valid Accounts)	Authentication event
4634	Logoff	—	Session ended

Security Monitors

Monitor 1: Mass File Deletion

mass_delete = {
    "name": "[FSxN] Mass File Deletion Detected",
    "type": "log alert",
    "query": 'logs("source:fsxn @event_type:4660").index("*").rollup("count").by("@user").last("5m") > 50',
    "message": """## Mass File Deletion Alert

A user has triggered more than 50 file deletion events within 5 minutes.

**User**: {{@user}}  |  **Count**: {{value}}

### Investigation Steps
1. Check affected paths: `source:fsxn @event_type:4660 @user:{{@user}}`
2. Verify if this is a scheduled cleanup or authorized bulk operation
3. Check the client IP for unexpected sources
4. Correlate with user's normal deletion patterns

@slack-storage-alerts""",
    "tags": ["source:fsxn", "team:storage", "severity:high"],
    "options": {
        "thresholds": {"critical": 50, "warning": 20},
        "notify_no_data": False,
        "renotify_interval": 60,
        "evaluation_delay": 60
    }
}

Why 50? In our test environment, normal daily deletions per user stay under 10. The threshold should be tuned based on your environment's baseline — run source:fsxn @event_type:4660 | stats count by @user over a week to establish what's normal.

Monitor 2: Abnormal Access Volume

abnormal_access = {
    "name": "[FSxN] Abnormal Access Volume",
    "type": "log alert",
    "query": 'logs("source:fsxn @result:\\"Audit Success\\"").index("*").rollup("count").by("@user").last("1h") > 1000',
    "message": """## Abnormal Access Volume Alert

More than 1000 successful file access events in 1 hour.
May indicate data exfiltration or unauthorized bulk access.

**User**: {{@user}}  |  **Count**: {{value}}

### Investigation Steps
1. Review patterns: `source:fsxn @user:{{@user}}`
2. Check if backup jobs or batch operations are running
3. Verify client IP and compare with known locations
4. Check accessed paths for sensitive content

@slack-security-alerts""",
    "options": {
        "thresholds": {"critical": 1000, "warning": 500},
        "evaluation_delay": 60
    }
}

Tuning note: Backup service accounts (svc-backup, svc-indexer) generate legitimate high-volume access. Exclude them with @user:-svc-* in the query or create a suppression rule.

Monitor 3: Access Failure Spike

failure_spike = {
    "name": "[FSxN] Access Failure Spike",
    "type": "log alert",
    "query": 'logs("source:fsxn @result:\\"Audit Failure\\"").index("*").rollup("count").by("@user").last("15m") > 10',
    "message": """## Access Failure Spike

More than 10 access failures in 15 minutes.
May indicate unauthorized access attempts or permission misconfiguration.

**User**: {{@user}}  |  **Count**: {{value}}

### Investigation Steps
1. Check failed paths: `source:fsxn @result:"Audit Failure" @user:{{@user}}`
2. Verify if permissions were recently changed
3. Check if user is accessing resources outside their scope
4. Correlate with AD group membership changes

@slack-security-alerts""",
    "options": {
        "thresholds": {"critical": 10, "warning": 5},
        "evaluation_delay": 60
    }
}

Saved Views

Five pre-configured views for common investigation patterns:

View	Query	When to use
FSxN File Deletions	`source:fsxn @event_type:4660`	Monitor triggered, investigating which files
FSxN Access Failures	`source:fsxn @result:"Audit Failure"`	Permission denied investigation
FSxN All Events	`source:fsxn`	General audit stream
FSxN Sensitive Share Access	`source:fsxn (@path:finance OR @path:hr OR @path:legal)`	Sensitive data access review
FSxN After-Hours Access	`source:fsxn` (filtered by time)	Off-hours activity detection

Each view includes pre-configured columns (user, path, client_ip, event_type) so analysts don't need to customize the table every time.

Facets for One-Click Filtering

Facets add clickable filters to the left sidebar. Create them from any log entry's detail panel:

Facet	Field	Why
Event Type	`@event_type`	Filter to specific operations (4660=delete, 4663=access)
Operation	`@operation_name`	Human-readable version (after pipeline applies)
User	`@user`	Filter by specific user under investigation
SVM	`@svm`	Isolate events to specific storage virtual machines
File Path	`@path`	Filter by directory or share
Client IP	`@client_ip`	Filter by source workstation/server
Result	`@result`	Quick split between success and failure events

Note: Fields are searchable with @field:value syntax even without Facets. Facets add UI convenience — they're not required for alerting or saved views.

Cost Reality

Component	Monthly Cost	Notes
Lambda (5-min schedule, ~1s execution)	~$0.50	8,640 invocations/month
Secrets Manager (2 secrets)	~$0.80	API key + APP key
EventBridge Scheduler	~$0.00	Free tier covers this
S3 AP reads	~$0.05	Depends on file count
Datadog Forwarder Lambda	~$0.10	CloudTrail event processing
S3 Archive storage	~$0.02	Glacier tier after 30 days
AWS total	~$1.50/month
Datadog log ingestion	~$0.10/GB	Conventional pricing
Total (1GB/month)	~$1.60/month

Datadog pricing caveat: Log ingestion pricing varies by plan (Conventional vs On-Demand) and contract. The ~$0.10/GB is the list price for conventional plans; actual cost depends on your contract, committed volume, and retention choices.

Cloud SIEM pricing: Cloud SIEM is a separate paid feature (~$0.20/GB analyzed for log detection). A 30-day free trial is available. Security Signals, Detection Rules, and the triage workflow require Cloud SIEM. Log Monitors (threshold alerts to Slack) work without it.

Cost optimization patterns

Filter before shipping — Drop low-value EventIDs (4658 Handle Close) at the Lambda level to reduce ingestion volume
Compress payloads — Enable gzip on the HTTP POST to reduce egress
Tune audit scope — Configure ONTAP vserver audit to monitor only the shares that matter
Index retention — Use Datadog's flexible retention (15/30/60/90 days) to match compliance needs vs cost

E2E Verification Results

Verified on Datadog AP1 (ap1.datadoghq.com) with paid plan, June 2026:

Component	Status
Lambda → Datadog Logs API v2	✅ HTTP 202
Log Pipeline (5 processors)	✅ Applied automatically
Category Processor (9 EventIDs)	✅ `@operation_name` populated
Status Remapper	✅ Failure events marked as error
Date Remapper	✅ Event timestamp used (not ingest time)
Attribute Remapper (usr.id)	✅ Enables Datadog user features
Mass Delete Monitor	✅ ALERT triggered (55 events > threshold 50)
Access Failure Monitor	✅ ALERT triggered (12 events > threshold 10)
Abnormal Access Monitor	✅ Active (OK — high threshold by design)
Anomaly Monitor (ML)	✅ Active (learning baseline)
Saved Views (5)	✅ Accessible from Views dropdown
Facets (8 custom)	✅ Created via UI
Dashboard (10 widgets)	✅ FSx ONTAP Audit Log Overview
Log-based Metrics (4)	✅ `fsxn.audit.*` in Metrics Explorer
Sensitive Data Scanner (5 rules)	✅ PII detection active
Cloud SIEM Detection Rules (4)	✅ Security Signals generated
CloudTrail Forwarder	✅ Logs arriving in Datadog
GeoIP Enrichment	✅ Country/city populated on client_ip
GuardDuty Correlation Rule	✅ Cross-service detection active
Workflow Automation	✅ Monitor → Slack + Case
Case Management (FSXN project)	✅ Project + template case
SOC Triage Runbook	✅ 7-step Notebook
Log Archive (S3 → Glacier)	✅ source:fsxn archived
Snapshot Remediation Lambda	✅ Deployed (fsxn-snapshot-remediation)

Log-based Metrics

Log-based metrics extract numerical values from logs at ingest time — you get metric-resolution dashboards and anomaly detection without paying for log retention beyond the evaluation window.

# Created via Datadog API (no UI needed)
metrics = [
    "fsxn.audit.delete_count",         # Delete events grouped by user/SVM
    "fsxn.audit.access_failure_count",  # Failures by user/SVM/client_ip
    "fsxn.audit.event_count",           # All events by event_type/SVM
    "fsxn.audit.unique_users",          # Active users
]

Use cases:

Anomaly detection on fsxn.audit.delete_count — Datadog's built-in anomaly monitor catches unusual deletion patterns without manual thresholds
SLO tracking — Define an SLO on access failure rate (fsxn.audit.access_failure_count / fsxn.audit.event_count)
Cost efficiency — Metrics are retained for 15 months at metric resolution (vs log retention at full-text cost)

Sensitive Data Scanner

Audit logs frequently contain PII in file paths (/hr/EMP-123456-performance-review.xlsx) and user contexts. Datadog's Sensitive Data Scanner catches these before they reach analysts.

Rule	What It Catches	Why It Matters
Employee ID (`EMP-\d{6}`)	Employee records in paths	PII exposure prevention
JP Phone (`0[789]0-\d{4}-\d{4}`)	Mobile numbers in filenames	Privacy protection
Email in Path	Customer emails in file names	Data minimization
Credit Card	CC numbers in documents	PCI-DSS technical control
My Number	Japanese national ID	Sensitive PII detection

Compliance note: Sensitive Data Scanner is a technical detection and redaction control. It does not constitute full APPI/GDPR/PCI-DSS compliance, which requires organizational measures (consent management, purpose limitation, data subject rights, etc.). Consult your legal/compliance team for regulatory assessment.

Matched patterns are partially redacted at ingest — the first 3 characters remain for identification, the rest is replaced with [REDACTED]. This preserves investigation capability while protecting the data subject.

Enhanced Dashboard

The SOC-focused dashboard provides at-a-glance visibility into:

Who is generating the most activity (top users, top IPs)
What operations dominate (sunburst by operation/SVM)
Where access is concentrated (hot paths)
When failures spike (timeline with per-user breakdown)
Trend from log-based metrics (delete rate over time)

Cloud SIEM Security Signals

Beyond log monitors (which notify via Slack/PagerDuty), Security Signals integrate with Datadog's SOC workflow — triage, investigation, and response all in one place.

Log Monitors vs Cloud SIEM Detection Rules

Aspect	Log Monitors	Cloud SIEM Detection Rules
Output	Alert notification (Slack/PagerDuty/email)	Security Signal (appears in Security Signals panel)
Triage	Manual (click alert → investigate)	Built-in triage workflow (Open → In Progress → Archived)
MITRE mapping	Manual tags	Native MITRE ATT&CK framework integration
Correlation	Single query	Cross-log-source correlation possible
Case integration	None	Direct "Create Case" from signal
Best for	Operational alerts (DevOps/SRE)	Security investigation (SOC/IR teams)

Recommendation: Use both. Log Monitors for immediate Slack notification to storage team. Cloud SIEM Detection Rules for SOC triage workflow and MITRE-mapped investigation.

Three detection rules generate Security Signals when FSxN audit logs match threat patterns:

Rule	MITRE ATT&CK	Trigger	Severity
Mass File Deletion	T1485 Data Destruction	>50 deletes/5min per user	Critical
Brute Force File Access	T1110 Brute Force	>20 failures/15min per user+IP	High
Permission Tampering	T1222 File Permissions Modification	>5 changes/10min per user	High

# Creating a Security Signal rule via API
rule = {
    "name": "FSxN: Mass File Deletion",
    "type": "log_detection",
    "queries": [{
        "query": "source:fsxn @event_type:4660",
        "groupByFields": ["@user"],
        "aggregation": "count",
        "name": "delete_events",
        "dataSource": "logs"
    }],
    "cases": [
        {"condition": "delete_events > 50", "status": "critical", "name": "Critical"},
        {"condition": "delete_events > 20", "status": "high", "name": "High"}
    ],
    "options": {
        "evaluationWindow": 300,
        "keepAlive": 3600,
        "maxSignalDuration": 86400,
        "detectionMethod": "threshold"
    },
    "message": "User {{@user}} deleted {{value}} files in 5 min. T1485.",
    "tags": ["source:fsxn", "technique:T1485-data-destruction"]
}

Each signal includes investigation steps and response guidance directly in the signal panel — analysts don't need to leave the Security Signals view to understand what happened.

Cloud SIEM Setup Steps

Navigate to Security → Cloud SIEM → Get Started
Skip Content Packs (FSxN is a custom source)
Select fsxn as a log source → Enable as Trial
Configure Cloud SIEM Index (default: 450 days retention)
Detection rules are automatically applied to incoming source:fsxn logs

Workflow Automation

When a critical monitor fires, the Workflow automatically triggers response actions:

Monitor Alert → @workflow-fsxn-security-alert-response → Slack notification + Investigation links

The workflow (fsxn-security-alert-response) is linked to monitors via the @workflow-<handle> mention syntax in the monitor message. No additional configuration needed — just add the mention to any monitor's notification body.

Extensions (via Datadog Workflow builder):

Auto-create Jira ticket with investigation context
Query Active Directory for user's manager
Invoke fsxn-snapshot-remediation Lambda for evidence preservation (deployed — see Automated Snapshot Remediation)

Investigation Notebook

A pre-built 5-step investigation template guides analysts through alert triage:

Step	Content	Widget
1. Identify Scope	Event type distribution timeline	Timeseries (bars)
2. User Timeline	Delete events by user over time	Timeseries (line)
3. Affected Files	Top 20 deleted paths	Top List
4. Client IP Analysis	Top 10 source IPs	Top List
5. Conclusion	Root cause, impact, action template	Markdown

Access at: Notebooks → "FSxN Audit Log Investigation Template"

Each step includes pre-configured queries — analysts just adjust the time window and user filter to match the alert context.

RBAC and Access Control

The FSxN Security Analyst role provides scoped access to audit logs:

Role	Permissions	Use Case
FSxN Security Analyst	`logs_read_data`, `logs_read_index_data`	SOC analysts investigating file access
Datadog Standard Role	Full access	Storage admins managing pipeline

Assign users to this role to grant log access without exposing infrastructure metrics or APM data. Combined with Saved Views, analysts see only the investigation tools relevant to their role.

OTel Collector Bridge

For teams already running OpenTelemetry Collector, an alternative delivery path routes FSxN logs through OTel with trace context injection:

# otel-bridge/collector-config.yaml (excerpt)
receivers:
  otlp:
    protocols:
      http: { endpoint: 0.0.0.0:4318 }

processors:
  resource:
    attributes:
      - key: service.name
        value: ontap-audit
        action: upsert
  transform:
    log_statements:
      - context: log
        statements:
          - set(attributes["ddsource"], "fsxn")

exporters:
  datadog:
    api:
      key: ${env:DD_API_KEY}
      site: ${env:DD_SITE:-ap1.datadoghq.com}

Benefits over direct Lambda→Datadog:

trace_id injection for distributed tracing correlation
Multi-backend fanout (Datadog + Grafana + S3 in parallel)
Attribute enrichment at collector level (team, environment tags)
Sampling for high-volume environments
Edge-side PII redaction — Mask sensitive fields before they leave your AWS account (stronger than relying solely on Datadog's Sensitive Data Scanner)

Edge redaction pattern: Use OTel's transform processor to hash user and client_ip and truncate path to directory level BEFORE export to Datadog. This ensures PII never crosses your network boundary — even if Datadog's SDS configuration is misconfigured or disabled, your data is already protected at the source.

Status: Config syntax verified. Integration testing requires deploying OTel Collector on ECS (see Part 7 for the ECS-based Collector pattern used with Grafana and Honeycomb).

Log Archives and Compliance

For regulatory retention (adjust based on your organization's audit policy), a dedicated CloudFormation template deploys an S3 archive with Glacier lifecycle:

aws cloudformation deploy \
  --template-file integrations/datadog/template-log-archive.yaml \
  --stack-name fsxn-datadog-archive \
  --parameter-overrides \
    DatadogExternalId=<from-datadog-integration-page> \
    RetentionDays=30 \
    GlacierRetentionDays=2555 \
  --capabilities CAPABILITY_NAMED_IAM

This separates concerns:

Detection — Hot logs in Datadog (15-30 day index retention)
Compliance — Cold archive in S3 → Glacier (7+ years)
Investigation — Rehydrate specific time ranges on demand

Anomaly Detection (ML-based)

Beyond static thresholds, the anomaly monitor on fsxn.audit.delete_count uses Datadog's agile algorithm to learn each user's baseline:

# Anomaly monitor — no manual threshold needed
"avg(last_4h):anomalies(
    sum:fsxn.audit.delete_count{*} by {user}.as_count(),
    'agile', 3, direction='above'
) >= 1"

When a user who normally deletes 5 files/day suddenly deletes 200, the alert fires — even though 200 is well below the static "50 in 5 minutes" threshold. This catches slow, sustained exfiltration that threshold-based monitors miss.

Baseline period: The anomaly algorithm needs ~2 weeks of data to build confidence. Deploy early — it stays silent during the learning period.

Cardinality Management

⚠️ Log-based metrics with group_by: user create one time series per unique user. For organizations with 10,000+ users:

Metric	Recommended group_by	Reason
`fsxn.audit.event_count`	svm only	Broad metric, low cardinality
`fsxn.audit.delete_count`	user, svm	Targeted, high signal
`fsxn.audit.access_failure_count`	user, svm, client_ip	Investigation-focused
`fsxn.audit.unique_users`	user	Intentionally per-user

Monitor cardinality in Metrics Summary (fsxn.audit.*). Datadog bills per unique tag-value combination — keep per-user grouping only on metrics where individual user behavior matters.

What's Different from Splunk/CrowdStrike?

Aspect	Datadog	Splunk (Part 8)	CrowdStrike (Part 15)
Delivery protocol	Logs API v2 (JSON)	HEC (JSON)	HEC (JSON)
Pipeline config	API-driven (Python)	props.conf / UI	LogScale parser YAML
Monitor creation	API (`POST /api/v1/monitor`)	Saved searches + alerts	CQL + alert actions
EventID mapping	Category Processor	eval/lookup	LogScale parser
Identity correlation	`usr.id` attribute	`user` field	Falcon Identity
Free tier	❌ (paid only for logs)	❌	❌
API management	✅ Full (Pipeline + Monitor + Dashboard)	Partial	Limited

The key Datadog advantage: everything is API-manageable. Pipeline, monitors, dashboards, and saved views can all be created, updated, and version-controlled through the Datadog API — making infrastructure-as-code for observability fully achievable.

Production Recommendations

Use the API, not the UI — Version-control your Pipeline, Monitor, and Dashboard definitions in a setup script. When you need to adjust thresholds or add EventIDs, it's a code change with review.
Separate APP key from API key — The API key ships logs; the APP key manages configuration. Store both in Secrets Manager with different IAM policies.
Exclude service accounts — Add @user:-svc-* to monitor queries or create Datadog suppression rules for known batch accounts.
Start with Warning, promote to Critical — Deploy monitors with Warning thresholds first, observe for 1-2 weeks, then tighten to Critical after confirming the baseline.
Connect to incident workflow — Route Critical monitors to PagerDuty/Slack and include investigation links (Saved View URLs) in the alert message.
Enable Log Archives for compliance — Configure S3 archive with Glacier lifecycle for FISC/SOC2 retention requirements. Rehydrate on demand for historical investigation.
Use Workflow Automation for response — Configure Datadog Workflows to automatically create Jira tickets, trigger Slack quick-actions, or invoke Lambda remediation when critical monitors fire.
Manage with Terraform for production — For enterprise/multi-org deployments, use the Datadog Terraform Provider to version-control Pipelines, Monitors, and Detection Rules. Separate API key (logs_write only) from APP key (admin-level, CI/CD pipeline only).
Apply IAM Permissions Boundary — In large organizations, apply a Permissions Boundary to the log-shipping Lambda role to prevent privilege escalation. The boundary should allow only s3:GetObject, s3:ListBucket, secretsmanager:GetSecretValue, and logs:* (CloudWatch).
Encrypt with customer-managed keys (CMK) — For regulated environments, enable Datadog Log Management Encryption with your KMS CMK, and configure the S3 archive bucket with SSE-KMS. This ensures audit logs are encrypted with keys you control at every stage.
Tune detection rules (anti-alert-fatigue) — Run all new Cloud SIEM rules in "Warning / dev" status for 2 weeks. Whitelist known service accounts with @usr.id:-svc-* in queries. Review false positives weekly and adjust thresholds or add suppression rules before promoting to Critical.
Data Integrity: Handle HTTP 429/5xx — Datadog Logs API v2 returns HTTP 202 (accepted, not indexed). On HTTP 429 (rate limit) or 5xx (transient), implement exponential backoff (base 1s, max 5 retries). After final failure, route the complete HEC payload to SQS DLQ for replay. Never drop logs silently.

Deployment

# 1. Deploy the CloudFormation stack (Lambda + Scheduler + DLQ + Alarms)
aws cloudformation deploy \
  --template-file integrations/datadog/template.yaml \
  --stack-name fsxn-datadog-integration \
  --parameter-overrides \
    FsxS3AccessPointArn=arn:aws:s3:<region>:<account>:accesspoint/<name> \
    DatadogApiKeySecretArn=<secret-arn> \
    DatadogSite=ap1.datadoghq.com \
  --capabilities CAPABILITY_NAMED_IAM

# 2. Deploy full observability (Pipeline + Monitors + Metrics + Scanner)
export DD_API_KEY_SECRET_ID="fsxn-datadog-api-key"
export DD_APP_KEY_SECRET_ID="datadog/fsxn-app-key"
export DD_SITE="ap1.datadoghq.com"
bash integrations/datadog/scripts/setup-full-observability.sh

# 3. (Optional) Deploy Log Archive for compliance
aws cloudformation deploy \
  --template-file integrations/datadog/template-log-archive.yaml \
  --stack-name fsxn-datadog-archive \
  --parameter-overrides DatadogExternalId=<external-id> \
  --capabilities CAPABILITY_NAMED_IAM

# 4. Create Facets (guided manual step)
bash integrations/datadog/scripts/setup-facets.sh

# 5. Enable Cloud SIEM (UI: Security → Cloud SIEM → Get Started → select fsxn)

Time from zero to full detection capability: ~45 minutes.

Pre-Deployment Checklist

Before deploying to production, confirm:

[ ] FSx audit configuration confirmed (XML format, rotation schedule)
[ ] S3 Access Point created and Lambda role authorized
[ ] Datadog API Key + APP Key stored in Secrets Manager
[ ] Datadog site region confirmed (AP1/US1/EU1)
[ ] Data classification sign-off: audit logs contain user PII (usernames, IPs, file paths) — confirm external transmission is approved per organization policy
[ ] Retention requirements defined (Datadog index retention + S3 archive lifecycle)
[ ] On-call/escalation path defined for Critical signals
[ ] Service accounts identified for monitor exclusion (svc-*)
[ ] Cloud SIEM trial or paid plan confirmed (for Security Signals)
[ ] Network path validated (VPC-external Lambda or NAT Gateway for S3 AP access)

Lessons Learned

1. API-first beats UI-first

Creating the Pipeline via API took 30 seconds. Doing it through the UI takes 5 minutes of clicking through dropdowns. More importantly, the API approach is repeatable, reviewable, and version-controlled.

2. Facets are UX, not functionality

Every field is searchable with @field:value whether or not you create a Facet. Facets just add sidebar convenience. Don't block on Facet setup — your monitors work without them.

3. Category Processor is the key transformation

Without it, analysts need to memorize Windows EventID codes. With it, @operation_name:Object Delete is immediately understandable. This single processor provides more investigative value than any other pipeline step.

4. Evaluation delay prevents false positives

The 60-second evaluation_delay on monitors accounts for ingestion latency. Without it, you get spurious alerts when log batches arrive slightly out of order.

5. Saved Views are investigation shortcuts

When a monitor fires at 3 AM, the on-call engineer needs to investigate immediately. Pre-built Saved Views with the right columns and filters eliminate setup time during incidents.

6. Anomaly detection needs baseline data

The anomaly monitor requires at least 2 weeks of historical data to build a reliable baseline. Deploy it early — it won't generate false positives during the learning period (it simply stays silent until confident).

7. Cloud SIEM and Log Monitors are complementary

Log Monitors give you fast Slack/PagerDuty notifications for the on-call team. Cloud SIEM Detection Rules give SOC analysts a structured triage workflow with MITRE mapping. Deploy both — they serve different audiences.

8. GeoIP is free threat intelligence

The GeoIP Processor is zero-cost and instantly useful. Even in a pure on-premises environment, unusual geography flags VPN misconfiguration, compromised credentials, or travel you weren't expecting.

9. Forwarder deployment enables an ecosystem

Once the Datadog Forwarder Lambda is deployed, enabling CloudTrail, GuardDuty, Lambda logs, or any other AWS log source is a single S3 notification configuration — no new infrastructure needed.

CloudTrail Correlation (Cross-Service Detection)

A dedicated detection rule correlates FSxN audit events with AWS CloudTrail context. When the signal fires, the investigation message includes pre-built queries for cross-service correlation:

source:cloudtrail @userIdentity.arn:*<username>*     → IAM actions
source:guardduty                                      → GuardDuty findings
source:vpc @network.client.ip:<suspicious-ip>         → Network flows

The rule (FSxN: Suspicious Deletion After IAM Role Assumption) fires on >30 deletions in 10 minutes and embeds a correlation checklist:

Was there a recent AssumeRole or ConsoleLogin from an unusual IP?
Did the user's permissions change in the last 24 hours?
Is the client IP in the corporate VPN range?
Are there GuardDuty findings for this account?

Note: This rule activates fully once CloudTrail logs are flowing to Datadog via the AWS Integration. The FSxN detection works immediately; CloudTrail queries return results after integration setup.

SOC Triage Runbook

A 7-step Notebook guides analysts from signal to resolution:

Step	Action	Output
1. Signal Context	Verify user, IP, time, volume	Timeseries widget
2. User Verification	Check AD status, role changes, maintenance windows	Manual checklist
3. Impact Assessment	Identify affected files and directories	Top List (paths)
4. Cross-Service Correlation	CloudTrail, GuardDuty, VPC Flow Logs	Query templates
5. Decision Matrix	Map conditions to response actions	Decision table
6. Response Actions	Disable account, snapshot, notify, ticket	Checklist
7. Post-Incident	Update thresholds, document, lessons learned	Checklist

Accessible at: Notebooks → "FSxN Security Signal Triage Runbook"

Case Management

Cases provide structured investigation tracking across the SOC team:

Project: FSXN — All FSx for ONTAP security investigations
Case creation: Manual from Security Signals, or auto-create via Workflow
Priority levels: P1 (critical mass deletion) through P4 (informational)
Lifecycle: Open → In Progress → Resolved → Closed

When a critical signal fires, the workflow creates a case automatically with:

Signal link and context
Affected user and IP
Investigation notebook link
Response checklist

Threat Intelligence (GeoIP Enrichment)

The Log Pipeline's GeoIP Processor automatically enriches every FSxN log with geographic data from the client_ip field:

@client_ip: 10.0.5.99 → @network.client.geoip: { country: "JP", city: "Tokyo", ... }

A detection rule flags access from unexpected countries:

# Rule: "FSxN: File Access from Unusual Geography"
# Query: source:fsxn -@network.client.geoip.country:JP -@network.client.geoip.country:US
# Threshold: >5 events/15min from non-JP/US IPs

This catches compromised credentials being used from abroad without needing an external threat intelligence feed — Datadog's built-in GeoIP database handles the enrichment at ingest time.

GuardDuty Correlation

A detection rule correlates FSxN deletion events with GuardDuty findings from the same source IP:

FSxN: >10 deletions/30min from IP X
  + GuardDuty finding for IP X (UnauthorizedAccess, Recon, Trojan)
  = Critical Security Signal with full context

The rule's investigation message includes pre-built GuardDuty queries:

source:guardduty @detail.resource.instanceDetails.networkInterfaces.privateIpAddress:{{@client_ip}}

GuardDuty findings flow to Datadog via the same Forwarder Lambda that handles CloudTrail — no additional setup needed.

Automated Snapshot Remediation

When mass deletion is confirmed (Critical signal reviewed by analyst), the Workflow invokes fsxn-snapshot-remediation Lambda:

Security Signal (Critical) → Analyst confirms → Workflow → Lambda → ONTAP REST API → Snapshot

The Lambda:

Receives volume name, SVM, and reason from the Workflow
Authenticates to ONTAP via Secrets Manager credentials
Creates a timestamped snapshot: remediation_20260614_215530_mass_deletion
Returns snapshot name and status for the Case record

# Lambda invocation payload (from Datadog Workflow)
{
    "volume_name": "finance_share",
    "svm_name": "ProductionSVM",
    "reason": "Mass deletion detected",
    "user": "CORP\\suspicious-user"
}

Configuration: Set ONTAP_MGMT_IP and ONTAP_CREDENTIALS_SECRET_ARN environment variables on the Lambda to point to your FSx for ONTAP management endpoint.

TLS note: The Lambda uses cert_reqs="CERT_NONE" for ONTAP REST API calls because FSx for ONTAP uses self-signed certificates by default. In production, upload the ONTAP CA certificate to the Lambda layer (/opt/certs/ontap-ca.pem) and configure urllib3.PoolManager(ca_certs="/opt/certs/ontap-ca.pem") to validate the connection.

Snapshot Storm Prevention (Cooldown)

To prevent runaway Snapshot creation during sustained mass-deletion events:

# Before creating snapshot, check for recent remediation snapshots
existing = http.request("GET",
    f"https://{mgmt_ip}/api/storage/volumes/{vol_uuid}/snapshots"
    f"?name=remediation_*&order_by=create_time desc&max_records=1",
    headers=base_headers)

if existing.status == 200:
    snaps = json.loads(existing.data).get("records", [])
    if snaps:
        last_snap_time = snaps[0].get("create_time", "")
        # Skip if a remediation snapshot was created in the last 15 minutes
        if is_within_cooldown(last_snap_time, minutes=15):
            return {"statusCode": 200, "body": "Skipped — cooldown active"}

ONTAP limits: Each volume supports up to 1023 Snapshots. The cooldown prevents hitting this limit during sustained attack scenarios.

Remediation Audit Trail

The Lambda invocation itself must be auditable — proving that the Snapshot was created by an authorized pipeline, not a rogue actor:

CloudTrail: Lambda invocation recorded with InvokedBy: workflow.datadoghq.com
ONTAP audit log: Snapshot creation appears as an administrative event with the API user
Datadog Case: Snapshot name and status recorded in the Case timeline
Lambda CloudWatch Logs: Full request/response logged with correlation ID

What's Next

This article covered the complete detection-to-response lifecycle. Future enhancements:

Datadog Threat Intel feeds — When Datadog makes the Threat Intel Indicators API available on AP1, feed internal IP reputation lists for richer enrichment
Multi-SVM snapshot orchestration — Extend the remediation Lambda to snapshot across multiple SVMs in parallel
Automated account lockout — Chain the Workflow to invoke AD lockout via Systems Manager Run Command

GitHub Repository

DEV Community: Yoshiki Fujiwara(藤原 善基)@AWS Community Builder

S3 Burst on ONTAP Files — Collect via S3 API, Consume via FlexCache NFS/SMB

Introduction

The Architecture

Test Environment

Why NFS First

SMB Shows Identical Results

SMB Considerations

Results — All Four Directions

What the Numbers Say

NFS Client Cache Effect

FlexCache Duality — It Works (With One Extra Step)

Result: Works After Enabling -is-s3-enabled true

Design Implication

When This Pattern Fits

When It Doesn't Fit

S3 Access Point Design Notes

S3 AP ≠ Amazon S3

Throughput is shared with NFS/SMB

Directory design matters

What It Costs — Egress First, Requests Second

The read count decides it

Requests matter too, but only below a certain object size

If the readers can move, move them

On the distribution side

Deploy

Conclusion

Embedding AI Agents into a File Portal — From AgentCore MCP to Multi-Agent Teams (Part 3)

Introduction

If You're Already Using ChatGPT / Copilot / Claude for Document Analysis

What I Built

Why Embed AI Agents in the Portal

AgentChat — 3-Mode Integrated Chat

3 Modes

Architecture

Tool Execution Visibility

Chat History Persistence

Multimodal Input

SemanticSearch — Vector Search

Keyword / Semantic Mode Toggle

Bedrock Knowledge Base Integration

Semantic Search Input Examples

Constraints

Agent Directory — Agent Catalog

Preset Agents

Custom Agent Creation (Agent Creator)

Agent Teams — Multi-Agent Collaboration

When to Use Multi-Agent vs Single Agent

Example Team Configuration

Roles

Team Creation Wizard

ActionApproval — Human-in-the-Loop

Why HITL Matters

Which Operations Require Approval

How the Approval Flow Works

Example: SnapMirror Break Approval Flow

UI Design

Bedrock Guardrails — Response Filtering

Applied Checks

Model, Cost, and Fallback

Model Configuration

Bedrock KB Configuration Parameters

Fallback Behavior

Chat History Retention Policy

Design Tradeoffs

On Streaming Responses

Security Model (AI Agent Specific)

Monitoring

Frequently Asked Questions

Staged Adoption Steps

Step 1: No KB, No Agent (File Search Only)

Step 2: Add Bedrock Knowledge Base

Step 3: Enable AgentChat (File Agent Mode)

Step 4: Multi-Agent Mode and HITL

Step 5: Agent Teams and Custom Agents

Resources

Summary and Next Steps

Embedding Storage Operations into a File Portal — From ARP/AI Incident Response to Regulatory Retention Management (Part 2)

Introduction

Why Embed Admin Operations in a Portal

DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

Result: Works After Enabling `-is-s3-enabled true`

Async Operations with `return_timeout=0`