DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

AWS Backup now copies FSx for ONTAP backups across Regions and accounts — a new option, measured Tokyo to Osaka

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sat, 29 Aug 2026 05:02:38 +0000

Introduction

How do you approach disaster recovery and backup for your file systems on AWS? Take FSx for ONTAP and a pair of Japanese Regions as the example: if Tokyo is primary, standing up a file system in Osaka and replicating into it continuously with SnapMirror is probably the first shape that comes to mind for anyone who has run ONTAP, or is about to evaluate it. Plenty of AWS users run exactly that in production, and as a way to hold availability it is still the best answer. RPO comes down from the 60 minutes AWS Backup offers at its shortest to SnapMirror's 5-minute minimum interval, switching between active and standby is breaking the relationship and promoting the destination, and cutting data back the other way is a snapmirror resync in the reverse direction.

The trouble was that "I just want the data sitting on the DR side, without operating a file system there until I actually restore" required the same shape. Amazon FSx for NetApp ONTAP volume backups were created and restored inside the file system's own Region and account, and could not cross a Region boundary. AWS Backup could already copy backups across Regions and accounts for FSx for Windows File Server, FSx for OpenZFS, and FSx for Lustre (AWS Backup extends Amazon FSx support to 5 additional AWS Regions and expands cross-Region and cross-account copy to 14 AWS Regions, April 2026), but FSx for ONTAP was not in that set at the time.

Then on 27 August 2026 that gap closed along the two paths below. One more shape became available to choose from in a backup strategy. The two were announced separately.

Path	What it does	Scheduling	Cross-account
FSx for ONTAP backup copy (`CopyBackup`)	Copies a volume backup to another Region	None. Run it each time	No. Same account only
AWS Backup (policy-based backup management service)	Copies FSx for ONTAP backups to another Region and another account	Yes. Copy rules in a backup plan	Yes. Requires AWS Organizations

So I copied a backup from Tokyo to Osaka, restored it at the destination, and confirmed the content matched. This post covers that procedure, along with the design and operational considerations that came with it.

Up front:

AWS Backup now copies FSx for ONTAP backups to another Region and another account (27 August 2026). The FSx for ONTAP CopyBackup also reaches another Region, but it is same-account and manual.
Moving them to another account needs a CMK. AWS managed keys cannot be used for a cross-account copy, and a file system's KMS key is set at creation — so it is a decision made before the file system exists (below).
What matters is that you no longer need a file system at the destination until you recover. The restore target is still the Region the backup lives in, so creating that file system at recovery time lands on your RTO — 20 minutes, measured (below).
CopyBackup has no scheduler. Recurring copies mean an AWS Backup backup plan or your own automation. The AWS Backup copy and restore were measured too — 6 m 31 s to 8 m 35 s to copy, 16 m 16 s to restore, at 9 MB.
This does not replace SnapMirror. If a minutes-level RPO and a failback procedure are requirements, SnapMirror still is the answer. Replication and copy are different operations and leave different things at the destination (below).
Restore time is proportional to capacity. My 13 minutes was a figure for 9.4 MiB. Restoring 10 TB on a minimum configuration is 22 hours at the floor, calculated from AWS published rates. I could not find a measured figure at 10 TB or above in public sources (below).
Cost inverts with scale. A Single-AZ capacity pool is 47.6% of backup storage ($0.0238 against $0.050, retrieved from the Price List API), so past roughly 4.8 TB a standing destination file system is lower monthly. Choosing to keep one running is not choosing the more expensive option (below).

Scope: within one account, copying a volume backup from ap-northeast-1 to ap-northeast-3, restoring it into a file system there, and verifying the content. Both CopyBackup and AWS Backup were measured.

Out of scope: cross-account copies (they require AWS Backup with AWS Organizations — I show the shape but did not measure it), building or failing back a SnapMirror relationship, durations at production data volumes, and reconciling inter-Region transfer charges against a bill.

Shape of the setup

(Dark theme: backup-copy-cross-region-en-dark@2x.png)

The right-hand side being empty in normal operation is the whole point. With SnapMirror, that right-hand side holds an equivalent file system at all times.

Layer	Day to day	When recovering
Tokyo (production)	file system + RW volume + backups	unchanged, or lost if the Region is affected
Osaka (destination)	the backup copy only. No file system	file system → SVM → restore into a new volume
Billing	Osaka pays backup storage only	Osaka's file system starts billing once it exists
Running the copy	`CopyBackup` each time, or an AWS Backup copy rule	—

On vendor neutrality: this post evaluates FSx for ONTAP backup copies on technical grounds only. The section on existing backup products touches other options, but not in a framing where one is better than another. Each suits a different context, and the goal here is to record what actually works and what does not.

Test environment

Item	Value
Date	2026-08-28 (`CopyBackup` path), 2026-08-29 (AWS Backup path)
Runs	One per step (n=1). Variance was not measured
Interface	AWS CLI v2 and the AWS Management Console. Japanese locale for the `CopyBackup` path, English locale for the AWS Backup path (the setting at capture time). CloudFormation untested
Source	`ap-northeast-1`, first-generation `SINGLE_AZ_1`, 1,024 GiB SSD, 128 MBps
Destination	`ap-northeast-3`, same shape, created for this test
ONTAP version	`9.17.1P7D1`
Source volume	FlexVol RW, 1 GiB, tiering `NONE`, storage efficiency off
Dataset	5 files / 9.4 MiB (nested directory, symlink, UTF-8 filename, one file at `0640`). The AWS Backup run used an equivalent 9,458,747 B set
AWS Backup	one vault per Region, one backup plan (one rule, one copy rule), selection scoped to the single verification volume ARN, existing `AWSBackupDefaultServiceRole`
KMS	Defaults in each Region: `aws/fsx` for the source file system, `alias/aws/backup` (`KeyManager: AWS`) for the AWS Backup vaults. CMKs untested, and this configuration cannot do a cross-account copy (below)
Clients	one `t3.micro` (AL2023) per Region, NFS
Actual spend	~$0.55 for the `CopyBackup` run (destination file system ran 1 h 24 m); under $2 for the AWS Backup run (destination file system ran ~1 h 21 m, two `t3.micro`)
Data used	Synthetic data from `/dev/urandom`. No real or personal data
Not verified	Cross-account copies (covered by the earlier post), CMKs, copying `AUTOMATIC` backups, incrementality, residue from a failed copy, deploying the CloudFormation template, restoring into an AD-joined SVM, reconciling inter-Region transfer charges, AWS Backup restore testing

Do not use these durations as an RTO basis for your environment. They are measured against 9.4 MiB, where fixed overhead dominates.

When this fits

You want a copy of your data in another Region but want to avoid running a file system there
Compliance requires retaining a copy that is logically separated from production
You want an isolated-account layer against credential compromise or a KMS key compromise
Recovery measured in tens of minutes to hours is acceptable
You already take volume backups and only want to add a destination for them
SnapMirror already covers availability, and you want a separate retention layer detached from production

When to look at something else for now

RPO in minutes is required (the guidance figure for backups is 60 minutes)
RTO in single-digit minutes is required (creating the destination file system took 20 minutes)
You need a documented path back to production (backup copies have none — see below)
FlexGroup volumes are in scope (copy is unsupported, and creation failed in my environment — see below)
You need copies spanning China Regions, or cross-account copies in the China Regions (unsupported)

If any of those apply, a SnapMirror-based design fits the requirement better. Running both together is also coherent, covered below.

The distinction that matters most — what changed is where a backup can live

Blur this and the runbook comes out wrong.

Claim	Status
A backup can be stored in another Region	now possible (`CopyBackup` or AWS Backup)
A backup can be stored in another account	now possible, but only through AWS Backup. The FSx for ONTAP `CopyBackup` stays inside one account
A backup can be restored to a file system in a different Region than the backup	no (unchanged)
A restore produces a new volume	unchanged. There is no restore-in-place path

using-backups.html still states that a backup can only be restored to a file system in the Region where the backup is stored. That is not a contradiction — copying increases the set of Regions where a backup is stored.

So recovery in Osaka is three steps:

A copied backup is already in Osaka (this is what you prepare in advance)
Create a file system and an SVM in Osaka
Restore as a new volume

Step 2 lands on your RTO. Not paying for a standby file system means paying for its creation when you need it. That is the trade you are choosing.

How `CopyBackup` and AWS Backup relate

Two different things are called "copy a backup to another Region" here, so it is worth separating them first. They are separate mechanisms. Neither calls the other, and their control planes and storage locations differ.

	FSx for ONTAP `CopyBackup`	AWS Backup
Whose API	Amazon FSx (`fsx:CopyBackup`)	AWS Backup (backup plans / copy jobs)
What it copies	An existing FSx for ONTAP volume backup, which must be `AVAILABLE`	A recovery point AWS Backup took itself
Where the copy lands	As an FSx for ONTAP backup in the destination Region	An AWS Backup backup vault
`BackupType`	`USER_INITIATED`	`AWS_BACKUP`
Scheduling	None. Called each time	Yes. Backup rules and copy rules
Cross-account	No	Yes, with AWS Organizations
Link back to the source	Keeps `SourceBackupId` and `SourceBackupRegion`	Followed through the recovery point ARN

Here is where the two get conflated. You cannot point CopyBackup at an AWS Backup backup vault, and an AWS Backup copy rule does not call fsx:CopyBackup on your behalf. Answering "I want CopyBackup on a schedule" with "use AWS Backup" is right not because the same call can be scheduled, but because the same goal is reachable through a different mechanism.

The restore constraint, on the other hand, is shared. Both paths ask you to pick an existing file system and SVM in the destination Region and both produce a new volume. The AWS Backup restore screen asks for a File system and a Storage virtual machine too (documented). So the benefit — no standby file system while nothing is wrong — holds for AWS Backup as well, and so does the cost: file system creation lands on the RTO either way.

On terminology: AWS Backup's own word is copy. How that differs from replication is the next section.

Replication and copy are different words for different things

Two words get used for "keep the data in another Region": replication and copy. They are different operations, and what sits at the destination differs. Without agreeing on that, a requirements conversation goes past itself.

	SnapMirror replication	AWS Backup / `CopyBackup` copy
What sits at the destination	A volume. A `DP` volume exists on the destination SVM	A backup (recovery point). No volume
When the source changes	It follows. Deltas are sent each schedule	It does not. A point-in-time image; later changes wait for the next backup
To use the destination	Break the relationship and promote it to become writable	A restore is required, and what you get is a new volume
Does the relationship persist	Yes. Having it in place is the normal state	No. Each copy is an independent artefact
What sets the RPO	The replication schedule (down to 5 minutes)	The backup interval (guidance: 60 minutes)
What the destination costs while idle	File system capacity and throughput	Backup storage only
Path back to production	`snapmirror resync` in the other direction	There is none

AWS's own wording splits the same way. The AWS Backup console and documentation call this operation copy throughout — the screens read Copy jobs, Copy rule, Copy type: Manual (screenshots above). The FSx for ONTAP API is named CopyBackup. Replication is the word for mechanisms where the destination exists and tracks the source, as in Amazon S3 Cross-Region Replication.

So "replicate cross-Region with AWS Backup" reads as though a volume at the destination is tracking the source. There isn't one. What is at the destination is a recovery point, and using it takes a restore. The difference lands directly on the RTO: replication promotes a volume that is already there, while a copy starts from creating a file system and an SVM and then restoring.

The reverse mistake is calling SnapMirror a backup. The destination tracks the source, so a file deleted on the source is gone from the destination's current state after the next transfer — what remains is whatever the destination's snapshots still hold. Retaining generations is the job of snapshots or SnapVault, not of the replication itself.

SnapVault is a third mechanism. ONTAP's SnapVault is a relationship that accumulates generations at the destination. It rides the same transport as SnapMirror with a different purpose. The backup copies in this post do not create an ONTAP SnapVault relationship.

Which one suits which requirement is in choosing between this and SnapMirror.

A short glossary

Only the terms this post uses. Skip if you already know them.

Term	What it is	Why it matters here
File system	The billing and performance unit. SSD capacity and throughput are provisioned here	This is what you no longer need at the destination
SVM (Storage Virtual Machine)	A logical server inside a file system, holding the NFS / SMB endpoints and auth config	Required to restore. A file system alone is not enough
Volume	Where data lives, and the unit of backup and restore	A restore produces a new volume
Junction path	Where a volume attaches in the SVM namespace (e.g. `/dr_restored`); the NFS mount path	Specified at restore
FlexVol / FlexGroup	Volume styles. FlexVol sits on one aggregate; FlexGroup spreads across several for capacity and parallelism	FlexGroup backup copies are unsupported
`RW` / `DP`	Volume type. `RW` is read-write, `DP` is read-only and used for SnapMirror destinations	`DP` cannot be backed up. A restoring volume transiently reads as `DP`
Tiering	Moving cold data from SSD to the cheaper capacity pool. `NONE` / `AUTO` / `SNAPSHOT_ONLY`	Affects read latency right after a restore
Snapshot vs backup	A snapshot lives inside the file system; a backup is an independent copy outside it	Only the backup can cross a Region boundary

If you run ONTAP on-premises

ONTAP concept	Relationship to this capability
SnapMirror	A different mechanism. Needs cluster peering and a standing destination file system
SnapVault	A different mechanism. Backup copies do not create a SnapVault relationship
Cluster peering	Not required for backup copies. Operationally this is the largest difference from SnapMirror
Driving it from ONTAP CLI / REST	Backup copies are an AWS-side feature with no corresponding ONTAP CLI command. Use the AWS CLI, API, or console
Trace left on the volume	A backup leaves a snapshot named `backup-<backup-id>` on the volume

Prerequisites and permissions

Prerequisites

Item	Condition
Source backup	must be `AVAILABLE`
Source volume type	`RW` (`DP`, LSM, and FlexCache destinations cannot be backed up at all)
Volume style	FlexVol. FlexGroup copies are unsupported
Partition	Between commercial Regions, between the two China Regions, and between the two GovCloud (US) Regions — but not across those sets
KMS	Encrypted with a key in the destination Region. No parameter needed if the default key is fine

A backup whose source volume has already been deleted can still be copied (measured). That is useful after an accidental deletion.

IAM

The requester must be able to reach the backup in the source Region. Following the shape in the AWS documentation:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CopyBackupAcrossRegions",
      "Effect": "Allow",
      "Action": "fsx:CopyBackup",
      "Resource": "arn:aws:fsx:*:<your-account-id>:backup/*"
    },
    {
      "Sid": "DescribeAndRestore",
      "Effect": "Allow",
      "Action": [
        "fsx:DescribeBackups",
        "fsx:DescribeFileSystems",
        "fsx:DescribeStorageVirtualMachines",
        "fsx:DescribeVolumes",
        "fsx:CreateVolumeFromBackup",
        "fsx:TagResource"
      ],
      "Resource": "*"
    }
  ]
}

The "Resource": "*" on the second statement is a simplification for readability. Narrow it in production. Some Describe* actions do not support resource-level scoping, but fsx:CreateVolumeFromBackup can be limited to the destination file system and SVM ARNs. To pin the destination Region, use the aws:RequestedRegion condition key:

{
  "Sid": "RestrictCopyDestination",
  "Effect": "Deny",
  "Action": "fsx:CopyBackup",
  "Resource": "*",
  "Condition": {
    "StringNotEquals": { "aws:RequestedRegion": ["ap-northeast-3"] }
  }
}

If you have a data residency requirement, put this guard in IAM or an SCP. "We have a rule that says not to" does not hold.

With a customer managed key (CMK), you will additionally need permissions such as kms:CreateGrant and kms:DescribeKey on the destination-Region key. I used the default key, so I have not verified this. If you are building on a CMK, prove it with a small volume first.

Values to replace with your own

Every command below works once you substitute these.

Placeholder	Meaning	How to get it
`<SRC_REGION>`	Source Region	e.g. `ap-northeast-1`
`<DST_REGION>`	Destination Region	e.g. `ap-northeast-3`
`<SRC_VOLUME_ID>`	The RW volume to back up	`aws fsx describe-volumes --region <SRC_REGION> --query 'Volumes[?OntapConfiguration.OntapVolumeType==`RW`].[VolumeId,Name]' --output table`
`<SRC_BACKUP_ID>`	The backup to copy	output of step 1
`<DST_SUBNET_ID>`	Subnet for the destination file system	`aws ec2 describe-subnets --region <DST_REGION> --query 'Subnets[].[SubnetId,AvailabilityZone,CidrBlock]' --output table`
`<DST_SG_ID>`	Security group for the destination file system	Must allow NFS / SMB. Sharing one SG with the client plus a self-referencing rule is enough
`<DST_FS_ID>`	Destination file system	output of step 4
`<DST_SVM_ID>`	Destination SVM	output of step 5
`<DST_VOLUME_SIZE_BYTES>`	Restored volume size, in bytes	Must be at least the data in the backup. Reusing the source `SizeInBytes` is safest. 1 GiB = `1073741824`

Two notes on sizing. The destination file system's SSD capacity must exceed the data being restored — restored data is written to SSD first. And a FlexVol cannot span more than one HA pair. For a large volume, settle the destination SSD capacity and generation (second-generation Single-AZ can add HA pairs) before you start.

Walkthrough — Tokyo to Osaka

1. Create a backup (skip if you already have one)

aws fsx create-backup \
  --region <SRC_REGION> \
  --volume-id <SRC_VOLUME_ID> \
  --tags Key=Name,Value=dr-source Key=Purpose,Value=cross-region-copy \
  --query 'Backup.{Id:BackupId,Life:Lifecycle}' --output table

Wait for AVAILABLE:

aws fsx describe-backups \
  --region <SRC_REGION> --backup-ids <SRC_BACKUP_ID> \
  --query 'Backups[0].{Life:Lifecycle,Pct:ProgressPercent}' --output table

2. Copy it

--region is the destination; --source-region is where the backup is now. This reads backwards at first — the copy is requested against the destination Region.

aws fsx copy-backup \
  --region <DST_REGION> \
  --source-backup-id <SRC_BACKUP_ID> \
  --source-region <SRC_REGION> \
  --copy-tags \
  --tags Key=Name,Value=dr-copy \
  --query 'Backup.{Id:BackupId,Life:Lifecycle,Src:SourceBackupId,SrcRegion:SourceBackupRegion}' \
  --output table

Add --kms-key-id <destination-region-key-arn> for a CMK. Incrementality is conditional on using the same KMS key, so changing keys mid-stream makes the next copy full.

Watch progress:

aws fsx describe-backups \
  --region <DST_REGION> --backup-ids <DST_BACKUP_ID> \
  --query 'Backups[0].{Life:Lifecycle,Pct:ProgressPercent}' --output table

COPYING → AVAILABLE means done. Mine took 7 m 15 s for a 9.4 MiB volume.

Day-to-day operation ends here. Steps 3 to 6 run only when you are recovering.

3. [Recovery] Check which generation the destination Region offers

Check the available deployment types before you need them. If first generation (SINGLE_AZ_1) is available, the minimum throughput is 128 MBps; where only second generation exists it is 384 MBps, which makes standby cost roughly 1.9× higher.

SINGLE_AZ_1 was accepted in Osaka (measured 2026-08-28). This varies by Region — confirm it for your destination before writing the runbook.

4. [Recovery] Create the destination file system

Billing starts here. Even at the minimum, Osaka costs roughly $0.37 per hour, and forgetting it runs about $270 a month. If you are doing this as a drill, open the teardown commands from step 8 first. During my own run I mistyped an option on delete-file-system and polled for 8 minutes while the command had already failed.

This is the 20 minutes that lands on RTO.

aws fsx create-file-system \
  --region <DST_REGION> \
  --file-system-type ONTAP \
  --storage-capacity 1024 \
  --storage-type SSD \
  --subnet-ids <DST_SUBNET_ID> \
  --security-group-ids <DST_SG_ID> \
  --ontap-configuration '{
    "DeploymentType": "SINGLE_AZ_1",
    "ThroughputCapacity": 128,
    "PreferredSubnetId": "<DST_SUBNET_ID>"
  }' \
  --tags Key=Name,Value=dr-restore-target \
  --query 'FileSystem.{Id:FileSystemId,Life:Lifecycle}' --output table

1,024 GiB and 128 MBps are the first-generation minimums. Raise them if your restore exceeds that. A restore pauses when SSD space runs out and resumes automatically once space is available.

5. [Recovery] Create the SVM

aws fsx create-storage-virtual-machine \
  --region <DST_REGION> \
  --file-system-id <DST_FS_ID> \
  --name dr_svm \
  --root-volume-security-style UNIX \
  --query 'StorageVirtualMachine.{Id:StorageVirtualMachineId,Life:Lifecycle}' --output table

For SMB you also need the Active Directory join here. If AD join is part of your recovery, verify separately that the destination Region can reach your domain controllers, and budget the time. My verification was NFS only.

6. [Recovery] Restore

A restore always produces a new volume. There is no in-place path.

aws fsx create-volume-from-backup \
  --region <DST_REGION> \
  --backup-id <DST_BACKUP_ID> \
  --name dr_restored \
  --ontap-configuration '{
    "StorageVirtualMachineId": "<DST_SVM_ID>",
    "JunctionPath": "/dr_restored",
    "SizeInBytes": <DST_VOLUME_SIZE_BYTES>,
    "OntapVolumeType": "RW",
    "TieringPolicy": { "Name": "NONE" }
  }' \
  --query 'Volume.{Id:VolumeId,Life:Lifecycle}' --output table

TieringPolicy: NONE follows the AWS recommendation for consistent read latency after a restore — it assumes you wait for the data to land on SSD before accessing it. Change it if you want the capacity pool.

Mine reached CREATED in 13 m 21 s for 9.4 MiB.

Three conditions drive restore and post-restore performance. Check these when it feels slow:

Condition	Detail
SSD capacity	Restored data is written to SSD first. The restore pauses when space runs out and resumes when it frees up
Generation	Second generation is readable during a restore (after metadata loads). First generation makes you wait for completion — mine was first generation
Background priority	Backup and restore are lower priority than client I/O and consume unused throughput capacity. They progress slowly while production I/O is saturated

The third one has an operational consequence: overlapping backups with a nightly batch window slows both. Separate the backup window from your peak.

7. Verify the content

Skip this and you cannot claim you restored anything. Order matters: take the hashes before you create the backup. Taking them afterwards mixes in every change since, and you can no longer separate that from a copy problem.

# On the source, BEFORE creating the backup
cd /mnt/src && find . -type f -exec sha256sum {} + | LC_ALL=C sort -k2 > /tmp/before.txt

# On the restored volume
cd /mnt/restored && find . -type f -exec sha256sum {} + | LC_ALL=C sort -k2 > /tmp/after.txt
diff /tmp/before.txt /tmp/after.txt && echo "MATCH"

Check permissions and symlinks too:

find . -mindepth 1 -printf "%y %m %s %p\n" | LC_ALL=C sort -k4

All five sha256 digests matched in my run, and mode 0640, the symlink target, and a UTF-8 (Japanese) filename all survived. I also used the ordering deliberately: a 2 MiB file added after the backup was absent from the restored volume, which is what makes it a point-in-time image rather than a mirror.

8. Delete what the test created

Forgetting the destination file system costs roughly $270 a month. The order matters.

# 1. volume (do not let it create a final backup)
aws fsx delete-volume --region <DST_REGION> --volume-id <DST_VOLUME_ID> \
  --ontap-configuration '{"SkipFinalBackup":true}'

# 2. SVM (after the volumes are gone)
aws fsx delete-storage-virtual-machine --region <DST_REGION> \
  --storage-virtual-machine-id <DST_SVM_ID>

# 3. file system (after the SVM is gone)
aws fsx delete-file-system --region <DST_REGION> --file-system-id <DST_FS_ID>

# 4. the copied backup, if you no longer need it
aws fsx delete-backup --region <DST_REGION> --backup-id <DST_BACKUP_ID>

Without SkipFinalBackup a final backup is created, and that backup then blocks the next deletion. I have been stuck on this before.

And confirm the deletion through the API. During this run I passed an unsupported option to delete-file-system and polled for 8 minutes while the command had already failed. Waiting without reading the response only advances the bill.

Measured through AWS Backup as well

Everything above is the CopyBackup path. The same route, Tokyo to Osaka, was also measured through AWS Backup. That is the path that carries scheduling and cross-account copies, so its behaviour is what matters for production.

Two ways round. An on-demand backup job followed by an on-demand copy job, and a backup plan with a copy rule left to fire on its schedule. Both used the same source volume (1 GiB, 9,458,747 bytes, 5 files).

Step	Elapsed
On-demand backup job	4 m 02 s
On-demand copy job, Tokyo to Osaka	8 m 35 s
Plan-triggered backup job	30 m 06 s, of which about 24 minutes was the start window
Copy rule copy job (automatic)	6 m 31 s
Restore from the AWS Backup console	16 m 16 s

The copy rule fired without being touched. The schedule came round, a job was created, the backup completed, and a copy job followed on its own to put a recovery point in the Osaka backup vault.

A plan-triggered job waits in `CREATED`

This changes an operational assumption. The on-demand job went to RUNNING immediately. The plan-triggered one stayed CREATED for about 24 minutes. When it starts inside the start window — 60 minutes in this configuration — is AWS Backup's decision.

So a runbook that reads "the scheduled time has passed, therefore the backup exists" is wrong by the length of that window. Confirm a generation by job state or by the recovery point, not by the clock.

The recovery point in Osaka points at the source

The Resource ID column names the Tokyo file system and volume. It sits in the Osaka vault and points at where it came from.

Open the recovery point and the copy is visible as such.

Source region reads Asia Pacific (Tokyo). That is the evidence this artefact came from a cross-Region copy. Copy type is Manual because this one was run on demand; the copy-rule one arrives as a separate recovery point.

The restore form makes you pick a destination file system and SVM

The "File system ID" at the top is the source. The destination is chosen separately, below. Read the two as the same thing and the restore target looks already decided. It is not.

Open the dropdown and only file systems in the destination Region are listed.

With no file system in Osaka this list is empty. Exactly the same constraint as the CopyBackup path: not holding a destination day to day is paid for here.

Storage efficiency is checked by default

Two things.

Volume size is pre-filled from the source (1024 MiB). The FSx for ONTAP console defaults the same field to 1 TiB (below), so AWS Backup is the more predictable of the two here.

"Enable storage efficiency" is checked by default. The source volume had it off, and the restored volume came back with StorageEfficiencyEnabled: true. The FSx for ONTAP console restore form pre-selects the source value, so the two consoles behave differently. Uncheck it to match the source.

Attribute	Source	After the AWS Backup restore
`OntapVolumeType`	`RW`	`RW`
`SizeInBytes`	1 GiB	1 GiB
`TieringPolicy`	`NONE`	`NONE`
`StorageEfficiencyEnabled`	`false`	`true` (the form's default)
`SecurityStyle`	`UNIX`	empty

SecurityStyle coming back empty had been seen once on the CopyBackup path. It reproduced through AWS Backup, so it is no longer a single-path observation.

The content matched

Mounted from an EC2 instance in Osaka and compared. All five sha256 values matched, and the symlink target, the 0640 mode, the UTF-8 filename, the nested directories and the mtimes were preserved.

The restored volume carries .snapshot/backup-<id>/. That <id> is the source FSx for ONTAP backup, not the Osaka recovery point ID. The destination volume's default snapshot policy starts running too, so hourly.* snapshots appear beside it.

What cannot be monitored, and what catches you on teardown

BackupSizeInBytes was 0 on both backup jobs. Not a size to alert on.

Restore progress read 0.00% for the whole 16 minutes, then 100%. Anything that estimates remaining time from the percentage will not work.

One thing caught me on teardown. A vault cannot be deleted until an EXPIRED recovery point is fully gone. After delete-recovery-point the entry stayed listed as EXPIRED for about six minutes, and delete-backup-vault refused with "contains recovery points" throughout. The underlying FSx for ONTAP backup was still AVAILABLE at that point. Teardown automation has to poll; taking the delete request's success as done leaves you stuck.

On the console language: the screens in this section are in English. That was the setting at capture time; the other sections show the FSx for ONTAP console in Japanese. Field names are quoted in English either way.

Cross-account copies were not measured. They require AWS Organizations, so this verification stayed inside one account.

For the cross-account half — an earlier post, and what this one adds

There is a post on the NetApp Community that goes further on cross-account copies than this one does.

Cross-Region and Cross-Account Backup for Amazon FSx for NetApp ONTAP (Jordan Roth, NetApp Community)

It walks the cross-account path I did not measure, having actually run it. It sets out the protection layers (Snapshot copies, Multi-AZ HA, SnapMirror, AWS Backup copies), four architecture patterns — fan-in into an isolated vault account, fan-out to several, crossing Region and account in one hop, and a clean room for incident work — and the organization, key and IAM prerequisites behind them. If cross-account is going to production, read that first. What follows here is only where it meets my own measurements.

What this post was missing — cross-account needs a customer managed key

This is the one that matters. A cross-account copy requires a customer managed key (CMK). The AWS Backup documentation states that for resource types not fully managed by AWS Backup, cross-account copy with AWS managed keys is not supported, because an AWS managed key's key policy is immutable and cannot be shared across accounts (Encryption for backups in AWS Backup). FSx for ONTAP sits on the "not fully managed" side.

My verification did not meet that condition. The destination vault was created with defaults, so its key was alias/aws/backup (KeyManager: AWS). The cross-Region copy still succeeded. That is looser than the documentation's general statement, and it is the reason for measuring rather than assuming.

	Key used	Result
Cross-Region copy (Tokyo to Osaka, one account)	Defaults on both the source file system and the vaults (`aws/fsx` / `aws/backup`)	Succeeded (measured)
Cross-account copy	AWS managed keys cannot be used	Not measured. Unsupported per the documentation

There is a design consequence. A file system's KMS key is chosen at creation, and update-file-system has no parameter for changing it (checked in the AWS CLI). So if a cross-account copy might ever be needed, the CMK decision happens before the file system exists. It is not the kind of thing to switch on later. The destination vault has its own constraint: the default vault cannot be used, because its key cannot be shared.

The Region exceptions are listed individually

Earlier I wrote "commercial Regions where both services are available". AWS Backup feature availability lists the exceptions one by one.

Exception	Scope
Cross-Region copy not supported	Middle East (Bahrain), Middle East (UAE) — for all four of FSx for ONTAP, Lustre, Windows File Server and OpenZFS
Cross-Region and cross-account copy not supported	Asia Pacific (New Zealand), China (Beijing), China (Ningxia) — FSx for ONTAP

The China Regions need reading carefully. The AWS Backup document history carries an entry dated the same 27 August 2026 saying cross-account backup copy and cross-account management are now supported in the China Regions. That is about AWS Backup generally; FSx for ONTAP is excluded there, per the table above. Merge the two same-day announcements and you reach the opposite conclusion.

On "what was possible before", that post and I read it differently

Its table of changes records cross-Region CopyBackup within one account as already available before the launch. This post has been saying a backup could not cross a Region boundary before. Rather than assert, here is what I checked.

What I checked	What it says
ONTAP User Guide, "Protecting your data with volume backups" (archived 2026-07-05)	The sub-page list has no "Copying backups" — only "Copying tags to backups". The body says a backup restores to a file system in the same Region where it is stored
`copy-backups.html` in the ONTAP User Guide	No snapshot at all in the Internet Archive
The FSx for ONTAP What's New	"Previously, you could create and restore backups in the same Region and account as your file system"

What I cannot test now is whether the CopyBackup API would have accepted an ONTAP backup before the launch. That API is not specific to FSx for ONTAP, and has supported cross-Region copies for FSx for Windows File Server and FSx for Lustre for years, so reading the API reference alone and concluding it was already possible is a reasonable reading. Documentation not describing something is not the same as an API refusing it, so I am leaving that unverified.

For a reader today the practical answer is the same either way. What is unambiguously new is the AWS Backup half: cross-Region copy through AWS Backup, and any cross-account copy, from 27 August 2026.

What this post adds

The parts that do not overlap.

Here	What
Measured durations	Backup, copy and restore on both paths, to the second — including a plan-triggered job waiting about 24 minutes in the start window
Two consoles behaving differently	The AWS Backup restore form checks storage efficiency by default; the FSx for ONTAP one pre-selects the source value
A single-path observation reproduced	`SecurityStyle` coming back empty
What cannot be monitored	`BackupSizeInBytes` reads 0; restore progress sits at `0.00%` for 16 minutes
Where teardown sticks	An `EXPIRED` recovery point blocks vault deletion for minutes
Cross-Region copy on default keys	It works with `aws/backup` (above)
Replication and copy as terms	Covered in its own section
Regional DR inside one country	Tokyo to Osaka, with the cost of keeping data in-country

The two things that caught me out

1. A restoring volume reads as `DP`, not `RW`

I read the attributes right after the restore reached CREATED and got OntapVolumeType: DP. DP volumes are read-only and cannot be backed up. I started writing "a restore produces a read-only volume".

Then I tried touch from the client. It succeeded. A re-read returned RW. To be sure, I ran a second restore passing OntapVolumeType: RW explicitly — and it also reported DP while CREATING.

State	`OntapVolumeType`
Restoring (`CREATING`)	`DP`
Restore complete (`CREATED`)	`RW`

It is transient. If monitoring or automation reads this field, it sees DP during a restore. Since DP volumes cannot be backed up, a naive "back up as soon as the restore finishes" step behaves differently depending on when it reads. Gate on Lifecycle reaching CREATED first.

Without the write probe I would have published the wrong claim.

2. FlexGroup backup creation failed asynchronously

The documented limitation is that copying backups of FlexGroup volumes is unsupported. It attaches to the copy and says nothing about creation. I misread it and built the test on the creation side.

CreateBackup against a FlexGroup volume was accepted and returned CREATING, then went FAILED about 30 seconds later. The message names no cause:

Backup failed. Please delete the backup and try again.

As a control, a FlexVol RW backup created in the same session with the same credentials succeeded — so this is not permissions.

But this is one observation on one volume. I am not claiming FlexGroup backups cannot be created. AWS documentation describes the restore behaviour of FlexGroup backups (adding constituents when the HA pair count differs), which does not read as creation being unsupported. Separately it states only that a SnapLock FlexGroup volume cannot be backed up. My target was a non-SnapLock FlexGroup, single constituent, one aggregate, 113 GiB provisioned.

The consequence: no AVAILABLE FlexGroup backup was produced, so the documented copy restriction was never exercised. If FlexGroup is in scope, verify it from the creation side in your own environment.

3. The attribute I called "not carried over" was a CLI default

Seeing StorageEfficiencyEnabled: true on the restored volume, I wrote that storage efficiency is not carried over by a restore. That was wrong too.

Running the same restore through the console showed the field pre-selected to the source value (disabled), and it restored as false. The initial true came from omitting the field in the CLI --ontap-configuration. It is an API default when omitted, not restore behaviour. State it explicitly if you script the restore.

That is twice in one exercise that I nearly published a behaviour inferred from a single observation. Running the same operation through a second interface is what exposed both.

Attribute	Source	Restored
`StorageEfficiencyEnabled`	`false`	`true` if omitted in the CLI, `false` when stated
`SecurityStyle`	`UNIX`	empty in the API response (both CLI and console; UNIX mode bits behaved correctly over NFS)

Check storage efficiency after a restore — omitting the field enables it even when the source had it off. The empty SecurityStyle reproduced across both paths, but against the same UNIX-style source; other security styles were not tested.

What the console shows that the CLI does not

Four behaviours only visible in the console.

A note on the screenshots: my console is set to Japanese, so that is what you will see. The field
positions and the behaviour are identical in any locale, and each caption names the field in English.
If you would rather follow along in English, switch your own console language under Unified
Settings → Language.

The copy form defaults to the same Region

Backups list. Select a backup, then **Actions → Copy backup* (バックアップをコピー), alongside restore
(バックアップを復元) and delete (バックアップを削除).*

Copy backup form. *送信先リージョン (destination Region) defaults to the current Region** — leaving it
alone produces an in-Region copy. タグをコピー is "copy tags".*

The destination Region list. Commercial Regions only; the China and GovCloud sets cannot be crossed.

With Asia Pacific (Osaka) selected, the **KMS キー ID (KMS key ID) changes to that Region's default
key. This is what "incrementality requires the same KMS key" means in practice: the key in the
destination Region.

While the copy runs

Destination Region's Backups list. ステータス reads コピー中 ("copying"), and the entry is labelled
*(コピーされたバックアップ) — "(copied backup)"*. Tag copying carried the name across.

The completed copy. *ソースバックアップ ID (source backup ID)** and ソースバックアップリージョン
(source backup Region) = ap-northeast-1 are the two fields that establish this is a copy. タグ shows
the copied tags.*

Restore needs more than a file system

The dialog is titled *バックアップからボリュームを作成 — "Create volume from backup"*. The title states
plainly that a restore produces a new volume. ファイルシステム (file system) starts empty.

The ファイルシステム dropdown lists **only file systems that already exist in the destination Region. If
you have not created one, there is nothing to select — which is where the "create the file system first"
step announces itself.

After picking the file system, *ストレージ仮想マシン (storage virtual machine) appears with a red 必須
("required")** and blocks submission. This is the concrete form of "the destination needs a file system
and an SVM". Note the SnapLock section at the bottom — irreversible, and not a field to set in
passing during a restore.*

The volume size default is 1 TiB

The same form filled in. *ボリュームサイズ (volume size) defaults to 1 TiB** even though the source
volume is 1 GiB — I changed the unit to GiB here. ボリューム名, ジャンクションパス, and ストレージ効率
are all pre-filled from the backup; size is not. On a 1,024 GiB destination the default consumes the
entire file system.*

The console shows `DP` too, and does not refresh

During the restore. *ライフサイクルの状態 (lifecycle) = 作成 ("creating")** and ONTAP ボリュームタイプ
(ONTAP volume type) = DP — with Read-Write (RW) selected in the form. ジャンクションパス and
ストレージ効率 are absent, and セキュリティスタイル (security style) is blank.*

The same page after a reload: *作成済み ("created")** and RW, with junction path, storage efficiency,
and snapshot policy now shown. The page does not auto-refresh — it kept showing DP after the API had
already returned RW, so judging completion by eye alone misses it. セキュリティスタイル stayed blank.*

Designing this for production

Everything above was one manual copy. Six things still need deciding.

1. Scheduling — `CopyBackup` has none

The AWS documentation describes the FSx for ONTAP path as "manually copy volume backups". It runs from the console, CLI, or API on demand and carries no scheduler. Two ways to make it periodic:

Path	Shape	Fits when
AWS Backup backup plan	Adding a copy rule to a plan's rule writes each backup to a backup vault in the chosen Region, and optionally the chosen account, as it is taken. On-demand copy jobs are available too	You want policy control, or the same protection applied across an organization. The only path if the copy has to land in another account
Your own automation	EventBridge Scheduler → Lambda → `create-backup` → wait → `copy-backup`	You want to keep using FSx for ONTAP backups directly and manage generations yourself

AWS Backup is a managed service that centralizes taking, copying, and retaining backups under policy. FSx for ONTAP joined its cross-Region and cross-account copy targets in August 2026. Applied across an organization it pairs with AWS Organizations, and the account boundary is defined by Organizations policies; the documentation describes fan-in (many production accounts into one isolated account) and fan-out (one production account into several isolated accounts).

The cross-Region copy and the restore were measured (Measured through AWS Backup as well). The copy rule fired without being touched and the restore went through. A plan-triggered job does wait inside the start window, though — about 24 minutes here — so the scheduled time is not when the generation exists.

AWS Backup's Region coverage here is "all commercial Regions where both services are available", but the feature availability page lists individual exceptions (below). That page also states that when a resource type has no checkmark in the copy columns, copy operations are unsupported in every scenario, including a same-Region same-account copy to a different vault.

If you build it yourself, calling copy-backup before create-backup completes fails — the source must be AVAILABLE. Poll, or use a Step Functions wait.

Whether AUTOMATIC backups can be copied is unverified. The What's New post says "new and existing backups" without qualifying the type. If automatic backups are your evacuation source, prove it with one generation first.

2. Make recovery declarative — CloudFormation can express the restore

Typing six steps during an incident is not realistic at any team size. This can be declarative.

AWS::FSx::Volume has a BackupId property ("the ID of the volume backup to use to create a new volume"). So the file system, the SVM, and the restore all fit in one template:

Parameters:
  CopiedBackupId:
    Type: String
    Description: ID of the backup in the destination Region

Resources:
  DrFileSystem:
    Type: AWS::FSx::FileSystem
    Properties:
      FileSystemType: ONTAP
      StorageCapacity: 1024
      StorageType: SSD
      SubnetIds: [!Ref DrSubnetId]
      SecurityGroupIds: [!Ref DrSecurityGroupId]
      OntapConfiguration:
        DeploymentType: SINGLE_AZ_1
        ThroughputCapacity: 128
        PreferredSubnetId: !Ref DrSubnetId

  DrSvm:
    Type: AWS::FSx::StorageVirtualMachine
    Properties:
      FileSystemId: !Ref DrFileSystem
      Name: dr_svm
      RootVolumeSecurityStyle: UNIX

  DrRestoredVolume:
    Type: AWS::FSx::Volume
    Properties:
      Name: dr_restored
      VolumeType: ONTAP
      BackupId: !Ref CopiedBackupId
      OntapConfiguration:
        StorageVirtualMachineId: !Ref DrSvm
        JunctionPath: /dr_restored
        SizeInBytes: 1073741824
        OntapVolumeType: RW
        TieringPolicy:
          Name: NONE

Keeping this template undeployed in the destination Region is the cold-standby-IaC shape. Zero cost while idle, one deploy at recovery, and no step to forget.

I have not deployed this template — I ran the same sequence through the CLI. Deploy it once and time it before it becomes your runbook. An undeployed recovery template is a draft, not a procedure.

3. RTO breakdown — "20 minutes" is one line of it

I measured file system creation (20 min) and the restore (13 min). Your RTO is not the sum of those two. Fill this in for your environment.

Step	My measurement	Yours
Decision to recover (detection to action)	not measured
Destination file system creation	~20 min
SVM creation	a few minutes
AD join (if using SMB)	not performed
Restore to `CREATED`	13 m 21 s (9.4 MiB)
Export policy / share reconfiguration	not measured
Client remount, DNS cutover	not measured
Business-side validation	not measured

If you use SMB, the AD join is in there. Confirm destination-Region reachability to your domain controllers while nothing is wrong. Discovering that mid-incident is a dead end.

Restore time gets longer with capacity

The 13 m 21 s and 16 m 16 s I measured were on a 9.4 MiB volume. Treat those as reference points only. Restore time for FSx for ONTAP backup, through either path, scales with capacity.

AWS publishes the rates (Backup and restore performance).

Operation	Mostly large files	Mostly small files
Backup creation	750 MBps	100 MBps
Restore	250 MBps	100 MBps

Two conditions come attached to those numbers, and both push an estimate to the longer side.

They are values "across several concurrent restores" — that is AWS's wording. Nothing says a single restore runs at that rate.
Backup and restore are background work and use only unused throughput capacity. Throughput capacity is therefore the ceiling. A 128 MBps file system will not reach 250 MBps.

Applying both, the calculation is min(published rate, throughput capacity). These are floors, on the optimistic assumption that client I/O is zero.

Logical data	128 MBps / large files	256 MBps+ / large files	Mostly small files (any throughput)
1 TB	2 h 10 m	1 h 6 m	2 h 46 m
5 TB	10 h 51 m	5 h 33 m	13 h 53 m
10 TB	21 h 42 m	11 h 6 m	27 h 46 m
20 TB	43 h 24 m	22 h 13 m	55 h 33 m
50 TB	108 h 30 m	55 h 33 m	138 h 53 m

Restoring 10 TB on the minimum configuration (128 MBps) is 22 hours at the floor. Mostly small files is 28 hours, and that one does not shrink by adding throughput, because the published 100 MBps binds first. At 50 TB it passes four days.

I could not find a measured figure at 10 TB or above in any public source. I searched AWS blogs, re:Post, the Knowledge Center, and the documentation, and found no article that publishes a measured restore duration with the capacity stated. The table above is calculated from AWS published rates — it is not something anyone measured. If you need a meaningful number for your own environment, the only way is to measure once at production-equivalent capacity and file mix.

And this can overturn "generation 1 is enough"

The section below says gen 1 is enough for a DR standby. At scale that reverses.

	Gen 1	Gen 2
Monthly at minimum throughput	$115.97 (128 MBps)	$772.99 (384 MBps)
Reads while restoring	No. You wait for completion	Yes, once metadata has loaded
Time to service at 10 TB	21 h 42 m (floor)	6–46 min (after 1–7% of metadata loads; the rest loads in the background)

Metadata is documented as 1–7% of the backup data, depending on file size — the smaller the files, the larger the share. At 10 TB, loading just that 1–7% first lets clients mount and start reading. A first access to a block that has not loaded yet costs tens of milliseconds, and it is cached on SSD after that.

So $657 a month of difference can turn a 21-hour outage into something closer to 30 minutes. If you hold 10 TB or more and your RTO is constrained in hours, choosing gen 2 for the standby too is a defensible call. Gen 1 is enough when either the capacity is small or the RTO can absorb waiting for the restore to finish.

A few TB or less, or an RTO in days → gen 1
10 TB or more with an RTO in hours → gen 2's reads-while-restoring earn their keep
Both are a longer RTO than SnapMirror (neither reaches a shape where the destination volume already exists)

4. Application consistency — this does not replace a database backup

A backup is a point-in-time copy of a volume. It does not quiesce an application.

Workload	What you need
File servers, shared storage	A volume point-in-time is usually enough
Databases (Oracle, SQL Server, …)	A database-level backup, or quiescing before the backup
Anything with constantly open writes	Application-level consistency checks after restore

Reading "we copy volume backups to another Region, so DR is covered" for a database workload can end with a volume that restores and a database that will not start.

5. Concurrency limits at scale

The limits are on operations in flight, not on how much you can keep.

Limit	Value	Consequence
Copies in flight (per volume, per destination Region, per KMS key)	5	Parallelizing one volume's generations stops at 5
Copies in flight (per account)	1,000	Firing off hundreds or thousands of volumes at once exceeds it. Excess requests are rejected
Backups per volume	4,091	Work your generation plan backwards from here

In a multi-tenant design with one volume per tenant, tenant count becomes your concurrency. Batch the submissions and retry rejections with exponential backoff.

6. Monitoring and control

Copies are asynchronous. When copy-backup returns, nothing has finished yet.

Decide	Options
Where failures are detected	With AWS Backup you can follow job state. For `CopyBackup`, polling `describe-backups` is the dependable route. Whether CloudWatch or EventBridge can carry this is unverified
How success is evidenced	Store the `describe-backups` response (`BackupId`, `SourceBackupId`, `SourceBackupRegion`, `CreationTime`)
Who approves a restore	A restore creates a new volume and destroys nothing, but it consumes destination SSD and starts billing. Put it in an approval flow where that is expected
Which destinations are allowed	Enforce it in IAM or an SCP, not in a written rule

One note on tags. --copy-tags carries source tags to the destination. If your tags name systems or departments, that naming appears in the isolated account too. Depending on why you are isolating, re-tagging may be the better choice.

Cost

The $0.55 I spent was a test bill. What is useful in your environment is a monthly figure, so this builds one up from unit rates.

Unit rates — queried from the AWS Price List API

The rates below were queried from the AWS Price List API on 2026-08-29 (effective 2026-07-01). Tokyo and Osaka were identical on all four of the items that appear in both.

Charged item	Tokyo / Osaka rate	Nature of the charge
SSD storage (Single-AZ)	$0.150 / GB-month	provisioned
SSD storage (Multi-AZ)	$0.300 / GB-month	provisioned
Capacity pool Standard (Single-AZ)	$0.0238 / GB-month	consumed
Capacity pool Standard (Multi-AZ)	$0.0476 / GB-month	consumed
Backup storage	$0.050 / GB-month	consumed (incremental)
Throughput capacity (Single-AZ, gen 1)	$0.906 / MBps-month	provisioned
Throughput capacity (Single-AZ, gen 2)	$2.013 / MBps-month	provisioned
Capacity pool read requests	$0.00037 / 1,000 requests	consumed
Capacity pool write requests	$0.0047 / 1,000 requests	consumed

Going through AWS Backup does not change the backup storage rate. FSx for ONTAP is not a resource type that AWS Backup fully manages, so the storage charge appears on the FSx for ONTAP side rather than on AWS Backup (Metering, costs, and billing for AWS Backup). The rate is therefore $0.050 / GB-month on the native path and on the AWS Backup path alike. Only when you use a logically air-gapped vault (LAG vault) does all of the storage and transfer land on the AWS Backup side.

FSx for ONTAP backup is not the cheap option

$0.050 / GB-month is 2.1× the Single-AZ capacity pool ($0.0238). Put the other way round, the capacity pool is 47.6% of backup storage — less than half.

AWS makes the same comparison. AWS Prescriptive Guidance puts fully managed backup at $0.05 / GB-month, against $0.03221 / GB-month for SnapVault aimed at a capacity pool at an SSD-to-pool ratio of 1:10 (Choose the right SMB file storage, US Region example).

That 47.6% is Single-AZ only. A Multi-AZ capacity pool is $0.0476, which is 95.2% of backup storage, and the advantage all but disappears. Carrying "the capacity pool is less than half" into a Multi-AZ estimate will put your numbers wrong.

Running one continuously has a floor

Read the rate table straight and it seems to say "if the capacity pool is cheap, then keeping a destination file system running is cheap too". It does not work out that way. The minimums on the provisioned charges set the floor for anything kept running.

Constraint	Value	Source
Minimum SSD capacity	1,024 GiB per HA pair	Quotas
Minimum throughput capacity (gen 1)	128 MBps	same
Minimum throughput capacity (gen 2, 1 HA pair)	384 MBps	same

Monthly cost of a Single-AZ minimum configuration:

Generation	Components	Monthly at minimum
Gen 1 (128 MBps)	1,024 GiB × $0.150 + 128 MBps × $0.906	$269.57
Gen 2 (384 MBps)	1,024 GiB × $0.150 + 384 MBps × $2.013	$926.59

The same "minimum configuration" differs by 3.4× between generations, because gen 2's minimum throughput is 384 MBps and its per-MBps rate is 2.2× gen 1. For backup and a DR standby, gen 1 is the choice (below).

All-tiering does not remove this floor either. Metadata always stays on SSD, so the 1,024 GiB SSD minimum remains provisioned.

It inverts with scale

Assumptions: 30 daily backup generations, 2% daily change rate, no dedupe or compression benefit assumed, Single-AZ gen 1 at the 128 MBps minimum, the Osaka rates above. The all-SSD column provisions capacity to the recommended 80% SSD utilisation.

Logical data	Backup copies (no destination FS)	Standing + all-tiering	Standing + all SSD
1 TB	$80	$293	$303
2 TB	$160	$317	$491
5 TB	$400	$389	$1,053
10 TB	$800	$508	$1,991
20 TB	$1,600	$746	$3,866
50 TB	$4,000	$1,460	$9,491

The crossover is around 4.8 TB. That is the $269.57 floor divided by the per-GB difference: 1.6 generations of backup at $0.050 ($0.080) against the capacity pool at $0.0238, a gap of $0.0562.

This is what backs up "we keep one running for RTO and RPO". Past 4.8 TB a standing destination file system is lower monthly, and a minutes-level RPO, a single-digit-minute RTO, and a failback path come with it. Improving recoverability is not an increase in monthly cost at that size. At 1 TB it is the other way round, and the gap between $80 and $293 has to be explained as a difference in requirements.

Why a DR standby takes generation 1

For backup and a DR standby, gen 1 is enough. The reason is the throughput increments.

	Gen 1	Gen 2 (1 HA pair)
Selectable throughput	128 / 256 / 512 / 1,024 / 2,048 MBps (Tokyo and Osaka cap at 2,048)	384 / 768 / 1,536 / 3,072 / 6,144 MBps
Monthly at minimum throughput	128 MBps × $0.906 = $115.97	384 MBps × $2.013 = $772.99

A standby's throughput is sized by SnapMirror transfer volume and the reads right after a failover, not by production load. Gen 2 still floors at 384 MBps for that use, which is 6.7× on the throughput line alone. With gen 1 you start at 128 MBps and move up to 256, then 512, when you need it.

Deployment type cannot be changed after creation (documented). Going from gen 1 (SINGLE_AZ_1) to gen 2 (SINGLE_AZ_2) means a restore from backup, or a migration over SnapMirror or AWS DataSync. Decide the generation before you build the standby.

Total monthly by throughput

The 4.8 TB crossover above assumes the 128 MBps minimum. Raise throughput and the crossover moves up with it. Here are the combinations laid out.

Assumptions: Single-AZ gen 1, all-tiering, SSD at the 1,024 GiB minimum, Tokyo / Osaka rates. The backup-copy column is 30 daily generations at a 2% change rate. Capacity pool request charges are not included.

Logical data	128 MBps	256 MBps	512 MBps	1,024 MBps	2,048 MBps	Backup copies
1 TB	$293	$409	$641	$1,105	$2,033	$80
5 TB	$389	$505	$736	$1,200	$2,128	$400
10 TB	$508	$624	$855	$1,319	$2,247	$800
20 TB	$746	$862	$1,093	$1,557	$2,485	$1,600
50 TB	$1,460	$1,576	$1,807	$2,271	$3,199	$4,000

Bold is cheaper than holding backup copies. Restated as the highest throughput that still comes in under the backup-copy figure:

Logical data	Backup copies / month	Highest throughput under that
1–2 TB	$80–$160	None. Even 128 MBps standing costs more
5 TB	$400	128 MBps
10 TB	$800	256 MBps
20 TB	$1,600	1,024 MBps
50 TB	$4,000	2,048 MBps (the gen 1 ceiling in Tokyo and Osaka)

At 50 TB you can provision the largest throughput gen 1 offers in-Region and still come in cheaper than holding backup copies. At 1–2 TB backup copies are cheaper at every throughput, and there the difference is a requirements conversation.

What the "zero while idle" of backup copies buys

With no destination file system in normal operation, there is no throughput capacity charge and no SSD capacity charge. That is a clear advantage on the monthly figure. The work it defers, though, reappears at the moment of failover, as urgent work.

And the unit of a backup is the volume, so a restore always produces a new volume. There is no path that overwrites an existing volume in place, on the CopyBackup route or the AWS Backup route. That is what makes steps 4 to 6 below unavoidable rather than optional: a new volume means a new junction path if the original still exists, and export policies, SMB shares, quotas, and the snapshot policy do not travel with it. The volume size default is a separate trap — it comes up as 1 TiB rather than the source's size, covered further up.

In an incident that same constraint cuts the other way and becomes an advantage: you can preserve a compromised volume exactly as it stands and restore beside it, so the original is still there to investigate.

Here is what a cold start to restored service actually needed, in the range I measured.

#	Step	Measured / state
1	Create the file system in the destination Region	20–22 min (measured, `SINGLE_AZ_1`, 128 MBps)
2	Create the SVM	a few minutes
3	Restore the volume from backup	13 m 21 s (`CopyBackup` path) / 16 m 16 s (AWS Backup path), both at 9 MB
4	Recreate export policies and SMB shares	Only the volume is restored. Share configuration does not come with it
5	AD-join the SVM if you use SMB	It is a new SVM, so it joins again
6	Repoint clients	The new file system has a different DNS name and different IPs. Either move a CNAME or remount on the client

Steps 1 and 3 finish if you wait. Steps 4 to 6 are a person deciding and typing. And they are needed mid-incident, not on a quiet afternoon. What you saved monthly is traded against doing that work under pressure.

In a SnapMirror shape, 1 to 3 do not exist, and the destination SVM in 5 is already joined. What is left is 6 and the break-and-promote, so the number of steps is not comparable to begin with.

Steps 4 to 6 can be automated ahead of time. Declare the restore in CloudFormation, script vserver cifs create and the share creation, put a CNAME in front of DNS. But "can be" and "confirmed to work during an incident" are different claims. Run it through once while nothing is wrong.

The price difference is the requirement difference

Here is what sits inside the $80 and the $293 at 1 TB.

	Backup copies	Standing (SnapMirror shape)
RPO	60 min (the automatic backup minimum)	5 min
RTO	file system creation 20–22 min + restore 13–16 min (measured)	promote the volume
Failback	no path	yes
Destination file system in normal operation	not needed	needed

If the requirement is a minute-level RPO, backup copies are not a candidate. And at recovery time the backup-copy side creates a destination file system too, so the same floor starts applying from then on. The accurate reading is a design whose cost structure differs between normal operation and recovery.

Not included above

Capacity pool request charges: not in the tables above. The rates are known, but I have not measured the conversion from SnapMirror transferred bytes to request counts, so I cannot produce a number. With all-tiering every write goes to the capacity pool, so write requests ($0.0047 / 1,000) may not be negligible.
Inter-Region data transfer: treated differently depending on the path. Split out into the next section
Source-side backup storage (unchanged — you only added somewhere to put a copy)
KMS request charges if you use a CMK
SnapMirror inter-Region transfer (SnapMirror is not measured in this post)

Inter-Region data transfer is treated differently per path

AWS Backup copies have a transfer line item. Tokyo to Osaka is $0.09 / GB (Price List API, AWSBackup, APN1-APN3-CrossRegion-WarmBytes-FSx, retrieved 2026-08-29). The AWS Backup pricing page gives the same transfer rate for Resource Group 3, which includes Amazon FSx, on a standard vault and on a logically air-gapped vault alike.

For native CopyBackup, the reasonable thing is to estimate as though it applies. I started to write "probably does not apply", checked again, and withdrew it. Here is why.

The argument I withdrew: backups sit in AWS-managed S3 and never traverse the customer VPC, so they are not subject to inter-Region transfer charges the way EC2 is. That does not hold. EBS snapshots are equally in an AWS-managed area and equally do not traverse a VPC, and a cross-Region snapshot copy does incur AWS Data Transfer charges. The AWS Storage Blog walks through tracking that charge in Cost Explorer, and the usage type that appears there is USW2-USE1-AWS-Out-Byte, the service is "EC2 - Other", and the usage type group is Data Transfer - Region to Region (Out) (Effectively track AWS data transfer costs for cross-region Amazon EBS Snapshot Copy).

That also exposes a hole in how I first checked it. I based it on "there is no transfer item in the FSx for ONTAP price list", but inter-Region transfer is listed under AWSDataTransfer, not under the originating service. EBS is exactly that case: there is no "snapshot transfer" item in the EBS price list, and the charge happens anyway.

And AWSDataTransfer has a generic, service-agnostic inter-Region SKU.

Direction	Usage type	Rate
Tokyo → Osaka (out)	`APN1-APN3-AWS-Out-Bytes`	$0.09 / GB
Osaka → Tokyo (in)	`APN1-APN3-AWS-In-Bytes`	$0.00 / GB

(Price List API, AWSDataTransfer, effective 2026-06-01, retrieved 2026-08-29)

The Amazon FSx SKU on the AWS Backup side is also $0.09 / GB. The same transfer rate appearing in both places makes it natural to read them as the same underlying charge.

To summarise, this is as far as it can be taken right now.

Question	State
AWS Backup cross-Region copies incur transfer	Listed in the price list ($0.09 / GB, Tokyo → Osaka)
Native `CopyBackup` incurs transfer	Estimate as though it does. A generic $0.09 / GB SKU exists, and the same shape (EBS snapshot copy) is charged in practice. But I have not confirmed it on a bill — unverified
"It is not in the FSx for ONTAP price list, so it is free"	This inference is wrong. Transfer is listed under `AWSDataTransfer`

In an estimate, put in the first full copy's capacity × $0.09 / GB. After that it is incremental, so changed volume × $0.09 / GB. A 10 TB first full is $900 — not a monthly figure, but a one-off of real size. The shortest way to settle it is to copy one small volume and read the UsageType in Cost Explorer. I have not done that.

On which account is billed for transfer, two AWS statements disagree. FSx for ONTAP is not a resource type AWS Backup fully manages, so that discrepancy lands directly on this case.

Source	Statement
AWS Backup Developer Guide	For resource types that are not fully managed, transfer appears on the destination account
AWS Backup pricing page	Transfer is charged to the account sending the data (the copy source)

Which one is correct I have not been able to confirm. If cost attribution has to go into the design of a cross-account setup, copy once first and check which account it lands on in the actual bill.

The trade-off of all-tiering

Keeping a standby cheap with all-tiering means reads after a failover start from the capacity pool. All does not pull read blocks back onto SSD, so performance immediately after a cutover is not the same as production sized for SSD. It is a choice between the monthly cost while idle and the performance immediately after cutover.

Prioritise cost while idle → all
Need production-equivalent performance from the moment of cutover → provision SSD as in production (the "all SSD" column above)

When you build your own estimate, replace the change rate and the generation count with measurements from your environment. The 2% is an assumption, not something I measured. Rates are revised, too. The figures above were retrieved on 2026-08-29 and are effective 2026-07-01, so check the current values on the FSx for ONTAP pricing page and the AWS Backup pricing page before you decide.

As a ransomware layer

An isolated copy is a useful layer, but it lacks two properties people expect of it.

Expectation	Reality
Put it in an isolated account and it cannot be deleted	Copies can be deleted. The isolation holds only if the destination account's IAM or vault policy prevents deletion
A backup copy satisfies an immutability requirement	It does not. Preventing deletion during a retention period is the domain of WORM features (Object Lock, SnapLock, Tamperproof Snapshot, Vault Lock)

WORM features are irreversible. Once enabled, nothing is deletable until retention expires, and the blast radius is routinely wider than the resource you named. Get the retention value and the blast radius approved separately before enabling one. I made a file system undeletable for six months this way.

On the other hand, a restore always producing a new volume is an advantage during an incident. You can preserve the compromised volume for investigation and restore into a separate volume to resume operations, without overwriting the original.

Common errors and how to isolate them

Decide which layer stopped before you start debugging. Get the layer wrong and you spend the incident suspecting something that works.

Layer	Look at	What to check
1. Source	The backup	`Lifecycle` is `AVAILABLE`; volume type is `RW`; style is FlexVol
2. Permissions	IAM / KMS	`fsx:CopyBackup` present; source-Region backup reachable; destination key permissions for a CMK
3. Copy	`describe-backups` at the destination	`COPYING`, `AVAILABLE`, or `FAILED`; concurrency limits
4. Restore	The destination volume	`Lifecycle`; enough SSD (it pauses when short)
5. Access	The client	Export policy, security group, SVM NFS / SMB endpoint

Messages I actually saw:

Message	Layer	What to do
`Backup <id> is being copied, can't be deleted`	3	Wait for the copy. There is a short delay even after it completes
`Backup failed. Please delete the backup and try again.`	1	Names no cause. Suspect FlexGroup and take a FlexVol control
`Volume with type DP is not backupable.`	1	That is a SnapMirror destination. Back up at the source instead
Rejection from exceeding copies in flight	3	5 per volume per destination Region per KMS key, 1,000 per account. Wait and retry
`Unknown options: --ontap-configuration` (`delete-file-system`)	—	A CLI version difference. Read the response. This cost me 8 minutes of polling

Whether a failed copy leaves a partial backup at the destination is unverified — I did not induce a failure deliberately. If you automate this, detect FAILED and delete it (the backup created by my FlexGroup failure did remain as FAILED, and it deleted cleanly).

Where this sits alongside an existing backup product or SaaS

If you already run a data protection product or backup SaaS, this update is not a migration question. It is a question of which layer you give to what. There are three paths, and combining them is normal.

Path	Managed by	Another Region	Another account	Fits
A. FSx for ONTAP backups	FSx for ONTAP API / console	yes (new)	no	Volume-level evacuation with the fewest moving parts
B. AWS Backup	Backup plans	yes	yes (needs Organizations)	Policy control, one view across services
C. Existing data protection product / SaaS	That product's policy	Depends on the product	Depends on the product	File- or application-level restore, catalog search, fitting existing workflows

A concrete way to split them

File-level restores from the existing product; whole-volume evacuation from FSx for ONTAP backup copies. One split that often works, and the reason is restore granularity. A restore from an FSx for ONTAP backup produces one new volume, which is a lot of machinery to return a single file a user deleted. Conversely, pushing whole volumes to another Region through an existing product tends to grow the transfer volume and the catalog. Where your product already handles cross-Region evacuation efficiently, the opposite split is equally reasonable.

The decisive question for AWS Backup is whether you need another account. Evacuating to an isolated account is out of reach for the FSx for ONTAP API alone; it needs AWS Backup and AWS Organizations. If you only need another Region, the FSx for ONTAP API is fewer moving parts.

What to ask, regardless of product

Ask along these axes rather than by product name:

Axis	What to establish
Unit of protection	Whole volume, file / directory, or application-consistent
Restore granularity	Can it return one file, or only a whole volume
Cross-Region / cross-account path	The product's own feature, via AWS Backup, or your own copy
What the restore target requires	Does a file system need to exist first
Where retention policy lives	The product or AWS. Is it managed in two places
Monitoring and alerting	Where failures surface, and whether that reaches your existing observability
Where the cost lands	Product licensing, AWS usage, or both

"Managed in two places" is where the accidents are. If FSx for ONTAP automatic backups and the product's policy each hold generations independently, changing retention on one side leaves the other holding an unintended number. The ceiling is 4,091 backups per volume, so plan backwards from it.

Storage is also billed twice. Holding the same data in both the product's repository and FSx for ONTAP backups charges for both sets of generations. If you combine them, pick a primary and make the generation counts differ:

Shape	Primary	Secondary
File-level restore matters most	Existing product, long retention, many generations	FSx for ONTAP backups short and few (recent whole-volume evacuation)
Cross-Region evacuation matters most	FSx for ONTAP backup copies, many generations	Existing product limited to in-Region file restores

Neither is correct in the abstract. It comes down to which granularity 90% of your restore requests arrive in. Counting your past restore tickets is the most reliable way to find out.

About monitoring

My verification polled describe-backups. I have not verified a path for detecting copy completion through CloudWatch metrics or EventBridge events. If you plan to wire this into an existing observability stack, that is a gap you will need to close yourself.

Choosing between this and SnapMirror

Neither is better. The AWS documentation states RPO and RTO figures for both, and they protect different things.

For context: cross-Region disaster recovery for FSx for ONTAP has been buildable with SnapMirror for years — a destination file system, cluster peering, asynchronous replication. Plenty of production environments run exactly that, and backup copies do not displace it. What got added is the option of not holding the destination while nothing is wrong.

Dimension	Backup copies	SnapMirror
Suits	Offline copies for compliance or isolation	Availability in a second Region
RPO guidance	60 minutes	as low as 5 minutes
RTO guidance	minutes to hours (size dependent)	single-digit minutes
Needed while idle	No destination file system	A running destination file system
Shape of recovery	Restore as a new volume. The original does not come back	Break the relationship and promote the destination
Path back to production	None. Continue on the recovery side, or copy back in the other direction	Delete the relationship and `snapmirror resync` from the DR side
Trade-off	Destination file system creation lands on RTO (20 min measured); RPO is as coarse as the backup interval. The unit of backup and restore is the volume, and with AWS Backup / FSx backups a restore always lands in a newly created volume — there is no restore-in-place path, so junction paths, share configuration and where clients point all have to be rebuilt	You pay for destination capacity and throughput continuously; needs cluster peering and does not work through NAT

In operation, the asymmetry in "path back to production" is what bites. SnapMirror can be re-established in reverse to fail back. Backup copies only restore, and what you get is a new volume. If a documented failback procedure is a requirement, backup copies do not substitute for it.

One caveat on snapmirror resync: user-created snapshots are not replicated by a resync. The preserve parameter is available on XDP relationships.

Running both

This is a natural combination: SnapMirror for availability, backup copies for isolated retention. One catch — a SnapMirror destination is a DP volume and cannot be backed up, so "back up the replica and evacuate that" does not work. Take backups at the source and copy those.

Regional DR within one country

Copying to Osaka keeps data inside Japan while placing it in a second Region.

Consideration	Detail
Where the data sits	Both Tokyo and Osaka are in-country
Idle cost	Backup storage only ($0.050/GB-month in Osaka, rate effective 2026-07-01)
Needed at recovery	A file system and SVM in Osaka. Creation time lands on RTO (20 min measured)
Generation availability	`SINGLE_AZ_1` was creatable in Osaka (measured 2026-08-28). This differs by Region
Trade-off	Coarser RPO and RTO than SnapMirror. A minute-level RPO requirement means keeping the destination running

How you satisfy a data residency requirement is a design question, not a legal or regulatory determination. Interpreting regulatory requirements belongs to your compliance function and the relevant authority. This post does not substitute for that.

A staged rollout

Do not start on a production volume. The order I would suggest:

Run the whole path on a small test volume (~1 GiB) — copy, destination file system, restore, content check, teardown. This is where the holes in your procedure show up
Time it including teardown. Destination file system creation lands on RTO, so this is the number you most want
Measure once at production data volume. My 9.4 MiB figures are dominated by fixed overhead and are not evidence for your environment
Check whether the second copy is incremental by comparing transfer volume and duration
Run periodic copies on a non-production volume and build the failure-detection path. This is where the real work is
Go to production, keeping the existing backup path running in parallel
Put restore drills into operations. Without them your RTO is an estimate

Do not go to production without step 5. Copies are asynchronous, and when one fails your command has already returned success.

A verification plan template

Useful to have on one page if a team is involved:

Item	What goes in it	Mine
Question being answered	"Does cross-Region evacuation work without a standing destination?"	same
Success criteria	Content matches after restore, and every line of the RTO breakdown is filled	Hash match achieved. RTO breakdown incomplete (AD and remount unmeasured)
Time required	Half a day for the small path, a day at production volume	~1.5 hours including teardown
Cost ceiling	Driven by destination file system runtime. Set it first	Capped at 3 hours / $2; actual $0.55
Stakeholders	Storage operations, the business (accepting the restored data), audit / control, networking (destination VPC, AD reachability)	—
Go criteria	Measured RTO under target / incrementality confirmed / failure detection working	Not met (incrementality and detection unverified)
No-Go criteria	RPO needed in minutes / failback required / FlexGroup in scope	—

By that standard I cannot call my own run a Go. Incrementality and failure detection are unverified, and the RTO breakdown does not include AD. What this post establishes is that the path works.

A recovery runbook skeleton

Naming the decision points keeps the day itself calm:

#	Decision or action	Branch
1	Is recovery needed (Region-level failure, or data loss)?	Data loss alone is an in-Region restore. You do not need the other Region
2	Which generation?	Check `CreationTime` on the copies. This is also where you see whether copying kept up
3	Create the destination file system	`deploy` the template, or steps 4–5 by hand
4	SMB in play?	Then AD join is required. Put the reachability check here
5	Restore	Wait for `Lifecycle: CREATED`. `OntapVolumeType` reads `DP` while restoring
6	Business validation	Data match and application startup. Databases need a consistency check
7	Return to the original Region?	Backup copies have no failback path. Either continue on the recovery side, or copy back in the other direction

Try it now

The same steps with only the values to substitute:

# Change these five lines for your environment
export SRC_REGION=ap-northeast-1
export DST_REGION=ap-northeast-3
export SRC_VOLUME_ID=fsvol-0123456789abcdef0
export SRC_BACKUP_ID=backup-0123456789abcdef0
export DST_BACKUP_ID=   # fill in after the copy

# 1) create a backup
aws fsx create-backup --region "$SRC_REGION" --volume-id "$SRC_VOLUME_ID" \
  --query 'Backup.BackupId' --output text

# 2) wait for AVAILABLE
aws fsx describe-backups --region "$SRC_REGION" --backup-ids "$SRC_BACKUP_ID" \
  --query 'Backups[0].Lifecycle' --output text

# 3) copy to another Region (--region is the destination, --source-region is the origin)
aws fsx copy-backup --region "$DST_REGION" \
  --source-backup-id "$SRC_BACKUP_ID" --source-region "$SRC_REGION" --copy-tags \
  --query 'Backup.BackupId' --output text

# 4) confirm the copy
aws fsx describe-backups --region "$DST_REGION" --backup-ids "$DST_BACKUP_ID" \
  --query 'Backups[0].{Life:Lifecycle,Src:SourceBackupId,SrcRegion:SourceBackupRegion}' --output table

If step 4 shows SourceBackupId and SourceBackupRegion, the copy is real. That is the whole day-to-day preparation.

Wrapping up

What this update removed is what you keep running day to day. Putting data in a second Region no longer requires a file system there.

You pay for that at recovery time instead — file system creation (20 minutes measured) plus an RPO as coarse as your backup interval. If minute-level RPO and a failback procedure are requirements, this does not replace SnapMirror. If isolated retention is the requirement, you now have an option that needs nothing standing by.

Three things I plan to do next:

Measure cross-account copies. The AWS Backup cross-Region copy and restore are measured now; cross-account needs AWS Organizations and remains documentation only
Measure incrementality at production data volume. At 9.4 MiB it disappears into fixed overhead
Establish a failure-detection path. Whether CloudWatch or EventBridge can carry copy state is unverified

The most useful thing I learned was smaller than the feature: I believed DP was a permanent property until I tried writing to the volume, and I believed storage efficiency was not carried over until I ran the same restore through the console. Two conclusions drawn from a single observation on a single interface, both wrong.

Thanks for reading.

Resources

AWS What's New: FSx for NetApp ONTAP now supports copying backups across AWS Regions and accounts
AWS What's New: AWS Backup adds cross-Region and cross-account backup support
AWS: Copying backups
AWS: Copying backups within the same AWS account — the backup-copy vs SnapMirror guidance and the RPO / RTO figures
AWS: Protecting your data with volume backups — restore target confined to the backup's Region, SSD-first restores, background priority
AWS: Restoring a backup to a new volume
AWS: CreateVolumeFromBackup
AWS CloudFormation: AWS::FSx::Volume — the BackupId property
AWS Backup: Creating backup copies across AWS Regions
AWS Backup: Creating backup copies across AWS accounts
AWS Backup: Encryption for backups in AWS Backup (why AWS managed keys cannot do a cross-account copy)
AWS Backup: Feature availability (the per-Region exceptions)
Cross-Region and Cross-Account Backup for Amazon FSx for NetApp ONTAP (NetApp Community: the cross-account walkthrough, four architecture patterns, clean room and restore testing)
AWS: Quotas (FSx for ONTAP)

Every resource used for this test has been deleted. The figures are measurements from one specific environment and configuration, and will differ with data volume and setup.

Using Databricks FILE type against data on NAS: what works today with FSx for ONTAP S3 Access Points

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Wed, 12 Aug 2026 15:07:29 +0000

A follow-up to Part 2: Databricks and FSx for ONTAP S3 Access Points in the "FSx for ONTAP S3 Access Points x Lakehouse Deep Dive" series. Some results changed since Part 2 — if you read that one, see "Why reads fail" below.

TL;DR

I tested whether images and PDFs already sitting on FSx for ONTAP can be referenced from
Databricks without copying them into the lakehouse. The result is: registration works, reads
do not.

If you only need the zero-copy verdict, read "Connecting to an FSx for ONTAP S3 Access Point"
and "Design tips" below. If you want the behaviour of the FILE type itself, start from the
top.

Databricks shipped FILE as a beta column type: a reference to an unstructured file held as a
single column in a Delta table. The value is a struct carrying uri / offset / size /
content_type / checksum. Because it holds the reference and the metadata as a struct, you
can pass it straight from a table into an AI function such as ai_parse_document. There are
two variants — FILE EXTERNAL, which holds only the reference, and FILE MANAGED, which
copies the bytes into Databricks.

I tested this against files on Amazon FSx for NetApp ONTAP, reached through an FSx for ONTAP S3
Access Point — the feature that exposes an ONTAP volume over an S3-compatible API.

Here is what I found:

Registration against an FSx for ONTAP S3 AP succeeds; reads do not. The cause sits in the session policy attached to the temporary credentials Unity Catalog vends, and there is no way to work around it from the user side
The FILE type itself works. But you have to choose between FILE EXTERNAL and FILE MANAGED before you ingest
If your design puts metadata in object tags, tag values are effectively ASCII. Most CJK strings are rejected while a minority are accepted. AWS Support has escalated that split to the service team as a potential defect
The operator surface is inconsistent. GROUP BY and DISTINCT are accepted; = and ORDER BY are rejected

Below: what is established for each of those four, and how to turn it into design decisions.
The templates and scripts are in the repository, so you can re-run the same tests in your own
account.

Repository: Yoshiki0705/fsxn-lakehouse-integrations

Test environment

Item	Value
Measured	2026-08-12
Storage	One FSx for ONTAP file system
Access Point	One, INTERNET origin, UNIX root identity
Workspace	One non-trial workspace, purpose-built, same account and region
Compute	Serverless SQL warehouse, and a classic DBR 18.2 cluster
Control	A S3 general bucket in the same account, running the same operations alongside
Method	SQL Statement Execution API, and boto3 from my workstation

Running the S3 general bucket control alongside matters. Looking only at the Access Point side, you
cannot separate a defect in your own environment from behaviour specific to the Access Point.
Except where stated otherwise, every conclusion below rests on the same operation succeeding
against S3 general bucket.

Note that the object tag results depend on the ONTAP tag validation implementation, so a
different ONTAP version may behave differently. The FSx console and describe-file-systems do
not expose the version; use the ONTAP REST API (GET /api/cluster?fields=version) to check
yours.

What the FILE type can and cannot do

First, the behaviour of the type itself. There is prior work on this feature from Databricks
Japan: パス文字列でもバイナリでもない。DatabricksのFILE型を試す
by taka_yayoi. Every operational caveat it lists reproduced in my environment, so I have
tabulated those first. (What follows is my paraphrase and my own measurements, not a
translation.)

Some terms first. FILE MANAGED copies bytes into a Databricks-side location called a
FileSpace. list_files enumerates files on a Volume and returns FILE values; create_file
constructs a FILE value explicitly. _object_metadata reads an object's tags and user
metadata.

Behaviour	Measured	Design implication
FileSpace on the same volume as the source	`CREATE TABLE` succeeds; `INSERT` fails with `Cannot get file metadata under managed storage`	Give the FileSpace its own volume
`checksum` on FILE values from `list_files`	Null on every row	Not usable for integrity checks
`checksum` under `FILE MANAGED`	Populated with `ETAG:"…"`, matching the `etag` from `_object_metadata` on S3 general bucket exactly as a string	Comes from the object store, so it is usable for reconciliation
Filenames under `FILE MANAGED`	Replaced with an opaque UUID; the extension does not survive either	Copy the name to a column before ingesting
Format of `uri`	Prefixed with `dbfs:`	Normalise before writing string comparisons
Serverless notebooks	Unsupported	Use the SQL Statement Execution API against a serverless SQL warehouse
Automatic garbage collection in beta	Does not run	The FileSpace grows on every re-run. Plan for manual deletion
The Previews page toggle	Used `FILE EXTERNAL` / `FILE MANAGED` without touching it (confirmed on 2 workspaces)	May not be required. Do not assume either way
`content_type` from `create_file`	`binary/octet-stream` for both Japanese and ASCII	Pass it explicitly if anything downstream branches on it

What this tells us

The difference between FILE EXTERNAL and FILE MANAGED is not only whether bytes are copied.
Whether checksum is populated and whether the filename survives both change. Reversing the
decision later means re-ingesting, so decide before you ingest.

On content_type, I got binary/octet-stream for both Japanese and ASCII input. For the same
files, list_files reported text/plain, so the two paths infer differently. If anything
downstream branches on content_type, pass it explicitly.

The state of operator support

Both the documentation and the prior article state that FILE columns cannot be used in grouping
expressions. Testing it, support splits more finely than that.

Operation	Result
`GROUP BY file`	accepted
`SELECT DISTINCT file`	accepted
`GROUP BY file.uri`	accepted (the documented approach)
`d.file = r.attachment`	rejected — The `=` does not support ordering on type "FILE"
`ORDER BY file`	rejected — The `sortorder` does not support ordering on type "FILE EXTERNAL"

What this tells us

GROUP BY and DISTINCT both need equality semantics to decide group membership. Both are
accepted while the equality operator itself is explicitly rejected. Since the error names the
type's operator surface, this does not look like row-count-dependent behaviour.

That said, this was confirmed on two rows. I cannot rule out that grouping degrades to identity
comparison rather than value equality. In practice, use GROUP BY file.uri and do not depend on
GROUP BY file being accepted. This is beta, so the behaviour may change.

Connecting to an FSx for ONTAP S3 Access Point

This is the main question. The answer is "registration works, reads do not", and the cause is
not a lack of Access Point support.

The four things registration needs

Item	Required form	Symptom when wrong
External ID in the trust policy	The Databricks account UUID	`403 Forbidden` from the storage provider. The metastore ID and the workspace ID do not work
Self-assume in the trust policy	Account root as the principal, with an `aws:PrincipalArn` condition naming the role ARN	Naming the role as its own principal fails at creation with `Invalid principal`, because IAM validates that the principal exists. Unity Catalog requires the role to be able to assume itself
Resource in the IAM permission policy	The access point ARN (`arn:aws:s3:<region>:<account>:accesspoint/<name>` and `.../object/*`)	`AccessDeniedException`. The alias-as-bucket-name form alone does not work
External location URL	The alias form (`s3://<alias>/`)	An ARN-style URL is rejected immediately with `url does not specify a valid bucket name`

What this tells us

There is an asymmetry worth committing to memory: the same Access Point must be written as an
ARN in the IAM policy and as an alias in the external location URL. The AWS CLI works with the
alias form of the ARN, so verifying connectivity with the CLI first and then writing the IAM
policy will trip you up.

The aws:PrincipalArn approach works because that key resolves to the role ARN rather than the
assumed-role session ARN. Rather than trust my memory of the condition key, I deployed the
template, assumed the role, and assumed it again with those credentials to confirm.

With all four satisfied, CREATE EXTERNAL LOCATION succeeds with skip_validation=False —
meaning Unity Catalog's own validation passed. The external volume on top can be created too.

Why reads fail

After registration, every read path returns 403 or an authorisation error. The same on
serverless SQL and on a classic DBR 18.2 cluster.

This is the test that isolated the cause. Unity Catalog will vend the temporary credentials it
uses if you ask for them. I requested them for both paths and used them from my workstation,
with no Databricks compute or network involved.

Vended for	`ListObjectsV2`	`HeadObject`
S3 general bucket control path	success	success
FSx for ONTAP S3 AP path	AccessDenied	403

Same role, same session, same network; the only variable is which path the credentials were
scoped to. The error states the cause explicitly:

is not authorized to perform: s3:ListBucket on resource:
"arn:aws:s3:<region>:<account>:accesspoint/<name>"
because no session policy allows the s3:ListBucket action

What this tells us

because no session policy allows is the decisive part. What is named is the session policy,
not the role's permissions. By session policy I mean the down-scoped policy passed to
AssumeRole to constrain the role's permissions on a per-request basis.

The policy body is generated by Unity Catalog and is not readable by the user. But since adding
an access-point-ARN allowance to the role changes nothing, the session policy appears not to
mention the access point ARN resource form at all. Only the overlap between the session policy
and the role policy is effective, so additions on the role side have no effect.

There is no workaround on the user side. Unity Catalog generates the policy, so the fix has
to happen there. Emitting the access point ARN form in the session policy should resolve it,
though I do not know what implementation constraints apply.

Note also that validation at registration time (skip_validation=False) passes while reads
return 403. What that validation checks is not published, so I cannot account for the
difference.

What changed since Part 2

Part 2 of this series (measured 2026-05) reported that listing top-level files and reading an
explicit file did work through a UC external location. In this run (2026-08-12), read_files,
list_files, to_file and dbutils.fs.ls all returned 403; no read path partially succeeded.

That environment is gone, so I cannot account for the difference conclusively. The likely
explanation is that the earlier partial successes were observed on an instance-profile path
rather than through UC-vended credentials — Part 2 lists instance-profile direct access as a
separate approach that works but sits outside UC governance, so the paths may not have been
cleanly separated.

This run had a S3 general bucket control alongside and includes the vended-credential test outside
Databricks entirely, so take these results as the current ones. I will annotate the Part 2
table accordingly.

This test has prerequisites. Enable External Data Access on the metastore and grant
EXTERNAL USE SCHEMA and EXTERNAL USE LOCATION. Both are disabled by default. They are the
controls that allow credentials to be used externally, so they are worth understanding
independently of this issue.

A table to work through when you see a 403

Arranged so that anyone hitting the same 403 can check top to bottom. None of these is the
cause.

What to suspect	Finding
Unity Catalog volume privileges	The failing volume had `READ VOLUME`; the working control volume had no privileges at all
IAM role permissions	Assumed the same role from my workstation and listed successfully. HEAD on a real object returned 200
HEAD on a missing key returning 403 (killing the `_delta_log` probe)	The Access Point returned 404 for every shape I tried — same as S3 general bucket
`x-amz-expected-bucket-owner` being rejected	Accepted by both the Access Point and the control bucket
A different endpoint	Both Databricks and the AWS CLI use `<alias>.s3.<region>.amazonaws.com`, which CNAMEs to `s3-r-w.<region>.amazonaws.com`. Same host
Insufficient compute role permissions	`sts:GetCallerIdentity` raised `NoCredentialsError` on the driver. The cluster has no default AWS credentials; the compute role is not in the path
The S3 gateway endpoint	No change after disassociating the route. The failure reproduces outside the VPC with vended credentials, so this path is not the problem

One note on the last row. A Databricks-managed VPC creates an S3 gateway endpoint at workspace
creation, and the private route tables point the S3 managed prefix list at it. That was not the
cause here, but for designs that reach an S3 AP from inside a VPC, check first whether the
alias resolves into the prefix list's range. I did not reconcile the resolved CIDRs against the
prefix list contents in this environment.

What object tags can actually hold

If you want metadata to live with the file, object tags are one option: tag at write time, read
from the table side later. That path has constraints.

The following was measured without Databricks in the picture — boto3 from my workstation
straight against the FSx for ONTAP S3 Access Point, with the same operations running against
the S3 general bucket control bucket.

Start with what works. PutObjectTagging, GetObjectTagging and DeleteObjectTagging all
function, as do x-amz-meta-* headers and x-amz-tagging on the same PutObject as the data.
The limits (10 tags per object, 128 characters for a key, 256 for a value) are the same values
S3 general bucket documents; I did not measure the boundaries.

Reading through a second access point on the same volume returned tags written through the
first. Tag retention is not scoped per access point. For existing NAS assets shared across
several paths, that property works in your favour. But I did not observe where the tags are
physically stored, so visibility from NFS or SMB, and retention across SnapMirror, FlexClone
and Snapshot restore, remain unverified.

Two constraints change how you design.

An object overwrite clears tags and user metadata. Nothing errors. A pipeline that rewrites
a file has to re-apply them in the same PutObject. My understanding is that S3 general bucket behaves
the same way, since PutObject replaces the object, so treat this as a general S3 design point
rather than something specific to FSx for ONTAP. I did not run a side-by-side control for this
particular case.

Tag values are effectively ASCII. Every printable Latin-1 character I tried
(U+00A1–U+00FF) was accepted. Above U+0100, most are rejected with InvalidTag — but not all.
A minority of CJK strings are accepted, deterministically, 6 runs out of 6.

To find the unit of validation, I tested single characters:

Input	Result
U+5206	`InvalidTag`
U+985E	`InvalidTag`
U+5206 U+985E	accepted
U+6771	`InvalidTag`
U+4EAC	`InvalidTag`
U+6771 U+4EAC	`InvalidTag`

What this tells us

Two two-character strings, every constituent character rejected on its own, and the two strings
disagree with each other. Validity is determined by the complete byte sequence, not as a
function of the characters in it — so this is not a per-character allowlist.

For the two-character pairs I tested, results were identical forwards and backwards, and
identical as a tag key and as a tag value. I did not test longer strings or other scripts.

I raised this with AWS Support. After checking it against the S3 and FSx for ONTAP
documentation, they gave the view that the pattern does not match any intentional validation
they could identify, and escalated it to the service team as a potential defect in the tag
validation layer.

Their reply also quoted the documented character set: letters, whitespace, and
+ - = . _ : / @. That does not predict the split under either reading. If "letters" means
Unicode letters, every string I tested qualifies yet half are rejected. If it means ASCII
letters, all of them should be rejected yet half are accepted.

The design guidance is simple: keep tags ASCII and keep localised text in a column. Today
the situation is "some strings pass and some do not", which is more awkward to handle than a
uniform restriction. I will update the repository and this article when a determination
arrives.

Design tips

The same material, ordered the way you actually make the decisions.

FILE type

Decide between FILE EXTERNAL and FILE MANAGED before ingesting. Checksum and filename handling differ, and changing course later means re-ingesting
Give the FileSpace its own volume. On the same volume, INSERT fails — and because the error surfaces at ingest rather than at configuration time, diagnosis is slower
Copy the filename to its own column before ingesting into FILE MANAGED. It is replaced with a UUID and the extension does not survive
Do not rely on content_type inference; pass it explicitly
Group by file.uri. Do not depend on GROUP BY file being accepted
Automatic garbage collection does not run in beta. Build manual FileSpace deletion into operations

Unity Catalog and FSx for ONTAP S3 Access Points

The external ID in the trust policy is the Databricks account UUID. The metastore ID does not work
The trust policy needs self-assume: account root as the principal with an aws:PrincipalArn condition naming the role ARN
The IAM permission policy needs the access point ARN, including .../object/*. The alias-form ARN alone does not work
The external location URL takes the alias form. ARN-style URLs are rejected
Reads do not work today. If zero-copy is a requirement, evaluate AWS-native engines that authorise in the caller's own IAM context alongside it (this repository has walkthroughs under integrations/athena, integrations/glue, integrations/emr-spark, integrations/redshift-spectrum and integrations/lake-formation)

Object tags

Keep tags ASCII and keep localised text in a column
Re-apply tags in the same PutObject when you rewrite a file. An overwrite clears them
Tag retention is not scoped per access point, so the same tags are readable through several access points

How to test

Run a S3 general bucket control alongside. Without it you cannot separate a defect in your environment from Access Point behaviour, and the first 403 looks like the answer
If the platform exposes credential vending, use those credentials from your own machine and compare. Isolating the problem is much faster without the vendor's compute in the path
Verify teardown by reconciling against a baseline. NAT Gateways do not disappear when you delete the workspace

Run it yourself

All of the above is in the repository as templates and scripts: one template, one script, one
runbook.

Artefact	Role
`integrations/databricks/uc-storage-credential-role.yaml`	The IAM role the Unity Catalog storage credential assumes, plus a S3 general bucket control bucket
`shared/scripts/probe_uc_external_location.py`	Seeds identical object tags on both sides, registers both external locations with validation on, reads `_object_metadata` through each, prints a verdict
`docs/en/databricks-verification-runbook.md`	Prerequisites, where each parameter comes from, the verdict branches, dependency-ordered teardown, measured cost

# 1. Baseline first. This is what makes "teardown looks clean" verifiable
python3 shared/scripts/audit_databricks_workspace_footprint.py \
  --region <region> --save /tmp/baseline.json

# 2. The IAM role and control bucket
aws cloudformation deploy --region <region> \
  --stack-name fsxn-databricks-uc-credential \
  --template-file integrations/databricks/uc-storage-credential-role.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides file://cfn-params/databricks-uc-storage-credential.json

# 3. Compare. --control-bucket is deliberately mandatory
.venv/bin/python shared/scripts/probe_uc_external_location.py \
  --profile <your-profile> --role-arn <from stack output> \
  --ap-alias <alias>-ext-s3alias --ap-name <name> \
  --control-bucket <from stack output> --region <region> --vend-check

# 4. Tear down and reconcile against the baseline
python3 shared/scripts/audit_databricks_workspace_footprint.py \
  --region <region> --compare /tmp/baseline.json

The script prints one of four verdicts. The first is the important one.

Inconclusive — the S3 general bucket control could not be read either, so this run says nothing about the Access Point. Fix the environment first
Registration succeeded, reads denied — the result in this article
Registration and reads both succeeded — the platform changed. Please tell me which release
Registration itself failed — almost certainly a missing Access Point ARN in the IAM policy

One note on the parameter file: a _comment key in a CloudFormation parameter file makes the
CLI fail with Unknown parameter in Parameters[n]. The description of each parameter lives in
cfn-params/README.md instead, so fill in values with that open alongside.

If you are evaluating a different platform

The specific bug is Databricks'. The shape is not, and the shape is the part worth taking away
if you are looking at a different product.

Managed lakehouse platforms generally reach into your cloud account by assuming a role you
create, then constrain that role at runtime with a down-scoped session policy so that
credentials handed to a query cannot roam. That is good design.

But the policy has to name your storage. If the storage is addressed through an access point
while the policy names it in bucket-form ARNs, the two do not match and authorisation fails —
however correct your own role policy is.

So the question to put to a platform you are evaluating, before you commit to an architecture,
can be narrow and answerable:

When you vend temporary credentials for an external location, which resource ARN forms does
the session policy contain? Specifically, does it include the S3 access point ARN form in
addition to the bucket form?

And if the platform exposes credential vending, you can verify it without asking anyone:
request the credentials, use them from your own machine, and compare a native bucket path
against an access point path. That is what --vend-check above does, and it was the only test
in this exercise that produced a conclusion with no room for interpretation.

Cost and cleanup

I created a test workspace and destroyed it the same day. The workspace mode that uses your own
AWS account creates a VPC with a NAT Gateway. A NAT Gateway runs roughly 45 USD a month per
gateway (hourly charge only, excluding data processing, at the August 2026 Tokyo region rate).

These resources are created directly rather than through a CloudFormation stack, so deleting
the workspace does not remove them. This is the easiest place to make a foolish mistake during
cleanup.

I recorded a baseline before starting and reconciled afterwards. Available NAT Gateways back to
zero; VPC, both IAM roles and the bucket gone, with no unassociated Elastic IPs.

I turned that check into a script (shared/scripts/audit_databricks_workspace_footprint.py). In
a shared account a plain listing is not a verdict, because most of what it shows belongs to
someone else. Run it once before creating anything and once after teardown; it exits non-zero if
anything exists that was not in the baseline. Running it confirmed this teardown was clean and
also surfaced leftover roles from an earlier workspace. The cost breakdown is in the runbook with
measured figures.

Current state and open items

Established

CREATE STORAGE CREDENTIAL / CREATE EXTERNAL LOCATION / CREATE EXTERNAL VOLUME succeed against an FSx for ONTAP S3 AP alias, with Unity Catalog validation left on
Reads through it return 403, on both serverless SQL and a classic DBR 18.2 cluster
The same failure reproduces outside Databricks, using vended credentials from a workstation
_object_metadata on S3 general bucket returns object tags and user metadata correctly
Object tag acceptance is determined by the complete byte sequence, and within the range tested is neither order- nor position-dependent

Not established

VPC-origin Access Points, and Access Points with WINDOWS identity
Whether _object_metadata would read tags through an Access Point if the session policy were fixed
ai_parse_document over files served from an Access Point
Throughput and latency of these paths
Whether GROUP BY on a FILE column is correct at scale, or merely accepted
Where object tags are physically stored; visibility from NFS or SMB; retention across SnapMirror, FlexClone, Snapshot restore and FabricPool tiering
The boundary values for tag limits (10 tags, 128-character keys, 256-character values)

Reporting status

The object tag behaviour has been escalated through AWS Support to the FSx for ONTAP service
team and is awaiting a determination. What I asked for is not a timeline but the distinction:
a defect that will be fixed, or an undocumented restriction that gets written down. Either
answer lets me state the constraint accurately.

The session policy issue and the operator inconsistency go to Databricks when this article
publishes.

This repository carried an entry from May 2026 saying Unity Catalog external locations do not
support S3 Access Points. Registration does work; the entry has been corrected. If you read the
earlier version, please note the change.

Closing

The FILE type is a well-built mechanism for bringing unstructured data under table governance.
For referencing files on an FSx for ONTAP S3 Access Point with zero copies, though, it does not
get there today. Registration works, so you can try it, but reads fail, and because the cause
sits in the session policy on the vending side, users cannot fix it.

If zero-copy is the requirement, AWS-native engines that authorise in the caller's own IAM
context are the more straightforward path for now — in exchange for assembling table-level
governance yourself with Lake Formation. If Unity Catalog governance is the requirement, then
for the time being data placement has to be part of the design.

The object tag character set is awaiting a determination. When it moves, I will update both the
article and the repository.

I hope this is useful to someone.

See you next time.

Previous: Part 2: Databricks and FSx for ONTAP S3 Access Points
Next: Part 8: OpenSharing and FSx for ONTAP
Series index: What works, what does not, and why

About this article

This series is based on verification in the fsxn-lakehouse-integrations repository
Author: Yoshiki Fujiwara
Affiliation: NetApp G.K. Japan
Role: Senior Cloud Solutions Architect - AWS
Topic: Amazon FSx for NetApp ONTAP / Hybrid Cloud / Generative AI Data Architecture
Level: 200-300

All test environments have been deleted. The behaviours described are measurements from
2026-08-12 in one specific configuration and may change with platform updates. Beta features are
involved, so please check against current documentation before making production decisions.

S3 Burst on ONTAP Files — Collect via S3 API, Consume via FlexCache NFS/SMB

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 10 Aug 2026 01:06:17 +0000

Introduction

You have an S3 ingestion pipeline. But the consumers only see an NFS mount. Sound familiar?

You want to deliver driving logs to a HiL test bench, stage design jobs for an EDA toolchain, push rendering assets to production nodes, or feed genomic sequencer output to an HPC cluster. In each case the writer speaks S3 API in the cloud while the reader speaks NFS/SMB on physical hardware. Without a way to connect these directly, you end up adding a sync job in between — and with it comes latency, cost, and one more thing to monitor.

This structure spans industries: automotive, semiconductor, media/VFX, oil and gas, life sciences, manufacturing, remote work, and IoT (see the full table in When This Pattern Fits below).

This post introduces a pattern that removes that sync job. Write to an Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP) S3 Access Point, and the data appears on a FlexCache NFS mount. No separate copy job and no scheduled replication between the two. FlexCache pulls the data that gets read from the origin and holds it on the cache side, so what this removes is the sync job you would otherwise own — not the data transfer itself. I measured all four directions to see how fast the visibility propagates.

Here's the conclusion up front:

S3 PutObject → FlexCache NFS read visibility: p50 8 ms. No separate copy or replication job; the data that gets read is pulled into the cache on demand
FlexCache overhead is +5 ms (vs. reading origin directly) — nearly transparent for same-region VPC peering
Reverse direction (NFS write → S3 AP read) is p50 44 ms, dominated by S3 API overhead. Not the main path in this design
FlexCache duality (NAS bucket S3 reads on FlexCache volumes) works, but only after -is-s3-enabled true is set on the cache volume in advanced privilege. My first attempt returned AccessDenied because that step was missing

Repository: Yoshiki0705/s3-burst-on-ontap-files

The Architecture

[S3 Client] --PutObject--> [S3 Access Point] --> [Origin Volume]
                                                       |
                                                  FlexCache
                                                       |
                                                [NFS/SMB Client]

Layer	What	Protocol
Collect (write)	S3 Access Point on FSx for ONTAP	S3 API
Source of truth	Origin volume	—
Distribution	FlexCache	Cluster/SVM peering
Consume (read)	Cache volumes	NFS/SMB only

The S3 Access Point attaches to the origin only. Cache volumes serve NFS/SMB. This single decision simplifies the design considerably:

One write path. The origin is authoritative; writes always go through the S3 AP
No S3 implementation differences pushed to the edge
Cache side requires only FlexCache + NFS/SMB — fewer platform constraints to hit

Test Environment

Item	Value
Date	2026-08-09
ONTAP version	9.18.1P3D1 (both clusters)
Configuration	SINGLE_AZ_1, 128 MBps × 2 clusters
Connectivity	VPC peering (same region, same account)
Mount	NFSv3, `actimeo=0` (client cache disabled)
Object size	64 bytes
Concurrency	1
Method	boto3 persistent session, same host, 30 iterations

Why NFS First

The initial measurements used NFS. Three reasons:

Client cache control. NFS actimeo=0 fully disables kernel attribute caching, isolating storage-side visibility latency. SMB oplocks/leases let the client cache independently, making it harder to measure the same thing at the same granularity.
Matching the identity to the security style. An access point authorizes every request as one file system identity, and AWS says to use a UNIX identity for UNIX security-style volumes and a Windows identity for NTFS ones (managing data access). NTFS volumes are supported too — this is not a UNIX-only feature, which corrects how an earlier version of this post put it. UNIX plus NFS was chosen here because it was the shortest path to a measurement, not because the other side is unavailable.
Primary use-case distribution. HiL test benches, rendering farms, and IoT analysis appliances are predominantly Linux + NFS. When the read side is SMB-dominant (Windows workstations in a production studio), the origin security style changes to NTFS — a different design path (see First Decisions).

SMB Shows Identical Results

I ran a separate verification with SMB (mount -t cifs, cache=none) on the same architecture:

Protocol	Mount method	p50	p90	max	n
SMB	`mount -t cifs`, `cache=none`	7 ms	8 ms	9 ms	30
NFS	`mount -t nfs`, `actimeo=0`	7 ms	8 ms	15 ms	30

With persistent mounts, SMB and NFS have identical visibility latency. No protocol difference.

Note: smbclient (establishing a new session per request) showed p50 43 ms. This is SMB session setup overhead — the same structural problem as the AWS CLI cold-start that inflated our first NFS→S3 measurement to 873 ms. Production environments with persistent connections won't see this.

SMB Considerations

FlexCache serves both NFS and SMB. This architecture doesn't exclude SMB — but note:

With a UNIX security-style origin, SMB clients receive UNIX permission-based access control, not NTFS ACLs
For SMB-primary workloads, NTFS security style is the natural choice, and an access point works there with a Windows identity. The trade-off is not availability but the identity you fix on the access point, which cannot be changed after creation
mixed security style is available in the API but officially not recommended by AWS — it's labeled "advanced users only." Permission type is determined by the last client that wrote, making troubleshooting difficult. This architecture doesn't use it
SMB needs a CIFS server on the SVM. An Active Directory join is not required: where a domain is not available, AWS documents setting up an SMB server in a workgroup (NTLM only, no Kerberos, and no GPO, VSS or SMB3 CA shares). If you do join AD, every data operation through the access point then needs a reachable domain controller, and HeadBucket succeeds even when it is not — so it cannot be used to check that

What I used here: a UNIX security-style origin with a UNIX identity, because the read side was NFS. For an SMB-primary workload the equivalent choice is an NTFS origin with a Windows identity. Decide it before the origin volume exists — whether the cache inherits the security style from the origin is unconfirmed on the on-premises path, and if it does, changing it later means rebuilding the serve layer.

actimeo=0 measures the minimum visibility latency by disabling client-side caching. Production deployments should use appropriate actimeo values for their workload — the default (~60 s) means subsequent reads hit kernel cache at ~0.05 ms, but changes aren't visible during that window.

Results — All Four Directions

#	Direction	p50	p90	p99	max
1	S3 AP PutObject → FlexCache NFS read	8 ms	9 ms	19 ms	19 ms
2	S3 AP PutObject → Origin NFS read (direct)	3 ms	5 ms	8 ms	8 ms
3	NFS write (Origin) → FlexCache NFS read	6 ms	7 ms	25 ms	25 ms
4	NFS write (Origin) → S3 AP GetObject	44 ms	49 ms	328 ms	328 ms

What the Numbers Say

Direction 1 is the main path. S3 write, FlexCache NFS read: p50 8 ms.

The +5 ms gap between directions 1 and 2 is FlexCache overhead. For same-region VPC peering, FlexCache is nearly transparent.

Direction 3 is faster than 1 (6 ms < 8 ms). NFS writes commit directly to the origin — no S3 API overhead — and FlexCache propagation alone is faster than the full S3-to-cache path.

Direction 4 (reverse) is slowest at 44 ms. S3 read-side processing dominates. This design keeps reads on NFS/SMB, so direction 4 isn't the main path.

NFS Client Cache Effect

Condition	p50
`actimeo=0` (cache disabled)	7 ms
`actimeo=60` (subsequent access)	0.05 ms (kernel cache hit)

With defaults, reads within 60 seconds of the last attribute check hit the kernel cache. The tradeoff: you won't see changes during that window. Tune per workload.

FlexCache Duality — It Works (With One Extra Step)

S3 access to NAS FlexCache volumes — "duality" — arrives in ONTAP 9.18.1, not 9.14.1 as an earlier version of this post said. I tested it on FSx for ONTAP 9.18.1P3D1.

Result: Works After Enabling `-is-s3-enabled true`

My initial test returned AccessDenied and I concluded it didn't work. After additional investigation, I found missing configuration step: S3 access must be explicitly enabled on the FlexCache volume:

set -privilege advanced
flexcache config modify -vserver <svm> -volume <fcache_vol> -is-s3-enabled true

After applying this setting:

Operation	Before (missing setting)	After (-is-s3-enabled true)
HeadBucket	✅	✅
ListObjectsV2	❌ AccessDenied	✅ KeyCount=1
GetObject	❌ AccessDenied	✅ Content verified

The fsxadmin role on FSx for ONTAP has access to advanced privilege commands including flexcache config modify. Documented at NetApp: Enable FlexCache duality.

Design Implication

FlexCache duality works, which means S3 reads from the cache side are possible. However, this architecture still recommends NFS/SMB on the cache side because:

ONTAP native S3 (NAS buckets) and AWS-managed S3 Access Points are different mechanisms
NAS buckets are read-only (no PutObject)
Requires additional configuration (advanced privilege + S3 user management)
No IAM integration or access point policy governance like S3 AP provides

FlexCache duality becomes an option when "S3 reads at the cache site" is a hard requirement. For AWS service integration (Lambda, Bedrock, etc.), the origin-side S3 AP remains the better fit. These are separate mechanisms, and the support status of one is not evidence for the other.

When This Pattern Fits

"Collect via S3 API in the cloud, consume via NFS/SMB at the edge" — this structure exists across industries.

Industry	Collect side	Consume side	Example
Automotive (AV/ADAS)	Driving logs and sensor data ingested to S3	HiL test benches replay via NFS	AWS + NetApp: Hybrid Cloud HiL
Semiconductor (EDA)	Design job I/O staged via S3	Toolchains (Synopsys, Cadence) run on NFS	EDA Scale with FSx for ONTAP
Media and VFX	Rendering assets collected to S3	Artist workstations mount SMB/NFS	FlexCache: distributed product development
Oil and Gas	Seismic survey data uploaded to S3	Interpretation workstations mount NFS	VDI for Subsurface O&G
Life Sciences	Genome sequencer output stored in S3	Bioinformatics HPC processes via NFS	Sequencer → S3 → NFS pipeline
Manufacturing / QA	Inspection camera images collected to S3	Line-side inspection software reads via NFS	Image → judgment → archive flow
Remote Work	Central design data updated via S3	Remote-site WorkSpaces access FlexCache NFS/SMB	FlexCache in AWS WorkSpaces
IoT / Edge	Sensor data streamed to S3	On-site analysis appliances read via NFS	Factory gateway → cloud → shopfloor

Common structure: Few write sites (often one), multiple read sites. Writes are bursty; reads touch only what's needed.

When It Doesn't Fit

Need full S3 semantics (versioning, event notifications, lifecycle): Use S3 natively
Object names aren't NAS-friendly (flat namespace, millions of keys with no directory separators): Performance degrades as root directory grows
Consumers need S3 reads at the cache site: possible from ONTAP 9.18.1 with duality (see above), but this architecture keeps the cache on NFS/SMB — the NAS bucket is a read-only view, needs advanced privilege, and has no IAM or access point policy governance
Need write-back from the cache: This pattern keeps cache read-centric
Need conditional writes (If-None-Match): Returns 501 NotImplemented. Handle exclusion at the application layer

S3 Access Point Design Notes

A few things to know when using the S3 AP as the collect layer.

S3 AP ≠ Amazon S3

The FSx for ONTAP S3 AP supports a subset of S3 operations. GetObject, PutObject, ListObjectsV2, HeadObject, DeleteObject, and MultipartUpload work. The following do not:

S3 Event Notifications (poll, or read the ONTAP native audit log. FPolicy + EventBridge is not a substitute — see the next section)
Lifecycle rules (use FabricPool)
Versioning (use ONTAP Snapshots)
Conditional writes If-None-Match (returns 501)
S3 Select, SSE-S3/KMS, Cross-AP Copy

Monitoring Coverage — FPolicy Does Not See This Path

An earlier version of this post offered FPolicy as the substitute for S3 Event Notifications.
I measured it, and it does not work. Correcting that here.

Measured 2026-08-26, ap-northeast-1, ONTAP 9.18.1P3D1, with the same result for both a UNIX-identity
and a WINDOWS-identity access point.

Mechanism	Operations through the S3 AP	Detail
FPolicy	Not notified	Zero notifications in a 90-second idle window, zero across nine S3 AP data-plane calls, while a file-protocol control on the same volume in the same session did fire. The protocols an FPolicy event accepts are `cifs`, `nfsv3` and `nfsv4` only — there is no value for the S3 path
FPolicy `mandatory`	Not blocked	With a synchronous engine and `mandatory=true`, an NFSv3 write returned `Permission denied` while PUT, GET, LIST and DELETE through the S3 AP on the same volume all succeeded
ONTAP native audit log	Recorded	As `Source=HTTP` for object operations and `Source=S3` for LIST. But `SubjectUserName` and `SubjectDomainName` are `Not Present` and `SubjectIP` is an AWS service-side address, so the requester is not recorded. `HeadObject` produced nothing across six calls. An audit ACE (SACL) is required
ARP	Detected	ARP 5.0. 150 high-entropy files written through the access point were recorded as suspect under `High Entropy`, with `attack_probability` at `moderate`

Two things follow for the design. First, detection, DLP and blocking that start from an FPolicy
notification do not reach this path — including the guarantee mandatory is chosen for, that an
unreachable engine stops the operation. Second, the audit log records the operation but not the
requester, so answering "who touched this file" from the audit trail alone is not possible; it needs
correlation with AWS CloudTrail.

ARP does detect, but attack_probability changes more than ten minutes after the write. Reading
none from a short observation window and concluding nothing was detected is a false negative. ARP
blocking is unmeasured.

Whether FPolicy, auditing or ARP fire on the cache side is unverified. Writes in this architecture
land on the origin, so the cache side remains a separate question.

Throughput is shared with NFS/SMB

S3 AP, NFS, and SMB all consume the same FSx for ONTAP provisioned throughput. In this architecture, origin and cache are separate clusters so this rarely matters — but if NFS clients also access the origin directly, account for the shared bandwidth. I have not measured concurrency, so I will not hand you a number: the ceiling follows from provisioned throughput divided by the bandwidth one request consumes, and that second term depends on your object size and request duration. Raise concurrency while watching the SlowDown (503) rate and p99, and stop below what you can absorb.

Directory design matters

S3 PutObject keys map directly to directory structure. What sets the ceiling is the volume's maxdir-size: reach it and the client gets ENOSPC and can no longer create files. It is a per-volume setting, and raising it could affect performance. Check the value on your own volume and partition by date, tenant, or hash prefix so the entries in one directory stay well short of it. Response time for readdir and ListObjectsV2 grows with entry count too, though I have not measured where that starts to hurt.

This directly affects NFS usability. If you flat-dump millions of objects without hierarchy, FlexCache NFS clients will struggle with ls and find. Design your S3 keys with "how does this look when I ls on NFS?" in mind.

Key recommendations:

Date-based partitioning (year=YYYY/month=MM/day=DD/) so the entries in one directory stay well short of maxdir-size
Separate ingest volumes from consumption volumes; apply FlexCache only to consumption
On the NFS side, use manifest files or path generation instead of directory traversal (find)

For full details, see the S3 AP Design Guide in the repository.

What It Costs — Egress First, Requests Second

I framed this section around S3 request pricing when I first wrote it. That was the wrong axis, and the modelling I did afterwards says so plainly. Corrected below.

The cost that hurts when your readers sit outside AWS is data transfer. Egress is charged on bytes leaving the Region, so reading the same file ten times moves ten times the bytes and pays ten times over. A cache removes the multiplier: it carries the working set once and every later read is served locally over NFS or SMB, never becoming an S3 request at all.

A worked case — 20 TiB dataset, 2 TiB monthly working set, 4 MiB objects, each file read 30 times, internet egress from ap-northeast-1:

Option	Monthly
Read S3 directly from on premises	$6,211 — egress alone is $5,693 (92%)
Copy the whole dataset down with DataSync	$2,847
FSx for ONTAP + FlexCache	$1,333

Storage and requests are rounding errors next to transfer here.

The read count decides it

Reads per file per month	Direct	This architecture	Ratio
1	$746	$1,333	0.6x
5	$1,680	$1,333	1.3x
10	$2,593	$1,333	1.9x
30	$6,211	$1,333	4.7x
100	$18,451	$1,333	13.8x

At one read per file the direct path wins — the same bytes move either way and there is no reason to carry a file system's fixed cost. The crossover is between five and ten. Where it falls is a property of your workload, not of the products.

Requests matter too, but only below a certain object size

I had assumed S3 GET charges were a co-equal problem. At 4 MiB objects they are not: the entire 5.2 million reads in the ten-read case cost $1.94, about a thousandth of the transfer bill. Through an access point it is $0.31, and through FlexCache $0.16 — the reads are NFS and SMB, so they generate no S3 requests, and only the origin-side capacity pool fetch on cache fill remains.

Where the assumption does hold is small objects. Holding bytes read constant and varying object size:

Object size	Reads/month	GET charges	Share of that option's transfer bill
8 KiB	2.68 B	$993	48%
64 KiB	336 M	$124	6%
256 KiB	84 M	$31	1.5%
4 MiB	5.2 M	$1.94	0.1%

So both charges need designing for, and they respond to opposite remedies:

Transfer falls by carrying fewer bytes — cache the working set, move the readers into AWS, or drop the unit rate with Direct Connect ($0.041/GB against $0.114 for the first 10 TB of internet egress). Making objects bigger does not help; the byte count is unchanged.
Requests fall by making fewer calls — batch at collection time so files are larger, and serve reads over a file protocol so they never become S3 calls. Negotiating transfer rates does not help when the money is on the request side.

Batch too far and you hit the collection side: a single PutObject caps at 5 GiB and a whole object at 50 GiB, and the 50 GiB check happens at CompleteMultipartUpload — after the entire payload has been transferred and paid for. Both figures being binary (5,368,709,120 and 53,687,091,200 bytes) is confirmed by the vendor, but the public documentation still reads "5 GB" and "50 GB" and is being corrected. The increase from 5 GiB to 50 GiB was also never announced through any channel. An unannounced tenfold change leaves every downstream document that quotes the old value wrong, with no signal to re-check. Batching past the consumer's read unit also sends bytes nobody reads. The repository's design guide sets the consumer's read unit as the reference for how far to go, with the measurement and monitoring steps.

If the readers can move, move them

Worth stating because it outweighs every storage choice. In-Region transfer is free, so putting the consumers in AWS deletes the entire egress line — $2,079 at ten reads on this workload. Reading S3 directly from EC2 comes to $514/month; S3 Files costs $515, ninety-four cents more, because 4 MiB objects sit above its size threshold and never reach its high-performance storage; FSx for ONTAP in the same Region is $1,053, twice the direct path, since the transfer gap that justified it is gone.

S3 Files cannot serve this architecture's consumers. It speaks NFSv4.1 and NFSv4.2 only — no NFSv3, no SMB — so an appliance pinned to v3 and any Windows stage are out, and the documented compute targets are EC2, Lambda, EKS and ECS. It belongs in the picture as the option you get after migrating the readers, not as a substitute for a cache.

This architecture is for the cases where the readers cannot move: equipment on site, proximity to whatever is being measured, capital already spent on it.

On the distribution side

A FlexCache cache volume cannot be tiered — an origin with FabricPool tiering can be cached, but the cache itself never tiers, so it sits entirely on SSD. That is affordable because it is sparse: NetApp's guidance is at least 10% of origin, which is also the create default. At 10% the distribution side runs 2.7x to 6.1x below a full second copy, with the copy given its capacity pool discount. Size it like a copy and it inverts — at 100% it costs more than the copy, precisely because it cannot tier.

Cache volumes are writable, which I understated earlier. Write-around is the default and withholds the client acknowledgement until the origin has committed; write-back, from ONTAP 9.15.1, commits at the cache and propagates asynchronously. The default being synchronous with respect to the origin matters for freshness: there is no window where a cache-side write is missing from the canonical copy.

One assumption drives more of these numbers than any other, so treat it carefully. Background storage efficiency does not run on data once it has been tiered — only savings applied while the block was on SSD are preserved, and a block tiered before efficiency ran keeps none (AWS, NetApp). Expecting better than 40% on a tiering-enabled volume is optimistic. The model takes the SSD rate from AWS's published per-workload figures, assumes the pool tier retains half of it, and carries a sensitivity table — because the assumption only ever flatters this architecture: ONTAP deduplication does not reduce an S3 storage bill.

Full breakdown, every rate with its effective date, the workload models and the cross-layer pitfalls: FinOps cost structure and the S3 AP Design Guide (Japanese). Cost tables are generated from a model in the repository, so a price change moves one declaration rather than fifty typed totals.

A verification copy that costs no capacity — FlexClone and the S3 Access Point

Everything above is about distribution, so I measured duplication as well.

I checked whether FlexClone works on a volume with an access point attached (2026-08-26, ap-northeast-1, ONTAP 9.18.1P3D1). NetApp records FlexClone as unsupported for ONTAP S3, so I expected the same restriction might appear on this path, but it did not appear as a restriction.

FlexClone has two granularities, and they behave quite differently.

	File granularity	Volume granularity
Where it lands	The same volume. It appears as a new key on the same access point	A separate volume
Authorization boundary	Same as the parent	Its own access point, its own policy, its own identity
Time before it is usable	None	A volume created through the ONTAP API took 599 to 1,177 s to appear on the AWS side, then the junction path has to be set before attaching
Visible over S3	Yes. `StorageClass=FSX_ONTAP`, sha256 matches the source	Yes. LIST / GET / PUT through the clone's own access point

At either granularity the clone shares data blocks with its parent. Cloning a 256 MiB file four times gives these figures:

Point	Logical	Physical
Just after one 256 MiB PUT	—	253,534,208 B
After four file-granularity clones	1,350,942,720 B	277,200,896 B

Logical grew to five copies, but physical grew by only about 23 MB.

FinOps — no capacity to provision per duplicate

FSx for ONTAP bills provisioned SSD capacity. If a duplicate shares blocks with its parent, there is no capacity to provision for it. Doing the same on S3 requires CopyObject, every duplicate carries full storage charges, and the copy requests are billed on top. That difference is structural, so a unit rate does not close it.

The saving does reach the invoice only as capacity you did not have to provision, though. Where headroom is already generous, the figure does not move.

Two notes on what the space figures assume. NetApp states that copies consume no storage except what is required for metadata until changes are written to the copy (FlexClone volumes, files, and LUNs), so changes written to the clone are not shared and consume new blocks for whatever was written. And deleting data on the parent does not return capacity while a clone or a snapshot still references it.

The operation that ends the sharing is a split (volume clone split start). I did not measure it, so this is the documentation: from ONTAP 9.4, on AFF systems where the volume guarantee is none, the split shares the physical blocks rather than copying the data, and space efficiency is preserved. After the split, however, both the parent and the clone require the full space allocation set by their volume guarantees. The space needed can be checked beforehand with volume clone show -estimate (splitting procedure).

Operational Excellence — no AWS-side work per duplicate

What helps operationally is that making a duplicate adds no AWS-side resource. A file-granularity clone appears as a new key under the existing access point, so there is nothing to attach and nothing to wait for. A verification dataset can be produced from the production bytes and handed to the S3 consumers already in place, such as Amazon Bedrock, AWS Glue and Amazon Athena, without a staging copy job.

Where separation is needed, volume granularity is the one to use. Attaching a separate access point to the clone keeps its identity and policy apart from the parent's, and a write to the clone did not appear on the parent. That gives a way to separate the authorization boundary without duplicating the bytes. The cost is the 10-to-20-minute wait noted above.

There are two operational cautions as well.

The first is that a file-granularity clone gives no observable failure. POST /api/storage/file/clone returns 202 and a job UUID, but the UUID resolved to 404 entry doesn't exist and appeared in no job listing. The same fsxadmin retrieves volume-create and volume-clone jobs as state=success, so this is not a permissions problem. A call naming a destination directory that does not exist also returned 202 and created nothing. Judge the outcome by inspecting the destination file.

The second is that teardown gains an ordering constraint. Delete the clone before the parent. While the deleted clone sits in ONTAP's volume recovery queue, the parent keeps clone.has_flexclone at true, and the AWS-side delete-volume returned DELETING and then silently went back to CREATED. Purging the recovery queue clears the flag and the same call works. That queue appears in neither the console nor the FSx for ONTAP API, so watching only the AWS side leaves you stuck here.

What I did not measure is LUN-granularity cloning: creating a LUN needs an iSCSI configuration, which is not on this architecture's path. The procedure and the controls are in the repository's interoperability page, and the cost structure in FinOps cost structure.

Deploy

CloudFormation and Terraform templates:

GitHub: Yoshiki0705/s3-burst-on-ontap-files

Conclusion

"Collect via S3, consume via NFS" — one volume, no separate copy or replication job. FSx for ONTAP S3 Access Point + FlexCache. Main path p50 8 ms. FlexCache adds ~5 ms for same-region — nearly transparent. Transfer still happens for whatever gets read; what is absent is the sync job.

This verification was done entirely on AWS (FSx for ONTAP to FSx for ONTAP over VPC peering), but the cache side isn't limited to AWS. AWS documents exactly three FlexCache configurations, and with FSx for ONTAP as the origin the cache is either on-premises ONTAP or FSx for ONTAP (replicating with FlexCache). Cloud Volumes ONTAP, ONTAP Select, Azure NetApp Files and Google Cloud NetApp Volumes are absent from that table, so I record them as unconfirmed rather than assuming they work because they are ONTAP-based — and unconfirmed is not the same as unsupported. The repository's Portability and Support Matrix pages track what's confirmed and what's next. Cross-platform verification is on the roadmap.

The reverse direction is outside that table too, and the verdict splits two ways. Another
cloud's file storage as the origin with FSx for ONTAP as the cache is unconfirmed for Google Cloud
NetApp Volumes and Azure NetApp Files — something that looking could still resolve. For Google Cloud
Filestore, Azure Managed Lustre, Azure Blob NFS and OCI File Storage it is out of scope as a
mechanism, because they are not ONTAP and the cluster and SVM peering FlexCache requires cannot
exist. Network reachability does not change the second group. I do not write the two with the same
word.

I wrote up the network underneath it separately. AWS Interconnect – multicloud is GA, but its
published Region pairs are eight for Google Cloud and one for OCI (us-east-1 to us-ashburn-1), and
neither includes a Japanese Region. Azure is stated as planned, which is neither GA nor Preview.
Starting from Japan therefore means joining Direct Connect to the other cloud's circuit inside an
interconnection provider's fabric, and whether that is possible is decided not by the Region-pair
table but by whether Direct Connect locations, the other cloud's connection locations and the
provider's footprint overlap. Taking the partner route does not add Regions to the managed
service — it switches to a different construction.

The arrows in the figure stop at the AWS VPC. Not drawing the next hop as a solid edge is what the figure is for.

Encryption also sits at two layers worth keeping apart. MACsec on the physical link and cluster
peering encryption over FlexCache traffic (ONTAP 9.6 or later, TLS 1.2 AES-256 GCM) are different
things, and the first does not remove the need for the second. Whether ONTAP itself offers MACsec
on an intercluster LIF is something I could find no statement for. The options and the Region
coverage are in cross-cloud connectivity.

FlexCache duality (S3 reads on the cache side) does work, once S3 access is enabled on the cache volume itself with -is-s3-enabled true in advanced privilege. The earlier AccessDenied was a missing setting, not a platform limitation. The architecture still keeps cache-side access to NFS/SMB, because ONTAP native S3 and the AWS-managed access point are separate mechanisms and the NAS bucket is a read-only view without IAM integration.

All test resources torn down. Numbers are from a specific test environment and vary by workload and configuration.

Embedding AI Agents into a File Portal — From AgentCore MCP to Multi-Agent Teams (Part 3)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:21:40 +0000

Let an AI agent work the NAS, with a human approving — Amazon Bedrock AgentCore and MCP (part 3 of 3)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1 I built the file portal foundation on the S3 Access Points of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP), and in Part 2 I embedded storage operations (ARP/AI incident response, Tamperproof Snapshots, regulatory retention management). At that point the UI was sufficient for anyone who already knew which button to press — but a different kind of friction was still there. "Where did I put last week's simulation results?" "Summarize the key points of this PDF in three lines." The intent is perfectly clear; the cost of translating it into system operations was what stayed high.

So I integrated AI agents into the portal. Express your intent in natural language, and the agent finds the files, reads them, analyzes the content, and proposes storage management operations when needed. Because storage management includes irreversible operations, the execution decision stays with a human through a Human-in-the-Loop (HITL) flow.

Here's the conclusion up front:

Combining the Amazon Bedrock Converse API with MCP tools lets you explore and analyze files on an S3 AP in natural language
Connecting a Bedrock Knowledge Base directly to an S3 AP means you can reach a file through semantic search without knowing its name
Wrapping destructive operations in a HITL approval modal keeps agent autonomy and safety in the same design
Multi-agent collaboration pays off only for tasks with several phases — "discover → analyze → judge"

In this article, I'll cover the 3-mode AgentChat design, file access via MCP tools, the HITL flow, and the implementation patterns for multi-agent collaboration.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Repository: solutions/amplify-portal/

Terminology note: "MCP" in this article refers to Model Context Protocol — a standard protocol that allows AI models to invoke external tools (file operations, database queries, etc.). AgentCore Gateway acts as the MCP server, exposing Lambda-based tools to the AI model.

If You're Already Using ChatGPT / Copilot / Claude for Document Analysis

When using general-purpose AI assistants (ChatGPT, GitHub Copilot, Claude, etc.) for file analysis, the typical flow is "upload file → AI analyzes → returns results." The portal agent takes a different approach.

Aspect	ChatGPT / Copilot (upload-based)	Portal Agent (NAS-connected)
File access	Manual upload (one file at a time)	Direct access to files on NAS (via S3 AP)
Scope	Only uploaded files	Cross-volume search across entire storage
Storage operations	Not possible	Proposes SnapLock enable, user block, etc. (with HITL approval)
ONTAP-specific features	Not supported	ARP/AI status checks, Snapshot operations, export-policy changes
Data movement	File content sent to external service	File content processed within same-region Bedrock (VPC-contained possible)
Destructive operation safety	N/A	HITL modal for human approve/reject

This portal does not attempt to replace general-purpose AI assistants. It adds NAS-connected file intelligence — file discovery, content understanding, and storage operation proposals. For general Q&A, ChatGPT and Copilot remain well-suited tools for those tasks.

Verification status: how far each feature in this article has actually been confirmed is recorded in verification results, split four ways (live E2E / live read / tests only / DemoMode). Agent and team execution sits in the "tests only" category: handler and component tests pass, but it has not been driven from a browser against a real system.

What I Built

The portal moved from "browse files and check operational state" to "ask in natural language, and the agent finds files, understands content, and proposes storage operations." Here's what was added:

AgentChat: Multi-tool AI chat with 3 switchable modes
SemanticSearch: Vector search via Bedrock Knowledge Base — resolves "where did I put that file?" in natural language
Agent Directory / Creator: Define custom agents and share them with your team
Agent Teams: Run multiple agents in collaboration
ActionApproval (HITL): Insert a human approval flow before destructive operations
Multimodal input: Drag and drop an image into the chat for file analysis

Why Embed AI Agents in the Portal

The admin features from Part 2 work well for people who already know "what to operate and how." But what I actually hear in practice sounds more like:

"Where did I put last week's simulation results?"
"Can you summarize what caused the error in this log file?"
"I want to run legal review on the contracts folder, but I don't know the steps"

These aren't cases of "not knowing how to operate the system." They're cases of "I know what I want, but translating that intent into system operations is tedious." This applies equally to non-technical team members:

"Where's the attachment for last month's expense report?"
"Summarize this PDF in 3 lines"
"Compile a file list from this folder for my weekly report"

The motivation for embedding AI agents was to eliminate this translation cost entirely. Express your intent in natural language, and the agent finds files, reads them, analyzes content, and proposes admin operations if needed. Users only need to think about "what they want to accomplish."

However, giving an agent unrestricted authority in a storage management context is dangerous. Enabling SnapLock is irreversible. Modifying an export-policy immediately cuts off user access. So I combine the Human-in-the-Loop (HITL) pattern: "the agent proposes, but humans approve execution."

AgentChat — 3-Mode Integrated Chat

A persistent AI chat in the portal's right panel, fully replacing the previous "Bedrock Q&A" (single-file question-answering).

3 Modes

Mode	Icon	Purpose
🧠 Knowledge	KB	Cross-volume semantic search. Answers "where's that file?"
📁 File Agent	agent	File read/list/analyze. Accesses S3 AP via MCP tools
🤖 Multi-Agent	multi	Multiple agents collaborate. File discovery → content analysis → operation proposal in sequence

Modes switch via pill buttons below the header. Welcome screen task cards change per mode, showing "what you can do in this mode."

Architecture

Browser (React)
    ↓ AppSync Query (agentQuery)
Lambda: Agent Orchestrator (outside VPC)
    ├── mode=kb   → Bedrock KB RetrieveAndGenerate
    ├── mode=agent → Bedrock Converse + MCP Tools
    └── mode=multi → Multi-agent collaboration
                         ↓ MCP Client
                    AgentCore Gateway
                         ↓ Lambda Invoke
                    MCP Tool Lambda (list_files / read_file / search_files)
                         ↓ S3 API
                    FSx for ONTAP S3 Access Point

Light theme shown. A dark theme version is available.

The VPC split principle applies here too. The Agent Orchestrator Lambda is placed outside VPC, accessing Bedrock API and S3 AP (Internet-origin). Admin operations calling ONTAP REST API are handled by a separate VPC-internal Lambda.

Tool Execution Visibility

When the agent invokes tools, a timeline appears in the chat:

🔧 Tool Calls (3)                          2.1s
├── ✅ 🔍 file-explorer: list_files         {"prefix": "/engineering/"}
├── ✅ 📄 file-explorer: read_file          {"key": "/engineering/thermal-spec-v3.pdf"}
└── ✅ 🧠 knowledge-analyst: analyze_file   {"key": "...", "question": "..."}

Each tool's input/output examples:

Tool	Input Example	Output Example
`list_files`	`{"prefix": "/engineering/", "max_keys": 20}`	File path list + size + last modified
`read_file`	`{"key": "/engineering/thermal-spec-v3.pdf"}`	File content (text-extracted)
`search_files`	`{"query": "thermal limit exceeded"}`	Matched files + relevant snippets

Displayed in <details> — expanded by default when 3 or fewer tool calls, collapsed when 4+. This balances Nielsen's "visibility of system status" with Wroblewski's "progressive disclosure."

Chat History Persistence

Conversation sessions auto-save to DynamoDB (2-second debounce). Past sessions can be recalled from the history panel to continue questioning.

// Auto-save: 2 seconds after message change → DynamoDB
useEffect(() => {
  if (messages.length === 0 || !currentSessionId) return;
  const timer = setTimeout(() => saveCurrentSession(), 2000);
  return () => clearTimeout(timer);
}, [messages, currentSessionId]);

The rationale: forcing an explicit "Save" button interrupts the natural flow of chat. Close the browser, and next time you open it, you can pick up from "where we left off yesterday."

Multimodal Input

Drag-and-drop images into the chat for Bedrock Vision analysis:

Circuit diagram image → "Where's the bottleneck in this circuit?"
Screenshot → "What's causing this error screen?"
Graph image → "Explain this performance degradation trend"

Supports JPEG/PNG/GIF/WebP under 5MB. Base64-encoded and sent to Bedrock Converse API.

SemanticSearch — Vector Search

Keyword search on file names has limits. When you want to find "the spec document about thermal design limits," you might find thermal-spec-v3.pdf by name, but TC-2024-0089.pdf? No chance.

Keyword / Semantic Mode Toggle

Mode toggle pills in the search bar:

🔍 [Search files...                        ] [🔍]
   [📂 Keyword]  [🧠 Semantic]

Keyword mode: S3 AP ListObjectsV2 + prefix filter. Auto-searches with 500ms debounce after 2+ characters
Semantic mode: Bedrock Knowledge Base RetrieveAndGenerate API. Vectorizes natural language queries for similarity search

Bedrock Knowledge Base Integration

FSx for ONTAP's S3 AP can be specified directly as a Knowledge Base data source (AWS official tutorial).

FSx for ONTAP Volume
    ↓ S3 AP (Data Source)
Bedrock Knowledge Base
    ↓ Embeddings (Titan V2)
OpenSearch Serverless (Vector Store)
    ↓ RetrieveAndGenerate
Results: related chunks + source file path + relevance score

Light theme shown. A dark theme version is available.

Results include scores (percentage display) and snippets. Clicking a result navigates to the file in the All Files view.

Semantic Search Input Examples

When KB is unconfigured, help text guides users:

💡 Semantic search examples:
• "Files with anomalies in last month's sales reports"
• "Test results exceeding thermal design specifications"
• "Description of the approval process for contract renewal"

Constraints

KB sync is asynchronous (minutes to tens of minutes lag after file changes)
Large file environments incur OpenSearch Serverless indexing costs
PDF/Office text extraction accuracy depends on file quality
If bedrockKbId is empty in portal-config.ts, semantic mode shows a "KB not configured" error

KB sync detection: Data source sync status can be checked via the bedrock-agent:GetIngestionJob API. Periodic sync can be scheduled in the Bedrock console. If immediate search reflection after file changes is required, consider an EventBridge + StartIngestionJob API configuration for on-demand sync triggers.

Agent Directory — Agent Catalog

A UI for discovering "what agents are available."

Preset Agents

Agent	Icon	Specialty
file-explorer	📁	File operations (list, read, search)
knowledge-analyst	🧠	Vector search + document analysis
safety-controller	🛡️	ARP status checks + containment action proposals
compliance-auditor	📋	Retention period and SnapLock configuration audits
ops-advisor	⚙️	EMS event analysis + operational recommendations

Custom Agent Creation (Agent Creator)

When presets aren't enough, define your own agents:

Agent Creation Wizard:
1. Set name and icon
2. Write system prompt
3. Select tools to use (checkboxes)
4. Test execution
5. Sharing settings (personal only / team-shared)

Created agents appear in the Agent Directory, available to team members (when shared). This lets you codify patterns like "this department frequently does this kind of analysis" as reusable agents.

Agent Teams — Multi-Agent Collaboration

For complex tasks that a single agent can't handle alone, multiple agents collaborate.

When to Use Multi-Agent vs Single Agent

Scenario	Recommended	Reason
"Summarize this PDF"	Single agent	Completes with one tool (read_file + analysis)
"Analyze all files in engineering/ and report security issues"	Multi-agent	Requires file discovery → content analysis → security judgment
"Audit files changed last month for compliance violations"	Multi-agent	File identification → content reading → regulatory assessment → report generation

Decision criteria: Multi-agent is effective when the task has multiple phases ("discover → analyze → judge") requiring different expertise at each phase. For simple tasks, multi-agent adds cost and latency without benefit — use single agent instead.

Example Team Configuration

Team: "Security Audit Team"
├── 🔍 file-explorer (Collaborator): Identify target files
├── 🛡️ safety-controller (Supervisor): Verify ARP/security state
└── 📋 compliance-auditor (Reviewer): Final compliance judgment

Roles

Role	Responsibility
Supervisor	Manages overall task progress. Directs other agents
Collaborator	Performs work based on instructions (file ops, analysis, etc.)
Reviewer	Reviews work results. Quality checks and approve/reject

Team Creation Wizard

Enter team name and description
Add agents from Agent Directory (minimum 2)
Assign roles to each agent
Sharing settings

Team execution flow:

User: "Analyze all simulation results in the engineering/ folder"
    ↓
Supervisor (safety-controller):
    → Instructs file-explorer: "Get file listing for engineering/"
    ↓
Collaborator (file-explorer):
    → list_files → read_file (multiple)
    → Result: 12 files found, 8 are simulation results
    ↓
Supervisor:
    → Instructs knowledge-analyst: "Create summaries for each file"
    ↓
Collaborator (knowledge-analyst):
    → Analyzes each file, generates summaries
    ↓
Reviewer (compliance-auditor):
    → "Verify no PHI data is included"
    → Result: 2 files may contain personal information → recommends guardrail application
    ↓
Final response to user:
    "Analyzed 8 of 12 files. Summaries below.
     Note: 2 files were filtered due to potential personal information."

Light theme shown. A dark theme version is available.

Each agent leverages its specialty while the Supervisor coordinates the overall flow. From the user's perspective, a single chat message completes a complex task.

ActionApproval — Human-in-the-Loop

When the AI agent proposes a destructive or irreversible operation, a modal requesting human approval appears before execution.

Why HITL Matters

In a storage management context, some operations are irreversible or have immediate impact. Enabling SnapLock can never be undone. A SnapMirror break severs the replication relationship. If an AI agent autonomously executes these, recovery may be impossible. The HITL pattern ensures the agent "proposes" but the final execution decision rests with a human.

Which Operations Require Approval

Operations flagged with isDestructive: true or isReversible: false require approval:

Operation	Icon	Irreversible?	Why Approval is Required
File deletion	🗑️	△ (recoverable from snapshot)	Prevent unintended bulk deletion
SnapLock enable	🔐	◎ (completely irreversible)	Once enabled, cannot be disabled
SnapMirror break	🔗	◎ (relationship must be re-established)	Replication severance requires manual recovery
User block	🚫	○ (reversible but immediate impact)	Risk of business disruption
IP block	🚫	○	Risk of service disruption
Retention period change	⏱️	△ (can only extend, never shorten)	Shortening is not possible
Threat containment	🛡️	○	May affect multiple users

How the Approval Flow Works

1. Agent proposes a destructive operation
2. UI displays modal (showing action, target, reason)
3. User reviews the details
4. [Approve] → operation executes / [Reject] → operation cancelled, agent notified

Example: SnapMirror Break Approval Flow

User: "Execute failover to the DR site"
    ↓
Agent: Determines SnapMirror break is required
    ↓
┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  SnapMirror break                │
│ Target:  vol_production → vol_dr         │
│ Reason:  DR failover execution           │
│                                          │
│ 🔴 This operation severs the replication │
│    relationship. Re-sync requires manual │
│    intervention.                         │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

UI Design

┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  User block                      │
│ Target:  DOMAIN\suspicious_user          │
│ Reason:  ARP detected abnormal write     │
│          patterns                        │
│                                          │
│ ⚠️ This operation will immediately cut   │
│    off the target user's SMB access      │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

Safe-side default: Reject button has autoFocus. Reflexively pressing Enter falls to the safe side. Clicking outside the modal also triggers "reject."

Irreversibility warning: Operations with isReversible: false show a red warning bar, ensuring users visually recognize the severity of the operation.

Bedrock Guardrails — Response Filtering

Bedrock Guardrails are applied to all AgentChat responses. While the PHI guardrail (Part 2) blocks based on path patterns, Guardrails filter response content itself.

Applied Checks

Check	Target	Behavior
PII detection	Input/Output	Masking (`*--**`)
Harmful content	Output	Block + substitute message
Topic denial	Input	Refuse response to specific topics
Grounding check	Output	Suppress ungrounded answers

When Guardrails are applied, a 🛡️ badge appears on the message. Users see "some information was filtered" with filter reason details available via tooltip.

Model, Cost, and Fallback

Model Configuration

Setting	Default	Changeable	Notes
Model	`amazon.nova-lite-v1:0`	○	Lowest cost. Use Claude 3.5 Haiku for quality
Max tokens	4096 (output)	○	Increase for long analyses
History sent	Last 10 messages	○	Balance with token consumption
Cost per request	~$0.001–$0.01	—	For Nova Lite. Larger context from file reads increases cost

Model is configurable via bedrockModelId in portal-config.ts. Recommended: Nova Lite for testing, Claude 3.5 Haiku for production.

Cost note: Token cost per conversation depends on tool call count and file sizes. A typical "file search + summarize" conversation costs ~$0.005–$0.02 with Nova Lite. To control context window size, adjust maxHistoryMessages in portal-config.ts (default: 10). Reducing history lowers cost but loses conversation context in longer sessions.

Data residency note: When using Bedrock in ap-northeast-1 (Tokyo), input data processing completes within the same region. Deploying FSx for ONTAP, the portal, and Bedrock in the same region ensures file contents never leave the region. Important for financial institutions and healthcare organizations with data residency requirements.

Security note: File contents are included in Bedrock API request bodies, but a VPC-contained configuration is possible. Bedrock does not use request data for model training (AWS Bedrock Data Protection). With VPC endpoint routing, data never leaves the VPC.

Bedrock KB Configuration Parameters

Setting	Recommended	Rationale
Embedding Model	Titan Embeddings V2	Japanese support, cost-efficient
Chunk Size	512 tokens	Appropriate granularity for file-level search
Chunk Overlap	64 tokens	Prevents context loss at chunk boundaries
Top K	5	Balance of precision and cost
Vector Store	OpenSearch Serverless	Auto-provisioned with Managed KB

These are starting-point recommendations. Adjust based on average file size and content characteristics.

Chunk Size tradeoff: Smaller chunks (256 tokens) improve recall but fragment context, reducing precision. Larger chunks (1024 tokens) provide richer context but allow irrelevant content to mix in. 512 tokens balances well for "file-level search." For technical documents (long sections), consider 768–1024.

Fallback Behavior

When Bedrock API is unavailable (regional outage, throttling, etc.), the agent chat displays "Cannot connect. Please use conventional search" and falls back to keyword search. The design ensures "agent unavailable ≠ portal unavailable."

Chat History Retention Policy

Set TTL on DynamoDB conversation history. Default: auto-deleted after 90 days. If auditing "who asked what" is required, separately retain CloudTrail API call logs.

Right to deletion: If a user requests deletion of their chat history, administrators can manually delete the relevant DynamoDB sessions. For GDPR/privacy law environments, implementing a self-service "Clear history" button is recommended.

Design Tradeoffs

Decision	Benefit	Tradeoff
AppSync Query (not Subscription)	Simple implementation, single Cold Start	No streaming response (waits for full completion)
Agent Orchestrator outside VPC	Short Cold Start (no ENI)	Cannot call ONTAP REST API (delegates to VPC Lambda)
DynamoDB session storage	History shared across browsers	Cost (write capacity), TTL management needed
HITL via modal	Reliably blocks destructive ops	UX interruption, reduced agent autonomy
Mode pill toggle (3 modes)	Optimal agent for each purpose	Learning cost of mode concept
Bedrock Guardrails always-on	Structural PII leak prevention	Latency increase (hundreds of ms)
Multi-agent (Agent Teams)	Handles complex tasks	Harder debugging, increased cost

On Streaming Responses

The current implementation uses AppSync Query (synchronous), so agent responses display only after full generation completes. For long responses (file analysis, etc.), users wait several seconds.

Token-level streaming via AppSync Subscription (WebSocket) is technically possible, but implementation complexity (reconnection handling, partial response parsing, error management) increases substantially. I've deferred this as a next step for when latency becomes a problem.

Security Model (AI Agent Specific)

Layer	Implementation	Intent
Authorization	Cognito Groups (authenticated)	All authenticated users can use chat
Tool permissions	Agent autonomously executes read-only tools	Write operations require HITL
Input filter	Bedrock Guardrails (input)	Prompt injection countermeasure
Output filter	Bedrock Guardrails (output)	PII masking
PHI path	isPhiPath() check	Don't pass `/dicom/` etc. file contents to agent
Cost control	Only last 10 messages sent as history	Token consumption cap
Audit	CloudTrail + DynamoDB session logs	Track "who asked what"

Monitoring

Recommended operational metrics:

Metric	Source	Alert Threshold
Agent Orchestrator error rate	Lambda CloudWatch Metrics	> 5% warrants investigation
Response latency (p99)	Lambda Duration	> 10s check model or tool side
Bedrock Throttling	Bedrock CloudWatch Metrics	ThrottledCount > 0 consider provisioned throughput
DynamoDB write throttling	DynamoDB ConsumedWriteCapacityUnits	On-demand auto-scales, typically no issue

Key design decision: The agent can propose storage-admin operations, but execution is authorized against the user's Cognito Group. If a regular user asks the agent "enable SnapLock," it fails with an authorization error. The agent explains "you don't have permission for this operation."

Compliance note: When acting on AI responses about regulated data, the final judgment responsibility lies with the human who approved the operation. Agent responses are "assistive information" — not substitutes for legal or regulatory judgment. Audit trail via CloudTrail + DynamoDB session logs records "who asked what, and what was approved."

Frequently Asked Questions

The following addresses common questions from different perspectives.

Cost note: "What's the token cost per conversation? How do I control context window size?" — With Nova Lite, ~$0.005–$0.02/conversation (varies by tool call count). Adjust maxHistoryMessages to balance cost vs. context retention.

Security note: "What data leaves the VPC? Can Bedrock see file contents?" — File contents are included in Bedrock API requests, but Bedrock does not use data for model training. With VPC endpoint configuration, data stays within the VPC. S3 AP access in Internet-origin configuration also completes within the same region.

Operations note: "How do I add a new MCP tool? What's the deploy process?" — New tools are implemented as Lambda functions and registered as targets in AgentCore Gateway. Deploy via sam deploy or Amplify CI/CD pipeline. See AgentCore MCP Tools Reference for details.

End-user note: "Can I just ask 'find my expense report from last month' and it works?" — Yes. With Knowledge mode and semantic search enabled, you can search by intent without knowing file names. File Agent mode can also traverse folder structures to locate files.

Scaling note: "How does this work with multiple Knowledge Bases across departments?" — KB data sources can be specified per-volume via S3 AP, so you can create separate KBs per department volume. The agent switches target KB via bedrockKbId configuration.

Front-end note: "How do I customize the chat UI? Can I add my own components?" — The chat UI is implemented as a React component (AgentChat.tsx), supporting style customization and custom message renderers. Tool call result displays are also extensible via the ToolCallTimeline component.

Staged Adoption Steps

All AI agent features support DemoMode. They work without FSx for ONTAP for evaluation.

Step 1: No KB, No Agent (File Search Only)

Leave bedrockKbId empty in portal-config.ts. Only keyword search available.

Step 2: Add Bedrock Knowledge Base

Register FSx for ONTAP S3 AP as KB data source
Set bedrockKbId in portal-config.ts
Semantic search mode becomes active

Step 3: Enable AgentChat (File Agent Mode)

Deploy MCP tool Lambda. list_files, read_file, search_files become available.

Step 4: Multi-Agent Mode and HITL

Add admin operation tools (block_user, enable_snaplock, etc.). HITL modal activates automatically.

Step 5: Agent Teams and Custom Agents

Define team-specific analysis patterns as agents. Share via Agent Directory.

Resources

Summary and Next Steps

I integrated AI agents into the "file portal + storage operations" foundation built in Parts 1 and 2:

Feature	What became possible
AgentChat (3 modes)	Complete file operations, analysis, and admin ops in natural language
SemanticSearch	Solve "where did I put that file?" with vector search
Agent Directory/Creator	Codify team-specific analysis patterns as agents
Agent Teams	Collaboratively process complex tasks with multiple agents
HITL (ActionApproval)	Execute destructive operations only after human approval
Bedrock Guardrails	Structurally prevent PII leaks and harmful responses

This three-part series documented the portal's evolution:

Part 1 covered the file portal foundation (S3 AP + Amplify Gen2 / Nextcloud)
Part 2 added storage operations (ARP/AI, Tamperproof, regulatory retention)
Part 3 integrated AI agents (natural language operations, semantic search, multi-agent)

Through the progression of "browse files" → "manage storage" → "talk to AI," the portal reached a point where daily operations and data utilization complete in the browser — without opening ONTAP System Manager or the CLI.

All code is published in the GitHub repository.

For next steps, I'd suggest starting with keyword search in DemoMode, then adding a Bedrock Knowledge Base for semantic search, and layering on MCP tools and HITL after that. When you move on to a production connection, see the PoC → Production Guide.

I hope this post helps someone out there.

See you next time.

Embedding Storage Operations into a File Portal — From ARP/AI Incident Response to Regulatory Retention Management (Part 2)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:20:41 +0000

Run detection through containment from the screen — ARP/AI, SnapLock, audit logs (part 2 of 3)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1, I looked at putting a file portal on top of the S3 Access Point of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP S3 AP, or S3 AP), comparing Amplify Gen2 and Nextcloud as two approaches. The portal at that point stopped at "browse the files" — the moment you wanted to know something about the storage layer, you had to go back to the ONTAP CLI or the REST API.

Once people started using it, the first two things that came back were "I want to check whether ARP has detected anything myself" and "is this snapshot actually locked?" Both requests came from team members outside storage administration, and the sticking point was the same: you cannot check any of it without a VPN connection to the management LIF. So I added ONTAP operations features to the Amplify Gen2 portal, up to the point where health checks and incident first response can be done entirely in a browser.

Here's the conclusion up front:

ONTAP System Manager-equivalent admin operations can be executed from a browser through AppSync + Lambda
Separating storage-admin from regular users with Cognito Groups gives you a safe split between read-only viewing and change operations
Managing ransomware response as four states — Detected → Contained → Investigating → Resolved — removes the hesitation from first response
Turning regulatory retention periods (FISC 7 years / SOX 5 years / HIPAA 6 years) into presets prevents day-count mistakes

In this article, I'll cover the design of the storage operations features embedded in the portal, the ONTAP REST API implementation behind them, and the points I was careful about when handling irreversible operations.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Relationship to existing tools: This portal makes the same storage management operations previously available only through the ONTAP CLI or REST API accessible from a browser with Cognito authentication. It is not a replacement for those — it's an additional layer that opens the same operations to people who should not hold cluster-administrator credentials.

The features added are as follows:

Storage Dashboard: a 4-card health overview
ARP/AI Incident Lifecycle: managed as four states — Detected → Contained → Investigating → Resolved
S3 Object Lock Configuration UI: select a bucket, specify a retention mode, apply
PHI Guardrail: structurally blocks AI processing for paths such as /dicom/
EMS Events: real-time ONTAP alert display
Regulatory Retention Presets: FISC 7-year / SOX 5-year / HIPAA 6-year
Welcome Modal: first-time user onboarding
Audit Log: file access trail via CloudTrail, viewable in the UI
FlexClone Restore: one-click recovery from snapshots
Athena Query: SQL analysis on NAS data

Scope: this article is an implementation record of adding "operations features usable by members outside storage administration" to the file portal built in Part 1. Part 3 covers AI agent integration.

Repository: solutions/amplify-portal/

Why Embed Admin Operations in a Portal

For FSx for ONTAP, the management interfaces you can reach without routing through an additional third-party SaaS are three: the AWS Management Console / FSx API, the ONTAP CLI over SSH, and the ONTAP REST API. ONTAP System Manager is not among them — System Manager support for FSx for ONTAP is delivered through the vendor's SaaS console, and that SaaS covers FSx for ONTAP only in its SaaS-connected mode (sources in management interfaces).

So learning anything about the ONTAP side means SSH to the ONTAP CLI or a REST call, both of which assume cluster-administrator credentials. When team members outside storage administration (security staff, compliance officers, data protection teams) want to "check ARP status" or "verify whether a snapshot is locked," the way to let them is to hand over an SSH key.

Motivation for embedding admin operations:

Security staff should be able to instantly check ransomware detection state and execute containment actions
Compliance officers should be able to verify retention settings and audit logs themselves
Operations teams should be able to check ONTAP alerts without opening another tool

Cognito Groups (storage-admin / authenticated) separate authorization — regular users are read-only, only admins can execute changes.

What You Used to Do vs What You Can Do Now

Operation	Previous Method	Portal Method
Check ARP threats and contain	CLI: `security anti-ransomware volume show` → block by hand	Dashboard detection → one-click containment
WORM-lock a snapshot	CLI: `volume snapshot modify-retention` or direct REST API call	Lock panel → select preset → apply
DR failover with SnapMirror	CLI: `snapmirror break` / `snapmirror resync`	Status list → click action button

Storage Dashboard — Admin Landing Page

The first thing admins see after login is a 4-card health dashboard. This follows ONTAP System Manager's "dashboard first" pattern.

┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ 💾 12       │  │ 🛡️ 8         │  │ 🔐 5        │  │ 📊 3.2x 　   │
│ Volumes     │  │ARP Protected│  │ Locked Snaps│  │ Efficiency  │
│ Avg: 62%    │  │✅ No threats│  │ Tamperproof │  │ 69% saved   │
└─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘

The 4 Cards

Card	Shows	Navigates to
Volume Capacity	Volume count + average utilization	Volume Manager
ARP/AI Protection	Protected volumes + threat count	ARP Admin
Locked Snapshots	Locked snapshot count	Snapshot Admin
Storage Efficiency	Dedup/compression ratio + savings	Efficiency Panel

Implementation

Four APIs fetched in parallel with Promise.allSettled. If one fails (e.g., ARP not configured), remaining cards still render.

const [volResp, arpResp, snapResp, effResp] = await Promise.allSettled([
  adminQuery({ action: "listVolumes", params: JSON.stringify({}) }),
  adminQuery({ action: "listArpVolumes", params: JSON.stringify({}) }),
  protectionQuery({ action: "listSnapshots", params: JSON.stringify({ maxResults: 50 }) }),
  adminQuery({ action: "getEfficiencyStats", params: JSON.stringify({}) }),
]);

Why Promise.allSettled over Promise.all: with Promise.all, one failure rejects everything. On a storage dashboard, "ARP not configured" is a normal state, not an error.

ARP/AI Incident Lifecycle

After ONTAP Autonomous Ransomware Protection (ARP/AI) detects a threat, the response flow is managed across 4 states.

State Transitions

🔴 Detected
  → 🟠 Contained
    → 🟡 Investigating
      → 🟢 Resolved

Light theme shown. A dark theme version is available.

What's Recorded at Each State

State	Timestamp	Additional Data
Detected	detectedAt	—
Contained	containedAt	blockedUsers[], blockedIps[], snapshotName
Investigating	—	notes (free text)
Resolved	resolvedAt	—

Containment Actions

Actions executable directly from the portal:

Action	ONTAP REST API	Effect
Block SMB User	name-mapping deny (win→unix)	Immediately denies target user's SMB access
Block NFS IP	export-policy deny rule	Immediately denies NFS access from target IP
Full Containment	snapshot + block + disconnect	Preserves snapshot + blocks all + disconnects sessions
Unblock	delete name-mapping/export-policy rules	Recovery after investigation

This implements the equivalent of NetApp DII Storage Workload Security blocking directly via ONTAP REST API.

Implementation

// useIncidentState hook — persists per-volume in localStorage
export type IncidentState = "none" | "detected" | "contained" | "investigating" | "resolved";

export function useIncidentState(volumeName: string) {
  const [incident, setIncident] = useState<IncidentRecord>(() => loadIncident(volumeName));

  const markContained = useCallback((snapshot?: string, users?: string[], ips?: string[]) => {
    transition("contained", { snapshotName: snapshot, blockedUsers: users || [], blockedIps: ips || [] });
  }, [transition]);

  // ...
}

The UI displays a state badge with the next action button changing dynamically:

[🔴 Detected] → [Execute Containment] button
[🟠 Contained] → [Start Investigation] button
[🟡 Investigating] → [Resolve] button
[🟢 Resolved] → (complete)

Current limitation: Incident state is stored in localStorage and not shared across browsers. There is also a risk of losing incident state if the browser crashes. DynamoDB persistence is recommended for production use so that multiple members see the same state.

Incident response note: After executing containment actions, integrate with your existing incident runbooks (PagerDuty, OpsGenie, etc.). Portal containment is a first-response action and should be positioned as part of your full incident management workflow.

Security note: Executing containment actions (SMB block, NFS IP block) requires membership in the storage-admin group. Unblocking requires the same privilege, so even if a regular user's browser session is compromised, the attacker cannot lift a block. Compromise of a storage-admin account itself is guarded by Cognito MFA.

Multi-tenant note: When multiple teams (tenants) share the same portal, ensure tenant A cannot view tenant B's ARP alerts by combining Cognito Groups with volume-level access control. In the current implementation, all storage-admin group members can view ARP status for all volumes. If tenant isolation is required, consider deploying separate portal instances per SVM.

PHI Guardrail — Structurally Blocking AI Processing

Prevents the "accidentally sent PHI data to an external AI" scenario at the UI level in HIPAA environments.

How It Works

Files in paths containing /dicom/, /phi/, /pii/, /hipaa/, or /protected-health/ have their AI processing button automatically disabled.

const isPhiPath = (path: string): boolean => {
  const lower = path.toLowerCase();
  return /\/(dicom|phi|pii|hipaa|protected-health)[\/-]/.test(`/${lower}`) ||
         lower.startsWith("dicom/") || lower.startsWith("phi/") || lower.startsWith("pii/");
};

UI display:

Normal folder: [⚡ Run AI Processing]     ← clickable
PHI folder:    [🚫 PHI — AI Blocked]      ← disabled, not clickable

Design Intent

Guardrail, not detect-and-respond: rather than "processed it, then discovered it was PHI," this prevents processing from starting at all
Cannot be bypassed even by admins: the button itself is disabled regardless of permissions
Depends on folder naming convention: path-pattern based, so it presupposes organizational folder structure rules

Important limitation: This guardrail blocks based on folder path only — it does NOT scan file contents. A file like /contracts/sensitive-patient.pdf placed outside PHI paths will not be blocked. For content-based PHI detection, combine with Bedrock Guardrails (response filtering) described in Part 3. Path guardrail + content filtering gives you ideal 2-layer defense.

HIPAA note: Using this portal in a HIPAA environment presupposes a Business Associate Agreement (BAA) with AWS. Cognito, Lambda, S3, Step Functions, and Bedrock are all HIPAA eligible services. The PHI guardrail is an additional UI-level defense — both the BAA and technical controls are required.

S3 Object Lock — WORM Protection for Output Buckets

AI processing results and compliance reports are stored in standard S3 buckets outside FSx for ONTAP. The portal provides a UI to configure S3 Object Lock on these output buckets.

3-Tab Lock Panel

Tab	Target	Purpose
ONTAP SnapLock	Volumes	WORM protection for NAS data (shared across NFS/SMB/S3 AP)
S3 Object Lock	S3 Buckets	WORM protection for AI processing results
Tamperproof Snapshot	Snapshots	Tamper prevention for specific snapshots

S3 Object Lock Configuration Flow

1. Fetch bucket list → select from dropdown
2. Choose mode:
   - Governance: Authorized users can override (recommended for AI output)
   - Compliance: No one, including root, can delete until retention expires (regulatory archives)
3. Specify retention days
4. [Apply] sets Object Lock Configuration

When to Use SnapLock vs S3 Object Lock

Characteristic	ONTAP SnapLock	S3 Object Lock
Target	Files on NAS volumes	Objects in S3 buckets
Access protocols	NFS/SMB/S3 AP	S3 API
Use case	Source data WORM protection	AI results/reports protection
Regulatory	SEC 17a-4, FISC, HIPAA, NARA	SEC 17a-4, HIPAA

Combining both achieves "source data AND processing results both tamper-proof."

Compliance framework note: The AWS services used by this portal (Cognito, Lambda, S3, Step Functions, Bedrock, AppSync) are in scope for ISMAP and SOC 2. FedRAMP environments for US government agencies require deployment in AWS GovCloud regions. Confirm compliance requirements with your legal and compliance teams — this portal is a technical implementation pattern and does not provide compliance judgments.

Regulatory Retention Period Presets

The Tamperproof Snapshot lock form offers retention periods as dropdown selections.

Preset List

Preset	Retention Days	Regulatory Basis
30 days	30	Short-term validation
90 days	90	Quarterly
1 year	365	Annual
SOX/J-SOX	1,825 (5 years)	Securities Exchange Act — financial records retention
HIPAA	2,192 (6 years)	Medical records minimum retention
FISC	2,557 (7 years)	Center for Financial Industry Information Systems standard, Chapter 9

Compliance note: FISC/SOX/HIPAA presets are provided for technical convenience. Whether the regulatory mapping and retention periods apply to your organization's specific situation should be validated with your legal and compliance teams. Regulatory interpretation varies by industry, business scope, and jurisdiction.

Why Presets Are Needed

SnapLock retention settings are irreversible. Once a snapshot is locked, the retention period cannot be shortened.

Prevents "how many days is 5 years?" calculation errors
Each preset shows a tooltip with regulatory basis
Minimizes risk of incorrect selection (can extend but never shorten)

<option value={1825} title="SOX/J-SOX: Financial records must be retained for 5 years">
  1,825 days — SOX/J-SOX (5 years)
</option>
<option value={2192} title="HIPAA: Medical records require minimum 6-year retention">
  2,192 days — HIPAA (6 years)
</option>
<option value={2557} title="FISC: Financial institution data requires 7-year retention">
  2,557 days — FISC (7 years)
</option>

EMS Events — ONTAP Alert Viewer

Retrieves events from ONTAP's Event Management System (EMS) with severity alert/error/emergency and displays them in the admin panel.

Backend Implementation

def _get_ems_events(http, headers, event):
    """ONTAP REST: GET /api/support/ems/events"""
    max_records = min(event.get("maxRecords", 20), 50)
    severity_filter = event.get("severity", "alert,error,emergency")

    query = f"/support/ems/events?max_records={max_records}"
    query += f"&severity={severity_filter}"
    query += "&order_by=time desc"
    query += "&fields=time,severity,message.name,message.text,node.name"

    data = _ontap_request(http, headers, "GET", query)
    events = [
        {
            "time": e.get("time", ""),
            "severity": e.get("severity", ""),
            "messageName": e.get("message", {}).get("name", ""),
            "messageText": e.get("message", {}).get("text", ""),
            "node": e.get("node", {}).get("name", ""),
        }
        for e in data.get("records", [])
    ]
    return {"events": events, "count": len(events)}

Example Events

Severity	Message Name	Meaning
emergency	ha.takeover.byPartner	HA partner takeover
alert	raid.disk.predictiveFailure	Disk predictive failure
error	scsiblade.san.netLIFDown	SAN LIF down
alert	arw.volume.attack	ARP detected ransomware attack

Storage administrators can check recent alerts directly in the portal without opening an ONTAP CLI session.

Refresh frequency note: EMS events are fetched on demand when the dashboard is displayed (not by periodic polling). If you need real-time alerting, push EMS events via EventBridge + SNS instead (see fsxn-observability-integrations). The portal is for "checking recent events"; for "detecting in real time," consider pairing it with a separate observability stack.

Welcome Modal — First-Time Onboarding

First-time users see a 3-step guided tour.

3 Steps

📂 File Browsing — Browse and search NAS files from your browser
⚡ AI Processing — Select files and trigger AI/ML workflows
🔒 Data Protection — Snapshots, WORM locks, ransomware protection

Implementation

export function WelcomeModal() {
  const [dismissed, setDismissed] = useState(() => {
    try { return localStorage.getItem("portal-welcome-dismissed") === "true"; }
    catch { return false; }
  });

  if (dismissed) return null;

  // 3-step carousel with dot navigation
  // "Don't show again" checkbox → localStorage persistence
}

Design Intent

Standard onboarding pattern in SaaS products
Often omitted in internal tools, but highly effective for conveying "what this portal can do" in 10 seconds
localStorage state management, works without backend changes

Resource Management — 20-Panel Card-Grid Admin

ONTAP admin operations organized into 5 categories × 20 panels.

Category Structure

Category	Panels
🗄️ Storage	Volumes / 🧬 FlexClone / ⚡ FlexCache / Qtrees / Quotas / Efficiency
🔐 Access Control	Export Policies / SMB Shares / 👤 Local Users / 🔀 Name Mapping / QoS
🛡️ Data Protection	ARP/AI / Snapshots / SnapLock / 📡 FPolicy / 🦠 Vscan / 🪞 SnapMirror
🖥️ Cluster	🔗 Peering / 🖥️ Cluster Information
🤖 Services	AI Settings

Each panel displays as a card grid, clicking navigates to the detail view. Follows System Manager's card-based navigation.

The Cluster category covers the part the AWS Management Console does not expose. Cluster peering and SVM peering had to be done from the ONTAP CLI or by hand-writing REST calls, which is a real operational cost for an AWS-centric team that otherwise never leaves the console.

New Panel Highlights

Vscan — "Zero to Configured" Setup Guidance

Vscan (antivirus scanning) has the highest setup barrier of any ONTAP feature — it requires external Windows/Linux servers and antivirus vendor licenses. When unconfigured, the Vscan panel displays a 5-step guided wizard:

Vendor selection (6-vendor comparison table with license page links)
NetApp Antivirus Connector download
EC2 deployment (architecture diagram + AWS Blog/GitHub samples)
ONTAP CLI commands (scanner-pool / policy / enable)
Verification in this panel

Even someone starting from "what is Vscan?" can follow the setup path directly from the portal.

One correction from the first version of this panel: the guidance only rendered while Vscan was disabled, so it disappeared the moment Vscan came up — exactly when you still want the scanner-side steps and the interoperability matrix link. It now stays reachable from a toolbar toggle in both states.

FlexClone — Instant Zero-Copy Clones

Create and split volume clones from the UI for ransomware recovery, forensics, or dev/test. Metadata-only copy completes in seconds with near-zero additional capacity.

SnapMirror — Replication Lifecycle Management

Monitor DR and cross-region replication relationships in the browser and execute actions (sync, break, resync, quiesce, resume, delete). Click a relationship to expand the last 10 transfer records (size, duration, success/failure). Lag times exceeding your RPO target display in red so they are noticed immediately.

DR planning note: SnapMirror Break/Resync operations are executable from the browser, but understand the RPO impact before clicking Break. After a break, the replication relationship is severed and resync requires delta transfer. For production environments, pair the confirmation dialog (implemented) with your team's DR runbook procedures.

FlexCache — Remote Read Cache CRUD

Create FlexCache volumes (async, with prepopulate path support), view the list (origin→cache arrow display), and delete (automated 3-step: unmount→offline→delete). When unconfigured, guidance is displayed with a datalist for selecting origin volumes.

Async polling note: FlexCache creation uses async polling to detect completion. If the user closes the browser during creation, the ONTAP-side operation continues. The created volume appears on the next list refresh — no data loss or inconsistent state results.

Cluster and SVM Peering — the gap the console leaves

Peering is a two-sided operation, and the authentication passphrase has to travel between the two clusters. The panel drives one side at a time.

The prerequisite that trips people up is the intercluster LIF, so it gets its own tab rather than being left as a footnote. Both clusters need at least one in up state, and TCP 11104, 11105 plus ICMP have to be allowed between their addresses — the intercluster LIF addresses, not the management LIF.

The flow is: check the LIFs, create the peer on one cluster with Generate passphrase, then enter that passphrase under Accept on the other. The generated value is shown once — ONTAP returns it in the creation response only, so the panel surfaces it in a banner you have to dismiss deliberately. Lose it and the peer has to be deleted and recreated.

SVM peers come after the cluster peer reaches available, and are accepted on the remote side without a passphrase. An SVM-level SnapMirror additionally needs the source subtype default and the destination dp_destination.

Cluster Information — nodes, licences, LIFs, protocols, DNS, jobs

Four tabs: overview (cluster name and version, nodes, licences), interfaces (LIF list with enable/disable), services (NFS, SMB and S3 state with enable/disable, plus the SVM DNS domains and servers), and jobs.

The overview tab is quieter than that list suggests. On the cluster I tested (ONTAP 9.17.1P7D1) both /cluster/nodes and /cluster/licensing/licenses returned zero records with no error, because AWS manages the cluster rather than the tenant. The first version of the panel rendered that as a bare "No nodes", which reads as a failure, so it now states that an empty list is expected here. The cluster name and ONTAP version come from /cluster and are still populated.

The jobs tab matters more than it looks. FlexCache creation, FlexClone split, SnapMirror transfers and peering all run as asynchronous ONTAP jobs, so this is where their progress and, more usefully, their failure reasons show up.

DNS note: an AD-joined SVM resolves its domain controllers through the servers set here. A wrong value breaks SMB, and on an AD-joined SVM it also makes S3 Access Point data operations fail with AccessDenied — while HeadBucket keeps succeeding, which sends you looking at IAM instead of the file system layer.

Destructive operations are gated in both layers

SnapMirror break, resync and delete, Vscan and FPolicy policy deletion, peer deletion, and disabling a LIF or a protocol all show an inline confirmation row before anything is sent.

The confirmation is not only in the UI. The Lambda refuses the same actions unless confirm=true is present, so a direct call that bypasses the browser is rejected identically. That split matters: a UI-only guard is a suggestion, not a control.

Getting this wrong is instructive. In the first cut, the Vscan and FPolicy delete buttons sent no confirm flag and had no confirmation row, and the handler did not check for one either — while the documentation already claimed the operation was confirm-gated. The button appeared to work and the docs looked right. The fix was to make all three agree, and to pin the contract in tests: for every confirm-gated action, assert it refuses without confirm and succeeds with exactly the parameters the UI sends. The second half of that assertion is the one that would have caught it.

Security Model

Layer	Implementation
Authentication	Cognito User Pool + MFA (TOTP/SMS)
Authorization (API)	Cognito Groups (`storage-admin` / `authenticated`)
Authorization (Files)	S3 AP + UNIX/Windows file system identity
Transport encryption	HTTPS (AppSync) + TLS 1.2 (ONTAP REST API)
Secrets management	Secrets Manager (ONTAP credentials)
Audit	CloudTrail S3 data events
WORM	SnapLock (Compliance/Enterprise) + S3 Object Lock
Ransomware protection	ONTAP ARP/AI + portal containment actions
PHI protection	Path-based guardrail (AI processing block)

Only users in the storage-admin group can execute admin operations (volume creation, SnapLock configuration, ARP containment, etc.). Regular users are limited to file browsing and AI processing.

Enterprise SSO note: Cognito User Pool supports SAML 2.0 and OIDC federation. To integrate with an existing IdP such as Okta, Azure AD, or Google Workspace, see Cognito's Adding SAML identity providers. Group mapping lets you map IdP groups to the Cognito storage-admin group.

Infrastructure protection note: Amplify Hosting is served via Amazon CloudFront, so AWS Shield Standard (DDoS protection) is automatically applied. If additional WAF rules are needed, attach AWS WAF to the CloudFront distribution.

Usage Scenarios

Scenario 1: Ransomware Response

1. Storage Dashboard shows 🚨 ARP Threats: 1
2. Navigate to ARP/AI panel → view threat details
3. Click [Contain] → SMB user blocked + snapshot taken
4. State transitions to 🟠 Contained
5. After forensic analysis, [Investigation Complete] → [Resolve]
6. Unblock to resume normal operations

Scenario 2: Compliance Audit

1. Lock panel → SnapLock tab to review Compliance volume list
2. Tamperproof tab to apply FISC 7-year lock to snapshots
3. S3 Object Lock tab to verify output bucket retention settings
4. Audit Trail to review operation history

Scenario 3: Daily Monitoring

1. Storage Dashboard — glance at 4 cards (30 seconds)
2. If any volume exceeds 85% capacity, drill into Volume Manager
3. Check EMS Events for recent alerts
4. No issues → done

Design Trade-offs

Design Decision	Benefit	Trade-off
localStorage (Incident State)	Simple deployment	Not shared across browsers; incident state can be lost if the browser crashes. DynamoDB migration recommended for production
PHI path regex	Simple, immediate	Requires folder naming convention compliance
Generic Dispatch (10 endpoints)	Avoids CFn 1MB limit	IAM policy granularity is coarser
VPC split	Cold start optimization	Two types of Lambda required
Promise.allSettled (Dashboard)	Resilient to partial failures	Failed cards show 0
Regulatory presets (hardcoded)	Ready to use immediately	Code change needed if regulations update

Rollback Procedure

Recovery paths if a portal update causes problems:

Situation	Rollback method
Frontend UI issue	One-click revert to the previous build from the Amplify Hosting console
Lambda function issue	`git revert` + `git push` triggers automatic redeploy (Amplify Gen2)
ONTAP configuration change issue	Restore directly via ONTAP REST API (export-policy, name-mapping, etc.)
Cognito configuration change	CDK stack rollback is not available. Revert manually to the previous state

Because frontend and backend are both under Git management, git revert + git push rolls back immediately. Some ONTAP-side changes (such as enabling SnapLock) are irreversible, so verification before execution matters.

8-Language Support (i18n)

All admin features included, the entire portal supports 8 languages:

Code	Language
`ja`	日本語
`en`	English
`ko`	한국어
`zh-CN`	简体中文
`zh-TW`	繁體中文
`fr`	Français
`de`	Deutsch
`es`	Español

Technical terms (ONTAP, SnapLock, FlexClone, S3 AP, ARP/AI, WORM, FISC, SOX, HIPAA) are not translated. Browser navigator.language auto-detection with instant switching via the header language picker.

Audit Log — Checking "Who Accessed What and When" in the UI

The most frequent request from compliance officers was "I want to check the file access trail myself." Grepping CloudTrail logs from the CLI is not realistic for anyone outside the security team.

The portal's Audit tab runs filtered queries:

Filter	Example
File path	Show only accesses under `/contracts/`
Event type	READ / WRITE / ALL
Time range	2026-07-01 – 2026-07-28

Architecture

AppSync Query → Lambda → Athena SQL → CloudTrail S3 Data Event logs

Light theme shown. A dark theme version is available.

CloudTrail S3 data events are registered as a Glue table and queried with Athena. Results display in table form, and access denials (AccessDenied) are highlighted in red.

Prerequisites

Enable data events for the S3 AP ARN on a CloudTrail Trail
Create the Athena table via Glue Crawler or a manual CREATE TABLE
Set ATHENA_AUDIT_DATABASE, ATHENA_AUDIT_TABLE, and ATHENA_AUDIT_OUTPUT as Lambda environment variables

If these are unconfigured, the UI displays "configuration required" guidance with the setup steps. The design treats this as onboarding rather than an error.

Data Retention

Query results are stored in the Athena workgroup output location (S3). Retention of the original CloudTrail logs is controlled by the S3 bucket lifecycle policy on the Trail's bucket. Set retention periods such as FISC 7 years or HIPAA 6 years according to your audit requirements.

Cost note: If you store audit trails in DynamoDB, storage costs grow in large environments. Set a retention policy (TTL) and archive old records to S3 Glacier. CloudTrail log retention is controlled separately via the Trail's lifecycle policy.

FlexClone Restore — Instant Recovery from Snapshots

When recovering from ransomware damage or accidental deletion, the requirement is "I want to go back to the snapshot from before the infection." Previously this meant creating a FlexClone from the CLI and attaching an S3 AP as a sequence of manual operations.

In the portal, from the 📸 button on the Files tab:

1. Enter the snapshot name (e.g., daily.2026-07-18_0010)
2. Click [Clone & Attach]
3. Step Functions automatically runs FlexClone creation → S3 AP attach
4. FlexClone information appears on the Results tab

FlexClone Status Display

🔄 FlexClone Volume
  Volume:  clone-uc6-20260718-abc123
  Parent:  vol_data
  Status:  🟢 online
  Created: 2026-07-18 15:00:00
  Size:    128 MB
  S3 AP:   clone-uc6-abc123-s3alias

Use Cases

Scenario	Purpose
Ransomware recovery	Immediate access to a pre-infection snapshot
Forensics	Isolate a point-in-time copy for investigation
Testing	Validate against a clone without touching production data

FlexClone is a metadata-only copy, so it completes in seconds with near-zero additional capacity. Differential capacity is consumed only as writes occur.

Accessibility note: Confirmation dialogs for FlexCache deletion and volume deletion are displayed inline. For screen reader users, confirmation messages are announced via role="alertdialog" and aria-describedby. Keyboard navigation (Tab/Enter/Escape) is fully supported.

Throughput note: A FlexClone volume shares the throughput budget with its parent volume. If clones are used actively in parallel in production, account for the performance impact on the parent volume. For read-oriented uses such as forensics this is typically not a concern.

Athena Query — SQL Analytics on NAS Data

When files on FSx for ONTAP are cataloged via Glue Crawler, you can run SQL queries using Athena. The portal's "Analytics" tab lets you enter a database name and SQL, and execute directly.

There is no need to open the AWS Athena console separately — everything completes inside the portal.

UI Layout

┌──────────────────────────────────────────────────┐
│ SQL Query (Athena)      [default        ]        │
├──────────────────────────────────────────────────┤
│ 💡 How to use: catalog files on FSx for ONTAP    │
│ with Glue Crawler, then analyze them here with   │
│ SQL.                                             │
│ ▸ 📝 View query examples (collapsible)           │
├──────────────────────────────────────────────────┤
│ SHOW TABLES IN default                           │
│                                                  │
│ [Run query]                                      │
└──────────────────────────────────────────────────┘

Usage Flow

First time: run SHOW TABLES IN default to discover available tables
Once you know table names: execute specific SELECT queries
Results: displayed in table form

Practical Query Examples

-- Find large files (capacity planning)
SELECT key, size FROM default.s3_objects
WHERE size > 1000000000
ORDER BY size DESC LIMIT 20

-- Total size of a specific folder (project sizing)
SELECT SUM(size) as total_bytes
FROM default.s3_objects
WHERE key LIKE 'engineering/%'

-- Files modified in last 7 days (change tracking)
SELECT key, last_modified
FROM default.s3_objects
WHERE last_modified > current_date - interval '7' day

The portal UI includes an expandable "View query examples" section, so even users unfamiliar with SQL can get started by copy-pasting.

Not just "browse the files" but "ask questions of the data" — directly from the portal.

File Formats Glue Crawler Recognizes

Category	Formats
Structured data	CSV, TSV, JSON, Parquet, ORC, Avro
Logs	Apache/NGINX logs, CloudTrail JSON
Documents	— (text extraction requires Textract or Bedrock)

CAD/EDA binary files cannot be cataloged by Glue Crawler, but file metadata (size, last modified, path) is still retrievable.

Athena Cost Optimization Tips

Athena bills on the volume of data scanned ($5/TB). Ways to keep costs down:

Date partitions (year=/month=/day=) reduce scan volume when a WHERE clause narrows the time range
With columnar formats such as Parquet/ORC, specifying only the columns you need instead of SELECT * reduces cost
Make it a habit to always add a LIMIT clause to exploratory queries
Setting a query limit on the Athena workgroup (for example 10 GB/day) caps unexpected scan volume

Cost Awareness

Rough additional cost if every feature in this article is enabled:

Resource	Estimate	Notes
VPC Lambda (Admin)	~$5/month	Depends on invocation frequency. Negligible for daily checks
CloudTrail S3 Data Events	~$10–$50/month	Proportional to file access volume
Athena queries	$5/TB scanned	Audit queries scan little; partitions reduce it further
Secrets Manager	$0.40/secret/month	One ONTAP credential
DemoMode	$0	No FSx for ONTAP needed. Runs on S3 alone

The cost of FSx for ONTAP itself (~$194/month at 128 MBps) was covered in Part 1. The portal's incremental cost depends on usage frequency, but evaluating in DemoMode adds nothing.

Deployment Time Estimates

Step	Time	Prerequisite
DemoMode deploy	~15 min	Amplify CLI installed
Add VPC + ONTAP connectivity	~30 min	FSx for ONTAP file system running
Audit Log configuration	~20 min	CloudTrail + Glue Crawler
Full feature deploy	~60 min	All of the above

What to Expect After Deployment

Once the DemoMode deploy completes, the portal is reachable at https://<branch>.amplifyapp.com. Cognito sign-up → log in → the file list shows test data from the S3 bucket
Once VPC connectivity is added, the Storage Dashboard shows actual volume counts and utilization, and protected volumes appear in the ARP panel
Once all features are deployed, the Audit Log tab can run CloudTrail queries and the Lock panel shows SnapLock/Object Lock settings

Environment Variables and Configuration Parameters

Parameter	Description	Example Value
`DEMO_MODE`	Run without FSx for ONTAP	`true` (PoC) / `false` (production)
`ONTAP_SECRET_ARN`	Secrets Manager ARN for ONTAP credentials	`arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:fsxn-admin-XXXXXX`
`ATHENA_AUDIT_DATABASE`	Glue database name for audit queries	`cloudtrail_logs`
`ATHENA_AUDIT_TABLE`	Athena table name for audit queries	`s3_data_events`
`ATHENA_AUDIT_OUTPUT`	S3 location for Athena query results	`s3://my-audit-results/athena-output/`

PoC → Production Checklist

After evaluating in DemoMode, use this checklist when moving to production connectivity:

Category	Verification
Network	VPC subnet contains the FSx for ONTAP ENIs; Security Group allows TCP 443 (management LIF). Note: NFS (2049) / SMB (445) are for data access and are not needed for the portal's admin operations
Auth	Cognito User Pool MFA enabled; `storage-admin` group members confirmed
Secrets	ONTAP credentials registered in Secrets Manager
Audit	CloudTrail Trail with S3 data events enabled for the S3 AP ARN
Backup	`portal-config.ts` values in Git (secrets excluded)
Cost	CloudTrail data event estimate confirmed (proportional to access volume)
SSO	(Optional) SAML/OIDC federation configured if using an enterprise IdP

Role-Based Documentation (8 Languages × 3 Guides)

Alongside the feature additions, role-specific documentation was built. Based on usability principles (visibility of system status, task-oriented structure, ease of error recovery), 3 guides are provided in 8 languages:

Guide	Target Role	Key Content
User Guide	End users	Sign in, file operations, AI processing, FAQ
Compliance Guide	Security/Compliance officers	ARP verification, WORM checks, audit trail, PHI guardrail validation, regulatory mapping (FISC/HIPAA/SOX/NIST/ISO)
Quick Reference	All roles	1-page cheat sheet (navigation, tasks by role, status indicators, troubleshooting)

Supported languages: 日本語, English, 한국어, 简体中文, 繁體中文, Français, Deutsch, Español

Design choices for the compliance officer guide:

All tasks executable without storage-admin privileges (read-only access is sufficient)
Each task includes "Evidence for auditors" instructions
Explicit "What you cannot do" escalation table showing who to contact

FlexCache / SnapMirror Management

Direct ONTAP REST API Operations from the Browser

FlexCache, FlexClone, SnapMirror, Vscan, FPolicy, cluster and SVM peering, and the cluster services are all managed through the browser UI — 110 actions in total. A VPC-deployed Lambda connects to the FSx for ONTAP management endpoint via HTTPS, calling ONTAP REST API with credentials from Secrets Manager.

Browser → AppSync (Cognito auth) → Lambda (in VPC) → ONTAP REST API
                                                       ↓
                                                Secrets Manager
                                                (fsxadmin credentials)

Light theme shown. A dark theme version is available.

FlexCache Create UI

Select the origin volume from a dropdown (datalist), specify cache name, size, and prepopulate paths, then create. The experience mirrors ONTAP System Manager — accessible from any browser with Cognito authentication.

SnapMirror Lifecycle Management

The replication relationship list shows state badges (✅ Snapmirrored / 🔴 Broken-off / 🔄 Transferring / ⏸️ Paused) with context-sensitive action buttons. DR failover (Break → Resync) is fully accessible from the browser. Lag times exceeding your RPO target display in red with a ⚠️ RPO warning.

FPolicy — Event Definitions and Notification Policies

FPolicy is where file access notification is configured: an event definition
names the protocol and the operations to watch, and a policy subscribes to those
events and points at an engine. The panel exposes all of it — create, enable,
disable, delete — across three tabs for policies, events and connections.

Note the Delete button on the enabled policy. ONTAP refuses to delete a policy
while it is enabled, so the panel disables the control rather than letting the
call fail. Disable first, then delete. The same ordering applies through the
REST API, and it is the kind of constraint that is easier to encode in the UI
than to explain in a runbook.

Where the Portal Is Self-Contained, and Where It Needs Something Else

The containment actions in this portal are a port of ontap_response.py from a
separate repository, fsxn-observability-integrations
— the module docstring records it. The ONTAP mechanisms are identical:
name-mapping deny, export-policy deny rule, protective snapshot, CIFS session
disconnect. What differs is the trigger, and the layers around it.

Self-contained in the portal

Capability	Requires
Review ARP/AI state, attack probability and suspect files	The portal reaching the ONTAP REST API
Block and unblock an SMB user or an NFS client IP	`storage-admin` group
Create a protective snapshot; lock a snapshot (WORM)	Same
Disconnect CIFS sessions	An AD-joined SVM
List active blocks and lift them individually	—
Block expiry and automatic lifting (24 hours by default)	The block ledger (DynamoDB) being reachable
Apply the same block across several SVMs (`svms` or `allSvms`)	—
Audit file access that went through the S3 Access Point	CloudTrail data events + Athena
FlexClone a snapshot to browse it; diff two generations	—

Every one of these starts when a person clicks. Nothing in the portal
contains a threat unattended — expiry is the one exception, and it only ever
works in the direction of ending a lockout.

Needs something outside the portal

Goal	What it takes
Contain without waiting for a human	An SNS topic plus a response Lambda
Be told about a detection instead of finding it	EMS → webhook → SIEM or an observability platform
Cut NFS off without the client cache window	A VPC NACL deny rule (network layer)
Judge a recovery point before restoring from it	A verification workflow (FlexClone + isolated scan)
Detect anomalies against a per-user ML baseline	A SIEM with anomaly detection, or a dedicated storage security product
Trace file access that arrived over NFS or SMB directly	An ONTAP audit log / FPolicy delivery pipeline

The audit boundary is the one most likely to be misread. The portal's audit trail
reads CloudTrail S3 data events for the S3 Access Point. Access that arrived over
NFS or SMB does not appear there at all — that needs ONTAP's own audit log or
FPolicy events. The two are complementary, not substitutes.

The delivery pipeline is still yours to build, but there is now somewhere for it
to deliver to. Once FPolicy (or Transfer Family) publishes to EventBridge, a
bridge Lambda writes what arrives into a notification table and a Folder
Watch section shows the events under the prefixes you are watching. It is
enabled by an admin toggle and off by default: until an admin states that a
publisher exists, turning it on would only ever show an inbox that cannot fill.
Filtering applies the Cognito group path boundary first and each user’s own
watches second. A watch is your own record, so you may register /, but that
cannot reveal anything outside the group boundary.

So the portal is the hands of incident response: it puts the ONTAP
containment actions in a browser, behind Cognito groups, with confirmation and an
audit trail. If you need something watching around the clock and moving those
hands for you, detection and response belong in a pipeline. Conversely, if you
already detect in a SIEM and only lack a way to stop it at the storage layer, an
SNS-triggered response Lambda fits that shape better than a portal button.

One click was too few

Writing this section is what exposed the gap worth fixing. A block removes a
principal's data access across the whole SVM. The first version executed on a
single click, with no confirmation and no confirm parameter in the payload —
the only guards were the Cognito group and a protected-accounts list. At the time
there was also no expiry, so a block stayed until a person lifted it.

There are now two gates. The UI asks, with wording specific to the action,
because the consequences differ: an NFS block is subject to client-side attribute
caching for up to 60 seconds, an SMB block is not, and disconnecting sessions on
its own does not stop the next login. The Lambda then refuses any call that
arrives without confirm: true, in front of the ONTAP client construction — a
dialog in a browser is a suggestion, and anything calling AppSync directly skips
it.

Session disconnect became its own action in the same pass. It existed in the
handler but only ever ran as step 4 of full containment, so an operator who had
already blocked a user had no way to cut the sessions that survived the block.
Order matters: block first, then disconnect. Disconnecting before the block is in
place just invites a successful reconnect.

Lessons Learned

Async Operations with `return_timeout=0`

FlexCache creation is an asynchronous ONTAP job taking 30-120 seconds. By specifying return_timeout=0, ONTAP returns immediately with 202 Accepted + job UUID, preventing Lambda timeout. The UI auto-refreshes the list at 10s/30s/60s intervals.

One Wrong Field Name Empties the Whole List

The SnapMirror list was requesting fields=...,last_transfer_type,last_transfer_size. last_transfer_size is not a field on /snapmirror/relationships, and ONTAP rejects the entire request when a single field name is unknown:

The value "last_transfer_size" is invalid for field "fields" (<field,...>)

So every relationship vanished — not with a visible error, but as an empty list. What made this survive the test suite is worth noting: the mock ONTAP in the unit tests returns records regardless of which fields are requested, so a test that asserts on the mapped response passes either way. The fix was to assert on the outgoing query string instead. Per-transfer byte counts were already available from /snapmirror/relationships/{uuid}/transfers, so nothing was lost by dropping the field.

Automated 3-Step FlexCache Deletion

Mounted FlexCache volumes cannot be deleted directly. The portal automates:

Remove junction path (unmount)
Set volume offline
Delete FlexCache (async)

Credential Synchronization

When changing the fsxadmin password, update both the FSx for ONTAP API and Secrets Manager simultaneously. Updating only one side causes authentication failures that can trigger ONTAP's account lockout mechanism. Recovery: aws fsx update-file-system password reset → Secrets Manager sync.

FSx for ONTAP Specifics

Item	FSx for ONTAP	On-prem ONTAP
Aggregate specification	Not needed (auto-selected)	Required
SVM creation	AWS API only	CLI/REST available
Intra-cluster FlexCache	Supported (no peering needed)	Supported
fsxadmin password reset	`aws fsx update-file-system`	`security login password`

Coexistence with Existing Tools

This portal is not a replacement for the ONTAP CLI or REST API. It makes a subset of those operations accessible from a browser UI with Cognito authentication — an additional management layer, not a substitute.

Scenario	Portal's Role
Using cloud file-sharing SaaS	AI processing and audit for large-scale NAS data only
Running self-hosted file sharing	Add S3 AP as External Storage for management operations
Using hybrid file services	Add data protection and ARP/AI visibility layer
Operating daily through the ONTAP CLI / REST API	Provide visibility to non-storage-admin team members

The portal provides a management layer for NAS-specific capabilities (Snapshot, FlexClone, FlexCache, SnapMirror, ARP/AI) accessible from the browser. Adoption can be incremental — management operations only, AI processing only, or the full suite — depending on existing workflows.

PoC → Production Migration Flow

After evaluating in DemoMode (no FSx for ONTAP required), migrate to production connectivity in three phases:

┌────────────────┐     ┌────────────────┐     ┌────────────────┐
│ Phase 1: PoC   │     │ Phase 2: VPC   │     │ Phase 3: Prod  │
│ (~15 min)      │     │   Connectivity │     │   Hardening    │
│                │     │ (~30 min)      │     │ (~60 min)      │
│ DemoMode=true  │ ──→ │ ONTAP mgmt LIF │ ──→ │ Least-privilege│
│ S3 bucket      │     │ Secrets Mgr    │     │ MFA required   │
│ Auth: Cognito  │     │ VPC Endpoint   │     │ WAF added      │
│                │     │                │     │ Audit enabled  │
└────────────────┘     └────────────────┘     └────────────────┘

Light theme shown. A dark theme version is available.

Phase	Time	Added Cost	What You Get	What Changes	What Stays the Same
1 (PoC)	~15 min	$0	UI/UX evaluation, AI processing test	Nothing (fresh deploy)	—
2 (VPC)	~30 min	Lambda VPC ~$5/mo	ONTAP admin operations, dashboard	Lambda VPC config added, Secrets Manager registered	Frontend UI, Cognito settings, S3 bucket config
3 (Prod)	~60 min	CloudTrail ~$10–50/mo	Audit, WAF, MFA, least-privilege IAM	IAM policy tightening, WAF attachment, MFA enforcement	Application code, ONTAP connection settings

Post-Deployment Verification

Expected state after each phase completes:

After Phase 1: you can log in to the portal, and DemoMode file listing and AI processing tests work. The Storage Dashboard still shows "not connected"
After Phase 2: the Storage Dashboard shows actual volume information, and ONTAP data appears in the ARP panel and EMS Events
After Phase 3: login is impossible without MFA, CloudTrail records all operations, and WAF applies rate limiting

Irreversible operations warning: The following operations cannot be undone once executed. Verify your organization's policies before enabling them in Phase 2+:

SnapLock Compliance enablement: Volume SnapLock type cannot be changed after creation

Tamperproof Snapshot retention period: Once set, the period cannot be shortened (only extended)

S3 Object Lock Compliance mode: No one, including root, can delete objects until retention expires

For detailed migration steps, see the PoC → Production Guide.

Authorization Model (PoC → Production)

Item	PoC	Production
AppSync auth	`allow.authenticated()`	`allow.groups(["storage-admin"])`
IAM resource scope	`"*"`	Specific ARN patterns
generateClient	`authMode: "userPool"` required	Same

In Amplify Gen2 with multiple auth providers, generateClient() must explicitly specify authMode: "userPool" to ensure the Cognito token is sent to AppSync. This is a recommended pattern per official documentation.

Resources

Summary and Next Steps

Building on the file portal from Part 1, ONTAP management operations and data protection features were embedded:

Feature	What it enables
Storage Dashboard	Health at a glance immediately after login
Incident Lifecycle	Ransomware response with state management
PHI Guardrail	Structurally prevents AI processing of regulated data
S3 Object Lock	Tamper prevention for AI results, configurable from UI
EMS Events	ONTAP alerts without CLI
Retention Presets	Just pick the regulation, get the right retention
Audit Log	"Who accessed what and when" — self-service for compliance
FlexClone Restore	Recover from snapshot in seconds
FlexCache CRUD	Create/delete read cache volumes from browser
SnapMirror Lifecycle	Sync, break, resync, quiesce — DR from a browser
Athena Query	Ask SQL questions against NAS data
Welcome Modal	First-time users productive in 10 seconds

From a portal that "just browses files" to one that "completes storage operations from the web." Daily monitoring and incident first response now finish in the browser without opening an ONTAP CLI session.

All code is available in the GitHub repository.

As a next step, after running DemoMode using the instructions in Part 1, try adding the VPC configuration to enable ONTAP connectivity. That gets you to a state where every feature in this article is usable. Migration details are collected in the PoC → Production Guide.

In Part 3, I'll write about embedding AI agents into this portal — completing file operations, analysis, and admin operations in natural language.

I hope this post helps someone out there.

See you next time.

Adding a File Portal to FSx for ONTAP S3 Access Points — Choosing Between Amplify Gen2 and Nextcloud (Part 1)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:16:29 +0000

Reach the NAS from a browser and from AI, without stopping existing mounts — choosing between Amplify Gen2 and Nextcloud (part 1 of 3)

A Japanese version of this article is available: 日本語版

Introduction

I've been asked more than a few times how to open files sitting on a NAS in a browser and run AI over them. The data lives on NFS/SMB volumes and runs to hundreds of terabytes — between the sheer volume, the latency, and data-residency rules, copying it out to an external service isn't realistic. Requiring an NFS mount instead puts the data out of reach for anyone who isn't an engineer. Closing that gap felt worth doing, so I built a verification environment for it.

What I used is Amazon FSx for NetApp ONTAP S3 Access Points (hereafter FSx for ONTAP S3 AP, or S3 AP — an access point that exposes NAS volumes for read/write through the S3 API). Because it reaches the data in place, you can add a browser UI and AI processing while leaving existing NFS/SMB workflows exactly as they are. I implemented the portal with two approaches: Amplify Gen2 as an AI-processing dashboard, and Nextcloud as a file-sharing UI.

Here's the conclusion up front:

S3 AP gives you browser access to NAS volumes with no data migration
Amplify Gen2 and Nextcloud aren't mutually exclusive — both can attach to the same volume at the same time
The deciding axis: Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
With DemoMode you can bring the portal up in under 30 minutes without FSx for ONTAP, and try file browsing and AI processing

In this article, I'll walk through the design of the two approaches and how to choose between them, the DemoMode startup steps, and the places where I got stuck during implementation.

Here's the overall picture first.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Figure 1: Overall architecture — two frontends reaching the same volume through one S3 Access Point

The part worth noticing is that neither frontend moves any data. The S3 Access Point reads the existing NFS/SMB volume in place, so the same files stay available to existing clients at the same time.

Approach	Role it plays
Amplify Gen2	AI processing dashboard (file selection → AI trigger → result display)
Nextcloud	File sharing UI (browsing, sync, sharing links)

Pick based on what you want to do. Both can connect to the same volume simultaneously, so you can start incrementally.

License: MIT License. Commercial use, modification, and redistribution permitted.

Repository: Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

Prerequisites and Positioning

Audience: Anyone with unstructured data on NAS who wants to protect and leverage that data. On-premises NAS, cloud NAS (FSx for ONTAP, EFS, Azure NetApp Files, etc.) — the platform doesn't matter.

File data on NAS shares several common challenges:

Challenge	Context
Data grows but stays underutilized	Files accumulate year over year — ownership unclear, labels missing, search difficult (CSA study)
Ransomware target	Shared drives are primary targets for encryption attacks. Without detection/stop mechanisms, damage spreads
Want AI but can't move data	Data volume, latency requirements, and data residency regulations make copying to external services impractical
Limited remote access	VPN + mount is required — browser-based access needs additional infrastructure
Audit & compliance overhead	Access logs exist but answering "who accessed what when" in a UI requires a separate analytics layer

This project implements approaches to these challenges using FSx for ONTAP S3 Access Points. FSx for ONTAP was chosen because it combines NFS/SMB/S3 multi-protocol access with integrated data protection: Snapshots, SnapLock (WORM), FlexClone, and ARP/AI (autonomous ransomware detection).

How you use the portal depends on your environment:

Your Environment	How to Use This Portal
Considering migration from on-prem NAS	Build FSx for ONTAP + S3 AP for browser access, AI processing, and data protection in one step
Already using FSx for ONTAP	Enable S3 AP on existing volumes — all portal features become available immediately
NAS + SaaS side by side	Keep SaaS as-is. Add AI processing, audit, and protection for NAS data
Using another cloud NAS	Connect via SnapMirror or DataSync to FSx for ONTAP, then add capabilities incrementally

What this portal provides:

Browser access to NAS files (no VPN)
AI analysis, classification, semantic search (Bedrock integration)
ONTAP-specific data protection (Snapshot, SnapLock, FlexClone, ARP/AI)
Audit trail visibility (CloudTrail + Athena)
Sharing links (Presigned URLs), QR code access

How it differs from SaaS file sharing:

No real-time co-authoring (like Google Docs simultaneous editing)
No desktop sync client (NFS/SMB mount serves this purpose)
No native mobile app (responsive web UI as alternative)

Note: Authentication is independent via Cognito (SAML/OIDC federation supported). No impact on existing SaaS contracts or licenses.

Current Setup	What this portal adds
Box for contracts	Tamperproof Snapshot on NAS originals (FISC 7-year retention)
SharePoint for design docs & reports	AI anomaly detection and classification on NAS files (one-click trigger)
Google Drive for team files	"Who accessed what when" audit trail (CloudTrail + Athena)
Dropbox for project file sync	Ransomware detection + automated containment (ARP/AI)
NFS/SMB direct access only	Browser-based file access for non-technical users + AI analysis

Industry-specific examples:

Financial services: "Hundreds of TB of trading logs and risk calculation results on NFS. Daily documents in SharePoint, but we need anomaly detection on logs and 7-year audit trail retention"
Manufacturing: "500TB of CAD files on NFS, daily document sharing via SharePoint. Add only AI quality inspection on CAD files through the portal"
Healthcare: "DICOM images stored on NAS, clinical data in a dedicated system. Need AI diagnostic support access to images with HIPAA-compliant retention management"
Media & entertainment: "Multiple PB of video assets (ProRes/MXF). Daily editing via NFS mount from Avid/Premiere, but need AI auto-tagging of content management metadata"
Legal: "Tens of thousands of contract PDFs accumulated on NAS. Need AI-powered content search, automated classification, and deadline visibility"
Research: "Hundreds of TB of genomics analysis and simulation results. Researchers want to search and preview results from a browser without using CLI"

Adding Web Experiences to NFS/SMB File Servers

NFS/SMB file servers have clear strengths: high throughput, low latency, multi-protocol support, and deep integration with workstation tools. These qualities make them the backbone of many organizations' workflows. Meanwhile, some of the web-based experiences that SaaS file sharing provides — browser access, AI integration, searchable audit trails — require additional tooling when working with file servers.

This portal adds those web experiences to existing file server environments without changing anything about how the servers operate:

Experience Added	Traditional Approach	How This Portal Delivers It
Browser access (no VPN)	VPN + mount configuration	S3 AP + Cognito auth (Internet-origin)
Natural language file search	grep or find on file names/paths	Bedrock Knowledge Base semantic search
Sharing links (expiring, password-protected)	Manual smbclient or email attachment	Presigned URL time-limited links + QR codes
Version history visibility	Snapshots exist but require CLI/API	Snapshot list in UI + FlexClone one-click restore
Audit trail in a UI	Parse auditd or FPolicy logs	CloudTrail + Athena self-service query
AI analysis & classification	Requires building a separate pipeline	Bedrock + Step Functions one-click trigger
Ransomware visibility	ONTAP ARP runs but checking needs the ONTAP CLI / REST API	ARP/AI dashboard in browser
Mobile / tablet access	Requires dedicated apps or VPN	Responsive web UI + QR code access
Compliance reporting	Run SnapLock CLI commands and compile manually	Lock panel with FISC/SOX/HIPAA presets + status overview

No data movement required. S3 AP reads and writes data directly on the NAS volume. Edit via NFS/SMB → browse via S3 AP in browser → AI results written back to the same volume. All the same data.

Note: Existing NFS/SMB workflows are unaffected. NFS mounts from CAD tools, SMB access from Avid, cron job file processing — all continue working exactly as before. S3 AP is an additional access path to the same volume.

Note: Authentication is independent from existing SaaS (Cognito or LDAP/SAML). No impact on existing SaaS contracts or licenses.

How to Choose

Selection Flow (Q1-Q3)

Q1. Do users need to operate files from a browser?
  → No: EventBridge Scheduler (scheduled batch) is sufficient
  → Yes: Go to Q2

Q2. Primary need: "file sharing & sync"? Or "AI/ML processing launch & result viewing"?
  → File sharing focused: Nextcloud (or Storage Browser for S3)
  → AI processing focused: Amplify Gen2 portal
  → Both: Use together (covered in this article)

Q3. Are users technical? Or non-engineers?
  → Technical: Any of the GUI portals above
  → Non-engineers: Add Amazon Quick (MCP, natural language operation)

Comparison

What you want to do	Suited approach	Trade-off
Trigger AI/ML processing from UI, see results in real time	Amplify Gen2	Custom code required, React knowledge assumed
Browse, upload, sync, and share files via web	Nextcloud	AI integration needs Webhook development
Zero-custom-code file operation UI	Storage Browser for S3	Limited features (browse/DL/UL only)
Natural language file search & analysis	Amazon Quick + AgentCore MCP	Preview stage, region availability limited
No frontend needed, batch is fine	EventBridge Scheduler	No interactive operation

How to choose: Amplify Gen2, Nextcloud, Storage Browser for S3, and Amazon Quick each suit a different context. The trade-offs above are listed symmetrically, including the constraints of the option I reached for, and the article records what actually worked on technical grounds. Read it as material for choosing against your own requirements.

Cost Estimates

Approach	Monthly Cost	Includes
Amplify Gen2 portal	~$18-46	Cognito/AppSync/Lambda (within Free Tier) + AI pay-per-use
Nextcloud (EC2)	~$70-110	EC2 + RDS + ALB. Local Docker = $0
Storage Browser for S3	~$5	Amplify Hosting only
Quick + AgentCore MCP	~$5-15	Lambda execution only
EventBridge direct	~$1-5	Lambda + Step Functions execution only

Cost note: FSx for ONTAP infrastructure (~$194/month, 128 MBps configuration) is common across all approaches. The portal itself adds only tens of dollars per month. At 100 users, Cognito/AppSync/Lambda costs remain within pay-per-use bounds (serverless). At 1000 users, FSx for ONTAP throughput scaling (128→256 MBps, +~$194/month) is the primary scaling cost.

Approach 1: Amplify Gen2 — AI Processing Dashboard

What is Amplify Gen2 (Key Points for NAS Users)

AWS Amplify Gen2 is a full-stack development framework that lets you define both frontend (UI) and backend (auth, API, database, storage) in TypeScript and deploy to AWS.

What this means for NAS/file server users:

Aspect	What Amplify Gen2 Handles
No infrastructure management	No servers to provision, patch, or scale. Everything runs serverless (Lambda, AppSync, Cognito)
Built-in authentication	Cognito provides email login, MFA, and enterprise SSO (SAML/OIDC) out of the box. No custom auth system to build
Direct AWS service integration	Connect to S3 Access Points, Bedrock (AI), Step Functions (workflows), Athena (analytics) with minimal code. No need to design IAM from scratch
Git-based deployment	`git push` triggers automatic build and deploy. No separate CI/CD pipeline required
Per-developer sandboxes	Isolated environments auto-created per developer. Team members work in parallel without conflicts
CDK extensibility	When standard features aren't enough, add any AWS resource (VPC Lambda, DynamoDB, etc.) via AWS CDK

In short: when you want "browser UI + AI processing + authentication on NAS data," Amplify Gen2 delivers this without building or maintaining web servers. You write UI in React; AWS managed services handle the backend.

Architecture

┌───────────────────────────────────────────────────┐
│  Amplify Gen2                                     │
│  ┌──────────┐  ┌───────────────────────────────┐  │
│  │ Cognito  │  │ AppSync (Generic Dispatch)    │  │
│  │ + MFA    │  │  fileQuery → ListFiles λ      │  │
│  │ + Groups │  │  adminQuery → ResourceMgmt λ  │  │
│  └──────────┘  │  arpQuery → ArpResponse λ     │  │
│                │  protectionQuery → Snapshot λ │  │
│                └──────────────┬────────────────┘  │
└───────────────────────────────┼───────────────────┘
                                │
          ┌─────────────────────┼────────────────────┐
          │                     │                    │
          ▼                     ▼                    ▼
┌──────────────────┐  ┌─────────────────┐  ┌────────────────┐
│ S3 Access Point  │  │ ONTAP REST API  │  │ Step Functions │
│ (Internet-origin)│  │ (inside VPC)    │  │ (AI/ML jobs)   │
└──────────────────┘  └─────────────────┘  └────────────────┘

Light theme shown. A dark theme version is available.

Figure 2: Amplify Gen2 architecture — Lambda outside the VPC reads and writes the volume through the S3 Access Point

Keeping Lambda outside the VPC and letting it call the S3 Access Point directly is the crux of this design. Here's why.

Key Design Decisions

VPC split architecture:

Non-VPC Lambda: Reads/writes files via S3 AP (Internet-origin). Fast cold starts.
VPC Lambda: Calls ONTAP REST API (management LIF). Must be inside VPC.
Never combine both in one Lambda (Internet-origin S3 AP unreachable via S3 Gateway VPC Endpoint).

Network note: VPC Lambda handles admin operations only. Required VPC Endpoints: Secrets Manager and CloudWatch Logs. S3 AP data access uses non-VPC Lambda, so NAT Gateway is not needed.

Generic Dispatch pattern: 155 operations consolidated into 10 generic endpoints (action + params: AWSJSON) to avoid CloudFormation 1MB limit.

Extensibility note: To add new operations, simply add an action entry to the Lambda handler's dispatch map. No AppSync schema or CloudFormation changes needed. See the PSTK Action Catalog for the full action list.

UI Layout (Sidebar)

┌───────────────┬─────────────────────────────────────┐
│ Browse        │                                     │
│  📂 All Files │  [Main Content Area]                │
│  ⭐ Favorites │                                     │
│  🕐 Recent    │                                     │
│  🔔 Watch     │                                     │
│  📤 Upload    │                                     │
│               │                                     │
│ AI & Proc.    │                                     │
│  ⚡ AI Proc.  │                                     │
│  🤖 AI Chat   │                                     │
│  🔍 Search    │                                     │
│  📋 History   │                                     │
│  📊 Analytics │                                     │
│  🗂️ Agent Dir │                                     │
│               │                                     │
│ Data Prot.    │  + Right Panel: AI Assistant        │
│  📸 Snapshots │  (appears on file selection)        │
│  🔒 Lock      │                                     │
│  🛡️ ARP/AI    │                                     │
│               │                                     │
│ Admin         │                                     │
│  🔧 Resources │                                     │
│  🔄 Version   │                                     │
│  🔍 Audit     │                                     │
└───────────────┴─────────────────────────────────────┘

4 groups × 17 sections. Same sidebar pattern as Google Drive and Box.

Key Features

File selection → AI processing trigger → real-time result display (5s polling)
Bedrock Q&A, Rekognition image analysis, Athena SQL, Textract OCR
FlexClone snapshot restore (one-click from UI)
Job execution history (DynamoDB, per-user tracking)
8-language i18n (JA/EN/KO/ZH-CN/ZH-TW/FR/DE/ES)
Keyboard navigation + ARIA labels + Dark/Light mode
Responsive (hamburger menu at 768px, tablet-ready)

Audit note: All operations are recorded in CloudTrail. Audit log retention is controlled via CloudTrail settings (default 90 days, extendable to 7 years for compliance). Self-service Athena queries available from the portal's Audit section.

Security note: Blast radius if a Cognito token is compromised — tokens are scoped by Cognito Groups via AppSync resolver authorization; access is limited to resources within the user's group. Default token expiry: 1 hour. MFA-required setting mitigates leakage risk.

Gotchas

AppSync APPSYNC_JS runtime: new Date() not available → use util.time.nowISO8601()
Data Source must be in the same CDK stack as AppSync API ("Data source not found" otherwise)
Bedrock Nova Lite: requires converse() API (invoke_model + inputText is Titan-only)
Presigned URLs: need explicit SigV4 + regional endpoint

Approach 2: Nextcloud — File Sharing UI

What is Nextcloud (Key Points for NAS Users)

Nextcloud is a self-hosted, open-source file sharing and collaboration platform. It provides Google Drive/Dropbox-like user experience on infrastructure you control.

What this means for NAS/file server users:

Aspect	What Nextcloud Handles
Data sovereignty	File data stays on your servers. No need to entrust data to external SaaS
Direct NAS connection	External Storage feature mounts S3 AP / NFS / SMB / WebDAV as additional folders. No data copy needed
Familiar experience	Desktop sync client (Windows/Mac/Linux), mobile apps, sharing links — SaaS-equivalent UX
No per-user licensing	AGPL-3.0, free to use. No per-seat pricing
Auth integration	Supports LDAP, SAML, OIDC. Connects to existing Active Directory

When you configure FSx for ONTAP's S3 AP as Nextcloud's External Storage backend, ONTAP volume files appear in Nextcloud's folder listing. Files saved via NFS/SMB are immediately visible in the browser and syncable to desktop clients.

Architecture

# Nextcloud's External Storage App mounts S3 AP as S3-compatible storage
Nextcloud (Docker) → External Storage App (S3 backend) → S3 AP → FSx for ONTAP

Light theme shown. A dark theme version is available.

Figure 3: Nextcloud architecture — the External Storage App mounts the S3 Access Point as S3-compatible storage

Pass the S3 AP alias as the bucket name, and ONTAP volume contents appear in Nextcloud's file listing.

Setup (~5 minutes)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or bucket name (e.g., "my-volume-ap-s3alias")
export S3_BUCKET=<YOUR_S3AP_ALIAS>

# Configure External Storage to mount ONTAP volume via S3 AP
make configure-s3

# Verify connection (success = file listing returned)
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from your ONTAP volume (or S3 bucket) via S3 AP.

Key Features

Desktop sync client (Windows/Mac/Linux)
Sharing links (password-protected, time-limited)
Comments & annotations
WebDAV access
LDAP/SAML authentication

Trade-offs

Strengths	Considerations
Mature NAS file sharing tool	AI/ML integration needs Webhook development
Desktop sync + mobile apps	EC2/RDS operational cost (production config)
Free under AGPL-3.0 (Docker as-is)	Source disclosure obligation if server code is modified

Gotchas

use_path_style=true required (S3 AP alias only works with path-style addressing)
occ files_external:config must be run once per parameter ("Too many arguments" otherwise)
Empty credentials cause IMDS (169.254.169.254) fallback timeout in Docker

Cost

Local Docker: $0. Production (EC2 + RDS + ALB): ~$70-110/month.

Coexistence Pattern: Amplify + Nextcloud

They are not mutually exclusive — both can access the same FSx for ONTAP volume simultaneously.

Feature Comparison

Feature	Nextcloud	Amplify Gen2
File browsing	✅	✅
File DL/UL	✅	✅ (Presigned URL, 50GB object limit)
Desktop sync	✅	△ (Use NFS/SMB instead)
Sharing links (time-limited)	✅	✅ (Presigned URL)
AI/ML processing workflows	⚠️ Webhook needed	✅
File Q&A (Bedrock)	❌	✅
FlexClone restore	❌	✅
ONTAP admin operations	❌	✅

Expected Usage Flow

1. Team uses Nextcloud daily for file browsing & sharing
   (same data visible as NFS/SMB users)

2. When AI processing is needed on a specific folder,
   launch it from the Amplify portal

3. Result files are written back to the same volume,
   instantly visible from Nextcloud and NFS

Light theme shown. A dark theme version is available.

Figure 4: Running both — AI processing and file management share one S3 Access Point

Authentication is independent (Nextcloud: LDAP/SAML, Amplify: Cognito). For gradual adoption, set up Nextcloud first, then add Amplify when processing needs arise.

Data Consistency

ONTAP guarantees strong consistency across multi-protocol access. A PutObject via S3 AP is immediately visible from NFS/SMB after write completion. The reverse is also true. Whichever portal writes, all see consistent data.

Storage operations note: Your existing procedures (volume creation, snapshot management, and so on) remain unchanged. This portal is an end-user-facing AI processing and file access layer; administrators carry on with the AWS Management Console / FSx API and the ONTAP CLI / REST API. Those three are the management interfaces reachable for FSx for ONTAP — ONTAP System Manager is not among them (management interfaces).

Comparison with Alternatives

Multiple approaches exist for "accessing NAS data from a browser." Each has trade-offs:

Approach	Suited for	Trade-off
This portal (S3 AP + Amplify/Nextcloud)	AI processing + admin ops without data movement	Custom code required, hours of initial setup
DataSync + S3 + any portal	Leveraging S3-native ecosystem	Data copy, sync lag, storage cost duplication
Transfer Family SFTP	Existing SFTP client compatibility	No web UI, AI integration needs extra development
EFS + Lambda	Serverless simplicity	No ONTAP-specific features (SnapLock, FlexClone, ARP)
Full SaaS (Box/SharePoint etc.)	Zero infrastructure, immediate availability	NAS data copy required, limited data protection features

The S3 AP characteristic is "access data via S3 API without moving it." No data copy means no consistency issues or double storage costs. However, it's not "sign up and use immediately" like SaaS.

How to choose: DataSync + S3, Transfer Family SFTP, EFS + Lambda, and SaaS file sharing each suit a different context. The table above also carries the constraints on the FSx for ONTAP S3 AP side (custom code required, a few hours to stand up). Read it as material for choosing against your own requirements.

Try It (~30 Minutes)

Prerequisites

Tool	Version	Check Command	Time
Node.js	20.x or later	`node --version`	0 min if installed
npm	10.x or later	`npm --version`	Bundled with Node.js
AWS CLI	2.x	`aws --version`	~2 min
AWS Account	—	`aws sts get-caller-identity`	0 min if authenticated
Docker	24.x or later	`docker --version`	Nextcloud only

FSx for ONTAP is NOT required for DemoMode. You can verify portal behavior with a regular S3 bucket.

Fastest Path (~5 min, copy-paste 4 lines)

If Node.js and AWS credentials are ready:

# Clone repository and navigate to portal directory
git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git && cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/amplify-portal

# Install dependencies (~2 min)
npm install

# Copy config (DemoMode default — no edits needed)
cp amplify/portal-config.example.ts amplify/portal-config.ts

# Start sandbox (first run also creates Cognito users, ~3 min)
npx ampx sandbox

What to expect: Terminal shows ✅ Deployment complete, then http://localhost:5173 opens the portal. You can browse files and access AI processing screens from the sidebar.

portal-config.ts Key Parameters

// portal-config.ts — parameter explanations
export const portalConfig = {
  // S3 AP alias (empty string = DemoMode, uses regular S3 bucket)
  // Production example: "my-ontap-vol-ap-s3alias"
  s3ApAlias: "",

  // VPC ID (empty = no VPC Lambda = DemoMode)
  // Production example: "vpc-0123456789abcdef0"
  vpcId: "",

  // Cognito group-to-S3AP mapping (different access per group)
  // Example: analysts group gets analysis AP only, admins get full access
  groupApMapping: {
    // "analysts": "analysis-vol-ap-s3alias",
    // "admins": "admin-vol-ap-s3alias"
  },
};

CI/CD note: Amplify Gen2 auto-deploys on git push. git push origin main triggers Amplify Hosting build/deploy. No separate CI/CD pipeline needed.

Nextcloud Setup (~5 min)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or S3 bucket name
export S3_BUCKET=<YOUR_BUCKET_OR_AP_ALIAS>

# Configure External Storage
make configure-s3

# Verify connection
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from the ONTAP volume (or S3 bucket) via S3 AP.

About DemoMode

Leave VPC settings empty in portal-config.ts to experience the portal UI and AI processing flow without FSx for ONTAP.

Feature	DemoMode	Production
Sidebar UI / navigation	✅	✅
File browsing	⚠️ (S3 bucket)	✅ (S3 AP)
AI processing	⚠️ (needs Step Functions)	✅
ONTAP admin panels	⚠️ (shows "connection required")	✅

For production connectivity, see PoC → Production Guide.

Role-Based Documentation (8 Languages)

Guide	Audience	Content
User Guide	End users	Sign in → file ops → AI → FAQ
Compliance Guide	Security/Compliance officers	ARP verification → WORM checks → audit trail
Quick Reference	All roles	1-page operations cheat sheet

Demo note: Minimum 30-minute demo setup is "4-line fastest path → DemoMode launch → file browsing → AI processing screen." No FSx for ONTAP pre-build required.

Operational Considerations

File-Level Access Control

S3 AP access is governed by ONTAP's UNIX permissions (UID/GID) or Windows ACLs. The File System Identity specified when creating the S3 AP determines "who" the access runs as.

# Example: separate team access with multiple S3 APs
S3 AP "portal-readonly"  → UID 1001 (read-only)
S3 AP "portal-analyst"   → UID 1002 (analysis team, specific dirs only)
S3 AP "portal-admin"     → UID 0    (admin, full access)

Throughput Sharing

S3 AP access shares the same FSx for ONTAP throughput budget as NFS/SMB. Normal portal usage (directory listing + individual reads) is not an issue, but if heavy parallel reads coexist with NFS workloads, monitor CloudWatch ThroughputUtilization.

Options if throughput is insufficient:

Scale up throughput capacity (128→256/512/1024/2048 MBps)
Set QoS policy to cap portal volume throughput

Multi-Account Environments

This portal is designed for single-account use. AWS Organizations considerations:

S3 AP: Cross-account access requires AP resource policy
Cognito: Multi-account user consolidation recommended via SAML federation
Tenant isolation: groupApMapping assigns different S3 APs per group

About Presigned URLs

AWS documentation lists Presign as "Not supported," but they work in practice (confirmed with AWS Support). Presigning is client-side SigV4 signature calculation, and since GetObject is Supported, there's no structural way to block it. However, production reliance is at your own risk since documentation says "Not supported." Set short expiry (5 minutes or less recommended).

Scaling Characteristics

Component	Scaling	Downtime
Lambda / AppSync / Cognito / DynamoDB	Automatic (serverless)	None
FSx for ONTAP	Manual throughput/storage expansion	Minutes during throughput change

Portal components are all serverless — the only scaling bottleneck is FSx for ONTAP throughput.

Data engineering note: File metadata is available via CloudTrail + S3 AP access logs. Queryable with Athena, but building a dedicated metadata catalog (Glue Data Catalog etc.) requires separate design.

Series Structure (3 Parts)

Part	Theme	Key Content
Part 1 (this article)	Portal foundation	Amplify Gen2 vs Nextcloud, coexistence patterns, DemoMode
Part 2	Storage operations	ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore
Part 3	AI agent integration	AgentChat, semantic search, multi-agent teams, HITL

FAQ

Q: I'm not using FSx for ONTAP yet. Can I try this?
Yes. DemoMode uses a regular S3 bucket to demonstrate the portal's UI and AI processing flow. FSx for ONTAP can be added later.

Q: Do I need to replace my existing Box/SharePoint?
No. This portal is an additional layer for AI processing, data protection, and audit on NAS data.

Q: How much does it cost?
DemoMode is free (within AWS Free Tier). Production portal add-on: ~$5–50/month. Main cost is FSx for ONTAP infrastructure (~$194/month minimum).

Q: What about security?
Cognito MFA + Groups role separation, HTTPS, Secrets Manager. Enterprise SSO (SAML/OIDC) supported via Cognito federation. Part 2 covers the security model in detail.

Q: Can it handle large files (multiple GB)?
S3 AP upload has a 50GB object size limit (a single PutObject is capped at 5GB, so Storage Browser switches to multipart above that). For files larger than 50GB, place them on the volume via NFS/SMB and use the portal for browsing/processing only.

Q: Does this affect my existing operational procedures?
No. This portal is an end-user-facing additional layer. Operating through the ONTAP CLI / REST API and the AWS Management Console continues exactly as before.

Verified Environment

Component	Version
Node.js	20.18.x
React	18.3.x
Amplify Gen2	1.x (as of 2026-07)
Python (Lambda)	3.12 (ARM64)
FSx for ONTAP	ONTAP 9.17.1
Region	ap-northeast-1
Verified	2026-07

Resources

GitHub: FSx for ONTAP S3 AP Serverless Patterns — This repository (MIT License)
User Guide (8 languages) — End-user daily operations guide
Compliance Guide (8 languages) — ARP/WORM/audit trail verification procedures
PoC → Production Guide — DemoMode to production connectivity migration
Scaling Guide — Capacity planning and throughput design
Nextcloud External Storage Setup — Nextcloud + S3 AP detailed configuration
S3 AP Compatibility Notes — API support status and known constraints
DemoMode Guide — Running without FSx for ONTAP
AWS: FSx for ONTAP S3 Access Points — Official documentation

Summary and Next Steps

Point	Conclusion
Data migration	Not needed — S3 AP reads the same volume NFS/SMB already uses
Portal choice	Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
Running both	Both can attach to the same volume at once. Authentication stays independent
How to try it	DemoMode takes about 30 minutes and needs no FSx for ONTAP
Existing workflows	NFS/SMB mounts and ONTAP CLI / REST API operations continue unchanged

For next steps, start with the 4-line fastest path in DemoMode and spend a few minutes in the sidebar UI and the AI processing screen. Once you can see which approach fits, the PoC → Production Guide covers moving to a production connection.

Part 2 covers storage operations (ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore), and Part 3 covers AI agent integration (AgentChat, semantic search, multi-agent teams, HITL).

I hope this post helps someone out there.

See you next time.

KNFSD File Cache FSx for ONTAP: NFS Read Speedup + S3 Access Points Dual-Path

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Thu, 23 Jul 2026 12:59:50 +0000

Introduction

In July 2026, AWS released KNFSD File Cache as Preview — an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems.

This is the first time an NFS read-acceleration layer has shipped as an official AWS solution. Working daily with FSx for ONTAP NFS performance for EDA and VFX workloads, my reaction was immediate: spin up a verification environment and see what happens.

What I tested was a Dual-Path architecture: KNFSD caching NFS reads from the same FSx for ONTAP volume that S3 Access Points expose to serverless Lambda processing. The question: can compute fleets read at local-NVMe speed while Lambda runs AI/ML workloads on the same data, without bandwidth contention?

Here's what I found:

Dual-Path E2E test passed: A file written via S3 AP was immediately readable through the KNFSD NFS cache
Cache effect: first read 55ms → subsequent reads 2ms (28x improvement), proxy cache serves 422-619 MB/s
NFSv4.1 is mandatory: NFSv3 re-export causes Stale file handle on file creation (kernel-level filehandle size constraint)
FSID_MODE=local + SQLite on FSx for ONTAP: FSID persistence at zero additional cost

This post covers every pitfall I hit during deployment, and why this architecture still makes sense despite those issues.

When to Consider This — and When Not To

This architecture suits workloads that:

Need NFS read bandwidth beyond FSx for ONTAP's provisioned throughput (EDA DRC/LVS, VFX rendering, genomics)
Want both NFS (high-speed reads) and S3 API (Lambda AI/ML post-processing) on the same data
Use Spot instances for burst compute and need the cache to stay warm through reclamation
Unify multiple NFS sources (FSx for ONTAP + on-premises NAS + other clouds) into one cache layer

Consider alternatives when:

FSx for ONTAP throughput scaling alone covers your bandwidth needs (steady-state workloads)
Write caching is needed (FlexCache offers configurable write-back/write-around)
You want fully managed operations with no EC2 to run (FlexCache or Amazon File Cache)
Production SLA is non-negotiable (KNFSD is Preview — wait for GA)

On vendor neutrality: This post evaluates KNFSD File Cache on technical merits only. FlexCache, Amazon File Cache, EFS, and other options each suit different contexts. The goal here is to document what actually works, not to rank solutions.

What is KNFSD File Cache?

KNFSD File Cache is an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems. NFS re-export means mounting a remote NFS share and publishing it as a new NFS export to downstream clients — the proxy sits transparently between source and consumers. Wētā FX (Avatar: The Way of Water) and ILM used the predecessor project in production.

NFS Source (FSx for ONTAP)
     │ NFS mount (NFSv4.1, fsc)
     ▼
KNFSD Proxy (EC2, NVMe L2 cache)
     │ NFS re-export (NFSv4.1)
     ▼
Compute Fleet (Spot × hundreds)

L1 cache: Linux Page Cache (RAM — 5-9 GB/s on 16 GB in this test)
L2 cache: FS-Cache (local NVMe — survives reboot, 106 MB/s measured)
Writes: write-through (immediately forwarded to source)
Scaling: ASG + DNS round-robin or NLB
Prerequisite: source must be an NFS v3/4.x server (parallel filesystems like Lustre/GPFS/WekaFS are not supported)

The key difference from FlexCache: KNFSD is a protocol-level transparent proxy that doesn't require the source to be ONTAP. It can unify FSx for OpenZFS, on-premises NetApp/Pure/Dell/DDN (A3I), and other-cloud NFS servers (via Direct Connect / Cloud Interconnect) into a single cache layer.

Test Environment

Component	Details
Region	ap-northeast-1
Source	FSx for ONTAP (128 MBps, Single-AZ, ONTAP 9.17.1)
KNFSD Proxy	m6gd.xlarge (arm64 Graviton, 16 GB RAM, 237 GB NVMe)
Kernel	7.1.3-knfsd (Packer custom build)
FSID Mode	local (SQLite on FSx for ONTAP NFS mount)
NFS Version	NFSv4.1 (source → proxy → client)
Test Client	t4g.micro (AL2023, 916 MB RAM)
S3 AP	Internet-origin, UNIX identity (root)

Cost note: KNFSD-specific incremental resources only (Packer Spot build ~$0.30 + m6gd.xlarge 1hr ~$0.29). Prerequisites: a running FSx for ONTAP (~$194/mo minimum) + VPC infrastructure. If you already operate FSx for ONTAP, the incremental evaluation cost is effectively < $1.

Dual-Path E2E Test Results (Verified)

Architecture

[Lambda / CLI]                    [Compute Fleet]
     │                                 │
     │ S3 PutObject                    │ NFS read (NFSv4.1)
     ▼                                 ▼
┌──────────────────┐         ┌──────────────────────┐
│ S3 Access Point  │         │ KNFSD File Cache     │
│ (Internet-origin)│         │ (m6gd.xlarge, NVMe)  │
└────────┬─────────┘         └──────────┬───────────┘
         │                              │
         │  ← same volume →             │ NFS mount (NFSv4.1)
         ▼                              ▼
┌──────────────────────────────────────────────────┐
│ FSx for ONTAP Volume (/vol1, UNIX security style)│
└──────────────────────────────────────────────────┘

Results

Test	Result	Details
S3 AP write → KNFSD NFS read	✅	Immediate reflection, content match
NFS write → S3 AP read (MD5)	✅	`c092ef65...` match both directions
S3 AP batch 50 files → NFS bulk read	✅	50 files in 107ms (2.1ms/file)
S3 AP multipart 50MB → NFS read	✅	MD5 verified
NFSv4.1 write via KNFSD	✅	Write-through reflects immediately
fsidd + SQLite persistence	✅	SQLite created on FSx for ONTAP
Proxy restart → client continues	✅	NFSv4.1 grace period, no remount needed

# Actual test log
$ aws s3 cp test.txt s3://arn:aws:s3:ap-northeast-1:XXXX:accesspoint/knfsd-dualpath-test/s3ap-e2e-1784803253.txt
# → upload: success

$ cat /mnt/knfsd/vol1/s3ap-e2e-1784803253.txt  # NFSv4.1 via KNFSD
dual-path-e2e-test-1784803253
# → read success ✅

Performance Numbers

Test environment note: These numbers are from m6gd.xlarge (237 GB NVMe, single drive). Production-recommended instances (im4gn.16xlarge: 30 TB NVMe RAID, i3en.24xlarge: 60 TB) are expected to deliver significantly higher L2 NVMe throughput (estimated 2-8 GB/s sequential).

3-Layer Cache Hierarchy

Layer	Throughput	Use Case
L1: Proxy RAM (page cache)	5.0-9.1 GB/s	Hot data, repeated reads
L2: NVMe (FS-Cache)	106 MB/s	Warm-up shortcut after proxy restart
Source: FSx for ONTAP (NFS fetch)	18-19 MB/s	Cache miss, initial fetch

Important note on source performance: The 18-19 MB/s figure reflects test-time NFS mount constraints (NFSv3 single-stream, rsize=64K) — not FSx for ONTAP's throughput limit (128 MBps). Switching the source mount to NFSv4.1 + rsize=1M is expected to significantly improve cache miss performance (proposed upstream in Issue #42).

L2 NVMe is 5.6x faster than source fetch. After a proxy reboot with empty RAM, data is served from NVMe cache immediately.

Throughput

Operation	Throughput
Sequential read (proxy L1 → client)	422-619 MB/s
Large write (write-through)	157-218 MB/s
S3 AP multipart upload (50 MB)	36.6 MiB/s

Latency

Operation	Cold	Cached	Improvement
10 MB file read	55 ms	2 ms	28x
4 KB small file	1.75 ms	1.5 ms	14%
S3 AP batch → NFS	—	2.1 ms/file	—

Working Set > Client RAM

With client RAM at 916 MB and a 500 MB dataset, reading after drop_caches still achieves 422-428 MB/s — served from the KNFSD proxy's 16 GB RAM cache. This is the core value proposition: small compute nodes can access datasets larger than their local memory at proxy-cache speed.

nconnect Caveat

nconnect	100 MB cold read
1 (default)	619 MB/s
16	184 MB/s

On t4g.micro (5 Gbps NIC), nconnect=16 is counterproductive. Only effective on 100 Gbps instances (c5n.18xlarge, hpc7g).

Cache Coherency

S3 AP write visibility through KNFSD depends on NFS attribute cache timeouts:

Parameter	Default	Effect
`acregmin`	3s	Minimum file attribute cache
`acdirmax`	60s	Maximum directory attribute cache

In our tests, S3 AP-written files were immediately visible through KNFSD (new files bypass directory attribute cache). For frequently-overwritten files, set acregmax=5-10 to reduce stale-data windows.

Every Pitfall We Hit (Documenting All of Them)

NFSv3 Breaks Writes (Most Important Finding)

This consumed the most debugging time. When mounting the KNFSD proxy via NFSv3, existing file reads succeed but new file creation returns Stale file handle.

The cause is documented in the Linux Kernel NFS re-export documentation:

reexport server's filehandle for the reexported object will be X+22 bytes

FSx for ONTAP filehandles are ~40+ bytes. After +22 bytes = ~62+ bytes. New file creation (which adds subdirectory inode information) pushes past NFSv3's 64-byte limit.

NFSv4.1 supports 128 bytes — ample headroom. Our Terraform validates against nfs_version="3" with a clear error message linking to the upstream issue.

Filed upstream as Issue #40.

Other Pitfalls

Problem	Cause	Fix
proxy-startup.sh immediate failure	No EC2 API access (no Public IP)	`assign_public_ip = true`
SSM `GetParametersByPath` denied	IAM missing path-level ARN	Include both `.../*` and `...` (Issue #41)
"No storage devices found"	`CACHEFILESD_DISK_TYPE=local_nvme`	Correct value: `local-nvme` (hyphen)
`FSID_MODE=static` breaks all writes	fsidd disabled, filehandle unresolvable	Switch to `FSID_MODE=local`
S3 AP create BadRequest	`--type One_Zone` is for OpenZFS	Use `--type ONTAP`
`exportfs: No fsid found`	fsidd first-time allocation warning	No operational impact (ignore)

FSID_MODE=local: SQLite on FSx for ONTAP

KNFSD's NFS re-export requires stable FSID management. FSID_MODE=static (fsidd disabled) is fundamentally broken — all write operations fail with Stale file handle.

The fix: use the Linux kernel's fsidd daemon with SQLite backend, placing the database file on the FSx for ONTAP NFS mount:

/srv/nfs/vol1/.knfsd/fsids.sqlite  ← on FSx for ONTAP (persistent, 99.99% SLA)

Aspect	local (SQLite on FSx)	external (RDS)
Additional cost	$0	$15-45/mo
Persistence	✅ (FSx for ONTAP SLA)	✅ (RDS SLA)
Multi-node	△ (single-node only)	✅
Complexity	Low	Medium-High
Best for	PoC / single-node	Production multi-node

FlexCache and KNFSD: Different Tools for Different Contexts

Criterion	FlexCache	KNFSD
Source must be ONTAP	✅ Native integration	○ Any NFS-compatible filer
Multiple source unification	Not possible	✅
Write-back/write-around	✅ (configurable)	△ (write-through only)
Burst read scaling	△ (within FSx throughput)	✅ (NVMe cache scales with ASG)
Managed operations	✅	△ (EC2 operations)
Protocol	NFSv3/4.x	NFSv4.1 (required for re-export)
Data protection integration	✅ (SnapMirror/SnapLock)	None (pass-through)
Node elasticity	△ (manual)	✅ (Auto Scaling)

FlexCache note: FlexCache also doesn't consume origin bandwidth on cache hits (same structure as KNFSD). However, in FSx for ONTAP's managed environment, auto-scaling FlexCache node count is not possible. For burst read bandwidth exceeding FSx max throughput, KNFSD offers more flexibility.

Amazon File Cache / FSx for Lustre: Amazon File Cache is a managed Lustre-compatible cache requiring the Lustre kernel client module on every node. KNFSD uses standard NFS clients (built into Linux/macOS), requiring no additional modules. However, Lustre's parallel I/O bandwidth is inherently higher than NFS re-export — if raw bandwidth is the priority and you can deploy Lustre clients on all nodes, File Cache / FSx for Lustre is the better fit.

Cost break-even guideline: KNFSD becomes economical when you need burst bandwidth beyond FSx for ONTAP's maximum throughput tier (4,096 MBps at ~$2,000/mo). For steady workloads where FSx throughput scaling suffices, increasing FSx throughput is simpler.

Target Workloads

Industry	Scenario	KNFSD Role	S3 AP Role
Semiconductor EDA	DRC/LVS burst	Tech file repeated reads	Verification result AI classification
VFX	Rendering	Texture/asset Fanout	Render quality AI check
Genomics	Reference genome + dbSNP	Hundreds of nodes on same reference	Variant AI classification
Financial Risk	Monte Carlo VaR	Market data sub-ms delivery	Anomaly scenario AI detection
Automotive CAE	Structural sim	Shared mesh data reads	Result comparison AI
Weather	NWP ensemble	GFS initial condition cache	Extreme weather AI detection

Step-by-Step Deployment

1. Build AMI (~25 min)

# Find your subnet (same AZ as FSx for ONTAP recommended)
aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-YOUR-VPC" \
  --query 'Subnets[].{Id:SubnetId,AZ:AvailabilityZone}' --output table

git clone https://github.com/awslabs/knfsd-file-cache.git /tmp/knfsd-file-cache
cd /tmp/knfsd-file-cache/image
packer build -var 'REGION=ap-northeast-1' -var 'ARCH=["arm64"]' \
  -var 'ASSOCIATE_PUBLIC_IP_ADDRESS=true' -var 'SUBNET=subnet-YOUR-SUBNET' .
# → Note the AMI ID (ami-0xxxxxxxxxx)

The AMI is reusable across deploy/destroy cycles. Rebuilds are only needed for kernel updates (~monthly). Normal deploys just pass the existing AMI ID to Terraform.

2. Terraform Deploy (~3 min)

cd infrastructure/knfsd-file-cache/terraform
cp terraform.tfvars.example terraform.tfvars
# Edit: vpc_id, subnet_ids, knfsd_ami_id, source_mounts

./scripts/preflight-check.sh   # Validates environment before deploy
terraform init && terraform apply

3. Mount from Client

sudo mkdir -p /mnt/knfsd
sudo mount -t nfs -o vers=4.1 <KNFSD_IP>:/vol1 /mnt/knfsd
echo test > /mnt/knfsd/hello.txt && cat /mnt/knfsd/hello.txt

⚠️ Always specify vers=4.1. Using vers=3 will cause Stale file handle on file creation.

4. Verify Cache Effect

dd if=/dev/urandom of=/mnt/knfsd/bench.dat bs=1M count=10; sync
sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
time cat /mnt/knfsd/bench.dat > /dev/null   # ~55ms (first read — fetched from source)
time cat /mnt/knfsd/bench.dat > /dev/null   # ~2ms  (subsequent — served from cache, 28x)

5. Dual-Path Test (if you have an S3 AP)

# Write via S3 AP
aws s3 cp test.txt "s3://arn:aws:s3:REGION:ACCOUNT:accesspoint/AP-NAME/file.txt"

# Read via KNFSD NFS (immediate reflection)
cat /mnt/knfsd/file.txt

Security note: The above test uses root (UID 0) as the S3 AP FileSystemIdentity. This is for testing only. In production, use a dedicated non-root user following least-privilege principles.

6. Cleanup

terraform destroy
aws ec2 deregister-image --image-id ami-xxx

Upstream Feedback

Five issues filed during this verification:

#	Summary
#38	Verification report + minimal example for existing source filers
#39	Private subnet support (graceful degradation without EC2 API)
#40	NFSv3 re-export filehandle overflow documentation
#41	IAM policy missing path-level ARN (bug fix)
#42	NFSv4.1 source mount support (end-to-end v4.1)

Filing upstream issues for things you find broken (or under-documented) is one of the most impactful things you can do with a Preview-stage OSS project. It ensures the maintainers have real-world data to prioritize GA fixes.

Summary and Next Steps

Dual-Path E2E test passed — S3 AP write → KNFSD NFS read works.

Three key findings:

NFSv4.1 mandatory (NFSv3 filehandle size limit breaks writes)
FSID_MODE=local + SQLite on FSx for ONTAP provides FSID persistence at $0
S3 AP writes reflect immediately through KNFSD NFS

What's next:

Wait for GA (SLA terms, support scope confirmation)
Multi-node (ASG + NLB) verification
EDA/VFX-specific file access pattern benchmarks
im4gn.16xlarge (30 TB NVMe) L2 bandwidth at production scale

KNFSD File Cache is Preview, but it clearly functions as a practical read-acceleration layer for burst compute workloads.

Small-scale starting point: m6gd.xlarge × 1 running 24/7 = ~$215/mo. Daytime-only (10h) = ~$65/mo.

Resources

KNFSD File Cache GitHub
AWS Launch Blog
Dual-Path Architecture Guide (JA) — 7 industry deep dives + cost estimates + observability
Terraform + Demo Guide
FSID Backend Options — SQLite/RDS/Aurora comparison
Linux Kernel NFS Re-export Docs
Verification Issues #38-#42

Preview notice: KNFSD File Cache is in Preview as of July 2026. The code is OSS (Apache 2.0) but AWS Preview Service Terms apply. Evaluate for non-production or supplementary use until GA. AWS Support coverage may not extend to KNFSD-specific issues — GitHub Issues is the primary channel. NVMe instance store cache data is protected by AES-256 hardware encryption and auto-erased on instance termination.

Mackerel's Log Feature Just Opened in Beta — Here's What It Takes to Wire It Into an OTLP Pipeline

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 20 Jul 2026 08:05:24 +0000

TL;DR

Mackerel — Hatena's Japan-origin observability platform — opened its log feature as public beta on July 16, 2026. In response, this repository added Mackerel integration for shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events via the existing Lambda/OTel Collector pipeline.

The OTel Collector path worked immediately — just add an exporter config entry. But the direct-send path (Lambda → vendor OTLP endpoint, no Collector) exposed two compatibility gaps:

Mackerel requires a custom Mackerel-Api-Key header, not Bearer/Basic
Mackerel's OTLP endpoint only accepts Protobuf, rejecting OTLP/JSON outright

This article covers the generic auth header primitive and limited Protobuf encoder added to close those gaps. Both delivery paths are confirmed E2E against a live Mackerel organization, but since the log feature is still in beta, production deployments should use the Collector path and should not treat Mackerel as the sole audit trail destination.

Scope: This article covers shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events to Mackerel via the existing Lambda/OTel Collector integration. It does not cover Collector high availability, Lambda retry/DLQ design, NAT Gateway routing, or log-loss recovery — see the base article (Part 5) and repository operations docs for those topics.

FSx for ONTAP
├─ File access audit (EVTX/XML)
├─ EMS events
└─ FPolicy events
         │
         ▼
Lambda: builds OTLP Logs Data Model
         │
         ├──▶ OTel Collector ──▶ Mackerel [recommended]
         │
         └──▶ Direct to Mackerel [verification path]
              - Mackerel-Api-Key header
              - Accept: */*
              - OTLP/Protobuf required

GitHub: Yoshiki0705/fsxn-observability-integrations
New files: integrations/mackerel/, integrations/otel-collector/otel-collector-config-mackerel.yaml
Modified: integrations/otel-collector/lambda/{handler,ems_handler,fpolicy_handler}.py

This is a standalone entry in the Serverless Observability for FSx for ONTAP series — it's not a continuation of the incident-response article, and doesn't assume you've read it. It builds on Part 5 (the vendor-neutral OTel Collector pattern), since that's exactly the mechanism a new backend plugs into.

Why Mackerel, and Why Now

This series has already shipped FSx for ONTAP audit logs to 9 observability platforms — Datadog, Splunk, Grafana Cloud, Honeycomb, and others — all confirmed E2E against real accounts. A few readers, mostly from Japan, have asked why Mackerel wasn't on that list. Fair question, and one I'd been wanting to answer for a while.

The short answer: it wasn't this repository's gap to close. Mackerel simply didn't have a log-sending feature yet. It's offered OpenTelemetry-based tracing (APM) for a while, but logs were a gap in Mackerel's own product.

There's another direct trigger behind this article. At SRE NEXT 2026, I had a chance to casually catch up with some longtime acquaintances on the Mackerel team and trade notes on Observability for Amazon FSx for NetApp ONTAP. That particular conversation didn't lead anywhere specific on its own, but the conference as a whole gave me useful ideas and motivation for infrastructure-focused development, which fed directly into picking this integration up.

That changed on July 16, 2026, when Mackerel opened its log feature as a public beta. A feature I'd been waiting on finally showed up, which is the actual reason this article exists.

That said, "just opened in beta" is worth taking seriously. The feature itself is new and explicitly non-GA — a genuinely different situation from adding an eighth or ninth already-mature vendor without much thought. This article walks through what actually broke while wiring it up, and why it still sits in a separate section instead of being quietly folded into the main vendor table.

Vendor-neutral note: this article evaluates Mackerel's OTLP log ingestion on its technical merits — the same way this repository's vendor-comparison.md treats Datadog, Grafana, Splunk, and the other 9. No platform in this comparison is framed as superior to another; each fits a different context, and the goal here is documenting what's actually supported, not ranking who's "better."

When this integration is a good fit:

You already use Mackerel for monitoring/APM
You want to integrate FSx for ONTAP events into your existing incident investigation workflow
You use an OTel Collector as a shared telemetry router
You can evaluate a beta feature in a non-production or supplementary capacity

When to consider other options for now:

Regulatory/audit requirements demand guaranteed data retention
This would be your sole security log destination
Log loss is unacceptable and formal support terms are required
Strict data residency or contract-path conditions apply

What Mackerel's Beta Actually Supports

Before writing any integration code, the first step was reading Mackerel's own documentation rather than assuming logs would work like traces. As of July 2026, the log-sending method documented by Mackerel's official help is OpenTelemetry-based. This article uses OTLP/HTTP accordingly. Some things turned out to be shared, some didn't:

	Mackerel Tracing (APM)	Mackerel Logs (beta, 2026-07-16)
Protocol	OTLP/HTTP	OTLP/HTTP (same)
Endpoint	`https://otlp-vaxila.mackerelio.com`	Same endpoint
Auth	`Mackerel-Api-Key` header (Write scope)	Same header, same scope
Required extra header	`Accept: /` (documented as required by Mackerel; the reason isn't publicly documented)	Same requirement
Grouping key	—	`service.namespace` + `service.name` resource attributes
Retention	(APM's own terms)	30 days planned at GA; beta operates under the same window but with no guarantee
Collector batch config	—	~3.5MB (`sending_queue.batch.max_size: 3500000` bytes). This is a Collector-side setting matching Mackerel's official config example, not an absolute API max request size

The practical upshot: if you already have an OTel Collector sending traces to Mackerel, adding logs is a second exporter pipeline entry, not a new integration pattern.

Mackerel's Logs feature top page. The beta notice (no data retention guarantee, unscheduled maintenance possible) is explicitly displayed.

The Architecture Question: Collector or Direct-Send?

The Collector path is recommended for most deployments. Direct-send is an option for minimal setups without a Collector, or for verifying protocol compatibility. Mackerel's own documentation also recommends placing a Collector in between for filtering and masking.

Criteria	Recommended Path
Production use, masking, multiple destinations, retry control	Collector
Minimize components in a small environment	Direct-send
Just want to try Mackerel integration first	Local Collector test
Handling security audit logs	Collector (filter sensitive attributes)

This repository's OTel Collector integration already ships FSx for ONTAP audit/EMS/FPolicy logs to Datadog, Grafana Cloud, and Honeycomb simultaneously from one Lambda codebase — the Lambda builds a backend-neutral OTLP payload, and the Collector's exporters config decides where it goes. Adding Mackerel as a fourth backend to that pattern needed zero Lambda changes:

# otel-collector-config-mackerel.yaml
# NOTE: This repo's verified Collector version (0.152.0) uses otlp_http. Older
# versions may require otlphttp (no underscore). Check your version's docs.
exporters:
  otlp_http/mackerel:
    endpoint: https://otlp-vaxila.mackerelio.com
    headers:
      Accept: "*/*"
      Mackerel-Api-Key: ${env:MACKEREL_APIKEY}
    sending_queue:
      batch:
        max_size: 3500000
        sizer: bytes
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [otlp_http/mackerel]

That's the whole integration for the Collector-mediated path. But this repository also supports a direct-send path — Lambda posts straight to a vendor's OTLP endpoint, skipping the Collector entirely — used today for Grafana Cloud via AUTH_MODE=basic. Checking whether that path also worked for Mackerel is where things got more interesting.

The Gap: A Header Mackerel Needs That the Direct-Send Path Couldn't Send

"Add an exporter and you're done" turned out not to cover every path. The direct-send auth logic in handler.py, ems_handler.py, and fpolicy_handler.py only supported two modes:

if AUTH_MODE == "basic":
    encoded = base64.b64encode(token.encode("utf-8")).decode("utf-8")
    auth_headers = {"Authorization": f"Basic {encoded}"}
else:
    auth_headers = {"Authorization": f"Bearer {token}"}

Mackerel's auth is neither. It's a bare custom header, Mackerel-Api-Key: <token>, with no Bearer or Basic wrapping. At that point, direct-send to Mackerel without a Collector simply wasn't possible.

The fix is a generic primitive rather than a Mackerel-specific branch:

AUTH_MODE = os.environ.get("AUTH_MODE", "bearer")  # "bearer", "basic", or "header"
AUTH_HEADER_NAME = os.environ.get("AUTH_HEADER_NAME", "Authorization")
EXTRA_HEADERS_JSON = os.environ.get("EXTRA_HEADERS_JSON", "")

# ...
elif AUTH_MODE == "header":
    auth_headers = {AUTH_HEADER_NAME: token}
# ...
if EXTRA_HEADERS_JSON:
    extra_headers = json.loads(EXTRA_HEADERS_JSON)
    auth_headers = {**(auth_headers or {}), **extra_headers}

AUTH_MODE=header sends the secret verbatim under any header name you specify. Mackerel happens to be the first consumer, but nothing about the option itself references Mackerel. EXTRA_HEADERS_JSON covers Mackerel's required Accept: */*, which isn't authentication at all — just a static header their backend depends on.

EXTRA_HEADERS_JSON constraint: This is for static, non-secret headers like Accept only. Do not specify auth headers (Authorization, Mackerel-Api-Key, etc.), Content-Type, Content-Length, or Host — these are controlled by AUTH_MODE/OTLP_CONTENT_TYPE settings or set automatically by the HTTP library.

The same fix went into ems_handler.py and fpolicy_handler.py too. The CloudFormation template passes these env vars to all three Lambda functions, so patching only one would have left the other two silently ignoring AUTH_MODE=header if someone set it.

12 new unit tests cover this across the three handlers (4 each in handler.py, ems_handler.py, and fpolicy_handler.py): custom-header auth, EXTRA_HEADERS_JSON merging, the case where no API key secret is configured but extra headers still apply, and invalid EXTRA_HEADERS_JSON producing a startup warning with extra headers disabled. cfn-lint and gitleaks run clean on the modified template and new files.

The Second Gap: Auth Was Fixed, But the Payload Format Wasn't

Fixing auth wasn't the finish line, though. The unit tests mock the HTTP layer entirely, so they passed without catching what a real API key call turned up next:

OTLP endpoint error 400: {"code":400,"message":"json is not supported yet"}

_send_otlp_payload always sends OTLP/JSON with Content-Type: application/json, but Mackerel's OTLP endpoint accepts Protobuf only and rejects JSON outright. The Collector-mediated path never surfaced this: the Collector decodes received OTLP data into its internal data model, then re-serializes via the otlphttp exporter whose default encoding is Protobuf (proto). So even though the Lambda sends OTLP/JSON to the Collector, the Collector sends OTLP/Protobuf to Mackerel. The direct-send Lambda code simply never had a Protobuf encoder.

This project's policy is to include no additional Python dependency packages in the Lambda deployment package. Only boto3 and urllib3 — both available in the Lambda Python runtime environment — are used; protobuf/opentelemetry-proto are not bundled. Adding those packages for one vendor felt disproportionate, so the fix is a small hand-rolled encoder (otlp_protobuf.py). This encoder is not a general-purpose OTLP implementation. It covers only the limited data structures this repository's FSx for ONTAP log builders produce: timeUnixNano, severityNumber, severityText, body.stringValue, attributes (stringValue only), resource attributes (stringValue only), and InstrumentationScope. It does not support the full OTLP Logs field set or all AnyValue types, and is not intended for reuse outside this project. Field numbers came straight from the official OTLP proto definitions (Apache License 2.0); the encoder's byte output was cross-checked against the official opentelemetry-proto generated Python classes in a throwaway virtualenv before it touched a real account. We monitor official proto definition updates and maintain compatibility tests against the generated classes.

OTLP_CONTENT_TYPE = os.environ.get("OTLP_CONTENT_TYPE", "json")  # "json" or "protobuf"

if content_type == "protobuf":
    headers = {"Content-Type": "application/x-protobuf"}
    body = encode_logs_data(payload)
else:
    headers = {"Content-Type": "application/json"}
    body = json.dumps(payload).encode("utf-8")

After adding OTLP_CONTENT_TYPE=protobuf, calling handler.py's actual build_otlp_payload and _send_otlp_payload functions — the same code the Lambda runs, not a reimplementation — against the real Mackerel API key succeeded. For the two sample records sent (ReadData/Success and Delete/Access Denied), this repository's generated audit attributes — operation type, result, file path, SVM name, user information, event timestamp — were confirmed searchable on Mackerel's log search UI.

Unit tests verify auth headers, Content-Type branching, and encoding output. However, they cannot detect vendor-side acceptance behavior, rate limits, network path issues, or search-indexing delays — those require E2E tests against the real endpoint, which we run separately. 20 more unit tests were added: 3 per handler for the new content-type path, plus 11 in a dedicated test_otlp_protobuf.py, bringing the OTel Collector integration's test suite to 110 passing tests.

Verifying one delivery path for a vendor doesn't verify another. Auth headers, payload wire format, and network reachability can each fail independently, and a unit-test suite that mocks the HTTP layer won't catch a payload-format mismatch that only a real vendor endpoint enforces.

Why This Still Isn't in the "9 Vendors, All Verified" List

Every other vendor in this repository has a checkmark for a reason: an actual payload was sent to a live account and confirmed to arrive. Mackerel now has that checkmark too. Sample FSx audit log payloads showed up in Mackerel's log search UI with this repository's generated audit attributes confirmed searchable, through both the Collector-mediated path and the direct-send path once OTLP_CONTENT_TYPE=protobuf was in place. This repo's README.md reflects that:

✅ E2E verified (open beta) — Confirmed end-to-end against a live account, but the backend platform's own feature is itself in open beta (no data retention guarantee, unscheduled maintenance possible) — see the linked integration README for beta constraints.

Verification Detail	Value
E2E verified	2026-07-18
Collector version	0.152.0 (OpenTelemetry Collector Contrib)
Region	ap-northeast-1
Repository commit	`main` branch HEAD at time of publication

E2E evidence: Searching by the fsxn-audit service shows 4 records — ReadData/Success, WriteData/Success, Open/Failure, Delete/Access Denied at WARN/INFO severity levels.

Log detail: Operation type, result, SVM name, file path, user info, client IP, and timestamp are all retained as searchable attributes.

So the code-side gate is cleared. There's a second reason the integration still doesn't sit in the main table, though, and it's not about this repository's code at all — it's about what Mackerel itself has published about its own feature:

No data-retention guarantee during the beta period
Unscheduled maintenance is possible
GA is planned for fall 2026, but the exact date isn't fixed yet

Cost note: As of July 2026, the beta period is free. GA pricing is planned as ingest-volume-based billing. Pricing details, measurement units, and free-tier terms may change after this article's publication — check Mackerel's official pricing page for current terms.

Folding this integration quietly into the "supported vendors" table with the same checkmark as the other 9 would let someone deploying it for security alerting (this repository's automated incident-response module, for instance) reasonably assume the same confidence level as Datadog or Splunk. That assumption would be wrong, not because of anything this repo's code does, but because the platform underneath it is explicitly pre-GA. So the integration stays in a separate "Emerging / Beta Vendors Under Evaluation" section of vendor-comparison.md, marked as E2E-verified-but-beta, until Mackerel's own log feature reaches GA.

One thing worth flagging for anyone reproducing this kind of verification: during investigation we also inspected GraphQL requests used internally by the browser UI, but we have not confirmed these are public/stable APIs. This repository's permanent E2E verification does not depend on them — final confirmation uses the official UI's log search screen combined with the Collector's own send metrics (otelcol_exporter_sent_log_records). Internal implementation details on Mackerel's side may change without notice.

Security Considerations

FSx for ONTAP audit logs may contain usernames, file paths, share names, and client IP addresses — content that may constitute sensitive or personally-identifiable information depending on your organization's policies. Before sending production data to any external observability service:

Confirm your organization's data classification, storage location, access control, and retention requirements
Use OTel Collector filter/transform processors to remove or mask sensitive attributes as needed (see this repository's PII redaction cookbook)
Do not use beta-period Mackerel as the sole storage destination for audit trails

On EXTRA_HEADERS_JSON specifically:

Never put secret values in it — API keys belong in Secrets Manager
Never put API keys directly in CloudFormation parameters or Lambda environment variables
Never log full header contents at runtime

Troubleshooting: 4-Layer Isolation

When issues occur, isolate across these four layers:

Layer	Target	What to Check
1. Generation	Lambda	OTLP record count, required attributes, timestamp, severity, resource attributes. If enabling debug logging, avoid outputting sensitive info like file paths or usernames
2. Transmission	Lambda / Collector exporter	HTTP response code, timeouts, retry results (400, 401, 403, 429, 5xx)
3. Acceptance	Collector metrics / OTLP response	`sent`/`failed` log records, partial success, drops
4. Consumption	Mackerel UI	service attribute match, search time range, filter conditions, indexing delay

Distinguishing Collector/HTTP transmission failures (layers 2-3) from post-acceptance search/display issues (layer 4) is critical. If Mackerel rejects with HTTP 400, that's a layer 2-3 problem, not layer 4. See the troubleshooting table in integrations/mackerel/docs/ for detailed guidance.

Recommended Adoption Steps

For evaluating this beta integration in stages:

E2E verification with sample logs using a local Collector (test-local-mackerel.sh)
Send non-production FSx for ONTAP audit logs in limited scope
Evaluate send volume, data loss, searchability, and sensitive-information handling
In production, implement dual-send to your existing log platform (CloudWatch Logs, S3, existing vendor)
During the beta period, do not use Mackerel as your sole audit trail destination
Re-evaluate for primary-path promotion once data retention terms, support conditions, and pricing are finalized at GA

What's Actually Usable Today

Both paths below are now confirmed working end-to-end:

# Collector-mediated (recommended, zero Lambda changes)
cp integrations/otel-collector/.env.mackerel.example .env.mackerel
# edit .env.mackerel with a Write-scoped Mackerel API key
bash integrations/otel-collector/scripts/test-local-mackerel.sh

# Direct-send (skips the Collector; needs AUTH_MODE=header AND OtlpContentType=protobuf)
aws cloudformation deploy \
  --template-file integrations/otel-collector/template.yaml \
  --stack-name fsxn-otel-integration \
  --parameter-overrides \
    OtlpEndpoint=https://otlp-vaxila.mackerelio.com \
    AuthMode=header \
    AuthHeaderName=Mackerel-Api-Key \
    ExtraHeadersJson='{"Accept":"*/*"}' \
    OtlpContentType=protobuf \
    ApiKeySecretArn=<secret-arn> \
    ... \
  --capabilities CAPABILITY_NAMED_IAM

OtlpContentType=protobuf is not optional for Mackerel — without it, the send fails with the JSON-rejection error described above, even with correct auth.

Full setup guides (bilingual) are in integrations/mackerel/docs/, including a troubleshooting table that specifically tells you to isolate Collector-side failures from Mackerel-side rejections before debugging the wrong layer.

Wrapping Up, and What's Next

So that's where Mackerel's log beta stands today. Getting there took two fixes that aren't visible when only the Collector-mediated path is tested — a custom auth header format, and a JSON-vs-Protobuf mismatch — and neither one surfaced until real credentials hit the direct-send path. A green unit-test suite doesn't tell you that.

The AUTH_MODE=header, EXTRA_HEADERS_JSON, and OTLP_CONTENT_TYPE=protobuf options added here are not Mackerel-specific code — they're reusable primitives for any future OTLP/HTTP backend that requires a custom auth header or Protobuf-only ingestion.

A few things to watch going forward:

Watch for Mackerel's GA announcement for its log feature (data retention terms, pricing) — this article and the linked docs will be updated, not silently left stale, once that happens.
Once GA lands with confirmed data retention conditions, support terms, and pricing, move Mackerel from "Emerging / Beta Vendors" into the main "Supported Vendors" comparison table.
If you're running Mackerel already, the setup guide's sample OTLP payload is a good starting point to validate your own organization's setup independently.

Mackerel's log feature still has some growing to do before GA, and I'm looking forward to seeing where it lands. Hope this was useful to someone wiring up the same thing.

Resources

Automated Access Blocking for FSx for ONTAP — From Ransomware Detection to Storage-Layer Deny

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 12 Jul 2026 12:11:54 +0000

Introduction

Problem: On Amazon FSx for NetApp ONTAP — a file system running on AWS — how do you automatically stop ransomware from spreading after detection (via ARP/AI, SIEM, or CloudWatch alarm)?
Solution: A Lambda function calls ONTAP REST API to immediately block the compromised user/IP at the storage layer and create a protective snapshot — the same containment-phase actions DII Storage Workload Security performs, but triggerable from any detection source, not just DII's own.
Scope: This is storage-layer access blocking plus evidence preservation. Host isolation, malware removal, credential rotation, and blocking lateral movement to other systems (eradication/recovery phases) are out of scope and still require human judgment.
Mechanism: SMB user blocking via name-mapping + session disconnect (immediate), NFS IP blocking via export-policy rules + NACL deny rules (immediate, network-layer) — all through REST API and AWS VPC APIs.
Trigger: Any source that can publish to SNS (CloudWatch Alarm, Datadog Monitor, Elastic SIEM, manual CLI).
Deploy: One CloudFormation stack. Tested with 52 unit tests (36 core + 16 network-layer). CLI helper for manual operations.
Cost: ~$0.51/month (Lambda + SNS + Secrets Manager). Add ~$14/month if VPC Endpoints are needed.
Timing: Under 2 minutes end-to-end (verified). Worst-case with Lambda cold start + VPC ENI attach: under 3 minutes.
Where this fits: In NIST CSF 2.0 terms, this module is a Respond-function tool (RS.MI mitigation + RS.AN analysis groundwork). It doesn't touch Govern, or the ML side of Detect — see the dedicated section below. (Companion modules in this repo now cover parts of Recover and Identify too — also covered below.)

GitHub: Yoshiki0705/fsxn-observability-integrations
Template: shared/templates/automated-response.yaml
Module: shared/python/ontap_response.py

This is Part 18 of the Serverless Observability for FSx for ONTAP series. It builds on the detection capabilities from Part 17 (CloudWatch Log Alarm) and Part 3 (ARP + Datadog).

The Gap Between Detection and Response

The previous articles in this series covered detection:

Part 3: ARP detects ransomware → EMS event → alert in ~30 seconds
Part 17: CloudWatch Log Alarm detects suspicious admin operations → alert in ~90 seconds

But an alert is just information. The real question is: between when detection fires and when a human completes their response, what should the system do automatically?

In a ransomware scenario, every second counts. If ARP detects file encryption and you get a Slack notification, but the SOC analyst is in a meeting, the attacker keeps encrypting files for 15-30 minutes before anyone reacts.

Where This Idea Came From

We have previously recommended dedicated storage security products — such as NetApp DII (Data Infrastructure Insights, formerly Cloud Insights) Storage Workload Security — to close this gap. DII's built-in per-user ML baselines detect anomalies and automatically block users or IPs at the storage layer, using the same ONTAP mechanisms (name-mapping, export-policy rules) this module uses. If you already run DII in your on-premises NetApp ONTAP environment, it remains a valid choice on AWS as well.

The friction shows up in a specific situation this series keeps running into: detection that originates outside DII. Part 3's ARP → Datadog pipeline, Part 17's CloudWatch Log Alarm, or any third-party SIEM monitor produces an alert. Based on DII's publicly available documentation, its response actions are designed around DII's own detection rather than an arbitrary SNS message from your existing observability stack — we didn't find a documented path to trigger a DII block from an external alert, and building one was outside the scope of this AWS-native series. There's also a practical entry-cost question: DII is a capacity-based licensed product with its own SaaS data plane, which is a different investment than "add a Lambda to a stack you already have."

So the question became: "Can the same containment-phase actions — blocking storage access, taking a snapshot, disconnecting sessions — be triggered by any event that can publish to an SNS topic, using only ONTAP REST API and AWS-native services?" That's what shared/python/ontap_response.py and this article implement. It's not a replacement for DII's ML detection — see the comparison table further down this article, and the fuller comparison plus FAQ ("Does this replace DII Storage Workload Security entirely?") in the Automated Response Guide on GitHub — it's a way to get the same storage-layer response when your detection already lives somewhere else.

Architecture

+--------------------------------------------------------------+
| Detection (existing - any source)                            |
|                                                              |
|  CloudWatch Log Alarm --+                                    |
|  ARP EMS -> Monitor ----+                                    |
|  FPolicy -> SIEM -------+-- SNS Trigger Topic                |
|  Manual CLI ------------+                                    |
|                                                              |
+--------------------------------------------------------------+
| Response (NEW - this article)                                |
|                                                              |
|  SNS -> Lambda (VPC) -> ONTAP REST API                       |
|                           |                                  |
|                           +- Block SMB user (name-mapping)   |
|                           +- Block NFS IP (export-policy)    |
|                           +- Block NFS IP (NACL — immediate) |
|                           +- Create snapshot (evidence)      |
|                           +- Disconnect CIFS sessions        |
|                           +- Notify (SNS -> email/PagerDuty) |
|                                                              |
+--------------------------------------------------------------+

The key design choice: detection and response are decoupled via SNS. Any system that can publish a JSON message to an SNS topic can trigger a storage-layer block. This means you're not locked into a single detection product.

How ONTAP User Blocking Works

SMB: Name-Mapping Deny

ONTAP resolves Windows users to UNIX users via name-mapping rules. By mapping a Windows user to an empty UNIX identity (" "), the SID-to-UNIX translation fails and all file operations are denied.

Before: CORP\jdoe → maps to → jdoe (UNIX) → access granted
After:  CORP\jdoe → maps to → " " (empty)  → access DENIED

This is an SVM-wide block — it affects every volume and share within the SVM.

Operational safety note: The deny mapping is inserted at position 1 by default, meaning it's evaluated before any existing name-mappings. If your SVM has existing name-mappings for service accounts or automation, use a higher index (the module uses index 1; customize via ontap_response.py if needed). Always test with health_check and a non-production user first. To immediately undo an accidental block: ./automated-response-cli.sh unblock-smb --domain CORP --user jdoe

ONTAP REST API call:

POST /api/name-services/name-mappings
{
  "direction": "win_unix",
  "index": 1,
  "pattern": "CORP\\jdoe",
  "replacement": " ",
  "svm": {"name": "svm-prod-01"}
}

NFS: Export-Policy Deny Rule

For NFS clients, blocking is done via export-policy rules with ro_rule: never and rw_rule: never:

POST /api/protocols/nfs/export-policies/{policy_id}/rules
{
  "clients": [{"match": "fsxn_auto_response,10.0.5.99"}],
  "ro_rule": ["never"],
  "rw_rule": ["never"],
  "superuser": ["never"],
  "protocols": ["any"],
  "index": 999
}

The fsxn_auto_response marker in the client match makes these rules identifiable and easy to clean up. Use a high index (e.g., 999) to avoid conflicts with existing rules.

Scope note: Both SMB and NFS blocks are SVM-wide — they affect ALL volumes and shares within the target SVM, not just the volume specified in the contain_* action. In multi-tenant SVMs (multiple applications sharing one SVM), this is the intended blast radius for containment. If you need volume-level granularity instead, use create_snapshot (per-volume) and coordinate with your network team for host-level isolation.

NFS: Why Export-Policy Alone Isn't Instant (and How We Fixed It)

The export-policy rule above takes effect on the server side immediately — but Linux NFS clients cache access decisions for up to 60 seconds (actimeo default). During that window, the attacker's existing mount can still read/write despite the ONTAP-layer deny.

Unlike SMB (where disconnect_smb_sessions forces re-authentication), NFS has no equivalent "kill this client's connection" API. (NFSv4 has lease management internally, but ONTAP does not expose a REST API endpoint for forced lease revocation of a specific client.)

Solution: Network-layer blocking via VPC NACL deny rules.

When FsxSubnetId is configured, contain_nfs_threat automatically applies both layers:

Layer	Mechanism	Effect	Bypass?
ONTAP	Export-policy deny rule	Persistent, survives NACL removal	Client cache (up to 60s)
Network	NACL deny rule (rule 50-99)	Immediate (packet-level)	Cannot be bypassed by client

# Deploy with network-layer NFS blocking enabled:
aws cloudformation deploy \
  --template-file shared/templates/automated-response.yaml \
  --stack-name fsxn-automated-response \
  --parameter-overrides \
    ... \
    FsxSubnetId=<subnet-where-fsx-enis-reside> \
  --capabilities CAPABILITY_NAMED_IAM

The NACL rule blocks ALL traffic from the attacker IP at the VPC level — no client-side cache, no retry window. The export-policy rule persists as the long-term block after the NACL is eventually removed during investigation.

Blast radius note: By default, the NACL rule blocks ALL protocols from the IP (not just NFS). If the attacker IP is also used for monitoring or management, set "block_all_ports": false in the SNS message to block only NFS port 2049.

Why NACL and not Security Group? Security Groups are allow-only — you cannot add a deny rule for a specific IP. NACLs support explicit deny and are evaluated before Security Groups, making them the correct mechanism for emergency IP blocking.

VPC Endpoint requirement: The Lambda calls the EC2 API (CreateNetworkAclEntry) to manage NACL rules. If your VPC has a NAT Gateway, this works automatically. If not, add an EC2 Interface VPC Endpoint (com.amazonaws.<region>.ec2) or set CreateVpcEndpoints=true and add EC2 manually. Without network access to the EC2 API, the NACL block will timeout while the ONTAP export-policy block still succeeds.

Session Disconnect

Blocking prevents new operations, but existing CIFS sessions can continue. The composite action also disconnects active sessions:

DELETE /api/protocols/cifs/sessions/{svm_uuid}/{identifier}/{connection_id}

Safety Features

Input Validation

The module validates all inputs before calling ONTAP:

Username injection prevention (blocks ;, |, &, `, $, newlines)
Protected account denylist (fsxadmin, administrator, vsadmin, system)
IP address format validation (valid IPv4 octets 0-255)

Snapshot Storm Prevention

A configurable cooldown (default: 15 minutes) prevents creating multiple snapshots during a sustained attack:

# If a snapshot with our prefix was created <15 min ago → skip
result = client.create_snapshot(
    svm_name="svm-prod",
    volume_name="vol_data",
    cooldown_minutes=15,  # Prevents storm
)
# result["status"] == "skipped" if cooldown active

Marker-Based Cleanup

All response rules include a consistent marker (fsxn_auto_response), making it straightforward to find and remove all automated blocks:

# Find all our rules
export-policy rule show -clientmatch *fsxn_auto_response*
vserver name-mapping show -direction win-unix -replacement " "

Evidence-handling note: The protective snapshot this module creates is evidence, and evidence needs a chain of custody, not just a timestamp. Trigger source, exact UTC time, and post-action API confirmation are already captured in CloudWatch Logs — but pre-action state (what the name-mapping or export-policy looked like before the block) isn't currently queried and logged, and the SNS trigger message itself isn't hashed. If this snapshot might ever need to hold up in an investigation, not just an ops postmortem, review the chain-of-custody gap table in the Security Addendum before treating "snapshot created" as "evidence preserved."

Time-Limited Blocks

Persistent blocks are a real operational risk — a false positive can lock out a legitimate user indefinitely if nobody notices. The companion automated-response-ttl.yaml stack deploys an EventBridge Scheduler that periodically checks for and auto-removes blocks older than a configurable TTL, so a missed alert doesn't turn into a standing lockout. Deploy it alongside the main stack.

Deployment

One Stack Deploy

aws cloudformation deploy \
  --template-file shared/templates/automated-response.yaml \
  --stack-name fsxn-automated-response \
  --parameter-overrides \
    OntapMgmtIp=<management-ip> \
    OntapCredentialsSecretArn=<secret-arn> \
    VpcId=<vpc-id> \
    SubnetIds=<subnet-1>,<subnet-2> \
    SecurityGroupId=<sg-id> \
    DefaultSvmName=svm-prod-01 \
    SharedPythonLayerArn=<layer-arn> \
    NotificationEmail=security-team@example.com \
  --capabilities CAPABILITY_NAMED_IAM

New parameter (from verification): SharedPythonLayerArn — provide the ARN of the fsxn-shared-python Lambda Layer containing ontap_response.py. Without this, the handler fails with ModuleNotFoundError. Build and publish the layer first:
bash shared/python/build-layer.sh
aws lambda publish-layer-version --layer-name fsxn-shared-python \
  --zip-file fileb://shared/python/dist/fsxn-shared-python-layer.zip \
  --compatible-runtimes python3.12 --query 'LayerVersionArn' --output text

The stack creates:

SNS Trigger Topic (entry point)
SNS Notification Topic (results)
Lambda function (VPC, Python 3.12)
DLQ with encryption
CloudWatch Log Group (365-day retention)
DLQ depth alarm
VPC Endpoints for Secrets Manager + SNS (if CreateVpcEndpoints=true)

DLQ alarm = urgent: A message in the DLQ means a containment action failed. Unlike most DLQs where retry-later is fine, here it means the attacker may still have access. Treat DLQ depth > 0 as a P1 incident requiring immediate human investigation. See the DLQ Replay Runbook for recovery steps.

Critical: VPC Endpoints Required. Lambda in a VPC cannot reach AWS APIs (Secrets Manager, SNS) without Interface VPC Endpoints or a NAT Gateway. The stack creates these by default (CreateVpcEndpoints=true). If your VPC already has them, set it to false. Without this, Lambda will timeout on every invocation.

Cost note: If your VPC already has a NAT Gateway, set CreateVpcEndpoints=false — the Lambda can reach AWS APIs through NAT without the additional ~$14/month VPC Endpoint cost. The $0.51/month figure is infrastructure cost only; operational cost (tuning detection rules, investigating false positives, running quarterly drills) is the larger ongoing investment.

The CloudFormation template embeds the Lambda code inline for a quick first deployment. For a GitOps-friendly setup, package ontap_response.py as a Lambda Layer instead using build-layer.sh and version it independently of the stack.

Comprehensive deployment guidance: For VPC Endpoint conflict avoidance, parameter file templates, and pre-deployment environment validation, see the Deployment Guide. It includes a pre-flight check script that detects existing VPC Endpoints before you deploy — the single most common source of deployment rollbacks in this project.

Connect Detection Sources

Subscribe the trigger topic to your detection:

Source	How to Connect
CloudWatch Log Alarm	Set alarm action to trigger topic ARN
Datadog Monitor	Use `@sns-<topic>` in notification
Elastic SIEM	Use SNS action in Kibana alert
Manual	`aws sns publish --topic-arn <arn> --message '<json>'`

Alert-tuning note: Whatever you wire to the trigger topic will fire on every match, and every fire generates a notification. A detection rule tuned for "catch everything" rather than "catch what matters" turns this pipeline into another source of alert fatigue — the Reliability note in the Automated Response Guide already flags the self-inflicted-DoS risk of an over-triggering rule; the analyst-facing version of that same risk is that a noisy rule buries the one block that actually mattered in a pile of routine ones. Tune the upstream detection rule's specificity before wiring it here, not after the first pager rotation complains.

CLI Helper

For manual operations and testing, use the included CLI wrapper:

export RESPONSE_TOPIC_ARN="arn:aws:sns:ap-northeast-1:123456789012:fsxn-automated-response-trigger"
export DEFAULT_SVM="svm-prod-01"

# Full containment (snapshot + block + disconnect)
./shared/scripts/automated-response-cli.sh contain-smb \
  --domain CORP --user jdoe --volume vol_data \
  --reason "ARP detection - arw.volume.state alert"

# Block a single IP
./shared/scripts/automated-response-cli.sh block-nfs \
  --ip 10.0.5.99 --reason "Mass deletion from FPolicy"

# Unblock after investigation
./shared/scripts/automated-response-cli.sh unblock-smb \
  --domain CORP --user jdoe

# Dry-run test (no publish)
./shared/scripts/automated-response-cli.sh test

If the same compromised identity or IP has access across multiple SVMs (e.g., a domain user with shares on both production and DR), use automated-response-multi-svm-cli.sh instead — it wraps the single-SVM CLI, fans out the same action across a comma-separated SVM list, and reports per-SVM success/failure:

./shared/scripts/automated-response-multi-svm-cli.sh contain-smb \
  --svms "svm-prod-01,svm-prod-02,svm-dr-01" \
  --domain CORP --user jdoe --volume vol_data \
  --reason "ARP detection - multi-SVM block"

Composite Actions

Individual actions (block_smb_user, block_nfs_ip, create_snapshot, disconnect) can be combined into a single message:

Action	Steps	Use Case
`contain_smb_threat`	Snapshot → Block SMB user → Disconnect sessions	Compromised AD user detected
`contain_nfs_threat`	Snapshot → Block NFS IP	Suspicious NFS client activity
`contain_multiprotocol_threat`	Snapshot → Block SMB user → Block NFS IP → Disconnect sessions	Volume accessed via both SMB and NFS — a single-protocol block lets the attacker switch protocols and continue

The contain_* naming maps to the Containment phase in NIST SP 800-61's incident handling model. This module only covers storage-layer access blocking and evidence preservation, though. Isolating the compromised host, removing malware, rotating credentials, and blocking lateral movement to other systems (eradication/recovery) remain out of scope and still require a human (or another IR tool) to handle.

Comparison with DII Storage Workload Security

Both approaches use the same underlying ONTAP mechanisms. The difference is where detection intelligence lives:

Aspect	DII Storage Workload Security	This Approach
Blocking mechanism	ONTAP REST API	ONTAP REST API (identical)
Detection	Built-in ML (per-user baselines)	Your choice of SIEM/observability
Context scope	Storage only	Storage + network + application
Data residency	SaaS (vendor cloud)	Your AWS VPC
Integration	Limited export	Native (detection originates from your tools)
Cost model	Capacity-based license	~$0.51/month (pay-per-use)
Setup time	Days (agent + collector + AD config)	30 minutes (CloudFormation)

Neither approach is "better" — they suit different contexts. If you already have a SIEM with anomaly detection (Datadog ML, Elastic ML Jobs, Splunk MLTK), or if your alert originates outside DII, this approach extends your existing investment to cover storage-layer access blocking. If you already run DII and your detection lives there too, its built-in ML and turnkey containment remain a solid choice — see the Automated Response Guide for the fuller writeup.

The table above compares mechanisms. The next section places both mechanisms inside a wider risk-management frame, because "which tool blocks the user" and "is your organization's ransomware posture actually sound" are different questions.

Where This Fits in Cyber Resilience: NIST CSF 2.0

It's tempting to read a 65-second block-and-snapshot flow as "ransomware handled." It isn't — it's one function, in one framework, among six. NIST CSF 2.0 organizes an organization's entire cybersecurity risk-management program into Govern, Identify, Protect, Detect, Respond, Recover, and NIST's dedicated ransomware profile (NIST IR 8374r1) maps ransomware-specific outcomes onto those six functions. Both AWS (Ransomware Risk Management on AWS Using the NIST CSF) and NetApp (Fortify your cybersecurity defenses with NIST framework) publish their own mappings for the same reason this section exists: to be explicit about which function a given tool covers, and which functions remain the organization's job regardless of tooling.

Placed on that map, this module and DII Storage Workload Security occupy the same narrow slice:

CSF 2.0 Function	What It Covers	Where This Module (or DII) Sits
Govern	Risk strategy, roles, policy, board oversight	Out of scope for both — this is an organizational responsibility no storage-layer tool automates
Identify	Asset/data inventory, classification	Out of scope for this module, but a companion module in the same repo — a Content-Level PII Classification Scanner using Amazon Comprehend — covers text/structured-data content discovery. Office/PDF content extraction is not implemented; DII's parent product line offers broader data classification separately
Protect	Safeguards that reduce likelihood/impact	Partially covered — ONTAP Snapshot/SnapLock, export-policy, name-mapping are the underlying safeguards both approaches call
Detect	Finding anomalies via monitoring	Not this module's job — it responds to a detection, it doesn't produce one (see Part 3 and Part 17 for the Detect side)
Respond	Mitigation, analysis, reporting on a detected incident	✅ This is what this module does — block, snapshot, disconnect, notify
Recover	Restoring systems and coordinating with stakeholders	The snapshot this module creates is Respond-phase evidence, not a verified recovery point on its own. A companion Verified-Clean Recovery Point Guide closes part of that gap — see below — though the restore itself and stakeholder-level coordination remain manual

Governance-reporting note: If you're reporting this capability up to a risk committee or board, resist framing it as "ransomware is covered." It's more accurate — and more useful for risk-register purposes — to say "we have automated Respond-phase containment; Govern-phase maturity, and the ML side of Detect, are tracked separately." CloudFormation-as-code and CloudWatch Logs give you the audit evidence (who deployed what, when a block fired and why) that a Govern program consumes as input, but they aren't a substitute for having the program itself.

Privacy note: The Content-Level PII Classification Scanner referenced in the Identify row above is worth a second look if your org is tracking data-minimization obligations. Its findings deliberately record only entity type, count, and confidence per file (EMAIL, SSN, CREDIT_DEBIT_NUMBER, and so on via Amazon Comprehend) — never the matched text itself — so a "this file has 3 SSN-pattern matches" report doesn't itself become a new PII exposure. That's useful for prioritizing where to apply DLP controls, but it isn't a substitute for a DPO's formal data classification review, and it currently can't see inside Office documents or PDFs (no text extraction yet) — plan around that blind spot if your regulated data lives mostly in those formats.

A point worth being precise about on the Recover side, since it's easy to overstate: a snapshot existing is a Protect-phase artifact, not proof that recovery works.

Recovery-sufficiency note: "We took a snapshot" and "we have a verified-clean recovery point" are two different maturity levels. Elastio's mapping of ransomware recovery to CSF 2.0 makes the same point industry-wide: RC.RP (Incident Recovery Plan Execution) is only credible once someone has actually restored from the recovery point and confirmed it's free of compromise — not merely confirmed the snapshot job succeeded. This module alone only delivers the first maturity level (the snapshot). A companion Verified-Clean Recovery Point Guide in this repo now automates a version of the second: it clones the candidate snapshot (FlexClone), scans the clone through an isolated S3 Access Point for ransomware-associated file extensions, and records a pass/fail verdict — without ever touching the production volume.

What "Verified Recovery" Looks Like in Practice

After the containment above creates its protective snapshot, you can verify it before restoring:

Candidate Snapshot
     │
     ▼ FlexClone (copy-on-write, instant, never modifies source)
     │
     ▼ Isolated S3 Access Point (VPC-scoped)
     │
     ▼ Scan for ransomware-associated extensions (.encrypted, .locked, etc.)
     │
     ▼ Verdict: clean / suspicious / error
     │
     ▼ Auto-cleanup (clone + access point deleted)

This is a single Step Functions workflow (5 Lambda functions). Deploy it and run a verification:

# Deploy the verification workflow
aws cloudformation deploy \
  --template-file shared/templates/restore-verification.yaml \
  --stack-name fsxn-restore-verification \
  --parameter-overrides \
    VpcId=<vpc-id> SubnetIds=<subnet-1>,<subnet-2> \
    SecurityGroupId=<sg-id> OntapMgmtIp=<mgmt-ip> \
    OntapCredentialsSecretArn=<secret-arn> \
  --capabilities CAPABILITY_NAMED_IAM

# Verify a specific snapshot before restoring
aws stepfunctions start-execution \
  --state-machine-arn <from stack outputs> \
  --input '{"svm_name":"svm-prod","volume_name":"vol_data","snapshot_name":"incident_response_20260712_005307"}'

The workflow completes in 2-5 minutes (depending on clone creation and access point attachment time). The verdict appears in the execution output and is logged to CloudWatch for audit.

What it doesn't do: This is a fast extension-pattern pre-filter — it flags a volume dominated by renamed/encrypted files. It is not a deep content inspection, and it does not exercise an actual end-to-end restore. Treat a "clean" verdict as a necessary first gate before committing to a restore, and still run periodic full restore drills into an isolated environment separately.

Restore-testing note: Be precise about what that verdict actually certifies, because it's easy to over-claim from the runbook name alone. The scan is a fast extension-pattern pre-filter — it flags a volume dominated by .encrypted/.locked-style renamed files — not a deep content inspection, and it does not exercise an actual restore end-to-end. Treat a "clean" verdict as a necessary first gate before a human commits to a restore, and still run periodic full restore-and-verify drills into an isolated environment separately. Don't let an automated pre-filter quietly become your only recovery test.

This module's protective snapshot, in other words, is Respond-phase evidence preservation, and the verification workflow above is a fast pre-filter toward Recover-phase confidence — neither is a substitute for a tested, human-verified restore. A fuller function-by-function breakdown — including where this repository's other components (ARP, EMS detection, Forensics dashboards, the recovery verification and PII scanning modules) land on Identify/Protect/Detect/Recover — lives in the Cyber Resilience Capability Map.

End-to-End Flow: ARP Detection → Auto-Block

Here's the full flow from ransomware detection to an automated storage-layer block. This flow has been verified end-to-end on ONTAP 9.17.1P7D1 (FSx for ONTAP, ap-northeast-1). Key timings from actual execution:

ARP/AI detection: triggered within minutes of ransomware-like file operations (no learning period — ARP/AI in ONTAP 9.16.1+ is immediately active)
contain_smb_threat Lambda execution: 1.8 seconds (snapshot + block + session disconnect in a single invocation)
SMB access denial confirmed after re-authentication

1. User "jdoe" starts encrypting files on vol_data
   (zip with password → delete original → new extension)
         │
2. ONTAP ARP/AI detects anomaly (entropy + extension changes)
   No learning period required (ONTAP 9.16.1+ / ARP/AI)
         │  Detection within minutes
         ▼
3. EMS event: callhome.arw.activity.seen (severity: alert)
   ONTAP auto-creates Anti_ransomware_attack_backup snapshot
         │
4. EMS Webhook → API Gateway → Lambda → Observability Platform
         │  ~30 seconds
         ▼
5. Observability Monitor fires (ARP alert detected)
         │
6. Monitor → SNS publish (contain_smb_threat)
         │  ~5 seconds
         ▼
7. Response Lambda executes (1.8s measured):
   a) Creates incident_response_20260712_005307 snapshot
   b) Blocks CORP\jdoe via name-mapping (deny on re-authentication)
   c) Attempts CIFS session disconnect
   d) Sends notification to security team
         │
8. Total time: detection + routing (~35s) + response (1.8s) ≈ under 2 minutes

SOC correlation tip: Always include incident_id and detection_source in your SNS message so that the storage-layer block can be correlated with the upstream detection in your SIEM timeline:
{"action":"contain_smb_threat","svm_name":"svm-prod","domain":"CORP",
 "username":"jdoe","volume_name":"vol_data",
 "reason":"ARP alert","incident_id":"INC-2026-0712-001",
 "detection_source":"datadog-monitor-arp"}
How immediate blocking actually works: The contain_smb_threat composite action achieves effective immediate cutoff through a two-step mechanism:

name-mapping deny — blocks all future authentication attempts (evaluated at session setup time, not per-I/O)

disconnect_smb_sessions — forcefully terminates the user's existing active sessions via DELETE /protocols/cifs/sessions/{svm}/{id}/{conn}

When the existing session is terminated, the Windows SMB client automatically attempts to re-establish the connection — at which point the name-mapping deny takes effect and access is refused. The net result: the user is cut off within seconds of the Lambda executing, not at the next natural session expiry.

If the disconnect call returns HTTP 404 (session already gone — e.g., the encrypting process finished before the Lambda fired), that's expected and logged but not treated as an error. The name-mapping deny still prevents any future reconnection.

ARP/AI verification note (ONTAP 9.16.1+): The CLI command security anti-ransomware volume attack simulate referenced in some ONTAP documentation does NOT exist in ONTAP 9.17.1. To trigger ARP/AI detection for testing, perform actual ransomware-like file operations: encrypt files with a password, delete originals, and add a new file extension. ARP/AI detects the high-entropy + deletion + never-seen-extension pattern. Detection threshold: 5+ distinct new extensions within 48 hours. See the ONTAP REST API Quick Reference for details.

What Blocking Looks Like: Visual Evidence

The following was captured during E2E verification on ONTAP 9.17.1P7D1 (July 2026).

NFS: Before and After

Before — user can list and read files normally:

$ ls /mnt/fsxn/
hr-salary.txt  project-spec.txt  reports/

$ cat /mnt/fsxn/hr-salary.txt
Confidential HR Record - Employee Salary Data

After — export-policy deny rule applied (immediate, same-subnet):

$ ls /mnt/fsxn/
ls: cannot access '/mnt/fsxn/': Permission denied

$ cat /mnt/fsxn/hr-salary.txt
cat: /mnt/fsxn/hr-salary.txt: Permission denied

SMB: Before and After (AD-joined SVM)

Before — domain user DEMO\testuser can access CIFS share:

PS> Get-ChildItem X:\
Mode   Length Name
----   ------ ----
d-----        reports
-a---- 46     hr-salary.txt
-a---- 43     project-spec.txt

PS> Get-Content X:\hr-salary.txt
Confidential HR Record - Employee Salary Data

After — block_smb_user executed (name-mapping → nobody, volume perms 750):

PS> Test-Path X:\
False
# ACCESS DENIED — drive not accessible after block

Step Functions: Recovery Point Verification Workflow

Step Functions console: All states succeeded (green). Verified-clean recovery point workflow completed end-to-end — waiting for FSx API to discover the ONTAP-created FlexClone. This is the ONTAP→FSx sync delay documented by AWS ("may take up to several minutes").

Detection & Forensics: Datadog Dashboard

Datadog Forensics Dashboard: Incident timeline (Detection → Response → Verification), user activity tracking, affected volumes, automated response actions log, ARP alert feed, and recovery verification results — all from a single pane.

Datadog Log Explorer: Real-time ARP detection events (callhome.arw.activity.seen, callhome.arw.volume.state) with structured attributes — entropy score, affected file count, new extensions detected.

File Access Audit Trail: Who (CORP\jdoe) accessed what (/vol_data/finance/budget-2026.docx) with what operation (WriteData|Delete) — the same forensic visibility DII Storage Workload Security provides.

Scrolled right: Client IP (198.51.100.99) visible alongside the structured JSON content — enabling "who accessed what from where" forensic queries.

Vendor-neutral note: The screenshots above use Datadog as a representative example, but this repository delivers the same logs to all 9 supported observability vendors (Datadog, Grafana, Splunk, Elastic, New Relic, Dynatrace, Sumo Logic, Honeycomb, OTel Collector). Deployable forensics dashboard definitions (user timeline, file access audit trail, affected volumes, client IP analysis) are provided for Datadog (JSON), Grafana (JSON), Splunk (SPL), and Elastic (KQL Saved Searches). See the per-vendor reference.

Testing

The module has 52 unit tests covering:

Category	Tests	What's Verified
SMB blocking	3	Success, SVM not found, API error
SMB unblocking	2	Success, not found
NFS blocking	2	Success, policy not found
NFS unblocking	2	Success, not found
Snapshots	5	Success, cooldown active/expired/disabled, volume not found
Session disconnect	3	By user, no sessions, missing params
Composite actions	3	Full sequence, partial failure, NFS containment
Input validation	7	Protected accounts, injection prevention, IP validation
Configurable protected accounts	1	`PROTECTED_ACCOUNTS_EXTRA` env var
List blocks	2	With results, empty
Health check	3	Success, SVM not found, unreachable
Error handling	3	Error attributes, timeout, malformed response
NACL blocking	16	Block IP, unblock, list active, find NACL, rule number allocation, marker validation

python3 -m pytest shared/python/tests/test_ontap_response.py shared/python/tests/test_network_block.py -v
# 52 passed in 0.06s

Verify Your Deployment

After deploying to your environment, run these checks within the first 30 days:

Day 1: health_check action succeeds (ONTAP reachable, credentials valid)
Day 1: Test create_snapshot on a non-production volume (confirm snapshot appears)
Week 1: Run a full contain_smb_threat drill against a test user on a test SVM
Week 1: Verify TTL cleanup removes the test block after configured minutes
Day 30: Confirm DLQ is empty (no silent failures), CloudWatch dashboard shows expected invocation count

If any of these fail, check the Troubleshooting section in Prerequisites.

EMS Webhook Integration: Payload Format

When connecting ARP EMS events to your detection pipeline, note the actual ONTAP EMS webhook payload format (verified on ONTAP 9.17.1P7D1):

{
  "message-name": "callhome.arw.activity.seen",
  "message-severity": "alert",
  "message-timestamp": "2026-07-12T00:42:06+00:00",
  "parameters": {
    "vserver-name": "svm-prod",
    "volume-name": "vol_data"
  }
}

Caution: Field names use hyphens (message-name), not camelCase (messageName) or snake_case (message_name). This matters for SIEM parsing rules and Lambda handler field validation.

The EMS event for ARP attack detection is callhome.arw.activity.seen (severity: alert), not arw.volume.state (which is a state-change notification for ARP enable/disable). The callhome.* prefix indicates an ONTAP AutoSupport-class event (internal telemetry classification, not external data transmission). Configure your detection rules to match the correct event name.

Resources

GitHub: ontap_response.py
GitHub: automated-response.yaml
GitHub: Automated Response Guide
GitHub: ONTAP REST API Quick Reference — Common pitfalls (svm.uuid duplicate, allow_privileged_access, ARP/AI behaviors), verified on ONTAP 9.17.1P7D1
GitHub: EMS Detection Capabilities — 30+ events, Push vs Pull delivery, latency comparison
GitHub: Cyber Resilience Capability Map — full NIST CSF 2.0 function mapping with FSx for ONTAP implementation and alternative implementation paths
GitHub: Verified-Clean Recovery Point Guide — FlexClone + isolated scan to verify a snapshot before restoring (CSF 2.0 RC.RP)
GitHub: Content-Level PII Classification Scanner — Amazon Comprehend-based PII discovery for file contents (CSF 2.0 Identify)
NetApp: ONTAP REST API Reference
AWS: FSx for ONTAP ARP
NIST Cybersecurity Framework (CSF) 2.0
NIST IR 8374r1 — Ransomware Risk Management: A CSF 2.0 Community Profile

Alerting on FSx for ONTAP Audit Logs — No Metric Filter Required, with CloudWatch Log Alarms

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Fri, 03 Jul 2026 16:15:43 +0000

TL;DR

Once FSx for ONTAP audit logs land in CloudWatch Logs, a common next question is: "How do I get alerted when someone touches a specific file?" Until recently you had to build a metric filter first. As of July 1, 2026, CloudWatch Log Alarms let you create an alarm directly from a Logs Insights query — no metric filter step.

CloudWatch Log Alarm (GA 2026-07-01): alarm straight from a log query. No metric filter.
How it works: a Logs Insights query matches events, count(*) turns them into a number, and the alarm fires when the count crosses a threshold.
5 detection presets: sensitive-path access / failed access / bulk delete / privileged-user activity / custom.
One-command deploy: bash shared/scripts/deploy-log-alarm.sh (working in ~15 minutes).
Cost: ~$6.6/month (100 MB/day). Slightly higher than the metric-filter approach, but you get log lines in the notification and retroactive queries.
E2E verified: CloudFormation deploy through state transition, in the Tokyo region.

GitHub: Yoshiki0705/fsxn-observability-integrations

This is Part 17 of the Serverless Observability for FSx for ONTAP series. It builds directly on the admin-audit pipeline from Part 14.

Two AWS Launches That Make This Work

This article sits on top of two recent CloudWatch launches. If you're arriving from outside this series, read these first — together they remove the EC2 syslog server and the metric-filter step that used to stand between ONTAP logs and an alert.

1. Managed syslog ingestion (June 2026) — how the logs get in

ONTAP emits its admin audit trail (and EMS events) as syslog. Historically, capturing that meant running an EC2 syslog server. In June 2026, CloudWatch Logs added managed syslog ingestion: you send syslog (RFC 5424 / RFC 3164 / Cisco FTD/ASA) over a VPC endpoint straight into a log group — no agent, no EC2.

For FSx for ONTAP, that means pointing ONTAP log-forwarding at the syslog VPC endpoint and having the admin audit log arrive in CloudWatch Logs directly.

What's New: Amazon CloudWatch Logs supports managed syslog ingestion
Setup guide: Setting up syslog ingestion — VPC endpoint, log group, resource policy, and the three transport options (TCP+TLS 6514 encrypted, TCP 1514 plaintext, UDP 514 best-effort)

This is the piece the previous article (Part 14 / Syslog VPCE setup) already wired up. If you followed it, your admin audit log is in CloudWatch Logs and this article's alarm has something to query.

2. CloudWatch Log Alarms (July 2026) — how you alert on them

The second launch is the subject of this article: alarms defined by a Logs Insights query rather than a metric.

What's New: Amazon CloudWatch supports creating alarms from log queries
Docs: Alarming on logs

Put together: ONTAP → managed syslog ingestion → CloudWatch Logs → Log Alarm → SNS. No EC2, no metric filter, no forwarding Lambda.

What Changed

Alerting on log content is a basic monitoring need. On CloudWatch it used to take a detour: create a metric filter, generate a custom metric from it, then attach an alarm to that metric — three steps. From "I want to detect this" to an alert actually firing was 15–30 minutes of setup.

Log Alarms collapse the middle steps. You give a Logs Insights query a threshold, and the alarm reads straight from the logs.

Aspect	Before (metric filter)	Log Alarm (NEW)
Setup steps	3 (filter → metric → alarm)	1 (Log Alarm only)
Query flexibility	Filter pattern syntax only	Full Logs Insights syntax
Log lines in notification	No	Yes (up to 50)
Retroactive to existing logs	No	Yes
IaC	2 resources	1 resource (`AWS::CloudWatch::LogAlarm`)

The part I appreciate most: the notification can include the matched log lines. When the alert lands, you already see who touched which file and what they did — you can form a first read before opening the console.

FSx for ONTAP Logs × Log Alarm

There are three FSx for ONTAP log/event families, and it's worth separating them because only some are a natural fit for Log Alarms.

Log type	What it records	Delivery path	Log Alarm target
Admin audit log	ONTAP management ops (CLI / REST API)	Managed syslog ingestion → CloudWatch Logs	✅ this article
File access audit log	NAS file/folder ops (NFS / SMB)	FSx for ONTAP S3 AP → Lambda	Separate pipeline
EMS events	ONTAP system events (capacity / HA / ARP)	Managed syslog ingestion → CloudWatch Logs, or EMS Webhook → Lambda	✅ when syslog-delivered

Admin audit log is the "who did what as an administrator" record — logins (success/failure), security login account and role changes, volume create/offline/delete, vserver config changes, and privileged operations like system node systemshell or set -privilege diagnostic.

Two ONTAP settings, don't conflate them: security audit decides what gets recorded (e.g., security audit modify -cliget on -httpget on -ontapiget on to capture read/GET operations — off by default, so without it your sensitive-file-access query on admin ops comes up empty), while cluster log-forwarding decides where it's sent. The exact commands are in the Syslog VPCE setup guide. Note cluster log-forwarding supports multiple destinations, so you can add CloudWatch alongside an existing on-prem SIEM without cutting over.

File access audit log is the "which user did what to which file" record, enabled via vserver audit create and emitted in Windows Security Event format (EVTX / XML). It's the right source for sensitive-folder access and mass-delete detection — but to use it with a Log Alarm you'd need a separate pipeline to land EVTX/XML into CloudWatch Logs. In this project, that content flows via the FSx for ONTAP S3 Access Point → Lambda path to each vendor instead.

⚠️ Critical scoping (read before you trust a preset name): the presets in this article run against the admin audit log (/syslog/fsxn-admin-audit). That log contains management-plane operations — it does not see end-user file I/O over NFS/SMB. So on this log group:

bulk-delete-operations detects admin-plane destructive ops (Snapshot delete, volume offline/delete) — not ransomware encrypting user files over SMB.

sensitive-file-access matches a path only when an admin command references it — not when a user opens that file.

For user-file ransomware/mass-delete/sensitive-access detection, use ONTAP ARP (Part 3) and FPolicy / file-access audit (Part 4). The same preset works against a file-access-audit log group if you land that data in CloudWatch Logs — but on /syslog/fsxn-admin-audit it only sees the admin plane.

EMS events (system events)

The third source is EMS (Event Management System) — ONTAP's internal system-event notifications. Where audit logs say "who did it", EMS says "what happened to the system", across seven severities (emergency → debug). Representative events this project normalizes:

Event	Severity	Meaning
`arw.volume.state` / `arw.vserver.state`	alert	ARP (Autonomous Ransomware Protection) state transition
`monitor.volume.full` / `wafl.vol.full`	alert	Volume space exhaustion
`wafl.quota.hardlimit.exceeded`	error	Quota hard-limit exceeded
`cf.fsm.takeoverStarted`	alert	HA takeover started
`net.linkDown`	alert	Network link down

EMS has two routes. This project's EMS Webhook path (HTTPS → API Gateway → Lambda) normalizes and ships events to any vendor/OTLP. Alternatively, ONTAP 9.x can forward EMS over syslog (event notification destination create -syslog ...) to the same CloudWatch Logs syslog VPC endpoint you built for the admin audit log — no separate EC2 syslog server — and then EMS becomes a Log Alarm target directly. If your need is "alert immediately on volume-full or an ARP state change", syslog-to-CloudWatch + a Log Alarm is the shortest path.

Architecture

FSx for ONTAP (ONTAP log-forwarding)
    │ Syslog TCP (TLS 6514)
    ▼
CloudWatch Logs managed syslog ingestion (VPC endpoint)
    │  → /syslog/fsxn-admin-audit
    │ Scheduled Query (5 min)
    ▼
CloudWatch Log Alarm
    │ count(*) > threshold → ALARM
    ▼
SNS → Email / Slack / PagerDuty (with log lines)

The actual log format

Writing a Log Alarm query is easier if you know what the line looks like. A real admin-audit line in CloudWatch Logs:

<190>Jul  2 03:17:37 FsxId...-02: [kern_audit:info:6392]
...:: FsxId...:ssh :: <source-ip>:unknown ::
FsxId...:fsx-control-plane:admin ::
system node systemshell -node * -command "top -d 1 -s 1"
:: Success

It reads like an incantation, but everything is there: when, which protocol (ssh/http), from where (source IP), who (user), what (command), and the outcome (Success/Failure). A query usually just needs one of those.

Why "Alert on a String" Actually Means "Count Then Compare"

Worth recalling: a CloudWatch alarm compares a number against a threshold. It is not natively a "fire when this string appears" trigger.

So how does a Log Alarm alert on content? It turns the string into a number first. A Logs Insights query narrows to matching events, count(*) converts them to a count, and the alarm fires when that count crosses the threshold. Read it as string → count → threshold and it clicks.

For example, to detect access to a confidential folder:

fields @timestamp, @message
| filter @message like /\/vol\/data\/confidential/

Set the aggregation to count(*) and the threshold to > 0, and a single access within the 5-minute window flips the alarm to ALARM. "Tell me the moment anyone touches the confidential folder" — done, with just that.

Deploy

The repo ships a deploy script, so a few environment variables bring up the whole set of resources.

Deploy script (recommended)

# Sensitive-file access detection (auto-creates the SNS topic)
DETECTION_TYPE=sensitive-file-access \
TARGET_PATTERN="/vol/data/confidential" \
CREATE_SNS_TOPIC=true \
SNS_TOPIC_NAME=fsxn-security-alerts \
  bash shared/scripts/deploy-log-alarm.sh

The script creates the SNS topic, deploys the CloudFormation stack, and prints the alarm name and console URL.

CloudFormation template

# shared/templates/cloudwatch-log-alarm.yaml (excerpt)
Resources:
  SensitiveFileAccessAlarm:
    Type: AWS::CloudWatch::LogAlarm
    Properties:
      AlarmName: fsxn-sensitive-file-access
      ComparisonOperator: GreaterThanThreshold
      Threshold: 0
      QueryResultsToEvaluate: 3
      QueryResultsToAlarm: 1
      TreatMissingData: notBreaching
      ScheduledQueryConfiguration:
        QueryString: |
          fields @timestamp, @message
          | filter @message like /\/vol\/data\/confidential/
        LogGroupIdentifiers:
          - /syslog/fsxn-admin-audit
        ScheduledQueryRoleARN: !GetAtt ScheduledQueryRole.Arn
        AggregationExpression: "count(*)"
        ScheduleConfiguration:
          ScheduleExpression: "rate(5 minutes)"
          StartTimeOffset: 300
      AlarmActions:
        - !Ref AlarmSNSTopic
      ActionLogLineCount: 5
      ActionLogLineRoleArn: !GetAtt LogLineRole.Arn

On QueryResultsToEvaluate / QueryResultsToAlarm (M-of-N): this is the flapping control — 3 / 1 fires fast but can flap on a single spike; 3 / 2 smooths transient blips at ~one extra interval of latency. Tune this before the threshold. Per-use-case recommendations are in the setup guide.

Detection presets

The template ships five presets for common patterns. Switch DetectionType and each gets an appropriate query and threshold.

DetectionType	Use case	Default threshold
`sensitive-file-access`	Access to a specific path	> 0
`failed-access-attempts`	Authentication/authorization failures	> 10
`bulk-delete-operations`	Mass deletion (ransomware signal)	> 50
`specific-user-activity`	Privileged-user monitoring	> 0
`custom`	Any Logs Insights query	your choice

⚠️ Regulated environments — one thing to get right first: ActionLogLineCount puts the matched log lines (usernames, file paths, client IPs — potentially PHI) into the SNS notification, which leaves the CloudWatch boundary. For healthcare/finance/public-sector data, default to ActionLogLineCount: 0 and let responders pivot into Logs Insights for the detail. This is a detection mechanism, not a compliance attestation — classify your fields and confirm APPI/FISC/ISMAP/HIPAA scope with your compliance team first. Full guidance, the regulated default, the alert-audit-trail requirements, and multi-account (StackSets) rollout are in the setup guide.

Use Cases

1. Admin-plane destructive-op detection (defense in depth)

To be precise about scope: on the admin audit log this preset catches management-plane destructive operations — a burst of Snapshot deletes, volume offline, volume delete — the kind of action an attacker with stolen admin credentials (or a mistaken operator) would take to remove recovery points. It does not see user-file encryption over SMB; that's ONTAP ARP's job at the storage layer (Part 3), with FPolicy (Part 4) for per-file operations. Layer all three: ARP for encryption, FPolicy for file ops, and this Log Alarm for admin-plane tampering (e.g., someone deleting the Snapshots you'd restore from).

DETECTION_TYPE=bulk-delete-operations \
ALARM_THRESHOLD=50 \
QUERY_RESULTS_TO_ALARM=2 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

Detection layer	Catches	Method	Latency
Storage layer (ARP)	User-file encryption	ML-based entropy analysis	Real-time
File ops (FPolicy)	Per-file create/write/delete/rename over NFS/SMB	Protocol-level intercept	~6 s (validated, Part 4)
Admin plane (Log Alarm)	Snapshot/volume destructive ops by admins	Count-based threshold on admin audit log	~5 min

Three different vantage points on the same attack: ARP sees the encryption, FPolicy sees the file operations, and this Log Alarm sees an admin deleting the Snapshots you'd recover from. Layering them means whatever slips past one is more likely caught by another.

In ATT&CK terms, that admin-plane view is T1490 Inhibit System Recovery — an attacker deleting your restore points so you can't roll back the encryption ARP detects (T1486). Two techniques, two controls: detect the Snapshot deletion here, and prevent it with SnapLock (WORM Snapshots that can't be deleted before expiry) so your recovery points survive the attempt. The full ATT&CK mapping, tamper-resistance guidance (who can delete the alarm and how to guard it), and a one-slide coverage map are in the setup guide.

Ops note (baseline first): Scheduled bulk operations — nightly backups, batch ETL, archive cleanups — can legitimately exceed a "50 deletes / 5 min" threshold and page on-call for nothing. Baseline your normal delete volume for a few days without an alarm action, then set the threshold above your routine peak (the exact baseline query is in the setup guide). This mirrors the ARP learning-period caveat from Part 3.

2. Compliance: notify on regulated-data access

For finance or healthcare, where "any touch of this data must be recorded and notified", a simple per-path detection works.

DETECTION_TYPE=sensitive-file-access \
TARGET_PATTERN="/vol/finance/" \
ALARM_THRESHOLD=0 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

Alert-fatigue note: A > 0 threshold on an actively-used path pages on every access and quickly becomes noise. Reserve > 0 for genuinely restricted paths (break-glass directories, quarantined data). For paths with legitimate regular access, prefer a rate threshold (e.g., access from an unexpected principal, or volume above a baseline) and route to a ticket/Slack channel rather than a pager.

3. Privileged-user monitoring

Keeping a record of every admin-account action uses the same mechanism.

DETECTION_TYPE=specific-user-activity \
TARGET_PATTERN="fsxadmin" \
ALARM_THRESHOLD=0 \
SNS_TOPIC_ARN=<YOUR_SNS_ARN> \
  bash shared/scripts/deploy-log-alarm.sh

E2E Validation (Tokyo Region)

Because it's a brand-new feature, I was skeptical it would behave as documented. So I deployed the template in a real Tokyo-region environment and drove the alarm through a state transition.

Item	Result
CloudFormation deploy	✅ Success
IAM role auto-creation	✅ ScheduledQueryRole + LogLineRole
Scheduled Query execution	✅ INSUFFICIENT_DATA → OK transition confirmed
Console display	✅ Shown as "Log alarm" type
Logs Insights query	✅ Matched (12 hits / 3,482 records scanned for `/volume/`; 472 hits for `ssh`)

In the console it shows up as a new "Log alarm" type, distinct from the metric alarms you already have. In the screenshot below, look at the Type column — the new alarm is labeled "Log alarm" rather than "Metric alarm".

Opening the alarm shows the Log Alarm detail page. Note the query configuration (the Logs Insights query string, the target log group /syslog/fsxn-admin-audit, and the 5-minute schedule) and the two IAM roles CloudFormation created automatically — ScheduledQueryRole (runs the query) and LogLineRole (attaches matched log lines to the notification).

Running the query in Logs Insights hits real audit data. The bar chart at the top shows the match count per interval, and the results table below lists the matched log lines — here the /volume/ filter returned 12 matches across 3,482 records scanned.

The alarm itself stays OK while there's no matching access — the screenshot shows the state after the initial INSUFFICIENT_DATA → OK transition, with an empty history graph because nothing has crossed the threshold yet. With threshold > 0, it flips to ALARM the moment a single access to the sensitive path appears.

Audit trail of the alert itself: for compliance you also need evidence that the alarm fired and who was notified. Alarm state history is retained 90 days (fixed); for multi-year evidence, route CloudWatch Alarm state-change events (EventBridge) to S3, and rely on CloudTrail for the "who configured this detection" record. Details and retention table: setup guide.

Gotchas

Gotcha	Workaround
AWS CLI not yet supported (no `put-log-alarm`)	Use CloudFormation
cfn-lint E3006 (resource type not yet in the spec)	Suppress per-resource (not a blanket disable); exact `Metadata` snippet in the setup guide
First evaluation takes 5–10 min	Just wait
Notification includes log lines (PII/PHI risk)	Set `ActionLogLineCount=0` in regulated environments (see the callout above)
Multi-node log streams (`FsxId...-01/-02`)	Query the whole log group, don't pin `@logStream`, or you miss half the traffic on HA takeover

The "AWS CLI doesn't have put-log-alarm yet" is a just-after-GA reality — for now CloudFormation or the console are the only creation paths. Platform/CoE note: drift detection won't cover a resource type the CLI can't yet describe, so treat CloudFormation as the single source of truth until the API surface completes.

Cost

Rough estimate for one alarm at a 5-minute cadence.

Logs/day	Log Alarm	Metric filter
100 MB	~$6.6/month	~$3/month
500 MB	~$33/month	~$3/month
1 GB	~$66/month	~$3/month

How the cost scales: the table above is per alarm. Each alarm runs its own Scheduled Query over the same log group, so cost is alarms × cadence × scan size — ten alarms on one log group is ~10×, not a flat add-on. Narrow queries with filter/limit and consolidate related detections to keep it bounded. Full breakdown in the setup guide.

Honestly, Log Alarms cost more than the metric-filter approach — the Scheduled Query scans logs on each run. But the difference is recoverable elsewhere: log lines in the notification cut investigation time, it applies retroactively to existing logs, and setup is a single step.

When the premium is worth it (a line you can put in a proposal): choose Log Alarms when the alert content matters for triage (you want the matched lines in the page), when the detection query needs full Logs Insights syntax a metric filter can't express, or when you need to apply it retroactively to existing logs. Stick with metric filters for high-volume, well-understood, purely numeric signals where a few dollars × many alarms adds up. For most teams the crossover is engineer time: if the log-line context saves even one 15-minute console dig per incident, the premium pays for itself.

If volume makes cost a concern, dropping the cadence to 15 minutes cuts it to a third, and adding limit to the query bounds the scan.

When to Use Which

Log Alarms aren't meant to replace every kind of monitoring. Depending on the need, metric filters or the OTel Collector fit better. Among this project's three delivery paths, the Log Alarm is the simplest entry point.

Path	When	Extra infra
Log Alarm (this article)	Simple threshold alerts	None (self-contained in CloudWatch)
Lambda → vendor	Dashboards, SIEM	Forwarding Lambda
OTel Collector	Multi-backend, PII redaction	Forwarding Lambda + Collector (ECS Fargate)

Concretely, the OTel Collector path's "extra infra" is two layers: three forwarding Lambdas (audit-log shipper, EMS handler, FPolicy handler) sending OTLP/HTTP, plus a resident otel/opentelemetry-collector-contrib task on ECS Fargate (with NAT Gateway for egress, Cloud Map for task IP resolution, ALB/autoscaling under load). Fan-out to Grafana/Honeycomb/Datadog and PII redaction are then a single change in the Collector's config YAML. The Log Alarm, by contrast, is the "everything stays inside CloudWatch" minimal option.

A Log Alarm is a first alert, not a full investigation tool. Let it handle "notice it first", and hand off the deep dive to vendor tooling like Datadog or Splunk. That division of labor is the realistic one.

For ONTAP operators, the setup guide covers the storage-side specifics: what security audit captures vs what log-forwarding sends, keeping an existing SIEM alongside CloudWatch (multiple destinations), ONTAP EMS native email/SNMP as an alternative to pushing to AWS, a dead-man's-switch heartbeat alarm for when syslog delivery stops, and DR-region deployment — ONTAP operational notes.

Cleanup

# Delete all Log Alarm stacks
bash shared/scripts/cleanup-log-alarm.sh --all -y

What's Next

CloudWatch Log Alarms aren't a flashy feature. But "turn what you noticed in the logs straight into an alert" lowers the bar for setting up monitoring — you're done with one query before you'd have finished building a metric filter, staring at a metric, and wiring an alarm. It pairs well with FSx for ONTAP audit logs, answering the "I just want to notice it first" needs of storage security without extra infrastructure. There's some just-after-GA roughness (the AWS CLI hasn't caught up), but CloudFormation works today, verified in a real environment.

Upcoming in the project:

Phase 4: Terraform module equivalents
Phase 4: CDK construct library
PagerDuty escalation for CloudWatch alarms — see pagerduty-escalation-guide

See the full ROADMAP.

Resources

GitHub: github.com/Yoshiki0705/fsxn-observability-integrations
Template: cloudwatch-log-alarm.yaml
Deploy script: deploy-log-alarm.sh
Setup guide (EN): cloudwatch-log-alarm.md
Runbook: log-alarm-triggered.md
Syslog VPCE setup: syslog-vpce-setup-guide.md

AWS references

What's New — CloudWatch supports creating alarms from log queries (2026-07)
What's New — CloudWatch Logs supports managed syslog ingestion (2026-06)
Docs — Alarming on logs
Docs — Setting up syslog ingestion

Series Navigation

Part 1: Why Your FSx for ONTAP Logs Deserve Better
Part 3: Event-Driven Ransomware Detection with ONTAP ARP + Datadog
Part 5: Escape Vendor Lock-in with OTel Collector
Part 13: 9 Vendors, One Architecture
Part 14: Can You Use System Manager with FSx for ONTAP?
Part 17: Alerting on Audit Logs with CloudWatch Log Alarms (this post)

If you deploy this, I'd love to hear how it went — drop a comment or open a GitHub issue.

GitHub: github.com/Yoshiki0705/fsxn-observability-integrations

42 Patterns, Category Architecture, and HA LifeKeeper Monitoring — FSx for ONTAP S3 Access Points, Phase 18

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 21 Jun 2026 16:54:22 +0000

TL;DR

Phase 18 restructures the entire repository from 41 flat directories into a categorized solutions/ hierarchy, adds HA LifeKeeper Monitoring as a new pattern category, introduces 5 category-specific architecture diagrams, and establishes modern Python project infrastructure. The repository now contains 42 deployable patterns organized for discoverability at scale.

Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

Why Restructure?

With 41 pattern directories at the repository root, navigating the project had become unwieldy. New contributors could not quickly find patterns by domain, the README required extensive scrolling, and adding new categories (HA, GenAI) had no clear placement convention.

Before (flat):

legal-compliance/  financial-idp/  semiconductor-eda/  sap-erp-adjacent/
flexcache-anycast-dr/  genai-kb-selfservice-curation/  event-driven-fpolicy/
... (41 directories at root)

After (categorized):

solutions/
├── industry/           # 28 UC patterns (UC1-UC28)
├── flexcache/          # 7 FlexCache/FlexClone patterns
├── genai/              # 2 GenAI patterns (UC29-UC30)
├── sap/                # SAP/ERP pattern
├── ha/                 # HA monitoring (new)
├── event-driven/       # 2 FPolicy event-driven patterns
└── edge/               # CDN/edge delivery

The Move: git mv with History Preservation

All 41 directories were moved using git mv, preserving full commit history. Key renames include:

Old Path	New Path	Reason
`sap-erp-adjacent/`	`solutions/sap/erp-adjacent/`	Category grouping
`dynamic-flexcache-render-workflow/`	`solutions/flexcache/dynamic-render-workflow/`	Shorter name
`genai-kb-selfservice-curation/`	`solutions/genai/kb-selfservice-curation/`	Strip prefix
`devops-flexclone-cicd/`	`solutions/flexcache/devops-cicd/`	Strip prefix
`content-edge-delivery/`	`solutions/edge/content-delivery/`	Category grouping

Use git log --follow <file> to trace history across the move.

HA LifeKeeper Monitoring — New Pattern

SIOS LifeKeeper is a Linux/Windows HA clustering solution that can be used on Amazon EC2 for application-aware failover scenarios. With FSx for ONTAP Multi-AZ as shared storage (NFS/iSCSI, depending on OS and configuration), this pattern focuses on observing LifeKeeper logs without putting monitoring agents on the HA nodes.

The new HA pattern (solutions/ha/lifekeeper-monitoring/) provides non-intrusive log analysis:

graph TB
    subgraph "HA Cluster"
        LK1[LifeKeeper Node 1<br/>Active]
        LK2[LifeKeeper Node 2<br/>Standby]
    end

    subgraph "Shared Storage"
        FSXN[FSx for ONTAP Multi-AZ]
        S3AP[S3 Access Point<br/>Read-only log access]
    end

    subgraph "Analysis Pipeline"
        SFN[Step Functions]
        DISC[Discovery Lambda<br/>Log classification]
        PROC[Processing Lambda<br/>Bedrock Root Cause Analysis]
        RPT[Report Lambda<br/>Health score + alerts]
    end

    LK1 -->|Log write| FSXN
    FSXN --> S3AP -->|Non-intrusive read| DISC
    SFN --> DISC --> PROC --> RPT
    PROC -->|Nova Pro| BEDROCK[Amazon Bedrock]
    RPT --> SNS[SNS Alert]

Key Design Decisions

Non-intrusive to HA nodes: No monitoring agent is installed on HA nodes. The S3 AP read path avoids host-level changes, while still consuming FSx/S3 API throughput like any other read workload.
Human-in-the-loop: AI analysis is advisory only. LifeKeeper's own health checks handle failover decisions.
Health Scoring: 0-100 score with deductions for failover events, comm path latency, and resource state anomalies.
Root Cause Analysis: Bedrock Nova Pro analyzes state transitions (ISP→OSF, ISS→ISP) to identify likely causes.

Implementation note: This pattern observes LifeKeeper logs and produces advisory analysis. It does not replace LifeKeeper cluster design, quorum/witness configuration, split-brain prevention, protocol-specific recovery kit setup, or application-level failover testing.

Example Operational Metrics

These are suggested evaluation metrics for future validation. Phase 18 verifies the DemoMode pipeline, not real-cluster failover triage time.

Category	Metric	Demo Target
Operations	Time from workflow start to structured triage report	< 10 min
Technical	Log discovery completeness	All configured LifeKeeper log paths in scope
Quality	False positive alert rate	< 5% (requires real-cluster validation)
Cost	Monthly monitoring cost	< $15 (5-min polling, 1 cluster)

Deploy (DemoMode)

DemoMode=true deploys without FSx for ONTAP — uses a regular S3 bucket with sample LifeKeeper logs:

cd solutions/ha/lifekeeper-monitoring
sam build && sam deploy --guided \
  --parameter-overrides DemoMode=true \
    S3AccessPointAlias=your-demo-bucket \
    OutputBucketName=your-output-bucket

DemoMode verification confirms: SAM template deploys successfully, Step Functions workflow executes all states, Discovery Lambda classifies sample logs correctly, and Processing Lambda generates a health score report. Bedrock analysis produces structured, advisory observations based on log patterns; failover decisions remain outside the AI workflow.

Verified in ap-northeast-1 on 2026-06-21. The sample log set intentionally contains failure-like events, resulting in a demo health score of 40/100. The score is not a benchmark for LifeKeeper or FSx for ONTAP; it demonstrates that the scoring pipeline detects and reports anomalies from sample logs.

Coming in Phase 19: Full E2E verification with a real SIOS LifeKeeper HA cluster (AWS Marketplace + FSx for ONTAP Multi-AZ) including live failover testing, actual state transition detection, and Bedrock RCA quality assessment against real-world failure scenarios.

Category Architecture Diagrams

Phase 18 adds 5 mermaid architecture diagrams to README.md (both JA and EN), each in a collapsible <details> block:

Category	Key Components
🏭 FlexCache	ONTAP REST API → HealthCheck → RouteDecision → DynamoDB routing → Create/Cleanup lifecycle
🤖 GenAI	FPolicy → SQS → EventBridge → Bedrock KB → RetrieveAndGenerate / Agentic tools
🛡️ HA	S3 AP non-intrusive read → Bedrock RCA → Health score → SNS alerts
⚡ Event-Driven	FPolicy Engine → ECS Fargate TCP → SQS → EventBridge rule routing
🌐 Edge/CDN	3 delivery modes (ORIGIN_PULL, OAC, PUBLISH_PUSH) → vendor-neutral CDN

All diagrams include accTitle and accDescr for screen reader accessibility.

Project Infrastructure Improvements

pyproject.toml (PEP 621)

Modern Python project metadata with unified tool configuration:

[tool.ruff]
target-version = "py312"
line-length = 120

[tool.pytest.ini_options]
addopts = "-v --tb=short --import-mode=importlib"

[tool.coverage.report]
fail_under = 80

Dependency Pinning

# requirements.txt — exact versions for reproducibility
boto3==1.43.29
urllib3==2.7.0
jsonschema==4.17.3

# requirements-dev.txt
pytest==9.1.0
hypothesis==6.155.2
moto==5.2.2
ruff==0.15.17
cfn-lint==1.51.4

.cfnlintrc

Project-wide cfn-lint configuration that discovers all templates under solutions/:

templates:
  - "solutions/**/template.yaml"
  - "solutions/**/template-deploy.yaml"
ignore_checks:
  - W3002  # Local CodeUri (sam build handles upload)
  - W1031  # Fn::Sub false positive with Secrets Manager ARNs
regions:
  - ap-northeast-1

Additional Tooling

File	Purpose
`.gitattributes`	Consistent line endings, language-specific diff drivers
`.github/PULL_REQUEST_TEMPLATE.md`	Project-specific PR checklist
`solutions/README.md`	Category navigation index
`CHANGELOG.md`	Keep a Changelog format, all releases
`CONTRIBUTING.md`	"Adding a New Pattern" section with category guide

CI/CD Changes

The CI pipeline was split to avoid pytest importlib mode namespace collisions when multiple patterns with identically-named handler.py files are collected together:

# Before: single pytest invocation (collision risk)
pytest shared/tests/ solutions/**/tests/ --cov=shared

# After: isolated invocations
- name: Run shared tests with coverage
  run: pytest shared/tests/ --cov=shared
- name: Run pattern tests
  run: pytest solutions/industry/*/tests/ solutions/flexcache/*/tests/ ...

Also added persist-credentials: false to all actions/checkout steps (zizmor security hardening).

Multi-Perspective Review

The restructuring and HA monitoring pattern were reviewed from partner delivery, storage architecture, HA operations, security, CI/CD, accessibility, and contributor onboarding perspectives. The review resulted in wording changes around neutrality, operational caveats, connector validation, HA safety boundaries, and repository discoverability.

What This Means for You

Role	Impact
Partner / SI / delivery team	Pattern selection is now intuitive by category — pick `solutions/industry/` for industry PoCs, `solutions/flexcache/` for distributed workloads
New contributor	`CONTRIBUTING.md` now includes "Adding a New Pattern" with required checklist and category selection guide
Existing repository users	`git log --follow <file>` still works. `sam build` and `sam deploy` are unchanged (run from the pattern directory)
CI/CD	Zero changes needed in your `samconfig.toml` or deployment scripts — CodeUri is relative to template.yaml

Infrastructure cost: Zero. This is a repository organization change; it does not deploy AWS resources by itself.

What's Next

Phase 16-17 blog articles (GenAI patterns) — ready to publish
dev.to series updated for directory restructuring (Phase 13 link fixed)
Next pattern candidates under evaluation

Getting Started

git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

# Browse patterns by category
ls solutions/

# Deploy HA LifeKeeper monitoring (DemoMode)
cd solutions/ha/lifekeeper-monitoring
sam build && sam deploy --guided --parameter-overrides DemoMode=true

# Run tests
make test-quick PYTHON=.venv/bin/python

Yoshiki Fujiwara

Amazon Quick Agentic Workspace Powered by FSx for ONTAP S3 Access Points — Phase 17

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sun, 21 Jun 2026 15:54:23 +0000

TL;DR

UC30 bridges the gap between file-based business data and AI-powered actions. Business users maintain structured and unstructured data on an FSx for ONTAP SMB share, while Amazon Quick (Index / Sight / Flows) consumes it through S3 Access Points and a serverless Action API — providing search, BI, and governed action workflows from a single workspace.

Where UC29 focuses on "self-service knowledge ingestion into Bedrock KB," UC30 focuses on unifying search, analytics, and action execution behind Quick Suite's agentic interface.

New in this release: generate_brief_with_web action augments internal context with real-time web search results via AgentCore Web Search Tool (GA June 2026), enabling briefs that combine primary internal data with current public context.

Repository: github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns (see solutions/genai/quick-agentic-workspace/ and samconfig.toml.example)

Amazon Quick × S3 AP Data Mapping

Quick Feature	Role	S3 AP Data	Implementation
Quick Index / Research	Unstructured file search	`index/<role>/` (md/pdf)	S3 AP as data source
Quick Sight	Structured BI & visualization	`analytics/<role>/` (csv)	Glue/Athena (Athena Query Lambda)
Quick Flows	Action automation	`flows/<role>/` (json)	Action API (API Gateway + Lambda + Bedrock)
Quick Flows + Web	Web-augmented briefs	`flows/<role>/` + web	Action API + AgentCore Web Search (opt-in)

Seven roles (sales / marketing / finance / IT / operations / legal / developers) share the same AI-dedicated volume — reusable from UC29.

Design note: FSx for ONTAP S3 Access Points are useful as an integration boundary, but they do not remove the need to validate each consuming service connector. The access path combines S3/IAM policy evaluation with file-system-level identity authorization.

Architecture

Amazon Quick provides a unified workspace — search (Index), BI (Sight), and action automation (Flows) — powered by FSx for ONTAP data via S3 Access Points.

Windows Explorer (drag & drop into quick-workspace/ SMB share)
  ├── index/<role>/ → Quick Index (unstructured search)
  ├── analytics/<role>/ → Glue/Athena → Quick Sight (BI)
  └── flows/<role>/ → Action API → Quick Flows (actions)

Action API (6 actions):
  API Gateway (IAM auth / SigV4)
  → Lambda (per-action authorization + HITL gate)
  → generate_brief           → Bedrock Converse (internal context only)
  → generate_brief_with_web  → Bedrock Converse + AgentCore Web Search (hybrid)
  → create_action_item       → SNS notification
  → request_approval         → DynamoDB (HITL entry)
  → approve                  → DynamoDB (admin only)
  → execute_approved         → DynamoDB check + execution (enforced HITL)

Hybrid RAG Flow (generate_brief_with_web)

Quick Flows request: {"action": "generate_brief_with_web", "params": {...}}
  ├─→ [1] Internal context (from params.context — FSx for ONTAP file content)
  ├─→ [2] AgentCore Web Search (us-east-1, MCP protocol)
  │       query from params.web_query or params.title
  │       → Amazon web index → snippets + URLs + titles + dates
  └─→ [3] Bedrock Converse (ap-northeast-1)
           system prompt: internal = primary, web = supplementary, untrusted
        → Unified brief with [Internal: ...] + [Web: title](URL) citations

Security Design

Authentication + Per-Action Authorization

The Action API uses IAM authentication (SigV4). The handler extracts the authenticated caller identity (requestContext.identity) — not self-declared body fields — and performs per-action authorization:

ACTION_AUTH_MODE=open (default/demo): No enforcement; audit fields still bound to authenticated caller

For production, use ACTION_AUTH_MODE=enforce and explicitly define AUTHORIZED_PRINCIPALS and ADMIN_PRINCIPALS.

ACTION_AUTH_MODE=enforce (production):

Read-only actions (generate_brief, generate_brief_with_web): always allowed

Mutating actions: caller must match AUTHORIZED_PRINCIPALS

Admin actions (approve): caller must match ADMIN_PRINCIPALS

Mismatch → 403 Forbidden

Enforced Human-in-the-Loop (HITL)

High-risk operations are gated by a DynamoDB approval store:

request_approval → persists record as pending_approval (enforced=true)
approve → admin transitions to approved (ConditionExpression prevents race)
execute_approved → only executes if record is approved; otherwise 409

Verified live: execute pre-approval → 409, post-approval → 200, re-execute → 409 (no replay).

Security note: Approval records have a 7-day TTL (DynamoDB Time-to-Live). Stale pending approvals auto-expire, preventing indefinite accumulation of unreviewed requests. Expired records cannot be approved or executed.

Additional Controls

Prompt injection defense: Both generate_brief and generate_brief_with_web treat context as untrusted data with explicit delimiter boundaries (<internal_context>, <web_search_results>)
Web query safety: Only params.web_query or params.title is sent to Web Search — never internal document content
Raw SQL disabled by default: ALLOW_RAW_SQL=false; role-level data boundaries enforced via Lake Formation (LF-TBAC) in production
Results bucket hardening: PublicAccessBlock + TLS-only + 30-day lifecycle
API throttling: Rate/burst limits against denial-of-wallet
Web Search citation obligation: Source URLs + titles are always included in responses (Acceptable Use Policy compliance)

Web-Augmented Brief Generation (opt-in)

GA at AWS Summit NYC 2026 (June 17, 2026). Powered by AgentCore Web Search Tool.

The Problem

Business briefs based solely on internal documents lack current market context. A sales brief about a product launch needs both the internal product spec and awareness of relevant public announcements published recently. A legal compliance brief needs both the internal policy document and the latest regulatory guidance.

The Solution

A new action generate_brief_with_web combines internal context with real-time web search results. The internal context remains the primary source; web results are supplemental, cited, and treated as untrusted input.

Usage

{
  "action": "generate_brief_with_web",
  "params": {
    "title": "Q3 Data Protection Regulatory Update",
    "context": "Internal operations follow FISC safety standards...",
    "web_query": "data protection regulation 2026 Japan financial services"
  }
}

Response

{
  "status": "completed",
  "action": "generate_brief_with_web",
  "title": "Q3 Data Protection Regulatory Update",
  "brief": "Based on internal FISC compliance documentation... Additionally, [Web: FISC 2026 Revision Summary](https://example.com/fisc) published on 2026-06-10 introduces...",
  "web_citations": [
    {"source": "https://example.com/fisc", "title": "FISC 2026 Revision Summary", "publishedDate": "2026-06-10"}
  ],
  "web_search_enabled": true,
  "guardrail_applied": true
}

Design Properties

Property	Detail
Internal context priority	Internal documents are the primary source; web supplements
Graceful degradation	Web Search failure → behaves like `generate_brief` (internal only)
Citation separation	Internal sources and web sources are visually distinct in the brief
Query safety	Only the `web_query` (or `title`) is sent externally — never internal content
Cross-region	Gateway in us-east-1 (Web Search Tool constraint); adds ~100-200ms
Authorization	Read-only action (same tier as `generate_brief`)
Prompt injection defense	Web results wrapped in `<web_search_results>` as untrusted data

Activation

sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Without these parameters, generate_brief_with_web still works but produces internal-only briefs (graceful degradation).

Verification Findings

Lake Formation + Athena

Athena queries running against Glue tables backed by S3 AP data — the foundation for Quick Sight analytics.

The UC30 CloudFormation stack with all resources (API Gateway, Lambda, DynamoDB ApprovalsTable, Athena WorkGroup) deployed.

The test account had Lake Formation governing the Data Catalog. The Athena Query Lambda's execution role required Lake Formation permission grants (DESCRIBE on DB, SELECT/DESCRIBE on tables) in addition to IAM. Production deployments should design LF-TBAC for role-based data visibility.

Amazon Quick × FSx for ONTAP S3 AP Integration Boundary

Amazon Quick's S3 KB connector accepts the S3 AP alias but authorization fails due to FSx for ONTAP's dual-layer auth — leading to the recommendation below.

Quick provides multiple data integration paths — for FSx for ONTAP data, Bedrock KB (UC29) or Athena-mediated access is the validated route in this repository.

Amazon Quick's S3 knowledge base connector accepts an FSx for ONTAP S3 AP alias as a "valid URL," but connection verification fails with "You do not have permissions to access the S3 bucket." This is not a missing IAM permission that can be simply added — it is a structural authorization path mismatch. Three factors block the standard connector path:

ARN format mismatch — FSx for ONTAP S3 APs use arn:aws:s3:{region}:{account}:accesspoint/{name} (not the arn:aws:s3:::{bucket} format). The connector's internal IAM evaluation likely targets the alias as a bucket name, which does not match IAM policy evaluation for the AP ARN.
AP resource policy rejects the principal — Adding Quick's data access role to the S3 AP resource policy returns MalformedPolicy: Invalid principal in policy, indicating a principal constraint on FSx for ONTAP S3 AP policies.
Layer 2: filesystem identity — Even if IAM layers were resolved, the ONTAP file-system identity (UNIX UID or AD user) associated with the S3 AP must independently permit read access to the target files.

For readers attempting this path: Use CloudTrail to capture the exact AccessDenied event and identify Quick's calling principal ARN. If the MalformedPolicy constraint can be resolved (potentially by configuring the S3 AP with an AD-based identity rather than UNIX root), the direct path may become viable. As of June 2026, Bedrock Knowledge Base → FSx for ONTAP S3 AP is the confirmed working route for RAG ingestion.

Evidence-based implementation guidance:

FSx for ONTAP → RAG: Bedrock KB (UC29) is the validated path in this repository
Quick Index: Stage to a standard S3 bucket for predictable connector behavior
Quick Sight (BI): Athena-mediated access works (QuickSight role needs Athena/Glue/LF/results-bucket permissions)
Direct Quick → FSx for ONTAP S3 AP (standard connector path): validated — does not work (tested with UNIX root identity; MalformedPolicy on AP resource policy). AD-based S3 AP identity configuration is tracked as a future hypothesis

Glue Tables

analytics/<role>/ CSVs are pointed to by Glue tables (sales_pipeline / it_incidents) created via Athena DDL. LOCATION uses S3 AP alias format: s3://<alias>/quick-workspace/analytics/<role>/. For scale, migrate to Parquet + partitioning to reduce Athena scanned costs.

Data Classification

Output	Classification	Rationale
Action API response (generate_brief)	INTERNAL	Source-derived summary; no external disclosure
Action API response (generate_brief_with_web)	INTERNAL	Contains internal citations; web portion is PUBLIC
Action API response (create/approve/execute)	INTERNAL	Business operation records
Athena query results (results bucket)	INTERNAL	Encrypted + 30-day lifecycle + TLS
DynamoDB ApprovalsTable	INTERNAL	Approval state metadata
SNS notifications	INTERNAL	Action summaries only; no file content
Web Search results (raw)	PUBLIC	External public information

Extend shared/data_classification.py for regulated workloads (CUI / FISC / HIPAA).

Cost

Component	Monthly estimate	Notes
Amazon Quick	Per-user/plan billing	Separate; unsubscribe when done
Athena	Scanned-data pricing	Reduce with Parquet
Lambda / API Gateway	Serverless pay-per-use	< $10 for moderate usage
Bedrock LLM (briefs)	Usage-based	Usage-based; verify the current model price
DynamoDB (approvals)	Pay-per-request	Minimal for approval records
AWS Budgets alarm	Free (SNS delivery cost only)	Created when `NotificationEmail` set
AgentCore Web Search (opt-in)	Per-query pricing (see AgentCore pricing)	Gateway invocation pricing
Cross-region transfer (opt-in)	< $0.02	us-east-1 ↔ ap-northeast-1

Teardown / rebuild: one-command idempotent scripts (scripts/teardown-uc29-uc30.sh / scripts/rebuild-uc29-kb.py)

Getting Started

git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git
cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/genai/quick-agentic-workspace

# Install dependencies
pip install -r requirements.txt        # or: uv pip install -r requirements.txt

cat samconfig.toml.example  # Review parameters

sam build && sam deploy --guided

# DemoMode=true runs without FSx for ONTAP (regular S3 bucket)

# Optional: Enable Web Search hybrid RAG
sam deploy --parameter-overrides \
  EnableWebSearch=true \
  AgentCoreGatewayId=<gateway-id> \
  AgentCoreGatewayRegion=us-east-1

Governance Note

This article is technical architecture guidance, not legal, compliance, or regulatory advice. Amazon Quick features, pricing, regional availability, and connector behavior are subject to change — verify with official documentation and your own account settings. S3 AP data source boundaries are at volume/prefix granularity. For per-user visibility control, use Quick's document-level ACL or Custom Permission-Aware RAG. Web Search Tool usage requires compliance with the Acceptable Use Policy (source citations must be retained and displayed in end-user output).

Yoshiki Fujiwara

DEV Community: Yoshiki Fujiwara(藤原 善基)@AWS Community Builder

AWS Backup now copies FSx for ONTAP backups across Regions and accounts — a new option, measured Tokyo to Osaka

Introduction

Shape of the setup

Test environment

When this fits

When to look at something else for now

The distinction that matters most — what changed is where a backup can live

How CopyBackup and AWS Backup relate

Replication and copy are different words for different things

A short glossary

If you run ONTAP on-premises

Prerequisites and permissions

Prerequisites

IAM

Values to replace with your own

Walkthrough — Tokyo to Osaka

1. Create a backup (skip if you already have one)

2. Copy it

3. [Recovery] Check which generation the destination Region offers

4. [Recovery] Create the destination file system

5. [Recovery] Create the SVM

6. [Recovery] Restore

7. Verify the content

8. Delete what the test created

Measured through AWS Backup as well

A plan-triggered job waits in CREATED

The recovery point in Osaka points at the source

The restore form makes you pick a destination file system and SVM

Storage efficiency is checked by default

The content matched

What cannot be monitored, and what catches you on teardown

For the cross-account half — an earlier post, and what this one adds

What this post was missing — cross-account needs a customer managed key

The Region exceptions are listed individually

On "what was possible before", that post and I read it differently

What this post adds

The two things that caught me out

1. A restoring volume reads as DP, not RW

2. FlexGroup backup creation failed asynchronously

3. The attribute I called "not carried over" was a CLI default

What the console shows that the CLI does not

The copy form defaults to the same Region

While the copy runs

Restore needs more than a file system

The volume size default is 1 TiB

The console shows DP too, and does not refresh

Designing this for production

1. Scheduling — CopyBackup has none

2. Make recovery declarative — CloudFormation can express the restore

3. RTO breakdown — "20 minutes" is one line of it

Restore time gets longer with capacity

And this can overturn "generation 1 is enough"

4. Application consistency — this does not replace a database backup

5. Concurrency limits at scale

6. Monitoring and control

Cost

Unit rates — queried from the AWS Price List API

FSx for ONTAP backup is not the cheap option

Running one continuously has a floor

It inverts with scale

Why a DR standby takes generation 1

Total monthly by throughput

What the "zero while idle" of backup copies buys

The price difference is the requirement difference

Not included above

Inter-Region data transfer is treated differently per path

The trade-off of all-tiering

As a ransomware layer

Common errors and how to isolate them

Where this sits alongside an existing backup product or SaaS

A concrete way to split them

What to ask, regardless of product

About monitoring

Choosing between this and SnapMirror

Running both

Regional DR within one country

A staged rollout

A verification plan template

A recovery runbook skeleton

DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

How `CopyBackup` and AWS Backup relate

A plan-triggered job waits in `CREATED`

1. A restoring volume reads as `DP`, not `RW`

The console shows `DP` too, and does not refresh

1. Scheduling — `CopyBackup` has none

Result: Works After Enabling `-is-s3-enabled true`