DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

What to build a file portal on — Nx Plugin for AWS 1.0, AWS Blocks and Amplify Gen 2, same four features (File Portal Part 7)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Wed, 16 Sep 2026 14:34:33 +0000

Introduction

Nx Plugin for AWS reached 1.0 and AWS Blocks appeared in preview, so starting an application on AWS now has three front doors. I took all three to the same scope, counted the resources the same way, deployed them for real, implemented four features on each, and deleted everything at the end.

This article stands on its own. It continues a series, but the only thing it assumes is that a file portal is running on an FSx for ONTAP S3 Access Point.

What prompted it was reading a Zenn article by an AWS SA that compared all three. Someone else's comparison is useful, but as a basis for a decision in my own repository it is thin: you have to deploy and delete before you can say you compared anything.

Part 1 was about the application layer — what to build a file portal with. This one is about the tooling underneath it.

What the verification covers:

The resource count of each generated output, and the fixed cost its defaults imply
Deploying each one for real and exercising it
The differences that appear once the same four features are implemented on all three
What survives a stack deletion, and the order in which to clean it up

Out of scope: which tool is better, Nx or monorepos as concepts, and keeping generated code current over time.

The conclusions first:

Choosing a tool is a decision you can take back. The data path is common to all three, and switching tools does not mean rebuilding the FSx for ONTAP side.
The difference is not "can it be built" but what gets fixed in place once you have chosen. All three deployed without trouble.
The generated defaults mean fixed monthly cost at zero requests.
What a stack deletion leaves behind comes in two kinds. Retain and deletion protection are the generator's defaults and can be released on the generating side. Log groups that never appear in the template belong to CDK's internals, and there is no route to release those. The KMS waiting period is the only thing that cannot be avoided.
For some resources, notably KMS keys, DELETE_COMPLETE means "scheduled for deletion after a 30-day wait" rather than "gone".

Switching tools does not touch the data foundation underneath

The components side by side, matched to one scope: web, auth, API and a table. The data path below the S3 Access Point is this portal's real configuration and is identical in all three.

Figure 1: components of the file portal composed on Amplify Gen 2

Amplify Gen 2 is what this portal actually runs on. No perimeter resources — AWS WAF, customer managed KMS keys — are created by default. Authentication is Amazon Cognito, on the default ESSENTIALS tier.

Figure 2: components of the file portal composed on the AWS Blocks production preset

What to read in the AWS Blocks figure is the absence of Amazon Cognito. Authentication is AuthBasic (DynamoDB plus JWTs), so it moves out of a managed service and into the tables. Two of the four tables are for auth, and all four carry deletion protection.

Figure 3: components of the file portal composed on Nx Plugin for AWS

For Nx Plugin for AWS the difference is more perimeter in the figure: two AWS WAF web ACLs on the path (a third sits in a separate us-east-1 stack), Amazon Cognito on PLUS with MFA required, and four KMS keys. Those three are the fixed-cost difference described later.

That the three figures match below the access point matters: switching tools does not mean rebuilding the FSx for ONTAP side. Nothing was created or deleted on the FSx for ONTAP side during this verification.

Reaching it was also identical on all three. An Internet-origin access point is reached by passing the alias as a bucket name to the standard S3 SDK — no endpoint override, no VPC attachment. A file on the NAS can be found, opened and written from a browser with no VPN and no SMB mount, and through phase 3 the functions do not even need to sit in a VPC.

The same four features on all three tools (2026-09-14)

Everything above compares the outputs as generated. What someone choosing a tool actually wants to know is the difference once their own application sits on it. So a few days later I implemented the same four features — sign in, list, read, upload (listFiles / readFile / uploadFile) — on each and ran them against the same FSx for ONTAP S3 Access Point.

The target was fixed too: the same prefix (13 objects), the same file (1,615 bytes, text/markdown; charset=utf-8), the same 24-byte write. All three returned the same values.

The listing

.md rows are listed with sizes and modification times" width="800" height="476">
The listing on the AWS Blocks build: thirteen files.

The listing on the Nx Plugin for AWS build, with the generated shadcn sidebar.

The listing on the Amplify Gen 2 portal. Being a portal in actual use, it carries extra features such as stars and download icons.

Reading

.md sit 1615 bytes and text/markdown; charset=utf-8, the same report is rendered in the page, and an Upload section follows below it" width="800" height="476">

The two above render the report in the page. The Amplify Gen 2 portal below does not: it hands .md to an AI Assistant panel instead, because that is how I wrote it.

An aside: a by-product of the capture pass was one accessibility defect in my own portal. The document icon on a file row announces itself as aria-label="Download <filename>", yet what it actually does is select the file for AI processing. The title attribute mentions both, but a screen reader reads out only Download, so someone using assistive technology is told the button downloads a file when it starts AI processing instead. It is unfixed as of writing, recorded as something for the portal side to correct. Without lining up three builds in the same order, I probably would not have noticed.

Uploading

The two above are listings right after a successful write. The Amplify screen below is a refusal: the same file name was sent twice and the server side declined to overwrite. That is a difference in what I implemented, not in the tools.

How the IAM permission differs by tool

Reaching the S3 Access Point is the same on all three, but the IAM needs care: all three require the access-point ARN form.

arn:aws:s3:<region>:<account-id>:accesspoint/<name>          # ListBucket
arn:aws:s3:<region>:<account-id>:accesspoint/<name>/object/* # GetObject, PutObject

Written in the bucket form (arn:aws:s3:::<alias>), the deploy succeeds and list, get and put all return AccessDenied at run time — a failure that is hard to read.

AWS Blocks in particular has FileBucket.fromExisting(alias), which internally calls s3.Bucket.fromBucketName(alias).grantReadWrite() and therefore generates the bucket form. Trust the block's grant and add nothing, and it passes locally and fails once deployed.

Which layer refuses a malformed input

Trying an input that walks out of the prefix (../escape.txt) showed that the refusing layer differs by tool.

Input	Nx Plugin for AWS	AWS Blocks
`../escape.txt`	HTTP 403 `{"message":"Forbidden"}` (blocked by WAF)	The application's own domain error
An empty file name	HTTP 400 plus the application's error	The application's own domain error

On the Nx build, AWS WAF stopped the request before the Lambda. Defence in depth on by default is a good thing, but the refusal arrives as a generic HTTP 403, so the starting point of an investigation moves from your own code to the WAF metrics — worth knowing before you are debugging at speed.

What this round measured (deploy time and resource count)

	Deploy time	Resources	Application Lambdas
AWS Blocks (production preset)	318 s	82	1 (the whole API)
Nx Plugin for AWS (sandbox)	307 s	100	5 (one per procedure)

Amplify Gen 2 is excluded from this round's figures (deploy time, resource count) because the comparison used the portal already in production rather than a fresh build.

Worth noting: a green local e2e run on AWS Blocks says nothing about whether the access point is reachable, because the mocked storage closes the loop inside the mock.

Deploying the starters as generated (2026-09-13)

Stopping at synth leaves out everything you learn while deleting resources. So all three were deployed to a real AWS account in their generated state, before any features were added.

Output	Resources	Deploy	Exercised
AWS Blocks (sandbox preset)	83	—	signUp and friends returned 200 over JSON-RPC
AWS Blocks (production preset)	117	1,228 s	all five operations 200, CloudFront delivery included
Nx Plugin for AWS	86	317 s	TOTP enrolled, then 200 from the API via the identity pool

All three deployed without trouble. The Blocks production preset takes longer because DynamoDB builds its GSIs one at a time.

Fixed cost that accrues at zero requests

A generator's defaults reach the monthly bill even when the template says nothing about them.
The figure is settled at the moment the tooling is chosen, which makes it a number to read
before starting rather than after.

With not a single request served, the floor for each tool looks like this (FSx for ONTAP's own cost is not included).

Output	WAF	KMS CMK	Monthly floor
Nx Plugin for AWS	3 ACLs + 6 rules = $21	4 keys, $4	from $25
AWS Blocks (production preset)	none	1 key, $1	$1
Amplify Gen 2 (starter)	none	none	$0

Nx starts higher because what production needs is present from the start. It does bite when something is stood up for a test and left running.

What a production portal still needs

The feature set beyond these four, arranged in phases. Which phase your requirements stop at also changes how you choose a tool.

Phase	Feature	VPC attachment	Newly required
1	Paging, search, sorting	not needed	nothing
2	Downloads and share links	not needed	presigned URLs and an expiry policy
3	Large file uploads	not needed	multipart, or similar
4	Showing ONTAP information	needed	reaching the management LIF, splitting the functions
5	Operating ONTAP	needed	audit logs, approval for irreversible operations
6	AI processing	depends	confidence thresholds, data classification

From phase 4 the ONTAP management LIF has to be reachable, which is where the decision to put a Lambda in a VPC arrives, and with it fixed cost such as a NAT gateway. If phase 5 or later is a possibility, splitting the functions into "touches the S3 Access Point" and "touches ONTAP" from the start is the design that pays off.

Tearing down the evaluation — what a stack deletion leaves

This is about putting the evaluation environment back, not about running the portal. A portal that
gets built runs for years; the deletion happens while you are still trying things out. Even so, how
much is left behind is worth knowing before you start.

Listing what remained after each stack deletion turned up resources that static analysis of the template (synth) could not show.

Output	Left after the stack deletion
Blocks (sandbox preset)	5 log groups
Blocks (production preset)	4 DynamoDB tables (deletion protection), 8 log groups, 1 KMS key (waiting)
Nx Plugin for AWS	1 Cognito user pool, 1 DynamoDB table, 4 KMS keys, 2 IAM roles, 7 log groups

Those eight and nine are the generator's defaults, not an AWS constraint. The generated code
is what applies Retain and deletion protection, so anything that may go can be released on the
generating side (RemovalPolicy.DESTROY in CDK; the deletion-protection properties on DynamoDB
and Cognito set to false). Defaults leaning towards keeping data is a production-shaped choice;
it conflicts with intent only when the deployment is a test.

Log groups are the awkward ones. The Lambda functions behind CDK and tooling custom resources create them on first execution with the default retention (never expire), which puts them outside CloudFormation's control and leaves them behind. One sweep at the end of the teardown catches them:

aws logs describe-log-groups --log-group-name-prefix "/aws/lambda/<stack-name>" \
  --query "logGroups[].[logGroupName,retentionInDays]" --output text

The removal order matters too. Nx's user pool cannot be deleted until deletion protection is off, and the SMS configuration needed to switch it off references one of the two IAM roles listed above as survivors. Delete those IAM roles first, while clearing up what the stack left behind, and the route to switching off deletion protection is gone. Delete the user pool first, then the roles.

Your own functions are a different case from CDK's. A Lambda you write can declare its log
group in the template, so it does not survive — measured, the declared ones went away on all
three. What survives belongs to CDK's internal custom resources, and there is no route to declare
those from your own code: the CustomResourceProvider base class declares an IAM role and a
Lambda function and no log group (custom-resource-provider-base.ts,
read in full on main, 2026-09-16).

No released version fixes this (checked 2026-09-16). Two feature requests are open upstream
(aws-cdk #26553,
aws-cdk #23909). A third,
aws-cdk #24815, is closed as completed — but
its closing comment, from an AWS maintainer, says the fixing PR was reverted, that the problem
is general to all custom resources, and that tracking moved to #23909. Closed is not shipped.

AWS Support has not been asked. Lambda documents the implicit creation and the never-expire
default, the CDK source shows what the provider declares, and CDK's primary channel is its
GitHub repository rather than AWS Support. What would be worth filing is the per-output residue
count measured here, which appears in neither issue.

Two behaviours that are easy to misread in operation

Both are documented, and both are still easy to read the wrong way.

Trap 1: a deployment mode cannot be dropped later

Nx generates deploy-sandbox as cdk deploy ... --express under the hood. If a deploy breaks, "drop --express and run it again" does not work: a stack last updated in express mode requires its later operations to be express too. The remedy is to pair --express --rollback from the start.

Trap 2: DELETE_COMPLETE does not always mean gone

DELETE_COMPLETE in CloudFormation does not always mean the resource is gone. For a KMS key it means deletion was scheduled — a 30-day wait by default — and that wait cannot be shortened afterwards. Confirm a teardown from the listed state rather than from the status.

Choosing between them

The three are not mutually exclusive; they suit different purposes.

Nx Plugin for AWS: production-leaning defaults and dependency management that holds up as projects multiply. Fixed cost is the highest of the three.
AWS Blocks: running the whole application locally with no AWS account (it is in preview, so treat a Block ID as immutable after deployment).
Amplify Gen 2: treating the backend definition, the sandbox and hosting as one thing.

Part 1's conclusion stands unchanged — Nextcloud when file sharing is the centre, Amplify Gen 2 when AI processing and ONTAP operations are. What this part settled is which tool the "build" side sits on.

Test environment

Item	Value
Measured	2026-09-13 (starters, live), 2026-09-14 (live, with the four features implemented)
Region	ap-northeast-1 (only Nx's CloudFront web ACL in us-east-1)
Outputs	Nx Plugin for AWS 1.0 (`@aws/nx-plugin@1.0.0`), AWS Blocks (preview), Amplify Gen 2 (`npm create amplify`)
Local	Node.js v26.4.0, npm 11.17.0
Pricing	AWS Price List API, ap-northeast-1, 2026-09-07
Teardown	everything deleted; only 5 KMS keys remain in their waiting period (`PendingDeletion` is not billed)

The record in the repository

The numbers, the sources and the teardown procedure live in the repository, in more detail than this article.

Choosing an application foundation — where the three sit, measurements up to synth, fixed cost, reproduction steps
Deploy verification and teardown — the 15-entry pitfall register with a source on every entry, the teardown order, and the live record
The same four features on all three — twelve captures across three builds by four screens, how the permission differs, which layer refuses, 9 more pitfalls, reproduction steps
Beyond the four features — what each phase assumes, the nature of the cost, which phases can be skipped
Documentation index

Closing

Everything in this part is about defaults. Choosing a tool fixes your monthly floor and your teardown procedure, and neither is written in the template.

What worked as a method was writing the same four features three times. The difference shows up not as how much code you write but as where you write it, and which layer refuses a bad request. Choosing a tool is, I think, choosing where you look first when something breaks.

I hope this record helps anyone weighing the same choice.

Everything in the test environment has been deleted (only KMS keys remain in their waiting period). The figures come from one specific environment and configuration and will differ with workload and setup. Prices are the published ap-northeast-1 rates as of 2026-09-07.

How far ONTAP features reach through an S3 Access Point — measuring qtree, quota and FlexClone (File Portal Part 6)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Fri, 04 Sep 2026 00:59:30 +0000

Introduction

This is the sixth part of the series, and Part 7 continues it. Part 1 built a file portal on Amazon FSx for NetApp ONTAP (FSx for ONTAP) S3 access points, Part 2 added ransomware response and WORM retention, Part 3 put AI agents on it, Part 4 delegated 182 storage operations, and Part 5 collected what the running cluster refused.

This part is what was left off the screen. That is the difference from last time: where Parts 4 and 5 were about operations you run from a screen, this one has two subjects. How much of ONTAP stays usable without going through the portal at all, and the operational work that was handed to a schedule instead of a button.

What this part does not cover: performance figures (this measures whether things are possible and under what condition, not throughput), running SnapLock or Object Lock (irreversible, so not enabled without an instruction naming the retention), and actual tiering into a capacity pool (needs a different aggregate configuration, which was not on this path).

The reach of ONTAP features on this path

When you read and write files over the S3 API through an FSx for ONTAP S3 access point, how much of ONTAP stays usable alongside it? You want qtrees to carve up directories, quotas to stop a tenant running away with the space, FlexClone to spin a verification copy. Does any of that sound like your environment?

Start looking into it and NetApp's documentation gives you pause. ONTAP S3 interoperability records qtrees, quotas, FlexClone, and a volume clone of a FlexGroup containing ONTAP S3 buckets as not supported — for the ONTAP S3 server.

That is where it stalls. The FSx for ONTAP S3 access point is an AWS mechanism, and what it attaches to is a volume, not a bucket. Attaching one does stand up an ONTAP S3 server on the SVM, and I/O through the access point goes through ONTAP's S3 protocol stack. So the two are not unrelated. Does the same restriction appear?

I measured it. Up front: everything I measured worked.

ONTAP feature	Result on the S3 access point path
Qtree	Can be created. Appears as an S3 prefix, and an object PUT into it lands inside the qtree
Quota	A tree quota refuses an S3 PUT
FlexClone (volume granularity)	The clone can be made, and the clone itself can take an S3 access point
FlexClone (file granularity)	A file written over S3 can be cloned, and the clone is visible as an object through the access point
FlexGroup volume	An S3 access point can be attached. PUT / GET / LIST and multipart all work
Clone of a FlexGroup	The clone can be created

The four items that table lists did not appear as restrictions on this path. Reading that table as "not supported there, therefore not supported here" is wrong, and so is the reverse: a "supported" row is not evidence about this path either. The subjects differ, so nothing is known until it is measured.

What actually tripped me up was not in the table at all. Three things.

An NTFS volume with a UNIX identity on an SVM with no CIFS server attaches, reaches AVAILABLE, and then refuses every data operation with AccessDenied. AVAILABLE says nothing about the health of the file system layer
A volume created on the ONTAP side takes ten to twenty minutes to appear in the AWS API. The documentation says "several minutes"
A volume that has once carried an S3 access point cannot be deleted from the ONTAP side, even after every access point is detached. Only aws fsx delete-volume removes it

Below is the walkthrough, and how the controls were set up.

Test environment

Item	Value
Date	2026-08-26 (UTC)
Region	ap-northeast-1
ONTAP version	9.18.1P3D1
Deployment type / throughput	SINGLE_AZ_1 / 128 MBps
Test volumes	UNIX security style 1 GiB × 2 (one with an access point, one as the control), NTFS 1 GiB × 1, FlexGroup 400 GiB × 1
Access point identity	UNIX `root`, internet origin
Cleanup	Throwaway volumes only. All deleted

A note on the identity: UNIX root was used to keep the measurement simple. Every request through an access point is authorized as that one file system identity, so specifying root removes any narrowing by file permissions. This is not a recommended configuration. Use a dedicated user with only the permissions the writes need, and split access points per purpose (FileSystemIdentity cannot be changed after creation).

The distinction that matters most — different subjects

The results below have to be read with these two kept apart.

Mechanism	What it is	Whose
ONTAP S3	The S3 server with which ONTAP serves buckets. This is what NetApp's table is about	NetApp
FSx for ONTAP S3 access point	An AWS access point attached to an FSx for ONTAP volume	AWS

They are not unrelated. Attaching one stands up an ONTAP S3 server on the SVM, and the I/O goes through ONTAP's S3 protocol stack (measured). That is exactly why NetApp's table is the right place to start.

Being the right place to start is not the same as being the conclusion. The target is a volume rather than a bucket, and the namespace and the authorization path both differ. "Not supported there, therefore not supported here" turns something unchecked into something checked. The reverse reading is the same error.

The walkthrough

Qtree — usable

Step	Result
Create a qtree in the volume with an access point	Succeeded
Control: the same qtree in the volume with no access point	Succeeded. No difference
`list-objects-v2 --delimiter /`	The qtree appears in CommonPrefixes
PUT into the qtree prefix	Succeeded
Inspect the qtree from ONTAP	The written file is inside the qtree directory

NetApp's "Qtrees: not supported" does not manifest as a refusal on this path. The qtree is created, it is visible as a prefix in the S3 namespace, and an object written into that prefix lands inside it. Together with the quota result below, it behaves as a qtree.

Incidentally, a volume with an access point attached carries an internal ____NTAP_S3_MAPPING directory at its root. It is visible over NFS and SMB, so anyone who inspects a collect-layer volume by hand will see it.

Quota — enforced against an S3 PUT

Step	Result
Set a tree quota on the qtree (space 1 MiB, files 10) and enable quotas on the volume	Succeeded
`quota report`	The rule is active, and the file written over S3 is counted in files used
PUT 15 small objects	8 succeeded, 7 refused. files used stopped at exactly 10/10
What the S3 client received on refusal	HTTP 507 `InsufficientCapacity` / `Maximum storage capacity of file system has been reached.`
Control: raise the files limit 10 → 50 and re-PUT the refused key with the same body	Succeeded

The control carries the conclusion. Raising the limit alone made the identical PUT succeed, so the refusal was the quota, not a capacity shortage and not a permissions artefact.

That the response mis-describes its cause matters to the design. The file system was not full; a qtree file-count quota was reached. An operator who sees a 507 and "maximum storage capacity of file system" will consider growing the file system. If you use quotas on this path, write down what that response actually means in the runbook.

FlexClone — both volume and file granularity

There are two granularities, and they behave differently.

	File granularity	Volume granularity
Where it lands	The same volume. It appears as a new key on the same access point	A separate volume
Authorization boundary	Same as the parent	Its own access point, its own policy, its own identity
Time before it is usable	None	599 to 1,177 s measured, for a volume created through the ONTAP API to appear on the AWS side
Visible over S3	Yes. `StorageClass=FSX_ONTAP`, sha256 matches the source	Yes. LIST / GET / PUT through the clone's own access point

At either granularity the clone shares data blocks with its parent. Cloning a 256 MiB file four times:

Point	Logical	Physical
Just after one 256 MiB PUT	—	253,534,208 B
After four file-granularity clones	1,350,942,720 B	277,200,896 B

Logical grew to five copies, physical by about 23 MB. FSx for ONTAP bills provisioned SSD capacity, so there is no capacity to provision per duplicate.

NetApp states that copies consume no storage except what is required for metadata until changes are written to the copy (FlexClone volumes, files, and LUNs). The operation that ends the sharing is a split (volume clone split start), which I did not measure. From ONTAP 9.4, on AFF systems where the volume guarantee is none, the documentation states that the split shares the physical blocks rather than copying data (splitting procedure).

A file-granularity clone gives no observable failure, however. POST /api/storage/file/clone returns 202 and a job UUID, but the UUID resolved to 404 entry doesn't exist and appeared in no job listing. The same fsxadmin retrieves volume-create and volume-clone jobs as state=success, so this is not a permissions problem. A call naming a destination directory that does not exist also returned 202 and created nothing. Judge the outcome by inspecting the destination file.

FlexGroup — access point attachment

Creation has conditions, learned by failing in order.

Step	Result
Create with default parameters	Failed. `Volumes of this type must be at least 50GB`
Retry at 50 GiB	Failed. `Aggregates not matching FabricPool requirements: aggr1`
Retry with the aggregate named explicitly	Failed. `Minimum size is "400GB"`
400 GiB, explicit aggregate, `tiering.policy=none`, thin	Succeeded

FlexGroup volumes can be created from the AWS Management Console, the AWS CLI and the Amazon FSx API (Managing volumes). Naming the aggregate explicitly was required because this one was created through the ONTAP REST API; that constraint does not appear on the AWS-side path.

What follows is the point: whether an access point on the AWS side attaches to a FlexGroup created on the ONTAP side.

Step	Result
Appears on the FSx for ONTAP side as `VolumeStyle=FLEXGROUP`	It does, with an `fsvol-` identifier
Attach an S3 access point to the FlexGroup	Succeeded. `Lifecycle=AVAILABLE`
PUT / GET / LIST	Succeeded. The GET content matched
12 MiB multipart upload	Succeeded. `StorageClass=FSX_ONTAP`
Snapshot the FlexGroup	Succeeded
Clone that FlexGroup	Succeeded. `style=flexgroup`, `is_flexclone=true`, online

That last row is the item NetApp's table records as "volume clone of the FlexGroup volume containing ONTAP S3 buckets: not supported".

What actually failed on this path

The constraints were not on NetApp's side of the table. Within what I measured, these three are the real traps.

A security-style and identity mismatch — failure after a successful attachment

Step	Result
Attach with an NTFS-security-style volume + UNIX identity (`root`)	Succeeded. `Lifecycle=AVAILABLE`
PUT through that access point	Refused. `AccessDenied`, body only `Access Denied`
Control: same identity, same caller, UNIX volume	PUT / GET / LIST all succeeded
CIFS server on this SVM	None. The UNIX-to-Windows mapping cannot resolve

AVAILABLE does not mean the file system layer is healthy. Attachable and usable are different properties: IAM and the access point policy are both passed, and the file system layer refuses after that. The body is only Access Denied and does not name the layer. A refusal at the identity layer says no identity-based policy allows ... instead, so the difference in the body is the signal.

Trying WINDOWS identity on the same SVM does not even complete the attachment.

Step	Result
Attach with WINDOWS identity on an SVM with no CIFS server	Failed. `did not stabilize` (`NotStabilized`), stack rolled back
Whether an attachment survives the rollback	It does not. Nothing was orphaned

The wait for something created in ONTAP to appear on the AWS side

A volume created through the ONTAP API is not immediately in the AWS-side describe-volumes. Without an fsvol- identifier, neither AWS::FSx::S3AccessPointAttachment nor create-and-attach-s3-access-point can reference it.

Measurement	Result
FlexGroup, polled every 20 s with no gaps	Appeared at 599 s (about 10 min), with an `fsvol-` identifier
A FlexClone volume, polled every 20 s with no gaps	Appeared at 1,177 s (about 19.6 min)
A separate run	Still absent at 1,258 s (about 21 min)

The three observations disagree, so this is not an upper bound. The order of magnitude is tens of minutes, which is not enough to bake a fixed wait into a design. AWS documents the following, and the measurement is longer than its "several minutes".

Amazon FSx periodically syncs with ONTAP to ensure consistency. If you create or modify volumes using NetApp applications, it may take up to several minutes for these changes to be reflected in the AWS Management Console, AWS CLI, API and SDKs.

— Managing FSx for ONTAP resources using NetApp applications

This delay has to be kept apart from absence. In an observation of about 2.5 minutes, a volume created through the ONTAP API shows no fsvol- identifier, and it would be easy to conclude from that that neither a clone nor a FlexGroup can take an access point. Wait, and it appears, and it attaches. 2.5 minutes sits inside the "a few minutes" AWS documents, so what is observable in that window is "not yet appeared".

It lags in the other direction too. Setting a junction path on the ONTAP side and attaching immediately fails with Amazon FSx is unable to attach S3access point because the volume is not mounted. The AWS-side JunctionPath was None at that moment and still was two minutes later. Setting it with aws fsx update-volume made it visible on the AWS side in about 40 seconds, and the attachment then succeeded. The error message was telling the truth. What was wrong was my assumption that setting it in ONTAP made it visible to AWS.

A residual NAS bucket refusing volume deletion on the ONTAP side

First, the terms. An S3 access point is an AWS-side mechanism and an object store NAS bucket is an ONTAP-side object, and the two are different things. Attaching an access point creates a corresponding NAS bucket on the ONTAP side.

What happened here is residue of the latter. After every access point was detached, the NAS bucket remained on the ONTAP side, and ONTAP refused to delete the volume because of it.

Cannot delete volume "..." in SVM "..." because it is associated with the following
object store NAS buckets: "amazon-fsx-<volume-id>"

What was checked	Result
Where the bucket name comes from	An exact match for the volume ID. It is created per volume, not per access point
After deleting every access point	Still refused, and still refused hours later
Bringing the volume back online and mounted	Still refused
Re-attaching an access point and detaching it in the correct order	Still refused. It is independent of the access point lifecycle
`aws fsx delete-volume`	Succeeded. Both the volume and the bucket went

NetApp's documented procedure for removing a NAS bucket configuration uses vserver object-store-server bucket delete, which could not be applied here because the target could not be listed.

Whether this residue is by design or a defect is unconfirmed. No AWS documentation I could find describes a NAS bucket outliving its access points and blocking a volume delete; the documented removal path is detach-and-delete-s3-access-point. If you hit the same state, delete-volume on the AWS side will complete the teardown, but ask support to establish the cause.

Not being listed is not the same as not existing. Laid out by reader:

Reader	How it shows this bucket / S3 server
ONTAP REST `/protocols/s3/buckets`	Does not list it
ONTAP CLI `vserver object-store-server bucket show`	Does not list it
ONTAP REST `/svm/svms` (the `s3` field)	Reports the S3 server as `enabled`
ONTAP REST `/protocols/s3/services`	Does not list the same SVM

Objects that AWS manages are hidden from the standard ONTAP S3 views. I confirmed the bucket's absence across two readers, and both shared the same blind spot.

If you have made clones, the teardown order needs one more step of care. delete-volume returned DELETING and then silently went back to CREATED (observed twice). The reason appears only in LifecycleTransitionReason on describe-volumes: Failed to delete volume because it has one or more clones. The clones had been deleted, and were sitting in ONTAP's volume recovery queue. The parent kept clone.has_flexclone at true, and the AWS-side delete reads that flag and refuses. Purging the recovery queue cleared the flag and the same delete-volume worked. That queue appears in neither the console nor the FSx for ONTAP API.

Putting it in a runbook

Translated into operational terms, the measurements come to five points.

#	What to write in the runbook
1	Match the security style to the identity. UNIX identity for UNIX volumes, WINDOWS identity for NTFS ones. Avoid a UNIX identity on NTFS even where the SVM does have a CIFS server
2	Do not use `AVAILABLE` as a connectivity check. Make the acceptance criterion a PUT and a read-back after attaching
3	Budget tens of minutes into automation that creates a resource in ONTAP and then handles it through the AWS API. Poll for the `fsvol-` identifier rather than sleeping a fixed interval
4	Write through the AWS management plane. Where a setting can be made from either side, as with the junction path, the AWS side reflects faster (about 40 seconds against more than two minutes)
5	Tear down through the AWS-side API. A volume that has carried an access point can keep a NAS bucket on the ONTAP side, so a procedure built on ONTAP's `volume delete` gets stuck there (`aws fsx delete-volume` completes)

What the schedule took over

Everything so far was about whether an operation is possible. Operations has another kind of work: checking capacity, taking stock of snapshots, revisiting tiering policy, confirming throughput is still right-sized. All of it is daily or weekly, and a button for it does not get pressed. That work suits a schedule better than a screen, so six patterns live in operations/.

ID	Pattern	What it does
OPS1	capacity-rightsizing	Capacity and throughput monitoring, tier recommendation, what-if cost
OPS2	storage-efficiency	Tracks deduplication and compression efficiency, recommends enabling them
OPS3	tiering-optimizer	Cold-data analysis, tiering policy recommendation and savings estimate
OPS4	snapshot-lifecycle	Retention policy compliance check and drift detection
OPS5	cost-optimization	Cost breakdown, unit economics ($/GB), growth projection
OPS6	qos-monitoring	QoS policy adherence, bandwidth contention risk

The shape is the same in all six. An EventBridge schedule starts a Step Functions workflow, a VPC Lambda collects from the ONTAP REST API and CloudWatch, another analyses, and a third writes the report. Unit tests total 74 across the six (capacity-rightsizing 27, snapshot-lifecycle 19, tiering-optimizer 11, storage-efficiency 6, cost-optimization 6, qos-monitoring 5).

The report has two destinations. With OutputDestination=FSXN_S3AP it is written to an FSx for ONTAP volume through an S3 access point, which means anyone with the NFS or SMB mount reads it in the file explorer they already use. On the real cluster the same 4,240-byte JSON appeared on the ONTAP volume as in the S3 bucket, the Step Functions execution took around 15 seconds, and four metrics were published to the FSxOps namespace.

A recommendation comes out like this.

{
  "recommendation_type": "downsize",
  "target": "vol_backup_temp",
  "current_value": "10.0% (1024 GB allocated, 102 GB used)",
  "recommended_value": "Shrink to 205 GB or enable autosize (grow_shrink)",
  "monthly_cost_delta_usd": -102.4
}

DemoMode=true runs the whole workflow on mock data with no file system, so the report layout and the shape of the recommendations can be reviewed first. Running cost itself — Lambda, Step Functions and S3 storage — came to roughly $1-5 a month for a daily run over a handful of file systems.

Progressive automation — four levels by design, two in the code

Honestly, here. The template's AutomationLevel accepts 0 through 3.

Level	Intended behaviour	Implemented
0	Report only	Yes (default)
1	Notify over SNS	Yes
2	Execute after human review	No
3	Auto-execute within guardrails	No

The code branches in exactly one place, automation_level >= 1, deciding whether to send SNS. So 2 or 3 deploys successfully and behaves identically to Level 1. The values the parameter accepts and the values the code distinguishes do not match.

The values listed in AllowedValues are not a declaration of what is implemented — accepting a value and honouring it are different things. It is the same point Part 4 made about not shipping buttons that cannot be pressed, showing up on the parameter side instead. Until the approval flow exists, narrowing AllowedValues to 0 and 1 would be the accurate move.

None of this replaces existing tooling. The vendor's CloudWatch dashboards and auto-resize solutions, and the SSD capacity auto-expansion AWS documents, differ in immediacy and in scope. The six here suit a flow where analysis and a recommendation come first and a person decides. Choose on the use case.

On how recommendations are treated: OPS4's snapshot deletion recommendations exclude anything below the minimum retention. A recommendation is an assistive signal, not a final decision. Which retention period satisfies your regulatory requirements is a legal and compliance judgement, and neither this article nor the pattern stands in for it.

What I did not measure

"Unverified" is not "cannot be done". Every result above turned out that way, so on this path nothing is known until it is measured.

Item	State
FlexClone at LUN granularity	Unverified. Creating a LUN needs an iSCSI configuration, which is not on this path
The FlexClone split	Unverified. Documentation only
An upper bound on the ONTAP-to-AWS reflection time	Unverified. The three observations disagree
FabricPool tiering	Unverified. Needs a different aggregate configuration
QoS / deduplication and compression / SnapMirror / Vscan	Unverified
SnapLock / Object Lock	Unverified. Irreversible, so not enabled without an instruction naming the retention value
Attaching on a FlexCache cache volume	Unverified. Requires a cluster peer and an SVM peer
OPS Levels 2 and 3	Not implemented. The approval flow and auto-execution are not in the code
Whether the OPS recommendations are sound	Not evaluated. They are produced; none was applied and measured
OPS across many file systems	Unmeasured. Only a handful were run

Conclusion

Two things were left off the screen, and both were written up here.

On the ONTAP side I took NetApp's table as the starting point and measured on the S3 access point path. The four items it lists as unsupported did not appear as restrictions here. Qtrees work, a quota refuses an S3 PUT, FlexClone works at both volume and file granularity, a FlexGroup takes an access point, and a clone of that FlexGroup can be created. The constraints were elsewhere: AVAILABLE says nothing about the file system layer, something created in ONTAP takes tens of minutes to appear on the AWS side, and a volume that has carried an access point can only be deleted from the AWS side.

On the operations side, the daily work went to a schedule rather than a screen. There is one mismatch between the declaration and the code here: AutomationLevel accepts 0 through 3 while the code distinguishes only up to 1.

Every failure in these six parts looked like a correct conclusion on its own. A conclusion is not settled until one more control, or a second reader, is added.

Thank you for reading these six parts. It started with showing files in a browser and ended outside the screen entirely. Part 7 picks up the layer underneath the portal: which tooling to build it on, and what a stack deletion leaves behind. I hope the series is useful to anyone operating FSx for ONTAP, or building this kind of admin screen.

All test resources were deleted. The figures come from one specific environment and configuration and will differ with workload and setup. This article does not stand in for governance or regulatory judgement.

FlexGroup creation and capacity rebalancing failed on the defaults — what the error codes said (File Portal Part 5)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Fri, 04 Sep 2026 00:59:05 +0000

Introduction

Part 4 described putting 182 ONTAP operations on a file portal: the design of delegation and record, and the treatment of buttons that cannot be pressed and buttons that were never built.

This part is what the running cluster refused while those 182 were being built. None of it was visible while reading the documentation; each one arrived as an error code. That is the difference from last time: Part 4 was about design, this one is about measurement.

The readers I have in mind are people writing code against the FSx for ONTAP ONTAP REST API. You do not need to be building a portal — if you call the same API, you stop in the same places.

What this part covers: FlexGroup creation, FlexGroup capacity rebalance, how to read the capacity numbers, deletes that refuse, and four pitfalls in the S3-compatible API. Each with the response the system actually returned.

What this part does not cover:

ONTAP internals
Absolute throughput figures (the subject here is whether something is possible and under what condition — but the defaults that decide whether a measured figure can be trusted are covered in a section below)
Running SnapLock compliance (irreversible, and it would make a test file system undeletable for months, so it was deliberately not done)

Up front:

Creating a FlexGroup through the ONTAP REST API requires naming the target aggregate. FSx for ONTAP aggregates have tiering enabled, so ONTAP's automatic placement skips them. Creating one from the AWS console or the Amazon FSx API does not need this step
The same root cause is documented on the FlexCache side as a flag (use_tiered_aggregate), and it had not reached the volume-creation side
The capacity rebalance max_runtime has a lower and an upper bound that are not in the API reference. With ONTAP's defaults it never starts
Starting a rebalance enables an irreversible setting. Stopping it does not undo it
space.used means a different quantity depending on the volume. Snapshots inside the reserve are not counted
A deleted FlexClone child volume stays invisible to the API while still blocking the parent volume's delete. There are two ways out, and both were measured
Two places the documentation and the running system disagreed, and both changed the screen's wording
Eight combinations of defaults return a plausible figure while measuring something other than what you intended. Three of them come from reads being served out of cache, and two from when you measured

Test environment

Item	Value
Dates	mid-August 2026 to 2026-09-02 (JST)
Region	ap-northeast-1
ONTAP version	9.18.1P3D1
Deployment type	SINGLE_AZ_1
Caller	ONTAP REST API from a VPC Lambda (Python 3.13)
Privilege	`fsxadmin`
Cleanup	All test volumes deleted

The distinction that matters most — ONTAP's words and AWS's words

Mix these up and searching for the error text will not find the answer.

Term	Where it appears
FabricPool, `use_tiered_aggregate`	ONTAP error text and REST field names
tiering, capacity pool tier	AWS documentation and console

They refer to the same mechanism. But the error arrives in ONTAP's vocabulary while the documentation explains it in AWS's, so searching the AWS documentation for the error string returns nothing. The portal's wording follows the AWS side; only API strings are quoted as-is.

What to watch when creating a FlexGroup

A FlexGroup presents several "constituents" — internally split real volumes — as one namespace.

To set the scope: FlexGroup volumes can be created from the AWS console, the AWS CLI and the Amazon FSx API (Managing volumes). What follows is the ONTAP REST API path, which is the one the portal uses.

ONTAP normally picks the aggregates if you do not name them. On FSx for ONTAP that automatic selection does not resolve.

Aggregates not matching FabricPool requirements: aggr1

An FSx for ONTAP file system has a primary storage tier (SSD) and a capacity pool storage tier, and infrequently accessed data is tiered to the latter (AWS documentation). Aggregates in that configuration fall outside ONTAP's automatic placement.

The conditions came out one failure at a time.

Step	Result
Create with default parameters	Failed. `Volumes of this type must be at least 50GB`
Retry at 50 GiB	Failed. `Aggregates not matching FabricPool requirements: aggr1`
Retry naming the aggregate	Failed. `Minimum size is "400GB"`
400 GiB, aggregate named, `tiering.policy=none`, thin	Succeeded

Naming the aggregate works. But that aggregate name appears in neither the AWS console nor the Amazon FSx API. It has to be fetched with GET /storage/aggregates. Nobody hands the tenant an aggregate name, so that extra call is mandatory.

The 400 GB floor is configuration-dependent: the documented minimum is 100 GB per constituent, so the floor moves with the constituent count (the default is 8 per HA pair). And this extra step is only needed on the ONTAP REST API path — if you do not need to choose the aggregate yourself, creating the volume from the Amazon FSx API is shorter.

The same root cause was already known on the FlexCache side. There is a flag, use_tiered_aggregate, defaulting to false. It had not reached the volume-creation side. The same cause surfacing elsewhere with a different face is a familiar shape in a managed service that wraps upstream software.

The screen carries this note:

When creating a FlexGroup from this screen, the aggregate is named explicitly rather than placed automatically. FSx for ONTAP aggregates have tiering enabled and are excluded from automatic placement (measured: without it, creation fails with "Aggregates not matching FabricPool requirements"). With the default four constituents the minimum is 400 GB.

The capacity rebalance that never starts with the defaults

A FlexGroup places files across constituents by hash, so they drift apart over time. When one fills, the whole volume returns "no space" even though the others have room. A capacity rebalance is what corrects that.

Running it against the real cluster produced two constraints on max_runtime that are not in the API reference.

Constraint	Detail	Error code
Lower bound	Under 30 minutes is refused	`144182221`
Upper bound	Must be shorter than the time remaining until the next snapshot	`13107433`

Together, the window to start is:

30 min <= max_runtime < (time from start until the next snapshot)

ONTAP's default max_runtime is 6 hours. On a volume with the default snapshot policy (hourly at :05), 6 hours always violates the upper bound. With the value the portal offered as its default, it never started once. With the default policy the only window is the 30 minutes from :05 to :35 each hour. On a frequent schedule such as 5min, nothing starts until the policy is detached.

The boundary was pinned with a one-second A/B.

Time	`maxRuntime`	Until next snapshot	Result
16:06:12	`PT1H` (60 min)	58m47s	Refused (`13107433`)
16:06:13	`PT30M` (30 min)	58m46s	Succeeded

60 minutes > 58m47s refused; 30 minutes < 58m46s succeeded. Because the two calls are one second apart, the deciding factor is the max_runtime value and not elapsed time. ONTAP's own message names both remedies — "reduce the -max-runtime or disable the snapshot policy" — because of this structure.

The full observation log is in the FlexGroup capacity rebalance record. State values absent from the reference's volume-side list turned up there too: idle means running with nothing to move, scheduled means a reservation exists. The portal had been displaying "running" as "unknown".

A note on irreversibility

Starting a rebalance enables granular data on that volume. It is required for the rebalance, and it creates two multi-part inodes per moved file (ONTAP documentation). Stopping does not undo it. The vendor's documentation lists no way back other than deleting the volume or restoring from a snapshot taken before it was enabled.

Stopping is not a rollback either: files already moved stay where they are. Both were measured (granular_data remains true after a stop). The screen's confirmation says both.

The space that does not come back after a delete

Read the usage figure in a volume list naively and you will get it wrong, because space.used means a different quantity depending on the volume.

Snapshots that fit inside the snapshot reserve (5% by default) are not counted in space.used
Anything past the reserve consumes the same space as live data and is counted

The measurements. A 100 GiB volume reported used of 18.1 MiB while holding 77.3 MiB of snapshots. That sits inside the 5% reserve (5 GiB), so the usage bar does not move. On a different volume with a 0% reserve, used was 83,677 MiB: 81,934 MiB of live data and 1,743 MiB of snapshots. On 8 of 11 volumes, snapshots exceeded live data.

So the list was split from one usage figure into three: live data, snapshots, and reserve spill. Whether the space is not coming back because of snapshots or because live data fills it can now be read off the screen.

Sometimes the cause is on another volume and its own row does not show it. That is FlexClone. A clone is based on one of the parent's snapshots, and that snapshot stays locked while the clone exists. Deleting files on the parent frees nothing while the locked snapshot still references the blocks. Reading the parent's row alone cannot tell you that, so it is written on the FlexClone panel instead.

What to check when a FlexClone volume will not delete

After deleting a child volume created with FlexClone, deleting the parent volume is refused with has one or more clones. I reproduced that on real hardware. The awkward part is that the API cannot see that child volume. It is not in the volume listing, fetching it by name returns entry doesn't exist, and yet only the parent's delete refuses, saying a clone exists.

The cause is ONTAP's volume recovery queue, which retains deleted volumes for 12 hours by default. An A/B in the same environment, varying only whether the clone was split first:

Steps	Parent delete
Delete an unsplit clone, then delete the parent	Failed (still failing after 7 and 15 minutes)
Split the clone, delete it, then delete the parent	Succeeded within seconds

The exit was measured too. GET /api/private/cli/volume/recovery-queue reads the queue. POST .../purge goes through with fsxadmin privilege, the entry left the queue in about 20 seconds, and the parent delete succeeded immediately afterwards.

This is a place where I published something wrong and have corrected it. The earlier text said purge needs diag privilege, is unreachable as fsxadmin, and therefore the only option is to wait 12 hours. That was wrong. I blamed privilege without trying it.

The screen's error carries the causation and both exits (split first, or purge). Purge cannot be undone, so it also says to confirm the volume was yours before running it.

Four pitfalls in the S3-compatible API

Behaviours worth knowing before you call an FSx for ONTAP S3 access point from a browser or an SDK. Knowing them changes how long the investigation takes.

The 501 failure of a "write only if absent" request

PutObject with if-none-match: * returns 501 NotImplemented, with the body A header you provided implies functionality that is not implemented.

Request	Result
`PUT` + `if-none-match: *`	501 NotImplemented
`PUT` + `x-amz-checksum-crc32` (no `if-none-match`)	200 OK
`GET` / `ListObjectsV2`	200 (neither header is sent)

CRC32 flexible checksums pass. Only if-none-match fails. Reads are unaffected, so the symptom presents as "the listing works but every write fails."

This is not somebody else's problem. Amplify's Storage Browser translates its "do not overwrite" option into that header, so dropping it in unchanged breaks every upload and folder creation. If you need overwrite protection, look the key up before writing instead. That does not give you if-none-match's atomicity, and two concurrent writes to the same key can both decide it is absent.

GOVERNANCE and COMPLIANCE, indistinguishable from the error text

S3 Object Lock has two retention modes. GOVERNANCE lets a user holding a specific permission (s3:BypassGovernanceRetention) delete inside the retention period; COMPLIANCE lets nobody delete until the retention expires. Deleting an object written with one day of retention, compared across both modes:

Operation	GOVERNANCE	COMPLIANCE
Delete without bypass	`AccessDenied ... object protected by object lock`	Identical text
Delete with `BypassGovernanceRetention`	Succeeds	Refused with identical text

Because the text is identical, the response does not tell you why the bypass did not work. The first things to read are get-object-lock-configuration and head-object's ObjectLockMode rather than the IAM policy, because the same text covers both a missing permission and a mode that has no bypass.

The lock indicator that does not change under automatic locking

Assigning a policy whose schedules carry a retentionPeriod leaves the locking feature flag at false; only the assigned policy name changes. Reading the state alone suggests nothing will be locked, while every snapshot that policy takes is in fact locked. The judgement needs the assigned policy's retentionPeriod as well.

The usable endpoints, derived from the API rather than a config file

An alias copied into a config file looks equally correct for an access point that has been deleted and for one that is MISCONFIGURED. fsx describe-s3-access-point-attachments returns Lifecycle and the Internet or VPC origin, so derive the inventory from there. Always follow the pagination: read only the first page and an access point that exists goes missing in a way indistinguishable from one that does not.

Two places the documentation and the running system disagreed

Two things behaved differently from what I read before implementing them. Both changed the screen's wording.

Subject	Expected from the documentation	Measured (ONTAP 9.18.1P3D1)
Deleting a QoS policy while assigned	The CLI reference says it is refused without `-force`	REST accepts it, and the volume's assignment is silently removed (every ceiling becomes 0, meaning unlimited)
Enforcement after deleting a quota rule	The REST reference says it stays in effect until enforcement is cycled off and on	The deleted rule's ceiling disappeared from the usage report immediately

The QoS confirmation text was rewritten. The correct warning is not "it cannot be deleted" but "deleting it removes the ceilings from every volume using that policy". A note was added that releasing a single volume means assigning none, not deleting the policy.

The eight defaults that can invalidate a measurement

Up to here the defaults decided whether an operation goes through. Measurement has defaults of the same shape. None of them returns an error: each returns a plausible figure while measuring something other than what you intended.

Where these figures come from: a different measurement from the environment above. 2026-09-01 to 02, ap-northeast-1, ONTAP 9.18.1P3D1, SINGLE_AZ_1, throughput tiers 128 MBps and 2048 MBps, client c5n.9xlarge. Repeatability across repeat runs of the same measurement was 0.02–0.14%.

The sixth is from a different measurement. 2026-09-06, ap-northeast-1, ONTAP 9.18.1P3D1,
SINGLE_AZ_2, throughput 6,144 MBps, an SMB SVM joined to AWS Managed AD, clients c5n.2xlarge and
c5n.9xlarge. A different generation and a different tier from the five above, so do not put the
figures side by side.

The sixth does not record the NVMe read cache state. The second generation at 6,144 MBps has
1,900 GB of NVMe read cache and it ships enabled (confirmed on a live file system on
2026-09-10). The difference Multichannel makes is a channel count and is unaffected by this gap,
but do not cite 942 and 1,824 as disk-path figures.

The seventh and eighth are from a third measurement. 2026-09-10, ap-northeast-1,
ONTAP 9.18.1P6, SINGLE_AZ_2, 1,536 MBps and 6,144 MBps, 8,192 GiB of SSD, client c5n.9xlarge.
The NVMe read cache state is part of the recorded conditions. It ships enabled at 6,144, so it
was disabled on both nodes and read back before measuring.

Default	What happens	How to notice	What to do
Linux NFS uses one TCP connection	Stops around 590 MB/s and does not respond to stream count	1 / 4 / 8 give the same figure	`nconnect=16`
`dd if=/dev/zero`	Zero blocks skip disk and return at 4x the published ceiling	Exceeds the published ceiling	Use incompressible data
Volume inline efficiency is on	Compressible or identical payloads collapse (4x on reads, under 5% on writes)	`space_savings.dedupe_percent` is high	Turn it off before writing
`DiskIopsConfiguration: AUTOMATIC`	3 IOPS/GiB, so IOPS becomes the limit	MB/s divided by IOPS is implausibly small	`USER_PROVISIONED`
A read only slightly over the cache size	The cache-served share dominates and the disk path is not measured	`DiskReadBytes` divided by `DataReadBytes` is a few per cent or less	Read at least twice the cache in one pass
SMB Multichannel ships disabled	One channel, and it holds while the session reports `dialect=3.1.1` (1 MiB sequential read 942 against 1,824 MB/s)	One established TCP connection to the SVM; `Get-SmbMultichannelConnection` returns nothing	`vserver cifs options modify -is-multichannel-enabled true`. It does not reach existing connections — restart the client's SMB service and remap
No window length decided	Measures while the burst allowance is still there. 2.0x on the same environment and the same shape (2,882 MB/s for 27 minutes, then 1,439 MB/s)	`FileServerDiskThroughputUtilization` is above 100%. It is a ratio against baseline, so it can be (median 203.5% measured)	Run until the allowance is gone. Read `FileServerDiskThroughputBalance` over the same window and record the balance
Reading straight after a fill	Competes with the fill's write-back and runs about 20% low for 90 seconds	Disagrees with the same shape measured a second time (39% measured)	Leave two minutes after the write finishes. Put the wait in the parameter file, not in a note in the runbook

How the fifth one surfaced. I assumed 280 GiB would not fit in cache. Read through Amazon CloudWatch's AWS/FSx namespace, DiskReadBytes divided by DataReadBytes came to 1.4 / 0.9 / 1.5%. 98.5-99.9% of the bytes read never touched disk.

The 2048 MBps tier's in-memory cache is 256 GB, that is 238 GiB (FSx for ONTAP performance; ap-northeast-1 falls under the first-generation Single-AZ "all other Regions" table). 280 GiB exceeds it by 18%, not by a factor of two or three. "Past the cache" and "not served from the cache" are different things.

And clearing the cache by a wide margin does not help if the window is short. In the seventh
measurement the working set was 30 times the in-memory cache, and the figure still came back 2.0x
high for the 27 minutes the burst allowance lasted. "Read at least twice the cache" answers the
fifth, not the seventh.

Inline efficiency can be restored, provided you wait.

# Rejected unless efficiency.op_state is idle
PATCH /api/storage/volumes/{uuid}  {"efficiency": {"compression": "inline"}}
PATCH /api/storage/volumes/{uuid}  {"efficiency": {"dedupe": "both"}}

But only turn it off on a volume you can discard. Immediately after writing, a background efficiency operation is running and the wait is not predictable.

Putting it in a runbook

#	What to write down
1	To create a FlexGroup through the ONTAP REST API, fetch the aggregate with `GET /storage/aggregates` and name it. If you do not need to choose the aggregate, create the volume from the Amazon FSx API instead
2	Set the rebalance `max_runtime` to at least 30 minutes and below the time to the next snapshot. Do not pass the default through
3	Do not treat starting a rebalance as an operation you can press at any time. `granular data` is enabled irreversibly
4	Do not show usage as a single number. Separate live data from snapshots, or nobody can reach the cause of "deleting files did not free space"
5	A teardown procedure for an environment with clones must include the split, or the recovery queue purge. "The clone was deleted, so the parent can go" does not hold
6	Errors arrive in ONTAP's vocabulary. Keep the AWS-to-ONTAP term mapping at hand before searching
7	A runbook for measuring performance states how to drop the defaults first. Leave any one of zero-filled data, inline efficiency, an unspecified `nconnect` or `AUTOMATIC` IOPS in place, and the figure produced is measuring something else

What I did not measure

"Unverified" is not "impossible". As last time, the boundary stated as a boundary:

Item	State
The actual effect of a rebalance (how far the skew is evened out)	Unmeasured. Only the conditions to start were measured
Performance impact after `granular data` is enabled	Unmeasured
A recovery queue with a changed retention	Unverified. Measured at the 12-hour default
Actual tiering into the capacity pool	Unverified. Needs a different aggregate configuration
Running SnapLock compliance	Deliberately not done. Irreversible, up to the file system
An alternative path for copies over 5 GiB (NFS or SMB)	Unmeasured. The portal stops at refusing

Part 6 covers the other side: what was left off the screen. How much of ONTAP stays usable alongside the S3 access point path, and the operational work that was handed to a schedule instead of a button.

Conclusion

Nothing in this part was visible while reading the documentation. Creating a FlexGroup through the ONTAP REST API needs the aggregate named, a capacity rebalance never starts with the defaults, space.used means a different quantity per volume, and a deleted FlexClone volume blocks the parent's delete while invisible through the API.

What they share is that the error text does not name the cause. That is why the causation and the exits live on the screen. As with the one-second A/B on the rebalance, another explanation looked just as correct until one control was added.

There is also one correction. I published "purge needs diag privilege, so wait 12 hours" and later found it goes through. I blamed privilege without trying it.

The figures come from one specific environment and configuration and will differ with workload and setup. The error codes were observed on ONTAP 9.18.1P3D1.

Putting FSx for ONTAP operations in a browser — the buttons I disabled, and the ones I never built (File Portal Part 4)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Fri, 04 Sep 2026 00:57:42 +0000

Introduction

Part 1 built the foundation of a file portal on top of Amazon FSx for NetApp ONTAP (FSx for ONTAP) S3 access points. Part 2 added ransomware response and WORM retention. Part 3 put AI agents on it.

Those parts were screens for people who handle files. From here the subject is the people who operate the storage itself. Volume create and resize, FlexCache, FlexClone, SnapMirror, snapshot policies, QoS, quotas, SMB shares. The portal now reaches 182 operations.

The readers I have in mind are infrastructure owners running FSx for ONTAP, and developers building this kind of admin screen with Amplify Gen2. ONTAP terms get a one-line gloss on first use, so ONTAP operations experience is not assumed. You can start here without Parts 1-3.

What this part covers is what had to be decided to put operations on a screen: the design of delegation and record, and the treatment of buttons that cannot be pressed and buttons that were never built.

What this part does not cover:

The ONTAP features themselves (ONTAP's own documentation is more accurate)
The failures the real cluster produced (that is an article's worth on its own, so it became Part 5)
Which retention period satisfies your regulatory requirements (a legal and compliance judgement that neither I nor the portal can stand in for)
Performance and scale limits of the screen (unmeasured; no concurrent-user test was run)

Up front:

Three management interfaces reach the file system, and two of them are for administrators inside the VPC. What the portal adds is not a path but delegation and record
The portal does not remove the path into the VPC. The path did not disappear; what holds it moved from a person to a Lambda
The number 182 is not maintained by hand. A script counts it and CI fails when it disagrees
A button that cannot be pressed says what is missing. A button that does nothing is indistinguishable from a broken one
The confirmation screen for an irreversible operation states what becomes undeletable, for how long, and what it costs meanwhile, because the effect does not stop at the resource named in the call
After writing this article I ran the screen and found the lock indicator was broken. While only one panel was being read, it looked correct

The repository is solutions/amplify-portal/.

Test environment

Item	Value
Date	2026-09-02 (JST)
Region	ap-northeast-1
ONTAP version	9.18.1P3D1
Portal	Amplify Gen2 sandbox: AppSync + Cognito + VPC Lambdas
Lambda runtime	Python 3.13
How the action count was taken	`scripts/portal_action_types.py --check` and `scripts/check_portal_action_params.py`
Cleanup	Test sandboxes deleted. One working stack retained

The distinction that matters most — a path that persists, a holder that moved

First, the premise. Without routing through an additional third-party SaaS, three management interfaces reach FSx for ONTAP.

Interface	Path	Reach
AWS Management Console / FSx API	IAM	The AWS-side operations on file systems, SVMs and volumes; backups
ONTAP CLI (SSH)	SSH to the management endpoint	Roughly an ONTAP cluster administrator
ONTAP REST API	HTTPS to the management endpoint	The CLI's operations, programmatically

The sources, and how anything outside this list is treated, are in Management interfaces. The CLI and REST management endpoints are reachable only from inside the VPC or through a Transit Gateway peered network.

One consequence follows directly. Consider an everyday request: add a volume, or raise one quota. Answering it needs fsxadmin-equivalent credentials and a path into the VPC. So every request lands on the infrastructure owner, and none of it can be handed to anyone else.

That separation is what the portal does.

Delegation. A Cognito group (storage-admin) decides who may run what, per operation. No SSH key and no ONTAP credential is handed out. The credential lives in Secrets Manager and only the Lambda reads it
A record. Who ran which operation is retained with the Cognito principal. Unlike an SSH session on a shared account, it attaches to a person

The portal does not remove the path into the VPC: the Lambda that calls the ONTAP REST API sits inside it. The path did not disappear; what holds it moved from a person to a Lambda. That distinction affects operational design, so the repository documentation words it the same way.

The reach of the screen

The portal reaches 182 actions across 12 endpoints. That number is not maintained by hand. A script reconciles the generated types against the handlers and prints it.

$ python3 scripts/portal_action_types.py --check
DISPATCH ACTION TYPES: PASS (182 actions declared, matching the handlers)

$ python3 scripts/check_portal_action_params.py
PORTAL ACTION PARAMS: PASS (187 literal call sites, 182 actions
across 12 endpoints, 1 call(s) not statically readable)

The reason it is not counted by hand is not that the number grew. Types do not cross the boundary between the screen and the Lambda. The portal's dispatch takes an untyped params, so code that sends {snapshotName, retentionDays} to an action reading snapshotId and expiryTime compiles, lints, renders a button, and fails on every click. CI now fails until the reconciliation agrees.

By domain.

Domain	Operations
Volume lifecycle	Create / resize / delete / bring online, for both FlexVol and FlexGroup
FlexGroup capacity rebalance	Read state / start / stop / schedule
FlexCache	Create / list / resize / write-back toggle / delete
FlexClone	Create / list / split
SnapMirror	Create / list / transfer history / update now / quiesce / resume / break / resync / abort / delete
Snapshots	List / create / delete / lock / policy create / assign / enable locking
SnapLock and WORM	Read configuration / change retention, S3 Object Lock retention
ARP/AI (ransomware protection)	State change / bulk enable / suspect list and clear / surge parameters
Containment	Block NFS IP / block SMB user / disconnect sessions / list active blocks
QoS	Policy create / update / delete / assign to and release from a volume
Quotas	Rule create / update / delete / usage report / per-volume enforcement
SMB	Share create, update, delete; local users and groups; members; name mapping
Qtrees	Create / update / rename / delete
Export policies	Policy and rule create / delete
FPolicy and Vscan	Policy and event create / enable / delete
Peering	Cluster peer and SVM peer create / accept / delete, intercluster LIFs
Cluster inventory	Nodes / licenses / LIFs / protocols / DNS / jobs, EMS events
S3	Bucket list, Object Lock status and retention

182 is not every ONTAP operation. Operations that need advanced privilege are absent from the REST API, so they cannot appear on this screen.

The plain statement of what cannot be done

The copy limit is the clearest case. It is easy to confuse with the upload limit, so the measured values are split out.

Operation	Limit	Measured
Upload (single `PutObject`)	5 GiB	`MaxSizeAllowed` = 5,368,709,120
Upload (multipart, whole object)	50 GiB	53,687,091,200. Checked only at `CompleteMultipartUpload`
Download (`GetObject`)	No limit	Objects over 50 GiB were retrieved
Copy (single `CopyObject`)	5 GiB	As above
Copy (`UploadPartCopy`)	Unusable	Documented as Supported; every form returned `NoSuchKey`

Uploads do reach 50 GiB (measured with multipart). Copies, though, have no path past 5 GiB. UploadPartCopy is listed as Supported in Access point compatibility and returned NoSuchKey in every form measured. Rename, move, copy, trash and restore in the portal are all CopyObject, so that 5 GiB becomes the screen's limit.

The units caught me out as well. The AWS documentation's "5 GB" and "50 GB" both turned out to be binary when measured. MaxSizeAllowed from PutObject is 5,368,709,120 = 5 × 1024³, not the decimal 5,000,000,000. The detail is in the object size limit verification.

Rather than surfacing an S3 error halfway through, the portal checks the size before the copy and refuses with a reason.

{key} is 6.2 GiB. A single copy is limited to 5 GiB, and the multipart copy that would lift the limit is not usable on this Access Point, so this operation cannot be completed from the portal. Move the file over NFS or SMB instead.

The guard's 5 GiB threshold is the same value as the measured MaxSizeAllowed.

On the same reasoning, no button was built for converting a FlexVol to a FlexGroup. The conversion is ONTAP CLI advanced-privilege only, with no REST equivalent. Instead the screen carries the prerequisites, the irreversibility, the treatment of existing snapshots, and what AWS recommends — a copy into a new FlexGroup with AWS DataSync rather than an in-place conversion. Writing down that something is absent, and why, is more useful than a button that cannot be pressed.

The free-space estimate for a clone split (volume clone split estimate) is the same. It exists only in the CLI, so it cannot appear here. Rather than omitting it silently, the screen says why it is missing.

What the confirmation screen states for an irreversible operation

SnapLock and snapshot locking cannot be undone. "Are you sure?" on its own does not tell the person pressing it what becomes undeletable, so the screen states this:

The SnapLock type (compliance) can only be set at creation and cannot be changed or removed afterwards

Under compliance, nobody can delete until retention expires. Not the account administrator, and not AWS

While one unexpired WORM file remains, this volume, its SVM, and the file system cannot be deleted

While the file system cannot be deleted, it keeps billing

A newly created empty volume can still be deleted. It stops being deletable when the first file is committed to WORM

The last two lines are the point. The effect of an irreversible operation does not stop at the resource; it propagates to the parent and to the bill. A test environment is the worst place to put a file system you cannot delete for six months.

Snapshot locking is written up with its asymmetry intact. A lock can only be extended, never shortened or released. The policy side is reversible: removing the retention, switching policies, or detaching stops new locks. But snapshots already locked stay until they expire. Compressing that asymmetry into one sentence makes one half of it wrong.

A retention count does not act as a ceiling either. Locked snapshots cannot be rotated out, so they accumulate past the count until expiry and consume the space. That is on the screen too.

Irreversible operations carry a type-to-confirm (the keyword typed in capitals). The API also requires acknowledgeIrreversible, so a call that bypasses the UI passes the same check.

What is written here is the technical blast radius of an operation. Which retention period satisfies your regulatory requirements is a legal and compliance judgement, and neither this article nor the portal stands in for it.

What I did not measure

"Unverified" is not "impossible". The boundary of this article, stated as a boundary:

Item	State
Behaviour under concurrent users	Unmeasured. Only single-operation latency was observed
Operations needing advanced privilege	Out of scope. Absent from the REST API, so they cannot be on the screen
Enabling SnapLock compliance from the screen	Deliberately not done. Irreversible, and it would make a test file system undeletable for months
Multi-tier delegation with more Cognito groups	Unverified. There is one tier, `storage-admin`
Long-term retention and search of the audit record	Unverified. The record is kept; no retention design was done

Part 5 covers what the real cluster refused while these 182 actions were being built: why creating a FlexGroup through the ONTAP REST API needs the target aggregate named, why a capacity rebalance never starts with ONTAP's defaults, and the two places the documentation and the running system disagreed.

Conclusion

Putting operations on a screen was not a matter of listing features. Three paths already reach the file system and none of them went away, so what a screen adds is delegation and record, not reach.

Half the screen's job turned out to be saying what cannot be done rather than showing what can. Name the missing input on a button that cannot be pressed, write down absent features with the reason, and give irreversible operations their blast radius. Where a limit is measurable — the 5 GiB copy — using the measured value directly in the guard keeps the screen and the storage saying the same thing.

And one display defect surfaced after the article was written. Until two panels were reconciled, the broken indicator looked correct.

The figures come from one specific environment and configuration and will differ with workload and setup. This article does not stand in for governance or regulatory judgement.

AWS Backup now copies FSx for ONTAP backups across Regions and accounts — a new option, measured Tokyo to Osaka

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Sat, 29 Aug 2026 05:02:38 +0000

Introduction

How do you approach disaster recovery and backup for your file systems on AWS? Take FSx for ONTAP and a pair of Japanese Regions as the example: if Tokyo is primary, standing up a file system in Osaka and replicating into it continuously with SnapMirror is probably the first shape that comes to mind for anyone who has run ONTAP, or is about to evaluate it. Plenty of AWS users run exactly that in production, and as a way to hold availability it is still the best answer. RPO comes down from the 60 minutes AWS Backup offers at its shortest to SnapMirror's 5-minute minimum interval, switching between active and standby is breaking the relationship and promoting the destination, and cutting data back the other way is a snapmirror resync in the reverse direction.

The trouble was that "I just want the data sitting on the DR side, without operating a file system there until I actually restore" required the same shape. Amazon FSx for NetApp ONTAP volume backups were created and restored inside the file system's own Region and account, and could not cross a Region boundary. AWS Backup could already copy backups across Regions and accounts for FSx for Windows File Server, FSx for OpenZFS, and FSx for Lustre (AWS Backup extends Amazon FSx support to 5 additional AWS Regions and expands cross-Region and cross-account copy to 14 AWS Regions, April 2026), but FSx for ONTAP was not in that set at the time.

Then on 27 August 2026 that gap closed along the two paths below. One more shape became available to choose from in a backup strategy. The two were announced separately.

Path	What it does	Scheduling	Cross-account
FSx for ONTAP backup copy (`CopyBackup`)	Copies a volume backup to another Region	None. Run it each time	No. Same account only
AWS Backup (policy-based backup management service)	Copies FSx for ONTAP backups to another Region and another account	Yes. Copy rules in a backup plan	Yes. Requires AWS Organizations

So I copied a backup from Tokyo to Osaka, restored it at the destination, and confirmed the content matched. This post covers that procedure, along with the design and operational considerations that came with it.

Up front:

AWS Backup now copies FSx for ONTAP backups to another Region and another account (27 August 2026). The FSx for ONTAP CopyBackup also reaches another Region, but it is same-account and manual.
Moving them to another account needs a CMK. AWS managed keys cannot be used for a cross-account copy, and a file system's KMS key is set at creation — so it is a decision made before the file system exists (below).
What matters is that you no longer need a file system at the destination until you recover. The restore target is still the Region the backup lives in, so creating that file system at recovery time lands on your RTO — 20 minutes, measured (below).
CopyBackup has no scheduler. Recurring copies mean an AWS Backup backup plan or your own automation. The AWS Backup copy and restore were measured too — 6 m 31 s to 8 m 35 s to copy, 16 m 16 s to restore, at 9 MB.
This does not replace SnapMirror. If a minutes-level RPO and a failback procedure are requirements, SnapMirror still is the answer. Replication and copy are different operations and leave different things at the destination (below).
Restore time is proportional to capacity. My 13 minutes was a figure for 9.4 MiB. Restoring 10 TB on a minimum configuration is 22 hours at the floor, calculated from AWS published rates. I could not find a measured figure at 10 TB or above in public sources (below).
Cost inverts with scale. A Single-AZ capacity pool is 47.6% of backup storage ($0.0238 against $0.050, retrieved from the Price List API), so past roughly 4.8 TB a standing destination file system is lower monthly. Choosing to keep one running is not choosing the more expensive option (below).

Scope: within one account, copying a volume backup from ap-northeast-1 to ap-northeast-3, restoring it into a file system there, and verifying the content. Both CopyBackup and AWS Backup were measured.

Out of scope: cross-account copies (they require AWS Backup with AWS Organizations — I show the shape but did not measure it), building or failing back a SnapMirror relationship, durations at production data volumes, and reconciling inter-Region transfer charges against a bill.

Shape of the setup

(Dark theme: backup-copy-cross-region-en-dark@2x.png)

The right-hand side being empty in normal operation is the whole point. With SnapMirror, that right-hand side holds an equivalent file system at all times.

Layer	Day to day	When recovering
Tokyo (production)	file system + RW volume + backups	unchanged, or lost if the Region is affected
Osaka (destination)	the backup copy only. No file system	file system → SVM → restore into a new volume
Billing	Osaka pays backup storage only	Osaka's file system starts billing once it exists
Running the copy	`CopyBackup` each time, or an AWS Backup copy rule	—

On vendor neutrality: this post evaluates FSx for ONTAP backup copies on technical grounds only. The section on existing backup products touches other options, but not in a framing where one is better than another. Each suits a different context, and the goal here is to record what actually works and what does not.

Test environment

Item	Value
Date	2026-08-28 (`CopyBackup` path), 2026-08-29 (AWS Backup path)
Runs	One per step (n=1). Variance was not measured
Interface	AWS CLI v2 and the AWS Management Console. Japanese locale for the `CopyBackup` path, English locale for the AWS Backup path (the setting at capture time). CloudFormation untested
Source	`ap-northeast-1`, first-generation `SINGLE_AZ_1`, 1,024 GiB SSD, 128 MBps
Destination	`ap-northeast-3`, same shape, created for this test
ONTAP version	`9.17.1P7D1`
Source volume	FlexVol RW, 1 GiB, tiering `NONE`, storage efficiency off
Dataset	5 files / 9.4 MiB (nested directory, symlink, UTF-8 filename, one file at `0640`). The AWS Backup run used an equivalent 9,458,747 B set
AWS Backup	one vault per Region, one backup plan (one rule, one copy rule), selection scoped to the single verification volume ARN, existing `AWSBackupDefaultServiceRole`
KMS	Defaults in each Region: `aws/fsx` for the source file system, `alias/aws/backup` (`KeyManager: AWS`) for the AWS Backup vaults. CMKs untested, and this configuration cannot do a cross-account copy (below)
Clients	one `t3.micro` (AL2023) per Region, NFS
Actual spend	~$0.55 for the `CopyBackup` run (destination file system ran 1 h 24 m); under $2 for the AWS Backup run (destination file system ran ~1 h 21 m, two `t3.micro`)
Data used	Synthetic data from `/dev/urandom`. No real or personal data
Not verified	Cross-account copies (covered by the earlier post), CMKs, copying `AUTOMATIC` backups, incrementality, residue from a failed copy, deploying the CloudFormation template, restoring into an AD-joined SVM, reconciling inter-Region transfer charges, AWS Backup restore testing

Do not use these durations as an RTO basis for your environment. They are measured against 9.4 MiB, where fixed overhead dominates.

When this fits

You want a copy of your data in another Region but want to avoid running a file system there
Compliance requires retaining a copy that is logically separated from production
You want an isolated-account layer against credential compromise or a KMS key compromise
Recovery measured in tens of minutes to hours is acceptable
You already take volume backups and only want to add a destination for them
SnapMirror already covers availability, and you want a separate retention layer detached from production

When to look at something else for now

RPO in minutes is required (the guidance figure for backups is 60 minutes)
RTO in single-digit minutes is required (creating the destination file system took 20 minutes)
You need a documented path back to production (backup copies have none — see below)
FlexGroup volumes are in scope (copy is unsupported, and creation failed in my environment — see below)
You need copies spanning China Regions, or cross-account copies in the China Regions (unsupported)

If any of those apply, a SnapMirror-based design fits the requirement better. Running both together is also coherent, covered below.

The distinction that matters most — what changed is where a backup can live

Blur this and the runbook comes out wrong.

Claim	Status
A backup can be stored in another Region	now possible (`CopyBackup` or AWS Backup)
A backup can be stored in another account	now possible, but only through AWS Backup. The FSx for ONTAP `CopyBackup` stays inside one account
A backup can be restored to a file system in a different Region than the backup	no (unchanged)
A restore produces a new volume	unchanged. There is no restore-in-place path

using-backups.html still states that a backup can only be restored to a file system in the Region where the backup is stored. That is not a contradiction — copying increases the set of Regions where a backup is stored.

So recovery in Osaka is three steps:

A copied backup is already in Osaka (this is what you prepare in advance)
Create a file system and an SVM in Osaka
Restore as a new volume

Step 2 lands on your RTO. Not paying for a standby file system means paying for its creation when you need it. That is the trade you are choosing.

How `CopyBackup` and AWS Backup relate

Two different things are called "copy a backup to another Region" here, so it is worth separating them first. They are separate mechanisms. Neither calls the other, and their control planes and storage locations differ.

	FSx for ONTAP `CopyBackup`	AWS Backup
Whose API	Amazon FSx (`fsx:CopyBackup`)	AWS Backup (backup plans / copy jobs)
What it copies	An existing FSx for ONTAP volume backup, which must be `AVAILABLE`	A recovery point AWS Backup took itself
Where the copy lands	As an FSx for ONTAP backup in the destination Region	An AWS Backup backup vault
`BackupType`	`USER_INITIATED`	`AWS_BACKUP`
Scheduling	None. Called each time	Yes. Backup rules and copy rules
Cross-account	No	Yes, with AWS Organizations
Link back to the source	Keeps `SourceBackupId` and `SourceBackupRegion`	Followed through the recovery point ARN

Here is where the two get conflated. You cannot point CopyBackup at an AWS Backup backup vault, and an AWS Backup copy rule does not call fsx:CopyBackup on your behalf. Answering "I want CopyBackup on a schedule" with "use AWS Backup" is right not because the same call can be scheduled, but because the same goal is reachable through a different mechanism.

The restore constraint, on the other hand, is shared. Both paths ask you to pick an existing file system and SVM in the destination Region and both produce a new volume. The AWS Backup restore screen asks for a File system and a Storage virtual machine too (documented). So the benefit — no standby file system while nothing is wrong — holds for AWS Backup as well, and so does the cost: file system creation lands on the RTO either way.

On terminology: AWS Backup's own word is copy. How that differs from replication is the next section.

Replication and copy are different words for different things

Two words get used for "keep the data in another Region": replication and copy. They are different operations, and what sits at the destination differs. Without agreeing on that, a requirements conversation goes past itself.

	SnapMirror replication	AWS Backup / `CopyBackup` copy
What sits at the destination	A volume. A `DP` volume exists on the destination SVM	A backup (recovery point). No volume
When the source changes	It follows. Deltas are sent each schedule	It does not. A point-in-time image; later changes wait for the next backup
To use the destination	Break the relationship and promote it to become writable	A restore is required, and what you get is a new volume
Does the relationship persist	Yes. Having it in place is the normal state	No. Each copy is an independent artefact
What sets the RPO	The replication schedule (down to 5 minutes)	The backup interval (guidance: 60 minutes)
What the destination costs while idle	File system capacity and throughput	Backup storage only
Path back to production	`snapmirror resync` in the other direction	There is none

AWS's own wording splits the same way. The AWS Backup console and documentation call this operation copy throughout — the screens read Copy jobs, Copy rule, Copy type: Manual (screenshots above). The FSx for ONTAP API is named CopyBackup. Replication is the word for mechanisms where the destination exists and tracks the source, as in Amazon S3 Cross-Region Replication.

So "replicate cross-Region with AWS Backup" reads as though a volume at the destination is tracking the source. There isn't one. What is at the destination is a recovery point, and using it takes a restore. The difference lands directly on the RTO: replication promotes a volume that is already there, while a copy starts from creating a file system and an SVM and then restoring.

The reverse mistake is calling SnapMirror a backup. The destination tracks the source, so a file deleted on the source is gone from the destination's current state after the next transfer — what remains is whatever the destination's snapshots still hold. Retaining generations is the job of snapshots or SnapVault, not of the replication itself.

SnapVault is a third mechanism. ONTAP's SnapVault is a relationship that accumulates generations at the destination. It rides the same transport as SnapMirror with a different purpose. The backup copies in this post do not create an ONTAP SnapVault relationship.

Which one suits which requirement is in choosing between this and SnapMirror.

A short glossary

Only the terms this post uses. Skip if you already know them.

Term	What it is	Why it matters here
File system	The billing and performance unit. SSD capacity and throughput are provisioned here	This is what you no longer need at the destination
SVM (Storage Virtual Machine)	A logical server inside a file system, holding the NFS / SMB endpoints and auth config	Required to restore. A file system alone is not enough
Volume	Where data lives, and the unit of backup and restore	A restore produces a new volume
Junction path	Where a volume attaches in the SVM namespace (e.g. `/dr_restored`); the NFS mount path	Specified at restore
FlexVol / FlexGroup	Volume styles. FlexVol sits on one aggregate; FlexGroup spreads across several for capacity and parallelism	FlexGroup backup copies are unsupported
`RW` / `DP`	Volume type. `RW` is read-write, `DP` is read-only and used for SnapMirror destinations	`DP` cannot be backed up. A restoring volume transiently reads as `DP`
Tiering	Moving cold data from SSD to the cheaper capacity pool. `NONE` / `AUTO` / `SNAPSHOT_ONLY`	Affects read latency right after a restore
Snapshot vs backup	A snapshot lives inside the file system; a backup is an independent copy outside it	Only the backup can cross a Region boundary

If you run ONTAP on-premises

ONTAP concept	Relationship to this capability
SnapMirror	A different mechanism. Needs cluster peering and a standing destination file system
SnapVault	A different mechanism. Backup copies do not create a SnapVault relationship
Cluster peering	Not required for backup copies. Operationally this is the largest difference from SnapMirror
Driving it from ONTAP CLI / REST	Backup copies are an AWS-side feature with no corresponding ONTAP CLI command. Use the AWS CLI, API, or console
Trace left on the volume	A backup leaves a snapshot named `backup-<backup-id>` on the volume

Prerequisites and permissions

Prerequisites

Item	Condition
Source backup	must be `AVAILABLE`
Source volume type	`RW` (`DP`, LSM, and FlexCache destinations cannot be backed up at all)
Volume style	FlexVol. FlexGroup copies are unsupported
Partition	Between commercial Regions, between the two China Regions, and between the two GovCloud (US) Regions — but not across those sets
KMS	Encrypted with a key in the destination Region. No parameter needed if the default key is fine

A backup whose source volume has already been deleted can still be copied (measured). That is useful after an accidental deletion.

IAM

The requester must be able to reach the backup in the source Region. Following the shape in the AWS documentation:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CopyBackupAcrossRegions",
      "Effect": "Allow",
      "Action": "fsx:CopyBackup",
      "Resource": "arn:aws:fsx:*:<your-account-id>:backup/*"
    },
    {
      "Sid": "DescribeAndRestore",
      "Effect": "Allow",
      "Action": [
        "fsx:DescribeBackups",
        "fsx:DescribeFileSystems",
        "fsx:DescribeStorageVirtualMachines",
        "fsx:DescribeVolumes",
        "fsx:CreateVolumeFromBackup",
        "fsx:TagResource"
      ],
      "Resource": "*"
    }
  ]
}

The "Resource": "*" on the second statement is a simplification for readability. Narrow it in production. Some Describe* actions do not support resource-level scoping, but fsx:CreateVolumeFromBackup can be limited to the destination file system and SVM ARNs. To pin the destination Region, use the aws:RequestedRegion condition key:

{
  "Sid": "RestrictCopyDestination",
  "Effect": "Deny",
  "Action": "fsx:CopyBackup",
  "Resource": "*",
  "Condition": {
    "StringNotEquals": { "aws:RequestedRegion": ["ap-northeast-3"] }
  }
}

If you have a data residency requirement, put this guard in IAM or an SCP. "We have a rule that says not to" does not hold.

With a customer managed key (CMK), you will additionally need permissions such as kms:CreateGrant and kms:DescribeKey on the destination-Region key. I used the default key, so I have not verified this. If you are building on a CMK, prove it with a small volume first.

Values to replace with your own

Every command below works once you substitute these.

Placeholder	Meaning	How to get it
`<SRC_REGION>`	Source Region	e.g. `ap-northeast-1`
`<DST_REGION>`	Destination Region	e.g. `ap-northeast-3`
`<SRC_VOLUME_ID>`	The RW volume to back up	`aws fsx describe-volumes --region <SRC_REGION> --query 'Volumes[?OntapConfiguration.OntapVolumeType==`RW`].[VolumeId,Name]' --output table`
`<SRC_BACKUP_ID>`	The backup to copy	output of step 1
`<DST_SUBNET_ID>`	Subnet for the destination file system	`aws ec2 describe-subnets --region <DST_REGION> --query 'Subnets[].[SubnetId,AvailabilityZone,CidrBlock]' --output table`
`<DST_SG_ID>`	Security group for the destination file system	Must allow NFS / SMB. Sharing one SG with the client plus a self-referencing rule is enough
`<DST_FS_ID>`	Destination file system	output of step 4
`<DST_SVM_ID>`	Destination SVM	output of step 5
`<DST_VOLUME_SIZE_BYTES>`	Restored volume size, in bytes	Must be at least the data in the backup. Reusing the source `SizeInBytes` is safest. 1 GiB = `1073741824`

Two notes on sizing. The destination file system's SSD capacity must exceed the data being restored — restored data is written to SSD first. And a FlexVol cannot span more than one HA pair. For a large volume, settle the destination SSD capacity and generation (second-generation Single-AZ can add HA pairs) before you start.

Walkthrough — Tokyo to Osaka

1. Create a backup (skip if you already have one)

aws fsx create-backup \
  --region <SRC_REGION> \
  --volume-id <SRC_VOLUME_ID> \
  --tags Key=Name,Value=dr-source Key=Purpose,Value=cross-region-copy \
  --query 'Backup.{Id:BackupId,Life:Lifecycle}' --output table

Wait for AVAILABLE:

aws fsx describe-backups \
  --region <SRC_REGION> --backup-ids <SRC_BACKUP_ID> \
  --query 'Backups[0].{Life:Lifecycle,Pct:ProgressPercent}' --output table

2. Copy it

--region is the destination; --source-region is where the backup is now. This reads backwards at first — the copy is requested against the destination Region.

aws fsx copy-backup \
  --region <DST_REGION> \
  --source-backup-id <SRC_BACKUP_ID> \
  --source-region <SRC_REGION> \
  --copy-tags \
  --tags Key=Name,Value=dr-copy \
  --query 'Backup.{Id:BackupId,Life:Lifecycle,Src:SourceBackupId,SrcRegion:SourceBackupRegion}' \
  --output table

Add --kms-key-id <destination-region-key-arn> for a CMK. Incrementality is conditional on using the same KMS key, so changing keys mid-stream makes the next copy full.

Watch progress:

aws fsx describe-backups \
  --region <DST_REGION> --backup-ids <DST_BACKUP_ID> \
  --query 'Backups[0].{Life:Lifecycle,Pct:ProgressPercent}' --output table

COPYING → AVAILABLE means done. Mine took 7 m 15 s for a 9.4 MiB volume.

Day-to-day operation ends here. Steps 3 to 6 run only when you are recovering.

3. [Recovery] Check which generation the destination Region offers

Check the available deployment types before you need them. If first generation (SINGLE_AZ_1) is available, the minimum throughput is 128 MBps; where only second generation exists it is 384 MBps, which makes standby cost roughly 1.9× higher.

SINGLE_AZ_1 was accepted in Osaka (measured 2026-08-28). This varies by Region — confirm it for your destination before writing the runbook.

4. [Recovery] Create the destination file system

Billing starts here. Even at the minimum, Osaka costs roughly $0.37 per hour, and forgetting it runs about $270 a month. If you are doing this as a drill, open the teardown commands from step 8 first. During my own run I mistyped an option on delete-file-system and polled for 8 minutes while the command had already failed.

This is the 20 minutes that lands on RTO.

aws fsx create-file-system \
  --region <DST_REGION> \
  --file-system-type ONTAP \
  --storage-capacity 1024 \
  --storage-type SSD \
  --subnet-ids <DST_SUBNET_ID> \
  --security-group-ids <DST_SG_ID> \
  --ontap-configuration '{
    "DeploymentType": "SINGLE_AZ_1",
    "ThroughputCapacity": 128,
    "PreferredSubnetId": "<DST_SUBNET_ID>"
  }' \
  --tags Key=Name,Value=dr-restore-target \
  --query 'FileSystem.{Id:FileSystemId,Life:Lifecycle}' --output table

1,024 GiB and 128 MBps are the first-generation minimums. Raise them if your restore exceeds that. A restore pauses when SSD space runs out and resumes automatically once space is available.

5. [Recovery] Create the SVM

aws fsx create-storage-virtual-machine \
  --region <DST_REGION> \
  --file-system-id <DST_FS_ID> \
  --name dr_svm \
  --root-volume-security-style UNIX \
  --query 'StorageVirtualMachine.{Id:StorageVirtualMachineId,Life:Lifecycle}' --output table

For SMB you also need the Active Directory join here. If AD join is part of your recovery, verify separately that the destination Region can reach your domain controllers, and budget the time. My verification was NFS only.

6. [Recovery] Restore

A restore always produces a new volume. There is no in-place path.

aws fsx create-volume-from-backup \
  --region <DST_REGION> \
  --backup-id <DST_BACKUP_ID> \
  --name dr_restored \
  --ontap-configuration '{
    "StorageVirtualMachineId": "<DST_SVM_ID>",
    "JunctionPath": "/dr_restored",
    "SizeInBytes": <DST_VOLUME_SIZE_BYTES>,
    "OntapVolumeType": "RW",
    "TieringPolicy": { "Name": "NONE" }
  }' \
  --query 'Volume.{Id:VolumeId,Life:Lifecycle}' --output table

TieringPolicy: NONE follows the AWS recommendation for consistent read latency after a restore — it assumes you wait for the data to land on SSD before accessing it. Change it if you want the capacity pool.

Mine reached CREATED in 13 m 21 s for 9.4 MiB.

Three conditions drive restore and post-restore performance. Check these when it feels slow:

Condition	Detail
SSD capacity	Restored data is written to SSD first. The restore pauses when space runs out and resumes when it frees up
Generation	Second generation is readable during a restore (after metadata loads). First generation makes you wait for completion — mine was first generation
Background priority	Backup and restore are lower priority than client I/O and consume unused throughput capacity. They progress slowly while production I/O is saturated

The third one has an operational consequence: overlapping backups with a nightly batch window slows both. Separate the backup window from your peak.

7. Verify the content

Skip this and you cannot claim you restored anything. Order matters: take the hashes before you create the backup. Taking them afterwards mixes in every change since, and you can no longer separate that from a copy problem.

# On the source, BEFORE creating the backup
cd /mnt/src && find . -type f -exec sha256sum {} + | LC_ALL=C sort -k2 > /tmp/before.txt

# On the restored volume
cd /mnt/restored && find . -type f -exec sha256sum {} + | LC_ALL=C sort -k2 > /tmp/after.txt
diff /tmp/before.txt /tmp/after.txt && echo "MATCH"

Check permissions and symlinks too:

find . -mindepth 1 -printf "%y %m %s %p\n" | LC_ALL=C sort -k4

All five sha256 digests matched in my run, and mode 0640, the symlink target, and a UTF-8 (Japanese) filename all survived. I also used the ordering deliberately: a 2 MiB file added after the backup was absent from the restored volume, which is what makes it a point-in-time image rather than a mirror.

8. Delete what the test created

Forgetting the destination file system costs roughly $270 a month. The order matters.

# 1. volume (do not let it create a final backup)
aws fsx delete-volume --region <DST_REGION> --volume-id <DST_VOLUME_ID> \
  --ontap-configuration '{"SkipFinalBackup":true}'

# 2. SVM (after the volumes are gone)
aws fsx delete-storage-virtual-machine --region <DST_REGION> \
  --storage-virtual-machine-id <DST_SVM_ID>

# 3. file system (after the SVM is gone)
aws fsx delete-file-system --region <DST_REGION> --file-system-id <DST_FS_ID>

# 4. the copied backup, if you no longer need it
aws fsx delete-backup --region <DST_REGION> --backup-id <DST_BACKUP_ID>

Without SkipFinalBackup a final backup is created, and that backup then blocks the next deletion. I have been stuck on this before.

And confirm the deletion through the API. During this run I passed an unsupported option to delete-file-system and polled for 8 minutes while the command had already failed. Waiting without reading the response only advances the bill.

Measured through AWS Backup as well

Everything above is the CopyBackup path. The same route, Tokyo to Osaka, was also measured through AWS Backup. That is the path that carries scheduling and cross-account copies, so its behaviour is what matters for production.

Two ways round. An on-demand backup job followed by an on-demand copy job, and a backup plan with a copy rule left to fire on its schedule. Both used the same source volume (1 GiB, 9,458,747 bytes, 5 files).

Step	Elapsed
On-demand backup job	4 m 02 s
On-demand copy job, Tokyo to Osaka	8 m 35 s
Plan-triggered backup job	30 m 06 s, of which about 24 minutes was the start window
Copy rule copy job (automatic)	6 m 31 s
Restore from the AWS Backup console	16 m 16 s

The copy rule fired without being touched. The schedule came round, a job was created, the backup completed, and a copy job followed on its own to put a recovery point in the Osaka backup vault.

A plan-triggered job waits in `CREATED`

This changes an operational assumption. The on-demand job went to RUNNING immediately. The plan-triggered one stayed CREATED for about 24 minutes. When it starts inside the start window — 60 minutes in this configuration — is AWS Backup's decision.

So a runbook that reads "the scheduled time has passed, therefore the backup exists" is wrong by the length of that window. Confirm a generation by job state or by the recovery point, not by the clock.

The recovery point in Osaka points at the source

The Resource ID column names the Tokyo file system and volume. It sits in the Osaka vault and points at where it came from.

Open the recovery point and the copy is visible as such.

Source region reads Asia Pacific (Tokyo). That is the evidence this artefact came from a cross-Region copy. Copy type is Manual because this one was run on demand; the copy-rule one arrives as a separate recovery point.

The restore form makes you pick a destination file system and SVM

The "File system ID" at the top is the source. The destination is chosen separately, below. Read the two as the same thing and the restore target looks already decided. It is not.

Open the dropdown and only file systems in the destination Region are listed.

With no file system in Osaka this list is empty. Exactly the same constraint as the CopyBackup path: not holding a destination day to day is paid for here.

Storage efficiency is checked by default

Two things.

Volume size is pre-filled from the source (1024 MiB). The FSx for ONTAP console defaults the same field to 1 TiB (below), so AWS Backup is the more predictable of the two here.

"Enable storage efficiency" is checked by default. The source volume had it off, and the restored volume came back with StorageEfficiencyEnabled: true. The FSx for ONTAP console restore form pre-selects the source value, so the two consoles behave differently. Uncheck it to match the source.

Attribute	Source	After the AWS Backup restore
`OntapVolumeType`	`RW`	`RW`
`SizeInBytes`	1 GiB	1 GiB
`TieringPolicy`	`NONE`	`NONE`
`StorageEfficiencyEnabled`	`false`	`true` (the form's default)
`SecurityStyle`	`UNIX`	empty

SecurityStyle coming back empty had been seen once on the CopyBackup path. It reproduced through AWS Backup, so it is no longer a single-path observation.

The content matched

Mounted from an EC2 instance in Osaka and compared. All five sha256 values matched, and the symlink target, the 0640 mode, the UTF-8 filename, the nested directories and the mtimes were preserved.

The restored volume carries .snapshot/backup-<id>/. That <id> is the source FSx for ONTAP backup, not the Osaka recovery point ID. The destination volume's default snapshot policy starts running too, so hourly.* snapshots appear beside it.

What cannot be monitored, and what catches you on teardown

BackupSizeInBytes was 0 on both backup jobs. Not a size to alert on.

Restore progress read 0.00% for the whole 16 minutes, then 100%. Anything that estimates remaining time from the percentage will not work.

One thing caught me on teardown. A vault cannot be deleted until an EXPIRED recovery point is fully gone. After delete-recovery-point the entry stayed listed as EXPIRED for about six minutes, and delete-backup-vault refused with "contains recovery points" throughout. The underlying FSx for ONTAP backup was still AVAILABLE at that point. Teardown automation has to poll; taking the delete request's success as done leaves you stuck.

On the console language: the screens in this section are in English. That was the setting at capture time; the other sections show the FSx for ONTAP console in Japanese. Field names are quoted in English either way.

Cross-account copies were not measured. They require AWS Organizations, so this verification stayed inside one account.

For the cross-account half — an earlier post, and what this one adds

There is a post on the NetApp Community that goes further on cross-account copies than this one does.

Cross-Region and Cross-Account Backup for Amazon FSx for NetApp ONTAP (Jordan Roth, NetApp Community)

It walks the cross-account path I did not measure, having actually run it. It sets out the protection layers (Snapshot copies, Multi-AZ HA, SnapMirror, AWS Backup copies), four architecture patterns — fan-in into an isolated vault account, fan-out to several, crossing Region and account in one hop, and a clean room for incident work — and the organization, key and IAM prerequisites behind them. If cross-account is going to production, read that first. What follows here is only where it meets my own measurements.

What this post was missing — cross-account needs a customer managed key

This is the one that matters. A cross-account copy requires a customer managed key (CMK). The AWS Backup documentation states that for resource types not fully managed by AWS Backup, cross-account copy with AWS managed keys is not supported, because an AWS managed key's key policy is immutable and cannot be shared across accounts (Encryption for backups in AWS Backup). FSx for ONTAP sits on the "not fully managed" side.

My verification did not meet that condition. The destination vault was created with defaults, so its key was alias/aws/backup (KeyManager: AWS). The cross-Region copy still succeeded. That is looser than the documentation's general statement, and it is the reason for measuring rather than assuming.

	Key used	Result
Cross-Region copy (Tokyo to Osaka, one account)	Defaults on both the source file system and the vaults (`aws/fsx` / `aws/backup`)	Succeeded (measured)
Cross-account copy	AWS managed keys cannot be used	Not measured. Unsupported per the documentation

There is a design consequence. A file system's KMS key is chosen at creation, and update-file-system has no parameter for changing it (checked in the AWS CLI). So if a cross-account copy might ever be needed, the CMK decision happens before the file system exists. It is not the kind of thing to switch on later. The destination vault has its own constraint: the default vault cannot be used, because its key cannot be shared.

The Region exceptions are listed individually

Earlier I wrote "commercial Regions where both services are available". AWS Backup feature availability lists the exceptions one by one.

Exception	Scope
Cross-Region copy not supported	Middle East (Bahrain), Middle East (UAE) — for all four of FSx for ONTAP, Lustre, Windows File Server and OpenZFS
Cross-Region and cross-account copy not supported	Asia Pacific (New Zealand), China (Beijing), China (Ningxia) — FSx for ONTAP

The China Regions need reading carefully. The AWS Backup document history carries an entry dated the same 27 August 2026 saying cross-account backup copy and cross-account management are now supported in the China Regions. That is about AWS Backup generally; FSx for ONTAP is excluded there, per the table above. Merge the two same-day announcements and you reach the opposite conclusion.

On "what was possible before", that post and I read it differently

Its table of changes records cross-Region CopyBackup within one account as already available before the launch. This post has been saying a backup could not cross a Region boundary before. Rather than assert, here is what I checked.

What I checked	What it says
ONTAP User Guide, "Protecting your data with volume backups" (archived 2026-07-05)	The sub-page list has no "Copying backups" — only "Copying tags to backups". The body says a backup restores to a file system in the same Region where it is stored
`copy-backups.html` in the ONTAP User Guide	No snapshot at all in the Internet Archive
The FSx for ONTAP What's New	"Previously, you could create and restore backups in the same Region and account as your file system"

What I cannot test now is whether the CopyBackup API would have accepted an ONTAP backup before the launch. That API is not specific to FSx for ONTAP, and has supported cross-Region copies for FSx for Windows File Server and FSx for Lustre for years, so reading the API reference alone and concluding it was already possible is a reasonable reading. Documentation not describing something is not the same as an API refusing it, so I am leaving that unverified.

For a reader today the practical answer is the same either way. What is unambiguously new is the AWS Backup half: cross-Region copy through AWS Backup, and any cross-account copy, from 27 August 2026.

What this post adds

The parts that do not overlap.

Here	What
Measured durations	Backup, copy and restore on both paths, to the second — including a plan-triggered job waiting about 24 minutes in the start window
Two consoles behaving differently	The AWS Backup restore form checks storage efficiency by default; the FSx for ONTAP one pre-selects the source value
A single-path observation reproduced	`SecurityStyle` coming back empty
What cannot be monitored	`BackupSizeInBytes` reads 0; restore progress sits at `0.00%` for 16 minutes
Where teardown sticks	An `EXPIRED` recovery point blocks vault deletion for minutes
Cross-Region copy on default keys	It works with `aws/backup` (above)
Replication and copy as terms	Covered in its own section
Regional DR inside one country	Tokyo to Osaka, with the cost of keeping data in-country

The two things that caught me out

1. A restoring volume reads as `DP`, not `RW`

I read the attributes right after the restore reached CREATED and got OntapVolumeType: DP. DP volumes are read-only and cannot be backed up. I started writing "a restore produces a read-only volume".

Then I tried touch from the client. It succeeded. A re-read returned RW. To be sure, I ran a second restore passing OntapVolumeType: RW explicitly — and it also reported DP while CREATING.

State	`OntapVolumeType`
Restoring (`CREATING`)	`DP`
Restore complete (`CREATED`)	`RW`

It is transient. If monitoring or automation reads this field, it sees DP during a restore. Since DP volumes cannot be backed up, a naive "back up as soon as the restore finishes" step behaves differently depending on when it reads. Gate on Lifecycle reaching CREATED first.

Without the write probe I would have published the wrong claim.

2. FlexGroup backup creation failed asynchronously

The documented limitation is that copying backups of FlexGroup volumes is unsupported. It attaches to the copy and says nothing about creation. I misread it and built the test on the creation side.

CreateBackup against a FlexGroup volume was accepted and returned CREATING, then went FAILED about 30 seconds later. The message names no cause:

Backup failed. Please delete the backup and try again.

As a control, a FlexVol RW backup created in the same session with the same credentials succeeded — so this is not permissions.

But this is one observation on one volume. I am not claiming FlexGroup backups cannot be created. AWS documentation describes the restore behaviour of FlexGroup backups (adding constituents when the HA pair count differs), which does not read as creation being unsupported. Separately it states only that a SnapLock FlexGroup volume cannot be backed up. My target was a non-SnapLock FlexGroup, single constituent, one aggregate, 113 GiB provisioned.

The consequence: no AVAILABLE FlexGroup backup was produced, so the documented copy restriction was never exercised. If FlexGroup is in scope, verify it from the creation side in your own environment.

3. The attribute I called "not carried over" was a CLI default

Seeing StorageEfficiencyEnabled: true on the restored volume, I wrote that storage efficiency is not carried over by a restore. That was wrong too.

Running the same restore through the console showed the field pre-selected to the source value (disabled), and it restored as false. The initial true came from omitting the field in the CLI --ontap-configuration. It is an API default when omitted, not restore behaviour. State it explicitly if you script the restore.

That is twice in one exercise that I nearly published a behaviour inferred from a single observation. Running the same operation through a second interface is what exposed both.

Attribute	Source	Restored
`StorageEfficiencyEnabled`	`false`	`true` if omitted in the CLI, `false` when stated
`SecurityStyle`	`UNIX`	empty in the API response (both CLI and console; UNIX mode bits behaved correctly over NFS)

Check storage efficiency after a restore — omitting the field enables it even when the source had it off. The empty SecurityStyle reproduced across both paths, but against the same UNIX-style source; other security styles were not tested.

What the console shows that the CLI does not

Four behaviours only visible in the console.

A note on the screenshots: my console is set to Japanese, so that is what you will see. The field
positions and the behaviour are identical in any locale, and each caption names the field in English.
If you would rather follow along in English, switch your own console language under Unified
Settings → Language.

The copy form defaults to the same Region

Backups list. Select a backup, then **Actions → Copy backup* (バックアップをコピー), alongside restore
(バックアップを復元) and delete (バックアップを削除).*

Copy backup form. *送信先リージョン (destination Region) defaults to the current Region** — leaving it
alone produces an in-Region copy. タグをコピー is "copy tags".*

The destination Region list. Commercial Regions only; the China and GovCloud sets cannot be crossed.

With Asia Pacific (Osaka) selected, the **KMS キー ID (KMS key ID) changes to that Region's default
key. This is what "incrementality requires the same KMS key" means in practice: the key in the
destination Region.

While the copy runs

Destination Region's Backups list. ステータス reads コピー中 ("copying"), and the entry is labelled
*(コピーされたバックアップ) — "(copied backup)"*. Tag copying carried the name across.

The completed copy. *ソースバックアップ ID (source backup ID)** and ソースバックアップリージョン
(source backup Region) = ap-northeast-1 are the two fields that establish this is a copy. タグ shows
the copied tags.*

Restore needs more than a file system

The dialog is titled *バックアップからボリュームを作成 — "Create volume from backup"*. The title states
plainly that a restore produces a new volume. ファイルシステム (file system) starts empty.

The ファイルシステム dropdown lists **only file systems that already exist in the destination Region. If
you have not created one, there is nothing to select — which is where the "create the file system first"
step announces itself.

After picking the file system, *ストレージ仮想マシン (storage virtual machine) appears with a red 必須
("required")** and blocks submission. This is the concrete form of "the destination needs a file system
and an SVM". Note the SnapLock section at the bottom — irreversible, and not a field to set in
passing during a restore.*

The volume size default is 1 TiB

The same form filled in. *ボリュームサイズ (volume size) defaults to 1 TiB** even though the source
volume is 1 GiB — I changed the unit to GiB here. ボリューム名, ジャンクションパス, and ストレージ効率
are all pre-filled from the backup; size is not. On a 1,024 GiB destination the default consumes the
entire file system.*

The console shows `DP` too, and does not refresh

During the restore. *ライフサイクルの状態 (lifecycle) = 作成 ("creating")** and ONTAP ボリュームタイプ
(ONTAP volume type) = DP — with Read-Write (RW) selected in the form. ジャンクションパス and
ストレージ効率 are absent, and セキュリティスタイル (security style) is blank.*

The same page after a reload: *作成済み ("created")** and RW, with junction path, storage efficiency,
and snapshot policy now shown. The page does not auto-refresh — it kept showing DP after the API had
already returned RW, so judging completion by eye alone misses it. セキュリティスタイル stayed blank.*

Designing this for production

Everything above was one manual copy. Six things still need deciding.

1. Scheduling — `CopyBackup` has none

The AWS documentation describes the FSx for ONTAP path as "manually copy volume backups". It runs from the console, CLI, or API on demand and carries no scheduler. Two ways to make it periodic:

Path	Shape	Fits when
AWS Backup backup plan	Adding a copy rule to a plan's rule writes each backup to a backup vault in the chosen Region, and optionally the chosen account, as it is taken. On-demand copy jobs are available too	You want policy control, or the same protection applied across an organization. The only path if the copy has to land in another account
Your own automation	EventBridge Scheduler → Lambda → `create-backup` → wait → `copy-backup`	You want to keep using FSx for ONTAP backups directly and manage generations yourself

AWS Backup is a managed service that centralizes taking, copying, and retaining backups under policy. FSx for ONTAP joined its cross-Region and cross-account copy targets in August 2026. Applied across an organization it pairs with AWS Organizations, and the account boundary is defined by Organizations policies; the documentation describes fan-in (many production accounts into one isolated account) and fan-out (one production account into several isolated accounts).

The cross-Region copy and the restore were measured (Measured through AWS Backup as well). The copy rule fired without being touched and the restore went through. A plan-triggered job does wait inside the start window, though — about 24 minutes here — so the scheduled time is not when the generation exists.

AWS Backup's Region coverage here is "all commercial Regions where both services are available", but the feature availability page lists individual exceptions (below). That page also states that when a resource type has no checkmark in the copy columns, copy operations are unsupported in every scenario, including a same-Region same-account copy to a different vault.

If you build it yourself, calling copy-backup before create-backup completes fails — the source must be AVAILABLE. Poll, or use a Step Functions wait.

Whether AUTOMATIC backups can be copied is unverified. The What's New post says "new and existing backups" without qualifying the type. If automatic backups are your evacuation source, prove it with one generation first.

2. Make recovery declarative — CloudFormation can express the restore

Typing six steps during an incident is not realistic at any team size. This can be declarative.

AWS::FSx::Volume has a BackupId property ("the ID of the volume backup to use to create a new volume"). So the file system, the SVM, and the restore all fit in one template:

Parameters:
  CopiedBackupId:
    Type: String
    Description: ID of the backup in the destination Region

Resources:
  DrFileSystem:
    Type: AWS::FSx::FileSystem
    Properties:
      FileSystemType: ONTAP
      StorageCapacity: 1024
      StorageType: SSD
      SubnetIds: [!Ref DrSubnetId]
      SecurityGroupIds: [!Ref DrSecurityGroupId]
      OntapConfiguration:
        DeploymentType: SINGLE_AZ_1
        ThroughputCapacity: 128
        PreferredSubnetId: !Ref DrSubnetId

  DrSvm:
    Type: AWS::FSx::StorageVirtualMachine
    Properties:
      FileSystemId: !Ref DrFileSystem
      Name: dr_svm
      RootVolumeSecurityStyle: UNIX

  DrRestoredVolume:
    Type: AWS::FSx::Volume
    Properties:
      Name: dr_restored
      VolumeType: ONTAP
      BackupId: !Ref CopiedBackupId
      OntapConfiguration:
        StorageVirtualMachineId: !Ref DrSvm
        JunctionPath: /dr_restored
        SizeInBytes: 1073741824
        OntapVolumeType: RW
        TieringPolicy:
          Name: NONE

Keeping this template undeployed in the destination Region is the cold-standby-IaC shape. Zero cost while idle, one deploy at recovery, and no step to forget.

I have not deployed this template — I ran the same sequence through the CLI. Deploy it once and time it before it becomes your runbook. An undeployed recovery template is a draft, not a procedure.

3. RTO breakdown — "20 minutes" is one line of it

I measured file system creation (20 min) and the restore (13 min). Your RTO is not the sum of those two. Fill this in for your environment.

Step	My measurement	Yours
Decision to recover (detection to action)	not measured
Destination file system creation	~20 min
SVM creation	a few minutes
AD join (if using SMB)	not performed
Restore to `CREATED`	13 m 21 s (9.4 MiB)
Export policy / share reconfiguration	not measured
Client remount, DNS cutover	not measured
Business-side validation	not measured

If you use SMB, the AD join is in there. Confirm destination-Region reachability to your domain controllers while nothing is wrong. Discovering that mid-incident is a dead end.

Restore time gets longer with capacity

The 13 m 21 s and 16 m 16 s I measured were on a 9.4 MiB volume. Treat those as reference points only. Restore time for FSx for ONTAP backup, through either path, scales with capacity.

AWS publishes the rates (Backup and restore performance).

Operation	Mostly large files	Mostly small files
Backup creation	750 MBps	100 MBps
Restore	250 MBps	100 MBps

Two conditions come attached to those numbers, and both push an estimate to the longer side.

They are values "across several concurrent restores" — that is AWS's wording. Nothing says a single restore runs at that rate.
Backup and restore are background work and use only unused throughput capacity. Throughput capacity is therefore the ceiling. A 128 MBps file system will not reach 250 MBps.

Applying both, the calculation is min(published rate, throughput capacity). These are floors, on the optimistic assumption that client I/O is zero.

Logical data	128 MBps / large files	256 MBps+ / large files	Mostly small files (any throughput)
1 TB	2 h 10 m	1 h 6 m	2 h 46 m
5 TB	10 h 51 m	5 h 33 m	13 h 53 m
10 TB	21 h 42 m	11 h 6 m	27 h 46 m
20 TB	43 h 24 m	22 h 13 m	55 h 33 m
50 TB	108 h 30 m	55 h 33 m	138 h 53 m

Restoring 10 TB on the minimum configuration (128 MBps) is 22 hours at the floor. Mostly small files is 28 hours, and that one does not shrink by adding throughput, because the published 100 MBps binds first. At 50 TB it passes four days.

I could not find a measured figure at 10 TB or above in any public source. I searched AWS blogs, re:Post, the Knowledge Center, and the documentation, and found no article that publishes a measured restore duration with the capacity stated. The table above is calculated from AWS published rates — it is not something anyone measured. If you need a meaningful number for your own environment, the only way is to measure once at production-equivalent capacity and file mix.

And this can overturn "generation 1 is enough"

The section below says gen 1 is enough for a DR standby. At scale that reverses.

	Gen 1	Gen 2
Monthly at minimum throughput	$115.97 (128 MBps)	$772.99 (384 MBps)
Reads while restoring	No. You wait for completion	Yes, once metadata has loaded
Time to service at 10 TB	21 h 42 m (floor)	6–46 min (after 1–7% of metadata loads; the rest loads in the background)

Metadata is documented as 1–7% of the backup data, depending on file size — the smaller the files, the larger the share. At 10 TB, loading just that 1–7% first lets clients mount and start reading. A first access to a block that has not loaded yet costs tens of milliseconds, and it is cached on SSD after that.

So $657 a month of difference can turn a 21-hour outage into something closer to 30 minutes. If you hold 10 TB or more and your RTO is constrained in hours, choosing gen 2 for the standby too is a defensible call. Gen 1 is enough when either the capacity is small or the RTO can absorb waiting for the restore to finish.

A few TB or less, or an RTO in days → gen 1
10 TB or more with an RTO in hours → gen 2's reads-while-restoring earn their keep
Both are a longer RTO than SnapMirror (neither reaches a shape where the destination volume already exists)

4. Application consistency — this does not replace a database backup

A backup is a point-in-time copy of a volume. It does not quiesce an application.

Workload	What you need
File servers, shared storage	A volume point-in-time is usually enough
Databases (Oracle, SQL Server, …)	A database-level backup, or quiescing before the backup
Anything with constantly open writes	Application-level consistency checks after restore

Reading "we copy volume backups to another Region, so DR is covered" for a database workload can end with a volume that restores and a database that will not start.

5. Concurrency limits at scale

The limits are on operations in flight, not on how much you can keep.

Limit	Value	Consequence
Copies in flight (per volume, per destination Region, per KMS key)	5	Parallelizing one volume's generations stops at 5
Copies in flight (per account)	1,000	Firing off hundreds or thousands of volumes at once exceeds it. Excess requests are rejected
Backups per volume	4,091	Work your generation plan backwards from here

In a multi-tenant design with one volume per tenant, tenant count becomes your concurrency. Batch the submissions and retry rejections with exponential backoff.

6. Monitoring and control

Copies are asynchronous. When copy-backup returns, nothing has finished yet.

Decide	Options
Where failures are detected	With AWS Backup you can follow job state. For `CopyBackup`, polling `describe-backups` is the dependable route. Whether CloudWatch or EventBridge can carry this is unverified
How success is evidenced	Store the `describe-backups` response (`BackupId`, `SourceBackupId`, `SourceBackupRegion`, `CreationTime`)
Who approves a restore	A restore creates a new volume and destroys nothing, but it consumes destination SSD and starts billing. Put it in an approval flow where that is expected
Which destinations are allowed	Enforce it in IAM or an SCP, not in a written rule

One note on tags. --copy-tags carries source tags to the destination. If your tags name systems or departments, that naming appears in the isolated account too. Depending on why you are isolating, re-tagging may be the better choice.

Cost

The $0.55 I spent was a test bill. What is useful in your environment is a monthly figure, so this builds one up from unit rates.

Unit rates — queried from the AWS Price List API

The rates below were queried from the AWS Price List API on 2026-08-29 (effective 2026-07-01). Tokyo and Osaka were identical on all four of the items that appear in both.

Charged item	Tokyo / Osaka rate	Nature of the charge
SSD storage (Single-AZ)	$0.150 / GB-month	provisioned
SSD storage (Multi-AZ)	$0.300 / GB-month	provisioned
Capacity pool Standard (Single-AZ)	$0.0238 / GB-month	consumed
Capacity pool Standard (Multi-AZ)	$0.0476 / GB-month	consumed
Backup storage	$0.050 / GB-month	consumed (incremental)
Throughput capacity (Single-AZ, gen 1)	$0.906 / MBps-month	provisioned
Throughput capacity (Single-AZ, gen 2)	$2.013 / MBps-month	provisioned
Capacity pool read requests	$0.00037 / 1,000 requests	consumed
Capacity pool write requests	$0.0047 / 1,000 requests	consumed

Going through AWS Backup does not change the backup storage rate. FSx for ONTAP is not a resource type that AWS Backup fully manages, so the storage charge appears on the FSx for ONTAP side rather than on AWS Backup (Metering, costs, and billing for AWS Backup). The rate is therefore $0.050 / GB-month on the native path and on the AWS Backup path alike. Only when you use a logically air-gapped vault (LAG vault) does all of the storage and transfer land on the AWS Backup side.

FSx for ONTAP backup is not the cheap option

$0.050 / GB-month is 2.1× the Single-AZ capacity pool ($0.0238). Put the other way round, the capacity pool is 47.6% of backup storage — less than half.

AWS makes the same comparison. AWS Prescriptive Guidance puts fully managed backup at $0.05 / GB-month, against $0.03221 / GB-month for SnapVault aimed at a capacity pool at an SSD-to-pool ratio of 1:10 (Choose the right SMB file storage, US Region example).

That 47.6% is Single-AZ only. A Multi-AZ capacity pool is $0.0476, which is 95.2% of backup storage, and the advantage all but disappears. Carrying "the capacity pool is less than half" into a Multi-AZ estimate will put your numbers wrong.

Running one continuously has a floor

Read the rate table straight and it seems to say "if the capacity pool is cheap, then keeping a destination file system running is cheap too". It does not work out that way. The minimums on the provisioned charges set the floor for anything kept running.

Constraint	Value	Source
Minimum SSD capacity	1,024 GiB per HA pair	Quotas
Minimum throughput capacity (gen 1)	128 MBps	same
Minimum throughput capacity (gen 2, 1 HA pair)	384 MBps	same

Monthly cost of a Single-AZ minimum configuration:

Generation	Components	Monthly at minimum
Gen 1 (128 MBps)	1,024 GiB × $0.150 + 128 MBps × $0.906	$269.57
Gen 2 (384 MBps)	1,024 GiB × $0.150 + 384 MBps × $2.013	$926.59

The same "minimum configuration" differs by 3.4× between generations, because gen 2's minimum throughput is 384 MBps and its per-MBps rate is 2.2× gen 1. For backup and a DR standby, gen 1 is the choice (below).

All-tiering does not remove this floor either. Metadata always stays on SSD, so the 1,024 GiB SSD minimum remains provisioned.

It inverts with scale

Assumptions: 30 daily backup generations, 2% daily change rate, no dedupe or compression benefit assumed, Single-AZ gen 1 at the 128 MBps minimum, the Osaka rates above. The all-SSD column provisions capacity to the recommended 80% SSD utilisation.

Logical data	Backup copies (no destination FS)	Standing + all-tiering	Standing + all SSD
1 TB	$80	$293	$303
2 TB	$160	$317	$491
5 TB	$400	$389	$1,053
10 TB	$800	$508	$1,991
20 TB	$1,600	$746	$3,866
50 TB	$4,000	$1,460	$9,491

The crossover is around 4.8 TB. That is the $269.57 floor divided by the per-GB difference: 1.6 generations of backup at $0.050 ($0.080) against the capacity pool at $0.0238, a gap of $0.0562.

This is what backs up "we keep one running for RTO and RPO". Past 4.8 TB a standing destination file system is lower monthly, and a minutes-level RPO, a single-digit-minute RTO, and a failback path come with it. Improving recoverability is not an increase in monthly cost at that size. At 1 TB it is the other way round, and the gap between $80 and $293 has to be explained as a difference in requirements.

Why a DR standby takes generation 1

For backup and a DR standby, gen 1 is enough. The reason is the throughput increments.

	Gen 1	Gen 2 (1 HA pair)
Selectable throughput	128 / 256 / 512 / 1,024 / 2,048 MBps (Tokyo and Osaka cap at 2,048)	384 / 768 / 1,536 / 3,072 / 6,144 MBps
Monthly at minimum throughput	128 MBps × $0.906 = $115.97	384 MBps × $2.013 = $772.99

A standby's throughput is sized by SnapMirror transfer volume and the reads right after a failover, not by production load. Gen 2 still floors at 384 MBps for that use, which is 6.7× on the throughput line alone. With gen 1 you start at 128 MBps and move up to 256, then 512, when you need it.

Deployment type cannot be changed after creation (documented). Going from gen 1 (SINGLE_AZ_1) to gen 2 (SINGLE_AZ_2) means a restore from backup, or a migration over SnapMirror or AWS DataSync. Decide the generation before you build the standby.

Total monthly by throughput

The 4.8 TB crossover above assumes the 128 MBps minimum. Raise throughput and the crossover moves up with it. Here are the combinations laid out.

Assumptions: Single-AZ gen 1, all-tiering, SSD at the 1,024 GiB minimum, Tokyo / Osaka rates. The backup-copy column is 30 daily generations at a 2% change rate. Capacity pool request charges are not included.

Logical data	128 MBps	256 MBps	512 MBps	1,024 MBps	2,048 MBps	Backup copies
1 TB	$293	$409	$641	$1,105	$2,033	$80
5 TB	$389	$505	$736	$1,200	$2,128	$400
10 TB	$508	$624	$855	$1,319	$2,247	$800
20 TB	$746	$862	$1,093	$1,557	$2,485	$1,600
50 TB	$1,460	$1,576	$1,807	$2,271	$3,199	$4,000

Bold is cheaper than holding backup copies. Restated as the highest throughput that still comes in under the backup-copy figure:

Logical data	Backup copies / month	Highest throughput under that
1–2 TB	$80–$160	None. Even 128 MBps standing costs more
5 TB	$400	128 MBps
10 TB	$800	256 MBps
20 TB	$1,600	1,024 MBps
50 TB	$4,000	2,048 MBps (the gen 1 ceiling in Tokyo and Osaka)

At 50 TB you can provision the largest throughput gen 1 offers in-Region and still come in cheaper than holding backup copies. At 1–2 TB backup copies are cheaper at every throughput, and there the difference is a requirements conversation.

What the "zero while idle" of backup copies buys

With no destination file system in normal operation, there is no throughput capacity charge and no SSD capacity charge. That is a clear advantage on the monthly figure. The work it defers, though, reappears at the moment of failover, as urgent work.

And the unit of a backup is the volume, so a restore always produces a new volume. There is no path that overwrites an existing volume in place, on the CopyBackup route or the AWS Backup route. That is what makes steps 4 to 6 below unavoidable rather than optional: a new volume means a new junction path if the original still exists, and export policies, SMB shares, quotas, and the snapshot policy do not travel with it. The volume size default is a separate trap — it comes up as 1 TiB rather than the source's size, covered further up.

In an incident that same constraint cuts the other way and becomes an advantage: you can preserve a compromised volume exactly as it stands and restore beside it, so the original is still there to investigate.

Here is what a cold start to restored service actually needed, in the range I measured.

#	Step	Measured / state
1	Create the file system in the destination Region	20–22 min (measured, `SINGLE_AZ_1`, 128 MBps)
2	Create the SVM	a few minutes
3	Restore the volume from backup	13 m 21 s (`CopyBackup` path) / 16 m 16 s (AWS Backup path), both at 9 MB
4	Recreate export policies and SMB shares	Only the volume is restored. Share configuration does not come with it
5	AD-join the SVM if you use SMB	It is a new SVM, so it joins again
6	Repoint clients	The new file system has a different DNS name and different IPs. Either move a CNAME or remount on the client

Steps 1 and 3 finish if you wait. Steps 4 to 6 are a person deciding and typing. And they are needed mid-incident, not on a quiet afternoon. What you saved monthly is traded against doing that work under pressure.

In a SnapMirror shape, 1 to 3 do not exist, and the destination SVM in 5 is already joined. What is left is 6 and the break-and-promote, so the number of steps is not comparable to begin with.

Steps 4 to 6 can be automated ahead of time. Declare the restore in CloudFormation, script vserver cifs create and the share creation, put a CNAME in front of DNS. But "can be" and "confirmed to work during an incident" are different claims. Run it through once while nothing is wrong.

The price difference is the requirement difference

Here is what sits inside the $80 and the $293 at 1 TB.

	Backup copies	Standing (SnapMirror shape)
RPO	60 min (the automatic backup minimum)	5 min
RTO	file system creation 20–22 min + restore 13–16 min (measured)	promote the volume
Failback	no path	yes
Destination file system in normal operation	not needed	needed

If the requirement is a minute-level RPO, backup copies are not a candidate. And at recovery time the backup-copy side creates a destination file system too, so the same floor starts applying from then on. The accurate reading is a design whose cost structure differs between normal operation and recovery.

Not included above

Capacity pool request charges: not in the tables above. The rates are known, but I have not measured the conversion from SnapMirror transferred bytes to request counts, so I cannot produce a number. With all-tiering every write goes to the capacity pool, so write requests ($0.0047 / 1,000) may not be negligible.
Inter-Region data transfer: treated differently depending on the path. Split out into the next section
Source-side backup storage (unchanged — you only added somewhere to put a copy)
KMS request charges if you use a CMK
SnapMirror inter-Region transfer (SnapMirror is not measured in this post)

Inter-Region data transfer is treated differently per path

AWS Backup copies have a transfer line item. Tokyo to Osaka is $0.09 / GB (Price List API, AWSBackup, APN1-APN3-CrossRegion-WarmBytes-FSx, retrieved 2026-08-29). The AWS Backup pricing page gives the same transfer rate for Resource Group 3, which includes Amazon FSx, on a standard vault and on a logically air-gapped vault alike.

For native CopyBackup, the reasonable thing is to estimate as though it applies. I started to write "probably does not apply", checked again, and withdrew it. Here is why.

The argument I withdrew: backups sit in AWS-managed S3 and never traverse the customer VPC, so they are not subject to inter-Region transfer charges the way EC2 is. That does not hold. EBS snapshots are equally in an AWS-managed area and equally do not traverse a VPC, and a cross-Region snapshot copy does incur AWS Data Transfer charges. The AWS Storage Blog walks through tracking that charge in Cost Explorer, and the usage type that appears there is USW2-USE1-AWS-Out-Byte, the service is "EC2 - Other", and the usage type group is Data Transfer - Region to Region (Out) (Effectively track AWS data transfer costs for cross-region Amazon EBS Snapshot Copy).

That also exposes a hole in how I first checked it. I based it on "there is no transfer item in the FSx for ONTAP price list", but inter-Region transfer is listed under AWSDataTransfer, not under the originating service. EBS is exactly that case: there is no "snapshot transfer" item in the EBS price list, and the charge happens anyway.

And AWSDataTransfer has a generic, service-agnostic inter-Region SKU.

Direction	Usage type	Rate
Tokyo → Osaka (out)	`APN1-APN3-AWS-Out-Bytes`	$0.09 / GB
Osaka → Tokyo (in)	`APN1-APN3-AWS-In-Bytes`	$0.00 / GB

(Price List API, AWSDataTransfer, effective 2026-06-01, retrieved 2026-08-29)

The Amazon FSx SKU on the AWS Backup side is also $0.09 / GB. The same transfer rate appearing in both places makes it natural to read them as the same underlying charge.

To summarise, this is as far as it can be taken right now.

Question	State
AWS Backup cross-Region copies incur transfer	Listed in the price list ($0.09 / GB, Tokyo → Osaka)
Native `CopyBackup` incurs transfer	Estimate as though it does. A generic $0.09 / GB SKU exists, and the same shape (EBS snapshot copy) is charged in practice. But I have not confirmed it on a bill — unverified
"It is not in the FSx for ONTAP price list, so it is free"	This inference is wrong. Transfer is listed under `AWSDataTransfer`

In an estimate, put in the first full copy's capacity × $0.09 / GB. After that it is incremental, so changed volume × $0.09 / GB. A 10 TB first full is $900 — not a monthly figure, but a one-off of real size. The shortest way to settle it is to copy one small volume and read the UsageType in Cost Explorer. I have not done that.

On which account is billed for transfer, two AWS statements disagree. FSx for ONTAP is not a resource type AWS Backup fully manages, so that discrepancy lands directly on this case.

Source	Statement
AWS Backup Developer Guide	For resource types that are not fully managed, transfer appears on the destination account
AWS Backup pricing page	Transfer is charged to the account sending the data (the copy source)

Which one is correct I have not been able to confirm. If cost attribution has to go into the design of a cross-account setup, copy once first and check which account it lands on in the actual bill.

The trade-off of all-tiering

Keeping a standby cheap with all-tiering means reads after a failover start from the capacity pool. All does not pull read blocks back onto SSD, so performance immediately after a cutover is not the same as production sized for SSD. It is a choice between the monthly cost while idle and the performance immediately after cutover.

Prioritise cost while idle → all
Need production-equivalent performance from the moment of cutover → provision SSD as in production (the "all SSD" column above)

When you build your own estimate, replace the change rate and the generation count with measurements from your environment. The 2% is an assumption, not something I measured. Rates are revised, too. The figures above were retrieved on 2026-08-29 and are effective 2026-07-01, so check the current values on the FSx for ONTAP pricing page and the AWS Backup pricing page before you decide.

As a ransomware layer

An isolated copy is a useful layer, but it lacks two properties people expect of it.

Expectation	Reality
Put it in an isolated account and it cannot be deleted	Copies can be deleted. The isolation holds only if the destination account's IAM or vault policy prevents deletion
A backup copy satisfies an immutability requirement	It does not. Preventing deletion during a retention period is the domain of WORM features (Object Lock, SnapLock, Tamperproof Snapshot, Vault Lock)

WORM features are irreversible. Once enabled, nothing is deletable until retention expires, and the blast radius is routinely wider than the resource you named. Get the retention value and the blast radius approved separately before enabling one. I made a file system undeletable for six months this way.

On the other hand, a restore always producing a new volume is an advantage during an incident. You can preserve the compromised volume for investigation and restore into a separate volume to resume operations, without overwriting the original.

Common errors and how to isolate them

Decide which layer stopped before you start debugging. Get the layer wrong and you spend the incident suspecting something that works.

Layer	Look at	What to check
1. Source	The backup	`Lifecycle` is `AVAILABLE`; volume type is `RW`; style is FlexVol
2. Permissions	IAM / KMS	`fsx:CopyBackup` present; source-Region backup reachable; destination key permissions for a CMK
3. Copy	`describe-backups` at the destination	`COPYING`, `AVAILABLE`, or `FAILED`; concurrency limits
4. Restore	The destination volume	`Lifecycle`; enough SSD (it pauses when short)
5. Access	The client	Export policy, security group, SVM NFS / SMB endpoint

Messages I actually saw:

Message	Layer	What to do
`Backup <id> is being copied, can't be deleted`	3	Wait for the copy. There is a short delay even after it completes
`Backup failed. Please delete the backup and try again.`	1	Names no cause. Suspect FlexGroup and take a FlexVol control
`Volume with type DP is not backupable.`	1	That is a SnapMirror destination. Back up at the source instead
Rejection from exceeding copies in flight	3	5 per volume per destination Region per KMS key, 1,000 per account. Wait and retry
`Unknown options: --ontap-configuration` (`delete-file-system`)	—	A CLI version difference. Read the response. This cost me 8 minutes of polling

Whether a failed copy leaves a partial backup at the destination is unverified — I did not induce a failure deliberately. If you automate this, detect FAILED and delete it (the backup created by my FlexGroup failure did remain as FAILED, and it deleted cleanly).

Where this sits alongside an existing backup product or SaaS

If you already run a data protection product or backup SaaS, this update is not a migration question. It is a question of which layer you give to what. There are three paths, and combining them is normal.

Path	Managed by	Another Region	Another account	Fits
A. FSx for ONTAP backups	FSx for ONTAP API / console	yes (new)	no	Volume-level evacuation with the fewest moving parts
B. AWS Backup	Backup plans	yes	yes (needs Organizations)	Policy control, one view across services
C. Existing data protection product / SaaS	That product's policy	Depends on the product	Depends on the product	File- or application-level restore, catalog search, fitting existing workflows

A concrete way to split them

File-level restores from the existing product; whole-volume evacuation from FSx for ONTAP backup copies. One split that often works, and the reason is restore granularity. A restore from an FSx for ONTAP backup produces one new volume, which is a lot of machinery to return a single file a user deleted. Conversely, pushing whole volumes to another Region through an existing product tends to grow the transfer volume and the catalog. Where your product already handles cross-Region evacuation efficiently, the opposite split is equally reasonable.

The decisive question for AWS Backup is whether you need another account. Evacuating to an isolated account is out of reach for the FSx for ONTAP API alone; it needs AWS Backup and AWS Organizations. If you only need another Region, the FSx for ONTAP API is fewer moving parts.

What to ask, regardless of product

Ask along these axes rather than by product name:

Axis	What to establish
Unit of protection	Whole volume, file / directory, or application-consistent
Restore granularity	Can it return one file, or only a whole volume
Cross-Region / cross-account path	The product's own feature, via AWS Backup, or your own copy
What the restore target requires	Does a file system need to exist first
Where retention policy lives	The product or AWS. Is it managed in two places
Monitoring and alerting	Where failures surface, and whether that reaches your existing observability
Where the cost lands	Product licensing, AWS usage, or both

"Managed in two places" is where the accidents are. If FSx for ONTAP automatic backups and the product's policy each hold generations independently, changing retention on one side leaves the other holding an unintended number. The ceiling is 4,091 backups per volume, so plan backwards from it.

Storage is also billed twice. Holding the same data in both the product's repository and FSx for ONTAP backups charges for both sets of generations. If you combine them, pick a primary and make the generation counts differ:

Shape	Primary	Secondary
File-level restore matters most	Existing product, long retention, many generations	FSx for ONTAP backups short and few (recent whole-volume evacuation)
Cross-Region evacuation matters most	FSx for ONTAP backup copies, many generations	Existing product limited to in-Region file restores

Neither is correct in the abstract. It comes down to which granularity 90% of your restore requests arrive in. Counting your past restore tickets is the most reliable way to find out.

About monitoring

My verification polled describe-backups. I have not verified a path for detecting copy completion through CloudWatch metrics or EventBridge events. If you plan to wire this into an existing observability stack, that is a gap you will need to close yourself.

Choosing between this and SnapMirror

Neither is better. The AWS documentation states RPO and RTO figures for both, and they protect different things.

For context: cross-Region disaster recovery for FSx for ONTAP has been buildable with SnapMirror for years — a destination file system, cluster peering, asynchronous replication. Plenty of production environments run exactly that, and backup copies do not displace it. What got added is the option of not holding the destination while nothing is wrong.

Dimension	Backup copies	SnapMirror
Suits	Offline copies for compliance or isolation	Availability in a second Region
RPO guidance	60 minutes	as low as 5 minutes
RTO guidance	minutes to hours (size dependent)	single-digit minutes
Needed while idle	No destination file system	A running destination file system
Shape of recovery	Restore as a new volume. The original does not come back	Break the relationship and promote the destination
Path back to production	None. Continue on the recovery side, or copy back in the other direction	Delete the relationship and `snapmirror resync` from the DR side
Trade-off	Destination file system creation lands on RTO (20 min measured); RPO is as coarse as the backup interval. The unit of backup and restore is the volume, and with AWS Backup / FSx backups a restore always lands in a newly created volume — there is no restore-in-place path, so junction paths, share configuration and where clients point all have to be rebuilt	You pay for destination capacity and throughput continuously; needs cluster peering and does not work through NAT

In operation, the asymmetry in "path back to production" is what bites. SnapMirror can be re-established in reverse to fail back. Backup copies only restore, and what you get is a new volume. If a documented failback procedure is a requirement, backup copies do not substitute for it.

One caveat on snapmirror resync: user-created snapshots are not replicated by a resync. The preserve parameter is available on XDP relationships.

Running both

This is a natural combination: SnapMirror for availability, backup copies for isolated retention. One catch — a SnapMirror destination is a DP volume and cannot be backed up, so "back up the replica and evacuate that" does not work. Take backups at the source and copy those.

Added (September 2026): read-only does not mean inert. Verified on ONTAP 9.18.1P5 in ap-northeast-1: a SnapMirror destination can be served read-only through an FSx for ONTAP S3 Access Point while replication is still running — no break, no clone. What gates the attachment is the junction path, not the volume type, and setting that junction path is an ONTAP-side operation: the Amazon FSx API refuses it on a DP volume, and UpdateVolume refuses it silently, returning HTTP 200 with the volume unchanged. After a transfer, new data was readable through the same access point in 15 seconds. Budget minutes to tens of minutes for the FSx control plane to report a newly mounted volume as attachable. Design considerations.

Regional DR within one country

Copying to Osaka keeps data inside Japan while placing it in a second Region.

Consideration	Detail
Where the data sits	Both Tokyo and Osaka are in-country
Idle cost	Backup storage only ($0.050/GB-month in Osaka, rate effective 2026-07-01)
Needed at recovery	A file system and SVM in Osaka. Creation time lands on RTO (20 min measured)
Generation availability	`SINGLE_AZ_1` was creatable in Osaka (measured 2026-08-28). This differs by Region
Trade-off	Coarser RPO and RTO than SnapMirror. A minute-level RPO requirement means keeping the destination running

How you satisfy a data residency requirement is a design question, not a legal or regulatory determination. Interpreting regulatory requirements belongs to your compliance function and the relevant authority. This post does not substitute for that.

A staged rollout

Do not start on a production volume. The order I would suggest:

Run the whole path on a small test volume (~1 GiB) — copy, destination file system, restore, content check, teardown. This is where the holes in your procedure show up
Time it including teardown. Destination file system creation lands on RTO, so this is the number you most want
Measure once at production data volume. My 9.4 MiB figures are dominated by fixed overhead and are not evidence for your environment
Check whether the second copy is incremental by comparing transfer volume and duration
Run periodic copies on a non-production volume and build the failure-detection path. This is where the real work is
Go to production, keeping the existing backup path running in parallel
Put restore drills into operations. Without them your RTO is an estimate

Do not go to production without step 5. Copies are asynchronous, and when one fails your command has already returned success.

A verification plan template

Useful to have on one page if a team is involved:

Item	What goes in it	Mine
Question being answered	"Does cross-Region evacuation work without a standing destination?"	same
Success criteria	Content matches after restore, and every line of the RTO breakdown is filled	Hash match achieved. RTO breakdown incomplete (AD and remount unmeasured)
Time required	Half a day for the small path, a day at production volume	~1.5 hours including teardown
Cost ceiling	Driven by destination file system runtime. Set it first	Capped at 3 hours / $2; actual $0.55
Stakeholders	Storage operations, the business (accepting the restored data), audit / control, networking (destination VPC, AD reachability)	—
Go criteria	Measured RTO under target / incrementality confirmed / failure detection working	Not met (incrementality and detection unverified)
No-Go criteria	RPO needed in minutes / failback required / FlexGroup in scope	—

By that standard I cannot call my own run a Go. Incrementality and failure detection are unverified, and the RTO breakdown does not include AD. What this post establishes is that the path works.

A recovery runbook skeleton

Naming the decision points keeps the day itself calm:

#	Decision or action	Branch
1	Is recovery needed (Region-level failure, or data loss)?	Data loss alone is an in-Region restore. You do not need the other Region
2	Which generation?	Check `CreationTime` on the copies. This is also where you see whether copying kept up
3	Create the destination file system	`deploy` the template, or steps 4–5 by hand
4	SMB in play?	Then AD join is required. Put the reachability check here
5	Restore	Wait for `Lifecycle: CREATED`. `OntapVolumeType` reads `DP` while restoring
6	Business validation	Data match and application startup. Databases need a consistency check
7	Return to the original Region?	Backup copies have no failback path. Either continue on the recovery side, or copy back in the other direction

Try it now

The same steps with only the values to substitute:

# Change these five lines for your environment
export SRC_REGION=ap-northeast-1
export DST_REGION=ap-northeast-3
export SRC_VOLUME_ID=fsvol-0123456789abcdef0
export SRC_BACKUP_ID=backup-0123456789abcdef0
export DST_BACKUP_ID=   # fill in after the copy

# 1) create a backup
aws fsx create-backup --region "$SRC_REGION" --volume-id "$SRC_VOLUME_ID" \
  --query 'Backup.BackupId' --output text

# 2) wait for AVAILABLE
aws fsx describe-backups --region "$SRC_REGION" --backup-ids "$SRC_BACKUP_ID" \
  --query 'Backups[0].Lifecycle' --output text

# 3) copy to another Region (--region is the destination, --source-region is the origin)
aws fsx copy-backup --region "$DST_REGION" \
  --source-backup-id "$SRC_BACKUP_ID" --source-region "$SRC_REGION" --copy-tags \
  --query 'Backup.BackupId' --output text

# 4) confirm the copy
aws fsx describe-backups --region "$DST_REGION" --backup-ids "$DST_BACKUP_ID" \
  --query 'Backups[0].{Life:Lifecycle,Src:SourceBackupId,SrcRegion:SourceBackupRegion}' --output table

If step 4 shows SourceBackupId and SourceBackupRegion, the copy is real. That is the whole day-to-day preparation.

Wrapping up

What this update removed is what you keep running day to day. Putting data in a second Region no longer requires a file system there.

You pay for that at recovery time instead — file system creation (20 minutes measured) plus an RPO as coarse as your backup interval. If minute-level RPO and a failback procedure are requirements, this does not replace SnapMirror. If isolated retention is the requirement, you now have an option that needs nothing standing by.

Three things I plan to do next:

Measure cross-account copies. The AWS Backup cross-Region copy and restore are measured now; cross-account needs AWS Organizations and remains documentation only
Measure incrementality at production data volume. At 9.4 MiB it disappears into fixed overhead
Establish a failure-detection path. Whether CloudWatch or EventBridge can carry copy state is unverified

The most useful thing I learned was smaller than the feature: I believed DP was a permanent property until I tried writing to the volume, and I believed storage efficiency was not carried over until I ran the same restore through the console. Two conclusions drawn from a single observation on a single interface, both wrong.

Thanks for reading.

Resources

AWS What's New: FSx for NetApp ONTAP now supports copying backups across AWS Regions and accounts
AWS What's New: AWS Backup adds cross-Region and cross-account backup support
AWS: Copying backups
AWS: Copying backups within the same AWS account — the backup-copy vs SnapMirror guidance and the RPO / RTO figures
AWS: Protecting your data with volume backups — restore target confined to the backup's Region, SSD-first restores, background priority
AWS: Restoring a backup to a new volume
AWS: CreateVolumeFromBackup
AWS CloudFormation: AWS::FSx::Volume — the BackupId property
AWS Backup: Creating backup copies across AWS Regions
AWS Backup: Creating backup copies across AWS accounts
AWS Backup: Encryption for backups in AWS Backup (why AWS managed keys cannot do a cross-account copy)
AWS Backup: Feature availability (the per-Region exceptions)
Cross-Region and Cross-Account Backup for Amazon FSx for NetApp ONTAP (NetApp Community: the cross-account walkthrough, four architecture patterns, clean room and restore testing)
AWS: Quotas (FSx for ONTAP)

Every resource used for this test has been deleted. The figures are measurements from one specific environment and configuration, and will differ with data volume and setup.

Using Databricks FILE type against data on NAS: what works today with FSx for ONTAP S3 Access Points

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Wed, 12 Aug 2026 15:07:29 +0000

A follow-up to Part 2: Databricks and FSx for ONTAP S3 Access Points in the "FSx for ONTAP S3 Access Points x Lakehouse Deep Dive" series. Some results changed since Part 2 — if you read that one, see "Why reads fail" below.

TL;DR

I tested whether images and PDFs already sitting on FSx for ONTAP can be referenced from
Databricks without copying them into the lakehouse. The result is: registration works, reads
do not.

If you only need the zero-copy verdict, read "Connecting to an FSx for ONTAP S3 Access Point"
and "Design tips" below. If you want the behaviour of the FILE type itself, start from the
top.

Databricks shipped FILE as a beta column type: a reference to an unstructured file held as a
single column in a Delta table. The value is a struct carrying uri / offset / size /
content_type / checksum. Because it holds the reference and the metadata as a struct, you
can pass it straight from a table into an AI function such as ai_parse_document. There are
two variants — FILE EXTERNAL, which holds only the reference, and FILE MANAGED, which
copies the bytes into Databricks.

I tested this against files on Amazon FSx for NetApp ONTAP, reached through an FSx for ONTAP S3
Access Point — the feature that exposes an ONTAP volume over an S3-compatible API.

Here is what I found:

Registration against an FSx for ONTAP S3 AP succeeds; reads do not. The cause sits in the session policy attached to the temporary credentials Unity Catalog vends, and there is no way to work around it from the user side
The FILE type itself works. But you have to choose between FILE EXTERNAL and FILE MANAGED before you ingest
If your design puts metadata in object tags, tag values are effectively ASCII. Most CJK strings are rejected while a minority are accepted. AWS Support has escalated that split to the service team as a potential defect
The operator surface is inconsistent. GROUP BY and DISTINCT are accepted; = and ORDER BY are rejected

Below: what is established for each of those four, and how to turn it into design decisions.
The templates and scripts are in the repository, so you can re-run the same tests in your own
account.

Repository: Yoshiki0705/fsxn-lakehouse-integrations

Test environment

Item	Value
Measured	2026-08-12
Storage	One FSx for ONTAP file system
Access Point	One, INTERNET origin, UNIX root identity
Workspace	One non-trial workspace, purpose-built, same account and region
Compute	Serverless SQL warehouse, and a classic DBR 18.2 cluster
Control	A S3 general bucket in the same account, running the same operations alongside
Method	SQL Statement Execution API, and boto3 from my workstation

Running the S3 general bucket control alongside matters. Looking only at the Access Point side, you
cannot separate a defect in your own environment from behaviour specific to the Access Point.
Except where stated otherwise, every conclusion below rests on the same operation succeeding
against S3 general bucket.

Note that the object tag results depend on the ONTAP tag validation implementation, so a
different ONTAP version may behave differently. The FSx console and describe-file-systems do
not expose the version; use the ONTAP REST API (GET /api/cluster?fields=version) to check
yours.

What the FILE type can and cannot do

First, the behaviour of the type itself. There is prior work on this feature from Databricks
Japan: パス文字列でもバイナリでもない。DatabricksのFILE型を試す
by taka_yayoi. Every operational caveat it lists reproduced in my environment, so I have
tabulated those first. (What follows is my paraphrase and my own measurements, not a
translation.)

Some terms first. FILE MANAGED copies bytes into a Databricks-side location called a
FileSpace. list_files enumerates files on a Volume and returns FILE values; create_file
constructs a FILE value explicitly. _object_metadata reads an object's tags and user
metadata.

Behaviour	Measured	Design implication
FileSpace on the same volume as the source	`CREATE TABLE` succeeds; `INSERT` fails with `Cannot get file metadata under managed storage`	Give the FileSpace its own volume
`checksum` on FILE values from `list_files`	Null on every row	Not usable for integrity checks
`checksum` under `FILE MANAGED`	Populated with `ETAG:"…"`, matching the `etag` from `_object_metadata` on S3 general bucket exactly as a string	Comes from the object store, so it is usable for reconciliation
Filenames under `FILE MANAGED`	Replaced with an opaque UUID; the extension does not survive either	Copy the name to a column before ingesting
Format of `uri`	Prefixed with `dbfs:`	Normalise before writing string comparisons
Serverless notebooks	Unsupported	Use the SQL Statement Execution API against a serverless SQL warehouse
Automatic garbage collection in beta	Does not run	The FileSpace grows on every re-run. Plan for manual deletion
The Previews page toggle	Used `FILE EXTERNAL` / `FILE MANAGED` without touching it (confirmed on 2 workspaces)	May not be required. Do not assume either way
`content_type` from `create_file`	`binary/octet-stream` for both Japanese and ASCII	Pass it explicitly if anything downstream branches on it

What this tells us

The difference between FILE EXTERNAL and FILE MANAGED is not only whether bytes are copied.
Whether checksum is populated and whether the filename survives both change. Reversing the
decision later means re-ingesting, so decide before you ingest.

On content_type, I got binary/octet-stream for both Japanese and ASCII input. For the same
files, list_files reported text/plain, so the two paths infer differently. If anything
downstream branches on content_type, pass it explicitly.

The state of operator support

Both the documentation and the prior article state that FILE columns cannot be used in grouping
expressions. Testing it, support splits more finely than that.

Operation	Result
`GROUP BY file`	accepted
`SELECT DISTINCT file`	accepted
`GROUP BY file.uri`	accepted (the documented approach)
`d.file = r.attachment`	rejected — The `=` does not support ordering on type "FILE"
`ORDER BY file`	rejected — The `sortorder` does not support ordering on type "FILE EXTERNAL"

What this tells us

GROUP BY and DISTINCT both need equality semantics to decide group membership. Both are
accepted while the equality operator itself is explicitly rejected. Since the error names the
type's operator surface, this does not look like row-count-dependent behaviour.

That said, this was confirmed on two rows. I cannot rule out that grouping degrades to identity
comparison rather than value equality. In practice, use GROUP BY file.uri and do not depend on
GROUP BY file being accepted. This is beta, so the behaviour may change.

Connecting to an FSx for ONTAP S3 Access Point

This is the main question. The answer is "registration works, reads do not", and the cause is
not a lack of Access Point support.

The four things registration needs

Item	Required form	Symptom when wrong
External ID in the trust policy	The Databricks account UUID	`403 Forbidden` from the storage provider. The metastore ID and the workspace ID do not work
Self-assume in the trust policy	Account root as the principal, with an `aws:PrincipalArn` condition naming the role ARN	Naming the role as its own principal fails at creation with `Invalid principal`, because IAM validates that the principal exists. Unity Catalog requires the role to be able to assume itself
Resource in the IAM permission policy	The access point ARN (`arn:aws:s3:<region>:<account>:accesspoint/<name>` and `.../object/*`)	`AccessDeniedException`. The alias-as-bucket-name form alone does not work
External location URL	The alias form (`s3://<alias>/`)	An ARN-style URL is rejected immediately with `url does not specify a valid bucket name`

What this tells us

There is an asymmetry worth committing to memory: the same Access Point must be written as an
ARN in the IAM policy and as an alias in the external location URL. The AWS CLI works with the
alias form of the ARN, so verifying connectivity with the CLI first and then writing the IAM
policy will trip you up.

The aws:PrincipalArn approach works because that key resolves to the role ARN rather than the
assumed-role session ARN. Rather than trust my memory of the condition key, I deployed the
template, assumed the role, and assumed it again with those credentials to confirm.

With all four satisfied, CREATE EXTERNAL LOCATION succeeds with skip_validation=False —
meaning Unity Catalog's own validation passed. The external volume on top can be created too.

Why reads fail

After registration, every read path returns 403 or an authorisation error. The same on
serverless SQL and on a classic DBR 18.2 cluster.

This is the test that isolated the cause. Unity Catalog will vend the temporary credentials it
uses if you ask for them. I requested them for both paths and used them from my workstation,
with no Databricks compute or network involved.

Vended for	`ListObjectsV2`	`HeadObject`
S3 general bucket control path	success	success
FSx for ONTAP S3 AP path	AccessDenied	403

Same role, same session, same network; the only variable is which path the credentials were
scoped to. The error states the cause explicitly:

is not authorized to perform: s3:ListBucket on resource:
"arn:aws:s3:<region>:<account>:accesspoint/<name>"
because no session policy allows the s3:ListBucket action

What this tells us

because no session policy allows is the decisive part. What is named is the session policy,
not the role's permissions. By session policy I mean the down-scoped policy passed to
AssumeRole to constrain the role's permissions on a per-request basis.

The policy body is generated by Unity Catalog and is not readable by the user. But since adding
an access-point-ARN allowance to the role changes nothing, the session policy appears not to
mention the access point ARN resource form at all. Only the overlap between the session policy
and the role policy is effective, so additions on the role side have no effect.

There is no workaround on the user side. Unity Catalog generates the policy, so the fix has
to happen there. Emitting the access point ARN form in the session policy should resolve it,
though I do not know what implementation constraints apply.

Note also that validation at registration time (skip_validation=False) passes while reads
return 403. What that validation checks is not published, so I cannot account for the
difference.

What changed since Part 2

Part 2 of this series (measured 2026-05) reported that listing top-level files and reading an
explicit file did work through a UC external location. In this run (2026-08-12), read_files,
list_files, to_file and dbutils.fs.ls all returned 403; no read path partially succeeded.

That environment is gone, so I cannot account for the difference conclusively. The likely
explanation is that the earlier partial successes were observed on an instance-profile path
rather than through UC-vended credentials — Part 2 lists instance-profile direct access as a
separate approach that works but sits outside UC governance, so the paths may not have been
cleanly separated.

This run had a S3 general bucket control alongside and includes the vended-credential test outside
Databricks entirely, so take these results as the current ones. I will annotate the Part 2
table accordingly.

This test has prerequisites. Enable External Data Access on the metastore and grant
EXTERNAL USE SCHEMA and EXTERNAL USE LOCATION. Both are disabled by default. They are the
controls that allow credentials to be used externally, so they are worth understanding
independently of this issue.

A table to work through when you see a 403

Arranged so that anyone hitting the same 403 can check top to bottom. None of these is the
cause.

What to suspect	Finding
Unity Catalog volume privileges	The failing volume had `READ VOLUME`; the working control volume had no privileges at all
IAM role permissions	Assumed the same role from my workstation and listed successfully. HEAD on a real object returned 200
HEAD on a missing key returning 403 (killing the `_delta_log` probe)	The Access Point returned 404 for every shape I tried — same as S3 general bucket
`x-amz-expected-bucket-owner` being rejected	Accepted by both the Access Point and the control bucket
A different endpoint	Both Databricks and the AWS CLI use `<alias>.s3.<region>.amazonaws.com`, which CNAMEs to `s3-r-w.<region>.amazonaws.com`. Same host
Insufficient compute role permissions	`sts:GetCallerIdentity` raised `NoCredentialsError` on the driver. The cluster has no default AWS credentials; the compute role is not in the path
The S3 gateway endpoint	No change after disassociating the route. The failure reproduces outside the VPC with vended credentials, so this path is not the problem

One note on the last row. A Databricks-managed VPC creates an S3 gateway endpoint at workspace
creation, and the private route tables point the S3 managed prefix list at it. That was not the
cause here, but for designs that reach an S3 AP from inside a VPC, check first whether the
alias resolves into the prefix list's range. I did not reconcile the resolved CIDRs against the
prefix list contents in this environment.

What object tags can actually hold

If you want metadata to live with the file, object tags are one option: tag at write time, read
from the table side later. That path has constraints.

The following was measured without Databricks in the picture — boto3 from my workstation
straight against the FSx for ONTAP S3 Access Point, with the same operations running against
the S3 general bucket control bucket.

Start with what works. PutObjectTagging, GetObjectTagging and DeleteObjectTagging all
function, as do x-amz-meta-* headers and x-amz-tagging on the same PutObject as the data.
The limits (10 tags per object, 128 characters for a key, 256 for a value) are the same values
S3 general bucket documents; I did not measure the boundaries.

Reading through a second access point on the same volume returned tags written through the
first. Tag retention is not scoped per access point. For existing NAS assets shared across
several paths, that property works in your favour. But I did not observe where the tags are
physically stored, so visibility from NFS or SMB, and retention across SnapMirror, FlexClone
and Snapshot restore, remain unverified.

Two constraints change how you design.

An object overwrite clears tags and user metadata. Nothing errors. A pipeline that rewrites
a file has to re-apply them in the same PutObject. My understanding is that S3 general bucket behaves
the same way, since PutObject replaces the object, so treat this as a general S3 design point
rather than something specific to FSx for ONTAP. I did not run a side-by-side control for this
particular case.

Tag values are effectively ASCII. Every printable Latin-1 character I tried
(U+00A1–U+00FF) was accepted. Above U+0100, most are rejected with InvalidTag — but not all.
A minority of CJK strings are accepted, deterministically, 6 runs out of 6.

To find the unit of validation, I tested single characters:

Input	Result
U+5206	`InvalidTag`
U+985E	`InvalidTag`
U+5206 U+985E	accepted
U+6771	`InvalidTag`
U+4EAC	`InvalidTag`
U+6771 U+4EAC	`InvalidTag`

What this tells us

Two two-character strings, every constituent character rejected on its own, and the two strings
disagree with each other. Validity is determined by the complete byte sequence, not as a
function of the characters in it — so this is not a per-character allowlist.

For the two-character pairs I tested, results were identical forwards and backwards, and
identical as a tag key and as a tag value. I did not test longer strings or other scripts.

I raised this with AWS Support. After checking it against the S3 and FSx for ONTAP
documentation, they gave the view that the pattern does not match any intentional validation
they could identify, and escalated it to the service team as a potential defect in the tag
validation layer.

Their reply also quoted the documented character set: letters, whitespace, and
+ - = . _ : / @. That does not predict the split under either reading. If "letters" means
Unicode letters, every string I tested qualifies yet half are rejected. If it means ASCII
letters, all of them should be rejected yet half are accepted.

The design guidance is simple: keep tags ASCII and keep localised text in a column. Today
the situation is "some strings pass and some do not", which is more awkward to handle than a
uniform restriction. I will update the repository and this article when a determination
arrives.

Design tips

The same material, ordered the way you actually make the decisions.

FILE type

Decide between FILE EXTERNAL and FILE MANAGED before ingesting. Checksum and filename handling differ, and changing course later means re-ingesting
Give the FileSpace its own volume. On the same volume, INSERT fails — and because the error surfaces at ingest rather than at configuration time, diagnosis is slower
Copy the filename to its own column before ingesting into FILE MANAGED. It is replaced with a UUID and the extension does not survive
Do not rely on content_type inference; pass it explicitly
Group by file.uri. Do not depend on GROUP BY file being accepted
Automatic garbage collection does not run in beta. Build manual FileSpace deletion into operations

Unity Catalog and FSx for ONTAP S3 Access Points

The external ID in the trust policy is the Databricks account UUID. The metastore ID does not work
The trust policy needs self-assume: account root as the principal with an aws:PrincipalArn condition naming the role ARN
The IAM permission policy needs the access point ARN, including .../object/*. The alias-form ARN alone does not work
The external location URL takes the alias form. ARN-style URLs are rejected
Reads do not work today. If zero-copy is a requirement, evaluate AWS-native engines that authorise in the caller's own IAM context alongside it (this repository has walkthroughs under integrations/athena, integrations/glue, integrations/emr-spark, integrations/redshift-spectrum and integrations/lake-formation)

Object tags

Keep tags ASCII and keep localised text in a column
Re-apply tags in the same PutObject when you rewrite a file. An overwrite clears them
Tag retention is not scoped per access point, so the same tags are readable through several access points

How to test

Run a S3 general bucket control alongside. Without it you cannot separate a defect in your environment from Access Point behaviour, and the first 403 looks like the answer
If the platform exposes credential vending, use those credentials from your own machine and compare. Isolating the problem is much faster without the vendor's compute in the path
Verify teardown by reconciling against a baseline. NAT Gateways do not disappear when you delete the workspace

Run it yourself

All of the above is in the repository as templates and scripts: one template, one script, one
runbook.

Artefact	Role
`integrations/databricks/uc-storage-credential-role.yaml`	The IAM role the Unity Catalog storage credential assumes, plus a S3 general bucket control bucket
`shared/scripts/probe_uc_external_location.py`	Seeds identical object tags on both sides, registers both external locations with validation on, reads `_object_metadata` through each, prints a verdict
`docs/en/databricks-verification-runbook.md`	Prerequisites, where each parameter comes from, the verdict branches, dependency-ordered teardown, measured cost

# 1. Baseline first. This is what makes "teardown looks clean" verifiable
python3 shared/scripts/audit_databricks_workspace_footprint.py \
  --region <region> --save /tmp/baseline.json

# 2. The IAM role and control bucket
aws cloudformation deploy --region <region> \
  --stack-name fsxn-databricks-uc-credential \
  --template-file integrations/databricks/uc-storage-credential-role.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides file://cfn-params/databricks-uc-storage-credential.json

# 3. Compare. --control-bucket is deliberately mandatory
.venv/bin/python shared/scripts/probe_uc_external_location.py \
  --profile <your-profile> --role-arn <from stack output> \
  --ap-alias <alias>-ext-s3alias --ap-name <name> \
  --control-bucket <from stack output> --region <region> --vend-check

# 4. Tear down and reconcile against the baseline
python3 shared/scripts/audit_databricks_workspace_footprint.py \
  --region <region> --compare /tmp/baseline.json

The script prints one of four verdicts. The first is the important one.

Inconclusive — the S3 general bucket control could not be read either, so this run says nothing about the Access Point. Fix the environment first
Registration succeeded, reads denied — the result in this article
Registration and reads both succeeded — the platform changed. Please tell me which release
Registration itself failed — almost certainly a missing Access Point ARN in the IAM policy

One note on the parameter file: a _comment key in a CloudFormation parameter file makes the
CLI fail with Unknown parameter in Parameters[n]. The description of each parameter lives in
cfn-params/README.md instead, so fill in values with that open alongside.

If you are evaluating a different platform

The specific bug is Databricks'. The shape is not, and the shape is the part worth taking away
if you are looking at a different product.

Managed lakehouse platforms generally reach into your cloud account by assuming a role you
create, then constrain that role at runtime with a down-scoped session policy so that
credentials handed to a query cannot roam. That is good design.

But the policy has to name your storage. If the storage is addressed through an access point
while the policy names it in bucket-form ARNs, the two do not match and authorisation fails —
however correct your own role policy is.

So the question to put to a platform you are evaluating, before you commit to an architecture,
can be narrow and answerable:

When you vend temporary credentials for an external location, which resource ARN forms does
the session policy contain? Specifically, does it include the S3 access point ARN form in
addition to the bucket form?

And if the platform exposes credential vending, you can verify it without asking anyone:
request the credentials, use them from your own machine, and compare a native bucket path
against an access point path. That is what --vend-check above does, and it was the only test
in this exercise that produced a conclusion with no room for interpretation.

Cost and cleanup

I created a test workspace and destroyed it the same day. The workspace mode that uses your own
AWS account creates a VPC with a NAT Gateway. A NAT Gateway runs roughly 45 USD a month per
gateway (hourly charge only, excluding data processing, at the August 2026 Tokyo region rate).

These resources are created directly rather than through a CloudFormation stack, so deleting
the workspace does not remove them. This is the easiest place to make a foolish mistake during
cleanup.

I recorded a baseline before starting and reconciled afterwards. Available NAT Gateways back to
zero; VPC, both IAM roles and the bucket gone, with no unassociated Elastic IPs.

I turned that check into a script (shared/scripts/audit_databricks_workspace_footprint.py). In
a shared account a plain listing is not a verdict, because most of what it shows belongs to
someone else. Run it once before creating anything and once after teardown; it exits non-zero if
anything exists that was not in the baseline. Running it confirmed this teardown was clean and
also surfaced leftover roles from an earlier workspace. The cost breakdown is in the runbook with
measured figures.

Current state and open items

Established

CREATE STORAGE CREDENTIAL / CREATE EXTERNAL LOCATION / CREATE EXTERNAL VOLUME succeed against an FSx for ONTAP S3 AP alias, with Unity Catalog validation left on
Reads through it return 403, on both serverless SQL and a classic DBR 18.2 cluster
The same failure reproduces outside Databricks, using vended credentials from a workstation
_object_metadata on S3 general bucket returns object tags and user metadata correctly
Object tag acceptance is determined by the complete byte sequence, and within the range tested is neither order- nor position-dependent

Not established

VPC-origin Access Points, and Access Points with WINDOWS identity
Whether _object_metadata would read tags through an Access Point if the session policy were fixed
ai_parse_document over files served from an Access Point
Throughput and latency of these paths
Whether GROUP BY on a FILE column is correct at scale, or merely accepted
Where object tags are physically stored; visibility from NFS or SMB; retention across SnapMirror, FlexClone, Snapshot restore and FabricPool tiering
The boundary values for tag limits (10 tags, 128-character keys, 256-character values)

Reporting status

The object tag behaviour has been escalated through AWS Support to the FSx for ONTAP service
team and is awaiting a determination. What I asked for is not a timeline but the distinction:
a defect that will be fixed, or an undocumented restriction that gets written down. Either
answer lets me state the constraint accurately.

The session policy issue and the operator inconsistency go to Databricks when this article
publishes.

This repository carried an entry from May 2026 saying Unity Catalog external locations do not
support S3 Access Points. Registration does work; the entry has been corrected. If you read the
earlier version, please note the change.

Closing

The FILE type is a well-built mechanism for bringing unstructured data under table governance.
For referencing files on an FSx for ONTAP S3 Access Point with zero copies, though, it does not
get there today. Registration works, so you can try it, but reads fail, and because the cause
sits in the session policy on the vending side, users cannot fix it.

If zero-copy is the requirement, AWS-native engines that authorise in the caller's own IAM
context are the more straightforward path for now — in exchange for assembling table-level
governance yourself with Lake Formation. If Unity Catalog governance is the requirement, then
for the time being data placement has to be part of the design.

The object tag character set is awaiting a determination. When it moves, I will update both the
article and the repository.

I hope this is useful to someone.

See you next time.

Previous: Part 2: Databricks and FSx for ONTAP S3 Access Points
Next: Part 8: OpenSharing and FSx for ONTAP
Series index: What works, what does not, and why

About this article

This series is based on verification in the fsxn-lakehouse-integrations repository
Author: Yoshiki Fujiwara
Affiliation: NetApp G.K. Japan
Role: Senior Cloud Solutions Architect - AWS
Topic: Amazon FSx for NetApp ONTAP / Hybrid Cloud / Generative AI Data Architecture
Level: 200-300

All test environments have been deleted. The behaviours described are measurements from
2026-08-12 in one specific configuration and may change with platform updates. Beta features are
involved, so please check against current documentation before making production decisions.

S3 Burst on ONTAP Files — Collect via S3 API, Consume via FlexCache NFS/SMB

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 10 Aug 2026 01:06:17 +0000

Introduction

You have an S3 ingestion pipeline. But the consumers only see an NFS mount. Sound familiar?

You want to deliver driving logs to a HiL test bench, stage design jobs for an EDA toolchain, push rendering assets to production nodes, or feed genomic sequencer output to an HPC cluster. In each case the writer speaks S3 API in the cloud while the reader speaks NFS/SMB on physical hardware. Without a way to connect these directly, you end up adding a sync job in between — and with it comes latency, cost, and one more thing to monitor.

This structure spans industries: automotive, semiconductor, media/VFX, oil and gas, life sciences, manufacturing, remote work, and IoT (see the full table in When This Pattern Fits below).

This post introduces a pattern that removes that sync job. Write to an Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP) S3 Access Point, and the data appears on a FlexCache NFS mount. No separate copy job and no scheduled replication between the two. FlexCache pulls the data that gets read from the origin and holds it on the cache side, so what this removes is the sync job you would otherwise own — not the data transfer itself. I measured all four directions to see how fast the visibility propagates.

Here's the conclusion up front:

S3 PutObject → FlexCache NFS read visibility: p50 8 ms. No separate copy or replication job; the data that gets read is pulled into the cache on demand
FlexCache overhead is +5 ms (vs. reading origin directly) — nearly transparent for same-region VPC peering
Reverse direction (NFS write → S3 AP read) is p50 44 ms, dominated by S3 API overhead. Not the main path in this design
FlexCache duality (NAS bucket S3 reads on FlexCache volumes) works, but only after -is-s3-enabled true is set on the cache volume in advanced privilege. My first attempt returned AccessDenied because that step was missing

Repository: Yoshiki0705/s3-burst-on-ontap-files

The Architecture

[S3 Client] --PutObject--> [S3 Access Point] --> [Origin Volume]
                                                       |
                                                  FlexCache
                                                       |
                                                [NFS/SMB Client]

Layer	What	Protocol
Collect (write)	S3 Access Point on FSx for ONTAP	S3 API
Source of truth	Origin volume	—
Distribution	FlexCache	Cluster/SVM peering
Consume (read)	Cache volumes	NFS/SMB only

The S3 Access Point attaches to the origin only. Cache volumes serve NFS/SMB. This single decision simplifies the design considerably:

One write path. The origin is authoritative; writes always go through the S3 AP
No S3 implementation differences pushed to the edge
Cache side requires only FlexCache + NFS/SMB — fewer platform constraints to hit

Test Environment

Item	Value
Date	2026-08-09
ONTAP version	9.18.1P3D1 (both clusters)
Configuration	SINGLE_AZ_1, 128 MBps × 2 clusters
Connectivity	VPC peering (same region, same account)
Mount	NFSv3, `actimeo=0` (client cache disabled)
Object size	64 bytes
Concurrency	1
Method	boto3 persistent session, same host, 30 iterations

Why NFS First

The initial measurements used NFS. Three reasons:

Client cache control. NFS actimeo=0 fully disables kernel attribute caching, isolating storage-side visibility latency. SMB oplocks/leases let the client cache independently, making it harder to measure the same thing at the same granularity.
Matching the identity to the security style. An access point authorizes every request as one file system identity, and AWS says to use a UNIX identity for UNIX security-style volumes and a Windows identity for NTFS ones (managing data access). NTFS volumes are supported too — this is not a UNIX-only feature, which corrects how an earlier version of this post put it. UNIX plus NFS was chosen here because it was the shortest path to a measurement, not because the other side is unavailable.
Primary use-case distribution. HiL test benches, rendering farms, and IoT analysis appliances are predominantly Linux + NFS. When the read side is SMB-dominant (Windows workstations in a production studio), the origin security style changes to NTFS — a different design path (see First Decisions).

SMB Shows Identical Results

I ran a separate verification with SMB (mount -t cifs, cache=none) on the same architecture:

Protocol	Mount method	p50	p90	max	n
SMB	`mount -t cifs`, `cache=none`	7 ms	8 ms	9 ms	30
NFS	`mount -t nfs`, `actimeo=0`	7 ms	8 ms	15 ms	30

With persistent mounts, SMB and NFS have identical visibility latency. No protocol difference.

Note: smbclient (establishing a new session per request) showed p50 43 ms. This is SMB session setup overhead — the same structural problem as the AWS CLI cold-start that inflated our first NFS→S3 measurement to 873 ms. Production environments with persistent connections won't see this.

SMB Considerations

FlexCache serves both NFS and SMB. This architecture doesn't exclude SMB — but note:

With a UNIX security-style origin, SMB clients receive UNIX permission-based access control, not NTFS ACLs
For SMB-primary workloads, NTFS security style is the natural choice, and an access point works there with a Windows identity. The trade-off is not availability but the identity you fix on the access point, which cannot be changed after creation
mixed security style is available in the API but officially not recommended by AWS — it's labeled "advanced users only." Permission type is determined by the last client that wrote, making troubleshooting difficult. This architecture doesn't use it
SMB needs a CIFS server on the SVM. An Active Directory join is not required: where a domain is not available, AWS documents setting up an SMB server in a workgroup (NTLM only, no Kerberos, and no GPO, VSS or SMB3 CA shares). If you do join AD, every data operation through the access point then needs a reachable domain controller, and HeadBucket succeeds even when it is not — so it cannot be used to check that

What I used here: a UNIX security-style origin with a UNIX identity, because the read side was NFS. For an SMB-primary workload the equivalent choice is an NTFS origin with a Windows identity. Decide it before the origin volume exists — whether the cache inherits the security style from the origin is unconfirmed on the on-premises path, and if it does, changing it later means rebuilding the serve layer.

actimeo=0 measures the minimum visibility latency by disabling client-side caching. Production deployments should use appropriate actimeo values for their workload — the default (~60 s) means subsequent reads hit kernel cache at ~0.05 ms, but changes aren't visible during that window.

Results — All Four Directions

#	Direction	p50	p90	p99	max
1	S3 AP PutObject → FlexCache NFS read	8 ms	9 ms	19 ms	19 ms
2	S3 AP PutObject → Origin NFS read (direct)	3 ms	5 ms	8 ms	8 ms
3	NFS write (Origin) → FlexCache NFS read	6 ms	7 ms	25 ms	25 ms
4	NFS write (Origin) → S3 AP GetObject	44 ms	49 ms	328 ms	328 ms

What the Numbers Say

Direction 1 is the main path. S3 write, FlexCache NFS read: p50 8 ms.

The +5 ms gap between directions 1 and 2 is FlexCache overhead. For same-region VPC peering, FlexCache is nearly transparent.

Direction 3 is faster than 1 (6 ms < 8 ms). NFS writes commit directly to the origin — no S3 API overhead — and FlexCache propagation alone is faster than the full S3-to-cache path.

Direction 4 (reverse) is slowest at 44 ms. S3 read-side processing dominates. This design keeps reads on NFS/SMB, so direction 4 isn't the main path.

NFS Client Cache Effect

Condition	p50
`actimeo=0` (cache disabled)	7 ms
`actimeo=60` (subsequent access)	0.05 ms (kernel cache hit)

With defaults, reads within 60 seconds of the last attribute check hit the kernel cache. The tradeoff: you won't see changes during that window. Tune per workload.

FlexCache Duality — It Works (With One Extra Step)

S3 access to NAS FlexCache volumes — "duality" — arrives in ONTAP 9.18.1, not 9.14.1 as an earlier version of this post said. I tested it on FSx for ONTAP 9.18.1P3D1.

Result: Works After Enabling `-is-s3-enabled true`

My initial test returned AccessDenied and I concluded it didn't work. After additional investigation, I found missing configuration step: S3 access must be explicitly enabled on the FlexCache volume:

set -privilege advanced
flexcache config modify -vserver <svm> -volume <fcache_vol> -is-s3-enabled true

After applying this setting:

Operation	Before (missing setting)	After (-is-s3-enabled true)
HeadBucket	✅	✅
ListObjectsV2	❌ AccessDenied	✅ KeyCount=1
GetObject	❌ AccessDenied	✅ Content verified

The fsxadmin role on FSx for ONTAP has access to advanced privilege commands including flexcache config modify. Documented at NetApp: Enable FlexCache duality.

Design Implication

FlexCache duality works, which means S3 reads from the cache side are possible. However, this architecture still recommends NFS/SMB on the cache side because:

ONTAP native S3 (NAS buckets) and AWS-managed S3 Access Points are different mechanisms
NAS buckets are read-only (no PutObject)
Requires additional configuration (advanced privilege + S3 user management)
No IAM integration or access point policy governance like S3 AP provides

FlexCache duality becomes an option when "S3 reads at the cache site" is a hard requirement. For AWS service integration (Lambda, Bedrock, etc.), the origin-side S3 AP remains the better fit. These are separate mechanisms, and the support status of one is not evidence for the other.

When This Pattern Fits

"Collect via S3 API in the cloud, consume via NFS/SMB at the edge" — this structure exists across industries.

Industry	Collect side	Consume side	Example
Automotive (AV/ADAS)	Driving logs and sensor data ingested to S3	HiL test benches replay via NFS	AWS + NetApp: Hybrid Cloud HiL
Semiconductor (EDA)	Design job I/O staged via S3	Toolchains (Synopsys, Cadence) run on NFS	EDA Scale with FSx for ONTAP
Media and VFX	Rendering assets collected to S3	Artist workstations mount SMB/NFS	FlexCache: distributed product development
Oil and Gas	Seismic survey data uploaded to S3	Interpretation workstations mount NFS	VDI for Subsurface O&G
Life Sciences	Genome sequencer output stored in S3	Bioinformatics HPC processes via NFS	Sequencer → S3 → NFS pipeline
Manufacturing / QA	Inspection camera images collected to S3	Line-side inspection software reads via NFS	Image → judgment → archive flow
Remote Work	Central design data updated via S3	Remote-site WorkSpaces access FlexCache NFS/SMB	FlexCache in AWS WorkSpaces
IoT / Edge	Sensor data streamed to S3	On-site analysis appliances read via NFS	Factory gateway → cloud → shopfloor

Common structure: Few write sites (often one), multiple read sites. Writes are bursty; reads touch only what's needed.

When It Doesn't Fit

Need full S3 semantics (versioning, event notifications, lifecycle): Use S3 natively
Object names aren't NAS-friendly (flat namespace, millions of keys with no directory separators): Performance degrades as root directory grows
Consumers need S3 reads at the cache site: possible from ONTAP 9.18.1 with duality (see above), but this architecture keeps the cache on NFS/SMB — the NAS bucket is a read-only view, needs advanced privilege, and has no IAM or access point policy governance
Need write-back from the cache: This pattern keeps cache read-centric
Need conditional writes (If-None-Match): Returns 501 NotImplemented. Handle exclusion at the application layer

S3 Access Point Design Notes

A few things to know when using the S3 AP as the collect layer.

S3 AP ≠ Amazon S3

The FSx for ONTAP S3 AP supports a subset of S3 operations. GetObject, PutObject, ListObjectsV2, HeadObject, DeleteObject, and MultipartUpload work. The following do not:

S3 Event Notifications (poll, or read the ONTAP native audit log. FPolicy + EventBridge is not a substitute — see the next section)
Lifecycle rules (use FabricPool)
Versioning (use ONTAP Snapshots)
Conditional writes If-None-Match (returns 501)
S3 Select, SSE-S3/KMS, Cross-AP Copy

Monitoring Coverage — FPolicy Does Not See This Path

An earlier version of this post offered FPolicy as the substitute for S3 Event Notifications.
I measured it, and it does not work. Correcting that here.

Measured 2026-08-26, ap-northeast-1, ONTAP 9.18.1P3D1, with the same result for both a UNIX-identity
and a WINDOWS-identity access point.

Mechanism	Operations through the S3 AP	Detail
FPolicy	Not notified	Zero notifications in a 90-second idle window, zero across nine S3 AP data-plane calls, while a file-protocol control on the same volume in the same session did fire. The protocols an FPolicy event accepts are `cifs`, `nfsv3` and `nfsv4` only — there is no value for the S3 path
FPolicy `mandatory`	Not blocked	With a synchronous engine and `mandatory=true`, an NFSv3 write returned `Permission denied` while PUT, GET, LIST and DELETE through the S3 AP on the same volume all succeeded
ONTAP native audit log	Recorded	As `Source=HTTP` for object operations and `Source=S3` for LIST. But `SubjectUserName` and `SubjectDomainName` are `Not Present` and `SubjectIP` is an AWS service-side address, so the requester is not recorded. `HeadObject` produced nothing across six calls. An audit ACE (SACL) is required
ARP	Detected	ARP 5.0. 150 high-entropy files written through the access point were recorded as suspect under `High Entropy`, with `attack_probability` at `moderate`

Two things follow for the design. First, detection, DLP and blocking that start from an FPolicy
notification do not reach this path — including the guarantee mandatory is chosen for, that an
unreachable engine stops the operation. Second, the audit log records the operation but not the
requester, so answering "who touched this file" from the audit trail alone is not possible; it needs
correlation with AWS CloudTrail.

ARP does detect, but attack_probability changes more than ten minutes after the write. Reading
none from a short observation window and concluding nothing was detected is a false negative. ARP
blocking is unmeasured.

Whether FPolicy, auditing or ARP fire on the cache side is unverified. Writes in this architecture
land on the origin, so the cache side remains a separate question.

Throughput is shared with NFS/SMB

S3 AP, NFS, and SMB all consume the same FSx for ONTAP provisioned throughput. In this architecture, origin and cache are separate clusters so this rarely matters — but if NFS clients also access the origin directly, account for the shared bandwidth. I have not measured concurrency, so I will not hand you a number: the ceiling follows from provisioned throughput divided by the bandwidth one request consumes, and that second term depends on your object size and request duration. Raise concurrency while watching the SlowDown (503) rate and p99, and stop below what you can absorb.

That reading is for the first generation (SINGLE_AZ_1). On the second generation the value specified means something else. It names the disk throughput baseline, and the network baseline is about twice it. On a file system specified at 1,536 MBps, a 1 MiB sequential read delivered 2,882 MB/s for 27 minutes (1.88x the value specified) and then 1,439 MB/s (0.94x). The drop happens inside a single 10-second interval once the burst allowance runs out, so a five-minute measurement reports 2.0x too high. Treating the value specified as the ceiling puts the estimate out in either direction (measurement record, Japanese).

Directory design matters

S3 PutObject keys map directly to directory structure. What sets the ceiling is the volume's maxdir-size: reach it and the client gets ENOSPC and can no longer create files. It is a per-volume setting, and raising it could affect performance. Check the value on your own volume and partition by date, tenant, or hash prefix so the entries in one directory stay well short of it. Response time for readdir and ListObjectsV2 grows with entry count too, though I have not measured where that starts to hurt.

This directly affects NFS usability. If you flat-dump millions of objects without hierarchy, FlexCache NFS clients will struggle with ls and find. Design your S3 keys with "how does this look when I ls on NFS?" in mind.

Key recommendations:

Date-based partitioning (year=YYYY/month=MM/day=DD/) so the entries in one directory stay well short of maxdir-size
Separate ingest volumes from consumption volumes; apply FlexCache only to consumption
On the NFS side, use manifest files or path generation instead of directory traversal (find)

For full details, see the S3 AP Design Guide in the repository.

What It Costs — Egress First, Requests Second

I framed this section around S3 request pricing when I first wrote it. That was the wrong axis, and the modelling I did afterwards says so plainly. Corrected below.

The cost that hurts when your readers sit outside AWS is data transfer. Egress is charged on bytes leaving the Region, so reading the same file ten times moves ten times the bytes and pays ten times over. A cache removes the multiplier: it carries the working set once and every later read is served locally over NFS or SMB, never becoming an S3 request at all.

A worked case — 20 TiB dataset, 2 TiB monthly working set, 4 MiB objects, each file read 30 times, internet egress from ap-northeast-1:

Option	Monthly
Read S3 directly from on premises	$6,211 — egress alone is $5,693 (92%)
Copy the whole dataset down with DataSync	$2,847
FSx for ONTAP + FlexCache	$1,333

Storage and requests are rounding errors next to transfer here.

The read count decides it

Reads per file per month	Direct	This architecture	Ratio
1	$746	$1,333	0.6x
5	$1,680	$1,333	1.3x
10	$2,593	$1,333	1.9x
30	$6,211	$1,333	4.7x
100	$18,451	$1,333	13.8x

At one read per file the direct path wins — the same bytes move either way and there is no reason to carry a file system's fixed cost. The crossover is between five and ten. Where it falls is a property of your workload, not of the products.

Requests matter too, but only below a certain object size

I had assumed S3 GET charges were a co-equal problem. At 4 MiB objects they are not: the entire 5.2 million reads in the ten-read case cost $1.94, about a thousandth of the transfer bill. Through an access point it is $0.31, and through FlexCache $0.16 — the reads are NFS and SMB, so they generate no S3 requests, and only the origin-side capacity pool fetch on cache fill remains.

Where the assumption does hold is small objects. Holding bytes read constant and varying object size:

Object size	Reads/month	GET charges	Share of that option's transfer bill
8 KiB	2.68 B	$993	48%
64 KiB	336 M	$124	6%
256 KiB	84 M	$31	1.5%
4 MiB	5.2 M	$1.94	0.1%

So both charges need designing for, and they respond to opposite remedies:

Transfer falls by carrying fewer bytes — cache the working set, move the readers into AWS, or drop the unit rate with Direct Connect ($0.041/GB against $0.114 for the first 10 TB of internet egress). Making objects bigger does not help; the byte count is unchanged.
Requests fall by making fewer calls — batch at collection time so files are larger, and serve reads over a file protocol so they never become S3 calls. Negotiating transfer rates does not help when the money is on the request side.

Batch too far and you hit the collection side: a single PutObject caps at 5 GiB and a whole object at 50 GiB, and the 50 GiB check happens at CompleteMultipartUpload — after the entire payload has been transferred and paid for. Both figures being binary (5,368,709,120 and 53,687,091,200 bytes) is confirmed by the vendor. When this was published the documentation still read "5 GB" and "50 GB" and a correction was said to be in progress; checking again on 2026-09-11, the 50 GiB figure now reads in GiB (access point API support). The page stating the 5 GiB single-PutObject limit, however, was not found that day. "Not found" is not "does not exist", so that one rests on the measurement alone. The increase from 5 GiB to 50 GiB was also never announced through any channel. An unannounced tenfold change leaves every downstream document that quotes the old value wrong, with no signal to re-check.

And that 50 GiB caps uploads, not retrievals. The documentation reads "Maximum object size is 50 GiB for uploads, but you can download objects larger than that". A file larger than 50 GiB cannot be created over S3, but one created over NFS / SMB can be read over S3. Even with collection centred on S3, writing just the large files on the file side is a documented way out. I have not measured this. That the documentation says so, and that a file larger than 50 GiB was created over NFS and fetched with GetObject, are different claims. Batching past the consumer's read unit also sends bytes nobody reads. The repository's design guide sets the consumer's read unit as the reference for how far to go, with the measurement and monitoring steps.

If the readers can move, move them

Worth stating because it outweighs every storage choice. In-Region transfer is free, so putting the consumers in AWS deletes the entire egress line — $2,079 at ten reads on this workload. Reading S3 directly from EC2 comes to $514/month; S3 Files costs $515, ninety-four cents more, because 4 MiB objects sit above its size threshold and never reach its high-performance storage; FSx for ONTAP in the same Region is $1,053, twice the direct path, since the transfer gap that justified it is gone.

S3 Files cannot serve this architecture's consumers. It speaks NFSv4.1 and NFSv4.2 only — no NFSv3, no SMB — so an appliance pinned to v3 and any Windows stage are out, and the documented compute targets are EC2, Lambda, EKS and ECS. It belongs in the picture as the option you get after migrating the readers, not as a substitute for a cache.

This architecture is for the cases where the readers cannot move: equipment on site, proximity to whatever is being measured, capital already spent on it.

On the distribution side

A FlexCache cache volume cannot be tiered — an origin with FabricPool tiering can be cached, but the cache itself never tiers, so it sits entirely on SSD. That is affordable because it is sparse: NetApp's guidance is at least 10% of origin, which is also the create default. At 10% the distribution side runs 2.7x to 6.1x below a full second copy, with the copy given its capacity pool discount. Size it like a copy and it inverts — at 100% it costs more than the copy, precisely because it cannot tier.

Cache volumes are writable, which I understated earlier. Write-around is the default and withholds the client acknowledgement until the origin has committed; write-back, from ONTAP 9.15.1, commits at the cache and propagates asynchronously. The default being synchronous with respect to the origin matters for freshness: there is no window where a cache-side write is missing from the canonical copy.

One assumption drives more of these numbers than any other, so treat it carefully. Background storage efficiency does not run on data once it has been tiered — only savings applied while the block was on SSD are preserved, and a block tiered before efficiency ran keeps none (AWS, NetApp). Expecting better than 40% on a tiering-enabled volume is optimistic. The model takes the SSD rate from AWS's published per-workload figures, assumes the pool tier retains half of it, and carries a sensitivity table — because the assumption only ever flatters this architecture: ONTAP deduplication does not reduce an S3 storage bill.

Full breakdown, every rate with its effective date, the workload models and the cross-layer pitfalls: FinOps cost structure and the S3 AP Design Guide (Japanese). Cost tables are generated from a model in the repository, so a price change moves one declaration rather than fifty typed totals.

A verification copy that costs no capacity — FlexClone and the S3 Access Point

Everything above is about distribution, so I measured duplication as well.

I checked whether FlexClone works on a volume with an access point attached (2026-08-26, ap-northeast-1, ONTAP 9.18.1P3D1). NetApp records FlexClone as unsupported for ONTAP S3, so I expected the same restriction might appear on this path, but it did not appear as a restriction.

FlexClone has two granularities, and they behave quite differently.

	File granularity	Volume granularity
Where it lands	The same volume. It appears as a new key on the same access point	A separate volume
Authorization boundary	Same as the parent	Its own access point, its own policy, its own identity
Time before it is usable	None	A volume created through the ONTAP API took 599 to 1,177 s to appear on the AWS side, then the junction path has to be set before attaching
Visible over S3	Yes. `StorageClass=FSX_ONTAP`, sha256 matches the source	Yes. LIST / GET / PUT through the clone's own access point

At either granularity the clone shares data blocks with its parent. Cloning a 256 MiB file four times gives these figures:

Point	Logical	Physical
Just after one 256 MiB PUT	—	253,534,208 B
After four file-granularity clones	1,350,942,720 B	277,200,896 B

Logical grew to five copies, but physical grew by only about 23 MB.

FinOps — no capacity to provision per duplicate

FSx for ONTAP bills provisioned SSD capacity. If a duplicate shares blocks with its parent, there is no capacity to provision for it. Doing the same on S3 requires CopyObject, every duplicate carries full storage charges, and the copy requests are billed on top. That difference is structural, so a unit rate does not close it.

The saving does reach the invoice only as capacity you did not have to provision, though. Where headroom is already generous, the figure does not move.

Two notes on what the space figures assume. NetApp states that copies consume no storage except what is required for metadata until changes are written to the copy (FlexClone volumes, files, and LUNs), so changes written to the clone are not shared and consume new blocks for whatever was written. And deleting data on the parent does not return capacity while a clone or a snapshot still references it.

The operation that ends the sharing is a split (volume clone split start). I did not measure it, so this is the documentation: from ONTAP 9.4, on AFF systems where the volume guarantee is none, the split shares the physical blocks rather than copying the data, and space efficiency is preserved. After the split, however, both the parent and the clone require the full space allocation set by their volume guarantees. The space needed can be checked beforehand with volume clone show -estimate (splitting procedure).

Operational Excellence — no AWS-side work per duplicate

What helps operationally is that making a duplicate adds no AWS-side resource. A file-granularity clone appears as a new key under the existing access point, so there is nothing to attach and nothing to wait for. A verification dataset can be produced from the production bytes and handed to the S3 consumers already in place, such as Amazon Bedrock, AWS Glue and Amazon Athena, without a staging copy job.

Where separation is needed, volume granularity is the one to use. Attaching a separate access point to the clone keeps its identity and policy apart from the parent's, and a write to the clone did not appear on the parent. That gives a way to separate the authorization boundary without duplicating the bytes. The cost is the 10-to-20-minute wait noted above.

There are two operational cautions as well.

The first is that a file-granularity clone gives no observable failure. POST /api/storage/file/clone returns 202 and a job UUID, but the UUID resolved to 404 entry doesn't exist and appeared in no job listing. The same fsxadmin retrieves volume-create and volume-clone jobs as state=success, so this is not a permissions problem. A call naming a destination directory that does not exist also returned 202 and created nothing. Judge the outcome by inspecting the destination file.

The second is that teardown gains an ordering constraint. Delete the clone before the parent. While the deleted clone sits in ONTAP's volume recovery queue, the parent keeps clone.has_flexclone at true, and the AWS-side delete-volume returned DELETING and then silently went back to CREATED. Purging the recovery queue clears the flag and the same call works. That queue appears in neither the console nor the FSx for ONTAP API, so watching only the AWS side leaves you stuck here.

What I did not measure is LUN-granularity cloning: creating a LUN needs an iSCSI configuration, which is not on this architecture's path. The procedure and the controls are in the repository's interoperability page, and the cost structure in FinOps cost structure.

Deploy

CloudFormation and Terraform templates:

GitHub: Yoshiki0705/s3-burst-on-ontap-files

Conclusion

"Collect via S3, consume via NFS" — one volume, no separate copy or replication job. FSx for ONTAP S3 Access Point + FlexCache. Main path p50 8 ms. FlexCache adds ~5 ms for same-region — nearly transparent. Transfer still happens for whatever gets read; what is absent is the sync job.

This verification was done entirely on AWS (FSx for ONTAP to FSx for ONTAP over VPC peering), but the cache side isn't limited to AWS. AWS documents exactly three FlexCache configurations, and with FSx for ONTAP as the origin the cache is either on-premises ONTAP or FSx for ONTAP (replicating with FlexCache). Cloud Volumes ONTAP, ONTAP Select, Azure NetApp Files and Google Cloud NetApp Volumes are absent from that table, so I record them as unconfirmed rather than assuming they work because they are ONTAP-based — and unconfirmed is not the same as unsupported. The repository's Portability and Support Matrix pages track what's confirmed and what's next. Cross-platform verification is on the roadmap.

The reverse direction is outside that table too, and the verdict splits two ways. Another
cloud's file storage as the origin with FSx for ONTAP as the cache is unconfirmed for Google Cloud
NetApp Volumes and Azure NetApp Files — something that looking could still resolve. For Google Cloud
Filestore, Azure Managed Lustre, Azure Blob NFS and OCI File Storage it is out of scope as a
mechanism, because they are not ONTAP and the cluster and SVM peering FlexCache requires cannot
exist. Network reachability does not change the second group. I do not write the two with the same
word.

I wrote up the network underneath it separately. AWS Interconnect – multicloud is GA for Google
Cloud (eight pairs) and OCI (one pair, us-east-1 to us-ashburn-1), and at Preview for Azure since
2026-08 with four pairs. No Japanese Region appears in any CSP's pairs, and a pair being listed
is not the same as that pair being GA. Azure also has a facing service, Azure Multicloud
Interconnect, though the only sources I found are Microsoft blogs rather than the Microsoft Learn
reference. Starting from Japan therefore means joining Direct Connect to the other cloud's circuit
inside an interconnection provider's fabric, and whether that is possible is decided not by the
Region-pair table but by whether Direct Connect locations, the other cloud's connection locations and
the provider's footprint overlap. Taking the partner route does not add Regions to the managed
service — it switches to a different construction.

The arrows in the figure stop at the AWS VPC. Not drawing the next hop as a solid edge is what the figure is for.

The same user guide carries AWS Interconnect – last mile. The name is close, but it does not
join two CSPs: it brings a carrier circuit to AWS. It is offered with Lumen, from the New Jersey
sites in us-east-1 to any AWS Region, or from anywhere in the continental United States over Lumen's
fabric (Regional Availability).
Where the serve side sits at a site it can form part of the path, but it is a different axis, so it
is not among the three ways in the figure above. The stage is documented; I have not measured it.

Encryption also sits at two layers worth keeping apart. MACsec on the physical link and cluster
peering encryption over FlexCache traffic (ONTAP 9.6 or later, TLS 1.2 AES-256 GCM) are different
things, and the first does not remove the need for the second. Whether ONTAP itself offers MACsec
on an intercluster LIF is something I could find no statement for. The options and the Region
coverage are in cross-cloud connectivity.

FlexCache duality (S3 reads on the cache side) does work, once S3 access is enabled on the cache volume itself with -is-s3-enabled true in advanced privilege. The earlier AccessDenied was a missing setting, not a platform limitation. The architecture still keeps cache-side access to NFS/SMB, because ONTAP native S3 and the AWS-managed access point are separate mechanisms and the NAS bucket is a read-only view without IAM integration.

All test resources torn down. Numbers are from a specific test environment and vary by workload and configuration.

Letting an AI agent operate on FSx for ONTAP data — AgentCore MCP, with a human approving (File Portal Part 3)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:21:40 +0000

Let an AI agent work the NAS, with a human approving — Amazon Bedrock AgentCore and MCP (part 3 of 6)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1 I built the file portal foundation on the S3 Access Points of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP), and in Part 2 I embedded storage operations (ARP/AI incident response, Tamperproof Snapshots, regulatory retention management). At that point the UI was sufficient for anyone who already knew which button to press — but a different kind of friction was still there. "Where did I put last week's simulation results?" "Summarize the key points of this PDF in three lines." The intent is perfectly clear; the cost of translating it into system operations was what stayed high.

So I integrated AI agents into the portal. Express your intent in natural language, and the agent finds the files, reads them, analyzes the content, and proposes storage management operations when needed. Because storage management includes irreversible operations, the execution decision stays with a human through a Human-in-the-Loop (HITL) flow.

Here's the conclusion up front:

Combining the Amazon Bedrock Converse API with MCP tools lets you explore and analyze files on an S3 AP in natural language
Connecting a Bedrock Knowledge Base directly to an S3 AP means you can reach a file through semantic search without knowing its name
Wrapping destructive operations in a HITL approval modal keeps agent autonomy and safety in the same design
Multi-agent collaboration pays off only for tasks with several phases — "discover → analyze → judge"

In this article, I'll cover the 3-mode AgentChat design, file access via MCP tools, the HITL flow, and the implementation patterns for multi-agent collaboration.

Who this is for: anyone who wants to run an AI agent against files on a NAS, and anyone interested in putting human approval in front of a destructive operation.

What this article does not cover: quality or latency comparisons between models (not measured), the cost of embeddings, and confirmation of the agents through real browser operation — as the verification status below records, they sit in the "automated tests only" category.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Repository: solutions/amplify-portal/

Terminology note: "MCP" in this article refers to Model Context Protocol — a standard protocol that allows AI models to invoke external tools (file operations, database queries, etc.). AgentCore Gateway acts as the MCP server, exposing Lambda-based tools to the AI model.

Where this agent sits

It does not replace a general-purpose AI assistant. General question answering is better served there. What is added here is discovery, content understanding and storage-operation proposals against the files on the NAS.

The properties on this side:

Files are reached on the NAS directly through the S3 AP. No upload
The scope is the whole volume, searchable across it
Storage operations (enabling SnapLock, blocking a user) can be proposed, and executing one goes through HITL approval
ONTAP-side operations are available: ARP/AI status, snapshot operations, export-policy changes
File contents are processed by Bedrock in the same Region

Verification status: how far each feature in this article has actually been confirmed is recorded in verification results, split four ways (live E2E / live read / tests only / DemoMode). Agent and team execution sits in the "tests only" category: handler and component tests pass, but it has not been driven from a browser against a real system.

What I Built

The portal moved from "browse files and check operational state" to "ask in natural language, and the agent finds files, understands content, and proposes storage operations." Here's what was added:

AgentChat: Multi-tool AI chat with 3 switchable modes
SemanticSearch: Vector search via Bedrock Knowledge Base — resolves "where did I put that file?" in natural language
Agent Directory / Creator: Define custom agents and share them with your team
Agent Teams: Run multiple agents in collaboration
ActionApproval (HITL): Insert a human approval flow before destructive operations
Multimodal input: Drag and drop an image into the chat for file analysis

Why Embed AI Agents in the Portal

The admin features from Part 2 work well for people who already know "what to operate and how." But what I actually hear in practice sounds more like:

"Where did I put last week's simulation results?"
"Can you summarize what caused the error in this log file?"
"I want to run legal review on the contracts folder, but I don't know the steps"

These aren't cases of "not knowing how to operate the system." They're cases of "I know what I want, but translating that intent into system operations is tedious." This applies equally to non-technical team members:

"Where's the attachment for last month's expense report?"
"Summarize this PDF in 3 lines"
"Compile a file list from this folder for my weekly report"

The motivation for embedding AI agents was to eliminate this translation cost entirely. Express your intent in natural language, and the agent finds files, reads them, analyzes content, and proposes admin operations if needed. Users only need to think about "what they want to accomplish."

However, giving an agent unrestricted authority in a storage management context is dangerous. Enabling SnapLock is irreversible. Modifying an export-policy immediately cuts off user access. So I combine the Human-in-the-Loop (HITL) pattern: "the agent proposes, but humans approve execution."

AgentChat — 3-Mode Integrated Chat

A persistent AI chat in the portal's right panel, fully replacing the previous "Bedrock Q&A" (single-file question-answering).

3 Modes

Mode	Icon	Purpose
🧠 Knowledge	KB	Cross-volume semantic search. Answers "where's that file?"
📁 File Agent	agent	File read/list/analyze. Accesses S3 AP via MCP tools
🤖 Multi-Agent	multi	Multiple agents collaborate. File discovery → content analysis → operation proposal in sequence

Modes switch via pill buttons below the header. Welcome screen task cards change per mode, showing "what you can do in this mode."

Architecture

Browser (React)
    ↓ AppSync Query (agentQuery)
Lambda: Agent Orchestrator (outside VPC)
    ├── mode=kb   → Bedrock KB RetrieveAndGenerate
    ├── mode=agent → Bedrock Converse + MCP Tools
    └── mode=multi → Multi-agent collaboration
                         ↓ MCP Client
                    AgentCore Gateway
                         ↓ Lambda Invoke
                    MCP Tool Lambda (list_files / read_file / search_files)
                         ↓ S3 API
                    FSx for ONTAP S3 Access Point

Light theme shown. A dark theme version is available.

The VPC split principle applies here too. The Agent Orchestrator Lambda is placed outside VPC, accessing Bedrock API and S3 AP (Internet-origin). Admin operations calling ONTAP REST API are handled by a separate VPC-internal Lambda.

Tool Execution Visibility

When the agent invokes tools, a timeline appears in the chat:

🔧 Tool Calls (3)                          2.1s
├── ✅ 🔍 file-explorer: list_files         {"prefix": "/engineering/"}
├── ✅ 📄 file-explorer: read_file          {"key": "/engineering/thermal-spec-v3.pdf"}
└── ✅ 🧠 knowledge-analyst: analyze_file   {"key": "...", "question": "..."}

Each tool's input/output examples:

Tool	Input Example	Output Example
`list_files`	`{"prefix": "/engineering/", "max_keys": 20}`	File path list + size + last modified
`read_file`	`{"key": "/engineering/thermal-spec-v3.pdf"}`	File content (text-extracted)
`search_files`	`{"query": "thermal limit exceeded"}`	Matched files + relevant snippets

Displayed in <details> — expanded by default when 3 or fewer tool calls, collapsed when 4+. This balances Nielsen's "visibility of system status" with Wroblewski's "progressive disclosure."

Chat History Persistence

Conversation sessions auto-save to DynamoDB (2-second debounce). Past sessions can be recalled from the history panel to continue questioning.

// Auto-save: 2 seconds after message change → DynamoDB
useEffect(() => {
  if (messages.length === 0 || !currentSessionId) return;
  const timer = setTimeout(() => saveCurrentSession(), 2000);
  return () => clearTimeout(timer);
}, [messages, currentSessionId]);

The rationale: forcing an explicit "Save" button interrupts the natural flow of chat. Close the browser, and next time you open it, you can pick up from "where we left off yesterday."

Multimodal Input

Drag-and-drop images into the chat for Bedrock Vision analysis:

Circuit diagram image → "Where's the bottleneck in this circuit?"
Screenshot → "What's causing this error screen?"
Graph image → "Explain this performance degradation trend"

Supports JPEG/PNG/GIF/WebP under 5MB. Base64-encoded and sent to Bedrock Converse API.

SemanticSearch — Vector Search

Keyword search on file names has limits. When you want to find "the spec document about thermal design limits," you might find thermal-spec-v3.pdf by name, but TC-2024-0089.pdf? No chance.

Keyword / Semantic Mode Toggle

Mode toggle pills in the search bar:

🔍 [Search files...                        ] [🔍]
   [📂 Keyword]  [🧠 Semantic]

Keyword mode: S3 AP ListObjectsV2 + prefix filter. Auto-searches with 500ms debounce after 2+ characters
Semantic mode: Bedrock Knowledge Base RetrieveAndGenerate API. Vectorizes natural language queries for similarity search

Bedrock Knowledge Base Integration

FSx for ONTAP's S3 AP can be specified directly as a Knowledge Base data source (AWS official tutorial).

FSx for ONTAP Volume
    ↓ S3 AP (Data Source)
Bedrock Knowledge Base
    ↓ Embeddings (Titan V2)
OpenSearch Serverless (Vector Store)
    ↓ RetrieveAndGenerate
Results: related chunks + source file path + relevance score

Light theme shown. A dark theme version is available.

Results include scores (percentage display) and snippets. Clicking a result navigates to the file in the All Files view.

Semantic Search Input Examples

When KB is unconfigured, help text guides users:

💡 Semantic search examples:
• "Files with anomalies in last month's sales reports"
• "Test results exceeding thermal design specifications"
• "Description of the approval process for contract renewal"

Constraints

This portal does not decide when a newly written file becomes searchable. The Knowledge Base is a bring-your-own resource and the portal only calls retrieve. Nothing in the portal starts an ingestion job — no EventBridge rule, no schedule. A newly written file is therefore not searchable until ingestion runs
The time to appear is unmeasured. With no trigger on the portal side, the timing belongs to whoever operates the KB
Large file environments incur OpenSearch Serverless indexing costs
PDF/Office text extraction accuracy depends on file quality
If bedrockKbId is empty, semantic mode shows a "KB not configured" error. Keyword search does not go through the KB and is not subject to any of this

KB sync detection: Data source sync status can be checked via the bedrock-agent:GetIngestionJob API. Periodic sync can be scheduled in the Bedrock console. If immediate search reflection after file changes is required, consider an EventBridge + StartIngestionJob API configuration for on-demand sync triggers.

Agent Directory — Agent Catalog

A UI for discovering "what agents are available."

Preset Agents

Agent	Icon	Specialty
file-explorer	📁	File operations (list, read, search)
knowledge-analyst	🧠	Vector search + document analysis
safety-controller	🛡️	ARP status checks + containment action proposals
compliance-auditor	📋	Retention period and SnapLock configuration audits
ops-advisor	⚙️	EMS event analysis + operational recommendations

Custom Agent Creation (Agent Creator)

When presets aren't enough, define your own agents:

Agent Creation Wizard:
1. Set name and icon
2. Write system prompt
3. Select tools to use (checkboxes)
4. Test execution
5. Sharing settings (personal only / team-shared)

Created agents appear in the Agent Directory, available to team members (when shared). This lets you codify patterns like "this department frequently does this kind of analysis" as reusable agents.

Agent Teams — Multi-Agent Collaboration

For complex tasks that a single agent can't handle alone, multiple agents collaborate.

When to Use Multi-Agent vs Single Agent

Scenario	Recommended	Reason
"Summarize this PDF"	Single agent	Completes with one tool (read_file + analysis)
"Analyze all files in engineering/ and report security issues"	Multi-agent	Requires file discovery → content analysis → security judgment
"Audit files changed last month for compliance violations"	Multi-agent	File identification → content reading → regulatory assessment → report generation

Decision criteria: Multi-agent is effective when the task has multiple phases ("discover → analyze → judge") requiring different expertise at each phase. For simple tasks, multi-agent adds cost and latency without benefit — use single agent instead.

Example Team Configuration

Team: "Security Audit Team"
├── 🔍 file-explorer (Collaborator): Identify target files
├── 🛡️ safety-controller (Supervisor): Verify ARP/security state
└── 📋 compliance-auditor (Reviewer): Final compliance judgment

Roles

Role	Responsibility
Supervisor	Manages overall task progress. Directs other agents
Collaborator	Performs work based on instructions (file ops, analysis, etc.)
Reviewer	Reviews work results. Quality checks and approve/reject

Team Creation Wizard

Enter team name and description
Add agents from Agent Directory (minimum 2)
Assign roles to each agent
Sharing settings

Team execution flow:

User: "Analyze all simulation results in the engineering/ folder"
    ↓
Supervisor (safety-controller):
    → Instructs file-explorer: "Get file listing for engineering/"
    ↓
Collaborator (file-explorer):
    → list_files → read_file (multiple)
    → Result: 12 files found, 8 are simulation results
    ↓
Supervisor:
    → Instructs knowledge-analyst: "Create summaries for each file"
    ↓
Collaborator (knowledge-analyst):
    → Analyzes each file, generates summaries
    ↓
Reviewer (compliance-auditor):
    → "Verify no PHI data is included"
    → Result: 2 files may contain personal information → recommends guardrail application
    ↓
Final response to user:
    "Analyzed 8 of 12 files. Summaries below.
     Note: 2 files were filtered due to potential personal information."

Light theme shown. A dark theme version is available.

Each agent leverages its specialty while the Supervisor coordinates the overall flow. From the user's perspective, a single chat message completes a complex task.

ActionApproval — Human-in-the-Loop

When the AI agent proposes a destructive or irreversible operation, a modal requesting human approval appears before execution.

Why HITL Matters

In a storage management context, some operations are irreversible or have immediate impact. Enabling SnapLock can never be undone. A SnapMirror break severs the replication relationship. If an AI agent autonomously executes these, recovery may be impossible. The HITL pattern ensures the agent "proposes" but the final execution decision rests with a human.

Which Operations Require Approval

Operations flagged with isDestructive: true or isReversible: false require approval:

Operation	Icon	Irreversible?	Why Approval is Required
File deletion	🗑️	△ (recoverable from snapshot)	Prevent unintended bulk deletion
SnapLock enable	🔐	◎ (completely irreversible)	Once enabled, cannot be disabled
SnapMirror break	🔗	◎ (relationship must be re-established)	Replication severance requires manual recovery
User block	🚫	○ (reversible but immediate impact)	Risk of business disruption
IP block	🚫	○	Risk of service disruption
Retention period change	⏱️	△ (can only extend, never shorten)	Shortening is not possible
Threat containment	🛡️	○	May affect multiple users

How the Approval Flow Works

1. Agent proposes a destructive operation
2. UI displays modal (showing action, target, reason)
3. User reviews the details
4. [Approve] → operation executes / [Reject] → operation cancelled, agent notified

Example: SnapMirror Break Approval Flow

User: "Execute failover to the DR site"
    ↓
Agent: Determines SnapMirror break is required
    ↓
┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  SnapMirror break                │
│ Target:  vol_production → vol_dr         │
│ Reason:  DR failover execution           │
│                                          │
│ 🔴 This operation severs the replication │
│    relationship. Re-sync requires manual │
│    intervention.                         │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

UI Design

┌──────────────────────────────────────────┐
│ ⚠️  The agent is requesting approval     │
├──────────────────────────────────────────┤
│                                          │
│ Action:  User block                      │
│ Target:  DOMAIN\suspicious_user          │
│ Reason:  ARP detected abnormal write     │
│          patterns                        │
│                                          │
│ ⚠️ This operation will immediately cut   │
│    off the target user's SMB access      │
│                                          │
│ [❌ Reject]              [✅ Approve]     │
└──────────────────────────────────────────┘

Safe-side default: Reject button has autoFocus. Reflexively pressing Enter falls to the safe side. Clicking outside the modal also triggers "reject."

Irreversibility warning: Operations with isReversible: false show a red warning bar, ensuring users visually recognize the severity of the operation.

Bedrock Guardrails — Response Filtering

Bedrock Guardrails are applied to all AgentChat responses. While the PHI guardrail (Part 2) blocks based on path patterns, Guardrails filter response content itself.

Applied Checks

Check	Target	Behavior
PII detection	Input/Output	Masking (`*--**`)
Harmful content	Output	Block + substitute message
Topic denial	Input	Refuse response to specific topics
Grounding check	Output	Suppress ungrounded answers

When Guardrails are applied, a 🛡️ badge appears on the message. Users see "some information was filtered" with filter reason details available via tooltip.

Model, Cost, and Fallback

Model Configuration

Item	Actual value	Where it is decided
Model	`amazon.nova-lite-v1:0`	The agent reads the `AGENT_MODEL_ID` environment variable. The ask-about-file side is hardcoded as `BEDROCK_MODEL_ID` in `backend.ts`
Max output tokens	2048 for the agent, 1024 for ask-about-file	Hardcoded in the handlers. Not a setting
History sent	Last 10 messages	Hardcoded in the frontend, `messages.slice(-10)` in `AgentChat.tsx`
Cost per request	~$0.001–$0.01	For Nova Lite. Larger context from file reads increases cost

To swap the model, pass AGENT_MODEL_ID for the agent. The ask-about-file side is hardcoded in backend.ts, so changing it means touching code. Nova Lite is the only model this verification used; no quality or latency comparison against another model was measured. Which model suits you is not something this article answers.

Cost note: Token cost per conversation depends on tool call count and file sizes. A typical "file search + summarize" conversation costs ~$0.005–$0.02 with Nova Lite. The number of history messages sent is hardcoded as messages.slice(-10) in AgentChat.tsx. Reducing it lowers cost and loses context in longer conversations.

Data residency note: When using Bedrock in ap-northeast-1 (Tokyo), input data processing completes within the same region. Deploying FSx for ONTAP, the portal, and Bedrock in the same region ensures file contents never leave the region. Important for financial institutions and healthcare organizations with data residency requirements.

Security note: File contents are included in Bedrock API request bodies, but a VPC-contained configuration is possible. Bedrock does not use request data for model training (AWS Bedrock Data Protection). With VPC endpoint routing, data never leaves the VPC.

Bedrock KB Configuration Parameters

Setting	Recommended	Rationale
Embedding Model	Titan Embeddings V2	Japanese support, cost-efficient
Chunk Size	512 tokens	Appropriate granularity for file-level search
Chunk Overlap	64 tokens	Prevents context loss at chunk boundaries
Top K	5	Balance of precision and cost
Vector Store	OpenSearch Serverless	Auto-provisioned with Managed KB

The cost that bites is recomputing embeddings. Every ingestion run creates embeddings for the files in scope, so the sync frequency and the scope are the bill. Not measured in this article.

These are starting-point recommendations. Adjust based on average file size and content characteristics.

Chunk Size tradeoff: Smaller chunks (256 tokens) improve recall but fragment context, reducing precision. Larger chunks (1024 tokens) provide richer context but allow irrelevant content to mix in. 512 tokens balances well for "file-level search." For technical documents (long sections), consider 768–1024.

Fallback Behavior

When Bedrock API is unavailable (regional outage, throttling, etc.), the agent chat displays "Cannot connect. Please use conventional search" and falls back to keyword search. The design ensures "agent unavailable ≠ portal unavailable."

Chat History Retention Policy

Set TTL on DynamoDB conversation history. Default: auto-deleted after 90 days. If auditing "who asked what" is required, separately retain CloudTrail API call logs.

Right to deletion: If a user requests deletion of their chat history, administrators can manually delete the relevant DynamoDB sessions. For GDPR/privacy law environments, implementing a self-service "Clear history" button is recommended.

Design Tradeoffs

Decision	Benefit	Tradeoff
AppSync Query (not Subscription)	Simple implementation, single Cold Start	No streaming response (waits for full completion)
Agent Orchestrator outside VPC	Short Cold Start (no ENI)	Cannot call ONTAP REST API (delegates to VPC Lambda)
DynamoDB session storage	History shared across browsers	Cost (write capacity), TTL management needed
HITL via modal	Reliably blocks destructive ops	UX interruption, reduced agent autonomy
Mode pill toggle (3 modes)	Optimal agent for each purpose	Learning cost of mode concept
Bedrock Guardrails always-on	Structural PII leak prevention	Latency increase (hundreds of ms)
Multi-agent (Agent Teams)	Handles complex tasks	Harder debugging, increased cost

On Streaming Responses

The current implementation uses AppSync Query (synchronous), so agent responses display only after full generation completes. For long responses (file analysis, etc.), users wait several seconds.

Token-level streaming via AppSync Subscription (WebSocket) is technically possible, but implementation complexity (reconnection handling, partial response parsing, error management) increases. I've deferred this as a next step for when latency becomes a problem.

Security Model (AI Agent Specific)

Layer	Implementation	Intent
Authorization	Cognito Groups (authenticated)	All authenticated users can use chat
Tool permissions	Agent autonomously executes read-only tools	Write operations require HITL
Input filter	Bedrock Guardrails (input)	Prompt injection countermeasure
Output filter	Bedrock Guardrails (output)	PII masking
PHI path	isPhiPath() check	Don't pass `/dicom/` etc. file contents to agent
Cost control	Only last 10 messages sent as history	Token consumption cap
Audit	CloudTrail + DynamoDB session logs	Track "who asked what"

Monitoring

Recommended operational metrics:

Metric	Source	Alert Threshold
Agent Orchestrator error rate	Lambda CloudWatch Metrics	> 5% warrants investigation
Response latency (p99)	Lambda Duration	> 10s check model or tool side
Bedrock Throttling	Bedrock CloudWatch Metrics	ThrottledCount > 0 consider provisioned throughput
DynamoDB write throttling	DynamoDB ConsumedWriteCapacityUnits	On-demand auto-scales, typically no issue

Key design decision: The agent can propose storage-admin operations, but execution is authorized against the user's Cognito Group. If a regular user asks the agent "enable SnapLock," it fails with an authorization error. The agent explains "you don't have permission for this operation."

Compliance note: When acting on AI responses about regulated data, the final judgment responsibility lies with the human who approved the operation. Agent responses are "assistive information" — not substitutes for legal or regulatory judgment. Audit trail via CloudTrail + DynamoDB session logs records "who asked what, and what was approved."

Frequently Asked Questions

The following addresses common questions from different perspectives.

Cost note: "What's the token cost per conversation? How do I control context window size?" — With Nova Lite, ~$0.005–$0.02/conversation (varies by tool call count). The history count is hardcoded as messages.slice(-10) in AgentChat.tsx. Lowering it lowers cost and loses context in long conversations. Adjust it to balance cost vs. context retention.

Security note: "What data leaves the VPC? Can Bedrock see file contents?" — File contents are included in Bedrock API requests, but Bedrock does not use data for model training. With VPC endpoint configuration, data stays within the VPC. S3 AP access in Internet-origin configuration also completes within the same region.

Operations note: "How do I add a new MCP tool? What's the deploy process?" — New tools are implemented as Lambda functions and registered as targets in AgentCore Gateway. Deploy via sam deploy or Amplify CI/CD pipeline. See AgentCore MCP Tools Reference for details.

End-user note: "Can I just ask 'find my expense report from last month' and it works?" — Yes. With Knowledge mode and semantic search enabled, you can search by intent without knowing file names. File Agent mode can also traverse folder structures to locate files.

Scaling note: "How does this work with multiple Knowledge Bases across departments?" — KB data sources can be specified per-volume via S3 AP, so you can create separate KBs per department volume. The agent switches target KB via bedrockKbId configuration.

Front-end note: "How do I customize the chat UI? Can I add my own components?" — The chat UI is implemented as a React component (AgentChat.tsx), supporting style customization and custom message renderers. Tool call result displays are also extensible via the ToolCallTimeline component.

Staged Adoption Steps

All AI agent features support DemoMode. They work without FSx for ONTAP for evaluation.

Step 1: No KB, No Agent (File Search Only)

Leave bedrockKbId empty in portal-config.ts. Only keyword search available.

Step 2: Add Bedrock Knowledge Base

Register FSx for ONTAP S3 AP as KB data source
Set bedrockKbId in portal-config.ts
Semantic search mode becomes active

Step 3: Enable AgentChat (File Agent Mode)

Deploy MCP tool Lambda. list_files, read_file, search_files become available.

Step 4: Multi-Agent Mode and HITL

Add admin operation tools (block_user, enable_snaplock, etc.). HITL modal activates automatically.

Step 5: Agent Teams and Custom Agents

Define team-specific analysis patterns as agents. Share via Agent Directory.

Resources

Summary and Next Steps

I integrated AI agents into the "file portal + storage operations" foundation built in Parts 1 and 2:

Feature	What became possible
AgentChat (3 modes)	Complete file operations, analysis, and admin ops in natural language
SemanticSearch	Solve "where did I put that file?" with vector search
Agent Directory/Creator	Codify team-specific analysis patterns as agents
Agent Teams	Collaboratively process complex tasks with multiple agents
HITL (ActionApproval)	Execute destructive operations only after human approval
Bedrock Guardrails	Structurally prevent PII leaks and harmful responses

This series documented the portal's evolution:

Part 1 covered the file portal foundation (S3 AP + Amplify Gen2 / Nextcloud)
Part 2 added storage operations (ARP/AI, Tamperproof, regulatory retention)
Part 3 integrated AI agents (natural language operations, semantic search, multi-agent)
Parts 4 to 6 continue into delegating operations and recording them, what the running cluster stopped, and what was left off the screen

Through the progression of "browse files" → "manage storage" → "talk to AI," the portal reached a point where daily operations and data utilization complete in the browser — without opening ONTAP System Manager or the CLI.

All code is published in the GitHub repository.

For next steps, I'd suggest starting with keyword search in DemoMode, then adding a Bedrock Knowledge Base for semantic search, and layering on MCP tools and HITL after that. When you move on to a production connection, see the PoC → Production Guide.

I hope this post helps someone out there.

See you next time.

Running ARP/AI and SnapLock on FSx for ONTAP from a browser — ransomware detection and WORM retention (File Portal Part 2)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:20:41 +0000

Run detection through containment from the screen — ARP/AI, SnapLock, audit logs (part 2 of 6)

A Japanese version of this article is available: 日本語版

Introduction

In Part 1, I looked at putting a file portal on top of the S3 Access Point of Amazon FSx for NetApp ONTAP (hereafter FSx for ONTAP S3 AP, or S3 AP), comparing Amplify Gen2 and Nextcloud as two approaches. The portal at that point stopped at "browse the files" — the moment you wanted to know something about the storage layer, you had to go back to the ONTAP CLI or the REST API.

Once people started using it, the first two things that came back were "I want to check whether ARP has detected anything myself" and "is this snapshot actually locked?" Both requests came from team members outside storage administration, and the sticking point was the same: you cannot check any of it without a VPN connection to the management LIF. So I added ONTAP operations features to the Amplify Gen2 portal, up to the point where health checks and incident first response can be done entirely in a browser.

Here's the conclusion up front:

ONTAP System Manager-equivalent admin operations can be executed from a browser through AppSync + Lambda
Separating storage-admin from regular users with Cognito Groups gives you a safe split between read-only viewing and change operations
Managing ransomware response as four states — Detected → Contained → Investigating → Resolved — removes the hesitation from first response
Turning regulatory retention periods (FISC 7 years / SOX 5 years / HIPAA 6 years) into presets prevents day-count mistakes

In this article, I'll cover the design of the storage operations features embedded in the portal, the ONTAP REST API implementation behind them, and the points I was careful about when handling irreversible operations.

Who this is for: anyone who wants to hand ransomware response and retention settings to people who are not the storage administrator. It is a record of how much to show someone who never touches the ONTAP CLI.

What this article does not cover: the interpretation of the regulations themselves (that is a legal and compliance judgement), absolute performance figures, tenant isolation design.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Relationship to existing tools: This portal makes the same storage management operations previously available only through the ONTAP CLI or REST API accessible from a browser with Cognito authentication. It is not a replacement for those — it's an additional layer that opens the same operations to people who should not hold cluster-administrator credentials.

The features added are as follows:

Storage Dashboard: a 4-card health overview
ARP/AI Incident Lifecycle: managed as four states — Detected → Contained → Investigating → Resolved
S3 Object Lock Configuration UI: select a bucket, specify a retention mode, apply
PHI Guardrail: refuses AI processing on a path-name match such as /dicom/. It does not look at contents
EMS Events: ONTAP alerts shown on screen (fetched when the dashboard is opened)
Regulatory Retention Presets: FISC 7-year / SOX 5-year / HIPAA 6-year
Welcome Modal: first-time user onboarding
Audit Log: file access trail via CloudTrail, viewable in the UI
FlexClone Restore: recovery from a snapshot, selected in the UI
Athena Query: SQL analysis on NAS data

Scope: this article is an implementation record of adding "operations features usable by members outside storage administration" to the file portal built in Part 1. Part 3 covers AI agent integration.

Repository: solutions/amplify-portal/

Why Embed Admin Operations in a Portal

For FSx for ONTAP, the management interfaces you can reach without routing through an additional third-party SaaS are three: the AWS Management Console / Amazon FSx API, the ONTAP CLI over SSH, and the ONTAP REST API. ONTAP System Manager is not among them — System Manager support for FSx for ONTAP is delivered through the vendor's SaaS console, and that SaaS covers FSx for ONTAP only in its SaaS-connected mode (sources in management interfaces).

So learning anything about the ONTAP side means SSH to the ONTAP CLI or a REST call, both of which assume cluster-administrator credentials. When team members outside storage administration (security staff, compliance officers, data protection teams) want to "check ARP status" or "verify whether a snapshot is locked," the way to let them is to hand over an SSH key.

Motivation for embedding admin operations:

Security staff should be able to check ransomware detection state without entering the ONTAP CLI, and execute containment actions
Compliance officers should be able to verify retention settings and audit logs themselves
Operations teams should be able to check ONTAP alerts without opening another tool

Cognito Groups (storage-admin / authenticated) separate authorization — regular users are read-only, only admins can execute changes.

What You Used to Do vs What You Can Do Now

Operation	Previous Method	Portal Method
Check ARP threats and contain	CLI: `security anti-ransomware volume show` → block by hand	Detected on the dashboard, contained from there
WORM-lock a snapshot	CLI: `volume snapshot modify-retention` or direct REST API call	Lock panel → select preset → apply
DR failover with SnapMirror	CLI: `snapmirror break` / `snapmirror resync`	Status list → click action button

Storage Dashboard — Admin Landing Page

The first thing admins see after login is a 4-card health dashboard. This follows ONTAP System Manager's "dashboard first" pattern.

┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ 💾 12       │  │ 🛡️ 8         │  │ 🔐 5        │  │ 📊 3.2x 　   │
│ Volumes     │  │ARP Protected│  │ Locked Snaps│  │ Efficiency  │
│ Avg: 62%    │  │✅ No threats│  │ Tamperproof │  │ 69% saved   │
└─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘

The 4 Cards

Card	Shows	Navigates to
Volume Capacity	Volume count + average utilization	Volume Manager
ARP/AI Protection	Protected volumes + threat count	ARP Admin
Locked Snapshots	Locked snapshot count	Snapshot Admin
Storage Efficiency	Dedup/compression ratio + savings	Efficiency Panel

Implementation

Four APIs fetched in parallel with Promise.allSettled. If one fails (e.g., ARP not configured), remaining cards still render.

const [volResp, arpResp, snapResp, effResp] = await Promise.allSettled([
  adminQuery({ action: "listVolumes", params: JSON.stringify({}) }),
  adminQuery({ action: "listArpVolumes", params: JSON.stringify({}) }),
  protectionQuery({ action: "listSnapshots", params: JSON.stringify({ maxResults: 50 }) }),
  adminQuery({ action: "getEfficiencyStats", params: JSON.stringify({}) }),
]);

Why Promise.allSettled over Promise.all: with Promise.all, one failure rejects everything. On a storage dashboard, "ARP not configured" is a normal state, not an error.

ARP/AI Incident Lifecycle

After ONTAP Autonomous Ransomware Protection (ARP/AI) detects a threat, the response flow is managed across 4 states.

State Transitions

🔴 Detected
  → 🟠 Contained
    → 🟡 Investigating
      → 🟢 Resolved

Light theme shown. A dark theme version is available.

What's Recorded at Each State

State	Timestamp	Additional Data
Detected	detectedAt	—
Contained	containedAt	blockedUsers[], blockedIps[], snapshotName
Investigating	—	notes (free text)
Resolved	resolvedAt	—

Containment Actions

Actions executable directly from the portal:

Action	ONTAP REST API	Effect
Block SMB User	name-mapping deny (win→unix)	Immediately denies target user's SMB access
Block NFS IP	export-policy deny rule	Immediately denies NFS access from target IP
Full Containment	snapshot + block + disconnect	Preserves snapshot + blocks all + disconnects sessions
Unblock	delete name-mapping/export-policy rules	Recovery after investigation

This implements the equivalent of NetApp DII Storage Workload Security blocking directly via ONTAP REST API.

Implementation

// useIncidentState hook — persists per-volume in localStorage
export type IncidentState = "none" | "detected" | "contained" | "investigating" | "resolved";

export function useIncidentState(volumeName: string) {
  const [incident, setIncident] = useState<IncidentRecord>(() => loadIncident(volumeName));

  const markContained = useCallback((snapshot?: string, users?: string[], ips?: string[]) => {
    transition("contained", { snapshotName: snapshot, blockedUsers: users || [], blockedIps: ips || [] });
  }, [transition]);

  // ...
}

The UI displays a state badge with the next action button changing dynamically:

[🔴 Detected] → [Execute Containment] button
[🟠 Contained] → [Start Investigation] button
[🟡 Investigating] → [Resolve] button
[🟢 Resolved] → (complete)

Current limitation: Incident state is stored in localStorage and not shared across browsers. There is also a risk of losing incident state if the browser crashes. DynamoDB persistence is recommended for production use so that multiple members see the same state.

Incident response note: After executing containment actions, integrate with your existing incident runbooks (PagerDuty, OpsGenie, etc.). Portal containment is a first-response action and should be positioned as part of your full incident management workflow.

Security note: Executing containment actions (SMB block, NFS IP block) requires membership in the storage-admin group. Unblocking requires the same privilege, so even if a regular user's browser session is compromised, the attacker cannot lift a block. Compromise of a storage-admin account itself is guarded by Cognito MFA.

Multi-tenant note: When multiple teams (tenants) share the same portal, ensure tenant A cannot view tenant B's ARP alerts by combining Cognito Groups with volume-level access control. In the current implementation, all storage-admin group members can view ARP status for all volumes. If tenant isolation is required, consider deploying separate portal instances per SVM.

PHI Guardrail — refusing AI processing by path name

Stops "PHI went to a managed AI service by accident" in a HIPAA environment, using a folder naming convention. The decision is a path-name match and nothing else — not the file contents, not the extension, not the tags.

Where this sits, first. It is a guard against a mistake, not the boundary itself. The boundary is IAM and the Cognito group path prefixes: a key the caller cannot reach cannot be handed to an AI either. The folder convention sits in front of that, to keep a key the caller can reach from being submitted (the predicate itself is portal_regulated_path.py).

Hiding a button does not stop a call to AppSync. So the check sits both in the browser and in the five endpoints that hand file contents outward — asking about a file, Textract, Comprehend, Rekognition label detection, and the agent's file read. The list of endpoints is pinned by a test, so a sixth one added later cannot ship without it.

How It Works

Files in paths containing /dicom/, /phi/, /pii/, /hipaa/, or /protected-health/ have their AI processing button automatically disabled.

const isPhiPath = (path: string): boolean => {
  const lower = path.toLowerCase();
  return /\/(dicom|phi|pii|hipaa|protected-health)[\/-]/.test(`/${lower}`) ||
         lower.startsWith("dicom/") || lower.startsWith("phi/") || lower.startsWith("pii/");
};

UI display:

Normal folder: [⚡ Run AI Processing]     ← clickable
PHI folder:    [🚫 PHI — AI Blocked]      ← disabled, not clickable

Design Intent

Guardrail, not detect-and-respond: rather than "processed it, then discovered it was PHI," this prevents processing from starting at all
Cannot be bypassed even by admins: the button itself is disabled regardless of permissions
Depends on folder naming convention: path-pattern based, so it presupposes organizational folder structure rules

Important limitation: This guardrail blocks based on folder path only — it does NOT scan file contents. A file like /contracts/sensitive-patient.pdf placed outside PHI paths will not be blocked. For content-based PHI detection, combine with Bedrock Guardrails (response filtering) described in Part 3. Path guardrail + content filtering gives you ideal 2-layer defense.

HIPAA note: Using this portal in a HIPAA environment presupposes a Business Associate Agreement (BAA) with AWS. Cognito, Lambda, S3, Step Functions, and Bedrock are all HIPAA eligible services (checked against the list dated 2026-09-03). That list is updated, so read the original at the point you adopt this. The PHI guardrail refuses in the browser and in the five endpoints that hand file contents to a managed AI service. Both the BAA and technical controls are still required.

S3 Object Lock — WORM Protection for Output Buckets

AI processing results and compliance reports are stored in standard S3 buckets outside FSx for ONTAP. The portal provides a UI to configure S3 Object Lock on these output buckets.

3-Tab Lock Panel

Tab	Target	Purpose
ONTAP SnapLock	Volumes	WORM protection for NAS data (shared across NFS/SMB/S3 AP)
S3 Object Lock	S3 Buckets	WORM protection for AI processing results
Tamperproof Snapshot	Snapshots	Tamper prevention for specific snapshots

S3 Object Lock Configuration Flow

1. Fetch bucket list → select from dropdown
2. Choose mode:
   - Governance: Authorized users can override (recommended for AI output)
   - Compliance: No one, including root, can delete until retention expires (regulatory archives)
3. Specify retention days
4. [Apply] sets Object Lock Configuration

When to Use SnapLock vs S3 Object Lock

Characteristic	ONTAP SnapLock	S3 Object Lock
Target	Files on NAS volumes	Objects in S3 buckets
Access protocols	NFS/SMB/S3 AP	S3 API
Use case	Source data WORM protection	AI results/reports protection
Regulatory	SEC 17a-4, FISC, HIPAA, NARA	SEC 17a-4, HIPAA

Combining both achieves "source data AND processing results both tamper-proof."

Compliance framework note: The AWS services used by this portal (Cognito, Lambda, S3, Step Functions, Bedrock, AppSync) are in scope for ISMAP and SOC 2 (as of 2026-09; scope changes). FedRAMP environments for US government agencies require deployment in AWS GovCloud regions. Confirm compliance requirements with your legal and compliance teams — this portal is a technical implementation pattern and does not provide compliance judgments.

Regulatory Retention Period Presets

The Tamperproof Snapshot lock form offers retention periods as dropdown selections.

Preset List

Preset	Retention Days	Regulatory Basis
30 days	30	Short-term validation
90 days	90	Quarterly
1 year	365	Annual
SOX/J-SOX	1,825 (5 years)	Securities Exchange Act — financial records retention
HIPAA	2,192 (6 years)	Medical records minimum retention
FISC	2,557 (7 years)	Center for Financial Industry Information Systems standard, Chapter 9

Compliance note: FISC/SOX/HIPAA presets are provided for technical convenience. Whether the regulatory mapping and retention periods apply to your organization's specific situation should be validated with your legal and compliance teams. Regulatory interpretation varies by industry, business scope, and jurisdiction.

Why Presets Are Needed

SnapLock retention settings are irreversible. Once a snapshot is locked, the retention period cannot be shortened.

Prevents "how many days is 5 years?" calculation errors
Each preset shows a tooltip with regulatory basis
Minimizes risk of incorrect selection (can extend but never shorten)

<option value={1825} title="SOX/J-SOX: Financial records must be retained for 5 years">
  1,825 days — SOX/J-SOX (5 years)
</option>
<option value={2192} title="HIPAA: Medical records require minimum 6-year retention">
  2,192 days — HIPAA (6 years)
</option>
<option value={2557} title="FISC: Financial institution data requires 7-year retention">
  2,557 days — FISC (7 years)
</option>

EMS Events — ONTAP Alert Viewer

Retrieves events from ONTAP's Event Management System (EMS) with severity alert/error/emergency and displays them in the admin panel. The filtering happens in the ONTAP query (message.severity), and at most the 50 most recent are fetched, newest first.

There is no duplicate suppression. A repeating message fills all 50 rows. It is read-only, with no acknowledge action. If you need counts collapsed, rows capped or events acknowledged, pair it with pushing EMS events out through EventBridge + SNS.

Backend Implementation

def _get_ems_events(http, headers, event):
    """ONTAP REST: GET /api/support/ems/events"""
    max_records = min(event.get("maxRecords", 20), 50)
    severity_filter = event.get("severity", "alert,error,emergency")

    query = f"/support/ems/events?max_records={max_records}"
    query += f"&severity={severity_filter}"
    query += "&order_by=time desc"
    query += "&fields=time,severity,message.name,message.text,node.name"

    data = _ontap_request(http, headers, "GET", query)
    events = [
        {
            "time": e.get("time", ""),
            "severity": e.get("severity", ""),
            "messageName": e.get("message", {}).get("name", ""),
            "messageText": e.get("message", {}).get("text", ""),
            "node": e.get("node", {}).get("name", ""),
        }
        for e in data.get("records", [])
    ]
    return {"events": events, "count": len(events)}

Example Events

Severity	Message Name	Meaning
emergency	ha.takeover.byPartner	HA partner takeover
alert	raid.disk.predictiveFailure	Disk predictive failure
error	scsiblade.san.netLIFDown	SAN LIF down
alert	arw.volume.attack	ARP detected ransomware attack

Storage administrators can check recent alerts directly in the portal without opening an ONTAP CLI session.

Refresh frequency note: EMS events are fetched on demand when the dashboard is displayed (not by periodic polling). If you need real-time alerting, push EMS events via EventBridge + SNS instead (see fsxn-observability-integrations). The portal is for "checking recent events"; for "detecting in real time," consider pairing it with a separate observability stack.

Welcome Modal — First-Time Onboarding

First-time users see a 3-step guided tour.

3 Steps

📂 File Browsing — Browse and search NAS files from your browser
⚡ AI Processing — Select files and trigger AI/ML workflows
🔒 Data Protection — Snapshots, WORM locks, ransomware protection

Implementation

export function WelcomeModal() {
  const [dismissed, setDismissed] = useState(() => {
    try { return localStorage.getItem("portal-welcome-dismissed") === "true"; }
    catch { return false; }
  });

  if (dismissed) return null;

  // 3-step carousel with dot navigation
  // "Don't show again" checkbox → localStorage persistence
}

Design Intent

Standard onboarding pattern in SaaS products
Often omitted in internal tools, but highly effective for conveying "what this portal can do" in 10 seconds
localStorage state management, works without backend changes

Resource Management — 20-Panel Card-Grid Admin

ONTAP admin operations organized into 5 categories × 20 panels.

Category Structure

Category	Panels
🗄️ Storage	Volumes / 🧬 FlexClone / ⚡ FlexCache / Qtrees / Quotas / Efficiency
🔐 Access Control	Export Policies / SMB Shares / 👤 Local Users / 🔀 Name Mapping / QoS
🛡️ Data Protection	ARP/AI / Snapshots / SnapLock / 📡 FPolicy / 🦠 Vscan / 🪞 SnapMirror
🖥️ Cluster	🔗 Peering / 🖥️ Cluster Information
🤖 Services	AI Settings

Each panel displays as a card grid, clicking navigates to the detail view. Follows System Manager's card-based navigation.

The Cluster category covers the part the AWS Management Console does not expose. Cluster peering and SVM peering had to be done from the ONTAP CLI or by hand-writing REST calls, which is a real operational cost for an AWS-centric team that otherwise never leaves the console.

New Panel Highlights

Vscan — "Zero to Configured" Setup Guidance

Vscan (antivirus scanning) has the highest setup barrier of any ONTAP feature — it requires external Windows/Linux servers and antivirus vendor licenses. When unconfigured, the Vscan panel displays a 5-step guided wizard:

Vendor selection (6-vendor comparison table with license page links)
NetApp Antivirus Connector download
EC2 deployment (architecture diagram + AWS Blog/GitHub samples)
ONTAP CLI commands (scanner-pool / policy / enable)
Verification in this panel

Even someone starting from "what is Vscan?" can follow the setup path directly from the portal.

One correction from the first version of this panel: the guidance only rendered while Vscan was disabled, so it disappeared the moment Vscan came up — exactly when you still want the scanner-side steps and the interoperability matrix link. It now stays reachable from a toolbar toggle in both states.

FlexClone — Instant Zero-Copy Clones

Create and split volume clones from the UI for ransomware recovery, forensics, or dev/test. Metadata-only copy completes in seconds with near-zero additional capacity.

SnapMirror — Replication Lifecycle Management

Monitor DR and cross-region replication relationships in the browser and execute actions (sync, break, resync, quiesce, resume, delete). Click a relationship to expand the last 10 transfer records (size, duration, success/failure). Lag times exceeding your RPO target display in red so they are noticed immediately.

DR planning note: SnapMirror Break/Resync operations are executable from the browser, but understand the RPO impact before clicking Break. After a break, the replication relationship is severed and resync requires delta transfer. For production environments, pair the confirmation dialog (implemented) with your team's DR runbook procedures.

FlexCache — Remote Read Cache CRUD

Create FlexCache volumes (async, with prepopulate path support), view the list (origin→cache arrow display), and delete (automated 3-step: unmount→offline→delete). When unconfigured, guidance is displayed with a datalist for selecting origin volumes.

Async polling note: FlexCache creation uses async polling to detect completion. If the user closes the browser during creation, the ONTAP-side operation continues. The created volume appears on the next list refresh — no data loss or inconsistent state results.

Cluster and SVM Peering — the gap the console leaves

Peering is a two-sided operation, and the authentication passphrase has to travel between the two clusters. The panel drives one side at a time.

The prerequisite that trips people up is the intercluster LIF, so it gets its own tab rather than being left as a footnote. Both clusters need at least one in up state, and TCP 11104, 11105 plus ICMP have to be allowed between their addresses — the intercluster LIF addresses, not the management LIF.

The flow is: check the LIFs, create the peer on one cluster with Generate passphrase, then enter that passphrase under Accept on the other. The generated value is shown once — ONTAP returns it in the creation response only, so the panel surfaces it in a banner you have to dismiss deliberately. Lose it and the peer has to be deleted and recreated.

SVM peers come after the cluster peer reaches available, and are accepted on the remote side without a passphrase. An SVM-level SnapMirror additionally needs the source subtype default and the destination dp_destination.

Cluster Information — nodes, licences, LIFs, protocols, DNS, jobs

Four tabs: overview (cluster name and version, nodes, licences), interfaces (LIF list with enable/disable), services (NFS, SMB and S3 state with enable/disable, plus the SVM DNS domains and servers), and jobs.

The overview tab is quieter than that list suggests. On the cluster I tested (ONTAP 9.17.1P7D1) both /cluster/nodes and /cluster/licensing/licenses returned zero records with no error, because AWS manages the cluster rather than the tenant. The first version of the panel rendered that as a bare "No nodes", which reads as a failure, so it now states that an empty list is expected here. The cluster name and ONTAP version come from /cluster and are still populated.

The jobs tab matters more than it looks. FlexCache creation, FlexClone split, SnapMirror transfers and peering all run as asynchronous ONTAP jobs, so this is where their progress and, more usefully, their failure reasons show up.

DNS note: an AD-joined SVM resolves its domain controllers through the servers set here. A wrong value breaks SMB, and on an AD-joined SVM it also makes S3 Access Point data operations fail with AccessDenied — while HeadBucket keeps succeeding, which sends you looking at IAM instead of the file system layer.

Destructive operations are gated in both layers

SnapMirror break, resync and delete, Vscan and FPolicy policy deletion, peer deletion, and disabling a LIF or a protocol all show an inline confirmation row before anything is sent.

The confirmation is not only in the UI. The Lambda refuses the same actions unless confirm=true is present, so a direct call that bypasses the browser is rejected identically. That split matters: a UI-only guard is a suggestion, not a control.

Getting this wrong is instructive. In the first cut, the Vscan and FPolicy delete buttons sent no confirm flag and had no confirmation row, and the handler did not check for one either — while the documentation already claimed the operation was confirm-gated. The button appeared to work and the docs looked right. The fix was to make all three agree, and to pin the contract in tests: for every confirm-gated action, assert it refuses without confirm and succeeds with exactly the parameters the UI sends. The second half of that assertion is the one that would have caught it.

Security Model

Layer	Implementation
Authentication	Cognito User Pool + MFA (TOTP/SMS)
Authorization (API)	Cognito Groups (`storage-admin` / `authenticated`)
Authorization (Files)	S3 AP + UNIX/Windows file system identity
Transport encryption	HTTPS (AppSync) + TLS 1.2 (ONTAP REST API)
Secrets management	Secrets Manager (ONTAP credentials)
Audit	CloudTrail S3 data events
WORM	SnapLock (Compliance/Enterprise) + S3 Object Lock
Ransomware protection	ONTAP ARP/AI + portal containment actions
PHI protection	Path-based guardrail (AI processing block)

Only users in the storage-admin group can execute admin operations (volume creation, SnapLock configuration, ARP containment, etc.). Regular users are limited to file browsing and AI processing.

Enterprise SSO note: Cognito User Pool supports SAML 2.0 and OIDC federation. To integrate with an existing IdP such as Okta, Azure AD, or Google Workspace, see Cognito's Adding SAML identity providers. Group mapping lets you map IdP groups to the Cognito storage-admin group.

Infrastructure protection note: Amplify Hosting is served via Amazon CloudFront, so AWS Shield Standard (DDoS protection) is automatically applied. If additional WAF rules are needed, attach AWS WAF to the CloudFront distribution.

Usage Scenarios

Scenario 1: Ransomware Response

1. Storage Dashboard shows 🚨 ARP Threats: 1
2. Navigate to ARP/AI panel → view threat details
3. Click [Contain] → SMB user blocked + snapshot taken
4. State transitions to 🟠 Contained
5. After forensic analysis, [Investigation Complete] → [Resolve]
6. Unblock to resume normal operations

Scenario 2: Compliance Audit

1. Lock panel → SnapLock tab to review Compliance volume list
2. Tamperproof tab to apply FISC 7-year lock to snapshots
3. S3 Object Lock tab to verify output bucket retention settings
4. Audit Trail to review operation history

Scenario 3: Daily Monitoring

1. Storage Dashboard — glance at 4 cards (30 seconds)
2. If any volume exceeds 85% capacity, drill into Volume Manager
3. Check EMS Events for recent alerts
4. No issues → done

Design Trade-offs

Design Decision	Benefit	Trade-off
localStorage (Incident State)	Simple deployment	Not shared across browsers; incident state can be lost if the browser crashes. DynamoDB migration recommended for production
PHI path regex	Simple to implement	Depends on the folder naming convention being followed. Contents are not inspected
Generic dispatch	Avoids CFn 1MB limit	IAM policy granularity is coarser
VPC split	Cold start optimization	Two types of Lambda required
Promise.allSettled (Dashboard)	Resilient to partial failures	Failed cards show 0
Regulatory presets (hardcoded)	Ready to use immediately	Code change needed if regulations update

Rollback Procedure

Recovery paths if a portal update causes problems:

Situation	Rollback method
Frontend UI issue	Revert to the previous build from the Amplify Hosting console
Lambda function issue	`git revert` + `git push` triggers automatic redeploy (Amplify Gen2)
ONTAP configuration change issue	Restore directly via ONTAP REST API (export-policy, name-mapping, etc.)
Cognito configuration change	CDK stack rollback is not available. Revert manually to the previous state

Because frontend and backend are both under Git management, git revert + git push rolls back immediately. Some ONTAP-side changes (such as enabling SnapLock) are irreversible, so verification before execution matters.

8-Language Support (i18n)

All admin features included, the entire portal supports 8 languages:

Code	Language
`ja`	日本語
`en`	English
`ko`	한국어
`zh-CN`	简体中文
`zh-TW`	繁體中文
`fr`	Français
`de`	Deutsch
`es`	Español

Technical terms (ONTAP, SnapLock, FlexClone, S3 AP, ARP/AI, WORM, FISC, SOX, HIPAA) are not translated. Browser navigator.language auto-detection with instant switching via the header language picker.

Audit Log — Checking "Who Accessed What and When" in the UI

The most frequent request from compliance officers was "I want to check the file access trail myself." Grepping CloudTrail logs from the CLI is not realistic for anyone outside the security team.

The portal's Audit tab runs filtered queries:

Filter	Example
File path	Show only accesses under `/contracts/`
Event type	READ / WRITE / ALL
Time range	2026-07-01 – 2026-07-28

Architecture

AppSync Query → Lambda → Athena SQL → CloudTrail S3 Data Event logs

Light theme shown. A dark theme version is available.

CloudTrail S3 data events are registered as a Glue table and queried with Athena. Results display in table form, and access denials (AccessDenied) are highlighted in red.

Prerequisites

Enable data events for the S3 AP ARN on a CloudTrail Trail
Create the Athena table via Glue Crawler or a manual CREATE TABLE
Set ATHENA_AUDIT_DATABASE, ATHENA_AUDIT_TABLE, and ATHENA_AUDIT_OUTPUT as Lambda environment variables

If these are unconfigured, the UI displays "configuration required" guidance with the setup steps. The design treats this as onboarding rather than an error.

Data Retention

Query results are stored in the Athena workgroup output location (S3). Retention of the original CloudTrail logs is controlled by the S3 bucket lifecycle policy on the Trail's bucket. Set retention periods such as FISC 7 years or HIPAA 6 years according to your audit requirements.

Cost note: If you store audit trails in DynamoDB, storage costs grow in large environments. Set a retention policy (TTL) and archive old records to S3 Glacier. CloudTrail log retention is controlled separately via the Trail's lifecycle policy.

FlexClone Restore — Instant Recovery from Snapshots

When recovering from ransomware damage or accidental deletion, the requirement is "I want to go back to the snapshot from before the infection." Previously this meant creating a FlexClone from the CLI and attaching an S3 AP as a sequence of manual operations.

In the portal, from the 📸 button on the Files tab:

1. Enter the snapshot name (e.g., daily.2026-07-18_0010)
2. Click [Clone & Attach]
3. Step Functions automatically runs FlexClone creation → S3 AP attach
4. FlexClone information appears on the Results tab

FlexClone Status Display

🔄 FlexClone Volume
  Volume:  clone-uc6-20260718-abc123
  Parent:  vol_data
  Status:  🟢 online
  Created: 2026-07-18 15:00:00
  Size:    128 MB
  S3 AP:   clone-uc6-abc123-s3alias

Use Cases

Scenario	Purpose
Ransomware recovery	Immediate access to a pre-infection snapshot
Forensics	Isolate a point-in-time copy for investigation
Testing	Validate against a clone without touching production data

FlexClone is a metadata-only copy, so it completes in seconds with near-zero additional capacity. Differential capacity is consumed only as writes occur.

Accessibility note: Confirmation dialogs for FlexCache deletion and volume deletion are displayed inline. For screen reader users, confirmation messages are announced via role="alertdialog" and aria-describedby. Keyboard navigation (Tab/Enter/Escape) is fully supported.

Throughput note: A FlexClone volume shares the throughput budget with its parent volume. If clones are used actively in parallel in production, account for the performance impact on the parent volume. For read-oriented uses such as forensics this is typically not a concern.

Athena Query — SQL Analytics on NAS Data

When files on FSx for ONTAP are cataloged via Glue Crawler, you can run SQL queries using Athena. The portal's "Analytics" tab lets you enter a database name and SQL, and execute directly.

There is no need to open the AWS Athena console separately — everything completes inside the portal.

UI Layout

┌──────────────────────────────────────────────────┐
│ SQL Query (Athena)      [default        ]        │
├──────────────────────────────────────────────────┤
│ 💡 How to use: catalog files on FSx for ONTAP    │
│ with Glue Crawler, then analyze them here with   │
│ SQL.                                             │
│ ▸ 📝 View query examples (collapsible)           │
├──────────────────────────────────────────────────┤
│ SHOW TABLES IN default                           │
│                                                  │
│ [Run query]                                      │
└──────────────────────────────────────────────────┘

Usage Flow

First time: run SHOW TABLES IN default to discover available tables
Once you know table names: execute specific SELECT queries
Results: displayed in table form

Practical Query Examples

-- Find large files (capacity planning)
SELECT key, size FROM default.s3_objects
WHERE size > 1000000000
ORDER BY size DESC LIMIT 20

-- Total size of a specific folder (project sizing)
SELECT SUM(size) as total_bytes
FROM default.s3_objects
WHERE key LIKE 'engineering/%'

-- Files modified in last 7 days (change tracking)
SELECT key, last_modified
FROM default.s3_objects
WHERE last_modified > current_date - interval '7' day

The portal UI includes an expandable "View query examples" section, so even users unfamiliar with SQL can get started by copy-pasting.

Not just "browse the files" but "ask questions of the data" — directly from the portal.

File Formats Glue Crawler Recognizes

Category	Formats
Structured data	CSV, TSV, JSON, Parquet, ORC, Avro
Logs	Apache/NGINX logs, CloudTrail JSON
Documents	— (text extraction requires Textract or Bedrock)

CAD/EDA binary files cannot be cataloged by Glue Crawler, but file metadata (size, last modified, path) is still retrievable.

Athena Cost Optimization Tips

Athena bills on the volume of data scanned ($5/TB). Ways to keep costs down:

Date partitions (year=/month=/day=) reduce scan volume when a WHERE clause narrows the time range
With columnar formats such as Parquet/ORC, specifying only the columns you need instead of SELECT * reduces cost
Make it a habit to always add a LIMIT clause to exploratory queries
Setting a query limit on the Athena workgroup (for example 10 GB/day) caps unexpected scan volume

Cost Awareness

Rough additional cost if every feature in this article is enabled. Unit prices vary by Region and change over time, so price an actual deployment from each service's pricing page:

Resource	Estimate	Notes
VPC Lambda (Admin)	~$5/month	Depends on invocation frequency. Negligible for daily checks
CloudTrail S3 Data Events	~$10–$50/month	Proportional to file access volume
Athena queries	$5/TB scanned	Audit queries scan little; partitions reduce it further
Secrets Manager	$0.40/secret/month	One ONTAP credential
DemoMode	$0	No FSx for ONTAP needed. Runs on S3 alone

The cost of FSx for ONTAP itself (~$194/month at 128 MBps) was covered in Part 1. The portal's incremental cost depends on usage frequency, but evaluating in DemoMode adds nothing.

Deployment Time Estimates

Step	Time	Prerequisite
DemoMode deploy	~15 min	Amplify CLI installed
Add VPC + ONTAP connectivity	~30 min	FSx for ONTAP file system running
Audit Log configuration	~20 min	CloudTrail + Glue Crawler
Full feature deploy	~60 min	All of the above

What to Expect After Deployment

Once the DemoMode deploy completes, the portal is reachable at https://<branch>.amplifyapp.com. Cognito sign-up → log in → the file list shows test data from the S3 bucket
Once VPC connectivity is added, the Storage Dashboard shows actual volume counts and utilization, and protected volumes appear in the ARP panel
Once all features are deployed, the Audit Log tab can run CloudTrail queries and the Lock panel shows SnapLock/Object Lock settings

Environment Variables and Configuration Parameters

Parameter	Description	Example Value
`DEMO_MODE`	Run without FSx for ONTAP	`true` (PoC) / `false` (production)
`ONTAP_SECRET_ARN`	Secrets Manager ARN for ONTAP credentials	`arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:fsxn-admin-XXXXXX`
`ATHENA_AUDIT_DATABASE`	Glue database name for audit queries	`cloudtrail_logs`
`ATHENA_AUDIT_TABLE`	Athena table name for audit queries	`s3_data_events`
`ATHENA_AUDIT_OUTPUT`	S3 location for Athena query results	`s3://my-audit-results/athena-output/`

PoC → Production Checklist

After evaluating in DemoMode, use this checklist when moving to production connectivity:

Category	Verification
Network	VPC subnet contains the FSx for ONTAP ENIs; Security Group allows TCP 443 (management LIF). Note: NFS (2049) / SMB (445) are for data access and are not needed for the portal's admin operations
Auth	Cognito User Pool MFA enabled; `storage-admin` group members confirmed
Secrets	ONTAP credentials registered in Secrets Manager
Audit	CloudTrail Trail with S3 data events enabled for the S3 AP ARN
Backup	`portal-config.ts` values in Git (secrets excluded)
Cost	CloudTrail data event estimate confirmed (proportional to access volume)
SSO	(Optional) SAML/OIDC federation configured if using an enterprise IdP

Role-Based Documentation (8 Languages × 3 Guides)

Alongside the feature additions, role-specific documentation was built. Based on usability principles (visibility of system status, task-oriented structure, ease of error recovery), 3 guides are provided in 8 languages:

Guide	Target Role	Key Content
User Guide	End users	Sign in, file operations, AI processing, FAQ
Compliance Guide	Security/Compliance officers	ARP verification, WORM checks, audit trail, PHI guardrail validation, regulatory mapping (FISC/HIPAA/SOX/NIST/ISO)
Quick Reference	All roles	1-page cheat sheet (navigation, tasks by role, status indicators, troubleshooting)

Supported languages: 日本語, English, 한국어, 简体中文, 繁體中文, Français, Deutsch, Español

Design choices for the compliance officer guide:

All tasks executable without storage-admin privileges (read-only access is sufficient)
Each task includes "Evidence for auditors" instructions
Explicit "What you cannot do" escalation table showing who to contact

FlexCache / SnapMirror Management

Direct ONTAP REST API Operations from the Browser

FlexCache, FlexClone, SnapMirror, Vscan, FPolicy, cluster and SVM peering, and the cluster services are all managed through the browser UI — 110 actions in total. A VPC-deployed Lambda connects to the FSx for ONTAP management endpoint via HTTPS, calling ONTAP REST API with credentials from Secrets Manager.

Browser → AppSync (Cognito auth) → Lambda (in VPC) → ONTAP REST API
                                                       ↓
                                                Secrets Manager
                                                (fsxadmin credentials)

Light theme shown. A dark theme version is available.

FlexCache Create UI

Select the origin volume from a dropdown (datalist), specify cache name, size, and prepopulate paths, then create. The experience mirrors ONTAP System Manager — accessible from any browser with Cognito authentication.

SnapMirror Lifecycle Management

The replication relationship list shows state badges (✅ Snapmirrored / 🔴 Broken-off / 🔄 Transferring / ⏸️ Paused) with context-sensitive action buttons. DR failover (Break → Resync) is fully accessible from the browser. Lag times exceeding your RPO target display in red with a ⚠️ RPO warning.

FPolicy — Event Definitions and Notification Policies

FPolicy is where file access notification is configured: an event definition
names the protocol and the operations to watch, and a policy subscribes to those
events and points at an engine. The panel exposes all of it — create, enable,
disable, delete — across three tabs for policies, events and connections.

Note the Delete button on the enabled policy. ONTAP refuses to delete a policy
while it is enabled, so the panel disables the control rather than letting the
call fail. Disable first, then delete. The same ordering applies through the
REST API, and it is the kind of constraint that is easier to encode in the UI
than to explain in a runbook.

Where the Portal Is Self-Contained, and Where It Needs Something Else

The containment actions in this portal are a port of ontap_response.py from a
separate repository, fsxn-observability-integrations
— the module docstring records it. The ONTAP mechanisms are identical:
name-mapping deny, export-policy deny rule, protective snapshot, CIFS session
disconnect. What differs is the trigger, and the layers around it.

Self-contained in the portal

Capability	Requires
Review ARP/AI state, attack probability and suspect files	The portal reaching the ONTAP REST API
Block and unblock an SMB user or an NFS client IP	`storage-admin` group
Create a protective snapshot; lock a snapshot (WORM)	Same
Disconnect CIFS sessions	An AD-joined SVM
List active blocks and lift them individually	—
Block expiry and automatic lifting (24 hours by default)	The block ledger (DynamoDB) being reachable
Apply the same block across several SVMs (`svms` or `allSvms`)	—
Audit file access that went through the S3 Access Point	CloudTrail data events + Athena
FlexClone a snapshot to browse it; diff two generations	—

Every one of these starts when a person clicks. Nothing in the portal
contains a threat unattended — expiry is the one exception, and it only ever
works in the direction of ending a lockout.

Needs something outside the portal

Goal	What it takes
Contain without waiting for a human	An SNS topic plus a response Lambda
Be told about a detection instead of finding it	EMS → webhook → SIEM or an observability platform
Cut NFS off without the client cache window	A VPC NACL deny rule (network layer)
Judge a recovery point before restoring from it	A verification workflow (FlexClone + isolated scan)
Detect anomalies against a per-user ML baseline	A SIEM with anomaly detection, or a dedicated storage security product
Trace file access that arrived over NFS or SMB directly	An ONTAP audit log / FPolicy delivery pipeline

The audit boundary is the one most likely to be misread. The portal's audit trail
reads CloudTrail S3 data events for the S3 Access Point. Access that arrived over
NFS or SMB does not appear there at all — that needs ONTAP's own audit log or
FPolicy events. The two are complementary, not substitutes.

The delivery pipeline is still yours to build, but there is now somewhere for it
to deliver to. Once FPolicy (or Transfer Family) publishes to EventBridge, a
bridge Lambda writes what arrives into a notification table and a Folder
Watch section shows the events under the prefixes you are watching. It is
enabled by an admin toggle and off by default: until an admin states that a
publisher exists, turning it on would only ever show an inbox that cannot fill.
Filtering applies the Cognito group path boundary first and each user’s own
watches second. A watch is your own record, so you may register /, but that
cannot reveal anything outside the group boundary.

So the portal is the hands of incident response: it puts the ONTAP
containment actions in a browser, behind Cognito groups, with confirmation and an
audit trail. If you need something watching around the clock and moving those
hands for you, detection and response belong in a pipeline. Conversely, if you
already detect in a SIEM and only lack a way to stop it at the storage layer, an
SNS-triggered response Lambda fits that shape better than a portal button.

One click was too few

Writing this section is what exposed the gap worth fixing. A block removes a
principal's data access across the whole SVM. The first version executed on a
single click, with no confirmation and no confirm parameter in the payload —
the only guards were the Cognito group and a protected-accounts list. At the time
there was also no expiry, so a block stayed until a person lifted it.

There are now two gates. The UI asks, with wording specific to the action,
because the consequences differ: an NFS block is subject to client-side attribute
caching for up to 60 seconds, an SMB block is not, and disconnecting sessions on
its own does not stop the next login. The Lambda then refuses any call that
arrives without confirm: true, in front of the ONTAP client construction — a
dialog in a browser is a suggestion, and anything calling AppSync directly skips
it.

Session disconnect became its own action in the same pass. It existed in the
handler but only ever ran as step 4 of full containment, so an operator who had
already blocked a user had no way to cut the sessions that survived the block.
Order matters: block first, then disconnect. Disconnecting before the block is in
place just invites a successful reconnect.

Lessons Learned

Async Operations with `return_timeout=0`

FlexCache creation is an asynchronous ONTAP job taking 30-120 seconds. By specifying return_timeout=0, ONTAP returns immediately with 202 Accepted + job UUID, preventing Lambda timeout. The UI auto-refreshes the list at 10s/30s/60s intervals.

One Wrong Field Name Empties the Whole List

The SnapMirror list was requesting fields=...,last_transfer_type,last_transfer_size. last_transfer_size is not a field on /snapmirror/relationships, and ONTAP rejects the entire request when a single field name is unknown:

The value "last_transfer_size" is invalid for field "fields" (<field,...>)

So every relationship vanished — not with a visible error, but as an empty list. What made this survive the test suite is worth noting: the mock ONTAP in the unit tests returns records regardless of which fields are requested, so a test that asserts on the mapped response passes either way. The fix was to assert on the outgoing query string instead. Per-transfer byte counts were already available from /snapmirror/relationships/{uuid}/transfers, so nothing was lost by dropping the field.

Automated 3-Step FlexCache Deletion

Mounted FlexCache volumes cannot be deleted directly. The portal automates:

Remove junction path (unmount)
Set volume offline
Delete FlexCache (async)

Credential Synchronization

When changing the fsxadmin password, update both the FSx for ONTAP API and Secrets Manager simultaneously. Updating only one side causes authentication failures that can trigger ONTAP's account lockout mechanism. Recovery: aws fsx update-file-system password reset → Secrets Manager sync.

FSx for ONTAP Specifics

Item	FSx for ONTAP	On-prem ONTAP
Aggregate specification	Not needed (auto-selected)	Required
SVM creation	AWS API only	CLI/REST available
Intra-cluster FlexCache	Supported (no peering needed)	Supported
fsxadmin password reset	`aws fsx update-file-system`	`security login password`

Coexistence with Existing Tools

This portal is not a replacement for the ONTAP CLI or REST API. It makes a subset of those operations accessible from a browser UI with Cognito authentication — an additional management layer, not a substitute.

Scenario	Portal's Role
Using cloud file-sharing SaaS	AI processing and audit for large-scale NAS data only
Running self-hosted file sharing	Add S3 AP as External Storage for management operations
Using hybrid file services	Add data protection and ARP/AI visibility layer
Operating daily through the ONTAP CLI / REST API	Provide visibility to non-storage-admin team members

The portal provides a management layer for NAS-specific capabilities (Snapshot, FlexClone, FlexCache, SnapMirror, ARP/AI) accessible from the browser. Adoption can be incremental — management operations only, AI processing only, or the full suite — depending on existing workflows.

PoC → Production Migration Flow

After evaluating in DemoMode (no FSx for ONTAP required), migrate to production connectivity in three phases:

┌────────────────┐     ┌────────────────┐     ┌────────────────┐
│ Phase 1: PoC   │     │ Phase 2: VPC   │     │ Phase 3: Prod  │
│ (~15 min)      │     │   Connectivity │     │   Hardening    │
│                │     │ (~30 min)      │     │ (~60 min)      │
│ DemoMode=true  │ ──→ │ ONTAP mgmt LIF │ ──→ │ Least-privilege│
│ S3 bucket      │     │ Secrets Mgr    │     │ MFA required   │
│ Auth: Cognito  │     │ VPC Endpoint   │     │ WAF added      │
│                │     │                │     │ Audit enabled  │
└────────────────┘     └────────────────┘     └────────────────┘

Light theme shown. A dark theme version is available.

Phase	Time	Added Cost	What You Get	What Changes	What Stays the Same
1 (PoC)	~15 min	$0	UI/UX evaluation, AI processing test	Nothing (fresh deploy)	—
2 (VPC)	~30 min	Lambda VPC ~$5/mo	ONTAP admin operations, dashboard	Lambda VPC config added, Secrets Manager registered	Frontend UI, Cognito settings, S3 bucket config
3 (Prod)	~60 min	CloudTrail ~$10–50/mo	Audit, WAF, MFA, least-privilege IAM	IAM policy tightening, WAF attachment, MFA enforcement	Application code, ONTAP connection settings

Post-Deployment Verification

Expected state after each phase completes:

After Phase 1: you can log in to the portal, and DemoMode file listing and AI processing tests work. The Storage Dashboard still shows "not connected"
After Phase 2: the Storage Dashboard shows actual volume information, and ONTAP data appears in the ARP panel and EMS Events
After Phase 3: login is impossible without MFA, CloudTrail records all operations, and WAF applies rate limiting

Irreversible operations warning: The following operations cannot be undone once executed. Verify your organization's policies before enabling them in Phase 2+:

SnapLock Compliance enablement: Volume SnapLock type cannot be changed after creation

Tamperproof Snapshot retention period: Once set, the period cannot be shortened (only extended)

S3 Object Lock Compliance mode: No one, including root, can delete objects until retention expires

For detailed migration steps, see the PoC → Production Guide.

Authorization Model (PoC → Production)

Item	PoC	Production
AppSync auth	`allow.authenticated()`	`allow.groups(["storage-admin"])`
IAM resource scope	`"*"`	Specific ARN patterns
generateClient	`authMode: "userPool"` required	Same

In Amplify Gen2 with multiple auth providers, generateClient() must explicitly specify authMode: "userPool" to ensure the Cognito token is sent to AppSync. This is a recommended pattern per official documentation.

Resources

Summary and Next Steps

Building on the file portal from Part 1, ONTAP management operations and data protection features were embedded:

Feature	What it enables
Storage Dashboard	Health at a glance immediately after login
Incident Lifecycle	Ransomware response with state management
PHI Guardrail	Refuses AI processing on regulated folders by path name
S3 Object Lock	Tamper prevention for AI results, configurable from UI
EMS Events	ONTAP alerts without CLI
Retention Presets	Just pick the regulation, get the right retention
Audit Log	"Who accessed what and when" — self-service for compliance
FlexClone Restore	Recover from snapshot in seconds
FlexCache CRUD	Create/delete read cache volumes from browser
SnapMirror Lifecycle	Sync, break, resync, quiesce — DR from a browser
Athena Query	Ask SQL questions against NAS data
Welcome Modal	First-time users productive in 10 seconds

From a portal that "just browses files" to one that "completes storage operations from the web." Daily monitoring and incident first response now finish in the browser without opening an ONTAP CLI session.

All code is available in the GitHub repository.

As a next step, after running DemoMode using the instructions in Part 1, try adding the VPC configuration to enable ONTAP connectivity. That gets you to a state where every feature in this article is usable. Migration details are collected in the PoC → Production Guide.

In Part 3, I'll write about embedding AI agents into this portal — completing file operations, analysis, and admin operations in natural language. Parts 4 to 6 continue into delegating operations, what the running cluster stopped, and what was left off the screen.

I hope this post helps someone out there.

See you next time.

Reaching an FSx for ONTAP NAS from a browser through an S3 Access Point — Amplify Gen 2 or Nextcloud (File Portal Part 1)

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 03 Aug 2026 10:16:29 +0000

Reach the NAS from a browser and from AI, without stopping existing mounts — choosing between Amplify Gen2 and Nextcloud (part 1 of 6)

A Japanese version of this article is available: 日本語版

Introduction

I've been asked more than a few times how to open files sitting on a NAS in a browser and run AI over them. The data lives on NFS/SMB volumes and runs to hundreds of terabytes — between the sheer volume, the latency, and data-residency rules, copying it out to an external service isn't realistic. Requiring an NFS mount instead puts the data out of reach for anyone who isn't an engineer. Closing that gap felt worth doing, so I built a verification environment for it.

What I used is Amazon FSx for NetApp ONTAP S3 Access Points (hereafter FSx for ONTAP S3 AP, or S3 AP — an access point that exposes NAS volumes for read/write through the S3 API). Because it reaches the data in place, you can add a browser UI and AI processing while leaving existing NFS/SMB workflows exactly as they are. I implemented the portal with two approaches: Amplify Gen2 as an AI-processing dashboard, and Nextcloud as a file-sharing UI.

Here's the conclusion up front:

S3 AP gives you browser access to NAS volumes with no data migration
Amplify Gen2 and Nextcloud aren't mutually exclusive — both can attach to the same volume at the same time
The deciding axis: Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
With DemoMode you can bring the portal up in under 30 minutes without FSx for ONTAP, and try file browsing and AI processing

In this article, I'll walk through the design of the two approaches and how to choose between them, the DemoMode startup steps, and the places where I got stuck during implementation.

Who this is for: anyone who wants to reach files on a NAS from a browser and run AI over them, and the frontend or full-stack developer building that screen. Operating the storage itself — volumes, SnapMirror, SnapLock — starts in Part 4.

What this article does not cover: absolute performance figures, production hosting steps (this is an Amplify sandbox), tenant isolation design.

Teardown: a volume that has carried an S3 Access Point can keep a NAS bucket on the ONTAP side after every access point is detached, and ONTAP then refuses to delete the volume. Tear down with aws fsx delete-volume (measured in Part 6).

Here's the overall picture first.

Light theme shown. A dark theme version is available, and every figure is listed in the architecture diagram index.

Figure 1: Overall architecture — two frontends reaching the same volume through one S3 Access Point

The part worth noticing is that neither frontend moves any data. The S3 Access Point reads the existing NFS/SMB volume in place, so the same files stay available to existing clients at the same time.

Approach	Role it plays
Amplify Gen2	AI processing dashboard (file selection → AI trigger → result display)
Nextcloud	File sharing UI (browsing, sync, sharing links)

Pick based on what you want to do. Both can connect to the same volume simultaneously, so you can start incrementally.

License: MIT License. Commercial use, modification, and redistribution permitted.

Repository: Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns

Prerequisites and Positioning

Audience: Anyone with unstructured data on NAS who wants to protect and leverage that data. On-premises NAS, cloud NAS (FSx for ONTAP, EFS, Azure NetApp Files, etc.) — the platform doesn't matter.

File data on NAS shares several common challenges:

Challenge	Context
Data grows but stays underutilized	Files accumulate year over year — ownership unclear, labels missing, search difficult (CSA study)
Ransomware target	Shared drives are primary targets for encryption attacks. Without detection/stop mechanisms, damage spreads
Want AI but can't move data	Data volume, latency requirements, and data residency regulations make copying to external services impractical
Limited remote access	VPN + mount is required — browser-based access needs additional infrastructure
Audit & compliance overhead	Access logs exist but answering "who accessed what when" in a UI requires a separate analytics layer

This project implements approaches to these challenges using FSx for ONTAP S3 Access Points. FSx for ONTAP was chosen because it combines NFS/SMB/S3 multi-protocol access with integrated data protection: Snapshots, SnapLock (WORM), FlexClone, and ARP/AI (autonomous ransomware detection).

How you use the portal depends on your environment:

Your Environment	How to Use This Portal
Considering migration from on-prem NAS	Build FSx for ONTAP + S3 AP for browser access, AI processing, and data protection in one step
Already using FSx for ONTAP	Create an S3 AP against the existing volume and connect. IAM, network reachability and a volume security style matching the access point identity are prerequisites — a mismatch attaches as `AVAILABLE` and then fails every data operation with `AccessDenied` (measured in Part 6)
NAS + SaaS side by side	Keep SaaS as-is. Add AI processing, audit, and protection for NAS data
Using another cloud NAS	Move onto FSx for ONTAP first. SnapMirror replicates between ONTAP systems and preserves permissions and snapshots; DataSync copies between unlike systems and preserves a different set of metadata. Which one you use changes what survives the move

What this portal provides:

Browser access to NAS files (no VPN or mount setup on the user side)
AI analysis, classification, semantic search (Bedrock integration)
ONTAP-specific data protection (Snapshot, SnapLock, FlexClone, ARP/AI)
Audit trail visibility (CloudTrail + Athena)
Sharing links (Presigned URLs), QR code access

How it differs from a browser-native file sharing service:

No online simultaneous editing. Co-authoring happens on the SMB/NFS mount side
No desktop sync client (NFS/SMB mount serves this purpose)
No native mobile app (responsive web UI as alternative)

Whatever file sharing service you already use, keep it. What this portal adds is browser access, AI processing, an audit trail and data protection for the data that stays on the NAS. Authentication is independent through Cognito (SAML/OIDC federation supported), so existing contracts and licences are unaffected.

The shape this fits is an environment where the files live in one place and the processing you want belongs somewhere else. Two of them:

Healthcare: DICOM images on the NAS, clinical data in a separate system. What gets added is browser access and retention management on the image side only. A deployable implementation of that shape is healthcare-dicom.
Manufacturing: CAD files on NFS, day-to-day document sharing left where it is, AI quality inspection added on the CAD files alone. The implementation is manufacturing-analytics.

Other sectors — financial document processing, media, legal, research — sit on the same base. The overview is industry workload mapping and the implementations are under solutions/industry/. This article does not cover them.

Adding Web Experiences to NFS/SMB File Servers

NFS/SMB file servers have clear strengths: high throughput, low latency, multi-protocol support, and deep integration with workstation tools. These qualities make them the backbone of many organizations' workflows. Meanwhile, some of the web-based experiences that SaaS file sharing provides — browser access, AI integration, searchable audit trails — require additional tooling when working with file servers.

This portal adds those web experiences to existing file server environments without changing anything about how the servers operate:

Experience Added	Traditional Approach	How This Portal Delivers It
Browser access (no VPN on the user side)	VPN + mount configuration	S3 AP + Cognito auth (Internet-origin). The path into the VPC remains, on the Lambda side
Natural language file search	grep or find on file names/paths	Bedrock Knowledge Base semantic search
Sharing links (expiring, password-protected)	Manual smbclient or email attachment	Presigned URL time-limited links + QR codes
Version history visibility	Snapshots exist but require CLI/API	Snapshot list in the UI, restored through FlexClone by selecting one
Audit trail in a UI	Parse auditd or FPolicy logs	CloudTrail + Athena self-service query
AI analysis & classification	Requires building a separate pipeline	Bedrock + Step Functions triggered from the screen
Ransomware visibility	ONTAP ARP runs but checking needs the ONTAP CLI / REST API	ARP/AI dashboard in browser
Mobile / tablet access	Requires dedicated apps or VPN	Responsive web UI + QR code access
Compliance reporting	Run SnapLock CLI commands and compile manually	Lock panel with FISC/SOX/HIPAA presets + status overview

No data movement required. S3 AP reads and writes data directly on the NAS volume. Edit via NFS/SMB → browse via S3 AP in browser → AI results written back to the same volume. All the same data.

Note: Existing NFS/SMB workflows are unaffected. NFS mounts from CAD tools, SMB access from Avid, cron job file processing — all continue working exactly as before. S3 AP is an additional access path to the same volume.

Note: Authentication is independent from existing SaaS (Cognito or LDAP/SAML). No impact on existing SaaS contracts or licenses.

How to Choose

Selection Flow (Q1-Q3)

Q1. Do users need to operate files from a browser?
  → No: EventBridge Scheduler (scheduled batch) is sufficient
  → Yes: Go to Q2

Q2. Primary need: "file sharing & sync"? Or "AI/ML processing launch & result viewing"?
  → File sharing focused: Nextcloud (or Storage Browser for S3)
  → AI processing focused: Amplify Gen2 portal
  → Both: Use together (covered in this article)

Q3. Are users technical? Or non-engineers?
  → Technical: Any of the GUI portals above
  → Non-engineers: Add Amazon Quick (MCP, natural language operation)

Comparison

What you want to do	Suited approach	Trade-off
Trigger AI/ML processing from UI, see results on screen	Amplify Gen2	Custom code required, React knowledge assumed
Browse, upload, sync, and share files via web	Nextcloud	AI integration needs Webhook development
Zero-custom-code file operation UI	Storage Browser for S3	Limited features (browse/DL/UL only)
Natural language file search & analysis	Amazon Quick + AgentCore MCP	Preview stage, region availability limited
No frontend needed, batch is fine	EventBridge Scheduler	No interactive operation

How to choose: Amplify Gen2, Nextcloud, Storage Browser for S3, and Amazon Quick each suit a different context. The trade-offs above are listed symmetrically, including the constraints of the option I reached for, and the article records what actually worked on technical grounds. Read it as material for choosing against your own requirements.

Cost Estimates

Approach	Monthly Cost	Includes
Amplify Gen2 portal	~$18-46	Cognito/AppSync/Lambda (within Free Tier) + AI pay-per-use
Nextcloud (EC2)	~$70-110	EC2 + RDS + ALB. Local Docker = $0
Storage Browser for S3	~$5	Amplify Hosting only
Quick + AgentCore MCP	~$5-15	Lambda execution only
EventBridge direct	~$1-5	Lambda + Step Functions execution only

Cost note: FSx for ONTAP infrastructure (~$194/month, 128 MBps configuration) is common across all approaches. The portal itself adds only tens of dollars per month. At 100 users, Cognito/AppSync/Lambda costs remain within pay-per-use bounds (serverless). At 1000 users, FSx for ONTAP throughput scaling (128→256 MBps, +~$194/month) is the primary scaling cost.

Figures vary by Region, configuration and exchange rate. Price an actual deployment from the FSx for ONTAP pricing page.

Approach 1: Amplify Gen2 — AI Processing Dashboard

What is Amplify Gen2 (Key Points for NAS Users)

AWS Amplify Gen2 is a full-stack development framework that lets you define both frontend (UI) and backend (auth, API, database, storage) in TypeScript and deploy to AWS.

What this means for NAS/file server users:

Aspect	What Amplify Gen2 Handles
No infrastructure management	No servers to provision, patch, or scale. Everything runs serverless (Lambda, AppSync, Cognito)
Built-in authentication	Cognito provides email login, MFA, and enterprise SSO (SAML/OIDC) out of the box. No custom auth system to build
Direct AWS service integration	Connect to S3 Access Points, Bedrock (AI), Step Functions (workflows), Athena (analytics) with minimal code. No need to design IAM from scratch
Git-based deployment	`git push` triggers automatic build and deploy. No separate CI/CD pipeline required
Per-developer sandboxes	Isolated environments auto-created per developer. Team members work in parallel without conflicts
CDK extensibility	When standard features aren't enough, add any AWS resource (VPC Lambda, DynamoDB, etc.) via AWS CDK

In short: when you want "browser UI + AI processing + authentication on NAS data," Amplify Gen2 delivers this without building or maintaining web servers. You write UI in React; AWS managed services handle the backend.

Architecture

┌───────────────────────────────────────────────────┐
│  Amplify Gen2                                     │
│  ┌──────────┐  ┌───────────────────────────────┐  │
│  │ Cognito  │  │ AppSync (Generic Dispatch)    │  │
│  │ + MFA    │  │  fileQuery → ListFiles λ      │  │
│  │ + Groups │  │  adminQuery → ResourceMgmt λ  │  │
│  └──────────┘  │  arpQuery → ArpResponse λ     │  │
│                │  protectionQuery → Snapshot λ │  │
│                └──────────────┬────────────────┘  │
└───────────────────────────────┼───────────────────┘
                                │
          ┌─────────────────────┼────────────────────┐
          │                     │                    │
          ▼                     ▼                    ▼
┌──────────────────┐  ┌─────────────────┐  ┌────────────────┐
│ S3 Access Point  │  │ ONTAP REST API  │  │ Step Functions │
│ (Internet-origin)│  │ (inside VPC)    │  │ (AI/ML jobs)   │
└──────────────────┘  └─────────────────┘  └────────────────┘

Light theme shown. A dark theme version is available.

Figure 2: Amplify Gen2 architecture — Lambda outside the VPC reads and writes the volume through the S3 Access Point

Keeping Lambda outside the VPC and letting it call the S3 Access Point directly is the crux of this design. Here's why.

Key Design Decisions

VPC split architecture:

Non-VPC Lambda: Reads/writes files via S3 AP (Internet-origin). Fast cold starts.
VPC Lambda: Calls ONTAP REST API (management LIF). Must be inside VPC.
Never combine both in one Lambda (Internet-origin S3 AP unreachable via S3 Gateway VPC Endpoint).

Network note: VPC Lambda handles admin operations only. Required VPC Endpoints: Secrets Manager and CloudWatch Logs. S3 AP data access uses non-VPC Lambda, so NAT Gateway is not needed.

Generic Dispatch pattern: 155 operations consolidated into 10 generic endpoints (action + params: AWSJSON) to avoid CloudFormation 1MB limit.

Extensibility note: Adding one operation needs no AppSync schema or CloudFormation change, because the endpoints take an action string and a params payload. It is not only the handler, though: the generated src/lib/dispatchActions.ts has to be regenerated (scripts/portal_action_types.py --emit), and a new UI string has to be added across 8 locales. CI fails until both agree. See the PSTK Action Catalog for the full action list.

UI Layout (Sidebar)

┌───────────────┬─────────────────────────────────────┐
│ Browse        │                                     │
│  📂 All Files │  [Main Content Area]                │
│  ⭐ Favorites │                                     │
│  🕐 Recent    │                                     │
│  🔔 Watch     │                                     │
│  📤 Upload    │                                     │
│               │                                     │
│ AI & Proc.    │                                     │
│  ⚡ AI Proc.  │                                     │
│  🤖 AI Chat   │                                     │
│  🔍 Search    │                                     │
│  📋 History   │                                     │
│  📊 Analytics │                                     │
│  🗂️ Agent Dir │                                     │
│               │                                     │
│ Data Prot.    │  + Right Panel: AI Assistant        │
│  📸 Snapshots │  (appears on file selection)        │
│  🔒 Lock      │                                     │
│  🛡️ ARP/AI    │                                     │
│               │                                     │
│ Admin         │                                     │
│  🔧 Resources │                                     │
│  🔄 Version   │                                     │
│  🔍 Audit     │                                     │
└───────────────┴─────────────────────────────────────┘

4 groups × 17 sections. The sidebar pattern is the one file management screens generally use.

Key Features

File selection → AI processing trigger → results shown on screen (5-second polling)
Bedrock Q&A, Rekognition image analysis, Athena SQL, Textract OCR
FlexClone snapshot restore (selected and run from the UI)
Job execution history (DynamoDB, per-user tracking)
8-language i18n (JA/EN/KO/ZH-CN/ZH-TW/FR/DE/ES)
Keyboard navigation + ARIA labels + Dark/Light mode
Responsive (hamburger menu at 768px, tablet-ready)

Audit note: All operations are recorded in CloudTrail. Audit log retention is controlled via CloudTrail settings (default 90 days, extendable to 7 years for compliance). Self-service Athena queries available from the portal's Audit section.

Security note: Blast radius if a Cognito token is compromised — tokens are scoped by Cognito Groups via AppSync resolver authorization; access is limited to resources within the user's group. Default token expiry: 1 hour. MFA-required setting mitigates leakage risk.

Gotchas

AppSync APPSYNC_JS runtime: new Date() not available → use util.time.nowISO8601()
Data Source must be in the same CDK stack as AppSync API ("Data source not found" otherwise)
Bedrock Nova Lite: requires converse() API (invoke_model + inputText is Titan-only)
Presigned URLs: need explicit SigV4 + regional endpoint

Approach 2: Nextcloud — File Sharing UI

What is Nextcloud (Key Points for NAS Users)

Nextcloud is a self-hosted, open-source file sharing and collaboration platform. It provides browser-based file operations and sharing on infrastructure you control.

What this means for NAS/file server users:

Aspect	What Nextcloud Handles
Data sovereignty	File data stays on your servers. No need to entrust data to external SaaS
Direct NAS connection	External Storage feature mounts S3 AP / NFS / SMB / WebDAV as additional folders. No data copy needed
Familiar experience	Desktop sync client (Windows/Mac/Linux), mobile apps, sharing links — SaaS-equivalent UX
No per-user licensing	AGPL-3.0, free to use. No per-seat pricing
Auth integration	Supports LDAP, SAML, OIDC. Connects to existing Active Directory

When you configure FSx for ONTAP's S3 AP as Nextcloud's External Storage backend, ONTAP volume files appear in Nextcloud's folder listing. Files saved via NFS/SMB are immediately visible in the browser and syncable to desktop clients.

Architecture

# Nextcloud's External Storage App mounts S3 AP as S3-compatible storage
Nextcloud (Docker) → External Storage App (S3 backend) → S3 AP → FSx for ONTAP

Light theme shown. A dark theme version is available.

Figure 3: Nextcloud architecture — the External Storage App mounts the S3 Access Point as S3-compatible storage

Pass the S3 AP alias as the bucket name, and ONTAP volume contents appear in Nextcloud's file listing.

Setup (~5 minutes)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or bucket name (e.g., "my-volume-ap-s3alias")
export S3_BUCKET=<YOUR_S3AP_ALIAS>

# Configure External Storage to mount ONTAP volume via S3 AP
make configure-s3

# Verify connection (success = file listing returned)
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from your ONTAP volume (or S3 bucket) via S3 AP.

Key Features

Desktop sync client (Windows/Mac/Linux)
Sharing links (password-protected, time-limited)
Comments & annotations
WebDAV access
LDAP/SAML authentication

Trade-offs

Strengths	Considerations
Mature NAS file sharing tool	AI/ML integration needs Webhook development
Desktop sync + mobile apps	EC2/RDS operational cost (production config)
Free under AGPL-3.0 (Docker as-is)	Source disclosure obligation if server code is modified

Gotchas

use_path_style=true required (S3 AP alias only works with path-style addressing)
occ files_external:config must be run once per parameter ("Too many arguments" otherwise)
Empty credentials cause IMDS (169.254.169.254) fallback timeout in Docker

Cost

Local Docker: $0. Production (EC2 + RDS + ALB): ~$70-110/month.

Coexistence Pattern: Amplify + Nextcloud

They are not mutually exclusive — both can access the same FSx for ONTAP volume simultaneously.

Feature Comparison

Feature	Nextcloud	Amplify Gen2
File browsing	✅	✅
File DL/UL	✅	✅ (Presigned URL, 50GB object limit)
Desktop sync	✅	△ (Use NFS/SMB instead)
Sharing links (time-limited)	✅	✅ (Presigned URL)
AI/ML processing workflows	⚠️ Webhook needed	✅
File Q&A (Bedrock)	❌	✅
FlexClone restore	❌	✅
ONTAP admin operations	❌	✅

Expected Usage Flow

1. Team uses Nextcloud daily for file browsing & sharing
   (same data visible as NFS/SMB users)

2. When AI processing is needed on a specific folder,
   launch it from the Amplify portal

3. Result files are written back to the same volume,
   instantly visible from Nextcloud and NFS

Light theme shown. A dark theme version is available.

Figure 4: Running both — AI processing and file management share one S3 Access Point

Authentication is independent (Nextcloud: LDAP/SAML, Amplify: Cognito). For gradual adoption, set up Nextcloud first, then add Amplify when processing needs arise.

Data Consistency

ONTAP keeps multi-protocol access consistent, but when a write becomes visible from the other protocol depends on the client's mount options. These are measured, on one volume reached from both an S3 Access Point and NFS (2026-08-09, ap-northeast-1, SINGLE_AZ_1, 128 MBps, NFSv3, concurrency 1):

Direction / operation	Result
S3 → NFS (64 B write)	p50 7 ms
NFS → S3	An order of magnitude higher (around a second)
Delete reflected, `actimeo=0`	7 ms
Delete reflected, default mount	2,171 ms
Completed multipart upload, default mount	Still not visible after 3 seconds

If you are writing code that reads through the other protocol straight after a write, match your mount options to the measurement conditions. Every direction and the full conditions are in the verification record. Reproducibility of this measurement is incomplete in one respect: the ONTAP version could not be identified.

Storage operations note: Your existing procedures (volume creation, snapshot management, and so on) remain unchanged. This portal is an end-user-facing AI processing and file access layer; administrators carry on with the AWS Management Console / Amazon FSx API and the ONTAP CLI / REST API. Those three are the management interfaces reachable for FSx for ONTAP — ONTAP System Manager is not among them (management interfaces).

Comparison with Alternatives

Multiple approaches exist for "accessing NAS data from a browser." Each has trade-offs:

Approach	Suited for	Trade-off
This portal (S3 AP + Amplify/Nextcloud)	AI processing + admin ops without data movement	Custom code required, hours of initial setup
DataSync + S3 + any portal	Leveraging S3-native ecosystem	Data copy, sync lag, storage cost duplication
Transfer Family SFTP	Existing SFTP client compatibility	No web UI, AI integration needs extra development
EFS + Lambda	Serverless simplicity	No ONTAP-specific features (SnapLock, FlexClone, ARP)
A browser-native file sharing service	No infrastructure to build	Reaching data on the NAS requires a copy. ONTAP-side Snapshot / SnapLock / FlexClone / ARP are outside the path

The S3 AP characteristic is "access data via S3 API without moving it." No data copy means no consistency issues or double storage costs. However, it's not "sign up and use immediately" like SaaS.

How to choose: DataSync + S3, Transfer Family SFTP, EFS + Lambda, and SaaS file sharing each suit a different context. The table above also carries the constraints on the FSx for ONTAP S3 AP side (custom code required, a few hours to stand up). Read it as material for choosing against your own requirements.

Try It (~30 Minutes)

Prerequisites

Tool	Version	Check Command	Time
Node.js	20.x or later	`node --version`	0 min if installed
npm	10.x or later	`npm --version`	Bundled with Node.js
AWS CLI	2.x	`aws --version`	~2 min
AWS Account	—	`aws sts get-caller-identity`	0 min if authenticated
Docker	24.x or later	`docker --version`	Nextcloud only

FSx for ONTAP is NOT required for DemoMode. You can verify portal behavior with a regular S3 bucket.

Fastest Path (~5 min, copy-paste 4 lines)

If Node.js and AWS credentials are ready:

# Clone repository and navigate to portal directory
git clone https://github.com/Yoshiki0705/FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns.git && cd FSx-for-ONTAP-S3AccessPoints-Serverless-Patterns/solutions/amplify-portal

# Install dependencies (~2 min)
npm install

# Copy config (DemoMode default — no edits needed)
cp amplify/portal-config.example.ts amplify/portal-config.ts

# Start sandbox (first run also creates Cognito users, ~3 min)
npx ampx sandbox

What to expect: Terminal shows ✅ Deployment complete, then http://localhost:5173 opens the portal. You can browse files and access AI processing screens from the sidebar.

portal-config.ts Key Parameters

// portal-config.ts — parameter explanations
export const portalConfig = {
  // S3 AP alias (empty string = DemoMode, uses regular S3 bucket)
  // Production example: "my-ontap-vol-ap-s3alias"
  s3ApAlias: "",

  // VPC ID (empty = no VPC Lambda = DemoMode)
  // Production example: "vpc-0123456789abcdef0"
  vpcId: "",

  // Cognito group-to-S3AP mapping (different access per group)
  // Example: analysts group gets analysis AP only, admins get full access
  groupApMapping: {
    // "analysts": "analysis-vol-ap-s3alias",
    // "admins": "admin-vol-ap-s3alias"
  },
};

CI/CD note: Amplify Gen2 auto-deploys on git push. git push origin main triggers Amplify Hosting build/deploy. No separate CI/CD pipeline needed.

Nextcloud Setup (~5 min)

# Start Nextcloud container
cd solutions/nextcloud-test
make up

# Set your S3 AP alias or S3 bucket name
export S3_BUCKET=<YOUR_BUCKET_OR_AP_ALIAS>

# Configure External Storage
make configure-s3

# Verify connection
make verify

# Open in browser
open http://localhost:8080  # admin / admin123

What to expect: Nextcloud file listing shows files from the ONTAP volume (or S3 bucket) via S3 AP.

About DemoMode

Leave VPC settings empty in portal-config.ts to experience the portal UI and AI processing flow without FSx for ONTAP.

Feature	DemoMode	Production
Sidebar UI / navigation	✅	✅
File browsing	⚠️ (S3 bucket)	✅ (S3 AP)
AI processing	⚠️ (needs Step Functions)	✅
ONTAP admin panels	⚠️ (shows "connection required")	✅

For production connectivity, see PoC → Production Guide.

Role-Based Documentation (8 Languages)

Guide	Audience	Content
User Guide	End users	Sign in → file ops → AI → FAQ
Compliance Guide	Security/Compliance officers	ARP verification → WORM checks → audit trail
Quick Reference	All roles	1-page operations cheat sheet

Demo note: Minimum 30-minute demo setup is "4-line fastest path → DemoMode launch → file browsing → AI processing screen." No FSx for ONTAP pre-build required.

Operational Considerations

File-Level Access Control

S3 AP access is governed by ONTAP's UNIX permissions (UID/GID) or Windows ACLs. The File System Identity specified when creating the S3 AP determines "who" the access runs as.

# Example: separate team access with multiple S3 APs
S3 AP "portal-readonly"  → UID 1001 (read-only)
S3 AP "portal-analyst"   → UID 1002 (analysis team, specific dirs only)
S3 AP "portal-admin"     → UID 0    (admin, full access)

Throughput Sharing

S3 AP access shares the same FSx for ONTAP throughput budget as NFS/SMB. Normal portal usage (directory listing + individual reads) is not an issue, but if heavy parallel reads coexist with NFS workloads, monitor CloudWatch ThroughputUtilization.

Options if throughput is insufficient:

Scale up throughput capacity (first-generation file systems offer 128 / 256 / 512 / 1,024 / 2,048 MBps, and 4,096 MBps in some Regions; second-generation uses a separate series from 384 MBps up)
Set QoS policy to cap portal volume throughput

Multi-Account Environments

This portal is designed for single-account use. AWS Organizations considerations:

S3 AP: Cross-account access requires AP resource policy
Cognito: Multi-account user consolidation recommended via SAML federation
Tenant isolation: groupApMapping assigns different S3 APs per group

About Presigned URLs

AWS documentation lists Presign as "Not supported," but they work in practice (confirmed with AWS Support). Presigning is client-side SigV4 signature calculation, and since GetObject is Supported, there's no structural way to block it. However, production reliance is at your own risk since documentation says "Not supported." Set short expiry (5 minutes or less recommended).

Scaling Characteristics

Component	Scaling	Downtime
Lambda / AppSync / Cognito / DynamoDB	Automatic (serverless)	None
FSx for ONTAP	Manual throughput/storage expansion	Minutes during throughput change

Portal components are all serverless — the only scaling bottleneck is FSx for ONTAP throughput.

Data engineering note: File metadata is available via CloudTrail + S3 AP access logs. Queryable with Athena, but building a dedicated metadata catalog (Glue Data Catalog etc.) requires separate design.

Series Structure (7 Parts)

Part	Theme	Key Content
Part 1 (this article)	Portal foundation	Amplify Gen2 vs Nextcloud, coexistence patterns, DemoMode
Part 2	Storage operations	ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore
Part 3	AI agent integration	AgentChat, semantic search, multi-agent teams, HITL
Part 4	Delegation and record	182 actions, buttons that cannot be pressed, the blast radius of irreversible operations
Part 5	What only the cluster told me	FlexGroup creation, capacity rebalance, delete refusals, defaults that invalidate a measurement
Part 6	What was left off the screen	How far ONTAP features reach over the S3 Access Point path, and the work handed to a schedule
Part 7	The tooling underneath	Nx Plugin for AWS 1.0, AWS Blocks and Amplify Gen 2 compared: resource counts, fixed monthly cost, and what a stack deletion leaves behind

FAQ

Q: I'm not using FSx for ONTAP yet. Can I try this?
Yes. DemoMode uses a regular S3 bucket to demonstrate the portal's UI and AI processing flow. FSx for ONTAP can be added later.

Q: Do I need to replace the file sharing service we already use?
No. This portal is an additional layer for AI processing, data protection, and audit on NAS data.

Q: How much does it cost?
DemoMode is free (within AWS Free Tier). Production portal add-on: ~$5–50/month. Main cost is FSx for ONTAP infrastructure (~$194/month minimum).

Q: What about security?
Cognito MFA + Groups role separation, HTTPS, Secrets Manager. Enterprise SSO (SAML/OIDC) supported via Cognito federation. Part 2 covers the security model in detail.

Q: Can it handle large files (multiple GB)?
S3 AP upload has a 50GB object size limit (a single PutObject is capped at 5GB, so Storage Browser switches to multipart above that). For files larger than 50GB, place them on the volume via NFS/SMB and use the portal for browsing/processing only.

Q: Does this affect my existing operational procedures?
No. This portal is an end-user-facing additional layer. Operating through the ONTAP CLI / REST API and the AWS Management Console continues exactly as before.

Verified Environment

Component	Version
Node.js	20.18.x
React	18.3.x
Amplify Gen2	1.x (as of 2026-07)
Python (Lambda)	3.12 (ARM64)
FSx for ONTAP	ONTAP 9.17.1
Region	ap-northeast-1
Verified	2026-07

Resources

GitHub: FSx for ONTAP S3 AP Serverless Patterns — This repository (MIT License)
User Guide (8 languages) — End-user daily operations guide
Compliance Guide (8 languages) — ARP/WORM/audit trail verification procedures
PoC → Production Guide — DemoMode to production connectivity migration
Scaling Guide — Capacity planning and throughput design
Nextcloud External Storage Setup — Nextcloud + S3 AP detailed configuration
S3 AP Compatibility Notes — API support status and known constraints
DemoMode Guide — Running without FSx for ONTAP
AWS: FSx for ONTAP S3 Access Points — Official documentation

Summary and Next Steps

Point	Conclusion
Data migration	Not needed — S3 AP reads the same volume NFS/SMB already uses
Portal choice	Nextcloud when file sharing is the center of gravity, Amplify Gen2 when AI processing and ONTAP admin operations are
Running both	Both can attach to the same volume at once. Authentication stays independent
How to try it	DemoMode takes about 30 minutes and needs no FSx for ONTAP
Existing workflows	NFS/SMB mounts and ONTAP CLI / REST API operations continue unchanged

For next steps, start with the 4-line fastest path in DemoMode and spend a few minutes in the sidebar UI and the AI processing screen. Once you can see which approach fits, the PoC → Production Guide covers moving to a production connection.

Part 2 covers storage operations (ARP/AI incident response, Tamperproof Snapshot, regulatory retention, Audit Log, FlexClone Restore), and Part 3 covers AI agent integration (AgentChat, semantic search, multi-agent teams, HITL). Parts 4 to 6 continue into delegating operations, what the running cluster stopped, and what was left off the screen.

I hope this post helps someone out there.

See you next time.

KNFSD File Cache FSx for ONTAP: NFS Read Speedup + S3 Access Points Dual-Path

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Thu, 23 Jul 2026 12:59:50 +0000

Introduction

In July 2026, AWS released KNFSD File Cache as Preview — an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems.

This is the first time an NFS read-acceleration layer has shipped as an official AWS solution. Working daily with FSx for ONTAP NFS performance for EDA and VFX workloads, my reaction was immediate: spin up a verification environment and see what happens.

What I tested was a Dual-Path architecture: KNFSD caching NFS reads from the same FSx for ONTAP volume that S3 Access Points expose to serverless Lambda processing. The question: can compute fleets read at local-NVMe speed while Lambda runs AI/ML workloads on the same data, without bandwidth contention?

Here's what I found:

Dual-Path E2E test passed: A file written via S3 AP was immediately readable through the KNFSD NFS cache
Cache effect: first read 55ms → subsequent reads 2ms (28x improvement), proxy cache serves 422-619 MB/s
NFSv4.1 is mandatory: NFSv3 re-export causes Stale file handle on file creation (kernel-level filehandle size constraint)
FSID_MODE=local + SQLite on FSx for ONTAP: FSID persistence at zero additional cost

This post covers every pitfall I hit during deployment, and why this architecture still makes sense despite those issues.

When to Consider This — and When Not To

This architecture suits workloads that:

Need NFS read bandwidth beyond FSx for ONTAP's provisioned throughput (EDA DRC/LVS, VFX rendering, genomics)
Want both NFS (high-speed reads) and S3 API (Lambda AI/ML post-processing) on the same data
Use Spot instances for burst compute and need the cache to stay warm through reclamation
Unify multiple NFS sources (FSx for ONTAP + on-premises NAS + other clouds) into one cache layer

Consider alternatives when:

FSx for ONTAP throughput scaling alone covers your bandwidth needs (steady-state workloads)
Write caching is needed (FlexCache offers configurable write-back/write-around)
You want fully managed operations with no EC2 to run (FlexCache or Amazon File Cache)
Production SLA is non-negotiable (KNFSD is Preview — wait for GA)

On vendor neutrality: This post evaluates KNFSD File Cache on technical merits only. FlexCache, Amazon File Cache, EFS, and other options each suit different contexts. The goal here is to document what actually works, not to rank solutions.

What is KNFSD File Cache?

KNFSD File Cache is an open-source (Apache 2.0) NFS caching proxy built on the Linux kernel's NFS re-export and FS-Cache subsystems. NFS re-export means mounting a remote NFS share and publishing it as a new NFS export to downstream clients — the proxy sits transparently between source and consumers. Wētā FX (Avatar: The Way of Water) and ILM used the predecessor project in production.

NFS Source (FSx for ONTAP)
     │ NFS mount (NFSv4.1, fsc)
     ▼
KNFSD Proxy (EC2, NVMe L2 cache)
     │ NFS re-export (NFSv4.1)
     ▼
Compute Fleet (Spot × hundreds)

L1 cache: Linux Page Cache (RAM — 5-9 GB/s on 16 GB in this test)
L2 cache: FS-Cache (local NVMe — survives reboot, 106 MB/s measured)
Writes: write-through (immediately forwarded to source)
Scaling: ASG + DNS round-robin or NLB
Prerequisite: source must be an NFS v3/4.x server (parallel filesystems like Lustre/GPFS/WekaFS are not supported)

The key difference from FlexCache: KNFSD is a protocol-level transparent proxy that doesn't require the source to be ONTAP. It can unify FSx for OpenZFS, on-premises NetApp/Pure/Dell/DDN (A3I), and other-cloud NFS servers (via Direct Connect / Cloud Interconnect) into a single cache layer.

Test Environment

Component	Details
Region	ap-northeast-1
Source	FSx for ONTAP (128 MBps, Single-AZ, ONTAP 9.17.1)
KNFSD Proxy	m6gd.xlarge (arm64 Graviton, 16 GB RAM, 237 GB NVMe)
Kernel	7.1.3-knfsd (Packer custom build)
FSID Mode	local (SQLite on FSx for ONTAP NFS mount)
NFS Version	NFSv4.1 (source → proxy → client)
Test Client	t4g.micro (AL2023, 916 MB RAM)
S3 AP	Internet-origin, UNIX identity (root)

Cost note: KNFSD-specific incremental resources only (Packer Spot build ~$0.30 + m6gd.xlarge 1hr ~$0.29). Prerequisites: a running FSx for ONTAP (~$194/mo minimum) + VPC infrastructure. If you already operate FSx for ONTAP, the incremental evaluation cost is effectively < $1.

Dual-Path E2E Test Results (Verified)

Architecture

[Lambda / CLI]                    [Compute Fleet]
     │                                 │
     │ S3 PutObject                    │ NFS read (NFSv4.1)
     ▼                                 ▼
┌──────────────────┐         ┌──────────────────────┐
│ S3 Access Point  │         │ KNFSD File Cache     │
│ (Internet-origin)│         │ (m6gd.xlarge, NVMe)  │
└────────┬─────────┘         └──────────┬───────────┘
         │                              │
         │  ← same volume →             │ NFS mount (NFSv4.1)
         ▼                              ▼
┌──────────────────────────────────────────────────┐
│ FSx for ONTAP Volume (/vol1, UNIX security style)│
└──────────────────────────────────────────────────┘

Results

Test	Result	Details
S3 AP write → KNFSD NFS read	✅	Immediate reflection, content match
NFS write → S3 AP read (MD5)	✅	`c092ef65...` match both directions
S3 AP batch 50 files → NFS bulk read	✅	50 files in 107ms (2.1ms/file)
S3 AP multipart 50MB → NFS read	✅	MD5 verified
NFSv4.1 write via KNFSD	✅	Write-through reflects immediately
fsidd + SQLite persistence	✅	SQLite created on FSx for ONTAP
Proxy restart → client continues	✅	NFSv4.1 grace period, no remount needed

# Actual test log
$ aws s3 cp test.txt s3://arn:aws:s3:ap-northeast-1:XXXX:accesspoint/knfsd-dualpath-test/s3ap-e2e-1784803253.txt
# → upload: success

$ cat /mnt/knfsd/vol1/s3ap-e2e-1784803253.txt  # NFSv4.1 via KNFSD
dual-path-e2e-test-1784803253
# → read success ✅

Performance Numbers

Test environment note: These numbers are from m6gd.xlarge (237 GB NVMe, single drive). Production-recommended instances (im4gn.16xlarge: 30 TB NVMe RAID, i3en.24xlarge: 60 TB) are expected to deliver significantly higher L2 NVMe throughput (estimated 2-8 GB/s sequential).

3-Layer Cache Hierarchy

Layer	Throughput	Use Case
L1: Proxy RAM (page cache)	5.0-9.1 GB/s	Hot data, repeated reads
L2: NVMe (FS-Cache)	106 MB/s	Warm-up shortcut after proxy restart
Source: FSx for ONTAP (NFS fetch)	18-19 MB/s	Cache miss, initial fetch

Important note on source performance: The 18-19 MB/s figure reflects test-time NFS mount constraints (NFSv3 single-stream, rsize=64K) — not FSx for ONTAP's throughput limit (128 MBps). Switching the source mount to NFSv4.1 + rsize=1M is expected to significantly improve cache miss performance (proposed upstream in Issue #42).

L2 NVMe is 5.6x faster than source fetch. After a proxy reboot with empty RAM, data is served from NVMe cache immediately.

Throughput

Operation	Throughput
Sequential read (proxy L1 → client)	422-619 MB/s
Large write (write-through)	157-218 MB/s
S3 AP multipart upload (50 MB)	36.6 MiB/s

Latency

Operation	Cold	Cached	Improvement
10 MB file read	55 ms	2 ms	28x
4 KB small file	1.75 ms	1.5 ms	14%
S3 AP batch → NFS	—	2.1 ms/file	—

Working Set > Client RAM

With client RAM at 916 MB and a 500 MB dataset, reading after drop_caches still achieves 422-428 MB/s — served from the KNFSD proxy's 16 GB RAM cache. This is the core value proposition: small compute nodes can access datasets larger than their local memory at proxy-cache speed.

nconnect Caveat

nconnect	100 MB cold read
1 (default)	619 MB/s
16	184 MB/s

On t4g.micro (5 Gbps NIC), nconnect=16 is counterproductive. Only effective on 100 Gbps instances (c5n.18xlarge, hpc7g).

Cache Coherency

S3 AP write visibility through KNFSD depends on NFS attribute cache timeouts:

Parameter	Default	Effect
`acregmin`	3s	Minimum file attribute cache
`acdirmax`	60s	Maximum directory attribute cache

In our tests, S3 AP-written files were immediately visible through KNFSD (new files bypass directory attribute cache). For frequently-overwritten files, set acregmax=5-10 to reduce stale-data windows.

Every Pitfall We Hit (Documenting All of Them)

NFSv3 Breaks Writes (Most Important Finding)

This consumed the most debugging time. When mounting the KNFSD proxy via NFSv3, existing file reads succeed but new file creation returns Stale file handle.

The cause is documented in the Linux Kernel NFS re-export documentation:

reexport server's filehandle for the reexported object will be X+22 bytes

FSx for ONTAP filehandles are ~40+ bytes. After +22 bytes = ~62+ bytes. New file creation (which adds subdirectory inode information) pushes past NFSv3's 64-byte limit.

NFSv4.1 supports 128 bytes — ample headroom. Our Terraform validates against nfs_version="3" with a clear error message linking to the upstream issue.

Filed upstream as Issue #40.

Other Pitfalls

Problem	Cause	Fix
proxy-startup.sh immediate failure	No EC2 API access (no Public IP)	`assign_public_ip = true`
SSM `GetParametersByPath` denied	IAM missing path-level ARN	Include both `.../*` and `...` (Issue #41)
"No storage devices found"	`CACHEFILESD_DISK_TYPE=local_nvme`	Correct value: `local-nvme` (hyphen)
`FSID_MODE=static` breaks all writes	fsidd disabled, filehandle unresolvable	Switch to `FSID_MODE=local`
S3 AP create BadRequest	`--type One_Zone` is for OpenZFS	Use `--type ONTAP`
`exportfs: No fsid found`	fsidd first-time allocation warning	No operational impact (ignore)

FSID_MODE=local: SQLite on FSx for ONTAP

KNFSD's NFS re-export requires stable FSID management. FSID_MODE=static (fsidd disabled) is fundamentally broken — all write operations fail with Stale file handle.

The fix: use the Linux kernel's fsidd daemon with SQLite backend, placing the database file on the FSx for ONTAP NFS mount:

/srv/nfs/vol1/.knfsd/fsids.sqlite  ← on FSx for ONTAP (persistent, 99.99% SLA)

Aspect	local (SQLite on FSx)	external (RDS)
Additional cost	$0	$15-45/mo
Persistence	✅ (FSx for ONTAP SLA)	✅ (RDS SLA)
Multi-node	△ (single-node only)	✅
Complexity	Low	Medium-High
Best for	PoC / single-node	Production multi-node

FlexCache and KNFSD: Different Tools for Different Contexts

Criterion	FlexCache	KNFSD
Source must be ONTAP	✅ Native integration	○ Any NFS-compatible filer
Multiple source unification	Not possible	✅
Write-back/write-around	✅ (configurable)	△ (write-through only)
Burst read scaling	△ (within FSx throughput)	✅ (NVMe cache scales with ASG)
Managed operations	✅	△ (EC2 operations)
Protocol	NFSv3/4.x	NFSv4.1 (required for re-export)
Data protection integration	✅ (SnapMirror/SnapLock)	None (pass-through)
Node elasticity	△ (manual)	✅ (Auto Scaling)

FlexCache note: FlexCache also doesn't consume origin bandwidth on cache hits (same structure as KNFSD). However, in FSx for ONTAP's managed environment, auto-scaling FlexCache node count is not possible. For burst read bandwidth exceeding FSx max throughput, KNFSD offers more flexibility.

Amazon File Cache / FSx for Lustre: Amazon File Cache is a managed Lustre-compatible cache requiring the Lustre kernel client module on every node. KNFSD uses standard NFS clients (built into Linux/macOS), requiring no additional modules. However, Lustre's parallel I/O bandwidth is inherently higher than NFS re-export — if raw bandwidth is the priority and you can deploy Lustre clients on all nodes, File Cache / FSx for Lustre is the better fit.

Cost break-even guideline: KNFSD becomes economical when you need burst bandwidth beyond FSx for ONTAP's maximum throughput tier (4,096 MBps at ~$2,000/mo). For steady workloads where FSx throughput scaling suffices, increasing FSx throughput is simpler.

Target Workloads

Industry	Scenario	KNFSD Role	S3 AP Role
Semiconductor EDA	DRC/LVS burst	Tech file repeated reads	Verification result AI classification
VFX	Rendering	Texture/asset Fanout	Render quality AI check
Genomics	Reference genome + dbSNP	Hundreds of nodes on same reference	Variant AI classification
Financial Risk	Monte Carlo VaR	Market data sub-ms delivery	Anomaly scenario AI detection
Automotive CAE	Structural sim	Shared mesh data reads	Result comparison AI
Weather	NWP ensemble	GFS initial condition cache	Extreme weather AI detection

Step-by-Step Deployment

1. Build AMI (~25 min)

# Find your subnet (same AZ as FSx for ONTAP recommended)
aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-YOUR-VPC" \
  --query 'Subnets[].{Id:SubnetId,AZ:AvailabilityZone}' --output table

git clone https://github.com/awslabs/knfsd-file-cache.git /tmp/knfsd-file-cache
cd /tmp/knfsd-file-cache/image
packer build -var 'REGION=ap-northeast-1' -var 'ARCH=["arm64"]' \
  -var 'ASSOCIATE_PUBLIC_IP_ADDRESS=true' -var 'SUBNET=subnet-YOUR-SUBNET' .
# → Note the AMI ID (ami-0xxxxxxxxxx)

The AMI is reusable across deploy/destroy cycles. Rebuilds are only needed for kernel updates (~monthly). Normal deploys just pass the existing AMI ID to Terraform.

2. Terraform Deploy (~3 min)

cd infrastructure/knfsd-file-cache/terraform
cp terraform.tfvars.example terraform.tfvars
# Edit: vpc_id, subnet_ids, knfsd_ami_id, source_mounts

./scripts/preflight-check.sh   # Validates environment before deploy
terraform init && terraform apply

3. Mount from Client

sudo mkdir -p /mnt/knfsd
sudo mount -t nfs -o vers=4.1 <KNFSD_IP>:/vol1 /mnt/knfsd
echo test > /mnt/knfsd/hello.txt && cat /mnt/knfsd/hello.txt

⚠️ Always specify vers=4.1. Using vers=3 will cause Stale file handle on file creation.

4. Verify Cache Effect

dd if=/dev/urandom of=/mnt/knfsd/bench.dat bs=1M count=10; sync
sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
time cat /mnt/knfsd/bench.dat > /dev/null   # ~55ms (first read — fetched from source)
time cat /mnt/knfsd/bench.dat > /dev/null   # ~2ms  (subsequent — served from cache, 28x)

5. Dual-Path Test (if you have an S3 AP)

# Write via S3 AP
aws s3 cp test.txt "s3://arn:aws:s3:REGION:ACCOUNT:accesspoint/AP-NAME/file.txt"

# Read via KNFSD NFS (immediate reflection)
cat /mnt/knfsd/file.txt

Security note: The above test uses root (UID 0) as the S3 AP FileSystemIdentity. This is for testing only. In production, use a dedicated non-root user following least-privilege principles.

6. Cleanup

terraform destroy
aws ec2 deregister-image --image-id ami-xxx

Upstream Feedback

Five issues filed during this verification:

#	Summary
#38	Verification report + minimal example for existing source filers
#39	Private subnet support (graceful degradation without EC2 API)
#40	NFSv3 re-export filehandle overflow documentation
#41	IAM policy missing path-level ARN (bug fix)
#42	NFSv4.1 source mount support (end-to-end v4.1)

Filing upstream issues for things you find broken (or under-documented) is one of the most impactful things you can do with a Preview-stage OSS project. It ensures the maintainers have real-world data to prioritize GA fixes.

Summary and Next Steps

Dual-Path E2E test passed — S3 AP write → KNFSD NFS read works.

Three key findings:

NFSv4.1 mandatory (NFSv3 filehandle size limit breaks writes)
FSID_MODE=local + SQLite on FSx for ONTAP provides FSID persistence at $0
S3 AP writes reflect immediately through KNFSD NFS

What's next:

Wait for GA (SLA terms, support scope confirmation)
Multi-node (ASG + NLB) verification
EDA/VFX-specific file access pattern benchmarks
im4gn.16xlarge (30 TB NVMe) L2 bandwidth at production scale

KNFSD File Cache is Preview, but it clearly functions as a practical read-acceleration layer for burst compute workloads.

Small-scale starting point: m6gd.xlarge × 1 running 24/7 = ~$215/mo. Daytime-only (10h) = ~$65/mo.

Resources

KNFSD File Cache GitHub
AWS Launch Blog
Dual-Path Architecture Guide (JA) — 7 industry deep dives + cost estimates + observability
Terraform + Demo Guide
FSID Backend Options — SQLite/RDS/Aurora comparison
Linux Kernel NFS Re-export Docs
Verification Issues #38-#42

Preview notice: KNFSD File Cache is in Preview as of July 2026. The code is OSS (Apache 2.0) but AWS Preview Service Terms apply. Evaluate for non-production or supplementary use until GA. AWS Support coverage may not extend to KNFSD-specific issues — GitHub Issues is the primary channel. NVMe instance store cache data is protected by AES-256 hardware encryption and auto-erased on instance termination.

Mackerel's Log Feature Just Opened in Beta — Here's What It Takes to Wire It Into an OTLP Pipeline

Yoshiki Fujiwara(藤原善基)@AWS Community Builder — Mon, 20 Jul 2026 08:05:24 +0000

TL;DR

Mackerel — Hatena's Japan-origin observability platform — opened its log feature as public beta on July 16, 2026. In response, this repository added Mackerel integration for shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events via the existing Lambda/OTel Collector pipeline.

The OTel Collector path worked immediately — just add an exporter config entry. But the direct-send path (Lambda → vendor OTLP endpoint, no Collector) exposed two compatibility gaps:

Mackerel requires a custom Mackerel-Api-Key header, not Bearer/Basic
Mackerel's OTLP endpoint only accepts Protobuf, rejecting OTLP/JSON outright

This article covers the generic auth header primitive and limited Protobuf encoder added to close those gaps. Both delivery paths are confirmed E2E against a live Mackerel organization, but since the log feature is still in beta, production deployments should use the Collector path and should not treat Mackerel as the sole audit trail destination.

Scope: This article covers shipping FSx for ONTAP file access audit logs, EMS events, and FPolicy events to Mackerel via the existing Lambda/OTel Collector integration. It does not cover Collector high availability, Lambda retry/DLQ design, NAT Gateway routing, or log-loss recovery — see the base article (Part 5) and repository operations docs for those topics.

FSx for ONTAP
├─ File access audit (EVTX/XML)
├─ EMS events
└─ FPolicy events
         │
         ▼
Lambda: builds OTLP Logs Data Model
         │
         ├──▶ OTel Collector ──▶ Mackerel [recommended]
         │
         └──▶ Direct to Mackerel [verification path]
              - Mackerel-Api-Key header
              - Accept: */*
              - OTLP/Protobuf required

GitHub: Yoshiki0705/fsxn-observability-integrations
New files: integrations/mackerel/, integrations/otel-collector/otel-collector-config-mackerel.yaml
Modified: integrations/otel-collector/lambda/{handler,ems_handler,fpolicy_handler}.py

This is a standalone entry in the Serverless Observability for FSx for ONTAP series — it's not a continuation of the incident-response article, and doesn't assume you've read it. It builds on Part 5 (the vendor-neutral OTel Collector pattern), since that's exactly the mechanism a new backend plugs into.

Why Mackerel, and Why Now

This series has already shipped FSx for ONTAP audit logs to 9 observability platforms — Datadog, Splunk, Grafana Cloud, Honeycomb, and others — all confirmed E2E against real accounts. A few readers, mostly from Japan, have asked why Mackerel wasn't on that list. Fair question, and one I'd been wanting to answer for a while.

The short answer: it wasn't this repository's gap to close. Mackerel simply didn't have a log-sending feature yet. It's offered OpenTelemetry-based tracing (APM) for a while, but logs were a gap in Mackerel's own product.

There's another direct trigger behind this article. At SRE NEXT 2026, I had a chance to casually catch up with some longtime acquaintances on the Mackerel team and trade notes on Observability for Amazon FSx for NetApp ONTAP. That particular conversation didn't lead anywhere specific on its own, but the conference as a whole gave me useful ideas and motivation for infrastructure-focused development, which fed directly into picking this integration up.

That changed on July 16, 2026, when Mackerel opened its log feature as a public beta. A feature I'd been waiting on finally showed up, which is the actual reason this article exists.

That said, "just opened in beta" is worth taking seriously. The feature itself is new and explicitly non-GA — a genuinely different situation from adding an eighth or ninth already-mature vendor without much thought. This article walks through what actually broke while wiring it up, and why it still sits in a separate section instead of being quietly folded into the main vendor table.

Vendor-neutral note: this article evaluates Mackerel's OTLP log ingestion on its technical merits — the same way this repository's vendor-comparison.md treats Datadog, Grafana, Splunk, and the other 9. No platform in this comparison is framed as superior to another; each fits a different context, and the goal here is documenting what's actually supported, not ranking who's "better."

When this integration is a good fit:

You already use Mackerel for monitoring/APM
You want to integrate FSx for ONTAP events into your existing incident investigation workflow
You use an OTel Collector as a shared telemetry router
You can evaluate a beta feature in a non-production or supplementary capacity

When to consider other options for now:

Regulatory/audit requirements demand guaranteed data retention
This would be your sole security log destination
Log loss is unacceptable and formal support terms are required
Strict data residency or contract-path conditions apply

What Mackerel's Beta Actually Supports

Before writing any integration code, the first step was reading Mackerel's own documentation rather than assuming logs would work like traces. As of July 2026, the log-sending method documented by Mackerel's official help is OpenTelemetry-based. This article uses OTLP/HTTP accordingly. Some things turned out to be shared, some didn't:

	Mackerel Tracing (APM)	Mackerel Logs (beta, 2026-07-16)
Protocol	OTLP/HTTP	OTLP/HTTP (same)
Endpoint	`https://otlp-vaxila.mackerelio.com`	Same endpoint
Auth	`Mackerel-Api-Key` header (Write scope)	Same header, same scope
Required extra header	`Accept: /` (documented as required by Mackerel; the reason isn't publicly documented)	Same requirement
Grouping key	—	`service.namespace` + `service.name` resource attributes
Retention	(APM's own terms)	30 days planned at GA; beta operates under the same window but with no guarantee
Collector batch config	—	~3.5MB (`sending_queue.batch.max_size: 3500000` bytes). This is a Collector-side setting matching Mackerel's official config example, not an absolute API max request size

The practical upshot: if you already have an OTel Collector sending traces to Mackerel, adding logs is a second exporter pipeline entry, not a new integration pattern.

Mackerel's Logs feature top page. The beta notice (no data retention guarantee, unscheduled maintenance possible) is explicitly displayed.

The Architecture Question: Collector or Direct-Send?

The Collector path is recommended for most deployments. Direct-send is an option for minimal setups without a Collector, or for verifying protocol compatibility. Mackerel's own documentation also recommends placing a Collector in between for filtering and masking.

Criteria	Recommended Path
Production use, masking, multiple destinations, retry control	Collector
Minimize components in a small environment	Direct-send
Just want to try Mackerel integration first	Local Collector test
Handling security audit logs	Collector (filter sensitive attributes)

This repository's OTel Collector integration already ships FSx for ONTAP audit/EMS/FPolicy logs to Datadog, Grafana Cloud, and Honeycomb simultaneously from one Lambda codebase — the Lambda builds a backend-neutral OTLP payload, and the Collector's exporters config decides where it goes. Adding Mackerel as a fourth backend to that pattern needed zero Lambda changes:

# otel-collector-config-mackerel.yaml
# NOTE: This repo's verified Collector version (0.152.0) uses otlp_http. Older
# versions may require otlphttp (no underscore). Check your version's docs.
exporters:
  otlp_http/mackerel:
    endpoint: https://otlp-vaxila.mackerelio.com
    headers:
      Accept: "*/*"
      Mackerel-Api-Key: ${env:MACKEREL_APIKEY}
    sending_queue:
      batch:
        max_size: 3500000
        sizer: bytes
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [otlp_http/mackerel]

That's the whole integration for the Collector-mediated path. But this repository also supports a direct-send path — Lambda posts straight to a vendor's OTLP endpoint, skipping the Collector entirely — used today for Grafana Cloud via AUTH_MODE=basic. Checking whether that path also worked for Mackerel is where things got more interesting.

The Gap: A Header Mackerel Needs That the Direct-Send Path Couldn't Send

"Add an exporter and you're done" turned out not to cover every path. The direct-send auth logic in handler.py, ems_handler.py, and fpolicy_handler.py only supported two modes:

if AUTH_MODE == "basic":
    encoded = base64.b64encode(token.encode("utf-8")).decode("utf-8")
    auth_headers = {"Authorization": f"Basic {encoded}"}
else:
    auth_headers = {"Authorization": f"Bearer {token}"}

Mackerel's auth is neither. It's a bare custom header, Mackerel-Api-Key: <token>, with no Bearer or Basic wrapping. At that point, direct-send to Mackerel without a Collector simply wasn't possible.

The fix is a generic primitive rather than a Mackerel-specific branch:

AUTH_MODE = os.environ.get("AUTH_MODE", "bearer")  # "bearer", "basic", or "header"
AUTH_HEADER_NAME = os.environ.get("AUTH_HEADER_NAME", "Authorization")
EXTRA_HEADERS_JSON = os.environ.get("EXTRA_HEADERS_JSON", "")

# ...
elif AUTH_MODE == "header":
    auth_headers = {AUTH_HEADER_NAME: token}
# ...
if EXTRA_HEADERS_JSON:
    extra_headers = json.loads(EXTRA_HEADERS_JSON)
    auth_headers = {**(auth_headers or {}), **extra_headers}

AUTH_MODE=header sends the secret verbatim under any header name you specify. Mackerel happens to be the first consumer, but nothing about the option itself references Mackerel. EXTRA_HEADERS_JSON covers Mackerel's required Accept: */*, which isn't authentication at all — just a static header their backend depends on.

EXTRA_HEADERS_JSON constraint: This is for static, non-secret headers like Accept only. Do not specify auth headers (Authorization, Mackerel-Api-Key, etc.), Content-Type, Content-Length, or Host — these are controlled by AUTH_MODE/OTLP_CONTENT_TYPE settings or set automatically by the HTTP library.

The same fix went into ems_handler.py and fpolicy_handler.py too. The CloudFormation template passes these env vars to all three Lambda functions, so patching only one would have left the other two silently ignoring AUTH_MODE=header if someone set it.

12 new unit tests cover this across the three handlers (4 each in handler.py, ems_handler.py, and fpolicy_handler.py): custom-header auth, EXTRA_HEADERS_JSON merging, the case where no API key secret is configured but extra headers still apply, and invalid EXTRA_HEADERS_JSON producing a startup warning with extra headers disabled. cfn-lint and gitleaks run clean on the modified template and new files.

The Second Gap: Auth Was Fixed, But the Payload Format Wasn't

Fixing auth wasn't the finish line, though. The unit tests mock the HTTP layer entirely, so they passed without catching what a real API key call turned up next:

OTLP endpoint error 400: {"code":400,"message":"json is not supported yet"}

_send_otlp_payload always sends OTLP/JSON with Content-Type: application/json, but Mackerel's OTLP endpoint accepts Protobuf only and rejects JSON outright. The Collector-mediated path never surfaced this: the Collector decodes received OTLP data into its internal data model, then re-serializes via the otlphttp exporter whose default encoding is Protobuf (proto). So even though the Lambda sends OTLP/JSON to the Collector, the Collector sends OTLP/Protobuf to Mackerel. The direct-send Lambda code simply never had a Protobuf encoder.

This project's policy is to include no additional Python dependency packages in the Lambda deployment package. Only boto3 and urllib3 — both available in the Lambda Python runtime environment — are used; protobuf/opentelemetry-proto are not bundled. Adding those packages for one vendor felt disproportionate, so the fix is a small hand-rolled encoder (otlp_protobuf.py). This encoder is not a general-purpose OTLP implementation. It covers only the limited data structures this repository's FSx for ONTAP log builders produce: timeUnixNano, severityNumber, severityText, body.stringValue, attributes (stringValue only), resource attributes (stringValue only), and InstrumentationScope. It does not support the full OTLP Logs field set or all AnyValue types, and is not intended for reuse outside this project. Field numbers came straight from the official OTLP proto definitions (Apache License 2.0); the encoder's byte output was cross-checked against the official opentelemetry-proto generated Python classes in a throwaway virtualenv before it touched a real account. We monitor official proto definition updates and maintain compatibility tests against the generated classes.

OTLP_CONTENT_TYPE = os.environ.get("OTLP_CONTENT_TYPE", "json")  # "json" or "protobuf"

if content_type == "protobuf":
    headers = {"Content-Type": "application/x-protobuf"}
    body = encode_logs_data(payload)
else:
    headers = {"Content-Type": "application/json"}
    body = json.dumps(payload).encode("utf-8")

After adding OTLP_CONTENT_TYPE=protobuf, calling handler.py's actual build_otlp_payload and _send_otlp_payload functions — the same code the Lambda runs, not a reimplementation — against the real Mackerel API key succeeded. For the two sample records sent (ReadData/Success and Delete/Access Denied), this repository's generated audit attributes — operation type, result, file path, SVM name, user information, event timestamp — were confirmed searchable on Mackerel's log search UI.

Unit tests verify auth headers, Content-Type branching, and encoding output. However, they cannot detect vendor-side acceptance behavior, rate limits, network path issues, or search-indexing delays — those require E2E tests against the real endpoint, which we run separately. 20 more unit tests were added: 3 per handler for the new content-type path, plus 11 in a dedicated test_otlp_protobuf.py, bringing the OTel Collector integration's test suite to 110 passing tests.

Verifying one delivery path for a vendor doesn't verify another. Auth headers, payload wire format, and network reachability can each fail independently, and a unit-test suite that mocks the HTTP layer won't catch a payload-format mismatch that only a real vendor endpoint enforces.

Why This Still Isn't in the "9 Vendors, All Verified" List

Every other vendor in this repository has a checkmark for a reason: an actual payload was sent to a live account and confirmed to arrive. Mackerel now has that checkmark too. Sample FSx audit log payloads showed up in Mackerel's log search UI with this repository's generated audit attributes confirmed searchable, through both the Collector-mediated path and the direct-send path once OTLP_CONTENT_TYPE=protobuf was in place. This repo's README.md reflects that:

✅ E2E verified (open beta) — Confirmed end-to-end against a live account, but the backend platform's own feature is itself in open beta (no data retention guarantee, unscheduled maintenance possible) — see the linked integration README for beta constraints.

Verification Detail	Value
E2E verified	2026-07-18
Collector version	0.152.0 (OpenTelemetry Collector Contrib)
Region	ap-northeast-1
Repository commit	`main` branch HEAD at time of publication

E2E evidence: Searching by the fsxn-audit service shows 4 records — ReadData/Success, WriteData/Success, Open/Failure, Delete/Access Denied at WARN/INFO severity levels.

Log detail: Operation type, result, SVM name, file path, user info, client IP, and timestamp are all retained as searchable attributes.

So the code-side gate is cleared. There's a second reason the integration still doesn't sit in the main table, though, and it's not about this repository's code at all — it's about what Mackerel itself has published about its own feature:

No data-retention guarantee during the beta period
Unscheduled maintenance is possible
GA is planned for fall 2026, but the exact date isn't fixed yet

Cost note: As of July 2026, the beta period is free. GA pricing is planned as ingest-volume-based billing. Pricing details, measurement units, and free-tier terms may change after this article's publication — check Mackerel's official pricing page for current terms.

Folding this integration quietly into the "supported vendors" table with the same checkmark as the other 9 would let someone deploying it for security alerting (this repository's automated incident-response module, for instance) reasonably assume the same confidence level as Datadog or Splunk. That assumption would be wrong, not because of anything this repo's code does, but because the platform underneath it is explicitly pre-GA. So the integration stays in a separate "Emerging / Beta Vendors Under Evaluation" section of vendor-comparison.md, marked as E2E-verified-but-beta, until Mackerel's own log feature reaches GA.

One thing worth flagging for anyone reproducing this kind of verification: during investigation we also inspected GraphQL requests used internally by the browser UI, but we have not confirmed these are public/stable APIs. This repository's permanent E2E verification does not depend on them — final confirmation uses the official UI's log search screen combined with the Collector's own send metrics (otelcol_exporter_sent_log_records). Internal implementation details on Mackerel's side may change without notice.

Security Considerations

FSx for ONTAP audit logs may contain usernames, file paths, share names, and client IP addresses — content that may constitute sensitive or personally-identifiable information depending on your organization's policies. Before sending production data to any external observability service:

Confirm your organization's data classification, storage location, access control, and retention requirements
Use OTel Collector filter/transform processors to remove or mask sensitive attributes as needed (see this repository's PII redaction cookbook)
Do not use beta-period Mackerel as the sole storage destination for audit trails

On EXTRA_HEADERS_JSON specifically:

Never put secret values in it — API keys belong in Secrets Manager
Never put API keys directly in CloudFormation parameters or Lambda environment variables
Never log full header contents at runtime

Troubleshooting: 4-Layer Isolation

When issues occur, isolate across these four layers:

Layer	Target	What to Check
1. Generation	Lambda	OTLP record count, required attributes, timestamp, severity, resource attributes. If enabling debug logging, avoid outputting sensitive info like file paths or usernames
2. Transmission	Lambda / Collector exporter	HTTP response code, timeouts, retry results (400, 401, 403, 429, 5xx)
3. Acceptance	Collector metrics / OTLP response	`sent`/`failed` log records, partial success, drops
4. Consumption	Mackerel UI	service attribute match, search time range, filter conditions, indexing delay

Distinguishing Collector/HTTP transmission failures (layers 2-3) from post-acceptance search/display issues (layer 4) is critical. If Mackerel rejects with HTTP 400, that's a layer 2-3 problem, not layer 4. See the troubleshooting table in integrations/mackerel/docs/ for detailed guidance.

Recommended Adoption Steps

For evaluating this beta integration in stages:

E2E verification with sample logs using a local Collector (test-local-mackerel.sh)
Send non-production FSx for ONTAP audit logs in limited scope
Evaluate send volume, data loss, searchability, and sensitive-information handling
In production, implement dual-send to your existing log platform (CloudWatch Logs, S3, existing vendor)
During the beta period, do not use Mackerel as your sole audit trail destination
Re-evaluate for primary-path promotion once data retention terms, support conditions, and pricing are finalized at GA

What's Actually Usable Today

Both paths below are now confirmed working end-to-end:

# Collector-mediated (recommended, zero Lambda changes)
cp integrations/otel-collector/.env.mackerel.example .env.mackerel
# edit .env.mackerel with a Write-scoped Mackerel API key
bash integrations/otel-collector/scripts/test-local-mackerel.sh

# Direct-send (skips the Collector; needs AUTH_MODE=header AND OtlpContentType=protobuf)
aws cloudformation deploy \
  --template-file integrations/otel-collector/template.yaml \
  --stack-name fsxn-otel-integration \
  --parameter-overrides \
    OtlpEndpoint=https://otlp-vaxila.mackerelio.com \
    AuthMode=header \
    AuthHeaderName=Mackerel-Api-Key \
    ExtraHeadersJson='{"Accept":"*/*"}' \
    OtlpContentType=protobuf \
    ApiKeySecretArn=<secret-arn> \
    ... \
  --capabilities CAPABILITY_NAMED_IAM

OtlpContentType=protobuf is not optional for Mackerel — without it, the send fails with the JSON-rejection error described above, even with correct auth.

Full setup guides (bilingual) are in integrations/mackerel/docs/, including a troubleshooting table that specifically tells you to isolate Collector-side failures from Mackerel-side rejections before debugging the wrong layer.

Wrapping Up, and What's Next

So that's where Mackerel's log beta stands today. Getting there took two fixes that aren't visible when only the Collector-mediated path is tested — a custom auth header format, and a JSON-vs-Protobuf mismatch — and neither one surfaced until real credentials hit the direct-send path. A green unit-test suite doesn't tell you that.

The AUTH_MODE=header, EXTRA_HEADERS_JSON, and OTLP_CONTENT_TYPE=protobuf options added here are not Mackerel-specific code — they're reusable primitives for any future OTLP/HTTP backend that requires a custom auth header or Protobuf-only ingestion.

A few things to watch going forward:

Watch for Mackerel's GA announcement for its log feature (data retention terms, pricing) — this article and the linked docs will be updated, not silently left stale, once that happens.
Once GA lands with confirmed data retention conditions, support terms, and pricing, move Mackerel from "Emerging / Beta Vendors" into the main "Supported Vendors" comparison table.
If you're running Mackerel already, the setup guide's sample OTLP payload is a good starting point to validate your own organization's setup independently.

Mackerel's log feature still has some growing to do before GA, and I'm looking forward to seeing where it lands. Hope this was useful to someone wiring up the same thing.

DEV Community: Yoshiki Fujiwara(藤原 善基)@AWS Community Builder

What to build a file portal on — Nx Plugin for AWS 1.0, AWS Blocks and Amplify Gen 2, same four features (File Portal Part 7)

Introduction

Switching tools does not touch the data foundation underneath

The same four features on all three tools (2026-09-14)

The listing

Reading

Uploading

How the IAM permission differs by tool

Which layer refuses a malformed input

What this round measured (deploy time and resource count)

Deploying the starters as generated (2026-09-13)

Fixed cost that accrues at zero requests

What a production portal still needs

Tearing down the evaluation — what a stack deletion leaves

Two behaviours that are easy to misread in operation

Trap 1: a deployment mode cannot be dropped later

Trap 2: DELETE_COMPLETE does not always mean gone

Choosing between them

Test environment

The record in the repository

Closing

How far ONTAP features reach through an S3 Access Point — measuring qtree, quota and FlexClone (File Portal Part 6)

Introduction

The reach of ONTAP features on this path

Test environment

The distinction that matters most — different subjects

The walkthrough

Qtree — usable

Quota — enforced against an S3 PUT

FlexClone — both volume and file granularity

FlexGroup — access point attachment

What actually failed on this path

A security-style and identity mismatch — failure after a successful attachment

The wait for something created in ONTAP to appear on the AWS side

A residual NAS bucket refusing volume deletion on the ONTAP side

Putting it in a runbook

What the schedule took over

Progressive automation — four levels by design, two in the code

What I did not measure

Conclusion

FlexGroup creation and capacity rebalancing failed on the defaults — what the error codes said (File Portal Part 5)

Introduction

Test environment

The distinction that matters most — ONTAP's words and AWS's words

What to watch when creating a FlexGroup

The capacity rebalance that never starts with the defaults

The space that does not come back after a delete

What to check when a FlexClone volume will not delete

Four pitfalls in the S3-compatible API

The 501 failure of a "write only if absent" request

GOVERNANCE and COMPLIANCE, indistinguishable from the error text

The lock indicator that does not change under automatic locking

The usable endpoints, derived from the API rather than a config file

Two places the documentation and the running system disagreed

The eight defaults that can invalidate a measurement

Putting it in a runbook

What I did not measure

Conclusion

Putting FSx for ONTAP operations in a browser — the buttons I disabled, and the ones I never built (File Portal Part 4)

Introduction

Test environment

The distinction that matters most — a path that persists, a holder that moved

The reach of the screen

The plain statement of what cannot be done

What the confirmation screen states for an irreversible operation

What I did not measure

Conclusion

AWS Backup now copies FSx for ONTAP backups across Regions and accounts — a new option, measured Tokyo to Osaka

Introduction

Shape of the setup

Test environment

When this fits

When to look at something else for now

The distinction that matters most — what changed is where a backup can live

How CopyBackup and AWS Backup relate

Replication and copy are different words for different things

A short glossary

If you run ONTAP on-premises

Prerequisites and permissions

DEV Community: Yoshiki Fujiwara(藤原善基)@AWS Community Builder

How `CopyBackup` and AWS Backup relate

A plan-triggered job waits in `CREATED`

1. A restoring volume reads as `DP`, not `RW`

The console shows `DP` too, and does not refresh

1. Scheduling — `CopyBackup` has none