TL;DR
We built a serverless Lambda pipeline that ships FSx for ONTAP audit logs to Sumo Logic's JP (Tokyo) region deployment. For Japanese enterprises with data residency requirements under APPI (Act on the Protection of Personal Information), this means audit logs never leave Japan.
FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → Sumo Logic HTTP Source (JP)
│
▼
┌───────────────────┐
│ Sumo Logic JP │
│ (Tokyo) │
│ │
│ • 500 MB/day FREE │
│ • Data stays in │
│ Japan │
│ • 7-day retention │
│ (free tier) │
└───────────────────┘
Key advantages:
- 500 MB/day free tier (~15 GB/month) — covers most FSx for ONTAP deployments at zero vendor cost
- JP region deployment — data residency in Tokyo
- Simplest auth model — URL-embedded token, no header management
- 30-minute end-to-end — HTTP Source URL is the only credential needed
Verified on Sumo Logic JP region. Logs searchable via _sourceCategory=aws/fsxn/audit.
This is Part 12 of the Serverless Observability for FSx for ONTAP series.
Why Sumo Logic for Japanese Enterprises?
For organizations operating under Japanese data protection regulations, the choice of observability platform often comes down to one question: where does the data physically reside?
| Requirement | Sumo Logic JP | Other Options |
|---|---|---|
| Data residency in Japan | ✅ Tokyo deployment | Varies by vendor |
| APPI compliance consideration | ✅ Data stays in JP | May require cross-border assessment |
| Free tier for validation | ✅ 500 MB/day | Most offer 14-day trials only |
| No agent installation | ✅ HTTP Source (agentless) | Some require collectors |
Sumo Logic's JP deployment (service.jp.sumologic.com) processes and stores all data within Japan, making it a straightforward choice for organizations that need to demonstrate data residency compliance.
Compliance note: This integration provides a technical path for data residency. Evaluate your specific regulatory requirements with your compliance team — data residency alone does not constitute full regulatory compliance.
Architecture
┌─────────────────────────────────────────────────────────┐
│ Event Sources │
├─────────────────────────────────────────────────────────┤
│ │
│ EventBridge Scheduler │
│ rate(5 minutes) ──→ Lambda │
│ │ lists new files via │
│ │ S3 Access Point │
│ │ (checkpoint in SSM) │
│ ▼ │
│ Sumo Logic HTTP Source │
│ (URL-embedded auth) │
│ │ │
│ EMS Webhook │ │
│ ──→ API GW ──→ Lambda ─────────────┤ │
│ (ems_handler) │ │
│ ▼ │
│ FPolicy Sumo Logic │
│ ──→ ECS Fargate ──→ SQS (Log Search, │
│ ──→ Bridge Lambda Dashboards, │
│ ──→ EventBridge Alerts) │
│ ──→ Lambda (fpolicy_handler) ──────────────────────────┤
└─────────────────────────────────────────────────────────┘
The Simplest Auth Model
Sumo Logic's HTTP Source embeds the authentication token directly in the URL. No separate auth headers, no API key management, no token rotation complexity:
https://collectors.jp.sumologic.com/receiver/v1/http/<TOKEN>
Lambda just POSTs JSON to this URL. That's it. The metadata (source category, name, host) is sent via X-Sumo-* headers:
headers = {
"Content-Type": "application/json",
"X-Sumo-Category": "aws/fsxn/audit",
"X-Sumo-Name": "fsxn-ontap-audit",
"X-Sumo-Host": "fsxn-ontap"
}
⚠️ Security: The HTTP Source URL contains the auth token. Store it in Secrets Manager, never log it, and never expose it in environment variables or source control.
Quick Start (30 Minutes)
1. Create Sumo Logic Account (JP Region)
- Sign up at service.jp.sumologic.com
- Select APAC: Tokyo (JP) as your deployment region
- Free tier: 500 MB/day, 7-day retention, full feature access for 30 days
2. Create Hosted Collector + HTTP Source
- Go to Manage Data → Collection → Add Collector
- Select Hosted Collector (name:
fsxn-audit-collector) - Add HTTP Logs & Metrics Source:
- Name:
fsxn-ontap-audit - Source Category:
aws/fsxn/audit - Timestamp Parsing: Auto-detect
- Name:
- Copy the generated HTTP Source URL
3. Store HTTP Source URL
aws secretsmanager create-secret \
--name "sumo-logic/fsxn-http-source" \
--secret-string '{"url":"https://collectors.jp.sumologic.com/receiver/v1/http/<YOUR_TOKEN>"}' \
--region ap-northeast-1
4. Deploy CloudFormation Stack
aws cloudformation deploy \
--template-file integrations/sumo-logic/template.yaml \
--stack-name fsxn-sumo-logic-integration \
--parameter-overrides \
S3AccessPointArn=arn:aws:s3:ap-northeast-1:123456789012:accesspoint/fsxn-audit-ap \
SumoLogicHttpSourceSecretArn=arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:sumo-logic/fsxn-http-source-XXXXXX \
S3BucketName=my-fsxn-audit-bucket \
--capabilities CAPABILITY_NAMED_IAM \
--region ap-northeast-1
5. Verify in Sumo Logic
Run this search query:
_sourceCategory=aws/fsxn/audit
Note: First-time indexing on a new JP region account takes ~10 minutes. Subsequent ingestion is near-instant.
Query Examples
Sumo Logic uses a pipe-based query language:
Basic Investigation
-- All failed access attempts with user and path
_sourceCategory=aws/fsxn/audit
| json "Result", "UserName", "ObjectName"
| where Result = "Failure"
| count by UserName, ObjectName
| sort by _count desc
-- Top operations by volume
_sourceCategory=aws/fsxn/audit
| json "Operation"
| count by Operation
| sort by _count desc
-- Access pattern timeline (5-minute buckets)
_sourceCategory=aws/fsxn/audit
| json "Operation", "UserName"
| timeslice 5m
| count by _timeslice, Operation
Security Investigation
-- After-hours access (outside 9am-6pm JST)
_sourceCategory=aws/fsxn/audit
| json "UserName", "Operation", "ObjectName"
| where _messagetime < today() + 9h OR _messagetime > today() + 18h
| count by UserName, Operation
-- Bulk delete detection (potential data exfiltration)
_sourceCategory=aws/fsxn/audit
| json "Operation", "UserName"
| where Operation = "Delete"
| timeslice 5m
| count by _timeslice, UserName
| where _count > 50
-- Sensitive path access
_sourceCategory=aws/fsxn/audit
| json "ObjectName", "UserName", "Result"
| where ObjectName matches "*confidential*" OR ObjectName matches "*restricted*"
| count by UserName, Result
Operational Monitoring
-- Log volume trend (capacity planning)
_sourceCategory=aws/fsxn/audit
| timeslice 1h
| count by _timeslice
| outlier _count
-- SVM activity comparison
_sourceCategory=aws/fsxn/audit
| json "SVMName"
| timeslice 15m
| count by _timeslice, SVMName
Sumo Logic Metadata Headers
Lambda sends structured metadata with each request:
| Header | Value | Purpose |
|---|---|---|
X-Sumo-Category |
aws/fsxn/audit |
Primary search dimension |
X-Sumo-Name |
fsxn-ontap-audit |
Source identification |
X-Sumo-Host |
fsxn-ontap |
Host-level grouping |
These headers enable efficient searching without parsing the log body:
-- Search by metadata (fast, no JSON parsing)
_sourceCategory=aws/fsxn/audit _sourceHost=fsxn-ontap
Cost Analysis: The Free Tier Advantage
Sumo Logic's free tier is the most generous for small-to-medium FSx for ONTAP deployments:
| Monthly Log Volume | Daily Average | Sumo Logic Tier | Monthly Cost |
|---|---|---|---|
| 1 GB | ~33 MB/day | Free | $0 |
| 5 GB | ~167 MB/day | Free | $0 |
| 10 GB | ~333 MB/day | Free | $0 |
| 15 GB | ~500 MB/day | Free (at limit) | $0 |
| 30 GB | ~1 GB/day | Professional | ~$108/month |
| Component | Monthly Cost (10 GB/month) |
|---|---|
| Lambda (5-min polling) | ~$3 |
| EventBridge Scheduler | ~$1 |
| Secrets Manager | ~$1 |
| Sumo Logic | $0 (within free tier) |
| Total | ~$5 |
For most FSx for ONTAP deployments generating < 15 GB/month of audit logs, the total cost is just the AWS infrastructure (~$5/month). The observability platform itself is free.
Sumo Logic Deployment Regions
| Region | URL | Data Residency |
|---|---|---|
| JP (Tokyo) | service.jp.sumologic.com |
Japan |
| US1 | service.sumologic.com |
US |
| US2 | service.us2.sumologic.com |
US |
| EU (Ireland) | service.eu.sumologic.com |
EU |
| AU (Sydney) | service.au.sumologic.com |
Australia |
| IN (Mumbai) | service.in.sumologic.com |
India |
| CA (Montreal) | service.ca.sumologic.com |
Canada |
Select the deployment matching your data residency requirements at account creation time. Region cannot be changed after account creation.
Gotchas & Lessons Learned
| # | Discovery | Impact |
|---|---|---|
| 1 | JP region new accounts have ~10 minute initial indexing lag | First search returns empty; wait and retry |
| 2 | Search queries use _sourceCategory (underscore prefix) |
Common mistake: sourceCategory without underscore returns nothing |
| 3 | HTTP Source URL contains embedded auth token | Rotate by creating a new HTTP Source, updating the Secrets Manager secret, then deleting the old Source |
| 4 | Free tier has 7-day retention only | Sufficient for real-time monitoring; archive to S3 for long-term |
| 5 | No built-in Firehose support | Lambda direct delivery only |
| 6 | Max 1MB per request (newline-delimited JSON) | Lambda batches accordingly |
| 7 | Region is permanent — cannot migrate data between deployments | Choose JP at signup for data residency |
Free Tier vs Professional
| Feature | Free (500 MB/day) | Professional |
|---|---|---|
| Daily ingestion | 500 MB | 1+ GB (configurable) |
| Retention | 7 days | 30-365 days |
| Users | 3 | Unlimited |
| Alerts | ✅ | ✅ |
| Dashboards | ✅ | ✅ |
| API access | ✅ | ✅ |
| Data residency | ✅ (region-specific) | ✅ |
Tip: Enable Field Extraction Rules (FER): After first ingestion, create a FER for
_sourceCategory=aws/fsxn/auditwith "Auto-parse JSON" enabled. This automatically extracts all JSON fields (UserName, Operation, ObjectName, etc.) as searchable metadata — no manual field definition needed. Go to Manage Data → Logs → Field Extraction Rules → Add Rule.
For most PoC and small production deployments, the free tier is sufficient. Upgrade when you need longer retention or higher volume.
Data Residency & Classification
Sumo Logic JP deployment keeps all data in Japan, but audit logs still contain PII fields:
| Field | Classification | Sumo Logic Handling |
|---|---|---|
UserName |
PII | Use RBAC to restrict search access; consider field extraction rules for masking |
ObjectName |
Sensitive | Path may reveal business context; restrict dashboard sharing |
ClientIP |
Internal | Generally acceptable |
For APPI compliance considerations:
- Data stays in JP region (no cross-border transfer for log data)
- Configure retention policies matching your regulatory requirements (7 days free tier vs 30-365 days paid)
- Implement Sumo Logic RBAC to restrict PII field access by role
See the Data Classification Guide for full field classification and regulatory mapping.
Production Readiness
This integration follows the project's Production Readiness Levels:
| Level | What You Get | Go/No-Go to Next |
|---|---|---|
| Level 1 (this Quick Start) | Audit poller + DLQ | Logs arrive, checkpoint advances, DLQ empty 24h |
| Level 2 | + Sumo dashboards + alerts | SLOs met 7 days, security review done |
| Level 3 | + DynamoDB ledger + poison-pill | SLOs met 30 days, compliance pack |
| Level 4 | + OTel Collector + redaction | Multi-backend, PII redaction, DR tested |
Full criteria: Pipeline SLO Definitions | DLQ Replay Runbook
Enterprise scale: For multi-account deployments across your Organization, see the StackSets deployment guide. For compliance evidence collection (ISMAP, FISC, SOC 2), see the Compliance Evidence Pack. For regulation-to-vendor retention mapping, see the Retention Policy Matrix.
CloudFormation Templates
| Template | Purpose | Key Parameters |
|---|---|---|
template.yaml |
FSx audit log poller | S3AccessPointArn, SumoLogicHttpSourceSecretArn |
template-ems.yaml |
EMS webhook handler | SumoLogicHttpSourceSecretArn |
template-fpolicy.yaml |
FPolicy EventBridge handler | SumoLogicHttpSourceSecretArn, EventBusName |
Resources
- GitHub: integrations/sumo-logic/
- Sumo Logic JP: service.jp.sumologic.com
- HTTP Source Docs: Configure HTTP Source
- Search Query Language: Search Operators
- Series GitHub: github.com/Yoshiki0705/fsxn-observability-integrations
Series Navigation
- Part 1: Why Your FSx for ONTAP Logs Deserve Better
- Part 2: Shipping FSx for ONTAP Logs to Datadog — The Serverless Way
- Part 3: Event-Driven Ransomware Detection with ONTAP ARP + Datadog
- Part 4: FPolicy File Activity Pipeline — ONTAP to Datadog via ECS Fargate
- Part 5: Escape Vendor Lock-in with OTel Collector
- Part 6: Direct-to-Grafana: Shipping Logs via OTLP Gateway
- Part 7: New Relic: 100GB Free Tier for FSx Audit Logs
- Part 8: EC2 to Serverless: Modernizing Splunk Integration
- Part 9: Data Sovereignty with Elastic
- Part 10: High-Cardinality Analysis with Honeycomb
- Part 12: JP Region with Sumo Logic (this post)
Questions about the Sumo Logic JP integration or data residency? Drop a comment below.
GitHub: github.com/Yoshiki0705/fsxn-observability-integrations
Top comments (0)