Enterprise Log Management in OCI: Moving Logs from OCI Logging to Object Storage Using Service Connector Hub

Introduction

Logs are one of the most valuable sources of information in any cloud environment. Whether you're troubleshooting application failures, investigating security incidents, or meeting compliance requirements, having access to historical logs is critical.

During one of my OCI implementations, I needed a centralized solution to retain logs generated by OCI services without relying on custom scripts or third-party tools. OCI provides a native solution using Logging, Service Connector Hub, and Object Storage, allowing logs to be automatically archived for long-term retention and future analysis.

In this article, I'll walk through the complete implementation and share some practical validation and troubleshooting steps I used during testing.

Understanding the Architecture

The solution consists of four OCI services working together:

OCI Logging
OCI Service Connector Hub
OCI Object Storage
OCI Logging Analytics (Optional)

High-level architecture:

Load Balancer
Compute Instance
VCN Flow Logs
Audit Logs
│
▼
OCI Logging
│
▼
Log Group
│
▼
Service Connector Hub
│
▼
Object Storage Bucket
│
▼
Logging Analytics (Optional)

The objective is simple:

• Collect logs in OCI Logging.
• Automatically move them to Object Storage.
• Retain them for future troubleshooting, auditing, or analytics.

Prerequisites

Before starting, ensure the following resources are available:

• OCI tenancy
• Compartment
• OCI Load Balancer (or any OCI service generating logs)
• Object Storage bucket
• Appropriate IAM permissions

Step 1: Create an Object Storage Bucket

Navigate to:

Storage
└── Object Storage & Archive Storage

Create a bucket.

Example:

Bucket Name: oci-log-archive
Storage Tier: Standard

Note : For production environments, consider enabling lifecycle policies to move older logs to Archive Storage.

Step 2: Create a Log Group

Navigate to:

Observability & Management
└── Logging
└── Log Groups

Create a dedicated log group.

Example:

lb-production-logs

I generally prefer separating logs by workload:

prod-network-logs
prod-security-logs
prod-application-logs

This makes troubleshooting significantly easier later.

Step 3: Enable Load Balancer Logging

Navigate to:

Networking
└── Load Balancer
└── Logs

Enable:

Access Logs
Error Logs

Select the log group created earlier.

Once enabled, OCI starts collecting load balancer events.

Step 4: Create Service Connector

Now we will create the Service Connector.

Go to:

Analytics & AI → Messaging → Connector Hub

Step 5: Configure the Source Connection

In the Source Connection section, select the logs that need to be archived.

Provide the following details:

Compartment: The compartment where the logs are generated
Log Group: The log group containing the logs
Logs: Select the required log source

For example, if you are archiving Load Balancer Access Logs, select the corresponding log group and access log entry.

Service Connector Hub also supports selecting multiple log sources. Additional logs can be added using the Add Another Log option.

For this implementation, I selected a single log source to keep the configuration simple and easy to validate.

Step 6: Configure the Target Connection

Next, configure the destination where the logs will be stored.

Select:

Target Service: Object Storage
Compartment: Compartment containing the bucket
Bucket: oci-logging-archive-bucket

Once the connector is created, Service Connector Hub automatically transfers matching log records from OCI Logging to the specified Object Storage bucket.

This bucket now becomes the centralized repository for archived logs and can later be integrated with OCI Logging Analytics for advanced searching, troubleshooting, and reporting.

Step 7: Validate Log Delivery

Generate additional traffic:

for i in {1..50}
do
curl http://
done

Wait several minutes.

Navigate to:

Storage
└── Object Storage
└── oci-log-archive

You should now see log objects being created automatically.

This confirms the Service Connector is functioning correctly.

Troubleshooting

Issue 1: No Logs in OCI Logging

Possible causes:

Logging not enabled
Incorrect log group
No traffic generated

Verification:

Load Balancer
└── Logs
└── Status = Enabled

Issue 2: Logs Appear in OCI Logging but Not Object Storage

Possible causes:

Connector inactive
IAM permission issue

Verification:

Service Connector Status = Active

Review IAM policies.

Issue 3: Empty Bucket

Possible causes:

Wrong destination bucket
Connector misconfiguration

Verification:

Review source and target configuration.

Real-World Use Cases

This architecture is useful for:

• Security Investigations
• Retain logs for incident response and forensic analysis.
• Compliance Requirements
• Store logs for audit purposes.

Cost Optimization

• Move older logs to Archive Storage using lifecycle policies.
• Centralized Operations
• Create a single location for operational log retention.

Best Practices

During implementation, I found the following practices helpful:

• Use dedicated log groups per workload.
• Create separate buckets for production and non-production environments.
• Enable lifecycle policies for long-term retention.
• Restrict bucket access using IAM policies.
• Periodically validate connector health.
• Integrate archived logs with Logging Analytics for deeper analysis.

Conclusion

OCI Service Connector Hub provides a simple and scalable way to automate log movement from OCI Logging to Object Storage. By combining these services, organizations can build a centralized log retention architecture without maintaining custom scripts or additional infrastructure.

The implementation is straightforward, operationally efficient, and suitable for production environments that require long-term log retention, compliance, and troubleshooting capabilities.