DEV Community: Yobitel

How do you connect CVAT seamlessly to Amazon S3 storage?

Yobitel — Thu, 30 Jul 2026 07:14:30 +0000

Connecting CVAT to Amazon S3 requires 3 verified steps:

Creating an S3 bucket with Block Public Access enabled
Creating an IAM user with AmazonS3ReadOnlyAccess or AmazonS3FullAccess permissions
Registering the bucket in CVAT under Cloud Storage → + using your IAM Access Key ID and Secret Access Key

CVAT outputs finished annotation datasets back to S3 once it is linked, avoiding manual file transfers.

Annotation tools require local access to media files, while production datasets are stored in object storage. Through CVAT's native Amazon S3 integration, annotators can access data directly from S3 buckets, eliminating the need for local copies, sync scripts, and storage overhead on the CVAT instance itself.

What is CVAT and why use Amazon S3 with it?

CVAT is an open-source platform for labelling image and video datasets used in computer vision and ML pipelines. Amazon S3 eliminates local storage bottlenecks by allowing distributed annotation teams to access shared datasets directly from the cloud.

CVAT supports every major annotation type used in computer vision:

Bounding boxes
Polygons
Polylines
Ellipses
Cuboids
Keypoints
Tags
Segmentation masks
3D point cloud annotations

It handles both image datasets and video files, and exports annotations in formats including

COCO
Pascal VOC
YOLO (Ultralytics and standard)
CVAT XML
Datumaro
LabelMe
KITTI
Cityscapes and more.

For teams annotating at scale, storing raw media locally creates two fundamental problems.

Storage capacity - annotation projects regularly involve tens of thousands of images or hours of video.
Collaboration - multiple annotators pulling from a shared local directory creates sync conflicts, bandwidth bottlenecks, and version control headaches.

Amazon S3 resolves both. A single bucket can store datasets of any size, S3 versioning protects source files from accidental modification, and CVAT's Cloud Storage module connects directly to S3 so annotators can access frames on demand without downloading anything locally. This integration is especially straightforward when CVAT runs as the Yobitel Image & Video Annotation Platform built on CVAT, deployed directly from AWS Marketplace. Co-locating the EC2 instance and S3 bucket in the same region reduces latency and data transfer costs from day one.

1. Create your Amazon S3 bucket

Create an S3 bucket in the AWS console, specify the bucket name and region, and enable Block All Public Access. It uses IAM credentials for bucket access, which should remain disabled for security.

Log in to your AWS account and navigate to the Amazon S3 console.
Click Create bucket. Enter a globally unique bucket name (e.g., cvat-annotation-data-yourorg). Select the AWS region closest to your CVAT EC2 instance. Co-location is critical for performance.
Under Block Public Access, enable Block all public access. CVAT authenticates with IAM credentials, so public access is not needed.
Leave all other settings at their defaults and click Create bucket.

Place the S3 bucket in the same AWS region as your CVAT EC2 instance. Cross-region data transfer adds latency on every frame load and incurs data transfer costs, especially impactful for high-frame-count video annotation tasks.

2. Upload your dataset to S3

Upload your image or video dataset to the S3 bucket using the AWS console or AWS CLI. Organise files under a logical prefix. A manifest.jsonl file is optional but recommended for cloud storage integrations. It improves task creation speed and enables caching mode.

Organise files under a prefix that separates raw media from annotation outputs:

cvat-annotation-data-yourorg/
├── project-alpha/
│ ├── raw/
│ │ ├── image_001.jpg
│ │ ├── image_002.jpg
│ │ └── manifest.jsonl ← is optional but recommended
│ └── annotations/
│ └── coco_export.zip

Upload large datasets via the AWS CLI:
bash aws s3 cp /local/path/to/dataset/ s3://cvat-annotation-data-yourorg/project-alpha/raw/ --recursive
CVAT manifest file

The manifest.jsonl file is a dataset descriptor in JSONL format, one JSON object per line that CVAT uses when reading data from cloud storage. Manifest files are mainly used when working with cloud storage to reduce the amount of network traffic used and speed up the task creation process.

For an image dataset, it looks like this:
json {"version": "1.0"} {"type": "images"} {"name": "image_001", "extension": ".jpg", "width": 1920, "height": 1080, "meta": {"related_images": []}, "checksum": "a1b2c3d4..."} {"name": "image_002", "extension": ".jpg", "width": 1920, "height": 1080, "meta": {"related_images": []}, "checksum": "b2c3d4e5..."}.
Note: The manifest tool generates an MD5 hash automatically, not entered manually.

Generate the manifest automatically using the CVAT Docker utility:

bash docker run -it --rm -u "$(id -u)":"$(id -g)" \ -v "${PWD}":"/local" \ --entrypoint python3 \ cvat/server \ utils/dataset_manifest/create.py --output-dir /local /local/<path/to/images>
Then upload the manifest to the same S3 prefix as your images:

bash aws s3 cp manifest.jsonl s3://cvat-annotation-data-yourorg/project-alpha/raw/
If your IAM user has read-only access, upload the manifest via the S3 console browser (drag and drop) instead of the CLI, since the CLI write requires s3:PutObject.

3. Configure IAM access permissions

CVAT officially supports two AWS-managed IAM policies:
AmazonS3ReadOnlyAccess for read-only annotation workflows, and AmazonS3FullAccess for workflows that also export annotation outputs back to S3. Create a dedicated IAM user with programmatic access, attach the appropriate policy, and save the Access Key ID and Secret Access Key.

Verified from the official CVAT docs at docs.cvat.ai:

Navigate to IAM in the AWS console and select Add users.
Set a User name (e.g., cvat-s3-access) and enable Access key – programmatic access.
Click Next: Permissions, then select Create group and name the group (e.g., cvat-s3-group).
Search for and attach the appropriate policy: AmazonS3ReadOnlyAccess for annotation-only workflows where CVAT only reads from S3, or AmazonS3FullAccess for full pipelines where CVAT also exports annotation datasets back to the bucket.
Complete user creation and save the Access Key ID and Secret Access Key — these are shown once only.

Policy	What It Grants	Best For
AmazonS3ReadOnlyAccess	`s3:Get`, `s3:List`, `s3:Describe*`	Read-only annotation workflows
AmazonS3FullAccess	All Amazon S3 actions	Full annotation pipelines with export back to Amazon S3
Custom Least-Privilege	Access limited to a specific S3 bucket ARN	Production and multi-tenant environments

AWS managed policies grant access to all S3 buckets in the account. For production, create a custom policy scoped to your specific bucket: apply s3:ListBucket at the bucket level (arn:aws:s3:::your-bucket-name) and s3:GetObject, s3:PutObject at the object level (arn:aws:s3:::your-bucket-name/*).

CVAT also supports anonymous access to publicly accessible S3 buckets. For production data or proprietary data, authenticated IAM access should be used when the data is sensitive.

4. Register the S3 bucket in CVAT

In CVAT, navigate to Cloud Storage from the top menu, click the + button, select AWS S3 as the provider, and fill in the bucket name, region, and IAM credentials. After clicking Submit, CVAT validates the connection. A green status confirms successful authentication.

Log in to your CVAT instance. In the top navigation menu, select Cloud Storage.
Click the + button in the upper-right area of the Cloud Storage page.
Fill in the form:

CVAT Field	What to Enter
Display Name	Any descriptive name for the storage connection
Provider	AWS S3
Bucket Name	Enter the exact S3 bucket name (case-sensitive)
Region	AWS Region where the bucket is located (for example, `eu-west-2`)
Access Key ID	IAM user's Access Key ID
Secret Access Key	IAM user's Secret Access Key
Prefix (Optional)	Folder path inside the bucket, such as `project-alpha/raw/`
Manifest Path (Optional)	Path to the `manifest.jsonl` file within the bucket

Click Submit. CVAT attempts to authenticate and list the bucket contents.
Check the connection status. A green status confirms successful authentication. A red status requires troubleshooting.

The most common cause of a red status is a region mismatch between what was entered in the CVAT form and the bucket's actual region. Other causes include incorrect credentials with trailing whitespace, a typo in the bucket name, or a missing s3:ListBucket permission on a custom policy.

5. Create annotation tasks using S3 as the data source

When creating a new CVAT task, switch the data source from Local files to Cloud storage, select the registered S3 bucket, specify a dataset prefix or filename pattern, and complete the task configuration. Annotators stream media directly from S3 during the annotation session. Full deployment steps for the AMI, including instance sizing and first login, are covered in the Yobitel CVAT user manual.

Click Create Task from the CVAT Tasks page.
Enter a task name and configure annotation labels.
Under Select files, switch from My computer to Cloud storage.
Select the registered S3 bucket from the dropdown. Browse contents or use the filename pattern field (e.g., project-alpha/raw/*.jpg) to filter files.
Select the files or directory to annotate, configure overlap and segment size, then click Submit & Open.
On task completion, use Actions → Export dataset to write annotation files back to S3 or download locally.

CVAT supports glob-style filename patterns when selecting cloud storage files. Use patterns like .jpg, batch_01/.png, or */frames/.jpeg to target specific file sets without browsing the full directory tree.

Troubleshooting common CVAT S3 errors:

Common Issues and Fixes

Error	Likely Cause	Recommended Fix
Red connection status	Region mismatch, incorrect credentials, or missing `ListBucket` permission	Verify the AWS Region, confirm the IAM credentials, and check the IAM policy.
Invalid `manifest.jsonl` file structure	Required fields such as `name` or file extension are missing	Regenerate the manifest using the CVAT Docker utility.
Task creation fails: "No media data found"	The filename pattern does not match any uploaded files	Verify the prefix path and confirm the files exist in the specified S3 location.
Slow frame loading	Cross-Region latency or missing manifest/caching	Deploy EC2 and S3 in the same AWS Region and use a `manifest.jsonl` file.
Access denied on export	IAM user has read-only permissions	Grant `s3:PutObject` permission or use AmazonS3FullAccess if appropriate.

Architecture value at scale

S3 integration decouples dataset storage from compute capacity, which has practical consequences for how annotation infrastructure scales. The running instance can be resized, replaced, or replicated without interrupting access to annotation data. Buckets feeding active tasks can simultaneously serve model training pipelines, data versioning systems, and QA review workflows, eliminating redundant copies across the machine learning stack.

For teams managing multi-annotator projects, parallel task assignments, or continuous active learning pipelines on AWS, centralising datasets in S3 is what makes CVAT a production-viable annotation platform rather than a single-user tool. Teams evaluating a managed deployment path can explore Yobitel's AWS Marketplace solutions as a starting point. For the full EC2 launch and login walkthrough, see how to annotate datasets through AWS using CVAT.

ERPNext on AWS Replaces 5 Business Tools With One Unified Platform

Yobitel — Wed, 29 Jul 2026 10:11:16 +0000

ERPNext on AWS replaces five business subscriptions, accounting, inventory, HR and payroll, CRM, and procurement, with one open source ERP running in your own AWS account. Yobitel's prebuilt AMI (Amazon Machine Image, a ready-to-launch server image with everything installed) brings it live in under 15 minutes, with no per-seat licensing and no vendor lock-in.

A typical growing business runs those five functions on five disconnected tools: QuickBooks for accounting, Cin7 or a spreadsheet for inventory, Gusto for payroll, HubSpot for the sales pipeline, and an email chain for purchase approvals. Five logins, five separate databases, five monthly invoices.

Disconnection creates daily rework. Finance invoices customers without seeing current stock levels. Payroll totals are retyped into the accounting system every month. A closed deal travels from the CRM to the order system by hand, and purchase records live in inboxes instead of ledgers.

ERPNext removes the rework by running all five functions on one shared dataset. Below is a tool-by-tool breakdown of what it replaces, how each module connects to the others, and what changes for your business when five systems become one.

1. ERPNext Accounting Connects Finance to Live Operations

What businesses currently use: QuickBooks, Xero, FreshBooks, or Sage for invoicing, expense tracking, bank reconciliation, and financial reporting.