DEV Community: VisuaLeaf

Learn how to use DuckDB for local analysis: open a database, query CSV and Parquet files, inspect relationships, edit data, and create charts.

VisuaLeaf — Thu, 03 Sep 2026 07:21:04 +0000

Roxana-Maria Haidiner

Sep 3

How to Use DuckDB for Local Data Analysis

#duckdb #sql #database #softwaredevelopment

8 min read

mongodump Tasks Automate and Schedule MongoDB Backups

VisuaLeaf — Mon, 31 Aug 2026 11:00:28 +0000

Running a MongoDB backup once is not difficult. With MongoDB Database Tools installed, the complete PowerShell command can be as short as this:

& "C:\Program Files\MongoDB\Tools\100\bin\mongodump.exe" `
  --uri="mongodb://127.0.0.1:27017" `
  --db="gui_test_db" `
  --gzip `
  --out="C:\MongoDBBackups\gui_test_db"

This example uses an unauthenticated local MongoDB deployment. Authenticated or remote deployments also require the appropriate credentials, authentication database, TLS settings, and connection options.

mongodump is MongoDB's command-line utility for creating a binary export of a database or collection. The problem is rarely the first run. It is repeating the same backup every day or week, keeping the settings consistent, using a fresh destination, and checking whether the operation actually completed.

That is where a saved task becomes useful. In VisuaLeaf, the same binary dump can be configured once, run manually when needed, scheduled, and checked later from its execution history.

Save the Same mongodump as a VisuaLeaf Task

Open *Task Manager*, create a new job, and choose the binary dump/restore operation. Select *Export to Dump* for the operation.

The full gui_test_db database saved as a reusable mongodump task in VisuaLeaf.

The task should have a name that still makes sense when you return to it later. I used:

Full Database mongodump – 4.6 GB (Gzip)

This is more useful than a generic name such as Backup or New Job. It tells me the scope, approximate source size, and whether compression is enabled.

For the source, I selected the complete gui_test_db database running locally on port 27017. Its uncompressed data size was approximately 4.6 GB, spread across four collections:

dataset_30000
large_collection_4GB
large_docs_10mb
medium_docs_1mb

The source panel displayed the database statistics before the job ran. This gave me a quick check that I had selected the full database before creating several gigabytes of output.

Back Up a Database or One Collection

The correct scope depends on why you need the backup.

Choose a *collection dump* when you only need one dataset, want to move a collection between environments, or need a smaller restore unit. Choose a *database dump* when the job should include all collections and their metadata.

Be deliberate here. A successful task using the wrong source is still the wrong backup.

Choose the Output Folder and Enable Gzip

I saved the VisuaLeaf output in its own folder and enabled *Compress with Gzip*. A separate folder for every run keeps completed backups distinct and prevents CLI files, task files, or partial attempts from being mixed.

The job summary also showed *auto threads* and *no timeout*. Auto threads leaves the worker configuration to VisuaLeaf. The gear icon opens the advanced settings if the task needs a specific thread limit or timeout. I left both settings at their defaults for this workflow.

A full gui_test_db backup configured as a binary dump task in VisuaLeaf.

After choosing the source and destination, save the job. The connection, database, folder, compression, and operation settings remain attached to the task, so the next backup does not require rebuilding the command.

Schedule and Monitor the MongoDB Backup

You can still start a saved job manually, but scheduling is the main reason to turn this workflow into a task. VisuaLeaf's Task Manager documentation lists one-time, hourly, daily, weekly, monthly, and custom cron schedules.

Choose the frequency based on how much recent data the application could afford to lose. A weekly backup is easy to manage, but it may be inadequate for a database that changes throughout the day.

Use a dated destination for recurring runs and decide how many older backups to keep. Otherwise, the task may work perfectly while the backup drive quietly runs out of space.

The full database dump is scheduled to run every day at 6:00 AM.

After a run, *Execution History* provides a detailed dump summary. In my test, it recorded:

Total duration, collections, documents, and database size.
The thread used for each collection.
Document count and size per collection.
Individual collection duration and processing speed.

The completed dump included four collections, 30,449 documents, and 4.72 GB of source data.

*Performance note:* On the same Windows machine, with Gzip enabled and the same 4.72 GB database, the VisuaLeaf task completed in 4 minutes 40 seconds. The equivalent direct mongodump CLI run took 6 minutes 12 seconds, so VisuaLeaf finished about 25% faster in this test. Results may vary depending on the hardware, storage, data, and task settings.

The equivalent direct mongodump CLI run completed in 6 minutes and 12 seconds and produced 3.1 GB of compressed output.

Before treating the exported files as a valid backup, check that the execution status is *Completed*. A cancelled or failed run may still leave partial files in the destination.

What Files Does the Backup Create?

A compressed directory dump creates one .bson.gz data file and one .metadata.json.gz file for each collection. The VisuaLeaf output for gui_test_db contained:

dataset_30000.bson.gz
dataset_30000.metadata.json.gz
large_collection_4GB.bson.gz
large_collection_4GB.metadata.json.gz
large_docs_10mb.bson.gz
large_docs_10mb.metadata.json.gz
medium_docs_1mb.bson.gz
medium_docs_1mb.metadata.json.gz

The BSON files contain the documents. The metadata files contain collection options and index definitions used during restoration. MongoDB describes this directory structure in its official [mongodump](https://www.mongodb.com/docs/database-tools/mongodump/) documentation.

The 4.6 GB test database produced approximately 3.18 GB of compressed output in VisuaLeaf. That result is specific to this data. A similarly sized database containing images, encrypted values, or already compressed content may shrink much less.

The compressed gui_test_db backup contains eight BSON and metadata files totaling 3.18 GB.

Can You Run mongodump in MongoDB Compass?

No. MongoDB Compass can export an individual collection as JSON or CSV, but MongoDB explicitly says Compass is not a backup tool. It does not create a full binary database dump containing compressed BSON files and collection metadata. For that, MongoDB directs users to its separate Database Tools. See the official Compass import and export documentation.

MongoDB Compass exports collection data as JSON or CSV; it does not create a binary database backup.

This is an important difference between exporting data and creating a backup. JSON can preserve MongoDB types when the correct Extended JSON format is used, while CSV can lose type information. Neither Compass export is the same workflow as a BSON database dump intended for mongorestore.

Restore the BSON Backup in VisuaLeaf

Creating compressed BSON files is only half of a backup test. You also need to confirm that MongoDB can restore them.

To avoid changing the original gui_test_db database, I created a separate VisuaLeaf restore job. Under *Source Configuration, I selected *Restore (BSON)** and chose the folder containing the .bson.gz and .metadata.json.gz files.

For the target, I selected the same MongoDB connection and entered a new database name: gui_test_db_restore. Because that database did not exist, VisuaLeaf indicated that it would create it when the restore job ran.

VisuaLeaf restore job configured to import a compressed BSON dump into a new MongoDB database.

This operation uses the binary dump-and-restore workflow. It reads the BSON data and collection metadata without converting the documents to JSON or CSV.

When restoring to a different deployment, check version compatibility first. MongoDB requires the source and destination to use the same major version or feature compatibility version. See the official mongodump behavior documentation.

After the job finishes, check the restored database rather than relying only on the Completed status. The restore took 7 minutes and 34 seconds and recreated all four collections with 30,449 documents:

Four collections.
30,449 documents in total.
30,000 documents in dataset_30000.
292 documents in large_collection_4GB.
57 documents in large_docs_10mb.
100 documents in medium_docs_1mb.
The expected collection indexes and options.

VisuaLeaf restored all four collections and 30,449 documents from the compressed BSON dump.

The document counts matched the source, confirming that the BSON files could be restored successfully. A production restore test should also verify indexes, collection options, and several real application queries.

Do not restore a test dump over the only copy of your original database. Use a new database name or a separate development environment.

A backup is not verified until the restore finishes successfully and the restored data matches the source.

If mongodump Does Not Run on Windows

Two small Windows problems appeared during this test.

If PowerShell says that mongodump is not recognized, MongoDB Database Tools may be installed without its bin directory being added to PATH. Using the complete path to mongodump.exe, as shown in the opening command, avoids that problem.

A connection error can also mean that the command is using the wrong host or port. One attempt used localhost:27019, while the current MongoDB instance was listening on 27017. Check the connection used by VisuaLeaf or MongoDB Compass before retrying.

Cancelling a dump may leave partial files in the destination. Delete those incomplete files or use a new output folder before starting another backup.

MongoDB Backup Limitations to Plan For

A mongodump backup still needs consistency planning, safe storage, and restore testing.

Limitation	Why it matters	What to do
Writes continue during the dump	A database-scoped dump may not represent one exact point in time.	`--oplog` cannot be combined with this `--db` command; it requires a full replica-set member dump. Use another backup method when exact point-in-time consistency is required.
Collection names differ only by capitalization	Directory dumps may conflict on case-insensitive Windows and macOS file systems.	Avoid ambiguous collection names or use archive output with the CLI.
The backup stays on the MongoDB machine	Disk failure, device loss, or ransomware could affect the database and backup together.	Keep another copy on separate or off-machine storage.
The backup is never restored	A Completed status and BSON files do not prove that the backup is usable.	Restore into a separate database and verify document counts, indexes, and collection options.
Old backups are never removed	Scheduled dumps can eventually fill the destination disk.	Define a retention policy and monitor available storage space.

MongoDB positions mongodump and mongorestore mainly for smaller deployments; larger or business-critical systems may require a different backup strategy.

Conclusion

A reliable MongoDB backup workflow does not end when files appear in a folder. It ends when those files can be restored, and the resulting database has been verified.

The command line remains useful for one-off dumps and scripts. In VisuaLeaf, you can save both export and restore jobs, schedule repeated backups, and review their status, duration, history, and logs. You still need off-machine copies, retention rules, consistency planning, and restore tests, but the recurring steps are easier to manage from one place.

PostgreSQL JSONB: Query, Update, and Index JSON Data

VisuaLeaf — Tue, 18 Aug 2026 11:20:25 +0000

JSONB allows you to store JSON data within a PostgreSQL table row. It is suitable when certain columns remain unchanged, while the remaining columns may vary across rows.

Let us take the support tickets table, where the status, priority, and the date a ticket was created cannot change. However, the client name, environment, tags, and even error details may be optional and have different formats. That is why the optional content may be stored in the JSONB field

This storage process is easy enough. However, you need answers to questions such as how to find a nested value, update a single field, or create an effective index for your database engine.

This article explores how to achieve that goal using a support_tickets table and the corresponding details column.

PostgreSQL JSONB operations used in this guide

We’ll use each of these against the same support_tickets.details column, so you can see how querying, updating, and indexing work together.

Create the support tickets table

CREATE TABLE support_tickets (
    ticket_id  BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    status     TEXT NOT NULL,
    priority   TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    details    JSONB NOT NULL
        CHECK (jsonb_typeof(details) = 'object')
);

I kept status, priority, and created_at outside JSONB because they have stable types and are useful for filtering and sorting. The less predictable ticket context goes into details.

Here is one record used in the test:

INSERT INTO support_tickets (status, priority, details)
VALUES (
    'open',
    'high',
    '{
      "customer": {
        "name": "Trevor Lisbon",
        "plan": "Professional"
      },
      "environment": {
        "browser": "Chrome",
        "os": "Windows 11",
        "appVersion": "4.8.2"
      },
      "tags": ["sync", "postgresql"],
      "error": {
        "code": "SYNC_TIMEOUT",
        "retryable": true
      }
    }'::jsonb
);

The CHECK constraint confirms that details contains a JSON object. It does not guarantee that customer.plan exists or that tags is always an array. JSONB validates the JSON format, not your complete application schema.

PostgreSQL table with regular columns and nested JSONB data displayed in VisuaLeaf.

Query nested JSONB values

PostgreSQL provides two operators that look similar but return different data types:

Use -> when you want an object or array to remain JSONB.

Use ->> when you need a scalar value for an ordinary SQL comparison.

This query returns the complete customer object and extracts the plan as text:

SELECT
    ticket_id,
    details -> 'customer' AS customer,
    details -> 'customer' ->> 'plan' AS plan
FROM support_tickets
WHERE ticket_id <= 4
ORDER BY ticket_id;

Expanding the customer JSONB object while plan is returned as text.

A common mistake is to use -> for the final value:

WHERE details -> 'customer' -> 'plan' = 'Professional'

The left side is JSONB, while Professional is being treated as a SQL string. PostgreSQL may return:

invalid input syntax for type json
Token "Professional" is invalid.

The clearer fix is:

WHERE details -> 'customer' ->> 'plan' = 'Professional'

To filter by a nested value, use ->> for the final part of the path:

SELECT
    ticket_id,
    status,
    details -> 'customer' ->> 'name' AS customer_name
FROM support_tickets
WHERE details -> 'error' ->> 'code' = 'SYNC_TIMEOUT';

Filtering tickets by a nested JSONB error code and returning the customer name as text.

If a row has no error object, PostgreSQL returns SQL NULL for that expression. It does not fail.

Search inside a JSONB array

You can search the tags array with the ? operator:

SELECT ticket_id, status, details -> 'tags' AS tags
FROM support_tickets
WHERE details -> 'tags' ? 'postgresql';

This assumes that tags contains an array of strings. If some rows store "tags": "postgresql" instead, the data is valid JSONB but has the wrong shape for this query.

Match part of a document with `@>`

The containment operator checks whether one JSONB value contains another:

SELECT ticket_id, status, details -> 'error' AS error
FROM support_tickets
WHERE details @> '{
  "error": {
    "code": "SYNC_TIMEOUT"
  }
}'::jsonb;

Using @> to find tickets containing the SYNC_TIMEOUT error object.

The structure must match the document. This does not work:

WHERE details @> '{"code": "SYNC_TIMEOUT"}'::jsonb

code is nested under error; it is not a top-level key. This matters when you add a GIN index, because containment queries can use that index directly.

Update a nested value with `jsonb_set`

Ticket 1 was reproduced in version 4.8.4. Instead of replacing the entire details document, I updated only environment.appVersion:

UPDATE support_tickets
SET details = jsonb_set(
    details,
    '{environment,appVersion}',
    to_jsonb('4.8.4'::text),
    false
)
WHERE ticket_id = 1
RETURNING
    ticket_id,
    details -> 'environment' AS environment;

The result shows appVersion as 4.8.4, while browser and os remain unchanged.

Updating environment.appVersion while preserving the other JSONB fields.

The final false means the key must already exist. The ::text cast is also important; without it, PostgreSQL may not know which type to convert:

could not determine polymorphic type because input has type unknown

When the parent object is missing

jsonb_set can create the final key, but not a missing parent object. Ticket 3 has no error object, so updating error.firstSeenAt would run without an error but change nothing.

This version creates the parent when needed:

UPDATE support_tickets
SET details = jsonb_set(
    details,
    '{error}',
    COALESCE(details -> 'error', '{}'::jsonb)
        || jsonb_build_object(
            'firstSeenAt',
            '2026-08-03T11:05:00Z'
        ),
    true
)
WHERE ticket_id = 3
RETURNING details -> 'error' AS error;

RETURNING lets you confirm that the field was actually added.

Creating the missing error object.

Add a GIN index for containment searches

For the indexing test, I added 40,000 generated tickets. The table contained 40,004 rows in total, while the original four remained available for the earlier examples.

Because the query uses @> against details, I created a containment-focused GIN index:

CREATE INDEX idx_support_tickets_details_gin
ON support_tickets
USING GIN (details jsonb_path_ops);

ANALYZE support_tickets;

jsonb_path_ops works well for containment searches, but it does not support every JSONB operator. For example, it cannot support the ? operator used in the tags query.

I then opened the same query in VisuaLeaf’s Explain view:

SELECT ticket_id
FROM support_tickets
WHERE details @> '{
  "error": {
    "code": "SYNC_TIMEOUT"
  }
}'::jsonb;

PostgreSQL found 402 matching tickets and used idx_support_tickets_details_gin through a Bitmap Index Scan.

The JSONB query uses the GIN index instead of scanning the full table.

The index worked for this query and dataset. PostgreSQL may still choose a sequential scan for a small table or when many rows match.

When to use JSONB

JSONB is useful for flexible data, but stable fields usually belong in regular columns.

In this example, environment and error fit inside JSONB. Fields such as status, priority, and created_at are better as regular columns.

I ran these examples in VisuaLeaf, using its SQL editor, nested JSONB table view, and visual query plan to inspect the results.

References

MongoDB Schema Design: Build or Reverse Engineer Visually

VisuaLeaf — Thu, 06 Aug 2026 11:45:16 +0000

MongoDB collections can become difficult to understand once they contain nested objects, arrays, references, and documents with slightly different structures.

Looking at one JSON document at a time shows you the data, but not the complete model. It is harder to see how collections connect, which fields use inconsistent BSON types, or whether a value is embedded or referenced elsewhere.

The opposite problem happens when the database does not exist yet. You may know which collections and fields you need, but reviewing the design across separate JSON files or application models is not easy.

VisuaLeaf’s Visual Schema supports both directions:

Generate a diagram from an existing MongoDB database.
Design a new schema from an empty diagram.
Keep the schema virtual or materialize it into real collections.

I tested both workflows using a travel-planning database containing collections such as travelers, places, flights, accommodations, restaurants, activities, carRentals, and vacationPlans.

VisuaLeaf generates a visual MongoDB schema showing collections, fields, BSON types, and relationships in the TravelPlannerDemo database.

Although this guide focuses on MongoDB, VisuaLeaf’s Visual Schema can also be used with SQL databases. If you’re deciding which database model fits your project or want to compare their structures, read MongoDB vs PostgreSQL: What’s the Difference?.

Reverse engineer an existing MongoDB database

If your database already contains data, you can generate a Visual Schema from its collections.

Connect to MongoDB, open *Visual Schema*, and select the database you want to analyze. VisuaLeaf reads the existing documents and creates a diagram containing:

collections and fields;
detected BSON types;
embedded objects;
arrays and their nested fields;
detected relationships between collections.

In my TravelPlannerDemo database, the generated diagram included the main travel collections and their nested structures.

For example, vacationPlans contains simple fields such as title, status, and createdAt, but it also includes more complex structures:

budget and dates as embedded objects;
itinerary, checklist, and documents as arrays;
destinationIds and travelerIds as arrays of references;
ownerTravelerId and selectedAccommodationId as individual references.

Instead of opening several documents to understand how these fields fit together, I could inspect the structure in one diagram.

This is particularly useful when you are working with a database you did not design, or when the structure has changed, but its documentation has not.

Review the detected structure

Here’s the trap most of us fall into: reverse-engineering a database shows what’s currently stored, not what was actually intended.

MongoDB allows documents in the same collection to have different fields and BSON types. For example, most documents might store a date correctly:

{
  createdAt: ISODate("2026-07-18T09:30:00Z")
}

while an older import stores it as a string:

{
  createdAt: "2026-07-18"
}

These values may look similar in a table, but one has the BSON type date and the other has the type string.

The same problem can occur with identifiers. A field such as ownerTravelerId may contain an ObjectId in most documents but a string in others. That difference matters when you try to connect it to travelers._id.

The generated schema represents the data VisuaLeaf analyzed. Rare fields may also be absent if they do not appear in the sampled documents. Review unusual records and any fields with multiple detected types before using the diagram as your final model.

Review and add relationships

MongoDB does not store relationships as enforced foreign keys. It stores the reference value, but the meaning of that value often needs to be confirmed.

In the travel database, relationships can connect fields such as:

vacationPlans.ownerTravelerId → travelers._id
vacationPlans.travelerIds[] → travelers._id
vacationPlans.destinationIds[] → places._id
vacationPlans.selectedAccommodationId → accommodations._id

Other collections use similar connections:

accommodations.placeId → places._id
restaurants.placeId → places._id
activities.placeId → places._id
flights.passengers.travelerId → travelers._id

Check both ends before accepting or adding a relationship. An ObjectId does not match a string containing the same characters.

For example, these two values use different BSON types:

ObjectId("66af9138ad73c91224345c10")

"66af9138ad73c91224345c10"

The diagram can document a relationship, but it does not make MongoDB enforce it. Your application can still insert an ownerTravelerId that does not exist in travelers.

The generated diagram may also need some visual cleanup. In a database with many related collections, lines can overlap and detected connections may not represent the relationship you intended. Reposition the collections and review each relationship rather than accepting every connection without checking.

A closer look at nested fields, BSON types, arrays, and relationships between `_vacationPlans___,__ _places___, and__ _accommodations`_ in VisuaLeaf.

Modify the generated schema

A reverse-engineered schema does not have to remain a read-only picture of the current database.

You can use it as a starting point and then:

add or remove fields;
correct BSON types;
define embedded objects and arrays;
add missing relationships;
remove incorrect relationships;
reorganize the diagram.

In this example, I added a new optional travelStyle field to vacationPlans. I also manually connected destinationIds to places._id to represent the reference between the two collections.

The same collection contains embedded data as well. The budget object belongs to one vacation plan, so storing it inside the document keeps the related values together. destinationIds, however, contains references to documents stored in the places collection.

vacationPlans
├── budget
│   ├── currency
│   ├── estimatedTotal
│   └── paidSoFar
├── dates
├── itinerary[]
├── ownerTravelerId
├── travelerIds[]
└── destinationIds[]

This shows both modeling approaches in the same collection: embedded data for information owned by the vacation plan and references for information stored elsewhere and reused.

Changing the virtual schema does not automatically migrate existing documents. If you rename a field, change its type, or move it into an embedded object, the records already stored in MongoDB remain unchanged.

The diagram describes the proposed model. Data migration remains a separate operation.

Design a new MongoDB schema from scratch

You can also use Visual Schema before the database exists.

Start with an empty diagram and add the collections you need. You can then define their fields, BSON types, embedded structures, arrays, and relationships.

For this example, I created a small tour-booking schema with two collections:

Collection	Example fields
`departures`	`_id`, `packageId`, `startDate`, `capacity`, `meetingPoint`, `status`
`reservations`	`_id`, `departureId`, `bookingReference`, `leadGuest`, `totalAmount`, `paymentStatus`, `createdAt`

The leadGuest field is an embedded object containing fields such as fullName and email. The relationship connects:

reservations.departureId → departures._id

This represents a one-to-many relationship: one departure can have multiple reservations.

Defining the schema visually makes several problems easier to notice:

references pointing to the wrong field;
incorrect BSON types;
fields that should be grouped inside an embedded object;
missing or unnecessary relationships;
inconsistent field names between collections.

After completing the design, save the schema and bind it to the MongoDB connection where you want to create the collections. The diagram remains a virtual model until you materialize it.

Keep the schema virtual or materialize it

The diagram can remain virtual if you only need it for documentation, planning, or discussing changes with your team.

Nothing is created in MongoDB until you choose to materialize it.

When the design is ready, save it and bind it to the MongoDB connection and target database. You can then click *Materialize* to review the collections and generated validators before applying the schema.

This allows both workflows:

Existing MongoDB database → Visual Schema

and:

Visual Schema → MongoDB database

You can therefore reverse engineer an existing database such as TravelPlannerDemo, modify its virtual schema, and materialize the result into a separate test database.

You can also begin with an empty diagram, as I did with TourBookingSchema, and create a new database from that design.

Choose whether to attach validators

Materializing a schema does not require you to enforce every field and BSON type.

The materialization screen lets you review the JSON Schema generated for each collection. You can then decide whether MongoDB should use it as a $jsonSchema validator.

If you want to create the collections without enforcing the schema, leave *Attach validator* disabled.

These are separate decisions:

The visual schema describes the intended structure.
Materialization creates the collections in MongoDB.
The generated JSON Schema describes the field and type rules.
The attached validator makes MongoDB enforce those rules during writes.

A field is mandatory only if it is included in the schema’s required array. Otherwise, its BSON type is checked only when the field is present.

This distinction matters when your application still writes inconsistent data.

Suppose the generated schema expects:

{
  createdAt: {
    bsonType: "date"
  }
}

If the application sends this instead:

{
  createdAt: "2026-08-03"
}

MongoDB will reject the write when the validator is attached. A document validation failure commonly returns MongoDB error code 121.

The solution is not necessarily to remove validation. First check what your application actually sends. You may need to fix the value before inserting it or adjust the schema if both types are intentionally supported.

Review the materialization plan

Before clicking *Apply to Database*, check:

the selected connection and database;
which collections will be created;
which collections already exist;
collection and field names;
BSON types;
embedded objects and arrays;
relationships in the visual diagram;
generated JSON Schemas;
which collections have the *Attach validator* enabled.

For the first test, use a new database instead of applying the model directly to an existing one.

For example:

TourBookingDemo

This makes the result easier to inspect and avoids mixing the materialized collections with the original TravelPlannerDemo data.

If you materialize changes against a database that already contains records, changing reservations.departureId from String to ObjectId in the diagram will not convert the stored values. Data migration remains a separate operation.

It also does not create enforced foreign keys. The relationship between reservations.departureId and departures._id remains part of the model, but your application must maintain valid references.

Verify the result

After applying the materialization plan, open the target database and confirm that the departures and reservations collections were created.

If you enabled *Attach validator*, inspect the validation settings and test them with:

one valid document that should be accepted;
one invalid document that should be rejected.

If you left the option disabled, the collections should still exist, but MongoDB will continue accepting documents with different structures.

Also remember that a validator does not replace indexes. Fields used for lookups and filtering may still need indexes, such as:

db.reservations.createIndex({
  departureId: 1,
  createdAt: 1
})

db.departures.createIndex({
  startDate: 1,
  status: 1
})

Validators control document structure. Indexes support query performance and can enforce uniqueness when created with the unique option.

Relationships explain how collections connect, but MongoDB does not enforce them as foreign keys.

One schema, two directions

Visual Schema is useful when you inherit an existing MongoDB database and need to understand its structure. It is also useful before you create the first collection.

In my travel-planning example, I could generate a diagram containing travelers, places, flights, accommodations, activities, vacationPlans, and the other related collections. I could inspect nested structures such as budget and itinerary, review the references, and correct the model where necessary.

The same workspace also supports the opposite workflow. You can start with an empty diagram, design the collections visually, and then decide whether to keep the model virtual or materialize it into MongoDB.

When you materialize it, attaching the generated JSON Schema validators remains optional.

*Want to turn your MongoDB database into a diagram—or your diagram into a database?*

Download VisuaLeaf and generate a schema from your existing MongoDB database, or design a new one from scratch.

Which MongoDB Client Handles Large Collections Best: VisuaLeaf vs Compass vs Studio 3T

VisuaLeaf — Tue, 04 Aug 2026 07:56:19 +0000

A MongoDB client can feel fast with a few hundred documents and become frustrating as soon as the result grows.

The database is not always the problem. MongoDB may finish the query while the desktop client is still fetching batches, converting BSON values, and building thousands of rows or JSON blocks.

I tested VisuaLeaf, MongoDB Compass, and Studio 3T with the same collection of 50,844 documents. Then I tested three related workflows: comparing collections, automating repeated work, and investigating an unindexed query.

Test setup

I ran all three applications on the same HP laptop with 16 GB of RAM and Windows 11 Pro. Each application was updated to the latest version available when I ran the test. All three connected to the same MongoDB database through the same internet connection and opened the same largeCollection. The collection contained nested objects and arrays, not only flat fields.

For the first test, I opened the collection without a filter, projection, or custom limit:

db.largeCollection.find({})

Here is what happened:

Test 1: What the large-result test actually shows

The three clients made different choices.

VisuaLeaf allowed one continuous result. That is useful when you need to scroll through a large set, select many documents, inspect distant rows, or export the current result without moving through pages.

VisuaLeaf loaded 50,000 documents in one result in 13.481 seconds.

Compass limited the amount of data visible at once. This protects the interface from an unexpectedly large query and works well when you only need to inspect a sample, edit one document, or confirm a filter. The tradeoff is that browsing the whole collection requires hundreds of pages.

MongoDB Compass displayed the collection in pages of up to 100 documents.

Studio 3T did not finish this particular operation before I stopped it. That does not prove that every version, display mode, or computer will behave the same way. It does mean that an unrestricted result was not practical in this setup.

Studio 3T became unresponsive while attempting to load 50,000 documents.

If a client struggles, reduce both the number of documents and the number of fields:

db.largeCollection.find(
  {},
  {
    _id: 1,
    company: 1,
    registered: 1
  }
).limit(1000)

This is usually the better daily workflow. A large result may be useful for inspection or export, but it still consumes database resources, network bandwidth, memory, and rendering time.

Test 2: Comparing two MongoDB collections

For this test, I used two controlled collections in the QA_Edit database: orders_compare_source and orders_compare_target.

Both collections started with 1,000 identical documents. I then modified 100 documents, removed 50, and added 50 in the target collection. I matched the documents by _id.

VisuaLeaf and Studio 3T both found 850 identical documents, 100 modified documents, 50 missing from the source, and 50 missing from the target.

A useful comparison must answer three questions:

Which documents exist on only one side?
Which fields changed inside matching documents?
What will be inserted, updated, or deleted during synchronization?

VisuaLeaf

VisuaLeaf has a dedicated Collection Compare workflow for schema and data differences. It keeps the comparison next to the document, query, and data-operation tools.

The important setup choice is the matching field. _id works only if both collections preserve the same values. If an import regenerated them, use a stable key such as customerId, orderId, or externalId.

VisuaLeaf found 850 identical, 100 modified, and 50 documents missing from each collection.

Studio 3T

Studio 3T provides the most detailed comparison controls of the three. Its Data Compare & Sync supports collections on different databases or servers, document- and field-level differences, result export, and synchronization in either direction.

Bi-directional synchronization is useful, but it increases the cost of a wrong choice. Decide which collection is the source of truth before applying changes.

Studio 3T returned the same differences using _id as the matching field.

MongoDB Compass

Compass does not provide the same dedicated collection-diff workflow. You can still build part of the comparison with an aggregation.

For example, this pipeline finds source documents without a matching _id in the target:

db.orders_compare_source.aggregate([
  {
    $lookup: {
      from: "orders_compare_target",
      localField: "_id",
      foreignField: "_id",
      as: "target"
    }
  },
  {
    $match: {
      target: {
        $eq: []
      }
    }
  },
  {
  $count: "missingInTarget"
  }
  }
])

This finds missing matches, not every field-level change. It also assumes both collections are in the same database. Comparing data on different servers requires a script, a temporary copy, or another tool.

Compass used an aggregation pipeline to find 50 source documents missing from the target.

Test 3: Automating Daily Orders

Large collections make small mistakes more expensive.

For this test, I used a daily orders JSON file.

In Studio 3T, I imported the file into a daily_orders collection. Studio 3T can also save and schedule tasks, so it works well when you want to repeat imports, exports, or other database jobs.

JSON import task: import a file into a collection

In VisuaLeaf, I used Task Manager to connect more than one step:

Import Daily Orders JSON → Add Orders to Main Collection → Export Sales Report

VisuaLeaf lets you connect tasks into a workflow, not just run one import.

The useful part is that tasks can have parent tasks. So the next task runs after the previous one is completed.

This is helpful when the work is not just “import this file.” In a real workflow, you may need to import data, map fields, move it into the main collection, mask sensitive data, and export a report after that.

For example, field mapping matters because orderedAt should be mapped to createdAt as a MongoDB Date, and values like email or phone numbers may need data masking before export.

So for me, Studio 3T is strong for saving and scheduling database tasks. VisuaLeaf is easier when I want to see a connected workflow in one place and control the order of the steps.

Compass is different. It is great for interactive work, but recurring workflows usually need something outside the desktop app, like a script, cron, Windows Task Scheduler, CI, or an Atlas service.

Test 4: Checking the Query Before Blaming the Client

The first test measured how each tool handled a large result on screen.

This test is different. Here, I wanted to check the work MongoDB was doing before the data even reached the client.

I started with a query that had no supporting index:

db.largeCollection.find({
  company: "ZENSOR"
})

The important part was the explain plan.

I checked:

scan stage in the winning plan: COLLSCAN or IXSCAN
totalDocsExamined
totalKeysExamined
nReturned
execution time

When the query uses COLLSCAN, MongoDB has to scan the collection to find matching documents. That means the slow part is not always the GUI. Sometimes the database is simply doing too much work.

A COLLSCAN is not automatically a problem. But in this case, scanning 50,844 documents to return only 51 showed that the query could benefit from an index.

VisuaLeaf detected a COLLSCAN, examining 50,844 documents to return 51.

Studio 3T reported a collection scan across 50,844 documents for 51 results.

Compass used a COLLSCAN, examining all 50,844 documents to find 51 matches.

After that, I created an index:

db.largeCollection.createIndex({
  company: 1
})

Then I ran the same explain operation again.

A lower execution time is useful, but the stronger evidence is that the winning plan now contains an IXSCAN. In this test, totalDocsExamined dropped from 50,844 to 51, matching the 51 returned documents.

VisuaLeaf’s Query Profiler is useful here because it brings slow queries, execution plans, index usage, scanned documents, returned documents, and optimization hints into one place.

After adding the company_1 index, VisuaLeaf examined 51 documents and 51 index keys.

Studio 3T has the deepest performance tools in this comparison, with Query Profiler, Visual Explain, and Index Manager.

With the company_1 index, Studio 3T used an index scan and examined 51 documents.

Compass also has Explain Plan, which works well when you already know which query you want to inspect.

After indexing company, Compass changed to IXSCAN and examined only 51 documents.

For this test, the main point was simple: before blaming the client, check if MongoDB is scanning too much data.

Which client fits which workflow?

If you mainly need to…	Best fit	Why
Open and work with a large continuous result	VisuaLeaf	It loaded 50,000 documents in 13.481 seconds and remained usable
Compare two collections visually	VisuaLeaf	It clearly separated identical, modified, and missing documents
Automate multi-step data workflows	VisuaLeaf	Import, copy, export, and scheduling can be managed in one workflow
Find inefficient queries in the same workspace	VisuaLeaf	Query Profiler combines plans, index usage, scanned documents, and optimization hints
Browse documents in small pages with a free official client	MongoDB Compass	Pagination keeps browsing controlled and predictable
Access advanced synchronization and administration controls	Studio 3T	It provides more granular diff, sync, and performance options

There is no single client for every MongoDB job. Compass is a good option for controlled, paginated browsing, while Studio 3T offers advanced administration and synchronization controls. But for everyday work with large results, collection comparisons, automated tasks, and query analysis, VisuaLeaf provides the most complete workflow in one place.

Choose the client around the operation that costs you time, not the longest feature list.

How to Copy and Sync a MongoDB Collection to PostgreSQL

VisuaLeaf — Tue, 28 Jul 2026 07:46:26 +0000

I tested a MongoDB-to-PostgreSQL sync with a visits collection.

At first, I thought the hard part would be the sync itself. It was not. The harder part was making PostgreSQL understand the MongoDB document shape.

MongoDB lets one document contain strings, dates, nested objects, arrays, and an _id. PostgreSQL wants fixed columns, clear data types, and a primary key.

So before running the sync, I had to decide what the PostgreSQL table should look like.

For this test, I used VisuaLeaf Mongo Sync.

I used it because I wanted to set up the workflow visually: source collection, target PostgreSQL table, field mapping, and sync status. I still had to create the PostgreSQL table and check the mapping carefully, but I did not have to write a custom sync script.

Source Collection

The source was:

database_compare_demo.visits

Each MongoDB document had simple fields like:

_id
visitId
status
visitDate
visitReason

It also had nested fields:

patient
doctor
symptoms
prescriptions
invoice
labResults
visitDetails
vitals
clinic

MongoDB visits collection with nested patient, doctor, lab results, and vitals data.

This is where the mapping matters. If you try to force every nested MongoDB field into separate SQL columns too early, the setup gets messy fast.

For this test, I used a PostgreSQL landing table.

PostgreSQL Table

I created one table:

CREATE TABLE public.mongo_visits_sync (
    mongo_id TEXT PRIMARY KEY,
    "visit_id" TEXT,
    status TEXT,
    "visit_date" TIMESTAMPTZ,
    "visit_reason" TEXT,

    patient JSONB,
    doctor JSONB,
    symptoms JSONB,
    prescriptions JSONB,
    invoice JSONB,
    "lab_results" JSONB,
    "visit_details" JSONB,
    vitals JSONB,
    clinic JSONB
);

I kept the simple fields as normal columns.

I stored the nested MongoDB objects and arrays as JSONB.

This was easier to test, and it kept the original document structure instead of flattening everything too early.

Why `mongo_id` Is Important

The PostgreSQL table needs a primary key.

In this setup, MongoDB _id maps to PostgreSQL mongo_id.

_id -> mongo_id

Without this, the sync cannot safely update existing rows.

One error I hit was:

SQL target has no key columns — cannot upsert idempotently

That means PostgreSQL did not have a usable key for the sync job.

The fix was to create mongo_id as the primary key and map MongoDB _id to it.

Field Mapping and Data Types

In VisuaLeaf, the mapping shows the MongoDB fields with simple source types like String, Object, and Array.

That matters because not every MongoDB field should become a normal text column in PostgreSQL.

Creating a new transformation mapping before generating the MongoDB to PostgreSQL fields.

For this test, I used this logic:

Field mapping from MongoDB fields to PostgreSQL columns, including objects and arrays.

The simple string fields went into normal PostgreSQL text columns.

The objects and arrays went into JSONB columns.

This was the important part. If I mapped an object or array into a normal text column, the sync would either fail or store the data in a way that is harder to query later.

For example:

patient is an object, so I mapped it to a JSONB column.

symptoms is an array, so I also mapped it to JSONB.

The main key was:

_id -> mongo_id

That gave PostgreSQL a stable primary key for updates and deletes during sync.

The Error I Had With Column Names

At first, my PostgreSQL table used names like:

lab_results
visit_details
visit_reason

But the sync tried to insert into:

labResults
visitDetails
visitReason

So PostgreSQL returned this error:

ERROR: column "labResults" of relation "mongo_visits_sync" does not exist

The sync failed because the MongoDB field labResults did not match the PostgreSQL column lab_results.

The problem was not PostgreSQL. The problem was the mismatch between the mapping and the real column names.

You can fix this in two ways:

Keep snake_case in PostgreSQL and map each field manually.
Use quoted camelCase column names in PostgreSQL.

Sync Setup

The source MongoDB database was a replica set.

That matters because continuous sync uses MongoDB change streams. Change streams work with replica sets and sharded clusters, not standalone MongoDB.

The setup was:

Mapping the MongoDB visits collection to the PostgreSQL target table.

I used Full sync because I wanted the first copy and the later changes.

Final review before saving the MongoDB to PostgreSQL sync job.

Checking the Initial Copy

After running the sync, I checked the row count:

SELECT COUNT(*) AS synced_rows
FROM public.mongo_visits_sync;

Then I checked a few fields:

SELECT
    "visit_id",
    status,
    "visit_date",
    patient ->> 'fullName' AS patient_name,
    doctor ->> 'fullName' AS doctor_name
FROM public.mongo_visits_sync;

Synced MongoDB visit documents shown in the PostgreSQL target table.

This confirmed that normal fields were copied into columns, and nested values were still queryable from JSONB.

Testing Insert, Update, and Delete

The first copy is not enough to prove the sync works.

So I tested changes from MongoDB too.

Live sync monitor showing inserts, updates, deletes, and connection status.

For the update test, I used:

db.visits.updateOne(
  { visitId: "VIS-SYNC-TEST" },
  {
    $set: {
      status: "completed",
      "visitDetails.notes": "Updated after initial sync"
    }
  }
);

Then I checked PostgreSQL:

SELECT
    "visit_id",
    status,
    "visit_details" ->> 'notes' AS notes
FROM public.mongo_visits_sync
WHERE "visit_id" = 'VIS-SYNC-TEST';

The updated status and note appeared in PostgreSQL.

I also tested insert and delete. The new document appeared in PostgreSQL, and the deleted document was removed from the table.

What I Would Watch For

The sync works, but the setup is not automatic magic.

You still need to be careful with:

primary keys
field mapping
camelCase vs snake_case column names
nested objects
arrays
JSONB fields
unmapped MongoDB fields

For a first version, one PostgreSQL landing table is usually easier than trying to normalize everything.

After the data is synced, you can decide if some fields should move into separate SQL tables later.

Conclusion

This kind of sync is useful when MongoDB is where the app data lives, but PostgreSQL is better for reporting, SQL queries, dashboards, or sharing data with teams that work mostly in relational databases.

The goal is not only to copy the collection once. The useful part is keeping PostgreSQL updated when documents are inserted, updated, or deleted in MongoDB.

For this test, the cleanest setup was to store simple fields as normal PostgreSQL columns and keep nested objects and arrays as JSONB.

Once the primary key, column names, data types, and field mapping were correct, VisuaLeaf Mongo Sync could handle the copy and the later changes without writing a custom sync script.

Want to Sync MongoDB to Atlas Instead?

If you want to copy a local MongoDB collection to Atlas and keep it in sync, I covered the full process in this detailed guide: How to Copy a Local MongoDB Collection to Atlas—and Keep It in Sync.

How to Import JSON into MongoDB and Export to CSV with Data Masking

VisuaLeaf — Thu, 23 Jul 2026 09:20:23 +0000

Every morning, an online store receives the previous day’s orders from a marketplace partner.

The file comes in JSON format. The company needs to add those orders to its main MongoDB orders collection. The sales manager also needs a CSV report that can be opened in Excel.

That sounds like a small task. Import the file, copy the documents, export the report.

But in practice, a few things can break the process.

A date can be imported as a string. A field can have the wrong name. One batch may use total, while the main collection uses totalAmount. A temporary collection can keep old records and trigger duplicate key errors. A CSV export can create null values because the mapping points to fields that do not exist.

And then there is customer data. The manager may need the sales numbers, but they probably do not need real customer names or internal customer IDs.

This article walks through a real daily workflow:

Import marketplace JSON
        ↓
Store the batch in a temporary MongoDB collection
        ↓
Copy the orders into the main orders collection
        ↓
Mask customer fields during export
        ↓
Create a CSV report

The goal is not just to move data from JSON to CSV. The goal is to make the process repeatable, easier to check, and safer to share.

The workflow

The workflow has three jobs:

Import Yesterday Orders
        ↓
Add Orders to Main
        ↓
Export Daily Sales Report

The important part is the parent relationship between the jobs.

Add Orders to Main depends on Import Yesterday Orders, so it only runs after the JSON file is imported successfully.

Export Daily Sales Report depends on Add Orders to Main, so the CSV is created only after the main orders collection has been updated.

This prevents the report from being generated when data is missing or incomplete.

The incoming JSON file

The partner sends a file with yesterday’s completed orders.

A single order looks like this:

{
  "orderId": "ORD-2026-07-201",
  "customerId": "CUST-1003",
  "customerName": "Sofia Rossi",
  "orderDate": "2026-07-21T08:20:00Z",
  "status": "completed",
  "paymentStatus": "paid",
  "currency": "EUR",
  "items": [
    {
      "sku": "EL-002",
      "productName": "Wireless Mouse",
      "category": "Electronics",
      "quantity": 1,
      "unitPrice": 24.99,
      "lineTotal": 24.99
    },
    {
      "sku": "EL-003",
      "productName": "Mechanical Keyboard",
      "category": "Electronics",
      "quantity": 1,
      "unitPrice": 79.99,
      "lineTotal": 79.99
    }
  ],
  "itemsSummary": "1x Wireless Mouse, 1x Mechanical Keyboard",
  "itemCount": 2,
  "subtotal": 104.98,
  "discount": 10,
  "shippingFee": 4.99,
  "totalAmount": 99.97,
  "couponCode": "WELCOME10"
}

There are two fields worth pointing out here.

The first is items. This is the real order structure. It keeps each product as a nested object with its own SKU, quantity, price, and line total.

The second is itemsSummary. This is not as rich as the items array, but it works better in a CSV report. Instead of putting a full JSON array into one spreadsheet cell, the manager sees a readable value:

1x Wireless Mouse, 1x Mechanical Keyboard

MongoDB handles nested arrays well. CSV does not. So for the database, keep the array. For the report, export the summary.

The first job imports the file into a temporary collection:

online_store.daily_sales_import

Why use a temporary collection?

The first job imports the JSON file into online_store.daily_sales_import instead of writing directly to orders.

This extra step is worth it.

The file comes from another system. Even if the partner usually sends the correct structure, a single change can break your report. A field can be renamed. A value can arrive as text instead of a number. A date can be formatted differently. Or the import job can use an old field mapping from another file.

The temporary collection gives you a place to check the batch before it becomes part of the main order history.

If it keeps old documents, the next run may copy the same orders again. That can cause duplicate key errors later.

Generate field mappings every time the JSON structure changes

This is the easiest step to skip, and it is also where a lot of bad imports start.

When you select the JSON file, click:

Generate Field Mappings

For this order file, the mapping should include:

_id           → _id
orderId       → orderId
customerId    → customerId
customerName  → customerName
orderDate     → orderDate
status        → status
paymentStatus → paymentStatus
currency      → currency
items         → items
itemsSummary  → itemsSummary
itemCount     → itemCount
subtotal      → subtotal
discount      → discount
shippingFee   → shippingFee
totalAmount   → totalAmount
couponCode    → couponCode

If you do not regenerate the mappings, the job may reuse fields from an older import.

For example, if the previous import was a customer file, the mapping may still expect fields like:

country
email
joinedAt
name
customerId
status

The import may still run. But the order fields will be missing, and the result will look wrong. You may see null values or documents that only contain a few shared fields like customerId and status.

That is not a MongoDB issue. It is a mapping issue.

Copy the batch into the main orders collection

The second job copies documents from:

daily_sales_import

into:

orders

This job should use Import Yesterday Orders as its parent.

A simple configuration is:

Source: online_store.daily_sales_import
Target: online_store.orders
Mode: Insert or Append
Parent: Import Yesterday Orders

Use insert or append because you are adding new orders to the order history.

Do not use replace mode on orders unless you really want to overwrite the full collection.

The main collection should keep all orders. The temporary collection should only hold the latest imported file.

Duplicate key errors are useful, but they still need a fix

If you run the same batch twice, MongoDB may return this error:

E11000 duplicate key error

That means MongoDB blocked a duplicate value for a unique field, often _id or orderId.

This is usually good. It stops the same order from being inserted twice.

But in a daily workflow, the error also tells you something is wrong with the process.

Common causes:

The temporary collection was not cleared.
The same JSON file was imported twice.
The partner sent duplicate order IDs.
The copy job tried to insert records already present in orders.

In testing, the temporary collection is often the problem.

If daily_sales_import contains old records and new records, the copy job tries to insert all of them. MongoDB accepts the new ones and rejects the duplicates.

The fix is simple: clear or replace the temporary collection before each new import.

Export the weekly report from the main collection

The company imports marketplace orders every day, so the main MongoDB orders collection stays up to date.

But the manager does not need a CSV file every morning. In this case, the manager needs a weekly sales report.

That changes the source of the export job.

If you export from daily_sales_import, you export only the latest imported batch.

If you export from orders, you export the main order history, including the orders that were imported during the week.

For this workflow, the export job uses:

Source: online_store.orders
Output: weekly-sales-report-{{yyyy-MM-dd}}.csv
Parent: Add Orders to Main

The parent relationship still matters. The weekly CSV should be created only after the latest imported orders have been added to the main orders collection.

This gives the manager one report with the updated sales data, instead of sending a separate file every day.

Mask customer data during export

The manager needs sales data, but they do not need the real customer name or internal customer ID.

Instead of changing the original MongoDB documents, the export job applies masking only in the CSV output.

In this workflow:

customerId   → hash(value)
customerName → maskName(value)

This keeps the original data intact in MongoDB, while the exported report hides sensitive customer details.

What maskName(value) does

The maskName(value) transformation keeps the first character and replaces the rest with asterisks.

Example:

Alex Ionescu → A***********
Mia Thompson → M***********

It does not create a fake name.

It masks the original value. The report still has a customer-name column, but the real name is hidden.

This is useful when the report needs to show that a customer exists, but not who the customer is.

What hash(value) does

The hash(value) transformation changes the customer ID into a hashed value.

Example:

CUST-1002 → 043a5f9b...

The same input should produce the same hashed output.

That means the report can still show that two orders belong to the same customer without exposing the original customer ID.

Example:

Original customerId: CUST-1002
Masked customerId:   043a5f9b...

Original customerId: CUST-1002
Masked customerId:   043a5f9b...

This is useful for analysis.

The manager can group orders by the masked ID, but they cannot see the real internal ID.

Schedule the workflow

Once each job works on its own, schedule the full workflow.

The final process is:

Receive marketplace JSON
        ↓
Import the daily batch
        ↓
Add orders to MongoDB
        ↓
Mask customer details
        ↓
Export the CSV report

Set the schedule to run every morning at 09:00.

The parent relationships keep the jobs in order.

The report does not run before the copy job. The copy job does not run before the import job.

That is the main reason to use a workflow instead of three separate jobs.

What this workflow actually solves

This workflow is not only about converting JSON to CSV.

It solves a real process that many teams deal with: marketplace data comes in one format, MongoDB stores the operational data, and the manager needs a spreadsheet they can review without seeing unnecessary customer details.

In this setup, the daily JSON file is imported into daily_sales_import. Then the new orders are copied into the main orders collection. At the end, the weekly CSV report is created from the updated orders collection.

Before the CSV is exported, sensitive customer fields are masked:

customerName → A***********
customerId   → 043a5f9b...

The original MongoDB documents stay complete. Only the exported report hides customer details.

This makes the process repeatable, easier to check, and safer to share.

Final result

At the end, you have one scheduled workflow that handles the daily marketplace file and keeps MongoDB up to date.

You avoid repeating the same import and export steps by hand. You reduce the chance of copying old batches again. You keep the report readable with fields like itemsSummary. And you share the sales data without exposing the original customer identities.

The small details still matter: generate the mappings when the JSON structure changes, keep field names consistent, store dates as DATE_TIME, and preview the export before trusting the CSV.

Once those pieces are set, the workflow is easier to run, easier to check, and safer to share.

I built this example in VisuaLeaf using Task Manager and export transformations.

If you work with MongoDB imports, scheduled exports, or masked reports, you can try it here: https://visualeaf.com/download

How to Copy and Sync a Local MongoDB Collection to Atlas

VisuaLeaf — Mon, 20 Jul 2026 14:04:46 +0000

Moving data from a local MongoDB database to Atlas usually means exporting files, importing them again, and repeating the same process when something changes.

I wanted to test a simpler way in VisuaLeaf: copy a local collection to MongoDB Atlas and keep the Atlas collection updated automatically.

For this example, I used a local visits collection containing seven documents. The documents included ordinary fields, nested objects, arrays, dates, and ObjectIds, so I could also check whether the original MongoDB structure remained intact in Atlas.

Here is the complete process, followed by an insert, update, and delete test.

What I used

My source and target were:

Role	Connection	Database	Collection
Source	`Local Replica Set`	`database_compare_demo`	`visits`
Target	`MongoDB Atlas`	`database_compare_demo`	`visits_atlas_sync`

I used a different target collection name so I could verify the copied data without mixing it with anything already stored in Atlas.

The local source was a replica set because continuous synchronization relies on MongoDB change streams. Change streams are available for replica sets and sharded clusters, not standalone MongoDB deployments. (MongoDB change streams)

If the source is standalone, the job can still perform an initial copy, but it cannot continue watching for later changes.

The local visits collection before it was copied to Atlas.

Preparing the Atlas connection

Before creating the sync job, I made sure VisuaLeaf could write to my Atlas cluster.

Before using a standard Atlas connection, I configured two access items:

A database user with permission to access the target database.
My computer’s public IP address added to the project’s IP access list.

Atlas database users are separate from the account used to sign in to the Atlas website. For this test, I used a database user with read and write access to database_compare_demo.

I added only my current IP address to the Atlas access list instead of allowing connections from every IP. Atlas only accepts standard client connections from addresses included in that list. (MongoDB Atlas connection prerequisites)

From the Atlas cluster’s *Connect* window, I copied the driver connection string. Its general format was:

mongodb+srv://<username>:<password>@<cluster-host>/

I used that URI to create a connection in VisuaLeaf and named it MongoDB Atlas. I tested the connection before continuing.

Creating the sync job

I opened *Mongo Sync* and created a new job with these details:

Setting	Value
Job name	`Local Visits to Atlas`
Description	`Copy local clinic visits to Atlas and keep later changes synchronized.`
Source connection	`Local Replica Set`
Source database	`database_compare_demo`

The source connection showed full change-stream support, which meant I could select Full sync later.

The local replica set and database_compare_demo database selected as the source.

Mapping the local collection to Atlas

In the Rules step, I added the Atlas connection as the target and configured one rule:

Setting	Value
Target connection	`MongoDB Atlas`
Target database	`database_compare_demo`
Source collection	`visits`
Target collection	`visits_atlas_sync`
Filter	Empty
Transformation	`None (pass-through)`

I left the filter empty because I wanted all seven documents.

I also used *None (pass-through)* instead of generating field mappings. Both sides were MongoDB, so I wanted to preserve each document exactly as it was, including its _id, nested objects, arrays, dates, and optional fields.

Field mappings are useful when renaming fields, changing types, masking values, or sending data into a different schema. They were unnecessary for this direct MongoDB-to-MongoDB copy.

The visits collection mapped to visits_atlas_sync with pass-through documents.

Choosing Full sync

The Options step offered three modes:

Mode	What it does	When to use it
Initial Only	Copies the documents that currently exist, then stops	One-time collection copy
Incremental Only	Watches only for changes made after the job starts	The target is already populated
Full sync	Copies existing documents and then watches for later changes	Initial copy plus continuous synchronization

I selected *Full sync* and enabled Start sync immediately after creation.

This mode first copies the seven documents already stored locally. After the initial phase finishes, it continues listening for new inserts, updates, and deletes.

Full sync copies existing documents and continues watching the local collection.

Reviewing and launching the job

The Review screen showed:

Local Replica Set / database_compare_demo as the source;
MongoDB Atlas / database_compare_demo as the target;
one rule from visits to visits_atlas_sync;
no filter or transformation;
Full sync with automatic start.

It also estimated an initial copy of approximately seven documents.

The complete local-to-Atlas configuration before launching the job.

I clicked *Create & launch* and waited for the initial phase to finish. Once the monitor showed that the job was running and waiting for changes, I checked the Atlas collection directly.

The initial copy is complete and the job is monitoring later changes.

Verifying the initial copy in Atlas

I opened database_compare_demo.visits_atlas_sync on the Atlas connection and counted its documents:

db.visits_atlas_sync.countDocuments({})

The result was 7, matching the local source collection.

I also opened several documents and checked their BSON types. The ObjectIds, dates, nested patient and doctor objects, and arrays were preserved because the job used pass-through documents rather than converting them into a new schema.

The initial count proved that the collection was copied. It did not yet prove that the Full sync job would handle later changes, so I tested each main operation separately.

Test 1: Inserting a new local document

I inserted one clearly labeled test visit into the local source:

Insert a new document into the local connection.

Then I searched for the same visitId in Atlas:

db.visits_atlas_sync.findOne({
  visitId: "VIS-ATLAS-TEST"
})

A new local visit appears automatically in the Atlas collection.

The new document appeared in the target collection without restarting or recreating the job. The target count increased from seven to eight.

Test 2: Updating the local document

Next, I updated both a top-level field and a nested field:

db.visits.updateOne(
  { visitId: "VIS-ATLAS-TEST" },
  {
    $set: {
      status: "completed",
      "doctor.roomNumber": "204",
      updatedAt: new Date()
    }
  }
)

I checked the Atlas document again:

Top-level and nested changes from the local document are visible in Atlas.

Test 3: Deleting the local document

Finally, I deleted the test document from the local collection:

db.visits.deleteOne({
  visitId: "VIS-ATLAS-TEST"
})

I verified the target one more time:

db.visits_atlas_sync.countDocuments({
  visitId: "VIS-ATLAS-TEST"
})

Deleting the local test document removes its synchronized Atlas copy.

The result returned to 0, and the full Atlas collection returned to seven documents.

When this workflow is useful

This type of Mongo Sync job makes sense when I want to:

copy selected local collections to an Atlas development or staging environment;
keep a cloud copy updated while continuing to write locally;
move a collection without repeatedly exporting and importing JSON files;
test an application against Atlas using recent local data;
synchronize only the collections needed for a specific workflow.

The important word is *selected*. This was a collection-level sync, not a complete production migration.

Live MongoDB sync from local replica set to Atlas.

How this differs from other Atlas migration methods

MongoDB provides other tools for different jobs:

mongodump and mongorestore are useful for a self-managed, one-time transfer using a BSON dump.
Atlas Live Migration is designed for moving a deployment into Atlas and managing a production cutover with minimal downtime.
This Mongo Sync workflow focuses on copying selected collections and continuing to apply their changes.

Atlas recommends its Live Migration workflow when performing a supported production migration. It uses MongoDB’s mongosync technology underneath and includes a cutover process for redirecting applications to the Atlas destination. (Atlas Live Migration)

I would not present a collection-level sync as a replacement for that process.

What I would still verify before using it in production

This small test verified document copying and three change types. It did not test:

high write volume or very large collections;
long network interruptions and recovery;
schema changes while the job was running;
independent writes made to both source and target;
users, roles, indexes, validation rules, or other database metadata;
a full application cutover from local MongoDB to Atlas.

I treated the local collection as the source of truth and used an empty target collection. That kept the test easy to understand and avoided conflicts with existing Atlas documents.

Final takeaway

The first copy was easy to check: 7 local documents appeared in Atlas.

But the more important part was what happened after that. When I inserted, updated, and deleted a local document, the same changes appeared in Atlas without exporting another file or recreating the job.

Because both sides were MongoDB, I could use pass-through mode and keep the original document structure.

For development, staging, testing, or selected collection sync, this is a practical way to move local MongoDB data to Atlas and keep it updated. For a full production migration, I would still follow MongoDB’s official migration process and plan the cutover separately.

Want to try it?

Download for Free and test Mongo Sync with your own local MongoDB database and Atlas cluster.

References

7 MongoDB Query Mistakes That Return the Wrong Results

VisuaLeaf — Tue, 14 Jul 2026 09:39:05 +0000

MongoDB queries look simple. You type a field, give it a value, hit run, and you get your data back.

But just because a query runs without throwing an error doesn't mean it worked right. Sometimes you get a blank screen. Sometimes you get way too many records. Other times, the data looks fine at first glance, but it doesn't actually match what you asked for.

Most of these slip-ups happen for one basic reason: the query structure doesn't match the way the data actually sits in the database.

To show you what we mean, we’ll use a clinic database with a collection called visits. Here is what a typical document looks like:

JSON

{
  "_id": "6871b6f9c3f1d1a4c2a10001",
  "status": "completed",
  "visitDate": "2026-07-01T09:30:00.000Z",
  "patient": { "name": "Anna Keller", "age": 34 },
  "doctor": { "name": "Dr. James Carter", "specialty": "Cardiology" },
  "symptoms": ["cough", "fever"],
  "prescriptions": [
    { "name": "Ibuprofen", "active": false },
    { "name": "Paracetamol", "active": true }
  ],
  "invoice": { "paid": true, "method": "card", "total": 250 }
}

You can run these examples right in the VisuaLeaf MongoDB Shell. Using visual tools makes a big difference because you can see exactly what MongoDB is returning in real time.

1. Forgetting the Curly Braces

This is just a quick typo, but it breaks things right away.

The Mistake:

db.visits.find(status: "completed")

The Correct Query

The find() tool always expects an object. Even if you are only looking for one specific thing, you still need to wrap that condition in curly braces {}.

2. Treating `$or` Like a Regular Object

This one trips a lot of people up because the broken version looks like it should work.

The Mistake:

db.visits.find({
  $or: {
    status: "completed",
    "invoice.paid": false
  }
})

What is wrong:

$or expects an array of conditions, but this query gives it one object.

The error will usually be something like:

MongoServerError: $or must be an array

The Correct Query

The first query is wrong because $or needs an array, not one regular object.

Each condition has to be written as its own object inside [].

The corrected query returns visits where the status is pending or completed.

3. Querying Nested Fields Like They Are Flat

MongoDB documents often have fields inside other fields.

In our visits collection, doctor is an object:

doctor: {
  name: "Dr. Michael Brown",
  specialization: "Neurology"
}

Query to avoid:

db.visits.find({
  doctor: "Neurology"
})

This asks MongoDB to find a doctor field that is exactly "Neurology".

But doctor is not a string. It is an object.

Better query:

db.visits.find({
  "doctor.specialization": "Neurology"
})

The dot tells MongoDB to go inside the doctor object and check the specialization field.

So instead of asking:

Is the whole doctor field equal to Neurology?

you are asking:

Is the doctor’s specialization Neurology?

4. Typing the Same Field Twice

Say you want to find visits with doctors from Cardiology or Dermatology.

Query to avoid:

db.visits.find({
  "doctor.specialization": "Cardiology",
  "doctor.specialization": "Dermatology"
})

Better query:

In JavaScript and MongoDB, you should not reuse the same key within a single object.

If you type "doctor.specialization" twice, the second value can overwrite the first one. So MongoDB may only search for Dermatology.

If one field can match more than one value, use $in.

5. Treating Dates Like Plain Text

Dates can look like regular text, but in MongoDB they are often stored as real Date values.

In our visits collection, the invoice date is stored inside the invoice object:

invoice: {
  issuedAt: ISODate("2026-05-18T10:15:00Z")
}

So this query will not work as expected:

db.visits.find({
  "invoice.issuedAt": "2026-05-18"
})

This searches for a string, not a Date, so it will usually return nothing.

Better query:

This returns invoices created on May 18, 2026.

The range matters because dates usually include time. A visit at 1:15 PM is still on May 18, but it is not equal to midnight.

So instead of asking:

Is the date exactly "2026-05-18"?

you are asking:

Is the date on or after May 18 and before May 19?

6. Expecting Text Search to Work Automatically

MongoDB does not search text like Google by default.

In our visits collection, the doctor name is stored like this:

doctor: {
  fullName: "Dr. Michael Brown"
}

So this query will not work:

db.visits.find({
  "doctor.fullName": "Michael"
})

This searches for an exact value. MongoDB checks if doctor.fullName is exactly "Michael".

But the real value is "Dr. Michael Brown".

Better setup:

For text search, you need to create a text index first:

db.visits.createIndex({
  "doctor.fullName": "text"
})

Then you can search inside the text field:

Now MongoDB can search for the word Michael inside the indexed field.

The important part is this:

$text search does not work unless the collection has a text index.

7. Reading `$or` Logic Backwards

This query runs fine, but it is easy to read it *the wrong way*:

db.visits.find({
  status: "completed",
  $or: [
    { "doctor.specialization": "Neurology" },
    { "invoice.paymentStatus": "unpaid" }
  ]
})

This does not mean:

completed OR Neurology OR unpaid

It actually means:

completed AND (Neurology OR unpaid)

Because status is outside the $or block, MongoDB treats it as required.

So MongoDB first looks for visits where status is completed. Then, from those visits, it checks if the doctor is specialized in Neurology or the invoice is unpaid.

If you want all three conditions to be part of the OR logic, move status inside the $or array.

Better query:

db.visits.find({
  $or: [
    { status: "completed" },
    { "doctor.specialization": "Neurology" },
    { "invoice.paymentStatus": "unpaid" }
  ]
})

Now the query means:

completed OR Neurology OR unpaid

Same fields. Different structure. Different result.

One More Thing to Do before Using Your Query

A MongoDB query is only as good as your understanding of the underlying document structure.

Before you spend hours rewriting a broken query, do one simple thing: Open a single raw document from your collection.

Look closely at the exact spelling of field names.
Check the data types (Are numbers stored as strings? Are dates stored as objects?).
Map out your nested objects.

This is exactly where a visual tool like VisuaLeaf saves the day. Instead of guessing from your code editor, you can test your queries inside the built-in shell and instantly toggle over to the Table View or Tree View to see exactly how your documents are laid out in real-time. No more guessing.

DBeaver Alternative for MongoDB Queries, Aggregations, and Visual Workflows

VisuaLeaf — Wed, 08 Jul 2026 13:29:52 +0000

DBeaver is a strong tool if you work with many databases.

It makes sense for teams using PostgreSQL, MySQL, SQL Server, Oracle, and MongoDB in the same place. But if most of your work is in MongoDB, the workflow matters more.

DBeaver does support MongoDB, but according to its documentation, the MongoDB driver is available in Lite, Enterprise, and Ultimate editions only. So this is not about saying DBeaver cannot work with MongoDB. It can, if you pay for those versions.

Many developers stick with DBeaver because they are comfortable writing SQL. But VisuaLeaf bridges that gap natively with its own SQL Mode, allowing you to write standard SQL queries that automatically translate into optimized MongoDB syntax.

The question is whether you want a broad database tool, or a workspace built more directly around MongoDB documents, queries, aggregations, schemas, and charts.

That is where VisuaLeaf fits better.

It is focused on MongoDB workflows, not every database type at once.

" width="800" height="480">

VisuaLeaf brings MongoDB queries, schema diagrams, charts, dashboards, and team workflows into one visual workspace.

Why MongoDB Users May Want a Different Tool

MongoDB work is usually more visual than people expect.

You are not only reading rows, but you are also checking what is inside each document.

You are opening nested objects, or you are filtering by fields that may or may not exist in every record.

You are building aggregation pipelines and trying to understand what changes after each stage.

A general database tool can help, but it may not always feel built for this kind of work.

VisuaLeaf is designed for the daily MongoDB tasks that users actually repeat:

-> querying data, reading documents, building aggregations, checking schemas, creating charts, and working with collections more visually.

Build MongoDB Queries Visually

Writing MongoDB queries by hand is useful.

But not every filter needs to start with code.

Sometimes you just want to find documents where status is paid, sort them by date, limit the results, and check the output.

In VisuaLeaf, you can build that query visually.

You add filters, combine conditions, sort results, and see the generated MongoDB query behind it.

That last part matters.

The tool does not hide the query from you. It helps you understand it.

So if you are learning MongoDB, you can see how the query is created.

And if you already know MongoDB, you can move faster without typing every condition from scratch.

" width="800" height="480">

Build MongoDB queries visually and keep the generated code visible.

Build Aggregation Pipelines Step by Step

Aggregation pipelines are powerful, but they can get hard to read fast.

A pipeline with one $match stage is simple.

A pipeline with $match, $group, $sort, $project, $lookup, and $unwind is different.

At that point, the problem is not only writing the syntax.

The problem is knowing what happened to the data after each stage.

VisuaLeaf lets you build aggregation pipelines visually and check the output as you go.

You can add a stage, see the result, then continue.

That makes it easier to find mistakes, test ideas, and explain the pipeline to someone else.

You are not guessing what the pipeline does. You can see it.

" width="800" height="480">

Build MongoDB aggregation pipelines step by step with live preview

Read MongoDB Documents More Clearly

MongoDB documents can be clean, but real data is often messy.

A document may have nested fields, arrays, dates, ObjectIds, embedded data, and references to other collections.

If everything is forced into a flat table, the structure can be hard to follow.

VisuaLeaf gives you different ways to browse MongoDB data, so you can inspect documents without losing the shape of the data.

You can look at the document structure, check nested values, and switch views depending on what you need.

This is useful when you open a collection you did not create yourself.

You should not have to guess what is inside the data. You should be able to see it clearly.

" width="800" height="480">

Browse MongoDB collections in the 3 different view modes.

See the MongoDB Schema

MongoDB is flexible, but that flexibility can make the structure harder to understand.

In SQL, tables and foreign keys usually show the structure.

In MongoDB, relationships may be embedded inside documents, stored as references, or only suggested by field names.

VisuaLeaf helps by showing collections, fields, nested structures, and relationships in a visual schema diagram.

This helps when you are learning a project, documenting a database, or explaining the structure to someone else.

Instead of opening collection after collection and trying to remember everything, you can see the schema in one place.

Visual schema diagrams make flexible NoSQL data easier to understand.

Turn Query Results Into Charts

Sometimes a result table is enough. Sometimes it is not.

If you want to see payments by method, orders by status, revenue by month, or expenses by category, a chart is easier to understand.

VisuaLeaf lets you turn MongoDB data into charts and dashboards without exporting the results into another tool.

You can query the data, check the result, and build a chart from the same workflow.

That makes it easier to understand patterns in the data, not just read raw documents.

" width="800" height="480">

Create charts and dashboards from real MongoDB data.

Mongo Shell for Developers

Visual tools are helpful, but developers still need direct access to the database.

VisuaLeaf includes a MongoDB shell, so you can run commands without leaving the workspace.

You can test a find() query, run an update, check an aggregation, or execute MongoDB commands directly with autocomplete and syntax highlighting.

The difference is that the shell stays connected to the rest of the MongoDB workflow.

You can write commands when you need full control, then switch back to visual results, query building, schema views, or charts when they help.

" width="800" height="480">

Run MongoDB commands with autocomplete, syntax highlighting, and visual results.

DBeaver vs VisuaLeaf for MongoDB Users

Feature	DBeaver	VisuaLeaf
MongoDB access	Available in Lite, Enterprise, and Ultimate	Free Community Edition + 14-day Pro trial
Query building	SQL and JavaScript workflow	Visual MongoDB query builder
Aggregations	Manual SQL or JavaScript workflow	Visual aggregation builder with stage preview
Charts	Available in paid editions	Charts and dashboards in Professional
Best fit	Teams using many database types	Developers focused on MongoDB

DBeaver is stronger when you need one tool for many database systems.

VisuaLeaf is a better fit when MongoDB is your main focus, and you want visual tools for queries, aggregations, schema, documents, and charts.

Final Thoughts

DBeaver is a practical choice if you need one tool for many databases.

But if you came here looking for a DBeaver alternative for MongoDB, you probably care more about the MongoDB workflow than the number of databases a tool supports.

That is where VisuaLeaf is different.

It gives you a MongoDB-focused workspace for browsing collections, building visual queries, creating aggregation pipelines, using the shell, viewing schema structure, and turning results into charts. And these are only some of the features. VisuaLeaf also includes tools for indexes, validation rules, collection comparison, query profiling, dashboards, saved queries, and more.

Try VisuaLeaf if you want a MongoDB workspace that feels more focused, visual, and easier to follow.

MongoDB vs PostgreSQL: The Arguments Everyone Gets Wrong

VisuaLeaf — Wed, 08 Jul 2026 13:08:13 +0000

Most of the Postgres vs Mongo arguments I see online are based on assumptions that are either super outdated or just kind of technically wrong. The same topics keep coming up over and over on reddit, youtube, or whatever, and I wanted to write some of them down.

"PostgreSQL JSONB is basically MongoDB"

JSONB lets Postgres store documents, but it doesn't turn Postgres into a document database. Mongo's storage engine, replication, sharding, aggregation framework, indexing, update semantics, and query engine were all designed around documents from day one. JSONB is bolted onto a relational engine.

It's still really useful. But "just use JSONB" is kind of like saying "just throw a truck bed on a sedan and now it's a pickup." Like, technically there's overlap, but they're still doing different things.

"MongoDB is schemaless"

Actually, that's one of the most common misconceptions.

Mongo lets you have flexible documents, but modern Mongo has schema validation, required fields, type enforcement, unique indexes, and most of the guardrails you'd want from a regular database.

Most Mongo deployments that actually work in production aren't truly schema-free. They just enforce schema differently (usually at the app layer or with validators).

"MongoDB loses data"

This one is basically a meme from like 2013.

Back then:

Default durability settings were weaker
A lot of deployments ran without replica sets
Some people disabled journaling (which... yeah)

Modern Mongo defaults are way safer. You get journaling, replica sets, majority-acknowledged writes, multi-document transactions, point-in-time recovery, and distributed replication. The "MongoDB loses data" thing is mostly people repeating something they heard a decade ago.

"MongoDB is only for lazy devs"

There are a lot of really smart database engineers and architects working at companies that run Mongo at scale. Choosing the "correct" database usually has to do with the workload, not with anyone being lazy, and some workloads just map to documents better than they map to normalized tables.

The misconception usually comes from MongoDB's flexibility, plus developers designing their databases recklessly without thinking about the future. In Postgres, it's much harder to do that because of all its guardrails. But that doesn't mean MongoDB is for lazy developers.

"Postgres is always better"

That's... debatable. Postgres is one of the best databases ever made, and it's usually a safe default. I'm not arguing against that. But like everything in software engineering, databases are tools, not teams to root for.

The question isn't "which one is better." It's "which one matches the workload better."

If your data is super relational, Postgres usually wins. If your data is document-shaped, Mongo usually wins. A lot of real systems end up using both.

Why JSONB vs BSON Benchmarks Are Kinda Misleading

You see a lot of benchmarks where someone compares Mongo documents against Postgres JSONB documents and concludes Mongo is faster. From this video: https://www.youtube.com/watch?v=ZZ2tx8iL3P4 (I am not the author of the video).

And for that test, the numbers are probably right. But the conclusion people pull from them often isn't.

There's two things going on with benchmarks like this.

1. The data models aren't really comparable.

JSONB isn't really Postgres's main data model. So when you benchmark Mongo documents against Postgres JSONB documents, what you're actually comparing is:

A document database
A relational database being used as a document database

A fairer Postgres benchmark would model the same data using normalized tables, foreign keys, real indexes, joins, the whole thing. A lot of workloads that suck in JSONB are actually really fast once you model them properly. The reverse is true too. Workloads that feel natural in Mongo get weird when you force them into rows and joins.

2. The durability defaults aren't comparable either.

In that video specifically, Mongo was running with its default writeConcern, which only flushes the journal every 100ms. Postgres was running with its default WAL fsync on every commit. So Mongo gets to acknowledge writes faster, because it's literally promising less durability per write. That's not "Mongo is faster than SQL." That's "Mongo is faster when you let it relax its durability." Crank Mongo up to j: true or w: "majority" with proper sync and the gap closes a lot.

Durability Isn't a Mongo Weakness Anymore

Like I mentioned above, the "Mongo loses data" thing is mostly an artifact of how people ran Mongo in the early 2010s.

Modern Mongo gives you journaling, replica sets, majority-acknowledged writes, multi-doc transactions, point-in-time recovery, and distributed replication.

A write with:

writeConcern: { w: "majority" }

is durable across multiple replica set members before it even gets acknowledged.

The actual durability gap between a properly configured Mongo cluster and a properly configured Postgres cluster is way smaller than people assume. Most "is it durable" arguments these days are really arguments about operational choices and consistency models, not about the storage engine.

Where Mongo Actually Wins

There are real workloads where Mongo is just the better tool.

Documents that get mutated a lot

Mongo is great when you're updating documents constantly and each update only touches a small piece of the doc. Stuff like:

Counters
User state
Device state
Telemetry that gets enriched over time
Session data

Postgres JSONB has to rewrite the whole JSONB value every time. Mongo often doesn't. This gap is huge in practice. Mongo can churn through partial updates at a rate that JSONB just can't match unless you pull the hot fields out into real columns.

Deeply nested data

Mongo handles nested objects, arrays, and embedded docs naturally. You don't have to join your way through five tables to assemble a real object.

Query patterns you can't predict yet

If your users can add arbitrary fields and then query them later, Mongo's flexibility is hard to beat. You don't have to know the schema ahead of time.

Horizontal write scaling

Mongo's sharding was built into the database from the start. Postgres can scale out too, but you usually end up with partitioning, extensions, distributed variants, or app-level sharding. None of those feel as native as Mongo's sharding does. That's one of MongoDB's biggest strengths.

Document-centric workloads

Mongo really shines when each request reads one doc, updates one doc, writes one doc. No joins, no fan-out, just the doc as the unit of work.

Big documents that change a lot

If your documents are large, mutated frequently, and accessed as a single unit, Mongo is basically designed for that.

Where Postgres Still Wins

Mongo's good at a lot, but Postgres still owns a few areas pretty cleanly.

Complex joins

Postgres's query planner is incredible. It's hard to overstate how good it is at joins. It actually outperforms MongoDB by a lot in a couple of public benchmarks.

Referential integrity

Foreign keys actually enforce relationships. Mongo can't really do this natively.

Reporting / analytics

SQL is still the language for analytics and BI. Pretty much every tool on the planet speaks it.

Heavily relational data

When your data is naturally graphy/relational, Postgres usually produces a cleaner design.

Multi-row transactions

Postgres's transaction model is one of its biggest strengths. Mongo has transactions too, but they're more limited.

A Java vs Python Way of Thinking About It

Here's an analogy that kind of clicks for me. You can almost think of SQL like Java and Mongo like Python.

Java has built-in types, strict structure, classes everywhere. It kind of forces you to organize your code and keep things modular. You can't really get away with being sloppy, because the language won't let you. SQL is similar. Schemas, types, constraints, foreign keys. The structure is enforced for you whether you want it or not.

Python is more like Mongo. Super flexible, super widely used, still really popular and not going anywhere. But if you don't have the self-discipline to make actual classes, structure your modules properly, and keep things organized, your codebase falls apart pretty fast. It's the same with Mongo. The flexibility is great until you stop enforcing schema at the app layer and your collections turn into a mess of inconsistent docs.

Neither is bad. Python is huge, Mongo is huge. They give you a lot of room to move fast. But the room to move fast is also the room to shoot yourself in the foot. Java and SQL take that choice away from you, which sucks when you want flexibility and is great when you need guardrails.

Different Tools, Different Assumptions

The way I think about it is that Postgres and Mongo aren't really competing for the same spot. They're built on different assumptions.

Postgres assumes:

Relationships matter
Consistency matters
Joins matter
Transactions matter

Mongo assumes:

Documents are the unit of work
Schema evolves
Updates are localized
Horizontal scaling matters

Neither one is universally right. The mistake is forcing your workload into the wrong set of assumptions, and then being mad when the database fights you. The best database is usually just the one whose assumptions most closely match what your app actually does.

A Quick Note on Where This Is Coming From

For context: I've been building a tool called VisuaLeaf for the last year and a half. It's a visual GUI for both MongoDB and PostgreSQL. Same workspace, same workflow, whichever database your data happens to live in. You can even migrate from sql databases to a mongodb database using VisuaLeafs Tasks/

Building it means I spend a kind of unreasonable amount of time inside both engines. A lot of these misconceptions come up in support tickets, Reddit threads, and questions from people picking between the two, so I figured I'd dump them in one spot instead of typing the same reply over and over.

PostgreSQL JSONB vs MongoDB BSON: The Real Architectural Tradeoffs

VisuaLeaf — Wed, 08 Jul 2026 13:04:12 +0000

Most teams pick between Postgres and Mongo by arguing about SQL vs documents, transactions, joins, or what everyone's using at the moment. The format on disk barely makes it into the discussion. That’s the wrong place to stop, because each database’s byte-level design carries the philosophy of the engine. BSON is a binary echo of MongoDB's runtime: a self-describing wire format that the server can scan, mutate, and ship easily. JSONB is a parse tree frozen into a Postgres tuple: optimized for read-time access, indifferent to write-time mutation, and beholden to MVCC.

Once you know what each format actually is, the rest of the comparison stops being a religious argument and starts being a set of engineering tradeoffs you can reason about. This piece walks through what BSON and JSONB are at the byte level, how they behave once they hit storage, what each one costs to index, update, and read back, and which workloads each one quietly punishes.

Why the format matters more than the API

Application developers see JSON. Both engines accept JSON on the way in and serve JSON on the way out, so it is tempting to assume the storage format is a cosmetic detail. It is not. The storage format dictates:

How much byte juggling the server does on every read.
Whether a partial update can mutate a value in place or has to rewrite the entire document.
What kinds of indexes can be built and how big they get.
How much write amplification you pay when a single field changes.
Whether the disk representation survives a torn page or a partial flush.

A query that runs in 1ms on a hot row can run in 30ms if the format forces the server to materialize and reparse the row. A field update that flips a boolean can rewrite 8KB of heap, generate WAL, trigger toast churn, and invalidate three indexes, or it can patch four bytes. The format decides which world you live in.

BSON: a binary echo of MongoDB's runtime

BSON stands for Binary JSON. BSON is a length-prefixed, type-tagged, ordered key-value format that was designed with three properties in mind: cheap to parse linearly, cheap to mutate in place when the new value is the same size, and rich enough to carry types that JSON cannot express.

The wire layout of a BSON document looks roughly like this:

int32  total_document_length_in_bytes
{ element }*
byte   0x00  // terminator

Each element is itself:

byte    type_tag    (0x01 double, 0x02 string, 0x03 embedded doc, 0x07 ObjectId, ...)
cstring field_name
<payload depending on type>

Two things follow from that layout. First, every document and every embedded subdocument carries its own length prefix, which means a parser can skip an entire subtree in one pointer arithmetic step. Second, fields are ordered. The same logical document can be encoded in different byte sequences depending on insertion order, and the server preserves that order.

The type tag set is wider than JSON's. BSON has dedicated tags for:

32 bit and 64 bit integers (JSON has only a generic number).
IEEE 754 decimal128 for financial workloads.
ObjectId (12 bytes: timestamp, machine id, counter).
UTC datetime (int64 milliseconds since epoch).
Binary blobs with a subtype byte.
UUID (a binary subtype).
Regular expressions.
A JavaScript code type, mostly historical.
MinKey and MaxKey sentinels used for index bounds.

Two consequences for engineers. One: BSON round trips numeric types faithfully. A column that stores 64 bit account ids does not silently become a double the way it would in pure JSON. Two: BSON carries metadata that JSON cannot, which is part of the reason a MongoDB driver feels chatty when you push it through a strict JSON pipeline.

The clever bit, and the one that distinguishes BSON from a hundred other binary JSON formats: every field's payload is preceded by enough information that a server can walk the document linearly without recursive parsing, and many field updates can be performed by patching the payload in place. If you change an int32 field from 7 to 8, the document length does not change, the field offset does not move, and the engine writes four bytes. That property is what lets WiredTiger keep update latency flat across a wide range of document sizes.

It also explains the cost of BSON. Field names are stored as raw cstrings in every document. A collection of a billion documents with a created_at field carries the literal bytes created_at\0 a billion times. There is no schema, no dictionary, no shared symbol table. Wide documents with long field names waste a lot of disk and a lot of memory.

JSONB: a parse tree frozen on disk

JSONB is the spiritual opposite of BSON. Where BSON is a wire format that happens to live on disk, JSONB is a storage format that happens to be transferable. Postgres parses incoming JSON, normalizes it (keys sorted, whitespace stripped, duplicate keys deduplicated with last write wins), and serializes the result into a binary structure that maps directly onto the engine's value tree.

There is no public byte spec the way there is for BSON. Internally, a JSONB value is a header plus a sequence of entries that describe each key value pair, plus the data area. The headers contain length and type bits so the engine can binary search keys inside an object, and offsets are stored every N entries to bound the cost of finding a specific key. That structure has two big implications.

First, JSONB is a read optimized format. Once the document is stored, looking up a key by name is logarithmic in the number of keys, not linear. For wide objects with hundreds of fields, this matters. BSON, by contrast, is linear: it walks the document until it finds the key, leveraging the length prefixes to skip subtrees.

Second, JSONB cannot be patched in place. The header layout, the offset cache, and the dedup pass all assume the value is being constructed from scratch. Updating a single boolean inside a 4KB JSONB document materializes a brand new 4KB JSONB document, writes a new heap tuple, marks the old one dead, and updates every index that references it. This is not a JSONB design flaw, it is the consequence of binding the format to Postgres's MVCC and TOAST machinery. JSONB does what makes Postgres fast at reads. Postgres pays for the rest at write time.

JSONB throws away two things compared to JSON. Key order is lost (objects are stored with keys sorted), and duplicate keys are collapsed. Most applications never notice, but if you are relying on either property, you want json (the text type) not jsonb. Almost nobody should be relying on either property.

JSONB also does not preserve numeric type fidelity in the same way BSON does. Postgres has its own numeric type which is arbitrary precision, and JSONB encodes numbers using that representation. You will not lose precision the way you would with a 64 bit float, but you also do not get a distinct int32 vs int64 tag the way BSON gives you.

Binary blobs and where the type systems leak

The gap between BSON and JSONB shows up the fastest when data falls outside JSON's native type system.

Binary blobs are the usual offender. BSON has a native binary type with a subtype tag, JSONB does not. In Postgres your options are base64 encoding the bytes into JSONB (ugly), or pulling them out into a separate bytea column and referencing them by id (cleaner). Splitting the document works fine when you own the schema, it becomes awkward when you are ingesting third party documents that already embed binary fields, like S3 events with attachments or message payloads with thumbnails.

Storage: WiredTiger pages vs heap tuples plus TOAST

The byte format is half the story. The other half is what the storage engine does with the bytes.

MongoDB stores BSON documents inside WiredTiger, a B+ tree storage engine that also backs a number of other databases. WiredTiger gives MongoDB two important properties:

Block compression by default. Snappy compresses each block; zstd is available and can roughly halve the disk footprint on text heavy collections. The compression happens at the storage layer, so the in memory representation is uncompressed but the on disk and on wire (replication) footprint is much smaller. For a workload that stores web event payloads or product catalogs, the ratio of on disk size to logical size is routinely 3:1 or better.
In place updates when possible. If a field changes and the new BSON encoding fits in the existing slot, WiredTiger patches the page. If the document grows beyond its allocated space, WiredTiger rewrites the document, which is the expensive path. Schema decisions matter here: documents with arrays that grow over time will rewrite often, and the standard advice is to size arrays carefully or break them out into separate collections.

WiredTiger maintains a journal (write ahead log) that commits to disk every 100ms by default, and snapshots dirty pages to disk at 60 second checkpoints. The two intervals together keep recovery time bounded without grinding the write path on every commit.

PostgreSQL stores JSONB inside a heap tuple, the same structure that holds every other Postgres row. The heap is a page based store (8KB pages by default) with row level MVCC. Every update writes a new tuple, links it to the old one, and lets autovacuum reclaim the dead copy later.

JSONB also interacts with TOAST (The Oversized Attribute Storage Technique). Any column value that exceeds approximately 2KB after compression gets pushed into a separate TOAST table and replaced in the main heap with a pointer. JSONB documents larger than that threshold therefore live in two places: the heap tuple holds a pointer, and the actual JSONB lives in toast chunks. Reading the document means following the pointer and reassembling the chunks. The default TOAST strategy for jsonb is EXTENDED, which means values are first compressed (using the cluster's default compression, pglz or lz4 since PG 14), and then chunked if still over threshold.

The two consequences are easy to miss:

A SELECT * on a table with TOASTed JSONB columns will fetch the entire chain even if you only wanted three scalar fields from the row. Project the columns you need or extract the JSONB fields explicitly.
Updating one field in a JSONB document materializes a new JSONB, which gets TOASTed again. The old TOAST chunks become dead and wait for autovacuum. On a heavy update workload against large JSONB documents, the TOAST table can balloon faster than the main heap.

Postgres 14 added LZ4 compression for TOAST, which is meaningfully faster than the legacy pglz both for compression and decompression. If your JSONB columns are large and updated often, switching to LZ4 is the single highest leverage TOAST change you can make.

Updates: in place vs MVCC tombstones

Spend ten minutes profiling a heavy update workload on each engine and the contrast jumps out.

In MongoDB, a field level update with $set, $inc, or $push is interpreted by the server and translated into a targeted mutation of the BSON document. If the new value fits, WiredTiger updates the page in place. If it does not fit, the document is rewritten. Index entries are only touched for the fields that actually changed and that are indexed. The journal records the delta, not the whole document.

In PostgreSQL, UPDATE on a JSONB column always rewrites the entire JSONB value. The jsonb_set function looks surgical at the SQL layer (it lets you set a specific path) but underneath, it builds a new JSONB and the row update replaces the old tuple. MVCC then leaves a dead tuple behind, autovacuum reclaims it later, and every index on the table that does not satisfy the HOT (Heap Only Tuple) conditions has to insert a new entry pointing at the new tuple. If your indexes reference fields inside the JSONB column, HOT is off the table and every update is amplified.

In practice this means two patterns dominate.

MongoDB rewards documents with mutable internals. A counter inside a document, a status field that flips often, an array that gets pushed to on every event: these are cheap operations. The flip side is that document growth is the silent killer. A document that starts at 1KB and grows to 50KB over a year is rewritten by WiredTiger every time it crosses its current allocation, which means the late life updates are dramatically more expensive than the early life updates.

PostgreSQL rewards documents that are read more than they are updated. Catalog data, feature flags, event payloads that are written once and read many times, configuration that changes weekly rather than per second: JSONB is excellent for these. The moment you start treating a JSONB document as a per row mutable object that changes on every request, you are fighting the engine.

The benchmark that exposes this gap is straightforward. Take a million row table or collection, each row holding a 4KB document with a view_count integer somewhere inside it. Increment the counter once per second per row across all rows. MongoDB may only need to modify the bytes associated with the changed field, avoiding the full document rewrite that JSONB incurs. Postgres rewrites 4KB of JSONB per document, generates 4KB of WAL per update, and starts producing dead tuples faster than autovacuum can clean them. The Postgres answer here is simple: do not store view_count in JSONB. Pull it into a regular bigint column. That fix is real and lasting, and it is also a tax on schemas that try to be JSON first.

Indexes: GIN, multikey, and what each can answer

Both engines let you index inside documents. The mechanisms are different, and the resulting indexes have different shapes.

PostgreSQL offers two index strategies for JSONB.

GIN indexes on the whole document. The default operator class supports containment (@>), key existence, and path queries. A more aggressive variant, jsonb_path_ops, supports only @> but produces a smaller, faster index. Both are inverted indexes that emit one entry per path through the document.

CREATE INDEX idx_events_data ON events USING GIN (data jsonb_path_ops);
SELECT * FROM events WHERE data @> '{"type": "signup"}';

B tree indexes on specific expressions. When you know the exact path you want to query, an expression B tree is dramatically smaller and faster.

CREATE INDEX idx_events_user_id ON events ((data->>'user_id'));
SELECT * FROM events WHERE data->>'user_id' = '12345';

MongoDB indexes work on field paths directly. The standard createIndex({user_id: 1, created_at: -1}) builds a B tree on a compound path. The syntax is different from Postgres, but the behavior is familiar.

Multikey indexes. When the indexed field is an array, MongoDB automatically creates one index entry per array element. The same B tree, traversed differently. A document with tags: ["postgres", "mongodb", "bson"] generates three index entries pointing at the same document, and any element match query can use the index. This is the killer feature for tags, references, and other array shaped data.

For fixed, known query shapes, Postgres ties or beats Mongo with expression B trees. For varied or array heavy data, Mongo's first class multikey support gives it more leverage with less ceremony.

One more thing Postgres has on Mongo here: partial indexes with arbitrary predicates. You can build an index that only covers the rows where some JSONB field has a specific value, or a date range, or whatever. Mongo has partial indexes too but the filter expressions you can use are way more limited, so anything more than a simple equality check usually has to live in the query side instead of the index.

Read paths and the cost of materialization

The end to end read path is where the two formats earn or lose their reputation.

A MongoDB find that hits an index and returns a single document does roughly the following: traverse the B tree to find the record id, fetch the page from WiredTiger, decompress the block, locate the BSON document inside the page, copy it into the response buffer, ship it over the wire. The document is already in BSON. If the driver supports it (most do), the bytes can travel from the storage page to the client with no parse on the server side. Field projection ($project in aggregation, or projection on find) walks the BSON in place and emits a trimmed version.

A PostgreSQL select that hits an index and returns a single row does roughly the following: traverse the B tree to find the heap tuple location, check the visibility map and the tuple's xmin/xmax to confirm the row is visible to this transaction, fetch the heap page, locate the tuple, follow any TOAST pointers to reassemble JSONB columns, materialize the row into the executor's tuple format, apply any projections, and emit. The visibility check is the MVCC tax. The TOAST chase is the wide column tax. The materialization is the format tax.

For point reads of small documents, both engines are fast and the differences are noise. For wide documents (tens of KB) or selective projections on a large document, BSON has a structural edge because the on disk format and the on wire format are the same, and projection is a walk. JSONB has to be unpacked and the projection produces new JSONB. The win is not huge in absolute terms, but it shows up on workloads that fan out reads.

Aggregations flip it though. Postgres has been working on its query planner for like 30 years. Parallel scans, hash joins, merge joins, a cost based optimizer that has seen basically every shape of analytical query a person can write. Mongo's aggregation pipeline has gotten way better in recent versions (SBE is faster, $lookup is actually usable now), but once you're doing real analytics with multiple joins and big group by aggregates, Postgres just wins. Not by a small margin either, you'll often see big multiples on the same workload.

When PostgreSQL JSONB is the right answer

Reach for JSONB when:

The bulk of your schema is relational and a few columns happen to be semi structured. Customer records with a metadata field. Products with a variable attributes blob. Orders with line items that have inconsistent shape across categories.
Reads dominate writes against the JSONB columns. Catalog data, configuration, audit payloads, settings.
You need joins. Postgres can join JSONB columns against regular columns against full text indexes against PostGIS geometries against time series partitions in one query. Mongo can do this through $lookup but it is not the same.
You need transactions across many rows. JSONB inherits Postgres's transaction model for free.
The team is already running Postgres and the operational cost of adding a second database is not justified.
You want strong typing in some columns and document flexibility in others, in the same table.

Workloads that fit this shape: SaaS application data, e commerce catalogs, CMS storage, configuration stores, audit logs that need to be queried by structured fields, anything where the analytical query layer is going to use SQL anyway.

When MongoDB BSON is the right answer

Reach for MongoDB when:

The unit of work is the document. Each request reads or writes one document, occasionally a few. Most operations do not span documents.
Documents are large, mutate frequently, and the mutations are field local. Telemetry events that get enriched over time. Game state. IoT device shadows. Anything where the same document gets touched many times and each touch is small.
The query patterns are varied and the schema is genuinely fluid. You do not know which fields users will query next month.
You need horizontal write scaling out of the box. Mongo's native sharding has gotten very good. Postgres has sharding solutions (Citus, partitioning) but they are bolted on rather than first class.
You need a document oriented secondary index pattern that Postgres expression indexes cannot model cleanly.
The team has Mongo expertise and the workload does not need SQL analytics.

Workloads that fit this shape: real time event streams with rich payloads, mobile and game backends, IoT device fleets, content management with deeply nested structures, anything where the data model is naturally a graph of nested objects and the queries are mostly key value style with occasional secondary lookups.

The practical takeaway

If you remember three things from this:

The format is the engine. BSON is what makes Mongo fast at field updates and fast at projection. JSONB is what makes Postgres flexible without giving up relational performance. The choice between them is not about JSON vs binary, it is about update behavior, index shape, and read path.
JSONB is a read optimized format. It pays for itself when you write once and read many. It punishes you when you treat a JSONB column as a mutable object that changes on every request. Pull hot mutable fields out into regular columns.
BSON is an update optimized format. It pays for itself when documents are touched often and the touches are local. It punishes you when documents grow unboundedly, when field names are long, and when the query pattern wants joins and aggregations.

Pick the engine whose architectural assumptions match your workload's access pattern. Everything else is a tax you will pay every day until you fix it.

Working with MongoDB documents?

Understanding BSON is useful, but seeing the structure of your collections makes the work much easier.

With VisuaLeaf, you can browse MongoDB documents, inspect schemas visually, build queries, create aggregation pipelines, and understand how your data is actually organized.

DEV Community: VisuaLeaf

Learn how to use DuckDB for local analysis: open a database, query CSV and Parquet files, inspect relationships, edit data, and create charts.

How to Use DuckDB for Local Data Analysis

mongodump Tasks Automate and Schedule MongoDB Backups

Save the Same mongodump as a VisuaLeaf Task

Back Up a Database or One Collection

Choose the Output Folder and Enable Gzip

Schedule and Monitor the MongoDB Backup

What Files Does the Backup Create?

Can You Run mongodump in MongoDB Compass?

Restore the BSON Backup in VisuaLeaf

If mongodump Does Not Run on Windows

MongoDB Backup Limitations to Plan For

Conclusion

PostgreSQL JSONB: Query, Update, and Index JSON Data

PostgreSQL JSONB operations used in this guide

Create the support tickets table

Query nested JSONB values

Match part of a document with @>

Update a nested value with jsonb_set

When the parent object is missing

Add a GIN index for containment searches

When to use JSONB

References

MongoDB Schema Design: Build or Reverse Engineer Visually

Reverse engineer an existing MongoDB database

Review the detected structure

Review and add relationships

Modify the generated schema

Design a new MongoDB schema from scratch

Keep the schema virtual or materialize it

Choose whether to attach validators

Review the materialization plan

Verify the result

One schema, two directions

Which MongoDB Client Handles Large Collections Best: VisuaLeaf vs Compass vs Studio 3T

Test setup

Test 1: What the large-result test actually shows

Test 2: Comparing two MongoDB collections

VisuaLeaf

Studio 3T

MongoDB Compass

Test 3: Automating Daily Orders

Test 4: Checking the Query Before Blaming the Client

Which client fits which workflow?

How to Copy and Sync a MongoDB Collection to PostgreSQL

Source Collection

PostgreSQL Table

Why mongo_id Is Important

Field Mapping and Data Types

The Error I Had With Column Names

Sync Setup

Checking the Initial Copy

Testing Insert, Update, and Delete

What I Would Watch For

Conclusion

Want to Sync MongoDB to Atlas Instead?

How to Import JSON into MongoDB and Export to CSV with Data Masking

The workflow

The incoming JSON file

Why use a temporary collection?

Generate field mappings every time the JSON structure changes

Copy the batch into the main orders collection

Duplicate key errors are useful, but they still need a fix

Export the weekly report from the main collection

Mask customer data during export

What maskName(value) does

What hash(value) does

Schedule the workflow

What this workflow actually solves

Final result

How to Copy and Sync a Local MongoDB Collection to Atlas

What I used

Preparing the Atlas connection

Creating the sync job

Mapping the local collection to Atlas

Choosing Full sync

Reviewing and launching the job

Verifying the initial copy in Atlas

Test 1: Inserting a new local document

Match part of a document with `@>`

Update a nested value with `jsonb_set`

Why `mongo_id` Is Important

2. Treating `$or` Like a Regular Object

7. Reading `$or` Logic Backwards