V-Order optimize parquet file through sorting, row group distribution, encoding and compression.
Disadvantage of V-order optimization is that it increase the write time by up to 15 but positive side is that it boost the data compression by 50% and data read time improve by 10% and also in some cases read time improve up to 50% as well.
parquet engine can read it as a regular parquet file.
there is not any impact on delta table other features, like Z-Order, vacuum, time travel, compaction etc.
V-Order is disabled by default
In Fabric runtime 1.3 and higher versions, the spark.sql.parquet.vorder.enable setting is removed. As V-Order is applied automatically during Delta optimization using OPTIMIZE statements, there's no need to manually enable this setting in newer runtime versions. If you're migrating code from an earlier runtime version, you can remove this setting, as the engine now handles it automatically.
Top comments (0)