DEV Community

Judy
Judy

Posted on

Merge multiple files, remove duplicates and sort the result #eg25

There are multiple csv files having the same structure in a directory:

Image description
file-februar.csv

Image description
Use Java to do this: Combine these files, remove duplicate rows and sort rows by From, and save the result as a new file:

Image description
Write the following SPL code:

=T(\"d:\result.csv\":directory@p(\"d:\file-*.csv\").(T(~)).merge@u().sort(From))

T() function parses a file or writes data of a file to a new file. directory@p searches for target files according to the wildcard characters and returns files with full paths. merge@u merges records by rows or fileds; @u option finds the union. The symbol ~ represents the current varialbe in a loop.
Read How to Call a SPL Script in Java to find how to integrate SPL into a Java application.
Source:https://stackoverflow.com/questions/71508457/java-datastructure-like-a-table-without-using-a-database

Top comments (1)

Collapse
 
esproc_spl profile image
Judy

SPL open source address:github.com/SPLWare/esProc