loading...
Cover image for  Kotlin CSV to Data Class Parser

Kotlin CSV to Data Class Parser

blackmo18 profile image Victor Harlan D. Lacson Updated on ・2 min read

Update: The current version is 0.4.1

Features

1. Simple And Direct

  • No hard configuration
  • No invasive annotations to data class
  • Custom mapping
  • Nullable Data Types

2. Primitive Types

  • Short
  • Int
  • Long
  • Float
  • Double
  • Boolean
  • String

3. Support for Java 8 Date Time Apis

  • LocalTime
  • LocalDateTime
  • LocalDate
  • Custom Formatting

Hello Everyone!!

Hi to those who love Kotlin

To this date, I haven't found one unique implementation on Kotlin CSV parser into Kotlin data class.

So I decided to create one, and it is now available on GitHub Kotlin-Grass.

Unlike some existing Java CSV parsers, it doesn't require to invade your data class with annotation to map CSV file columns to data class fields.

Tho before its usage it requires another awesome Kotlin library for reading the CSV, namely Kotlin-CSV. Since this one is implemented in Kotlin Multiplatform I don't intend to reinvent the wheel for reading CSV File.

Let's get back to my Kotlin-Grass, a CSV to data class parser. It is already available at Jcenter for download.

repositories {
  jcenter()
}

dependencies {
    implementation("com.vhl.blackmo:kotlin-grass-jvm:0.4.1")
    implementation("com.github.doyaaaaaken:kotlin-csv-jvm:0.11.0 ")
}

Its usage is quite simple and direct nothing more than a plug and play

for example, we have a CSV file with the following data

short,int,long,float,double,boolean,string
0,1,2,3.0,4.0,true,hello

the data class would simply look like

data class PrimitiveTypes(
    val short: Short,
    val int: Int,
    val long: Long,
    val float: Float,
    val double: Double,
    val boolean: Boolean,
    val string: String
)

Yes! No annotation is required. It will automatically map the first line as header to data class fields.

The usage is just like

val csvContents = csvReader().readAllWithHeader(file)
val dataClasses = grass<PrimitiveTypes>().harvest(csvContents)

or the other way

csvReader.open(file) {
    val csvContents = readAllWithHeaderAsSequence()
    val dataClasses = grass<PrimitiveTypes>().harvest(csvContents)
}

You get a list or sequence of object parsed into a data class.

Currently, it is in the development stage. Compared to Java OpenCsv reading and parsing 32,000 lines Kotlin-Grass is behind around 200 milliseconds.

Yes I know 200 milliseconds is quite big for a software. It's still sprouting, there is more to come and fun in Kotlin

So, guys, I would love to get your feedback.

Also your PRs and feature requests are much appreciated if you had any issues.

Happy Hacking

Posted on by:

blackmo18 profile

Victor Harlan D. Lacson

@blackmo18

Software Developer Ethusiast, Java, Kotlin, Multiplatform

Discussion

pic
Editor guide
 

Cool, csv-parsing is a nice, self-contained but useful problem domain!
I recently wrote a csv to Kotlin data class parser for my work at 'Which?' as well 😅. I called my project KSV.

CSV contain some interesting corner cases that you might want to consider:

  • column names (the comma-separated elements of the header) might contain spaces, so while the property name is a good default, you might want to add an annotation that let's you define a different name (e.g. one that contains a space)
  • column names and values might be surrounded by quotes so that they can contain a comma (so your line splitting has to be a bit smarter than simply splitting by comma )
  • you don't seem to support nullable types currently, it makes sense to map empty strings in the csv to null, if the type you map to is nullable
  • while the separator in csv is often a comma and the quote a double quote, it's a good idea to make that configurable (e.g. because german uses a comma instead of a dot for floats, their csv often uses a semicolon instead)

advanced features:

  • support conversion to LocalDateTime (you need to specify the expected format)
  • support conversion to user-defined types (the user must be able to add string-to- converters)

I thing those are the main additional features I can think of right now. How did you publish to jcenter? Is it free (for open source)? I tried to google a free jar repository, but ended up suggesting Gradle's source dependency mechanism

 

Hi, thanks and I really appreciate the key points you shared.

  • Regarding the custom delimiters, surrounded with quotes, Kotlin-Csv already handles the scenarios. That's why I mainly focused on my Kotlin-Grass as a parser itself.

  • Custom Mapping is also supported on initial release, instead of relying on annotation I resorted to DSL mappings.

 val grass = grass<DateTimeTypes> {
        customKeyMap = mapOf("hour" to "time", "birthdate" to "date")
    }

which is I think neat and there is no need to annotate your data class

  • Support on Java Date Time API conversion is also available.
  • On User-defined types, I never yet encountered a common use case. I guess the user can do extension functions instead without polluting data class of unwanted logic but data itself. If it is not that complicated 😖🤣

With the release of version 0.3.0, nullable values, and white spaces were already covered.

Publishing to jcenter I never thought would be just easy, although I have done so much research since my structure is for kotlin-multiplatform. You just need to create bintray account, then publish your library in there. Once your library is uploaded you can request to link your packages to jcenter.

Apply necessary plugin in Gradle build file, like maven publish. You can check my repo's build file, it is configured for publishing in bintray.

 

cool, seems like you got all the features I can think of then 👍