Spring Batch read CSV file and write to database example — Complete Guide

A practical, in-depth guide to Spring Batch read CSV file and write to database example with examples.

INTRO

Every day, enterprises receive massive CSV dumps from partners, legacy systems, or IoT devices. Turning those flat files into relational data isn’t as trivial as “read‑line‑by‑line and INSERT”. You quickly run into performance bottlenecks, transaction management headaches, and error‑handling nightmares. The classic “read whole file into memory then bulk insert” works for a few hundred rows, but it collapses under real‑world loads of millions of records.

Spring Batch was built to solve exactly this class of problems. It gives you a declarative, fault‑tolerant pipeline that can chunk data, retry failed records, and parallelize work without you having to reinvent the wheel. Yet, many developers still stare at the documentation and wonder how to wire a CSV reader to a JDBC writer in a clean, testable way. That’s the gap this teaser aims to fill.

In the full guide you’ll see a production‑ready configuration that reads a CSV with custom delimiters, maps each line to a POJO, validates the data, and writes it to a relational table using batch inserts. The example is deliberately minimal so you can copy‑paste it into a Spring Boot project, but it also dives deep into the knobs you’ll need to turn when you move from a demo to a production job.

WHAT YOU'LL LEARN

• How to configure FlatFileItemReader with a DelimitedLineTokenizer and a BeanWrapperFieldSetMapper for clean POJO mapping.
• Setting up JdbcBatchItemWriter with named parameters and a configurable batchSize to maximize DB throughput.
• Defining a Step with chunk‑oriented processing, including skip‑policy and retry logic for malformed rows.
• Building a Job that can be launched via Spring Boot, the command line, or a scheduler like Quartz.
• Strategies for handling large files: multi‑threaded steps, partitioning, and resource cleanup.
• Common pitfalls (e.g., transaction boundaries, CSV encoding issues) and how to avoid them.

A SHORT CODE SNIPPET

@Configuration
@EnableBatchProcessing
public class CsvToDbJobConfig {

@Bean
public FlatFileItemReader<Person> reader() {
return new FlatFileItemReaderBuilder<Person>()
.name("personItemReader")
.resource(new ClassPathResource("people.csv"))
.delimited()
.names("firstName", "lastName", "email")
.fieldSetMapper(new BeanWrapperFieldSetMapper<>() {{
setTargetType(Person.class);
}})
.build();
}

@Bean
public JdbcBatchItemWriter<Person> writer(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Person>()
.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
.sql("INSERT INTO person (first_name, last_name, email) VALUES (:firstName, :lastName, :email)")
.dataSource(dataSource)
.assertUpdates(false)
.build();
}

@Bean
public Step csvToDbStep(StepBuilderFactory stepBuilderFactory,
FlatFileItemReader<Person> reader,
JdbcBatchItemWriter<Person> writer) {
return stepBuilderFactory.get("csvToDbStep")
.<Person, Person>chunk(500)
.reader(reader)
.processor(item -> item) // no-op processor, placeholder for validation
.writer(writer)
.faultTolerant()
.skipPolicy(new AlwaysSkipItemSkipPolicy())
.build();
}

@Bean
public Job importUserJob(JobBuilderFactory jobBuilderFactory, Step csvToDbStep) {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.flow(csvToDbStep)
.end()
.build();
}
}

KEY TAKEAWAYS

• Chunk‑oriented processing is the heart of Spring Batch; it lets you balance memory usage and DB round‑trips by controlling the chunk size.
• Fault tolerance (skip, retry, and rollback) should be configured early; it prevents a single bad row from killing an entire run.
• Declarative configuration using the builder API keeps the job definition readable and testable, while still exposing all the low‑level knobs you might need later.
• Scalability isn’t an afterthought—Spring Batch provides partitioning and multi‑threaded steps that can be added with a few extra beans when your CSV grows beyond a few hundred thousand rows.

👉 Read the complete guide with step-by-step examples, common mistakes, and production tips:

Spring Batch read CSV file and write to database example — Complete Guide