Processing File concurrently using Goroutines

Problem

• How to Read File
• How to Process File (i.e Channel) Concurrently with Goroutines

Solution

Reading a File

• Spin up a goroutine to read data from file in chunks using

scanner.Scan()

Processing Channel

• Spin Up N goroutine
• Each one will take a Chunk from the Channel
• Extract data from the Chunk and execute processFn

func ProcessFile(file io.Reader, numWorkers int, chunkSize int, processFn func(data []byte)) 

Github Repo for code

• Sandy10247 / go-process-file

  A Sample Repo, to demonstrate File Processing concurrently

  go-process-file

  A Sample Repo, to demonstrate File Processing concurrently

  Execution

  

  Dev to Article

  Processing File concurrently with using Goroutines

  
  
  

  
  

  View on GitHub

  
  

Github Repo

Inspired by