To add to Kasey's comments about using LINQ, this would help you out considerably. Your line parser is basically just a function, right? There's no need to stand up an entire interface for that. It's basically a Func<string, TOut>, where TOut is whatever your desired output type. You're also using methods vs. functions in there. In other words, you have a lot of void return types instead of functions that return the actual values that you want. I would prefer to see something like
var values =
File.ReadLines(@"..\..\example-input.csv")
.Where(x => x.StartsWith("MNO"))
.Select(LineParser.ParseLine);
That will read in lines lazily, filter out any line that starts with MNO, and then parse your lines out. On top of that, you can parallelize it by sticking a .AsParallel() in before the Select statement, if you're concerned about speed. I'd be curious to see how these affected your processing speed.
- Performance & optimisation is my wheelhouse
- Passionate about technical excellence
- Extreme programming & lean software development
- Trust in the scientific method
Interesting idea. I will admit that I did not consider LINQ, if I get time I will try your approach (and if you have time feel free to send in a pull request).
To add to Kasey's comments about using LINQ, this would help you out considerably. Your line parser is basically just a function, right? There's no need to stand up an entire interface for that. It's basically a
Func<string, TOut>, where TOut is whatever your desired output type. You're also using methods vs. functions in there. In other words, you have a lot ofvoidreturn types instead of functions that return the actual values that you want. I would prefer to see something likeThat will read in lines lazily, filter out any line that starts with MNO, and then parse your lines out. On top of that, you can parallelize it by sticking a
.AsParallel()in before theSelectstatement, if you're concerned about speed. I'd be curious to see how these affected your processing speed.Hi Jason,
Interesting idea. I will admit that I did not consider
LINQ, if I get time I will try your approach (and if you have time feel free to send in a pull request).Cheers,
Indy
Hi Jason, do you have memory consumption using linq?