DEV Community

Discussion on: Processing One Billion Rows in PHP!

Collapse
 
cviniciussdias profile image
Vinicius Dias

Did you consider using spl or even ext-ds instead of arrays? Maybe it would help a little bit as well.

Collapse
 
realflowcontrol profile image
Florian Engelhardt

That is a good idea, thanks for letting me know. I'll see if I find time. If you'd like to give it a shot, you can find a GitHub repo holding the source at github.com/realFlowControl/1brc

Although profiling shows me that most of the time (>80% of wall time) is spend in reading the data from the file via fgets() and then converting (split on ; and type cast). Nevertheless, shoving of some time with a better data structure would be nice, also a nice opportunity to dive into ext-ds