DEV Community

Cover image for Memory efficient parsing in GO
Cyber
Cyber

Posted on

Memory efficient parsing in GO

Instead of greedily reading the whole stream in memory and passing it to json.Unmarshal:

json.Unmarshal(bigBlobOfData, &myStruct)
Enter fullscreen mode Exit fullscreen mode

We can delegate the stream reading to our dear masters of golang:

json.NewDecoder(myIOReader).Decode(&myStruct)
Enter fullscreen mode Exit fullscreen mode

And read multi giga(tera?)bytes files with no sweat. Also works with xml.NewDecoder.

Discussion (0)