It's not reason for ":(", it's for ":)". Because it means you can write normal-looking easy code and it will work.
Your example, even though fictional, has nothing to do with my comment: if you have to process files as a whole, then you do, and no amount of black magic will help you. If you don't, then the question is whether it's a text or a binary file, as I said. And then you should use line or block buffering as needed.
If you're really strapped for memory, the first optimization I'd suggest is not using Python. Python is so dynamic that common data structures easily take up many times more memory than in "normal" languages with value semantics.
Like what? Like "I have a binary file 4GiB in size and I'm just gonna spit it to stdout 1KiB at a time"? Not to mention that you don't do any decoding at all, so bytes objects are written to your screen raw, which isn't what you want, no matter the usecase. And not to mention "if it doesn't exist in the moment I check, I won't do anything, even though it might exist later when I'd actually try to read from it"?
Sorry, I know you're trying to salvage your post, but "like a pro" doesn't mean that. A pro should know the terrain of possibilities they might encounter, and this is something you won't encounter. Ever. If you do, I'll eat my hat. :-P
Now, if you actually need to process a binary file in chunks (not "pretend it's text and write it on the screen"), that's why block buffering is for. Learn to use it. docs.python.org/3/library/io.html#... You're in fact implementing another buffer on top of a builtin one, which really doesn't help your memory nor your speed.
Let me just tell you one more thing. I do this all the time around the Net (explainxkcd.com/wiki/index.php/SIW...). Usually people stick to their guns and refuse to admit they are wrong. DEV is the only community where people thank me for correcting them. Kudos for that! B-)
Nobody is perfect. We are all learning to be better :D Though the post itself is not good enough, at least we had a meaningful conversation here and I get a better solution.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
It's not reason for ":(", it's for ":)". Because it means you can write normal-looking easy code and it will work.
Your example, even though fictional, has nothing to do with my comment: if you have to process files as a whole, then you do, and no amount of black magic will help you. If you don't, then the question is whether it's a text or a binary file, as I said. And then you should use line or block buffering as needed.
If you're really strapped for memory, the first optimization I'd suggest is not using Python. Python is so dynamic that common data structures easily take up many times more memory than in "normal" languages with value semantics.
I think I got what you mean here.
is fast and low memory consumed (Just learnt it from you and tried by myself, thanks)
And I agree with you that we should write the code as simple as possible in most cases, because having black magic here makes code less readable.
But I still think this technique is worth to mention and good to know, in case somebody needs it for some extreme cases.
Like what? Like "I have a binary file 4GiB in size and I'm just gonna spit it to stdout 1KiB at a time"? Not to mention that you don't do any decoding at all, so bytes objects are written to your screen raw, which isn't what you want, no matter the usecase. And not to mention "if it doesn't exist in the moment I check, I won't do anything, even though it might exist later when I'd actually try to read from it"?
Sorry, I know you're trying to salvage your post, but "like a pro" doesn't mean that. A pro should know the terrain of possibilities they might encounter, and this is something you won't encounter. Ever. If you do, I'll eat my hat. :-P
Now, if you actually need to process a binary file in chunks (not "pretend it's text and write it on the screen"), that's why block buffering is for. Learn to use it. docs.python.org/3/library/io.html#... You're in fact implementing another buffer on top of a builtin one, which really doesn't help your memory nor your speed.
You are right. Thanks a lot for these helpful comments. It’s definitely a good lesson learnt
Let me just tell you one more thing. I do this all the time around the Net (explainxkcd.com/wiki/index.php/SIW...). Usually people stick to their guns and refuse to admit they are wrong. DEV is the only community where people thank me for correcting them. Kudos for that! B-)
Nobody is perfect. We are all learning to be better :D Though the post itself is not good enough, at least we had a meaningful conversation here and I get a better solution.