Hey everyone! I’ve been learning more deeply about Node.js recently, and instead of studying it in isolation, I wanted to use it to solve a real pr...
For further actions, you may consider blocking this person and/or reporting abuse
Did exactly the same thing for a project of mine - backend application written in Laravel (running on AWS Lambda), image compression by a separate Lambda function written in JS, using Sharp (triggered via an S3 image upload event) ... !
What a coincidence :-)
What a small world! Great minds think alike. Lambdas for image processing are just so efficient and cost-effective. How’s the performance of Sharp treating you @leob ? I’ve been loving how fast it handles the compression.
Small world indeed - and yes, Sharp works well !
I always love this kind of development, it's not a A vs B, but A + B. Understanding the strength and weakness of any tool makes you a great engineer. I have a similar case too, not related to image compression though.
It was an async heavy operation, I had to move the feature into nodejs, the laravel and nodejs communicates seemlessly. We can also see that in things like broadcasting with Pusher.
Laravel code organization and maintainability is what keeps making me to use it.
Exactly @sadiqsalau! Choosing the right tool for the right job is key. I completely agree with the 'A + B' mindset. Laravel’s DX (Developer Experience) and structure are unmatched, but knowing when to delegate makes all the difference.
This post reads to me like a case of I want to apply my new knowledge as soon as possible. While at the base it is a good reaction. The most important question you should ask yourself is, does the application needs this?
This is the initial problem. You mentioned in the post that one of the main things is resizing the images. I assume because huge images are not needed.
So instead of allowing huge images, why not halt images that are too big from being uploaded?
This saves resources on the server with a minimum amount of code changes.
My question here is, when will the user that uploaded the image will be able to see or use it?
If it is an image that is a part of something, a gallery for example, are you going to keep that thing unpublished until the image has gone through the queue?
Having async flows has effects on the user experience, server resources are not the main driver of an application.
It is good you mention the trade-offs, there are consequences to every decision.
The main thing I want to communicate is apply new knowledge when it is the best solution for the application. Not because it is fresh in your mind. It is a trap I caught myself in more than a few times. The method that works for me not falling into the trap, is building a few experiments and then move on.
Hi @xwero . Thank you for asking. Great questions. Allow me to answer.
Why not halt images that are too big from being uploaded - I agree with you. And of course, I already put some validation here. However, because multiple images in one post might take more space, this is where the problem happens. Even if I limit the image uploading to 1MB per image, and one post needs 20 to 30 images, it could take 20 to 30MB. We have thousands of articles, posts, and other things that need a gallery upload. Therefore, image compression size is needed here. At the same time, we keep the quality of the images. I'm converting all image formats to WebP and keeping 80% of the image quality.
When will the user who uploaded the image be able to see or use it - They can see it right away. What I do for this part is, first, we keep all the original images. Then, in the background, we are queuing the image compression. After finishing, we remove the original image and replace it with the compressed version. Before I implement this method, I put a skeleton image, but it seems not user-friendly. Admin wanted to see it right away. So this is what I do. The process is lighthing fast.
What about yours? What is the trap you're facing, and did you solve it? I also wanted to learn from your experience.
Now you are just changing the parameters of the use case. If you have a 1 MB upper limit, why would you need to resize the images? In your example the resizing is done by setting a maximum width, and that width guarantees a filesize you can live with to serve the users of the application.
When you already set that size at 1 MB, it doesn't matter if a gallery or a post has one image or thirty. The filesize of the images it not going to change that much by resizing because that is the median filesize you know the files are after resizing to the preferred maximum width.
Sure setting the image quality and converting the image to a more suitable webformat helps bringing down the filesize, but they are secondary performance tweaks.
I guess this answer is related to the 1 MB upload limit. Because if it isn't, do you really want to serve 30 - 50 MB images on admin pages?
If you want a more performant way than Imagick there is php-vips which is based on the same binary as Sharp. But the question here is do you use Shap because you want to use a Node library or are the PHP solutions really the bottleneck?
Sharp is only the best solution when it is the latter case.
You know how compression works, right?
A 1 MB image can be reduced to less than 200 KB. Has that not changed much? One post has 30 images. I have 1000 posts. Calculate for me how much space I save by compressing the images. Not to mention the bandwidth saved when people visit our website.
And why limit the option to do the compression to PHP when I can do it better with NodeJS? Of course, I chose Node.js because of the bottleneck. Otherwise, I can use another PHP or Laravel package to solve this.
Again, what is your trap, and what is your solution for your trap? I am waiting for your cases for me to understand also. I might use your solution if it solves my problem, which is to reduce the filesize minimum as we can and to handle the bottleneck.
The final reduction was not the point of my questions. What is the reason that made you choose 1 MB as the upload limit? In the post you are mentioning phone and design images, those are not 1 MB. So they already went through a compression process that was executed by the uploader.
Are you sure? Did you know about the library I linked in my comment? From another comment you made it seems the only verification you did is reading the Sharp documentation.
The trap is not exploring all the possible options. I found the PHP libvips library by clicking on the libvips link in the Sharp documentation. And in the libvips repository README I found a list of language libraries.
Sharp is not even on that list. That is why I question if the path you choose is the best one.
I don't comment to pick a fight, I comment because those are the questions I would ask myself when I need to do find a solution.
Someone shared this PHP package with me (github.com/Intervention/image-driv...). Before I use NodeJS Slash, I'm thinking about the PHP Native image built-in function (ImageMagick). I expect the performance is not on par with the Slash. Since Slash mentioned in their documentation that they outperform ImageMagick 4-5x times.
After I look at the intervention docs, I think I will do some research and testing.
But overall, my sharing is not about "NodeJS vs PHP".. Or "NodeJS beats PHP". It's just who can give me the best compression with the best performance.