DEV Community

Get advice while stuck on a project

Dan on September 12, 2022

Hi, Not sure if this is the correct place to ask or not, but said I would try! I am creating a web app using a quarkus server, I am at a part of t...
Read full post
Collapse
 
cicirello profile image
Vincent A. Cicirello •

Sorry that I'm not offering an answer to your question. This is more a suggestion on increasing chance that someone does....

Try adding the tag #help to your post. It is DEV's tag for posts that are, well, asking for help with something. Some users might follow that tag if they especially like being helpful. Or I sometimes scroll through recent #help posts in case I can answer something.

You might also add the #webdev tag since your question concerns a web app. I think that one is widely followed.

Collapse
 
ddold profile image
Dan •

Thanks for your suggestions

Collapse
 
tiguchi profile image
Thomas Werner •

It sounds like you want to associate an image upload with meta information. The way I go about it is by representing a file upload with a data structure that can be managed through an API endpoint.

Let's assume you call that data structure MediaFile or something like that. It can be created, queried and managed using a /media endpoint. You would start a new media file upload session by posting the file's meta information to the /media endpoint:

POST /media

{
    "name": "Some Example",
    "description": "About image",
    "expirationDate": ...,
    ...
}
Enter fullscreen mode Exit fullscreen mode

Your API would store that information in the database, and return the persisted version of that, together with its assigned ID property:

{
    "id": "1234",
    "status": "incomplete"
    ...
}
Enter fullscreen mode Exit fullscreen mode

You now have initialized a media file file object that needs to be completed and finalized by uploading the actual image data using a dedicated endpoint:

POST /media/1234/upload

...binary data....
Enter fullscreen mode Exit fullscreen mode

Ideally that endpoint can process partial file uploads, to keep upload request time consistent and lower the risk of server or client side timeout problems that usually occur with larger file sizes.

You can utilize the Content-Range HTTP header for specifying the byte range of the chunk you are uploading. Server side you would store that chunk as a temporary file.

Lastly you want to notify the backend that all chunks were uploaded, so it can merge the chunks together and do some final post-processing (e.g. stripping location meta data from uploaded images to protect user privacy, or scale image down):

POST /media/1234/finalize
Enter fullscreen mode Exit fullscreen mode

The response would be the completed media file upload, that also contains the download URL from your server:

{
    "id": "1234",
    "status": "complete",
    "url": "https://your-server.com/images/1234.jpg",
    "name": "..."
    ....
}
Enter fullscreen mode Exit fullscreen mode

The Twitter API has a very similar endpoint for uploading media files. You can take a look to get some inspiration for specifics:

developer.twitter.com/en/docs/twit...

As to where to store the file: I would definitely use the file system for that. It's superior to DBs in doing that because it's designed for quickly and efficiently storing and accessing files.

You can keep either a consistent file naming convention, where you can easily reconstruct the file path based on the object ID for example, or you could also store the file path in the DB. You can also utilize something like AWS S3 or similar to store your uploaded files.

Using the file system has also the advantage that you can serve those files using more conventional means using a web server like NGINX.

Collapse
 
ddold profile image
Dan •

Just so I am following you correctly.....
On the UI I am on a page where I fill in the name, description etc. and select a image to upload.
Once I click Save, there is a POST request sent to /media and it is just the MediaFile DataStructure?
Then this is saved in the DB and returned.
Once returned then the actual image is upladed to the endpoint /media/<MediaFile.id>/upload and saved in the filesystem. The MediaFile object is updated to include the saved location of the file and then it is returned.

Is that what your saying/suggesting?
Effectively there is two calls made to the endpoint until there is a success

Collapse
 
tiguchi profile image
Thomas Werner •

That's the idea, yes

Thread Thread
 
ddold profile image
Dan •

Brilliant, thanks so much for the advice, this has had me stuck for a long time!