I always felt confused by the terms “Blob, Buffer, ArrayBuffer, Uint8Array.”
Everyone has their own definition, but no one can clearly define them. It always ends up in a nonsensical technical explanation.
Today, I’ll put a human-readable definition on these terms. This will help my future self and other engineers to have a better understanding.
Let me introduce you to the strange world of memory management in JavaScript.
Before we dive into the subject — if you have any questions during the lecture — feel free to ask me on Twitter: @artiumWs
Concepts
Before starting, let’s define some basic concepts: Memory, Memory Span, Binary Data, and Raw Data.
Memory
Memory (RAM) is fast storage used by a computer to quickly access and store data.
Memory span
A memory span refers to a contiguous block of memory.
If memory is a street, a memory span would be neighbouring houses.
Binary data
Binary data is the representation of information in binary form — bunch of 0 and 1
Raw Data
Raw data is the unprocessed information consumed by a program — text, images, files …
Keep these definitions in mind. They will be useful for understanding the following sections.
In-code object
Blob object
A Blob is an immutable object referencing raw data.
This is an object created in your code that contains raw data information like size in bytes and MIME type (image/png, application/pdf, application/json, …).
File object
A File extends from Blob properties and is created when using an input element.
It’s a Blob with more information like: last modified date and file name. This object is automatically created when a file is uploaded to your app using an input element or a drag-and-drop event.
Those object cannot be modified directly! They are the in-code reference to the raw data.
This is to ensures their integrity when manipulating them.
Manipulation
Those objects provide two methods to access the data: arrayBuffer and stream
arrayBuffer
arrayBuffer is method that load the binary data of the object in a memory span and return a in-code representation of a memory span.
This representation cannot be modified directly! We need to reference it first with a typed array to access its binary data. Typed arrays are defined later.
Caution: It loads the whole file in your memory at once. In case the file size is to big (>5Mb), this operation can freeze your app. In such case, use the next method.
stream
stream is a method that load the binary data in chunks. This process is called buffering.
Chunks returned are directly typed arrays so they can be directly manipulated to process you data on fly.
Typed array
Typed array (uInt8Array, int16array …) references the arrayBuffer to access and modify its binary data.
Multiple typed array can reference the same arrayBuffer. In such cases, the latest modification will overwrite the previous ones.
Bonus
Buffer (Node.Js)
Buffer is a Node.js object that extend from the typed array properties. It is designed to deal with communication features.
Use case:
Buffers are designed to handle communication logic.
eg: Optimise data send through requests.
Typed arrays are designed to handle processing logic.
eg: Adding a filter to an image
Recap
Let’s recap this article through the following use case: “Uploading an image on a web app”
- An image is stored on the your computer (hard drive, SSD, …)
- Upload this image to your web app using an
<input type="file">element - The browser creates a
Fileobject that references the image.
4.a Use the arrayBuffer method to load the image into memory span and reference it
5.a Attach a typed array to this arrayBuffer to read or write binary data
6.a Process the binary data
4.b Use the stream method to read chunk of stored image as a typed array
5.b Process the binary data chunks
I hope this article helps you gain a clearer understanding of the topic.
If you have any questions, feel free to ask me on Twitter: @artiumWs
Sources
https://nodejs.org/api/buffer.html
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Typed_arrays
https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts
https://developer.mozilla.org/en-US/docs/Web/API/Streams_API
https://developer.mozilla.org/en-US/docs/Web/API/File
https://developer.mozilla.org/en-US/docs/Web/API/Blob
https://stackoverflow.com/questions/11821096/what-is-the-difference-between-an-arraybuffer-and-a-blob#comment41489269_11821109
https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream
Top comments (0)