I have been working with OCaml and contributing to the OCaml TIFF library since the 8th of December, 2025. While applying for the internship, I submitted what we envisaged as a timeline to complete the project at hand. My proposed timeline looked like this:
- Week 1-4 Orientation and Setup: Learning OCaml concepts using the Real World OCaml book, having a walkthrough on the project, and learning more about geospatial data and how it's managed in TIFF files.
- Week 5-6: Refactoring the code into low-level and high-level libraries
- Week 7-8: Building in cache support to enable writing TIFF files
- Week 9-10: Add advanced support for Geospatial data
- Week 11-12: Continuous work on the project, especially if goals were not attained during the internship scope
When I came up with this seemingly perfect plan to contribute to this project, I didn't yet have a full grasp of what I needed to know. So, over time, I had to adjust my approach to meet the project's goals.
How it all played out
As per the original plan, I spent the first few weeks learning OCaml by reading the Real World OCaml book and the OCaml Programming textbook. To avoid getting stuck in a cycle of continuous learning and reading, my mentor, Patrick Ferris, recommended that I go through the project and just read code while learning.
As someone new to the programming language, it might be difficult to understand anything at first, but the goal in itself is not to understand the entire project, but to become a little more familiar with how data flows and is manipulated throughout the project.
Getting familiar with the project
A good place to start when working on a new project is with the test files, and that is what I did.
The project's benchmark tests were initially used to measure the runtime of functions that read TIFF files using the Unix library, an interface to the Unix file system; however, the project also utilizes OCaml's Eio library. The Eio library is better suited for concurrent programs, and it provides a higher-level, cross-platform API. Since we used two backends, I updated the benchmark tests to compare the runtimes when reading the TIFF files with the Unix and Eio backends; PR: Add support for eio library.
Currently, the only way to know the contents of a TIFF file after reading it with the OCaml tiff library is through the tests. To effectively write and add new tests for the TIFF files, I used Python's tifffile library to generate TIFF files with specific features and then wrote new tests that assert that those values are read correctly by the ocaml-tiff library. The next step was to make sure ocaml-tiff interoperates with Raven. Raven provides OCaml with scientific computing capabilities and makes it easy to manipulate data. I swapped out the use of the owl-base library for Raven's Nx library in the test [PR: Change the use of owl-base to nx in tests].
The pluggable backends in Nx allow implementation for different hardware. It also provides better performance compared to Owl and is an equivalent of Python's Numpy. Reading the documentation for the various libraries helped me understand how they worked better and guided me on what to implement. I also found the OCaml Discourse forum very useful for finding answers to my questions.
Writing a TIFF File
TIFF is a tag-based file format for storing and interchanging raster images. Digital images come in two formats, raster and vector. Raster images are produced when scanning or photographing an object and are generally what you think about when you think of images.
To effectively write a TIFF file, it is important to understand the file structure. I read through the TIFF specification docs to get familiar with the different parts of the TIFF file, the kind of data stored in it, and what each part means.
The approach I used to write a TIFF file was to first successfully use an existing TIFF file to create a new one.
The first part of a TIFF file is the Image File Header.
- Bytes 0-1: a string that specifies the file's byte order: "II" for little-endian and "MM" for big-endian.
- A magic number used to identify the file as a TIFF file. 42 is used for a normal TIFF file, and 43 for Big TIFFs. If the file is a Big TIFF file, this value will occupy 6 bytes; otherwise, 2 bytes.
- The offset for the first Image File Directory(IFD), that is, the location in the file where the IFD is found, is the last thing stored in the header. 8 bytes are used in a Big TIFF, and 4 bytes for a simple TIFF.
In the File module, I added a new definition for the writer, File.wo, similar to that used for reading the TIFF.
type wo = file_offset:Optint.Int63.t -> Cstruct.t list -> unit
The definition specifies that to write to a file, the user needs to pass the file offset, that is, the location in the file at which the writing is to start, and a buffer containing the data to be written at that location.
Since the header is the first thing in the file, its offset is always zero. So, to write the header, I determined all the data that had to be stored in the header, stored it in a buffer, and then wrote that buffer to the file at offset=0.
let write_header wo header =
let buf = Cstruct.create 16 in
Cstruct.blit_from_string
(match header.byte_order with Endian.Little -> "II" | Endian.Big -> "MM")
0 buf 0 2;
(match header.kind with
| Tiff ->
Endian.set_uint16 ~offset:2 header.byte_order buf 42;
Endian.set_uint32 ~offset:4 header.byte_order buf
(header.offset |> Optint.Int63.to_int32)
| Bigtiff ->
Endian.set_uint16 ~offset:2 header.byte_order buf 43;
Endian.set_uint64 ~offset:8 header.byte_order buf
(header.offset |> Optint.Int63.to_int64));
wo ~file_offset:Optint.Int63.zero [ buf ]
I also defined functions in the Unix and Eio module that'll be used for writing. Since ocaml-tiff uses Unix and an Eio backend, defining File.wo provides an easy struct required by the various backends.
TIFF files are tag-based, and an IFD entry is used to store those tags and their values. The entry contains:
- Tag: A unique number used to identify the entry
- Field: A number indicating the datatype of the entry's value
- Count: Number of values in the entry
- Value/offset: Can contain the actual data if it is less than 4 bytes for a TIFF and 8 bytes for a Big TIFF. If the data is larger, it'll contain its location in the file instead. Possible values for the entry tag and field are found in the TIFF specs.
At this stage, I was using the IFD entries read from one file to write to the new file. Writing an entry whose value was not immediately stored in its IFD entry was quite challenging when copying the IFD to the new file. In such cases, I had to write the entry, then read the entry's data at the offset specified in the entry and write it to the file at that offset. It doesn't sound as complicated now, but it was a hassle when I was trying to figure it out :)
IFD entries are separate from the actual image data. The data is stored in strips, and an important IFD entry is the StripOffset, which contains the various locations of the strips of data. The image data must not come immediately after the IFD entries, so the offset values are used to read and write the data accurately. The StripByteCount is also used to determine the length of a strip at an offset location.
I used the same idea I described for writing the header to write the IFD entries and the image data to the file. The complete implementation for duplicating a TIFF file can be seen in this PR
The final step is to be able to make a TIFF file from scratch, which will be the discussion for the next post.
Important to Note - Gotcha Moments
- Most times, the StripOffset value would not fit into an IFD entry since it's larger than the required size. At first, I thought writing the values of strip offsets meant writing the image data; however, it's just the location of the actual image data. That means writing the StripOffset entry actually means writing at an offset the offsets of dataπ
- Optional arguments are best put in front; Never put your optional arguments at the end.
- Endianness specifies the byte order in the file; least significant byte to the most significant byte for little-endian, and most significant byte to least significant byte for big-endian. The file is either a Big TIFF file or a TIFF file, depending on the magic number in the header and not on the endian values.
Top comments (0)