BuckleScript best practice: public and private modules

IN A previous post I mentioned that 'OCaml doesn't really have a way to hide file modules'. Turns out that at the compiler level, this is accurate; but at the build system level, not quite as accurate.

Hiding a module interface

OCaml power users have long been able to maintain private modules in their projects by taking advantage of the fact that the compiler can only 'see' modules that publish a .cmi (compiled interface) file. A module's .cmi file is an efficient intermediate representation of the module's interface, that the compiler uses to figure out what's in the module during the build. If there is no .cmi, the compiler can't 'see' the module at all.

In BuckleScript

You can take advantage of this nifty trick in BuckleScript to keep as many private modules as you like. The BuckleScript build system has a feature that allows you to list the 'public' modules of your project; once this is done BuckleScript will enforce that only those modules' .cmi files are published and visible to consumer projects. (The build system for native OCaml, dune, also offers a similar feature.)

The way it works is, you list the public modules in your bsconfig.json file, in the public field inside the sources field. For example:

{
  "name": "@myname/myproject",
  "version": "0.1.0",
  "sources": [
    {
      "dir": "src",
      "subdirs": true,
      "public": ["Myname_Myproject"]
    }
  ],
  ...
  "warnings": {
    "error" : "+49+101"
  },
  ...
}

This means that the src subdirectory is considered by BuckleScript to be a source directory, all modules found by recursively walking the directory hierarchy will be compiled, and that the public modules (in this case just the one) are the ones listed. I'll explain the warnings field later.

A bit more detail about how public behaves:

• If it's not present, BuckleScript assumes all modules are public
• If it is present, only the listed modules are public

Modular FTW

You may be thinking, won't we usually have a lot of public modules we want to list here? Won't this get messy? Well, this is where the modular project structure pays off! There you saw how to set up a single toplevel module that aliases all the modules that you want to publish for your users. Remember, the aliases look like this:

/** [src/Myname_Myproject.re] - this is the toplevel module documentation. */

/** Module-level documentation for the [Add] module */
module Add = Myname_Myproject_Add;

/** Module-level documentation for the [Subtract] module */
module Subtract = Myname_Myproject_Subtract;

/** Module-level documentation for the [Multiply] module */
module Multiply = Myname_Myproject_Multiply;

/** Module-level documentation for the [Divide] module */
module Divide = Myname_Myproject_Divide;

By virtue of these aliases, the implementation modules Myname_Myproject_Add, and so on will also automatically be published. So again, you usually just need to list the one toplevel module in your public array. Any modules reachable (by alias or by implementation) from that toplevel module, whether now or in future, will get the same benefit. (You may want to publish multiple toplevel modules in rare cases.)

Useful private modules

If you can't immediately think of what kinds of modules you'd want to keep private, a couple of useful ones are test and demo modules. You can put your unit tests in the same src/ directory hierarchy as the runtime code itself, and they will be completely hidden from users. You can build out a complete demo app in the same src/ directory structure as the library you're developing–again, no one will be able to access it.

For example, tests:

/** [src/Myname_Myproject/__tests__/Myname_Myproject_Add_Test.re] */

module Add = Myname_Myproject_Add;

let () = {
  assert(Add.int(1, 1) == 2);
  assert(Add.float(1., 0.) == 1.);
};

Naming strategy

Notice that I used the same full file naming conventions as for the public modules. Private modules still need to be unambiguously named within their projects. You don't want the possibility of a clash with another module within the same project.

Of course if you're confident that the private modules are completely unambiguous, then you can give them shorter names. For example that's what I've done in my work-in-progress project for Hyperapp bindings, re-hyperapp. The Demo.re module and Demo subdirectory are unambiguous within the project, so they get shorter names. The Yawaramin_ReHyperapp module is public, so it gets a fully-qualified name. (The ReactDOMRe and ReasonReact modules are also public, but they need those exact names, because they're shims to make Reason's JSX work with Hyperapp.)

The no cmi file warning–and error

In practice, as I mentioned earlier, BuckleScript uses your build specification to 'hide' the private modules by simply not outputting their .cmi files. When a consumer project tries to reference a private module, they will get a compile warning by default:

$ bsb -w
>>>> Start compiling
[2/2] Building src/Main.mlast.d
[1/1] Building src/Main.cmj

  Warning number 49
  (No file name)

  no cmi file was found in path for module MyPrivateModule

And when they try to actually access the contents of a private module, they'll get the usual module or file MyPrivateModule can't be found compile error. In other words, there's no actual way to use the module.

While this will prevent obvious errors, it would be somewhat better if BuckleScript would make warning 49 an error by default, so that users could not even refer to a private module. Users can of course try out the error for themselves, by setting the warnings field in their projects as I show above. This will turn the warning into a compile error:

>>>> Start compiling
[2/2] Building src/Main.mlast.d
[1/1] Building src/Main.cmj

  Warning number 49
  (No file name)

  no cmi file was found in path for module MyPrivateModule

  We've found a bug for you!
  /Users/yawar/src/public-test/src/Main.re

  Some fatal warnings were triggered (1 occurrences)

>>>> Finish compiling(exit: 1)

This error pinpoints the source file and forces a fix on the spot.

Privacy in the wild

Overall, while not as elegant as its module-and-interface system, OCaml's private modules feature is quite useable. Certainly, outside of quite industrial-scale languages like Java and Scala (and the Standard ML Basis system), it's difficult to think of any languages which offer a 'private modules' feature. In my opinion it's another of the signs of how ready OCaml (and ReasonML) is as an 'industrial-strength' language.