If you’re new to Julia, check out the Learn Julia crash course 🚀
Although you may not have heard of it, the Julia programming language has caused a stir in the development community. Born in 2012, the speedy and easy-to-use language has recently become a favorite among machine learning and AI researchers, some of which are wondering if this could be the underdog that soon usurps Python.
Regardless of Python’s fate, the fact remains that Julia’s popularity will continue to grow as AI continues to weave its way into every aspect of our lives. To that end, I wanted to write a few introductory articles on Julia to help newcomers get up to speed as fast as possible. In this article, I will be exploring the difference between two files that are integral to Julia’s package management system: Manifest.toml and Project.toml.
Intro to Pkg
Like many other popular development languages, Julia comes with its own built-in package manager, Pkg, which you can use to add, remove, and update packages used in your project. Unlike many other languages however, Julia’s Pkg was built with the concept of environments baked in. An environment is simply a set of packages that can be used by a project. You can read more about why this approach is tactful here, but in summary, it allows you to build projects without having to worry what your global dependency set looks like. Each project can have its own set of packages with their own versions. Likewise, projects can share an environment if you have a trusty set of packages you find yourself using all the time.
When you first start using Julia, a default environment named after your Julia version is used to manage your packages.
# Find your Julia version in the terminal by running this
julia --v
# Output should look like this: julia version 1.8.5
You can see the environment being used by accessing Pkg in the REPL interface by typing “]”. In my case, I’m using environment v1.8.
Create a New Environment
There are a few ways to create a new environment and which one you choose will depend on how far along your project is.
If you haven’t started yet, you an create a new project using the Pkg interface. Simply access Pkg by typing “]” and then type “generate {ProjectName}”. When the new project is created, it will contain a new Project.toml file. You can cd into the new directory and activate the new environment following these steps:
Generate the new project (generate Test)
Enter the shell REPL (;)
Enter the new project directory (cd Test)
Enter the Pkg REPL (])
Activate the environment (activate .)
You’ll notice in the above screenshot that the environment in parenthesis changes after the new environment is activated.
We can make the process of activating a new environment even faster by skipping the step where we navigate into the new directory. Julia easily lets you specify the path to the environment you want at the command line (ex. “activate ./Test”).
It doesn’t take much imagination to see how this could be a convenient way to test out newer versions of packages without needing to replace your current environment.
What is Project.toml?
The Project.toml file is a file that lists the direct dependencies of a project along with some other information, including the project’s authors, version, and name. If you’ve used Flutter before, this file is similar to the Pubspec.yaml. Without any packages, an empty Project.toml file will look like this:
name = "Test"
uuid = "076e85b2-218e-40bb-815d-431d02e4467a"
authors = ["Joseph Muller <jtmuller5@gmail.com>"]
version = "0.1.0"
When you add packages to the project using the Pkg REPL, they’ll be added underneath the metadata:
name = "Test"
uuid = "076e85b2-218e-40bb-815d-431d02e4467a"
authors = ["Joseph Muller <jtmuller5@gmail.com>"]
version = "0.1.0"
[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
Each package in the Julia package ecosystem has its own UUID which will appear in the Project.toml next to the package name. These values are assigned to the package when it is regestered in the Julia General registry and they never change, even if the version of the package does.
What is Manifest.toml?
If you’ve used other package managers before, you may be wondering where the package versions are stored. While the Project.toml stores the IDs of the project’s direct dependencies, the Manifest.toml tracks the entire dependency tree, including both the direct and indirect dependencies and the versions of each. Because of this, the Manifest.toml is always a larger file. For example, after adding just the CSV package to a fresh project, my Manifest.toml is already 200 lines long!
When a new project is first created, the Manifest.toml is absent. Its only added when the first package is added. If you remove all dependencies at a later date, the Manifest.toml will remain but it will be completely empty.
In general, the Manifest.toml should not be manually edited as it is managed by Julia’s package manager. To change the desired version of a package, you can instead type the following command into Pkg:
add PackageName@vX.Y.Z
The available package versions can generally be found on the package’s home page. The versions for the CSV package can be found on the Releases page, for example.
Bringing it All Together
The official Julia docs succinctly explain why these two files are essential:
Given a Project.toml + Manifest.tomlpair, it is possible to instantiate the exact same package environment, which is very useful for reproducibility.
On this note, both files should be included in version control (VC) for normal projects. When another developer pulls your code, that can very quickly stand up an identical environment to the one you were using and start working.
If instead you are developing your own package for others to use, the Manifest.toml should not be included in VC since users will each have their own dependency trees.
Up Next
Keep an eye on the AI Co-Founded blog for new Julia articles. I will be documenting as much of my learning process as possible there, as well as on Medium and dev.to. Happy coding!
Top comments (0)