Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris
If you are you completely new to Node.js or maybe you've just spun up an Express app in Node.js, but barely know anything else about Node - Then this first part in a series is for YOU.
In this part we will look at:
- Working with file paths, it's important when working with files and directories that we understand how to work with paths. There are so many things that can go wrong in terms of locating your files and parsing expressions but Node.js does a really good job of keeping you on the straight and narrow thanks to built-in variables and great core libraries
- Working with Files and Directories, almost everything in Node.js comes in an async, and sync flavor. It's important to understand why we should go with one over the other, but also how they differ in how you invoke them.
- Demo, finally we will build some demos demonstrating these functionalities
The file system
The file system is an important part of many applications. This means working with files, directories but also dealing with different access levels and paths.
Working with files is in Node.js a synchronous or an asynchronous process. Node.js is single-threaded which means if we need to carry things out in parallel we need an approach that supports it. That approach is the callback pattern.
References
- Node.js docs - file system This is the official docs page for the file system
-
Overview of the fs module
Good overview that shows what methods are available on the
fs
module - Reading files Shows all you need to know about reading files
- Writing files Docs page showing to how to writ files
- Working with folders Shows how to work with folders
- File stats If you need specific information on a file or directory like creation date, size etc, this is the page to learn more.
- Paths Working with paths can be tricky but this module makes that really easy.
- Create a Node.js app on Azure Want to know how to take your Node.js app to the Cloud?
- Log on to Azure programmatically using Node.js This teaches you how to programmatically connect to your Azure resources using Node.js
Paths
A file path represents where a directory or file is located in your file system. It can look like this:
/path/to/file.txt
The path looks different depending on whether we are dealing with Linux based or Windows-based operating system. On Windows the same path might look like this instead:
C:\path\to\file.txt
We need to take this into account when developing our application.
For this we have the built-in module path
that we can use like so:
const path = require("path");
The module path
an help us with the following operations:
- Information, it can extract information from our path on things such as parent directory, filename and file extension
- Join, we can get help joining two paths so we don't have to worry about which OS our code is run on
- Absolute path, we can get help calculating an absolute path
- Normalization, we can get help calculating the relative distance between two paths.
Demo - file paths
Pre-steps
- Create a directory for your app
-
Navigate to your directory
cd <name of dir>
-
Create app file, Now create a JavaScript file that will contain your code, the suggestion is
app.js
-
Create file we can open, In the same directory create a file
info.txt
and give it some sample data if you want
Information
Add the following code to your created app file.
const path = require("path");
const filePath = '/path/to/file.txt';
console.log(`Base name ${path.basename(filePath)}`);
console.log(`Dir name ${path.dirname(filePath)}`);
console.log(`Extension name ${path.extname(filePath)}`);
Now run this code with the following command:
node <name of your app file>.js
This should produce the following output
Base name file.txt
Dir name /path/to
Extension name .txt
Above we can see how the methods basename()
, dirname()
and extname()
helps us inspect our path to give us different pieces of information.
Join paths
Here we will look into different ways of joining paths.
Add the following code to your existing application file:
const join = '/path';
const joinArg = '/to/my/file.txt';
console.log(`Joined ${path.join(join, joinArg)}`);
console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)
Above we are joining the paths contained in variables join
and joinArg
but we are also in our last example testing out concatenating using nothing but directory names and file names:
console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)
Now run this using
node <name of your app file>.js
This should give the following output:
Joined /path/to/my/file.txt
Concat /path/user/files/file.txt
The takeaway here is that we can concatenate different paths using the join()
method. However, because we don't know if our app will be run on a Linux of Windows host machine it's preferred that we construct paths using nothing but directory and file names like so:
console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)
Absolute path
Add the following to our application file:
console.log(`Abs path ${path.resolve(joinArg)}`);
console.log(`Abs path ${path.resolve("info.txt")}`);
Now run this using
node <name of your app file>.js
This should give the following output:
Abs path /to/my/file.txt
Abs path <this is specific to your system>/info.txt
Note, how we in our second example is using the resolve()
method on info.txt
a file that exist in the same directory as we run our code:
console.log(`Abs path ${path.resolve("info.txt")}`);
The above will attempt to resolve the absolute path for the file.
Normalize paths
Sometimes we have characters like ./
or ../
in our path. The method normalize()
helps us calculate the resulting path. Add the below code to our application file:
console.log(`Normalize ${path.normalize('/path/to/file/../')}`)
Now run this using
node <name of your app file>.js
This should give the following output:
Normalize /path/to/
Working with Files and Directories
There are many things you can do when interacting with the file system like:
- Read/write files & directories
- Read stats on a file
- Working with permissions
You interact with the file system using the built in module fs
. To use it import it, like so:
const fs = require('fs')
I/O operations
Here is a selection of operations you can carry out on files/directories that exist on the fs
module.
-
readFile()
, reads the file content asynchronously -
appendFile()
, adds data to file if it exist, if not then file is created first -
copyFile()
, copies the file -
readdir()
, reads the content of a directory -
mkdir()
, creates a new directory, -
rename()
, renames a file or folder, -
stat()
, returns the stats of the file like when it was created, how big it is in Bytes and other info, -
access()
, check if file exists and if it can be accessed
All the above methods exist as synchronous versions as well. All you need to do is to append the Sync
at the end, for example readFileSync()
.
Async/Sync
All operations come in synchronous and asynchronous form. Node.js is single-threaded. The consequence of running synchronous operations are therefore that we are blocking anything else from happening. This results in much less throughput than if your app was written in an asynchronous way.
Synchronous operation
In a synchronous operation, you are effectively stopping anything else from happening, this might make your program less responsive. A synchronous file operation should have sync as part of the operation name, like so:
const fileContent = fs.readFileSync('/path/to/file/file.txt', 'utf8');
console.log(fileContent);
Asynchronous operation
An Asynchronous operation is non-blocking. The way Node.js deals with asynchronous operations is by using a callback model. What essentially happens is that Node.js doesn't wait for the operation to finish. What you can do is to provide a callback, a function, that will be invoked once the operation has finished. This gives rise to something called a callback pattern.
Below follows an example of opening a file:
const fs = require('fs');
fs.open('/path/to/file/file.txt', 'r', (err, fileContent) => {
if (err) throw err;
fs.close(fd, (err) => {
if (err) throw err;
});
});
Above we see how we provide a function as our third argument. The function in itself takes an error err
as the first argument. The second argument is usually data as a result of the operation, in this case, the file content.
Demo - files and directories
In this exercise, we will learn how to work with the module fs
to do things such as
- Read/Write files, we will learn how to do so in an asynchronous and synchronous way
- List stats, we will learn how to list stat information on a file
- Open directory, here we will learn how to open up a directory and list its file content
Pre-steps
- Create a directory for your app
-
Navigate to your directory
cd <name of dir>
-
Create app file, Now create a JavaScript file that will contain your code, a suggestion is
app.js
-
Sample file, In the same directory create a file
info.txt
and give it some sample data if you want -
Create a sub directory with content, In the same directory create a folder
sub
and within create the filesa.txt
,b.txt
andc.txt
Now your directory structure should look like this:
app.js
info.txt
sub -|
---| a.txt
---| b.txt
---| c.txt
Read/Write files
First, start by giving your app.js
file the following content on the top:
const fs = require('fs');
const path = require('path');
Now we will work primarily with the module fs
, but we will need the module path
for helping us construct a path later in the exercise.
Now, add the following content to app.js
:
try {
const fileContent = fs.readFileSync('info.txt', {
encoding: 'utf8'
});
console.log(`Sync Content: ${fileContent}`);
} catch (exception) {
console.error(`Sync Err: ${exception.message}`);
}
console.log('After sync call');
Above we are using the synchronous version of opening a file. We can see that through the use of a method ending in sync.
Follow this up by adding the asynchronous version, like so:
fs.readFile('info.txt', (err, data) => {
if (err) {
console.log(`Async Error: ${err.message}`);
} else {
console.log(`Async Content: ${data}`);
}
})
console.log('After async call');
Now run this code with the following command:
node <name of your app file>.js
This should produce the following output
Sync Content: info
After sync call
After async call
Async Content: info
Note above how the text After sync call
is printed right after it lists the file content from our synchronous call. Additionally note how text After async call
is printed before Async Content: info
. This means anything asynchronous happens last. This is an important realization about asynchronous operations, they may be non-blocking but they don't complete right away. So if the order is important you should be looking at constructs such Promises and Async/await.
List stats
For various reasons, you may want to list detailed information on a specific file/directory. For that we have stat()
method. This also comes in an asynchronous/synchronous version.
To use it, add the following code:
fs.stat('info.txt', (err, stats) => {
if (err) {
console.error(`Err ${err.message} `);
} else {
const { size, mode, mtime } = stats;
console.log(`Size ${size}`);
console.log(`Mode ${mode}`);
console.log(`MTime ${mtime}`);
console.log(`Is directory ${stats.isDirectory()}`);
console.log(`Is file ${stats.isFile()}`);
}
})
Now run this code with the following command:
node <name of your app file>.js
This should produce the following output
Size 4
Mode 33188
MTime Mon Mar 16 2020 19:04:31 GMT+0100 (Central European Standard Time)
Is directory false
Is file true
Results above may vary depending on what content you have in your file info.txt
and when it was created.
Open a directory
Lastly, we will open up a directory using the method readdir()
. This will produce an array of files/directories contained within the specified directory:
fs.readdir(path.join(__dirname, 'sub'), (err, files) => {
if (err) {
console.error(`Err: ${err.message}`)
} else {
files.forEach(file => {
console.log(`Open dir, File ${file}`);
})
}
})
Above we are constructing a directory path using the method join()
from the path
module, like so:
path.join(__dirname, 'sub')
__dirname
is a built-in variable and simply means the executing directory. The method call means we will look into a directory sub
relative to where we are executing the code.
Now run this code with the following command:
node <name of your app file>.js
This should produce the following output
Open dir, File a.txt
Open dir, File b.txt
Open dir, File c.txt
Summary
In summary, we have covered the following areas:
-
Paths, we've looked at how we can work with paths using the built-in
path
module -
Files & Directories, we've learned how we can use the
fs
module to create, update, remove, move etc files & directories.
There is lots more to learn in this area and I highly recommend looking at the reference section of this article to learn more.
Top comments (6)
Amazing explanation! I was looking for this the other day.
Thank you for that :) Happy it's helpful
This is very easy to understand, thanks.
Thank you Music. Glad it helps
Really helpful and so well explained 😁
Thank you, Charlie. Happy it's helpful