DEV Community

sangjun_park
sangjun_park

Posted on • Edited on

Basic Linux for AI Developers

1. Brief Introduction to Linux

Linux is a crucial OS for AI development due to its open-source nature, stability, efficiency, and compatibility with AI frameworks. It offers scalability, strong community support, and cost-effectiveness, making it an ideal platform for AI projects from development to deployment.

2. Essential Linux Commands

There are lots of commands in linux, However, I'll introduce something that is used most commonly.

Navigating the file system: ls, cd, pwd & permission command: chmod, chown

ls

the 'ls' command in Linux stands for "list" and is used to display the contents of a directory. It shows the files, directories, and sometimes additional details such as file permissions, size, and modification date.

Here are some example of basic structure of files and directories.

project/
├── data/
│   ├── dataset.csv
│   ├── processed_data/
│   └── raw_data/
├── models/
│   └── model_v1.h5
├── scripts/
│   └── train.py
├── .gitignore
├── README.md
└── venv/
    ├── bin/
    ├── include/
    ├── lib/
    └── pyvenv.cfg
Enter fullscreen mode Exit fullscreen mode

If we type 'ls' in this case,

ls
Enter fullscreen mode Exit fullscreen mode

It will show like this.

$ ls 
data models scripts .gitignore README.md venv
Enter fullscreen mode Exit fullscreen mode

Then, we'll now use ls -l command to watch some detailed view of the directories and files.

$ ls -l 
drwxr-xr-x 2 user user 4096 Sep 21 12:34 data 
drwxr-xr-x 2 user user 4096 Sep 21 12:35 models 
drwxr-xr-x 2 user user 4096 Sep 21 12:36 scripts 
-rw-r--r-- 1 user user 220 Sep 21 12:33 .gitignore 
-rw-r--r-- 1 user user 345 Sep 21 12:33 README.md 
drwxr-xr-x 6 user user 4096 Sep 21 12:37 venv
Enter fullscreen mode Exit fullscreen mode

In Linux, file permissions are represented by string of 10 characters that control who can read, write, or execute a file or directory.

drwxr-xr-x

# we can separate this by
1 3.  3.  3.  chars
d rwx r-x r--
Enter fullscreen mode Exit fullscreen mode

  • First character(d) represents the type of file.

    • d: directory
    • -: regular file (non-directory)
    • l: symbolic link (special type of file that acts as a reference or pointer to another file or directory)

  • Next three characters(rwx) represents the permissions for the owner.

    • r: stands for read permission
    • w: stands for write permission
    • x: stands for execute permission (for directories, it means the ability to list or traverse the directory)

  • Next three characters(r-x) represents the permissions for the group.

    • r, w, x: same meaning as I explained it before
    • -: no permission. In this case, there are '-' in second one so it means it doesn't have any write permission.

  • Final three characters(r--) represents the permissions for others.

    • In this case, the others have only read permission and don't have any write or execute permission.
chmod

this command stands for "change mode". It's a command used in Unix and Unix-like operating systems to change the access permissions of file system objects(files and directories).

chmod allows to set permissions for three categories of users. The 'owner', 'group', and 'others'. And for each category, you can set three types of permissions which is called Read(r), Write(w), and Execute(x).

The method of using chmod has two types of things. The first one is symbolic method, and the second one is numeric method.

The format of symbolic method is like this.

Format: chmod [who] [operation] [permissions] filename
Enter fullscreen mode Exit fullscreen mode
  • Who: u (user/owner), g (group), o (others), a (all)
  • Operation: + (add), - (remove), = (set exactly)
  • Permissions: r (read), w (write), x (execute)

For example,


chmod u+x file.sh

(Add execute permission for the owner in file.sh)


chmod go-rw file.txt

(Remove read and write permissions for group and others)

The format of numeric method is like this.

Format: chmod [assigned_3_numbers] filename
Enter fullscreen mode Exit fullscreen mode

Each permission is assigned a number:

  • Read (r) = 4
  • Write (w) = 2
  • Execute (x) = 1

And sum these numbers for each category(owner, group, others).

For example,


chmod 755 file.sh

- **Owner:** 7 (4+2+1 => read, write, and execute)
- **Group:** 5 (4+1 => read, execute) 
- **Others:** 5 (4 + 1 => read, execute)
Enter fullscreen mode Exit fullscreen mode

There are some command like this.


chmod -R 755 directory

(Recursively set permissions for a directory and its contents)

chown

this command stands for "change owner". It's a command used in Unix and Unix-like OS to change the owner and group of files and directories. "chown" command allows to change the user owner, or the group owner, or both the user and group owners simultaneously.

The basic syntax of the chown command is like this:

chown [options] <owner>:<group> <file/directory>`
Enter fullscreen mode Exit fullscreen mode
  • <owner>: The new owner of the file or directory.
  • <group>: (Optional) The new group for the file or directory.
  • <file/directory>: The file or directory whose ownership you want to change.
  • [options]: additional flags or parameters that modify the behavior of the command.

Here are some of the examples that could explain chown well.

example 1: Change the Owner of a Single File

sudo chown user1 file1.txt
Enter fullscreen mode Exit fullscreen mode

This command changes the owner of file1.txt to user1 while keeping the group unchanged.

example 2: Change Both Owner and Group of a File

sudo chown user1:group1 file1.txt
Enter fullscreen mode Exit fullscreen mode

This changes the owner of file1.txt to user1 and the group to group1.

example 3: Change Ownership of All Files in a Directory Recursively

sudo chown -R user1:group1 /home/user1/
Enter fullscreen mode Exit fullscreen mode

This command changes ownership for all files and subdirectories within /home/user1/, making user1 the owner and group1 the group for every item inside the directory.

example 4: Change Ownership Using a Reference File

sudo chown --reference=ref_file.txt target_file.txt
Enter fullscreen mode Exit fullscreen mode

This command changes the ownership of target_file.txt to match the ownership of ref_file.txt. It's useful if you want to apply the same ownership attributes as a reference file.

example 5: Suppress Errors During Ownership Change:

sudo chown -f user1:group1 /some/protected/file
Enter fullscreen mode Exit fullscreen mode

-f option: forces the command to ignore errors, such as permission issues, and completes the operation without displaying error messages.

example 6: Verbose Output to See What's Being Changed:

sudo chown -v user1:group1 file1.txt
Enter fullscreen mode Exit fullscreen mode

This command displays a message for each file whose ownership is changed, providing a clear overview of the modification.

File and directory manipulation: mkdir, touch, cp, mv, rm

mkdir

The command 'mkdir' stands for "make directory". It's used in command-line interfaces to create a new directory (folder) in a file system. The purpose of 'mkdir' is to create one or more new directories. If you want to use 'mkdir', just typically type 'mkdir' followed by the name of the directory named "new_folder" in your current location.

There are several ways to make directory with using "mkdir".

1. Create a single directory

mkdir documents
Enter fullscreen mode Exit fullscreen mode

2. Create multiple directories at once:

mkdir photos videos music
Enter fullscreen mode Exit fullscreen mode

3. Create a directory with spaces in the name:

mkdir "My Projects"
Enter fullscreen mode Exit fullscreen mode

Viewing and editing file contents: cat, less, nano

cat

The 'cat' command is a versatile utility in Unix-like operating systems. The 'cat' stands for "concatenate", which hints at its core functionality. And it's commonly used to display the contents of a file on the terminal.

The Basic syntax looks like this

cat [options] [file(s)]
Enter fullscreen mode Exit fullscreen mode

The key functions of 'cat' is like: display file contents, concatenate multiple files, create new files, and append content to existing files

for example, here are some of the examples of 'cat'

a) View a file's contents

cat myfile.txt
Enter fullscreen mode Exit fullscreen mode

b) Display contents of multiple files

cat file1.txt file2.txt
Enter fullscreen mode Exit fullscreen mode

c) Create a new file with content (You can then type content and press Ctrl+D to save and exit)

cat > newfile.txt
Enter fullscreen mode Exit fullscreen mode

d) Append content to an existing file

cat >> existingfile.txt
Enter fullscreen mode Exit fullscreen mode

e) Display file with line numbers:

cat -n myfile.txt
Enter fullscreen mode Exit fullscreen mode
less

The 'less' command is another utility for viewing file contents in Unix-like operating systems, but it offers more advanced features compared to 'cat'. 'less' is a pager program that allows you to view the contents of a file one screen at a time.

Here are some of the examples

Viewing a File:

less filename.txt
Enter fullscreen mode Exit fullscreen mode

This command opens filename.txt in less mode, allowing to navigate through the file using keyboard shortcuts.

if you want to view multiple files, you can use it like this

less file1.txt file2.txt file3.txt
Enter fullscreen mode Exit fullscreen mode

This allows you to view multiple files in sequence. Press ':n' to move to the next file and ':p' to go to the previous file.

if you want to view compressed files,

less file.gz
Enter fullscreen mode Exit fullscreen mode

'less' can directly read compressed files without needing to decompress them first.

you can also display line numbers such as using like this

less -N filename.txt
Enter fullscreen mode Exit fullscreen mode

This shows line numbers alongside the content.

If you want to start at a specific line, you can try like this

less +100 filename.txt
Enter fullscreen mode Exit fullscreen mode

This highlights all occurrences of "search term" in the file.

Read from standard input

command | less
Enter fullscreen mode Exit fullscreen mode

This allows you to pipe the output of another command into 'less' for easier viewing.

nano

The 'nano' command is a simple, user-friendly text editor for Unix-like operating systems. It's often considered easier to use than more complex editors like vim or emacs, making it a good choice for beginners or for quick edits.

Here's an overview of the 'nano' command. The basic usage is look like this

nano filename.txt
Enter fullscreen mode Exit fullscreen mode

This opens the file 'filename.txt' in the nano editor. If the file doesn't exist, nano will create it.

Top comments (0)

Read next

anthonymax profile image

❄️Introducing the HMPL Template Language

Anthony Max -

qingkouwei profile image

HarmonyOS Next - Mastering Log Knowledge in One Article

kouwei qing -

samuel_udeh profile image

State Isolation: Layout vs Workspace

Samuel Udeh -

jaloplo profile image

Think You Know Teams Chats? Discover the Workflows

Jaime López -