DEV Community: Andrew Malkov

How I Use "imcopy" to Backup My Creator Files to My NAS: A Practical Guide

Andrew Malkov — Tue, 16 Jan 2024 19:04:06 +0000

As a creator, my digital assets like code, videos, photos, and graphics are the lifeblood of my work. Ensuring their safety and accessibility is paramount. This is where "imcopy", a helpful command-line tool, comes into play. It has simplified my backup process, allowing me to efficiently synchronize files between my work directories and my Synology Network Attached Storage (NAS). Let me walk you through how I use "imcopy" with practical YAML configuration examples.

Understanding "imcopy"

"imcopy" is a CLI tool designed for copying and synchronizing files between directories. It stands out with its blazing-fast file transfers, customizable ignore patterns, and options for parallelism and file overwrite behaviors.

Why "imcopy" for Backups?

Speed: It offers fast file transfers, crucial for large media files.
Customization: I can specify exactly which files to backup and which to ignore. I can define multiple destination folders and backup my files to NAS and external disk simultaneously.
Flexibility: It works equally well on Windows and Linux, fitting into my multi-OS workflow.
Ease of Use: The tool uses gitignore-like patterns for specifying ignore rules, making it intuitive for us who is familiar with git.
Centralized Configuration: All configurations for copying tasks are consolidated in one place - the YAML file - which simplifies management and tracking.
Open Source: Being an open-source tool, it offers transparency, community support, and the opportunity for customization and improvement.

How to Find the Tool

To download and start using "imcopy", follow these steps:

Visit the GitHub Repository: Navigate to the imcopy GitHub repository where the tool is hosted.
Clone or Download: You can clone the repository to your local machine using Git, or download the source code as a ZIP file.
Installation Instructions: Inside the repository, you will find detailed instructions on how to install, set up and use imcopy on your system in the README file or in docs.
Stay Updated: Keep an eye on the repository for the latest updates, bug fixes, and new features. Consider starring the repository to get notifications.

My Backup Workflow with "imcopy"

I use "imcopy's" advanced option for my backups. This requires a YAML configuration file where I can specify multiple source and destination folders, along with detailed settings for each backup task.

YAML Configuration: The Heart of My Backup

The YAML configuration file for "imcopy" is structured into three sections: directories, ignorePatterns, and global settings.

Directories
Here, I define the source and destination paths. For instance, I backup my code from D:\Apps to \\NasStorage\AppsBackup and F:\Backups\Apps.
Ignore Patterns
I use this to skip temporary files like bin and obj directories or node_modules, which don't need to be backed up.
Global Settings
I generally set parallelism to 16 for efficient use of my system's resources.

YAML File Example

Here's a snippet from my actual YAML file (I made it short for a demo):

directories:
  - source: D:\Apps
    destinations:
      - \\NasStorage\AppsBackup
      - F:\Backups\Apps
    ignorePattern: dev_files
    overwriteBehavior: ifNewer
    removeBehavior: remove
  - source: D:\youtube
    destinations:
      - \\NasStorage\YoutubeBackup
      - F:\Backups\Youtube
    ignorePattern: youtube_files
    overwriteBehavior: ifNewer
    removeBehavior: remove

ignorePatterns:
  - name: dev_files
    patterns:
      - ".git/**"
      - ".vs/**"
      - "[Bb]in/**"
      - "[Oo]bj/**"
      - "node_modules/**"
      - "local.settings.json"
      - ".env"
      - ".env.local"
      - "dist/**"
      - "build/**"
      - "_storage_emulator/**"
      - ".localConfigs"
  - name: youtube_files
    patterns:
      - "Adobe Premiere Pro Auto-Save/**"

parallelism: 16
verbose: true

Executing the Backup

With my YAML file ready, running the backup is a simple command:

imcopy --file D:\imcopy\backup.yaml

Backup Scenarios

I run a nightly backup of my creator files. Sometimes, I backup only specific projects manually using a modified YAML files.

Conclusion

"imcopy" has simplified my backup process. It's efficient, customizable, and reliable—everything a creator needs to safeguard their digital assets. Whether it's code, multimedia files, or documents, "imcopy" ensures that all my valuable work is mirrored safely to my NAS and external drive with minimal effort and maximum efficiency and speed.

Tips for Setting Up Your "imcopy" Backup

Analyze Your Files: Understand which files are crucial and need regular backup. This helps in setting up effective ignore patterns.
Test with --dry-run: Before running your first backup, use the --dry-run option to simulate the backup process. This helps ensure that your settings are correct and no important file is left out.
Regularly Update Your YAML Configuration: As your project directories evolve, make sure to update your YAML file to reflect these changes.
Automate Your Backups: Consider automating the "imcopy" command to run at regular intervals using cron jobs (Linux) or Task Scheduler (Windows).

My Experience: Peace of Mind

Since integrating "imcopy" into my workflow, I've experienced a significant reduction in the stress related to data loss. Knowing that all my creative work is securely backed up, I can focus more on the creative aspects of my projects without worrying about data integrity. The ability to customize and automate the entire process means my backups are always up-to-date and reflective of my current work status.

Whether you are a developer, a photographer, a videographer, or a graphic designer, I highly recommend giving "imcopy" a try for your backup needs. It’s not just about backing up data; it’s about ensuring the continuity and safety of your creative journey.

How to use Google OSV Scanner

Andrew Malkov — Fri, 16 Dec 2022 19:16:25 +0000

3 days ago (December 13th 2022) Google launched the Open Source Vulnerability (OSV) Scanner, a free tool that gives open source developers easy access to vulnerability information relevant to their project.

The problem

Software projects are commonly built on top of a mountain of dependencies—external software libraries you incorporate into a project to add functionalities without developing them from scratch. Each dependency potentially contains existing known vulnerabilities or new vulnerabilities that could be discovered at any time. There are simply too many dependencies and versions to keep track of them manually, so automation is required.

Scanners provide this automated capability by matching your code and dependencies against lists of known vulnerabilities and notifying you if patches or updates are needed. Scanners bring incredible benefits to project security, which is why the 2021 U.S. Executive Order for Cybersecurity included this type of automation as a requirement for national standards on secure software development.

OSV Scanner

The OSV-Scanner generates reliable, high-quality vulnerability information that closes the gap between a developer’s list of packages and the information in vulnerability databases.

Running OSV-Scanner on your project will first find all the transitive dependencies that are being used by analyzing manifests, SBOMs, and commit hashes. The scanner then connects this information with the OSV database and displays the vulnerabilities relevant to your project.

Please continue reading here: https://www.anmalkov.com/blog/how-to-use-google-osv-scanner

Kubernetes tutorial | Install Kubernetes on Raspberry Pi FAST in 2022

Andrew Malkov — Fri, 11 Feb 2022 16:41:33 +0000

I've just uploaded a new video where I show how to install your own Kubernetes cluster on your own Raspberry Pi. In this video I'm using Raspberry Pi OS x64 that was recently released.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/rOXkutK8ANc

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Kubernetes tutorial - Learn Kubernetes FAST by doing

Use Serilog with Minimal API in .NET 6

Andrew Malkov — Fri, 19 Nov 2021 16:47:30 +0000

If you want to replace the standard logging with Serilog in Minimal API you have to do just a couple of steps.

At first, you have to add a reference to a NuGet package Serilog.AspNetCore:

dotnet add package Serilog.AspNetCore

At second, you need to register Serilog as your logger in Program.cs:

. . .

// remove default logging providers
builder.Logging.ClearProviders();
// Serilog configuration        
var logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateLogger();
// Register Serilog
builder.Logging.AddSerilog(logger);

. . .

That's it.

You can find more information and full code example here: Use Serilog with Minimal API in .NET 6

Timer in GitHub CLI

Andrew Malkov — Fri, 17 Sep 2021 13:43:56 +0000

The new GitHub CLI 2.0 includes extensions now.

I always wanted to have a timer or stopwatch in my terminal. This will allow me to use it to timebox my tasks and increase my productivity by minimizing switching from a terminal to a different timer application, like browser, and back to terminal.
It will be also cool if this timer can be big enough on the screen, so I can use it as a timer for my demo sessions to let attendees know how much time left for a break.

So, I decided to build it as a GitHub CLI extension. This will allow me to use it easily in all of my machines.

You can find it on GitHub:
https://github.com/anmalkov/gh-timer

How to install

First of all you have to install GitHub CLI 2.0, if you don't have it yet. Install GitHub CLI now.

After that you can install gh-timer extension:

gh extension install anmalkov/gh-timer

options

gh timer [units] [number] [mode] [clock]

[units]

-s, --sec - seconds

-m, --min - minutes

[mode]

-w, --stopwatch - use a stopwatch mode, otherwise use a timer mode

[clock]

-b, --big - show a big clock, otherwise show a small clock

usage examples

gh timer -s 10
run a timer for 10 seconds

gh timer -m 5
run a timer for 5 minutes

gh timer -m 5 -b
run a timer for 5 minutes and show big clock

gh timer -m 60 -w
run a stopwatch for 1 hour

gh timer -h
show help

If you like it, please don't forget to give it a star on GitHub!

If you have any questions or would like to contribute - you are welcome to contact me!

See you soon.

C# tutorial for beginners | 2 | Write your first C# program

Andrew Malkov — Sat, 06 Mar 2021 11:43:58 +0000

In this C# tutorial series you will learn how to get started programming with C#. You will write your own programs as well as make your own web site.
I can make you successful so you can be a programmer as well.

I've just uploaded the second episode. In this episode you will learn fundamentals of programming as well as basics of how to work with Visual Studio. If you will follow the steps from this video, at the end you will write your first program and you will know how to find, copy and run it.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/KTfilr_pnk0

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: C# tutorial - Learn C# FAST

Docker tutorial | What is Docker | What is container | How Docker works

Andrew Malkov — Thu, 25 Feb 2021 08:54:16 +0000

In this Docker tutorial series you will learn how to work with Docker. You will learn basic as well as advanced topics about containers and Docker. You will know how to containerize your apps, deploy them to production and troubleshoot your containers.

I've just uploaded the third episode where you can learn basic and advanced aspects of containers and Docker.
I show you what are the benefits of containers. Then, I talk about the main container's components and terms. After that, I dig inside a container image and a container itself. And at the end, I describe how Docker is architected and how it works.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/GSw9OHOloOY

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Docker tutorial - Learn Docker FAST by doing

C# tutorial for beginners | 1 | Setup development environment

Andrew Malkov — Wed, 17 Feb 2021 18:22:28 +0000

I've just uploaded the first episode where I show how to setup the development environment.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/vhG2-lIS3EE

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: C# tutorial - Learn C# FAST

Docker tutorial - Install Docker on Ubuntu fast

Andrew Malkov — Mon, 15 Feb 2021 06:53:21 +0000

I've just uploaded a new video where I show how to install Docker on Ubuntu Linux.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/5Dhx_ssK8oY

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Learn containers fast by doing

Install Docker on Windows 10 FAST

Andrew Malkov — Sat, 13 Feb 2021 07:23:21 +0000

I've just uploaded a new video where I show how to install Docker on Windows 10 and configure Docker to use WSL2.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/ERbcZ_V6dn4

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Learn containers fast by doing

Install Podman and Buildah on Windows 10 FAST using WSL2

Andrew Malkov — Tue, 09 Feb 2021 12:59:01 +0000

I've just uploaded a new video where I show how to install and configure Podman and Buildah on Windows 10 using WSL2 and Ubuntu 20.04.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/iOxILJIYsuY

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Learn containers fast by doing

wsl2 - Install Ubuntu on Windows 10 FAST with Windows Terminal

Andrew Malkov — Sat, 06 Feb 2021 20:52:33 +0000

I've just uploaded a new video where I show how to install Ubuntu 20.04 on Windows 10 or your favorite Linux distribution using Windows Subsystem for Linux 2, or just shortly WSL2.
This will allow you later to install and configure your favorite containers management tools like buildah, podman or Docker.

Please check it out and subscribe to my channel to be notified when my new videos will be released.

Thank you in advance.

Link to video: https://youtu.be/tqE9zcA8L3U

Link to my personal blog: https://www.anmalkov.com

Link to my playlist: Learn containers fast by doing