DEV Community

Cover image for How to work with CAR files with NestJS
Rodrigo Burgos
Rodrigo Burgos

Posted on

How to work with CAR files with NestJS

What is a CAR File?

A CAR file is a representation of a set of blocks in a DAG (Directed Acyclic Graph) structure. It's widely used in decentralized storage protocols like IPFS for its ability to store data efficiently while ensuring content addressability via CIDs (Content Identifiers).

Steps to Compact Multiple Files into a CAR File

  1. Prepare the Files
    Convert the files into byte arrays or readable streams.
    Ensure the files are properly labeled with metadata if needed.

  2. Use ipfs-car or nft.storage Tools
    Tools like ipfs-car are libraries to create, manipulate, and encode CAR files.

  3. Create a CAR Encoder
    Use the createFileEncoderStream, createDirectoryEncoderStream, or CAREncoderStream to encode the files into CAR format.

Implementation Example

Here’s how to implement it in a NestJS service using ipfs-car:

File controller:

import { Controller, Post, UploadedFiles, UseInterceptors } from '@nestjs/common';
import { FilesInterceptor } from '@nestjs/platform-express';
import { FileService } from './file.service';

@Controller('file')
export class FileController {
  constructor(private readonly fileService: FileService) {}

  @Post('upload-car')
  @UseInterceptors(FilesInterceptor('files'))
  async uploadFiles(@UploadedFiles() files: Express.Multer.File[]) {
    const rootCID = await this.fileService.packFilesIntoCar(files);
    console.log(`Root CID: ${rootCID}`);
    return { rootCID };
  }
}
Enter fullscreen mode Exit fullscreen mode

File Service:

import { Injectable } from '@nestjs/common';
import { Blob } from 'nft.storage';
import { CAREncoderStream } from 'ipfs-car';

@Injectable()
export class FileService {
  constructor() {}

  public async packFilesIntoCar(files: Express.Multer.File[]): Promise<string> {
    let rootCID = '';

    try {
      const blobs = files.map((file) => {
        const fileBytes = new Uint8Array(file.buffer);
        return new Blob([fileBytes], { type: file.mimetype });
      });

      // Start encoding files into CAR format
      const encoder = new CAREncoderStream();

      // Add all files to encoder
      for (const blob of blobs) {
        const fileStream = blob.stream();
        const writer = encoder.getWriter();
        for await (const chunk of fileStream) {
          writer.write(chunk);
        }
        writer.close();
      }

      const rootBlock = await encoder.getRootBlock();
      rootCID = rootBlock.cid.toString();

      return rootCID;
    } catch (error) {
      throw new Error(`Error creating CAR file: ${error.message}`);
    }
  }
}

Enter fullscreen mode Exit fullscreen mode

Explanation

File Controller:

Exposes an endpoint to upload files (/upload-car).

  • Intercepts files via FilesInterceptor (Multer middleware).
  • Calls the packFilesIntoCar method from the FileService. File Service:

Converts files to Blob objects.

  • Uses CAREncoderStream to encode files into a CAR file.
  • Returns the root CID of the CAR archive.

CAREncoderStream:

  • Writes file blocks to the CAR archive. Generates a root CID for referencing the CAR file.

Benefits

  • Data Integrity: Each block in the CAR file has its unique CID.
  • Compatibility: CAR files are natively supported in IPFS and Filecoin.
  • Efficiency: Combines multiple files into a single compact archive.

Additional Enhancements

  • Save the CAR file locally: Use Node.js fs module to write the CAR stream into a file.

  • Upload CAR to Decentralized Storage: Integrate with nft.storage, Web3.Storage, or similar APIs.

  • Add Metadata: Include additional metadata like file names, timestamps, or descriptions to make the archive more useful.

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay