Puppeteer Docker

Hi,

i am running into an error while performing html to pdf generation using puppeteer ,i am using docker to deploy on aws but when i excute my code i am getting error

"Read-only file system : '/var/task/chrome-linux64.zip'"

Thanks

Comments

[Jędrzej Grabala]

The error "Read-only file system: '/var/task/chrome-linux64.zip'" indicates that your Puppeteer process is trying to write to a location in the file system that is mounted as read-only. This is a common issue when deploying applications in environments like AWS Lambda where the file system may be restricted.

To address this issue, you can take the following steps:

1. Use a Writable Directory AWS Lambda provides a writable temporary directory at /tmp that you can use to store files. You can configure Puppeteer to use this directory for any file operations.

Modify your Puppeteer setup to extract the browser to the /tmp directory. Here’s how you can do it in your code:

const puppeteer = require('puppeteer-core');
const chrome = require('chrome-aws-lambda');

(async () => {
  const browser = await puppeteer.launch({
    executablePath: await chrome.executablePath,
    args: chrome.args,
    headless: chrome.headless,
    // Specify a writable directory
    userDataDir: '/tmp/puppeteer'
  });

  const page = await browser.newPage();
  await page.goto('https://example.com');
  const pdf = await page.pdf({ path: '/tmp/example.pdf' });

  // Do something with the PDF, e.g., upload to S3

  await browser.close();
})();

1. Use Headless Chromium for AWS Lambda You might want to use the chrome-aws-lambda package, which provides an optimized Chromium binary for AWS Lambda. This package simplifies the setup and ensures that Puppeteer works seamlessly in the AWS Lambda environment.

Install the necessary packages:

npm install puppeteer-core chrome-aws-lambda

1. Docker Configuration If you are using Docker to deploy your application on AWS (such as with AWS Fargate or ECS), ensure that your Docker container configuration allows writing to the necessary directories. You can specify volume mounts and ensure that the relevant directories are writable.

Here’s an example Dockerfile:

FROM node:14-slim

# Install dependencies
RUN apt-get update && apt-get install -y \
    libnss3 \
    libxss1 \
    libasound2 \
    fonts-liberation \
    libappindicator3-1 \
    xdg-utils \
    && rm -rf /var/lib/apt/lists/*

# Set up Puppeteer
RUN npm install puppeteer-core chrome-aws-lambda

# Copy your application code
COPY . /app
WORKDIR /app

# Entrypoint
CMD ["node", "your-script.js"]

1. AWS Lambda Layers If you are deploying on AWS Lambda, consider using AWS Lambda Layers to include the Chromium binary and Puppeteer dependencies. This approach can simplify deployment and ensure your Lambda function stays within the deployment size limits.

You can create a layer with the necessary binaries and dependencies, and then configure your Lambda function to use this layer.

1. Error Handling and Logging Ensure you have proper error handling and logging in place to capture any issues during the execution. This can help you diagnose and resolve issues more effectively.

Example Lambda Function with Layers
Here’s an example of how to set up an AWS Lambda function with Puppeteer using layers:

const chromium = require('chrome-aws-lambda');
const puppeteer = require('puppeteer-core');

exports.handler = async (event) => {
  let result = null;
  let browser = null;

  try {
    browser = await puppeteer.launch({
      executablePath: await chromium.executablePath,
      args: chromium.args,
      headless: chromium.headless,
    });

    const page = await browser.newPage();
    await page.goto('https://example.com');
    const pdf = await page.pdf({ path: '/tmp/example.pdf' });

    // Upload the PDF to S3 or handle it as needed

    result = 'PDF generated successfully';
  } catch (error) {
    console.error('Error generating PDF:', error);
    result = 'Error generating PDF';
  } finally {
    if (browser !== null) {
      await browser.close();
    }
  }

  return {
    statusCode: 200,
    body: JSON.stringify(result),
  };
};

Hope it helps, good luck !

[Let Alliance]

Thanks for the quick reply ,

i am using PuppeteerSharp for document generation
and below is my backend code

public void ProcessRecord(ReportModel report, ILambdaContext context)
{
var mq = new MessageQueue { Message = JsonConvert.SerializeObject(report) };
_messageQueue.Add(mq);
var status = "Completed";
if (string.IsNullOrEmpty(report.GetDocumentPath))
return;

Log.Information("Proceed with Lambda");
report.BucketName = SMSettings.AppSettings.S3BucketName;
var request = new GetObjectRequest
{
    BucketName = SMSettings.AppSettings.S3BucketName,
    Key = "template/document/" + report.GetDocumentPath + "/" + report.DocumentTemplate
};
try
{
    //context.Logger.Log(request.ToJsonString());

    GetObjectResponse response = S3Client.GetObjectAsync(request).Result;
    var s3ObjectKey = report.SaveDocumentPath + "/" + report.DocumentName + ".pdf";
    var responseBody = string.Empty;

    using (StreamReader reader = new StreamReader(response.ResponseStream))
    {
        responseBody = reader.ReadToEnd();
    }

    Log.Information(responseBody);
    if (!string.IsNullOrWhiteSpace(report.HeaderPath))
        responseBody = ReplaceTemplateHeaderFooter(report, responseBody, true);
    if (!string.IsNullOrWhiteSpace(report.FoooterPath))
        responseBody = ReplaceTemplateHeaderFooter(report, responseBody, false);
    Log.Information(responseBody);
    using (Stream responseStream = response.ResponseStream)
    {

        var bytes = ReadStream(responseStream);
        Log.Debug("Loading puppeteer");
        var browserLauncher = new HeadlessChromiumPuppeteerLauncher(new LoggerFactory()
                    .AddSerilog(Log.Logger));
        //ChromiumExtractor.ChromiumPath = "/usr/bin/google-chrome-stable";

        //var bfopt = new BrowserFetcherOptions() { };

        //new BrowserFetcher(bfopt).DownloadAsync().Wait();

        var options = new LaunchOptions
        {
            Headless = true,
            //Args = ["--no-sandbox", "--disable-setuid-sandbox"],
            ExecutablePath = "/tmp/Chrome"
            //Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "chromium", "chrome")
        };

        Log.Information(ChromiumExtractor.ChromiumPath);


        var browser1 = Puppeteer.LaunchAsync(options).Result;

        //context.Logger.Log("executablepath  - " + Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "chromium", "chrome"));
        Log.Information(ChromiumExtractor.ChromiumPath);

        var page1 = browser1.NewPageAsync().Result;

        PdfOptions objnav = new PdfOptions();
        objnav.Format = PaperFormat.A4;

        if (report.IsCustomMargin)
        {
            objnav.PrintBackground = true;
            objnav.MarginOptions.Bottom = "2cm";
            objnav.MarginOptions.Top = "2cm";
            objnav.MarginOptions.Left = "1cm";
            objnav.MarginOptions.Right = "1cm";
            //objnav.PrintBackground = true;
        }
        var generatedhtml = ReplaceTags(responseBody, report.DocumentData, report);
        page1.DefaultNavigationTimeout = 0;
        page1.SetContentAsync(generatedhtml, new NavigationOptions { WaitUntil = new[] { WaitUntilNavigation.Networkidle0 } }).Wait();

        var fileStream = page1.PdfStreamAsync(objnav).Result;
        UploadFileStreamSync(fileStream, report.BucketName, s3ObjectKey);
        Log.Information(s3ObjectKey);


        //dev express

        //RichEditDocumentServer server = new RichEditDocumentServer();
        //server.Options.Export.Html.CssPropertiesExportType = DevExpress.XtraRichEdit.Export.Html.CssPropertiesExportType.Inline;
        //server.Options.Export.Html.EmbedImages = true;
        //server.HtmlText = generatedhtml;
        //using (var pdfFileStream = new MemoryStream())
        //{
        //    server.ExportToPdf(pdfFileStream);
        //    UploadFileStreamSync(pdfFileStream, report.BucketName, s3ObjectKey);
        //}

        //server.ExportToPdf("Document_PDF.pdf");

        //dev express end


        //using (var page = browser.NewPageAsync().Result)
        //{
        //    var html = responseBody;
        //    PdfOptions objnav = new PdfOptions();
        //    objnav.Format = PaperFormat.A4;

        //    if (report.IsCustomMargin)
        //    {
        //        objnav.PrintBackground = true;
        //        objnav.MarginOptions.Bottom = "2cm";
        //        objnav.MarginOptions.Top = "2cm";
        //        objnav.MarginOptions.Left = "1cm";
        //        objnav.MarginOptions.Right = "1cm";
        //        //objnav.PrintBackground = true;
        //    }

        //    page.DefaultNavigationTimeout = 0;
        //    var generatedhtml = ReplaceTags(html, report.DocumentData, report);
        //    Log.Information("generatedhtml " + generatedhtml);
        //    page.SetContentAsync(generatedhtml, new NavigationOptions { WaitUntil = new[] { WaitUntilNavigation.Networkidle0 } }).Wait();
        //    //page.GetContentAsync().Wait();
        //    Log.Information("Waiting for Lambda Upload");
        //    var resultFile = page.PdfStreamAsync(objnav).Result;
        //    //var resultFile = page.PdfStreamAsync().Result;
        //    UploadFileStreamSync(resultFile, report.BucketName, s3ObjectKey);
        //    Log.Information(s3ObjectKey);
        //    _messageQueue.Remove(mq);
        //}
    }

}
catch (Exception e)
{
    context.Logger.LogError(e.ToString());
    Log.Error(e, "");
    status = "Error";

}
var send = AWSSecretManager.GetPublishEventKey("documentgenerated");
_eventBus.PublishAsync(new MessageViewer { Message = new { report.DocumentName, report.SaveDocumentPath, report.MessageId, Status = status }.ToJsonString(), MessageType = MessageType.Document.ToString() }, send).Wait();

}

and below is docker file code :

ARG CHROME_VERSION="81.0.4044.138-1"
RUN apt-get update && apt-get -f install && apt-get -y install wget gnupg2 apt-utils
RUN wget --no-verbose -O /tmp/chrome.deb dl.google.com/linux/chrome/deb/poo... \
&& apt-get update \
&& apt-get install -y /tmp/chrome.deb --no-install-recommends --allow-downgrades fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf \
&& rm /tmp/chrome.deb

Add user, so we don't need --no-sandbox.

same layer as npm install to keep re-chowned files from using up several hundred MBs more space

RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser

Run everything after as non-privileged user.

USER pptruser

END PUPPETEER RECIPE

ENV PUPPETEER_EXECUTABLE_PATH "/usr/bin/google-chrome-unstable"

and now i am getting below error :

System.AggregateException: One or more errors occurred. (Failed to launch browser! )

---> PuppeteerSharp.ProcessException: Failed to launch browser!

at PuppeteerSharp.States.ProcessStartingState.StartCoreAsync(LauncherBase p) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/States/ProcessStartingState.cs:line 83

at PuppeteerSharp.States.ProcessStartingState.StartCoreAsync(LauncherBase p) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/States/ProcessStartingState.cs:line 89

at PuppeteerSharp.Launcher.LaunchAsync(LaunchOptions options) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Launcher.cs:line 65

at PuppeteerSharp.Launcher.LaunchAsync(LaunchOptions options) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Launcher.cs:line 98

--- End of inner exception stack trace ---

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)

at PIB.DocumentGenerator.Data.Services.DocumentProcessService.ProcessRecord(ReportModel report, ILambdaContext context) in /var/task/PIB.DocumentGenerator.Data/Services/DocumentProcessService.cs:line 120

Failed to launch browser! /var/task/Chrome/Linux-122.0.6261.111/chrome-linux64/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file