DEV Community: Aaron

How To: Open Source Policy Automation via Phylum Extensions

Aaron — Wed, 07 Sep 2022 23:51:48 +0000

Policy Automation in Open Source Software

One of the biggest challenges with application security fundamentally comes from the amount of data that must be reasoned about. Nowhere is this more true than the moderation of issues stemming from open source packages - it is easy for teams to be totally inundated with issues and findings.

One of Phylum's key design goals was to help address this issue: Provide users with the tools they need in order to proactively reduce the number of false-positives they face, by allowing better automation from start to finish. It's with these tools we start to really enable the building blocks of automation - and to that end, allowing better integration into the overall development ecosystem.

Starting with Policy Management

While Phylum provides tools today to help proactively manage policy, they may not be as granular or robust as some organizations would desire. To that end, we will explore the development of a custom policy framework to enable automated triage of issues, as well as proactive screening of major engineering hygiene and compliance problems.

We will start here with a slightly more in-depth, custom version of the existing NPM shim extension - a tool that enforces default project policy when installing NPM packages. This custom extension will do some additional custom validation before allowing the installation process to continue.

First, per the API documentation we will start with the manifest file for our new extension - PhylumExt.toml:

name = "npm-policy"
description = "Example custom policy extension"
entry_point = "main.ts"
[permissions]
  read = ["./package.json", "./package-lock.json"]
  write = ["./package.json", "./package-lock.json"]
  run = ["npm"]

The Beginning of the Extension

Now, we will jump over and start working on our custom policy extension. Starting from the very beginning, we will create a basic skeleton in main.ts that essentially emulates the behavior of the linked shim extension. In the current implementation of the extension API, the first two command line arguments (the binary path + first value - what would usually be contained in arg[0] and arg[1]) are effectively consumed before the extension itself is invoked. This means that once our extension is installed, since the registered name is npm-policy, if we were to invoke the CLI with the following command: phylum npm-policy install packageX, Deno.args[0] will contain install and Deno.args[1] will contain packageX.

Starting with a rough analog to the npm policy shim, we get something similar to the following:


import { PhylumApi } from 'phylum';

const LOCKFILE_NAME = "./package-lock.json";
const PACKAGEFILE_NAME = "./package.json";

type NullableString = string | null;


// This method will return a promise containing a null value if the file is 
// missing, and the containing text otherwise. 
async function readTextFileIfExists(path: string): Promise<NullableString> {
  try {
    // We will attempt to read the contents of an existing
    // lockfile, if one exists. This will let us restore
    // the data after we perform our analysis.
    return await Deno.readTextFile(path); 
  } catch(e) {
    // Pass - we will just know that we won't need to _restore_ the lockfile after
    //  the operation is complete if we have no file content.
    return null;
  }

}


// Method to attempt: generating a new lockfile, parsing said lockfile, and
// performing analysis on the contained package information.
async function preAnalyzeNpm(subcmd: string, args: string[]): Promise<Object> {
  try {
    // Attempt to leverage npm to generate a new lockfile based on the
    // provided arguments
    await Deno.run({
      cmd: ['npm', subcmd, '--package-lock-only', ...args],
      stdout: 'piped',
      stderr: 'piped',
    }).status();

    // Attempt to parse the new lockfile - if that fails, we will return
    // an error. 
    const lockFile = await PhylumApi.parseLockFile(LOCKFILE_NAME, 'npm');
    if(!lockFile.packages.length)
       return {pass: false, message: 'error: no packages found in lockfile'};

    // Analyze the underlying packages, and get back the job information
    const jobId = await PhylumApi.analyze('npm', lockFile.packages);
    const results = await PhylumApi.getJobStatus(jobId);

    // Give back the results
    return results;

  } catch(e) {
    // An error occurred along the way - likely with command execution or similar
    return {pass: false, message: `error: analysis attempt failed! ${e}`};
  }
}


async function runNpmAnalysis(): Promise<Object> {
  // First, we will capture the existing text within both the 
  // package and lockfiles.
  const packageList = readTextFileIfExists(PACKAGEFILE_NAME);
  const lockfile = readTextFileIfExists(LOCKFILE_NAME);
  // We can't _really_ proceed without a package.json.
  if(!packageList)
    return {pass: false, message: "package.json is missing"};

  // Attempt the analysis
  const data = preAnalyzeNpm(Deno.args[0], Deno.args.slice(1));
  try {
    // We now need to restore the lockfile and package.json. 
    // Since we dont know if we want to allow the installation to
    // proceed just yet.
    await Deno.writeTextFile(PACKAGEFILE_NAME);
    if(lockfile)
      await Deno.writeTextFile(LOCKFILE_NAME);
    else
      await Deno.remove(LOCKFILE_NAME);
  } catch(e) {
    console.error(`Error: failed to restore either package or lockfile! ${e}`);
  }

  // Give back the data to the user.
  return data;
}



const args = new Set(['install', 'isntall', 'update', 'udpate']);

if(Deno.args.length >= 1 && args.has(Deno.args[0])) {
  // Attempt the analysis if we meet the criteria above
  const res = await runNpmAnalysis();
  // Now we will do some basic validation - this basically
  // gets us parity with the current extension:
  if(!res.pass) {
    if(res.message) {
      console.error(`An error occurred while attempting to check - ${res.message}`)
      Deno.exit(-1);
    }

    console.error("the installation attempt triggered a policy failure!");
    Deno.exit(-2);
  }

  if('complete' !== res.status) {
    console.warn("Scan is incomplete - please try again shortly!");
    Deno.exit(-3);
  }
  // TODO: Add more validation 
}


console.log("[phylum] installation ok to proceed...");

const status = await Deno.run({cmd: ['npm', ...Deno.args]}).status();

Deno.exit(status);

At this point we can now start to drill into specific issues, add logic to filter for certain types of license, or even apply local environment checks. Probably the best place to start our journey would be the Phylum API documentation around the actual analysis object - this gives us a spot to start digging into the types of findings that may come back. From that perspective, we will start by adding a set of new methods that we will ultimately plumb through to handle the extra validations. From this view, we should make sure that this API is flexible and easy to extend. Something we can start with might be as follows:


// First, we will define our "filter" type to handle examination of each
// package analysis result that comes back from our request. We will return
// an error message to display to the user if the check fails, or null if 
// the package is ok to use.
type Filter = (pkg: Object) => NullableString;

// Our validation method will take a list of packages (from our API response),
// and a list of "filter" methods (as defined above). It will return an error
// message to display to the user if the checks fail, or null if all pass.
const packageValidator = (pkgList: Object[], fltList: Filter[]): NullableString => {
  let errorMessages = [];

  // We will walk through the list of packages here
  for(const pkg of pkgList) {
     // For each package, we will apply each provided
     // filter method, and add the result to our list
     // of errors if a problem is found.
     for(const flt of fltList) {
       const res = flt(pkg);
       if(res)
         errorMessages.push(res);
     }
  }

  // Now we will return either a consolidated error list, or null.
  if(errorMessages.length)
    return errorMessages.join("\n");

  return null;
}

So now we have a general framework we can apply rules to - now, if we take the operative block of code from the first example where we left our // TODO: initially, we can start by slotting in our new filter method:


// ...

// TODO: populate with new filters!
const filterMethods: Filter[] = [];

if(Deno.args.length >= 1 && args.has(Deno.args[0])) {
  // Attempt the analysis if we meet the criteria above
  const res = await runNpmAnalysis();
  // Now we will do some basic validation - this basically
  // gets us parity with the current extension:
  if(!res.pass) {
    if(res.message) {
      console.error(`An error occurred while attempting to check - ${res.message}`)
      Deno.exit(-1);
    }

    console.error("the installation attempt triggered a policy failure!");
    Deno.exit(-2);
  }

  if('complete' !== res.status) {
    console.warn("Scan is incomplete - please try again shortly!");
    Deno.exit(-3);
  }
  // Now, we slot in our filter method, and provide the package list from
  // our previous analysis
  const filteredResult = packageValidator(res.packages, filterMethods);
  if(filteredResult) {
    // If we have landed here, that means at least one error was returned.
    // In that case, we will print the error message to the user and exit.
    console.error(filteredResult);
    Deno.exit(-4);
  }

  // Otherwise, our new filter methods have passed - this means we are free to
  // continue as before.
}

// ...

Now, we are at the point where it makes sense to start actually adding in filters. For this, we can start with a view of what the actual package structure will look like - we should receive a call to each of our filters for each package structure returned by the API. This will have a set of fields describing various attributes of the package, and also, optionally a list of issues. To keep things simple, we will start by writing a rule that filters packages with certain, restrictive licenses - this could include GPL or AGPL software - so it simply can't be installed anymore from here on out.


// List of validators to check licenses against
const disallowedLicenses = [/(A|L)?GPL-\d+.*/, /CC-BY-.*/];

const licenseValidator = (pkg: Object): NullableString => {
  // Now we will walk through the list of disallowed license regexes,
  // and if any match, we will return an error. 
  for(const license of disallowedLicenses)
    if(license.test(pkg.license))
      return `disallowed license ${pkg.license} in ${pkg.name}:${pkg.version}`;

  return null;
}

Perhaps in addition to this, we want to add in some filters to help manage against actual issues returned by the API; in this case, we will keep the example simple and focus on filtering malware issues above a threshold of "medium", and all other "critical" issues identified:


const disallowedMalware = new Set(['medium', 'high', 'critical']);

const issueValidator = (pkg: Object): NullableString => {
  // Issues are currently defined as:
  // {
  //    tag: "UNIQUE_ID",
  //    title: "issue title",
  //    description: "Issue description"
  //    severity: "low | medium | high | critical",
  //    domain: "malicious_code | author | engineering | license | vulnerability"
  // }
  for(const issue of pkg.issues) {
    // If we encounter _any_ critical issues, we will return now
    if('critical' === issue.severity)
      return `critical issue ${issue.title} found in ${pkg.name}:${pkg.version}`;

    // Similarly, we will check to see if the issue is both a malware finding,
    // and has a severity of medium or higher.
    if(issue.domain === 'malicious_code' && disallowedMalware.has(issue.severity))
      return `potential malware ${issue.title} found - ${pkg.name}:${pkg.version}`;
  }

  // If we made it here, none of the issues have created an error we need to report
  return null;
}

Now, we can go back and slot in our two new filter methods, and should have some validation during the installation process while utilizing this new extension:



// ...

const filterMethods: Filter[] = [licenseValidator, issueValidator];

if(Deno.args.length >= 1 && args.has(Deno.args[0])) {
// ...

  // Now, with the update at the top to filterMethods, we will apply 
  // our two new filters to each returned package.
  const filteredResult = packageValidator(res.packages, filterMethods);
  if(filteredResult) {

// ...

Now this is certainly not an entirely comprehensive solution, but should provide enough to begin building stronger policy capabilities on top of Phylum's tooling and API.

You can read many more articles in the same vein on our blog here, and sign up to use Phylum and try out this example (among others).

How To: Issue Triage via Phylum Extensions

Aaron — Wed, 07 Sep 2022 23:48:54 +0000

Extension Development

With the release of the Phylum community edition, we have added support for Software Supply Chain Extensions - scriptable extensions to the product which enable a layer of automation and customizability on top of what Phylum provides.

Motivation

Better automation is critically important to maintaining development velocity and ensuring that major problems aren't missed - in short, it is the entire cornerstone of DevOps. Without strong automation (and tooling to support it), CI/CD is no longer "continuous". With this in mind, we developed an automation framework to enable the development of tooling to help facilitate orchestration, findings management, and triage automation (among other things). As such, we strive to be "automation first," and provide better tooling to help design toward more proactive solutions.

Several guiding principles were applied in selecting the correct tools to enable this vision: the mechanism for managing extensions must be flexible, it should provide security controls (as a software supply chain security company, we want to ensure that the tooling from community is also adequately secure) and sandboxing, and should be simple to utilize.

Tooling

With this in mind we chose to leverage Deno as the base framework for our automation tooling. It runs in a sandbox with a robust permission management system, comes with a feature-rich standard library, and supports many languages (through its ability to run Web Assembly). Additionally, it provides first-class integrations with Rust (as the Deno embeddable sandbox itself ships as a crate), and is easily extensible.

Extensions In Motion

Given that the API itself has relatively robust documentation, and is not extraordinarily complex, we won't focus too heavily on that specifically for this exercise - rather, we'll spend more time focused on what we can actually do with the extension framework.

To that end, we will work through the exercise of building an alerting tool - the end result being an extension that can filter issues, and push critical alerts into Jira via REST API. This is also quite well documented, and should serve as a good example to showcase where the extension framework can be helpful.

Getting Started

As we start working on this problem, we will first add a manifest to outline what resources our extension will need to operate properly:

name = "jira"
description = "blindly put critical issues into jira!"
entry_point = "main.ts"
[permissions]
  env = ["JIRA_USER_EMAIL", "JIRA_USER_TOKEN"]
  net = ["example-domain.atlassian.net"]

Where example-domain should be replaced with whatever subdomain is appropriate.

Next, we will begin to build out our actual extension - starting in main.ts, which we identified as our extension's entry point (above). Here we will start by:

Parsing the lockfile provided by the user.
Performing an analysis on the loaded packages.
Collecting the returned issues that are of critical severity.


import { PhylumApi } from "phylum";


// First, we will check to ensure that we have the appropriate arguments -
// for this example, we will assume that it takes the form:
// phylum jira <ecosystem> <lockfile-path>
// NOTE: the phylum CLI itself utilizes the first two of those arguments before
//       calling into Deno, so `Deno.args[0]` will be the value provided for 
//       <ecosystem>, and `Deno.args[1]` will contain <lockfile>.
if(Deno.args.length < 2) {
  console.error("Usage: phylum jira <ecosystem> <lockfile>");
  Deno.exit(1);
}

const ecosystem = Deno.args[0];
const lockfile = Deno.args[1];
// Parse the provided lockfile
const lockData = await PhylumApi.parseLockfile(lockfile, ecosystem);

// Perform analysis
const jobId = await PhylumApi.analyze(ecosystem, lockData.packages);
const status = await PhylumApi.getJobStatus(jobId);

let criticalIssues = {};
// Now, we will loop through the packages and check the issues that came back:
for(const package of status.packages) {
  // Issues are as follows:
  // { 
  //   tag: "<unique issue ID>", 
  //   title: "issue title", 
  //   description: "issue description",
  //   severity: "low | medium | high | critical",
  //   domain: "malicious_code | vulnerability | author | license | engineering" 
  // }
  const issues = package.issues.filter(issue => issue.severity === 'critical');
  if(issues.length)
    criticalIssues[`${package.name}:${package.version}`] = issues;
}

Now we can refer back to the docs for the Jira API we will be hitting in order to generate Jira issues for each critical issue returned by the Phylum API. In practice, we would likely want to pull existing issues and de-duplicate (if the issue we are about to create already exists for an existing package/version), and would want to paginate for situations where 50 or more packages with critical issues are identified, but for the sake of keeping this example simple, we will just attempt bulk creation.


import { encode } from "https://deno.land/std/encoding/base64.ts";

// ...

let bodyData = {
  issueUpdates: []
};

// First we will loop through our packages with critical issues, and generate
// an entry for each:
for(const packageId of Object.keys(criticalIssues)) {
   // Populate whatever fields are appropriate given the package/version
   // tuple provided as the key, and the issue information as the value
}

const email = Deno.env.get("JIRA_USER_EMAIL");
const token = Deno.env.get("JIRA_USER_TOKEN");

if(!email || !token) {
  console.error("Please ensure that you provide the user email and token via env!")
  Deno.exit(2);
}

const encodedToken = encode(`${email}:${token}`);

const result = await fetch("https://example-domain.atlassian.net", {
  method: "POST",
  headers: {
    'Authorization': `Basic ${encodedToken}`,
    'Accept': 'application/json',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify(bodyData)
});

if(200 !== result.status) {
  console.error(`Request to create issues failed! Got a ${result.status} back.`);
  Deno.exit(3);
}

const data = result.json();

if(data.errors.length > 0) {
  // Handle errors
}

Followed by a phylum extension install ./jira-extension and now, execution!

You can also view this article (and many more) on our site - where you can also access the free, community version of our product.