DEV Community: Mitchell

[Part 4/100] Security? Who is she

Mitchell — Tue, 09 Aug 2022 14:44:00 +0000

Ignoring the 11 months since Part 3 was posted, it would be nice to share a tale between yours truly & Laravel, whomst've have tried their best avoiding a line of code that I believe is exploitable, present for over 6 years.

The year was 2021; upon discovering what I perceived to be an exploitable piece of code that existed within Laravel since at least 2016, I did what any good Samaritan would do.
Attempted to get a CVE issued against the framework.

The Exploit

Is a line of code within Scheduler's Command Builder that does not escape shell arguments.

I would never call myself an expert on security, but to my untrained eyes it appeared to resonate the profile of an command injection.

https://owasp.org/www-community/attacks/Command_Injection

return $event->user && ! windows_os() ?
   'sudo -u '.$event->user.' -- sh -c \''.$command.'\'' : 
   $command;

(Formatted for readability) https://github.com/illuminate/console/blob/6678d8f63c1f0dd8fb1c98010d9893673f444e84/Scheduling/CommandBuilder.php#L73

With assistance of this line of code, using the Laravel Framework we're able to schedule the execution of any arbitrary shell command (Outside of Laravel).

Timeline

Feb 2021

I submit a vulnerability @ Snyk with details surrounding the supposed exploit

Response (excerpt)

I recognise that the lines you pointed to show unescaped inputs. However, your description does not clarify how an attacker can alter these input parameters in the context of this package and how it's implemented by its users.

It would be helpful if you could provide the details showing the possible attack path, or, better yet, a working PoC of an injected command.

Fair enough.

6 hours later

Created & replied with this PoC

VeryStrongFingers / laravel-shell-escape-poc

Laravel - Shell Escape PoC

Proof of Concept to prove Laravel package vendors can exploit the command scheduler to run self-scheduled arbitrary shell commands.

Meaning the host can execute an arbitrary command in a child process shell, invoked by the Laravel scheduler.

This package uses head -n 1 /etc/passwd > /tmp/really-cool.log for the example.

Affecting all Laravel versions above 5.4 (Lumen is untested) when the scheduler artisan schedule:run is used.

Installation

Setup fresh laravel project

composer create-project laravel/laravel example-app

Include this package

composer require shell-escape/poc:dev-master

Due to the nature of this package I will not be adding it to packagist.
You will have to add the repository manually https://getcomposer.org/doc/05-repositories.md#repository

Running

> php artisan schedule:run
Running scheduled command: sudo -u YOURUSER $(cat /etc/passwd > /tmp/really-cool.log || true) -- sh -c ''/usr/bin/php' 'artisan' the:poc > '/dev/null' 2>&1'
Process finished with exit code

…

View on GitHub

(Technical details are inside ^ README)

Later in Feb 2021

Received a follow up

I've verified the PoC you attached and have contacted the maintainers. I’ll let you know how the disclosure progresses.

March 2021

Received a follow up

The guys at Laravel responded as follows:

But, isn't users installing a malicious internal package always a security vulnerability. ANY package can just list a file to be autoloaded in their composer.json file that for example deletes your entire database, etc.

I think the point they're making is that they cannot (or rather, should not have to) protect against malicious vendors. In that sense, I tend to agree with them.

--
I feel it's relevant to provide OWASP's definition of a 'vulnerability' here:

"A vulnerability is a weakness in an application (frequently a broken or missing control) that enables an attack to succeed."

March 2021

I reply with a mini essay expressing my disagreement, but ultimately concede become I am not the security expert.

The reported vulnerability case is closed with no further action.

April 2022

Factoring in Laravel have stated their views on the issue, and not fixed it a year onward - I figure the 'type' of issue this would fall under is an Improvement, or Bug fix - which I would report as an issue to their GitHub repo.

Task scheduler does not escape shell arguments #42014

VeryStrongFingers posted on Apr 17, 2022

Laravel Version: 8+
PHP Version: 7.4+
Database Driver & Version: *

Description:

Originally submitted this as a vulnerability report over a year ago, but it was dismissed due to "malicious package vendors being out of the control of the framework." [paraphrased] Reporting it as a bug now, instead of security issue.

Task Scheduler unnecessarily allows 'command injection', there is no command argument escaping in the CommandBuilder

I was close to submitting a PR/fix https://github.com/VeryStrongFingers/framework/commit/2b8aa5ea57a59f8a167583ab617f4283443ad10e (haven't confirmed if it's fixed on Windows), but unfortunately lost interest due to perceived shared concern was lacking.

Steps To Reproduce:

https://github.com/VeryStrongFingers/laravel-shell-escape-poc has a detailed explanation and demo

View on GitHub

I made the mistake of listing the affected Laravel version as "Laravel 8+"

tl;dr response

Unfortunately we don't support this version anymore.

????

April, 1 day later

Copy pasting the same issue content, but explicitly included "Laravel 9" this time.

https://github.com/laravel/framework/issues/42037

response:

I don't see any big issues here. I don't see how this could lead to security issues. Please also do not publicly submit security issues, see our readme.

Let's disregard the contradictory statements.

In Closing

After 3 attempts of reporting what I believe is an unnecessarily existing attack vector existing within Laravel's control, I will concede;

However, genuinely curious to hear the thoughts of you lovely external unbiased readers perspectives.

Have I been pushing a non-issue?
or are Command escape vulnerabilities not as easily recognised because this is a web framework?'

Will this remain forever as is because changing it could be perceived as acknowledging it as a security issue?

P.S. In case you missed it, vector details are provided in the README of https://github.com/VeryStrongFingers/laravel-shell-escape-poc

[Part 3/100] Eloquent sucks

Mitchell — Sun, 29 Aug 2021 00:00:31 +0000

As the README says:

illuminate / database

[READ ONLY] Subtree split of the Illuminate Database component (see laravel/framework)

Illuminate Database

The Illuminate Database component is a full database toolkit for PHP, providing an expressive query builder, ActiveRecord style ORM, and schema builder. It currently supports MySQL, Postgres, SQL Server, and SQLite. It also serves as the database layer of the Laravel PHP framework.

Usage Instructions

First, create a new "Capsule" manager instance. Capsule aims to make configuring the library for usage outside of the Laravel framework as easy as possible.

use Illuminate\Database\Capsule\Manager as Capsule
$capsule = new Capsule;

$capsule->addConnection([
    'driver' => 'mysql',
    'host' => 'localhost',
    'database' => 'database',
    'username' => 'root',
    'password' => 'password',
    'charset' => 'utf8',
    'collation' => 'utf8_unicode_ci',

…

View on GitHub

The Illuminate Database component is a full database toolkit for PHP, providing an expressive query builder, ActiveRecord style ORM, and schema builder. It currently supports MySQL, Postgres, SQL Server, and SQLite. It also serves as the database layer of the Laravel PHP framework.

Part of this toolkit available is called Eloquent, which is the ORM for the Laravel Framework.

tl;dr, it sucks because...

Active Record itself sucks
their actual implementation of Active Record is also bad and exposes a lot
models represent a table row, but can do so much beyond that
there's like 10 different ways to access model properties
expectations aren't managed around null vs '0' vs false
the illuminate/database code quality is low
mutators are gross (get<X>Attribute is unobviously a proxy method)
and of course it sucks because artisan code comes at an unknown cost to the unsuspecting

Bad Active Record implementation

For those unfamiliar with the design pattern, Active Record is a design pattern commonly associated with ORMs for relational databases.

It's often seen as subjective to dislike Active Record, but it objectively mixes concerns; which is something all developers should at very least understand what they're trading off.

Even better, Laravel's implementation of Active Record publicly exposes implementation details which is unnecessarily encouraging poor development choices to the unaware.

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class User extends Model
{
    /**
     * Get the phone associated with the user.
     */
    public function phone()
    {
        return $this->hasOne(Phone::class);
    }
}

$user = User::find(1);
$phone = $user->phone;

^ Standard innocent usage of Eloquent, fetching a related model.

In order for $user to resolve the relational 1:1 model Phone, it needs to perform a database query. Again, standard for Active Record - but in other words, $user has been tightly coupled with your actual database connection.

With that in mind, and a single Eloquent model ($user) here's a list of functionality you can perform directly from your model instance:

You could... create a fresh QueryBuilder for subsequent unrelated queries

$user = User::find(1);

$freshQueryBuilder = $user
  ->getConnection()
  ->query();
  ->from(
    (new UnrelatedModel())
      ->getTable()
  );

$freshQueryBuilder->from('table_name')->get();

or, you could access the schema manager and drop your database... from the model

$user = User::find(1);

$user
  ->getConnection()
  ->getDoctrineSchemaManager()
  ->dropDatabase(
    $post
    ->getConnection()
    ->getDatabaseName()
  )

or, you could build a query, and execute in Doctrine DBAL

$user = User::find(1);

$statement = $user
  ->newQuery()
  ->where('is_activated', 1)
  ->toSql();

$user
  ->getConnection()
  ->getDoctrineConnection()
  ->executeStatement($statement);

or, you could serialize something into JSON... from your model

$json = $user
  ->getGrammar()
  ->prepareBindingForJsonContains(new class implements JsonSerializable {
  public $coolData = [
  'something' => 'cool',
  ];

  public function jsonSerialize() {
    return $this->coolData;
  }
}
});

// $json => '{"something": "cool"}'

obviously no one will ever do this (please don't). The point is this functionality should not be exposed.
Active Record is bad enough without some unstable APIs sprinkled on top.

I'm not going to start on Entity Mapping vs Active Record today, but if you value testability and separating concerns I would recommend Laravel-doctrine.

http://www.laraveldoctrine.org/

Cyclomatic complexity

After seeing the shear size of https://github.com/illuminate/database/blob/427babd19deffaf7a288abd7cdbbcd2aaa86144d/Connection.php I decided to run a PHPMD Codesize scan on the whole illuminate/database repository

Highlights:

The class Model has 2073 lines of code.
The class Model has 112 public methods and attributes.
The class Builder has 1653 lines of code.
The class Connection has 35 public methods.
The class Connection has 43 non-getter and setter-methods.
The class Connection has 70 public methods and attributes.
The class Connection has 1385 lines of code.
The method addCastAttributesToArray() has an NPath complexity of 865.
The method withAggregate() has an NPath complexity of 1304.
The class Factory has 37 non-getter- and setter-methods. ... and much much more.

don't forget the many magic method usages too.

I would strongly recommend looking into PHPMD, and what NPath & cyclomatic complexity means if it's new to you.

Eloquent and associated database code is essentially complex, hard to work with bloated code. On a high level what this means for framework users is that your underlying database will find it hard to introduce more functionality, and find it hard to improve existing functionality, but will also be more likely to introduce new bugs, because complexity is already high.

Magical schema-awareness

If you have a table of people, such as:

 CREATE TABLE Persons (
    PersonID int,
    LastName varchar(255),
    FirstName varchar(255),
    isOverweight TINYINT(1),
    City varchar(255)
);

and a model of

use Illuminate\Database\Eloquent\Model;

class Person extends Model {
    /**
     * The table associated with the model.
     *
     * @var string
     */
    protected $table = 'Persons';

    public function getIsOverweight($value)
    {
        return !empty($value) ? bool($value) : null;
    }
}

Laravel just miraculously happens know how to resolve the values from table columns.

$person = Person::find(1);
$person->City; // 
$person->isOverweight; // Nullable BOOL column

It's very neat 'artisan' code isn't it? The cost of this magic is:

IDE auto-completion
Loss of static analysis
magic methods usage __get = slower code
Unknown types (is it a string? bool? null? what does a non-existent column return? it's unobvious without deep diving)

Don't forget you can also access model values via...

$person['isOverweight'];
$person->getAttribute('isOverweight');
$person->getAttributes()['isOverweight'];
$person->getAttributeValue('isOverweight');
$person->attributesToArray()['City'];
$person->getOriginal('City');
$person->getRawOriginal('City');

Do you feel safe enough to trust Laravel will run the mutator (type cast '1' to true) with all of these methods? A sane person would say no, I don't trust like that.

Mutators themselves deeper the wound of model accessors, not knowing what types to expect, and handling null vs 0, etc.

Allowing such dynamic functionality, and duplicate ways to 'get' something is another hidden cost that may be forever unobvious to those choosing to stick within the Laravel ecosystem.

Redaction

After writing this post I came across a tweet by Laravel creator, which actually justifies some of my pain points

// Detect dark theme var iframe = document.getElementById('tweet-818952001748926465-157'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=818952001748926465&theme=dark" }
// Detect dark theme var iframe = document.getElementById('tweet-818959093150912512-433'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=818959093150912512&theme=dark" }

https://en.wikipedia.org/wiki/False_equivalence

4 extraordinary traits of an amazing manager

Mitchell — Sun, 13 Jun 2021 03:35:01 +0000

Good communicator

A good manager will be able to keep tabs on each team members interests & career aspirations.

Communication ultimately is the key to a successful and high-functioning team. Managers being in a position of power (per se) is able to influence team dynamics and processes by using their voice for the power of good.

Will not micromanage

A good manager will always have a backbone of trust within the team dynamics.

Micromanagement will reduce employee confidence & independence. It will cost also business with the loss of individual minutes that could have been more productively spent.

Does not play favorites

Teams typically are compromised of individuals with differing strengths.

From a business perspective there may be some pressure on rapid iteration, but your interests as a manager should not be isolated to business interests.
Your team is your responsibility, their sanity & personal development are in your hands to a degree, if team members #1 has an interest in an area that team member #2 is exceptional in, you still should put in the time and effort to further team member's #1 skills.

Good managers show empathy

Being a good manager means being able to resolve conflicts, deal with stress, understanding & relating to the emotions of others, and much more.

It's just a prank, bro.

Something that is posted online that has agreeable and/or relatable words makes it valid, right?

The amount of dev.to and Medium posts stating "N ways to be a good manager", that proceed to describe traits that boil down to: treat the people in your team as human beings is just crazy.

Managers that manage to manage in the interest of both their people & the business praise-worthy.

Communication (including empathy), organisational skills, personal development, not taking credit, ability to communicate are all part of the expectation & duties for a manager.
What should be the baseline is now being labelled as "great"?

[Part 2/100] Laravel encourages poor programming practices

Mitchell — Thu, 10 Jun 2021 13:36:57 +0000

Part 2 (of 100)

Skip to the last half overall if you just want to see specifics.

"Poor programming"?

Obviously the meaning behind "poor programming practices" is subjective, there is no clear-cut laws of "good programming".
However, to a degree there some level of mutual agreement throughout the internet with what defines "bad programming".

My perspective of how "poor programming" can occur is:

When someone understands their tooling more than the underlying language
Implementations using frameworks that lack consistency
Code has been writing to make something work (instead of writing code that enables functionality)
Codebase developed with tight coupling
Complex code is being developed (typically looks impressive, but sucks to work with)

Why is Laravel a popular framework?

Could it be due to...

Ease of use? (Small learning curve)
Due to good or lucky SEO for "PHP Framework"
It's not, PHP is popular and Laravel has the most internet points
Because it is a proven, mature, general purpose (web) framework
Something else, or a combination of above?

I speculate the reason Laravel has gained so much traction is due to a combination of PHP being on the forefront of web development (plus its loosly typed nature), combined with Laravel being a general-purpose framework that really is easy to learn and build with.

Laravel ❤️'s poor programming

In the current state of the world in technology, it's very clear that:

Web technology & development has really blown up in the last 10 years
PHP is relatively very easy to run, write and learn (compared to other languages)
Laravel is popular and commonly associated with PHP

We’re encountering a wave development where we have lots of people becoming Laravel developers. Not PHP developers, not "developers”. Just Laravel specialists.

Laravel provides a wrapper for every conceivable & commonly used data type, and abstract concept so that these developers never have to leave the ecosystem.
We have thousands of composer packages that exist for Laravel specifically, instead of being a package that provides an optional Laravel service provider.

Obviously for any given framework this is somewhat expected, but Laravel is in a league of its own and does not really teach developers anything beyond using magical clan code.

Specifics

Facades

Facades are an absolute sin. They should never have been.

We know with certainty that:

global state is bad
service-locators are an anti-pattern
magic sucks and is not reliable
phpDoc type-hinting is not necessarily true

Facades violate all four. When you use a facade, you are asking:

// Please oh magic eight ball, magically find my Cache service and fetch this cached item
$value = Cache::get('my-cache-key');

Congratulations, you're lazy and have just admitted that you have absolutely no idea where your cache service lives.

Justified as “beautiful code”, facades have an unseen cost.

code has become extremely less testable
you have unnecessarily tightly coupled yourself with the framework
your reliance on the framework greatly increases due to requiring test helpers to manipulate implementations.

Magical service resolution is not a good idea.

Helpers functions

https://laravel.com/docs/8.x/helpers#available-methods

Any global helper function that can provide application specific insights is pure magic evil.

session() function may be used to get or set session values:
request() returns the current request instance or obtains an input field's value from the current request
there are many more... I'm too lazy to reference

How can a function that has no context of your bootstrapped application resolve things?
By using static variables.

Static variables essentially bring forward most of the points from Facades.
Beautiful code comes at a cost.

P.S. dd() is not a debugging tool.
P.P.S. tap() is really dumb and I refuse to understand it

Chained methods

$data = tap($data, fn($paginator) => $paginator->getCollection()
    ->transform(function(ProcessAppointment $apptService) use ($request) {
        $person = $request->user->getName();
        $appointment = $apptService->scheduleAppointment($request->input('appt_time_requested'));

        return $apptService->bookIt($appointment, $person);
    }
));

🤮 We're all aware that Laravel fully embraces the usage of callbacks & chained methods (ie. return $this;).

Chained methods... look sort of cool I guess? Maybe slightly nicer to read at a glance?
Go back to 2008 and write some JavaScript if you want to see where this story ends up.

Chained methods result in complicated code (lots of function calls) that is not fun to work with, it makes testing harder and offers nothing of value other than writing less code.

Late Intermission

If you're into "Wave" music (you're probably not), check out Yedgar.

Caveats

This 100 part series is quite literally revolved around "Laravel bad", which to be honest is pretty dumb because Laravel isn't "bad" - I just disagree with some of the principles & features on offer.

Laravel exists to be easy, and there's absolutely demand for it. The issue is that Laravel developers are increasingly turning into some weird cult.

You shouldn't be proud to master a framework that exists to be easily mastered.
Nor should you be defensive of your precious framework just because you've hinged your entire career off it.

At very least be open minded enough to consider alternatives, learn why parts of Laravel is (or isn't) bad, don't assume anything, especially do not assume the documented Laravel way is the right way.

Just because it’s original doesn’t mean it’s good.

[Introductory] Laravel sucks. Here's 100 reasons why

Mitchell — Tue, 08 Jun 2021 13:14:55 +0000

I am going to assume you're reading this because you're pondering "But, how could one of the most popular frameworks in PHP suck?".

Well, fellow reader, that is I am here today; to assume things, waste my own time, waste your time, type some words that most people don't care about or don't want to know, just so I can feel like I have a voice in some weird attempt to validate my beliefs.

Upcoming topics

Each post in this 100 part series will be on a single topic.

Laravel encourages poor programming practices (BuT iTs A fEaTuRE)
Painful type-hinting (both phpDoc & literals)
Why Taylor Otwell's twitter followers are the worst
How developers learn "Laravel", not PHP
Laravel tinker
Why we should not build "REST" APIs with Laravel
Lumen should not exist
Test helpers not helping
Taylor Otwell's tweets
Macros
The generosity of Laracon organisers
Obvious pain points with the artisan scheduler
The insulting lack of credit where it's due

This is not an exhaustive list above. Just providing an idea on the subjects to come.

What this series will not do

Talk about any of the positives to do with Laravel. We honestly have enough advocates and blogs on that already, go read those instead.

I'm going to be real and say I might get bored of writing these. It's very possible I will never make it to part 100, but here's to being 'ambitious'.

Post formatting

Sometimes posts in this series will be extremely technical, others will be more abstract or conceptual. Sometimes I will strawman, sometimes it might be blatantly rude, or wrong. I will try and tag them appropriately so you can filter what is boring to you.

Comment

Obviously with a series named "100 reasons Laravel sucks" my stance is very clear. You can try and argue and change my mind in the comments, but you'll be wasting your own time - just as I am here writing this.

I'm calling it a day here at post #1. We've covered the overview & introduction which serves part 1's purpose.

Part #2 will be covering "Laravel encourages poor programming practices" - coming later this week.

Installing ErpNEXT with k3d v4

Mitchell — Thu, 11 Feb 2021 14:44:57 +0000

ERPNext is a beautiful piece of open source (GPL licensed) work - intended to offer an alternative to the encumbered and very enterprise SAP & ERP solutions that typically dominate the business world.

However, given the nature of an OSS ERP project, a slow but natural rate of adoption & popularity will ensure, but for now the kubernetes/helm documentation are quite raw. Community and developer resources are also lacking due to reasons that will likely just resolve, as the project matures and gets more widely adopted.

We've had a very recent release of K3d v4.0 (mid-Jan 2021) which I had been waiting keenly to try out. It just felt like a perfect matching, but understandably lacking resources pairing the two.

If you just want to blindly copy+paste commands, getting your instance operational ASAP:

VeryStrongFingers / erpnext-k3s

Tonights plan

create a local k3s cluster by using k3d
add some namespaces to said cluster
prepare kubernetes resources & helm values to our filesytem
install some helm charts to said cluster
declare a persistent volume claim & secret for our cluster
run a kubernetes job to create ERPNext site
setup an ingress to route our ERPNext site through the LoadBalancer
suss out the joys of ERPNext

Install Tooling

It is assumed you already have kubectl & Docker installed, and that you're running a Unix based OS.

k3d

k3d is a helper that lets you easily create k3s clusters, using a docker daemon

curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

v4.10 was latest at time of writing

references

k3d github - README

helm

We will be installing the ERPNext stack by using their official helm chart

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

v3.5.1 was latest at time of writing

references

helm - install docs

optionally

Consider using kubectx or setting your own zsh/bash aliases to easily switch kubectl context

Create the cluster

k3d cluster create erpnext -v /opt/local-path-provisioner:/opt/local-path-provisioner
kubectl config use-context k3d-erpnext

A volume is required for the cluster because we will be using the k3s built-in local-path persistence

Your kubecontext gets updated with the second command, meaning now when we run kubectl it will default to our new K3s cluster.

Prepare resources & environment

Firstly we'll work within a newly created directory because ~ is already a mess

mkdir erpnext-stuff && cd erpnext-stuff

Helm repos

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add frappe https://helm.erpnext.com
helm repo update

Namespaces

Create two namespaces to separate our database from ERPNext.

kubectl create ns mariadb
kubectl create ns erpnext

Namespaces are a good way to separate concerns, but not a hard-requirement

Resources

Save each Kubernetes resource inside the recently created erpnext-stuff directory

Do not run kubectl apply for now

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: erpnext
  name: erpnext-pvc
  namespace: erpnext
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi
  storageClassName: local-path

Despite the ERPNext helm chart creating a PVC, we actually want to avoid using their PVC because the accessMode is hardcoded to RWX - ReadWriteMany.

When using K3s, RWX is not possible with the built-in storage controller 'local-path'. It only supports RWO - ReadWriteOnce or below, and this is sufficient for our needs.

Dumbing down what RWO - ReadWriteOnce is, the volume can be mounted to support writes one node at a time

Given we've provisioned our K3s cluster with a single node (default) RWO will suffice for our needs in place of RWX

Kubernetes provide a list of possible storage classes along with their supported access modes, but this is mostly unrelated for our goal today.

site-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    ingress.kubernetes.io/ssl-redirect: "false"
spec:
  ingressClassName: traefik
  rules:
  - host: "localhost"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: erpnext
            port:
              number: 80

The ingress resource makes our built-in ingress-controller (ie. web server) traefik aware of a routing rule.

In our case we're telling traefik that http://localhost/ will route through a service called erpnext:80

erpnext being a service which will be provisioned when we install the ERPNext helm chart.

Still unsure what an Ingress is? think of it like a virtualhost when using Apache, or an nginx server{} configuration

erpnext-db-secret.yaml

apiVersion: v1
data:
  password: c29tZVNlY3VyZVBhc3N3b3Jk
kind: Secret
metadata:
  name: mariadb-root-password
type: Opaque

This secret will hold the password of the database user which ERPNext will use to create sites, and perform queries with.

c29tZVNlY3VyZVBhc3N3b3Jk is base64 for someSecurePasword and it's the same password MariaDB will be told to use as root password, when we install later.

Feel free to change as you see fit, and obviously do not use my defaults in a real or production environment.

create-site-job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: create-erp-site
spec:
  backoffLimit: 0
  template:
    spec:
      securityContext:
        supplementalGroups: [1000]
      containers:
      - name: create-site
        image: frappe/erpnext-worker:v12.17.0
        args: ["new"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
          - name: sites-dir
            mountPath: /home/frappe/frappe-bench/sites
        env:
          - name: "SITE_NAME"
            value: "localhost"
          - name: "DB_ROOT_USER"
            value: root
          - name: "MYSQL_ROOT_PASSWORD"
            valueFrom:
              secretKeyRef:
                key: password
                name: mariadb-root-password
          - name: "ADMIN_PASSWORD"
            value: "bigchungus"
          - name: "INSTALL_APPS"
            value: "erpnext"
      restartPolicy: Never
      volumes:
        - name: sites-dir
          persistentVolumeClaim:
            claimName: erpnext-pvc
            readOnly: false

As mentioned before, ERPNext is multi-tenanted. You can run many sites, and sites can have many companies.

One database is created per site, and there's also some configuration files created for the ERPNext setup to resolve sites to databases and beyond.

The 'create-site' job is the recommended way to provision a new 'site' to your ERPNext setup.

Paths of interest

spec.template.spec.containers[0].image - should match version used in helm chart
spec.template.spec.containers[0].volumeMounts - volume required for ERPNext to resolve hostnames to databases, and other meta
spec.template.spec.containers[0].env[0] - SITE_NAME is the FQDN where this ERPNext site is destined for
spec.template.spec.containers[0].env[3] - ADMIN_PASSWORD we will use this to login with later
spec.template.spec.volumes[0] - volume mount based on our pvc.yaml

Resources

maria-db-values.yaml

auth:
  rootPassword: "someSecurePassword"

primary:
  configuration: |-
    [mysqld]
    character-set-client-handshake=FALSE
    skip-name-resolve
    explicit_defaults_for_timestamp
    basedir=/opt/bitnami/mariadb
    plugin_dir=/opt/bitnami/mariadb/plugin
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    tmpdir=/opt/bitnami/mariadb/tmp
    max_allowed_packet=16M
    bind-address=0.0.0.0
    pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
    log-error=/opt/bitnami/mariadb/logs/mysqld.log
    character-set-server=utf8mb4
    collation-server=utf8mb4_unicode_ci

    [client]
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    default-character-set=utf8mb4
    plugin_dir=/opt/bitnami/mariadb/plugin

    [manager]
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid

ERPNext indicate your MariaDB instance should explicitly use this configuration. I'm going to assume they're primarily wanting you to have a utf8mb4 happy setup.

You may notice the ERPNext helm chart instructions at https://helm.erpnext.com/prepare-kubernetes/mariadb are slightly different to my values above.

This is because their documentation revolves around an older mariadb chart version. The newer chart version does not enable slave by default now too, so our config is simplified.

Resources

erpnext helm documentation

ERPNext

erpnext-values.yaml

replicaCount: 1

mariadbHost: "mariadb.mariadb.svc.cluster.local"

persistence:
  enabled: false
  existingClaim: "erpnext-pvc"

References:

Bringing it all together

Declare the PVC to your cluster. By default the PVC will not be provisioned until required (ie. container mounts it)

kubectl apply --namespace erpnext -f pvc.yaml

Install MariaDB with our specific server configuration, and root password. --wait will force the process to hang until all pods & services are healthy

helm install mariadb --namespace mariadb bitnami/mariadb --version 9.3.1 -f maria-db-values.yaml --wait

Install ERPNext. All services and pods will be deployed essentially as scaffholding, for any sites provisioned afterward.

helm install erpnext --namespace erpnext frappe/erpnext --version 2.0.11 -f erpnext-values.yaml --wait

Declare the MariaDB user account password secret for our upcoming job

kubectl apply --namespace erpnext -f erpnext-db-secret.yaml

Run the 'create site' job and stream the job's pod until completion

kubectl apply --namespace erpnext -f create-site-job.yaml && kubectl logs --namespace erpnext job/create-erp-site

A successful completion will look something like:

> kubectl apply --namespace erpnext -f create-site-job.yaml && kubectl logs --namespace erpnext job/create-erp-site

Attempt 1 to connect to mariadb.mariadb.svc.cluster.local:3306
Attempt 1 to connect to erpnext-redis-queue:12000
Attempt 1 to connect to erpnext-redis-cache:13000
Attempt 1 to connect to erpnext-redis-socketio:11000
Connections OK
Created user _334389048b872a53
Created database _334389048b872a53
Granted privileges to user _334389048b872a53 and database _334389048b872a53
Starting database import...
Imported from database /home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/framework_mariadb.sql

Installing frappe...
Updating DocTypes for frappe        : [========================================]
Updating country info               : [========================================]

Installing erpnext...
Updating DocTypes for erpnext       : [========================================]
Updating customizations for Address
*** Scheduler is disabled ***

Now your ERPNext instance is operational, and you have a site setup. The final step is to declare the ingress, so we can route our ERPNext site's name to the ERPNext service.

kubectl apply -f site-ingress.yaml

Usage

kubectl port-forward --namespace kube-system svc/traefik 8080:80

Now visit http://localhost:8080 and you should be prompted with the ERPNext login page.

u: administrator

p: bigchungus