DEV Community: Tim Geyssens

One-pager powered by json content files and synced to git

Tim Geyssens — Mon, 15 Sep 2025 18:15:58 +0000

Recently I took Warp for a testdrive. I have a couple of friends who have a eco-glamping and where in need of a website. So perfect opportunity to see the current state of AI in webdev.

And colour me impressed. I was able to create a one pager powered by a tailored CMS. The frontend is plain html/css/js powered by json files that are managed by the php backend.

Now as an additional challenge I wanted to make sure the content was also source controlled. Every *.json change and updated file in /images (the path where user uploaded files are stored) should be stored in git.

It took me a couple of prompts to end up with a php scripts that uses the github content api to check the status and if necessary update the content files.

Running this as a scheduled task on a Plesk interface from a cheap hosting solution.

`

<?php
/**
* GitHub Sync Script for DenBerendries - Using Contents API
*
* This script uses the GitHub Contents API which handles encoding automatically
* and syncs both JSON files in root and images in /images directory
*/

// Configuration
$GITHUB_TOKEN = '???';
$GITHUB_USERNAME = 'timgeyssens';
$GITHUB_REPO = 'DenBerendries';
$BRANCH = 'main';
$COMMIT_MESSAGE = 'Auto commit from server on ' . date('Y-m-d H:i:s');

// Set content type to JSON
header('Content-Type: application/json');

// Check if script is called via HTTP or CLI
$isHttp = (php_sapi_name() !== 'cli');

// Function to make GitHub API requests
function githubApiRequest($url, $token, $method = 'GET', $data = null) {
    $ch = curl_init();
    $headers = [
        'Authorization: token ' . $token,
        'User-Agent: PHP-GitHub-Sync-Script',
        'Content-Type: application/json',
    ];

    curl_setopt($ch, CURLOPT_URL, 'https://api.github.com' . $url);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);

    if ($method === 'POST') {
        curl_setopt($ch, CURLOPT_POST, true);
        if ($data) curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
    } elseif ($method === 'PUT') {
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
        if ($data) curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
    } elseif ($method === 'PATCH') {
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PATCH');
        if ($data) curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
    }

    $response = curl_exec($ch);
    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);

    return ['code' => $httpCode, 'body' => json_decode($response, true)];
}

// Function to find JSON files in root and images in /images directory
function findFilesToSync() {
    $files = [];
    $basePath = realpath('.');

    // Supported image extensions
    $imageExtensions = ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'bmp'];

    // 1. Find JSON files in root directory
    $rootIterator = new DirectoryIterator($basePath);
    foreach ($rootIterator as $file) {
        if ($file->isFile() && !$file->isDot()) {
            $extension = strtolower(pathinfo($file->getFilename(), PATHINFO_EXTENSION));

            // Check if it's a JSON file
            if ($extension === 'json') {
                // Get relative path from the base directory
                $relativePath = substr($file->getRealPath(), strlen($basePath) + 1);

                // Normalize path separators for GitHub
                $relativePath = str_replace('\\', '/', $relativePath);

                $files[] = $relativePath;
            }
        }
    }

    // 2. Find image files in /images directory
    $imagesPath = $basePath . DIRECTORY_SEPARATOR . 'images';
    if (is_dir($imagesPath)) {
        $iterator = new RecursiveIteratorIterator(
            new RecursiveDirectoryIterator($imagesPath, RecursiveDirectoryIterator::SKIP_DOTS),
            RecursiveIteratorIterator::SELF_FIRST
        );

        foreach ($iterator as $file) {
            if ($file->isFile()) {
                $extension = strtolower(pathinfo($file->getFilename(), PATHINFO_EXTENSION));

                // Check if it's an image file
                if (in_array($extension, $imageExtensions)) {
                    // Get relative path from the base directory
                    $relativePath = substr($file->getRealPath(), strlen($basePath) + 1);

                    // Normalize path separators for GitHub
                    $relativePath = str_replace('\\', '/', $relativePath);

                    $files[] = $relativePath;
                }
            }
        }
    }

    return $files;
}

// Function to get file SHA from GitHub
function getFileShaFromGitHub($token, $username, $repo, $branch, $filePath) {
    $url = "/repos/$username/$repo/contents/$filePath?ref=$branch";
    $response = githubApiRequest($url, $token);

    if ($response['code'] === 200) {
        return $response['body']['sha'];
    }

    return null;
}

// Function to sync files using Contents API
function syncToGitHub($token, $username, $repo, $branch, $commitMessage) {
    $result = [];

    // Find all files to sync
    $allFiles = findFilesToSync();
    $result['all_files'] = count($allFiles);

    if (count($allFiles) === 0) {
        return ['success' => true, 'message' => 'No files found to sync'];
    }

    $changedFiles = [];
    $skippedFiles = [];

    foreach ($allFiles as $file) {
        if (file_exists($file)) {
            $localContent = file_get_contents($file);
            $localSha = sha1("blob " . strlen($localContent) . "\0" . $localContent);

            // Get the remote SHA
            $remoteSha = getFileShaFromGitHub($token, $username, $repo, $branch, $file);

            // If file doesn't exist in remote or SHA is different, it's changed
            if (!$remoteSha || $remoteSha !== $localSha) {
                $changedFiles[] = $file;
            } else {
                $skippedFiles[] = $file;
            }
        }
    }

    $result['changed_files'] = count($changedFiles);
    $result['skipped_files'] = count($skippedFiles);
    $result['files'] = $changedFiles;

    // If no files changed, return early without creating a commit
    if (count($changedFiles) === 0) {
        return ['success' => true, 'message' => 'No changes to commit', 'action' => 'none'];
    }

    // Update each changed file
    $successCount = 0;
    $errorCount = 0;
    $errors = [];

    foreach ($changedFiles as $file) {
        if (file_exists($file)) {
            $content = file_get_contents($file);
            $sha = getFileShaFromGitHub($token, $username, $repo, $branch, $file);

            $data = [
                'message' => $commitMessage,
                'content' => base64_encode($content),
                'branch' => $branch
            ];

            // If file exists, include the SHA for update
            if ($sha) {
                $data['sha'] = $sha;
            }

            $response = githubApiRequest("/repos/$username/$repo/contents/$file", $token, 'PUT', $data);

            if ($response['code'] === 200 || $response['code'] === 201) {
                $successCount++;
            } else {
                $errorCount++;
                $errors[] = "Failed to update $file: " . json_encode($response['body']);
            }
        }
    }

    if ($errorCount > 0) {
        $result['success'] = false;
        $result['message'] = "Completed with $successCount successes and $errorCount errors";
        $result['errors'] = $errors;
    } else {
        $result['success'] = true;
        $result['message'] = "Sync completed successfully. Updated $successCount files.";
    }

    $result['action'] = 'pushed';
    return $result;
}

// Handle the request
try {
    $startTime = microtime(true);

    // If called via HTTP with a custom message
    if ($isHttp && isset($_GET['message'])) {
        $COMMIT_MESSAGE = $_GET['message'];
    }

    $result = syncToGitHub($GITHUB_TOKEN, $GITHUB_USERNAME, $GITHUB_REPO, $BRANCH, $COMMIT_MESSAGE);
    $result['execution_time'] = round(microtime(true) - $startTime, 2) . ' seconds';

    if ($isHttp) {
        echo json_encode($result, JSON_PRETTY_PRINT);
    } else {
        // CLI output
        if ($result['success']) {
            if ($result['action'] === 'pushed') {
                echo "✅ Sync completed successfully!\n";
                echo "📊 All files: " . $result['all_files'] . "\n";
                echo "📊 Changed files: " . $result['changed_files'] . "\n";
                echo "📊 Skipped files: " . $result['skipped_files'] . "\n";
                echo "⏱️  Execution time: " . $result['execution_time'] . "\n";

                // List files if there are not too many
                if ($result['changed_files'] > 0 && $result['changed_files'] <= 20) {
                    echo "\n📄 Files synced:\n";
                    foreach ($result['files'] as $file) {
                        echo "  - $file\n";
                    }
                }
            } else {
                echo "✅ No changes to commit. Everything is up to date.\n";
                echo "📊 Files checked: " . $result['all_files'] . "\n";
                echo "⏱️  Execution time: " . $result['execution_time'] . "\n";
            }
        } else {
            echo "❌ Sync completed with errors:\n";
            echo "📊 Successes: " . ($result['changed_files'] - count($result['errors'])) . "\n";
            echo "📊 Errors: " . count($result['errors']) . "\n";
            echo "⏱️  Execution time: " . $result['execution_time'] . "\n";

            foreach ($result['errors'] as $error) {
                echo "❌ $error\n";
            }
        }
    }
} catch (Exception $e) {
    $error = ['success' => false, 'message' => 'Exception: ' . $e->getMessage()];

    if ($isHttp) {
        echo json_encode($error, JSON_PRETTY_PRINT);
    } else {
        echo "❌ Exception: " . $e->getMessage() . "\n";
    }
}

// If called via HTTP, add some HTML styling
if ($isHttp) {
    echo <<<HTML
    <!DOCTYPE html>
    <html>
    <head>
        <title>GitHub Sync Result</title>
        <style>
            body {
                font-family: Arial, sans-serif;
                margin: 20px;
                background: #f5f5f5;
            }
            .container {
                max-width: 800px;
                margin: 0 auto;
                background: white;
                padding: 20px;
                border-radius: 5px;
                box-shadow: 0 2px 5px rgba(0,0,0,0.1);
            }
            .success {
                color: #2ecc71;
                font-weight: bold;
            }
            .error {
                color: #e74c3c;
                font-weight: bold;
            }
            .info {
                color: #3498db;
                font-weight: bold;
            }
            pre {
                background: #f8f8f8;
                padding: 15px;
                border-radius: 5px;
                overflow: auto;
            }
        </style>
    </head>
    <body>
        <div class="container">
            <h1>GitHub Sync Result</h1>
            <p>Repository: <a href="https://github.com/$GITHUB_USERNAME/$GITHUB_REPO" target="_blank">$GITHUB_USERNAME/$GITHUB_REPO</a></p>
            <p>Scanning: JSON files in root directory and images in /images directory</p>
            <p>Commit message: $COMMIT_MESSAGE</p>
        </div>
    </body>
    </html>
HTML;
}

Making use of the dropzone in your weweb components

Tim Geyssens — Wed, 26 Jun 2024 08:47:48 +0000

In this article I'll explain how to make use of the dropzone when building an app on the nocode/lowcode weweb.io platform.

Dropzone?

Just to get this clear, the name might suggest that it has to do with file uploads but in this context it is not!

A dropzone will enable you to make the components you create in weweb even more versatile and dynamic. Since by integrating a dropzone into a component it still allows you to add additional items to the component when implementing it in your app, so on the instance of the component (this will get clear in the use case and example). See it as a placeholder you can populate when making use of your component. All to make sure we do not have to repeat our selves by creating a lot of similar items/components.

If you are not familiar with components make sure to check the documentation on those since we will extend on that topic (components, component variables and component workflows are important concepts to have a grip on before starting with this article).

The use case

Let say we have to implement these 3 contact forms on our app.

Form A consists of a name/email and message (so used as a contact form).

Form B consists of a *name/email a department selector and message *(for contacting a specific department)

And form C consists of a name/email a product name and serial number (for a contact relating to a specific product)

All forms need to expose the form data as a json object. Since that will be used down the line.

You can see there is a lot of overlap between the forms.

They all have name/email/message and the submit button (highlighted in yellow).

Instead of creating 3 different forms we can simply create a single form component containing the mutual items.

And extend the instances of the component if necessary with additional items (in this case extra form inputs).

Ground work

Lets first setup the form component with a variable. The goal of the form is that it will expose the form fields as an object.

{ "name": "Tim", "email": "tim@weweb.io", "message": "<p>drop it like it's hot</p>" }

When implementing the form component we can then send the exposed data to the workflow/service we wish.

A simple contact form containing the 3 inputs (the inputs have been given a name, so it is easy to keep track when we are binding afterwards).

Now we have to make sure that the component exposes the data. By adding a variable to the component called "data_to_submit" of type object (since this will contain several values) and we will be exposing this outside the component so we can handle the data when the form is in use.

The variable setup should look like this

Next we need to populate the variable with the values of the inputs.

Adding a workflow on each input (so the name, email and message input)

We want to set the value of a property **of the exposed data to the value of the corresponding input**.

So that we end up with this result

{ "name": "Tim", "email": "tim@weweb.io", "message": "<p>drop it like it's hot</p>" }

The workflow will trigger on change and it will c*hange the data_to_submit variable*.

But it will only partially change it (when triggered on change of the** name input** it should only change the name property, when triggered on email only the email property,...)

Select the data_to_submit variable, chose to partial update and set the path to the corresponding property you wish to update (name for name, email for email,...)

Finally we can bind it to the input values by selecting the correct form field component variable (aka the name of the input followed by value).

Now we have a working form component that exposes data containing all the form input values!

Component workflow

Of course we need to make sure items added in the dropzone can call a workflow on the form component. So define the following.

The workflow needs 2 parameters (that will be populated from items in the dropzone). The key and value of the property on the data_to_submit we want to fill.

So add those 2.

With those in place we can setup the rest of the workflow.

Allow execution from outside and from a dropzone. To make sure the workflow is accessible.

Binding the path the the key param **and the **value to the value param created earlier.

Hello dropzone

Just add it in the place where you want to be able to add additional form fields when using the form component.

Connecting the dots

Now we are able to execute the workflow on the form component from items inside the dropzone.

Not in the component editor but in your instance of the component you can add extra items into the dropzone.

You can see that the dropzone is visible with a dotted line.

And you can just add what you want, making the component a lot more flexible and dynamic!

I'll add a new field with the name field_from_instance and will execute the component workflow on change of the value.

For the key param I just set something static but for the value I bind it to the event value. (so since this is triggered on change of an input the event value will be the value of the input).

If you now check the variables in the debugger you will notice that the data_to_submit will also update with the data from the dropped in input.

Of course** keys need to be unique**. So a unique key per input.

And that is it, a base contact form component that features a dropzone to allow further customisation when using the component in your app.

Umbraco Forms Fieldtype with custom client and serverside validation

Tim Geyssens — Fri, 25 Nov 2022 09:12:14 +0000

Umbraco Forms allows you to add custom fieldtypes, I won´t go into detail on how to do that since you can find detailed docs over at our umbraco

As you can see it is also easy to add server side validation on a fieldtype level.

Let´s say you want have a phone number field and validate the number agains on api (to make sure it exists)

 public override IEnumerable<string> ValidateField(Form form, Field field, IEnumerable<object> postedValues, HttpContext context, IPlaceholderParsingService placeholderParsingService)
        {
            var returnStrings = new List<string>();

            var service = new NumverifyService();

            if (postedValues.Any() && !service.Verify(postedValues.First().ToString()))
            {
                returnStrings.Add(ValidPhoneNumber);
            }

            returnStrings.AddRange(base.ValidateField(form, field, postedValues, context, placeholderParsingService));

            return returnStrings;
        }

So that is the server side validation sorted.

For client side I´ll call the same NumverifyService from a api controller

    public class FormApiController : UmbracoApiController
    {
        private readonly NumverifyService _service;

        public FormApiController()
        {
            _service = new NumverifyService();
        }


        [HttpGet]
        public bool Check()
        {

            return _service.Verify(HttpContext.Current.Request.Params[0]);
        }
    }

Once thing to notice here is that I don´t use a param in the check method since form fields are named based on a guid...not on the alias of the form field. So I´ll just fetch the first request param.

So now how do you hook this up with the custom fieldtype? And do remove validation (Remote Validation is a technique that uses client side script to validate user input on the server without posting the entire form)

Since by default forms is using unobtrusive validation it can be handled by adding these attributes to the element. in the form field razor view

       data-val-remote="Not a valid phone number"
       data-val-remote-additionalfields="*.@Model.Name"
       data-val-remote-url="/Umbraco/Api/FormApi/Check/"

the error message, the field to send to the remote url and the url of the api method.

So no need for additional js...

A simple send umbraco page as test mail content app

Tim Geyssens — Mon, 10 Jan 2022 09:23:02 +0000

Content Apps are companions to the editing experience when working with content or media in the Umbraco backoffice.

So unlike prop editors they don't store any data against the current document.

A usecase I recently encountered was the ability to generate newsletters from Umbraco and send them with a third party tool (so import the outputted html with a mailing service like mailchimp).

But the editors also wanted the ability to first send a single test mail from within Umbraco before migrating the markup.

The solution consists of an API Controller that fetches the rendered output and sends that as an email.



using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Mail;
using System.Text;
using System.Threading.Tasks;
using Umbraco.Core.Logging;
using Umbraco.Web.WebApi;
using Umbraco.Web;
using Umbraco.Core.Models.PublishedContent;
using System.Web.Http;

namespace Acme.Core.Controllers.API
{
    public class NewsLetterController : UmbracoAuthorizedApiController
    {
        private readonly ILogger _logger;
        private readonly IProfilingLogger _profilingLogger;
        public NewsLetterController(ILogger logger, IProfilingLogger profilingLogger)
        {

            this._logger = logger;
            this._profilingLogger = profilingLogger;
        }

        [HttpGet]
        public bool SendTestMail(int newsletterId, string email)
        {
            var doc = this.Umbraco.Content(newsletterId);

            if (doc != null)
            {
                string markup = string.Empty;

                if (this.GetHtmlByUrl(doc.Url(mode: UrlMode.Absolute), out markup))
                {
                    var message = new MailMessage("noreply@acme.com", email);
                    message.Subject = doc.Name;
                    message.Body = markup;
                    message.IsBodyHtml = true;
                    var client = new SmtpClient();
                    client.Send(message);

                    return true;
                }
            }

            return false;
        }

        private bool GetHtmlByUrl(string url, out string fullHtml)
        {
            using (this._profilingLogger.DebugDuration<NewsLetterController>("GetHtmlByUrl(" + url + ")", "GetHtmlByUrl(" + url + ") done"))
            {

                if (!Uri.TryCreate(url, UriKind.Absolute, out Uri uri))
                {
                    fullHtml = "";
                    return false;
                }

                try
                {
                    var httpTimeout = 1000;

                    var cookieDictionary = new Dictionary<string, string>();
                    var webRequest = (HttpWebRequest)WebRequest.Create(uri);
                    httpTimeout *= 1000;
                    webRequest.Timeout = httpTimeout;
                    webRequest.UserAgent = "NewsLetterController";

                    if (cookieDictionary != null && cookieDictionary.Count > 0)
                    {
                        var container = new CookieContainer();
                        var domain = webRequest.Address.DnsSafeHost;
                        foreach (var cookie in cookieDictionary)
                        {
                            container.Add(new Cookie(cookie.Key, cookie.Value, "/", domain));
                        }

                        webRequest.CookieContainer = container;
                    }

                    var webResponse = (HttpWebResponse)webRequest.GetResponse();
                    using (var sr = new StreamReader(webResponse.GetResponseStream()))
                    {
                        fullHtml = sr.ReadToEnd();
                    }

                    return true;
                }
                catch (WebException ex)
                {
                    this._logger.Error<NewsLetterController>(ex, $"Error in NewsLetterController retrieval. Url: {url}");
                    fullHtml = string.Empty;
                }

                return false;
            }
        }
    }
}

And the actual content app, that outputs a text input and a submit button.

View



<div ng-controller="Custom.SendPageAsMail as vm">
  <umb-box>
    <umb-box-header title="Send test mail"></umb-box-header>
    <umb-box-content>
      <input type="text" ng-model="vm.emailAddress" class="umb-property-editor umb-textstring textstring" required no-dirty-check />
      <a class="control__button btn">
        <i class="icon icon-message" ng-click="sendTestMail()">Send</i>
      </a>
    </umb-box-content>
  </umb-box>
</div>

Controller (also populates the input with current user email)



angular.module("umbraco")
    .controller("Custom.SendPageAsMail", function ($scope, editorState, $http, userService) {
        var vm = this;
        vm.CurrentNodeId = editorState.current.id;
        vm.emailAddress = "";

        function _sendTestMail() {

            $http.get('/Umbraco/backoffice/Api/NewsLetter/SendTestMail?newsletterId=' + vm.CurrentNodeId + '&email=' + vm.emailAddress).then(function (response) {

            });


        }

        function _init() {

            $scope.sendTestMail = _sendTestMail;

            userService.getCurrentUser().then(function (user) {
                vm.emailAddress = user.email;
            });
        }

        _init();



});

Package manifest



{

    "contentApps": [
    {
        "name": "Send testmail", 
        "alias": "sendpageasmail", 
        "weight": 0, 
        "icon": "icon-message",
        "view": "~/App_Plugins/SendPageAsMail/SendPageAsMail.html",
        "show": [

            "+content/newsLetter",
            "+contentType/newsLetter"
        ]

    }
    ],

    "javascript": [
        "~/App_Plugins/SendPageAsMail/SendPageAsMail.controller.js"
    ]
}

A Flexible Umbraco Embed Provider for url to iframe stuff

Tim Geyssens — Thu, 18 Nov 2021 13:05:05 +0000

A feature that I introduced in Umbraco way back in 2012 (damn I'm getting old) is an easy way to embed third party media by simply providing the url. Behind the scenes that takes advantage of the oembed format but of course not all sites that offer embeddable content have an api endpoint ... sometimes it's also just going from url to an iframe without extra stuff... but you still want to give editors the easy "url only" way of embedding.

So for a recent project I extended the default Umbraco behaviour with a fresh Embed Provider

    public class UrlToIframeEmbedProvider: EmbedProviderBase
    {
        // no ApiEndpoint!
        public override string ApiEndpoint => String.Empty;
        public override string[] UrlSchemeRegex
        {
            get
            {
                var urls = new List<string>();
                var doc = new XmlDocument();
                doc.Load(System.Web.Hosting.HostingEnvironment.MapPath("~/config/UrlToIframeEmbedProvider.xml"));

                if (doc != null)
                {
                    foreach(XmlNode node in doc.SelectNodes("//embed"))
                    {
                        urls.Add(node.Attributes["regex"].Value);
                    }
                }

                return urls.ToArray();
            }

        }

        public override Dictionary<string, string> RequestParams => new Dictionary<string, string>();

        public override string GetMarkup(string url, int maxWidth, int maxHeight)
        {
            var markup = string.Empty;
            var doc = new XmlDocument();
            doc.Load(System.Web.Hosting.HostingEnvironment.MapPath("~/config/UrlToIframeEmbedProvider.xml"));

            if (doc != null)
            {
                foreach (XmlNode node in doc.SelectNodes("//embed"))
                {
                    var m = Regex.Match(url, node.Attributes["regex"].Value, RegexOptions.IgnoreCase);
                    if (m.Success)
                    {
                        markup = node.InnerText;
                        break;
                    }
                }
            }

            if (!string.IsNullOrEmpty(markup))
            {
                return string.Format(markup, url, maxWidth,maxHeight);
            }else
            {
                return markup;
            }
        }
    }

You can now simply populate the config file with different url regexes and the embed codes using the string.format placeholders

{0}: the url
{1}: the width
{2}: the height

<?xml version="1.0" encoding="utf-8" ?>
<customEmbeds>
  <embed regex="site\.com/*">
    <![CDATA[<iframe width="{1}" height="{2}" style="width: {1}px; height: {2}px; border: none; overflow: hidden;"
        src="{0}" webkitAllowFullScreen="true" mozallowfullscreen="true" allowFullScreen="true"
        frameborder="0" scrolling="no" allowtransparency="true" ></iframe>  ]]>
  </embed>
</customEmbeds>

Umbraco HQ AMA session, get your questions in!

Tim Geyssens — Tue, 02 Feb 2021 11:20:52 +0000

During some recent events HQ asked for questions from the community but they never got answered, so I launched an attempt at having an open AMA session.

Will be hosted by the great folks of Umbracoffee

Get your question(s) in -> google form
View the current questions -> google sheet

The open source sponsorship experiment

Tim Geyssens — Fri, 18 Dec 2020 11:47:07 +0000

As you may have noticed. You can now also sponsor an open source contributor as a company (a.k.a get an invoice)...

For the full details read the post over at github

It's no secret that finding a business model with open source is a hard nut to crack... especially when it is a smaller niche product/addon.

So what happens a lot is that open source projects simply turn into abandonware (creator loses interest). As a dev with the code still available you could spend time into adding/fixing what you want of course... but plenty of folks around that just use it without the tech knowledge required to update.

Years ago I made a package that allows you to prevent conflicting edits in Umbraco v7.

Short video here:
https://screencast-o-matic.com/watch/co6Vjvf8Lc

Now there have been a couple of folks requesting a v8 version...

As an experiment I now want to see if I can get $100 in sponsorship, once I hit that target I'll make the v8 version.

You can find my sponsor page over at https://github.com/sponsors/TimGeyssens

And different tiers:

So 2 companies in the $50 tier or 20 in the $5 tier (or another combo) would work! I don't expect individual personal dev contributions. Compared to doing the dev inhouse this is peanuts.

Let's see how this works out.

Setting up an Umbraco Baseline-ish flow with Azure Devops

Tim Geyssens — Tue, 08 Dec 2020 21:39:23 +0000

In my previous post I outlined my experiences with Umbraco Cloud. It boils down to the fact that if you are a team of devs and follow a gitflow or similar release management workflow Cloud just isn't an option.

So I was investigating alternatives that allow a release management workflow... but I still want a baseline and auto upgrades.

Azure devops

DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.

Devops consists of a lot more but in this post I'll outline how I'll use CI

Continuous Integration (CI) is the process of automating the build and testing of code every time a team member commits changes to version control.

and CD

Continuous Delivery (CD) is the process to build, test, configure and deploy from a build to a production environment.

Of course when developing Umbraco sites you keep your code in sourcecontrol adding CI into the mix we can make sure the code "builds" when new changes are commited.

Staging site

For simplicity I'll keep everyting in Azure devops. But you can also have this in for example github.

Say I have a Umbraco project with a Core + Web + Test project.

You can then first setup a pipeline that builds and deploys the test version (from the release branch). So new features get developed on feature branches, when they need to go to the client for review you create a pull request or merge the feature branch with release.

Setting this up is pretty simple..just add a new pipeline (and I tend to pick the classic visual editor)

Next you need to select the source... when picking azure you just need to select from dropdowns (which project/repo/branch), easy!

After that you can select a pipeline template (I pick the azure web app for asp.net, since Umbraco is a asp.net web app and I want to deploy to azure)..

Picking that template most stuff is already in place.

The only step that needs attention is selecting the azure subscription and App service you want to deploy to... (so of course you need an azure subscription with an existing app service).

Once all that is selected you now have a working CI, CD process for a test environment. So codes get's pushed to the release branch... that triggers a build and deploy...

One small side note, best is to enable xml transformation on the Azure App Service deploy task, that way you can have *.Release.config file transforms (like for settings that are different between your dev env and the actual test env). Stuff like conn strings can be also set that way but also on devops or in azure... (depending on how you want it configured). Plus of not having it in sourcecontrol is that there is an extra level of security (sensitive settings shouldn't go in source control).

Live sites

Of course now we have a staging version of the baseline site... but I also want child websites using that base...

So adding a new pipeline that will build an artifact (webdeploy package) from the main branch. Following the same steps as before but removing some tasks and adding 1 new one

This will now run when new code is commited to the main branch... each time a new artifact will be generated.

Important here is the Artifact name (we need this name later) and also the File or directory path, for this I used an available variable $(System.DefaultWorkingDirectory)\ (not sure if that is the correct way but it works!)

Next is setting up the release pipeline, so in the releases part of pipelines just add a new one (this should get you started with an artifact and stage)

The artifact then points to the latest pipeline (where we create the artifact), we also need to specify the artifact name here. (needs to match with what is on the pipeline)

Finaly a task per child is setup... this just uses a Azure App Service deploy task that points to the correct app service (again you need to have the app service created on your portal)

Again stuff like conn string shoudln't be set in the web.config in source control but you can set these in devops or in the azure portal.

So now when the build of the artifact is finished the deploy will happen to my 2 child sites using the same base...

Basics

Of course this outlines the basics... I also have uSync in the mix that handles the version control of a lot of umbraco stuff (like doctypes,datatypes,...) and auto imports these on the child sites...
Then with silent upgrade also a amazing free addon to Umbraco from Kevin Jump added we can first test an upgrade localy and if that succeeds roll it out as part of a release ... just add the nuget package and set (possible with a config transform)

Further

Now I have 1 codebase for all the sites... with a theme picker this could work in having a different look and feel for each child site... so a content defined theme... if you do want a different code base that is also possible. New git repo, add the base as an origin and you can pull/push from the base... of course you then need to setup a new build/release pipeline for that case...

You could also have staging sites for all the child one... just create a release artifact and add a new release pipeline...deploying to staging locations...

Of course blog storage is also possible, again, each child would have it's own config...

Key is to develop with the concept of a base site in mind, so make everything as flexible as possible, might even introduce some interfaces so you can switch different bits depending on config. For example an applicant tracking system, child a might be using umbraco content, child b an external api... making that swappable still allows a single code base...

If you want to fetch content from live to develop/test against you can add uSync complete that contains uSync publisher. Or just download the db and run on that locally...

Pros

Plug and play in your existing dev flow (doesn't matter where you have your sourcecode, github, bitbucket,...)
Superbe logging of each step, if something goes wrong you have detailed logs available in devops... you can see them as the build/deploy is in progress
Possible to run unit tests and don't deploy if these fail
Not a single point of failure, say the hosting goes down you can setup a deploy to a new endpoint... devops goes down, use a publish profile directly from vs
No sudden price increase when you hit certain usage limits (pageviews/node/domain/bandwidth/media) it just scales depending on what you use.
You can take advantage of the already existing task in devops to further finetune the CI CD , like a site warmup step, thanks Sjors!
Forum support included ;)
Kevin provides stellar support (even for his free addons)

Cons

You need some devops/azure knowledge (but plenty of documentation out there, since it is all coupled to your microsoft account it's super easy to select the correct stuff...)
Setting up a new child takes some manual work (about 30 mins) create app service, db, update the release pipeline...(you could look into automating this if you have the knowledge/time)

Overall

Unless Umbraco Cloud introduces a significant price drop and would allow for a release management workflow I wouldn't look back twice. Azure Devops is a really mature product that makes the whole continuous Integration/delivery a breeze...

From what I hear other folks are also using Octopus Deploy and Github actions to get a similar workflow.

Is Umbraco Cloud a good fit for your Umbraco project?

Tim Geyssens — Wed, 25 Nov 2020 12:39:08 +0000

I've been around a long time in the Umbraco dev community, one of the struggles in the early years was having a decent dtap flow

Umbraco is a db driven asp.net project. Some things are stored on filesystem like js/css/templates but others are in the db. Also in v2-7.x everything in the db only had a unique integer id. Once you then had multiple environments and you do some further development these id's won't match...
Live gets new content, dev gets new doctypes... so the id of these new doctypes wouldn't be the same accross environments. Often deploying stuff between environments also consisted of doing manual changes to keep everything in sync...so not exactly an easy/automated process. And also not possible to put all bits in sourcecontrol...

Courier

Enter Umbraco Courier at some point HQ released Courier, a commercial addon that allows you to transfer changes between environments...this even allowed revisions/dependencies checking... but as you know Umbraco is highly customizable and you can add new sections/prop editors/data types... and those can work with content/media/other ids in a nested structure... So to make everything sync at the push of a button is a super complex thing...Courier worked good with pretty standard setups but it had/has it pains...

uSync

Enter uSync since Courier wasn't cutting it for all cases a community member released uSync. This serializes the db stuff to disk (so you can have it in sourcecontrol) and can import those disk items back into Umbraco/db on out of sync environment... I'm not sure about all the details it had initialy but I don't think it was able to serialize everything...but these days it does... even content...

Cloud

Enter Cloud of course running umbraco is more then just source control so HQ released Cloud, which allows you to spin up a new Umbraco instance in a couple of minutes ... add different environments and sync between these. Also has auto upgrades, sourcecontrol of course and other features like Baselines... so it is more then just hosting...At that time the pricing model was also simpel...pay per env.. no limits on bandwidth/nodes/storage/domains

Initial finding

When I first started exploring Cloud it looked great, for a client that wanted a decent dtap I tested the different options at that time (4-5 years ago) and made the recommendation to go on cloud with a big multisite project.

Dtap

THe development of the project was going ok.... the developers used the grid with doctype grid editor... so far so good. The initial sites then whent live ... and a new env was added to do further dev.

Of course since there wasn't a recommended approach for building the backend, grid and doctype grid editor was in place so id's of items could be nested pretty deep (in json stored in the db).

Unfortunately during these early Cloud months that caused a lot of pain...deploys would fail, site would go down... pushing that deploy button was always a super stressy moment. And if it was down you would need to contact support to get it back...Combined with some cloud outages the decision was made to move the site to Azure... currenlty those have been running smoothly on azure with USync combo... Also there wasn't a really good way of keeping a main repo (out of cloud) in sync and pushing from a CI way to cloud.

Further attemps

In the following years I did a couple of other deploys to Cloud, since then the platform matured and 1 item that got added was the uid (so instead of having a different int id accross envs, items now had a single unique umbraco uid that was equal on all). This worked ok in a single developer setup...but of course sticking with Vanilla Umbraco as much as possible, since adding third party bits in some cases also needs extra work to make it work with Deploy/Cloud... + the main repo was also the Cloud one.

Again some cloud outages and bugs that got in due to auto upgrades was not making it a 100% smooth operation. Combined with the new limits on pageviews/content nodes/domain names/bandwidth/storage... the azure move was done again. Since that scales (from a price point of view) in a more fluent way then a sudden increase from 40$ to 210$ a month.

Recent evaluation

In the past months I also checked the state of Cloud. It isn't a secret that in the past month 4 big outages happened that had an effect on all sites.. you can check the history on https://status.umbraco.io/. And the post on the official blog.

Also there are posts that cover how to prepare for cloud outages that are still valid today...

One of the plans was to use baselines in order to setup different sites. But it appears that this isn't as fluent as I imagined it would be. With multiple devs a lot of collision errors happen: https://our.umbraco.com/documentation/Umbraco-Cloud/Troubleshooting/Deployments/Structure-Error/ there is even a video series on youtube describing on how to handle this. As you see this needs powertools (kudu and powershell) so the complexity that is supposed to hide is still needed...Also if you need support in getting these fixed you need to have the correct plan...

Also there is no way to do feature branches, have these on staging for a client to test and then deploy the ones you wish to live... it's all or nothing... So if you need to deploy a hotfix... you can't go trough staging to deploy to live since that would deploy everything... some details on that here on the forum

Verdict

If you are a solo dev and the client has budget to spend on Cloud (doesn't worry about the price increase) it can make sense if you don't want to setup your own env on azure or similar. If the Cloud platform becomes stable that is.

If you are a team of devs, have a bit of devops knowledge, want more control and have gitflow or similar approach. Use Azure devops and add uSync in the mix. No need for the extra layer...

Also be aware that currently Umbraco Cloud is run on VMs... Like seen in the last outages if something goes wrong this can/will affect a lot of sites.

Wishlist

What I would like to see is an Umbraco Devops... that you can just fit in your existing flow to get a decent DTAP. Imagine you signup to a new umbraco.io project, you just point it to your existing repo/branch...(from github for example), then you setup the build with optional tests... could be that cloud has default tests for core/heavily used packages... you could also write/record your own. Once the build and tests are succesfull you can setup where you want to deploy the site. To Azure, a vm, a dedicated server, ftp,...Then Deploy could be used to restore data between backends... (pulling live content to dev to continue dev on a new feature)

Of course this is all based on my experiences with the platform over the past years...feel free to share your thoughts or tips on how to get it smooth...

UI-O-Matic 3.1 for Umbraco the Hacktoberfest 2020 edition is out!

Tim Geyssens — Wed, 25 Nov 2020 08:46:59 +0000

Hacktoberfest has come to an end... and it is super to see folks have contributed to UI-O-Matic

So I merged all the pull requests in a 3.1 release!

What is new:

Again, thanks to the folks spending time on this and helping out in making this happen.

Get it from nuget: https://www.nuget.org/packages/Nibble.Umbraco.UIOMatic/

How to join the fun and get started with contributing to the umbraco community starter project/kit

Tim Geyssens — Fri, 09 Oct 2020 08:19:35 +0000

Hacktoberfest is in town! By submitting 4 prs to open source projects tagged with #hacktoberfest you can earn the necessary credits to complete the challenge! As a bonus Umbraco is also taking part ... read all about it on the blog https://umbraco.com/blog/hacktoberfest-2020/, if you have 1 approved PR on any of the repo's that are marked by HQ you earn the credits for some swag (or plant a tree :) )

Umbraco Community starter project/kit

Before october launched the idea of a community powered Umbraco v8 starter kit/project. We had/have discussions on the new Discussions feature of github and decided that we'll be building a site for a non profit org... So this project will be able to get used as inspiration or you can also deploy it for non profit orgs that are looking for an online presense...idea is that with a color palette picker we'll be able to change the look and feel to match the branding of the org.

Get started

Sounds good right? So how can you get going?

Fork, clone, and start contributing... you don't need VS if you are only interesed in design/frontend/docs

Here is an intro to the PR process on Github

Keeping your fork up to date

Also make sure to keep your fork up to date (so if other peoples pr get's merged, you also need that in your fork)

Design

An design start has been made: a mobile view of the homepage.

You can find preview render and the figma source file here:
https://github.com/TimGeyssens/Umbraco-v8-Community-Website-StarterKit/tree/main/design

There is a UI Kit with components :)
https://github.com/TimGeyssens/Umbraco-v8-Community-Website-StarterKit/tree/main/design/UI%20Kit

Design tasks are marked with "design" on the tracker: https://github.com/TimGeyssens/Umbraco-v8-Community-Website-StarterKit/issues

Frontend

Frontend is currently setup with gulp and tailwind. But you should be able to contribute without visual studio. So also on Mac.

You can find the markup in the markup folder.

Tasks will also be added to the issue tracker and marked with "design".

Backend

For this you'll need windows and visual studio. The solution can be found in the source folder and contains 3 projets (core, web, tests)

You just run the web project from vs and then from the backend (username/pw in readme) you first run a uSyn import to get everyting in sync!

Tasks are marked with backend (and go from easy to more challenging). Key is here to keep to the best practises...

Docs

And last but not least you can also write docs, those will be in the docs folder.

Now let's plant some trees!

The Last Crusade

Tim Geyssens — Fri, 02 Oct 2020 15:28:46 +0000

This is the final follow up to Has Umbraco turned into the "tourist trap" of open source .net

That caused some actions from HQ... Kim(the CEO) got in touch and I gave it 2 months where I didn't complain about anything... now at the DF20 my session about this wasn't picked... but was invited for the roundtable that took place around umbraco community... getting the agenda 2 hours in advance, no questions that got collected from the community where asked.

So I'll just stop trying to put energy in this but just wanted to share the thoughts I mailed Kim and Niels...

Mail1

Hey Kim and Niels

Since the kids are in bed now I have some time for a more detailed mail. As Niels might tell you I also reached out to him to make amends.

From our phone call I can hear you are definitely full of ideas and have a vision for a bright Umbraco future.

Since my "crusade" in March several people from the community reached out with several thoughts. And why not let that benefit you.

I am totally not a business guy but I do have a good touch with the unique Umbraco community. I also think I can give you some special insights... (since I know you aren't a twitter/slack guy). I've been around for a loooong time and also know of several HQ aspects.

But my true passion is for the community

Like look at this one https://twitter.com/timgeyssens/status/1296782437603848195/photo/11 , it is a picture from CG 2009 where the MVP's are announced.

What I find cool about this is that 4 of those folks are stil active with Umbraco. Got me thinking why is that? Since that is a long time to be using a piece of software.

For me personally Umbraco (and Niels of course) showed me that you can be a developer and also have fun (not just be the dull guy behind the computer). So even though there might be mistakes along the road what get's people to stay for the long term and not look back is that unique fun community feeling.

I'm sure you have seen/felt it at cg, but cg is a small subset of the community. And especially with this years COVID situation even that subset missed out...

But a digital business that can have these long standing relationships... that is golden I think.

Those cheesy quirky things are what put a smile on peoples faces... and that is a powerful to do.

Another thing from this week is this: https://twitter.com/KieronBos/status/1296379233351143426

Kieron contacted me on slack about a simple question, will a package I created (configtree) work on umbraco v... (conversation as attachment , diffent screenshots from slack, sorry don't know how to get 1 big one)

So as you can see I went the extra mile and also had some fun while helping him. I think Kieron will definitely remember this now and maybe even cut me some slack when something goes wrong, or turn his positive energy towards another community member.

Niels even had a Umbraco is % amount crazy on the site once (dunno if that is still on there), but that is what makes umbraco standout.

Will there be tech options in the future that are cooler, probably...
Options that are more feature rich, probably...
Options that are better priced... of course

That is a hard thing to predict and keep ahead of competition...

So I truly believe that you need to focus on having people committed and not even think about looking at other options. If you keep the Umbraco spirit and go the extra mile... people will stay on board for the long run. And they will also evangelize the platform with other folks...

If you know where to look, the community is full of great ideas https://twitter.com/Steve_Gibe/status/1296816981493141504

Surprise them, act on an impulse... and I'm betting most will forget the frustrations and cut you some slack.

So make the Umglourious Basterds happen, it might be hard to measure the ROI but I do think it is worth a try. And I think the community would be surprised if you announced that you turned the annoying guy into a valuable asset.

Again give it some thought, I don't expect you to get back to me immediately ... a community that would go through fire is priceless.

Have a good weekend!
Cheers,
Tim

Follow up mail:

Final idea on the Umglourious Basterds

Nothing set in stone of course but imagine having the team like this:

Chief Unicorn (Niels): making sure the community is up to date with the folk at HQ, like videos with HQ members showing what the job they do consists of...
Hacker (Warren): having Warren spend a day a week exploring the new features, ecxiting hack options with new tech...blogging about that on .com (so it has more reach then twitter)
Instructor (Paul Seal , from codeshare.co.uk): sponsor him to keep his youtube channel populated with new umbraco focused tutorials, how much he needs to get some vids a week, dunno...maybe 1 day of work in a week...
1st Lieutenant (Me): taking care of the frustration "killing" and overal keep the good vibe and crazyness in the Umbraco community.

Make a fun photoshop collage of them and ...

Response

Kim is a nice guy no doubt about it... and we want the same end result, a great Umbraco community... with a super succesfull HQ backing it...

But the response was: great ideas but no budget...

Guess Marketing comes at the first place...

Will I still using Umbraco, yes... still passionate about the project? Not that much, some side stuff like hacktober fest and the package team + the .net core version spark my interest but for the rest it's just a plain software comp now...