DEV Community: Michael Kennedy

Implementing 2FA under ASP.Net Core

Michael Kennedy — Wed, 06 Aug 2025 19:59:22 +0000

Enforcing authentication within an application using the [Authorize] attribute is pretty simple once you've done it a few times, but setting up full 2FA using an authenticator app felt like a dark art, the sort of thing best being left to the likes of Azure authentication or Auth0.

In hindsight, implementing in this way is pretty simple.

Provide a QR code that a user can scan.
Test that it works.
Intercept completion of login and prompt users for an OTP code.

QR Code generation

The QR code scanned within the authenticator app follows a standard URL format using the otpauth:// protocol, it encodes a secret which is used by the authenticator when generating the login code.

You could use a NuGet package to generate the full URL, but I wanted closer control over the issuer properties so I ended up encoding these myself:

public const string Issuer = "eMarketing Portal";
public const string IssuerWebsite = "emarketingportal";

public string GenerateOtpUrl(string emailAddress, string userSecret)
{
    var encodedIssuer = Uri.EscapeDataString(IssuerWebsite);
    var encodedLabel = Uri.EscapeDataString($"{Issuer}:{emailAddress}");

    return $"otpauth://totp/{encodedLabel}?secret={userSecret}&issuer={encodedIssuer}";
}

This is the bare minimum required to generate the OTP code. The one drawback is that there's no way to tell the authenticator app which logo to display, I'd split out the issuer name and website domain into separate fields to try to facilitate this, but to no avail.

There are other optional properties that we can add, these are somewhat dependent on your security requirements:

Algorithm
The encryption method used generating the OTP code. Defaults to SHA1. Other alternatives are SHA256 and SHA512.
Digits
The number of digits within the generated code. Defaults to 6.
Period
The OTP expiration period in seconds. Defaults to a value of 30.

I opted to generate the secret and QR code server-side as an additional security precaution. In my case, I used the QRCoder NuGet package to generate the code as this gave me the colour coding options I was looking for and allowed me to embed a logo within the generated image, but really anything will do.

public string GenerateBase64QrCode(string otpUri)
{
    using var qrGenerator = new QRCodeGenerator();
    using var qrCodeData = qrGenerator.CreateQrCode(otpUri, QRCodeGenerator.ECCLevel.H);
    using var qrCode = new QRCode(qrCodeData);
    var logo = new Bitmap(new MemoryStream(Resources.logo_eM_square));

    using var qrImage = qrCode.GetGraphic(
        pixelsPerModule: 20,
        darkColor: Color.FromArgb(45, 78, 118),
        lightColor: Color.White,
        icon: logo,
        iconSizePercent: 40,
        20
    );

    using var ms = new MemoryStream();
    qrImage.Save(ms, System.Drawing.Imaging.ImageFormat.Png);
    var qrBytes = ms.ToArray();

    string base64Image = Convert.ToBase64String(qrBytes);
    return $"data:image/png;base64,{base64Image}";
}

Initial test

It's common to force the user to test their OTP code before enabling it within the application. This is all about protecting ourselves and giving the user piece of mind - if they've validated a code from their authenticator app at the point of configuring 2FA then they know it works.

As developers, we already know that everything will work as long as the instructions have been followed, but by adding extra validation we give our users a greater level of confidence.

I'm validating my users OTP code using OtpNet, but as with the QR code, any similar package will do the same job.

public bool ValidateUserOtp(string secret, string otpCode)
{
    var totp = new Totp(Base32Encoding.ToBytes(secret));
    return totp.VerifyTotp(otpCode, out _, VerificationWindow.RfcSpecifiedNetworkDelay);
}

Trigger on login

Finally, we want to make sure that we trigger validation of the users OTP code every time they login, we'll want to make sure that:

The OTP verification screen is displayed immediately after the user logs in.
That there's no way of circumventing the OTP verification.
That we don't show the OTP verification screen when inappropriate (i.e. after resuming a browser session when already authenticated).

The first step is to track when the user has logged in, as soon as a user has authenticated by conventional means, we'll record their last login date.

var user = await userService.GetUserForAuthAsync(context.Principal);

if (user != null)
{
    await userService.UpdateLastLogin(
        user,
        context.Principal.Claims.Select(x => new KeyValuePair<string, string>(x.Type, x.Value)).ToList());
}

Next, we'll want to redirect users to the OTP screen after login, but we'll also need to cover off other scenarios - for example, a user logs in, but doesn't complete the OTP screen, returning later.

To handle this, we'll want to intercept user navigation and force a redirect until the OTP code has been provided, or the user logs out. We can do this in .Net Core by adding app.UseMiddleware to program.cs.

When we configure this command, we pass it a class containing an Invoke function:

public class UserIdentificationMiddleware(RequestDelegate next)
{
    public async Task Invoke(HttpContext context)
    {
        // do stuff

        await next(context);
    }
}

In this example we can check the request path being accessed by querying context.Request.Path.Value, assuming that the user isn't already accessing the OTP authentication page, or something like the logout screen then we can execute something like this:

var user = await userService.GetUserForAuthAsync(context.User);
var isAuthenticated = context.User?.Identity?.IsAuthenticated ?? false;

if (isAuthenticated && 
    user.OtpEnabled && // has the user enabled 2FA?
    (user.LastLoginDate.Value > user.LastOtpVerificationDate.Value))
{
    context.Response.Redirect("/VerifyLogin");
    return;
}

await next(context);

In addition to recording the users last login date, we're also recording the last time they authenticated their OTP code, we can use this alongside some server-side caching to check which event occurred last and redirect the user accordingly.

Once the user has provided their OTP password, we update the last verification date and the user can navigate as normal.

If we wanted to, we could take this approach a step further to force users to complete their account configuration before using a product or by redirecting to an error page if the account has been disabled.

Applying Visual Themes with SCSS

Michael Kennedy — Sun, 17 Sep 2023 15:02:56 +0000

When you have a large legacy web application, implementing a dark mode theme is a pretty daunting concept. Especially when your CSS is all over the place and you have a number of third-party components to worry about.

The concept of implementing dark mode in our eMarketing product was a natural continuation of conversations the team had had around usage of dark mode in HTML emails and email clients.

In the past, the team hadn't given too much consideration to the organisation of CSS. This was in part due to the lack of any overarching design system. As a result, I had appointed myself as our de facto UX expert and the task of figuring out the best way of implementing CSS themes fell to me.

Clean Styling

There's no point in even attempting to implement colour schemes in a web product unless the CSS is clean. Modern frameworks rightly separate out HTML, JavaScript and CSS into separate files but our 15 year old application (which was based on an even older ASP solution) hadn't followed these common processes.

This multi-step process was probably the most time consuming element of the implementation of visual themes, but well worth it even as an unrelated task.

We reviewed all of our existing CSS files, split component specific styles into their own separate files. There is no right approach here, what matters most is consistency. I split our CSS between external and internal components; so I created sub-folders for each component provider (i.e. BootStrap, Telerik, etc) and sub-folders for application areas (i.e. application settings, user security, etc) before migrating our CSS to files within these folders. Anything remaining that couldn't easily fit into a category was migrated to catch-all CSS.

Once this process had been completed, we installed & configured WebCompiler to automatically compile our styles from the SCSS.

Finally, we reviewed the usage of inline CSS and CSS style blocks within our CSHTML content (there was a lot of this) and migrated anything theme specific to the new SCSS files.

There would be more clean-up work on this later, but this put is in a good place for future development and allowed us to split our generated CSS between common application elements which are required all the time and screen specific styles which just needed to be loaded alongside certain screens.

Because of the amount of CSS being moved around, it's wise to test changes and compare screens against unmodified versions. This allows us to catch any precedence issues caused by moving things about.

Create SCSS Variables and Review Colour Scheme

While the team had taken care to ensure that the existing colour scheme was consistent throughout the product, inevitably there were areas where variations had been inadvertently introduced within our CSS.

This was now the perfect time to review all colour codes and fonts within our SCSS files and question any inconsistencies. This gave us the opportunity to not only streamline the product appearance but to keep the number of CSS variables as low as possible.

Even with this, after our first pass, we still created over 100 CSS variables. The impact of this clean-up on product appearance was instantly obvious.

Theme Generation

The next step was to create our first theme from the SCSS variables. I declared each theme in its own file, this isn't strictly necessary but it makes maintenance significantly easier.

Each file consists of a list of variables and a theme object which uses these variables, looking something like the below example.

// branding colours
$branding-primary: #105CC6;
$action-edit-button: #00a2e8;

// fonts
$font-primary:"Helvetica Neue", Helvetica, Arial, sans-serif;
$font-legend: Arial, Helvetica, sans-serif;
$font-header: Calibri, Arial, sans-serif;
$font-code: Lucida Console;

$standard-theme: ( 
    // branding
    branding-primary: $branding-primary,
    branding-primary-dark: darken($branding-primary, 15%),
    branding-primary-edit: $action-edit-button,
    branding-primary-edit-dark: darken($action-edit-button, 20%),

    // base fonts
    font-primary: $font-primary,
    font-legend: $font-legend,
    font-title-area: $font-header,
    font-code: $font-code
);

To generate the final CSS, we need to take our themes and use an SCSS mixin to define additional CSS rules that trigger their usage.

We achieved this by specifying a CSS class name within the <html> DOM element which would dictate the name of the current theme. The standard theme is enabled by the theme-standard class and the dark theme is enabled by the theme-dark class etc.

// import themes from external files
@import 'standard.scss';
@import 'dark.scss';

// add themes to an array
$themes: (
    standard: $standard-theme,
    dark: $dark-theme,
);

@mixin theme() {
    $array: $themes;

    @each $theme, $map in $array {
        html.theme-#{$theme} & {
            $array-map: () !global;

            @each $key, $submap in $map {
                $value: map-get(map-get($array, $theme), '#{$key}');
                $array-map: map-merge($array-map, ($key: $value)) !global;
            }

            @content;
            $array-map: null !global;
        }
    }
}

@function themeValue($key) {
    @return map-get($array-map, $key);
}

We can then modify our SCSS to include a reference to the themeValue function, this will trigger generation of multiple CSS lines, one for each possible theme.

body {
    @include theme() {
        font-family: themeValue(font-primary);
    }
    font-variant: none;
}

Theme Selection

Finally, the only thing left to was to configure our theme. This needs to happen as high up the page load as possible in order to prevent any flashes of un-styled content.

We could of course, just apply the style class to the <html> DOM object and be done with it; however, most applications provide a "System Theme" option and it won't take too much work to implement this ourselves.

In this example, we already support theme-standard and theme-dark, we'll now introduce theme-system as a third option and apply the light or dark theme automatically.

// are we using the system theme
const isSystemTheme = theme == "theme-system";
if (isSystemTheme)
{
    // detect colour scheme
    if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
        theme = "theme-dark";
    }
    else
    {
        theme = "theme-standard";
    }
}

setVisualTheme(theme, isSystemTheme);

We'll then clear the existing theme and apply the new one. We'll want to note if we're using the default system theme, this will allow us to change the theme without refreshing the page.

function setVisualTheme(themeName, isSystemTheme) {
    // remove any existing theme
    const $html = $("html");
    $html.attr("class", function(i, c){
        return c.replace(/(^|\s)theme-\S+/g, '');
    });

    $("html").prop("usingSystemTheme", isSystemTheme);

    // set new theme
    $("html").addClass(themeName);
}

We can intercept changes to the system theme by using the JavaScript below. If the operating system theme is changed, then this will be automatically passed to the browser window and we can ensure that this update is reflected on the page:

window.matchMedia("(prefers-color-scheme: dark)")
    .addEventListener("change", event => {
        if ($("html").prop("usingSystemTheme")) {
            let targetTheme = "theme-standard";

            if (event.matches) {
                targetTheme = "theme-dark";
            }

             setVisualTheme(targetTheme, true);
         }
    });

As a final step, we might also want to update the colour of the browser header to match the theme colour. This provides a more cohesive feel in mobile environments.

We can do this by adding an additional meta value within our document:

<meta name="theme-color" content="#4285f4" />

We can't extract the theme colour directly from CSS, but we can apply the appropriate class to a hidden DOM object and then read that colour using JavaScript and apply it to the theme-color object. This approach also resolved an issue with a third party component which required the colour scheme to be set using JavaScript.

In conclusion, implementing visual themes in a product that hadn't ever been written with this in mind was tricky work; however the majority of work was in cleaning up the existing CSS implementation.

There were a number of points along the journey where the benefit of what we were doing was obvious and this was before we'd even implemented our first colour scheme.

If this journey proves anything, it's how important it is to segregate our UI components and associated CSS. This is something modern frameworks do very well, but can be lacking in legacy applications.

The 5 Rules of ARIA

Michael Kennedy — Mon, 28 Aug 2023 09:25:28 +0000

As a general rule, no ARIA is better than bad ARIA. This is because bad ARIA can cause screen readers and other assistive technologies to misunderstand how a page fits together and this can make web pages unusable for end users.

When using ARIA, following the 5 rules of ARIA can help prevent us from making costly mistakes.

Don't use ARIA!

Aria should only be used to fill in gaps that HTML does not provide. ARIA should be the last resort, don't use it if native HTML elements will do the same thing.

Don't replace native semantics

Don't define an item as one type and use ARIA to override the type to something else. Instead of <div role="button">, use <button>, this already provides interactivity such as keyboard press & simplifies code requirements.

Interactive controls must be keyboard accessible

Everything you can do with a mouse, you should be able to do with keyboard. For example, if a mouse is required to operate menus then it should be possible to operate these from the keyboard - this extends to using home/end to select the first and last items, typing to search, escape to clear selection, etc.

Beware of `role="presentation"` and `aria-hidden="true"`

Users can get stuck if keyboard users can see items, but screen readers can't. Therefore hiding items from ARIA needs to be done with extreme care.

All interactive elements must have an accessible names

All interactive elements such as links, buttons, inputs, etc should have accessible names. This helps assistive technologies to understand how everything fits together.

For example, we can use the for attribute to draw a link between an <input> and the associated <label> control.

Colour Contrast of Overlapping Text Labels

Michael Kennedy — Tue, 22 Aug 2023 14:52:29 +0000

When using the standard Bootstrap progress bar control, any text labels need to be centred over the individual bars. When there are multiple progress controls on-screen, this can look odd.

If our text is particularly long, and the viewable area of the progress bar is particularly small, then we need to force a minimum width and carefully consider our wording. This can be tricky in a multi-lingual product.

The easiest way to handle this, is by centring the text. Perhaps by superimposing an additional label over the control. However, such an approach is dependent on the text colour contrast when straddling two progress objects.

The Bootstrap control typically doesn't require us to fill the unfilled whitespace at the end of the progress bar. However, if we add this ourselves and assign some extra CSS, then we can work some magic and make it look like the text colour is automatically changed to prevent clashing with the background.

For this example, I've used clip-path to restrict the visible areas of our elements and have preserved accessibility by moving the aria-label property to the progress container. Finally, I've used the aria-hidden attribute to prevent screen readers from seeing the text label multiple times.

<div class="progress progress-with-labels" aria-label="50% Example" aria-valuenow="50" aria-valuemin="0" aria-valuemax="100">
    <div class="progress-foreground progress-bar" style="clip-path: inset(0 50% 0 0);" aria-hidden="true">50% Example</div>
    <div class="progress-background" style="clip-path: inset(0 0 0 50%);" aria-hidden="true">50% Example</div>
</div>

.progress-with-labels {
    width: 100%;
    position: relative;

    .progress-foreground {
        display: flex;
        justify-content: center;
        align-items: center;
        width: 100%;
        color: white;
    }

    .progress-background {
        position: absolute;
        display: flex;
        justify-content: center;
        align-items: center;
        left: 0;
        right: 0;
        top: 0;
        bottom: 0;
        background: #e9ecef;
        color: black;
    }
}

With this approach, we can achieve a final output looking similar to the below, with all of our text labels centred and easily readable.

Reusable MVC Razor Components with HTML Extension Methods

Michael Kennedy — Mon, 04 Jul 2022 07:34:59 +0000

Component reuse is one of the intrinsic features of modern JavaScript frameworks. React, Angular and their counterparts all guide us toward the idea of creating small, reusable elements in order to maintain a clean codebase.

MVC performs a similar task, but the architecture doesn't make it as intuitive. As the name suggests, it provides reusable components as standard; unfortunately these components can be relatively unsophisticated making easy to fall into the trap of creating massive views and forgetting about reusability.

On occasion, the time required to migrate existing solutions to new frameworks is prohibitive, but that doesn't mean that we can't provide an easy to use component library within the confines of MVC Razor.

By simplifying the our source code we can take back control of styling, reduce time to onboard new team members and standardise component usage to prevent subtle differences seeping through in different product areas.

Approaches will vary depending on the complexity of your components, but we can take inspiration from providers of third party libraries and make our own HTML helpers accessible directly through Visual Studio IntelliSense.

What are HTML Helpers?

For anyone not familiar with the concept, we can use extension methods to expand on the functionality of any data type. This Microsoft document explains it in more detail, but essentially we declare something like this:

namespace MyExtensionMethods
{
    public static class MyExtensions
    {
        public static int WordCount(this string str)
        {
            return str.Split(new char[] { ' ', '.', '?' },
                StringSplitOptions.RemoveEmptyEntries).Length;
        }
    }
}

Whenever the namespace is available we can execute the WordCount function against any string type:

var stringValue = "There are four words";
var wordCount = stringValue.WordCount(); // outputs 4.

This same concept can be applied to the HTML helpers made available by Razor, we can leverage the same approach to make it easier to create & recreate common components; improving discoverability for new and existing developers alike.

@(Html.MyHtmlHelper()
    .GenerateThisControl("controlId")
    .EnableOptions(true)
);

Basic Usage

A basic example might be something like this, which expands on the functionality of Html.Raw() to escape quotation marks for inclusion in DOM attributes:

using System.Web;
using System.Web.Mvc;

namespace MyExtensionMethods
{
    public static class HtmlStringHelper
    {
        public static IHtmlString RawHtmlString(
            this HtmlHelper htmlHelper, string value)
        {
            return !string.IsNullOrWhiteSpace(value) 
                ? htmlHelper.Raw(EscapeQuotes(value))
                : "";
        }
    }
}

An example like this is of course quite limited. We can use it to perform basic operations in day-to-day work, but it's not something which will reduce code complexity by any measurable amount.

But we can expand on this to return entire views:

namespace MyExtensionMethods
{
    public static class HtmlStringHelper
    {
        public IHtmlString LoadingAnimation(
            this HtmlHelper htmlHelper)
        {
            return _htmlHelper.Partial(
                "~/Views/HtmlHelpers/LoadingAnimation.cshtml");
        }
    }
}

There's still not much benefit to this last example, certainly nothing that we can't achieve by documenting the location of the loading animation view, but this shows us that we can:

Use HTML Helpers to store content outside of the core solution. We could make this available as part of a DLL and perhaps directly render the HTML output from an embedded file. This would let us simplify create a component library which can be accessed by multiple projects.
Guide developers through the generation of HTML content by providing additional configuration options without the need to go digging for the right model or view name.

Helper Factory

To provide more advanced functionality, we'll have to expose additional options via helper factories.

This approach follows previous examples, except this time we're returning a class which implements IHtmlString instead of a partial view or string.

First, we add something like this to our existing helper:

public static HtmlHelperFactory MyProductName(this HtmlHelper helper)
{
    return new HtmlHelperFactory(helper);
}

Which is implements a HtmlHelperFactory class:

namespace MyExtensionMethods.HtmlHelpers.Factory
{
    public class HtmlHelperFactory
    {
        private readonly HtmlHelper _htmlHelper;

        public HtmlHelperFactory(HtmlHelper htmlHelper)
        {
            _htmlHelper = htmlHelper;
        }

        public TooltipBuilder Tooltip(string id, string text)
        {
            return new TooltipBuilder(_htmlHelper, id, text);
        }
    }
}

What we have here will allow us to generate a new tooltip component within our CSHTML files by entering something like @Html.MyProductName().Tooltip("id", "This is my tooltip").

We can imagine that this would perhaps generate a little "i" icon, with the specified tooltip text automatically escaped and assigned within the title attribute, perhaps using a third party component to make the tooltip look pretty.

Instead of creating the HTML content within the helper, we can shift it to a builder class, provide it with some additional options and create an appropriate view model.

public class TooltipBuilder : IHtmlString
{
    private readonly HtmlHelper _htmlHelper;
    private readonly TooltipModel _model = new TooltipModel();

    public TooltipBuilder(
        HtmlHelper htmlHelper, 
        string id = "", 
        string tooltipText = "")
    {
        _model.Id = id;
        _model.Text = tooltipText;
        _htmlHelper = htmlHelper;
    }

    public TooltipBuilder Colour(string value)
    {
        _model.Colour = value;
        return this;
    }

    public TooltipBuilder Mode(TooltipBuilderMode value)
    {
        _model.Mode = value;
        return this;
    }

    public string ToHtmlString()
    {
        var returnData = _htmlHelper
            .Partial(
                "~/Views/HtmlHelpers/Tooltip.cshtml", 
                _model
            )
            .ToHtmlString();

        return returnData;
    }
}

We can expect the generated tooltip to be pretty simplistic, perhaps a couple of <span>'s and a custom font used to generate an appropriate icon.

But what we've done here is to standardise this simple element and ensure that every instance is rendered in the same way. Should we ever want to change the icon then we'll only ever need to update a single line of code.

We've also given other developers a list of possible options in a manner which lets us chain operations together.

@(Html.MyProductName()
    // provide required fields
    .Tooltip("id", "This is my tooltip")
    // provide optional parameters
    .Colour("#575757")
    .Mode(TooltipBuilderMode.FloatBottom)
)

Html Attributes

While the tooltip example is pretty basic, we might not want every instance of the same object to behave in exactly the same way, perhaps we want to allow users to specify some additional classes on the parent container, or maybe we want to bind the tooltip value to a model.

In which case we might expand on TooltipBuilder by introducing some extra operations and enhanced processing:

public TooltipBuilder(
    HtmlHelper htmlHelper, 
    string id = "", 
    string defaultTooltipText = "")
{
    _model.Id = id;
    _model.Text = defaultTooltipText;
    _htmlHelper = htmlHelper;
}

// provide generic attribute options
public TooltipBuilder HtmlAttributes(
    IDictionary<string,object> value)
{
    if (value != null)
    {
        value.ToList().ForEach(v => SetHtmlAttribute(v));
    }
    return this;
}

// provide more specific guided attribute options
public TooltipBuilder TitleBindingAttribute(string value)
{
    SetHtmlAttribute(new KeyValuePair<string,object>(
        "data-bind", 
        $"attr: {{ title: {value} }}"
    ));

    return this;
}

// combine lists into a single property for processing
private void SetHtmlAttribute(
    KeyValuePair<string, object> attribute)
{
    if (!string.IsNullOrWhiteSpace(attribute.Value?.ToString() ?? ""))
    {
        if (_model.HtmlAttributes.ContainsKey(attribute.Key))
        {
            _model.HtmlAttributes[attribute.Key] = attribute.Value;
        }
        else
        {
            _model.HtmlAttributes.Add(attribute.Key, attribute.Value);
        }
    }
}

public string ToHtmlString()
{
    var returnData = _htmlHelper
        .Partial(
            "~/Views/HtmlHelpers/Tooltip.cshtml", 
            _model
        )
        .ToHtmlString();
    return returnData;
}

With this, we can allow more in-depth customisation while still retaining control over the appearance and behaviour of the tooltip:

@(Html.MyProductName()
    .Tooltip("id", "This is my tooltip")
    .SetHtmlAttribute(
        [{@class = "myClassName"}] 
    )
    .TitleBindingAttribute("tooltipTextField")
)

Grouping Commands

Finally, we may find ourselves in a situation where we have so many options within our helper that we want to group them by function to make them easier to work with. (Perhaps hard to believe within the confines of the tooltip example)

We can do this by creating a configurator class and using an Action delegate to preserve the simplicity of creating our control within a single razor command.

First we need a basic option factory:

public class TooltipOptionFactory
{
    internal TooltipOptions Options = 
        new TooltipOptions();

    public TooltipOptionFactory Disabled(bool value)
    {
        Options.Disabled = value;
        return this;
    }

    public TooltipOptionFactory Visible(bool value)
    {
        Options.Visible = value;
        return this;
    }
}

Then we need to integrate it with our TooltipBuilder class:

public TooltipBuilder Options(
    Action<TooltipOptionFactory> configurator)
{
    var optionFactory = new TooltipOptionFactory();
    configurator.Invoke(optionFactory);
    _model.Options = optionFactory.Options;
    return this;
}

Which will allow us to use it within our CSHTML like this:

@(Html.MyProductName()
    .Tooltip("id", "This is my tooltip")
    .Options(o => {
        o.Visible(true);
        o.Disabled(false);
    })
)

These are just the first stepping stones in creating a library of reusable Razor components. While there's clearly some overhead in creating a full library of these components the future time savings from taking this approach are immeasurable.

As with many approaches, it's much easier to begin a project with something like this in mind rather than shoehorning it into an established product; as someone who recently spent two weeks fixing issues caused by the upgrade of a third party component library - I really wish our development team had taken this approach 10 years ago, but now we have the tools to correct that mistake.

Insert an Ellipsis into the Middle of a String

Michael Kennedy — Sat, 25 Jun 2022 19:41:23 +0000

The text-overflow: ellipsis; CSS property is one of those commands that's tricky to get right the first time, but once you've figured out the correct overflow and white-space options to go alongside it then it's easy to remember.

Occasionally we'll come across a scenario where we'd prefer to add an ellipsis somewhere in the middle of the text. Displaying long URL's on-screen is a good example of this where it might be easier if we can draw particular attention to the domain name at the start and the and the targeted file at the end without breaking the user flow.

This isn't possible in CSS alone, and even if it were then we wouldn't have much control over where the ellipsis gets added.

I was able to fashion a simplistic solution to this using jQuery, although the approaches would be broadly similar regardless of your chosen flavour of scripting language.

if (displayText.Length > 40) {
    var start = displayText.Substring(0, displayText.Length - 25);
    var end = displayText.Substring(displayText.Length - 25);
    $targetObject = $`
       <div class='valueContainer middleEllipsis' title='{displayText}'>
          <div class='start'>{start}</div>
          <div class='end'>{end}</div>
       </div>`;
}
else {
    // fallback to a standard approach
    $targetObject = $`
        <div class='valueContainer' 
             style='overflow:none;text-overflow:ellipsis;white-space:nowrap;'>
            {displayText}
        </div>`;
}

The approach here, is to take your text value and split it at an appropriate juncture; we can then render both within separate elements and use a little CSS to ensure that everything flows properly even if the screen is too wide to require an ellipsis.

.middleEllipsis {
    display: flex;
    flex-direction: row;
    flex-wrap: nowrap;
    justify-content: flex-start; 
}

.middleEllipsis .start {
    overflow: hidden;
    text-overflow: ellipsis;
    white-space: nowrap;
    flex-shrink: 1; 
}

.middleEllipsis .end {
    white-space: nowrap;
    flex-basis: content;
    flex-grow: 0;
    flex-shrink: 0; 
}

Updating the Collation of an Existing MSSQL Database

Michael Kennedy — Sat, 25 Jun 2022 13:26:12 +0000

When products are installed outside of a controlled development environment we expect the unexpected, that's why we have logging. Product databases are typically an area where this isn't as much of an issue. Beyond client data, the variables are restricted to SQL version numbers, hardware and the server database collation...

As much as some might advise otherwise, clients will install multiple products on a single SQL server to reduce licencing costs.

When we don't have control over the server collation, products that make heavy use of #temporary tables or that query system databases such as msdb can run into problems. System databases will tend to use the same collation as the server and if we're not careful we'll encounter the dreaded collation error:

Cannot resolve the collation conflict between “SQL_Latin1_General_CP1_CI_AS” and “Latin1_General_CI_AS” in the equal to operation.

This situation commonly occurs when an existing database is restored to a SQL server where the target collations don't match.

Preventative Programming

As developers, the first thing that we can do to prevent this is by deliberately working environments where the collation of the development database has been deliberately set to something different to the server collation.

This forces us to automatically consider database collations when writing our stored procedure and function code, substituting collate database_default whenever necessary.

Core Database Collation

If the proverbial ship has sailed then we can find ourselves in a position where we have to manually update the collation of large client databases.

At first glance it might look like we just need to open database properties and change the collation but it's far from being that simple.

When we update the collation in this manner we're actually changing the default collation, newly created tables and columns will use this collation but existing objects will be unaffected.

Updating the collation of an existing database is tricky, but it can be boiled down to a simple process.

Preparation

Take a database backup (because not to do so is to invite disaster).

Set recovery mode to Simple.

USE [master]
GO
ALTER DATABASE [SampleDatabase] SET RECOVERY SIMPLE WITH NO_WAIT
GO

Set the database to single user mode (only necessary in an environment where another system could be accessing the database).
```
USE [master]
GO
ALTER DATABASE [SampleDatabase] SET SINGLE_USER WITH NO_WAIT
GO
```

Target Columns

Once our database backup has been configured we'll need to work out which database columns need to be updated. We can achieve this by querying system tables to retrieve a list of columns which don't use the new database collation.

declare @newCollationName sysname
set @newCollationName = 'Latin1_General_CI_AI'

select
    *
from
(
    select
        s.[name] as [schema],
        o.[name] as [table],
        c.[name] as [column],
        t.[name] as [datatype],
        case
            when 
                c.[max_length] = -1 AND
                (
                    lower(t.[name]) = 'char' OR
                    lower(t.[name]) = 'nchar' OR
                    lower(t.[name]) = 'varchar' OR
                    lower(t.[name]) = 'nvarchar'
                )
            then 'max'
            when lower(t.[name]) = 'varchar' OR lower(t.[name]) = 'char'  then convert(varchar,c.[max_length])
            when lower(t.[name]) = 'nvarchar' OR lower(t.[name]) = 'nchar'  then convert(varchar,c.[max_length]/2)
            else null
        end as [length],
        c.[collation_name] as [collation]
    from sys.objects o
    inner join sys.columns c on (c.[object_id] = o.[object_id])
    inner join sys.types t on (t.[system_type_id] = c.[system_type_id] AND t.[user_type_id] = c.[system_type_id])
    inner join sys.schemas s on (s.[schema_id] = o.[schema_id])
    where 
        o.[is_ms_shipped] = 0 and 
        o.[type] = 'U' and
        c.[collation_name] is not null AND
        lower(c.[collation_name]) != lower(@newCollationName)
    union select
        s.[name] as [schema],
        o.[name] as [table],
        c.[name] as [column],
        'cs' as [datatype],
        'cs' as [length],
        'cs' as [collation]
    from sys.objects o
    inner join sys.columns c on (c.[object_id] = o.[object_id])
    inner join sys.types t on (t.[system_type_id] = c.[system_type_id] AND t.[user_type_id] = c.[system_type_id])
    inner join sys.schemas s on (s.[schema_id] = o.[schema_id])
    where 
        o.[is_ms_shipped] = 0 and 
        c.is_computed = 1
) s
order by
    s.[table],
    s.[column]

Incompatible Objects

Now we have a list of database columns which need to be updated, but we still can't act on it, we'll need to drop all items associated with these columns first - Functions, Indexes, Unique Constraints, Foreign/Primary keys and Computed columns will all need to be dropped before we can attempt to change the collation.

The products I work with make use of a standalone database upgrade tool which compares a targeted database directly against the expected database schema and will automatically re-create any missing items.
In other scenarios it will be necessary to script these objects before dropping them in order to recreate them later.

Executing the below SQL will give us the information we need. (where #targetColumns is the output of the previous SELECT operation)

--drop table valued functions
select
    'DROP FUNCTION [' + s.[name] + '].[' + o.[name] + ']' as command,
    1 as [priority]
from sys.objects o
inner join sys.schemas s on (s.[schema_id] = o.[schema_id])
where o.[type] = 'TF'

--indexes to drop
union SELECT 
    'DROP INDEX [' + s.[name] + '].[' + t.[name] + '].[' + ind.[name] + ']' as command,
    2 as [priority]
FROM sys.indexes ind 
INNER JOIN sys.index_columns ic ON  ind.object_id = ic.object_id and ind.index_id = ic.index_id 
INNER JOIN sys.columns col ON ic.object_id = col.object_id and ic.column_id = col.column_id 
INNER JOIN sys.tables t ON ind.object_id = t.object_id 
inner join sys.schemas s on (s.[schema_id] = t.[schema_id])
inner join #targetColumns tcol 
on 
(
    lower(tcol.[schema]) = lower(s.[name]) AND
    lower(tcol.[table]) = lower(t.[name]) AND
    lower(tcol.[column]) = lower(col.[name])
)
WHERE 
    ind.is_primary_key = 0 AND 
    ind.is_unique = 0 AND 
    ind.is_unique_constraint = 0 AND 
    t.is_ms_shipped = 0 
group by
    s.[name],
    t.[name],
    ind.[name]

--drop unique constraints
union SELECT 
    'ALTER TABLE [' + s.[name] + '].[' + t.[name] + '] DROP CONSTRAINT [' + ind.[name] + ']' as command,
    2 as [priority]
FROM sys.indexes ind 
INNER JOIN sys.index_columns ic ON  ind.object_id = ic.object_id and ind.index_id = ic.index_id 
INNER JOIN sys.columns col ON ic.object_id = col.object_id and ic.column_id = col.column_id 
INNER JOIN sys.tables t ON ind.object_id = t.object_id 
inner join sys.schemas s on (s.[schema_id] = t.[schema_id])
inner join #targetColumns tcol 
on 
(
    lower(tcol.[schema]) = lower(s.[name]) AND
    lower(tcol.[table]) = lower(t.[name]) AND
    lower(tcol.[column]) = lower(col.[name])
)
WHERE 
    ind.is_primary_key = 0 AND 
    ind.is_unique = 1 AND 
    t.is_ms_shipped = 0 
group by
    s.[name],
    t.[name],
    ind.[name]

--foreign keys to drop
union select
    'ALTER TABLE [' + s.fk_schema + '].[' + s.[fk_table] + '] DROP CONSTRAINT [' + s.[fkname] + ']' as command,
    3 as [priority]
from
(
    select 
        fk.name as [fkname],
        schema_name(tab.schema_id) as [schema],
        tab.name as [table],
        col.name as [column],
        schema_name(tab.schema_id) as [fk_schema],
        tab.name as [fk_table]
    from sys.tables tab
        inner join sys.columns col 
            on col.object_id = tab.object_id
        inner join sys.foreign_key_columns fk_cols
            on fk_cols.parent_object_id = tab.object_id
            and fk_cols.parent_column_id = col.column_id
        left outer join sys.foreign_keys fk
            on fk.object_id = fk_cols.constraint_object_id
        left outer join sys.tables pk_tab
            on pk_tab.object_id = fk_cols.referenced_object_id
        left outer join sys.columns pk_col
            on pk_col.column_id = fk_cols.referenced_column_id
            and pk_col.object_id = fk_cols.referenced_object_id
    union select 
        fk.name as [fkname],
        schema_name(pk_tab.schema_id) as [schema],
        pk_tab.name as [table],
        pk_col.name as [column]  ,  
        schema_name(tab.schema_id) as [fk_schema],
        tab.name as [fk_table]
    from sys.tables tab
        inner join sys.columns col 
            on col.object_id = tab.object_id
        inner join sys.foreign_key_columns fk_cols
            on fk_cols.parent_object_id = tab.object_id
            and fk_cols.parent_column_id = col.column_id
        left outer join sys.foreign_keys fk
            on fk.object_id = fk_cols.constraint_object_id
        left outer join sys.tables pk_tab
            on pk_tab.object_id = fk_cols.referenced_object_id
        left outer join sys.columns pk_col
            on pk_col.column_id = fk_cols.referenced_column_id
            and pk_col.object_id = fk_cols.referenced_object_id
) s
inner join #targetColumns t 
on 
(
    lower(t.[schema]) = lower(s.[schema]) AND
    lower(t.[table]) = lower(s.[table]) AND
    lower(t.[column]) = lower(s.[column])
)

--primary keys to drop
union SELECT 
    'ALTER TABLE [' + s.[name] + '].[' + t.[name] + '] DROP CONSTRAINT [' + ind.[name] + ']' as command,
    4 as [priority]
FROM sys.indexes ind 
INNER JOIN sys.index_columns ic ON  ind.object_id = ic.object_id and ind.index_id = ic.index_id 
INNER JOIN sys.columns col ON ic.object_id = col.object_id and ic.column_id = col.column_id 
INNER JOIN sys.tables t ON ind.object_id = t.object_id 
inner join sys.schemas s on (s.[schema_id] = t.[schema_id])
inner join #targetColumns tcol 
on 
(
    lower(tcol.[schema]) = lower(s.[name]) AND
    lower(tcol.[table]) = lower(t.[name]) AND
    lower(tcol.[column]) = lower(col.[name])
)
WHERE 
    ind.is_primary_key = 1 AND
    t.is_ms_shipped = 0 
group by
    s.[name],
    t.[name],
    ind.[name]

--columns to drop
union select
    'ALTER TABLE [' + s.[name] + '].[' + o.[name] + '] DROP COLUMN [' + c.[name] + ']' as command,
    6 as [priority]
from sys.objects o
inner join sys.columns c on (c.[object_id] = o.[object_id])
inner join sys.types t on (t.[system_type_id] = c.[system_type_id] AND t.[user_type_id] = c.[system_type_id])
inner join sys.schemas s on (s.[schema_id] = o.[schema_id])
where 
    o.[is_ms_shipped] = 0 and 
    c.is_computed = 1

In larger databases, dropping this many items in one go will still add data to the database log even with simple logging mode enabled.
Our execution of this process includes a truncation of database logs every 100 records, this is particularly useful when executing in an environment where disk space is at a premium.

Update Column Collation

Once all associated items have been dropped, we can simply iterate through the list of columns and update the database collation.

ALTER TABLE [{schema}].[{table}] 
ALTER COLUMN [{column}] {data type}({length}) 
COLLATE {new collation name}

Update Database Collation

Now we're in a position where we can update the core database collation. This is easy enough to do within SQL Management Studio, but we can also achieve this by executing the following SQL command:

ALTER DATABASE {database name} COLLATE {new collation name};

At this point we can recreate previously dropped items and take the database out of single user mode. Depending on the number of items that were changed, it may also be prudent to reindex the database.

Database Reindexing (MSSQL)

Michael Kennedy — Sat, 25 Jun 2022 11:01:06 +0000

There are a number of ways to reindex a database. This is my preferred method.

First, execute the below to check the fragmentation level of the active database:

declare @fragmentation float

set @fragmentation = 20

SELECT 
    S.name as [Schema],
    T.name as [Table],
    I.name as [Index],
    DDIPS.avg_fragmentation_in_percent
FROM 
    sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS DDIPS
INNER JOIN 
    sys.tables T with (nolock) 
on 
(
    T.object_id = DDIPS.object_id
)
INNER JOIN 
    sys.schemas S with (nolock) 
on 
(
    T.schema_id = S.schema_id
)
INNER JOIN 
    sys.indexes I with (nolock) 
ON 
(
    I.object_id = DDIPS.object_id AND 
    DDIPS.index_id = I.index_id
)
WHERE 
    DDIPS.database_id = DB_ID() and 
    I.name is not null AND 
    DDIPS.avg_fragmentation_in_percent > @fragmentation
ORDER BY 
    DDIPS.avg_fragmentation_in_percent desc

Run the database reindex by executing the following:

declare @fragmentation float

set @fragmentation = 20

declare @schema_name sysname,
        @table_name sysname,
        @index_name sysname

DECLARE cursor_tables 
    CURSOR FOR 
        SELECT 
            S.name as 'Schema',
            T.name as 'Table',
            I.name as 'Index'
        FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS DDIPS
        INNER JOIN sys.tables T with (nolock) on (T.object_id = DDIPS.object_id)
        INNER JOIN sys.schemas S with (nolock) on (T.schema_id = S.schema_id)
        INNER JOIN sys.indexes I with (nolock) ON (I.object_id = DDIPS.object_id AND DDIPS.index_id = I.index_id)
        WHERE 
            DDIPS.database_id = DB_ID() and 
            I.name is not null AND 
            DDIPS.avg_fragmentation_in_percent > @fragmentation
        ORDER BY 
            DDIPS.avg_fragmentation_in_percent desc

open cursor_tables

fetch next from cursor_tables
into @schema_name, @table_name, @index_name

while @@fetch_status = 0
begin
    exec
    (
        '
            ALTER INDEX [' + @index_name + '] ON [' + @schema_name + '].[' + @table_name + ']
            REBUILD WITH(FILLFACTOR = 80, SORT_IN_TEMPDB = ON, STATISTICS_NORECOMPUTE = ON)
        '
    )

    exec
    (
        '
            ALTER INDEX [' + @index_name + '] ON [' + @schema_name + '].[' + @table_name + ']
            REORGANIZE
        '
    )

    fetch next from cursor_tables
    into @schema_name, @table_name, @index_name
end

close cursor_tables
deallocate cursor_tables

And finally, update statistics by running this

EXEC sp_updatestats

Article created mostly for personal reference; there are of course a few different ways in which these scripts could be improved, but when providing them to clients the simplest solution is often the best.

Securing User Logins with MVC and JWT

Michael Kennedy — Sun, 19 Jun 2022 15:24:31 +0000

Ensuring that an MVC application is fully secured can feel daunting. At first glance, the most important part of the login process is the login screen itself, and yes this is the natural starting point, but it's only the beginning.

Login Screen

In our case, we're making use of the standard login fields (username, password & login button) which we've expanded upon by providing a SSO sign-in options supporting both Azure and local Active Directory integration.

When a user attempts to login, we validate the credentials within our product database and return basic status information, perhaps a success code or an error message. This an area where want our error messages to remain ambiguous; we can retain an element of security by not differentiating between users who don't exist or where a login has been attempted with an incorrect password. This simple approach makes it harder to ascertain if an incorrect username has been entered and makes it twice as hard for any hacking attempt to be successful.

We can take this a step further & prevent brute force hacking attempts by temporarily blocking users or perhaps IP addresses where a number of incorrect logins have been attempted within a short time period.

JWT Configuration

A big problem is that once a user has logged in, we'll still need to verify their access rights each time an operation is performed. In a modern environment where a single page can make multiple AJAX calls this can amount to a high number of repeated database calls or a lot of cached user data.

While it initially sounds unavoidable, we can drastically improve product performance by keeping those database calls down.

This is where JWT comes in.

Each instance of our application is provided with a random Client Secret. I configured ours so that it's automatically inserted into web.config at the point of installation.

This unique value is then used to provide the user with two tokens:

Access Token

The access token is a short-lived token typically living for 5-15 minutes. It contains commonly accessed user information, perhaps a username, configuration options and security access rights.

This information is encoded into an alphanumeric string and can optionally be encrypted.

// using Microsoft.IdentityModel.Tokens;
// using System.IdentityModel.Tokens.Jwt;

/* 
    Get the client secret.

    Our usage of the client secret ensures that the request
    and access tokens are generated using completely unique
    values.

    In this example we're using a hard-coded constant, but
    this could be achieved any number of ways.
*/
var key = Encoding.ASCII.GetBytes(_jwtConfig.Secret +
    ACCESS_TOKEN_SECRET_SUFFIX);

var tokenHandler = new JwtSecurityTokenHandler();

var claims = new Dictionary<string, object>
{
    // Add user configuration and security details here
};

var tokenDescriptor = new SecurityTokenDescriptor
{
    Subject = new ClaimsIdentity(new[] {
        new Claim("id", activeUser.Id.ToString()),
        new Claim("userName", activeUser.UserName.ToString())
    }),
    IssuedAt = DateTime.UtcNow,
    Claims = claims,
    CompressionAlgorithm = CompressionAlgorithms.Deflate,
    Expires = DateTime.UtcNow.AddMinutes(_jwtConfig.AccessTokenValidMinutes),
    SigningCredentials = new SigningCredentials(
        new SymmetricSecurityKey(key), 
        SecurityAlgorithms.HmacSha256Signature, 
        SecurityAlgorithms.HmacSha256Signature
    )
};

var token = tokenHandler.CreateToken(tokenDescriptor);
return tokenHandler.WriteToken(token);

Refresh Token

The refresh token has a longer life, perhaps 4 hours. It won't contain any sensitive information beyond a user identifier and even this can be rendered unnecessary.

The purpose of this token is simply to preserve a user login outside of an webserver session and to enable regeneration of the access token without the need to store usernames and passwords within cookies.

// using Microsoft.IdentityModel.Tokens;
// using System.IdentityModel.Tokens.Jwt;

var tokenHandler = new JwtSecurityTokenHandler();
expiryDate = _jwtConfig.RefreshTokenValidMinutes;
var key = Encoding.ASCII.GetBytes(_jwtConfig.Secret +
    REFRESH_TOKEN_SECRET_SUFFIX);

var claims = new Dictionary<string, object>
{
    { "expiry", expiryDate },
    /* 
        We provide several login methods which aren't user
        specific, this will be lost when the access token
        expires, so we'll save it to the refresh token.
    */
    { "loginMode", activeUser.LoginStatus },
    /*
        We also require a separate user CRM login so this
        information also gets encoded in our refresh token.
    */
    { "crmRoles", activeUser.ConnectedCrmFunctionality },
    { "isAuthenticatedToCrm", activeUser.IsAuthenticatedToCrm },
};

var tokenDescriptor = new SecurityTokenDescriptor
{
    Subject = new ClaimsIdentity(new[] {
        new Claim("id", activeUser.Id.ToString()),
        new Claim("userName", activeUser.UserName.ToString())
    }),
    IssuedAt = DateTime.UtcNow,
    Claims = claims,
    CompressionAlgorithm = CompressionAlgorithms.Deflate,
    Expires = expiryDate,
    SigningCredentials = new SigningCredentials(
        new SymmetricSecurityKey(key),
        SecurityAlgorithms.HmacSha256Signature,
        SecurityAlgorithms.HmacSha256Signature)
    };

var token = tokenHandler.CreateToken(tokenDescriptor);
return tokenHandler.WriteToken(token);

Login

Once a users' credentials have been confirmed the JWT access and refresh tokens are returned to the client as cookies.

These cookies should be set to expire at the same time as the associated JWT token, configured with HttpOnly enabled and the SameSite mode set to Lax. This will secure the cookies so that they can't be read by JavaScript and can only be included in requests sent to the originating server.

One thing of note is that cookie sizes are typically restricted to 4Kb so we need to be careful about what information is included within the access token, keeping it to critical & commonly used values only and ensuring that the encoding method in place removes unnecessary text (i.e. don't encode values using JSON with long property names).

Depending on application flow, it may be necessary to add cookies not only to the response object but also to the request object. This will permit the access token to be regenerated at the start of a call and for the new value to persist without the need to make an additional server-side call.

var cookie = new HttpCookie(cookieName, value)
{
    Expires = DateTime.Now.AddMinutes(expiry),
    HttpOnly = httpOnly,
    SameSite = SameSiteMode.Lax
};

if (!context.Response.Cookies.AllKeys.Contains(cookieName))
{
    context.Response.Cookies.Add(cookie);
}
else
{
    /* If cookie already exists just update it otherwise we'll add
    more data to the HTTP response and potentially trigger an 
    overflow */
    context.Response.Cookies.Set(cookie);
}

Securing Server-Side Functions

Now that the JWT tokens are available as cookies we can secure our server-side functions. We achieve this by creating an instance of AuthorizeAttribute which in this example we'll call ProductAuthorizeAttribute.

First we need to reference this attribute in FilterConfig.cs, this will make sure that all functions are secured by default.

public class FilterConfig
{
    public static void RegisterGlobalFilters(GlobalFilterCollection filters)
    {
        /*
            Set order to a value greater than 0 so that
            we can override default security on a
            per-function basis.
        */
        filters.Add(new ProductSecurityAttribute(), 255);
    }
}

We'll want to prevent security from being triggered on certain operations, such as the login screen. This can be done by adding the AllowAnonymous attribute to any functions that don't need to be secured.

public class LoginController : BaseAsyncController
{
    [AllowAnonymous]
    public ActionResult Index()
    {

If we want to apply additional security on a given function we can do this by applying the ProductSecurityAttribute to it and specifying any additional properties.

[ProductSecurityAttribute(Operation = SecurityType.Create)]
public async Task<ActionResult> ActionName()
{

We can then setup the security attribute so that it automatically reads and decodes the JWT tokens and returns the user to a login or access denied screen if appropriate.

public class ProductSecurityAttribute : AuthorizeAttribute
{
    private bool _triggerTimeout;

    public ProductSecurityAttribute()
    {
        /*
        We need to set the attribute order to 0 so that 
            it gets processed before the default attribute 
            configured in FilterConfig.cs.
        */
    Order = 0;
    }

    // Extra security properties here 

    /*
        Now we override the authorise command and
        add our own logic 
    */
    protected override bool AuthorizeCore(HttpContextBase httpContext)
    {
        var hasAccess = false;

        // decode cookies
        var accessCookie = _cookieHelper.Get(_currentContext, ACCESS_TOKEN_NAME);
        var refreshCookie = _cookieHelper.Get(_currentContext, REFRESH_TOKEN_NAME);

        // decode access token
        var jwtUtils = new JwtUtils(_jwtConfig, HttpContext.Current, _cookieHelper);
        var accessToken = jwtUtils.VerifyAccessToken(accessTokenValue);

        /* 
            If the access token has expired then generate a 
            new one using the refresh token.
        */
        if (accessToken.LoginStatus == UserLoginStatus.ExpiredAccessToken || 
            accessToken.LoginStatus == UserLoginStatus.TokenMissing) 
        {
            var refreshToken = jwtUtils.VerifyRefreshToken(refreshTokenValue);

            /*  
                If the access token has expired but the refresh
                token is fine then regenerate both tokens.
            */

            if (refreshToken.Id > 0 &&
                refreshToken.LoginStatus != UserLoginStatus.ExpiredAccessToken &&
                refreshToken.LoginStatus != UserLoginStatus.ExpiredRefreshToken &&
                refreshToken.LoginStatus != UserLoginStatus.Failure)
            {
                // regenerate access & refresh tokens
            }                   
        }

        /*
            If all tokens have expired, return false and set
            _triggerTimeout = true.

            Otherwise, execute custom security code to work out if 
            the user has the necessary access
        */  
        if (hasAccess)
        {
            // user has access
            return true;
        }
        else
        {
            // redirect to access denied screen
            SetAccessDeniedModel(httpContext, itemIdValue);
            return false;
        }
    }       

    /*
        When authentication fails SecurityAttribute automatically
        calls this function which we can override to define
        how the application behaves when login fails.
    */
    protected override void HandleUnauthorizedRequest(AuthorizationContext filterContext)
    {
        // child actions can't execute a redirect
        var isChildAction = filterContext.IsChildAction ||
            filterContext.HttpContext.Request.IsAjaxRequest();

        if (_triggerTimeout)
        {
            // The user login has timed out.
            var redirectParams = new StringBuilder();
            redirectParams.Append("?logout=true");

            if (HttpContext.Current.Request.ApplicationPath !=
                 HttpContext.Current.Request.CurrentExecutionFilePath.TrimEnd('/'))
            {
                redirectParams.Append($"&missingSession&targetPage={HttpContext.Current.Request.CurrentExecutionFilePath}");
            }

            /*
                If this is a child action then return an
                error state otherwise we can redirect the
                user to the login screen. 

                If we throw 401 in an environment using IIS 
                windows authentication then the browser will 
                prompt the user for windows credentials so we're
                using a 405 instead.
            */  
            var redirectPath = $"~/Login{redirectParams}";
            filterContext.Result = isChildAction
                ? (ActionResult)new HttpStatusCodeResult(System.Net.HttpStatusCode.MethodNotAllowed, redirectPath)
                : new RedirectResult(redirectPath);
        }
        else
        {
            // The user is logged in but access has been denied.
            var redirectPath = "~/AccessDenied?redirect=true";
            filterContext.Result = isChildAction
                ? (ActionResult)new HttpStatusCodeResult(System.Net.HttpStatusCode.Forbidden, redirectPath)
                : new RedirectResult(redirectPath); // redirects the particular response, not everything
        }
    }
}

Securing AJAX Calls

SecurityAttribute is able to redirect calls to an access denied or login screen, but we can't do that as easily when an AJAX call fails and if a page is AJAX heavy then we'll need to process both response types.

$.ajaxSetup({
    // Prevent ajax calls from caching the response
    cache: false, 
    headers: antiForgeryTokenHeader(),
    // Intercept the AJAX complete function
    complete: function (xhr) {
        if (xhr.status == 405) {
            /*
                User session has timed out.
                Redirect user to the login screen,
                encode the target page in the URL 
                so we can redirect on login
            */
            var rootVal = "@(Url.Content("~"))";
            var currentHref = window.location.href;
            var targetPage = currentHref.substring(currentHref.indexOf(rootVal));
            var targetPageParam = "";
            if (targetPage != rootVal) {
                targetPageParam = "&targetPage=" + targetPage;
            }
            window.location.href = "@(Url.Content("~/Login"))?logout=true&missingSession" + targetPageParam;
            setLocalStorageUserState("logout");
        }
        else if (xhr.status == 403) {
            // access denied message
            window.location.href = "@(Url.Content("~/AccessDenied?redirect=true"))";
        }
        // ignore signalr notifications for service status etc
        else if (xhr.responseText != '{ "Response": "pong" }') 
        {
            // process user activity timeout
            userTimeoutReset();
        }
    }
});

Making Token Values Available to the Browser

The standard approach should always be to create the JWT cookie using the HttpOnly parameter because this will ensure that an attacker can't leverage JavaScript to read the access and/or request tokens.

However there are scenarios where it might be necessary to read values included within the access token from within the browser.

In this scenario it's useful to know that a JWT token is split into Header, Payload and Signature. We can split these into separate cookies and allow payload data to be read from within JavaScript but re-combine them within server operations in order to validate the token.

Forcing Session Timeout

We can use this method (or another cookie) to expose the amount of time until the users session is due to expire and display a warning message on-screen before automatically logging the user out.

function userTimeoutReset() {
    //reset any active timeout
    userTimeoutElapsed(false); 

    var timeoutDate = get_cookie("t");
    if (timeoutDate != null && timeoutDate.length > 0) {
        /* 
            Calculate ticks until we should start 
            displaying the timeout warning
            with 2 minutes leeway
        */
        var timeoutTicks =
            new Date(timeoutDate) -
            new Date() -
            ((USER_SECURITY_DEFAULT_TIMEOUT_PERIOD_MINUTES + 2) * 60000);

        userTimeout = setTimeout(function() {
            userTimeoutElapsed(true);
        }, timeoutTicks);
    }
}

function userTimeoutElapsed(timeoutElapsed)
{
    if (userTimeout != null)
    {
        clearTimeout(userTimeout);
    }

    if (userInterval != null) {
        clearInterval(userInterval)
    }

    if (timeoutElapsed) {
        userTimeoutCountdownPeriod = USER_SECURITY_DEFAULT_TIMEOUT_PERIOD_MINUTES * 60;
        setLocalStorageUserState("resetTimeoutCounter");
        userInterval = window.setInterval(function () {
            if (userTimeoutCountdownPeriod > 0) {
                userTimeoutCountdownPeriod = userTimeoutCountdownPeriod - 1;
            }

            $("#userTimeout").html("Inactivity timeout in " + userTimeoutCountdownPeriod + " seconds");

            if (userTimeoutCountdownPeriod == 0) {
                window.location.href = ROOT + "Login?logout=true&timeout=true&targetPage=" + window.location.pathname + window.location.search;
                setLocalStorageUserState("logout");
            }
        }, 1000);
    }
    else {
        setLocalStorageUserState("abortTimeout");
    }
}

Persisting Logout to all Tabs

Finally, if the user has multiple tabs open then we'll need to maintain logout timer between all tabs to avoid the user being automatically logged out on a tab that's been ignored for a few hours.

We can use browser local storage to send messages between tabs.

function setLocalStorageUserState(value) {
    if (window.localStorage != null) {
        /* 
            Reset the existing value 

            The change event won't trigger if the 
            value is the same so we'll need to
            do it twice.
        */
        window.localStorage.setItem("UserState", ""); 
        window.localStorage.setItem("UserState", value);
    }
}

/* 
    Listen for events from other tabs, 
    process result in a manner which 
    will prevent getting into an 
    endless loop of tab communication
*/

window.addEventListener('storage', (event) => {
    if (event.storageArea != localStorage) 
        return;

    /* 
        We're processing a message from another 
        tab, prevent this tab from sending messages 
        to other tabs & sending us into a loop
    */
    if (event.key == "UserState" && event.newValue != "") {
        // if the user has an active session
        if (window.location.href.indexOf("Login?logout") == -1) {
            if (event.newValue === 'logout') {
                // another page has been redirected to the login screen
                window.location.href = ROOT + "Login?logout=true&targetPage=" + window.location.pathname + window.location.search;
            }
            else if (event.newValue === 'abortTimeout') {
                // action was performed on another tab while we were counting down to timeout, abort this
                userTimeoutElapsed(false);
            }
            else if (event.newValue === 'resetTimeoutCounter') {
                // action was performed on another tab, reset the timeout counter
                userTimeoutElapsed(true);
            }
        }
    }
});

HTML Email Editors: A Design Case Study

Michael Kennedy — Sat, 02 Jan 2021 12:37:45 +0000

Applications that allow creation of stunning responsive emails are everywhere yet it's rare to find any that appeal to users of all experience levels.

As a professional, writing a responsive website is an easy process. Take a dash of HTML, mix with JavaScript and CSS, finally sprinkle with a bundle of experience and we end up with something pretty amazing.

But even for professional designers, repeating the same tasks within an HTML email is exponentially harder, mixing techniques from years gone by and modern practices in order to provide an optimal experience for all email clients.

This previous article discusses the complexity of Responsive HTML email design and the difficulties that can arise for the most experienced of us. With this in mind, how do we go about producing an editor that's perfect, or at least as close to perfect as we can get.

User Profiles and Experience

There are lots of applications that allow experienced users to produce stunning responsive emails. There are also countless apps that allow inexperienced users to produce equally stunning emails. However, it's rare to come across an HTML email editor that appeals to both groups of users.

The reason for this is simple to understand once you consider the level of knowledge possessed by the two user profiles. The difference comes down to the relationship between senior and junior roles.

At the novice level, users might have an idea of what they want a mailing to look like but not the experience to produce it quickly; they need help to build something to allow them to concentrate more on the content than layout and behaviour.

At the expert level our users are more likely to have a well-defined idea of what their mailing should look like and how it should be laid out, but often find that the restrictions of the editor prevent their vision from being fully realised.

More often than not, what we end up with is an app that provides the building blocks but often requires consultants or even paid developers in order that vision to make it to a users' inbox. All this comes with a hefty expense for clients.

So what HTML Email editors are out there, do they achieve a balance between the two approaches or do they just provide simple editors and force their clients to pay hundreds if not thousands for something more bespoke?

User Expectation

Before we consider how a responsive email application ought to work, we must first determine what users expect to see and how established practices mould that expectation. Providing something familiar helps break down the walls between our novice & expert user groups, it offers a starting point with which everyone is familiar allowing us to tag on extra features that users discover as they become more intimate with the processes we'll put in place.

Until a few years ago, the common denominator for all users would have been something like Microsoft Outlook. This is still common for business users, however with cloud computing and mobile devices becoming more common, simplified email clients like Gmail mean that users don't necessarily all have the same level of experience.

So our lowest common denominators range between the simplistic UI provided by Gmail/Outlook.com and the text editors such as Microsoft Word that allow creation of more advanced layouts. But; we're not producing a PDF document here, we're looking at something that can be displayed in varying sizes, on varying devices and that's before we even consider DPI and device text size options.

Existing HTML Email Editors

Email HTML editors come in all shapes & sizes, but they can be broadly split into two categories - Text editors and layout designers.

Users are familiar with the former, embodied in the likes of Outlook and Gmail. There are other options, but the restrictions of HTML in general and specifically in email make it harder to achieve a design guaranteed to work on every device.

There are many, many existing HTML email designers out there, but for the sake of simplicity we'll narrow it down to some of the common ones.

MailChimp

MailChimp comes closest to mimicking a traditional email interface, the initial screen is split nicely into sender, recipient, subject and content areas.

While there are a few oddities it provides a decent, if somewhat limited editor. Perfect for users who want to produce something quickly without any fuss.

Campaign Monitor

Campaign Monitor ups the game a little. It doesn't provide quite the same traditional email interface, opting to hide recipient selection until the entire mail has been designed; a common approach among marketers but not suitable for every situation.

Its editor is a little more advanced than MailChimp, and provides a few nice extra touches like image cropping. Extra brownie points for its collection of ready-to-go layouts.

BeeFree (or Pro)

What BeeFree provides is far more of an email editor rather than a fully-fledged mailing system. It provides a more than generous selection of pre-configured layouts but generated content then has to be download for use in a separate mail client.

This editor is far more refined than its counterparts, providing a well defined section editor and drag/drop interface, and options for padding, line height, hexadecimal colour codes alongside a brilliant image editor.

The editor receives extra kudos for its ability to show or hide elements in desktop/mobile mode. Now, you might argue that hiding items in mobile mode is the wrong approach, and in most circumstances you'd be right, but without deeper knowledge of HTML then there really isn't any other option here.

Mosaico

Mosaico takes a similar approach to the Bee editor, ignoring traditional mailing elements and producing pure downloadable HTML content.

The editor provides preconfigured, draggable sections. Editing of section layout is limited, however it does provide a range of stylings that can be applied to individual content items and full HTML editing of certain areas.

The Mosaico editors' handling of colour schemes is where it shines, providing the ability to swap the colour scheme on any particular section at the click of a button. There may only be two colour schemes available in the default view but the editor hints at the ability to add further schemes.

Of these well-known tools, there's no one clear best-of-breed option. They all offer good WYSIWYG editors, each with their own advantages. They're all brilliant tools in their own right, but in their desire to simplify the approach they miss out on the bigger picture and the pixel perfect designs that so many users are looking to achieve.

Features of the ideal HTML editor

We know what users are likely to expect, and for the purposes of a design review, we have an idea of what other editors can achieve (and more importantly, what they're missing out on). But we want a fully functioning HTML email editor, usable by all and that requires some more in-depth thought about how our editor should work.

The most important thing (and something that should ordinarily go without saying) is that our editor needs to be as close to WYSIWYG as is possible.
There will always be differences in the way that certain mail clients render our generated HTML, but users need to be able to make a change and see its immediate impact on the design while maintaining a high degree of certainty as to how it will look to the mail recipient.

We should be assisting this further, with at a minimum, the ability to swap the editor mode between different screen sizes and view modes in order to simulate how a mailing may look on a smaller screen or in dark mode.
A user won't necessarily know all details of a mailing at the point of creating it; as with product design, the details can be in flux right up until the final design is complete.

With this in mind, we want users to be able to view sender, subject, recipients and the email design all at the same time and (perhaps even to include some additional fields like the preheader and send time etc). But we don't want this overshadowing the design view, ideally these basic fields should be visible alongside the mailing content.
Pre-defined layouts look pretty and help to inspire users, but they can be restricting. So we'll want the choice to select between several layouts (or a blank layout), similar to how modern versions of Microsoft Word operates.
To get past the restrictiveness of such layouts, we'll want to complement our functionality with draggable rows which can be positioned anywhere within existing content in order to provide responsive email elements. For example we might want a tri-column section which automatically stacks on smaller devices; we might want to allow content to be dragged into a very specific area, preventing changes to all other areas.
In order to maintain compatibility with all mail clients, our sections will need to be table-based. In turn, this opens us up to the possibility of introducing another area that users of common office applications will be used to: the ability to split or merge cells in order to further redefine the layout.

Splitting or combining sections will update the manner in which responsive email design will operate so an element of caution is required. Also many HTML emails tend to use multiple levels of nested tables, we'll need a method of separating those used for content from those added purely for layout purposes.
With our layout in place, we'll need to provide the ability to add content components, making adding content as easy as possible at a minimum we'll need to provide text, image and buttons with the default mode being to add text.

We'll want to give consideration to dividers, bullet points, social media and perhaps even content pulled from an RSS feed. Some mail clients support gifs and videos, so we may also want to consider this, but there are other implications around using these sort of features which would need to be considered.

When dragging items we'll need to implicitly create a table structure around our objects to allow them to be positioned properly, this would also facilitate easier repositioning of components.
To help users retain similar designs throughout multiple mailings users will want to re-use sub-sections of designs elsewhere in much the same way as adding an image or text area to a mailing. We can allow users to build collections of design elements for later use, even allowing importing of design elements from previous mailings.
On selecting a section area or a content item we'll need to provide a list of common options available in most text and HTML editors.

This will require separating standard text editor functions into a standard menu/ribbon, or perhaps inline editor while the HTML editing functions (which typically would require more screen space) are held elsewhere.

At the very least we'll want to expose the standard font options, alignment, colour options (foreground, background & text highlight), padding options & table manipulation functions. But we'll want to keep it simple, with context-sensitive options.
CSS is a huge part of HTML content, advanced clients tend to setup standard CSS styles for their emails, sometimes reusing stylesheets from other locations such as company websites.

Most email clients require CSS in HTML email to be inlined; to simplify the lives of developers we'll need to provide the ability to add CSS (or even SCSS/LESS) directly to an email and automatically inline it at an appropriate time.

Exceptions to this are media queries (usually produced to work alongside dark mode and responsive design) which will need to be handled separately. We can handle separation of media queries from the core CSS automatically, but users will need some method of editing these options.
Full HTML access can be dangerous, but there are clients who will have their designs professionally built and will then want to import that content into an editor for easier manipulation. Equally there will be those who know exactly what they want and how to go about it.

Most HTML email editors tend to concentrate on simpler functionality, but ours needs to be opened up to users of all levels in a safe manner.

Editor Layout

So here I am, typing into a Microsoft Word document with both the left and right-hand panels open alongside my document, and it occurs that when there is enough screen space to play with, this is the ideal layout for our HTML editor, not only because it's an interface which will be familiar to users, but because it allows us to better guide them through the creative process.

We start on the left, with our non-design items (subject, sender, etc) and drag/drop components, move to our full design and WYSIWYG editor and provide a final panel in which a selected component can be edited. This allows the users flow to move left-to-right with the items more likely to be used at the beginning on the left and everything else on the right.

There is of course, a bit of rearranging to be done for smaller screens, where the visibility of content should be prioritised; the nature of such devices means that it will always be easier to create an HTML email when using a larger screen.

In this screen design, we've placed a number of tabs on the left-hand side with the following options:

Email - containing typical settings for email subject, recipients, sender, etc.
Layout - containing draggable layout sections (single column, multiple columns, 60/40 split, etc).
Design - containing draggable content items and user-defined snippets.
CSS - for configuration of pre-inlined CSS to be applied to the entire mailing.

In earlier designs, this section contained a button to expand the section over the design area, however given that this area could well include the need for a scrollbar this could cause the user interface to appear cluttered. But with the possible length of fields such as the subject and sender name more screen space would be useful to users. We could achieve similar functionality by automatically expanding the area of the emails tab as soon as a user selects a field.

The right-hand area is displayed only when an item within the editor is selected:

Containing CSS and HTML attributes appropriate to the item (those that will work within HTML email).
Toward the bottom of the section we have options to swap between complexity modes, standard, advanced and expert - these may be better off being placed within the "View" tab of the main editor if usage can be expanded outside of this area. This would have the effect of making users search for advanced options and reduce the likelihood of making advanced changes to areas that they don't fully understand (particularly important for HTML emails).

Finally, the central area takes a more traditional word-processor approach, with a central defined content area and options to zoom in and out in order to view the content in its entirety.

A ribbon style menu at the top of the screen provides additional options:

Edit - standard edit toolbar containing font options, alignment, text colour, paste formatting options, etc.
View mode options, allowing users to swap between simulated design views for desktop & mobile views. Also provides functionality to view without CSS media queries to simulate appearance under desktop applications like Outlook.
Send options - containing final send operations, validation, configuration of plain text, send time, etc.

In the examples of the left-hand panel here, the first tab shows our primary email options, required fields are pushed to the top (and will be identified with an appropriate icon) while less important items are further down. Expandable to any number of possible options, although given the plethora of CRM systems available, recipient selection may be better off being defined as a completely separate screen.

The simplistic drag and drop interface initiated from the second tab provides the ability to build the layout of the email design with built-in responsive design elements. While not displayed here, the appearance of the third tab is would follow a similar design approach with the exception that design elements need to be dragged into an existing layout element rather than directly onto the canvas.

As discussed above, the right-hand panel contains configuration options for all HTML attributes and CSS settings within stylised accordion settings. In smaller screen designs, this panel is repositioned to the left of the screen and in even smaller screens the left pane is collapsed entirely until required.

The complexity of these options will be driven by settings within the view tab. Standard mode provides most options but keeps the configuration simple (i.e. allowing padding to be set to a single value), advanced opens it up to the usual options (i.e. allowing top/bottom/left/right padding values to be set independently) while Expert mode allows users to set the padding values directly (i.e. users can type whatever they deem appropriate, including usage of CSS variables which will be inlined into HTML content as part of a separate process).

The design view is intended to operate in as close a manner to a standard text editor as possible, the key differences being that administrators are able to define sections that cannot be edited, this ensures that email designs will always be able to follow a set design pattern. HTML content is hidden except when running under Expert mode which provides full access to the underlying HTML of individual sections and components.

Finally, in addition to editing emails directly, the designer doubles as a template editor, giving expert users access to the entire HTML content and allowing them to define areas where layout areas can be added by standard users. This allows provision of functionality impossible in all HTML email editors where clients want to add double borders and other design elements around the entire content - something that the adding of layout sections can't deal with on its own, particularly where older email clients are concerned.

Conclusion

In conclusion, creating a pretty, responsive HTML emails has always been a complex task. Balancing behaviour so that users with all levels of experience can construct their own content will always be something of a tight-rope walk, but given enough consideration of the options we make available to our users anything is possible.

Responsive HTML Email: First Steps

Michael Kennedy — Tue, 26 May 2020 09:59:36 +0000

For most of us, producing a professional email at work is easy - open your chosen mail client; select your recipients; add a subject line and type your content. If you're anything like me the hardest part is tweaking the wording so that it puts your message across with the right professional ambience.

This process suits most of us during our day-to-day lives. Most email applications allow us to go further than basic text, quite often providing full text editor functionality, this is more than enough functionality for the scenarios which most of us are ever likely to encounter.

But there's a step beyond that which we don't usually have to think about, fully formatted HTML emails like those from Amazon, eBay and countless other services giving us account updates or sending marketing communications.

Going about producing an email with this level of professionalism is a whole step above what most email applications are capable of.

We might assume that all we need to write an HTML email is some basic HTML knowledge. To a certain extent this is true; but - unlike modern web browsers which receive tend to have high-pace release schedules and automatic updates - email clients tend to remain static.

Put simply, email clients are stuck in the past, if we want to write a professional looking HTML email, we need to take that onboard from the get-go.

Email Clients

Email clients come in three distinct categories - traditional windows applications (such as Outlook or Apple Mail), web based applications (such as Gmail or Outlook.com) and mobile applications (like Apple iPhone or Samsung Mail).

In 2019, Litmus reported that usage of web based email clients and mobile applications were evenly matched with mobile apps remaining marginally ahead of web clients for the past three years. Usage of windows clients has remained at around 18% for the last four years but interestingly the data shows a potential upward trend.

The other important piece of information that this report shows us, is that Microsoft Outlook is the third most used email client; this is up from #5 the previous year and accounted for 9.2% of all recorded email opens.

The reason that this is of particular importance is because Outlook uses Microsoft Word to render emails* and has done since 2007. As a consequence, Outlook suffers from a bunch of behavioural quirks which adds an extremely high level of complexity to HTML emails. Microsoft have previously indicated that they may be willing to address this, however as of Outlook 2019, little has changed; and even if they had swapped to a web based rendering engine we'll still be picking up the pieces from older installed versions for years to come.

I'm still running Outlook 2013 on my personal laptop (although I now rarely use it), my office PC runs Outlook 2016 which gets used all the time. However I can't see myself going through the hassle of upgrading it any time soon.

* Except Office 2011 on Mac which used a separately bundled email application.

Email Content

Conventional web development practices advise positioning elements using divs and floats. More recent practices expand on that with advanced CSS functionality such as Grid and Flexbox.

None of this more advanced functionality can be applied to emails because Outlook cannot possibly support them. We're also restricted because of the nature of displaying an email within a browser:

Use of CSS is mostly restricted to inline styles (because we don't want styles from the opened mailing breaking the layout of the parent page).
Use of JavaScript (imagine the security implications).
Use of iFrames - (Limited use in some clients*, unlikely to be of wide use because of the implications of introducing CSS style tags and JavaScript).
Embedded audio or video content is unsupported by most email clients (I get annoyed at social media videos that auto-play, doing so within an email would be a sure-fire way of a sender getting themselves blocked).
Forms - a number of email clients treat forms as a security risk, some will trigger spam filters.
SVG Images are largely unsupported by the majority of email clients.

* As noted by Campaign Monitor

With all these restrictions in mind and to support the lowest common denominator; we have to go back in time and properly format our content using nested tables. We'll also need to be more cautions about our choice of formatting options, resisting the temptation to use more recent CSS options and where possible defer to HTML attribute equivalents such as align, valign, cellpadding, cellspacing, width, border etc.

Email CSS Styling

In the past, support for CSS styling within email clients has been quite patchy. Although this has been gradually improving, we're still greatly restricted by the email clients that our recipients will have installed.

A large percentage of web based clients now support <style> tags (best placed within the <head> section) while a small percentage support externally referenced CSS files using <link> tags.

As a best practice, all CSS styles should be applied directly to elements using inlined styles. In practice it's useful to create CSS styles in the usual way and use a third party tool such as premailer to automatically inline our styles and reduce bugs. Style tags should be used with caution, tho they still have their place when it comes to responsive email design.

Unsurprisingly, support for CSS styles within emails isn't as wide ranging as it is in normal web development, we have a core group of CSS options available to us, after which we'll need to take an approach of progressive enhancement when adding other functionality.

Safe CSS Properties

CSS properties which are well supported across all email clients or that can be used without fear of causing display issues in other clients.

Well Supported CSS Properties

CSS properties which are supported by the majority of email clients, can be used with the appropriate fallback or using a pattern of progressive enhancement.

CSS Properties to Avoid

Font Styling

As with fonts on websites, your selected font depends on the fonts available on the users system; it's always best to select a fallback when working with a less common font. The same goes for web-fonts, which not all email clients support, again requiring that a suitable fall-back is in place using inline CSS.

We also need to be conscious that CSS support for font styling options under older email clients can be a little erratic. To be absolutely certain of applying bold, underline, italic and strike-through styles we need to fallback to <strong>, <u>, <em> and <del> syntax.

See this article by Mailchimp which digs deeper into font usage in HTML email.

Responsive Email Design

Responsive design is best conducted when creating emails with a predefined maximum width in the region of 600-800 pixels, this range covers the width available in the majority of applications. There's little point in expending time on sizes not generally required by users particularly with the reduced levels of content in email communications.

The 600px figure goes back to the days of 1024px x 768px screens and the size of Microsoft Outlooks viewable area in those days with around 424px taken up by outlooks sidebar and the vertical scroll for email content.

With responsive design we have a bit more flexibility, but we still have to consider other non-technical elements such as the users attention span over longer lines and how much harder it is to trace back to the beginning of the next line without losing your place.

Many email clients still require their content to be held in a base table, so as far as responsive design is concerned, this has to be our starting point.

We'll configure a parent table with 100% width, this would perhaps contain a default background colour which differentiates itself from the mail client. We'll then add a child table, this table is assigned a max-width within the desired range and forced to the centre of the viewable area.

Note that wherever possible we're positioning our components using older style attributes rather than CSS styles.

Our responsive options when building HTML emails are greatly restricted without the likes of Flexbox or CSS Grid to work with. We do have the ability to write basic @media queries but should we use them?

Writing a responsive email isn't like putting together a responsive website where you have maybe three or four major browsers to worry about, most of which are based on Chromium anyway.

Device types, widths, pixel density etc are far more varied, just because we can use media queries doesn't mean that we should use them for anything more than progressive enhancement.

Our alternative to media queries is to use an approach termed as "fluid-hybrid". With this process we don't need to worry about matching screen designs to different sizes. We create a parent "ghost" table which has a width value of 100% and a max-width of our maximum defined email width defined in pixels (say 600px), into that table we add our "responsive" elements in the form of additional table elements or containers each with the same 100% width value and an appropriate max width (say 2 equal columns each with a max-width value of 350px).

Taking this approach means that our email content will always fill the available space and will naturally drop from columns to rows when the screen width is not wide enough.

Other Areas of Concern

Regardless of which approach we take, there are a number of areas that we may need to give consideration to:

When supplementing inline styles by using CSS style tags, be sure to make use of the `!important` flag where there may be overriding properties within the inlined CSS, as this will take priority.
Split any responsive CSS code into separate style tags, if there are any errors or compatibility issues then the client may strip the entire style section from the email source, separating CSS into sections will reduce the likelihood of this causing further problems.
When configuring media queries on mobile devices it's easy to concentrate on device width as we would do normal responsive web development, but pixel density varies more widely on mobile devices. This should also be considered when selecting images to ensure that users have nice crisp and clear experiences regardless of device
When using GIFS, remember that some mail clients will only display the first frame.

The Future: Dark Mode

With the increasing use of dark mode options within email clients (with Gmail recently introducing this as an option), dark mode in email design is becoming more prevalent.

Email clients are still figuring out the best way of implementing dark mode, some just change the email interface while others provide varying degrees of colour inversion.

The core problem around dark mode is display of images and HTML elements traditionally designed to work on a lighter background. For example a darker logo on a transparent background won't be readable in dark mode. Most of these sort of issues can be tackled with a little forethought but this is really just scratching the surface of what will be possible in the future.

For reference, Litmus have produced a guide on how to tackle dark mode in emails here.

Final Thoughts

Know your audience: With the inclusion of an invisible 1x1 pixel image we can read the client user agent string and retrieve some basic information without breaching GDPR, CASL or any other data protection regulations.

This can be decoded to give us information about the type of device our recipients are using and the application they're opening it with.

While there are some scenarios where an email client may not automatically render images, this approach should give us enough information to identify the range of devices our emails are being opened with. We can then adjust our approach accordingly.

Test, test and test some more: If budget allows, use an email testing service, such as Litmus or Email on Acid which will automatically test your email content to give you an idea of how it would appear under different devices.

Don't forget about plain text content: We can produce fancy looking emails, but we can't forget that even now, not all email clients can read HTML content. Perhaps users have HTML content turned off for accessibility purposes, perhaps they're still using an old blackberry, perhaps they're reading an email on an iWatch. Even if it's automatically converted from HTML. Plain text email conent may not be as important as it once was, but it's still important.

Don't over complicate: The idea of keeping to a simplistic design doesn't just apply to HTML email content. The range of devices and applications available makes it a far greater concern than normal.

In the end, there's no point in producing a fancy email design which will only work for 20% of users. When in doubt, keep designs simplistic to reduce the likelihood of display issues in different clients.

Content is still king. For all we can achieve with HTML email, if the end result is that if we force users to think about how they're going to read an email then it's not worth it.

Thank you for reading, If you got this far please feel free to leave a comment, alternatively if you were even remotely entertained please hit the like button, or follow me on Twitter.

Grief-inspired Design

Michael Kennedy — Sun, 17 May 2020 14:39:48 +0000

Product design doesn’t have anything in common with processing intense grief. Does it? Sometimes we have to be forced into taking a step back in order to see the obvious.

Everyone grieves in their own way; everyone feels it differently. In our recent experience, we didn’t have any advanced warning, no time to prepare; this wasn’t something that can be classified as an act of god. It was deliberate. Our experience opens our eyes to how vulgar and unforgiving the world can be.

Spaghetti-brained grief

As a software developer, I spend my week staring at complex code, muttering and swearing (sometimes under my breath 😉) until everything compiles.

Grief complicates even the simplest task, it makes everything feel insurmountable and makes us question ourselves even when we have no need to. In such a situation, we gravitate towards effortless solutions in what usually ends up feeling like a meaningless effort to reassert a level of control over life.

Writing software isn’t just code, in the right mindset it’s an art form. Admittedly most users don’t get to see our art because it’s hidden away in compiled DLL’s and other encoded files.

When suffering from grief our thoughts are jumbled like the worst spaghetti code you’ve ever seen. As with coding, the only way to fix the problem is to search for the smallest class, function or module, take that as a starting point and simplify everything from that point onward.

It was that search for simplicity which fuelled the return to my creative side. As Headspace states:

Some of our greatest cultural riches are the byproduct of grief: the pyramids at Giza were burial mounds. Homer’s “Iliad” is a tragedy. Christian texts begin with the loss of Eden. From a safe distance, we admire and appreciate the creative fruits grief has harvested over the course of millennia; collectively, humanity owes a whole lot to heartache.

This simple statement was something that echoed in my mind, albeit on not anywhere near as grand a scale.

In one way or another, we all yearn for simplicity, the feeling of grief merely accentuates that feeling, bringing it to the forefront. Grief makes the smallest things feel that much harder to achieve, it creates a feeling of distance between ourselves and the things around us which if we’re not careful, can snowball into something crippling. A sentiment with which I became innately familiar.

Return to work

In the office, among the multitude of our internal systems is our development task tracking application; it’s a bespoke tool written in ASP with an appearance which wouldn’t have looked out of place in the latter half of the 1980's.

As an internal system, it was always on the back-burner as far as bug fixes and enhancements were concerned, despite being used by the majority of the team on a daily (if not hourly) basis. More often than not, any updates are hastily applied so that more time can be devoted to the products that keep the company running.

Because of this, its usability is poor. Sure, the various behavioural oddities and workarounds become second nature, but in the midst of spaghetti-brained grief the tool I’ve been using every single working day for the last who-knows-how-many-years was making my life harder.

I realised that I could actually get the data I wanted more quickly by querying the database directly than using the front-end application. During a grief fulled rant on the subject a trusted colleague pointed out the obvious — we’re software developers, so if we can’t fix it who can?

Ordinarily I probably would’ve thought of this as not being my problem, after all, we’ve got enough work to do as it is. But grief does funny things to you...

Grief-inspired design

It’s a known fact, that there simply aren’t enough hours in the day. This is very true of the software development world, where sometimes we’re forced to pick between doing something the right way, or doing it quickly.

But a project inspired by grief can’t function in this way. Everything needed to be perfect and that’s something that can’t happen with an internal system.

So, this was to be an unapproved, out-of-hours project. Projects of this nature allow us to experiment with different concepts and ideas, we learn more in this sort of situation, where the constraints of time and money don’t apply but there is still a purpose to the work at hand.

Before long, I had transitioned through half a dozen different design prototypes and rediscovered my creative side, realising that this was the sort of venture that I so badly needed.

In time, I came to the conclusions that anything I built needed to follow a few basic tenants:

A simplistic colour pallet. Two or three colours supported by application areas defined by black outlines and white or grey interiors.
Minimal interface. The existing functionality needed to be preserved, but there’s no point in littering screens with operation buttons which add clutter.
Multi purpose components. A pop-up isn’t just a pop-up, it provides a status indicator, a tabstrip isn’t just a navigational tool, it provides statistical information. The possibilities go on…
All pertinent information has to be available at a glance, minimising additional navigation and providing an immediate overview to the end user.
We can draw attention to product areas with animations, but using fancy animation to reveal product areas only serve to slow the users progress (I really should’ve known better before trying this).

The existing tool wasn't overly complex, but it does provide a plethora of options for use by the various teams. Beyond a few more basic options, not everything is used by everyone. I could have introduced use options to define different user types, but as a personal project I didn’t want to mess too much with the underlying data structure.

This restriction meant that the majority of my work had to be restricted to the search process, but as this was the area of the application I had the biggest problem with, this wasn’t an issue.

My issues with the existing search functionality could largely be broken down into two categories:

Complexity of the initial search screen and the difficulty of editing a search once it had been executed.
The amount of information returned in search results which returned full descriptions of change requests (which could be quite long) while not providing any other information.

Both categories need to balance the amount of data displayed on-screen with simplicity of the user interface.

Finding balance

Balance between functionality and data is a core tenant of any search application. Hell; finding balance is a core tenant of life.

Following the design tenants outlined above, I split the existing search options into eight core categories, and then organised these categories in order of priority.

These categories are represented on the search screen with the highest priority options displaying on the first line (the number of options automatically adjusts according to screen size). Additional options are hidden behind an expandable section; this approach allows all users immediate access to core search options while advanced users can expand the options list to two rows of settings — a selection which is remembered in subsequent sessions.

Search functions are displayed using pop-ups; highlighting indicates the selected options. Initially users are only able to search for a single product, client or employee, but additional drill-down search options allow searching for multiple values.

Results are split into a collection of tabs based on current status, these tabs provide item counts to provide users with a quick overview of current statistics.

A further set of sub-tabs reveals additional options for advanced users.

The display of search results follows the same minimalistic principles. Results are contained within a traditional grid but columns are merged in order to provide display large amounts of text in a more natural manner.

The provided list of columns are sensitive to the existing context, while additional options are provided to allow users to expand upon these defaults.

A quick-scroll area is provided on screen to allow users to read through the entire description without navigating away from the list of search results.

Additional features allow users to drill down to reveal further information. Tool-tips reveal user contact details while selecting individual records within the search results screen will provide access to further information and full task history.

This approach removes the need to navigate around separate application areas, everything is provided to the user within a single screen but is carefully hidden in order to prevent overloading users with too much information in one go.

There are of course additional screens which expand beyond those demonstrated here, but the design process follows the same basic principles of simplicity.

Conclusion

Grief makes a mess of our thought processes, in my attempt to straighten out some of that mess, I stumbled across similarities with basic design principles which helped me on my way but these principles aren’t really any different from recommended UX practices.

The process of working through grief requires simplification of thoughts, segregating the those that matter from those that don’t, stepping back from complicated tasks and allowing us to approach them when we’re ready.

It’s by no means a process that’s as easy as it sounds. Like good design, it takes time to achieve and there are many false-starts.

My grief has helped me come to the realisation that life needs to be simple, that the applications I write need to at least appear to be simple. Balancing the that simplicity with a hidden complexity is a skill that needs to be practised.

My only hope is that I don’t become as practised in processing grief as I am at designing applications and writing code.

DEV Community: Michael Kennedy

Implementing 2FA under ASP.Net Core

QR Code generation

Initial test

Trigger on login

Applying Visual Themes with SCSS

Clean Styling

Create SCSS Variables and Review Colour Scheme

Theme Generation

Theme Selection

The 5 Rules of ARIA

Don't use ARIA!

Don't replace native semantics

Interactive controls must be keyboard accessible

Beware of role="presentation" and aria-hidden="true"

All interactive elements must have an accessible names

Colour Contrast of Overlapping Text Labels

Reusable MVC Razor Components with HTML Extension Methods

What are HTML Helpers?

Basic Usage

Helper Factory

Html Attributes

Grouping Commands

Insert an Ellipsis into the Middle of a String

Updating the Collation of an Existing MSSQL Database

Preventative Programming

Core Database Collation

Preparation

Target Columns

Incompatible Objects

Update Column Collation

Update Database Collation

Database Reindexing (MSSQL)

Securing User Logins with MVC and JWT

Login Screen

JWT Configuration

Access Token

Refresh Token

Login

Securing Server-Side Functions

Securing AJAX Calls

Making Token Values Available to the Browser

Forcing Session Timeout

Persisting Logout to all Tabs

HTML Email Editors: A Design Case Study

Applications that allow creation of stunning responsive emails are everywhere yet it's rare to find any that appeal to users of all experience levels.

User Profiles and Experience

User Expectation

Existing HTML Email Editors

Features of the ideal HTML editor

Editor Layout

Conclusion

Responsive HTML Email: First Steps

Email Clients

* Except Office 2011 on Mac which used a separately bundled email application.

Email Content

* As noted by Campaign Monitor

Email CSS Styling

Safe CSS Properties

Well Supported CSS Properties

CSS Properties to Avoid

Font Styling

Responsive Email Design

Other Areas of Concern

The Future: Dark Mode

Final Thoughts

Thank you for reading, If you got this far please feel free to leave a comment, alternatively if you were even remotely entertained please hit the like button, or follow me on Twitter.

Grief-inspired Design

Product design doesn’t have anything in common with processing intense grief. Does it? Sometimes we have to be forced into taking a step back in order to see the obvious.

Spaghetti-brained grief

Return to work

Grief-inspired design

Finding balance

Conclusion

Beware of `role="presentation"` and `aria-hidden="true"`