DEV Community: André Santos

Straight A’s in WebPagetest with Umbraco

André Santos — Tue, 27 Nov 2018 21:40:06 +0000

Before launching a new website, there’s a checklist I go through, to make sure that everything is ready. One of the items in my checklist is to test the website against WebPagetest.

WebPagetest is a tool that was originally developed by AOL for use internally and was open-sourced in 2008 under a BSD license. The online version at www.webpagetest.org is run for the benefit of the performance community with several companies and individuals providing the testing infrastructure around the globe.

This tool tests any website against 6 major performance affecting factors, and provides a myriad of graphs and logs that make abundantly clear what might be slowing down your site.

In this post I’ll provide ways to make your site get straight A’s in WebPagetest.

Setup

To start this off, let’s setup our environment. We’ll just need the following:

Visual Studio 2017
Microsoft Azure account

In Visual Studio, let’s create a new empty ASP.NET Web Application project. Then, we’ll need the latest and greatest Umbraco NuGet package (I used version 7.12.4). Once it finishes installing, just launch the website and install Umbraco with all defaults. This will bootstrap Umbraco with the Starter Website, which we’ll use as our “guinea pig” for WebPagetest.

Next: publish it! We can use the publish wizard to automatically create our new WebApp and SQL Database in Azure. Before installing Umbraco in Azure, we’ll need to change the Web.config so that the install wizard is run again (I use Filezilla to change it in Azure):

Clear the Umbraco version number:

<add key="umbracoConfigurationStatus" value="" />

Clear the Umbraco connection string:

<add name="umbracoDbDSN" connectionString="" providerName="System.Data.SqlClient" />

With this changes in place, we’re good to go. This time, we’ll not use the defaults in the Umbraco install wizard, since we’ll want to use the SQL Database we’ve just created in Azure.

First test

For our tests, we’ll use the people page of the Starter Website. This is the score I got with a standard (S0) database and a basic WebApp:

Since this is a very small site, only used for demonstration purposes, half of the metrics are already cleared! However, this is not usually the case for bigger websites. For this reason, I’ll still present some solutions to improve the grade for these metrics.

First Byte Time

This test measures the time it takes for the first byte to reach the client’s browser after the initial http request.

There are two main factors that influence this result:

Server power
The webpage complexity (integrations with external services, complex logic involved, etc)

How to get an A

The easiest way to mitigate this problem is by caching. You can see how to do output caching in Umbraco by reading this old post of mine: https://blogit.create.pt/andresantos/2016/06/30/umbraco-and-donut-output-cache/.

Keep-alive Enabled

Keep alive is a method to allow the same tcp connection for HTTP conversation instead of opening a new one with each new request.

How to get an A

This setting is active by default in IIS, so, it’s also active by default in Azure WebApps, so it’s easy to get an A!

Compress Transfer

Gzip compresses your webpages, style sheets and javascripts, before sending them over to the browser. This drastically reduces transfer time since the files are much smaller.

How to get an A

Just add this setting to your Web.config file:

<httpCompression dynamicCompressionEnableCpuUsage="0" dynamicCompressionDisableCpuUsage="90" noCompressionForHttp10="false" noCompressionForProxies="false">
    <staticTypes>
        <add mimeType="text/*" enabled="true" />
        <add mimeType="message/*" enabled="true" />
        <add mimeType="application/javascript" enabled="true" />
        <add mimeType="application/font-woff" enabled="true" />
        <add mimeType="application/font-woff2" enabled="true" />
        <add mimeType="application/vnd.ms-fontobject" enabled="true" />
        <add mimeType="application/octet-stream" enabled="true" />
        <add mimeType="*/*" enabled="false" />
    </staticTypes>
    <dynamicTypes>
        <add mimeType="text/*" enabled="true" />
        <add mimeType="message/*" enabled="true" />
        <add mimeType="application/javascript" enabled="true" />
        <add mimeType="*/*" enabled="false" />
    </dynamicTypes>
</httpCompression>

Compress Images

Image compression is minimizing the size in bytes of a graphics file without degrading the quality of the image to an unacceptable level.

How to get an A

In Umbraco, to get an A in this grade, you need to do two things:

Crop every image and use srcsets where you can
Use the PostProcessor plugin for Image Processor

Cropping an image is easy in Umbraco:

 <div class="employee-grid\_\_item\_\_image" style="background-image: url('@person.Photo.GetCropUrl(width: 323, height: 300, quality: 85)')"></div>

In order to use the PostProcessor plugin, you just need to install it via nuget: https://www.nuget.org/packages/ImageProcessor.Web.PostProcessor/1.3.1.25.

Cache Static Content

Static content is content that changes rarely. For this reason it can be cached in the user’s browser to avoid downloading the same file over and over again.

How to get an A

Just set the time it takes for the content to expire in the user’s browser and add extra mime types if you want:

<staticContent>
    <clientCache cacheControlMode="UseMaxAge" cacheControlMaxAge="7.24:00:00" />
    <remove fileExtension=".air" />
    <mimeMap fileExtension=".air" mimeType="application/vnd.adobe.air-application-installer-package+zip" />
    <remove fileExtension=".svg" />
    <mimeMap fileExtension=".svg" mimeType="image/svg+xml" />
    <remove fileExtension=".woff" />
    <mimeMap fileExtension=".woff" mimeType="application/x-font-woff" />
    <remove fileExtension=".woff2" />
    <mimeMap fileExtension=".woff2" mimeType="application/x-font-woff2" />
    <remove fileExtension=".less" />
    <mimeMap fileExtension=".less" mimeType="text/css" />
    <remove fileExtension=".mp4" />
    <mimeMap fileExtension=".mp4" mimeType="video/mp4" />
    <remove fileExtension=".json" />
    <mimeMap fileExtension=".json" mimeType="application/json" />
</staticContent>

Effective use of CDN

A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos.

How to get an A

In Umbraco, you can achieve this last grade by doing two things:

Use an Azure Blob Storage for media storage by installing this nuget package: https://github.com/JimBobSquarePants/UmbracoFileSystemProviders.Azure.
Create an Azure CDN for serving this blobs through a content delivery network.

After creating an Azure CDN service and waiting about an hour for it to be available, the following presents my configuration for the media assets to be provided by it:

<?xml version="1.0" encoding="utf-8"?>
<security>
  <services>
    <service name="LocalFileImageService" type="ImageProcessor.Web.Services.LocalFileImageService, ImageProcessor.Web"/>
    <!--Disable the LocalFileImageService and enable this one when using virtual paths. -->
    <service prefix="media/" name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value="media"/>
        <setting key="MaxBytes" value="8194304"/>
        <setting key="Timeout" value="30000"/>
        <setting key="Host" value="https://<umbracositename>.blob.core.windows.net/media"/>
      </settings>
    </service>
  </services>
</security>

<?xml version="1.0" encoding="utf-8"?>
<caching currentCache="AzureBlobCache">
  <caches>
    <cache name="AzureBlobCache" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache, ImageProcessor.Web.Plugins.AzureBlobCache" maxDays="365">
      <settings>
        <setting key="CachedStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net" />
        <setting key="CachedBlobContainer" value="cache" />
        <setting key="UseCachedContainerInUrl" value="false" /> 
        <setting key="SourceStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net" />
        <setting key="SourceBlobContainer" value="media" />
        <setting key="StreamCachedImage" value="false" />
        <setting key="CachedCDNRoot" value="https://<cdnrootname>.azureedge.net" />
        <setting key="CachedCDNTimeout" value="1000" />
      </settings>
    </cache>
  </caches>
</caching>

Conclusion

So, if you followed this tips correctly, you’ll be able to run WebPagetest and get the same result as I did:

You can find the complete report here: https://www.webpagetest.org/result/181127_2A_bea6941dcd20d38ab54c29409fca9363/.

The post Straight A’s in WebPagetest with Umbraco appeared first on Blog IT.

KendoUI Grid, OData, WebAPI and Entity Framework

André Santos — Wed, 05 Jul 2017 10:42:53 +0000

In the latest project I was involved with, there was a need to present database tables data directly in the Umbraco’s backoffice, where all CRUD operations must be supported. This tables could be something like 1000 to 1 million rows and both the website and database would be hosted in Azure, in a WebApp and SQL Database respectively. Users should be able to create complex filters and order the data as they wish as well as update the data individually and in batch.

From the get-go there were some challenges we needed to overcome in order to create an usable system, that was user-friendly, fast, reliable, and where no conflitcts would come up when data was being updated from multiple users.

The solution was a new Umbraco Package we called Search Manager (since the database tables ultimately feed an Azure Search index) that uses Telerik’s Kendo UI Grid, WebAPI and Entity Framework. The following presents the process it took to reach the end result.

Step 1

We created a simple Umbraco Package that implemented a Kendo UI Grid which got its data from a WebAPI method that returned all rows from a table using Petapoco.

This allowed us to have the complete table data in memory using a Kendo UI Grid. It worked great but had major problems:

A table with around 40k rows was actually a 40 MBs json download in order to populate the grid
Users could be working with outdated data, since it was loaded in memory and changes could have been made to the source

Step 2

We started using the server paging feature of the Kendo Grid. This dropped the download size greatly, but we still had problems:

Even though we are working with only a subset of results client-side, we were still getting the complete table data server-side, using petapoco.

We needed an easy way to translate the client-side queries, such as filters, paging, order, etc to SQL queries.

Step 3

We changed the type of queries being generated by the Grid to OData. Using the OData WebAPI nuget package we are able to map the odata query to a C# object and then directly apply the query to any IQueryable data.

Petapoco does not provide any IQueryable data support, so we changed our ORM to Entity Framework, which does, and voilá!

Finally

With everything in place, we are able to have a filtered subset of results in a Grid, that gets exactly what it needs to display from a dynamically generated SQL query.

It is fast because we only get exactly what we need:

An OData query generated by the Kendo UI Grid looks like this:

http://localhost:14231/umbraco/backoffice/SearchManager/PricesApi/Read?%24inlinecount=allpages&%24top=20&%24skip=40&%24orderby=ProviderName+desc&%24filter=(NatureProcCode+eq+%27exames%27+and+AvgAmtClaimed+gt+10)

Which is translated to the follwing SQL query:

exec sp_executesql N'SELECT 
    [Project1].[Approved] AS [Approved], 
    [Project1].[DiffWeightedRate] AS [DiffWeightedRate], 
    [Project1].[SubmitPrvId] AS [SubmitPrvId], 
    [Project1].[PracticeSeq] AS [PracticeSeq], 
    [Project1].[NatureProcCode] AS [NatureProcCode], 
    [Project1].[GroupProcCode] AS [GroupProcCode], 
    [Project1].[Network] AS [Network], 
    [Project1].[AmtClaimedWeighted] AS [AmtClaimedWeighted], 
    [Project1].[ManualPrice] AS [ManualPrice], 
    [Project1].[IsImported] AS [IsImported], 
    [Project1].[DiffCalculatedRate] AS [DiffCalculatedRate], 
    [Project1].[AmtContract] AS [AmtContract], 
    [Project1].[AvgAmtClaimed] AS [AvgAmtClaimed], 
    [Project1].[ProviderName] AS [ProviderName], 
    [Project1].[ClinicName] AS [ClinicName]
    FROM ( SELECT 
        [Extent1].[Approved] AS [Approved], 
        [Extent1].[DiffWeightedRate] AS [DiffWeightedRate], 
        [Extent1].[SubmitPrvId] AS [SubmitPrvId], 
        [Extent1].[PracticeSeq] AS [PracticeSeq], 
        [Extent1].[NatureProcCode] AS [NatureProcCode], 
        [Extent1].[GroupProcCode] AS [GroupProcCode], 
        [Extent1].[Network] AS [Network], 
        [Extent1].[AmtClaimedWeighted] AS [AmtClaimedWeighted], 
        [Extent1].[ManualPrice] AS [ManualPrice], 
        [Extent1].[IsImported] AS [IsImported], 
        [Extent1].[DiffCalculatedRate] AS [DiffCalculatedRate], 
        [Extent1].[AmtContract] AS [AmtContract], 
        [Extent1].[AvgAmtClaimed] AS [AvgAmtClaimed], 
        [Extent1].[ProviderName] AS [ProviderName], 
        [Extent1].[ClinicName] AS [ClinicName]
        FROM (SELECT 
    [PriceGrid].[Approved] AS [Approved], 
    [PriceGrid].[DiffWeightedRate] AS [DiffWeightedRate], 
    [PriceGrid].[SubmitPrvId] AS [SubmitPrvId], 
    [PriceGrid].[PracticeSeq] AS [PracticeSeq], 
    [PriceGrid].[NatureProcCode] AS [NatureProcCode], 
    [PriceGrid].[GroupProcCode] AS [GroupProcCode], 
    [PriceGrid].[Network] AS [Network], 
    [PriceGrid].[AmtClaimedWeighted] AS [AmtClaimedWeighted], 
    [PriceGrid].[ManualPrice] AS [ManualPrice], 
    [PriceGrid].[IsImported] AS [IsImported], 
    [PriceGrid].[DiffCalculatedRate] AS [DiffCalculatedRate], 
    [PriceGrid].[AmtContract] AS [AmtContract], 
    [PriceGrid].[AvgAmtClaimed] AS [AvgAmtClaimed], 
    [PriceGrid].[ProviderName] AS [ProviderName], 
    [PriceGrid].[ClinicName] AS [ClinicName]
    FROM [dbo].[PriceGrid] AS [PriceGrid]) AS [Extent1]
        WHERE ([Extent1].[NatureProcCode] = @p__linq__0) AND ([Extent1].[AvgAmtClaimed] > @p__linq__1)
    )  AS [Project1]
    ORDER BY [Project1].[ProviderName] DESC, [Project1].[AmtClaimedWeighted] ASC, [Project1].[AmtContract] ASC, [Project1].[Approved] ASC, [Project1].[AvgAmtClaimed] ASC, [Project1].[ClinicName] ASC, [Project1].[DiffCalculatedRate] ASC, [Project1].[DiffWeightedRate] ASC, [Project1].[GroupProcCode] ASC, [Project1].[IsImported] ASC, [Project1].[ManualPrice] ASC, [Project1].[NatureProcCode] ASC, [Project1].[Network] ASC, [Project1].[PracticeSeq] ASC, [Project1].[SubmitPrvId] ASC
    OFFSET @p__linq__2 ROWS FETCH NEXT @p__linq__3 ROWS ONLY  option (Recompile)',N'@p__linq__0 varchar(8000),@p__linq__1 decimal(2,0),@p__linq__2 int,@p__linq__3 int',@p__linq__0='exames',@p__linq__1=10,@p__linq__2=40,@p__linq__3=20

It’s user-friendly and reliable because users can filter the data as if they were using an excel file on steroids
It avoids conflicts because we only get a subset of the results each time and each change is commited in a single transaction

Code samples:

grid = $("#grid").kendoGrid({
                dataSource: new kendo.data.DataSource({
                    transport: transport,
                    schema: schema,
                    serverFiltering: true,
                    serverPaging: true,
                    serverSorting: true,
                    type: "odata",
                    pageSize: 20
                }),
                dataBound: onDataBound,
                pageable: true,
                filterable: true,
                sortable: true,
                scrollable: false,
                navigatable: true,
                resizable: true,
                batch: true,
                columnMenu: true,
                editable: true,
                toolbar: [
                    { template: '<a class="k-button k-button-icontext k-grid-save-batch" href="\\#""><span class="k-icon k-i-update"></span>Guardar Alterações</a>' },
                    { name: "cancel", text: "Cancelar" },
                ],
                columns: [...]
            });

[HttpGet]
        public IHttpActionResult Read(ODataQueryOptions<PriceGrid> opts)
        {
            Mapper.CreateMap<PriceGrid, PriceGridBO>();

            using (var context = new EntitiesContext())
            {
                using (var qh = new HintScope(context, QueryHintType.Recompile))
                {
                    List<PriceGrid> results = opts
                        .ApplyTo(context.PriceGrids)
                        .Cast<PriceGrid>()
                        .ToList();

                    List<PriceGridBO> boEntities = results.Select(x => Mapper.Map<PriceGrid, PriceGridBO>(x)).ToList();

                    PriceResponse response = new PriceResponse()
                    {
                        Entities = boEntities,
                        Total = Request.ODataProperties().TotalCount.HasValue ? Request.ODataProperties().TotalCount.Value : 0
                    };

                    return Ok(response);
                }
            }
        }

The post KendoUI Grid, OData, WebAPI and Entity Framework appeared first on Blog IT.