DEV Community: Sitecore

Sitecore Search - Debugging Crawling Errors

Rob Earlam — Tue, 27 Feb 2024 21:45:06 +0000

The latest video in my series on #Sitecore #Search was published this week, discussing how you can debug crawling errors.

Sitecore Search SDK - Breaking Change

Rob Earlam — Fri, 02 Feb 2024 00:00:00 +0000

Updating packages in Sitecore’s Developer Portal

Recently my colleague Mark van Aalst was updating the package versions for the Sitecore Developer Portal. This also included updating the version of the Sitecore Search React SDK from v2.0.0 to v2.2.2, among other packages that were updated at the same time. Once this had been deployed, we got reports that faceting was no longer working on the search results page. We quickly rolled back and started to look into what had happened.

Breaking changes in the Faceting UI component

I reached out to the team that owns the SDK and apparently there was a breaking change introduced in a previous version and we needed to update how we were leveraging the UI components being used to render the Search Results page.

Previously we were wrapping out Facets in the AccordionFacets.Root component, like so

<AccordionFacets.Root defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    ….
</AccordionFacets.Root>

However, this approach is no longer valid, we should now be using SearchResultsAccordionFacets component instead. As follows:

<SearchResultsAccordionFacets defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    …
</SearchResultsAccordionFacets>

This was a fairly straightforward change, which meant we could bump the Developer Portal up to be running on the latest version of the Search SDK 🎉.

To see the complete set of changes we had to make for the Developer Portal, you can view the PR that was opened for this fix here.

Converting a Unicorn project to Sitecore Content Serialization (SCS)

Jason St-Cyr — Wed, 29 Nov 2023 13:01:00 +0000

Today I want to cover an important piece when migrating to XM Cloud: transitioning Unicorn serialization formats to the new Sitecore Content Serialization (SCS) format. In this article, I'll walk you through some of the reasons for the switch, but mostly focus on the tools and process for migrating your Unicorn configurations to SCS.

I want to throw a big shout-out to the Unicorn community that helped me with this article. I haven't used Unicorn before and I had to lean on a lot of smart folks to gather the information together. Thanks to Mark Cassidy, Kamruz Jaman, Michael West, Richard Seal, Justin Laster, Dylan Young, Mihaly Arvai, Martin Miles, and probably others that I forgot!

Why migrate from Unicorn to SCS?

First things first, let's address the "why" behind making this transition. While Unicorn is an excellent choice for serialization on Sitecore XM or Sitecore XP solutions, there are some reasons to make a move to SCS:

Future-Proofing: As Sitecore continues to evolve, it's clear that SCS is the direction that Sitecore is heading in. By transitioning to SCS now, you're ensuring your development workflow is aligned with the Sitecore roadmap.
Migration to XM Cloud: While you have options for serialization tools in self-hosted installations of Sitecore XM/XP, XM Cloud only fully supports SCS flows today. There are ways to keep Unicorn in your flow (which I'll cover next) but XM Cloud is designed for developers using SCS.

But what if I want to stay on Unicorn with XM Cloud?

This is absolutely not recommended by Sitecore, and I wouldn't personally recommend it for any customer use cases, but I did figure out a way to use Items as Resources (IAR) and the Sitecore CLI to support Unicorn serialization files, but still deploy to XM Cloud.

👉 Read about deploying Unicorn to XM Cloud with IAR ▶

Should I stay or should I go?

If you are not planning to migrate to XM Cloud now or in the future, or you are okay with the outlined workaround for using Unicorn with XM Cloud, then Unicorn might still be the right choice for you! The rest of this article assumes that you have already made the choice to convert to SCS and will walk through the steps.

What are the differences between Unicorn and SCS

Unicorn and SCS are not the same tools, so it's important to know the impact of moving from Unicorn to SCS. What will you will gain as extra capabilities? What things will you no longer have after the migration? While much is similar, in the below table are the unique features of each.

Unicorn features lost when moving to XM Cloud	Sitecore Content Serialization (SCS) features you gain
Open Source project strongly supported by the community	Supported by Sitecore Support
Supports database specified at an individual file level	Create Items as Resources (IAR) packages
Real-time transparent syncs (similar to SCS 'watch' but more immediate)	Can connect to multiple environments from same system
	Easily integrate as part of DevOps CI/CD
	Visual Studio plugin available (licensing required)

The Conversion Journey: A step-by-step migration guide

Now that we understand the "why", let's go through a sample plan for migrating your Unicorn configs to SCS. Here's a step-by-step guide to help you make a smooth transition:

Step 1: Evaluate Your Existing Unicorn Project

Before you begin the migration, take a good look at your existing Unicorn project. Familiarize yourself with the folder structure, configurations, and the types of items being serialized. This knowledge will come in handy as you plan your SCS migration.

This is also a good time to identify any irrelevant or inconsistent serialization configurations. Over time, items may have been added into serialization that are either no longer required or should no longer be managed through source control.

Very important: Make sure to check which default Sitecore items are overridden! In some projects, there are custom security settings or other changes applied to areas of the content tree that are normally shipped with the product. These are often overlooked when migrating to SCS.

A caveat about XM Cloud: The content items shipped with XM Cloud are delivered using Items as Resources (IAR). Depending on your sync choices, your configuration could prevent updates from the Sitecore XM Cloud team from being deployed. Content in the database (i.e. your sync'd items) will override those IAR values. Examine your overrides to make sure they are absolutely necessary.

Step 2: Install SCS and Prerequisites

Ensure that your local Sitecore development environment is compatible with Sitecore Content Serialization (SCS) and ready to push to Sitecore XM Cloud. Install the necessary Sitecore modules and SCS-related tools.

Read about the requirements in the docs:

Step 3: Convert Unicorn configs to SCS

There are a few tools that can help take your existing Unicorn configs and start converting them to SCS configuration files. The Unicorn and SCS configurations are quite similar, so the example scripts tend to find your serialization.config files and then convert them to the equivalent JSON module file.

A sample Unicorn config file

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <unicorn>
      <configurations>
        <configuration
          name="Project.HelixBase"
          description="HelixBase site content"
          dependencies="Foundation.*,Feature.*,Project.Common"
          patch:after="configuration[@name='Foundation.Serialization']"
          extends="Helix.Feature.Base">
          <predicate>
            <include name="Project.HelixBase.Templates" database="master" path="/sitecore/templates/Project/HelixBase" />
            <include name="Project.HelixBase.Renderings" database="master" path="/sitecore/layout/renderings/Project/HelixBase" />
            <include name="Project.HelixBase.Layouts" database="master" path="/sitecore/layout/layouts/Project/HelixBase" />
            <include name="Project.HelixBase.PlaceholderSettings" database="master" path="/sitecore/layout/placeholder settings/Project/HelixBase" />
            <include name="Project.HelixBase.Models" database="master" path="/sitecore/layout/models/Project/HelixBase" />
            <include name="Project.HelixBase.Content" database="master" path="/sitecore/content/HelixBase"/>
            <include name="Project.HelixBase.Media" database="master" path="/sitecore/media library/Project/HelixBase">
              <exclude children="true"/>
            </include>

          </predicate>
        </configuration>
      </configurations>
    </unicorn>
  </sitecore>
</configuration>

(Example Serialization.config from Mihaly Arvai: https://gist.github.com/Mitya88/5c9e29b9d04f1eff42700568f5bd8541/)

After converting to SCS

{
    "namespace":  "Project.HelixBase",
    "references":  [
                       "Foundation.*",
                       "Feature.*",
                       "Project.Common"
                   ],
    "items":  {
                  "includes":  [
                                   {
                                       "name":  "Project.HelixBase.Templates",
                                       "path":  "/sitecore/templates/Project/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.Renderings",
                                       "path":  "/sitecore/layout/renderings/Project/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.Layouts",
                                       "path":  "/sitecore/layout/layouts/Project/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.PlaceholderSettings",
                                       "path":  "/sitecore/layout/placeholder settings/Project/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.Models",
                                       "path":  "/sitecore/layout/models/Project/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.Content",
                                       "path":  "/sitecore/content/HelixBase",
                                       "database":  "master"
                                   },
                                   {
                                       "name":  "Project.HelixBase.Media",
                                       "path":  "/sitecore/media library/Project/HelixBase",
                                       "database":  "master",
                                       "scope":  "SingleItem"
                                   }
                               ]
              }
}

(Example module.json from Mihaly Arvai: https://gist.github.com/Mitya88/38b508019016e6998972350d8f3aaedd)

NOTE: The above code samples are directly from the article by Mihaly Arvai (linked below).

Check out these community scripts that you can extend for your needs to convert your Unicorn to SCS!

Step 4: Test and Iterate

Before you declare victory, you need to thoroughly test your new SCS configs to ensure that all serialized content is being correctly deployed and pulled against your local instance. Verify that the items, templates, and configurations work as expected.

This is where you may need to adjust the logic or configurations used to convert your Unicorn configurations if you are not seeing the results you expect.

There are a few tools you can use to start checking your serialization configs:

Run a validation of your config\
The sitecore ser validate command can be used to look for any problems in your config. If any are found, you can then run sitecore ser validate --fix to fix common problems. See the docs on validation.

Get a sync explanation report\
By using the sitecore ser explain command, you can get detailed information on why certain paths are included in the sync. See the docs on the explain subcommand.

Step 5: Update Your Deployment Pipeline

With your content serialization configurations finalized, you will now need to update your deployment pipeline for content. This might involve adjusting your local deployment process, continuous integration pipelines, and ensuring all team members are aware of the new workflow.

Watch this video by Rob Earlam to see how the Sitecore DevRel team converted their CI/CD DevOps flow to work with XM Cloud and the Sitecore CLI:

Step 6: Test Deployments

With your full flow now updated, you will need to test that your updates through CI/CD pipelines are being sent (or ignored) correctly as you make changes to your content and serialize them into your source control repository.

Remember to test these scenarios:

Adding new content that should be deployed
Adding new content that should NOT be deployed
Making a change to content that should override the target version
Making a change to content that should NOT override the target version
Deleting a piece of content from all environments

Wrapping it up!

Migrating from Unicorn to Sitecore Content Serialization can be a manual process, but there are community tools that can help you get there faster! In either case, it requires careful consideration and planning. By following the steps outlined above, you can transition your project to SCS and start working with XM Cloud!

Converting a TDS project to Sitecore Content Serialization (SCS)

Jason St-Cyr — Tue, 28 Nov 2023 13:14:06 +0000

Today I want to dive into an important piece when migrating to XM Cloud: transitioning from a traditional TDS (Team Development for Sitecore) project to the new Sitecore Content Serialization (SCS) format. In this article, I'll walk you through some of the reasons for the switch, but mostly focus on the tools and process for migrating your TDS project to SCS.

Why Migrate from TDS to SCS?

First things first, let's address the "why" behind making this transition. While TDS has been a trusted companion for many developers over the years, there are some reasons to make a move to SCS:

Performance: SCS is designed with performance in mind. It works natively with the Sitecore item provider and is optimized for speed, reducing the time it takes to sync items during development. Ongoing performance improvements are being done with SCS to make this even faster.
Future-Proofing: As Sitecore continues to evolve, it's clear that SCS is the direction that Sitecore is heading in. By transitioning to SCS now, you're ensuring your development workflow is aligned with the Sitecore roadmap.
Migration to XM Cloud: While you have options for serialization tools in self-hosted installations of Sitecore XM/XP, XM Cloud does not support TDS. If you want to migrate to XM Cloud, you should convert to SCS.

If you are not planning to migrate to XM Cloud now or in the future, then TDS might still be the right choice for you. This article assumes that you have already made that choice to convert to SCS and walks through the steps.

The Conversion Journey: a step-by-step migration guide

Now that we understand the "why", let's go through a sample plan for migrating your TDS project to SCS. Here's a step-by-step guide to help you make a smooth transition:

Step 1: Evaluate Your Existing TDS Project

Before you begin the migration, first assess your existing TDS project. Understand the scope of your project, the templates, and items you need to migrate, and any custom configurations that are in place.

Step 2: Install SCS and Prerequisites

Read about the requirements in the docs:

Step 3: Create an SCS-Compatible Serialization Folder

In your project, you will want to create a dedicated folder structure to store serialized content for SCS. SCS uses YAML files for serialization, so this folder will hold YAML files representing your Sitecore items.

Step 4: Convert your TDS project definition to SCS

There are a few tools that can help take your existing definitions and start converting them to SCS configuration files. In general, a TDS project and an SVS configuration have some similar concepts regarding serialization configuration:

TDS	SCS	Description
NeverDeploy	ignored	TDS serialization contains items that should not be deployed but exist to keep hierarchy relations.
SVS ignores the routes instead.
DeployOnce	CreateOnly	This setting ensures a content item is only created if it does not exist. Existing items are not overwritten.
AlwaysUpdate	CreateUpdateAndDelete	This setting ensures a content item will always be changed, regardless of the state in the target system.
NoChildSynchronization	SingleItem	This setting ensures that only the content item specified is updated. None of the items that have it as a parent will be affected.
KeepAllChildrenSynchronized	ItemAndDescendants	This setting forces the item and ALL children and sub-children through the item tree to be affected.
KeepDirectDescendantsSynchronized	ItemAndChildren	This setting will force the update across the item and the immediate children but will not change content items further down the content tree.

I found a few community tools that can parse through the TDS project definitions and generate SCS configuration JSON files by using this mapping.

Get SCS Paths from TDS Project

A PowerShell script by Sitecore MVP Aaron Bickle to generate SCS JSON files. Does not support role migration.

Get the Gist

Sitecore TDS to SCS Serialization Converter

A .NET application by Sitecore Solution Architect Joseph Long that you can configure to parse through TDS projects and generate SCS JSON files. Supports role migration but does not migrate role inheritance.

View GitHub Repo

These tools may not cover every single use case. You will likely need to extend them to your needs. You should also note that these convert the project configurations, but they do not deploy content during this step. You will do this in the next step.

You could also do this manually, but I would recommend using a tool to get started so you have less work to do overall.

Step 5: Test and Iterate

This is where you may need to adjust the logic or configurations used to convert your TDS projects if you are not seeing the results you expect.

There are a few tools you can use to start checking your serialization configs:

Run a validation of your config: The sitecore ser validate command can be used to look for any problems in your config. If any are found, you can then run sitecore ser validate --fix to fix common problems. See the docs on validation.
Get a sync explanation report: By using the sitecore ser explain command, you can get detailed information on why certain paths are included in the sync. See the docs on the explain subcommand.

Step 6: Update Your Deployment Pipeline

With your content serialization configurations finished, you will now need to update your deployment pipeline for content. This might involve adjusting your local deployment process, continuous integration pipelines, and ensuring all team members are aware of the new workflow.

Watch this video by Rob Earlam to see how the Sitecore DevRel team converted their CI/CD DevOps flow to work with XM Cloud and the Sitecore CLI:

Step 7: Test Deployments

Remember to test these scenarios:

Adding new content that should be deployed
Adding new content that should NOT be deployed
Making a change to content that should override the target version
Making a change to content that should NOT override the target version
Deleting a piece of content from all environments

Wrapping it up!

Migrating from TDS to Sitecore Content Serialization (SCS) can be a manual process, but there are tools that can help you get there faster! In either case, it requires careful consideration and planning. By following the steps outlined above, you can transition your project to SCS and start working with XM Cloud!

Deploy Unicorn serialization to XM Cloud using IAR – A proof of concept

Jason St-Cyr — Wed, 15 Nov 2023 13:02:00 +0000

Sitecore has been pretty clear that the only supported option for deploying items to XM Cloud was using Sitecore Content Serialization (SCS) format. But people love using Unicorn and there were enough clues lying around that Unicorn might work? But I couldn't be sure. To make my life as a DevRel professional at Sitecore easier, I was sort of hoping this Proof of Concept with Items as Resources (IAR) wasn't going to work.

But it did.

So I'm writing this article so you can see how it works and also highlight some risks with trying to deploy Unicorn serialization to XM Cloud!

⚠ IMPORTANT⚠
Sitecore does not officially support Unicorn deployment to XM Cloud.
The approach in this article was done to prove something is POSSIBLE, but also explain why you probably still shouldn't do it.
Please?

Jason, a concerned Sitecorian

Why write this article, then?

If this is something you shouldn't be doing, why document how to do it? Well, the fact of the matter is, I didn't know if it could be done or not. And when I talked to other people much more fluent in Unicorn than I, they also didn't know for sure. In fact, I couldn't find a single resource or example from Sitecore or others that showed why Unicorn could NOT be pushed to XM Cloud. There were plenty of statements saying that only the Sitecore CLI and Sitecore Content Serialization (SCS) format were supported. Nobody had figured out why, though.

So I'm going to show you how to make it work, and then explain the risks.

What's the TL;DR?

The challenge: Take an existing Unicorn serialization file and have the item get created in XM Cloud.

The pieces that make it work:

Deploy pipeline in GitHub actions (or your favorite DevOps tool) that invokes the Sitecore CLI
The Sitecore CLI itemres --unicorn command to create an IAR .dat file from Unicorn serialization
A placeholder .dat file in App_Data that is part of the solution being deployed.

The result: It works. (With some things that are not 'clean') Definitely not supported by Sitecore, and could stop working at any time. Be aware of the risks.

Follow along with the GitHub repo

So you can see exactly how this all came together, you can find my fork of the XM Cloud foundation head in my GitHub: https://github.com/jst-cyr/xmcloud-unicorn-deployment

The key files to reference are:

PART 1: Adding the Unicorn serialization files

In my example, I have a single Unicorn serialization file that is used to test out the concept.

Here are the steps I took:

I forked the xmcloud-foundation-head that is used by XM Cloud. This provided me a base solution that is ready to deploy to XM Cloud.
I found a simple serialized file from a 5-year old project that my colleague Dylan Young had in his GitHup repo. This file was a template folder so had limited references to other items and did not have a complex subitems tree I needed to worry about. The purpose was to have an example, but in a real scenario you would have many files across many projects/features that you might have to locate and add to your XM Cloud solution.
I added the serialized file into a folder that I named 'unicorn'. This was for testing purposes and made the later IAR generation step much easier, but you likely would have a more complex setup in a real situation.

NOTE: I did not need to reference it in the project, I only needed to add it to source control.
I updated the Parent ID value and the Path to match with the folder in XM Cloud. In the repository I took the file from, it was also referencing a 'project' folder, but it didn't match with the default one shipped with XM Cloud.

To make my life easier, and avoid needing to ship a new parent folder, I updated it to match with the /sitecore/templates/Project folder that ships with XM Cloud. This actually didn't occur to me originally until my deployment failed and the item didn't get created, so I'm telling you now so you don't make my mistake!

Important note: My use case was purposefully simple. No parent children to deploy, no child items, using an existing Project folder, and only pushing a single item. Your solution will not necessarily be that simple and you may have many files that you have to analyze for conflicting folder paths and invalid ID values.

That being said, whether you want to use Unicorn or convert to SCS, you'll likely need to figure out those updates.

PART 2: Adding a placeholder item resources DAT file

I'll be honest, I didn't do this as my second step when I was first putting this together. This was actually my last step because I had no idea I needed to do this until things didn't work for me.

The goal: This file is going to be replaced in the DevOps pipeline with a real IAR DAT file and will tell XM Cloud to deploy it.

The XM Cloud deployment will only look at the files you have referenced in your project. If you have a file in the right place, but it's not referenced by the project build, it isn't going to be pushed over to XM Cloud.

I chose to do a text file that had the right file name (items.master.unicorn.dat). This is an invalid DAT file and I did this so that if XM Cloud deploy throws an error I would know that the file got there, but my actual IAR file had not. (Basically my equivalent of a "Here!" write to the console log).

PART 3: Adding GitHub Action for triggering deployments on push to the branch

I stole this idea from our XM Cloud Introduction example repo. My colleague Rob Earlam had built out a multisite DevOps pipeline with GitHub actions and it was easy enough to trim it down to just what I needed to push a project to XM Cloud. This was a good starting point.

From what is in that repo, I created .github/workflow/CI-CI_XM_Cloud.yml

name: CI-CD - XM Cloud
on:
  workflow_dispatch:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  deploy:
    uses: ./.github/workflows/deploy_xmCloud.yml
    if: github.repository_owner == 'jst-cyr' && ((github.event.pull_request.head.repo.full_name == github.repository) || (github.event_name == 'push'))
    with:
      environmentName: Unicorn
    secrets:
      XM_CLOUD_CLIENT_ID: ${{ secrets.XM_CLOUD_CLIENT_ID }}
      XM_CLOUD_CLIENT_SECRET: ${{ secrets.XM_CLOUD_CLIENT_SECRET }}
      XM_CLOUD_ENVIRONMENT_ID: ${{ secrets.XM_CLOUD_ENVIRONMENT_ID }}

This file will trigger on PRs and pushes to the repository and run the deploy_xmCloud.yml GitHub action (more on that later). Important pieces:

github.repository_owner == 'jstcyr'

This ensures if somebody forks the repo and does a PR that it doesn't trigger a deployment to my environment.

environmentName: Unicorn

This can be any value, but I use it in deploy_xmCloud.yml to add it to the action name. This is useful in multi-site scenarios where we might deploy multiple sites and we want to see in the GitHub Actions dashboard which site is being pushed.

XM_CLOUD_CLIENTID

The value here is used to authenticate with the Sitecore CLI. Essentially my username. You can get your Client ID from the XM Cloud Deploy App.

XM_CLOUD_CLIENT_SECRET

The value here is used to authenticate with the Sitecore CLI. The password, if you will. You can your client secret when you get your client ID.

XM_CLOUD_ENVIRONMENT_ID

This specifies which environment the deployment should connect to. If you use the console to create your environment, you can easily get the environment ID there. If you already have an environment, you can get it from the environment details page.

I've used GitHub secrets here for sensitive environment/auth information so that I don't have to commit the values to the repository. I strongly suggest you do this for your XM Cloud deploy actions!

PART 4: Building the Deployment GitHub action

With deployments being triggered, we now need the Sitecore CLI actions to actually push our solution to the environment. I again stole this from our XM Cloud Introduction repo and have a very simple job:

name: Deploy the solution and items to an XM Cloud instance

on:
  workflow_call:
    inputs:
      environmentName:
        required: true
        type: string
    secrets:
      XM_CLOUD_CLIENT_ID:
        required: true
      XM_CLOUD_CLIENT_SECRET:
        required: true
      XM_CLOUD_ENVIRONMENT_ID:
        required: true

jobs:
  deploy:
    name: Deploy the XM Cloud ${{ inputs.environmentName }} Site
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-dotnet@v2
        with:
          dotnet-version: "6.0.x"
      - run: dotnet tool restore
      - run: dotnet sitecore --help
      - name: Authenticate CLI with XM Cloud
        run: dotnet sitecore cloud login --client-credentials --client-id ${{ secrets.XM_CLOUD_CLIENT_ID }} --client-secret ${{ secrets.XM_CLOUD_CLIENT_SECRET }} --allow-write
      - name: Deploy the CM assets to XM Cloud
        run: dotnet sitecore cloud deployment create --environment-id ${{ secrets.XM_CLOUD_ENVIRONMENT_ID }} --upload

In this deploy job, it does the following steps:

Ensure the Sitecore CLI is installed and ready
Authenticate against XM Cloud using the client ID and client secret that were passed in
Trigger a new deployment to the specified environment

At this point, when I commit code, it detects the change and pushes my project to XM Cloud.

PART 5: Generating the Unicorn IAR file

One small line of code can be added to that deployment job to make the magic happen now:

run: dotnet sitecore itemres unicorn --output src/platform/App_Data/items/master/unicorn --path "src/unicorn" --overwrite

By running this command before we trigger the deployment, we can replace the placeholder DAT file with our generated one. The commands and parameters work as follows:

itemres: This command generates an Item as Resources (IAR) package
unicorn: This subcommand tells it to expect Unicorn serialization format
--output: This specifies the path where we want the IAR file to be placed after it is created. We want this to match to the source path of where we put our placeholder DAT file. You might notice I don't specify the file name as it automatically uses the folder path in App_Data to create the file name. (items/master/unicorn becomes items.master.unicorn.dat)
--path: This tells the command where in the file system to find the files to process into the DAT file.

PART 6: Validating the deployment

Now we need to check that everything went over. First we check to see that the file showed up in our deployment logs on XM Cloud.

From the 'Build' section of the deployment we see:

This tells me that a DAT file with the right name got there, but it doesn't tell me if it was processed.

From the 'CM instance logs' Log file:

Since there was no error here, that means the file is not the invalid text file that was in my project, and is a valid IAR file that contains 1 item. However, that does NOT mean it successfully was imported (I learned that the hard way).

For instance, if the IAR is a valid format, this part will show correctly. However, I originally had a parent that didn't exist in the database. So no errors occurred, but the item wasn't pushed. So I need to check in Content Editor.

From the Content Editor UI:

\
I can see here that there is a Common folder in the Project templates folder, and that the Item ID matches the item I wanted to create that was in my Unicorn serialization.

SUCCESS!

Wait... it worked?

What I had hoped to learn, when doing this investigation, was that the Unicorn-based IAR files could not be uploaded to XM Cloud and there was no way to get Unicorn serialization format to work. That would make my life a lot easier with the statement that Sitecore does not support Unicorn serialization with XM Cloud because it's a lot easier to say something isn't supported when it also just plain doesn't work. So life for me will be a little more complicated and nuanced!

In doing this proof of concept, I did identify some risks with this solution.

⚠ Risk #1: No support
It works, but the way to get this to work is not supported by Sitecore. That means there is no way to get help from Sitecore about using this type of a flow if something goes wrong and no obligation from Sitecore to ensure it keeps working.

⚠ Risk #2: Brittle dependency
My approach relies on a mechanism of forcing files into a specific folder structure on the environment using the project file. That happens to work with XM Cloud's deploy app at the time of writing, but because this is SaaS the entire backend of how deployments work could shift on me and this could get broken. Also, because it's not supported, there would likely be no notifications sent out that it would no longer be working. You can always fix things later, when it breaks, I suppose? It's one of those "buyer beware" risks that some folks are willing to take on.

⚠ Risk #3: Unknown serialization conversion complexity
My sample scenario used a single template folder with a simple parent relationship. I used out of the box parent folders and templates with no dependencies. Even then, I needed to edit a few things manually on the file to get it to work with XM Cloud's default items. I can imagine a large project of serialized items could become cost-prohibitive to manually get working.

This is not really an issue with using Sitecore CLI, Unicorn, or the IAR approach, but more to point out what I got working here didn't really analyze the effort needed to migrate the serialization files to work with XM Cloud. You should factor that into your plans as an unknown risk factor to your timeline.

⚠ Risk #4: The steps provided might take more effort for a real project
The itemres command I used in the Sitecore CLI was pointed at a single folder of Unicorn serialized files. A real solution would have a Helix structure with many different locations that the files could be contained in for each feature. I did not check for how much work it would be to get that more complex structure pulled together. It's still technically possible to get it to work, but you should assume there will be more effort than shown here to get all of your serialized projects configured, working, and deployed through your DevOps pipeline.

⚠ Risk #5: Unclean project structure
In order to tell XM Cloud to grab that IAR file, I needed to add placeholder files to my .NET solution. The whole approach feels wrong, adding a bunch of invalid DAT files into the project (and source control) just to make deployments work. I suppose you could manage a different project file just for the placeholder DATs, just to make it cleaner for devs who are working locally, but it screams of bad practice. This risk doesn't affect your effort levels or whether the solution would work, but it does impact your developer experience.

Conclusion? Maybe not a good idea?

If you are going to work with XM Cloud, and you love Unicorn, you might be tempted to try to figure out a way to make it work. I did this experiment so that you can see what the challenges are, what steps you can take, and make an informed decision. I feel that converting your Unicorn serialization over to SCS is a much easier approach and there are tools for that already. I would definitely recommend taking that migration to SCS approach if you are building against XM Cloud, but if you really love Unicorn and are okay with the risks involved at least there is a way to make it work!

Sitecore XM Cloud and Vercel Deployment Protection

Jason St-Cyr — Mon, 06 Nov 2023 15:04:34 +0000

Recently, Vercel made Deployment Protection the default for all new projects in Vercel. This is great news! This means that all your deployments will automatically be able to be secured with authentication, passwords, or trusted IPs.

However, your Sitecore XM Cloud build might not be set up yet to work with Vercel's Deployment Protection. You may encounter a 401 authentication error (unauthorized) when accessing static assets for things like CSS where your Next.js application is trying to use the VERCEL_URL to access these assets.

Updating your Next.js application to use deployment protection

If you are planning on launching a new project on XM Cloud, you'll likely want to take a look at these docs so that you can make sure JSS app is loading these protected assets correctly:

Use Vercel's cloud platform features to limit access to your site deployments

But what if I don't want this Vercel feature?

You might have scenarios where you'd like to have overall security but in certain use cases you just want to bypass it altogether. Perhaps you are running End To End (E2E) testing, or maybe you have a particular preview environment you are sharing around to others for review. Vercel has a few steps you can take to bypass the deployment protection: https://vercel.com/docs/security/deployment-protection/methods-to-bypass-deployment-protection

You can also turn off the protections completely in the Project Settings ▶ Deployment Protection tab of your Vercel project.

]

SUGCON India 2023 goes off with a bang 💥

Rob Earlam — Thu, 31 Aug 2023 00:00:00 +0000

Last week I had the pleasure of attending SUGCON India 2023 in Delhi, India. This was the first time we’d run a SUGCON in India since 2019 and it was great to be back once more, interacting with the local Sitecore community. There was a lot of effort in involved in pulling the event off, and I do want to start by thanking all of the organisers for the time they invested in making this event the success that it was.

I was privileged to be chosen to deliver one of the keynote sessions this year, where I talked about a recent project I worked on where we integrated Sitecore Search into our Developer Portal. This was very well received and led to many questions afterwards, and many further conversations throughout the event. It was great to see the amount of interest there is in Sitecore Search from the attendees!

The quality of the sessions that were delivered at the event was of a very high standard. They ranged from high-level sessions covering Composable DXP’s in general and the power they can bring to customers, all the way down to technical deep dives on specific features and use-cases. Some of my highlights were:

The session by Pieter Brinkman titled Digital Experience is a team sport, where he talked about the different roles that play a part in successful Composable DXP implementation – it’s not just all about the developer!
George Smith & Sheetal Jain delivered an awesome session called Beyond Buzzwords: Composable Tech and Generative AI for Real. Here they showed how OrderCloud, Sitecore Content Hub & Dall-E can be combined to allow customers to generate art on the fly and have it printed on a t-shirt for sale in an online store, cool stuff!
Exploring the Experience Edge GraphQL Schema was a really interesting session delivered by Alistair Deneys. Here he gave a true deep-dive into the schema implemented for Edge and how you can leverage it in some really interesting ways. There’s also a cool surprise at the end of the session that I won’t give away here, but be sure to check it out if you get the chance.
Women in contemplating Darwin's Theory - To Break the Conventional Wheel was a great session by Varalakshmi M D & Raveena Mathur where they talked about the important role women have in ensuring successful outcomes for customers.
Andy Cohen delivered a great session titled Build for XM Cloud according to the Architect of XM Cloud. Here he talked about the best practices for building with XM Cloud... he might have ruffled some feathers with some of the recommendations, but it was great to see it all layed out!

The closing keynote was also a highly enjoyable part of the event. Each of the representatives from Sitecore (Pieter, Nicole, Andy, Alistair and myself) held a panel discussion and open Q&A session. This gave the attendees the chance to ask any lingering questions that they hadn’t had the chance to ask already. The topics were very wide from specific feature questions on the products through to general questions about the best way that Sitecore can support the local Indian community… and of course the age-old question of Sandboxes and how people can get access to Sitecore’s newer SaaS products – we hear you and are working on a solution!

As much as I enjoyed everything I mentioned above, as with every event I seem to attend nowadays, the true highlight was getting to interact with the local community we have on the ground. It’s amazing to see the passion they have for Sitecore and our products. It was great to see so many familiar faces from the events back in 2018 & 2019, but it was also great to see so many new faces as well, a good balance that shows how healthy the Sitecore community in India is.

I just want to finish off by saying that of course trips like this aren’t always about work though, we always try to make sure that we get out and see some of the places we visit and India is no different. On this trip, we made the journey down to see the Taj Mahal after the event had finished, and it truly is an amazing site to see. Really all of the descriptions of it you hear just cannot do it justice, and it was great to experience it with the awesome team we had on-site.

So, whether it’s the always friendly people, the amazing effort from the organisers & speakers, the food, the noise or the general sights you see around the city – India is one of my favourite places to visit and certainly one of my favourite SUGCON events.

Here’s looking forward to doing it all again in 2024!

Migrating the XM Cloud Introduction Repo to a new Nuget feed.

Rob Earlam — Thu, 27 Jul 2023 00:00:00 +0000

Today I had to go through the process of migrating the XM Cloud Introduction repository to a new Nuget feed. This was required as the old feed was no longer available, and we needed to move to a new one on short notice. This post will cover the steps I needed to complete to migrate the solution to the new feed.

Machine Tidy Up

Now for anyone who hasn't run a Sitecore site before you can most likely skip this step. However, for anyone who has you should tidy up any references to the old feed that have been configured on your machine. In my case, this was the PowerShell Repository used to restore the SitecoreDockerTools module used by this, and other Sitecore repositories.

Remove PS Repository

The first thing to check is whether you have the old Sitecore Feed registered as a PowerShell Repository, you can do this by running the following command Get-PSRepository and checking where there is a record named SitecoreGallery. If you have one registered, then you can remove it by running Unregister-PSRepository -Name SitecoreGallery.

Remove PS Modules

Next, I tidied up any previous versions of the SitecoreDockerTools module installed from the old feed. You can test whether you have any versions of that module installed by running the Get-Module -ListAvailable and look for any named SitecoreDockerTools. If you have those installed, you can quickly remove all installed versions of that package by running Uninstall-Module -Name SitecoreDockerTools -AllVersions.

Update the Repository

So now we’ve tidied our developer machine, we can update the repository to reference the new feed. This involved making two changes, an update to the init.ps1 script used to initialise the repository, and then an update to the nuget.config to define the new location for the NuGet packages in use by the solution.

Updating the init.ps1 script

The first thing the script does is install the SitecoreDockerTools PowerShell module, so we need to update the location that it is installed from, in our case, we needed to use v2 of the Nuget feed to get it to work, so we needed to change it to https://nuget.sitecore.com/resources/v2/. After making this change the init script ran successfully and installed the module from the new feed.

Updating the nuget.config

The final change we need to make is to update the nuget.config file in the root of the repository that controls where custom NuGet packages are installed from. We had a couple of older references in there that we removed and condensed down to the single new feed reference. This resulted in us having the following line in our config:

<add key="Sitecore" value="https://nuget.sitecore.com/resources/v3/index.json" />

Conclusion

Updating the XM Cloud Introduction repository to the new feed location was straight forward in the end. I would recommend any developers who have been running existing Sitecore solutions before follow the tidy-up steps at the start to ensure they remove any legacy references they have to the old feed. You can see the changes we had to make in the following commit. There may be other changes that you will need to complete to bring your codebase in line, but hopefully, this will help to get you started with what tasks are required for you.

MyGet outage impacts builds and deployments

Jason St-Cyr — Wed, 26 Jul 2023 17:29:28 +0000

Many development teams are seeing NuGet build issues and errors today with the MyGet outage. You can resolve these issues by transitioning to our new NuGet package provider.

Learn more about how to transition to our new #Sitecore package provider in this support article: https://support.sitecore.com/kb?id=kb_article_view&sysparm_article=KB1002999

Sitecore Search - Filtering vs Faceting

Rob Earlam — Tue, 25 Jul 2023 23:28:45 +0000

The latest video in my series on #Sitecore #Search was published this week, discussing Filtering vs Faceting.

Sitecore Search – Configuring your Domain

Rob Earlam — Mon, 10 Jul 2023 00:51:26 +0000

The second video in my series on #Sitecore #Search was published last week, showing how you can do the initial domain configuration

Sitecore Architect’s Guide to SaaS Migration – Classic XP with Simple Personalization

Jason St-Cyr — Wed, 31 May 2023 11:55:00 +0000

In this part of the migration series, I am going to look at migrating an existing Sitecore Experience Platform (XP) solution that has been built as a classic Sitecore MVC implementation with simple personalization implemented.

The scenario breakdown:

Single MVC site built on Sitecore XP 8.2
Basic session-based personalization in use, primarily using Geo IP to determine user location and campaign IDs in the URL
No SXA or Sitecore JSS in use
No custom site search, using third party search engine for site search
Development team is all .NET developers, no desire to move to a JavaScript framework

Follow the series to look at different Sitecore XM and XP scenarios and how you can gradually migrate your Sitecore platform solution over to a composable DXP architecture.

The goal is to walk through the example migration and highlight for you:

What challenges will you face along the way?
What options are available and when do they make sense?
What are the benefits of making certain changes to your architecture?

Every project is a little different, so hopefully this series can help you understand some of the questions to ask, and what options you have, to guide you in the right direction.

Continue reading "Sitecore Architect’s Guide to SaaS Migration – Classic XP with Simple Personalization"