Using Azure WebJobs with Sitecore XM/XP

Introduction

Sitecore offers Out Of The Box (OOTB) scheduled jobs, allowing developers to automate some of the recurring tasks, such as maintenance of, say, a scheduled batch import process, for example, the one described in my Sitecore “Batch” Connector, part 2: Import Content into Sitecore XM/XP post.

All posts in this `Sitecore “Batch” Connector` series

There are multiple ways to run scheduled jobs in Sitecore, of the most commonly used ones are the scheduled tasks and PowerShell-driven. In most cases, these were my go-to choices, but they run in the context of the Sitecore instance, which may or may not be a good thing.

I needed to run a scheduled task, based on some .NET custom code alongside Sitecore, in the same Azure Web App instance. My code happened to have many dependencies, which conflicted with Sitecore's own, so running my code in the same process turned out to be troublesome and risky.

While it is possible to achieve some level of separation with DLL redirects or, say, run my own code in a separate Azure App or an Azure Function, I decided to leverage existing already existing CM instance and run my scheduled job as an Azure Web Job, which then proved to work well and reliably.

In essence, while Sitecore's OOTB scheduled jobs serve the automation purpose for Sitecore instances hosted on Azure Web Apps, when regular Sitecore Jobs may not be the best fit.

This post describes the steps I have taken to get it to work.

Azure WebJobs

Azure Web Jobs is a feature of Azure App Service that enables the running of programs or scripts in your app service plan. These jobs can run continuously, on a schedule, or on demand, providing a higher degree of flexibility. Moreover, Azure Web Jobs supports a variety of script types including .exe, .bat, .cmd, .sh, .php, .py, .js, and more, widening the horizon of automation capabilities.

Implementing Azure Web Jobs in a Sitecore Instance on Azure Web Apps: To harness the benefits of Azure Web Jobs in a Sitecore instance running on Azure Web Apps, follow these steps:

Create Azure Web Job project in Visual Studio

In Visual Studio, create a new project.
Select the ‘Azure WebJob (.NET Framework)’ template.
Name your project and click OK.
Use the following template for your Program.cs, which would be the same as with regular console app
Implement your custom logic

using Newtonsoft.Json;
using Serilog;
using Serilog.Events;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;

namespace GVI.ContentHub.Sync.WebJob
{
	// To learn more about Microsoft Azure WebJobs SDK, please see https://go.microsoft.com/fwlink/?LinkID=320976
	internal class Program
	{
		static void Main()
		{
			try
			{
			///...your implementation here
			}
			catch (Exception ex)
			{
				Log.Error(ex, $"Error running {nameof(WebJob)}");
				//throw;
			}
			finally
			{
				Log.CloseAndFlush();
			}
		}
	}
}

Sitecore integration

The WebJob implementation has no dependencies on the Sitecore CM, running in the same Azure Web App instance, which is why I chose it in the first place, but I needed to save the results of this processing job in a folder, from which Sitecore read and process them, going forward. The good news is, that my Azure Web Job is meant to be running on the same instance as Sitecore CM, so, even though Sitecore CM and WebJob would in separate processes, the local filesystem is shared, so one (my web job) can save files to be processed by another.

The following code snippet saves the content from entitiesBatchobject into a JSON file into incomingFolderPath, which is a folder under Sitecore’s App_Data folder, so it can be picked up and processed by a PowerShell script scheduled task later on:

See the [TODO] post for more details on that Sitecore PowerShell script.

string strNum = i.ToString("D3");
var fileName = $"{entityConfiguration?.EntityDefinition}_{startTime.ToString("yyyyMMddTHHmmss")}_{strNum}.json";

var filePath = Path.Combine(incomingFolderPath, fileName);
var json = JsonConvert.SerializeObject(entitiesBatch);

File.WriteAllText(filePath, json);
Log.Information($"Saved {entitiesToSave} entities of type {entityConfiguration.EntityDefinition} entities to {filePath}");

Develop and test the web job locally

The web job is not all the different from a console app, in fact, a console app can be easily packaged and deployed as an Azure WebJob. So, implementing and debugging the web jobs locally is not all that different from the process of building and debugging console apps in .NET.

This article is a good walkthrough on developing web jobs. You don't have to create and use the storage account in Azure... unless you need it. The above code snippet can serve as a good starting point for your implementation.

Publish the WebJob to Azure

Publish from Visual Studio
- Right-click on the project in Solution Explorer.
- Select ‘Publish as Azure WebJob…’
- Follow the prompts to publish the web job to your Azure account.
Upload zipped Web Job to Web App instance in Azure portal

Scheduling the WebJob

In the Azure portal, navigate to your Web App.
Under Settings, find the ‘WebJobs’ section.
Here you can add a schedule for your WebJob, or run it on demand. Here's an example of a cron schedule for running a web job every 5 minutes.

{
  "schedule": "0 */5 * * * *"
}

Refer to Microsoft documentation for more details on scheduling of the web jobs, or cheat sheets like this one for quick answers.

Monitoring the WebJob

Azure provides a WebJobs dashboard for monitoring the execution of webjobs in Azure portal. You can access logs, view job history, and troubleshoot issues right from the dashboard.

Content Hub

Sitecore

Integrations

Sergey Yatsenko