CH Upstream Integrations part 3: Example Logic App for Syncing an External System with Sitecore

CH Downstream Integrations part 1: Example Azure Functions to Read Content Hub Entities

Intro

A collection of code snippets and Azure Functions for Content Hub Downstream Integrations

Read Sergey's blog posts at xcentium.com

Introduction: Using Azure Logic Apps in Integration Flows

The following is a bunch of code snippets, which I wrote for my Azure Functions, intended to search and read content from Sitecore Content Hub. I thought this might be useful as a collection of code snippets for later reuse in Content Hub integration projects. Feel free to copy-paste as needed :)

This post is the 1st part in a 3-piece series, describing an integration approach that allows connecting Content Hub with pretty much any external system via APIs. In this case, Content Hub is the source of truth, from which content gets published to other systems, so this is an example of downstream integration.

I'm leveraging Azure Functions as integration building blocks and Azure Logic Apps to compose them together into integration flows, where data from the external system would be extracted, transformed, or processed and pushed to Content Hub via its APIs. I chose to use Content Hub Web Client SDK, which is a .NET abstraction on top of the Sitecore CH REST API because it helps to deal with CH API throttling, and does a few more helpful things.

I believe Logic Apps is a good way to visually orchestrate various building blocks (Azure Functions) together with very little to no code required: easy to build, and easy to change, but of course, this isn't the only way. I'm sharing all source code here, so others can use it for building the custom integration solutions for Sitecore Content Hub.

All posts in this collection:

• Part 1 (this post). Example Azure Functions: Describes example functions for reading content entities and their IDs from Content Hub
• Part 2. Solr Indexer Logic App: Describes example Logic App to publish content changes from Sitecore Content Hub to an external system. In this case, I am extracting, transforming, and then saving Content Hub content into Solr, so this Logic App is effectively a Solr Indexer.
• Part 3. A Content Crawler for Full Index Rebuild: Another Logic App, which reads IDs of entities in Content Hub and pushes them to an Azure Service Bus Queue, so they would be picked up and processed by the Logic App from part 2.

It's worth noting Sitecore that recently announced Sitecore Connect along with other new great products, so consider using Sitecore Connect before implementing your custom solution.

Useful Information

Relations in Sitecore Content Hub

Querying and Reading Content Hub Data with Web SDK Client

Useful Code Snippets

Initialize WebClient

given request headers/// <summary>
/// Initialize WebClient with credentials provided in a given request headers.
/// </summary>
/// <param name="request"></param>
/// <returns>WebClient</returns>
public static IWebMClient InitClient(HttpRequestMessage request)
{
    var clientInfo = Utils.ExtractClientInfo(request.Headers);
    Uri endpoint = new Uri(clientInfo.baseUrl);
    OAuthPasswordGrant oauth = new OAuthPasswordGrant
    {
        ClientId = clientInfo.clientId,
        ClientSecret = clientInfo.clientSecret,
        UserName = clientInfo.userName,
        Password = clientInfo.password
    };

    IWebMClient client = MClientFactory.CreateMClient(endpoint, oauth);
    return client;
}

Get Entity by ID

/// <summary>
/// Get Content Hub Entity by ID
/// </summary>
/// <param name="client"></param>
/// <param name="entityId">ID of the Entity</param>
/// <param name="loadConfiguration">Entity load configuration, contolling how much data to load from CH.</param>
/// <param name="log">optional logger</param>
/// <returns>Entity or null if not found.</returns>
public static async Task<IEntity> GetEntity(IWebMClient client, long entityId, IEntityLoadConfiguration loadConfiguration, TraceWriter log = null)
{
    try
    {
        //Read Entitty and load its fields and relations as per specified LoadConfiguration
        var entity = await client.Entities.GetAsync(entityId, EntityLoadConfiguration.Full);

        if (entity != null)
        {
            //Log and return the results
            log?.Info($"Found entitity. ID: {entity.Id}", "GetEntity");
            return entity;
        }

        return null;
    }
    catch (Exception ex)
    {
        //Log and re-throw the exception
        log?.Error($"error message: {ex.Message}", ex, "GetEntity");
        throw ex;
    }
}

Extract Entity fields, including its public links

/// <summary>
/// Read entity properties into name-value pairs
/// </summary>
/// <param name="entity">Source Entity</param>
/// <returns>name-value pairs representing all entity properties</returns>
public static Dictionary<string, object> ExtractEntityData(IEntity entity)
{
    var e = new Dictionary<string, object>
    {
        { "Id", entity.Id },
        { "Identifier", entity.Identifier },
        { "DefinitionName", entity.DefinitionName },
        { "CreatedBy", entity.CreatedBy },
        { "CreatedOn", entity.CreatedOn },
        { "IsDirty", entity.IsDirty },
        { "IsNew", entity.IsNew },
        { "IsRootTaxonomyItem", entity.IsRootTaxonomyItem },
        { "IsPathRoot", entity.IsPathRoot },
        { "IsSystemOwned", entity.IsSystemOwned },
        { "Version", entity.Version },
        { "Cultures", entity.Cultures }
    };

    var relativeUrl = entity.GetPropertyValue<string>("RelativeUrl");
    var versionHash = entity.GetPropertyValue<string>("VersionHash");

    // Construct public link Urls if Entity happends to be an Asset and have public links set on it
    if (!string.IsNullOrEmpty(relativeUrl) && !string.IsNullOrEmpty(versionHash))
    {
        var publicLink = $"api/public/content/{relativeUrl}?v={versionHash}";
        e.Add("PublicLink", publicLink);
    }

    foreach (var property in entity.Properties)
    {
        try
        {
            var propertyValue = entity.GetPropertyValue(property.Name);
            e.Add(property.Name, propertyValue);
        }
        catch (Exception ex) when (ex.Message == "Culture is required for culture sensitive properties.")
        {
            var propertyValue = entity.GetPropertyValue(property.Name, CultureInfo.GetCultureInfo("en-US"));
            e.Add(property.Name, propertyValue);
        }
    }

    return e;
}

Search Entities using Linq Query

/// <summary>
/// Search and load all entities matching the search query (function)
/// </summary>
/// <param name="client">Content Hub Web SDK client</param>
/// <param name="queryFunction"></param>
/// <param name="log">optional logger - ignore if no logging is needed</param>
/// <returns>List of fully loaded entities matching the search criteria. Warning: be careful - this could grow huge</returns>
public static async Task<IList<IEntity>> SearcEntities(IWebMClient client, Func<QueryableEntities<IQueryableEntity>, IQueryable<IQueryableEntity>> queryFunction, IEntityLoadConfiguration loadConfiguration, TraceWriter log = null)
{
log?.Info($"Search query function: {queryFunction}");
//Initialize the query with given search criteria (function)
Query query = Query.CreateQuery(queryFunction);
try
{
var results = new List<IEntity>();
//Deal with paging when there are more than 50 results (default page size)
IEntityIterator iterator = client.Querying.CreateEntityIterator(query, loadConfiguration);
while (await iterator.MoveNextAsync())
{
var entities = iterator.Current.Items;
if (entities != null && entities.Any())
{
results.AddRange(entities);
}
}

if (results != null && results.Any())
{
//Log and return the results
log?.Info($"Found entities. Count: {results.Count}, IDs: {String.Join(",", results.Select(e => e.Id))}", "SearcEntitiesByFieldFalue");
return results;
}

return null;
}
catch (Exception ex)
{
//Log and re-throw the exception
log?.Error($"error message: {ex.Message}", ex, "SearcEntities");
throw;
}

}

Example Azure Functions to Read Content from Sitecore Content Hub

1. Read Entity and its Relations from Content Hub

//#r "Newtonsoft.Json"

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using Stylelabs.M.Framework.Essentials.LoadConfigurations;
using Stylelabs.M.Sdk.WebClient;
using SY.ContentHub.AzureFunctions.Models;
using System;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;

namespace SY.ContentHub.AzureFunctions
{
	/// <summary>
	/// Get Entity by ID and then if found, load entities from specified relations 
	/// </summary>
	public static partial class GetEntityById
	{
		[FunctionName("GetEntityById")]
		public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestMessage req, TraceWriter log)
		{
			//Read and parse request payload
			log.Info("GetEntityById invoked.", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var content = req.Content;
			string requestBody = content.ReadAsStringAsync().Result;
			log?.Info($"Request body: {requestBody}", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var requestObject = JsonConvert.DeserializeObject<GetEntityByIdRequest>(requestBody);

			try
			{
				//Initialize WebClient 
				IWebMClient client = Utils.InitClient(req);
				//Get Entity by ID and load all its properties and relations
				var entity = await Utils.GetEntity(client, requestObject.entityId, EntityLoadConfiguration.Full, log);

				if (entity == null)
				{
					return req.CreateResponse(HttpStatusCode.NotFound, $"Entity with Id {requestObject.entityId} not found.");
				}

				//Load all related entities, linked to specified relation fields
				var relations = await Utils.GetRelatedEntities(client, entity, requestObject.relations, log);

				//Create response object
				var result = new EntityInfo
				{
					//Read entity properties into name-value pairs
					Entity = Utils.ExtractEntityData(entity),
					Relations = relations
				};

				//Serialize response object into JSON and return that as HTTP response
				var resultJson = JsonConvert.SerializeObject(result);
				return new HttpResponseMessage(HttpStatusCode.OK)
				{
					Content = new StringContent(resultJson, Encoding.UTF8, "application/json")
				};

			}
			catch (ArgumentException argEx)
			{
				var message = $"Invalid request body or missing required parameters. Error message: {argEx.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.BadRequest)
				{
					Content = new StringContent($"Invalid request body or missing required parameters. Error message: {argEx.Message}")
				};
			}
			catch (Exception ex)
			{
				var message = $"'Error message':{ex.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.InternalServerError)
				{
					Content = new StringContent($"'Error message':{ex.Message}")
				};
			}
		}
	}
}

2. Search entities by Field Name and Entity Definition

//#r "Newtonsoft.Json"

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using Stylelabs.M.Base.Querying.Linq;
using Stylelabs.M.Framework.Essentials.LoadConfigurations;
using Stylelabs.M.Sdk.Contracts.Base;
using Stylelabs.M.Sdk.WebClient;
using SY.ContentHub.AzureFunctions.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;

namespace SY.ContentHub.AzureFunctions
{
	/// <summary>
	/// Find Entities by field value and definition name
	/// Optionally include related Entities using specified relations.
	/// </summary>
	public static partial class SearchEntitiesByFieldValue
	{
		[FunctionName("SearchEntitiesByFieldValue")]
		public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestMessage req, TraceWriter log)
		{
			//Read and parse request payload
			log.Info("SearchEntitiesByFieldValue invoked.", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var content = req.Content;
			string requestBody = content.ReadAsStringAsync().Result;
			log?.Info($"Request body: {requestBody}", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var requestObject = JsonConvert.DeserializeObject<SearchEntitiesByFieldValueRequest>(requestBody);

			try
			{
				//Initialize CH Web SDK client
				IWebMClient client = Utils.InitClient(req);

				//Search for Entities matching the requested field value
				IList<IEntity> entities;

				if (string.IsNullOrEmpty(requestObject.entitySearch.entitySearchField.fieldValue) || string.IsNullOrEmpty(requestObject.entitySearch.entitySearchField.fieldName))
				{
					entities = await Utils.SearcEntities(client,
											(entities =>
											from e in entities
											where e.DefinitionName == requestObject.entitySearch.entitySearchField.definitionName
											select e),
											EntityLoadConfiguration.Full, log);
				}
				else
				{
					entities = await Utils.SearcEntities(client,
											(entities =>
											from e in entities
											where e.Property(requestObject.entitySearch.entitySearchField.fieldName) == requestObject.entitySearch.entitySearchField.fieldValue
											where e.DefinitionName == requestObject.entitySearch.entitySearchField.definitionName
											select e),
											EntityLoadConfiguration.Full, log);
				}


				if (entities == null || entities.Count == 0)
				{
					return req.CreateResponse(HttpStatusCode.NotFound, $"No entities found for field {requestObject.entitySearch.entitySearchField.fieldName} with value {requestObject.entitySearch.entitySearchField.fieldValue}");
				}


				var response = new List<EntityInfo>(); 
				foreach (var entityObject in entities)
				{
					var relations = await Utils.GetRelatedEntities(client, entityObject, requestObject.relations, log);

					var entity = new EntityInfo
					{
						Entity = Utils.ExtractEntityData(entityObject),
						Relations = relations
					};

					response.Add(entity);
				}

				var resultJson = JsonConvert.SerializeObject(response);
				return new HttpResponseMessage(HttpStatusCode.OK)
				{
					Content = new StringContent(resultJson, Encoding.UTF8, "application/json")
				};

			}
			catch (ArgumentException argEx)
			{
				var message = $"Invalid request body or missing required parameters. Error message: {argEx.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.BadRequest)
				{
					Content = new StringContent($"Invalid request body or missing required parameters. Error message: {argEx.Message}")
				};
			}
			catch (Exception ex)
			{
				var message = $"'Error message':{ex.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.InternalServerError)
				{
					Content = new StringContent($"'Error message':{ex.Message}")
				};
			}
		}
	}
}

3. Search entities by Field Name and Entity Definition but Get Their IDs only

//#r "Newtonsoft.Json"

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using Stylelabs.M.Base.Querying.Linq;
using Stylelabs.M.Sdk.WebClient;
using SY.ContentHub.AzureFunctions.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;

namespace SY.ContentHub.AzureFunctions
{
	/// <summary>
	/// Get a list of IDs for all entities matching the specified search criteria.
	/// </summary>
	public static partial class SearchEntitityIDsByFieldValue
	{
		[FunctionName("SearchEntitityIDsByFieldValue")]
		public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestMessage req, TraceWriter log)
		{
			//Read and parse request payload
			log.Info("SearchEntitityIDsByFieldValue invoked.", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var content = req.Content;
			string requestBody = content.ReadAsStringAsync().Result;
			log?.Info($"Request body: {requestBody}", MethodBase.GetCurrentMethod().DeclaringType.Name);
			var requestObject = JsonConvert.DeserializeObject<SearchEntitityIDsByFieldValueRequest>(requestBody);

			try
			{
				//Initialize CH Web SDK client
				IWebMClient client = Utils.InitClient(req);

				//Search for Entities matching the requested field value
				IList<long> entityIDs;
				if (string.IsNullOrEmpty(requestObject.entitySearch.entitySearchField.fieldValue) || string.IsNullOrEmpty(requestObject.entitySearch.entitySearchField.fieldName))
				{
					//Search for IDs of all entities matching given definition
					entityIDs = await Utils.SearchEntityIDs(client,
											(entities =>
											from e in entities
											where e.DefinitionName == requestObject.entitySearch.entitySearchField.definitionName
											select e),
											log);
				}
				else
				{
					//Search for IDs of all entities matching given definition AND field value
					entityIDs = await Utils.SearchEntityIDs(client,
											(entities =>
											from e in entities
											where e.Property(requestObject.entitySearch.entitySearchField.fieldName) == requestObject.entitySearch.entitySearchField.fieldValue
											where e.DefinitionName == requestObject.entitySearch.entitySearchField.definitionName
											select e),
											log);
				}

				if (entityIDs == null || entityIDs.Count == 0)
				{
					return req.CreateResponse(HttpStatusCode.NotFound, $"No entities found for field {requestObject.entitySearch.entitySearchField.fieldName} with value {requestObject.entitySearch.entitySearchField.fieldValue}");
				}

				var resultJson = JsonConvert.SerializeObject(entityIDs);
				return new HttpResponseMessage(HttpStatusCode.OK)
				{
					Content = new StringContent(resultJson, Encoding.UTF8, "application/json")
				};

			}
			catch (ArgumentException argEx)
			{
				var message = $"Invalid request body or missing required parameters. Error message: {argEx.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.BadRequest)
				{
					Content = new StringContent($"Invalid request body or missing required parameters. Error message: {argEx.Message}")
				};
			}
			catch (Exception ex)
			{
				var message = $"'Error message':{ex.Message}";
				log.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.InternalServerError)
				{
					Content = new StringContent($"'Error message':{ex.Message}")
				};
			}
		}
	}
}

Useful Links

• Sitecore Content Hub

• Azure Functions

• Content Hub REST API

• Web Client SDK

• Sitecore Documentation on Authentication for steps to set up an OAuth client

• Source Code on GitHub - Visual Studio Solution