CH Upstream Integrations part 2: An Azure Function to Update or Create a Relation Between Content Hub Entities

Intro

March 3rd, 2022

The following post describes an example UpsertRelation Azure Function, which finds two specified entities in Sitecore Content Hub (using provided search criteria) and updates or creates a relation between them.

Read other Sergey's blogs at xcentium.com

This post is the 2nd part in a 4-piece series, describing an integration approach that allows to. Content Hub and connect Sitecore with pretty much any external system. I'm composing a number of Azure Functions, both provided by Azure and my custom-built ones to build integration flows using Azure Logic Apps. Upstream integrations pull data from one or more external systems, extracted, transform and process as needed, and pushed changes into Sitecore Content Hub via its APIs. I chose to use Content Hub Web Client SDK, which is a .NET abstraction on top of the Sitecore Content Hub REST API because it simplifies the development of .NET client code and helps to deal with CH API throttling, and does a few more helpful things.

I believe Logic Apps is a good way to visually orchestrate various building blocks (Azure Functions) together with very little to no code required: easy to build, and easy to change, but of course, this isn't the only way. I'm sharing all source code here, so others can use it for building the custom integration solutions for Sitecore Content Hub.

All posts in this series:

Part 1: Describes the UpsertEntity function, which will update or create a new entity from the payload data
Part 2 (this post): Describes the UpsertRelation Azure Function, which looks up two Entities to be associated in Content Hub and creates or updates a relation between them.
Part 3: Example Azure Logic App, combining above functions into example integration flow, which updates and/or creates entities and relations between them in Content Hub as source data is changing (in close to the real-time manner)
Part 4: Useful Azure Building Blocks for building Cloud Integrations with Sitecore Content Hub

It's worth noting Sitecore that recently announced Sitecore Connect along with other new great products, so consider using Sitecore Connect before implementing your custom solution.

An Example Schema in Content Hub: Two Entities and Relation Between Them

I am using Content Hub's OOTB PCM schema intended to be used for the management of commerce products and their catalogs. In this example M.PCM.Product entity is linked to M.PCM.ProductFamily via PCMProductFamilyToProduct relation. In Part 3 I provide a fully working example Logic App, which is an integration flow, where various entities and relationships can be created/updated in the Content hub.

Authentication: Setting OAuth Client in Content Hub

The code below relies on an "OAuth client", which needs to be configured in Content Hub prior to running. Refer to Sitecore Documentation on Authentication for steps to set up an OAuth client in your Content Hub instance(s).

An Azure Function to Update or Create A Relation in Content Hub

The UpsertRelation function will need to field values by which to look up two ends of relation in CH. Since these entities originate in the source system, there are fields that are the unique keys in the source system, which are saved in CH, so they can be later used for lookup later one when those entities might be updated or connected with each other going forward.

Here's what the below-described function does at, a very high level:

The request to execute function includes all necessary details about two Content Hub entities to be connected with each other. Alternative an existing relation in CH can be deleted if the "deleted" root node is set to true in the request payload
Content Hub instance information, such as its endpoint URL and authentication details are stored in settings, in this case in Function App settings
The function deserialize the request to extract the information needed to find and connect two entities in CH
Content Hub endpoint Url and Authentication details come from Azure Function Settings. Using Azure Key Vault might be a better idea security-wise - I'm leaving this out, but thought this was worth mentioning.
Search for parent and child entities in CH using their definition names and search key/value pairs provided in the request. The idea is that the key/value pair is unique in the source system, which knows nothing about Content Hub - this data is saved in Content Hub, so can be looked up later.
If given relation doesn't exist then it will be created.
If the given relation already exists and the "deleted" node is set to true then the relation will be deleted from CH, so entities will no longer be associated.

Azure Logic Apps allow to perform some basic retry and error-handling logic to do logging, and can be integrated Service Bus queue to decouple the source and destination ends of the integration.

Configuration Settings (Parameters in Azure Logic App)

Refer to post 3 in this series for more details on Azure Logic App configuration settings. A total of five settings are required by this Azure Function to establish a connection to the Content Hub: they include Content Hub root Url ([https://domain_name], nothing after that) and four authentication settings for the above-mentioned OAuth Client.

Function Input: the Request Body

Here's an example payload request for the UpsertRelation function

And these fields should be used like so:

continueOnEmptySearchFields: continue with no errors if search values for parent and/or child entities are not specified, otherwise thrown an error. This comes in handy in cases when this function is a part of the sync pipeline, where parent and/or child entities don't exist or weren't created on previous steps and it's okay to skip the relation then
continueOnNoFoundEntities: similarly to the above case, saliently continue without throwing errors if search values had been specified, but parent and/or child entity had not been found in CH
deleted: delete an existing relation in CH or skip the creation of a new one
relationFieldName: name of the relation field in CH
keepExistingRelations: if multiple relations already exist on the found parent entity - keep them and append a new one when this value is set to true, otherwise delete all existing ones and add a new one.
entitySearch node contains lookup information for parent and child entities to be associated with each other: names and values of the fields, by which this Entity will be looked up in the Content Hub.
- fieldName: name of the field as it appears in CH (actual field name, not the display name)
  - fieldType: name of the field type as it appears in CH
  - fieldValue: field value
Azure Function Source Code

//#r "Newtonsoft.Json"

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using Stylelabs.M.Framework.Essentials.LoadConfigurations;
using Stylelabs.M.Framework.Essentials.LoadOptions;
using Stylelabs.M.Sdk.Contracts.Base;
using Stylelabs.M.Sdk.WebClient;
using Stylelabs.M.Sdk.WebClient.Authentication;
using SY.ContentHub.AzureFunctions.Models;
using System;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Threading.Tasks;

namespace SY.ContentHub.AzureFunctions
{
	/// <summary>
	/// Update an existing or create a new relation between two specified entities in Content Hub
	/// </summary>
	public static partial class UpsertRelation
	{
		[FunctionName("UpsertRelation")]
		public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestMessage req, TraceWriter log)
		{
			//Read and parse request payload
			log.Info("UpsertRelation invoked.");
			var content = req.Content;
			string requestBody = content.ReadAsStringAsync().Result;
			log.Info($"Request body: {requestBody}");
			var requestObject = JsonConvert.DeserializeObject<UpsertRelationRequest>(requestBody);

			try
			{
				requestObject.Validate();

				if (requestObject.continueOnEmptySearchFields
					&& (
					string.IsNullOrEmpty(requestObject.entitySearch.parentEntitySearchField.fieldValue)
					|| string.IsNullOrEmpty(requestObject.entitySearch.childEntitySearchField.fieldValue)))
				{
					var message = $"Skipping relation creation as continueOnEmptySearchFields is set to true. Relation Name: {requestObject.entityData.relationFieldName}, Parent entity search value: {requestObject.entitySearch.parentEntitySearchField.fieldValue}, Child entity search value: {requestObject.entitySearch.childEntitySearchField.fieldValue}";
					log?.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
					return new HttpResponseMessage(HttpStatusCode.OK)
					{
						Content = new StringContent(message)
					};
				}

				//Initialize CH Web SDK client
				var clientInfo = Utils.ExtractClientInfo(req.Headers);
				Uri endpoint = new Uri(clientInfo.baseUrl);
				OAuthPasswordGrant oauth = new OAuthPasswordGrant
				{
					ClientId = clientInfo.clientId,
					ClientSecret = clientInfo.clientSecret,
					UserName = clientInfo.userName,
					Password = clientInfo.password
				};

				IWebMClient client = MClientFactory.CreateMClient(endpoint, oauth);

				//Query for single Entity that matches the search criteria for parent entity for the relation
				IEntity parentEntity = await Utils.SearchSingleEntity(client,
					requestObject.entitySearch.parentEntitySearchField.fieldName,
					requestObject.entitySearch.parentEntitySearchField.fieldValue,
					requestObject.entitySearch.parentEntitySearchField.definitionName,
					EntityLoadConfiguration.Full, log);

				//Query for single Entity that matches the search criteria for child entity for the relation
				IEntity childEntity = await Utils.SearchSingleEntity(client,
					requestObject.entitySearch.childEntitySearchField.fieldName,
					requestObject.entitySearch.childEntitySearchField.fieldValue,
					requestObject.entitySearch.childEntitySearchField.definitionName,
					EntityLoadConfiguration.Minimal, log);

				if (parentEntity != null && parentEntity.Id != null && parentEntity.Id.HasValue
					&& childEntity != null && childEntity.Id != null && childEntity.Id.HasValue)
				{
					log?.Info($"Parent Entity ID: {parentEntity.Id}, Child Entity ID: {childEntity.Id}");
					IRelation relation = null;
					string message = "";

					//Get a hold of Relation field in Parent entity
					relation = parentEntity.GetRelation(requestObject.entityData.relationFieldName, RelationRole.Parent);
					log?.Info($"Parent Relation from parent: {relation}");
					if (relation == null)
					{
						relation = parentEntity.GetRelation(requestObject.entityData.relationFieldName, RelationRole.Child);
						log?.Info($"Child Relation from child: {relation}");
					}

					try
					{
						//Update, create or delete the relation between entities, based on request values
						if (relation != null)
						{
							var ids = relation.GetIds().ToList();
							//If relation already exist then update/add or delete when "deleted" field is set to true in the request
							if (!ids.Contains(childEntity.Id.Value))
							{
								if (requestObject.deleted)
								{
									message = $"Skipping relation add because it is marked deleted. Relation Name: {requestObject.entityData.relationFieldName}, Parent Entity ID: {parentEntity.Id}, Child Entity ID: {childEntity.Id}";
								}
								else
								{
									ids.Add(childEntity.Id.Value);
									relation.SetIds(ids);
									long addedResult = await client.Entities.SaveAsync(parentEntity);
									message = $"Successfully added relation. Relation Name: {requestObject.entityData.relationFieldName},Parent Entity ID: {parentEntity.Id}, Child Entity ID: {childEntity.Id}";
								}

							}
							//If relation don't exist then add it, unless the "deleted" field is set to true in the request
							else
							{
								if (requestObject.deleted)
								{
									ids.Remove(childEntity.Id.Value);
									relation.SetIds(ids);
									long deletedResultId = await client.Entities.SaveAsync(parentEntity);
									message = $"Successfully deleted relation because it is marked deleted. Relation Name: {requestObject.entityData.relationFieldName},Parent Entity ID: {parentEntity.Id}, Child Entity ID: {childEntity.Id}";
								}
								else
								{
									message = $"Relation already exists - skipping. Relation Name: {requestObject.entityData.relationFieldName}, Parent Entity ID: {parentEntity.Id}, Child Entity ID: {childEntity.Id}";
								}
							}

							log?.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
							return new HttpResponseMessage(HttpStatusCode.OK)
							{
								Content = new StringContent(message)
							};
						}
						else
						{
							relation = parentEntity.GetRelation(requestObject.entityData.relationFieldName, RelationRole.Child);
							log?.Info($"Child Relation: {relation}");

							message = $"Relation not found. Parent Entity ID: {parentEntity.Id}, Relation Field Name: {requestObject.entityData.relationFieldName}";
						}
					}
					catch (Exception ex)
					{
						log?.Error($"Exception: Type: {ex.GetType()}, Message: {ex.Message}", ex, MethodBase.GetCurrentMethod().DeclaringType.Name);
						throw;
					}


					log?.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
					return new HttpResponseMessage(HttpStatusCode.NotFound)
					{
						Content = new StringContent(message)
					};
				}
				else

				{
					if (requestObject.continueOnNoFoundEntities)
					{
						var message = $"Skipping relation creation as continueOnNoFoundEntities is set to true.  Parent entity search value: {requestObject.entitySearch.parentEntitySearchField.fieldValue}, Child entity search value: {requestObject.entitySearch.childEntitySearchField.fieldValue}";
						log?.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
						return new HttpResponseMessage(HttpStatusCode.OK)
						{
							Content = new StringContent(message)
						};
					}
					else
					{
						var message = $"One or both ends of the relation are not found. Parent Entity ID Found: {parentEntity?.Id}, Child Entity ID: {childEntity?.Id}";
						log?.Error(message, null, MethodBase.GetCurrentMethod().DeclaringType.Name);
						return new HttpResponseMessage(HttpStatusCode.NotFound)
						{
							Content = new StringContent(message)
						};
					}
				}
			}
			catch (ArgumentException argEx)
			{
				var message = $"Invalid request body or missing required parameters. Error message: {argEx.Message}";
				log?.Info(message, MethodBase.GetCurrentMethod().DeclaringType.Name);
				return new HttpResponseMessage(HttpStatusCode.BadRequest)
				{
					Content = new StringContent(message)
				};
			}
			catch (Exception ex)
			{
				var message = $"'Error ':{ex.Message}";
				return new HttpResponseMessage(HttpStatusCode.InternalServerError)
				{
					Content = new StringContent(message)
				};
			}
		}
	}
}

Useful Links
- Sitecore Content Hub

Sitecore

Content Hub

Integrations

Sergey Yatsenko