CH Upstream Integrations part 3: Example Logic App for Syncing an External System with Sitecore Content Hub

Intro

An Azure Logic App to Index Content Hub content with Apache Solr

Read Sergey's blog posts at xcentium.com

Introduction: Using Azure Logic Apps in Integration Flows

This post is the 2nd part in a 3-piece series, describing an integration approach that allows connecting Content Hub with pretty much any external system via APIs. In this case, Content Hub is the source of truth, from which content gets published to other systems, so this is an example of downstream integration.

I'm leveraging Azure Functions as integration building blocks and Azure Logic Apps to compose them together into integration flows, where data from the external system would be extracted, transformed, or processed and pushed to Content Hub via its APIs. I chose to use Content Hub Web Client SDK, which is a .NET abstraction on top of the Sitecore CH REST API because it helps to deal with CH API throttling, and does a few more helpful things.

I believe Logic Apps is a good way to visually orchestrate various building blocks (Azure Functions) together with very little to no code required: easy to build, and easy to change, but of course, this isn't the only way. I'm sharing all source code here, so others can use it for building the custom integration solutions for Sitecore Content Hub.

All posts in this collection:

• Part 1 . Example Azure Functions: Describes example functions for reading content entities and their IDs from Content Hub
• Part 2 (this post). Solr Indexer Logic App: Describes example Logic App to publish content changes from Sitecore Content Hub to an external system. In this case, I am extracting, transforming, and then saving Content Hub content into Solr, so this Logic App is effectively a Solr Indexer.
• Part 3. A Content Crawler for Full Index Rebuild: Another Logic App, which reads IDs of entities in Content Hub and pushes them to an Azure Service Bus Queue, so they would be picked up and processed by the Logic App from part 2.

It's worth noting Sitecore that recently announced Sitecore Connect along with other new great products, so consider using Sitecore Connect before implementing your custom solution.

Useful Information

Relations in Sitecore Content Hub

Querying and Reading Content Hub Data with Web SDK Client

Azure Logic App

Our example Azure Logic App will index Content Hub Entities of type M.Asset as they get updated or created in Content Hub.

Here's how the indexing process goes:

• A trigger is fired upon the M.Asset entity created or updated in Sitecore Content Hub. This trigger will send a message, which includes Entity ID to the specified Azure Service Bus Queue

• Our Logic App is triggered by a new message in a given Service Bus Queue and performs the following steps:

  • Read and JSON-parse content of given entity from Content Hub
  • Read related Public Entities entities to extract public links and populate an array of public link URLs. In this case, retrieval of public links and creating a Solr array from them is akin to the concept of the computed field in Sitecore
  • Generate property formatter request JSON for Solr from M.Asset content and its public links
  • Submit a POST request with above-generated JSON payload to Solr to create or update the index record, representing the given M.Asset

  High-Level View

When a message is received in a queue (auto-complete)

The Service Bus Trigger is one of the Azure Out-Of-The-Box triggers for initiating Azure Logic App execution. This way messages sent out by Content Hub are picked up and processed one by one (multiple instances of the same Logic App can be configured to run in parallel). In this case, I configured a trigger to check for new messages in a given queue 12 times per minute, or every 5 seconds

Extract Entity ID

This is another Azure OOTB step, originally called JSON Parse. The schema can be generated by clicking on "Use sample payload to generate schema". The full message sent to Service Bus by Content Hub might look like this:

{
 "saveEntityMessage": {
  "EventType": "EntityUpdated",
  "TimeStamp": "2020-11-26T21:29:03.719Z",
  "IsNew": false,
  "TargetDefinition": "M.Asset",
  "TargetId": 72193,
  "TargetIdentifier": "jQ_9Xryr6UihojWlrwrd0g",
  "CreatedOn": "2021-11-26T01:35:19.9869541Z",
  "UserId": 11743,
  "Version": 7,
  "ChangeSet": {
  "PropertyChanges": [
    {
     "Culture": "(Default)",
     "Property": "Title",
     "Type": "System.String",
     "OriginalValue": "<p>Example Asset Title.</p>\n",
     "NewValue": "<p>Asset Title</p>\n"
    }
   ]
}

...but since we're only interested in retrieving the ID of the changed Entity, I'll shorted it like below and use this payload to generate a schema in the above "Extract Entity ID" step.

{
 "saveEntityMessage": {
  "TargetId": 72193
 }
}

Get Entity Content by ID

This one is my custom-built Azure Function, described in this post [TODO: insert link], which takes entityId as a parameter and then reads Entity with a given ID from Content Hub and returns its JSON representation in the response.

The Headers content base URL of Content Hub instance and OAuth client credentials. More details on configuring the OAuth client in Content Hub can be found in Sitecore documentation

Extract Entity Definition Name

Another "Parse JSON" step is to retrieve Entity Definition from Entity JSON. Here again, since we're only interested in reading the Definition field, the sample JSON I used to generate the schema will look like this:

{
 "Entity": {
  "Properties": [
      "DefinitionName":"M.Asset"
  ]
 }
}

Initialize Public Links variable

An array of public links to be populated on later steps

Switch

Branch logic based on DefinitionName value, retrieved above from the Entity JSON

M.Asset branch

Below steps are executed if EntityDefinition == "M.Asset"

M.Aset. Extract Properties

Parse JSON to read and extract some of its field values, which are to be posted to Solr at a later step. Here's an example of shortened JSON, which I used to generate the schema for this step:

{
    "Entity": {
        "Properties": {
            "Id": 34085,
            "Identifier": "Mid6uFXVWE6hKTHZDWDdWA",
            "DefinitionName": "M.Asset",
            "FileName": "depositphotos_96284060-stock-photo-contract-waiting-for-a-notary.jpg",
            "FileSize": 0.02,
            "Description": "",
            "PublishStatus": "Published",
            "Title": "depositphotos_96284060-stock-photo-contract-waiting-for-a-notary.jpg",
            "ApprovedBy": "sergey.yatsenko@xcentium.com"
        },
        "Objects": {            
            "MainFile": {
                "locations": {
                    "local": [
                        "local-9c63348885c94c048b01f26c889fa6d3"
                    ]
                },
                "properties": {
                    "filesize": 0.02,
                    "filesizebytes": "15182",
                    "filename": "depositphotos_96284060-stock-photo-contract-waiting-for-a-notary.jpg",
                    "width": "600",
                    "height": "400",
                    "group": "Images",
                    "extension": "jpg",
                    "content_type": "image/webp",
                    "resolution": "600x400",
                    "colorspace": "sRGB",
                    "megapixels": "0.24"
                }
            }
        }
    },
    "Relations": {
        "AssetToPublicLink": [
            40759,
            34130
        ],
        "AssetTypeToAsset": [
            34270
        ]
    }
}

M.Asset. Get Public Links

a "for each" loop, which is for each ID presented in AssetToPublicLink relation (see JSON above), reads PublicLink entity from Content Hub, parses its content, constructs the URL of each like in the below step, and appends it to a resulting array.

M.Asset. Update SOLR

And finally, construct JSON payload and POST that to Solr instance.

• Note that the Solr URL should include Solr code as a segment, so it may look like this: https://[your_solr_instance]/solr/sitecore_master_index/update?commitWithin=1000
• And Authorization, if you have it configured on your Solr instance may look like "Basic [your_password]"

Useful Links

• Sitecore Content Hub

• Azure Functions

• Content Hub REST API

• Web Client SDK

• Sitecore Documentation on Authentication for steps to set up an OAuth client

• Source Code on GitHub - Visual Studio Solution