AEM as Cloud Service Assets Service



In AEM as Cloud Service, Asset Ingestion & Processing has been changed. It offers “AEM Cloud Assets Service” a program powered by “Asset Microservices” which provides a scalable & resilient processing of assets using cloud service & handling of Dynamic Media.

It’s a well thought architecture which resolves load & performance related issues which are normally observed on Author & Publish Instances during asset ingestion & processing. AEM as Cloud Service no longer supports famously known “DAM Update Asset Workflow”.

 

Benefits

The key benefits of asset microservices are

1.     Scalable architecture for seamless processing of resource-intensive operations.

2.     Efficient indexing & text extractions that does not impact AEM environments.

3.     Minimize the need for workflows to handle asset processing in AEM environments. This frees up the resources, minimize the load on AEM & provides scalability.

4.     Improved resilience of asset processing, Potential issues while handling atypical files, such as corrupted files or extremely large files do not impact deployments performance.

5.     Simplified configuration of asset processing for the admins.

6.     Asset processing is managed & maintained by Adobe to provide best known configuration for handling renditions, metadata & text extraction for various file types.

7.     Ability to configure post-processing workflow to add user-specific actions & integrations.

 

Architecture


AEM as Cloud Service Assets Service (Courtesy: Adobe)

Let’s divide the architecture into topics to understand in detail

Direct Binary Access

·       This feature enables AEM to upload/download binaries directly to/from a supported Oak blob storage without streaming the binaries through application and Oak [AEM is based on Apache Jackrabbit Oak].

·       To use this feature underlying storage provider must support the capability & the corresponding Oak Blob Store must implement feature support.

·       This feature frees AEM applications from the network, memory & disk resource utilization involved in transferring binary files thus by eliminating potential bottleneck of network capacity on AEM application server & it’s deployment environment.

·       It provides application with scalability, high-bandwidth access directly to the cloud storage.

·       This API is independent of cloud provider & relies on standard HTTP binary upload & download.

·       The following dialog shows basic architecture for binary storage & access. It involves three parties A remote client, Server Application (AEM) & Cloud Binary Storage.


Basic Architecture Diagram of Binary Cloud Storage & Access (Courtesy: Apache)

 

Binary Cloud Storage

·       In AEM On-Premises “crx-quickstart/repository/datastore” folder is used to store binary data with three storage options as below

o   File Data Store

o   Amazon S3 Cloud Data Store

o   Azure Cloud Data Store.

·       In AEM Assets Service, “Direct Binary Access” feature is implemented in below storage options.

o   Amazon S3 Cloud (S3DataStore)

o   Azure Blob Storage (AzureDataStore)

 

Asset Processing

·       Clients, such as web browsers or Adobe Asset Link, send an upload request to Experience Manager & start uploading the binary directly to the binary storage.

·       When the direct binary upload completes, the client notifies Experience Manager.

·       Experience Manager sends a processing request to asset microservices. The request content depends on the processing profiles configuration in the Experience Manager that specify which renditions to generate.

·       Asset microservices back-end receives the request, dispatches it to one or more microservices based on the request. Each microservice accesses the original binary directly from the binary cloud storage.

·       Result of the processing, such as renditions are stored in the binary cloud storage.

·       Experience Manager is notified that the processing is complete along with direct pointers to the generated binaries (renditions). The generated renditions are available in Experience Manager for the uploaded asset.

·       When Experience Manager calls “Dynamic Media”, it serves the requested asset by pulling the asset binaries from “Binary Cloud Storage”.

 

Custom Asset Post Processing

·       There are use cases where assets ingested/uploaded requires additional processing, for such cases “Custom Post-Processing Workflows” can be used.

·       Post-processing workflows are regular Adobe Experience Manager workflow models, created & managed in AEM Workflow Editor.

·       AEM can be configured to automatically trigger the post-processing workflows after asset processing completes.

 

Asset Upload

In AEM Assets Service, assets can be directly uploaded to the cloud storage by using HTTP API as stated above. The steps to upload binary files are as below.

1.     Submit an HTTP request to upload a new binary.

2.     Put the content of the binary to one or more URIs provided by the initiate request.

3.     Submit an HTTP request to inform the server the status of the uploaded binary.

AEM as Cloud Service Asset Upload (Courtesy: Adobe)


Initiate Upload

·       Submit an HTTP Post request to the desired folder where asset to be create/updated.

·       Include “.initiateUpload.json” to indicate that the request is to initiate upload of a binary file.

·      

FoFor Example: if the path for asset to be uploaded in “/assets/folder” then POST request is

https://[aem_server]:[port]/content/dam/assets/folder.initiateUpload.json 

·       The content type of the request body should be application/x-www-form-urlencoded form data, containing the following fields:

o   (string) fileName: Required. The name of the asset as it appears in Experience Manager.

o   (number) fileSize: Required. The file size, in bytes, of the asset being uploaded.

 

Upload Binary

·       The output of initiating an upload includes one or more upload URI values.

·       If more than one URI is provided, the client splits the binary into parts and make PUT requests of each part to each URI, in order to use all URIs.

·       Ensure that the size of each part is within the minimum and maximum sizes as specified in the initiate response

·       CDN edge nodes help accelerate the requested upload of binaries.

 

Complete Upload

·       After all the parts of a binary file are uploaded, submit an HTTP POST request to the complete URI provided by the initiation data.

·       The content type of the request body should be application/x-www-form-urlencoded form data, containing the following fields.

 

Open-source Upload Library

Adobe provides two open source libraries & tools for upload of assets.

·       Open-source aem-upload library

·       Open-source command-line tool

 

References

·       https://experienceleague.adobe.com/docs/experience-manager-cloud-service/assets/asset-microservices-overview.html?lang=en

·       https://experienceleague.adobe.com/docs/experience-manager-cloud-service/assets/admin/developer-reference-material-apis.html?lang=en

·       https://jackrabbit.apache.org/oak/docs/features/direct-binary-access.html

 

 

Comments

Post a Comment

Popular posts from this blog

Headless content management in AEM

      Adobe Experience Manager, commonly referred to as AEM, is a cloud-native, API-first content management system (CMS) and Digital Asset Management (DAM) platform that enables you to structure and deliver headless content across multiple channels. AEM enables headless delivery of immersive and optimized media to customers that can automatically adapt to any platform or device.        This headless CMS will help you create immersive content that allows marketing teams to manage channel-agnostic content using an enterprise-level DAM. It supports a content as a service model that doesn’t need you to recreate your content for different channels. Moreover, businesses seeking content delivered as packed HTML ( HTML/CSS/JS) can acquire the same with AEM.        Taking the above into consideration, in this article, we’ll be shedding some light on everything you need to know about AEM and its unique headless content management feature. Headless CMS Vs. Traditional CMS       To unders
Read more

AEM as Cloud Service Migration Tools

  AEM as Cloud Service providing useful tools for migrating from AEM On-Premises. This article covers various tools available & their purpose. The tools can be downloaded from adobe “Software Distribution” portal.   Tools Best Practices Analyzer (BPA) Tool
Read more

AEM as Cloud Service Project Structure - 1

Image
  In AEM as Cloud Service, Project Structure has changed & is one of the important exercises to be done if any projects wish to migrate from On-Premises to AEM as CS. In this article we will explore on project structure, code repository, run modes & tools and tips. This section describes the changes required to "AEM On-Premises Maven Project to be AEM as Cloud Service Compatible Project" . 1.      AEM as Cloud Service brings the concept of  mutable  &  immutable content. The dependencies are established to create non-conflicting, deterministic deployments & that they are packaged in deploy-able structure. 2.      AEM application deployments must be comprised of a single AEM Package. This package should in turn contain sub packages that comprises everything to run the application.   Mutable vs Immutable 1.      /apps & /libs are considered  immutable  areas of AEM as they cannot be changed (CRUD Operations) after AEM starts (i.e. at run-time).
Read more