External DAM module
Connector Pack | DAM Connector Pack |
---|---|
License |
|
Issues |
|
Maven site |
|
Latest |
1.1.5 |
Module structure
artifactID | Description |
---|---|
|
Parent reactor. |
|
Provides the foundation for integrating with S3 and Bynder and a sample component for using assets in the Pages app. |
|
Provides a default implementation to connect to Bynder. |
|
Provides a default implementation to connect to Amazon S3. |
Installing with Maven
Bundled modules are automatically installed for you.
If the module is unbundled, add the following to your bundle including your project’s <dependencyManagement>
section and your webapp’s <dependencies>
section.
If the module is unbundled but the version is managed by the parent POM, add the following to your webapp’s <dependencies>
section.
<dependency>
<groupId>info.magnolia.external.dam</groupId>
<artifactId>magnolia-external-dam</artifactId>
<version>1.1.5</version> (1)
</dependency>
1 | Should you need to specify the module version, do it using <version> . |
<dependency>
<groupId>info.magnolia.external.dam</groupId>
<artifactId>magnolia-external-dam-bynder</artifactId>
<version>1.1.5</version> (1)
</dependency>
1 | Should you need to specify the module version, do it using <version> . |
<dependency>
<groupId>info.magnolia.external.dam</groupId>
<artifactId>magnolia-external-dam-s3</artifactId>
<version>1.1.5</version> (1)
</dependency>
1 | Should you need to specify the module version, do it using <version> . |
Connection configuration
This section explains how to configure the connection between the External DAM module and your external asset management solution.
Essentially, you:
-
Provide connection credentials for your external solution.
-
Enable the subapp to display in the Magnolia Assets app.
-
Configure the cache if necessary.
You can create or edit the configuration in the JCR or the File System (YAML) under <module-name>/external-dams/<definition-name>
.
Before you configure the connection, make sure you know:
-
Which solution you want to connect to. Default implementations are provided for:
-
AWS S3
-
Bynder
-
-
Connection credentials for your solution.
If you want to connect to another third-party tool, extend the Magnolia DAM API and create your own asset provider |
Magnolia Professional Services has developed several additional Incubator module connectors. For example: Aprimo, Celum, Cloudinary and Frontify. Incubator modules are only available to Magnolia Enterprise customers. |
AWS S3 connection configuration
To use AWS S3 assets in Magnolia, you must have a working AWS account.
You need AWS S3 credentials to connect your S3 buckets and display their content in Magnolia. Credentials consist of two parts:
-
AWS access key ID
-
AWS secret access key
You can store your AWS credentials using:
-
The Magnolia Passwords app
-
The AWS default credential provider chain
Using the Magnolia Passwords app
Add the credentials to your Magnolia instance in the Password manager app.
For example:
|
|
|
|
|
Using the AWS default credential provider chain
-
Add the following configuration properties to your
WEB-INF/config/default/magnolia.properties
file:magnolia.aws.validateCredentials=false magnolia.aws.useCredentials=false
-
Set your AWS credentials by following the instructions in the AWS documentation: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html
The magnolia-aws-foundation module handles all Amazon connections from Magnolia. This module is installed automatically by Maven when you install the External DAM module.
|
Configuring an AWS S3 bucket whitelist
By default, all the buckets in the S3 account you connect to are displayed. If you do not want to display all your buckets, you can configure a whitelist in the Resource files app using regex.
For example:
whitelistedBuckets:
- "^a.*"
- a-particular-bucket
Adding a filter capability to S3 buckets
If you want to restrict access to a single bucket either because it’s used in different use cases or for some confidentiality-related reasons, you can add a bucket prefix that grants access only to that specific bucket without revealing its actual path which might contain unwanted access information. This way the user will see the bucket and folders will be seen as the only folders in the Magnolia instance.
To make use of the filter, simply add the following bucket prefix to your config.yaml
file:
whitelistedBuckets:
- BUCKET_NAME
bucketPrefixes:
BUCKET_NAME: “FOLDER_NAME/SUBFOLDER_NAME”
Bynder connection configuration
To use Bynder assets in Magnolia, you must have a working Bynder account.
API token
You need a permanent OAuth 2.0 API token from Bynder to access the assets.
To generate a token, see this Bynder documentation: https://help.bynder.com/system/oauth2-permanent-tokens.htm.
You can use the token value directly in the yaml configuration or add the permanent token to your Magnolia instance in the Password manager app and reference the path in the configuration.
For example:
|
|
|
Connecting
Add your connection configuration under /dam-bynder/config.yaml
.
bynderPortalURL: https://magnoliaintegration.getbynder.com
permanentToken: <client_secret_or_path_to_password_manager>
Connection properties
Property | Description | ||
---|---|---|---|
|
required URL to your Bynder service. |
||
|
required
|
Enabling your subapp
By default, the AWS S3 and Bynder subapps are disabled.
To enable them, decorate the Magnolia Assets app under <dam-connector>/decorations/dam
and set the enabled
property to true
.
Bynder enabled configuration example:
providers:
bynderProvider:
identifier: bynder
enabled: true
class: info.magnolia.external.dam.bynder.BynderAssetProvider
renderers:
bynder:
class: info.magnolia.external.dam.bynder.BynderAssetRenderer
proxyMode: true
mediaTypes:
audio: 'audio/*'
application: 'application/*'
video: 'video/*'
image: 'image/*'
S3 enabled configuration example:
providers:
s3Provider:
identifier: s3
enabled: true
class: info.magnolia.external.dam.s3.datasource.S3AssetProvider
Configuring a single asset provider for link fields
You can configure a link field to limit your users to selecting assets from only one asset provider.
This is possible from External DAM 1.0.7 (DAM module 3.0.8). Previously, you had to use the aggregated damLinkField and users had to choose from all the enabled asset providers.
|
This example shows a dialog definition with three link fields, each limited to different asset providers:
form:
properties:
bynder:
$type: damLinkField
chooserId: dam-bynder:chooser
datasource:
class: info.magnolia.dam.app.data.AssetDatasourceDefinition
name: bynder
s3:
$type: damLinkField
chooserId: dam-s3:chooser
datasource:
class: info.magnolia.dam.app.data.AssetDatasourceDefinition
name: s3
jcr:
$type: linkField
converterClass: info.magnolia.ui.editor.converter.JcrAssetConverter
datasource:
$type: jcrDatasource
workspace: website
Configuring the cache
This connector pack uses Caffeine, a high-performance Java cache library, to manage caching for external assets via a Magnolia helper module called magnolia-addon-commons-cache
.
For Bynder and S3 caching, you can specify the behavior of the caches. Essentially, you can configure all the parameters available in CaffeineSpec.
For example:
-
Set a maximum cache size.
-
Define how often the cache expires.
Additionally, you can:
-
Enable or disable the cache.
You configure caching through decoration under /<dam-connector>/decorations/addon-commons-cache/config.yaml
.
Bynder default configuration
cacheConfigurations:
bynder:
caches:
bynder-media: maximumSize=500, expireAfterAccess=10m
bynder-all: expireAfterAccess=60m
bynder-size: expireAfterAccess=60m
enabled: true
S3 default configuration
/dam-s3/decorations/addon-commons-cache/config.yaml
cacheConfigurations:
s3:
caches:
s3-objects: maximumSize=500, expireAfterAccess=10m
s3-buckets: expireAfterAccess=60m
s3-count: maximumSize=500, expireAfterAccess=10m
s3-pages: maximumSize=500, expireAfterAccess=10m
Cache configuration properties
Property | Description |
---|---|
|
required Use CaffeineSpec properties to specify the cache behavior such as the maximum size of the cache and when cache entries expire for each cache.
For example:
|
|
required, Set to |
Bynder loading times Note that the Bynder API may be very slow in some regions. For example, one image upload may take up to 20 seconds in EU regions. This may cause some actions to fail in the Magnolia UI and the cache to be invalidated. For more information, see https://github.com/Bynder/bynder-js-sdk/issues/43. |
Manually flushing the cache
If you want to flush the cache, you can either set the cache expiry to a very short time (see above) or configure a script in the Groovy app to flush the cache manually. For example:
import info.magnolia.objectfactory.Components
cacheManager = Components.getComponent(info.magnolia.addon.commons.cache.CacheManager)
cacheManager.evictCache("s3-objects")
Configuring pagination to improve response times
Magnolia retrieves pages of objects from Amazon S3 and Bynder and then obtains the assets you require from those pages. The pages are cached.
You can configure the size of a page of objects using the property pageSize
.
Bynder has an additional property you can use if you set the page size to more than 1000 objects.
pageSize: 100
Property | Description | ||
---|---|---|---|
|
optional The maximum number of objects per request retrieved from your external system.
|
||
|
optional, default is Maximum number of objects that Bynder returns if you specify a
|
Bynder-specific configuration
This section provides instructions specific to configuring the Bynder connector.
(See below for S3.)
Configuring the proxy service to protect private asset URLS
To protect the URLs of private Bynder assets rendered in Magnolia, you must use the proxy service provided.
The proxy service is also required if you want to use Bynder image variations. |
The proxy service enforces security by generating a Magnolia URL that hides the private Bynder URL.
If you do not use the proxy service, private Bynder URLS are exposed in published Magnolia pages for example.
When using the proxy, if you want to make a Bynder asset public, for example an image asset used in the Pages app, you must publish the image as well as the page in Magnolia.
You can enable or disable the proxy service using the proxyMode
parameter. The functionality is provided via the BynderAssetRenderer.
The proxy service is only available with Magnolia 6.2.6+. If you use an earlier version of Magnolia Core, you must disable the proxy service. |
providers:
bynderProvider:
identifier: bynder
enabled: false
class: info.magnolia.external.dam.bynder.BynderAssetProvider
renderers:
bynder:
class: info.magnolia.external.dam.bynder.BynderAssetRenderer
proxyMode: true
mediaTypes:
audio: 'audio/*'
application: 'application/*'
video: 'video/*'
image: 'image/*'
Property | Description |
---|---|
|
optional, default is Enables or disables the proxy service.
|
Using Bynder image derivatives
You must enable proxy mode to use Bynder derivatives in Magnolia.
Bynder provides image derivatives (referred to as variations or renditions in Magnolia) by default, such as:
-
small
-
medium
-
large
-
original
-
thumbnail
These derivatives can be used as the rendition names in the Magnolia DAM templating function (damfn).
For example, to retrieve the small
Bynder derivative:
<!DOCTYPE html>
<html xml:lang="${cmsfn.language()}" lang="${cmsfn.language()}">
<head>
[@cms.page /]
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>${content.windowTitle!content.title!}</title>
</head>
<body class="sample-dam ${cmsfn.language()}">
<div class="container">
<h3>Bynder ID: ${content.assetId}</h3>
<h4>small</h4>
<img src="${damfn.getAssetLink(content.assetId, "small")}"/>
</div>
</body>
</html>
Only image type Bynder asset derivatives can be used in Magnolia.
To see which derivatives are available for a given Bynder image, look in the Size variations field in the asset details view:
Magnolia falls back to the original
derivative if the requested derivative is unavailable.
Retrieving Bynder derivatives differs from using typical Magnolia image variations:
|
For more information about Bynder derivatives, we recommend the following pages in the Bynder documentation:
AWS S3-specific configuration
This section provides instructions specific to configuring the S3 connector.
Configuring the maximum search time for S3
You can configure the maximum time Magnolia Periscope spends searching for S3 assets to be displayed in the Magnolia interface when a user searches for an asset in the Find Bar.
maxSearchTimeInMilis: 10000
Property | Description |
---|---|
|
optional The maximum time Magnolia Periscope spends searching for S3 assets. By default, the maximum search time is 10 seconds. For example: maxSearchTimeInMilis: 10000 |
Configuring presigned URLs to enable previewing for S3 assets
When you upload assets via Magnolia to S3, they are set to private by default. If you want to keep the preview functionality available, you must use presigned urls.
presignedSignatureDurationInMinutes: 60
Property | Description |
---|---|
|
optional Determines how long a presigned URL is valid. We recommend that the duration is at least equal to the expiration of the caches set for s3-objects`, For example: |
Configuring the AWS regions list
When creating an S3 bucket in the Assets app, only AWS-enabled regions are displayed to users.
If your users want to use a disabled region, you must enable it first in AWS as described in Managing AWS Regions in the AWS documentation.
Then you must remove the newly enabled region from the list of disabled regions in the Magnolia configuration so that it appears in the chooser.
disabledRegions:
- "ap-east-1"
- "af-south-1"
- "eu-south-1"
- "me-south-1"
- "cn-north-1"
- "cn-northwest-1"
- "us-gov-east-1"
- "us-gov-west-1"
- "us-iso-east-1"
- "us-isob-east-1"
Usage
The magnolia-external-dam
submodule provides a sample Text and external image page component so that editors and marketers can select and use external assets in the Pages app.
See DAM Connector Pack for an example.
If an asset is uploaded into the DAM app that has been compromised, for example, an image file containing malicious code, an error message is displayed. |
Release history
External DAM module 1.1.5
Released on July 13, 2023.
Due to an installation issue in 1.1.4, we recommended you use this version instead.
External DAM module 1.1.4
Released on June 16, 2023.
This release brings some necessary implementations, improvements, and an important bug fix.
-
S3 buckets have a filter capacity now that enables sharing only a single bucket using a prefix key. (EXTDAM-369)
-
commons-beanutils
were replaced bycommons-beanutils2
. (EXTDAM-374)
Bug fix
-
Using an S3 link within a component was preventing component to be created properly in JCR. (EXTDAM-358)
External DAM module 1.1.3
Released on March 25, 2022.
This release brings an important security fix by updating a third party library and one improvement:
-
Kotlin Standard Library updated to 1.6.10.
-
Notification improvement when publishing S3/Bynder assets.
External DAM module 1.0.10
Released on March 25, 2022.
This release brings an important security fix by updating a third party library:
-
Kotlin Standard Library updated to 1.6.10.
External DAM module 1.1.2
Released on February 11, 2022.
This maintenance release fixes a bug for customers using Bynder Assets with associated links that contain special characters. With this fix, special characters are encoded to avoid loading issues. (EXTDAM-327)
External DAM module 1.1.1 and 1.0.9
Released on December 22, 2021.
This maintenance release fixes a bug for Cloud customers using the delivery endpoint where, when working with S3 as your external asset management solution, the time.now
timestamp was appended to the URL.
External DAM module 1.1.0
Released on November 15, 2021.
This release brings a number of improvements and bug fixes.
Support for Bynder image derivatives
Bynder provides different image derivatives (referred to as image variations in Magnolia) such as large, medium, small. This functionality is now integrated into Magnolia and can be used via the DAM templating function (damfn).
For example, to retrieve the small
Bynder derivative:
damfn.getAssetLink(bynderAsset, "small")
Bynder derivatives do not function in the same way as standard Magnolia image variations as described in Defining image variations. You do _not have to configure the theme to use Bynder derivatives. |
You must configure the Bynder proxy service to use variations. Only image assets have variations. |
To see which image variations are available for a given Bynder asset, look in the asset details view:
See Using Bynder image derivatives for more information.
For more information about Bynder derivatives, see Derivatives (Create Smaller Versions of Assets) in the Bynder documentation.
We do not support image variations for S3 assets. You must upload different versions of the same image if you need to use variations. |
Improved caching of S3 bucket regions
We have improved S3 caching of bucket regions in Magnolia to fix an issue causing performance problems when using the delivery endpoint for Cloud customers.
Name column filters
When selecting external assets, you can now use a filter at the top of the Name column for both Bynder and S3 to reduce the number of assets displayed.
Note that in the case of S3, you must open the bucket of your choice before filtering within it. Filtering at the root level does not produce results.
AWS Regions are listed by code and unavailable regions are not displayed
When creating an S3 bucket in the Assets app, the list of regions available are ordered alphabetically and unavailable regions are no longer displayed.
For more information about AWS disabled regions, see Managing AWS Regions |
Demo Text and External Image component removed
Previously the DAM Connector Pack provided a sample Text and External Image component in the Travel demo to help get users started using external assets in pages.
This sample component has now been removed. We recommend you use the standard MTK Text and Image component instead.
S3 asset names can no longer be edited
To avoid S3 asset link references breaking, it is no longer possible to rename S3 assets in Magnolia.
The Asset name field is now disabled in the asset details view.
Bug fix
-
Sorting by name does not work for S3 and Bynder. EXTDAM-150
External DAM module 1.0.8
Released on Oct 27, 2021.
This release improves S3 caching of bucket regions in Magnolia to fix an issue causing performance problems when using the delivery endpoint for Cloud customers.
External DAM module 1.0.7
Released on June 1, 2021.
This release brings a number of improvements.
One asset provider per link field
From External DAM 1.0.7 (DAM module 3.0.8), you can configure a single asset provider for link fields to limit your users to selecting assets from only one asset provider.
Previously, you had to use the aggregated damLinkField
and users had to choose from all the enabled asset providers.
External DAM module 1.0.6
Released on April 26, 2021.
This release brings a number of improvements and bug fixes.
Bynder and AWS S3 subapps disabled by default
By default, the Bynder and AWS S3 subapps are disabled and do not appear in the Magnolia Magnolia Assets app.
You must enable the subapp for your external solution <dam-connector>/decorations/dam
.
When upgrading from an earlier version of External DAM, subapps that were previously displayed are disabled until you enable them again. |
AWS default credential provider chain
As an alternative to using the static Magnolia Password manager app to store your AWS S3 credentials, you can now use the AWS default credential provider chain.
External DAM module 1.0.5
Released on March 30, 2021.
This release delivers a third party library update:
-
Kotlin Standard Library updated to 1.4.21.
External DAM module 1.0.4
Released on January 28, 2021.
This release brings a number of improvements and bug fixes.
Proxy service to protect private asset URLS (Bynder)
To protect the URLs of private Bynder assets rendered in Magnolia, you must use the proxy service provided.
When using the proxy, if you want to make a Bynder asset public, for example an image asset used in the Pages app, you must publish the image as well as the page in Magnolia.
You can enable or disable the proxy service using the proxyMode
parameter.
The proxy service is only available with Magnolia Core 6.2.6. If you use an earlier version of Magnolia Core, you must disable the proxy service. |
See EXTDAM-160.
Publish S3 assets from within Magnolia
You can now publish S3 assets via the Magnolia UI. By default, all S3 assets are private.
If you want to use S3 assets on a public page, you must publish them to make them public.
See EXTDAM-63.
Enable previewing for S3 assets
When you upload assets via Magnolia to S3, they are set to private by default. If you want to keep the preview functionality available, you must use presigned urls.
See EXTDAM-157.
Folder paths for AWS S3
Your location in the S3 folder hierarchy is now displayed in the bottom left-hand corner of the asset browser.
See EXTDAM-190.
Bug fix: User sessions overlap when browsing S3 assets
When you have two or more users trying to browse assets in S3 the sessions can overlap each other. Users may enter the app in a folder opened by another user in another session instead of the top level folder.
See EXTDAM-210.
External DAM module 1.0.3
Released on December 1, 2020.
This release brings a number of improvements and bug fixes.
Bynder SDK 2.2.1
The Bynder SDK version has been reverted from 2.2.4 to 2.2.1.
This resolves the issue discovered in version External DAM 1.0.3 that blocks users from uploading assets to Bynder via Magnolia.
See EXTDAM-189.
Publish Bynder assets from within Magnolia
You can now publish Bynder assets via the Magnolia UI.
By default, all Bynder assets are private. If you want to use Bynder assets on a public page, you must publish them to make them public.
Once Bynder assets are marked as published they cannot be unpublished or made private again in Magnolia. |
See EXTDAM-62.
External DAM module 1.0.2
Released on November 18, 2020.
This release brings compatibility for Magnolia 6.2.4 and a number of improvements and bug fixes.
We have discovered an issue with Bynder SDK version 2.2.4 and External DAM 1.0.2 that blocks users from uploading assets to Bynder via Magnolia. Do not use this release if you are a Bynder connector user. External DAM 1.0.3 corrects the issue and will be released soon. |
Improved handling of large numbers of S3 assets
Magnolia has improved how large numbers (thousands) of assets in a given S3 folder are rendered.
Magnolia has a specialized method that counts the total number of assets in your S3 system instead of creating the corresponding internal objects. Since S3 only allows batches of 1000 assets to be retrieved in a given call, we iterate as many API calls as necessary to retrieve the assets stored in S3.
For example, Magnolia must make 100 API calls to retrieve 100 000 assets. Depending on your internet bandwidth, cache and other factors, this could take around 1 minute to complete. For better response times, we recommend that you store a maximum of 10 000 assets per S3 bucket.
See EXTDAM-170 and EXTDAM-138.
Configure the maximum search time for S3
You can configure the maximum time Magnolia Periscope spends searching for S3 assets to be displayed in the Magnolia interface using a new parameter called maxSearchTimeInMilis
under dam-s3/config.yaml
.
By default, the maximum search time is 10 seconds (maxSearchTimeInMilis
: 10000
).
See EXTDAM-184.
Mime types
If you upload an asset that does not have a suffix, Magnolia now infers the mime type from the binary.
To avoid extra API calls to S3, we recommend you include the suffix whenever possible.
See EXTDAM-126.
Bug fixes
See EXTDAM-167.
External DAM module 1.0.1
Released on June 22nd, 2020.
This release brings compatibility for Magnolia 6.2.1 and a number of minor improvements and bug fixes:
-
EXTDAM-105 - (Bynder) Allow upload another image when editing
-
EXTDAM-106 - (Bynder) Set the asset’s name with the name of the upload file
-
EXTDAM-121 - Cache is not applied when browsing through app-chooser
-
EXTDAM-119 - Adjust to Periscope changes in 6.2.1
-
EXTDAM-104 - Show notification to the user when an async action started
-
EXTDAM-108 - (Bynder) Correct notify message with apostrophe
-
EXTDAM-112 - (Bynder) Edit asset fails with exception
-
EXTDAM-113 - (Bynder) Changing the image is not reflected in the tree unless the name is updated
-
EXTDAM-114- Periscope search is case-sensitive
-
EXTDAM-115 - (S3) Double-click on bucket or folder opens the editor
-
EXTDAM-116 - (S3) Strange behavior deleting a folder
-
EXTDAM-120 - (Bynder) The item deletion does not happen immediately
External DAM module 1.0
Released on April 6, 2020.
Initial release of the DAM Connector Pack for integrating with Bynder and Amazon S3, so that you can use your assets directly in Magnolia.
See DAM Connector Pack for an overview of the functionality provided.
DAM Connector Pack compatibility
Module version | Magnolia CMS version |
---|---|
1.1.0+ |
6.2.13+ |
1.0.3+ |
6.2.4 |
1.0.2 |
6.2.4 |
1.0.1 |
6.2.1 |
1.0 |
6.2 |
A note about connector compatibilty The third-party versions listed below are the versions Magnolia has developed and tested against. If you don’t see a particular version listed, it simply means that we do not routinely test on it. If you are using another version of the third-party tool and run into compatibility issues, please contact our Support. |
Module version | Bynder SDK |
---|---|
1.0.4 |
Tested against 2.2.1 |
1.0 |
Tested against 2.2.0 |
Module version | AWS Java SDK |
---|---|
1.0.1 |
2.13.25 |
1.0 |
Tested against 2.9.7 |