Crownpeak DXM provides a facility for external content or data to be imported from an SFTP source and brought into the content management lifecycle. SFTP is used so that this process can operate on a pull model, which is preferable from a performance and scalability point of view.
NOTE: We only support SFTP imports now. Support for FTP has been dropped for security reasons but there are legacy configurations fields are still visible.
SFTP Imports can be used to implement many different use cases. The first step is deciding what data will be imported and how the data will be used in the CMS. Because of the possibility to perform bulk operations in an import, it is common to import text-based data that has some structure, for example:
Delimited content -- commonly comma- or tab-separated
The import can also bring in binary file and create digital assets (non-templated assets).
Before you can configure the SFTP import, you will need to have a number of elements prepared.
A template that has an ftp_import.aspx handler
A model blueprint that uses your template
A folder where you import content will be placed -- this does not need to be the final location of content created during the import process.
At a minimum, the template needs to have the ftp_import.aspx handler defined. Although the template could have other handlers defined, it is highly recommended that you have a separate import template to the content template(s) your import will be used.
Once you have extracted fields, you can use Asset.CreateNewAsset to create new assets; or Asset.Load and Asset.SaveContent to update existing assets. What you can also do is treat each import as a single asset rather than as a batch, in which case you will be extracting fields and calling asset.SaveContent. You will almost certainly want to use asset.Rename as well.
Set the FTPInputContext AutoCreateAsset property to false if you don't want the FTP import process to create an asset for the underlying FTP import file. However, it can be useful to have this in the CMS so that if the import process fails or has errors, it can be re-run after any corrective action has been taken.
Once you have the import template ready, create a model blueprint that uses your template. The purpose of this model is to ensure that the asset that is created from the FTP import picks up the correct template and workflow settings.
There is nothing special about this folder, but it is needed during the configuration phase.
For batch imports, it is recommended that the folder is a generic container, "imports" for example. If you are importing individual assets, then the folder could of course be more descriptive, "articles" for example.
SFTP import configurations can be set up in the Settings > Publishing > SFTP Imports panel.
Click on the "Add FTP Import" button to create a new configuration, or click on an existing one to edit. Once you are editing a particular configuration, these are the properties you can edit:
The name of the import configuration. Used for reference purposes only.
Allows you to pick a date and time after which the import will not be triggered.
Pick the frequency the import configuration will be triggered by the DXM scheduler. Options range from 1 day to 5 minutes
Max number of files PER DAY
The maximum number of files that can be imported by this configuration in any given day. This is useful for limiting the import in the event there is a sudden content surge or a problem with the upstream system.
Max number of files
The maximum number of files this import configuration can import in total. Again, useful if you need to place a limit on the process to protect against runaway processes.
Owner of the imported files
Pick the existing CMS user that the new assets will appear to be created by.
Choose from one of the actions the import process should perform on the remote file after import. Most configurations will either move the file to a "remove" or processed folder so that the file isn't imported again on the next cycle; or will delete the remove file.
Remote "remove" folder
Required (if the "Move to remote remove folder" action is selected)
The name of the remote folder to move the import file to.
The hostname or IP address of the remote SFTP server.
SFTP Remote Folder
The folder to change to on the remote server to start lookng for import candidates.
If not specified, the process will use the default folder after login.
The username to use when authenticating with the SFTP server.
The password to use with the SFTP username when authenticating with the SFTP server.
This property must be checked as we no longer support FTP configurations.
If checked, this indicates that the import should NOT run the ftp_import.aspx handler but rather just create a digital asset with the imported file.
Send errors to email
An email address that a notification of any errors or faults in an import will be sent to.
Although not required, it is strongly recommened that you use this facility.
Once you have entered all the configuration, you can test the connection; or trigger an import usng the buttons at the bottom of the configuration panel. This is a crucial part of the development and configuration process so that you can be sure that all the information is correct and also that the DXM servers can establish a connection to the remote server.
Once you save the import configuration, DXM will generate a unique token for the configuration. This is visible in the import overview panel as well as the individual configuration edit panel. You will need this token in the next section.
Once you have the DXM configuration set up and have the generated token, you can start delivering files into the SFTP server folder. For text files, the filename must include the generated configuration token in order for DXM to match up files and import configurations, for example: articles-20190812-TK9qMId7zxgvGRSJ.xml
NOTE: If you have a binary import configuration, you do NOT need to have the token appear in the filename. For this reason, it is strongly recommended that digital assets for import are placed in their own folder on the SFTP server.