In Crownpeak DQM, every Asset (HTML page, PDF Document etc.) has a page expiry set.
The page expiry allows DQM to crawl a site several times without seeing an asset before it is removed. There are occasions when some assets are not available because of a server issue, or because pages are temporarily taken down we scan a website. The page expiry feature can be set if you do not want to immediately remove these assets from a dashboard because the assets that are not available may not have been removed on purpose.
Previously the default page expiry is set to ensure that at least two scans are made before assets are removed. For example, for a dashboard that scans weekly, page expiry is usually set at 15 days to ensure that a scan is made on the 7th and 14th day. This means each asset will stay in our system for a minimum of 15 days, even if the asset is removed from the live site.
The default has now been updated to 5 days for weekly scans. This means that pages and other asset types are updated after every scan.
It is possible to see the page last scanned from the file details in page analyzer, this will provide you with an idea on how long the asset will stay before it is removed from the platform.
TipThere are cases where page URLs are changed or removed on purpose. The new URL will be treated as a new asset, and although the old URL does not exist anymore, it will stay in our system for the duration of the page expiry set on the dashboard before being removed. If a large amount of pages are removed it is possible for us to run a complete refresh of a dashboard to clear out these old URLs which is what we are doing now. It is also possible for us to update the page expiry depending on how often websites are updated. To do this please create a support case.