DeltaGeneration - Get the generated or deleted Files

[MichaelaReydt]

As described in the first blogposting about the DeltaGeneration , FirstSpirit 5 introduces a new mechanism, called DeltaGeneration, which simplifies things a lot.

The DeltaGeneration makes it possible to find out and to generate just the elements, which were changed since the last successful generation. How a DeltaGeneration can be configured is explained in the linked blogposting. (It is recommended to read the linked blogposting to understand the current blogposting.)

The information, which files were changed or deleted since the last generation, could be necessary - for example - to inform search engines and to let them update their search indices. Another use case might be to replace the FullGeneration by the DeltaGeneration. In this case the webserver needs the information which files were changed or deleted, because the synchronization of all files (as in the FullGeneration) won't be executed.

The DeltaGeneration offers commands to find out these files. Therefore the schedule-script of the DeltaGeneration has to be extended.

Generated Files

To get a list of all generated files the following command has to be added at the end of the schedule-script after the execution of the generate task:

changeSet.configureGenerateTask();
generatedFilesList = DeploymentUtil.getGeneratedFiles(context);

Note

If the checkbox "Clear generation directory beforehand" inside the dialog of the generation task isn't activated, this command will return all files, which are currently located within the generation folder.

If the list should only contain the files, which has been generated during the DeltaGeneration, the checkbox has to be activated.

With an iteration over this list of generatedFiles the several entries can be separated and for example be printed to the logfile.

generatedFilesList = DeploymentUtil.getGeneratedFiles(context);
it = generatedFilesList.iterator();

if(it.hasNext()) {
   context.logInfo("Generated files found.");
}
else {
   context.logInfo("No generated files found.");
}
 
while(it.hasNext()) {
   generatedFile = it.next();
   context.logInfo(" generatedFile: " + generatedFile);
}

Deleted Files

Such a list is also available for the deleted files. Therefore the following line has to be added to the script:

changeSet.configureGenerateTask();
deletedPageInfo = changeSet.getDeletedPageInfos();

Note

This line just works when persistent pages were deleted. So it is necessary to generate the paths for the pages inside the project by a SEO URL-Creator.

Further information to SEO URLs are available in the Release Notes for FirstSpirit 5.0.

The iteration in this step is equivalent to the iteration over the list of generatedFiles. Just the expression inside the loop isn't analog:

deletedPageInfos = changeSet.getDeletedPageInfos();
it = deletedPageInfos.iterator();

if(it.hasNext()) {
context.logInfo("Information for deleted Pages found.");
}
else {
  context.logInfo("No Information for deleted Pages found.");
}
 
while(it.hasNext()) {
  deletedPageInfo = it.next();
  context.logInfo("deletedPageInfo nodeId " + deletedPageInfo.getNodeId() 
  + " path " + deletedPageInfo.getPath());
}

Logfile-Information

After elements were changed inside a project and the schedule-task was executed the information of the generated or deleted files are displayed in the script-logfile.

Exemplary the logfile-information for the generated files:

code-snippet from the schedule-script (see above):

if(it.hasNext()) {
 context.logInfo("Generated files found.");
}
[...]
while(it.hasNext()) {
 generatedFile = it.next();
 context.logInfo(" generatedFile: " + generatedFile);
}

logfile-information:

INFO  04.04.2013 09:42:43.459 (de.espirit.firstspirit.impl.access.ScriptContextImpl): Generated files found.
INFO  04.04.2013 09:42:43.459 (de.espirit.firstspirit.impl.access.ScriptContextImpl): generatedFile: /de/example.html
INFO  04.04.2013 09:42:43.461 (de.espirit.firstspirit.impl.access.ScriptContextImpl): generatedFile: /en/example.html

API-Classes

The classes, which are needed for a DeltaGeneration, can be found inside the Developer-API.

Their names are:

• DeploymentUtil (includes the method "getGeneratedFiles")
• DeltaGeneration.ChangeSet (includes the method "getDeletedPageInfos")
• DeltaGeneration

[linde]

Very interesting feature. I tried it in a standard Mithras energy project. After changing a page in the pagestore the 2 pagrefs were generated (de and en), very good. The list of generated Files always contains all generated HTML pages. This seems to be wrong. I tried is in a script task before the genration task and after the generation task with the same result. I expected to get a list with 4 files (2 pagerefs 2 languages). Waht's worng there?

[feddersen]

Have you enabled the "Clear generation directory beforehand" checkbox as mentioned in post?

[linde]

That was the reason, the checkbox was disabled. The function seems to return a file list of the generation directory. I tried some release scenarios with pages, pagerefs an and pagereffolder. After releasing a pagereffolder the generation task was executed but nothing was generated, so changes in the navigation structure were not deployed. How can I consider this scenario in a delta deployment?

[feddersen]

Have you set appropriate leveRules, as described in the first blog post? See DeltaGeneration - What is that and why I might need it?

[linde]

I haven't seen the description of the level rules. I will try this. Thanks.

[vasanth]

I have follwoed the steps in the DeltaGeneration - What is that and why I might need it? as well as the above steps. For the first its works fine but for the second time i didn't change any file started the deployment but still displays the old generated file information.

Is there any thing i am missing ???

Thanks

[feddersen]

Have you enabled the "Clear generation directory beforehand" checkbox?

[rrichter]

Hi,

I tried the following DeploymentSchedule with three Actions

• Prepare Delta
• Generate Delta
• Postprocess Delta

"Prepare Delta" consists of the following lines

import de.espirit.firstspirit.access.schedule.*;
 
deltaGeneration = DeploymentUtil.createDeltaGeneration(context);
changeSet = deltaGeneration.calculateChangeSet();
changeSet.configureGenerateTask();
 
context.logInfo("***** isFullGenerate: " + changeSet.isFullGenerate());

"Generate Delta" is a generation task with the options described above (including the option "Default URLs (SEO)"). Works well with the nodes set by the "Prepare Delta"-Action

In "Postprocess Delta" I calculate the generated files (works well).

But I am not able to calculate the deletedPageInfos in this step, because I would have to execute another calculateChangeSet() that destroys the info from the "Prepare Delta"-Task.

Is there something wrong, or do I have to calculate the deletedPageInfos in the "Prepare Delta"-Task?

What I want to do is to add a file "to-delete.props" to the generation directory that contains the filenames of the files that have to be deleted.

Is this possible in the "Postprocess Delta"-Task?

Regards,

(a coding colleague of) Raphael.

[feddersen]

If you need the getDeletedPageInfos after the generation, retrieve it in the "prepare delta" task and write the result into the schedule context, so that you can read it again in your "postprocess delta" task.

[rrichter]

... my colleague is for ever and ever indebted to you.

The "Prepare Delta" now contains the following lines:

import de.espirit.firstspirit.access.schedule.*;
 
deltaGeneration = DeploymentUtil.createDeltaGeneration(context);
changeSet = deltaGeneration.calculateChangeSet();
changeSet.configureGenerateTask();
context.setProperty("deletedPageInfos",changeSet.getDeletedPageInfos());

The "Postprocess Delta" contains the following lines:

import de.espirit.firstspirit.access.schedule.*;
generatedFiles = DeploymentUtil.getGeneratedFiles(context);
it = generatedFiles.iterator();

if(it.hasNext()) {
context.logInfo("Found some generatedFiles.");
}
else {
  context.logInfo("No generatedFiles found.");
}
 
while(it.hasNext()) {
generatedFile = it.next();
  // do some dirty things with the info about the generated file
}

deletedPageInfos = context.getProperty("deletedPageInfos");

it = deletedPageInfos.iterator();
if(it.hasNext()) {
  context.logInfo("Some deletedPages found.");
}
else {
  context.logInfo("No deletedPages found.");
}
while(it.hasNext()) {
  deletedPageInfo = it.next();
  // do some nasty things with the info about the deleted file 
}

It works. Thanks.

[vasanth]

Thank you Christoph Feddersen. Its working fine

[lewandowskil]

Hi,

When I try to delete one site, which was generated in, e.g., 2 languages and afterwards try to delete this site I iterate over deletedPageInfos, but it delivers only the path to my master language, e.g., /de/site/test.html. I expected to receive also /en/site/test.html.

May I missed anything?

Regards

Lukas

[lewandowskil]

Sorry, my generate configuration was incorrect. Works now as expected.

[markus_reinsch]

Because of a known bug the query for the deletedPageInfos returns no result at the moment. This posting will be updated after a fix of the bug.

Due to the comments the bug seems to be fixed.

Can you please provide the exact FS version for this fix?

[feddersen]

The bug has been fixed with 5.0.317, so you should be fine with the current release (5.0.318).

[bIT_sosswald]

Hi,

is there a possibility to retrieve not only the path to the generated files but also the UID of the pageRef or something? The method getGeneratedFiles only returns a list of strings with the path to the generated files.

For deleted files it is at least possible to retrieve the Sitestore node id for each deleted page. (See here)

After a delta generation was performed my module has to do some processing of data which it has to extract out of the pageRefs of the generated pages.

Greetings

Sandro

[bIT_sosswald]

See here and here.

[Geetha]

Hello,

Good day

Iam trying to do partial deployment using delta generation.Please help me .

Question 1:

I have modified the EN languages and excecuted the script for

• Generate Delta
• Postprocess Delta

But it contains both EN and DE generatedFiles.How can i set only for EN ~language ???

Question 2:

UseCase:

Added one reference page in site store and exceuted the script .Works fine.

Again aded the same reference page and excecuted the script.It throws Error as below

URI conflict detected for SSNodeEntry-2525948-1909967-EN-, unified URI:

Question 3:

Is there any method or api for partailDeployment like

context.logInfo("***** isFullGenerate: " + changeSet.isFullGenerate());

Thanks in Advance!