ddukers
I'm new here

Partial generation for 1 datarecord of an datasource starts generation for all records

Hi,

In our project we have a news datasource and created a newsdetail page and 2 news overview pages. A workflow is used to release and publish newsitems to our site.

We use a partial generation step to only generated the details page for the one item plus the overview pages. The workflow uses a java class to release dependent media content and determine the needed startNodes for the partial generation.  These startNodes are then supplied to the generation task of the scheduled entry.

This all seems to work just fine when editing and publishing a single item. In the generation log you can see 1 newsdetail page plus the overview pages are being generated.

However it was noticed that sometimes it would still generate the newsdetails pages for all available newsitems (this could result in severall hundreds of pages).

At first it seemed to be fairly random but we managed to find a good reproduction path. It seems to be related to publishing 2 items in sequence.

For the first item the generation will only process the needed pages as expected. The second scheduledEntry will be submitted while the generation of pages for the previous news item are still being generated (as seen on the server monitoring page under activities). The schedule is configured to not allow for parallel execution. The scheduled entry for the second news item will be listed under 'planned scheduled entries' on the server monitoring page.

Once the processing of the first item is finished this second scheduled entry is picked up. The generation task in this scheduled entry will generate all the details pages instead of just a single detail page.

I already checked the java class to verify the startNodes supplied to the generation task as we first thought the wrong (or too many starting nodes were provided to the scheduled entry).

The supplied starting nodes for both sheduled entries look similair (see below) and would not explain this behaviour.

startNodes first scheduled entry:

[<PAGEREF editor="485027" htmlname="nieuwsdetails_1" id="60817" pageref="60744" releaseRevision="1919884" releasedby="485027" revision="1919884" uniquedescription="nieuwsdetails_1">

    <LANG displayname="Nieuwsdetails" language="NL"/>

    <CONTENTPARAMETER count="1" query="362914" source="News" templateid="207">

        <PARAM class="java.lang.Integer" name="row_id" value="24077"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

, <PAGEREF editor="70970" htmlname="nieuwsoverzicht_1" id="60818" pageref="60740" releaseRevision="630673" releasedby="128" revision="630673" uniquedescription="nieuwsoverzicht_1">

    <LANG displayname="Nieuwsoverzicht" language="NL"/>

    <CONTENTPARAMETER count="25" query="2528" source="News" templateid="228">

        <ORDER asc="0" name="publication_date"/>

        <PARAM class="java.lang.String" name="department" value="AZ"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

, <PAGEREF editor="2489" htmlname="rijksnews_1" id="480980" pageref="479606" releaseRevision="1917407" releasedby="128" revision="1917407" uniquedescription="rijksnews_1">

    <LANG displayname="News AZ XML" language="NL"/>

    <CONTENTPARAMETER count="10" pagecount="1" query="378303" source="News" templateid="43597">

        <PARAM class="java.lang.String" name="department" value="AZ"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

]

startNodes second scheduled entry:

[<PAGEREF editor="485027" htmlname="nieuwsdetails_1" id="60817" pageref="60744" releaseRevision="1919900" releasedby="485027" revision="1919900" uniquedescription="nieuwsdetails_1">

    <LANG displayname="Nieuwsdetails" language="NL"/>

    <CONTENTPARAMETER count="1" query="362914" source="News" templateid="207">

        <PARAM class="java.lang.Integer" name="row_id" value="24080"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

, <PAGEREF editor="70970" htmlname="nieuwsoverzicht_1" id="60818" pageref="60740" releaseRevision="630673" releasedby="128" revision="630673" uniquedescription="nieuwsoverzicht_1">

    <LANG displayname="Nieuwsoverzicht" language="NL"/>

    <CONTENTPARAMETER count="25" query="2528" source="News" templateid="228">

        <ORDER asc="0" name="publication_date"/>

        <PARAM class="java.lang.String" name="department" value="AZ"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

, <PAGEREF editor="2489" htmlname="rijksnews_1" id="480980" pageref="479606" releaseRevision="1917407" releasedby="128" revision="1917407" uniquedescription="rijksnews_1">

    <LANG displayname="News AZ XML" language="NL"/>

    <CONTENTPARAMETER count="10" pagecount="1" query="378303" source="News" templateid="43597">

        <PARAM class="java.lang.String" name="department" value="AZ"/>

    </CONTENTPARAMETER>

    <PAGE_LANG_SPEC language="NL" showinpagegrp="1" showinsitemap="1"/>

</PAGEREF>

]

Is there any documentation available on the generation process (especially about partial generation)? I would like to get a better understanding of how this proces works (for example how does it decides which pages to generate based on the starting nodes).

According the help file the "FirstSpirit Manual for Developers (Advanced)" should have a topic on partial generation but this document does not seem to be available yet.

Also i would like to know if it would be possible in a way to 'debug' the generation action on the server.

Regards,

Daniel

0 Kudos
1 Reply
ddukers
I'm new here

resolved the issue by a rework of the publication proces.  In the new publication proces we do not use the newsdetail pageRef but a copy of this pageRef.  After the generation the specific newsdetail page has to be renamed to ensure the filename is correct.

After the publication the copy of the pageRef is again removed.

This prevents the modification of the query by another publication.