Offline Manifests with FirstSpirit

cthomas · ‎12-13-2013

Isn’t it annoying when you need information from a website but can not get it because there is no internet connection available? Wouldn’t it be great if the browser was able to store the information once fetched locally? Well, it can!

This article shows a way to create offline manifests for your projects. Offline manifests - also called appcache or cache manifest - are a feature of HTML5 which instructs the browser on the client side to cache files from the internet on the local filesystem. The website performance is thus increased due to reduced loading times and it is possible to create web-applications that can be used even when there is no internet connection available.

Using the pre-generation capabilities of FirstSpirit it is a breeze to create the necessary files, just follow our step-by-step guide. Before we start, let us highlight some examples of where the offline manifest comes in handy:

Web applications that should also work offline.
Increase website performance by caching non dynamic content, like media files or JavaScript files.
Download and store big JSON or XML files, for example for a glossary look up.
Prepare a website demonstration (like a clickdummy) for a tablet where it must be ensured that the presentation works even without network connection.
Set up an interactive kiosk and decrease the maintenance effort by using 3 lines of JavaScript to let the interactive kiosk update itself whenever a server-connection is available.

Step-by-Step

Following this easy step-by-step guide you will be up and running with your offline web-application in no time. Basically it is only the manifest file which needs to be created using some clever logic.

Take into account that this is only one way of doing things. If you are creating offline manifests in another way, please share your approach in the comments.

Step 1: Create a page template for the manifest file

Create a new page template with the following content in the template store. This template is the base for the manifest file which is generated in a later step. Give the template the reference name “appcache” as this is going to be used in the script mentioned in step 5.

Form

The form has only one input component that is used to select a fallback page. The fallback page is shown when the user navigates to a page which is not stored offline. It makes sense to create a custom page for this - similar to a 404 page - that informs the user about the issue. You should be using absolute paths for links and media elements on the fallback page as the page is delivered in the context of the missing page, not from its own location, so relative paths might be wrong.

<CMS_MODULE>
  <FS_REFERENCE name="pt_fallback" hFill="yes" sections="no" upload="no" useLanguages="no">
    <FILTER>
      <ALLOW type="pageref"/>
    </FILTER>
    <LANGINFOS>
      <LANGINFO lang="*" label="Fallback page"/>
    </LANGINFOS>
    <PROJECTS>
      <LOCAL name=".">
        <SOURCES>
          <FOLDER name="root" store="sitestore"/>
        </SOURCES>
      </LOCAL>
    </PROJECTS>
  </FS_REFERENCE>
</CMS_MODULE>

Properties

On the properties pane, change the file extension for the HTML set to “mf” for example. You are free to use a different extension but need to make sure that the MIME type is correctly set as described below.

Make sure that the webserver delivers files using the .mf extension with the MIME type “text/cache-manifest”. Please refer to your webserver documentation on how to set this property. If you want to use the cache manifest with the internal Jetty of FirstSpirit you can set the property in the file [FS_ROOT]/server/jetty/webdefault.xml.

HTML

The HTML part ensures that the manifest file will only generate one time and only if dv_useAppcache is true (we will later see why). The timestamp is used to give a hint to the browser that the manifest has changed and it need to update the files stored locally.

$CMS_IF(!isSet(dv_useAppcache) || !dv_useAppcache.equals("true") || #global.language != #global.project.getMasterLanguage())$$CMS_SET(#global.stopGenerate, true)$$CMS_END_IF$CACHE MANIFEST
# $CMS_VALUE(#startdate.format("dd.MM.yyyy HH:mm:ss.SSS"))$

Step 2: Add a page and pageref

Now, as the template is all set up, create a page based on it in the page store and add a reference to the page in the site structure. The page reference should be called appcache as we are going to reference it later on.

Step 3: Link the manifest file with your website

To instruct the browser to store everything defined in the manifest an attribute needs to be added to the <HTML>-tag of all pages that should trigger the download. It is possible to add the attribute on all pages but generally sufficient to add it only to the hompage. Simply replace the <HTML>-tag as follows considering further attributes that may already be present.

<html$CMS_IF(isSet(dv_useAppcache) && dv_useAppcache.equals("true"))$ manifest="$CMS_REF(pageref:"appcache", lang:#global.project.masterLanguage)$"$CMS_END_IF$>

Again you see the use of the dv_useAppcache variable which is explained in the next step. In addition the pageref appcache created is used here as a reference to the manifest.

Step 4: Create a schedule entry

It is finally time to assemble the pieces and create a schedule entry in the ServerManager (aka Server- and project configuration) that is used to generate and deploy the offline version of the project.

For the sake of simplicity it is best to copy an already existing deployment schedule entry and edit it. Switch to the Actions tab, edit the generation action, switch to the Extended tab and add the variable dv_useAppcache with the value true.

Now you see what the variable dv_useAppache we talked about before is used for: It simply triggers wether or not the offline version is generated.

Step 5: Generate the content of the manifest file

We are almost there! We are only missing a way to actually generate the content of the manifest file. To do this please see the attached script offline_manifest.bsh which needs to be added as a new script action to the schedule task created in step 4. The script needs to be added directly after the generation action.

If you want to learn more about the “magic” of the script please refer to the comments contained in it.

Ready, test, go!

Make sure that the generation task includes the manifest file and the fallback page, then execute the task. After a successful deployment you can visit the webserver serving the pages and browse to one of the pages having the <HTML>-tag for the manifest set, as described in step 3. The browser will now download all contents specified in the manifest in the background. Give it a couple of seconds to finish the task, you can watch the progress in the console of your browser.

Finally disconnect from the net and browse your website. You will see that all content specified in the manifest are available offline and when you browse to a page that is not mentioned in the manifest you are being redirected to the fallback page specified.

The offline manifest is a browser feature underlying security permissions so it is sometimes necessary to test and debug what you have done when things are not working as expected. The easiest way is to do that in Google Chrome with activated DevTools (Shortcut: F12) and/or Mozilla’s Firefox (Shortcut: F12). In Google Chrome you can see information about appcached elements and their metadata by visiting chrome://appcache-internals/.

Additional Resources

Appcache Facts - Facts around offline manifests and other resources.
W3C Editor’s Draft for Offline Web applications
Wikipedia article on cache manifests

As stated earlier, this is but one way of creating the cache manifest and there are many more. Are you already using a feature like this in one of your projects? Please share your approach with us in the comments to get a bigger picture.