Jetspeed Distributed Cache Configuration

Jetspeed can be optionally configured to use distributed cache replication with ehcache. By default, distributed cache replication is disabled but it can be enabled through the Jetspeed configuration. The assembly folder contains cache.xml to control the ehcache configuration via Spring.

Jetspeed uses cache-replication for managing all of its distributed caches, such as the page-manager and preferences caches. The ehcache.xml (found in /WEB-INF/classes/) has more detailed cache configuration details. It has been modified to enable distributed cache replication for all caches including the page manager and preferences caches.

The following examples are taken from the distributed-ehcache.xml (in /WEB-INF/classes/). If you want to switch to complete distributed cache usage, it is easier to swap the usages of ehache.xml with the distributed.ehcache.xml completely in cache.xml (in /WEB-INF/assembly/) as shown below.

<!-- Cache Manager -->
<bean id="cacheManagerConfig" class="org.apache.jetspeed.cache.impl.EhCacheConfigResource">
	<!-- <property name="defaultConfigResource" value="distributed-ehcache.xml"/> -->
	<property name="defaultConfigResource" value="ehcache.xml"/>
</bean>

The Listener and Provider factory caches are required for cache replication. You shouldn't have to enter values in this file but instead use the jetspeed override.properties to set them. Below are parameters which can be set for configuring the distributed cache. The port and hostname can be the same for all nodes in the cluster under most default configurations.

<cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
                                 properties="hostName=${org.apache.jetspeed.ehcache.hostname},
                                 port=${org.apache.jetspeed.ehcache.port}"/>

<cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
                                 properties="peerDiscovery=automatic,
                                 multicastGroupAddress=${org.apache.jetspeed.ehcache.group.address},
                                 multicastGroupPort=${org.apache.jetspeed.ehcache.group.port},
                                 timeToLive=${org.apache.jetspeed.ehcache.group.ttl}"/>

<cache name="preferencesCache"
       maxElementsInMemory="10000"
       maxElementsOnDisk="1000"
       eternal="false"
       overflowToDisk="false"
       timeToIdleSeconds="28800"
       timeToLiveSeconds="28800"
       memoryStoreEvictionPolicy="LFU">
	<cacheEventListenerFactory
	        class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
	        properties="replicateAsynchronously=true, replicatePuts=false,
	                    replicateUpdates=false, replicateUpdatesViaCopy=false,
	                    replicateRemovals=true"/>
 </cache>

<cache name="pageManagerPathCache"
       maxElementsInMemory="${org.apache.jetspeed.ehcache.pagemanager.maxelements}"
       eternal="false"
       overflowToDisk="false"
       timeToIdleSeconds="${org.apache.jetspeed.ehcache.pagemanager.element.ttl}"
       timeToLiveSeconds="${org.apache.jetspeed.ehcache.pagemanager.element.ttl}"
       memoryStoreEvictionPolicy="LFU">
	<cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
	                           properties="replicateAsynchronously=true,
	                                       replicatePuts=false,
	                                       replicateUpdates=false,
	                                       replicateUpdatesViaCopy=false,
	                                       replicateRemovals=true"/>
</cache>

    <!-- Cache Manager -->
    <bean id="cacheManagerConfig" class="org.apache.jetspeed.cache.impl.EhCacheConfigResource">
        <!-- <property name="defaultConfigResource" value="distributed-ehcache.xml"/> -->
        <property name="defaultConfigResource" value="ehcache${server.ref}.xml"/>
    </bean> 	

java .... -Dserver.ref=_NODE56 

Testing Jetspeed Distributed Cache

First, verify that the following page-manager test works on your development machine:

mvn test -P test -Dtest=TestDatabasePageManagerCache

Not only must this test pass, but the following message should not appear in the console/shell or in the test log files:

"Server page managers not distributed: possible system limitation... test skipped"

If you see the above message, check firewall settings to allow UDP/4446. That is sometimes needed on Lunix systems if firewalls are enabled to allow UDP multicast loopback. Some older TCP stacks do not support UDP multicast loopback at all. This is why this test does not outright fail even though distributed caches can not be setup.

Now for the full test: setup two independent tomcat J2 images. Edit webapps/jetspeed/WEB-INF/assembly/cache.xml for Tomcat image #1:

<bean id="cacheManagerConfig" class="org.apache.jetspeed.cache.impl.EhCacheConfigResource">
    <property name="defaultConfigResource" value="distributed-ehcache.xml"/>
    <property name="defaultHostname" value="localhost"/>
    <property name="defaultPort" value="40001"/>
</bean>

Edit webapps/jetspeed/WEB-INF/assembly/cache.xml for Tomcat image #2:

<bean id="cacheManagerConfig" class="org.apache.jetspeed.cache.impl.EhCacheConfigResource">
    <property name="defaultConfigResource" value="distributed-ehcache.xml"/>
    <property name="defaultHostname" value="localhost"/>
    <property name="defaultPort" value="40002"/>
</bean>

Startup browsers against each server. Use different host names to ensure cookies are managed separately (i.e. '127.0.0.1' and 'localhost'). View the same page in each browser. Edit the page in one browser. Upon refresh, changes should be visible in second browser.

When distributed caches are run on multiple virtual or physical servers, the default port assigned above does not need to be incremented for each machine. If the servers are deployed on multiple networks, care must be taken to ensure multicast traffic on UDP/4446/230.0.0.1 and TCP/40001 is routed between the servers. Additionally, multicast propagation TTL may have to be specified to enable the UDP packets to jump between networks. The default value is "1", which is supposed to limit packets to one subnet. (See the org.apache.jetspeed.ehcache.group.ttl property).

Here is a fully specified cacheManagerConfig bean with its distributed cache default values for normal installation:

<bean id="cacheManagerConfig" class="org.apache.jetspeed.cache.impl.EhCacheConfigResource">
	<property name="defaultConfigResource" value="distributed-ehcache.xml"/>
	<property name="defaultGroupAddress" value="230.0.0.1"/>
	<property name="defaultGroupPort" value="4446"/>
	<property name="defaultGroupTTL" value="1"/>
	<property name="defaultHostname" value="localhost"/>
	<property name="defaultPort" value="40001"/>
</bean>

Any of these can be configured to reflect requirements of the production network.