Collecting Portal Data

PortalStatistics provides APIs for collecting portal application events. The following event types are supported out of the box:

• Portlet rendering statistics.
• Page rendering statistics.
• User access statistics.

    %h %l %u %t \"%r\" %>s %b

• %h - remote host
• %l - remote log name
• %u - remote user
• %t - time in common log time format
• %r - first line of HTTP request
• %s - HTTP status code
• %b - number of bytes sent ("-" if no bytes sent).

Statistics data is stored in 3 relational tables as illustrated below:

Each event type is mapped to a data holder that gets persisted in the statistics tables illustrated above.
Jetspeed-2 persists the collected data in batches through the BatchedStatistics component.

PortalStatistics Configuration

The PortalStatistics component is configured in WEB-INF/assembly/statistics.xml and requires the following configuration parameters:

• logToCLF: Whether to log to a Common Log Format file.
• logToDatabase: Whether to record the statistics in the database.
• maxRecordToFlush_Portlet: The maximum number of portlet events that can be in the portlet statistics FIFO before a flush occurs.
• maxRecordToFlush_User: The maximum number of user events that can be in the user statistics FIFO before a flush occurs.
• maxRecordToFlush_Page: The maximum number of page events that can be in the page statistics FIFO before a flush occurs.
• maxTimeMsToFlush_Portlet: The maximum length of time between FIFO flushes for the portlet statistics FIFO.
• maxTimeMsToFlush_User: The maximum length of time between FIFO flushes for the user statistics FIFO.
• maxTimeMsToFlush_Page: The maximum length of time between FIFO flushes for the page statistics FIFO.
• jetspeedDSEntry: The reference to the Jetspeed data source.

The FIFO parameters may be adjusted depending on site specific needs. For example, tuning the events and times down makes the system less likely to loose events in the case of a failure, and also makes the user experience viewing the statistics more responsive to recent events. On the other hand, tuning them higher will reduce the overhead to the database. Generally newer sites with fewer users should use smaller tuning parameters and larger more mature sites should use larger ones.

A sample configuration is provided below:

    <bean id="PortalStatistics" 
      class="org.apache.jetspeed.statistics.impl.PortalStatisticsImpl"
      init-method="springInit"
      destroy-method="springDestroy">
      <!-- logToCLF -->
      <constructor-arg index='0' type="boolean"><value>false</value></constructor-arg>
      <!-- logToDatabase -->
      <constructor-arg index='1' type="boolean"><value>true</value></constructor-arg>	
      <!-- maxRecordToFlush_Portal -->
      <constructor-arg index='2'><value>300</value></constructor-arg>
      <!-- maxRecordToFlush_User -->
      <constructor-arg index='3'><value>50</value></constructor-arg>	
      <!-- maxRecordToFlush_Page -->
      <constructor-arg index='4'><value>100</value></constructor-arg>	
      <!-- maxTimeMsToFlush_Portal -->
      <constructor-arg  index='5'><value>300000</value></constructor-arg>		
      <!-- maxTimeMsToFlush_User -->
      <constructor-arg  index='6'><value>5000</value></constructor-arg>
      <!-- maxTimeMsToFlush_Page -->
      <constructor-arg  index='7><value>60000</value></constructor-arg>
      <!-- jetspeedDSEntry -->
      <constructor-arg  index='8'><ref bean="JetspeedDS"/></constructor-arg>
    </bean>

Where are the Event Recorded?

Portal statistics events collection is injected at multiple points in the portal engine. Some examples are provided below.

For user events, the SecurityValveImpl logs a user login event in getSubject(RequestContext request).

    statistics.logUserLogin(request, 0);

For portlet events, the RenderingJobImpl invoked in RenderingJobbuildRenderingJob logs a portlet access event.

    statistics.logPortletAccess(requestContext, fragment.getName(), PortalStatistics.HTTP_OK, end - start);

Retrieving Aggregate Statistics

The PortalStatistics component exposes a queryStatistics method that given a StatisticsQueryCriteria will return AggregateStatistics.

    AggregateStatistics queryStatistics(StatisticsQueryCriteria criteria)

Using Aggregate Statistics

As illustrated in viewing statistics, Jetspeed-2 provides a default reporting portlet for view statistics. To query statistics, a StatisticsQueryCriteria must be set. According to this criteria the PortalStatisticsqueryStatistics() method will return an AggregateStatistics.

        StatisticsQueryCriteria criteria = statistics.createStatisticsQueryCriteria();
        ...
        criteria.setUser(user);
        criteria.setListsize("5");
        criteria.setSorttype("count");
        criteria.setSortorder("desc");
        criteria.setTimePeriod(timeperiod);
        criteria.setQueryType(queryType);
        AggregateStatistics stats = statistics.getDefaultEmptyAggregateStatistics();
        ...
        statistics.forceFlush();
        stats = statistics.queryStatistics(criteria);