Using the JSR 362 TCK

The JSR 362 Technology Compliance Kit (TCK) is intended to be used by portal vendors wishing to achieve JSR 362 compatibility for their product. If you do not meet this description, you do not need to read this section.

Overview

The TCK contains portlets and a test driver that are used to perform the compatibility test. The portlets contain the code that actually exercises the portlet API. The driver uses Selenium 2 driven by Junit to access the system under test through a selenium webdriver browser.

You can click through the tests on the portal pages displayed by the system under test, or you can use the test driver to automatically click through the tests and collect the results. Since there are many test cases (>6000), you will generally use the test driver.

A test portlet may define multiple test cases. The test case names must be unique in the system, and must be of a form that can be used as portlet names and as Java Properties file keys.

Each portlet test module defines its own test case list and page file containing the page definitions expected by the tests contained in the module.

The build process generates a consolidated test case list and consolidated page file containing all test cases and page definitions for all test modules. The consolidated test case list maps the test case to the name of the page on which the test case is located.

The test driver works by reading the consolidated test case list. For each test case, the driver accesses the specified page. It logs in if necessary, clicks any available test case link, and collects the test case results from the browser window.

The test driver accesses a TCK page by looking for and clicking a link whose link text is equal to the page name. If the test driver cannot find the link on current page, it will access the test server login URL configured in the portlet-tck_3.0/POM.xml file.

The test driver checks whether a login is necessary by checking for the existence of an entry field that has the HTML element ID for the user name field. If the user name field is available, the driver enters the user name and password into the corresponding entry fields and submits the form. The username HTML element ID, password element ID, username and password are configurable in the portlet-tck_3.0/POM.xml file.

The driver checks for links to be clicked and results to be collected by checking for specific HTML element IDs on the page. The HTML element IDs are based on the test case name. From the point of view of the test driver there can be several relevant element IDs on the page for each test case:

  • "<testcasename>-setup" - Test setup link
  • "<testcasename>-clickme" - Test execution link
  • "<testcasename>-results" - Test results field; "Succeeded" or "Failed"
  • "<testcasename>-details" - Test details field; provides failure reason

After accessing the page on which the test case is to be found, the test driver first looks for a setup link. If found, it clicks the link and waits for the page to load.

The driver then looks for a test execution link. If found, it clicks the link, waits for the new page to load, and then checks for the result elements.

The driver reads the contents of the HTML elements that have the results and details element IDs and analyzes the results string to determine test case success or failure.

Project Structure

The TCK source code is located in the subdirectory ./pluto/portlet-tck_3.0. The project consists of the following modules:

common
Common code used by the test portlets and the test driver. This module also contains the XSL stylesheets used during the test portlet module build to generate the test case list and page configuration for the module.
deploy
The directory deploy/target/deploy-files contains the portlet war files and the page file to be deployed on the system under test.
driver
The maven-based test driver. When this module is built, it generates a consolidated test case list and page file for the test cases and pages defined by all test case portlet modules.
TestModule1
Intended to be a sample for writing TCK portlets. Defines several portlets, each of which is placed on its own page.
TestModule2
Intended to be a sample for writing TCK portlets. Defines several portlets, all of which are placed on the same portal page. Some tests produce links that are clicked by the test driver when the test is run. Some tests set public render parameters that are used by another test portlet.
TestModule3
Intended to be a sample for writing TCK portlets. Defines two portlets in separate portlet applications for public render parameter testing. Both portlets are placed on the same page. When the test driver clicks a test link in one portlet, the test results are displayed in the second portlet.
V2*Tests
Modules with this naming scheme contain test cases to demostrate backward compatibilty with the Portlet Specification version 2.0. Portlets whose class names contain the string "_SIG_" contain generated code for verifying the method signatures. These portlets should not be changed by hand. The remaining portlets contain additional V2 verification code.
V3*Tests
Modules with this naming scheme test v3.0 specific funtionality.

The Page File

The page file defines the page names for the TCK and the portlets expected to be on each page. This is an XML file in the form of a Pluto driver configuration file. The schema for this file is located in the Pluto project at the following location:

./pluto/pluto-portal-driver/src/main/resources/pluto-portal-driver-config.xsd

After the TCK has been built, complete page file for the TCK pages is contained in the following file:

./pluto/portlet-tck_3.0/deploy/target/deploy-files/pluto-portal-driver-config.xml

It is expected that vendors will extract the portlet-to-page mapping information from the page file in order to create the necessary configuration files for the portal under test.

When creating pages for your system, note that the test driver expects links to all test case pages to be available. The test driver searches for the page links by link text, so it is important that the links have the names defined in the page file.

Accessing the version 3.0.0 TCK Source Code

The source code is located in the Apache Pluto repository.

First, use git to clone the repository: git clone https://github.com/apache/portals-pluto.git pluto

Then switch into the 'pluto' directory and checkout the source for Pluto 3.0.0:

git checkout pluto-3.0.0

The version 3.0.0 TCK source code is now located in the 'portlet-tck_3.0' directory.

Building the TCK

Building the TCK produces the test portlets, a list of test cases in XML format, an XML-based page file containing the page configuration expected by the test cases, and the test driver itself.

To build portlet war files that do not contain any special configuration, use the command:

mvn clean install

To build portlet war files that are configured for deployment on Pluto, use the command:

mvn clean install -Ppluto

These commands can be executed either from the Pluto project base directory or from the ./pluto/portlet-tck_3.0 directory.

Deploying the TCK

Use the page file which has the default name of deploy/target/deploy-files/pluto-portal-driver-config.xml to create pages for your system. The WAR files in the deploy/target/deploy-files contain the test portlet code. Deploy these files on your system.

If you are testing with Apache Pluto, you can simply copy the page file into the Pluto webapps/pluto/WEB-INF directory in order to set up the pages. To deploy the WAR files on Pluto, copy them into the Pluto webapps directory.

Running the TCK

Configure the parent POM file ./pluto/portlet-tck_3.0/pom.xml to address your system under test. See the properties section in the pom.xml file. You can set the host name and port for the system under test, and can also select the browser to be used by the Selenium WebDriver code. Note, however, that if you use a browser other than "firefox" or "HTMLUnit" you will have to download and install additional WebDriver software manually, and you may need to add additional configuration properties (See the test.browser.webDriver property in the parent POM).

At the time of this writing, the HTMLUnit driver does not support JavaScript adequately for JSR 362 TCK purposes. You may need to try different drivers to find the appropriate WebDriver driver for your environment.

All TCK tests were successfully executed through the test driver against Pluto using the ChromeDriver web driver. See the portlet-tck_3.0/POM.xml file for a configuration example.

Run the TCK by switching to the driver subdirectory and execute the command:

mvn test -Prun-tck

If you want to run a subset of the tests, use the "-Dtest.module=" option. The driver will then only execute those tests whose test case name contains the specified string. For example, if you want to run the signature verification tests, use the command:

mvn test -Prun-tck -Dtest.module=SIG

You can also exclude specific tests by placing an exclamation point before the text to be matched. For example, if you want to exclude the signature verification tests, use the command:

mvn test -Prun-tck -Dtest.module=!SIG

The 'test.dryrun' flag will cause the test cases to be listed without executing them. For example, the following command will list all of the test cases whose test case name contains 'ResourceResponse', but will not execute them:

mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.dryrun=true

The test driver will wait for a response for a default time of 3 seconds. If this value proves to be too short, you can change it in the portlet-tck_3.0/POM.xml file, or you can specify a timeout value using the 'test-timeout' command line option. The timeout value is specified in seconds, as follows:

mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.timeout=5

The test driver has an ignore list containing test case names that are not required to pass in order for a portal product to be JSR 362 compliant. The ignore list is an XML file in properties file format located at:

portlet-tck_3.0/driver/src/main/resources/xml-resources/ignoredTestCases.xml

By default, the test driver will ignore and will not execute all test cases in the ignore list. If you wish to execute all test cases including those in the ignore list, set the command line flag 'test.ignore' to false:

mvn test -Prun-tck -Dtest.ignore=false

Passing the TCK

TCK execution is self-policing. If you are a portal vendor and wish to claim JSR 362 compatibility for your product, you should be able to demonstrate that this TCK can be executed against your product with no errors.

Contesting a Test Case

When defining and implementing the TCK test cases, care was taken to assure proper implementation and to test only required functionality.

However, mistakes may have occurred. If you feel that a test case is incorrectly implemented or is invalid as defined, please proceed as follows.

In either case, please open an issue at the Pluto Jira:

https://issues.apache.org/jira/browse/PLUTO/

Prefix the summary with 'TCK:' in order to allow TCK issues to be readily identified. Please be sure to specify the exact name(s) of the test case(s) in question.

If you feel that the test case was implemented incorrectly, you may correct the implementation and issue a pull request through github here:

https://github.com/apache/portals-pluto/pulls

If you feel that the test case is invalid as defined, you may update the ignore list and issue a pull request.

If we agree that the test case is invalid or incorrectly implemented, we will fix it or integrate your fix into the Pluto master and version 3.0 branches.