This website uses cookies to ensure you get the best experience. Learn More.
Improving Test Performance on Liferay
Recently, we upgraded the JSF Portlet Bridge Test Compatibility Kit (TCK) from Selenium 1.0 to our new test framework which uses Selenium 2.53. The TCK contains about 220 tests, and on Liferay 7.0 before we upgraded, it took around 8 minutes and 30 seconds to execute. In contrast, the TCK took 1 minute and 30 seconds to run on Apache Pluto, the portlet container reference implementation. In order to speed up test execution for Liferay Portal, we decided to try a few simple tricks and changes to reduce the total test time, and… we succeeded! Here’s a breakdown for how fast each browser ran the TCK on my machine before our upgrade, after our upgrade, and after we tweaked the TCK with some Liferay specific tricks:
1.0
2.53
7.0
21.0
46.0
* Headless browser.
As you can see, we made some pretty big improvements just by upgrading to a modern version of Selenium. Running tests with HtmlUnit also provided a nice boost over other browsers. Before I talk in-depth about HtmlUnit though, I want to begin by explaining the Liferay-specific tweaks and tricks that we used to speed up the TCK. These techniques are specific to Liferay but not to Selenium or HtmlUnit, so they may be useful in improving the performance of Liferay portlet tests regardless of the test framework or browsers used.
The main performance boost is thanks to Liferay’s custom “exclusive” window state.1 In Liferay, any portlet can be accessed in the exclusive state with a render URL containing the portlet’s id (p_p_id) and p_p_state=exclusive:
exclusive
p_p_id
p_p_state=exclusive
http://localhost:8080/my/portlets/page?p_p_id=myPortletName_WAR_myPortletWarName&p_p_lifecycle=0&p_p_state=exclusive
Liferay’s exclusive window state feature was created when Liferay only supported Portlet 1.0, and it provides similar functionality to the RESOURCE_PHASE of the Portlet 2.0 lifecycle. When Liferay Portal returns a portlet’s markup in the exclusive state, the markup contains only the portlet’s markup fragment without <html>, <head>, or <body> tags. Thankfully, every browser we use for testing handles the incomplete markup gracefully and renders the portlet testable. Providing the bare-bones portlet markup, the exclusive state doesn’t render any portal markup or frontend resources (such as CSS or JS). This greatly reduces the time it takes for a browser to load the page. The exclusive window state is useful for quickly testing backend functionality and rendering. I would not recommend testing portlets with complex JavaScript or CSS in the exclusive state. In fact, if you want to test a portlet with any external JavaScript resources at all, you will need to make sure that the resource’s script tag is rendered in the portlet’s body (<div>) markup rather than the portal’s <head> section (which isn’t rendered).2 Certainly, using the exclusive state will not work for all tests, but it can provide significant speed improvements for certain tests.
<html>
<head>
<body>
<div>
If you cannot easily move your portlet’s resources into your portlet’s body or if you rely on Liferay Portal’s frontend features (JS and/or CSS), try using the “pop_up” window state instead. In Liferay, any portlet can be accessed in the pop up state with a render URL containing the portlet’s id (p_p_id) and p_p_state=pop_up:
pop_up
p_p_state=pop_up
http://localhost:8080/my/portlets/page?p_p_id=myPortletName_WAR_myPortletWarName&p_p_lifecycle=0&p_p_state=pop_up
When a portlet is in the pop up state, the portal renders the <html>, <head>, and <body> tags including JavaScript and CSS resources. However, as with the exclusive state, the portal does not render any portal markup or portlets besides the one referenced in the URL. Even though using the exclusive state yielded greater performance benefits, using the pop up state still significantly sped up our tests (probably by about 25%).
Of course, neither of these two states should be used for testing portlets which may interact with each other, since they only render a single portlet. These states also don’t help if you are trying to test other window states (as we do in the TCK). However, while upgrading the TCK, we found another way you can speed up your tests if you are using Selenium.
Of all the Selenium compatible browsers that we test with, HtmlUnit is by far the fastest. And since HtmlUnit is headless, it can be run on a CI server without a window manager. HtmlUnit is not perfect. Its JavaScript engine has trouble running more complicated or cutting edge code, so I wouldn’t recommend it for testing newer browser features like the History API (which SennaJS uses) or HTML5. Nonetheless, it is an excellent browser for testing simple pages quickly. So how do you use HtmlUnit? Well if you are using Selenium, simply add the HtmlUnit dependency to your project (make sure you get the latest one):
HtmlUnit
<dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>htmlunit-driver</artifactId> <version>2.23.2</version> <scope>test</scope> </dependency>
Then just change your WebDriver implementation to HtmlUnitDriver:
WebDriver
HtmlUnitDriver
WebDriver webDriver = new HtmlUnitDriver();
The End.
Well… not really. HtmlUnit is a simple tool, but unfortunately configuring it to behave like every other browser is over-complicated and poorly documented. First, you will need to specify the following dependencies to avoid several NoClassDefFoundError/ClassNotFoundException issues:
NoClassDefFoundError
ClassNotFoundException
<dependency> <groupId>xml-apis</groupId> <artifactId>xml-apis</artifactId> <version>1.4.01</version> <scope>test</scope> </dependency> <dependency> <groupId>org.eclipse.jetty.websocket</groupId> <artifactId>websocket-client</artifactId> <version>9.2.18.v20160721</version> <scope>test</scope> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.2</version> <scope>test</scope> </dependency>
Second, I recommend disabling (or filtering) HtmlUnit’s incredibly verbose logging:
// Uncomment to enable HtmlUnit's logs conditionally when the log level is FINER or higher. // if (logLevel.intValue() > Level.FINER.intValue()) { LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog"); Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF); Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF); // }
Third, you should extend HtmlUnitDriver:
public class CustomHtmlUnitDriver extends HtmlUnitDriver {
Extending HtmlUnitDriver allows you to force HtmlUnit not to throw exceptions when JavaScript errors occur, and it allows you to silence CSS errors:
@Override protected WebClient modifyWebClient(WebClient initialWebClient) { WebClient webClient = super.modifyWebClient(initialWebClient); // Don't throw exceptions when JavaScript Errors occur. webClient.getOptions().setThrowExceptionOnScriptError(false); // Uncomment to filter CSS errors. // if (logLevel.intValue() > Level.FINEST.intValue()) { webClient.setCssErrorHandler(new SilentCssErrorHandler()); // } return webClient; }
Fourth, you should note that HtmlUnit does not load any images by default,3 so any tests which require images to be loaded should manually call a method like the one below to load them:
public void loadImages() { HtmlPage htmlPage = (HtmlPage) lastPage(); DomNodeList<DomElement> imageElements = htmlPage.getElementsByTagName("img"); for (DomElement imageElement : imageElements) { HtmlImage htmlImage = (HtmlImage) imageElement; try { // Download the image. htmlImage.getImageReader(); } catch (IOException e) { // do nothing. } } }
Finally, always initialize HtmlUnit with JavaScript enabled (and emulate a popular browser):
WebDriver webDriver = new CustomHtmlUnitDriver(BrowserVersion.FIREFOX_45, true);
If you want a complete example of how we use HtmlUnit in Liferay Faces, see HtmlUnitDriverLiferayFacesImpl.java and Browser.java.
HtmlUnitDriverLiferayFacesImpl.java
Browser.java
Hopefully, the advice in this post will help you speed up your tests in Liferay. If you’ve found any other tricks that improve your tests’ performance, please post them in the comments below.
2.25
