JT Dev

The response time of websites is important, not just for the users, but also for SEO (Search Engine Optimalization) of a website. According to Google:

Speeding up websites is important — not just to site owners, but to all Internet users. Faster sites create happy users and we’ve seen in our internal studies that when a site responds slowly, visitors spend less time there. But faster sites don’t just improve user experience; recent data shows that improving site speed also reduces operating costs. Like us, our users place a lot of value in speed — that’s why we’ve decided to take site speed into account in our search rankings. We use a variety of sources to determine the speed of a site relative to other sites.

The response time of a website is the time between the HTTP request send from the webbrowser to the webserver and the corresponding HTTP reponse send from the webserver back to the browser.
There are many factors that determine this response time, such as geographical distance, Internet connection, the speed of the computer the browser is running on, and the performance of the webserver. The webservers performance is usually determined by database queries and the amount of data that has be to send back. But I was wondering how much the rendering performance of webframeworks impact the response time and how they compare to eachother.

In this post I’ll show how each you can render a list of products using each framework. This can be handy if you need a quick look at a framework, and see how it compares codewise to others. For each framework, I’ve measured the response times against the number of concurrent users and the number of product objects to render. I’ve also looked quickly at the memory consumption of each framework.

Please note, I’m not an expert in these frameworks, but I tried to use the most common solutions to render objects.
Also note that this post won’t show which framework is faster in general, it will only tell you which one is faster under the given environment/conditions and the given test case.
Another difficulty is that some frameworks have builtin features that shouldn’t be tested, like automatically opening database connection on each request. Some frameworks might have builtin security features that will increase the response times, or some might have builtin caching enabled by default. I haven’t looked at all these extra features that shouldn’t be part of the test case, so the results will only give a coarse indication of how fast a framework is in the given environment/test-case.

Test case

The test only consist of one page that is tested. This page contains a 2 column HTML layout using div tags. One column is the sidebar, the other the content area. The content area needs to render a list of products with it’s associated categories in a HTML table. The page does not contain components that need to be stateful (like forms etc.).
See GitHub for an example.

Each Product (class) contains the following properties:

-name: String
-price: Integer
-description: String
-categories: Hash-based set of Category

Each Category (class) contains the following properties:

-name: String

All solutions use an (static) in-memory List to retrieve products, so no database access was needed or set.

Framework features

For each framework, the page needs to use the following framework features:
-rendering of a list of products, each rendered as a custom component so that the layout of each product can be reused on other pages. For each product, it’s associated categories are shown.
-Separation of the page into a template that can also be used for other pages, and the content that is unique for each page (this content is surrounded by the template).
-Injection of the page name (including the number of products) in the title tag, and in the master template page name.

Measurements

The following variables are used in the response time measurements:
-Number of concurrent users
-Number of products

The memory usage is measured only for the 16 concurrent user with 1000 products test. Usage is measered by roughly looking at the average values using Windows Task Manager. CPU utilization is also measured in this way.

Each framework should return approximately the same HTML page size. Differences can occure because the way I wrote the template pages. For example: there is a difference in size between:

<g:each var="cat" in="${product.categories}">$${cat.name}, </g:each>

and

<g:each var="cat" in="${product.categories}">
	$${cat.name},
</g:each>

I measured the page sizes to make sure that there weren’t big differences. Big differences could impact performance.

The HTTP benchmark was performed by the tool Apache JMeter 2.4, which also measured the response times. Within JMeter, a Response Assertion was used to make sure that the correct page was returned (and not some error pages). The Response Assertion tested whether the result contained the string “Company” (which is part of the layout). No sessions/cookies nor HTTP request managers were set in JMeter, so each HTTP request was performed independently of the other requests.

Test system

-OS: Windows 7 Professional SP1 64bit
-CPU: AMD Phenom II X4 955
-MEM: 4 GB RAM
-Java: SE Update 24, 32bit
-Webserver for JVM based frameworks: apache-tomcat-7.0.12-windows-x86, run with -Xmx1024m

Tomcat was set to Xmx 1024mb, which gives the Java JVM at most 1024mb of available RAM to use. This should be enough to minimize garbage collection.

When settting Tomcat environment options under Windows, don’t use quotes!
set JAVA_OPTS=”-Xmx1024m”
won’t work, even though it is used in books like “Tomcat – The Definitive Guide”. Use instead:
set JAVA_OPTS=-Xmx1024m
You can verify this in the “Tomcat Web Application Manager” under “Server Status”, look for “Max memory”.

file: WEB-INF/web.xml (URL mappings)

....
<servlet>
	<servlet-name>products</servlet-name>
	<servlet-class>www.ProductsServlet</servlet-class>
</servlet>

<servlet-mapping>
	<servlet-name>products</servlet-name>
	<url-pattern>/products</url-pattern>
</servlet-mapping>
.....

web.xml is used to map the uri /products to the ProductsServlet class, which acts as a MVC controller.

file: WEB-INF/classes/www/ProductsServlet.java (controller)

public class ProductsServlet extends HttpServlet {
	@Override
	protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		request.setAttribute("products", Service.getProducts());

		RequestDispatcher view = request.getRequestDispatcher("/products.jsp");
		view.forward(request, response);
	}
}

The Servlet container (Tomcat) sends incoming HTTP requests to ProductsServlet (MVC controller), which retrieves the product data as a List object, and puts it in the request object as attributes and dispatches/forwards/pushes the request to products.jsp (MVC view).

file: products.jsp (view)

<%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@ taglib prefix="company" tagdir="/WEB-INF/tags" %>
<%@ taglib prefix="company2" uri="company" %>

<company:header title="Products Listing" />

<table>
	<c:forEach var="product" items="${products}">
		<tr>
			<td><company:product product="${product}" /></td>
			<td>
				<c:forEach var="cat" items="${product.categories}">
					${cat.name},
				</c:forEach>
			</td>
		</tr>
	</c:forEach>
</table>

<company:footer />

Products.jsp is the view, which receives data pushed from the ProductsServlet (controller).
The taglib lines define the location of JSP based tags.
The c:forEach tags are used to loop over the products and categories. The “company:product” tag is used as a custom component, which can be implemented as a JSP Tag File (prefix “company”), or as a TLD (Tag Library Descriptor) component (prefix “company2″). The company:header tag and company:footer tag are implemented as Tag Files, which surround the page content with the default layout that can be shared among other pages.

file: WEB-INF/tags/header.tag (heading template)

<%@ attribute name="title" required="true" rtexprvalue="true" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="default.css" media="screen" rel="stylesheet" type="text/css" />

		<!-- Title needs to be injected -->
		<title>${title}</title>
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<!-- Page name needs to be injected -->
					<div><h2>${title}</h2></div>
					<div class="innertube">

This Tag File is the first part of the default layout that can be shared among other pages. The first line defines data that can be passed to this Tag File.

(The footer heading template is similar to this file, but now shown here.)

file: WEB-INF/tags/product.tag (JSP based component)

<%@ tag body-content="empty" %>
<%@ attribute name="product" required="true" rtexprvalue="true" type="domain.Product" %>
<div class="product">
	<img src="${product.name}.jpg" />
	<span class="productname">${product.name}</span>, <span class="price">$${product.price}</span>
</div>

This is the Tag File based custom component used to render products. The attribute tg is used to define the parameters that can be passed to this file.

An alternative to the Tag File based custom component is the use of TLD (Tag Library Descriptor) + Java based code. This alternative isn’t tested in this post, but it’s probably faster than the Tag File based custom component. The code of this alternative is shown below.

(alternative, not tested) file: WEB-INF/default.tld (component declaration)

<?xml version="1.0" encoding="UTF-8"?>
<taglib xmlns="http://java.sun.com/xml/ns/j2ee"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
	web-jsptaglibrary_2_0.xsd" version="2.0">
	<tlib-version>1.0</tlib-version>
	<uri>company</uri>
	<tag>
		<name>product</name>
		<tag-class>www.ProductTag</tag-class>
		<body-content>empty</body-content>

		<attribute>
			<name>product</name>
			<required>true</required>
			<rtexprvalue>true</rtexprvalue>
		</attribute>
	</tag>
</taglib>

This is the default TLD (Tag Library Descriptor), which declares the company2:product tag used in products.jsp. It references to the www.ProductTag Java class file.

(alternative, not tested) file: WEB-INF/classes/www/ProductTag.java (Java based component)

public class ProductTag extends SimpleTagSupport {
	private Product product;

	public void setProduct(Product product) {
		this.product = product;
	}

	@Override
	public void doTag() throws JspException, IOException {
		JspWriter w = getJspContext().getOut();
		w.println("<div class=\"product\">");
		w.println("<img src=\"" + product.getName() + ".jpg\" />");
		w.println("<span class=\"productname\">" + product.getName() + "</span>,");
		w.println("<span class=\"price\">$" + product.getPrice() + "</span></div>");
	}
}

This writes a product as a custom component, it matches the TLD.

file: wicketapp/WicketApplication.java

public class WicketApplication extends WebApplication {
	...
	@Override
	public void init() {
		...
		mountPage("products", ProductsPage.class);
	}
}

The mountPage method maps the URI /products to the class ProductsPage.

file: wicketapp.TemplatePage.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="default.css" media="screen" rel="stylesheet" type="text/css" />
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<div><h2 wicket:id="pageName">Producs listing</h2></div>
					<div class="innertube">

						<wicket:child />

					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

This defines a Wicket page that can be used as a template for other Wicket pages; it serves as a base class for those other pages.

This is the template that is shared by other pages (subclasses). Together with TemplatePage.java, TemplatePage.html forms a single page.
The wicket:id is used to bind the HTML tag to a Wicket component. This allows TemplatePage.java to access and modify the HTML element/component, including it’s body. In this case, the wicket:id named “pageName” is used to allow TemplatePage.java to set the name of the page.

The wicket:child element is used to define the place where the content of subclass pages (such as the ProductsPage) will be shown.

file: wicketapp/TemplatePage.java

public class TemplatePage extends WebPage {

	private Model pageNameModel = new Model();

    public TemplatePage(final PageParameters parameters) {
    	// h2 page name
    	add(new Label("pageName", pageNameModel));
    }

    public void setPageName(String name) {
    	pageNameModel.setObject(name);
    }
}

This is the Java code that is associated with TemplatePage.html. It adds a Label component to the page, which is passed the wicket:id value from TemplatePage.html, and a model object.
Wicket uses models to attach data to components. In case of the Label component is given a model object that is able to retrieve the name of the page.

These models are required because Wicket is designed to let pages be stateful and in Wicket the retrieval of data for a component if seperated from the component itself. This allows pages to be serialized without requiring all the attached model data to be serialized.

file: wicketapp/ProductsPage.html

<wicket:head>
	<title>Product Listing</title>
</wicket:head>

<wicket:extend>
	<table>
		<tr wicket:id="products">
			<td>
				<div wicket:id="product" />
			</td>
			<td>
				<wicket:container wicket:id="categories">
					<span wicket:id="category"></span>,
				</wicket:container>
			</td>
		</tr>
	</table>
</wicket:extend>

ProductsPage.java extends TemplatePage.java, this will cause ProductsPage.html’s wicket:extend body to be placed at TemplatePage.html’s wicket:child tag’s location.
The wicket:id tags are used to bind the HTML tags to a Wicket component. This allows ProductsPage.java to access and modify those HTML elements/components, including the bodies.
The wicket:container tag is used as a grouping/container, which is bound to a repeating view component that is used to iterate over the categories.
The wicket:head tags allows child pages and components to add tags to the HTML head.

file: wicketapp/ProductsPage.java

public class ProductsPage extends TemplatePage {
	public ProductsPage(final PageParameters parameters) {
		super(parameters);
		ListView<Product> productsLV = new ListView<Product>("products", Service.getProducts()) {
			@Override
			protected void populateItem(ListItem<Product> li) {
				Product product = li.getModelObject();
				ProductPanel productPanel = new ProductPanel("product", product);
				li.add(productPanel);

				List<Category> categories = new ArrayList<Category>(product.getCategories());

				ListView<Category> categoriesLV = new ListView<Category>("categories", categories) {
					@Override
					protected void populateItem(ListItem<Category> catLi) {
						Category category = catLi.getModelObject();
						catLi.add(new Label("category", category.getName()).setRenderBodyOnly(true));
					}
				};
				li.add(categoriesLV);
			}
		};
		add(productsLV);
	}
}

ProductsPage.java created the products page together with ProductsPage.html. It is a subclass of TemplatePage (which defines the template that is shared by other pages). It has a ListView componenent which iterates over the products. Each iteration is defined in the populateItem method. Each iteration creates a ProductPanel (a custom component) and a label containing the categories of each product. A nested LiewView is used to iterate over the categories. The strings such as “product”, “categories” and “category” are the component instance names, which match the wicket:id tag values used in ProductsPage.html.
Wicket uses models to attach data, e.g. a list of products, to components. In case of the Label component, the category.getName() string is automatically wrapped in a model.
These models are required because Wicket is designed to let pages be stateful and in Wicket the retrieval of data for a component if seperated from the component itself. This allows pages to be serialized without requiring all the attached model data to be serialized.

file: wicketapp/ProductPanel.html

<wicket:panel>
	<!-- This part/layout of each product needs to be made as a custom component, so that it can be reused at other pages -->
	<div class="product">
		<img wicket:id="image" />
		<span wicket:id="productname" class="productname">A name of product1....</span>, <span wicket:id="price" class="price">$150</span>
	</div>
</wicket:panel>

This is the layout of the custom product component. The wicket:id tags give HTML elements unique names, so that the HTML element can be accessed/modified/replaced as a Wicket component in ProductPanel.java.

file: wicketapp/ProductPanel.java

public class ProductPanel extends Panel {

	public ProductPanel(String id, Product product) {
		super(id);

		WebComponent img = new WebComponent("image");
		String imgUrl = product.getName() + ".jpg";
		img.add(new SimpleAttributeModifier("src", imgUrl));
		add(img);

		Label nameLabel = new Label("productname", product.getName());
		add(nameLabel);

		Label priceLabel = new Label("price", product.getPrice().toString());
		add(priceLabel);
	}
}

This is the Java code of the custom product component. This custom component is composed of three child Wicket components, each consisting of a wicket:id name and associated product data.
The SimpleAttributeModifier is used to add the src HTML attribute of the img HTML element in ProductPanel.html.

production settings

set to production mode using:
System.setProperty(“wicket.configuration”, “deployment”);

file: views/layouts/main.gsp

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="${createLinkTo(dir:'', file:'default.css')}" media="screen" rel="stylesheet" type="text/css" />
		<title><g:layoutTitle default="Grails" /></title>
        <g:layoutHead />
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<div><h2>${pageName}</h2></div>
					<div class="innertube">

						<g:layoutBody />

					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

This defines the default layout that can be shared among other pages. The g:layoutBody tag defines the location where content will be placed.
g:layoutHead can be used to allow pages to inject content in the header.
The createLinkTo method makes sure that the webapp directory is automatically added to the URL, so that incorrect use of slashes won’t cause wrong relative urls to default.css.

file: controllers/grailsapp/ProductsController.groovy

class ProductsController {
    def index = {
		[products: s.Service.getProducts()]
	}
}

The method index will be mapped to the URI /products/index, or in short just /products (because it’s the index). Multiple methods (called actions) can be added in this way in the same controller class.
The index method returns a “model”, which is a map that the view uses when rendering. The keys (“products” in this case) within that map translate to variable names accessible by the view.

file: views/products/index.gsp

<g:set var="pageName" value="Product listing" scope="request" />

<html>
	<head>
		<meta name="layout" content="main"></meta>
		<title>Product listing</title>
	</head>
	<body>
		<table>
			<g:each var="product" in="${products}">
				<tr>
					<td><g:render template="/shared/product" model="[product:product]" /></td>
					<td>
						<g:each var="cat" in="${product.categories}">$${cat.name}, </g:each>
					</td>
				</tr>
			</g:each>
		</table>
	</body>
</html>

The layout meta tag points to the main layout. The html and body tag will be removed from the final output, because those from the main layout will be used.
The g:each tag is used to iterate over the products and categories.
The s:set tag sets a variable so that it can be used in the shared layout.
The g:render tag is used to render products as custom component/elements using Grails templates.

file: views/shared/_product.gsp

<div class="product">
	<img src="${product.name}.jpg" />
	<span class="productname">${product.name}</span>, <span class="price">${product.price}</span>
</div>

This is the custom component (Grails template) to render products.

production settings

grails prod war

file: config/routes.rb

Railsapp::Application.routes.draw do
	...
	resources :products
end

This line makes the products controller including all methods (actions) available under URI pattern /products/actionName. Since we are using the index action, which is the default, the URI will default to /products.

file: app/views/layouts/application.html.erb

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="/default.css" media="screen" rel="stylesheet" type="text/css" />

		<%= stylesheet_link_tag :all %>
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<!-- Page name needs to be injected -->
					<div><h2><%= @pagename %></h2></div>
					<div class="innertube">
						<%= yield %>

					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

This is the shared template that can be used by multiple pages. The yield method is used to specify the location of where the content of the pages will be placed.

file: app/controller/products_controller.rb

class ProductsController < ApplicationController
	skip_before_filter :verify_authenticity_token
	protect_from_forgery :except => :index

	def index
		@products = ProductsHelper::Service.getProducts
	end
end

The skip_before_filter and protect_from_forgery method calls are needed to disable some options that might influence benchmark performance, these options aren’t enabled by default on other frameworks.

The index method is the method that is called at each HTTP request to our page. It creates the @products instance variable on each request, which can than be accessed by the view.

file: app/views/products/index.html.erb

<% @pagename = "Product Listing" %>

<table>
	<% @products.each do |product| %>
	<tr>
		<td class="productname"><%= render "shared/product", :product => product %></td>
		<td class="categories">
			<% product.categories.each do |category| %>
				<%= category.name %>,
			<% end %>
		</td>
	</tr>
	<% end %>
</table>

As shown above, Rails uses plain Ruby code embedded between <% tags to render Rails code.
The @pagename is used to set the pagename so that it can be used in the shared template/layout.
The render method is used to render product as a custom componenent (called partials in Rails).

file: app/views/shared/_product.html.erb

<div class="product">
	<img src="<%= product.name %>.jpg" />
	<span class="productname"><%= product.name %></span>, <span class="price">$<%= product.price %></span>
</div>

This is a custom component (called a partial in Rails) to render a product.

production settings

Rails Webrick:
-rails server -e production

Rails Mongrel:
-cmdline: gem install mongrel
-cmdline: bundle install
-add line to gem Gemfile: gem “mongrel”, ‘>= 1.2.0.pre2′
-cmdline: rails server -e production

JRuby-Rails Webrick:
-jruby script\rails server

JRuby-Rails Trinidad:
-jruby -S trinidad -e production

file: views/main.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
    <head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

        <title>#{get 'title' /}</title>

        <link rel="stylesheet" media="screen" href="@{'/public/default.css'}">
        #{get 'moreStyles' /}
        <link rel="shortcut icon" type="image/png" href="@{'/public/images/favicon.png'}">
        #{get 'moreScripts' /}
    </head>

	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<!-- Page name needs to be injected -->
					<div><h2>#{get 'pagename' /}</h2></div>
					<div class="innertube">
						#{doLayout /}
					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

The get tags are used to retrieve variables that are set elsewhere, like in the pages. The doLayout tag specifies the location where content of each page will be shown.

file: controller/Products.java

public class Products extends Controller {
   public static void index() {
		List<Product> products = Service.getProducts();
        render(products);
    }
}

The index method of the controller that is called on each http request to /products/index.

file: views/products/index.html

#{extends 'main.html' /}
#{set title:'Product listing' /}
#{set pagename: 'Product listing' /}

<table>
	#{list items:products, as:'product'}
		<tr>
			<td>#{product product /}</td>
			<td>
				#{list items:product.categories, as:'category'}
					${category.name},
				#{/list}
			</td>
		</tr>
	#{/list}
</table>

The extends tag specifies the shared layout that can also be used by other pages. The set title tag sets the title variable so that it can be used in the shared layout.
The list tag is used to iterate over the products. Instead of a list tag, Groovy code (like Grails) can also be used in the views.
The product tag is a custom component to render a product.

file: views/tags/product.html

<div class="product">
	<img src="${_arg.name}.jpg" />
	<span class="productname">${_arg.name}</span>, <span class="price">$${_arg.price}</span>
</div>

This is the custom component (called a tag) to render a product.

production settings

set to production via the file application.conf:
application.mode=prod
%prod.application.mode=prod

play war -o k:\wars\play –zip

file: webapp/templates-hidden/default.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="default.css" media="screen" rel="stylesheet" type="text/css" />

		<!-- Title needs to be injected -->
		<title>Products listing</title>
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<!-- Page name needs to be injected -->
					<div><h2><lift:bind name="pagename">.....</lift:bind></h2></div>
					<div class="innertube" id="content">
						<lift:bind name="content"></lift:bind>
					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

This is the layout that can be shared by multiple pages.
The lift:bind tag is used as a placeholder where data can be placed by other pages or code.

file: bootstrap.liftweb.Boot.scala

class Boot {
	def boot {
		...
		// Build SiteMap
		val entries = List(
			...
			Menu.i("Products listing") / "products",
			...
		)
		...
	}
}

A menu is added to the SiteMap list, this is required for the page to be accessible via an URL. In this case, the name of the menu is “Products listing”, and the URI is /products. It will look for a page at webapp/products.html.

file: webapp/products.html

<lift:surround with="default">
	<lift:bind-at name="pagename">aaaa</lift:bind-at>

	<lift:bind-at name="content">
		<table class="lift:ProductsSnippet.showProducts">
			<tr>
				<td class="productname"></td>
				<td class="categories"></td>
			</tr>
		</table>
	</lift:bind-at>

</lift:surround>

The lift:surround places the body of this tag in the lift:bind tag that has the attribute name=”content” of the template named “default” (webapp/templates-hidden/default.html).
The lift:ProductsSnippet.showProducts value in the class attribute causes Lift to pass the table tag including it’s body to a Scala method in the class “ProductsSnippet” named “showProducts” as an XML object (NodeSeq), or it applies a function that ProductsSnippet.showProducts might return on the table tag. This allows showProducts to perform XML transformations/manipulations at the given table tag.

ProductSnippet.scala

package code
package snippet

import scala.xml.{NodeSeq, Text}
import net.liftweb.util._
import net.liftweb.common._
import java.util.Date
import code.lib._
import Helpers._
import _root_.s._

class ProductsSnippet {

  def showProducts = {
	val products = Service.products
	"tr" #> (in => products.flatMap{ p =>
		(".productname *" #> <div class="product"><img src={p.name + ".jpg"} /><span class="productname">{p.name}</span>, <span class="price">${p.price}</span></div> &
		".categories *" #> p.categories.map(_.name).mkString(", "))(in)
	})
  }
}

showProducts is a function. The goal of showProducts is to transform the table tag of products.html (including it’s body) to the desired HTML output. In this case, we want the custom product HTML layout (which we call a component) to be placed within the td tag, and we want the categories of each product to be listed in the other td tag.
To do this, we have to iterate over each product, create a custom component/layout for each product, and place it in the td tag. And for each product, we have to iterate over the categories.

showProducts can be difficult to understand if you don’t have experience with functional programming. I’ll try to explain what happens.
Lift takes the table tag of products.html (including it’s body) and calls the showProducts function. showProducts won’t immediately act on the table tag, but it will return a function that will be the result of the function call to the showProducts function. Then Lift passes the table tag as a parameter to the returned function, which will then return the desired HTML output in the form of a NodeSeq object, which will than be placed at the corresponding location in products.html.

(Instead of having showProducts to return a function, it’s also possible to let it receive the HTML (a NodeSeq) immediately, and return the transformed HTML (a NodeSeq). This sounds easier, but the syntax of Lift when returning a function, is shorter and might be easier to write.)

The val keyword declares the “products” variable, which is a List object. “#>” is a method/operator that is added to the String class using Scala’s implicit conversions (via an import statement). The #> method returns a function, but can also be given a function as input parameter.

The first #> method call is used to bind the “tr” tag to HTML (a NodeSeq). The map and flatMap methods are used to translate each product or category to the corresponding HTML output.
The #> method call at “.productname *” binds the new HTML to the body of a HTML element that has “productname” as value of the class attribute.

production settings

set JAVA_OPTS=-Drun.mode=production -Xmx1024m

sbt ~package

file: app/products.scala (mvc controller)

object Products extends Controller {
    import views.Products._

	def index(n: Int) = html.index(Service.getProducts(n))
}

Defines the URI /products/index, and maps it to the index method. The index method takes a parameter ‘n’ that is mapped to the URL querystring parameter ‘n’. Conversions to an integer are applied automatically.
In Play-Scala the view pages/templates are seen as functions (in this case html.index function) that can be passed data (in this case a list of products).

file: app/views/main.scala.html

@(title:String = "", pageName:String = "")(body: => Html)

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<link href="default.css" media="screen" rel="stylesheet" type="text/css" />

        <title>@title</title>
        <link rel="stylesheet" media="screen" href="@asset("public/stylesheets/main.css")">
        <link rel="shortcut icon" type="image/png" href="@asset("public/images/favicon.png")">
        <script src="@asset("public/javascripts/jquery-1.5.2.min.js")" type="text/javascript"></script>
	</head>
	<body>
		<div id="maincontainer">
			<div id="topsection">
				<div class="innertube"><h1>Company Title .....</h1></div>
			</div>
			<div id="contentwrapper">
				<div id="contentcolumn">

					<!-- Page name needs to be injected -->
					<div><h2>@pageName</h2></div>
					<div class="innertube">

						<!-- The content that is unique for each page -->
						@body
					</div>
				</div>
			</div>

			<div id="leftcolumn">
				<div class="innertube">
					<h3>Side bar.....</h3>
				</div>
			</div>

			<div id="footer">footer.....</div>
		</div>
	</body>
</html>

In Play-scala pages/templates/tags are seen as functions that can have parameters, so that data can be passed to a page/template/tag.
The first line defines a title, pageName and body parameter. Data can be passed from controllers and other pages/templates/tags.
The “@” sign refers to Scala code.

file: app/views/Products/index.scala.html

@(products: Seq[s.Product])
@import views.tags.html._

@main("Product listing title", "Product Listing page") {
	<table>
		@for(product <- products) {
			<tr>
				<td>@productTag(product)</td>
				<td>
					@for(category <- product.categories.toList) {
						$@category.name,
					}
				</td>
			</tr>
		}
	</table>
}

The first line defines the parameters that can be passed to this page. “main” is a function that refers to the main template. It is passed a title, pageName and the body.

file: app/views/tags/productTag.scala.html

@(product: s.Product)

<div class="product">
	<img src="@(product.name).jpg" />
	<span class="productname">@product.name</span>, <span class="price">$@product.price</span>
</div>

The custom product component (called a tag).

production settings

set to production via the file application.conf:
application.mode=prod
%prod.application.mode=prod

Two production environments were tested:
a) Play-scala running under Tomcat
play war -o k:\wars\play –zip

b) Play-scala running under it’s default builtin webserver called Netty.
JAVA_OPTS=-Xmx1024m
play run

"u": the number of concurrent users
"p": the number of products to render
"ms": response time in milliseconds
"KB": returned page size in kilobytes.
"%": CPU utilization
"mb": RAM memory usage of the webapp

Grails 1.3.7 - Tomcat
1u, 1000p, 129ms, 205,77 KB, UTF-8
2u, 1000p, 150ms
4u, 1000p, 185ms
8u, 1000p, 428ms
16u, 1000p, 848ms, 155mb, 85%
1u, 0p, 1ms
1u, 1p, 1ms
1u, 10p, 2ms
1u, 100p, 14ms
1u, 500p, 66ms
1u, 1000p, 129ms
1u, 5000p, 629ms

Grails 1.4.0.BUILD-SNAPSHOT (3 June 2011) - Tomcat
1u, 1000p, 107ms, 205,79 KB
2u, 1000p, 115ms
4u, 1000p, 132ms
8u, 1000p, 307ms
16u, 1000p, 622ms, 233mb, 85%
1u, 0p, 1ms
1u, 1p, 1ms
1u, 10p, 2ms
1u, 100p, 12ms
1u, 500p, 55ms
1u, 1000p, 107ms
1u, 5000p, 525ms

Grails 1.4.0.BUILD-SNAPSHOT2 (3 June 2011) - Tomcat
1u, 1000p, 47ms, 205,82 KB
2u, 1000p, 56ms
4u, 1000p, 67ms
8u, 1000p, 149ms
16u, 1000p, 373ms, 197mb, 75%
1u, 0p, 1ms
1u, 1p, 1ms
1u, 10p, 1ms
1u, 100p, 5ms
1u, 500p, 24ms
1u, 1000p, 47ms
1u, 5000p, 229ms

Grails 1.4.0.BUILD-SNAPSHOT (3 June 2011) - Jetty (default builtin)
Same results as Tomcat.

Rails 3.0.7 - Webrick 1.3.1
1u, 1000p, 102ms, 229,08 KB, UTF-8
2u, 1000p, 177ms
4u, 1000p, 254ms
8u, 1000p, 715ms
16u, 1000p, 1438ms, 40mb, 25%
1u, 0p, 15ms
1u, 1p, 15ms
1u, 10p, 15ms
1u, 100p, 16ms
1u, 500p, 52ms
1u, 1000p, 102ms
1u, 5000p, 445ms

JRuby 1.6.2 - Rails 3.0.7 - Webrick 1.3.1
1u, 1000p, 139ms, 229,08 KB
2u, 1000p, 268ms
4u, 1000p, 537ms
8u, 1000p, 1082ms
16u, 1000p, 2252ms, 135mb, 26%
1u, 0p, 2ms
1u, 1p, 2ms
1u, 10p, 4ms
1u, 100p, 16ms
1u, 500p, 71ms
1u, 1000p, 139ms
1u, 5000p, 716ms

Rails 3.0.7 - Ruby 1.9.2-p180 - Mongrel 1.2.0.pre2
1u, 1000p, 110ms, 229,08 KB
2u, 1000p, 224ms
4u, 1000p, 425ms
8u, 1000p, 583ms
16u, 1000p, 1722ms, 45mb, 25%
1u, 0p, 15ms
1u, 1p, 15ms
1u, 10p, 16ms
1u, 100p, 18ms
1u, 500p, 61ms
1u, 1000p, 110ms
1u, 5000p, 581ms

JRuby 1.6.2 - Rails 3.0.7 - Embedded Tomcat (Trinidad 1.2.0)
1u, 1000p, 142ms, 229,08 KB
2u, 1000p, 144ms
4u, 1000p, 152ms
8u, 1000p, 308ms
16u, 1000p, 621ms, 322mb, 95%
1u, 0p, 2ms
1u, 1p, 2ms
1u, 10p, 3ms
1u, 100p, 16ms
1u, 500p, 73ms
1u, 1000p, 142ms
1u, 5000p, 699ms

JSP 2.1 - JSTL 1.2 - Servlet 2.5 - Tomcat
1u, 1000p, 45ms, 213,58 KB
2u, 1000p, 48ms
4u, 1000p, 54ms
8u, 1000p, 108ms
16u, 1000p, 223ms, 51mb, 85%cpu
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 4ms
1u, 500p, 23ms
1u, 1000p, 45ms
1u, 5000p, 225ms

Static HTML - Tomcat
1u, 1000p, 0s, 213,58 KB
2u, 1000p, 1ms
4u, 1000p, 2ms
8u, 1000p, 5ms
16u, 1000p, 14ms, 72%cpu, 592mb
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 0ms
1u, 500p, 0ms
1u, 1000p, 0ms
1u, 5000p, 4ms

Wicket 1.5-RC3 - Tomcat
1u, 1000p, 251ms, 205,73 KB
2u, 1000p, 506ms
4u, 1000p, 503ms
8u, 1000p, 498ms
16u, 1000p, 499ms, 51mb, 25%cpu
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 2ms
1u, 100p, 19ms
1u, 500p, 119ms
1u, 1000p, 251ms
1u, 5000p, 1885ms

Wicket from trunk (for 1.5-RC5) - Tomcat
1u, 1000p, 251ms, 205,73 KB
2u, 1000p, 263ms
4u, 1000p, 348ms
8u, 1000p, 688ms
16u, 1000p, 1366ms, 158mb, 70%cpu
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 2ms
1u, 100p, 19ms
1u, 500p, 113ms
1u, 1000p, 246ms
1u, 5000p, 1955ms

Wicket from trunk (for 1.5-RC5), RepeatingView version (from Martin G.) - Tomcat
1u, 1000p, 179ms, 220,41 KB
2u, 1000p, 212ms
4u, 1000p, 265ms
8u, 1000p, 544ms
16u, 1000p, 1084ms, 146mb, 70%
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 1ms
1u, 100p, 14ms
1u, 500p, 82ms
1u, 1000p, 179ms
1u, 5000p, 1510m

Wicket from trunk (for 1.5-RC5), Stateful url version (from Martin G.) - Tomcat
NOTE: used with JMeter Cookie manager enabled, otherwise the results will be the same as: Wicket from trunk (for 1.5-RC5), RepeatingView version (from Martin G.) - Tomcat
1u, 1000p, 118ms, 209,67 KB
2u, 1000p, 129ms
4u, 1000p, 141ms
8u, 1000p, 272ms
16u, 1000p, 598ms, 683mb, 90%
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 1ms
1u, 100p, 10ms
1u, 500p, 57ms
1u, 1000p, 118ms
1u, 5000p, 1198ms

Play 1.2.1 - Tomcat
1u, 1000p, 222ms, 197,98 KB, UTF-8
2u, 1000p, 259ms
4u, 1000p, 330ms
8u, 1000p, 676ms
16u, 1000p, 1309ms, 85mb, 85%
1u, 0p, 0ms
1u, 1p, 1ms
1u, 10p, 3ms
1u, 100p, 22ms
1u, 500p, 102ms
1u, 1000p, 222ms
1u, 5000p, 1054ms

Play-Scala 0.9.1 - Play 1.2.2RC1 - Tomcat
1u, 1000p, 26ms, 226,52 KB
2u, 1000p, 30ms
4u, 1000p, 85ms
8u, 1000p, 185ms
16u, 1000p, 406ms, 115mb, 62%
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 3ms
1u, 500p, 14ms
1u, 1000p, 26ms
1u, 5000p, 192ms

Play-Scala 0.9.1 - Play 1.2.2RC1 - Netty
1u, 1000p, 15ms, 226,52 KB
2u, 1000p, 16ms
4u, 1000p, 20ms
8u, 1000p, 42ms
16u, 1000p, 89ms, 572mb, 80%
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 1ms
1u, 500p, 8ms
1u, 1000p, 15ms
1u, 5000p, 79ms

Play-Japid 0.8.3.1 - Play 1.2.1 - Tomcat
1u, 1000p, 4ms, 206,05 KB
2u, 1000p, 6ms
4u, 1000p, 13ms
8u, 1000p, 28ms
16u, 1000p, 63ms, 105mb, CPU too inaccurate to measure
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 0ms
1u, 500p, 2ms
1u, 1000p, 4ms
1u, 5000p, 39ms

Play-Japid 0.8.3.1 - Play 1.2.1 - Netty (builtin)
1u, 1000p, 3ms, 206,03 KB
2u, 1000p, 4ms
4u, 1000p, 5ms
8u, 1000p, 12ms
16u, 1000p, 25ms, 533mb, CPU too inaccurate to measure
1u, 0p, 0ms
1u, 1p, 0ms
1u, 10p, 0ms
1u, 100p, 0ms
1u, 500p, 1ms
1u, 1000p, 3ms
1u, 5000p, 17ms

Lift 2.3 - Scala: 2.8.1 - Tomcat
1u, 1000p, 99ms, 220,74 KB, UTF-8
2u, 1000p, 115ms
4u, 1000p, 138ms
8u, 1000p, 280ms
16u, 1000p, 850ms, starts at 120mb, increases continuously, probably due to a session explosion (each requests creates a new session), 55%
1u, 0p, 10ms
1u, 1p, 10ms
1u, 10p, 10ms
1u, 100p, 17ms
1u, 500p, 52ms
1u, 1000p, 99ms
1u, 5000p, 633ms

Tapestry 5.2.5
1u, 1000p, 20ms, 171,41 KB
2u, 1000p, 24ms
4u, 1000p, 30ms
8u, 1000p, 65ms
16u, 1000p, 192ms, 147mb, 65%
1u, 0p, 1ms
1u, 1p, 1ms
1u, 10p, 1ms
1u, 100p, 3ms
1u, 500p, 10ms
1u, 1000p, 20ms
1u, 5000p, 100ms

Context Framework 0.8.1 / Benchmark 1.0-SNAPSHOT
1u, 1000p, 60ms, 186,88 KB
2u, 1000p, 68ms
4u, 1000p, 133ms
8u, 1000p, 367ms
16u, 1000p, 768ms, 144mb, 41%
1u, 0p, 1ms
1u, 1p, 1ms
1u, 10p, 2ms
1u, 100p, 7ms
1u, 500p, 31ms
1u, 1000p, 60ms
1u, 5000p, 325ms

Grails 1.3.7 - Freemarker 0.3 - Freemarker plugin 0.6.0
1u, 1000p, 22ms, 205,8 KB
2u, 1000p, 32ms
4u, 1000p, 43ms
8u, 1000p, 99ms
16u, 1000p, 299ms, 200mb, 60%
1u, 0p, 2ms
1u, 1p, 2ms
1u, 10p, 2ms
1u, 100p, 4ms
1u, 500p, 12ms
1u, 1000p, 22ms
1u, 5000p, 112ms

JMeter impacts performance results because JMeter needs to collect data and calculate statistics. This is more prominent the faster the response times are, because then more data will be generated. CPU utilization of JMeter sometimes reaches 50%. In these cases, my CPU utilization of the associated framework will be very inaccurate.

During testing, I noticed some high peeks in response time. This might be due to Garbage collection of Ruby. I measured the response time without including these extreme pauses.
This is more difficult to test when performing a 2 concurrent user test. In this case, Rails was showing 2 groups of response times, where the second group was about half the response time of the first group. The groups seems to interleaf in series.

Commenters have pointed out that Rails should be tested under other webservers than Webrick.

cpu utilization about 53%, where other webapp have about 85%
Lift required more initial benchmarking time before reaching stable response times, otherwise the average responsetime will drop slowly.
At 16u, the response times fluctuate a lot. I have seen long running averages at 670ms, but also at 950ms, then slowly creeping up, then slowly down etc. This might be due to barbage collection and the effect of starting at 0 sessions or measuring when a higher amount of sessions are present (even though I’m using JMeter without cookies/sessions enables). Because of this, I have set it to 850ms.
A LOT of samples are needed.

During testing at 8u and 16u, wicket gave multiple warnings/exceptions:
WARN – PageAccessSynchronizer – “http-apr-8080″-exec-16 failed to acquire lock to page 0, attempted for 1 minute out of allowed 1 minute
Caused by: org.apache.wicket.page.CouldNotLockPageException: Could not lock page 0. Attempt lasted 1 minute

A post commenter (@12) pointed that this is due to a bug that is fixed in the latest trunk version of Wicket. This trunk is tested under “Wicket-trunk” in this post.

Tapestry iterates over the categories using plain Java API (StringBuilder) instead of Tapestry. Because according to the writer of the test code (Kaosko) “the general guideline for minimizing logic in the template”. I expect that the performance of Tapestry will be lower when Tapestry solutions are used to iterate over the categories.
The benchmark test case used to test all frameworks, should be changed to use custom components for categories instead of using plain Strings to render categories.

Also note that even though I try the HTML output to be roughly the same in size, the output size of Tapestry was lower then the rest: 171,41 KB. Tapestry seems to remove excessive whitespaces.

As an example, Play-scala-netty is used to show the effects, but the same probably applies to other frameworks.
During testing, I accidentaly forgot to check the page sizes of Play-scala, and tested Play-scala. Later I saw that the page size was: 373,04 KB (for 1000 products). To reduce this size to approximately the same size in which I tested other frameworks, I removed some unneeded whitespaces from products.scala.html, by changing:

@for(category <- product.categories.toList) {
	$@category.name,
}

to

@for(category <- product.categories.toList) { $@category.name, }

The page size then was reduced to: 226,52 KB.

Here the differences in response time for:
users, products, 226.52kb vs 373,04kb
1u, 1000p, 15ms vs 21ms
2u, 1000p, 16ms vs 23ms
4u, 1000p, 20ms vs 30ms
8u, 1000p, 42ms vs 64ms
16u, 1000p, 89ms vs 137ms
1u, 0p, 0ms vs 0ms
1u, 1p, 0ms vs 0ms
1u, 10p, 0ms vs 1ms
1u, 100p, 1ms vs 2ms
1u, 500p, 8ms vs 10ms
1u, 1000p, 15ms vs 21ms
1u, 5000p, 79ms vs 111ms

(With Tomcat gzip compression enabled, the page size is reduced to a page size of about 9kb.)