Page load times and performance

[craig]

I have the displeasure of maintaining a mango instance over a satellite link. The following has made the pages load much faster,from 30+ second page load times to 2.5s (once cache is primed). Fast links and servers may not notice much.

1. Enable gzip compression in tomcat:
   in conf/server.xml

    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" 
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/xml,text/javascript,application/x-javascript,application/javascript" />

2. I rebuilt the dojo.js file to include every module that mango uses. This saves several round trips which take a long time with 800ms latency. While the file is bigger, it is now gzipped so actually is smaller. Here is the profile file which goes in buildscripts/profiles with the dojo source distribution available from http://build.dojotoolkit.org/0.4.3/

var dependencies = [ 
	"dojo.lang.common",
	"dojo.lang.array",
	"dojo.lang.extras",
	"dojo.lang.declare",
	"dojo.lang.func",
	"dojo.event.*",
	"dojo.string.common",
	"dojo.string.extras",
	"dojo.io.*",
	"dojo.io.BrowserIo",
	"dojo.io.cookie",
	"dojo.json",
	"dojo.html.*",
	"dojo.html.display",
	"dojo.html.layout",
	"dojo.html.util",
	"dojo.lfx.shadow",
	"dojo.dnd.HtmlDragMove",
	"dojo.widget.ContentPane",
	"dojo.namespaces.dojo",
	"dojo.widget.FloatingPane",
	"dojo.widget.html.layout",
	"dojo.widget.Dialog",
	"dojo.widget.ResizeHandle",
	"dojo.widget.SplitContainer",
	"dojo.widget.Tree",
	"dojo.widget.TreeNode",
	"dojo.widget.TreeSelector",
	"dojo.widget.TreeBasicController",
	"dojo.dnd.TreeDragAndDrop"

];

// NOTE: this MUST be included or a list of files must be output via print()
// manually.
load("getDependencyList.js");

Then dojo 0.4 is built by

ant -Dprofile=mango clean release intern-strings strip-resource-comments

3. I've added an HttpResponseHeaderFilter which adds an Expires header so that static content may be cached. This is particularly useful because it saves a round trip for every icon and also allows the dojo.js file to be cached.

=== added file 'src/com/serotonin/mango/web/filter/HttpResponseHeaderFilter.java'
--- src/com/serotonin/mango/web/filter/HttpResponseHeaderFilter.java	1970-01-01 00:00:00 +0000
+++ src/com/serotonin/mango/web/filter/HttpResponseHeaderFilter.java	2009-10-25 22:27:03 +0000
@@ -0,0 +1,46 @@
+package com.serotonin.mango.web.filter;
+
+import java.io.IOException;
+import java.util.Enumeration;
+
+import javax.servlet.Filter;
+import javax.servlet.FilterChain;
+import javax.servlet.FilterConfig;
+import javax.servlet.ServletException;
+import javax.servlet.ServletRequest;
+import javax.servlet.ServletResponse;
+import javax.servlet.http.HttpServletResponse;
+/**
+ * http://juliusdev.blogspot.com/2008/06/tomcat-add-expires-header.html
+ * http://onjava.com/pub/a/onjava/2004/03/03/filters.html
+ *
+ */
+public class HttpResponseHeaderFilter implements Filter {
+
+	FilterConfig fc;
+	
+	@Override
+	public void init(FilterConfig filterConfig) throws ServletException {
+		this.fc = filterConfig;
+	}
+	
+	@Override
+	public void destroy() {
+		this.fc = null;
+	}
+
+	@Override
+	public void doFilter(ServletRequest request, ServletResponse servletResponse,
+			FilterChain chain) throws IOException, ServletException {
+
+		 HttpServletResponse response =	(HttpServletResponse) servletResponse;
+
+	      for (Enumeration<String> e = fc.getInitParameterNames(); e.hasMoreElements();) {
+	        String headerName = e.nextElement();
+	        response.addHeader(headerName, fc.getInitParameter(headerName));
+	      }
+	      // pass the request/response on
+	      chain.doFilter(request, response);
+	}
+
+}

=== modified file 'war/WEB-INF/web.xml'
--- war/WEB-INF/web.xml	2009-10-20 21:08:00 +0000
+++ war/WEB-INF/web.xml	2009-10-25 22:43:37 +0000
@@ -116,6 +116,25 @@
     <url-pattern>*.shtm</url-pattern>
   </filter-mapping>
 
+	<filter>
+	  <filter-name>HttpResponseHeaderFilter</filter-name>
+	  <filter-class> com.serotonin.mango.web.filter.HttpResponseHeaderFilter</filter-class>
+	  <init-param>
+	    <param-name>Cache-Control</param-name>
+	    <param-value>max-age=5184000</param-value>
+	  </init-param>
+	</filter>
+
+	<filter-mapping>
+	  <filter-name>HttpResponseHeaderFilter</filter-name>
+	  <url-pattern>/resources/*</url-pattern>
+	</filter-mapping>
+	
+	<filter-mapping>
+	  <filter-name>HttpResponseHeaderFilter</filter-name>
+	  <url-pattern>/images/*</url-pattern>
+	</filter-mapping>
+
   
   <!-- 
     Servlet definitions.

4. The DWR javascript files appear to be dynamically generated for every page load but are in fact static scripts, with the expcetion of engine.js, which is 99.9% static. Jawr (https://jawr.dev.java.net/integration/dwr.html) lets the dwr scripts be gzipped and cached and takes care of the dynamic portion of engine.js. DWR must be upgraded from 2.0.1 which ships with mango 1.7 to dwr 2.0.5. DWR 3 will take care of this without jawr but isn't released or ready. You will also have to download jawr and add it to WEB-INF/lib.

=== modified file 'war/WEB-INF/tags/page.tag'
--- war/WEB-INF/tags/page.tag	2009-10-23 08:49:32 +0000
+++ war/WEB-INF/tags/page.tag	2009-11-04 03:45:29 +0000
@@ -41,23 +41,34 @@
   <jsp:invoke fragment="styles"/>
   
   <!-- Scripts -->
-  <script type="text/javascript">var djConfig = { isDebug: true };</script>
+  
+<%@ taglib uri="http://jawr.net/tags" prefix="jwr" %>
+<jwr:script src="/jsBundle/CommonDwr.js"/>
+
+
+  
+  <script type="text/javascript">var djConfig = { isDebug: false };</script>
   <!-- script type="text/javascript" src="http://o.aolcdn.com/dojo/0.4.2/dojo.js"></script -->
   <script type="text/javascript" src="resources/dojo/dojo.js"></script>
-  <script type="text/javascript" src="dwr/engine.js"></script>
-  <script type="text/javascript" src="dwr/util.js"></script>
-  <script type="text/javascript" src="dwr/interface/MiscDwr.js"></script>
+ <!--  <script type="text/javascript" src="dwr/engine.js"></script>
+  <script type="text/javascript" src="dwr/util.js"></script> 
+  <script type="text/javascript" src="dwr/interface/MiscDwr.js"></script>-->
   <!-- <script type="text/javascript" src="resources/soundmanager2-nodebug-jsmin.js"></script> -->
   <script type="text/javascript" src="resources/common.js"></script>
+  
+ <c:forEach items="${dwr}" var="dwrname">
+  	<jwr:script src="/jsBundle/${dwrname}.js"/>
+</c:forEach>  
+  
   <c:forEach items="${dwr}" var="dwrname">
-    <script type="text/javascript" src="dwr/interface/${dwrname}.js"></script></c:forEach>
+    <!-- <script type="text/javascript" src="dwr/interface/${dwrname}.js"></script>--></c:forEach>
   <c:forEach items="${js}" var="jsname">
     <script type="text/javascript" src="resources/${jsname}.js"></script></c:forEach>
   <script type="text/javascript">
     mango.i18n = <sst:convert obj="${clientSideMessages}"/>;
   </script>
   <c:if test="${!simple}">
-    <script type="text/javascript" src="resources/header.js"></script>
+    <script type="text/javascript" src="resources/header.js"></script> 
     <script type="text/javascript">
       dwr.util.setEscapeHtml(false);
       dwr.engine.setErrorHandler(dwrErrorHandler);
@@ -172,4 +183,4 @@
 </c:if>
 
 </body>
-</html>
\ No newline at end of file
+</html>

=== modified file 'war/WEB-INF/web.xml'
--- war/WEB-INF/web.xml	2009-10-20 21:08:00 +0000
+++ war/WEB-INF/web.xml	2009-11-04 03:45:29 +0000
@@ -252,5 +252,26 @@
     <error-code>404</error-code>
     <location>/exception/404.jsp</location>
   </error-page>
-  
+
+        <servlet>
+                <servlet-name>JavascriptServlet</servlet-name>
+                <servlet-class>net.jawr.web.servlet.JawrServlet</servlet-class>
+                
+                <!-- Location in classpath of the config file -->
+                <init-param>
+                        <param-name>configLocation</param-name>
+                        <param-value>/jawr.properties</param-value>
+                </init-param>
+				<init-param>
+				<param-name>mapping</param-name>
+				<param-value>/jsBundle/</param-value>
+				</init-param>         
+                <load-on-startup>3</load-on-startup>
+        </servlet>
+            
+        <servlet-mapping>
+                <servlet-name>JavascriptServlet</servlet-name>
+				<url-pattern>/jsBundle/*</url-pattern>
+        </servlet-mapping> 
+
 </web-app>

=== added file 'war/WEB-INF/classes/jawr.properties'
--- war/WEB-INF/classes/jawr.properties	1970-01-01 00:00:00 +0000
+++ war/WEB-INF/classes/jawr.properties	2009-11-04 03:45:14 +0000
@@ -0,0 +1,74 @@
+# Common properties
+jawr.debug.on=false
+jawr.gzip.on=true
+jawr.gzip.ie6.on=false
+jawr.charset.name=UTF-8
+
+
+jawr.dwr.mapping=/dwr/
+jawr.js.bundle.basedir=/js
+jawr.js.use.cache=true
+# you may have to join this line back into one long line 
+jawr.js.bundle.names=MangoDwr, WatchListDwr, DataSourceListDwr, CompoundEventsDwr,\
 DataPointDetailsDwr, DataPointEditDwr, DataSourceEditDwr, EmportDwr, EventHandlersDwr, \
MailingListsDwr, PointHierarchyDwr, PointLinksDwr, PublisherEditDwr, PublisherListDwr, ReportsDwr,\
 ScheduledEventsDwr, SystemSettingsDwr, UsersDwr, ViewDwr
+
+jawr.js.bundle.MangoDwr.id=/jsBundle/CommonDwr.js
+jawr.js.bundle.MangoDwr.mappings=dwr:_engine, dwr:_util, dwr:MiscDwr
+jawr.js.bundle.MangoDwr.global=true;
+
+jawr.js.bundle.WatchListDwr.id=/jsBundle/WatchListDwr.js
+jawr.js.bundle.WatchListDwr.mappings=dwr:WatchListDwr
+
+jawr.js.bundle.DataSourceListDwr.id=/jsBundle/DataSourceListDwr.js
+jawr.js.bundle.DataSourceListDwr.mappings=dwr:DataSourceListDwr
+
+jawr.js.bundle.CompoundEventsDwr.id=/jsBundle/CompoundEventsDwr.js
+jawr.js.bundle.CompoundEventsDwr.mappings=dwr:CompoundEventsDwr
+
+jawr.js.bundle.DataPointDetailsDwr.id=/jsBundle/DataPointDetailsDwr.js
+jawr.js.bundle.DataPointDetailsDwr.mappings=dwr:DataPointDetailsDwr
+
+jawr.js.bundle.DataPointEditDwr.id=/jsBundle/DataPointEditDwr.js
+jawr.js.bundle.DataPointEditDwr.mappings=dwr:DataPointEditDwr
+
+jawr.js.bundle.DataSourceEditDwr.id=/jsBundle/DataSourceEditDwr.js
+jawr.js.bundle.DataSourceEditDwr.mappings=dwr:DataSourceEditDwr
+
+jawr.js.bundle.EmportDwr.id=/jsBundle/EmportDwr.js
+jawr.js.bundle.EmportDwr.mappings=dwr:EmportDwr
+
+jawr.js.bundle.EventHandlersDwr.id=/jsBundle/EventHandlersDwr.js
+jawr.js.bundle.EventHandlersDwr.mappings=dwr:EventHandlersDwr
+
+jawr.js.bundle.EventsDwr.id=/jsBundle/EventsDwr.js
+jawr.js.bundle.EventsDwr.mappings=dwr:EventsDwr
+
+jawr.js.bundle.MailingListsDwr.id=/jsBundle/MailingListsDwr.js
+jawr.js.bundle.MailingListsDwr.mappings=dwr:MailingListsDwr
+
+jawr.js.bundle.PointHierarchyDwr.id=/jsBundle/PointHierarchyDwr.js
+jawr.js.bundle.PointHierarchyDwr.mappings=dwr:PointHierarchyDwr
+
+jawr.js.bundle.PointLinksDwr.id=/jsBundle/PointLinksDwr.js
+jawr.js.bundle.PointLinksDwr.mappings=dwr:PointLinksDwr
+
+jawr.js.bundle.PublisherEditDwr.id=/jsBundle/PublisherEditDwr.js
+jawr.js.bundle.PublisherEditDwr.mappings=dwr:PublisherEditDwr
+
+jawr.js.bundle.PublisherListDwr.id=/jsBundle/PublisherListDwr.js
+jawr.js.bundle.PublisherListDwr.mappings=dwr:PublisherListDwr
+
+jawr.js.bundle.ReportsDwr.id=/jsBundle/ReportsDwr.js
+jawr.js.bundle.ReportsDwr.mappings=dwr:ReportsDwr
+
+jawr.js.bundle.ScheduledEventsDwr.id=/jsBundle/ScheduledEventsDwr.js
+jawr.js.bundle.ScheduledEventsDwr.mappings=dwr:ScheduledEventsDwr
+
+jawr.js.bundle.SystemSettingsDwr.id=/jsBundle/SystemSettingsDwr.js
+jawr.js.bundle.SystemSettingsDwr.mappings=dwr:SystemSettingsDwr
+
+jawr.js.bundle.UsersDwr.id=/jsBundle/UsersDwr.js
+jawr.js.bundle.UsersDwr.mappings=dwr:UsersDwr
+
+jawr.js.bundle.ViewDwr.id=/jsBundle/ViewDwr.js
+jawr.js.bundle.ViewDwr.mappings=dwr:ViewDwr
+
+

[mlohbihler]

Brilliant stuff, Craig. Thanks for sharing.

[mlohbihler]

Hey Craig,

Much of this has been ported into the next version of Mango. There doesn't seem to be a way to enable GZIP by web app, so that part will have to be the responsibility of the installer.

The dojo profile seems to have a dramatic effect on page load times, so thanks for that.

Regarding the headers, i folded the DWR stuff into this as well instead of going the jawr route. I'm pretty sure the 0.1% of engine.js that is dynamic is not used by Mango, so it is safe to cache the file. (Just for comet i think.) So far everything looks good (and faster!).

Here's the web.xml config i used:

  <filter-mapping>
    <filter-name>CacheHeaders</filter-name>
    <url-pattern>/resources/*</url-pattern>
  </filter-mapping>

  <filter-mapping>
    <filter-name>CacheHeaders</filter-name>
    <url-pattern>/images/*</url-pattern>
  </filter-mapping>

  <filter-mapping>
    <filter-name>CacheHeaders</filter-name>
    <url-pattern>/dwr/interfaces/*</url-pattern>
  </filter-mapping>

  <filter-mapping>
    <filter-name>CacheHeaders</filter-name>
    <url-pattern>/dwr/engine.js</url-pattern>
  </filter-mapping>

  <filter-mapping>
    <filter-name>CacheHeaders</filter-name>
    <url-pattern>/dwr/utils.js</url-pattern>
  </filter-mapping>

[mlohbihler]

Oh, i also added audio and graphics to cached directories.