Generate client object classes based on XML Schema ( .xsd files) (JAXB, Maven, Web Service)

To consume some xml based web services, sometimes we need to be able to generate java objects based on the schema that the service provider provides.

The JAXB is one tool that could help us do this and also JAXB is fully integrated with Maven.

Official Manual:
http://mojo.codehaus.org/jaxb2-maven-plugin/xjc-mojo.html

====================================================== ======================================================
Example:

pom.xml:

<dependencies>

 <dependency>
   <groupId>junit</groupId>
   <artifactId>junit</artifactId>
   <version>4.10</version>
   <scope>test</scope>
 </dependency>
 <dependency>
   <groupId>javax.xml.bind</groupId>
   <artifactId>jaxb-api</artifactId>
   <version>2.1</version>
 </dependency>
 <dependency>
   <groupId>com.sun.xml.bind</groupId>
   <artifactId>jaxb-impl</artifactId>
   <version>2.1</version>
 </dependency>
</dependencies>

<build>
 <pluginManagement>
   <plugins>
     <plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-compiler-plugin</artifactId>
       <version>2.3.1</version>
       <configuration>
         <source>1.6</source>
         <target>1.6</target>
       </configuration>
     </plugin>
   </plugins>
 </pluginManagement>
 <plugins>
   <plugin>
     <groupId>org.codehaus.mojo</groupId>
     <artifactId>jaxb2-maven-plugin</artifactId>
     <version>1.5</version>
     <configuration>
       <outputDirectory>${project.basedir}/src/main/java</outputDirectory>
       <schemaDirectory>${project.basedir}/src/main/schemas/</schemaDirectory>
       <bindingDirectory>${project.basedir}/src/main/schemas/bindings/</bindingDirectory>
       <extension>true</extension>
       <forceRegenerate>true</forceRegenerate>
     </configuration>
     <executions>
       <execution>
         <id>xjc-xxx</id>
         <goals>
           <goal>xjc</goal>
         </goals>
         <configuration>
           <schemaFiles>XXXXXXX.xsd</schemaFiles>
         </configuration>
       </execution>
     </executions>
   </plugin>
 </plugins>
</build>

We can have multiple executions, and each execution parse one xsd file. Binding file example: binding.xjb

<jaxb:bindings version="1.0" 

   xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
   xmlns:xs="http://www.w3.org/2001/XMLSchema" 

   xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc">

   <jaxb:globalBindings>
       <xjc:simple />
   </jaxb:globalBindings>

   <jaxb:bindings schemaLocation="..\cda\C32_POCD_MT000040.xsd">
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.ObservationMedia']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.RegionOfInterest']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.Section']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
   </jaxb:bindings>
</jaxb:bindings>

====================================================== ====================================================== The block:

   <jaxb:globalBindings>
       <xjc:simple />
   </jaxb:globalBindings>

The block is very useful. It could make the generated java code much clean, concise and easy to use.
For example (Get from Reference# 1):

1. Some content model definitoins, such as A,B,A, which used to cause an XJC compilation error and required manual intervention, now compile out of the box without any customization.
2. Some content model definitions that used to bind to a non-intuitive Java class now binds to a much better Java class:

   // schema
   <xs:complexType name="foo">
    <xs:choice>
     <xs:sequence>
      <xs:element name="a" type="xs:int" />
      <xs:element name="b" type="xs:int" />
     </xs:sequence>
     <xs:sequence>
      <xs:element name="b" type="xs:int" />
      <xs:element name="c" type="xs:int" />
     </xs:sequence>
    </xs:choice>
   </xs:complexType>
   
   // before
   class Foo {
       List<JAXBElement<Integer>> content;
   }
   
   // in <xjc:simple> binding
   class Foo {
       Integer a;
       int b; // notice that b is effectively mandatory, hence primitive
       Integer c;
   }
   

3. When repetable elements are bound, the method name will become plural.

   // schema
   <xs:complexType name="person">
    <xs:sequence>
     <xs:element name="child"  type="xs:string" maxOccurs="unbounded" />
     <xs:element name="parent" type="xs:string" maxOccurs="unbounded" />
    </xs:sequence>
   </xs:complexType>
   
   // before
   public class Person {
       protected List<String> child;
       protected List<String> parent;
   }
   
   // in <xjc:simple> binding
   public class Person {
       protected List<String> children;
       protected List<String> parents;
   }
   

(Once again, readers are warned that this is an experimental binding mode, and therefore the binding is subject to change in future versions of the JAXB RI without notice. Please send feedbacks on this binding to users@jaxb.dev.java.net) 4. It'll format the generated java class name, get rid of the suffix 'Type'. Sometimes, the provided xsd file may have definition like this:

<xs:element name="item" type="itemType"/>  
...   
<xs:complexType name="itemType">
    <xs:attribute name="id" type="xs:string" use="required">
    ...     
</xs:complexType>

Which is not a good practice. It will generate java object ItemType.java. But actually Item.java is what we want. Which should be defined like this :

<xs:element name="item">
    <xs:complexType>
    <xs:attribute name="id" type="xs:string" use="required">
        ...
    </xs:complexType>
</xs:element>

If it's this case, we may not want to update the schema file one by one. In stead, we can use the simple mode, then we can simply get what we want. NOTE: we must run the cmd with -extension. In the configuration of pom, we have to set <extension>true</extension>.
====================================================== ====================================================== This block:

   <jaxb:bindings schemaLocation="..\cda\C32_POCD_MT000040.xsd">
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.ObservationMedia']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.RegionOfInterest']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
       <jaxb:bindings
           node="//xs:complexType[@name='POCD_MT000040.Section']/xs:attribute[@name='ID']">
           <jaxb:property name="ID1"></jaxb:property>
       </jaxb:bindings>
   </jaxb:bindings>

It redefines the name of the target attribute.
In this case, in xs:complexType name='POCD_MT000040.ObservationMedia. There are one attribute with name 'id' and another attribute with name 'ID'. If we do not redefine the second attribute, when generating java class, it will throw exception:  "trying to create the same field twice: id"
At last, run maven to generate sources. This plug is run in phase 'generate-sources'. So we can run "mvn clean generate-sources"
Good tutorials and references about customization binding: 1. https://jaxb.java.net/nonav/2.1/docs/vendorCustomizations.html#simple 2. http://www.onjava.com/pub/a/onjava/2003/12/10/jaxb.html?page=2

WSDL Client Stub Generation (JAVA, Maven, Apache CXF)

To consume WSDL-based web service. The easiest way to do is to generate client stub to make the service call.

Apache CXF is one service framework for web services. Details could be found in official web-site:
http://cxf.apache.org/docs/overview.html

We use maven to manage client stub code, to integrate Apache CXF, we can add dependency:

<properties>
 <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 <cxf.version>2.7.5</cxf.version>
</properties>

<dependencies>
 <dependency>
   <groupId>junit</groupId>
   <artifactId>junit</artifactId>
   <version>3.8.1</version>
   <scope>test</scope>
 </dependency>
 <dependency>
   <groupId>org.apache.cxf</groupId>
   <artifactId>apache-cxf</artifactId>
   <version>${cxf.version}</version>
   <type>pom</type>
 </dependency>
</dependencies>

To generate client stub, we can use the 'cxf-codegen-plugin':

<plugin>
 <groupId>org.apache.cxf</groupId>
 <artifactId>cxf-codegen-plugin</artifactId>
 <version>${cxf.version}</version>
 <executions>
   <execution>
     <id>generate-sources</id>
     <phase>generate-sources</phase>
     <configuration>
       <sourceRoot>${basedir}/src/main/code</sourceRoot>
       <wsdlOptions>
         <wsdlOption>
           <wsdl>WSDL_URL/RELATIVE_PATH(e.g src/main/resources/wsdl/XXX.wsdl)</wsdl>

           <extraargs>
             <extraarg>-impl</extraarg>
             <extraarg>-verbose</extraarg>
             <extraarg>-client</extraarg>
             <extraarg>-p</extraarg>
             <extraarg>com.test.service.message</extraarg>
             <!-- <extraarg>-p</extraarg> <extraarg>SOME_NAMING_SPACE1=com.test.service.message.space1</extraarg>

          <extraarg>-p</extraarg> <extraarg>SOME_NAMING_SPACE2=com.test.service.message.space2</extraarg> 
          -->
           </extraargs>
         </wsdlOption>
         <wsdlOption>
           <wsdl>WSDL_URL/RELATIVE_PATH(e.g src/main/resources/wsdl/XXX2.wsdl)</wsdl>
           <extraargs>
             <extraarg>-impl</extraarg>
             <extraarg>-verbose</extraarg>
             <extraarg>-client</extraarg>
             <extraarg>-p</extraarg>
             <extraarg>com.test.service.directory</extraarg>
           </extraargs>
         </wsdlOption>
       </wsdlOptions>
     </configuration>
     <goals>
       <goal>wsdl2java</goal>
     </goals>
   </execution>
 </executions>
</plugin>

"<sourceRoot>${basedir}/src/main/code</sourceRoot>" =>Specify where to put the generated code. Instead of using this, in each "<wsdl>", can use extra args "-d" "path" to specify path for each wsdl. "<extraarg>-p</extraarg>  <extraarg>com.test.service.message</extraarg>" => Specify the package name of the generated code. In this case, if not specified, if there are multiple naming space in the wsdl, there may be multiple packages. Sometimes, it's hard to manage. So sometimes we may want to merge all the classes into one package and have one 'ObjectFactory'.
BUT I have one issue left over here, which is although I specify the package name in this way, other packages will still be generated and kept under the directory that I specified. So the result will be, in addition to the two package that I specified, other default packages will also be generated. Which is not what I want. 
If anyone knows how to update this, please leave a comment. Thank you very much!
So, because of that issue, if I want to generate a jar file that contains only two packages that I specify, I need to assemble the jar file myself. Because the plug cxf-codegen-plugin executes in phase generate-source and all the code it generates will be treated as compile source.
SO in pom.xml. I need to specify:

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-assembly-plugin</artifactId>
 <version>2.3</version>
 <executions>
   <execution>
     <id>assembly</id>
     <phase>package</phase>
     <goals>
       <goal>single</goal>
     </goals>
     <configuration>
       <finalName>${project.artifactId}</finalName>
       <appendAssemblyId>true</appendAssemblyId>
       <descriptors>
         <descriptor>src/main/assembly/service-jar.xml</descriptor>
       </descriptors>
     </configuration>
   </execution>
 </executions>
</plugin>

service-jar.xml:

<assembly

  xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
  <id>self-gen</id>
  <formats>
      <format>jar</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <dependencySets>
      <dependencySet>
          <outputDirectory>/</outputDirectory>
          <useProjectArtifact>true</useProjectArtifact>
          <includes>
              <include>${project.groupId}:${project.artifactId}</include>
          </includes>
          <unpack>true</unpack>
          <unpackOptions>
              <includes>
                      <include>com/test/service/message/*</include>
                      <include>com/test/service/directory/*</include>
              </includes>
          </unpackOptions>
          <scope>runtime</scope>
      </dependencySet>
  </dependencySets>
</assembly>

In this case, after we run "mvn clean package", we can get two jars. We use the target jar with classifier 'self-gen'.
Then we can write client code based on this jar file.  The client code which makes API calls could refer to the client java class that automatically generated.

Apache HTTP Server Setup as Load Balancer

Environment:

Win 7 64 bits, Apache Http Server (httpd) 2.4.3, Apache Tomcat 7.0.29.

1.       Download Apache Http Server:

http://httpd.apache.org/download.cgi#apache24

http://www.apachefriends.org/en/xampp-windows.html

In this test case, I use the XAMPP(1.8.1) with Apache Server version 2.4.3.

2.       Setup the XAMPP:

Run %XAMPP_HOME%/setup_xampp.bat

3.       Configure the Apache Server:

=====“httpd.conf”=====

#Add below modules under the module list:

LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so

LoadModule slotmem_shm_module modules/mod_slotmem_shm.so

#Uncomment below modules:

mod_proxy: mod_proxy_http, mod_proxy_ftp, mod_proxy_ajp, mod_proxy_balancer, and mod_proxy_connect

#Update the listen port:

#

# Listen: Allows you to bind Apache to specific IP addresses and/or

# ports, instead of the default. See also the <VirtualHost>

# directive.

#

# Change this to Listen on specific IP addresses as shown below to

# prevent Apache from glomming onto all bound IP addresses.

#

#Listen 0.0.0.0:80

#Listen [::]:80

Listen 127.0.0.1:90

NOTE: Multiple ‘ip:port’ can be listed down below, but do not put port only. Also, do not put ‘0.0.0.0:port’. Otherwise, exception will be thrown:

“C:\xampp-win32-1.8.1-VC9\xampp>apache_start.bat

Diese Eingabeforderung nicht waehrend des Running beenden

Bitte erst bei einem gewollten Shutdown schliessen

Please close this command only for Shutdown

Apache 2 is starting ...

(OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted.  : AH00072: make_sock: could not bind to address 0.0.0.0:80

AH00451: no listening sockets available, shutting down

AH00015: Unable to open logs”

If want to check which port is in use by other process, type in cmd:

netstat -a –n

#Update the Server Name:

#

# ServerName gives the name and port that the server uses to identify itself.

# This can often be determined automatically, but we recommend you specify

# it explicitly to prevent problems during startup.

#

# If your host doesn't have a registered DNS name, enter its IP address here.

#

ServerName localhost:90

=====“httpd-ssl.conf”=====

#

# When we also provide SSL we have to listen to the

# standard HTTP port (see above) and to the HTTPS port

#

# Note: Configurations that use IPv6 but not IPv4-mapped addresses need two

#       Listen directives: "Listen [::]:443" and "Listen 0.0.0.0:443"

#

#Listen 0.0.0.0:443

#Listen [::]:443

Listen 127.0.0.1:443

=====“httpd-proxy.conf”=====

#

# Implements a proxy/gateway for Apache.

# # Required modules: mod_proxy, mod_proxy_http

#

<IfModule proxy_module>

<IfModule proxy_http_module>

#

# Reverse Proxy

#

ProxyRequests Off

<Proxy balancer://mycluster>

                  BalancerMember http://127.0.0.1:8080

                  BalancerMember http://127.0.0.1:9080

                  ProxySet lbmethod=byrequests

</Proxy>

#Optional  start

<Location /balancer-manager>

   SetHandler balancer-manager

</Location>

ProxyPass /balancer-manager !   #Not proxy balancer-manager

#Optional  end

ProxyPass / balancer://mycluster/

</IfModule>

</IfModule>

NOTEI: The ‘mycluster’ is the identifier of the load balancer, can pick any value as long as starts with ‘balancer://’

NOTEII: The ‘balancer-manager’ is one optional configuration, this is one web UI of the load balancer manager. You can choose to use it, or not. URL is http://localhost:90/balancer-manager

NOTEIII: IMPORTANT! ProxyPass /balancer-manager !   , this is to not proxy balancer-manager to tomcat.

NOTEIV: IMPORTANT! ProxyPass / balancer://mycluster/, this is to proxy url from base ‘/’, and the trailing slash is very important. If not have, you’ll have 500 http error code. If you check the error log, below warning will be shown:

“[Tue Jun 18 11:32:39.296798 2013] [proxy:warn] [pid 36488:tid 1732] [client 127.0.0.1:59079] AH01144: No protocol handler was valid for the URL /error/HTTP_BAD_GATEWAY.html.var. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.”

Configuration Summary

You can also group things together by using ‘VirtualHost’.

=====”httpd-vhosts.conf”=====

<VirtualHost *:90>

        ProxyRequests off

        ServerName localhost:90

        <Proxy balancer://mycluster>

                BalancerMember http://127.0.0.1:8080

                BalancerMember http://127.0.0.1:9080

                # Security "technically we aren't blocking

                # anyone but this the place to make those

                # chages

                Order Deny,Allow

                Deny from none

                Allow from all

                # Load Balancer Settings

                # We will be configuring a simple Round

                # Robin style load balancer.  This means

                # that all webheads take an equal share of

                # of the load.

                ProxySet lbmethod=byrequests

        </Proxy>

        # balancer-manager

        # This tool is built into the mod_proxy_balancer

        # module and will allow you to do some simple

        # modifications to the balanced group via a gui

        # web interface.

        <Location /balancer-manager>

                SetHandler balancer-manager

                # I recommend locking this one down to your

                # your office

                Order deny,allow

                Allow from all

        </Location>

        # Point of Balance

        # This setting will allow to explicitly name the

        # the location in the site that we want to be

        # balanced, in this example we will balance "/"

        # or everything in the site.

        ProxyPass /balancer-manager !

        ProxyPass / balancer://mycluster/

</VirtualHost>

4.       Start two Tomcat instances

5.       Start Apache Http server (MUST run as administrator)

Either run the bat file %XAMPP_HOME%/apache_start.bat

Or start XAMPP’s control panel and start the server.

If any error happens, the error log is a good place that could help debug.

6.       Test load balance

For example, I run the web app that test the central session.

http://localhost:90/centralcache-test/testlargeusersession

Bingo. Happy ending.

This is a very very basic configuration of apache http server as load balancer. Main goal is to make it work. For further usage, please refer to the official web site.

http://httpd.apache.org/docs/2.4/

CentralCache Management(Memcache) Issue 1 -- net.spy.memcached.internal.CheckedOperationTimeoutException

One main aspect to evaluate one web app is its scalability, whether it can sustain heavy load.
So when we decide to build one web app, each technology that we plan to use must go through enough tests, not only functional test, but also load test. Otherwise, we may lose at the start point.

When I test the performance of tomcat-memcache central session management, I save&get small session data and large session data with both light load and heavy load.

When load is light, everything goes fine. But when load increase, and session data is around 10K, exception will be thrown:

net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: localhost/127.0.0.1:11212
        at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:160)
        at de.javakaffee.web.msm.LockingStrategy.onAfterBackupSession(LockingStrategy.java:294)
        at de.javakaffee.web.msm.MemcachedSessionService.backupSession(MemcachedSessionService.java:1062)
        at de.javakaffee.web.msm.RequestTrackingHostValve.backupSession(RequestTrackingHostValve.java:243)
        at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:168)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
        at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1770)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

Regarding this issue, I found one discussion page:
https://code.google.com/p/spymemcached/issues/detail?id=136
But I did not see effect solution. Maybe switch to other memcache client lib will be a better solution.
Somebody says that increase the spymemcache's timeout may help, but I haven't got time to test it.
Compared to memcache, now I prefer mongodb as the backup of session. At least it's client lib is more stable and under enough test.

So If we need to setup the central cache for web app, we need to customize and test the lib well based on our requirement. I think no one wants to face this issue in production.

Central Session Management (Tomcat, Memcache/MongoDB, Non-Sticky Session)

For all application server clusters, there are two elementary issues that need to be resolved:

1. How to dispatch requests to the cluster nodes.
2. Use one session management strategy, to ensure that if one node fails, other nodes can still obtained the session data, to achieve fault-tolerant cluster (failover)

For the first issue, the easiest way is to use uniform hash function to dispatch the request to each node. But for application server, session is used. To make sure one session’s request is properly handled, normally we dispatch the requests related to one session to same node to process, this is “Sticky Session” load balance. And the way randomly dispatch request is called “non-sticky session” load balance.

The “sticky session” sometimes restricts the load balance and even the web app’s design and development. To avoid “sticky session”, we need to find appropriate session management strategy.

There are several options for session share.

Session Share Options

Tomcat Clustering

It simply copied session to all the tomcat instances. If there are many instances, this may cause a lot network traffic. 

Save session related data to database

Actually some apps choose this way to make session management under control.

There are two ways to do this,

1.  Customize 'session' in the app. In this case, "session" is not HTTP session any more, it just means data that used as session data. 
   
   Each Tomcat instances can access the central db to get the session related data and load the data into its own cache/session. Also each instances can write data into db for session share purposes.
   
   Each request need to call the db for twice, one get session, one update session.
   Good point: data is fully under control.
   Bad point: database needs to be periodically clean up.
2. Extends Tomcat Manager class and customize your own session manager, backup your tomcat session data into MongoDB.
   
   Reference implementation:
   https://github.com/dawsonsystems/Mongo-Tomcat-Sessions
   https://github.com/dwelch2344/mongo-tomcat-session-manager
   
   Personally I like this way, because MongoDB is a mature product and in additional, its client lib(driver) is also robust. And it's fast. Although the speed is not that fast compared to backup method like memcache which backup session in memory, but it's still fast enough and the data is more safe. But still, old topic, all the decision should be made based on our own requirement.
   
   Little Note: In this way, we do not need extra periodical job to clear the session in db. We can override the backgroundProcess() method inheritted from Tomcat's Manager class.

Session in Cookie

This way is to save session data in the client side instead of server side. Each time client make the request, send the session back to server. In this way, developers can make control on session consume. But known issues is security concern. 

Central Cache (External Cache)

External Cache is a separate service that runs on the machine in the LAN. Use external cache can realize central session management. 

It stores the session in memory and all the tomcat instances can access the central cache to read session and set session. There is only one copy of for each session and all the tomcat instances share the same one.

Normally, there should be at least one central cache node and one backup cache node.

There are some implementations for central cache,  memcached (open-source) and Terracotta (open-source, commercial). Central cache has been proved to be fast and reliable, and many large web sites use central cache for session management.

But to set up central cache, may need professional system engineer to setup environment, maintain and monitor.

Summary

The core idea of using Central Cache and MongoDB is actually same, just backup session into one central place for reference. The only difference is cache saves data in memory and db saves data in hard drive.

We can pick one way based on requirement. Both ways have reference implementation on tomcat session manager.

Central Session Mangement - MongoDB Test Case

To test the central cache, I setup the environment and build one test web app.

central-session-management-mongo-test.zip

Structure:

Environment:

Tomcat 7,

Java Default Serializer,

mongodb-win32-x86_64-2.0.7 (Windows 64 bits version)

#Start mongodb

1. mongobin \>mongod.exe --dbpath=”DATA_PATH”

2. In another console start MongoDB client console by executing mongo.exe then execute the following commands.

   use tomcatsessions

   db.addUser("sessionadmin", "sessionadmin234")

3. Stop the MongoDB server (i.e. press Ctrl+C)

4. When the prompt shows, restart the MongoDB server with the --auth parameter:

   mongobin\>mongod.exe  --auth  --dbpath=”DATA_PATH”

 

#Start two tomcat instances

1. Run tomcat-instance\apache-tomcat-7.0.29-1\bin\startup.bat

   Port: 8080

1. Run tomcat-instance\apache-tomcat-7.0.29-2\bin\startup.bat

   Port: 9080

   

Test Case:

#Put String in the session

Access:

 http://localhost:8080/centralcache-test/SessionPage1

 1. Get the session, if not exist, create a new one.

 2. Add attribute "currentUser": "marym" into session

 Page will show:

 Current session Id: ${session id}

 CurrentUser: marym

Then Access:

 http://localhost:9080/centralcache-test/SessionPage2

 1. Get the session, if not exist, create a new one.

 2. Print attribute "currentUser" in the session.

 Page will show:

 Current session Id: ${session id}

 CurrentUser: marym

 

#Put User Object in the session

User{username,password}

Access:

 http://localhost:8080/centralcache-test/login

 http://localhost:9080/centralcache-test/login

 

 If user has not logged in, it will direct to login page. 

 If user has logged in (session has valid attribute "currentUser"), it will display the logged in user's username.  

 If user logout, the "currentUser" in session will be removed. (Not destroy the whole session)

 

Two tomcats share the session. If shut down the first memcache node at port 11211, 

the backup node at port 11212 will be used. Session will not be lost.

Note: Different client side apps do not share session. E.g. IE, Chrome, FireFox do not share session.

#Test Put 10K around session data into session

 http://localhost:8080/testlargeusersession

 http://localhost:9080/testlargeusersession

#Test Put 100bytes around session data into session

 http://localhost:8080/testsmallusersession

 http://localhost:9080/testsmallusersession

 

#Test Multi-thread Support

Open Jmeter Test Case.

Run.

I test 10000 concurrent threads with session data around 10K. Works pretty good.

Central Cache Test Case

To test the central cache, I setup the environment and build one test web app.

central-session-test.zip

Structure:

Environment:

Tomcat 7,
memcached-session-manager-1.6.4

memcached-amd64 (Windows 64 bits version)

#Start two memcache nodes

1. Run memcached-amd64\run-instance1.bat

   Port: 11211

2. Run memcached-amd64\run-instance2.bat

   Port: 11212

#Start two tomcat instances

1. Run tomcat-instance\apache-tomcat-7.0.29-1\bin\startup.bat

   Port: 8080

1. Run tomcat-instance\apache-tomcat-7.0.29-2\bin\startup.bat

   Port: 9080

   

Test Case:

#Put String in the session

Access:

 http://localhost:8080/centralcache-test/SessionPage1

 1. Get the session, if not exist, create a new one.

 2. Add attribute "currentUser": "marym" into session

 Page will show:

 Current session Id: ${session id}

 CurrentUser: marym

Then Access:

 http://localhost:9080/centralcache-test/SessionPage2

 1. Get the session, if not exist, create a new one.

 2. Print attribute "currentUser" in the session.

 Page will show:

 Current session Id: ${session id}

 CurrentUser: marym

#Put User Object in the session

User{username,password}

Access:

 http://localhost:8080/centralcache-test/login

 http://localhost:9080/centralcache-test/login

 If user has not logged in, it will direct to login page. 

 If user has logged in (session has valid attribute "currentUser"), it will display the logged in user's username.  

 If user logout, the "currentUser" in session will be removed. (Not destroy the whole session)

Two tomcats share the session. If shut down the first memcache node at port 11211, 

the backup node at port 11212 will be used. Session will not be lost.

Note: Different client side apps do not share session. E.g. IE, Chrome, FireFox do not share session.

Sample configuration:

1. Add dependencies (jars that should be placed under tomcat\lib):

    

    For Tomcat 7

• memcached-session-manager-1.6.4.jar
• memcached-session-manager-tc7-1.6.4.jar
• spymemcached-2.8.12.jar
• couchbase-client-1.1.2.jar

    For Tomcat 6

• memcached-session-manager-1.6.4.jar
• memcached-session-manager-tc6-1.6.4.jar
• spymemcached-2.8.12.jar
• couchbase-client-1.1.2.jar

    
    I've put all these dependencies in the zip file central-session-test.zip

2. Tomcat /conf/context.xml  (All instances use the same conf, non-sticky session)

    #Use Javolution Serializer

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
   memcachedNodes="n1:localhost:11212,n2:localhost:11213"
   sticky="false"
   sessionBackupAsync="false"
   lockingMode="uriPattern:/path1|/path2"
   requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
   transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
/>

      Extra Dependencies(jars that should be placed under tomcat\lib):

• javolution-5.4.3.1.jar
• msm-javolution-serializer-1.6.4.jar

 
 #Use Kryo Serializer

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
   memcachedNodes="n1:localhost:11212,n2:localhost:11213"
   sticky="false"
   sessionBackupAsync="false"
   lockingMode="uriPattern:/path1|/path2"
   requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
   transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
/>

      Extra Dependencies(jars that should be placed under tomcat\lib):

• msm-kryo-serializer-1.6.4.jar
• kryo-1.04.jar
• kryo-serializers-0.10.jar
• asm-3.2.jar
• reflectasm-1.01.jar
• minlog-1.2.jar

     Configure Kryo's buffer size.
      By default the initial buffer size is 100K, and maxium is 2M.
      If the session data is big, then we need to update these two values by adding one system property.

      In Tomcat's startup.bat file:

      set JAVA_OPTS=%JAVA_OPTS% -Dmsm.kryo.buffersize.initial=10444800 -Dmsm.kryo.buffersize.max=104448000

3. Start MemCache

Unzip memcached-amd64.zip, Go to memcached-amd64 directory.

memcached -p 11211 -u memcached -m 64 -M -vv

memcached -p 11212 -u memcached -m 64 -M -vv

-vv tells memcached to write lots of stuff to console, so you'll see when a session is requested or stored in the output of memcached.

4. Start Tomcat
    ->Tomcat Base/bin/start.bat

Reference:
---memcached-session-manager WIKI ---SetupAndConfiguration

---Various versions of Memcached for Windows

---memcached-session-manager Source Code

---Memcached on Windows and telnet interface

Friendly Reminder:

Please take care on the version number of extra lib that added into Tomcat's library. There are many difference versions of jar. If use wrong version, it may cause jar conflict or class missing.

In my above example. I use memcached-session-manager-1.6.4, then the extra dependency jar files that I listed is based on version 1.6.4 of  memcached-session-manager. If use other different version of memcached-session-manager, please check the dependencies' version and adjust if needed.


Little Notes:

=====Issue 1=====

If you wanna to install the Memcached for Windows and the follow instruction in Memcached on Windows and telnet interface, it will throw exception. "Failed to ignore SIGHUP: Result too large". To resolve this, can follow below steps:

1) run CMD as an administrator
2) type SC create memcached binpath= "c:\memcached\145\memcached.exe -m 512
-d"
3) type NET START memcahed

Although you get an error, the program started (check task manager or connect to it using telnet).

=====Issue 2=====

If you configure like this:

Tomcat context.xml:

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
   memcachedNodes="n1:localhost:11212"
   sticky="false"
   sessionBackupAsync="false"
   lockingMode="uriPattern:/path1|/path2"
   requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
   transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
/>

Error:

java.lang.NoSuchMethodError: net.spy.memcached.MemcachedClient.set(Ljava/lang/String;ILjava/lang/Object;)Lnet/spy/memcached/internal/OperationFuture;

at de.javakaffee.web.msm.BackupSessionTask.storeSessionInMemcached(BackupSessionTask.java:227)

at de.javakaffee.web.msm.BackupSessionTask.doBackupSession(BackupSessionTask.java:194)

at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:119)

at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:50)

at de.javakaffee.web.msm.BackupSessionService$SynchronousExecutorService.submit(BackupSessionService.java:346)

at de.javakaffee.web.msm.BackupSessionService.backupSession(BackupSessionService.java:205)

at de.javakaffee.web.msm.MemcachedSessionService.backupSession(MemcachedSessionService.java:1059)

at de.javakaffee.web.msm.RequestTrackingHostValve.backupSession(RequestTrackingHostValve.java:229)

at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:154)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)

at java.lang.Thread.run(Thread.java:619)

Solution:

Update memcached jar to spymemcached-2.8.12.jar or spymemcached-2.8.4.jar

=====Issue 3=====

memcached-session-manager provides several serializers. The testing of this part will be processed later, applicability, efficiency, performance, etc.