Wednesday, March 24, 2010

Killer Servlet: a performance tuning and garbage collection solution

An applet is a Java application executed by a web browser to show dynamic graphics on the front-end. After the advent of Rich Internet Application (RIA) technologies, applets are disappearing from the tech landscape but once upon a time it was the only way to show rich contents on the front-end. A servlet in brief is a piece of Java code executed on the server side (typically a web server with an embedded servlet engine) to process request, collect data from data source or rendering a well formatted HTML page on the browser.
An applet-servlet communication is typically done where one can not avoid having a thick client such as an applet on the front-end posting requesting and / or receiving feed from the server side. For example, suppose an applet is continuously drawing and updating a graph based on the feeds received from the servers on the stock prices.
Whenever a user is opening an applet on his/her browser, the browser is downloading the applet and starts executing it. The applet draws a nice graph on the front-end (browser) and on the background connects to a server to fetch stock prices. When we had such a situation back in 2001/2002 (original situation has been modified a little to keep the privacy of the project intact) the architecture followed to implement this is as follows.
Among the technical components there were

• A Stock Price Applet with
o a nice graph panel
o a thread to render the graph on the front-end
o a thread to communicate with a servlet at the back-end

• A Stock Price Servlet (HTTPServlet) on the server with a “while” loop in the doPost() method and having some sub-components as below.
o a JMS Listener attached to the Stock price JMS topic
o a shared message cache where the subscriber stored the messages and the servlet used to read from.

• A Stock price JMS Topic on the JMS server

• Multiple Stock price publishers connected with different external Stock servers on the outer end and with the Stock price JMS topis on the inner end (in the server). Figure 1 explains the architecture.


Figure 1

The applet used to run on the browser and communicate (by opening a http stream) with the StockPriceServlet at the back-end. The servlet used to instantiate the JMS subscriber on receiving the applet’s request. The JMS subscriber was connected with the Stock price JMS topic to listen to Stock price update messages. On the other hand the JMS publishers kept adding messages as and when received from the external servers. To keep the http session alive between the applet and the servlet an infinite “while” loop is placed in the doPost() method of the servlet. The communication was a server push within an http session continuation.

Everything was nice and smooth but after serving a certain number of clients (i.e. applets) the server was getting slower and slower. It was a real-time application and delays to a certain extent was permissible but not beyond that.
So we started investigating the reason behind it. We discovered that the StockPriceServlets those were serving the applets on the front-end were kept alive even after the applet window was closed. The reason was the infinite loop in the doPost() method. The JMS subscriber was still alive listening to the JMS topic and throwing the messages to the servlet. The StockPriceServlet was writing the messages on the http response stream without any errors/exceptions because it was unable to sense that the receiver on the other side of the stream (i.e. the applet) has been stopped.

There was an option to put a timeout on the servlet so after a certain period pf time it dies out but that was not optimal. With an increased frequency of updates received by the stock servers the system might choke anytime.
Another option was to send a completion request from the applet to the servlet so that it can come out of the loop. But once the servlet receives the first request from the applet it starts the JMS subscriber and goes inside the loop. Http communications are stateless therefore a new request will invoke a new servlet.

Looking for more options we figured out a way to stop the StockPriceServlet and free up the resources with a programmatic technique. It was quite clear that the servlet needed to be killed on the completion of the applet’s activities. So an event or update needs to be thrown back to the servlet notifying that the applet has been stopped. Based on this update the servlet had to be killed. So we felt the need of an agent on the server side who will receive the notification from the applet’s end. The agent needs to accomplish the following activities.

1. Receives the completion notification from the applet.
2. Identifies the StockPriceServlet instance that was serving the applet.
3. Notify the servlet by updating a flag that makes the servlet to break out from the infinite loop and stop the subscriber attached to it.

So the agent took birth as the KillerServlet (please don’t laugh :) it’s the birth of the terminator ;). Activity 1 can be easily achieved by sending a http request from the applet to the KillerServlet. The most critical was the step 2 i.e. to identify the servlet instance. All servlet instances were maintained by the servlet container and hence getting a hold with them is tough. Well, getting the reference of a servlet from the ServletContext method was easy and official during 2001/2002 , but the issue was to identify the exact subject for kill i.e. a correlation. To establish a correlation the participants in the communication channel i.e. the applet and the servlet should be marked / tagged with a unique id (a correlation id) that can be used later to get a reference of the participants.
Figure 2 explains the modified architecture to include the solution. The solution technique was conceptualized as below.

Figure 2

1. The applet connects with the StockPriceServlet.
2. The StockPriceServlet generates a unique id (correlation id) and sends back the same to the applet with the first response before entering the loop. The StockPriceServlet should have a flag to terminate the while loop e.g. “while(!completed){}” instead of “while(true){}”. The StockPriceServlet should also have a unique id generator (preferably a singleton object).
3. The StockPriceServlet sets an attribute to the ServletContext (ServletContext.setAttribute()) with its own reference against the unique id.
4. Once the applet receives the unique id it should store the same.
5. Once the applet is stopped or terminated it should be able to sense the same (may be by using a JavaScript or a close button) and invoke the KillerServlet with the unique id.
6. The KillerServlet receives the unique id and looks up for the same in the ServletContext. (ServletContext.getAttribute()).
7. On receiving the appropriate reference to the StockPriceServlet instance the KillerServlet should update the completion flag to “true” in the StockPriceServlet instance.
8. Once the while loop is terminated, the StockPriceServlet stops the JMS subscriber and cleans up the message cache.

We did not want to invoke the destroy method to avoid touching the lifecycle controls and left the rest of things on the servlet container.

No comments: