Prevent Clusterphobia

Clustering improves scalability, but isn't completely portable between application servers

When your television ad is aired and thousands of shoppers suddenly descend upon your Web site, the scalability of your e-business solution will be put to the test. Clustering is an effective scalability approach, but making it work requires a thorough understanding of the clustering techniques your application server vendor supports.

Scalability does not simply refer to the size of your system or the number of users it can handle. More specifically, scalability refers to the ability of your application to handle dramatic fluctuations in traffic. The majority of Web site outages are caused by e-business applications that can't scale. Web site crashes can be devastating - just ask eBay, whose 22-hour service outage in June 1999 interrupted 2.3 million auctions and sparked an almost immediate 9.2 percent drop in its stock value.

A typical multitiered e-business solution based on the Java 2 Enterprise Edition (J2EE) consists of several components that contribute to the scalability of the overall system, such as the Web server, application server, network, directory and security servers, firewalls, existing business servers, and database servers. Among those components there are various techniques for making the system scalable, including batching requests, aggregating data, managing connections, caching, using faster or special-purpose machines, and clustering. In this column, we'll consider clustering of J2EE application servers.

J2EE Portability vs. Clustering

Introducing a cluster of machines, with each node hosting one or more replicas of a J2EE application, or subsets thereof, achieves two important scalability goals. First, spreading user requests across multiple nodes increases the responsiveness of the overall system. This capability is generally referred to as load balancing. Second, creating replicas improves system availability by providing failover safety. In case of a failure on a node, user requests will be redirected to a redundant server and continue without interruption.

The biggest challenge for architects and developers is that not every J2EE-compliant application server handles clustering in the same way, and therefore e-business applications may behave differently when they're deployed in a cluster on different application servers.

That statement appears to contradict the main advantage of J2EE - portability. Supposedly, applications developed in accordance with the J2EE specification are portable across J2EE-compliant application servers. You should be able to take a J2EE application originally written for BEA Systems' WebLogic and deploy it on Net-scape/Sun Microsystems's iPlanet application server, Iona's iPortal application server, the Borland AppServer, or any other J2EE-compliant application server. And you can - as long as your application is written for a single state, or nonclustered environment.

The J2EE specification is silent on the subject of clustering, which allows application server vendors to approach clustering in different ways, without violating the specification. As an application architect, you must understand the different vendor approaches and design your application with those differences in mind. If you simply assume that your e-business solution will be completely portable even in a cluster, you'll be unpleasantly surprised when your application doesn't work as designed when deployed in a different application server environment.

The State of the Shopping Cart

Consider an e-business solution with a typical shopping cart metaphor. If the application is written for a single deployment, in a nonclustered form, the user's request to place a copy of The Field Guide to the Stars and Planets in a shopping cart comes back to the same instance of Web application where the session was initiated. The application was designed according to the J2EE guidelines and was developed to ensure that the state of the shopping cart will be maintained in the servlet's HTTP session. It works, but may not be scalable.

An e-business application deployed in a clustered environment lets the user sign on and shop, happily unaware that thousands of others are also shopping the site, and that a server just crashed. Behind the scenes, information about the user's shopping cart was updated on redundant servers, which provides failover if one server crashes during a user's shopping expedition. Clustering techniques keep the session state (or shopping cart in this case) available to other redundant nodes. One important difference between application server vendors is exactly how they make that session state available.

Servlets or Beans?

When developing an e-business application according to Enterprise JavaBeans (EJB) specifications, your application server architecture will typically feature a Web container and an EJB container. The Web container handles Java server pages and servlets. The EJB container deploys entity and session beans. According to J2EE design recommendations, all Web-client requests are handled by servlets, which use a set of session beans. The session beans, which implement the business rules, work with a set of domain objects implemented as entity beans to do the work and return information to the user. As an application architect, you have a design choice - you can store a user's shopping cart in the Web container as part of the servlet at the HTTP session level, or you can store it in the stateful session beans within the EJB container, at the EJB level. Both approaches comply with the J2EE design specification, and if you are deploying the application in a single node configuration, both approaches will work on any J2EE-compliant application server.

In a clustered environment, however, you must know at which level your application server replicates and preserves the state among redundant containers and which techniques it follows to accomplish this, and then develop your application accordingly. For example, BEA WebLogic 6.0 preserves the state at both the HTTP session level and the stateful session bean level. Previous releases did not provide for the replication of stateful session beans.

Borland takes a different approach. The Borland AppServer is optimized to preserve the state among clustered servers only at the EJB level, because replicating the HTTP session state among redundant servers is expensive in terms of system resources. If you developed your application to rely on the HTTP session state to preserve the HTTP session in a cluster, and you deploy on a server that only preserves the state at the EJB level, your clustering will not work.

Beyond not being scalable, the application may behave very differently than designed when deployed in a clustered configuration on a different application server. Also, if your company switches application server vendors, significant application redesign could be required to take full advantage of the clustering capabilities of the new servers.

Reaching New Levels

There are many differences between application server approaches to clustering, each of which has an impact on the scalability of the application. For example, there are three different ways to make the session state available at the HTTP session level: replicating it in memory, storing it in the file system, or storing it in a database (Java Database Connectivity [JDBC] persistence). Within each of those three methods are other differences, too. If the session is stored in a database, what type of database is it? What capabilities are available for optimizing the access?

There are also relevant differences at the EJB level. For instance, what is the approach used for making a selection among a set of redundant servers within a cluster? One set of differences has to do with the location of the selection logic. The selection logic is intended to keep track of available redundant servers and decide which server to select for servicing a request. This selection logic may be located at the client itself, in a naming service, or at a centralized location.

Again, some application servers support more than one of these approaches, while others support only one. Those differences go a step further: Which algorithms or metrics are supported to distribute requests among available servers? It might be a simple round-robin scheme where servers are selected in a cyclical manner, with or without the facility for user-specified weights.

Your application server may allow you to tune distribution of requests based on metrics related to server load such as CPU load, disk input/output, memory thrash, or number of requests queued. Your application server may also let you control distribution of requests at a more granular level, such as the performance of individual application components. In this case some of the options may include availability of cached results, average execution time, frequency of execution, time lapsed since last execution, or component response time. Finally, your application server might allow you to manage the frequency of broadcast intervals for updating any of these metrics and synchronizing the component states among clustered servers. Better yet, the vendor may leave it up to you to implement your own, more advanced algorithm for finely tuned load balancing.

Clustering Around the Application Server

These are just some examples of the many differences between application servers, and clearly as an application architect you need to be aware of these differences. An application architecture designed with special attention to the needs of scalable/clustered solutions can help avoid the worst of the potential scalability surprises resulting from application server differences. For example, in general, your J2EE application design should minimize the number of remote calls that are necessary, because remote calls are far more expensive than local calls. However, when designing and developing for clustering, this widely applicable principle takes on a far more important meaning, because a clustered configuration by definition requires more remote calls than one that is not clustered.

A number of tasks within a J2EE application can involve remote invocations. Examples of such tasks include naming service lookups, EJB invocations from clients such as servlets, method calls among EJBs, and multiple database connections within the scope of a single transaction. You can minimize the need for remote calls in several ways. For instance, you can reduce the number of naming service lookups through linked EJB references. You can reduce the number of EJB invocations from clients such as servlets by wrapping the calls to needed EJBs in distinct umbrella session beans for each application service needed by the clients. You can also reduce the need for remote calls by colocating the EJBs involved in a task into a single EJB container. Or you can reduce the need for multiple JDBC connections to more than one instance of a particular database within the scope of a transaction by accessing one database and then using your database facility to access the second database.

An awareness of the differences among application servers, combined with an application design that takes clustering into consideration from the beginning, should yield applications that are less vulnerable to the differences between application servers.

Should the J2EE specification include strict guidelines on clustering? While standards and portability are positive forces, so is the opportunity for competition. Encouraging creativity and different approaches within general guidelines yields better products than those that would result from specifying all the differences away. Just be sure you aren't misled by the notion of portability.

Ajaz Rana, Ph.D. [ajaz.rana@marchFIRST.com] is senior architect in MarchFirst's New York office.
Chris Gomes & Daniel Eng are application architects in MarchFirst's New York office.
Albie Collins is a technology partner in MarchFirst's New York office.