Javascript Refactoring For Safer Faster Better AJAX

MD5 for AJAX • Google Maps • JavaScript Crosswords • Javascript Refactoring • Javascript Performance • AJAX Without XML • Javascript Closures • JSFL Flash Extensions

Introduction

JavaScript is a very powerful and widely used programming language. It can be found today on almost any web site, even in the most plain-looking text-only pages. JavaScript code validates form fields, handles events, constructs urls, controls browser and manipulates page content and styles. It even communicates with dedicated servers in real-time to receive fresh XML data.

For many years JavaScript was treated as a "toy" scripting language for addressing odd web browser quirks. Writing JavaScript code was conducted casually and not considered to be a serious software engineering. But recently, with the release of major JavaScript-heavy services like Google Maps {1} and the overwhelming success of AJAX web applications {2}, this picture started to change. The amount of JavaScript code written by developers is continuously increasing, so is increasing it's impact on the overall quality of web applications. Well written, reliable and fast JavaScript code is not just nice to have, but rather must have. Any development shop that does not professionalize JavaScript development efforts, should expect a proliferation of bugs, coding errors, poor performance, browser incompatibility problems and customer support issues related to the use of JavaScript. How can one quickly convert casual software development to professional software engineering? There may not be one simple answer here, but if your current JavaScript project is in trouble or if you are about to start a new one - this article is for you.

The problem

Quite often I run into development teams with a lot of JavaScript that suffers from poor code design and desperately needs refactoring. Refactoring is the gradual modification of a software system to improve its internal logical structure, but mostly preserve its functionality. The definitive guide to refactoring by Martin Fowler {3} and handbook to improving general structure of code by Steve McConnell {4} have in existence there for several years, focusing on compiled languages. But JavaScript is a dynamically-typed functional language that is typically used without compilation. Nevertheless, an interpreter and the dynamic nature of JavaScript does not excuse JavaScript developers from not following the best practices of software engineering. Used properly, JavaScript can provide a competitive advantage over statically typed languages like C++, Java or C#.

What are the common problems, or anti-patterns found in JavaScript code? One of the biggest problems is left-and-right use of global variables and functions. The biggest evil of global variables are unexpected side-effects when you make code changes. Global variables frequently lead to further design problems and can host a huge number of bugs. They make development, testing, support and troubleshooting a nightmare and must be eliminated. Good code design that avoids global variables and makes code modular is the only path to reliable interactive web-applications.

Let's consider one example of a typical AJAX application. For simplicity, it might have just one HTML page with a link. When a user clicks this link, the JavaScript code in the page requests an XML document from the server, receives it, and shows its contents to the user. All of this is done on the same page without a page reload. The source for web page and JavaScript code can be found in old/test.htm and old/oyXMLRPC.js respectively.

The code is quite simple and works fine on both Firefox 1.0+ and IE 6.0+. If you are generally familiar with JavaScript and have used AJAX, a quick look at the source is quite sufficient for our discussion. Otherwise, review Listing 1 or other sources to learn more about JavaScript prototype-based objects and public and private access to object variables and functions.

Listing 1. Private and public access to object variables and functions in JavaScript.

// declaration
function myClass(){

    // variables
    var privateVar = "I am private";
    this.publicVar = "I am public";

    // functions
    var privateFunction = function(){
        alert("I am private");
    }

    this.publicFunction = function(){
        alert("I am public");
    }
}

// usage
var myInstance = new myClass();

myInstance.publicVar = "new value";
alert(myInstance.privateVar);             // undefined!

myInstance.publicFunction();
myInstance.privateFunction();             // error!

Listing 2 shows a general code structure focusing on global variables, functions and their dependencies. While this code works, it needs a serious refactoring before going to production! Lets review it and spot specific code design problems. First of all, its quite easy to see that it uses two global variables:

req, which holds the current request object being processed
ready, which holds a boolean flag indicating whether or not the request has been completed

What is slightly harder to see is that several functions in this example, make heavy use of these global variables. These functions are:

loadXMLDoc(url), which submits the request to a server
processStateChange(), which handles the request state changes
function processXML(), which handles processing of received the server response

Such dependency of functions on global variables makes these functions themselves global and not reusable in other contexts or for other purposes. For example, as your web application matures, you might want to handle several XMLHttpRequests concurrently. But it won't work. All concurrent requests will use the same req and ready variables resulting in the race conditions and errors. Or you might want to reuse the above JavaScript code in another similar project. It won't work either. Someone with a lot of natural creativity and perseverance will just duplicate all the code to overcome the design problems. Thus illustrating a classic example of bad initial design forcing further compromises and poor overall product quality.

Listing 2. General code structure of a sample application before refactoring (derived from old/oyXMLRPC.js ).

var req;
var ready;

function loadXMLDoc(url) {
// this function uses req, ready, processStateChange()
}

function processStateChange() {
// this function uses uses req, ready, processXML()
}

processXML() {

}

The solution

The good news is in many cases refactoring is quite easy to do. Let's start with the list of general design rules and goals for the new version.

avoid global variables
allow for multiple XMLHttpRequests to be handled concurrently
encapsulate browser specific code
improve error handling and error reporting
add basic performance metrics
create one JavaScript file that is directly reusable in other projects

There are several large goals here. The attempt to achieve all of them might strike you as a major application rewrite, rather than just refactoring. Simply consider the rest of the source code involved in your large application and you will realize that we are only focusing on a small piece of a big puzzle.

Any big refactoring requires skills, source code control, unit testing, and detailed planning. Then it's possible to achieve all the set design goals in several short iterations as illustrated by Listings 3-6:

In step 1, place all of the global variables and functions into a new object oyXMLRPCProvider. This object follows Asynchronous Completion Token pattern {5} and encapsulates handling of one request. If several concurrent requests are needed, several instances of oyXMLRPCProvider can be created.

Listing 3. Results of refactoring step 1.

function oyXMLRPCProvider(){
var req;
var ready;

    var loadXMLDoc = function (url) {
        // this function uses req, ready, processStateChange()
    }

    var processStateChange = function () {
        // this function uses uses req, ready, processXML()
    }

var processXML = function () {

}
}

In step 2, declare all variables private; review all functions and declare them either public or private. Specifically, make processStateChange() function private. Take processXML() function out of the provider object and replace it with empty event handler onComplete(). Now any function can be attached externally to the provider instance to handle the completed request. By convention all private function names starts with "internal" prefix.

Listing 4. Results of refactoring step 2.

function oyXMLRPCProvider(){
var req;
var ready;

    this.loadXMLDoc = function (url) {
        // this function uses req, ready, processStateChange()
    }

    var processStateChange = function () {
        // this function uses uses req, ready, onComplete()
    }

this.onComplete = function () {
}

}

In step 3, log important information and improve error handling in the provider class. Similar to the step above, add empty onLog and onError event handlers. Once again, any external function can now be used to handle log and error messages. Add quick defense function internalCanMsg() to prevent the log from being flooded with too many messages. Add timestamp to each message, so performance can be monitored during testing. Log submitted url, state changes, status codes, size of response text, etc. Note how variable oThis is used to reach public function from the private function.

Listing 5. Results of refactoring step 3.

function oyXMLRPCProvider(){

    // place here all code from previous step

    var oThis = this;

    // checks to see if we have too many messages in log
    var internalCanMsg = function(){
        msgCount++;
        return msgCount < 100;
    }

    // adds message to internal log
    var internalOnLog = function(msg){
        if(oThis.onLog && internalCanMsg()) {
            oThis.onLog(msg);
        }
    }

    // adds message to internal error handler
    var internalOnError = function(msg){
        if(oThis.onError && internalCanMsg()) {
            oThis.onError(msg);
        }
    }

    this.onLog = function (msg){
    }

    this.onError = function (msg){
    }

}

In step 4, add supporting functions, including: getVersion(), isSupported(), isBusy(), getUrl(), abort(), etc. Rename other functions to reflect their purpose.

Listing 6. Results of refactoring step 4.

Good intuition and a strong desire to adhere to the design goals drove the developer in this refactoring. While performed intuitively without any reference books or tools nearby, many well known patterns {3} were practically used, including:

Replace Date Value with Object
Replace Method with Method Object
Encapsulate Field
Self Encapsulate Field
Remove Setting Method
Introduce Assertion
Replace Magic Number with Symbolic Constant

The source for a new version of web page and JavaScript can be found in new/test.htm and new/oyXMLRPC.js respectively. All design goals are met, and the new code is easily integrated back into the complete system. Now we are ready for production!

Prototypes instead of closures

While very convenient for the refactoring, using closures might be inefficient from the performance and memory usage perspective. When instance methods are added as closures in the constructor a new closure is instantiated for each object instance. This overhead should not be a problem unless you plan to create thousands of object instances. This is not likely for oyXMLRPCProvider, but maybe true for some other classes. In such cases it is better to extend objects by using JavaScript object prototypes.

By adding a prototype to an object, we can extend all instances of an object at the same time. It takes only minutes for me to rewrite an implementation of oyXMLRPCProvider to work off prototypes. Simply convert every closure this.methodFoo = function() {} into a prototype method oyXMLRPCProvider.prototype.methodFoo = function (){} and you are done. Also change all references to local variables and functions to starts with "this".

The only thin place here is in assigning object method to this.req.onreadystatechange. You have to really understand JavaScript scoping rules as they apply to the closures and prototypes. The trick here is to declare a new local variable to hold "this" value, and to wrap an object callback method into a closure. The source for a new version of web page and JavaScript can be found in new/test.htm and new/oyXMLRPC.js respectively.

Listing 7.Working around JavaScript scoping rules to assign this.req.onreadystatechange.


oyXMLRPCProvider.prototype.submit = function(_url){	
	...
	...
	...
	var _this = this;
	...
	...
	this.req.onreadystatechange = function() {_this.internalRequestComplete(); };
	...
	...
	...
}

Final word

None of the code snippets, templates, prototypes, tools, libraries, or frameworks available out there can eliminate a need for good code design. Even if all of those libraries were written perfectly without bugs or design flaws, there would still be code that you would have to write on top of these libraries. To have solid control of your own code, stop treating it as a "toy" and continuously invest time in following basic software design principles. As you can see from examples in this article, significant improvements to code can be made with relatively small effort. After all, there is no need to find the best design that follows all the patterns out there. Just find an adequate design that provides basic modularity and code organization. Do not forget other parts of your software system that do not use JavaScript, they will benefit from good design too!

Effective refactoring starts with insightful design goals. Set bold and demanding engineering goals and targets for your software projects and people. It will stretch and develop your mind and will lead you to powerful solutions, confidence, agility and a lot of innovation. It will make your team stronger and capable of building safer, faster, better software products!

References

http://maps.google.com
http://ajaxpatterns.org/Ajax_Frameworks
Martin Fowler: Refactoring.
Steve McConnell: Code Complete.
Douglas Schmidt, Michael Stal, Hans Rohnert, Frank Buschmann: Pattern-Oriented Software Architecture, Volume 2, Patterns for Concurrent and Networked Objects.

Version Control

Responding to reader comments (Paul), removed unused local variables from the constructor of the prototype-based version of oyXMLRPCProvider() on Dec 13, 2006.
Responding to reader comments, added prototype-based version of oyXMLRPCProvider() on 7 Mar, 2006.
First version created on 26 Sep, 2005.

MD5 for AJAX • Google Maps • JavaScript Crosswords • Javascript Refactoring • Javascript Performance • AJAX Without XML • Javascript Closures • JSFL Flash Extensions

Finding the right web host can be hard, which providers dedicated hosting as well as wireless broadband connection.

Comments (6)

Comment by Gabriel Sroka — September 13, 2007 @ 12:32 am

Pavel,
In internalRequestComplete(), you define
var STATE_COMPLETED = 4;
var STATUS_200 = 200;
which is great. I’ve seen dozens of AJAX frameworks that use magic numbers without any definitions.

I’ve gone one or two steps further. I’ve defined “enum” type objects, such as
var State = {loading: 3, done: 4};
var Status = {ok: 200, notFound: 404};
so you can use
if (req.readyState == State.done) {
and
if (status == Status.ok) {

Also, I always add comments with references to URLs. For example, the State values come from
http://www.w3.org/TR/XMLHttpRequest/#xmlhttprequest
and Status.ok=200 comes from
http://tools.ietf.org/html/rfc2616
Comment by Ramar — February 4, 2008 @ 6:35 am

i want to software for working javascript
Comment by frank kootte — February 17, 2008 @ 3:34 pm

Your example describing private and public variables and functions is incorrect.

The following is a correct example:

var func = function ()
{
var private = ”privateString;
var private2 = function(){alert(‘privateFunction’);}
this.priviliged = ‘priviligedString’
var priviliged2 = function(){alert(‘priviligedFunction’);}
}

func.prototype.public = ‘publicString’;
func.prototype.public2 = function(){alert(‘publicFunction’);}
Comment by frank kootte — February 17, 2008 @ 3:35 pm

var priviliged2 = function(){alert(’priviligedFunction’);}

should be

this.priviliged2 = function(){alert(’priviligedFunction’);}

sorry.
Comment by jim groen — March 31, 2008 @ 10:47 am

Found memory leak in IE7

Add this.req = null;

Change:
if (this.status == STATUS_200) {
this.internalOnLog(“internalRequestComplete: calling callback on content with length ” + this.req.responseText.length + ” chars”);
if(this.onComplete) {
this.onComplete(this.req.responseText, this.req.responseXML);
}
this.internalOnLog(“internalRequestComplete: complete on ” + new Date());
this.req = null; // Fix IE memory leak / garbage collection
}
Comment by kailash thakur — November 8, 2008 @ 5:44 am

i want a software of wepon desgining