A Study of Ajax Performance Issues

Coach Wei: Direct from Web 2.0
Home
Syndicate
Main Page
Previous: The Common Culture Root of Web 2.0 and Barack Obama
A Study of Ajax Performance Issues
bycoachwei on Tue 22 Jan 2008 11:28 AM EST  | Permanent Link  | Cosmos
Introduction:
Ajax application performance largely depends on the performance of JavaScript execution and browser DOM operations. I've heard various people saying various things about Ajax performance. Some people say JavaScript is just too slow. Some people say that the problem is not JavaScript but rather Browser DOM being too slow. Some people would then argue that browser DOM operations are fine. The problem is Internet Explorer that is unbelievably slow.
What is the reality? Having heard enough but haven’t seen enough data to clearly justify one argument against the other ones, I decided to take some time and do some study in this area.
The Study:
Using the work produced by Robert Buffone at http://www.rockstarapps.com and a 2GB RAM/1.66GHz Dual CPU Windows XP machine, a JavaScript performance study is conducted. The results are shown in Appendix 1:
1.      Vertically, in each column, Appendix 1 shows how a particular operation performs in comparison to other operations on the same browser. Using an empty function call as the “base” for comparison, Appendix 1 shows the comparative percentage of time that other operations consume relative to that of the base operation. Note that for each browser, the time that most operations take is comparable to that of an empty function call. However, certain operations are significantly more expensive, some times even a few million percent more expensive, than the base empty function call;
2.      Horizontally, on each row, Appendix 1 shows how different browsers perform with regard to the same operation. Safari typically outperforms FireFox and FireFox outperforms Internet Explorer. However, there are exceptions that Internet Explorer outperforms other browsers.
What Are The Ajax Performance Issues?
What are noteworthy from the study are:
a)      JavaScript performance can be greatly enhanced via better implementations, without API change or new standard. In the above comparison, Safari generally beats Internet Explorer by a few hundred percent or even more in terms of performance, which is fairly dramatic.
b)      Internet Explorer really needs to enhance its JavaScript runtime performance to make it on par with other browsers. The significantly lower performance and the significant market share of Internet Explorer together create a major challenge for Ajax performance.
c)      That being said, other browsers also have performance bottlenecks.
More specifically, here is a list of Ajax performance issues:
1.    Array performance on all browsers in general
2.    HTML DOM performance in general
3.    Calculating Computed Box Model and Style
4.    FireFox: “eval”, Object Creation and “in” Operation
5.    “String” Manipulation Performance on IE
6.    Safari: “pop” Operation Performance
1.    “Array” Is Slow on All Browsers
Array-related operations are expensive for all browsers. Comparing to other operations, the same browser takes considerable more time for array-related operations. The differences can be a few orders of magnitude some times. Specifically:
1.    For IE7: “pop”, “push”, “shift”, “join” etc are all very expensive that can take ten times or even a few hundred times longer to complete than other operations;
2.    For FireFox: “pop” and “push” operations perform fairly well in comparison to other operations (“pop” and “push” operations cost 160% of the base operation on FireFox. On IE, they cost 532% and 3523% of the base operation respectively). “shift” and “join” are very expensive. The operation “creating an empty array using ‘new Array()’ constructor” costs 662% of the base operation while this number is only 119% and 145% on IE and Safari. This is related to the issue that it is very expensive for FireFox to create new objects. Other browsers do not seem to have this problem.
3.    For Safari, “pop”, “shift” and “join” are expensive operations.
2.  HTML DOM Operation Performance in General
All HTML DOM related JavaScript calls are significantly more expensive than most other operations across all browsers. On Safari, DOM calls cost from a few hundred percent to a few thousand percent of the base operations. On Internet Explorer, the cost can be as higher as tens of thousands of percentage. Given that DOM operation is essential to almost every browser-based application, improvement in this area can significantly broaden the applicability of Ajax applications. The table below shows the performance of DOM operations on three major browsers (the percetage is calculated relative to a base operation on that browser,See Appendix 1 for details): IE7 FireFox Safari HTML DOM Operation Time(ns) % Time(ns) % Time(ns) % Change text using innerHTML 469.0 9979% 234.0 6000% 109.0 7032% Create a text node on HTML Dom 1093.0 23255% 156.0 4000% 110.0 7097% Change the class name of an element 422.0 8979% 47.0 1205% 109.0 7032% getElementById 86.8 1846% 15.7 401% 3.9 252% getElementsByTagName("div") 153.1 3257% 18.0 462% 5.5 352% getElementsByName 93.8 1995% 44.6 1142% 4.7 303% placeDiv.getAttribute("id") 29.7 632% 46.8 1200% 5.5 352% placeDiv.attributes["id"] 31.3 665% 225.0 5769% 6.3 403%
3. Calculating Computed Box Model and Computed Style
Getting computed box model calculations out of IE is not for the faint of optimization. According to a post by Alex Russell at Dojo Foundation (see reference http://alex.dojotoolkit.org/?p=644),  when they profiled Dojo widgets for Dojo 1.0, they noted very quickly that getting box-model data out of the browser for any element is hugely costly on every browser, but on IE the cost was not just big… it was enormous. Our best guess right now is that the properties on the currentStyle property are re-calculated when they’re requested from script and not cached in the bound object when layout happens. The resulting performance penalty requires that developers nearly never manage layout in code, severely constraining the layouts which are attempted by toolkits like Dojo.
Why Is This Important?
Current browsers provide limited support for layout management. A lot web sites use “table” for managing layout, while some others use CSS for doing layout. Though both table and CSS are capable of serving the purpose of layout, layout support is not a first class citizen. There is not out-of-box support for common layout managers (such as docking, border layout). There is no layout object and there is no programmatic API for layout management. As a result, providing better layout management has been a task undertaken by JavaScript toolkits. Calculating the computed box model and style is essential for JavaScript toolkits to be able to manage layout.
4. FireFox Specific Performance Issues
Comparing to other browsers, FireFox performs very poorly for the following operations:
1.    “eval” of object or function: These two operations take Safari 9.4ns and 22.7ns, IE7 172ns and 94ns. But they take FireFox 546ns and 749ns.
2.    Object creation: Operations like “var myObject = new MyObject (17, 250);” and “var slowCar = {m_tireSize:17, m_maxSpeed:250};” are very expensive on FireFox comparing to other browsers. The differences between them are: 11.7ns and 8.6ns for IE7, 3.2ns and 2.4ns for Safari, 23.4ns and 23.4ns for FireFox which are almost 10 times more expensive than Safari and a few times more expensive than IE7.
3.    “in” operation: Operation “looping an array through ‘in’” takes IE 10.3ns and Safari 7.8ns, but 62.8ns for FireFox. Again, a difference of almost 10 times.
5. IE Specific Performance Issues
IE performs worse than FireFox and Safari in general. Beyond the areas that all browsers have problems, String manipulation deserves particular attention for IE.
String compares and concatenation on IE6 are super bad. According to Robert Buffone, “I left out IE 6 for the exercise because it is so bad on everything it would just make every JavaScript coder extremely mad”.
IE7 has made progress on this. But it still lags behind by a few hundred percent comparing to other browsers.
Why is this important?
1.    Every application involves String manipulation. Part of the appeal of Ajax is the benefit of leveraging client side processing power, which naturally leads to a lot more String manipulations on the client side;
2.    As pointed out by reference (http://alex.dojotoolkit.org/?p=644), most non-trivial blocks of JavaScript code today rely on innerHTML to bootstrap some new chunk of DOM in response to user action due in large part to the cross-browser speed and size advantages of innerHTML vs. raw DOM methods for equivalent DOM structures. Because building innerHTML is essentially String manipulation, the performance of string manipulation heavily influences the performance of Ajax applications.
6. Safari Specific Performance Issues
Safari in general performs really well comparing to other browsers. The only operation that Safari performs significantly worse than other browsers is “pop” method. In summary, if every browser can match Safari’s performance, it would be a significant step forward for web development.
Where Do We Go From Here?
Obviously, we would like to see browser vendors take a serious look into the following issues and put them on their roadmap:
1.    In all major browsers, performance with Array and HTML DOM needs improvement in general.
2.    Browsers need to provide API support for Computed Box Model and Style;
3.    FireFox needs to improve performance of “eval”, object creation and “in” operation
4.    Internet Explorer needs to improve performance in general to be at least on par with other browsers. Beyond that, “String” manipulation on IE needs continued improvements;
5.    Safari: “pop” operation performance needs improvement
6.    Just-in-time (JIT) compiler: This maybe a bigger task than an incremental fix of some existing features, however, it is worthy of every penny.  JIT will not only fix the String manipulation issue, it will enable JavaScript to truly shine in matching the performance of native applications. The amount of client side logic (aka, JavaScript code) needs to grow in order to accommodate the growth of application complexity, for which JavaScript runtime performance problem can be a major bottleneck.
But how can we get browser vendors to listen to us?
First of all, help make some noise! The community needs to come together and your participation is crucial in making something happen.
Secondly, let’s look around and get some coordinated acts together. At OpenAjax Alliance (http://www.openajax.org), we are starting a task force called “Runtime Advocacy Task Force”. At the OpenAjax 2007 September Member Meeting, the members of OpenAjax Alliance discussed the challenges for future Ajax growth and adoption, in particular, issues directly related to the various Ajax runtime environments. The collective wisdom was that OpenAjax Alliance should do something in this area to help creating a better Ajax eco-system. The work is still in its early stage, but watch for progress over the next month or so.
Testing setup:
·         Windows XP SP2,  DELL Latitude D820,  2GB RAM, 1.66GHz Dual CPU
·         IE 7.0.5730.11
·         FireFox 2.0.0.11
·         Safari 3.0.2 (522.13.1)
·         Application:http://www.rockstarapps.com/samples/performance/
Internet Explorer 7
FireFox
Safari
Description
Time (ns)
% of base
Time (ns)
% of base
Time (ns)
% of base
Normal Empty function call (Base Operation)
4.7
100%
3.9
100%
1.6
100%
Basic Function Calls
Function call using function.call(this)
5.5
116%
2.4
60%
1.6
100%
Normal Empty function using apply
5.5
117%
7.0
179%
2.4
152%
Normal Empty function using apply with 3 parameters
7.0
149%
7.1
181%
2.4
152%
Eval a function
172.0
3660%
546.0
14000%
9.4
603%
Eval an object
94.0
2000%
749.0
19205%
22.7
1461%
Basic Operations
Access Properties through a getter
13.3
282%
6.3
160%
5.5
352%
Access Properties directly
4.7
100%
3.2
81%
2.4
152%
Simple string concatenation
4.7
100%
2.3
59%
1.6
100%
Simple string compare
3.9
83%
2.4
60%
0.8
48%
Change string to upper case
11.7
249%
3.9
100%
4.7
303%
Replace string reg expression
12.5
266%
7.1
181%
9.4
603%
String concat with integer
7.1
150%
4.7
119%
2.4
152%
String concat with float
6.3
133%
4.7
121%
2.4
152%
Index of Bob in string, not found, length = 71
6.3
133%
6.3
160%
2.4
152%
Match of Bob in string, not found, length = 71
14.9
316%
25.0
640%
3.9
252%
charAt(10) in string
11.7
249%
6.3
160%
3.1
200%
Create object constructor initialized
11.7
249%
23.4
600%
3.2
203%
create simple object
8.6
182%
23.4
600%
2.4
152%
0%
Variable declaration
4.0
84%
2.3
59%
0.8
48%
Multiple variable declaration, multiple var
3.9
83%
2.4
60%
2.4
152%
Multiple variable declaration single var
3.9
83%
2.4
60%
1.6
100%
Variable declaration set to null
3.9
83%
2.4
60%
1.6
100%
0%
Variable assignment++
4.7
100%
5.5
140%
1.6
103%
Variable assignment + 1
5.5
116%
7.1
181%
1.6
100%
Four levels of property access
5.5
116%
5.5
140%
1.6
100%
Three levels of property access
4.7
100%
4.7
121%
2.4
152%
Two levels of property access
4.7
100%
4.7
119%
1.6
100%
One level of property access
3.9
83%
4.7
121%
1.6
103%
0%
Using the typeof function
3.9
83%
4.7
119%
1.6
100%
Array Operations
Array access
7.1
150%
7.0
179%
1.6
100%
Array index value change
3.9
83%
4.7
121%
1.6
103%
Empty Array index value change
8.6
183%
8.6
221%
6.3
403%
Empty Array add three values
10.2
216%
11.7
300%
3.1
200%
Empty Array with set size
11.0
233%
11.0
281%
3.2
203%
Empty Array using constructor
9.4
199%
25.8
662%
2.3
148%
Push element onto an Array
25.0
532%
6.3
160%
1.5
98%
Pop element of an Array
165.6
3523%
6.3
160%
249.7
16112%
Push element onto an Array
6.2
132%
7.1
181%
1.4
92%
Shift elements off the front of an Array
2030.0
43191%
5620.0
144103%
1250.0
80645%
Join the array into a string
125000
2659574%
47000
1205128%
16000
1032258%
Push element onto an Array
7.1
150%
7.0
179%
3.9
252%
Sorting of an Array
93.0
1979%
45.3
1162%
32.9
2119%
Math.max(7.25,7.30)
5.5
116%
4.7
121%
3.9
252%
Math.min(7.25,7.30)
5.5
117%
4.7
121%
3.2
203%
HTML DOM Operations
Change text using innerHTML
469.0
9979%
234.0
6000%
109.0
7032%
Create a text node on HTML Dom
1093.0
23255%
156.0
4000%
110.0
7097%
Change the class name of an element
422.0
8979%
47.0
1205%
109.0
7032%
getElementById
86.8
1846%
15.7
401%
3.9
252%
getElementsByTagName("div")
153.1
3257%
18.0
462%
5.5
352%
getElementsByTagName("spa")
142.2
3024%
18.8
481%
5.5
355%
getElementsByName
93.8
1995%
44.6
1142%
4.7
303%
placeDiv.getAttribute("id")
29.7
632%
46.8
1200%
5.5
352%
placeDiv.attributes["id"]
31.3
665%
225.0
5769%
6.3
403%
var atts = placeDiv.attributes; atts["id"].name
11.7
249%
18.8
482%
4.7
303%
var atts = placeDiv.attributes; atts.id.name
12.5
266%
18.8
482%
3.9
252%
Try/Catch
Function call using function.call(this)
7.1
150%
4.0
101%
2.4
152%
Try catch
7.1
150%
6.3
160%
5.5
352%
Try catch with throw
17.2
365%
7.8
200%
5.5
355%
Try catch finally
9.4
199%
7.8
200%
4.7
303%
Try catch finally with throw
16.5
350%
11.0
281%
6.3
403%
If statement and Date object
Create a date object
12.5
266%
10.2
260%
3.1
200%
Create a date object and call getDate()
28.1
598%
50.8
1303%
10.2
658%
The switch statement
4.7
99%
2.4
60%
2.4
152%
An if statement
4.0
84%
2.4
60%
2.4
152%
Simple if else
4.1
86%
2.2
56%
1.9
121%
turinary operator
3.9
83%
2.3
60%
2.0
131%
Compare of matching string ==
4.7
100%
7.1
181%
2.4
152%
Compare of matching string ===
4.7
100%
7.0
179%
2.4
152%
Compare of non-matching string
4.7
100%
3.9
100%
2.4
152%
Compare  TEST_INT == 5
4.1
86%
4.2
108%
1.7
111%
Compare  5 == 5
4.1
87%
2.0
52%
1.6
101%
Looping Performance Statistics
Loop through an array using in
10.3
219%
62.8
1610%
7.8
503%
Loop through an array using for (i++)
6.3
133%
10.6
273%
3.4
222%
Loop through an array using for (i < myArray.length)
6.2
133%
10.9
281%
3.1
201%
Loop through an array using for (i = 0; i < len; i++)
5.0
106%
4.7
120%
4.1
262%
Loop through an array using for (i = 0; i < len; i++)
5.3
113%
5.0
128%
4.1
262%
Loop through an array using while {i++}
5.3
113%
5.3
136%
4.7
302%
Loop through an array using while (i++ < len)
5.0
106%
5.0
128%
4.4
283%