Flash Lite Animation: Benchmark Experiment

Introduction
Creating flash lite menus is really a balance between visual aesthetics and possibly slowing down your menu due to the limited processing speed a cellphone has. As of lately, I've been wondering, do some phones run flash lite applications more efficiently than others? Although most applications are not visually extensive, I figured it would still be interested to find out which ones are more suitable for this.

Acknowledgements:
I would personally like to thank Matt Pollitt, Hayden Porter, Jameel Mulani, vivek, DenisR5, evilhomura89 and lithium3r for providing their phones as test subjects for this experiment.

Methods
The benchmark consists of a mini flash application that animates a black circle that moves in a circular motion. Based on calculations (100 fps, the animation lasts 200 frames), the optimal duration of this clip should last 2 seconds. Indeed, if you run the swf on a modern computer, it clocks out at around 2000 ms. The animation is sized for a w300i (my phone), but should work for any flash enabled phone since flash can dynamically resize their movies.

After the animation is complete, it will display the time taken to complete the animation as well as a variable I've defined as "time ratio":

This ratio = (actual time to complete animation) / 2000.

Thus, an optimal ratio would be 1. Likewise, if you have a ratio of 3, it means that your phone takes 3 times as long as how the flash would run on normally. Thus, the time ratio is a measure of how fast your cellphone processor is able to process the animation.

Materials:
The following phones were subjected to this test:

Nokia e61	Nokia 3650
Noka n93	Sony E W300i (2)
Nokia e50	Nokia 6131
Nokia 3250	Nokia 5300
Nokia n80	Nokia 7373
Nokia n91	Sony E W810i (3)
Nokia e70	Sony E W850i (3)
Sony E W600i

*Parentheses beside the phone model represent the number of phones of that model tested. If unstated otherwise, there was only one model tested.

The files used were created in Flash 8, named Benchmark.swf.

Procedure:
Each phone uploaded the swf file to their phone and ran the swf file in "Normal" and "Fullscreen" Mode. The Time Ratio for the Normal viewing mode was noted and repeated 5 times. Likewise, the Time Ratio for the Fullscreen viewing mode was ran 5 times as well. Afterwards, the average of the time ratios for each category were calculated and tabulated.

Observations:
All of the aforementioned phones were tested with the benchmark in two viewing modes (Normal, Full screen) and their results are shown in graphical form in increasing time ratios:

Results:

Nokia vs. Sony Ericsson
The wide range of phones being tested was really unexpected. The results are especially interesting when comparing the Nokia vs Sony phones. From the graph, one can see that the Time Ratio phones for Nokia phones are significantly lowered than Sony Ericsson phones.

Normal vs. Fullscreen
Also, SE phones seemed to suffer more from running the flash lite benchmark full screen whereas Nokia phones exhibited a negligible difference. This may be due to the fact that these SE phones only have browser compatibility for Flash Lite which probably places more load on the processing as compared to running it standalone.

Screen Size vs. Time Ratio (Sony Ericsson phones)
It's also interesting to see that the Time Ratio for SE phones seems to be strongly correlated with the screen size of the phone. I guess this is similar to running graphic intensive flash files at greater resolutions on computer.

Observations:
It is clear that flash lite files running on cellular phones are much slower if they involve graphic intensive animations. Moreover, the time it takes to complete the animation may be 3-fold as long or more. These results are in no way conclusive: For all I know, some of these values may not be representative of the processing speed of the phone model in general, so there is a certain degree of error associated with these results. Also, only one swf file was used for testing, which had a resolution of 128 x 160. A potential error could be that stretching the movie may have caused slower processing, though this does not seem to be the case.

One of the test phones according to the user seemed to achieve faster time ratios when certain numpad keys were pressed down while the benchmark ran. I’m not sure how this happens, but I’ll look into it in the future.

This is just the beginning of benchmark tests. There are still many variables to consider such as selected frame rate: perhaps there is a frame limit on animations? Does faster frame rates necessarily mean your phone will process faster? Does the type of animation (changing position, alpha, etc) change the render time? We've already looked at one of these comparisons, and that is Bitmap vs. Vector animations, which you can find here.

So why is this important? It is important for developers to consider the types and extent of animations they want to place in their applications. Although this benchmark is a crude exaggeration of animation, it is still important to consider when adjusting the frame rate of your animation. Usually when creating animations on a moderately fast computer, a FPS of 50 is sufficient to create the smoothest animation the eye can see. However, you MUST keep in mind that the cellphone playing these SWFs have a much more limited processing capability, and you will find that animations play on a much slower scale.

Conclusion:
From these tests, one can see the wide range of processing capabilities of Flash Lite enabled phones. For now, Nokia seems to have an upper hand in processing animations, and may just be due to the fact that their models support standalone compatibility whereas SE phones are browser-only. Once again, I'd like to extend my thank you to those who have helped me collecting data and providing motivation.