Sunday, 25 March 2007

W550i Frame Rate Profile


After showing interest in my frame rate profile benchmark, Andi, who owns an SE w550i decided to run the benchmark on his phone and collect some data. Thanks to him, I have the privelege to share the results with everyone, including the data in Normal and Fullscreen mode:


From these graphs, you can see that the results are not much different from what I came up with for the W300i: Here's a scatterplot to illustrate:

So there you have it. It seems like the W550i also runs most effective at around 40-45 fps. Though theoretically, the 'best' time ratio would be at fps=1, it really doesn't serve a purpose when you want to create animations.
I'm looking for Nokia phones to test this with. Are you interested? phirewerkz at gmail dot com

Wednesday, 21 March 2007

Choosing the Frame Rate (W300i Time Profile)


So what is a good frame rate to choose for flash lite animations? Does a faster fps necessarily mean a faster, more optimal animation? It's a balance between smoothness of the animation vs the capacity of the cellphone to render. Let's take a look at one example:

Remember the Benchmark Animation Results? Well, all those tests were performed at 100fps. However, what happens if we choose different ones? What about 50, 120? Will the amount of time taken to render differ? I took out the w300i again for some testing.

This time, we have the same animation, but I've changed the frame rate. They increase as follows: 15,20,25,30,35,40,41,42,43,44,45,50,60,70,...120. Why did you I choose it this way? You can see from the graph below that there is a huge change in the rendering time between 40 and 45fps. The graph compares the theoretical (dark bars) vs. the actual time (light bars):


I was honestly really surprised at this! What's up with the huge change in render time? This data doesn't look too conclusive though. Let's run a couple more trials and calculate the time ratio. Recall, time ratio:

Time Ratio = Actual Time/Theoretical Time

After calculating this, let's look at the results:



Wow. There seems to be two extremely linear portions, showing a direct correlation between render time and frame rate: The higher frame rate, the RELATIVELY longer it takes to render. I'm NOT saying it takes longer it render, but the ratio is increased. The two linear portions are separated by a large dip, specifically 40-45 fps.

So from this graph, you can see that higher FPS doesn't necessarily mean a better relative render time. Moreover, the optimal FPS seems to lie around the 44-45 FPS range. This is quite surprising.

Remember: this is only for the W300i. I'd love to create some more time profiles for other model phones. If you'd like to contribute, please let me know! phirewerkz at gmail dot com.


Sunday, 18 March 2007

Flash Lite Animation: Benchmark Experiment


Introduction
Creating flash lite menus is really a balance between visual aesthetics and possibly slowing down your menu due to the limited processing speed a cellphone has. As of lately, I've been wondering, do some phones run flash lite applications more efficiently than others? Although most applications are not visually extensive, I figured it would still be interested to find out which ones are more suitable for this.

Acknowledgements:
I would personally like to thank Matt Pollitt, Hayden Porter, Jameel Mulani, vivek, DenisR5, evilhomura89 and lithium3r for providing their phones as test subjects for this experiment.

Methods
The benchmark consists of a mini flash application that animates a black circle that moves in a circular motion. Based on calculations (100 fps, the animation lasts 200 frames), the optimal duration of this clip should last 2 seconds. Indeed, if you run the swf on a modern computer, it clocks out at around 2000 ms. The animation is sized for a w300i (my phone), but should work for any flash enabled phone since flash can dynamically resize their movies.


After the animation is complete, it will display the time taken to complete the animation as well as a variable I've defined as "time ratio":

This ratio = (actual time to complete animation) / 2000.

Thus, an optimal ratio would be 1. Likewise, if you have a ratio of 3, it means that your phone takes 3 times as long as how the flash would run on normally. Thus, the time ratio is a measure of how fast your cellphone processor is able to process the animation.

Materials:
The following phones were subjected to this test:
Nokia e61 Nokia 3650
Noka n93 Sony E W300i (2)
Nokia e50 Nokia 6131
Nokia 3250 Nokia 5300
Nokia n80 Nokia 7373
Nokia n91 Sony E W810i (3)
Nokia e70 Sony E W850i (3)
Sony E W600i
*Parentheses beside the phone model represent the number of phones of that model tested. If unstated otherwise, there was only one model tested.

The files used were created in Flash 8, named Benchmark.swf.

Procedure:
Each phone uploaded the swf file to their phone and ran the swf file in "Normal" and "Fullscreen" Mode. The Time Ratio for the Normal viewing mode was noted and repeated 5 times. Likewise, the Time Ratio for the Fullscreen viewing mode was ran 5 times as well. Afterwards, the average of the time ratios for each category were calculated and tabulated.

Observations:
All of the aforementioned phones were tested with the benchmark in two viewing modes (Normal, Full screen) and their results are shown in graphical form in increasing time ratios:



Results:

Nokia vs. Sony Ericsson
The wide range of phones being tested was really unexpected. The results are especially interesting when comparing the Nokia vs Sony phones. From the graph, one can see that the Time Ratio phones for Nokia phones are significantly lowered than Sony Ericsson phones.

Normal vs. Fullscreen
Also, SE phones seemed to suffer more from running the flash lite benchmark full screen whereas Nokia phones exhibited a negligible difference. This may be due to the fact that these SE phones only have browser compatibility for Flash Lite which probably places more load on the processing as compared to running it standalone.

Screen Size vs. Time Ratio (Sony Ericsson phones)
It's also interesting to see that the Time Ratio for SE phones seems to be strongly correlated with the screen size of the phone. I guess this is similar to running graphic intensive flash files at greater resolutions on computer.



Observations:
It is clear that flash lite files running on cellular phones are much slower if they involve graphic intensive animations. Moreover, the time it takes to complete the animation may be 3-fold as long or more. These results are in no way conclusive: For all I know, some of these values may not be representative of the processing speed of the phone model in general, so there is a certain degree of error associated with these results. Also, only one swf file was used for testing, which had a resolution of 128 x 160. A potential error could be that stretching the movie may have caused slower processing, though this does not seem to be the case.

One of the test phones according to the user seemed to achieve faster time ratios when certain numpad keys were pressed down while the benchmark ran. I’m not sure how this happens, but I’ll look into it in the future.

This is just the beginning of benchmark tests. There are still many variables to consider such as selected frame rate: perhaps there is a frame limit on animations? Does faster frame rates necessarily mean your phone will process faster? Does the type of animation (changing position, alpha, etc) change the render time? We've already looked at one of these comparisons, and that is Bitmap vs. Vector animations, which you can find here.

So why is this important? It is important for developers to consider the types and extent of animations they want to place in their applications. Although this benchmark is a crude exaggeration of animation, it is still important to consider when adjusting the frame rate of your animation. Usually when creating animations on a moderately fast computer, a FPS of 50 is sufficient to create the smoothest animation the eye can see. However, you MUST keep in mind that the cellphone playing these SWFs have a much more limited processing capability, and you will find that animations play on a much slower scale.

Conclusion:
From these tests, one can see the wide range of processing capabilities of Flash Lite enabled phones. For now, Nokia seems to have an upper hand in processing animations, and may just be due to the fact that their models support standalone compatibility whereas SE phones are browser-only. Once again, I'd like to extend my thank you to those who have helped me collecting data and providing motivation.