nazarcybulskij / mobile_bechmarks_

Mobile_Bechmarks

mobile_bechmarks_'s Issues

Misleading benchmarks

There a couple of things wrong with this benchmark.

1. The benchmark does not make use of the calculated value.

   Running the same algorithm in Rust as is would make use of dead-code elimination, resulting in an effective computation time of 0 ns. 🏆

2. The benchmark computes the same constant value in every iteration, which may be optimized to a single evaluation multiplied by iteration count.

   Benchmarks should make sure the compiler is not simply applying this optimization, and as a result avoiding measuring actual work.

3. The "getOneByPi" benchmarks are inconsistent and not actually benchmarking the same algorithm.

   For ObjC it uses an int as loop variable, while for Swift it uses a double. This adds an unnecessary int-to-double conversion on every loop iteration for the ObjC benchmark.

4. By running the Swift benchmark from SwiftUI, while running the ObjC benchmark from UIKit you're effectively including the given UI frameworks implementation details.

   Do not run benchmarks within UI frameworks, or at least make sure to run them in the same(!) UI framework.

When run in isolation (getOneByPi benchmark) we get these numbers:

C: 10.124000 ms
C (fixed): 9.951000 ms
Swift: 10.008999999999908 ms
Rust: 10.008999999999903 ms

These numbers are way too close to judge upon.

I only looked at getOneByPi as most implementations wouldn't even compile as is, but the issues described here apply to all benchmarks.

In its current state the benchmarks and Medium post misleading at best.

Apart from the fact that this kind of number crunching is in no way representative for the performance of a mobile framework. For benchmarking languages doing hot-loop primitive maths is not going to give you much insight into the overall performance of a given language in realistic scenarios. And for benchmarking frameworks, having the measured code not call into said frameworks at all doesn't benchmark anything useful either.

Not the best look for an evangelism post.

will the latest react-native version do better?

such as 0.64 version

React source needs refactoring.

Hi,
your article was very impressive.
But It would be better comparing, when you some modify source.
Would you split out event handler source
from in render function?

In render function react makes new event handler every state changes.