Environment

Last revision on master branch (830d048).

• Rust toolchain version: 1.64.0 (a55dd71d5 2022-09-19) x86_64-pc-windows-msvc
• Operating system: Windows 10 10.0

Description

The following value triggers an error:

1.0902420340782359E+57

How to reproduce

• cargo test --package dragonbox --test binary64_test test_regression
  => OK
• add this test to test_regression()

check!(1.0902420340782359E+57);

• cargo test --package dragonbox --test binary64_test test_regression
  => panic

assertion failed: (exp as usize) < TABLE_SIZE
thread 'test_regression' panicked at 'assertion failed: (exp as usize) < TABLE_SIZE', src\div.rs:71:5
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library\std\src\panicking.rs:584
1: core::panicking::panic_fmt
at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library\core\src\panicking.rs:142
2: core::panicking::panic
at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library\core\src\panicking.rs:48
3: dragonbox::div::divisible_by_power_of_5<24>
at .\src\div.rs:71
4: dragonbox::is_product_integer_fc
at .\src\lib.rs:469
5: dragonbox::compute_nearest_normal
at .\src\lib.rs:325
6: dragonbox::to_decimal
at .\src\lib.rs:553
7: dragonbox::to_chars::to_chars
at .\src\to_chars.rs:38
8: dragonbox::buffer::impl$5::write_to_dragonbox_buffer
at .\src\buffer.rs:112
9: dragonbox::Buffer::format_finite
at .\src\buffer.rs:52
10: dragonbox::Buffer::format
at .\src\buffer.rs:30
11: binary64_test::to_chars
at .\tests\binary64_test.rs:35
12: binary64_test::test_regression
at .\tests\binary64_test.rs:115
13: binary64_test::test_regression::closure$0
at .\tests\binary64_test.rs:106
14: core::ops::function::FnOnce::call_once<binary64_test::test_regression::closure_env$0,tuple$<> >
at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52\library\core\src\ops\function.rs:248
15: core::ops::function::FnOnce::call_once
at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library\core\src\ops\function.rs:248
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

Misc

I have changed the Cargo.toml to edition 2021 but I don't think this should have any impact.

I noticed that the original C++ source code has received many updates since the port to Rust, but I'm not sure they're relevant to that particular problem. I currently don't have the tools to check this particular value with the original C++ Dragonbox code, unfortunately; I will post an update if I manage to do that test.

Issue

decimal_length_minus_1 uses a very naive digit counting algorithm, when a few faster algorithms exist, which can be fast, or relatively quick and economical. There's a few approaches to speed up decimal_length_minus_1, which currently uses a very naive algorithm. This can be optimized significantly. These algorithms determine the decimal length, so decimal_length_minus_1 would just subtract 1 from these values. The assembly generated for various different algorithms is on Compiler Explorer here.

Implementations

1). Fast, but requires a bit of static storage, is described here.

This computes a fast log2 , and uses a pre-computed table to determine rounding of the value.

#[inline]
pub fn fast_log2(x: u32) -> usize {
    32 - 1 - (x | 1).leading_zeros() as usize
}

#[inline]
pub fn fast_digit_count(x: u32) -> usize {
    const TABLE: [u64; 32] = [
        4294967296,
        8589934582,
        8589934582,
        8589934582,
        12884901788,
        12884901788,
        12884901788,
        17179868184,
        17179868184,
        17179868184,
        21474826480,
        21474826480,
        21474826480,
        21474826480,
        25769703776,
        25769703776,
        25769703776,
        30063771072,
        30063771072,
        30063771072,
        34349738368,
        34349738368,
        34349738368,
        34349738368,
        38554705664,
        38554705664,
        38554705664,
        41949672960,
        41949672960,
        41949672960,
        42949672960,
        42949672960,
    ];
    let shift = unsafe { *TABLE.get_unchecked(fast_log2(x)) };
    let count = (x as u64 + shift) >> 32;
    count as usize
}

This requires a bit of static storage, but computes the value extremely cheaply, and optimizes to an add and shr instruction, along with a table lookup.

2). A more economical solution, but still fast algorithm is the following:

pub fn fast_digit_count_v2(x: u32) -> usize {
    const TABLE: [u32; 9] = [9, 99, 999, 9999, 99999, 999999, 9999999, 99999999, 999999999];
    let mut y = (9 * fast_log2(x)) >> 5;
    y += (x > unsafe { *TABLE.get_unchecked(y) }) as usize;
    y + 1
}

These are all significantly more efficient than the current algorithm (which has been shifted by 1):

fn fast_digit_count_v3(v: u32) -> i32 {
    if v >= 100000000 {
        9
    } else if v >= 10000000 {
        8
    } else if v >= 1000000 {
        7
    } else if v >= 100000 {
        6
    } else if v >= 10000 {
        5
    } else if v >= 1000 {
        4
    } else if v >= 100 {
        3
    } else if v >= 10 {
        2
    } else {
        1
    }
}

A quick look at the optimization results can be found here. The second solution is likely the best, since it's effectively very cheap, requires minimal static storage, the storage requirements can be reduced due to the small number of digits required (never >= 10^9).

Solution

A simple implementation of decimal_length_minus_1 would therefore be:

#[inline]
pub fn fast_log2(x: u32) -> i32 {
    32 - 1 - (x | 1).leading_zeros() as i32
}

pub fn decimal_length_minus_1(x: u32) -> i32 {
    const TABLE: [u32; 9] = [9, 99, 999, 9999, 99999, 999999, 9999999, 99999999, 999999999];
    let mut y = (9 * fast_log2(x)) >> 5;
    y += (x > unsafe { *TABLE.get_unchecked(y) }) as i32;
    y
}

This can never be "unsafe", since the maximum value from fast_log2) is 31, and (9 * 31) >> 5 is 8.