"Scalar Quantization 101" blog: dot product expansion has a spurious dim factor on the α² term

#585 · open · 0 comments

View on GitHub ↗

prashbhat

The Search Labs post [Scalar Quantization 101](https://www.elastic.co/search-labs/blog/scalar-quantization-101) (Oct 25, 2023) derives the float dot product in terms of the quantized int8 vectors. The final expanded formula is printed as: ``` dim × α² × dotProduct(int8, int8′) + Σᵢ min × α × int8ᵢ + Σᵢ min × α × int8′ᵢ + dim × min² ``` The `dim ×` on the first term appears to be a typo. Starting from the post's own per-dimension expansion: ``` float32ᵢ × float32′ᵢ ≈ α² × int8ᵢ × int8′ᵢ + α × min × int8ᵢ + α × min × int8′ᵢ + min² ``` and summing over `i = 0 … dim−1` gives: ``` α² × dotProduct(int8, int8′) + Σᵢ min × α × int8ᵢ + Σᵢ min × α × int8′ᵢ + dim × min² ``` Since `dotProduct(int8, int8′)` already sums over all dimensions, multiplying it by `dim` would double-count the dimensionality — the formula as printed is only correct when `dim = 1`. The `dim` factor legitimately appears only on the `min²` term, where the per-dimension constant `min²` is summed `dim` times. The prose immediately below the formula repeats the error: "dim×α² … can be stored as a single float value" — the precomputable constant should be just `α²`. For what it's worth, Lucene's implementation of this exact scoring agrees: the stored `constMultiplier` in the scalar-quantized similarity is `α × α`, with no `dim` factor, plus the per-vector offset corrections for the two cross terms. Suggested fix: drop `dim ×` from the first term of the expanded formula and from the "dim×α² can be stored as a single float value" sentence. cc @benwtrent

Comments