Research about using Montgomery form in `modexp` about zksync_era_precompiles

[fkrause98]

There are some approaches for modexp, modexp has the following signature:

fn modexp(base: BigInt, exp: BigInt, modulo: Int) -> BigInt

Montgomery

The issue with the Montgomery algorithm is that we need to precompute a constant $R \in \mathbb{Z}$ such that $R \gt n$ and $gcd(R, n) = 1$ (where n is the modulo argument).
This is fine when we know $n$ beforehand, since we can just choose and hardcode $R$ but when $n$ is unknown this becomes problematic.
Problematic because we need to find a coprime, large number to an arbitrary $n$.
There are possible solutions to this though, if we split $n$ into two cases:

1. If $n \equiv 1 \mod 2$, we could simply choose $R = 2^{\log_{2}(n)+1}$
2. If $n \equiv 0 \mod 2$, this is not really clear but Montgomery Reduction With Even Modulus proposes a solution using The Chinese Remainder Theorem but of course we would have to go and implement it.

Maybe there are more solutions with Montgomery but I have not found them.

Barret

An alternative to Montgomery's is the Barret Reduction.
Barret still needs to precompute 2 constants: $k$ such that $2^{k} \gt n$ and $r = \frac{4^k}{n}$.
The election of $k$ can be done simply taking $k$ = $log_{2}(n)$ which can be done through shifts.
$r$ can also be computed with shifts and a division since $4^{k} = 2^{2k} = 1 \gg 2k$.
Also, as an advantage, the barret reduction does not need a special form like Montgomery's form
when operating.
The downside of this approach is that $0 \leq \text{base} \leq n^2$ as a precondition must hold.

Useful links:

• Montgomery arithmetic from a software perspective
• Montgomery reduction with even modulus
• Comparison of three modular reduction functions
• Algorithmica's chapter on Montgomery

[ilitteri]

Closed as the research finished.