Wrong result for digamma integral representation about sympy

[oscarbenjamin]

Looks like this comes from meijerg:

In [11]: integrate((1 - t**z) / (1 - t), (t, 0, 1), meijerg=False)
Out[11]: 
1          
⌠          
⎮  z       
⎮ t  - 1   
⎮ ────── dt
⎮ t - 1    
⌡          
0          

In [12]: integrate((1 - t**z) / (1 - t), (t, 0, 1), meijerg=True)
Out[12]: 
     ⎛ 2⋅ⅈ⋅π          ⎞             ⎛ 2⋅ⅈ⋅π          ⎞             
  z⋅Φ⎝ℯ     , 1, z + 1⎠⋅Γ(z + 1)   Φ⎝ℯ     , 1, z + 1⎠⋅Γ(z + 1)    
- ────────────────────────────── - ──────────────────────────── + ∞
             Γ(z + 2)                        Γ(z + 2)              

In [13]: integrate((1 - t**z) / (1 - t), (t, 0, 1))
Out[13]: ∞ + ⅈ⋅π

[oscarbenjamin]

The result from meijerg=True looks like it would be correct if the oo was EulerGamma:

In [58]: integrate((1 - t**z) / (1 - t), (t, 0, 1), meijerg=True).simplify()
Out[58]: 
   ⎛ 2⋅ⅈ⋅π          ⎞    
- Φ⎝ℯ     , 1, z + 1⎠ + ∞

The exp_polar there if replaced by 1 would give -lerchphi(1, 1, z+1) which is maybe equivalent to digamma(z+1) (i.e. polygamma(0, z+1)).

[rogerbalsach]

@oscarbenjamin , I am not familiar with the Lerch transcendent, but evaluating the indefinite integral I get the following

import sympy as sym

t = sym.Symbol('t', nonnegative=True)
z = sym.Symbol('z')
sym.Integral((1 - t**z) / (1 - t), t).doit().simplify()

which results in -t**(z + 1)*lerchphi(t, 1, z + 1) - log(t - 1).
This seems to agree with WolframAlpha.
So, it looks like whatever the problem is, it must be in the evaluation of the limits.
This seems plausible by observing that at t=1 we have log(1 - 1) = oo and at t=0 we have log(0 - 1) = I*pi.

[oscarbenjamin]

which results in -t**(z + 1)*lerchphi(t, 1, z + 1) - log(t - 1).

That's coming from meijerg's indefinite integration:

In [8]: integrate((1 - t**z) / (1 - t), t, meijerg=True).simplify()
Out[8]: 
⎧ ⎛ z + 1  ⎛   2⋅ⅈ⋅π          ⎞             ⎞           
⎪-⎝t     ⋅Φ⎝t⋅ℯ     , 1, z + 1⎠ + log(t - 1)⎠  for t > 1
⎨                                                       
⎪ ⎛ z + 1  ⎛   2⋅ⅈ⋅π          ⎞             ⎞           
⎩-⎝t     ⋅Φ⎝t⋅ℯ     , 1, z + 1⎠ + log(1 - t)⎠  otherwise

In [9]: integrate((1 - t**z) / (1 - t), t).simplify()
Out[9]: 
   z + 1                            
- t     ⋅Φ(t, 1, z + 1) - log(t - 1)

So, it looks like whatever the problem is, it must be in the evaluation of the limits.

Maybe. Note that meijerg's definite integration is not really handled by computing the indefinite integral in the form shown. I think it would actually compute the definite integral using G-functions and then try to express the result in ordinary looking functions at the end.

[rogerbalsach]

Ok. First things first, this is the first time I look into the sympy code for integration, so I might miss a lot of things (for example, I don't really know what meijerg's method is).
I've been playing a bit with the debugger for the following code

import sympy as sym

t = sym.Symbol('t', nonnegative=True)
z = sym.Symbol('z')
f = (1 - t**z) / (1 - t)
I = sym.Integral((1 - t**z) / (1 - t), (t, 0, 1))

I.doit()

and it seems that it is doing the following:
First, it computes the indefinite integral

sympy/sympy/integrals/integrals.py

Lines 619 to 620 in c9a2f11

	antideriv = self._eval_integral(
	function, xab[0], **eval_kwargs)

which evaluates to -t**(z + 1)*z*lerchphi(t, 1, z + 1)*gamma(z + 1)/gamma(z + 2) - t**(z + 1)*lerchphi(t, 1, z + 1)*gamma(z + 1)/gamma(z + 2) - log(t - 1) , which is the same expression I had before.
The first thing I think is relevant is that I don't get any piecewise function, and as @oscarbenjamin mentioned

In [8]: integrate((1 - t**z) / (1 - t), t, meijerg=True).simplify()
Out[8]: 
⎧ ⎛ z + 1  ⎛   2⋅ⅈ⋅π          ⎞             ⎞           
⎪-⎝t     ⋅Φ⎝t⋅ℯ     , 1, z + 1⎠ + log(t - 1)⎠  for t > 1
⎨                                                       
⎪ ⎛ z + 1  ⎛   2⋅ⅈ⋅π          ⎞             ⎞           
⎩-⎝t     ⋅Φ⎝t⋅ℯ     , 1, z + 1⎠ + log(1 - t)⎠  otherwise

In [9]: integrate((1 - t**z) / (1 - t), t).simplify()
Out[9]: 
   z + 1                            
- t     ⋅Φ(t, 1, z + 1) - log(t - 1)

but I don't think this is a problem since log(t - 1) = log(1 - t) + I*pi, thus it should not matter for the definite integral.

After that, it indeed tries to evaluate the primitive in the two limits in line

sympy/sympy/integrals/integrals.py

Line 717 in c9a2f11

evalued = Add(*others)._eval_interval(x, a, b)

Then, for t = 0, the integral evaluates to I*pi (sometime I also get terms of the type 0**z). Or, if we use log(1 - t), then it evaluates to 0.

The problem then comes for 't = 1', which is evaluated at

sympy/sympy/core/expr.py

Line 916 in c9a2f11

C = limit(self, x, c, "+" if left else "-")

This limit evaluates to infinite, which is the wrong part.
Indeed, after simplifying the expression, as I mentioned before, the primitive can be written as
$$-t^{1 + z} \Phi(t, 1, 1 + z) - \log(1 - t)$$

The limit as $t \to 1^-$ is indeed the expected result of $\psi(z+1) + \gamma_E$. This can be seen from one of the series expansions of the Lerch transcendent
$$z^a \Phi(z, 1, a) \to \sum_{k=1}^\infty \zeta(1-k, a) \frac{\log^k(1)}{k!} + [\psi(1) - \psi(a) - \log(-\log(1))]\frac{\log^0(1)}{0!} = -\gamma_E - \psi(a) - \log(0)$$

Thus, in conclusion, it looks like the limit is the reason for the wrong value.
As I said at the beginning, I don't know much about how sympy handles integration, and I have not passed any fancy parameters like meijerg, so maybe in that case the integral is handled differently, but from your response, it seems to fail in a similar way.

[oscarbenjamin]

This limit evaluates to infinite, which is the wrong part.

What exactly is the expression whose limit is being computed? Can you reproduce the problem just using limit rather than integrate?

[rogerbalsach]

The expression is the primitive function: -t**(z + 1)*z*lerchphi(t, 1, z + 1)*gamma(z + 1)/gamma(z + 2) - t**(z + 1)*lerchphi(t, 1, z + 1)*gamma(z + 1)/gamma(z + 2) - log(t - 1)

Indeed this can be reproduced with the following code:

import sympy as sym

t = sym.Symbol('t', positive=True)
z = sym.Symbol('z')

F = (-t * t**z * z * sym.lerchphi(t, 1, z + 1) * sym.gamma(z + 1) / sym.gamma(z + 2)
     - t * t**z * sym.lerchphi(t, 1, z + 1) * sym.gamma(z + 1) / sym.gamma(z + 2)
     - sym.log(t - 1))
# An equivalent form
F = -t**(z+1)*sym.lerchphi(t, 1, z+1) - sym.log(t - 1)

L = sym.Limit(F, t, 1, '-')
L.doit()

This return infinite, but as I justified, it should be digamma(z+1) + GammaEuler (- I*pi).
The limit is also wrong in WolframAlpha.

[oscarbenjamin]

So it looks like the bug is in limit...

In [13]:  L.args[0].subs(z, S(1)/2).subs(t, 0.9999).n()
Out[13]: 0.61365563825509 - 3.14159265358979⋅ⅈ

In [14]:  L.args[0].subs(z, S(1)/2).limit(t, 1, '-')
Out[14]: ∞