Regression: lambdify of Derivative instance now throws PrintMethodNotImplementedError about sympy

[moorepants]

Ah, sorry I misunderstood. You want an expression that doesn't NameError. Yes. So my notes may not be related at all (I now realize).

[ericpre]

Thank you very much for your help, this was useful and is appreciated!

I don't understand everything but I suspect that our use case is not justified - this feature is a "good to have" but at the same time, we can also overwrite it manually when it doesn't work well (or not at all).

In the cases I reported in this issue, the failures introduced in 1.13.0rc1 are all valid and are an improvement to identify what was previously not working.

[nicholasferguson]

found hack solution... changed 'strict' :False
It was 'strict':None

That error msg needs to be more precise. It eats up research time, in a terrible fashion.

class CodePrinter(StrPrinter):

_default_settings: dict[str, Any] = {
    'order': None,
    'full_prec': 'auto',
    'error_on_reserved': False,
    'reserved_word_suffix': '_',
    'human': True,
    'inline': False,
    'allow_unknown_functions': False,
    'strict': False #None  # True or False; None => True if human == True
}

[oscarbenjamin]

It eats up research time, in a terrible fashion.

Development time is eaten up by people opening issues without doing the obvious things like showing some code to reproduce the issue.

[nicholasferguson]

https://github.com/nicholasferguson/SympyANDmpmath

[bjodah]

@nicholasferguson I authored the error message. The goal of this error message was of course to waste less research time (since sympy used to silently generate broken code), but we (I) might have unintentionally introduced a bug, so thanks for opening an issue (I think we all want the same thing here, and everyone involved are donating their time).

From your repository I could create this reproducer:

import mpmath
from sympy import Symbol, zeta, lambdify, Derivative

s = Symbol('s')
z = zeta(s)
zp = Derivative(z, s)
f = lambdify(s, zp, modules=['mpmath', {'Derivative': lambda expr, z: mpmath.zeta(z, derivative=1)}])
print(f(2))

which fails for the merge commit (b93a982) of gh-25913 (but passes when doing git checkout b93a982~1).

If you print the docstring of our generated function f for the older version of sympy it looks like this:

>>> print(f.__doc__)
Created with lambdify. Signature:

func(s)

Expression:

Derivative(zeta(s), s)

Source code:

def _lambdifygenerated(s):
    return (  # Not supported in Python with mpmath:
  # Derivative
Derivative(zeta(s), s))


Imported modules:

I see that there's a printer setting called "allow_unknown_functions" (which looks like it is being enabled by lambdify) that alters when CodePrinter calls the method _print_not_supported:

sympy/sympy/printing/codeprinter.py

Line 458 in 2565eb3

if expr.is_Function and self._settings.get('allow_unknown_functions', False):

So .is_Function needs to be True, let's check if that holds for our Derivative instance:

>>> zp.is_Function
False

So, with a bit of luck it would be enough to change that if-statement to if (expr.is_Function or expr.is_Derivative) ....

EDIT: (and if the fix turns out to be more complicated, we could resort to passing strict=False from lambdify)

[bjodah]

While trying to fix this I realize that this code doesn't make much sense:

>>> f = lambdify(s, zp, modules=['mpmath', {'Derivative': lambda expr, z: mpmath.zeta(z, derivative=1)}])

any Derivative instance would map to mpmath.zeta, an expression containing different derivatives would return non-meaningful results.

The proper fix would be to first replace the particular derivative of interest with a Function instance (e.g. Function('my_zeta_derivative')(s)). And then pass this function as a known function to lambdify. That this ever "worked" is mostly an accident.

Furthermore, the previous generated code performed unnecessary work (it evaluated the primitive function at the point of interest). I have a local patch of codeprinter.py which generates a "functioning" lambdify callable:

(Pdb) print(f.__doc__)
Created with lambdify. Signature:

func(s)

Expression:

Derivative(zeta(s), s)

Source code:

def _lambdifygenerated(s):
    return Derivative(zeta(s), (s, 1))


Imported modules:

but this is rather pointless. If we can come up with a legitimate use of passing 'Derivative' as a known function, I can use that as a test case for my patch, and submit it as a PR. But the original code snippet to reproduce the issue is not suitable since it doesn't do what you might expect.

The proper way to customize how derivatives are evaluated in the lambdified function would probably be along the lines of:

>>> from sympy import symbols, zeta, lambdify, Function
>>> import mpmath
>>> s = symbols('s')
>>> expr = 42 + 17*zeta(s).diff(s)
>>> zeta_diff = Function('zeta_diff')(s)
>>> expr2 = expr.replace(zeta(s).diff(s), zeta_diff)
>>> expr2
17*zeta_diff(s) + 42
>>> f = lambdify(s, expr2, ['mpmath', {'zeta_diff': lambda arg: mpmath.zeta(arg, derivative=1)}])
>>> f(2)
mpf('26.061679676630654')

[nicholasferguson]

That is why Aaron Meurer in his blog, which in repository is a pdf file .... writes

We now use lambdify to convert the SymPy expressions Z and D into functions that are evaluated using mpmath. A technical difficulty here is that the derivative of the zeta function
does not have a closed-form expression. mpmath's zeta can evaluate
but it doesn't yet work with sympy.lambdify (see SymPy issue 11802). So we have to manually define "Derivative" in lambdify, knowing that it will be the derivative of zeta when it is called. Beware that this is only correct for this specific expression where we know that Derivative will be Derivative(zeta(s), s).

[asmeurer]

This is the issue referenced in the blog post #11802

[bjodah]

Thank you @asmeurer, I opened gh-26678. Let me know what you think.

[ericpre]

Here is an example that breaks with 1.13.0rc1 and a similar error message:

import sympy

expr_str = "A*heaviside(x-n,0.5)"
expr = sympy.sympify(expr_str)

x = [symbol for symbol in expr.free_symbols if symbol.name == "x"][0]
variables = sympy.symbols([s.name for s in expr.free_symbols])
real_variables = sympy.symbols([s.name for s in variables], real=True)

# Replace symbols by real symbols to help with differentiation
# according to git comment 9 years ago
expr = expr.subs(
    {orig: real_ for (orig, real_) in zip(variables, real_variables)}
)

parameters = [var for var in real_variables if var.name != "x"]
eval_expr = expr.evalf()

for p in parameters:
    grad_expr = sympy.diff(eval_expr, p)
    f_grad = sympy.utilities.lambdify(
        real_variables,
        grad_expr.evalf(),
        modules="numpy",
        dummify=False,
    )

Error Traceback using bjodah:CodePrinter-customization-point-Derivative

Traceback (most recent call last):

  Cell In[1], line 23
    f_grad = sympy.utilities.lambdify(

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:880 in lambdify
    funcstr = funcprinter.doprint(funcname, iterable_args, _expr, cses=cses)

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:1171 in doprint
    str_expr = _recursive_to_string(self._exprrepr, expr)

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:966 in _recursive_to_string
    return doprint(arg)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:172 in doprint
    lines = self._print(expr).splitlines()

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:575 in _print_Mul
    a_str = [self.parenthesize(x, prec) for x in a]

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:575 in <listcomp>
    a_str = [self.parenthesize(x, prec) for x in a]

  File ~\mambaforge\lib\site-packages\sympy\printing\str.py:38 in parenthesize
    return self._print(item)

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\str.py:470 in _print_Subs
    self._print(expr), self._print(old), self._print(new))

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:470 in _print_Derivative
    return self._print_not_supported(expr, extra_info=f"\nPrinter {self.__class__.__name__} has no method: {meth_name}")

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:592 in _print_not_supported
    raise PrintMethodNotImplementedError(

PrintMethodNotImplementedError: Unsupported by <class 'sympy.printing.numpy.NumPyPrinter'>: <class 'sympy.core.function.Derivative'>
Printer NumPyPrinter has no method: _print_Derivative_heaviside
Set the printer option 'strict' to False in order to generate partially printed code.

Similar error when replacing expr_str = "A*heaviside(x-n,0.5)" with expr_str = "A * arctan(k * (x - x0))"

Example using arctan

import sympy


expr_str = "A * arctan(k * (x - x0))"
expr = sympy.sympify(expr_str)

x = [symbol for symbol in expr.free_symbols if symbol.name == "x"][0]
variables = sympy.symbols([s.name for s in expr.free_symbols])
real_variables = sympy.symbols([s.name for s in variables], real=True)

# Replace symbols by real symbols to help with differentiation
# according to git comment 9 years ago
expr = expr.subs(
    {orig: real_ for (orig, real_) in zip(variables, real_variables)}
)

parameters = [var for var in real_variables if var.name != "x"]
eval_expr = expr.evalf()

for p in parameters:
    grad_expr = sympy.diff(eval_expr, p)
    f_grad = sympy.utilities.lambdify(
        real_variables,
        grad_expr.evalf(),
        modules=["numpy"],
        dummify=False,
    )

Error traceback using bjodah:CodePrinter-customization-point-Derivative

Traceback (most recent call last):

  Cell In[1], line 23
    f_grad = sympy.utilities.lambdify(

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:880 in lambdify
    funcstr = funcprinter.doprint(funcname, iterable_args, _expr, cses=cses)

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:1171 in doprint
    str_expr = _recursive_to_string(self._exprrepr, expr)

  File ~\mambaforge\lib\site-packages\sympy\utilities\lambdify.py:966 in _recursive_to_string
    return doprint(arg)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:172 in doprint
    lines = self._print(expr).splitlines()

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:575 in _print_Mul
    a_str = [self.parenthesize(x, prec) for x in a]

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:575 in <listcomp>
    a_str = [self.parenthesize(x, prec) for x in a]

  File ~\mambaforge\lib\site-packages\sympy\printing\str.py:38 in parenthesize
    return self._print(item)

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\str.py:470 in _print_Subs
    self._print(expr), self._print(old), self._print(new))

  File ~\mambaforge\lib\site-packages\sympy\printing\printer.py:331 in _print
    return printmethod(expr, **kwargs)

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:470 in _print_Derivative
    return self._print_not_supported(expr, extra_info=f"\nPrinter {self.__class__.__name__} has no method: {meth_name}")

  File ~\mambaforge\lib\site-packages\sympy\printing\codeprinter.py:592 in _print_not_supported
    raise PrintMethodNotImplementedError(

PrintMethodNotImplementedError: Unsupported by <class 'sympy.printing.numpy.NumPyPrinter'>: <class 'sympy.core.function.Derivative'>
Printer NumPyPrinter has no method: _print_Derivative_arctan
Set the printer option 'strict' to False in order to generate partially printed code.

[bjodah]

Thank you @ericpre, this example is very useful. When I try to dig deeper here I do wonder if the previous generated functions actually worked? I tried to call f_grad in your example with some real valued arguments but using earlier released versions of sympy, I get NameError when calling those:

slightly modifed version of ericpre's reproducer

import sympy

def main(expr_str):
    expr = sympy.sympify(expr_str)

    x = [symbol for symbol in expr.free_symbols if symbol.name == "x"][0]
    variables = sympy.symbols([s.name for s in expr.free_symbols])
    real_variables = sympy.symbols([s.name for s in variables], real=True)

    # Replace symbols by real symbols to help with differentiation
    # according to git comment 9 years ago
    expr = expr.subs(
        {orig: real_ for (orig, real_) in zip(variables, real_variables)}
    )

    parameters = [var for var in real_variables if var.name != "x"]
    eval_expr = expr.evalf()

    print(f"{expr_str=} {real_variables}")
    print('='*40)
    print("")
    for p in parameters:
        grad_expr = sympy.diff(eval_expr, p)
        gef = grad_expr.evalf()
        f_grad = sympy.utilities.lambdify(
            real_variables,
            gef,
            modules="numpy",
            dummify=False,
        )
        print(f"{p=}\n  {grad_expr=}\n  {gef=}")
        n = len(real_variables)
        res = f_grad(*[i/(n+1) for i in range(n)])

if __name__ == '__main__':
    for expr_str in ["A*heaviside(x-n,0.5)", "A * arctan(k * (x - x0))"]:
        main(expr_str)
        print("\n"*3)

output using e.g. sympy-1.11

expr_str='A*heaviside(x-n,0.5)' [A, n, x]                                                                                                                                                                                                      
========================================                                                                                                                                                                                                       
                                                                                                                                                                                                                                               
p=A                                                                                                                                                                                                                                            
  grad_expr=heaviside(-n + x, 0.5)                                                                                                                                                                                                             
  gef=heaviside(-n + x, 0.5)                                                                                                                                                                                                                   
p=n                                                                                                                                                                                                                                            
  grad_expr=-A*Subs(Derivative(heaviside(_xi_1, 0.5), _xi_1), _xi_1, -n + x)                                                                                                                                                                   
  gef=-A*Subs(Derivative(heaviside(_xi_1, 0.5), _xi_1), _xi_1, -n + x)                                                                                                                                                                         
Traceback (most recent call last):                                                                                                                                                                                                             
  File "/home/bjorn/vc/sympy/gh26663_2156407342.py", line 38, in <module>                                                                                                                                                                      
    main(expr_str)                                                                                                                                                                                                                             
  File "/home/bjorn/vc/sympy/gh26663_2156407342.py", line 34, in main                                                                                                                                                                          
    res = f_grad(*[i/(n+1) for i in range(n)])                                                                                                                                                                                                 
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                 
  File "<lambdifygenerated-2>", line 4, in _lambdifygenerated                                                                                                                                                                                  
NameError: name 'Subs' is not defined 

If they did work, do you think you would be able to modify your example so that the function objects can be successfully evaluated with a previous release of sympy?

[ericpre]

I can reproduce the error with older version of sympy (1.12), which indicates that this may be a different issue.
This functionality wasn't tested for the two expressions given in the example above, so it is very possible that it hasn't been working for a while. To give an idea if the reproducer is representative of the current usage, I added two expressions (gaussian and polynomial) that is covered in hyperspy test suite (and known to work) and they works fine in the reproducer.

Reproducer updated with example of expression that works

import sympy

def main(expr_str):
    expr = sympy.sympify(expr_str)

    x = [symbol for symbol in expr.free_symbols if symbol.name == "x"][0]
    variables = sympy.symbols([s.name for s in expr.free_symbols])
    real_variables = sympy.symbols([s.name for s in variables], real=True)

    # Replace symbols by real symbols to help with differentiation
    # according to git comment 9 years ago
    expr = expr.subs(
        {orig: real_ for (orig, real_) in zip(variables, real_variables)}
    )

    parameters = [var for var in real_variables if var.name != "x"]
    eval_expr = expr.evalf()

    print(f"{expr_str=} {real_variables}")
    print('='*40)
    print("")
    for p in parameters:
        grad_expr = sympy.diff(eval_expr, p)
        gef = grad_expr.evalf()
        f_grad = sympy.utilities.lambdify(
            real_variables,
            gef,
            modules="numpy",
            dummify=False,
        )
        print(f"{p=}\n  {grad_expr=}\n  {gef=}")
        n = len(real_variables)
        res = f_grad(*[i/(n+1) for i in range(n)])

if __name__ == '__main__':
    for expr_str in [
            "A * (1 / (sigma * sqrt(2*pi))) * exp(-(x - centre)**2 / (2 * sigma**2))", 
            "a1*x**1+a0",
            "A*heaviside(x-n,0.5)",
            "A * arctan(k * (x - x0))"
            ]:
        main(expr_str)
        print("\n"*3)

@francisco-dlp, any idea when the gradient stopped working with the arctan component in hyperspy?

[moorepants]

Here is another example of this failing but only when using cse=True:

In [8]: import sympy as sm

In [9]: t, a, b = sm.symbols('t, a, b')

In [10]: f = sm.Function('f')(t)

In [11]: expr = a*f.diff(t, 2) + b*f.diff(t) + a*b*f + a**2

In [12]: expr
Out[12]: a**2 + a*b*f(t) + a*Derivative(f(t), (t, 2)) + b*Derivative(f(t), t)

In [13]: func = sm.lambdify((f.diff(t, 2), f.diff(t), f, a, b), expr)

In [14]: func(2.0, 3.0, 4.0, 5.0, 6.0)
Out[14]: 173.0

In [15]: func?
Signature: func(_Dummy_40, _Dummy_41, _Dummy_42, a, b)
Docstring:
Created with lambdify. Signature:

func(arg_0, arg_1, f, a, b)

Expression:

a**2 + a*b*f(t) + a*Derivative(f(t), (t, 2)) + b*Derivative(f(t), t)

Source code:

def _lambdifygenerated(_Dummy_40, _Dummy_41, _Dummy_42, a, b):
    return _Dummy_40*a + _Dummy_41*b + _Dummy_42*a*b + a**2


Imported modules:
File:      ~/src/sympy/<lambdifygenerated-2>
Type:      function

In [16]: func = sm.lambdify((f.diff(t, 2), f.diff(t), f, a, b), expr, cse=True)
---------------------------------------------------------------------------
PrintMethodNotImplementedError            Traceback (most recent call last)
Cell In[16], line 1
----> 1 func = sm.lambdify((f.diff(t, 2), f.diff(t), f, a, b), expr, cse=True)

File ~/src/sympy/sympy/utilities/lambdify.py:880, in lambdify(args, expr, modules, printer, use_imps, dummify, cse, docstring_limit)
    878 else:
    879     cses, _expr = (), expr
--> 880 funcstr = funcprinter.doprint(funcname, iterable_args, _expr, cses=cses)
    882 # Collect the module imports from the code printers.
    883 imp_mod_lines = []

File ~/src/sympy/sympy/utilities/lambdify.py:1171, in _EvaluatorPrinter.doprint(self, funcname, args, expr, cses)
   1168     else:
   1169         funcbody.append('{} = {}'.format(self._exprrepr(s), self._exprrepr(e)))
-> 1171 str_expr = _recursive_to_string(self._exprrepr, expr)
   1173 if '\n' in str_expr:
   1174     str_expr = '({})'.format(str_expr)

File ~/src/sympy/sympy/utilities/lambdify.py:966, in _recursive_to_string(doprint, arg)
    963 from sympy.core.basic import Basic
    965 if isinstance(arg, (Basic, MatrixBase)):
--> 966     return doprint(arg)
    967 elif iterable(arg):
    968     if isinstance(arg, list):

File ~/src/sympy/sympy/printing/codeprinter.py:172, in CodePrinter.doprint(self, expr, assign_to)
    169 self._not_supported = set()
    170 self._number_symbols = set()
--> 172 lines = self._print(expr).splitlines()
    174 # format the output
    175 if self._settings["human"]:

File ~/src/sympy/sympy/printing/printer.py:331, in Printer._print(self, expr, **kwargs)
    329     printmethod = getattr(self, printmethodname, None)
    330     if printmethod is not None:
--> 331         return printmethod(expr, **kwargs)
    332 # Unknown object, fall back to the emptyPrinter.
    333 return self.emptyPrinter(expr)

File ~/src/sympy/sympy/printing/str.py:57, in StrPrinter._print_Add(self, expr, order)
     55 l = []
     56 for term in terms:
---> 57     t = self._print(term)
     58     if t.startswith('-') and not term.is_Add:
     59         sign = "-"

File ~/src/sympy/sympy/printing/printer.py:331, in Printer._print(self, expr, **kwargs)
    329     printmethod = getattr(self, printmethodname, None)
    330     if printmethod is not None:
--> 331         return printmethod(expr, **kwargs)
    332 # Unknown object, fall back to the emptyPrinter.
    333 return self.emptyPrinter(expr)

File ~/src/sympy/sympy/printing/codeprinter.py:565, in CodePrinter._print_Mul(self, expr)
    563     a_str = [self.parenthesize(a[0], 0.5*(PRECEDENCE["Pow"]+PRECEDENCE["Mul"]))]
    564 else:
--> 565     a_str = [self.parenthesize(x, prec) for x in a]
    566 b_str = [self.parenthesize(x, prec) for x in b]
    568 # To parenthesize Pow with exp = -1 and having more than one Symbol

File ~/src/sympy/sympy/printing/codeprinter.py:565, in <listcomp>(.0)
    563     a_str = [self.parenthesize(a[0], 0.5*(PRECEDENCE["Pow"]+PRECEDENCE["Mul"]))]
    564 else:
--> 565     a_str = [self.parenthesize(x, prec) for x in a]
    566 b_str = [self.parenthesize(x, prec) for x in b]
    568 # To parenthesize Pow with exp = -1 and having more than one Symbol

File ~/src/sympy/sympy/printing/str.py:38, in StrPrinter.parenthesize(self, item, level, strict)
     36     return "(%s)" % self._print(item)
     37 else:
---> 38     return self._print(item)

File ~/src/sympy/sympy/printing/printer.py:331, in Printer._print(self, expr, **kwargs)
    329     printmethod = getattr(self, printmethodname, None)
    330     if printmethod is not None:
--> 331         return printmethod(expr, **kwargs)
    332 # Unknown object, fall back to the emptyPrinter.
    333 return self.emptyPrinter(expr)

File ~/src/sympy/sympy/printing/codeprinter.py:582, in CodePrinter._print_not_supported(self, expr)
    580 def _print_not_supported(self, expr):
    581     if self._settings.get('strict', False):
--> 582         raise PrintMethodNotImplementedError("Unsupported by %s: %s" % (str(type(self)), str(type(expr))) + \
    583                          "\nSet the printer option 'strict' to False in order to generate partially printed code.")
    584     try:
    585         self._not_supported.add(expr)

PrintMethodNotImplementedError: Unsupported by <class 'sympy.printing.numpy.SciPyPrinter'>: <class 'sympy.core.function.Derivative'>
Set the printer option 'strict' to False in order to generate partially printed code.

[moorepants]

Note that it also fails in SymPy 1.12.1 but with a NameError when trying to evaluate the numerical function:

In [7]: func = sm.lambdify((f.diff(t, 2), f.diff(t), f, a, b), expr, cse=True)

In [8]: func(2.0, 3.0, 4.0, 5.0, 6.0)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[8], line 1
----> 1 func(2.0, 3.0, 4.0, 5.0, 6.0)

File <lambdifygenerated-2>:6, in _lambdifygenerated(_Dummy_39, _Dummy_40, _Dummy_41, a, b)
      1 def _lambdifygenerated(_Dummy_39, _Dummy_40, _Dummy_41, a, b):
      2     x0 = _Dummy_41
      3     return (  # Not supported in Python with SciPy and NumPy:
      4   # Derivative
      5   # Derivative
----> 6 a**2 + a*b*x0 + a*Derivative(x0, (t, 2)) + b*Derivative(x0, t))

NameError: name 'Derivative' is not defined

In [9]: func?
Signature: func(_Dummy_39, _Dummy_40, _Dummy_41, a, b)
Docstring:
Created with lambdify. Signature:

func(arg_0, arg_1, f, a, b)

Expression:

a**2 + a*b*f(t) + a*Derivative(f(t), (t, 2)) + b*Derivative(f(t), t)

Source code:

def _lambdifygenerated(_Dummy_39, _Dummy_40, _Dummy_41, a, b):
    x0 = _Dummy_41
    return (  # Not supported in Python with SciPy and NumPy:
  # Derivative
  # Derivative
a**2 + a*b*x0 + a*Derivative(x0, (t, 2)) + b*Derivative(x0, t))


Imported modules:
File:      ~/src/sympy/<lambdifygenerated-2>
Type:      function

[bjodah]

Thanks! So in order to debug this, I'd need some help finding an expression that:

1. for an older release of sympy: works with lambdify (i.e. no NameError when evaluated)
2. for master branch (or sympy-1.13.0rc1): fails with PrintMethodNotImplementedError

I have not yet seen such an expression.

[moorepants]

This expression expr = a*f.diff(t, 2) + b*f.diff(t) + a*b*f + a**2 fails with NameError in SymPy 1.12.1 and fails with the new error PrintMethodNotImplementedError in master. But I guess that is a better behavior now. Your new error handling finds the error when calling lambdify() and gives a more informative error.

The bug is that it works without cse=True but doesn't with cse=True. I may have opened a different issue about this because I remember trying to fix it but it would take a bit of surgery.

[bjodah]

Thank you @moorepants , the failure with cse=True should definitely be fixed. I'll see if I can figure out how to patch lambdify.

[bjodah]

@moorepants the failure case with cse=True is hopefully addressed in 8fb5fb6 (in gh-26678)

[moorepants]

Thanks for also looking into that. Much appreciated.

[ericpre]

Updating the example above, does it make sense that it works with atan but not arctan?

Atan versus Arctan

import sympy

def main(expr_str):
    expr = sympy.sympify(expr_str)

    x = [symbol for symbol in expr.free_symbols if symbol.name == "x"][0]
    variables = sympy.symbols([s.name for s in expr.free_symbols])
    real_variables = sympy.symbols([s.name for s in variables], real=True)

    # Replace symbols by real symbols to help with differentiation
    # according to git comment 9 years ago
    expr = expr.subs(
        {orig: real_ for (orig, real_) in zip(variables, real_variables)}
    )

    parameters = [var for var in real_variables if var.name != "x"]
    eval_expr = expr.evalf()

    print(f"{expr_str=} {real_variables}")
    print('='*40)
    print("")
    for p in parameters:
        grad_expr = sympy.diff(eval_expr, p)
        gef = grad_expr.evalf()
        f_grad = sympy.utilities.lambdify(
            real_variables,
            gef,
            modules="numpy",
            dummify=False,
        )
        print(f"{p=}\n  {grad_expr=}\n  {gef=}")
        n = len(real_variables)
        res = f_grad(*[i/(n+1) for i in range(n)])

if __name__ == '__main__':
    for expr_str in [
            "A * atan(k * (x - x0))",
            "A * arctan(k * (x - x0))"
            ]:
        main(expr_str)
        print("\n"*3)

[bjodah]

@ericpre the difference is that "atan" is recognized as the trigonometric function, while "arctan" is not:

>>> srepr(sympify('arctan(x)'))
"Function('arctan')(Symbol('x'))"

>>> srepr(sympify('atan(x)'))
"atan(Symbol('x'))"

parse_expr can offer more customization:

>>> parse_expr('atan(x)', transformations=(), local_dict={'x': x})
atan(x)
>>> parse_expr('atan(x)', transformations=(), local_dict={'x': x})
...
NameError: name 'arctan' is not defined

EDIT: and atan(x).diff(x) becomes 1/(x**2 + 1), while arctan(x).diff(x) becomes Derivative(arctan(x), (x, 1)).

[ericpre]

Thank you @bjodah, things are getting slowly more clear and for the case of the arctan, it seems that it was a correct failure and it possibly never worked before!

With the heaviside function, it seems that this is similar, the heaviside wasn't recognised correctly, however, after fixing it (replacing heaviside with Heaviside), there is a different error:

import sympy

expr = sympy.sympify('A*Heaviside(x-n,0.5)')

expr_grad_n = expr.evalf().diff("n")
variables = sympy.symbols([s.name for s in expr.free_symbols])

f_grad = sympy.utilities.lambdify(variables, expr_grad_n.evalf(), dummify=False)
n = len(variables)
res = f_grad(1, 2, 3)

gives the following error:

Traceback (most recent call last):

  Cell In[1], line 10
    res = f_grad(1, 2, 3)

  File <lambdifygenerated-1>:2 in _lambdifygenerated
    return -A*DiracDelta(-n + x)

NameError: name 'DiracDelta' is not defined

I get the same error with sympy 1.12 and 1.13.0rc1, so this doesn't seem to be a regression. Should I open a separate issue?

[bjodah]

@ericpre glad to hear.

Yes, please open new issues if you find a failure mode that was present even before PrintMethodNotImplementedError. As for DiracDelta I remember that its support in lambdify has been discussed (you can search for DiracDelta and lambdify, I couldn't immediately find the comment I was thinking of) and it was decided that we should not generate code for it by default.

The rationale is rather simple: people generally want useful (floating point) results when using lambdify, having e.g. a python code printer emit something like (float('inf') if x==0 else 0.0) is generally not helpful, and it can be quite hard to debug later in the process if expressions are big ("why is my nonlinear solver not converging?").

If this really is something that a user wants, then they can pass this explicitly as a user-known function to lambdify.