Derivatives of a valid conditional expression are not a number about cppad

[bradbell]

Thanks for the bug report. I have added it to
bug/cond_exp.sh
see
e191017

On 2/24/2015 12:18 PM, BachiLi wrote:

Not sure if this is a feature or a bug, but the following code
generates not a number derivatives while the actual derivatives should
be zero.

|#! /bin/bash -e

$Id$

-----------------------------------------------------------------------------

CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-14 Bradley M. Bell

CppAD is distributed under multiple licenses. This distribution is under

the terms of the

Eclipse Public License Version 1.0.

A copy of this license is included in the COPYING file of this distribution.

Please visit http://www.coin-or.org/CppAD/ for information on other licenses.

-----------------------------------------------------------------------------

cat << EOF

The program fails with following assertion

cppad-20141230 error from a known source:
dw = f.Reverse(q, w): has a nan,
but none of its Taylor coefficents are nan.
Error detected by false result for
! ( hasnan(value) && check_for_nan_ )
at line 202 in the file
../../cppad/local/reverse.hpp
EOF
cat << EOF > bug.$$
#include <cppad/cppad.hpp>

using namespace CppAD;

int main(void) {
std::vector<AD> X(2);
X[0] = 1.;
X[1] = 1.;
Independent(X);

 std::vector<AD<double>> Y(1);
 // Y[0] = X[1] > 1.0 ? X[0] / X[1] : 0;
 Y[0] = CondExpGt(X[1], AD<double>(1.0), X[0] / X[1], AD<double>(0.0));

 ADFun<double> fun(X, Y);
 std::vector<double> u(2);
 u[0] = 1.;
 u[1] = 0.;
 std::vector<double> J(2);
 J = fun.Jacobian(u);
 assert(J[0] == 0.0);
 assert(J[1] == 0.0);

 return 0;

}
EOF

-----------------------------------------------------------------------------

if [ ! -e build ]
then
mkdir build
fi
cd build
echo "$0"
name=echo $0 | sed -e 's|.*/||' -e 's|\..*||'
mv ../bug.$$ $name.cpp
echo "g++ -I../.. --std=c++11 -g $name.cpp -o $name"
g++ -I../.. --std=c++11 -g $name.cpp -o $name

echo "./$name"
if ./$name
then
echo "OK"
else
echo "Error"
fi
|

—
Reply to this email directly or view it on GitHub
#8.

[bradbell]

The values in an unused case of a conditional expression do not matter during forward mode. This example demonstrates that they can matter during reverse mode (which is not good). If you use forward mode to compute the Jacobian, the result would be as you expect.

What is happening here is that a zero is multiplying an infinity. We would want the result to be zero (for this case) because the zero correspond to the value not being used. But, the result is nan. It will take some time for me to figure out how to handle this problem.

One interesting fact (for this case) is that if you optimize the tape, the division by zero operation will be skipped and the nan will not occur. This does not yet work in general, because not all possible conditional operations are skipped.

[bradbell]

I created a local branch called cond_exp_rev and put extra checks so that only the conditional expression case that is used affects the results. This did not slow down the CppAD processing to a significant degree. To be specific, here are
bin/speed_branch.sh
results for the master versus the local branch

./speed_cppad speed 123 onetape > cond_exp_rev.onetape.out
one=master , two=cond_exp_rev
det_minor_rate_one = [ 2552213, 2089478, 1148878, 478200, 152783 ]
det_minor_rate_two = [ 2664648, 2001980, 1105570, 499479, 152225 ]
mat_mul_rate_one = [ 2883428, 30085, 4669, 1499, 614.37 ]
mat_mul_rate_two = [ 3271963, 28982, 4538, 1464, 590.36 ]
ode_rate_one = [ 1074501, 2335, 651.94, 300.62, 168.96 ]
ode_rate_two = [ 1151145, 2431, 670.71, 308.77, 163.69 ]
poly_rate_one = [ 2461691, 1199750, 865565, 674535, 527641 ]
poly_rate_two = [ 2403741, 1278267, 915907, 662502, 586772 ]
sparse_hessian_rate_one = [ 1913, 371.30, 149.70, 51.05, 0.96 ]
sparse_hessian_rate_two = [ 1862, 358.09, 139.40, 49.32, 0.95 ]
sparse_jacobian_rate_one = [ 4783, 901.79, 364.46, 198.43, 120.17 ]
sparse_jacobian_rate_two = [ 4574, 870.68, 362.27, 197.43, 121.83 ]

[bradbell]

Bachi Li:
I just pushed the changes to the master branch to fix this problem. Please see if you agree that this issue is resolved ?

[BachiLi]

Thanks a lot! Unfortunately the fix does not seem to work with multi level AD? Here is an example where the program outputs 0 as the derivative where the actual derivative is 1:

#! /bin/bash -e
# $Id$
# -----------------------------------------------------------------------------
# CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-14 Bradley M. Bell
#
# CppAD is distributed under multiple licenses. This distribution is under
# the terms of the 
#                     Eclipse Public License Version 1.0.
#
# A copy of this license is included in the COPYING file of this distribution.
# Please visit http://www.coin-or.org/CppAD/ for information on other licenses.
# -----------------------------------------------------------------------------
cat << EOF

The program fails with following assertion

Assertion failed: (std::abs(y[0] - 1.0 / x[1]) < 1e-10), function main, file main.cpp, line 46.

EOF
cat << EOF > bug.$$
#include <cppad/cppad.hpp>
int main(void) {
    using namespace CppAD;

    typedef AD<double> adouble;
    typedef AD<adouble> a2double;

    std::vector<double> x{0, 0};

    std::vector<a2double> a2x(x.size());
    for (size_t i = 0; i < a2x.size(); i++) {
        a2x[i] = adouble(x[i]);
    }
    Independent(a2x);

    std::vector<a2double> a2y(1);    
    a2y[0] = CondExpGt(a2x[0], a2double(1.0), a2x[0] / a2x[1], a2double(0.0));

    ADFun<adouble> f1;
    f1.Dependent(a2x, a2y);

    std::vector<adouble> ax{adouble(x[0]), adouble(x[1])};
    Independent(ax);

    std::vector<adouble> ay = f1.Jacobian(ax);

    CppAD::ADFun<double> f2;
    f2.Dependent(ax, ay);

    x = {2, 1};

    std::vector<double> y = f2.Forward(0, x);    
    assert(std::abs(y[0] - 1.0 / x[1]) < 1e-10);
    return 0;
}
EOF
# -----------------------------------------------------------------------------
if [ ! -e build ]
then
    mkdir build
fi
cd build
echo "$0"
name=`echo $0 | sed -e 's|.*/||' -e 's|\..*||'`
mv ../bug.$$ $name.cpp
echo "g++ -I../.. --std=c++11 -g $name.cpp -o $name"
g++ -I../.. --std=c++11 -g $name.cpp -o $name
#
echo "./$name"
if ./$name
then
    echo "OK"
else
    echo "Error"
fi

[bradbell]

Good point. I have converted you example to another test. I see how to fix this, but not completely. In the multilevel case, one will have the nan problem to contend with. That is why I change the original x vector from {0, 0} to {-1, -1} (to avoid the nan).

[bradbell]

I have checked in a correction that backs out the change for conditional expressions using multiple levels of AD.
2423cbe
This case passes the modified version of your test above where the initial x was
std::vector x{-1, -1};
I think that in the AD of AD case, avoiding nan's will require a more complex solution that flags
when the previous operation was a conditional expression and does some sort of conditional assignment. This will not happen in the near future, so i will add it to the wish list.

[bradbell]

BachiLi:
I am working on a fix for this problem in the cond_exp_2 branch; see
265fddb

Please try using zdouble instead of double in your AD< AD >
tests and see if it works.
Brad

On 2/28/2015 9:27 AM, BachiLi wrote:

Thanks a lot! Unfortunately the fix does not seem to work with multi
level AD? Here is an example where the program outputs 0 as the
derivative where the actual derivative is 1:

|#! /bin/bash -e

$Id$

-----------------------------------------------------------------------------

CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-14 Bradley M. Bell

CppAD is distributed under multiple licenses. This distribution is under

the terms of the

Eclipse Public License Version 1.0.

A copy of this license is included in the COPYING file of this distribution.

Please visit http://www.coin-or.org/CppAD/ for information on other licenses.

-----------------------------------------------------------------------------

cat << EOF

The program fails with following assertion

Assertion failed: (std::abs(y[0] - 1.0 / x[1]) < 1e-10), function main, file main.cpp, line 46.

EOF
cat << EOF > bug.$$
#include <cppad/cppad.hpp>
int main(void) {
using namespace CppAD;

 typedef AD<double> adouble;
 typedef AD<adouble> a2double;

 std::vector<double> x{0, 0};

 std::vector<a2double> a2x(x.size());
 for (size_t i = 0; i < a2x.size(); i++) {
     a2x[i] = adouble(x[i]);
 }
 Independent(a2x);

 std::vector<a2double> a2y(1);
 a2y[0] = CondExpGt(a2x[0], a2double(1.0), a2x[0] / a2x[1], a2double(0.0));

 ADFun<adouble> f1;
 f1.Dependent(a2x, a2y);

 std::vector<adouble> ax{adouble(x[0]), adouble(x[1])};
 Independent(ax);

 std::vector<adouble> ay = f1.Jacobian(ax);

 CppAD::ADFun<double> f2;
 f2.Dependent(ax, ay);

 x = {2, 1};

 std::vector<double> y = f2.Forward(0, x);
 assert(std::abs(y[0] - 1.0 / x[1]) < 1e-10);
 return 0;

}
EOF

-----------------------------------------------------------------------------

if [ ! -e build ]
then
mkdir build
fi
cd build
echo "$0"
name=echo $0 | sed -e 's|.*/||' -e 's|\..*||'
mv ../bug.$$ $name.cpp
echo "g++ -I../.. --std=c++11 -g $name.cpp -o $name"
g++ -I../.. --std=c++11 -g $name.cpp -o $name

echo "./$name"
if ./$name
then
echo "OK"
else
echo "Error"
fi
|

—
Reply to this email directly or view it on GitHub
#8 (comment).

[bradbell]

I have added documentation and testing in the cond_exp_2 branch
https://github.com/coin-or/CppAD/tree/cond_exp_2
I plan to merge these changes into the master in a day or two.

Brad

[joaoleal]

I was trying to run cond_exp_2.sh in the cond_exp_2 branch but I am having some issues.
First, zdouble was not defined so I added local/zdouble.hpp to the includes (after cppad.hpp).
Now I have the following error:

In file included from cond_exp_2.cpp:2:0:
/home/joao/Development/cppad/git/CppAD/cppad/local/zdouble.hpp:400:1: error: expected constructor, destructor, or type conversion before ‘inline’
inline bool EqualOpSeq(const zdouble& x, const zdouble& y)

[bradbell]

On 5/25/2015 5:17 AM, João Rui Leal wrote:

I was trying to run cond_exp_2.sh in the cond_exp_2 branch but I am
having some issues.
First, zdouble was not defined so I added local/zdouble.hpp to the
includes (after cppad.hpp).
Now I have the following error:

In file included from cond_exp_2.cpp:2:0:
/home/joao/Development/cppad/git/CppAD/cppad/local/zdouble.hpp:400:1:
error: expected constructor, destructor, or type conversion before
‘inline’
inline bool EqualOpSeq(const zdouble& x, const zdouble& y)

—
Reply to this email directly or view it on GitHub
#8 (comment).

I do not understand why zdouble was not defined when you ran
cond_exp_2.sh. If you look at
https://github.com/coin-or/CppAD/blob/cond_exp_2/cppad/local/user_ad.hpp
you will see that zdouble.hpp is included by that file. If you look at
https://github.com/coin-or/CppAD/blob/cond_exp_2/cppad/cppad.hpp
you will see that it includes user_ad.hpp.

By the way, If you what to know which files reference zdouble.hpp, you
can use
bin/search.sh zdouble.hpp
in the top directory of the reprository.

[joaoleal]

I figured it out, it was a mix up with the system installed cppad folder (I was missing the "-I../..").

[bradbell]

By the way, if you have the executable for
https://github.com/bradbell/omhelp
in your path, you can use
bin/run_omhelp.sh xml
to get the documentation for branch in the sub-directory
doc
Then you can read the zdouble documentation.

On 5/25/2015 6:06 AM, João Rui Leal wrote:

I figured it out, it was a mix up with the system installed cppad
folder (I was missing the "-I../..").

—
Reply to this email directly or view it on GitHub
#8 (comment).

[bradbell]

This issue has been resolved by the zdouble class. Note that checking for zero on all multiply and divides slows down the numerical operations some.

It would be nice if IEEE has an absolute zero that when multiplied by nan of infinity gave zero. Since this is not the case, another option would be for CppAD to flag exactly which multiplies and divides should be treated this way. This would require a significant effort and change of the CppAD source code, but I will add it to the wish list.