Support of Oracle native NUMBER, to enable generic lossless data copying. about ocilib

[vrogier]

This could be a possibility but it requires some design. It cannot just add an OCI_Number type on its own. This would implies to add as well a bunch of new methods for manipulating them. And also adding a new C++ class with a bunch of operator overloads..

The issue is only limited to few "magic oracle values" such as positive and negative infinity.

I required more thinking about it. It will not be implemented in the (really) short term

[dbpm]

That is not just for magic values. As I stated before - both float and double are lossy types, some numeric values can't be presented using number->double, double->number conversions.
The same is true for string->number conversions - for some cases neither TM9 nor 999999.0999 approach is not able to handle values.

I can provide the examples, if that is required.

But at the moment what do we need - is to retreive 1:1 binary representation of NUMBER and to pass it to direct path API or as bind.

From first sight it can be the same way which was used for OCI_Date, OCI_Timestamp.

[vrogier]

Hi,

Exposing NUMBER directly implies adding the following methods:

boolean OCI_API OCI_BindNumber
boolean OCI_API OCI_BindArrayOfNumbers
unsigned int OCI_API OCI_GetNumber
unsigned int OCI_API OCI_GetNumber
unsigned int OCI_API OCI_ElemGetNumber
boolean OCI_API OCI_ElemSetNumber
boolean OCI_API OCI_RegisterNumber
unsigned int OCI_API OCI_ObjectGetNumber
boolean OCI_API OCI_ObjectSetNumber

And also add support for NUMBER manipulation using new wrappers around OCI functions:

OCINumberAbs()
OCINumberAdd()
OCINumberArcCos()
OCINumberArcSin()
OCINumberArcTan()
OCINumberArcTan2()
OCINumberAssign()
OCINumberCeil()
CINumberCmp()
OCINumberCos()
OCINumberDec()
OCINumberDiv()
OCINumberExp()
OCINumberFloor()
OCINumberFromInt()
OCINumberFromReal()
OCINumberFromText()
OCINumberHypCos()
OCINumberHypSin()
OCINumberHypTan()
OCINumberInc()
OCINumberIntPower()
OCINumberIsInt()
OCINumberIsZero()
OCINumberLn()
OCINumberLog()
OCINumberMod()
OCINumberMul()
OCINumberNeg()
OCINumberPower()
OCINumberPrec()
OCINumberRound()
OCINumberSetPi()
OCINumberSetZero()
OCINumberShift()
OCINumberSign()
OCINumberSin()
OCINumberSqrt()
OCINumberSub()
OCINumberTan()
OCINumberToInt()
OCINumberToReal()
OCINumberToText()
OCINumberTrunc()

Adding also documentation and testing.
About 70 new methods and some internal updates !

I will try to estimate the effort asap for an eventual ETA.

Regards,

Vincent

[dbpm]

I'd say that new wrapping methods starting from OCINumberAbs() to OCINumberTrunc() are redundant a bit. If it will be possible to access the OCINumber part straight using OCI_Number field - that would be enough too.

From other hand - next call looks much better and cleaner

return OCI_NumberTrunc(value);

rather than

ORA_CHECK(OCINumberTrunc(value->number));
return value;

Adding such handing is vital. I remember case when we tried to rewrite some ETL logic in C, from PL/SQL, but it was failed due lossy nature of double type - for some cases it lead to a bit wrong results in financial math. Having native lossless datatype could solve that case.

[vrogier]

Hi,

I've been working on supporting OCINumber type.

I've added the following:

New Type OCI_Number

// Instances management
OCI_NumberCreate()
OCI_NumberFree()
OCI_NumberArrayCreate()
OCI_NumberAssign()

// fetching
OCI_GetNumber()
OCI_GetNumber2()

// binding
OCI_BindNumber()
OCI_BindArrayOfNumbers()
OCI_RegisterNumber()

// conversions
OCI_NumberFromText()
OCI_NumberToText()

// handling numbers in collections and objects
OCI_ElemGetNumber()
OCI_ElemSetNumber()
OCI_ObjectGetNumber()
OCI_ObjectSetNumber()

I still need to work on at least some methods to manipulate them and exposing maybe the number parts.
And add demo, documentation about it
I will comit next week :)

Example with what is already implemented :

int main()
{
    OCI_Connection *cn;
    OCI_Statement  *st;
    OCI_Resultset  *rs;
    OCI_TypeInfo   *tc;
    OCI_TypeInfo   *to;
    OCI_Number     *nm;
    OCI_Coll       *cl;
    OCI_Elem       *el;
    OCI_Object     *ob;
    OCI_Number     *no;

    unsigned int i, n;
    otext buf[128] = "";

    if (!OCI_Initialize(err_handler, NULL, OCI_ENV_DEFAULT))
        return EXIT_FAILURE;

    cn = OCI_ConnectionCreate("db12c", "usr", "pwd", OCI_SESSION_DEFAULT);
    st = OCI_StatementCreate(cn);
    tc = OCI_TypeInfoGet(cn, "num_coll", OCI_TIF_TYPE);
    to = OCI_TypeInfoGet(cn, "test_num_t", OCI_TIF_TYPE);
    cl = OCI_CollCreate(tc);
    ob = OCI_ObjectCreate(cn, to);
    el = OCI_ElemCreate(tc);
    nm = OCI_NumberCreate(cn);

    // Testing fetching numbers (as number and using implicit conversion number to string)
    OCI_Prepare(st, "select value from test_number");
    OCI_Execute(st);
    rs = OCI_GetResultset(st);
    while (OCI_FetchNext(rs))
    {
        OCI_Number *n = OCI_GetNumber(rs, 1);
        const otext *s = OCI_GetString(rs, 1);

        OCI_NumberToText(n, NULL, sizeof(buf), buf);

        printf("%s - %s\n", buf, s);
    }

    // Testing binding number
    OCI_Prepare(st, "begin :1 := :1 *2 ; end;");
    OCI_BindNumber(st, ":1", nm);
    OCI_NumberFromText(nm, "1234.4321", NULL);
    OCI_Execute(st);
    OCI_NumberToText(nm, NULL, sizeof(buf), buf);
    printf("%s\n", buf);

    // Testing registering number for a returning into statement
    OCI_Prepare(st, "update test_number set value  = value *2 returning value into :1");
    OCI_RegisterNumber(st, ":1");
    OCI_Execute(st);
    rs = OCI_GetResultset(st);
    while (OCI_FetchNext(rs))
    {
        printf("%s\n", OCI_GetString(rs, 1));
    }

    // testing Collections
    OCI_NumberFromText(nm, "1111.2222", NULL);
    OCI_ElemSetNumber(el, nm);
    OCI_CollAppend(cl, el);
    OCI_NumberFromText(nm, "3333.4444", NULL);
    OCI_ElemSetNumber(el, nm);
    OCI_CollAppend(cl, el);
    printf("%d\n", OCI_CollGetCount(cl));
    OCI_ElemFree(el);

    for (i = 1, n = OCI_CollGetCount(cl); i <= n; i++)
    {
        OCI_Elem   *e = OCI_CollGetElem(cl, i);
        OCI_Number *n = OCI_ElemGetNumber(e);
        OCI_NumberToText(n, NULL, sizeof(buf), buf);
        printf("%s\n", buf);
    }

    // Testing objects
    OCI_NumberFromText(nm, "5555.6666", NULL);
    OCI_NumberToText(nm, NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_ObjectSetNumber(ob, "value", nm);
    no = OCI_ObjectGetNumber(ob, "value");
    OCI_NumberToText(no, NULL, sizeof(buf), buf);
    printf("%s\n", buf);

    OCI_Cleanup();
}

[dbpm]

Example above looks gorgeous (though, as usual).
I hope next code:

OCI_NumberFromText(nm, "~", NULL);
OCI_NumberToText(nm, NULL, sizeof(buf), buf);

OCI_NumberFromText(nm, "-~", NULL);
OCI_NumberToText(nm, NULL, sizeof(buf), buf);

works too? Sorry, can't test by my own yet :) Could you create git branch for the enhancement?

[vrogier]

Apparently, OCI does no support this ! even sql to_number() method does not support it !

Regarding numbers "magic values", we have only '0', '' and '-'
I presume that i can make the checks myself in order to handle this:)

I will add support for it. I will make a pull request by monday.

[vrogier]

Hi,

I've added support for infinite values.

Here is the updated demo code and output :)

Code available by monday in the repo :)

Output:

3.14 - 3.14
4.14 - 4.14
2468.8642
6.28
8.28
2
1111.2222
3333.4444
5555.6666
5555.6666
pos infinite = [~]
neg infinite = [-~]
1.2
~
5.6
-~
9.2

Code:

int main()
{
    OCI_Connection *cn;
    OCI_Statement  *st;
    OCI_Resultset  *rs;
    OCI_TypeInfo   *tc;
    OCI_TypeInfo   *to;
    OCI_Number     *nm;
    OCI_Coll       *cl;
    OCI_Elem       *el;
    OCI_Object     *ob;
    OCI_Number     *no;
    OCI_Number    **ar;

    unsigned int i, n;
    otext buf[128] = "";

    if (!OCI_Initialize(err_handler, NULL, OCI_ENV_DEFAULT))
        return EXIT_FAILURE;

    cn = OCI_ConnectionCreate("db12c", "usr", "pwd", OCI_SESSION_DEFAULT);
    st = OCI_StatementCreate(cn);
    tc = OCI_TypeInfoGet(cn, "num_coll", OCI_TIF_TYPE);
    to = OCI_TypeInfoGet(cn, "test_num_t", OCI_TIF_TYPE);
    cl = OCI_CollCreate(tc);
    ob = OCI_ObjectCreate(cn, to);
    el = OCI_ElemCreate(tc);
    nm = OCI_NumberCreate(cn);
    ar = OCI_NumberArrayCreate(cn, 5);

    // Testing fetching numbers (as number and using implicit conversion number to string)
    OCI_Prepare(st, "select value from test_number");
    OCI_Execute(st);
    rs = OCI_GetResultset(st);
    while (OCI_FetchNext(rs))
    {
        OCI_Number *n = OCI_GetNumber(rs, 1);
        const otext *s = OCI_GetString(rs, 1);

        OCI_NumberToText(n, NULL, sizeof(buf), buf);

        printf("%s - %s\n", buf, s);
    }

    // Testing binding number
    OCI_Prepare(st, "begin :1 := :1 *2 ; end;");
    OCI_BindNumber(st, ":1", nm);
    OCI_NumberFromText(nm, "1234.4321", NULL);
    OCI_Execute(st);
    OCI_NumberToText(nm, NULL, sizeof(buf), buf);
    printf("%s\n", buf);

    // Testing registering number for a returning into statement
    OCI_Prepare(st, "update test_number set value  = value *2 returning value into :1");
    OCI_RegisterNumber(st, ":1");
    OCI_Execute(st);
    rs = OCI_GetResultset(st);
    while (OCI_FetchNext(rs))
    {
        printf("%s\n", OCI_GetString(rs, 1));
    }

    // testing Collections
    OCI_NumberFromText(nm, "1111.2222", NULL);
    OCI_ElemSetNumber(el, nm);
    OCI_CollAppend(cl, el);
    OCI_NumberFromText(nm, "3333.4444", NULL);
    OCI_ElemSetNumber(el, nm);
    OCI_CollAppend(cl, el);
    printf("%d\n", OCI_CollGetCount(cl));

    for (i = 1, n = OCI_CollGetCount(cl); i <= n; i++)
    {
        OCI_Elem   *e = OCI_CollGetElem(cl, i);
        OCI_Number *n = OCI_ElemGetNumber(e);
        OCI_NumberToText(n, NULL, sizeof(buf), buf);
        printf("%s\n", buf);
    }

    // Testing objects
    OCI_NumberFromText(nm, "5555.6666", NULL);
    OCI_NumberToText(nm, NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_ObjectSetNumber(ob, "value", nm);
    no = OCI_ObjectGetNumber(ob, "value");
    OCI_NumberToText(no, NULL, sizeof(buf), buf);
    printf("%s\n", buf);

    // Testing fetching infinite values to string
    OCI_Prepare(st, "SELECT utl_raw.cast_to_number('FF65'), utl_raw.cast_to_number('00') from dual");
    OCI_Execute(st);
    rs = OCI_GetResultset(st);
    while (OCI_FetchNext(rs))
    {
        printf("pos infinite = [%s]\n", OCI_GetString(rs, 1));
        printf("neg infinite = [%s]\n", OCI_GetString(rs, 2));
    }

    // Testing array of numbers
    OCI_NumberFromText(ar[0], "1.2", NULL);
    OCI_NumberFromText(ar[1], "~", NULL);
    OCI_NumberFromText(ar[2], "5.6", NULL);
    OCI_NumberFromText(ar[3], "-~", NULL);
    OCI_NumberFromText(ar[4], "9.2", NULL);

    OCI_NumberToText(ar[0], NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_NumberToText(ar[1], NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_NumberToText(ar[2], NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_NumberToText(ar[3], NULL, sizeof(buf), buf);
    printf("%s\n", buf);
    OCI_NumberToText(ar[4], NULL, sizeof(buf), buf);
    printf("%s\n", buf);

    OCI_NumberArrayFree(ar);
    OCI_NumberFree(nm);
    OCI_ElemFree(el); 
    OCI_CollFree(tc);
    OCI_ObjectFree(ob);

    OCI_Cleanup();

    return EXIT_SUCCESS;
}

[vrogier]

Hi,

Basic support for OCINumber in the C API is now committed.

It must now be extended and be implemented in the C++ API !

Regards,

Vincent

[vrogier]

Hi,

I still need to add methods for basic ops (add, sub, mul and div) and fix a little issue when OCI_Number is created with null connection.
After that, ocinumber support will be complete in the C api.
I am working on its support in the C++ api.

Regards,

Vincent

[vrogier]

Hi,

I've finalized basic support for OCINumber i both C and C++ API.
Can you let me know if its okay like this in the git repo ?

thanks

vincent

[dbpm]

It's not possible to add one OCI_Number to another OCI_Number using the OCI_NumberAdd (and other functions) without excessive memcpy in OCI_NumberSetNativeValue.
Though it's not critical, I'd reimplement the macro.

[dbpm]

Complex testing for my project depends on this issue #42
Especially OCI_DirPathSetNumber(), OCI_DirPathSetDate(), OCI_DirPathSetTimestamp()

I already have own implementation and regressions, but can't switch to new ocilib without the dirpath functionality.

Could you implement issue #42, I'd provide final tests for both #41 and #42 then?