sap / go-hdb Goto Github PK
View Code? Open in Web Editor NEWSAP HANA Database Client for Go
License: Apache License 2.0
SAP HANA Database Client for Go
License: Apache License 2.0
Hi Dev,
I am trying to test out the this driver to pull out data from TCURR table (from HANA SPS6). From what I understand is that in that table, UKURS, FFACT, and TFACT are all Decimals in HANA DB. However when on Go, I tried fetch using row.Scan to Float64, but alas, getting error like this
2016/08/18 10:18:31 sql: Scan error on column index 5: converting driver.Value type []uint8 ("@\xe2\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0060") to a float64: invalid syntax
From what i understand is that the driver consider the Decimal of HANA become UINT8, is this normal behavior or is there any configuration to set so Decimal become Float(32 or 64)?
Thanks.
Is there a way to do bulk update just like bulk insert?
I am attempting to select data and write it to a csv file. Handling of most types works fine but handling of decimal types does not appear to be trivial.
...
n := len(cols)
vals := make([]interface{}, n)
dest := make([]interface{}, n)
for i := 0; i < n; i++ {
dest[i] = &vals[i]
}
for rows.Next() {
err = rows.Scan(dest...)
check(err, w)
for i, v := range vals {
switch v := v.(type) {
...
}
}
}
In code like the above decimal types are coming through as []byte.
Even trying to convert this representation to a string that I can write has proven elusive.
Am I doing something wrong?
Hello, I am trying to simply add binary data and get an error:
sql: converting argument $2 type: column converter for data [1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6] type [16]uint8 is not implemented
I used the BulkInsert sample and added a second binary field to the test table:
CREATE TABLE TEST (TESTCOL1 INT, TESTCOL2 BINARY(16));
the insert is for testing only:
stmt, err := db.Prepare("bulk insert into test values (?,?)") // Prepare bulk query.
myid := [16]byte {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6}
if _, err := stmt.Exec(i,myid); err != nil {
Is there something special to insert binary data?
Thanks!
When I print the columns type of SECONDDATE column in a query, returned type is TIMESTAMP.
Example query:
select CURRENT_DATE "date1",CURRENT_TIME "date2", CURRENT_TIMESTAMP "date3",CAST (CURRENT_TIMESTAMP AS SECONDDATE) "date4" from DUMMY
I think is necessary to return types without generalize to know and do integrations.
Hi,
I can't find anywhere in the documentation any information about how to connect to tenant databases. I'm guessing this can be done with a DSN option but I don't know how.
Thanks in advance.
Using v0.99.1, selecting a column ALPHANUM
always returns the column length in index 0.
Example:
If column value is a
with length 1
is returned [1 97]
If column value is a
with length 115
is returned [115 97]
I don't know if a bug or the expected behavior, if second, there will be documented.
I noticed that if wrong Dsn is provided (e.g. new line character inside password string, eg passw0rd\n
) then error message will be returned. This message would contain user and password in a plain text.
Line 70 in 81ab053
When trying to execute
CALL proc.myProc(?);
we get an invalid memory address or nil pointer dereference
error.
This seems to be related to the use of the question mark, so maybe the driver is interpreting it as a placeholder in a prepared statement? If so, is it possible to escape it?
In Ubuntu we have been facing a test failure on armhf (32 bits):
# github.com/SAP/go-hdb/internal/protocol [github.com/SAP/go-hdb/internal/protocol.test]
src/github.com/SAP/go-hdb/internal/protocol/convert_test.go:66:56: constant -2147483649 overflows int
src/github.com/SAP/go-hdb/internal/protocol/convert_test.go:67:56: constant 2147483648 overflows int
FAIL github.com/SAP/go-hdb/internal/protocol [build failed]
The failed test seems to not work on 32 bits architectures. Please, try to make sure the test suite works fine on it as well.
Hi, following the link SAP - Go (golang) Support a made the code below:
package main
import (
"database/sql"
_ "github.com/SAP/go-hdb/driver"
"log"
)
const (
driverName = "hdb"
hdbDsn = "hdb://user:password@hostname:port"
)
func main() {
db, err := sql.Open(driverName, hdbDsn)
if err != nil {
log.Print("error on sql open => " ,err)
}
err = db.Ping()
if err != nil {
log.Print("error on db.Ping() => " ,err)
}
}
but I have the following problem:
2019/11/04 14:59:24 error on db.Ping() => SQL Error 4321 - only secure connections are allowed
I, tried too this:
hdbDsn = "hdb://user:password@hostname:port?encrypt=true"
but is not possible connect to Hana database.
Someone could help me?
Att,
Hi Colleagues,
Could I ask you how to use the parameter value list to execute the bulk insert? We always get the following error.
sql: converting argument $1 type: unsupported cesu8Type conversion: []interface {} [100 706105387999 9656 429 cNWEO 7224270694 2465882072]
//bulkInsertParser retuns the value string: '100','7061053879999656','429','cNWEO','7224270694','2465882072'
bulkInsertValue := strings.Split(bulkInsertParser(bulkInsertGetValue, conn), ",")
var inter = make([]interface{}, colCnt)
inter[0] = 100
for m := 1; m < colCnt; m++ {
inter[m] = bulkInsertValue[m]
}
if _, err := stmt.ExecContext(ctx,inter); err != nil {
log.Fatal(err)
}
Dear SAP Engineers:
There was a strange problem,like this:
we try to get data from a HANA CUB,through two ways(sql1 & sql2),the first one can running correctly,the second one was running aborted,but those two sql-statements all are well in the HANA STUDIO.
//date format:yyyy-MM-dd
func BrandWithChannel(yourDate string)(list []_BrandWithChannel ,err error){
// utils.HANA_DRIVER, utils.HANA_DNS ------>
// const (
// HANA_DRIVER = "hdb"
// HANA_DNS = "hdb://SYSTEM:[email protected]:30015"
// ) <------ utils.HANA_DRIVER, utils.HANA_DNS
conn, _ := sql.Open(utils.HANA_DRIVER, utils.HANA_DNS)
//defer db.Close()
if err !=nil {
fmt.Println(err.Error())
}
var sql1 = fmt.Sprintf(`SELECT BEHVO,BVTXT,KTOKD
,CC_YEAR_SUM , CC_MONTH_SUM , CC_DAY_SUM ,
CC_TBYEAR_SUM , CC_TBMONTH_SUM , CC_HBMONTH_SUM "
from "_SYS_BIC"."MF_SALES.DAILYSALE/SALES_D_BEHVO" ('PLACEHOLDER' = ('$$IP_TO_DATE$$', '%s'))`
, yourDate)
var sql2 = fmt.Sprintf(`SELECT BEHVO,BVTXT,KTOKD
,sum(CC_YEAR_SUM) AS CC_YEAR_SUM, sum(CC_MONTH_SUM) AS CC_MONTH_SUM, sum(CC_DAY_SUM) AS CC_DAY_SUM,
sum(CC_TBYEAR_SUM) AS CC_TBYEAR_SUM, sum(CC_TBMONTH_SUM) AS CC_TBMONTH_SUM, sum(CC_HBMONTH_SUM) AS CC_HBMONTH_SUM
from "_SYS_BIC"."MF_SALES.DAILYSALE/SALES_D_BEHVO" ('PLACEHOLDER' = ('$$IP_TO_DATE$$', '%s'))
GROUP BY BEHVO,BVTXT,KTOKD`
, yourDate)
fmt.Printf("my sql1:%s \r\n",sql1)
fmt.Printf("my sql2:%s \r\n",sql2)
rows,err := conn.Query(sql2)
if err!=nil {
fmt.Printf("my sql:%s \r\nwhat happened:%s",sql2,err.Error())
return nil,err
}
for rows.Next() {
var temp _BrandWithChannel
var _CC_YEAR_SUM []uint8
var _CC_MONTH_SUM []uint8
var _CC_DAY_SUM []uint8
var _CC_TBYEAR_SUM []uint8
var _CC_TBMONTH_SUM []uint8
var _CC_HBMONTH_SUM []uint8
arr := []interface{}{
&temp.BrandCode,&temp.BrandAlias,&temp.ChannelCode,
&_CC_YEAR_SUM,&_CC_MONTH_SUM,&_CC_DAY_SUM,
&_CC_TBYEAR_SUM,&_CC_TBMONTH_SUM,&_CC_HBMONTH_SUM,
}
if err = rows.Scan(arr...);err != nil {
fmt.Printf("my sql:%s \r\nwhat happened:%s",sql2,err.Error())
return nil, err
} else {
temp.CC_YEAR_SUM = utils.ToFloat64(_CC_YEAR_SUM)
temp.CC_MONTH_SUM = utils.ToFloat64(_CC_MONTH_SUM)
temp.CC_DAY_SUM = utils.ToFloat64(_CC_DAY_SUM)
temp.CC_TBYEAR_SUM = utils.ToFloat64(_CC_TBYEAR_SUM)
temp.CC_TBMONTH_SUM = utils.ToFloat64(_CC_TBMONTH_SUM)
temp.CC_HBMONTH_SUM = utils.ToFloat64(_CC_HBMONTH_SUM)
list = append(list, temp)
}
}
defer rows.Close()
defer conn.Close()
return list,nil
}
[
{
"BrandCode": "SU",
"BrandAlias": "SU",
"ChannelCode": "Z015",
"ChannelAlias": "",
"CC_YEAR_SUM": 306703016.39,
"CC_MONTH_SUM": 14295292.31,
"CC_DAY_SUM": 655552,
"CC_TBYEAR_SUM": 279843503.64,
"CC_TBMONTH_SUM": 13457145.040000001,
"CC_HBMONTH_SUM": 9654517.27
},
.......
]
my sql:SELECT TOP 10 BEHVO,BVTXT,KTOKD
,sum(CC_YEAR_SUM), sum(CC_MONTH_SUM), sum(CC_DAY_SUM)
,sum(CC_TBYEAR_SUM), sum(CC_TBMONTH_SUM), sum(CC_HBMONTH_SUM)
from "_SYS_BIC"."MF_SALES.DAILYSALE/SALES_D_BEHVO" ('PLACEHOLDER' = ('$$IP_TO_DATE$$', '2017-10-10'))
GROUP BY BEHVO,BVTXT,KTOKD
what happened:Invalid CESU-8
hdb.protocol 2018/06/11 18:43:54 session.go:106: Connection read error local address 192.168.93.79:55780 remote address 192.168.2.34:30015: read tcp 192.168.93.79:55780->192.168.2.34:30015: i/o timeout
// Read implements the io.Reader interface.
func (c *sessionConn) Read(b []byte) (int, error) {
//set timeout
if err := c.conn.SetReadDeadline(time.Now().Add(c.timeout)); err != nil {
return 0, err
}
n, err := c.conn.Read(b)
if err != nil {
errLogger.Printf("Connection read error local address %s remote address %s: %s", c.conn.LocalAddr(), c.conn.RemoteAddr(), err)
c.isBad = true
c.badError = err
return n, driver.ErrBadConn
}
return n, nil
}
I have a table field with data type TIMESTAMP
.
Inserting data with 'CURRENT_TIMESTAMP' will store date in local time, but when using golang time.Now()
will store it in UTC.
time.Now() or time.Now().Local()
return something like 2020-04-20 18:34:53.566207 +0700 WIB
, in HANADB it is 2020-04-20 11:34:53
.
Is there a way to make HanaDB store it as golang local time rather than UTC?
Thanks.
In our HANA2 database we have a Column table with several NCLOB fields, some of them allow NULL data and when running a query with go-hdb Query("SELECT POTENTIALLY_NULL FROM A_TABLE")
the program hangs for close to 5 minutes before erroring out with the following
i/o timeout
2018/01/24 16:18:13 SQL HdbError 259 - invalid table name: Could not find table/view A_TABLE in schema SYSTEM: line 1 col 36 (at pos 35)
I am explicitly setting the SCHEMA to the correct value and the included code works fine if the field I select is NOT NULL.
The JDBC driver handles this with no issues.
I have ran WireShark and it seems that using the go-hdb driver the packets just stop flowing.
package main
import (
"database/sql"
"fmt"
"log"
"net/url"
_ "github.com/SAP/go-hdb/driver"
)
func getDbURL(host string, port int, username string, password string) string {
dsn := &url.URL{
Scheme: "hdb",
User: url.UserPassword(username, password),
Host: fmt.Sprintf("%s:%d", host, port),
}
return dsn.String()
}
func main() {
db, err := sql.Open("hdb", getDbURL("my.hana.host", 30015, "SYSTEM", "SouperSecretPassword"))
if err != nil {
log.Fatal(err)
}
if err := db.Ping(); err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec("SET SCHEMA MY_NOT_SYSTEM_SCHEMA")
if err != nil {
log.Fatal(err)
}
_, err = db.Query("SELECT POTENTIALLY_NULL FROM A_TABLE")
if err != nil {
log.Fatal(err)
}
}
Go Version: go version go1.9.3 darwin/amd64
go-hdb Version: const DriverVersion = "0.9.5"
Thanks for writing the go-hdb library!!
The plan to use Decimal with big.Rat restrict driver capability.
For example. Is mandatory create a definition for output var type Decimal. But this restrict custom queries for manipulation by the developer.
If an app make custom queries and output is a array interface with pointers (like this example):
rows, _ := db.Query("SELECT aDecimalColumn from...") // Note: Ignoring errors for brevity
cols, _ := rows.Columns()
// Create a slice of interface{}'s to represent each column,
// and a second slice to contain pointers to each item in the columns slice.
columns := make([]interface{}, len(cols))
columnPointers := make([]interface{}, len(cols))
for i, _ := range columns {
columnPointers[i] = &columns[i]
}
for rows.Next() {
// Scan the result into the column pointers...
if err := rows.Scan(columnPointers...); err != nil {
return err
}
// Handle columns....
}
The value of Decimal is a []uint8 with length 16. I tested and this driver don't provide a case to convert this to Decimal or big.Rat. This is because a dynamic app cannot be developed with this driver.
I tested with other Go drivers (Oracle, SQLite3, MySQL, DB2, MSSQL, Firebase and PostgreSQL) and with these are fine, some return in float64, others are []uint8 but are a string, and GoDror(Oracle) returns in his Number driver type with is a string
Drivers tested:
github.com/denisenkom/go-mssqldb
github.com/go-sql-driver/mysql
github.com/godror/godror
github.com/ibmdb/go_ibm_db
github.com/lib/pq
github.com/mattn/go-sqlite3
github.com/nakagami/firebirdsql
The question are:
Answers need to be in documentation.
Hi,
While packaging this library for Fedora, we noticed that README.md
and NOTICE
have executable permission on them in the releases; just wanted to open a ticket with you to let you know that you might want to chmod -x
them. :)
-Ed
Using v0.100.14, I do a select with all datatypes supported for SAP HANA DB to support them in Dixer.
Currently, upgrading to version v0.102.1 of this driver, I receive a panic when I select when a column is DECIMAL.
I'm using version 2.00.040.00.1553674765 in testing environment.
I confirm this panic is triggered in versions v0.101.2+
panic: Missing FieldType for typeCode tcFixed8
goroutine 24 [running]:
github.com/SAP/go-hdb/internal/protocol.typeCode.fieldType(0x51, 0x200, 0xc00036fd00)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/typecode.go:232 +0x478
github.com/SAP/go-hdb/internal/protocol.decodeRes(0xc00041cfc0, 0x51, 0x20, 0xc000514000, 0x200, 0x8)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/fieldtype.go:140 +0x35
github.com/SAP/go-hdb/internal/protocol.(*resultset).decode(0xc0000702d0, 0xc00041cfc0, 0xc0003b0cf0, 0xc00036fe00, 0x60446e)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/result.go:182 +0xe5
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).readPart(0xc000338280, 0xe36240, 0xc0000702d0, 0xc000506360, 0xc00041cfe0)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:569 +0x68
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).read(0xc000338280, 0xe36240, 0xc0000702d0, 0x0, 0xc00036fe78)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:485 +0x4e
github.com/SAP/go-hdb/internal/protocol.(*Session).QueryDirect.func1(0xc0003b0cf0)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/session.go:180 +0x92
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).iterateParts(0xc000338280, 0xc00036ff28, 0x102, 0xc00036ff18)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:635 +0x17f
github.com/SAP/go-hdb/internal/protocol.(*Session).QueryDirect(0xc000070030, 0xc000325520, 0x1b, 0xd8f701, 0x1373c20, 0x0, 0x0, 0xc0000886b8)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/session.go:171 +0x1c6
github.com/SAP/go-hdb/driver.(*Conn).QueryContext.func2(0xc00005e190, 0xc000325520, 0x1b, 0xc000088730, 0xc000088750, 0xc00045e180)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/driver/connection.go:503 +0x59
created by github.com/SAP/go-hdb/driver.(*Conn).QueryContext
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/driver/connection.go:502 +0x5c5
Builds of 0.12.1 fail on s390x because archEndian
is only defined in internal/protocol/littleendian.go
rather than defined in a package everything pulls in and set via init()
in littleendian.go
. Build failure is here:
https://kojipkgs.fedoraproject.org//work/tasks/7285/27867285/build.log
If I get a chance later tonight or tomorrow, I'll send a PR, but the fix is pretty straightforward.
It would be nice to assist debugging particular methods to be able to turn on and off the sqlTrace
e.g.
driver.Trace(true)
Are there any plans for adding other authentication methods such as the hdbuserstore key? It's been listed under the Todo items for some time, but we weren't sure if there's still interest.
Hi hope this is an easy one.
Does this error indicate something i didn't do?
server version 0.00.000.00 is not supported
Thanks.
Encountered this when the go-hdb was pulled in as a subdependency and started panicking during our app's testing when we upgraded from 0.9.2 to 0.9.4. In short, it looks like the Go version check added to driver/runtime.go expects the releases of Golang to use semantic versioning, when they don't. It's checking to make sure there are three parts—major, minor and patch—but for people running the Aug. 24 release (c8aec40
), called only "go1.9"
, this check fails. I'm assuming similar behavior would pop up with other releases such as go1.9beta2
.
Hi.
It happens that I have a database connection called Hana (Hana DB) with the programming language Go (Golang). Using the Iris framework.
What happens is that Go has problems with null data. You have to use special data types in the struct as (using the sql library that comes by default when installing Go) sql.NullString
The question is, I have a GetEmployeeData () function called when connecting to a certain URL and returning a JSON. The problem is that it does not return anything, a CONNECTION error REJECTED appears and in the console it prints an error: hdb.protocol 2018/07/13 16:31:06 option.go:120: type code tcNull not implemented exit status 1 The code was already tested in another SQL call in another function and it works, but this SQL I suppose that when returning 1 only Null or more it gives error.
The code is the following:
`func GetEmployeeData(ctx iris.Context) {
const (
driverName = "hdb"
hdbDsn = "hdb://SYSTEM:[email protected]:30015"
)
//db, err := sql.Open(driverName, hdbDsn)
//Crea una abstraccion de conexion a base de datos pero no conecta
//Si no que prepara todo para la conexion
db, err := sql.Open(driverName, hdbDsn)
if err != nil {
log.Fatal(err)
}
//Declaramos variable de QUERY
//var peticion = GetCompanyData()
//fmt.Println(peticion)
// var companyData = GetCompanyData()
var request = `Select
"U_idcodigo",
"U_idapenom",
"U_idnumint",
"U_ctestado",
"U_idemail",
"U_cttipfun",
"U_uocodcar",
TO_CHAR("U_uocarasi",'dd/mm/yyyy'),
"U_uocodfun",
TO_CHAR("U_uofunasi",'dd/mm/yyyy'),
"U_idestciv",
"U_idsexo",
"U_idnacion",
"U_codcat",
"U_codesp",
TO_CHAR("U_idfecnac",'dd/mm/yyyy'),
"U_idfotogr",
"U_dodirecc","
U_docodpai",
"U_doregion",
"U_doprovin",
"U_docomuna",
"U_docodpos",
"U_donummov",
"U_cttipcon",
TO_CHAR("U_ctfecing",'dd/mm/yyyy'),
"U_uolugpag",
"U_uolugtra",
"U_uocodger",
"U_uocoddep",
"U_uocodsec",
"U_rpforpag",
"U_rpcodban",
"U_rpctacte",
"U_rptipocta",
"U_rpunisal",
"U_rpmoneda",
TO_VARCHAR("U_rpsalar"),
"U_tijor",
TO_VARCHAR("U_hdia"),
TO_VARCHAR("U_cthorcon"),
"U_sindi",
"U_jubil",
"U_turno",
TO_CHAR("U_turnoasi",'dd/mm/yyyy'),
"U_comentar"
FROM SBODemoCL."@A1A_MAFU"
Where "U_idcodigo" = '7579684-6'`
rows, err := db.Query(request)
if err != nil {
log.Fatal(err)
fmt.Println("No funciono la peticion")
}
defer rows.Close()
defer db.Close()
fmt.Println("Funciono la peticion")
type Employee struct {
CodigoEmpleado sql.NullString `json:"CodigoEmpleado"` //CODIGO UNICO DE EMPLEADO
Nombres sql.NullString `json:"Nombres"` //APELLIDO Y NOMBRE
NroInterno sql.NullString `json:"NroInterno"` //NUMERO INTERNO EN EMPRESA
Estado sql.NullString `json:"Estado"`
Email sql.NullString `json:"Email"` //CORREO ELECTRONICO
TipoF sql.NullString `json:"TipoF"` //TIPO DE FUNCIONARIO
Cargo sql.NullString `json:"Cargo"` //EL CARGO QUE OCUPA EL EMPLEADO
FechaCargo sql.NullString `json:"FechaCargo"` //FECHA DE ASIGNACION DE CARGO
Funcion sql.NullString `json:"Funcion"` //QUE FUNCION CUMPLE
FechaFuncion sql.NullString `json:"FechaFuncion"` //CUANDO SE LE ASIGNO SU FUNCION
Civil sql.NullString `json:"Civil"` //ESTADO CIVIL
Sexo sql.NullString `json:"Sexo"` //SI ES MASCULINO O FEMENINO
Nacionalidad sql.NullString `json:"Nacionalidad"` //SU NACIONALIDAD
Categoría sql.NullString `json:"Categoría"` //SU CATEGORIA
Especialidad sql.NullString `json:"Especialidad"` //SU ESPECIALIDAD
Nacimiento sql.NullString `json:"Nacimiento"` //FECHA DE NACIMIENTO
Fotografia sql.NullString `json:"Fotografia"` //RUTA DE MAPA DE BITS DE IMAGEN SOBRE EL EMPLEADO
Direccion sql.NullString `json:"Direccion"` //DIRECCION DE CASA DEL EMPLEADO
Pais sql.NullString `json:"Pais"` //PAIS EN DONDE SE ENCUENTRA ACTUALMENTE
Region sql.NullString `json:"Region"` //REGION DONDE SE ENCUENTRA ACTUALMENTE
Provincia sql.NullString `json:"Provincia"` //PROVINCIA DONDE SE ENCUENTRA ACTUALMENTE
Comuna sql.NullString `json:"Comuna"` //COMUNA DONDE SE ENCUENTRA ACTUALMENTE
C_Postal sql.NullString `json:"C_Postal"` //SU CODIGO POSTAL
Celular sql.NullString `json:"Celular"` //NUMERO DE CELULAR
TipoContrato sql.NullString `json:"TipoContrato"` //TIPO DE CONTRATO
FechaIngreso sql.NullString `json:"FechaIngreso"` //FECHA DE INGRESO A LA COMPAÑIA
LugarPago sql.NullString `json:"LugarPago"` //LUGAR DONDE NORMALMENTE COBRA EL EMPLEADO
LugarTrabajo sql.NullString `json:"LugarTrabajo"` //LUGAR DONDE NORMALMENTE TRABAJA EL EMPLEADO
Gerencia sql.NullString `json:"Gerencia"` //¿Quien es su genente? No es claro
Departamento sql.NullString `json:"Departamento"` //¿Si tiene departamento? ¿O la ubicacion de su departamento? ¿O con departamento se refieren a su partido o barrio?
Seccion sql.NullString `json:"Seccion"` //¿? Tampoco me queda claro
FormaPago sql.NullString `json:"FormaPago"` //COMO SE LE PAGA NORMALMENTE AL EMPLEADO
Banco sql.NullString `json:"Banco"` //EL BANCO QUE SE UTILIZA PARA DEPOSITAR EL SALARIO DEL EMPLEADO
CuentaBanco sql.NullString `json:"CuentaBanco"` //CUENTA BANCARIO DONDE SE DEPOSITA EL SALARIO
UnidadSalarial sql.NullString `json:"UnidadSalarial"` //¿? Su unidad salarial
Moneda sql.NullString `json:"Moneda"` //CON QUE TIPO DE MONEDA SE DEPOSITA SU SUELDO (EUROS, DOLARES, PESOS, ETC...)
Sueldo sql.NullString `json:"Sueldo"` //SUELDO BASICO DEL EMPLEADO
TipoJornada sql.NullString `json:"TipoJornada"` //TIPO DE JORNADA DEL EMPLEADO (CANTIDAD DE HORAS QUE TRABAJA)
HorasDia sql.NullString `json:"HorasDia"` //HORAS DIARIAS QUE ESTIPULA EL CONTRATO
HorasMes sql.NullString `json:"HorasMes"` //HORAS MENSUALES QUE ESTIPULA EL CONTRATO
Sindicato sql.NullString `json:"Sindicato"` //SINDICATO AL QUE PERTENECE EL EMPLEADO
Jubilido sql.NullString `json:"Jubilido"` //¿? Puede ser si es que esta jubilado o no
Turno sql.NullString `json:"Turno"` //¿Turno de que?
FechaTurno sql.NullString `json:"FechaTurno"` //FECHA EN QUE SE LE ASIGNO EL TURNO
Comentarios sql.NullString `json:"Comentarios"` //Comentarios sobre el empleado
}
// numero := 0
//
// ac := accounting.Accounting{
// Symbol: "", //El símbolo
// Precision: 2, // ¿Cuántos "centavos" queremos? (también llamado precisión)
// Thousand: companyData["ISeparator"], //Separador de miles
// Decimal: companyData["FSeparator"], //Separador de decimales
// }
employ := new(Employee)
for rows.Next() {
err := rows.Scan(
&employ.CodigoEmpleado,
&employ.Nombres,
&employ.NroInterno,
&employ.Estado,
&employ.Email,
&employ.TipoF,
&employ.Cargo,
&employ.FechaCargo,
&employ.Funcion,
&employ.FechaFuncion,
&employ.Civil,
&employ.Sexo,
&employ.Nacionalidad,
&employ.Categoría,
&employ.Especialidad,
&employ.Nacimiento,
&employ.Fotografia,
&employ.Direccion,
&employ.Pais,
&employ.Region,
&employ.Provincia,
&employ.Comuna,
&employ.C_Postal,
&employ.Celular,
&employ.TipoContrato,
&employ.FechaIngreso,
&employ.LugarPago,
&employ.LugarTrabajo,
&employ.Gerencia,
&employ.Departamento,
&employ.Seccion,
&employ.FormaPago,
&employ.Banco,
&employ.CuentaBanco,
&employ.UnidadSalarial,
&employ.Moneda,
&employ.Sueldo,
&employ.TipoJornada,
&employ.HorasDia,
&employ.HorasMes,
&employ.Sindicato,
&employ.Jubilido,
&employ.Turno,
&employ.FechaTurno,
&employ.Comentarios,
)
if err != nil {
log.Fatal(err)
}
// f, err := strconv.ParseFloat(valor6, 64)
// salario := ac.FormatMoney(f)
}
err = rows.Err()
if err != nil {
log.Fatal(err)
fmt.Println("Ocurrio un error")
}
ctx.JSON(employ)
}`
The libraries that are imported are the following:
`import (
"database/sql"
"fmt"
"log"
"strconv"
"github.com/kataras/iris"
"github.com/leekchan/accounting"
// Register hdb driver.
_ "github.com/SAP/go-hdb/driver"
)`
The error calls an option.go file. All the code of option go is the following:
` /*
Copyright 2014 SAP SE
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package protocol
import (
"fmt"
"github.com/SAP/go-hdb/internal/bufio"
)
type booleanType bool
func (t booleanType) String() string {
return fmt.Sprintf("%t", t)
}
type intType int32
func (t intType) String() string {
return fmt.Sprintf("%d", t)
}
type bigintType int64
func (t bigintType) String() string {
return fmt.Sprintf("%d", t)
}
type doubleType float64
func (t doubleType) String() string {
return fmt.Sprintf("%g", t)
}
type stringType []byte
type binaryStringType []byte
func (t binaryStringType) String() string {
return fmt.Sprintf("%v", []byte(t))
}
//multi line options (number of lines in part header argumentCount)
type multiLineOptions []plainOptions
func (o multiLineOptions) size() int {
size := 0
for _, m := range o {
size += m.size()
}
return size
}
//pointer: append multiLineOptions itself
func (o *multiLineOptions) read(rd *bufio.Reader, lineCnt int) {
for i := 0; i < lineCnt; i++ {
m := plainOptions{}
cnt := rd.ReadInt16()
m.read(rd, int(cnt))
*o = append(*o, m)
}
}
func (o multiLineOptions) write(wr *bufio.Writer) {
for _, m := range o {
wr.WriteInt16(int16(len(m)))
m.write(wr)
}
}
type plainOptions map[int8]interface{}
func (o plainOptions) size() int {
size := 2 * len(o) //option + type
for _, v := range o {
switch v := v.(type) {
default:
outLogger.Fatalf("type %T not implemented", v)
case booleanType:
size++
case intType:
size += 4
case bigintType:
size += 8
case doubleType:
size += 8
case stringType:
size += (2 + len(v)) //length int16 + string length
case binaryStringType:
size += (2 + len(v)) //length int16 + string length
}
}
return size
}
func (o plainOptions) read(rd *bufio.Reader, cnt int) {
for i := 0; i < cnt; i++ {
k := rd.ReadInt8()
tc := rd.ReadB()
switch TypeCode(tc) {
default:
outLogger.Fatalf("type code %s not implemented", TypeCode(tc))
case tcBoolean:
o[k] = booleanType(rd.ReadBool())
case tcInteger:
o[k] = intType(rd.ReadInt32())
case tcBigint:
o[k] = bigintType(rd.ReadInt64())
case tcDouble:
o[k] = doubleType(rd.ReadFloat64())
case tcString:
size := rd.ReadInt16()
v := make([]byte, size)
rd.ReadFull(v)
o[k] = stringType(v)
case tcBstring:
size := rd.ReadInt16()
v := make([]byte, size)
rd.ReadFull(v)
o[k] = binaryStringType(v)
}
}
}
func (o plainOptions) write(wr *bufio.Writer) {
for k, v := range o {
wr.WriteInt8(k)
switch v := v.(type) {
default:
outLogger.Fatalf("type %T not implemented", v)
case booleanType:
wr.WriteInt8(int8(tcBoolean))
wr.WriteBool(bool(v))
case intType:
wr.WriteInt8(int8(tcInteger))
wr.WriteInt32(int32(v))
case bigintType:
wr.WriteInt8(int8(tcBigint))
wr.WriteInt64(int64(v))
case doubleType:
wr.WriteInt8(int8(tcDouble))
wr.WriteFloat64(float64(v))
case stringType:
wr.WriteInt8(int8(tcString))
wr.WriteInt16(int16(len(v)))
wr.Write(v)
case binaryStringType:
wr.WriteInt8(int8(tcBstring))
wr.WriteInt16(int16(len(v)))
wr.Write(v)
}
}
}
`
Does anyone know how to fix the error?
I'm using this driver to connect to HANA. I want to be able to pass the application ID of my program to HANA on the connection string. Example: "hbd://user:password@endpoint:port?SESSIONVARIABLE:APPLICATION=my_app_id"
I know this is possible in the cgo driver that comes with the HANA client, however changing from this to the cgo driver in our project would create several complications.
Are there any plans to add this functionality?
When using this driver I am unable to get a proper value for RowsAffected or LastInsertId
##RowsAffected
> SET SCHEMA MITCHELL;
Affected Rows 0
Last Insert ID: 0
> INSERT INTO TESTING VALUES ('moose2');
Affected Rows 0
Last Insert ID: 0
> INSERT INTO TESTING VALUES ('moose2ddd');
Affected Rows 0
Last Insert ID: 0
> select * from testing;
+-----------+
| DAA |
+-----------+
| googl |
| moose |
| moose2 |
| moose2 |
| moose2 |
| moose2ddd |
+-----------+
> CREATE COLUMN TABLE T2 (ID INTEGER NOT NULL GENERATED BY DEFAULT AS IDENTITY, NAME VARCHAR(10), PRIMARY KEY (ID));
Affected Rows 0
Last Insert ID: 0
> insert into "MITCHELL"."T2" ("NAME") values('K')
>>> ;
Affected Rows 0
Last Insert ID: 0
Are these features implemented?
I tested with main_test.go with long address as below.
flag.StringVar(&TestDSN, "dsn", "hdb://SYSTEM:Manager1@3754cf52-415f-445a-9d15-54041f4e891b.hana.dpuser-haas-hc-dev.dev-aws.hanacloud.ondemand.com:443", "database")
it failed with
hdb.protocol 2020/03/10 14:07:37 session.go:106: Connection read error local address 10.60.47.19:47438 remote address 3.126.219.43:443: EOF
hdb.protocol 2020/03/10 14:07:40 session.go:106: Connection read error local address 10.60.47.19:47462 remote address 3.126.219.43:443: EOF
hdb.protocol 2020/03/10 14:07:42 session.go:106: Connection read error local address 10.60.47.19:47463 remote address 3.126.219.43:443: EOF
2020/03/10 14:07:42 main_test.go:69: driver: bad connection
When trying to connect to the HANA DB in SAP Cloud Platform, I get the following message: SQL Error 4321 - only secure connections are allowed
. I have not found any security-related options though. Is this not supported? Do I need to use the other Go HANA driver instead?
I am trying to connect to an HDI schema in SAP Cloud Foundry if it matters.
The database itself is available. I can connect to it with Java/JDBC without any problems. I get the above message only when trying to use this driver.
Hello colleagues,
I have the following situation, which I'm not sure is a bug or just me not understanding something, but it started happening with release 0.100.9
We're using the driver via the standard go sql package.
So we have a db client dbClient, err := hana.Connect(connectionConfig)
We create a prepared statement insertStmt, err := dbClient.Prepare(insertStmtSQL)
And we call ExecContext() on the prepared statement. _, err = insertStmt.ExecContext(ctx, ...)
However sometimes the ctx passed to ExecContext is being cancelled.
And we're expecting that only the current ExecContext is being cancelled. And up untill 0.100.9 our expectations were true.
However since 0.100.9 we see session.go:295: Kill session 112685822327579
in the logs.
And any further uses of the PreparedStatement (insertStmt in above snipper) and the dbClient object fail with
hdb.protocol 2020/11/13 09:09:32 session.go:138: Connection read error local address x.x.x.x:xxxxx remote address x.x.x.x:xxxxx: read tcp x.x.x.x:xxxxx->x.x.x.x:xxxxx: use of closed network connection
hdb.protocol 2020/11/13 09:09:32 session.go:154: Connection write error local address x.x.x.x:xxxxx remote address x.x.x.x:xxxxx: set tcp 1x.x.x.x:xxxxx: use of closed network connection
I traced down the issue to this particular line that's been added in 0.100.9:
https://github.com/SAP/go-hdb/blob/main/driver/connection.go#L384
Hi,
While I was playing around with the driver it seems that I stumbled upon a bug.
Here is sample of code:
...
db, err := sql.Open(driver.DriverName, cnnstr)
...
defer db.Close()
rows, err := db.Query(`SELECT
(TABLE_A.HOST ||' -> '|| TABLE_A.SERVICE) as metric,
TABLE_B.TIMESTAMP AS time,
TABLE_B.MEASURE AS value
FROM HOSTS AS TABLE_A
JOIN HISTORY AS TABLE_B
ON TABLE_A.ID = TABLE_B.SERVICE_ID
WHERE TABLE_B.TIMESTAMP > ADD_DAYS(NOW(), -1)
ORDER BY TABLE_B.TIMESTAMP DESC LIMIT 20;`);
defer rows.Close()
...
columns, err := rows.Columns()
...
columnTypes, _ := rows.ColumnTypes()
for i := 0; i < len(columnTypes); i++ {
fmt.Println(fmt.Sprintf("Column %d has NAME: %s", i+1, columnTypes[i].Name()))
}
fmt.Println(fmt.Sprintf("COLUMNS ARE %d, COLUMN NAMES: %s", len(columns), columns))
...
This results in the following:
Column 1 has NAME:
Column 2 has NAME: TIMESTAMP
Column 3 has NAME: MEASURE
COLUMNS ARE 3, COLUMN NAMES: [ TIMESTAMP MEASURE]
I expect that the columns of the result table should have names: "metric", "time" and "value" as they were given as aliases.
Could you please have a look?
-Acho
The hana studio connection requires an instance number and a tenant database name for multiple containers. DSN query parameters do not appear to be supported.
Using v0.100.5
Querying a column type BINTEXT
panic the app.
Connect to db and query rows:
rows, err := db.Query(`select bintext_column from tbl`)
Error:
panic: Missing FieldType for typeCode tcBintext
goroutine 11 [running]:
github.com/SAP/go-hdb/internal/protocol.typeCode.fieldType(...)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/typecode.go:214
github.com/SAP/go-hdb/internal/protocol.decodeRes(0xc00010e240, 0x35, 0x1, 0xc00003e980, 0x200, 0x8)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/fieldtype.go:174 +0x241
github.com/SAP/go-hdb/internal/protocol.(*resultset).decode(0xc00006cf60, 0xc00010e240, 0xc000012360, 0x4, 0xc000131dd0)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/result.go:198 +0xf6
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).readPart(0xc00005ad20, 0x6fd300, 0xc00006cf60, 0xc0000481e0, 0xc00010e260)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:452 +0x6b
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).read(0xc00005ad20, 0x6fd300, 0xc00006cf60, 0x0, 0xc000131 C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:368 +0x4e
github.com/SAP/go-hdb/internal/protocol.(*Session).QueryDirect.func1(0xc000012360)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/session.go:381
+0x93
github.com/SAP/go-hdb/internal/protocol.(*protocolReader).iterateParts(0xc00005ad20, 0xc000131f10, 0x102, 0xc000131f00)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/protocol.go:518 +0x178
github.com/SAP/go-hdb/internal/protocol.(*Session).QueryDirect(0xc000048240, 0x6ab356, 0x15, 0x0, 0x0, 0x0, 0x0)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/internal/protocol/session.go:372
+0x216
github.com/SAP/go-hdb/driver.(*conn).QueryContext.func1(0xc00003e8d0, 0x6ab356, 0x15, 0xc00003e950, 0xc00003e960, 0x6fe000, 0xc000012018, 0xc00001c360)
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/driver/connection.go:267 +0x4d
created by github.com/SAP/go-hdb/driver.(*conn).QueryContext
C:/Users/sdelacruz/Documents/go/pkg/mod/github.com/!s!a!p/[email protected]/driver/connection.go:266 +0x5e
A complete reproducible program is not possible because the driver cannot insert a value in BINTEXT column yet, see #41 (comment)
When trying to bulk insert multiple rows, the driver will return error unexpected part kind partKind(x)
, x
being an integer. For example, using a two-table column where the first one is a primary key:
stmt.Exec("0", "ABC")
stmt.Exec("1", "DEF")
stmt.Exec("2", "GHI")
stmt.Exec() // no error
stmt.Exec("0", "ABC")
stmt.Exec("1", "DEF")
stmt.Exec("2", "GHI")
stmt.Exec() // partKind error
The last line will produce the part kind error instead of reporting Primary Key violation. In our actual software, we're also facing the issue that the table turns out empty, even though it should contain the 3 rows inserted by the first four lines. I could not, however, reproduce this in an isolated Go source.
I use the example,then show me "version 2.00.024.06.1538035880 is not supported"
I am trying to use the go driver to get decimals out of Hana. Can you please give me an example of pulling a decimal field and then converting it so that it shows up as the decimal value? I am seriously confused since I am new to Go. I can pull other datatypes without issue, but decimals come in all screwed up with strange unicode looking codes that say they are not valid unicode.
When I pull the data:
If the field type is decimal: ACWP_HRS_EXCESS_RUNTIME:"\ufffd\ufffd\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000080"
If the field type is double:
DBL_ACWP_HRS_EXCESS_RUNTIME: 6.392
How do I get the decimal to show 6.392 as well?
Thanks,
Scott
This is not a strictly go-hdb issue, not sure how else to reach go-hdb developers though. Feel free to close as invalid.
Any thoughts on Gorm https://github.com/jinzhu/gorm support? It seems that all that would be needed is implementation of Dialect interface in Gorm (not here, that's why this issue is not strictly go-hdb issue). As far as I can tell Gorm is one of the better, maybe best, ORM for Go.
Note: also created issue https://github.com/jinzhu/gorm/issues/615 in Gorm issue tracker where it's appropriate but you'd never see it.
A column with data type TEXT
or BINTEXT
is returned like a NCLOB
with this query:
types, _ := rows.ColumnTypes()
for _, v := range types {
log.Printf("%s", v.DatabaseTypeName())
}
So, for insert, I treated this like a Lob with lobexample_test.go
and this error is returned:
sql: converting argument $1 type: convert named value datatype error: 0 - DtUnknown
While debugging I found that the function DataType()
in typecode.go
returns DtUnknown
because the TypeCode
is 32 corresponding to tcNlocator
I did changes to treat this TypeCode
like a Lob
but this error was returned from hxe:
SQL Error 1033 - error while parsing protocol: no such data type: type_code=32, index=1
I created a table in SAP HANA DB with both Boolean and Tinyint datatypes, on retrieving DatabaseTypeName using "coltypes[i].DatabaseTypeName()" , I am getting TINYINT for field that is of Boolean datatype.
Is this a defect? I have to process values of fields based on datatype. I am not able to differentiate between tinyint field and boolean field.
I have this query:
SELECT m_disks.used_size / m_disks.total_size * 100 AS DISK_UTILIZATION
The data that gets returned form HANA looks like this:
"DISK_UTILIZATION": "7q\u0002\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000080"
I can work around this by using a CAST AS INT
in my SQL query, but it seems this might be an issue with the driver.
BLOB and LOB are returned with unexported variable *protocol.binaryLobChunkWriter
and cannot be used.
Also TEXT and BINTEXT are returned with unexported variable *protocol.charLobChunkWriter
and cannot be used.
If I print the value in the []interface{} I can see the returned value in the column in this style &{0xc0003ba000 445873439899649 8 8 0 true 0 <nil> [115 97 110 116 105 97 103 111]}
where the array is []bytes (driver.Value returned in []uint8, but doesn't matter) and this can be converted to santiago
easily.
In this case, I think is better return it like BINARY and VARBINARY datatypes are returned.
Hi, dear
I meet a trouble when I read decimal data from HANA.
I follow this document( https://godoc.org/github.com/SAP/go-hdb/driver#example-Decimal )
Here is a segment of code show us how to read decimal data from HANA :
var out Decimal // Declare scan variable.
if err := db.QueryRow(fmt.Sprintf("select * from %s.%s", TestSchema, tableName)).Scan(&out); err != nil {
log.Fatal(err)
}
fmt.Printf("Decimal value: %s", (*big.Rat)(&out).String()) // Cast scan variable to *big.Rat to use *big.Rat methods.
But I meet a trouble when I read decimal data as this way. When I run my code, the error is :
Scan error on column index 3, name "O_TOTALPRICE": unsupported Scan, storing driver.Value type []uint8 into type *main.Decimal
I think that means there is a mismatch of data type.
Here is my code :
tableRows, err := db.Query(tableQuery)
if err != nil {
log.Fatal(err)
}
defer tableRows.Close()
for tableRows.Next() {
order := orders{}
err := tableRows.Scan(&order.o_orderkey, &order.o_custkey, &order.o_orderstatus,
&order.o_totalprice, &order.o_orderdate, &order.o_orderpriority,
&order.o_clerk, &order.o_shippriority, &order.o_comment)
if err != nil {
log.Fatal(err)
}
fmt.Println(order)
}
Who can help on this? Thanks a lot.
Concurrent bulk inserts for 10 tables based on one connection get the error as below.
We don't close the connection before the end of the test.
hdb.driver 2021/01/06 18:01:46 connection.go:724: close: INSERT INTO BLKTEST.BSET(MANDT,BUKRS,BELNR,GJAHR,BUZEI) VALUES (?, ?, ?, ?, ?) - not flushed records: 249)
hdb.driver 2021/01/06 18:01:47 connection.go:724: close: INSERT INTO BLKTEST.CKMI1(MANDT,KALNR,BDATJ,POPER,AWTYP,AWREF,AWORG) VALUES (?, ?, ?, ?, ?, ?, ?) - not flushed records: 1)
hdb.driver 2021/01/06 18:01:51 connection.go:724: close: INSERT INTO BLKTEST.USRBF2(MANDT,BNAME,OBJCT,AUTH) VALUES (?, ?, ?, ?) - not flushed records: 1)
hdb.driver 2021/01/06 18:01:51 connection.go:724: close: INSERT INTO BLKTEST.USRBF2(MANDT,BNAME,OBJCT,AUTH) VALUES (?, ?, ?, ?) - not flushed records: 1)
hdb.driver 2021/01/06 18:01:51 connection.go:724: close: INSERT INTO BLKTEST.USRBF2(MANDT,BNAME,OBJCT,AUTH) VALUES (?, ?, ?, ?) - not flushed records: 1)
hdb.driver 2021/01/06 18:01:52 connection.go:724: close: INSERT INTO BLKTEST.BSID(MANDT,BUKRS,KUNNR,UMSKS,UMSKZ,AUGDT,AUGBL,ZUONR,GJAHR,BELNR,BUZEI) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) - not flushed records: 1)
hdb.driver 2021/01/06 18:01:52 connection.go:724: close: INSERT INTO BLKTEST.USRBF2(MANDT,BNAME,OBJCT,AUTH) VALUES (?, ?, ?, ?) - not flushed records: 169)
hdb.driver 2021/01/06 18:01:52 connection.go:724: close: INSERT INTO BLKTEST.USRBF2(MANDT,BNAME,OBJCT,AUTH) VALUES (?, ?, ?, ?) - not flushed records: 249)
I'm using go-hdb for the hana_sql_exporter to get sap/hana metrics for prometheus. With versions up to v0.99.2 a normal scrape with a few hundred selects on different tenants takes about 200ms. With go-hdb versions > v0.99.2 the same scrape takes more than 400ms.
It seems, that since v0.100 a lot of time is spent for crypto algorithms like shown in the following pprof top:
I modified the code for documentation (#38 (comment)) for tried with driver.NullDecimal instead driver.Decimal:
db, err := sql.Open("hdb", connstring)
if err != nil {
log.Fatal(err)
}
defer db.Close()
query := "select decimal_column from test"
rows, err := db.Query(query)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
//The 1 is the columns len returned in query
recordPointers := make([]interface{}, 1)
//For Decimal Column
var t driver.NullDecimal
recordPointers[0] = &t
for rows.Next() {
// Scan the result into the column pointers...
if err := rows.Scan(recordPointers...); err != nil {
log.Fatal(err)
}
for _, v := range recordPointers {
switch x := v.(type) {
case *driver.NullDecimal:
if x.Valid {
d := (*big.Rat)(x.Decimal)
log.Print(d.FloatString(6))
} else {
log.Print("NULL")
}
default:
log.Print("is not Decimal")
}
}
}
When decimal_column is NULL the code return NULL
but when is not null return the error:
sql: Scan error on column index 0, name "decimal_column": invalid decimal value <nil>
I expect the decimal value.
If driver.Decimal is used then I got the value but when column is NULL then the same error is returned.
% cat main.go
package main
import (
"database/sql"
"log"
"github.com/SAP/go-hdb/driver"
)
func main() {
connector := driver.NewBasicAuthConnector("HOST_NAME:PORT", "USER", "PASSWORD")
connector.SetTimeout(60)
hanaDB := sql.OpenDB(connector)
defer hanaDB.Close()
if err := hanaDB.Ping(); err != nil {
log.Fatal(err)
}
}
% go run main.go
2020/10/14 13:36:32 server version 0.00.000.00 is not supported
exit status 1
% cat go.mod
module https://github.com/xtez/hana
go 1.15
require github.com/SAP/go-hdb v0.101.1
% cat go.sum
github.com/SAP/go-hdb v0.101.1 h1:+8zjTmPL6bt6FsG+bZZluh0jc86QfUIwaydMHaRVjuI=
github.com/SAP/go-hdb v0.101.1/go.mod h1:sfLJoH5jM9mbyEKfNCfq1jnSOi5qYPeR+hvJprEfvpk=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
Additional Info:
SELECT VERSION FROM "SYS"."M_DATABASE";
VERSION
1.00.122.25.1559112397
I am trying to execute a stored procedure through the go-hdb module and get the following error:
SQL Error 1347 - Not recommended feature: DDL statement is used in Dynamic SQL (current dynamic_sql_ddl_error_level = 1)
Unfortunately I don't know the Create Statement of the prepared statement because it is created by default in my HANA SBSS instance. I was trying to execute the SYS_XS_SBSS.CREATE_BINDING_CREDENTIAL
procedure which is documented here, at least what kind of input and output parameters it requires.
It sounds like SBSS creates a new HANA user, the node-hdb guys seem to have ran into the same issue and were able to fix it.
Can you please fix that in the go-hdb connector as well?
{"_func":"HanaChaos/Executor.runTest","_level":"info","_loc":"sqljob.go:58","_time":"2020-09-01T03:21:05.517832857-07:00","msg":"Run Test Case:INSERT INTO DDL5 VALUES($2) "}
{"_func":"HanaChaos/Executor.runTest","_level":"info","_loc":"sqljob.go:60","_time":"2020-09-01T03:21:05.518816453-07:00","msg":"Run Test Case:INSERT INTO DDL5 VALUES($2) "}
{"_func":"HanaChaos/Executor.bulkInsert","_level":"info","_loc":"sqljob.go:96","_time":"2020-09-01T03:21:05.519362595-07:00","msg":"BULK INSERT INTO DDL5 VALUES(?, ?)"}
{"_func":"HanaChaos/Executor.bulkInsert","_level":"fatal","_loc":"sqljob.go:106","_time":"2020-09-01T03:21:05.679429537-07:00","_trace":"goroutine 15 [running]:\nruntime/debug.Stack(0x7b4789, 0x5, 0xc000759a50)\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x9d\nHanaChaos/Logging.preLogFatal(0xc000f345b0)\n\t/home/lzhang/go/src/HanaChaos/Logging/logger.go:66 +0x17a\nHanaChaos/Logging.Fatal(0xc000759c98, 0x1, 0x1)\n\t/home/lzhang/go/src/HanaChaos/Logging/logger.go:126 +0x26\nHanaChaos/Executor.bulkInsert(0xc0002e8300, 0xc000246120, 0x1b)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:106 +0x35b\nHanaChaos/Executor.runTest(0xc0002222a0, 0xc000222270, 0x0, 0x0)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:65 +0x1ce\nHanaChaos/Executor.(*SqlJob).Run(0xc000228080, 0xc00022e080, 0x2, 0x0)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:21 +0x44\nHanaChaos/Worker.Worker.Start.func1(0xc00007a420, 0xc000092900, 0xc000092960)\n\t/home/lzhang/go/src/HanaChaos/Worker/job.go:26 +0x108\ncreated by HanaChaos/Worker.Worker.Start\n\t/home/lzhang/go/src/HanaChaos/Worker/job.go:18 +0x53\n","msg":"sql: converting argument $2 type: unsupported lobCESU8Type conversion: string 0u5jwx7z9z"}
{"_func":"HanaChaos/Executor.bulkInsert","_level":"fatal","_loc":"sqljob.go:106","_time":"2020-09-01T03:21:05.680060927-07:00","_trace":"goroutine 15 [running]:\nruntime/debug.Stack(0x7b4789, 0x5, 0xc000759a50)\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x9d\nHanaChaos/Logging.preLogFatal(0xc000f348e0)\n\t/home/lzhang/go/src/HanaChaos/Logging/logger.go:66 +0x17a\nHanaChaos/Logging.Fatal(0xc000759c98, 0x1, 0x1)\n\t/home/lzhang/go/src/HanaChaos/Logging/logger.go:126 +0x26\nHanaChaos/Executor.bulkInsert(0xc0002e8300, 0xc000246120, 0x1b)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:106 +0x35b\nHanaChaos/Executor.runTest(0xc0002222a0, 0xc000222270, 0x0, 0x0)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:65 +0x1ce\nHanaChaos/Executor.(*SqlJob).Run(0xc000228080, 0xc00022e080, 0x2, 0x0)\n\t/home/lzhang/go/src/HanaChaos/Executor/sqljob.go:21 +0x44\nHanaChaos/Worker.Worker.Start.func1(0xc00007a420, 0xc000092900, 0xc000092960)\n\t/home/lzhang/go/src/HanaChaos/Worker/job.go:26 +0x108\ncreated by HanaChaos/Worker.Worker.Start\n\t/home/lzhang/go/src/HanaChaos/Worker/job.go:18 +0x53\n","msg":"sql: converting argument $2 type: unsupported lobCESU8Type conversion: string e1itpn70fg"}
This query can be executed well
DELETE FROM EMPLOYEE.REGISTRATION WHERE REGISTER_DATE = '2020-01-06';
But when using query param or prepared statement
params := []interface{}{'2020-01-06'}
_, err := tx.Exec(`DELETE FROM EMPLOYEE.REGISTRATION WHERE REGISTER_DATE = ?`, params...)
it will throw the following error. It happens for insert into
as well.
Error inserting data line 0:
sql: converting argument $1 type:
unsupported time conversion type error string 2020-01-06
Not sure if it's entirely correct but gorm code use QueryRow for INSERT, it apparently works with bunch of other databases (mysql, posgresql, ms sql etc.) so I guess it should work.
The problem seems to be that when running INSERT the result fieldSet is nil (in driver.go defaultQuery after s.session.Query s.resultFieldSet is nil) and it is passed to newQueryResult which panic at line (s.resultFieldSet is passed as fieldSet):
columns := make([]string, fieldSet.NumOutputField())
Stack trace and some additonal logs (two comments marked ELVIS are from the two functions mentioned above):
hdb 2015/08/28 15:55:42 driver.go:266: INSERT INTO "USER" ("COMPANY","BIRTHDAY_MONTH","USER_ID","BIRTHDAY_YEAR","BIRTHDAY_DAY","GENDER","FIRST_NAME","MIDDLE_NAME","LAST_NAME","TITLE","DISPLAY_NAME","JOB_TITLE") VALUES (?,?,?,?,?,?,?,?,?,?,?,?) [SHMU 5 17 1984 1 male TestUser <nil> TestUserPost Vazeny pan F. Novak Weather man]
hdb 2015/08/28 15:55:42 driver.go:274: ELVIS: defaultQuery: s.resultFieldSet [<nil>]
hdb 2015/08/28 15:55:42 driver.go:387: ELVIS: newQueryResult: fieldSet [<nil>]
(/usr/local/go/src/runtime/asm_amd64.s:402)
[2015-08-28 15:55:42] [145.58ms] INSERT INTO "USER" ("COMPANY","BIRTHDAY_MONTH","USER_ID","BIRTHDAY_YEAR","BIRTHDAY_DAY","GENDER","FIRST_NAME","MIDDLE_NAME","LAST_NAME","TITLE","DISPLAY_NAME","JOB_TITLE") VALUES ('SHMU','5','17','1984','1','male','TestUser',NULL,'TestUserPost','Vazeny pan','F. Novak','Weather man')
2015/08/28 15:55:42 http: panic serving 127.0.0.1:56219: runtime error: invalid memory address or nil pointer dereference
goroutine 13 [running]:
net/http.func·011()
/usr/local/go/src/net/http/server.go:1130 +0xbb
github.com/SAP/go-hdb/internal/protocol.(*FieldSet).NumOutputField(0x0, 0x8c0070)
/home/erik/work/aintu/go/src/github.com/SAP/go-hdb/internal/protocol/field.go:106 +0xd5
github.com/SAP/go-hdb/driver.newQueryResult(0xc208079360, 0x0, 0x0, 0xc2080c1340, 0x7fe97ca370f8, 0xc2081388c8, 0x0, 0x0, 0x0, 0x0)
/home/erik/work/aintu/go/src/github.com/SAP/go-hdb/driver/driver.go:388 +0x122
github.com/SAP/go-hdb/driver.(*stmt).defaultQuery(0xc208118a80, 0xc208124180, 0xc, 0xc, 0x0, 0x0, 0x0, 0x0)
/home/erik/work/aintu/go/src/github.com/SAP/go-hdb/driver/driver.go:276 +0x3ca
github.com/SAP/go-hdb/driver.(*stmt).Query(0xc208118a80, 0xc208124180, 0xc, 0xc, 0x0, 0x0, 0x0, 0x0)
/home/erik/work/aintu/go/src/github.com/SAP/go-hdb/driver/driver.go:255 +0x102
database/sql.rowsiFromStatement(0x7fe97ca37460, 0xc208031b00, 0x7fe97ca37420, 0xc208118a80, 0xc208170a00, 0xc, 0x10, 0x0, 0x0, 0x0, ...)
/usr/local/go/src/database/sql/sql.go:1489 +0x3a4
database/sql.(*DB).queryConn(0xc208078e60, 0xc208031b00, 0x9254e8, 0xc20814cf70, 0xcd, 0xc208170a00, 0xc, 0x10, 0xc, 0x0, ...)
/usr/local/go/src/database/sql/sql.go:978 +0x451
database/sql.(*Tx).Query(0xc20802ea00, 0xc20814cf70, 0xcd, 0xc208170a00, 0xc, 0x10, 0x84fbb0, 0x0, 0x0)
/usr/local/go/src/database/sql/sql.go:1259 +0x10b
database/sql.(*Tx).QueryRow(0xc20802ea00, 0xc20814cf70, 0xcd, 0xc208170a00, 0xc, 0x10, 0xd9)
/usr/local/go/src/database/sql/sql.go:1266 +0x66
github.com/jinzhu/gorm.Create(0xc20807e780)
/home/erik/work/aintu/go/src/github.com/jinzhu/gorm/callback_create.go:88 +0x919
... rest of the stack trace omitted, not relevant ...
As far as I can tell the QueryRow should not result in panic and work just like in other databases but I didn't find any documentation that would say what exactly QueryRow should do for non-SELECT statements.
From http://golang.org/pkg/database/sql/ description of ErrNoRows variable: "ErrNoRows is returned by Scan when QueryRow doesn't return a row. In such a case, QueryRow returns a placeholder *Row value that defers this error until a Scan." it seems that it's valid for QueryRow query to not return any rows.
QueryRow documentation (same place) suggest the same, saying that QueryRow returns at most one row.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.