Memory leakage when query on z/os db2 about node-ibm_db

[bimalkjha]

In positionDaoWithGrouping.js, instead of ibmdb.open, use pool.open.

[stevehu]

@bimalkjha Thanks for the quick response. It doesn't work for me as the memory usage increases faster than ibmdb.open. I am just wondering if anyone else has the same issue or this is just for z/os db2.

[bimalkjha]

The memory usage increases on db2 for z/OS or on test server where you run index.js? What is the OS running on this test server?

Also, please help to reproduce the issue. I tried your shared express app, but it does not seem working for me.

linuxcli: defect> node index.js
Example app listening at http://0.0.0.0:3000

When I opened linuxcli:3000 in mozila, I see the text "Cannot GET / " only.

[stevehu]

I tested both Windows and Ubuntu Linux and the result is the same. I will create a repo to check in the app once I am in the office. Thanks.

[stevehu]

@bimalkjha You have to use the following url to trigger the express get as it is defined in the server.js. I am sorry I didn't make it clear. Thanks.

http://localhost:3000/v1/parties/1/positions

[stevehu]

@bimalkjha I have created the repo to reproduce this issue.

https://github.com/networknt/express-db2

[bimalkjha]

@stevehu I am able to repro the issue and found the each call of positionDaoWithGrouping is creating a new ODBC instance and hence increasing the memory consumption. Only one instance of ODBC is sufficient. I have just committed a fix for this issue in intermediate_fixes branch. Please use it. Also, you need to use options with ibmdb.open() to reuse the ODBC instance as below:

var ibmdb = require("../"); //, Pool = ibmdb.Pool, pool = new Pool();
var options = {};

module.exports = exports = function (req, res) {
ibmdb.open(cn, options, function (err, connection) {

[stevehu]

@bimalkjha I have tried the odbc.js and passed an empty options object into ibmdb.open as you suggested. It is getting better as the memory increase slows down; however, it is still increasing gradually. I think there are still leakage somewhere and wondering if you could try to run your server a little bit long to reproduced it. Thanks.

[bimalkjha]

@stevehu I am looking into code to find any more places for leakage.. btw, can you try below code for positionDaoWithGrouping.js? It should consume less memory and give faster client response.

var ibmdb = require("ibm_db"); 
var connection = {};
ibmdb.open(cn, function(err, conn){
  if(err) {
    console.log(err);
    return;
  }
  connection = conn;
});

module.exports = exports = function (req, res) {
      connection.query(sql, function (err, rows) {
        if (err) {
          console.log(err);
          return;
        }
        res.send(rows);
      });
};

[stevehu]

@bimalkjha It will speed up a lot in the express application in order to reproduce the issue. Good idea!!! In our code, we are using https://github.com/coopernurse/node-pool to cache 10 connections so that we don't create connections each time. I will update the express app in my repo. Thanks.

[bimalkjha]

@stevehu I would request to try https://github.com/ibmdb/node-ibm_db/tree/intermediate_fixes . I have updated odbc.cpp to free a leaking 1KB of memory per query.

[stevehu]

@bimalkjha I have tried intermediate_fixes branch and the memory usage is still increasing gradually. One think I realized that the version number in the branch is 0.0.5 but the version on public npm is 0.0.8. I didn't compare master and the hotfix branch as I am assuming you branch out from master which is released to public npm. Thanks.

[stevehu]

retest both odbc.js and odbc.cpp fixes from master branch and there are still memory leak. Thanks.

[stevehu]

@bimalkjha I saw big improvement after the odbc.js fix but the second fix in odbc.cpp (1Kb per query) is not that significant. Also, I can feel the more records returned from the database, the faster the memory increases. It might not be accurate as I don't have hard fact. After the two fixes, the memory usage is still increasing pretty fast so I guess there are even bigger leak than the odbc.js fix or there are multiple small ones. Let me know if I can help. Thanks.

[bimalkjha]

@stevehu Thanks for the feedback. We are looking into code to find further area of improvement.

[bimalkjha]

@stevehu Use latest driver code from github using npm install git+https://[email protected]/ibmdb/node-ibm_db.git and update your application to close the ibmdb too as below:

ibmdb.open(cn, function (err, connection) {
    if (err) {
      console.log(err);
      return;
    }
    connection.query("select 1 from sysibm.sysdummy1", function (err1, rows) {
      if (err1) console.log(err1);
      else console.log(rows);
      connection.close(function(err2) {if(err2) console.log(err2);ibmdb.close(connection);});
    });
});

[stevehu]

@bimalkjha I have installed the new driver and the memory leak is gone without change my code. Thanks a lot for your help. Also I have observed that performance dropped from 713 to 504 for 5 minutes load test.

[bimalkjha]

@stevehu Thanks for your patience and confirmation.

[stevehu]

@bimalkjha I rebuilt my vm yesterday and installed the latest driver from master branch and the memory leak came back again. I am wondering what has been changed can cause it. One variable in play is my database now has 1000s rows instead of several the first time we tested. Thanks.

[bimalkjha]

@stevehu it might be possible that some older existing driver in the system is causing it. Can you remove older installations and have fresh installation? Also, when you run npm install ibm_db, check the ibm_db version. It should be 0.0.9. If it is other than it, means older version from the system is getting picked up. If npm install ibm_db doesn't work, try npm install git+https://[email protected]/ibmdb/node-ibm_db.git command. Thanks.

[stevehu]

@bimalkjha This is a brand new virtual machine and I did checked the version is 0.0.9. Upon inspecting the commit log, I saw you have checked in something after this Jun 2, 2015 commit. I am trying to clone it and revert to that version to test it again. Also, could you please confirm the following code is correct. I see you have remove the empty options from README.

var dbConfig = require('./dbConfig.json');

var sql = 'SELECT ACC_NUM from ' +
  dbConfig.schema + '.VWDSC_ACC_SEC_PSN fetch first 5 rows only';

var cn = 'DATABASE=' + dbConfig.database + ';HOSTNAME=' + dbConfig.host +
  ';PORT=' + dbConfig.port + ';PROTOCOL=TCPIP;UID=' + dbConfig.userId +
  ';PWD=' + dbConfig.passwd + ';';

var ibmdb = require("ibm_db");
var options = {};

module.exports = exports = function (req, res) {

  ibmdb.open(cn, options, function (err, connection) {
    if (err) {
      console.log(err);
      return;
    }
    connection.query(sql, function (err1, rows) {
      if (err1) {
        console.log(err1);
      } 
      connection.close(function(err2) {
        if (err2) {
          console.log(err2);
        }
      });
      res.send(rows);
    });
  });
};

[bimalkjha]

yes code seems correct. We have updated code to do the same thing internally what empty options was doing. Passing options is optional.

[bimalkjha]

@stevehu Please use latest [email protected] driver and verify the issue.

I have used memwatch-next to monitor the application and see is there any memory leak, but I do not see any leaks. The heapUsed grows, but after sometime get reclaimed by javascript garbage collector and heapUsed get reduced. memwatch has not reported any memory leak on linuxamd64 so far using the latest ibm_db driver. If you want, I can share my application. Please check and let us know the result. Thanks.

[stevehu]

@bimalkjha I am using 0.0.10 from master branch for my testing and so far it is stable on both standalone express app and my integrated app. Due to the massive change, I have to test all the scenarios in order to confirm the current version works. Thanks.