Comments (24)
In positionDaoWithGrouping.js, instead of ibmdb.open, use pool.open.
from node-ibm_db.
@bimalkjha Thanks for the quick response. It doesn't work for me as the memory usage increases faster than ibmdb.open. I am just wondering if anyone else has the same issue or this is just for z/os db2.
from node-ibm_db.
The memory usage increases on db2 for z/OS or on test server where you run index.js? What is the OS running on this test server?
Also, please help to reproduce the issue. I tried your shared express app, but it does not seem working for me.
linuxcli: defect> node index.js
Example app listening at http://0.0.0.0:3000
When I opened linuxcli:3000 in mozila, I see the text "Cannot GET / " only.
from node-ibm_db.
I tested both Windows and Ubuntu Linux and the result is the same. I will create a repo to check in the app once I am in the office. Thanks.
from node-ibm_db.
@bimalkjha You have to use the following url to trigger the express get as it is defined in the server.js. I am sorry I didn't make it clear. Thanks.
http://localhost:3000/v1/parties/1/positions
from node-ibm_db.
@bimalkjha I have created the repo to reproduce this issue.
https://github.com/networknt/express-db2
from node-ibm_db.
@stevehu I am able to repro the issue and found the each call of positionDaoWithGrouping is creating a new ODBC instance and hence increasing the memory consumption. Only one instance of ODBC is sufficient. I have just committed a fix for this issue in intermediate_fixes branch. Please use it. Also, you need to use options with ibmdb.open() to reuse the ODBC instance as below:
var ibmdb = require("../"); //, Pool = ibmdb.Pool, pool = new Pool();
var options = {};
module.exports = exports = function (req, res) {
ibmdb.open(cn, options, function (err, connection) {
from node-ibm_db.
@bimalkjha I have tried the odbc.js and passed an empty options object into ibmdb.open as you suggested. It is getting better as the memory increase slows down; however, it is still increasing gradually. I think there are still leakage somewhere and wondering if you could try to run your server a little bit long to reproduced it. Thanks.
from node-ibm_db.
@stevehu I am looking into code to find any more places for leakage.. btw, can you try below code for positionDaoWithGrouping.js? It should consume less memory and give faster client response.
var ibmdb = require("ibm_db");
var connection = {};
ibmdb.open(cn, function(err, conn){
if(err) {
console.log(err);
return;
}
connection = conn;
});
module.exports = exports = function (req, res) {
connection.query(sql, function (err, rows) {
if (err) {
console.log(err);
return;
}
res.send(rows);
});
};
from node-ibm_db.
@bimalkjha It will speed up a lot in the express application in order to reproduce the issue. Good idea!!! In our code, we are using https://github.com/coopernurse/node-pool to cache 10 connections so that we don't create connections each time. I will update the express app in my repo. Thanks.
from node-ibm_db.
@stevehu I would request to try https://github.com/ibmdb/node-ibm_db/tree/intermediate_fixes . I have updated odbc.cpp
to free a leaking 1KB of memory per query.
from node-ibm_db.
@bimalkjha I have tried intermediate_fixes branch and the memory usage is still increasing gradually. One think I realized that the version number in the branch is 0.0.5 but the version on public npm is 0.0.8. I didn't compare master and the hotfix branch as I am assuming you branch out from master which is released to public npm. Thanks.
from node-ibm_db.
retest both odbc.js and odbc.cpp fixes from master branch and there are still memory leak. Thanks.
from node-ibm_db.
@bimalkjha I saw big improvement after the odbc.js fix but the second fix in odbc.cpp (1Kb per query) is not that significant. Also, I can feel the more records returned from the database, the faster the memory increases. It might not be accurate as I don't have hard fact. After the two fixes, the memory usage is still increasing pretty fast so I guess there are even bigger leak than the odbc.js fix or there are multiple small ones. Let me know if I can help. Thanks.
from node-ibm_db.
@stevehu Thanks for the feedback. We are looking into code to find further area of improvement.
from node-ibm_db.
@stevehu Use latest driver code from github using npm install git+https://[email protected]/ibmdb/node-ibm_db.git and update your application to close the ibmdb too as below:
ibmdb.open(cn, function (err, connection) {
if (err) {
console.log(err);
return;
}
connection.query("select 1 from sysibm.sysdummy1", function (err1, rows) {
if (err1) console.log(err1);
else console.log(rows);
connection.close(function(err2) {if(err2) console.log(err2);ibmdb.close(connection);});
});
});
from node-ibm_db.
@bimalkjha I have installed the new driver and the memory leak is gone without change my code. Thanks a lot for your help. Also I have observed that performance dropped from 713 to 504 for 5 minutes load test.
from node-ibm_db.
@stevehu Thanks for your patience and confirmation.
from node-ibm_db.
@bimalkjha I rebuilt my vm yesterday and installed the latest driver from master branch and the memory leak came back again. I am wondering what has been changed can cause it. One variable in play is my database now has 1000s rows instead of several the first time we tested. Thanks.
from node-ibm_db.
@stevehu it might be possible that some older existing driver in the system is causing it. Can you remove older installations and have fresh installation? Also, when you run npm install ibm_db
, check the ibm_db version. It should be 0.0.9. If it is other than it, means older version from the system is getting picked up. If npm install ibm_db
doesn't work, try npm install git+https://[email protected]/ibmdb/node-ibm_db.git
command. Thanks.
from node-ibm_db.
@bimalkjha This is a brand new virtual machine and I did checked the version is 0.0.9. Upon inspecting the commit log, I saw you have checked in something after this Jun 2, 2015 commit. I am trying to clone it and revert to that version to test it again. Also, could you please confirm the following code is correct. I see you have remove the empty options from README.
var dbConfig = require('./dbConfig.json');
var sql = 'SELECT ACC_NUM from ' +
dbConfig.schema + '.VWDSC_ACC_SEC_PSN fetch first 5 rows only';
var cn = 'DATABASE=' + dbConfig.database + ';HOSTNAME=' + dbConfig.host +
';PORT=' + dbConfig.port + ';PROTOCOL=TCPIP;UID=' + dbConfig.userId +
';PWD=' + dbConfig.passwd + ';';
var ibmdb = require("ibm_db");
var options = {};
module.exports = exports = function (req, res) {
ibmdb.open(cn, options, function (err, connection) {
if (err) {
console.log(err);
return;
}
connection.query(sql, function (err1, rows) {
if (err1) {
console.log(err1);
}
connection.close(function(err2) {
if (err2) {
console.log(err2);
}
});
res.send(rows);
});
});
};
from node-ibm_db.
yes code seems correct. We have updated code to do the same thing internally what empty options was doing. Passing options is optional.
from node-ibm_db.
@stevehu Please use latest [email protected]
driver and verify the issue.
I have used memwatch-next to monitor the application and see is there any memory leak, but I do not see any leaks. The heapUsed grows, but after sometime get reclaimed by javascript garbage collector and heapUsed get reduced. memwatch has not reported any memory leak on linuxamd64 so far using the latest ibm_db driver. If you want, I can share my application. Please check and let us know the result. Thanks.
from node-ibm_db.
@bimalkjha I am using 0.0.10 from master branch for my testing and so far it is stable on both standalone express app and my integrated app. Due to the massive change, I have to test all the scenarios in order to confirm the current version works. Thanks.
from node-ibm_db.
Related Issues (20)
- Bug Report_Getting " PrintDeps.exe Bad image" pop-ups while running tests from terminal using ibm_db package with Playwright with JavaScript HOT 8
- string returned after date type requested HOT 1
- Fetch multiple rows HOT 2
- adm-zip can it be moved to dev dependencies HOT 2
- queryResult deferred callback result changed with version 3.2.3 HOT 5
- Node ibm_db Installation Error with Python 3.12 HOT 2
- conn.query callback being fired before all queries are finished. (possibly) HOT 1
- Release v3.2.4 HOT 1
- Bug Report - Could not locate the bindings file. HOT 2
- Having trouble get a valid npm install in a GitLab CI Pipeline HOT 2
- libcrypto.so.1 error HOT 3
- Connection Pooling not working as expected .maxpool and settimeout having issue HOT 5
- I am tryingto install ibm_db in linux and getting error HOT 5
- GSKIT Error HOT 3
- No longer building on npm install HOT 11
- [IBM][CLI Driver] SQL1042C An unexpected system error occurred. SQLSTATE=58004 HOT 6
- Q: z/OS license being cached ? HOT 1
- npm install not working with podman HOT 7
- There is no build folder after installation of ibm_db HOT 5
- unhandledrejection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-ibm_db.