Comments (12)
If you want to reuse the connection I would recommend you to use the Connection pool. Because connections are not threadsafe and managing multiple connections would be a tricky task. While connection pools keeps them keep-alive, managing them and makes access to them threadsafe.
Also, few comments regarding you code:
You don't need to make connection and prepared statement as final
if you use them in try () catch
, they are effectively final anyway.
You don't need to preparedStatement.clearBatch()
in case preparedStatement was in try (preparedStatement)
block as it will be closed and cleared by GC anyway.
from clickhouse4j.
Hello. Code in the first snippet looks correct. From what I understand - you don't use Connection Pool. Is that correct? What driver version do you use?
Also, second snippet doesn't close the connections. While first one - does.
from clickhouse4j.
Yes first one close connection every batch insert in second one I try to reuse connections if they are not closed. Yes I don't use Connection Pool and I try clickhouse4j 1.1.1 and 1.4.1 with same results.
from clickhouse4j.
@rado-r do you use the code without try () catch
in multi threaded env?
from clickhouse4j.
@rado-r I did a quick fix - https://github.com/blynkkk/clickhouse4j/releases/download/clickhouse4j-1.4.1/clickhouse4j-1.4.2-SNAPSHOT.jar
Could you please check?
from clickhouse4j.
@rado-r hello. do you have any updates? Also, what Java version do you have?
from clickhouse4j.
@doom369 thank's for very fast response I test it and now it is fixed. I test old released version with connection pool (Hikari) but it don't releases ports too but it was only hundreds of ports. After this fix it is 0 ports. So gread job.
We use Java 8.
from clickhouse4j.
@rado-r so you used the provided build and it is fixes the issue, is that correct?
How many open connections did you saw? Were they in CLOSE_WAIT
state?
from clickhouse4j.
@doom369 yes provided build fix the issue.
Before fix I try to insert 2 billion records and connector uses all ports of docker there wasn't any free port to continue.
netstat -pnt |grep CLOSE_WAIT| wc -l
28232
from clickhouse4j.
What do you think when it will be on maven?
from clickhouse4j.
We will try to release today. However, this situation is very strange. Because in this driver we rely on HTTPUrlConnection
(while original driver relies on apache commons connection pool) in which connection pool is handled on the level of JVM. In old JVM there was a bug with non-closed connections. However, it was fixed many days ago, so now it should work. Also, on our envs (we have dozens of them) it is not the case. So it would be cool to find the actual cause.
Could you please provide also exact version of JVM and OS?
from clickhouse4j.
It should be
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
in docker openjdk:8-jre
but it is customised docker image so I'm not 100% sure.
from clickhouse4j.
Related Issues (20)
- Lz4 should use the fastest instance (current safe instance is used)
- Remove param descriptions to minimize driver size.
- Provide non-default plugable HTTP Clients
- Remove ExternalData support HOT 6
- Unable to connect to clickhouse through HAProxy HOT 7
- Provide a better method for setting the arrays
- Empty Authorization header is submitted if no auth is defined HOT 2
- Password with special characters not properly URL encoded HOT 1
- Is CopyManager thread safe ? HOT 1
- Error when insert `Array(String)` in Spark
- socket_timeout not working
- Merge changes back to official JDBC driver HOT 2
- how can i implements batch delete? HOT 1
- Improve JAR packaging (re-packaged net.jpountz.lz4)
- Server log filled with errors HOT 2
- ConnectionTimeout This property cannot be set and will be overridden by the default value
- SpringBoot 3.0.1: LoggerFactory is not a Logback LoggerContext but Logback is on the classpath
- Is project still active? HOT 1
- unable to catch any exception that occurs during copyFromDB call HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clickhouse4j.