Comments (6)
from java-spanner.
This is caused by an infinite loop that can occur during preparing sessions in the session pool. The session pool will try to prepare a session with a read/write transaction when a session is released into the pool. This process follows this logic:
- A session is released into the pool (either as part of the session pool initialization or because a session is returned to the pool by a client).
- If the current fraction of write-prepared sessions in the pool is less than the configured
writeFraction
, a BeginTransaction RPC will be scheduled. - The BeginTransact RPC will fail with a 'Database does not exist.' error. This will cause the session to be released into the pool without a prepared read/write transaction.
- The release of the session into the pool will start the process at step 1 again. As no BeginTransaction call will succeed, the process will continue until the Spanner instance is closed.
from java-spanner.
@olavloite Thanks for fixing this! I cherry-picked googleapis/google-cloud-java@7c66ee4, and it seems to have fixed the issue for me.
from java-spanner.
- The BeginTransact RPC will fail with a 'Database does not exist.' error. This will cause the session to be released into the pool without a prepared read/write transaction.
@skuruppu
An orthogonal question related to this:
What should the client do when a database not found error is returned ?
options:
- execute backend rpcs all customer requests and return NOT_FOUND (for database admin APIs for this specific database that got deleted)
- for read / write APIs - should this invalidate the session pool (since sessions are no longer valid for a deleted db) and handle all subsequent requests with NOT_FOUND errors ?
- this forces the customer to create a new db client after first NOT_FOUND error even though they recreate the database with the same name (but the backend database has obviously changed).
DatabaseId db = DatabaseId.of(project, instance, database);
DatabaseClient dbClient = spanner.getDatabaseClient(db);
- this forces the customer to create a new db client after first NOT_FOUND error even though they recreate the database with the same name (but the backend database has obviously changed).
from java-spanner.
@skuruppu @snehashah16
Regarding what to do when a Database not found
error is returned: We could also allow the read/write APIs handle this in the same way as the database admin APIs do, i.e. keep sending RPCs and let these return NOT_FOUND errors. If the user re-creates the database with the same name, the session pool will automatically heal as Session not found
errors are automatically retried by the client by discarding the old session and creating a new session.
from java-spanner.
My take on this is that since the Java client connection is at a DB level, if the DB is deleted, then this invalidates the current connection. Essentially option 2 that @snehashah16 mentioned so that the customer is expected to create a new client if they recreate a DB with the same name.
I'm a bit weary of sending RPCs continuously until NOT_FOUND errors are no longer returned. In most cases, I suspect once a DB is deleted, users don't recreate the same DB shortly after in a production app. They probably do such things in tests but for that we would expect them to rerun the tests with a new connection.
Let me know if I misunderstood something here.
from java-spanner.
Related Issues (20)
- Library Compatibility with Protobuf-Java 4.26.x+
- spanner.DatabaseClientImplTest: testSessionPoolExhaustedError_containsStackTraces failed HOT 3
- ITEmulatorConcurrentTransactionsTest is flaky with multiplexed sessions HOT 1
- ITReadOnlyTxnTest is flaky with multiplexed sessions
- OpenTelemetry traces show a missing SpanID HOT 1
- OpenTelemetrySpanTest.transactionRunner is flaky with multiplexed sessions
- Fix RESOURCE_EXHAUSTED admin quota limit exceed failures in test pipelines.
- Flaky failure in com.google.cloud.spanner.it.ITDmlReturningTest.dmlReturningWithExecuteQuery[Dialect = POSTGRESQL]
- Flaky test failure in AsyncTransactionManagerTest.asyncTransactionManagerWaitsUntilAsyncUpdateHasFinished with MultiplexedSessions=true
- Flaky failure in ITDmlReturningTest.dmlReturningWithExecuteQuery[Dialect = GOOGLE_STANDARD_SQL]
- spanner.it.slow.ITBackupTest: test02_RetryNonIdempotentRpcsReturningLongRunningOperations failed HOT 1
- spanner.it.slow.ITBackupTest: test03_Delete failed HOT 1
- spanner.SessionPoolOptionsTest: testToBuilder[min sessions = 500, max sessions = 600] failed HOT 1
- spanner.BatchCreateSessionsTest: testSpannerReturnsAllAvailableSessionsAndThenNoSessions failed HOT 1
- spanner.connection.SavepointMockServerTest: testKeepAlive[dialect = GOOGLE_STANDARD_SQL, useVirtualThreads = true] failed HOT 2
- spanner.connection.SavepointMockServerTest: testKeepAlive[dialect = GOOGLE_STANDARD_SQL, useVirtualThreads = false] failed HOT 2
- spanner.connection.SavepointMockServerTest: testKeepAlive[dialect = POSTGRESQL, useVirtualThreads = false] failed HOT 2
- SavepointMockServerTest is flaky
- spanner.SessionPoolOptionsTest: testToBuilder[min sessions = 600, max sessions = 500] failed HOT 1
- BatchCreateSessionsTest is flaky
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from java-spanner.