yoomoney / db-queue Goto Github PK

Could the QueueConsumer interface be updated to automatically TaskExecutionResult.fail() on an uncaught exception? The pattern of

try {
   // Process
} catch (e: Exception) {
  // log exception
  // return TaskExecutionResult.fail()
}

isn't compatible with top-level Spring exception handlers and leads to the catch and log pattern.

Hi,

What do you think about dynamic configuration any dbqueue params? It would be very useful for example when you increase thread count on during high load system and another decrease when it is not necessary without restart application. One more example when you set zero betweenTasksTimeout to empty the queue and when queue is empty return old params in back.

Thanks,
Leventsov Ivan

Is the same row in queue table picked twice or more for task processing if processing mode is USE EXTERNAL EXECUTOR and task takes time to executes?
if so how to avoid it?

Even though I wanted to work on introducing priorities within one worker thread as we discussed in #1, the support for MSSQL got higher priority for us. So I focused on implementing mssql driver for db-queue.

So far, I have managed to implement the main classes: MssqlQueueDao, MssqlQueuePickTaskDao, and accompanying test classes. However, I want to clear out a couple of things before I make the PR:

• are you willing to accept a PR to support MSSQL directly in db-queue?
• a: if yes, how should we manage the databases in tests? right now, embedded postgres is used. However, that can't be the case for mssql. Should we assume that both of them will be executed externally? (via docker in case of travis-ci).
• b: if no, because of the way that DatabaseDialect is implemented and used, users of library can not add support for other databases. I think we need to have an implementation registry of sort so dialects can be added dynamically.

Hi,

I was looking into the project and was wondering is there a way to have multiple consumers/subscribers for single queue and single task?

I have in mind the case that I'm registering the task and have registered for the given queue 2 consumers in the same app and both receive/process enqueued task.

Thanks

There is an insufficient number of connections in docker image for oracle. We were able to increase connections and sessions by rebuilding the docker image. This image is only available in internal storage.

Hey there,

Despite looking into the code samples and available documentation I wasn't able to find a way to configure the max number of retries for a task. Is there functionality like this available?

Thanks in advance

Is there any reason why QueueLoop tries to catch Throwable that includes critical Errors such as OOM? It just eats them.

Hi there,

I was looking into these two variables in https://github.com/yandex-money-tech/db-queue/blob/c1c16397be1ea83ceb4772c34ba5cf52ff7bdfca/src/main/java/ru/yandex/money/common/dbqueue/settings/QueueSettings.java#L27-L30 but from what I understand, they're not actually used anywhere in the code, i.e. the loop policy doesn't care about these values.

I believe you must have had ideas to use them somehow that you introduced them in the code. I might be able to work on this if you share those ideas.

Hi, sorry for opening an issue but I didn't see another way to contact the maintainers.

I would like to use this lib as part of a larger monolith that can also operate in clustered mode.

If that is the case, can the library be configured to operate consistently across all nodes of the cluster
so that each node can both insert jobs and work on jobs without conflict?

Every node of the cluster is hooked up to the same SQL db.

Would you be interested in including priority in tasks (JMS also has this)? I have created a preliminary branch for this that seems to do what I need: https://github.com/kennywest/db-queue/tree/add_priority

Thanks for considering.

I'm having issues with understanding how to use this library in a "spring way" with @configuration... I've found a nice example on maven for "db-queue-scheduler". Do you also have one for "db-queue" ?

How to avoid duplicate creation of job based on "payload"? In other words, the requirement is if there is already a task running with same "payload" , it should not be scheduled/accepted in queue.

Hi,

I'd like to use this package but my environment uses a different DB access technology, without Spring.

Could you please move the DAOs to a different, Spring-specific DAO JAR? That way, I could write my own implementation of the DAO interface, without any dependency on Spring. This would also require that the factory class is not part of the DAO interface any more.

Cheers
Matthias

Currently only an out of the box implementation is provided for Spring. A plain JDBC alternative would be great for non-Spring projects.

To avoid duplication the DB-specific logic should probably be moved to the core and only the different DB access implementations should in separate modules.