katanaml / katana-skipper Goto Github PK
View Code? Open in Web Editor NEWSimple and flexible ML workflow engine
Home Page: https://katanaml.io/
License: Apache License 2.0
Simple and flexible ML workflow engine
Home Page: https://katanaml.io/
License: Apache License 2.0
Hi,
I want to build an OCR project on top of your project. Can I know the commit ID of most stable version?
Thanks.
I have a question regarding the availability of microservices created, how can we check if a service is running or not ? So, for each service, we have an event receiver running using: python main.py, what if one service failed to run for some reason, how can we check by code if a certain service is running or not? In case the service is not running we need to show a message saying that this service is not running. Thank you.
Hi
Thank you for the wonderful katana-skipper.
I am trying to digest the library and execute the docker-compose.yml.
But it seems like it is not working.
Would appreciate it if you could take a look
I found that cache the EventProducer can improve performace 40%. I tried but it block may request when increase the speed test. Do you have suggest to fix that
Hello Andrej,
I would like to ask about how to move from docker-compose to Kubernetes, do we have to use some tools like kompose or other tools, I appreciate if you could guide me a little bit about how to perform this conversion to run our services on Skipper not using docker compose but kubernetes. Thank you.
Hello, Can you provide more detail about the logger service and how to get the benefit of it, how to track services in case of any error or failure in some services? And what is the difference between logger and workflow? What I have understood is that both of them are used to log some information related to services, but it is not clear to me what kind of logging info each one can provide. Thank you.
When I run the start command on docker I get the following error in the data-service container. Would greatly appreciate guidance on how to fix this issue.
`
data-service
katanaml/data-service
RUNNING
Traceback (most recent call last):
File "main.py", line 19, in
main()
File "main.py", line 15, in main
'http://127.0.0.1:5001/api/v1/skipper/logger/log_receiver'))
File "/usr/local/lib/python3.7/site-packages/skipper_lib/events/event_receiver.py", line 16, in init
credentials=credentials))
File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 360, in init
self._impl = self._create_connection(parameters, _impl_class)
File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.AMQPConnectionError
Traceback (most recent call last):
File "main.py", line 19, in
main()
File "main.py", line 15, in main
'http://127.0.0.1:5001/api/v1/skipper/logger/log_receiver'))
File "/usr/local/lib/python3.7/site-packages/skipper_lib/events/event_receiver.py", line 16, in init
credentials=credentials))
File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 360, in init
self._impl = self._create_connection(parameters, _impl_class)
File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.ProbableAuthenticationError: ConnectionClosedByBroker: (403) 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfi`
How do we add a new service with a new queue called translator
?
translator
.task_type
and expect a type translator
and a payload
SERVICE=translator
and QUEUE_NAME=skipper_translator
I am able to call the new endpoint and it returns:
task_id: "-",
task_status: "Success",
outcome: "<starlette.responses.JSONResponse object at 0x7ff2672dbed0>"
However the container is never triggered.
What am I missing?
Hi there, I am developing a microservice that takes more than 60 seconds to process some data with AI, sadly this causes the rabbitmq server to disconnect my client for not acking the message within 60 seconds.
how I can modify this timeout setting?
I've tried the rabbitmq docker-compose yml to set env variable:
but still the connection setting have the timeout at 60 seconds.
How we were able to get around this since this is aimed for model processing which generally takes way more than 60 seconds?
Other way would be configure the pika connector and set the heartbeat setting manually, but the library doesn't allow it by default.
I would like to know how everyone get around this.
Thanks!
rabbitmq-service | 2023-11-21 13:45:42.224 [error] <0.19553.0> closing AMQP connection <0.19553.0> (172.27.0.6:53454 -> 172.27.0.4:5672):
rabbitmq-service | missed heartbeats from client, timeout: 60s
rabbitmq-service | 2023-11-21 13:45:42.225 [info] <0.19791.0> Closing all channels from connection '172.27.0.6:53454 -> 172.27.0.4:5672' because it has been closed
Hello,
Thanks for sharing your ML workflow. I appreciate if you could explain the difference between event_producer and exchange_producer. event_producer is used to produce an event to rabbitmq, but exchange_producer is not clear to me. Can't we use event_producer in place of exchange_producer?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.