Comments (8)
Got it -- thanks @kashifkhan ! That makes sense.
from azure-sdk-for-python.
Thank you for your feedback. Tagging and routing to the team member best able to assist.
from azure-sdk-for-python.
Hi @cabal-daniel,
our typical usage pattern is to use the context manager ( with ) or simply managing the opening / closing of the client by yourself, so the code ends up like this.
It shouldn't impact anything but you will have sockets left open, other pieces of code that might be running. In general it is a good idea to call close just in case.
from azure.identity import DefaultAzureCredential
from azure.servicebus import ServiceBusClient, ServiceBusMessage
from datetime import datetime, timedelta
client = ServiceBusClient(
fully_qualified_namespace='foobar',
credential=DefaultAzureCredential(),
)
start = datetime.now()
sender_context = client.get_queue_sender(queue_name='hello')
checkpoint_1 = datetime.now()
sender.send_messages(ServiceBusMessage('hi')) # this will open up the connection if not open
checkpoint_2 = datetime.now()
sender = sender_context.close()
checkpoint_3 = datetime.now()
from azure-sdk-for-python.
Hi @cabal-daniel. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.
from azure-sdk-for-python.
@kashifkhan the problem is that opening the connection and closing the connection takes a considerable amount of time
initiating took 1255.869ms
sending took 77.747ms
closing took 804.139ms
For our pods, it seems easier to just keep the connection open and detect if the connection is closed on the other end then re-open so sending messages takes 77ms rather than 3000ms if we're using SB to build a DAG.
from azure-sdk-for-python.
@cabal-daniel thank you for that context. There are some things to note here
- Would you be able to try using the exact credential for your environment.
DefaultAzureCredential
can add some overhead as it tries to determine which credential to use. That itself should save you some time on connects - An AMQP connection is pretty expensive, so a first connection will always take time. After that it should be quite speedy. The clients are designed to open a connection when a desired operation is called. In the example above, the first
send_messages
will take some time as it has to open a connection, while subsequent calls will be fast as the connection is already open. - Calling
__exit__
is the same assender.close()
, so there is no short circuit here. - There is also a keep alive running in the background and if the service drops the connection due to timeouts, the client will reconnect and be ready.
- If the socket dies or something in the client goes wrong, unless it's something the client doesnt handle then it has to be re-initiated.
from azure-sdk-for-python.
@kashifkhan thank you for this helpful information.
Could you comment on the following type of architecture for distributed compute?
- Start worker
- Open all the queue senders / AMPQ connections
- Worker does it's job for a long time
- SIGTERM
- Use the
atexit
library to close all the connections
Between steps 2 to 5, the workers could be alive for days or weeks.
from azure-sdk-for-python.
In general the strategy looks fine, just wanted to add a few notes:
- Simply calling
get_queue_sender
/get_topic_sender
wont open the connection. An operation such assend_messages
will do that and then subsequent calls will be faster as they will benefit from an open connection. - Switching from
DefaultAzureCredential
to the exact credential you are looking for such asManagedIdentityCredential
etc will speed up first connection time. - As long as there is a consistent flow of messages the clients will work without having to do anything special. Keep alive running is also key
- The clients are designed to run for a long time. It will retry a lot of failures etc. on your behalf. If something goes wrong that it can't handle, an outside process goes wrong, errors that are not retryable etc. etc. then they have to recreated or that scenario has to be handled per the application logic.
- Calling close on the senders/receivers will also close down the connection for you
from azure-sdk-for-python.
Related Issues (20)
- can't find back the documentation of the azure functions decorator package HOT 1
- Copying blob within account with BlobClient.upload_blob_from_url fails with ErrorCode:CannotVerifyCopySource HOT 3
- add the possibility to track metrics every few iterations/periods of time HOT 4
- Azure AI Search - Support for scalar quantization seems incomplete HOT 4
- ManagedIdentityCredential failing with version 1.17.0 HOT 7
- Documentation of the previous version of Python SDK is not visible/accessible for HOT 4
- Get service bus sender client by name regardless of type (queue/topic) HOT 1
- Effective Identity with Service Principal in Python HOT 6
- ValueError: Management Operation error occurred. When a task exceeds 240s with 5min lock out period HOT 14
- AzureAppConfigurationProvider should be able to lazy load secrets HOT 6
- azure-communication-callautomation needs linting updates for pylint version 3.1.0 HOT 1
- azure-ai-documentintelligence AIO capability is not working HOT 6
- Is there no way to stream or wait for environment creation with ml_client HOT 3
- [bct] Add changelog reporting information in the README
- [bct] Rename tool
- SyntaxWarning: invalid escape sequence - azure-mgmt-network HOT 2
- yntaxWarning: invalid escape sequence - azure-mgmt-keyvault HOT 7
- Using Azure SDK for Python to run Workflows in Azure Purview HOT 2
- Missing websocket-client dependency for azure-servicebus HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azure-sdk-for-python.