GithubHelp home page GithubHelp logo

jhart-r7 / awsretry Goto Github PK

View Code? Open in Web Editor NEW

This project forked from linuxdynasty/awsretry

1.0 2.0 0.0 14 KB

Decorate your AWS Boto3 Calls with AWSRetry.backoff(). This will allows your calls to get around the AWS Eventual Consistency Errors.

Python 100.00%

awsretry's Introduction

AWSRetry - Boto3 Retry/Backoff Decorator

AWSRetry is a Python Decorator that can be used to wrap boto3 function calls. This function was built out of the need to get around a couple of common issues when working with AWS API's.

  • Query API Request Rate
  • Eventual Consistency Model.

Exceptions that will get retried when encountered

  • RequestLimitExceeded
  • Unavailable
  • ServiceUnavailable
  • InternalFailure
  • InternalError
  • ^w+.NotFound

This list can be extended. (http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html)

Quick Start

Install awsretry.

$ pip install awsretry

I will assume you know about setting up Boto3 Credentials, if not you can read the instructions here http://boto3.readthedocs.io/en/latest/guide/configuration.html

Keyword Arguments that AWSRetry.backoff accepts

  • tries = The number of times to try before giving up. Default = 10
  • delay = The initial delay between retries in seconds. Default = 3
  • backoff = backoff multiplier e.g. value of 2 will double the delay each retry. Default = 1.1
  • added_exceptions = Other exceptions to retry on, beyond the defaults. Default = list()

Examples

Write a quick function that implements AWSRetry.backoff()

#!/usr/bin/env python

import botocore
import boto3
from awsretry import AWSRetry


@AWSRetry.backoff()
def get_instances():
    client = boto3.client('ec2')
    try:
        instances = client.describe_instances()
        return instances
    except botocore.exceptions.ClientError as e:
        raise e:

instances = get_instances()

Write a quick function that will overwrite the default arguments.

#!/usr/bin/env python

import botocore
import boto3
from awsretry import AWSRetry


@AWSRetry.backoff(tries=20, delay=2, backoff=1.5, added_exceptions=['ConcurrentTagAccess'])
def create_tags():
    client = boto3.client('ec2')
    try:
        resources = ['1-12345678891234']
        tags = [{'Key': 'service'}, {'Value': 'web-app'}]
        instances = client.create_tags(Resources=resources)
    except botocore.exceptions.ClientError as e:
        raise e:

create_tags()

Development

Assuming that you have Python and virtualenv installed, set up your environment and install the required dependencies like this instead of the pip install awsretry defined above:

$ git clone https://github.com/linuxdynasty/awsretry.git
$ cd awsretry
$ virtualenv venv
...
$ . venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .

Running Tests

You can run the tests by using tox which implements nosetest or run them directly using nosetest.

$ tox
$ tox tests/test_awsretry.py
$ tox -e py27,py36 tests/
$ nosetest

awsretry's People

Contributors

epoelke avatar linuxdynasty avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.