async-aws / aws

AWS SDK with readable code and async responses

AsyncAws client

If you are one of those people that like the Amazon PHP SDK but hate the fact that you need to download Guzzle, PSR-7 and every AWS API client to use it?

This is the library for you!

See full documentation on https://async-aws.com.

Packages overview

Package name	Badges	BC check	Unreleased
async-aws/core
async-aws/app-sync
async-aws/athena
async-aws/cloud-formation
async-aws/cloud-front
async-aws/cloud-watch
async-aws/cloud-watch-logs
async-aws/code-build
async-aws/code-commit
async-aws/code-deploy
async-aws/cognito-identity-provider
async-aws/comprehend
async-aws/dynamo-db
async-aws/ecr
async-aws/elasti-cache
async-aws/event-bridge
async-aws/firehose
async-aws/iam
async-aws/iot
async-aws/iot-data
async-aws/kinesis
async-aws/kms
async-aws/lambda
async-aws/location-service
async-aws/media-convert
async-aws/rds-data-service
async-aws/rekognition
async-aws/route53
async-aws/s3
async-aws/scheduler
async-aws/secrets-manager
async-aws/ses
async-aws/sns
async-aws/sqs
async-aws/ssm
async-aws/sso
async-aws/step-functions
async-aws/timestream-query
async-aws/timestream-write
async-aws/translate
async-aws/x-ray
Integrations
async-aws/async-aws-bundle
async-aws/dynamo-db-session
async-aws/illuminate-cache
async-aws/illuminate-queue
async-aws/monolog-cloud-watch
async-aws/simple-s3

The main repository is not tagged and cannot be installed as a composer package.

aws's Issues

Document pagination feature

Related to #97

Update changelog and release core 0.2

That allows us to test other clients more easily.

Assert file is valid when dumping code

Notging prevent dumping an invalid file which leads to uncompilable-file.

This could be avoided by linting the code (php -l) after dumping it, and reverting the file in case of issue.

Implement all "incomplete tests"

There are test stubs that needs some love. Basically:

1. Add some Input data and verify the HTTP request
2. Add some HTTP response data and very Result

• CloudFormation
• S3
• Ses
• Sns
• Sts
• Sqs
• Lambda

All input object should be able to be constructed without arguments

Ie PutObjectRequest.

Dont throw exception twice

If an exception is thrown on ->resolve(), it can be caught. But then on __destruct(), we will call resolve again and a new message is thrown.

Implement AssumeRoleWithWebIdentityCredentialProvider

see https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-sts-2011-06-15.html#assumerolewithwebidentity

Add CI that runs test with lowest deps

Add link to the documentation in the operation docblock

Future idea: Automatic releases

We could do automatic minor releases every Wednesday if all of this conditions are true:

• The only changes made are from the github action bot
• The BC check is green
• The CI check is green
• If it is a AWS autogenerated service, (sqs, s3, etc)

Questions:

• How to write with change logs?
• Should it be minor releases?

Handle Enum

The api2-json file provides hints on enum. (ie. the Runtime Type in in Lambda)

We should:

• provide const for the values
• take those value in "validators"

class ListLayerVersionsRequest
{
    /**
     * A runtime identifier. For example, `go1.x`.
     *
     * @var string|null
     */
    private $CompatibleRuntime;

    public function validate(): void
    {
        foreach ($this->CompatibleRuntime as $value) {
            if (!isset(Runtime::AVAILABLE_RUNTIMES[$value])) {
                throw new InvalidArgument(sprintf('The CompatibleRuntime "%s" in "%s" is not a Valide "Runtime". Available values are "%"', $value, __CLASS__, array_keys(Runtime::AVAILABLE_RUNTIMES)));
            }
        }
    }
}

// Class is named "Runtime" because of the name of the shape
class Runtime 
{
    public const NODEJS = 'nodejs';
    public const NODEJS_4_3 = 'nodejs4.3';
    public const NODEJS_6_10 = 'nodejs6.10';
    public const NODEJS_4_3_EDGE = 'nodejs4.3-edge';
    // ..

    // = "AVAILABLE_" + className.upper()
    public const AVAILABLE_RUNTIME = [
        self::NODEJS = true,
        self::NODEJS_4_3 = true,
        self::NODEJS_6_10 = true,
        self::NODEJS_4_3_EDGE = true,
        // ..
    ];
}

Start using changelogs

Should we do it the symfony way? Ie both changelog in root and individual packages?

Alternatively we can just put changlog on the packages.

Use official SDK to build Objects

The official SDK contains JSON file describing every method/request/response

ie. https://github.com/aws/aws-sdk-php/blob/efdbdde0968355917470a255c0d2e509c49f7476/src/data/sqs/2012-11-05/api-2.json#L214-L229

To provide a library compatible with official Documentation, I suggest to use those files to generates our DTO/Objects.

The content of simple-aws is:

• low level logic to perform request, deal with response, authentication, pagination
• a developper tool to generate the Request/Response Functions based on SDK JSON files (similar to https://github.com/thecodingmachine/safe)

Optionnaly it will also provides:

• helper classes that provides a non-official but more friendly contract ($s3->putFile($path)) (ie like psr/cache vs psr/simple-cache)

What do you think?

Look for datatype "map"

We look for "structure" or "list" on many places, but we fail to look for "map".

We should always consider these 3 types of combined types.

Use enums in doc blocks

We should simplify doc blocks by doing something like:

/**
 * @param FooEnum::* $value
 */

See https://psalm.dev/docs/annotating_code/typing_in_psalm/

---

This will require us to replace our FooEnum::AVAILABLE_... with a static function. Ie FooEnum::exists(string $value): bool

Do not convert stream body to String

see https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

[S3] How to create a directory

The following code:

require_once __DIR__.'/vendor/autoload.php';

$s3 = new \AsyncAws\S3\S3Client([
    'region' => 'eu-central-1',
]);
$bucket = 'async-aws-test';

$result = $s3->putObject([
    'Bucket' => $bucket,
    'Key' => 'foo/',
]);

$result->resolve();

Will give:

Uncaught AsyncAws\Core\Exception\Http\ClientException: HTTP/1.1 411 Length Required returned for "https://s3.eu-central-1.amazonaws.com/async-aws-test/foo/".

Use examples-1.json from SDK to generate tests

Periodically check the AWS Api contract

Add a GithubAction that dry-run the generation based on the latest available version, and trigger an alert/or create a PR when something changes.

That should fixes #257 (comment)

Configuration Fallback misbehaving

I have a Bref based lambda function processing files stored in an s3 bucket on object creation events. Files are accessed via Flysystem v2.

Composer require:

"bref/bref": "^0.5.16",
"async-aws/flysystem-s3": "^0.2.0",

When creating the S3Client I'm using these params:

$client = new S3Client([
    'accessKeyId' => $config['username'],
    'accessKeySecret' => $config['password'],
    'region' => $config['region'],
]);

When trying to list files I'm getting this error

Code: InvalidToken
Message: The provided token is malformed or otherwise invalid.

From what I see in async-aws/core Configuration.php falls back to all env vars if an option is not specified. The Lambda has env vars for

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN

but I'm using a different user for accessing the s3 bucket when creating the client. I think when doing so the Configuration should not use the sessionToken from env vars.

Support for streamable input

It is just a minor thing. We should remove return type hint on parameters like PutObjectRequest::getBody().

We should also update the constructor doc block.

The correct signature is array|string|resource|\Traversable|\Closure

PR created nightly don't triggers check

more information: peter-evans/create-pull-request#48 (comment)

Remove Symfonys Http request from result object

We should remove it after the result is initialised.

This will make the Result object more pretty but we will possibly loose debugging info after resolving a successful request.

Throw exception if not all require parameters exists in a request object

This should not be valid:

$input = new \AsyncAws\S3\Input\GetObjectRequest();
$input->setBucket('bref-example');
$result = $s3->getObject($input);

Currently it will be a "listObject", which is way wrong..

Always type hint with DateTimeInterface

HeadObjectOutput::LastModified should be a \DateTimeImmutable

Like everything else marked as a "timestamp"

Handle "Metadata"

On the S3 HeadObjectOutput there is a parameter "Metadata" that is never set.

I think, that all parameters on the object is the metadata..

Psalm issue: IniFileProvider $homedir

I added it to the baseline, but it should be fixed.

There is a FIXME in the code in IniFileProvider::loadProfiles().

See #53 (comment)

[S3] PutObjectAcl should not have a body when using canned ACL

Looking at these examples: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html#API_PutObjectAcl_Examples

If one is using the x-amz-acl header, then the request body should be empty. Currently we get an error because the body contains:

<?xml version="1.0" encoding="UTF-8"?>

Should Result::$client be private?

The AsyncAws\Core\Result has some private properties and some protected. Should we be consistent with these?

Should we be defensive and do all private?

Or are we okey with using protected?

Add interface on StreamableBody

Restructure file hierarchy

├── CHANGELOG.md
├── composer.json
├── docs
│   ├── authentication.md
│   ├── ...
│   └── waiter.md
└── src
    ├── CodeGenerator
    ├── Core
    ├── Service
    │   ├── CloudFormation
    │   ├── Lambda
    │   ├── S3
    │   ├── Ses
    │   ├── Sns
    │   └── Sqs
    └── Integration
        ├── Flysystem
        |   └── S3
        └── Symfony
            ├── SqsTransport
            └── Bundle

Handle pagination

Find a solution for steaming responses

Currently, the only way to stream a response is to:

$result = ...;
$info = $result->getInfo();
foreach($httpClient->stream($info['response']) as $chunk) {
  // ...
}

Do we want to provide a better way?

How to test PRs?

If someone submit a PR with a patch for a bug, how can that be tested before the PR is merged?

T

[S3] Key prefixes

Should we treat these the same?

'Key' => 'foo.png',

'Key' => '/foo.png',

Currently we expect no leading slash. The problem occurs when we copy objects:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_Examples

That assumes we do one with leading slash and one without:

        $input = new CopyObjectRequest([
            'Key' => 'my-second-image.jpg',
            'Bucket' => 'my-bucket',
            'CopySource' => '/bucket/my-image.jpg',
        ]);

Update PHP extensions in composer.json

Each client is different. Some need ext-SimpleXML and some need ext-json. The composer.json should be updated accordingly.

Allow regenerate all services

The command:
./generate update —all

Should update all operations for all services.

`sendMessage` method not found when mocking `SqsClient`

When trying to mock SqsClient from the official AWS SDK i get the following error:

Trying to configure method "sendMessage" which cannot be configured because it does not exist, has not been specified, is final, or is static

because the method does not belong class but is somehow added to it. Please keep this in mind when releasing the stable version and dont make the same structure as the official SDK.

provide a VarExporter Caster

provide a VarExporter Caster to created readable dumps. (without request, client, ...)

[Core] Release 0.3.0

This I think core is pretty much feature complete.

Sign request when body is resource

Related PRs that might be helpful: #125 and #126

Create a Symfony Bundle

provides:

• semantic configuration (covers client's Configuration options)
• autowiring

AsyncAws, Async-Aws or Async Aws

How do we refer to the name of the organization?

I created #161 to make sure we are consistent.

What do you think is best?

Package ideas

We could write integrations to

• Symfony SES Mailer: symfony/symfony#35992
• Symfony SQS Messenger: symfony/symfony#36094
• Symfony SSM EnvLoader
• Symfony SNS Messenger
• Symfony SNS Notifier
• Monolog Cloudwatch handler
• BackupManager S3: backup-manager/backup-manager#168
• PHP-Enqueue SQS adapter: php-enqueue/enqueue-dev#1046
• Flysystem S3 adapter
• DynamoDb session handler: #569

I just list a few of them here. I dont say we should focus on any of them.

Rest Input/Result test are not just about body

The automated test generate stubs for Body, but in RestJson and RestXml, Uri and Headers are important too.

see for instance https://github.com/async-aws/aws/blob/master/src/Service/Lambda/tests/Unit/Input/AddLayerVersionPermissionRequestTest.php

Generated tests are put in the wrong folder

ie. tests are put in
src/Service/S3/src/Tests/Unit/Input/GetObjectAclRequestTest.php
instead of
src/Service/S3/tests/Unit/Input/GetObjectAclRequestTest.php

[Automatic PR] What if PR is not merged

What if:

Day 0: Update in official SDK
Day 1: Bot creates a PR with updates
Day 2: We still dont merge that PR
Day 3: Update in official SDK
Day 4: ??

Will the PR be updated?
Will a new PR be created?

What if we merge the PR but not delete the branch?

Hide "promise/async" complexity

I like the way symfony/http-client handle async calls, by processing the call when needed and hiding the complexity to the user.

as a user, I'd love to use something like:

$bucket = '...';
$client = (new AwsClient())->s3();
$objects = $client->listObjects(['bucket' => $bucket]);  // async 

// do something else...

foreach ($objects as $object) { // listObjects resolved here
  $client->deleteObject(['bucket' => $bucket, 'key' => $objet->getKey()]); // async
}

// deletions resolved at destruct

For DX, and IDE autocompletion, I think that each method should return an Dedicated Object with Getter TypeHinted.

$bucket = '...';
$key = '...';

$client = (new AwsClient())->s3();

$object = $client->getObject(['bucket' => $bucket, 'key' => $key']); // async
$object->getContentLength(); // resolved here