mirkokiefer / aws-lib Goto Github PK

We should test at least one API Method per client.

In aws.js, there is a call function that actually send http/https request.
Sometimes, the connection itself returns error. In this case, because there is no handler on connection object, node.js program dies without any chance to restore.

I think that we should handle connection error, like

var req = obj.connection.request( ....... //create request
});
req.on('error', function(err) { <process error and call callback function } ); //ADDED: add error handler
req.write(body); //write body

I'm setting the "region" option to "DE" like this

var amz = aws.createProdAdvClient(/*...*/);

amz.call('ItemLookup', {ItemId: 'foobar', host: 'ecs.amazonaws.de', region: 'DE'}, function(result) {
    console.log(JSON.stringify(result, null, 4));
});

and I expect the request being sent to Amazon to look like this

ItemId=foobar&host=ecs.amazonaws.de&Region=DE

but what actually happens is

ItemId=foobar&host=ecs.amazonaws.de&region=DE&Region=US

I'm already investigating where the key gets lowercased, but maybe someone can help. It's very confusing, because in prodAdv.js it is still uppercased.

query["Region"] = obj.region

The coffeescript below results in a steady increase in used memory. Screenshot of debugger profile: http://cl.ly/2f1t3i3e1Q3w0q0b0T3P

#sanity check -- is aws-lib a big memory leak?
require "v8-profiler"

aws = require "./aws-lib"

awsOptions = 
  host: "queue.amazonaws.com"
  path: "/181376977363/test"
  secure: true
awsKey = "..."
awsSecret = "..."


sendReceiveDelete = ->
  sqs = aws.createSQSClient awsKey,awsSecret,awsOptions
  msg = "My lucky number is #{Math.floor(Math.random()*100000000)}."
  sqs.call "SendMessage",{ MessageBody: msg }, (result) ->
    console.log "SEND: #{JSON.stringify result}"
    sqs.call "ReceiveMessage", {}, (result) ->
      console.log "RECEIVE: #{JSON.stringify result}"
      if result.ReceiveMessageResult
        receivedMessage = result.ReceiveMessageResult.Message
        if receivedMessage
          sqs.call "DeleteMessage", receivedMessage, (result) ->
            console.log "DELETE: #{JSON.stringify result}"
      process.nextTick sendReceiveDelete

sendReceiveDelete()

I get the following error message:

"SignatureDoesNotMatch","Message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

I already check my Access Key and Secret Key twice. Still the same error. Any idea?

I'm trying to access the Product Advertising API for amazon.de (German), but so far all the library returns are the amazon.com (USA) results. My best guess so far was based on this code in prodAdv.js.

So what I do is...

var prod = aws.createProdAdvClient(accessKey, secretAccessKey, associateTag, {"region":"DE"});

but that does not seem to work. Any help?

Hi,
I'm received from amazon message that error: "The Receipt Handle is invalid".

sqs = aws.createSQSClient(awsKey, awsPrivateKey, options);
var message = '';
sqs.call("ReceiveMessage", outbound, function (result, m) {
message = m.ReceiveMessageResult.Message;
DeleteMessage(message);
});

function DeleteMessage(message) {
var outboundDel = {
ReceiptHandle: JSON.stringify(message.ReceiptHandle)
};
sqs = aws.createSQSClient(awsKey, awsPrivateKey, options);
sqs.call("DeleteMessage", outboundDel, function (result, a) {
// ERROR RECEIVED BY AMAZON "The Receipt Handle is invalid".
});
}

What's my error?

Thank's for your help! =)

The "keywords" member contains an array of single-quoted strings. JSON requires double quotes.

aws.js, line 36 should be

   "Content-Type": "application/x-www-form-urlencoded; charset=utf-8",

to process non-ascii characters properly.
refer this article: http://buyitnw.appspot.com/developer.amazonwebservices.com/connect/thread.jspa?messageID=220215

When an exclamation point is in my SimpleDB expressions I get:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

This is easily fixed by adding the following line near the bottom of aws.js (with the others):

stringToSign = stringToSign.replace(/!/g,"%21");

Does Amazon specify exactly which characters need to be encoded somewhere? Or are they following a standard? It would be nice to know that no other characters will break the signatures.

I am working on a script to manage my aws instances. The first part looks like this ...

http        = require 'http'
aws         = require "aws-lib"

srvr = http.createServer (req, res) -> 

    req.ec2 = aws.createEC2Client 'AKI...ZOQ', 'lgHJcg+...HYz2',
        host: 'ec2.us-west-1.amazonaws.com'

    req.ec2.call "DescribeInstances", {}, (resp) ->
        console.log 'DescribeInstances', resp

srvr.listen 80

This was working the first few hours while I was debugging the rest of my code. Now it started giving me this error ...

{ Errors:  { Error:  { Code: 'AuthFailure', Message: 'AWS was not able to validate the provided access credentials' } } }

Using the command line ec2-describe-instances still works with the same security keys. Has anyone seen a problem like this?

I have a question about security. How is it that aws-lib only requires the two keys, while the aws api command line requires those plus the certificate etc.?

As you can tell I'm new to using the aws api. Any help would be appreciated.

It would be great to see an example using SNS.

I have this:
function createClient() {
return aws.createSQSClient(config.awskeyid, config.secretkey, {secure:false});
}
function sendMessage(message, callback) {
var sqs = createClient();
sqs.call("SendMessage", {MessageBody:message }, callback);
}

but I got this error:
"Error":{"Type":"Sender","Code":"InvalidAction","Message":"The action SendMessage is not valid for this endpoint.","Detail":{}

so, what's wrong? and how should I select the queue I want to send message to? Thanks. :)

ses.call('SendEmail', send_args, function(err, result) {
console.log(result);
});

Consider above code, when SendEmail request is sent to aws, aws either responds with error or success. But aws-lib does not report error as err, it just returns result back.

Right now everyone i thik parse the result to figure out what happened. But correct way to find result type is using http response code.

We need to use http error code, if code is >= 400 or <600, it should return it as err.

Link to api for easy access
http://docs.amazonwebservices.com/ses/latest/APIReference/API_SendRawEmail.html

Hi,
We had experienced hanging sockets to the Amazon servers when using the Product Advertising API. This led to our application being deadlocked as soon as 5 sockets were hanging (5 is the default number of sockets for node's https agent).

When using lsof you could see it like this:

node      20371       user   11u     IPv4         abc        0t0        TCP xxxx:49297->72.21.215.235:https (ESTABLISHED)
node      20371       user   12u     IPv4         cde        0t0        TCP xxxx:45374->72.21.215.235:https (ESTABLISHED)
...

Using a fork with the casualsteps#1 pull request and setting the timeout to 10 seconds fixed the issue for us.

Would be nice to see this in the main codebase.

CC @kenleytomlin @ogoffart @jturcotte

I can't seem to pull back customer reviews. Everything else that I request comes back, but customer reviews are simply missing. Here's what I'm doing:

var client = aws.createProdAdvClient(key, secret, tag);

var parameters =
{
    SearchIndex : "Beauty",
    BrowseNode: "11058281",
    ResponseGroup: "Small,ItemAttributes,Images,Reviews,EditorialReview,OfferSummary"
};

client.call("ItemSearch", parameters, function(err, result)
{
    // output result
});

Each item has an ItemAttribute, EditorialReviews, etc., but CustomerReviews is simply missing. I've run the same query (same ResponseGroup parameters, BrowseNode, AWS credentials, etc.), using a .Net AWS library, and it pulls the customer reviews, but I can't seem to get it to work with aws-lib.

What might I be missing?

Please add support to the CloudWatch APIs

When running in AWS with an instance role, setting accessKeyId and secretAccessKey to null does not result in the key, secret, and token being read correctly from the Instance Metadata service.

Or rather these values are being obtained, but they are being assigned to the variables that are then not referenced.

Here in lib/aws.js you are setting securityToken, accessKeyId, and secretAccessKey prior to the values in obj being reset with the values obtained from Instance Metadata.

  var securityToken = obj.token;
  var signHeader = obj.signHeader;
  var host = obj.host;
  var accessKeyId = obj.accessKeyId;
  var path = obj.path;
  var agent = obj.agent;
  var secretAccessKey = obj.secretAccessKey;
  var secure = obj.secure == null ? true : false;
  var connection = secure ? https : http;

Then when you reference the scope variables later they are still null even though you have updated obj.

See PR: https://github.com/livelycode/aws-lib/pull/84

Would be great if someone could write a few tests for this so that I can merge it:
https://github.com/livelycode/aws-lib/pull/53

Basically, I have a Queue set up where SendMessage has it's principal set to "Everybody (*)". However, when I pass null for the accessKeyId & secretAccessKey I get the following error:

secretAccessKey and accessKeyId must be set

If I set the accessKeyId & secretAccessKey to a blank string the result is null. Actually, no matter what I do I get a null result. I've set the accessKeyId & secretAccessKey to be blank, incorrect, and correct and I get a null result. The only time that the message actually shows up in the queue is when the accessKeyId & secretAccessKey are set to my AWS info.

Is there any way to call SendMessage without credentials using your library? FYI, I can successfully do it when I manually call SendMessage via a https.request call.

I've not changed any of my code, so maybe Amazon has changed something, anyway now I get the error code SignatureDoesNotMatch whenever I make a request via createProdAdvClient. :/

Not sure if this library is maintained anymore, judging by the lack of response to other issues.

Is anyone alive out there? :(

Amazon has an SDK available for Node and the browser here: https://github.com/aws/aws-sdk-js

How does this library compare to the official repo? What benefits does it provide in comparison? I'm asking out of curiosity.

Thanks!

@mirkok can you publish the latest version of the lib in npm?

Hi guys,
My apology if i am posting this at the wrong place.
I am trying to move content from IIS to s3 bucket. I have already setup Nginx to proxy-pass to S3 bucket with my AWS Secret Keys and AWS Access Keys. My configuration is as follow:

server {
listen 80;
server_name localhost;

    #charset koi8-r;

    #access_log  logs/host.access.log  main;

           location /customerfolder1 {


            proxy_pass http://mybucketname.s3.amazonaws.com/customerfolder1/;
            aws_access_key **********;
            aws_secret_key **********/***********;
            s3_bucket mybucketname;

            proxy_set_header Authorization $s3_auth_token;
            proxy_set_header x-amz-date $aws_date;
            proxy_intercept_errors on;

 }
    location /customerfolder2 {


            proxy_pass http://mybucketname.s3.amazonaws.com/customerfolder2/;
            aws_access_key *********;
            aws_secret_key **********/*********;
            s3_bucket mybucketname;

            proxy_set_header Authorization $s3_auth_token;
            proxy_set_header x-amz-date $aws_date;
    }

My problem is this:
... each time i try accessing content in s3 from a browser, i usually get the error below:

However, when i take out the trailing slash and the 'CustomerFolder' , i don't get this error.

i.e if it is; proxy_pass http://mybucketname.s3.amazonaws.com;

Please some one help me. I have been trying for mouths to figure out what the problem was without any breakthrough.

Hello,

I am not able to authenticate to eu-central-1, but it works on eu-west-1

"AWS was not able to validate the provided access credentials"

I saw elsewhere threads about similar problem but i don't know if it's the same with this lib ?

Thank you

Amazon's search by ItemLookup cannot be called with prodAdv function

Hey there,

Great work on the library so far! I'd like to use it, but it makes me nervous that the callbacks don't use the standard Node callback convention of (err, result), instead doing (result). This worries me because I have no way of catching asynchronous errors now. Would you consider updating these and properly "throwing" async errors by passing them to the callback?

On a more practical level, I'm a big fan of Streamline.js, which lets me write async calls as if they were synchronous, while the code still compiles to be fully async. This, and other tools, rely on the callbacks being in the standard Node callback convention, too.

Thanks in advance! And let me know if you have any questions or if I can help in any way.

I would like to retrieve 100 results for a keyword search with Amazon's Product Advertising API. By default the number of results is limited to 10 and I understand that paging is required to get more. There is a MoreSearchResultsUrl field provided in the response. How can I use that?

curl comes back empty for both GET and POST, my webbrowser shows a website with results. If I get the right results, I assume it should be XML which I somehow need to run through aws-lib to get the regular JSON results.

I think it's really strange that Amazon itself refers to this 4 year old library and that Amazon's JS SDK does not support the Product Advertising API.

Hi,

Using this code:

iam.call("ListUsers", {}, function(err, result) {
    if (result["Error"]) {
            console.log(JSON.stringify(result.ListUsersResult));

I can get a list of IAM users, however I need to also retrieve the secret key!

Is this possible using your library?

Best Regards.

I'm not sure if this is possible, but it would be great if the library dependency on Node.JS could be removed so that the library could be used from javascript environments other than Node.JS.

Hi,

I just followed your example code like this:

var api = aws.createProdAdvClient(credentials.id,credentials.key,credentials.tag);
api.call('ItemSearch', {SearchIndex:'Books', Keywords:'Java'}, function(err,result){
  if(err){
    console.log(err);
  } else {
    console.log(result);
  }

but I am getting an error like this:
{ [Error: connect EHOSTUNREACH]
code: 'EHOSTUNREACH',
errno: 'EHOSTUNREACH',
syscall: 'connect' }

Do I have to setup my server in a specific way in order to communicate with amazon via your API?

hi there, i just tried out this library and run into an error.

by testing the example for the amazon product advertising api i used:

prodAdv = _this.aws.createProdAdvClient(_this.options.aws_apikey, _this.options.aws_apisecret, _this.options.aws_user);

prodAdv.call("ItemSearch", {SearchIndex: "Books", Keywords: "Javascript"}, function(err, result) {
    console.log(result);
});

the response is:

{ '@': { xmlns: 'http://webservices.amazon.com/AWSECommerceService/2011-04-01' },
  OperationRequest: 
   { RequestId: 'xxxxx',
     Arguments: { Argument: [Object] },
     RequestProcessingTime: '0.0074920000000000' },
  Items: 
   { Request: 
      { IsValid: 'False',
        ItemSearchRequest: [Object],
        Errors: [Object] } } }

what am i doing wrong?

i used for yourAssociationTag the username i've created under https://console.aws.amazon.com/iam/home?#users

ec2.call 'DescribeSpotPriceHistory',
    {'StartTime': today_start, 'EndTime': today_end, 'Filter.1.Name': 'availability-zone', 'Filter.1.Value.1': az},
    (result) -> 
        # more code here

This stopped working with newer versions of xml2js. The callback is called with result as null.

@nivertech

Calls from federated users need a security token (session) sent in addition to standard params.

https://ec2.amazonaws.com/?Action=DescribeInstances
&InstanceId.0=I-45fa2e72
&Signature=Dqlp3Sd6ljTUA9Uf6SGtEExwUQEXAMPLE
&SignatureVersion=2
&SignatureMethod=HmacSHA256
&Timestamp=2010-03-31T12%3A00%3A00.000Z
&SecurityToken=Security Token Value
&AWSAccessKeyId=Access Key ID provided by AWS Security Token Service

http://docs.amazonwebservices.com/STS/latest/UsingSTS/UsingTokens.html

We don't have to replicate Amazon's docs for the supported API calls but I think it would help to have links in the readme.

According to the documentation, initializing the clients without key id and secret will cause the client to use the IAM role of the hosting EC2 machine for authentication. Unfortunately this does not work and when I try to instantiate a client with no parameters, I get the following error:

Uncaught TypeError: Not a buffer
  at new Hmac (crypto.js:226:17)
  at Object.Hmac (crypto.js:224:12)
  at Object.exports.hmacSha256 (/home/ubuntu/aws-lib/lib/utils.js:5:21)
  at signQuery (/home/ubuntu/aws-lib/lib/aws.js:166:18)
  at addQueryProperties (/home/ubuntu/aws-lib/lib/aws.js:121:34)
  at /home/ubuntu/aws-lib/lib/aws.js:71:15
  at /home/ubuntu/aws-lib/lib/metadata.js:59:9
  at IncomingMessage.<anonymous> (/home/ubuntu/aws-lib/lib/metadata.js:24:22)
  at IncomingMessage.EventEmitter.emit (events.js:117:20)
  at _stream_readable.js:910:16
  at process._tickCallback (node.js:415:13)

Also, when running the test without credentials, all the tests fail with this message.

I'm getting the following error for any ec2.call I try to make after upgrading to node 0.4.0 - I have no problems with same code under node 0.2.6. Any ideas?

node.js:116
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: ECONNRESET, Connection reset by peer
at Client._readImpl (net.js:142:14)
at Client._onReadable (net.js:608:22)
at IOWatcher.onReadable [as callback] (net.js:156:10

You must update this to inlcude the signature as requested from the AWS API. I looked at the code for this and it makes no attempt to sign the request.

Here is the JAVA code sample they use on their DOCS. Can't find the NODE equivalent on your module. http://docs.aws.amazon.com/AWSECommerceService/latest/DG/AuthJavaSampleSig2.html

How does one specify a particular region?

Thanks!

Hi,

I have a test script that calls the ListVerifiedEmailAddresses, GetSendStatistics and GetSendQuota SES APIs, all of which work with the ses-secure branch of mrduncan's fork.

Now, after seeing that you've pulled in this code and published version 0.0.4 to npm (thank you!), I thought I'd try it out.

Calling the ListVerifiedEmailAddresses API gives the same result, but both GetSendStatistics and GetSendQuota APIs yield this:

{ Message: 'Unable to determine service/operation name to be authorized' }

FYI, I'm testing all this on the stable 0.2.6 version of nodejs.

Thanks!

When I try example ec2 I get this error:

node ec2.js 

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: socket hang up
    at createHangUpError (http.js:1104:15)
    at Socket.<anonymous> (http.js:1187:27)
    at Socket.emit (events.js:88:20)
    at Array.0 (net.js:320:10)
    at EventEmitter._tickCallback (node.js:192:40)

I use node 0.6.7 on ubuntu 11.10

any idea ?

I know it's only just been released, but may as well have a ticket for it, no?

REST API details here:

http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/API.html

Is the "Filter.n.Name / Filter.n.Value" parameter supported? For example:

var aws = require("aws-lib");

ec2 = aws.createEC2Client("KEY", "SECRET");

var params = {};
params['Owner'] = 'self';
params['Filter.1.Name'] = 'tag-value';
params['Filter.1.Value'] = 'foo';

ec2.call("DescribeSnapshots", params, function(result, err) {
  // Results in error
})

Results in an "UnknownParameter" error, stating "The parameter Filter is not recognized" ... like the ".1.Name" is getting stripped at some point. It appears that the correct query is constructed.