engmzizo / copy-dynamodb-table Goto Github PK

Copy Dynamodb table to another in the same or different zone , It is 100% safe. Speed depends on your destination table user-defined write provisioned throughput

JavaScript 100.00%

copy-dynamodb-table's People

Contributors

Stargazers

Watchers

copy-dynamodb-table's Issues

RCU and WCU not used

Hey there!

I'm sure that this is probably an issue on my end, but I wonder why my code is not working. I updated my tables to have 100 RCUs (source) and 200 WCU (target). However, the output is Copied 25 items at a time.

Am I missing a configuration here?

What can I do so that the lib actually uses the tables RCUs and WCUs?

Thank you

TypeError: fn is not a function

Issue: Copying fails with the error TypeError: fn is not a function
Steps to Reproduce: Run the copy script.
Line causing the error: https://github.com/enGMzizo/copy-dynamodb-table/blob/master/index.js#L188
Traceback:

Copied 576957 items/Users/thomas/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^
TypeError: fn is not a function
    at /Users/thomas/node_modules/copy-dynamodb-table/index.js:188:16                                                                                                                     
    at Response.<anonymous> (/Users/thomas/node_modules/copy-dynamodb-table/index.js:254:5)    at Request.<anonymous> (/Users/thomas/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/Users/thomas/node_modules/aws-sdk/lib/sequential_executor.js:105:20) 
    at Request.emit (/Users/thomas/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/thomas/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/Users/thomas/node_modules/aws-sdk/lib/request.js:22:10) 
    at AcceptorStateMachine.runTo (/Users/thomas/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /Users/thomas/node_modules/aws-sdk/lib/state_machine.js:26:10 
    at Request.<anonymous> (/Users/thomas/node_modules/aws-sdk/lib/request.js:38:9)

This has happened both at the beginning of a run and 576957 records into a run. I believe this may be happening when the provisioned throughput is consumed, but am not positive.

Callback function is not called

Hi !
I am starting using your great tool.
It seems to work well except my callback function is not called.
I have looked briefly at the code and I wonder if it doesn't come from those lines

if (options.source.active && options.destination.active) { // both tables are active
    return startCopying(options, function (err, data) {
      if (err) {
        return fn(err, data)
      }
      if (options.continuousBackups) {
        setContinuousBackups(options, fn)
      }
    })
  }

I am in the case where both tables are active and options.continuousBackups is false.

Getting error " Unexpected key 'TableClassSummary' found in params " while running the code.

I was trying to replicate a table using the copy-dynamodb-table and encountered this issue.
I copy pasted the demo code present in the README and replaced all the placeholders with the appropriate keys.
As soon as i ran node ./replicateDB.js , i got this error:

UnexpectedParameter: Unexpected key 'TableClassSummary' found in params at ParamValidator.fail (C:\Users\default\project\node_modules\aws-sdk\lib\param_validator.js:50:37) at ParamValidator.validateStructure (C:\Users\default\project\node_modules\aws-sdk\lib\param_validator.js:78:14) at ParamValidator.validateMember (C:\Users\default\project\node_modules\aws-sdk\lib\param_validator.js:89:21) at ParamValidator.validate (C:\Users\default\project\node_modules\aws-sdk\lib\param_validator.js:34:10) at Request.VALIDATE_PARAMETERS (C:\Users\default\project\node_modules\aws-sdk\lib\event_listeners.js:132:42) at Request.callListeners (C:\Users\default\project\node_modules\aws-sdk\lib\sequential_executor.js:106:20) at callNextListener (C:\Users\default\project\node_modules\aws-sdk\lib\sequential_executor.js:96:12) at C:\Users\default\project\node_modules\aws-sdk\lib\event_listeners.js:86:9 at finish (C:\Users\default\project\node_modules\aws-sdk\lib\config.js:396:7) at C:\Users\default\project\node_modules\aws-sdk\lib\config.js:414:9 { code: 'UnexpectedParameter', time: 2022-09-22T11:17:32.658Z } { Table: { AttributeDefinitions: [ [Object] ], TableName: 'BackupTable', KeySchema: [ [Object] ], TableClassSummary: { TableClass: 'STANDARD' }, BillingMode: 'PAY_PER_REQUEST' } }

DynamoDB WCU is not utilized

Dears:

First I have to say, this is a really helpful tool.

However when I used it to copy my source table to my target table, on CloudWatch I found that only 25 WCU was consumed per second. (Both my source and target tables have more than 150 RCU and more than 1000 WCU)

I am assuming the limitation of 25 request per BatchWriteItem API was met. Is it as expected?

Kind Regard :')

Add support for ondemand dynamodb tables

Hey Guys,
I am getting errors while trying to copy Ondemand tables but works fine with provisioned tables.

{ ValidationException: 2 validation errors detected: Value '0' at 'provisionedThroughput.writeCapacityUnits' failed to satisfy constraint: Member must have value greater than or equal to 1; Value '0' at 'provisionedThroughput.readCapacityUnits' failed to satisfy constraint: Member must have value greater than or equal to 1
at Request.extractError (/Users/prasanjit_shome/node_modules/aws-sdk/lib/protocol/json.js:51:27)
at Request.callListeners (/Users/prasanjit_shome/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/Users/prasanjit_shome/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/Users/prasanjit_shome/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/Users/prasanjit_shome/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/Users/prasanjit_shome/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Users/prasanjit_shome/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/Users/prasanjit_shome/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/Users/prasanjit_shome/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/Users/prasanjit_shome/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
message: '2 validation errors detected: Value '0' at 'provisionedThroughput.writeCapacityUnits' failed to satisfy constraint: Member must have value greater than or equal to 1; Value '0' at 'provisionedThroughput.readCapacityUnits' failed to satisfy constraint: Member must have value greater than or equal to 1',
code: 'ValidationException',
time: 2019-04-25T19:54:24.873Z,
requestId: 'L9MVKUQ57HSNAU189FOUGCIJDBVV4KQNSO5AEMVJF66Q9ASUAAJG',
statusCode: 400,
retryable: false,
retryDelay: 32.77870743752317 }
{ Table:
{ AttributeDefinitions: [ [Object], [Object] ],
TableName: 'XXXXXXXXXXXXXXXXX,
KeySchema: [ [Object], [Object] ],
ProvisionedThroughput: { ReadCapacityUnits: 0, WriteCapacityUnits: 0 },
StreamSpecification: { StreamEnabled: true, StreamViewType: 'NEW_IMAGE' },
SSESpecification: { Enabled: true } } }

Module crashes when copying tables where item count is a multiple of 25

When copying a table that happened to have exactly 700 elements, the module crashed with the following error:

Copied 675 items{ ValidationException: 1 validation error detected: Value '{prod-checkinresponses=[]}' at 'requestItems' failed to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 25, Member must have length greater than or equal to 1]
    at Request.extractError (<PROJECT_DIR>/node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.callListeners (<PROJECT_DIR>/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (<PROJECT_DIR>/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (<PROJECT_DIR>/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (<PROJECT_DIR>/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (<PROJECT_DIR>/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at <PROJECT_DIR>/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (<PROJECT_DIR>/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (<PROJECT_DIR>/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (<PROJECT_DIR>/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
  message: '1 validation error detected: Value \'{prod-checkinresponses=[]}\' at \'requestItems\' failed to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 25, Member must have length greater than or equal to 1]',
  code: 'ValidationException',
  time: 2018-03-15T23:05:03.272Z,
  requestId: '<REQUEST ID>',
  statusCode: 400,
  retryable: false,
  retryDelay: 29.387693618990397 }

Other tables copied over just fine. After changing the Limit field passed to scan in the module code from 25 to 24, the copy succeeded. It seems there's a bug that occurs when the item count is a multiple of the limit, which in this case is 25.

Copy indices also

How do you copy table schema?

Copy only schema

Hello, is there any flag to pass to not copy the data, only the schema?

My use case:

My environments are in different accounts, so i have a account for dev, another for uat and other for prod.

So my first situation is to keep synced the schema from dev to prod (at least for the first creation, for migration i should think in another job)

Than after that, i will create a cron job to sync data down, bring data from prod to uat and than to dev

if your lib can replicate only the schema, i can use it for both cases.

Today im using .json schema template and calling aws cli to create the table, it works, but i'm looking for a more robust or better way

Copy table settings , backup and TTL

AWS-SDK version update to latest?

soon the aws-sdk version 2 will be deprecated according to the node version. so please update the aws-sdk version too.

Read config from .aws/config

Hi,

Good day.

Nice to have feature. Reading the credentials from the aws cli config profile file.

Regards.

How to copy to/from local dynamodb

Is it possible to use a local dynamodb as a source or target?

LSI ProvisionedThroughput is not existed

Hi, I found errors in index.js for table.LocalSecondaryIndexes[i].ProvisionedThroughput is undefined. I modified the following code for LSI and it works :

  if (table.LocalSecondaryIndexes[i].ProvisionedThroughput){
    delete table.LocalSecondaryIndexes[i].ProvisionedThroughput.LastIncreaseDateTime
    delete table.LocalSecondaryIndexes[i].ProvisionedThroughput.LastDecreaseDateTime
    delete table.LocalSecondaryIndexes[i].ProvisionedThroughput.NumberOfDecreasesToday
  }

Across Account

What would it take to do this across accounts? I mean copy tables from one account to another account?

How to filter the data being copied?

Hi, is there a way to filter the data that is being copied so I can choose to not copy the records I don't need based on some condition?

Let's say I want to only copy records from one table to another where the CustomerID is the one I specify instead of coping all records.
If I try to do that in the transform function, I get a MultipleValidationErrors error.

It would be helpful if I could define filter expressions to be used with options.source.dynamoClient.scan in your scan method.

How to specify item limit when copying

Great library firstly. I would like to know how to specify the limit to copy the items into the target databse?

Thanks.

Error during copy - Unexpected key 'LatestStreamArn' found in params

I'm trying to copy a table from one account to another. My setup:

var copy = require('copy-dynamodb-table').copy
 

var sourceAWSConfig = {
  accessKeyId: 'AK...7Q',
  secretAccessKey: 'at...BJ',
  region: 'us-east-1'
}
 
var destinationAWSConfig = {
  accessKeyId: 'AK...YA',
  secretAccessKey: 'A0...eZ',
  region: 'us-east-1' // support cross zone copying
}
 
copy({
    config: sourceAWSConfig,
    source: {
      tableName: 'Opportunities', // required
      config: sourceAWSConfig // optional , leave blank to use globalAWSConfig
    },
    destination: {
      tableName: 'Opportunities', // required
      config: destinationAWSConfig // optional , leave blank to use globalAWSConfig
    },
    log: true,// default false
    create : true // create destination table if not exist
  },
  function (err, result) {
    if (err) {
      console.log('error:', err)
    }
    console.log('result:', result)
  })

produces an error:

* UnexpectedParameter: Unexpected key 'LatestStreamLabel' found in params
* UnexpectedParameter: Unexpected key 'LatestStreamArn' found in params
    at ParamValidator.validate (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:40:28)
    at Request.VALIDATE_PARAMETERS (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:125:42)
    at Request.callListeners (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at callNextListener (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:85:9
    at finish (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:320:7)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:338:9
    at Credentials.get (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/credentials.js:126:7)
    at getAsyncCredentials (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:332:24)
    at Config.getCredentials (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:352:9)
  message: 'There were 2 validation errors:\n* UnexpectedParameter: Unexpected key \'LatestStreamLabel\' found in params\n* UnexpectedParameter: Unexpected key \'LatestStreamArn\' found in params',
  code: 'MultipleValidationErrors',
  errors:
   [ { UnexpectedParameter: Unexpected key 'LatestStreamLabel' found in params
    at ParamValidator.fail (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:77:14)
    at ParamValidator.validateMember (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:125:42)
    at Request.callListeners (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at callNextListener (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:85:9
    at finish (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:320:7)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:338:9
       message: 'Unexpected key \'LatestStreamLabel\' found in params',
       code: 'UnexpectedParameter',
       time: 2018-02-27T21:55:35.455Z },
     { UnexpectedParameter: Unexpected key 'LatestStreamArn' found in params
    at ParamValidator.fail (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:77:14)
    at ParamValidator.validateMember (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:125:42)
    at Request.callListeners (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at callNextListener (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/event_listeners.js:85:9
    at finish (/Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:320:7)
    at /Users/matthew/projects/anotherprojectname/myprojectname/node_modules/aws-sdk/lib/config.js:338:9
       message: 'Unexpected key \'LatestStreamArn\' found in params',
       code: 'UnexpectedParameter',
       time: 2018-02-27T21:55:35.455Z } ],
  time: 2018-02-27T21:55:35.456Z }
result: { Table:
   { AttributeDefinitions: [ [Object] ],
     TableName: 'Opportunities',
     KeySchema: [ [Object] ],
     ProvisionedThroughput: { ReadCapacityUnits: 20, WriteCapacityUnits: 20 },
     StreamSpecification: { StreamEnabled: true, StreamViewType: 'NEW_AND_OLD_IMAGES' },
     LatestStreamLabel: '2017-07-27T17:13:42.958',
     LatestStreamArn: 'arn:aws:dynamodb:us-east-1:213123123:table/Opportunities/stream/2017-07-27T17:13:42.958' } }

How to use promise

Hi,

Sorry if this question is stupid, but does this library support Promise? How to use it

Thanks

Dynamodb OnDemand pricing model changes

If the source table is on demand pricing model enabled the destination table creation will be failed.

I found that in the describeTable returning

  "ProvisionedThroughput": {
    "LastIncreaseDateTime": "2018-11-29T17:41:51.174Z",
    "NumberOfDecreasesToday": 0,
    "ReadCapacityUnits": 0,
    "WriteCapacityUnits": 0
 },
"BillingModeSummary": {
    "BillingMode": "PAY_PER_REQUEST",
    "LastUpdateToPayPerRequestDateTime": "2018-12-11T12:58:01.922Z"
  }

see API_DescribeTable
where createTable function is expecting

"BillingMode": "PAY_PER_REQUEST"

and ProvisionedThroughput also should not be defined for this billing mode.

see API_CreateTable

Beside this, one of the important use case of this lib is copy table from AWS to dynamodb local for testing and debug purposes.

However, dynamodb local hasn't support this new billing model yet. Any idea how we could detect if the dest table if a local db so that we could ignore the BillingMode attribute and fake the ProvisionedThroughput to bypass this issue?

Use the ProvisionedThroughputCapacity from the config

I am trying to use the library and when i include the readCapacity and writeCapacity in the destination config , these values are not honored for both the table and the globalIndex .

A valid table name is considered invalid due to the table name validation

Test 1:

sourceTable: xxx1xxx-yy-zzzzz

source.tableName must follow AWS naming rules (3-255 length, and only the following characters: a-z, A-Z, 0-9, _-.)
{ count: 0, status: 'FAIL' }

Test 2:

sourceTable: xxx1abc-yy-123
source.tableName must follow AWS naming rules (3-255 length, and only the following characters: a-z, A-Z, 0-9, _-.)
{ count: 0, status: 'FAIL' }

Table creation if it not exist

Hi, is destination table auto-creation exist in current lib version?
It will be rely great feature!

How to copy data from/to multiple tables at once?

Where is this intended to be run, in a Lambda?

I've setup a Lambda to run this but I must be doing something wrong. I've hardcoded the API keys and secret in to my code (taken from the AWS Console https://aws.amazon.com/blogs/security/how-to-find-update-access-keys-password-mfa-aws-management-console/).

The Lamba runs but nothing is happening with the code, no errors are reported and nothing in the logs. I'm running the Lambda via the Test button in the web interface which should pass in a dummy event but I can't see where this code needs any of the environment vars it would get from running it in any other way and again its not complaining about anything so I think that should be ok?

What am I doing wrong?

var copy = require('copy-dynamodb-table').copy;

var globalAWSConfig = { 
  accessKeyId: 'MYKEY',
  secretAccessKey: 'MYSECRET',
  region: 'eu-west-1'
};

function copyTable( from, to ) {
    console.info( "Copying table " + from + " to " + to );
    
    copy({
        config: globalAWSConfig,
        source: {
            tableName: from, // required
        },
        destination: {
            tableName: to, // required
        },
        log: true, // default false
        create : true // create destination table if not exist
    },
    function (err, result) {
        if (err) {
            console.log(err);
        }
        console.log("Result", result );
    });
    
    console.info( "Done?");
}

exports.handler = async (event) => {
    console.info( event );
    copyTable( "my-source-table-dev", "my-target-table-live" );

    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

Seems to fail when copying from a global replicated table

Copying from a table that is replicated to a 2nd region into a new table that is not replicated fails per below. Copying from a non-replicated table to a replicated table works. In both cases, the source and destination tables are in the same region

MultipleValidationErrors: There were 2 validation errors:
    * UnexpectedParameter: Unexpected key 'GlobalTableVersion' found in params
* UnexpectedParameter: Unexpected key 'Replicas' found in params
at ParamValidator.validate (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:40:28)
at Request.VALIDATE_PARAMETERS (/Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:132:42)
at Request.callListeners (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at callNextListener (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
at /Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:86:9
at finish (/Users/user/dev/test/node_modules/aws-sdk/lib/config.js:386:7)
at /Users/user/dev/test/node_modules/aws-sdk/lib/config.js:404:9
at SharedIniFileCredentials.get (/Users/user/dev/test/node_modules/aws-sdk/lib/credentials.js:127:7)
at getAsyncCredentials (/Users/user/dev/test/node_modules/aws-sdk/lib/config.js:398:24)
at Config.getCredentials (/Users/user/dev/test/node_modules/aws-sdk/lib/config.js:418:9) {
    code: 'MultipleValidationErrors',
        errors: [
        UnexpectedParameter: Unexpected key 'GlobalTableVersion' found in params
    at ParamValidator.fail (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:77:14)
    at ParamValidator.validateMember (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/Users/user/dev/test/node_modules/aws-sdk/lib/config.js:386:7)
    at /Users/user/dev/test/node_modules/aws-sdk/lib/config.js:404:9 {
        code: 'UnexpectedParameter',
            time: 2020-08-11T21:37:47.341Z
    },
    UnexpectedParameter: Unexpected key 'Replicas' found in params
    at ParamValidator.fail (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:77:14)
    at ParamValidator.validateMember (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/Users/user/dev/test/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:132:42)
    at Request.callListeners (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at callNextListener (/Users/user/dev/test/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at /Users/user/dev/test/node_modules/aws-sdk/lib/event_listeners.js:86:9
    at finish (/Users/user/dev/test/node_modules/aws-sdk/lib/config.js:386:7)
    at /Users/user/dev/test/node_modules/aws-sdk/lib/config.js:404:9 {
        code: 'UnexpectedParameter',
            time: 2020-08-11T21:37:47.341Z
    }
],
    time: 2020-08-11T21:37:47.341Z
}
{
    Table: {
        AttributeDefinitions: [ [Object] ],
            TableName: 'blah',
            KeySchema: [ [Object] ],
            ProvisionedThroughput: { ReadCapacityUnits: 5, WriteCapacityUnits: 5 },
        StreamSpecification: { StreamEnabled: true, StreamViewType: 'NEW_AND_OLD_IMAGES' },
        GlobalTableVersion: '2019.11.21',
            Replicas: [ [Object] ]
    }
}

Possible to bubble up aws-sdk errors?

Hey @enGMzizo, I love this tool, already copied a few tables over into new global table versions. I have an issue with one table, it doesn't seem to be copying and there is no output after running the function. I am wondering how can I bubble up the errors from the aws-sdk?

If I debug the copy() function, I end up @ aws-sdk/lib/event_listeners.js and I see the error in resp.error.message and the status code is 400.

resp.error={
  message: "One or more parameter values were invalid: Type mismatch for Index Key individual_id Expected: N Actual: S IndexName: individual_id-index",
  code: "ValidationException",
  time: {
  },
  requestId: "***",
  statusCode: 400,
  retryable: false,
  retryDelay: 2.9211740648415696,
}

I think it would be helpful if the message was bubbled up, but not sure how to do that with js. Any help would be appreciated. I have tried wrapping copy() in a try catch but got no output.

engmzizo / copy-dynamodb-table Goto Github PK

copy-dynamodb-table's People

Contributors

Stargazers

Watchers

Forkers

copy-dynamodb-table's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs