GithubHelp home page GithubHelp logo

willhlaw / node-firestore-backup-restore Goto Github PK

View Code? Open in Web Editor NEW

This project forked from steadyequipment/node-firestore-backup

91.0 91.0 24.0 133 KB

Google Firebase Firestore backup and restore tool

JavaScript 100.00%

node-firestore-backup-restore's People

Contributors

andraus avatar arpitkotecha avatar chaoranxie avatar dkbyo avatar pmarconi avatar willhlaw avatar yoiang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-firestore-backup-restore's Issues

.DS_Store prevents successfull restore

When I try to restore using this command:
firestore-backup-restore --backupPath /Users/amjad/Desktop/untitledfolder/iOrder --restoreAccountCredentials /Users/amjad/Desktop/iorder--credentials.json

I get the following error:

The behavior for Date objects stored in Firestore is going to change
AND YOUR APP MAY BREAK.
To hide this warning and ensure your app does not break, you need to add the
following code to your app before calling any other Cloud Firestore methods:

  const settings = {/* your settings... */ timestampsInSnapshots: true};
  const firestore = new Firestore(settings);

With this change, timestamps stored in Cloud Firestore will be read back as
Firebase Timestamp objects instead of as system Date objects. So you will also
need to update code expecting a Date to instead expect a Timestamp. For example:

  // Old:
  const date = snapshot.get('created_at');
  // New:
  const timestamp = snapshot.get('created_at');
  const date = timestamp.toDate();

Please audit all existing usages of Date when you enable the new behavior. In a
future release, the behavior will change to the new behavior, so if you do not
follow these steps, YOUR APP MAY BREAK.
Restoring to collection /Users/amjad/Desktop/untitledfolder document /Users/amjad/Desktop/untitledfolder/iOrder/.DS_Store
undefined:1



SyntaxError: Unexpected token 

I have activated show hidden files on my MAC, I couldn't locate the .DS_Store in the restore folder. I'm stuck and cannot restore my backup files

Can we restore only from local backup and not downloading it again from the source Firestore account?

Currently, the backup and restore works together like

➜  fs firestore-backup-restore -a source_service_account_key.json -B bakup --restoreAccountCredentials target_service_account_key.json

This makes a dependency of restore on source account. This is not useful if we only want to restore based on the current backup on the filesystem. This makes the local storage not so useful.

Requirement
We should be able to restore based on -B and -a2 flags only. Currently, it is not possible

firestore-backup-restore -B bakup --restoreAccountCredentials target_service_account_key.json
Missing: accountCredentials - Google Cloud account credentials JSON file

  Usage: firestore-backup-restore [options]


  Options:

    -V, --version                            output the version number
    -a, --accountCredentials <path>          Google Cloud account credentials JSON file
    -B, --backupPath <path>                  Path to store backup.
    -a2, --restoreAccountCredentials <path>  Google Cloud account credentials JSON file for restoring documents.
    -P, --prettyPrint                        JSON backups done with pretty-printing.
    -h, --help                               output usage information

Is the backup log sounds misleading?

Hi,

During the backup noticed, something like

Backing up Document '/users/7Loope6Pz9YiBEfs3LjFPVnafqv1' with type information
Restoring to collection /users document 7Loope6Pz9YiBEfs3LjFPVnafqv1...

After seeing the phrase Restoring to collection /users, i was like wait, am i restoring ? I was just backing up right. Later i realized its just means its storing the document on the storage. Felt its kinda misleading. It could be just me.

Authentication users backup

Hi
First, thank you for this backup/restore solutions.
Did a clone and worked fine.
I have a dought, the collection Users is backup but not the actual "Authentication users".
So if clone the DB and want to use it, when registering /signin, firebase will give a new User ID. Therefore, the user will not have is previous config and other data access.

Any solutions?
Thank you

Add the ability to apply a transformation function to documents before backup or restoration

Add a "schema migration" feature that add the ability to apply a transformation function before either writing to disk (so user can verify the schema migration) or before restoring to Firestore.

The proposal idea is to have a flag for this, called --transformFn or -T for the command line, where the user can indicate the path of a file from where the transformation function will be exported.

And for example will be executed like this:

firestore-backup-restore --accountCredentials cred.json --backupPath backups --transformFn transformFn.js

This transformation function should return a promise, and receive a object as parameter, which will contain the following fields:

accountDb {Object} Firestore database instance from where the backup is made
restoreAccountDb {Object} Firestore database instance where the backup is restored
collectionPath {Array} Array that contains the path of the collection
docId {String} Id of the document, where the function will be applied
docData {Object} Document data with "backup" format, specified below*

docData will have the format NAME: { "value": VALUE, "type": TYPE },
For example:

Task {
  "name": { "value": "Buy some apples", "type": "string" }
  "difficulty": { "value": "easy", "type": "string" },
}

The allowed types are:
'string', 'number', 'boolean', 'object', 'array', 'null', 'timestamp', 'geopoint', 'documentReference'

Example of transformation function:

  • To rename a field called MyCompany, to company in all the documents of Companies
const transformFn = async ({
  accountDb,
  restoreAccountDb,
  collectionPath,
  docId,
  docData
}) => {
  const operationByCollection = {
    Companies: function(companyDoc) {
      const docResult = companyDoc;
      if (!companyDoc.MyCompany) return Promise.resolve(docResult);
      docResult.company = {
        value: docResult.MyCompany.value,
        type: 'documentReference'
      };
      delete docResult.MyCompany;
      return Promise.resolve(docResult);
    }  
  };
  const collectionName = collectionPath[collectionPath.length - 1] // take the last one
  if (!operationByCollection[collectionName]) {
    console.log(`There is not a transformation for ${collectionName}`);
    return docData;
  }
  return await operationByCollection[collectionName](docData);
};

export default transformFn;

Socket hang up error during restore

I am restoring a DB with about 100K records and 77MB, I will hit this error soon after the restore starts.

!!! Uh-Oh, error saving collection xxx.json { Error: Getting metadata from plugin failed with error: socket hang up
at /abc/server/firebase/scripts/node_modules/grpc/src/client.js:554:15 code: 16, metadata: Metadata { _internal_repr: {} } }

Looks like it's the same issue as described here, the write request exceed the Firestore's quota and the server is rejecting most of them. Anyone else hit this issue?

I hit this for backup as well, but not as frequent.

SyntaxError: Unexpected token

When I run --restoreAccountCredentials command I just get this in my terminal.

undefined:1
SyntaxError: Unexpected token

Please help.

Restore option not working

Hello, I had previously made a backup and now I want to restore but unfortunatelly I keep getting this error:
var childFieldObject = value.reduce(function (acc, cur) {
^

TypeError: Cannot read property 'reduce' of undefined
at /home/developer/.nvm/versions/node/v8.11.3/lib/node_modules/firestore-backup-restore/build/lib/FirestoreDocument.js:79:36

thanks in advance

Backup and restore a DocumentReference field from database to another one

I currently get an error when wanting to restore a document that contains a document reference, using the command line

firestore-backup-restore --accountCredentials path/to/account/credentials/app1.json --backupPath /backups/myDatabase --restoreAccountCredentials path/to/restore/credentials/app2.json

In my tests, I get the following error

Error! Restoring to collection /User document Prp9Qpe5iU1Kzm2p44Ag - Error: Reference value app: "app1"
PathElement {
   type: "User"
   name: "Prp9Qpe5iU1Kzm2p44Ag"
}
  in different app partition
Error! Restoring to collection/User/Prp9Qpe5iU1Kzm2p44Ag/info document bWMsubqe8rBhDpu2ik1V - Error: Reference value app: "app1"
PathElement {
   type: "User"
   name: "Prp9Qpe5iU1Kzm2p44Ag"
}
  in different app partition

I think this error happens because we try to save a reference to a document belonging to another database

Backup empty collection folders

Tried to backup using all the options, but the result is the same: folders created with the name of the collections but no file inside
What am i doing wrong?
Thanks

sorting the keys before writing documents to file

is there any way to sort the keys before writing the documents to file? Basically, I am committing the backup files to version control but every time I do a new backup it shows all the documents are changed.

From what I can tell, following code is all that's needed

var stringify = require('json-stable-stringify');
fileContents = stringify(documentBackup, { space: 2 });

instead of

fileContents = JSON.stringify(documentBackup, null, 2);

Add feature to delete all documents in all collections or to replace when restoring

When migrating data or backing up and restoring data, we want to completely replace the database when restoring the data. So there should be an option to clear a firestore database. The option should be made clearly available as a standalone operation and an option when restoring data from disk or from another firestore instance.

The idea is from this comment by @MrDrProfX.

@pmarconi Awesome. Something I'm wondering about is what will happen to new documents/fields and collections that have been added since the backup. Two options could be:

Replace: delete the entire target firestore db and completely replace it with the disk backup
Merge: merge the target firestore db with the disk backup
Perhaps this could be an additional command line argument. Personally I would rely on the merge for my project.

See reference docs for deleting in batch: https://firebase.google.com/docs/firestore/manage-data/delete-data#collections

Checklist

  • New option --delete to use with -a, --accountCredentials
  • New option --replace to use with -a2, --restoreAccountCredentials

Optional

  • Safeguards such as prompt user "Are you sure?" or ensure a backup is created before deleting

Backup fails for subcollections unless document exists for parent document

My database is set up like this:

-versions
--v1
---tasks
----task1
----task2

I tried creating a backup of my database, however, it fails -- that is, it creates the "v1/tasks" folder, but it's left empty.

I figured this might be because the "v1/tasks" path is just an empty container -- there is no actual document at that location.

To fix it, I tried creating a document (with a meaningless field) at "versions/v1". As expected, the backup then succeeded. (because then this code was able to run: https://github.com/willhlaw/node-firestore-backup-restore/blob/master/index.js#L187)

It would be nice if the library did not require a "placeholder" document for the subcollections under a path to be able to be backed up.

Why show "throw new errors.AssertionError({"

Details
AssertionError [ERR_ASSERTION]: missing path
at Module.require (module.js:594:3)
at require (internal/module.js:11:18)
at Object. (/Users/y/.npm-global/lib/node_modules/firestore-backup-restore/node_modules/@google-cloud/firestore/build/src/index.js:24:20)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)

Backup of Collection with other collection but with empty document fields

I have detected that if you have a collection that contains an empty document with no fields but with another collection, the data is not saved.

Collection_1

  • Document_1
    • Fields
      • Field_1
      • Field_2
    • Collections
      • Collection_1_1
        • ...
  • Document_2
    • Fields
    • Collections
      • Collection_2_1
        • ...

The Collection_2_1 is not saved because the Document_1 doesn't have fields, if you add a field to the Document_2, the data is saved in the backup process

Error: Argument "collectionPath" is not a valid ResourcePath. Path must be a non-empty string.

Hi! I need you help please

I was trying to restore a recent backup and I am getting this error:

Error: Argument "collectionPath" is not a valid ResourcePath. Path must be a non-empty string.
    at Object.exports.(anonymous function) [as isResourcePath] (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\node_modules\@google-cloud\firestore\src\validate.js:86:15)
    at Firestore.collection (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\node_modules\@google-cloud\firestore\src\index.js:296:14)
    at saveDocument (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\lib\FirestoreDocument.js:27:32)
    at C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:211:63
    at Array.forEach (native)
    at restoreBackup (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:192:34)
    at C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:197:28
    at Array.forEach (native)
    at restoreBackup (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:192:34)
    at Object.<anonymous> (C:\Users\Pablo\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:239:21)

Am I doing anything wrong?
firestore-backup-restore --backupPath myBackupPath --restoreAccountCredentials blablabla.json

Thanks

Update tool to use gcloud auth instead of requiring credentials.json

Right now, this tool requires a service account credentials.json file to be specified to be able to access Google Firestore rules API. I think we should update it to allow gcloud auth instead or as an alternative.

However, I have verified it works if we do the following:

  1. npm install -g @google-cloud/cloud-sdk
  2. open new terminal so gcloud is on the PATH
  3. run gcloud auth application-default login and gcloud config set project [PROJECT_NAME]

After that, we do not need credentials.json if we update firestore-backup-restore to initialize firebase with:

  1. Firebase.initializeApp({ credential: Firebase.credential.applicationDefault(), databaseURL: 'https://[PROJECTDB_NAME].firebaseio.com' })
  • Try to InitializeApp with default application credential if no path to credentials.json given
  • Update README with gcloud auth instructions

Allow --excludeCollections option from node-firstore-backup

This would be pretty helpful as several of my collections are huge log collections that don't necessarily need to be backed up (especially when I'm about to perform a chance that only impacts a specific collection). This would be very nice to have!

Firestore error/warning message

I'm getting this Firestore error/warning at the top of the cmd window when running the plugin:

The behavior for Date objects stored in Firestore is going to change
AND YOUR APP MAY BREAK.
To hide this warning and ensure your app does not break, you need to add the
following code to your app before calling any other Cloud Firestore methods:

  const firestore = new Firestore();
  const settings = {/* your settings... */ timestampsInSnapshots: true};
  firestore.settings(settings);

With this change, timestamps stored in Cloud Firestore will be read back as
Firebase Timestamp objects instead of as system Date objects. So you will also
need to update code expecting a Date to instead expect a Timestamp. For example:

  // Old:
  const date = snapshot.get('created_at');
  // New:
  const timestamp = snapshot.get('created_at');
  const date = timestamp.toDate();

Please audit all existing usages of Date when you enable the new behavior. In a
future release, the behavior will change to the new behavior, so if you do not
follow these steps, YOUR APP MAY BREAK.

Is this message being generated by Firestore in response to something coded within the plugin?

Why is the backup operation calling restoreDocument?

I am running the backup command firestore-backup-restore --accountCredentials [...] --backupPath Backup1, yet while the backup is running, I keep seeing lines like this one:
Restoring to collection /versions/v6-prod/tasks document FcJgIhpH6JgDoXMlepQUVLPfXmT2...

Looking at the source, it seems the restoreDocument function is called from here: https://github.com/willhlaw/node-firestore-backup-restore/blob/master/index.js#L242

Why is it "restoring" a document when it's just supposed to be doing a backup?

Are these interim "restores" actually setting data on the live database? If so, I'm concerned that a future bug in the library could cause backup operations to actually mess up data on the live database, when I would want it to only read from the database.

Add feature to restore with merge option

To prevent destroying existing data fields and to support more of a schema migration from dev to staging, add a merge feature when restoring documents from either disk or another firestore instance.

The idea is from this comment by @MrDrProfX.

@pmarconi Awesome. Something I'm wondering about is what will happen to new documents/fields and collections that have been added since the backup. Two options could be:

Replace: delete the entire target firestore db and completely replace it with the disk backup
Merge: merge the target firestore db with the disk backup
Perhaps this could be an additional command line argument. Personally I would rely on the merge for my project.

Checklist

  • New option --merge to use with -a2, --restoreAccountCredentials that sets documents with {merge: true} option

JavaScript heap out of memory

I'm trying to restore my backup to another Firestore database, but at the end of restore process I received that information, how can I handle it?
My database have between 60k˜80k documents

captura de tela 2018-08-14 as 10 18 40

Remove --backupPath from being required when cloning from one Firestore to another

When cloning data from from Firestore in --accountCredentials to Firestore specified in --accountRestoreCredentials, --backupPath is still needed or you get an error. The backup path should not be required.

  • "Missing: backupPath - Path to store backup" condition is removed
  • Clone operation* can run without requiring --backupPath

*Clone operation

firestore-backup-restore --accountCredentials path/to/account/credentials/file.json --restoreAccountCredentials path/to/restore/credentials/file.json

Not able to restore when data is backed up using -J -S flags

I'm using firestore-backup-restore --accountCredentials <key.json> --backupPath <backup_dir> --prettyPrint -J --stable to back up Firestore. It works great !
However, Restoring it using this command firestore-backup-restore --backupPath <backup_dir> --restoreAccountCredentials <key.json>, fails to restore data back to Firestore. It gives Unsupported type, undefined from ... error in restoring logs.

NPM Install, Node 8.11.3

Getting an issue with the install on a new Ubuntu server

sudo npm install -g firestore-backup-restore
/usr/local/bin/firestore-backup-restore -> /usr/local/lib/node_modules/firestore-backup-restore/bin/firestore-backup-restore.js

[email protected] install /usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc
node-pre-gyp install --fallback-to-build --library=static_library

node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v57 ABI, glibc) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error EACCES: permission denied, mkdir '/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node'
gyp WARN EACCES user "root" does not have permission to access the dev dir "/home/dev/.node-gyp/8.11.3"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/.node-gyp"
gyp WARN install got an error, rolling back install
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/.node-gyp'
gyp ERR! System Linux 4.15.0-1014-gcp
gyp ERR! command "/usr/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--library=static_library" "--module=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc/grpc_node.node" "--module_name=grpc_node" "--module_path=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc" "--napi_version=3" "--node_abi_napi=napi"
gyp ERR! cwd /usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc
gyp ERR! node -v v8.11.3
gyp ERR! node-gyp -v v3.7.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc/grpc_node.node --module_name=grpc_node --module_path=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc --napi_version=3 --node_abi_napi=napi' (1)
node-pre-gyp ERR! stack at ChildProcess. (/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack at emitTwo (events.js:126:13)
node-pre-gyp ERR! stack at ChildProcess.emit (events.js:214:7)
node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:925:16)
node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
node-pre-gyp ERR! System Linux 4.15.0-1014-gcp
node-pre-gyp ERR! command "/usr/bin/node" "/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--library=static_library"
node-pre-gyp ERR! cwd /usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc
node-pre-gyp ERR! node -v v8.11.3
node-pre-gyp ERR! node-pre-gyp -v v0.10.3
node-pre-gyp ERR! not ok
Failed to execute '/usr/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc/grpc_node.node --module_name=grpc_node --module_path=/usr/local/lib/node_modules/firestore-backup-restore/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc --napi_version=3 --node_abi_napi=napi' (1)
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/firestore-backup-restore/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build --library=static_library
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/dev/.npm/_logs/2018-08-03T09_15_39_437Z-debug.log

Argument "collectionPath" must point to a collection, but was "${collectionPath}"

My backup looks great on my hard drive, thanks. But when I tried to restore a collection (not the entire database) I used this command:

firestore-backup-restore --backupPath /Users/TDK/LanguageTwo/backups/firestore/Dictionaries/Chinese/Words/ --restoreAccountCredentials /Users/TDK/LanguageTwo/myproject-firebase-adminsdk-qwerty12345.json

I got this error message:

Argument "collectionPath" must point to a collection, but was "${collectionPath}". Your path does not contain an odd number of components.

Dictionaries and Words are collections, so I'm pointing to a collection. Any idea what the problem is?

Update the docs to clarify functionality

Motivation from #3 (comment).

  • Clarify operations to retrieve and backup to disk (backup), retrieve and restore to another firestore (clone), and push from backup to another firestore (restore).
  • Add new commands that better match the intended action
  • Deprecate current commands

Slow?

I'm just wondering if I'm doing something wrong.
I'm trying to backup a database with around 100k items. The backup is already taking 60mins and we are not even half way through?

I see that https://github.com/steadyequipment/node-firestore-backup has an -L option to enable several requests to be made in parallel, but we don't seem to have this in this package?

-L, --requestCountLimit - The maximum number of requests to be made in parallel.

unable to backup latest firestore

has anyone seen this error - running the latest firestore-tools; never tried this tool before yesterday so I'm not sure if my environment would have worked with previous versions.
Using basic parameters firestore-backup-restore -a /path/dev.json -B /path/backup/date
The service json file does work.

TypeError [ERR_INVALID_ARG_TYPE]: The "id" argument must be of type string. Received type object
at Module.require (internal/modules/cjs/loader.js:644:11)
at require (internal/modules/cjs/helpers.js:20:18)
at Object. (/usr/local/lib/node_modules/firestore-backup-restore/node_modules/@google-cloud/firestore/build/src/index.js:24:20)
at Module._compile (internal/modules/cjs/loader.js:702:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
at Module.load (internal/modules/cjs/loader.js:612:32)
at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
at Function.Module._load (internal/modules/cjs/loader.js:543:3)
at Module.require (internal/modules/cjs/loader.js:650:17)
at require (internal/modules/cjs/helpers.js:20:18)

Error while Cloning or Restoring documents with 'reference' type fields from one project to another (related to closed issue #9)

firestore-backup-restore --accountCredentials /source.json --backupPath /path --restoreAccountCredentials /target.json

I have just tried it out in versions 1.2.1, 1.3.1 and the error is still there.

Error! Restoring to collection /releases document ylMxpcTV13Tj5JzVbBP6 - Error: 3 INVALID_ARGUMENT: Reference value app: "arccompsdemo"
PathElement {
type: "versionCode"
name: "kWuGZp1knaowVxgKgkxH"
}
in different app partition

Cannot find module './package.json'

node: v6.11.1
package version: 1.1.2

i am receiving the next message when i try to generate a backup Cannot find module './package.json'

module.js:471
throw err;
^

Error: Cannot find module './package.json'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object. (C:\Users\usuario\AppData\Roaming\npm\node_modules\firestore-backup-restore\build\index.js:47:15)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)

Unable to npm install latest version

I was able to install 1.1.2 but that version doesn't include using -B -a2 commands for restoring a backup. When I go to npm install the current version I get a Command failed and Permission denied error.
debug.txt

Running "npm install" fails

When trying to install this package globally, I'm getting an error in npm.

C:\Users\Stephen>npm install -g firestore-backup-restore
npm ERR! prepareGitDep 2> npm WARN install Usage of the `--dev` option is deprecated. Use `--only=dev` instead.
npm ERR! prepareGitDep npm ERR! write after end
npm ERR! prepareGitDep npm ERR! write after end
npm ERR! prepareGitDep
npm ERR! prepareGitDep npm ERR! A complete log of this run can be found in:
npm ERR! prepareGitDep npm ERR!     C:\Users\Stephen\AppData\Roaming\npm-cache\_logs\2018-07-27T06_10_40_133Z-debug.log
npm ERR! prepareGitDep
npm ERR! code ENOPACKAGEJSON
npm ERR! package.json Non-registry package missing package.json: @google-cloud/firestore@github:googleapis/nodejs-firestore.
npm ERR! package.json npm can't find a package.json file in your current directory.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Stephen\AppData\Roaming\npm-cache\_logs\2018-07-27T06_10_59_431Z-debug.log

The debug log file contents are here:

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'install',
1 verbose cli   '-g',
1 verbose cli   'firestore-backup-restore' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session fdc375477b115df1
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 http fetch GET 200 https://registry.npmjs.org/firestore-backup-restore 301ms
8 silly pacote tag manifest for firestore-backup-restore@latest fetched in 396ms
9 timing stage:loadCurrentTree Completed in 922ms
10 silly install loadIdealTree
11 silly install cloneCurrentTreeToIdealTree
12 timing stage:loadIdealTree:cloneCurrentTree Completed in 1ms
13 silly install loadShrinkwrap
14 timing stage:loadIdealTree:loadShrinkwrap Completed in 1ms
15 silly install loadAllDepsIntoIdealTree
16 silly resolveWithNewModule [email protected] checking installable status
17 http fetch GET 200 https://registry.npmjs.org/babel-preset-env 239ms
18 http fetch GET 304 https://registry.npmjs.org/colors 246ms (from cache)
19 http fetch GET 304 https://registry.npmjs.org/babel-runtime 251ms (from cache)
20 silly pacote range manifest for colors@^1.1.2 fetched in 256ms
21 silly resolveWithNewModule [email protected] checking installable status
22 silly pacote range manifest for babel-runtime@^6.26.0 fetched in 261ms
23 silly resolveWithNewModule [email protected] checking installable status
24 http fetch GET 304 https://registry.npmjs.org/mkdirp 259ms (from cache)
25 http fetch GET 304 https://registry.npmjs.org/json-stable-stringify 262ms (from cache)
26 http fetch GET 304 https://registry.npmjs.org/babel-cli 267ms (from cache)
27 http fetch GET 200 https://registry.npmjs.org/commander 267ms
28 silly pacote range manifest for mkdirp@^0.5.1 fetched in 266ms
29 silly resolveWithNewModule [email protected] checking installable status
30 silly pacote range manifest for babel-preset-env@^1.6.1 fetched in 273ms
31 silly resolveWithNewModule [email protected] checking installable status
32 silly pacote range manifest for json-stable-stringify@^1.0.1 fetched in 270ms
33 silly resolveWithNewModule [email protected] checking installable status
34 silly pacote range manifest for babel-cli@^6.26.0 fetched in 277ms
35 silly resolveWithNewModule [email protected] checking installable status
36 silly pacote range manifest for commander@^2.11.0 fetched in 273ms
37 silly resolveWithNewModule [email protected] checking installable status
38 http fetch GET 200 https://registry.npmjs.org/firebase-admin 273ms
39 silly pacote range manifest for firebase-admin@^5.4.3 fetched in 284ms
40 silly resolveWithNewModule [email protected] checking installable status
41 http fetch GET 304 https://registry.npmjs.org/babel-preset-flow 985ms (from cache)
42 silly pacote range manifest for babel-preset-flow@^6.23.0 fetched in 987ms
43 silly resolveWithNewModule [email protected] checking installable status
44 verbose prepareGitDep undefined: installing devDeps and running prepare script.
45 error prepareGitDep 2> npm WARN install Usage of the `--dev` option is deprecated. Use `--only=dev` instead.
45 error prepareGitDep npm ERR! write after end
45 error prepareGitDep npm ERR! write after end
45 error prepareGitDep
45 error prepareGitDep npm ERR! A complete log of this run can be found in:
45 error prepareGitDep npm ERR!     C:\Users\Stephen\AppData\Roaming\npm-cache\_logs\2018-07-27T06_10_40_133Z-debug.log
46 silly fetchPackageMetaData error for @google-cloud/firestore@github:googleapis/nodejs-firestore Non-registry package missing package.json: @google-cloud/firestore@github:googleapis/nodejs-firestore.
47 timing stage:rollbackFailedOptional Completed in 1ms
48 timing stage:runTopLevelLifecycles Completed in 78846ms
49 verbose stack Error: Non-registry package missing package.json: @google-cloud/firestore@github:googleapis/nodejs-firestore.
49 verbose stack     at BB.join (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\pacote\lib\finalize-manifest.js:160:23)
49 verbose stack     at tryCatcher (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
49 verbose stack     at Holder$5._callFunction (eval at generateHolderClass (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\join.js:92:16), <anonymous>:14:44)
49 verbose stack     at Holder$5.checkFulfillment (eval at generateHolderClass (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\join.js:92:16), <anonymous>:29:30)
49 verbose stack     at Promise.eval (eval at thenCallback (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\join.js:14:16), <anonymous>:6:20)
49 verbose stack     at Promise._settlePromise (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\promise.js:566:21)
49 verbose stack     at Promise._settlePromise0 (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\promise.js:614:10)
49 verbose stack     at Promise._settlePromises (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\promise.js:693:18)
49 verbose stack     at Promise._fulfill (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\promise.js:638:18)
49 verbose stack     at Object.<anonymous> (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\bluebird\js\release\nodeback.js:42:21)
49 verbose stack     at emitNone (events.js:111:20)
49 verbose stack     at Object.emit (events.js:208:7)
49 verbose stack     at Object.Parser.on._ (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\tar\lib\parse.js:75:14)
49 verbose stack     at emitTwo (events.js:126:13)
49 verbose stack     at Object.emit (events.js:214:7)
49 verbose stack     at Object.[emit] (C:\Users\Stephen\AppData\Roaming\nvm\v8.11.2\node_modules\npm\node_modules\tar\lib\parse.js:229:12)
50 verbose cwd C:\Users\Stephen
51 verbose Windows_NT 10.0.15063
52 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "-g" "firestore-backup-restore"
53 verbose node v8.11.2
54 verbose npm  v5.10.0
55 error code ENOPACKAGEJSON
56 error package.json Non-registry package missing package.json: @google-cloud/firestore@github:googleapis/nodejs-firestore.
57 error package.json npm can't find a package.json file in your current directory.
58 verbose exit [ 1, true ]

Node version: 8.11.2
NPM version: 5.10.0

Anyone know how to solve? (For now, I'll try cloning the repo and running npm install from there.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.