Comments (4)
This is expected behavior; it's how the BigQuery API is defined. If there are insertion errors not related to transport, the call itself will succeed, and the errors are returned as part of the response. See https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll#response for more info on the response data type. (If there are transport-related errors such as authentication failed or network problems, then you should get an {:error, reason}
response.)
from elixir-google-api.
Thanks @dazuma for the fast feeback. The question is, can I rely on "insert_errors: nil" to be sure that no data was inserted on BigQuery ? Or... mabye, are there cases that we can insert data, even with errors ? I ask, because I am writing a transaction feature to have sync data between BQ and Postgres, and in case BQ fails, I have to rollback Postgres
from elixir-google-api.
If you get a success response, and insert_errors
is set to an empty list, then yes, you can rely on the rows having been inserted correctly. Otherwise, the insert_errors
list will have entries for the rows that failed. Note that it is possible for a partial success to have taken place: rows that failed will have a corresponding entry in the insert_errors
list, while rows were successfully inserted will not. For more details, I suggest studying https://cloud.google.com/bigquery/troubleshooting-errors#streaming
One other thing to be aware of if data consistency is critical. If you get a HTTP error or some other condition that results in an {:error, reason}
response, then the actual end state may be indeterminate. (For example, it is possible that all the inserts succeeded, but due to network issues, the response never made it back to you.) See https://cloud.google.com/bigquery/streaming-data-into-bigquery#dataconsistency for some techniques you can use to handle this case.
from elixir-google-api.
Closing. Feel free to re-open if your question hasn't been answered yet.
from elixir-google-api.
Related Issues (20)
- WigWamWoodys HOT 1
- The task "google_apis.auth" could not be found HOT 3
- elixir-google-api broke with latest gcloud pub/sub emulator 0.7.1
- Speech: support v1p1beta1 HOT 2
- The Hex Docs are not being built HOT 2
- STOP GENERATING THOUSANDS OF MODULES!!! HOT 7
- Recommend configuration of Tesla adapter in README
- Copying Object in GCS
- Retail version does not match the doc HOT 5
- google_api_play_integrity package is not available on hex.pm HOT 1
- Version 0.25.0 of `dfa_reporting` is missing on hex.pm
- Unclear attribute, transaction, for GoogleApi.Firestore.V1.Model.CommitRequest
- Is this still being maintained? HOT 1
- Synthesis failed for IDS HOT 33
- Is the new Google Identity's API available in this library? HOT 1
- GoogleCloudMlV1_PredictRequest Mount Wrong URL HOT 1
- Empty API References Page for v0.16.1 release for Google API {Spanner, Notebooks, Manufacturers} HOT 1
- Synthesis failed for Transcoder HOT 10
- is batch uploads supported? HOT 1
- dfareporting v4 has been released HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elixir-google-api.