google-marketing-solutions / argon Goto Github PK

I've seen there was an issue related to this but was closed as no answer was given and I'd like to re open the discussion cause I'm having the same problem.

_"The POST works and a table is created with the correct header/files but there is no data inserted into the table.

Have tried this on a few Reach reports with the same issue and confirmed there is indeed data with both single and multiple file flag enabled."_

I can confirm the service account has access to the report and I can see the File ID's being processed in the logs. I guess the problem has to do with the last couple lines of the reports where it says "Grand Total" and the 2 following lines, which makes the function "crash" when it compares the schema to the one in BQ and therefore it doesn't ingests any data.

Please see attached a screenshot of the reach reports last lines and the error in the CF logs.

I have been trying to redeploy for the last 3 days, without success. The Cloud Function Logs shows this:

Provided module can't be loaded.
Did you list all required modules in the package.json dependencies?
Detailed stack trace: Error: Cannot find module 'split2'
Require stack:
 - /workspace/argon.js
 - /workspace/index.js
 - /layers/google.nodejs.functions-framework/functions-framework/node_modules/@google-cloud/functions-framework/build/src/loader.js
 - /layers/google.nodejs.functions-framework/functions-framework/node_modules/@google-cloud/functions-framework/build/src/main.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
at Function.Module._load (internal/modules/cjs/loader.js:667:27)
at Module.require (internal/modules/cjs/loader.js:887:19)
at require (internal/modules/cjs/helpers.js:74:18)
at Object.<anonymous> (/workspace/argon.js:19:15)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Module.require (internal/modules/cjs/loader.js:887:19)
Could not load the function, shutting down.

Is the first time I see this issue. I have successfully deploy Argon in the past. The source of the issue seems to be 'split2', imported in https://github.com/google/argon/blob/104181130a3f12a0de2bf5e82935d0419a847992/package.json#L28

There's a report file (in csv) available, however the data.items is always empty (see it after putting some logging in place), and it throws this error.
Cannot read property 'isValid' of undefined.
It happens in the very first condition of for loop.

for (const item of data.items) { if (item.status.match(REPORT_AVAIL_PATTERN)) { const reportDate = item.dateRange.endDate; if (reportDate in lookbackDates) { reportFiles[item.id] = item; delete lookbackDates[reportDate]; // remove found dates } latestDate = DateTime.fromFormat(reportDate, DATE_FORMAT); } }

What are we missing here?

Hi there,

As Argon is currently still using V1 of the API which will be replaced by 1.1. on April 15th 2021, will Argon get updated?
Please see the message we received below.

Thank you! :)

As previously announced, we will be sunsetting these deprecated services of the API on April 15, 2021. Requests to these services will no longer work after this date. To avoid an interruption in service, you must migrate to either a newer version of the DBM API or the Display & Video 360 (DV360) API, depending on the services you currently use.

If you are using the DBM API version 1 Reporting service, you must migrate to the DBM API version 1.1 Reporting service. If you are using any version of the DBM API SDF Download or Line Item service, you must migrate to the DV360 API.
To learn about changes between versions and get tips for migrating, visit the API developer site and read the Google Ads Developer Blog post about this upcoming sunset. Also consider subscribing to the Developer Blog to stay up to date about new releases, upcoming sunsets, and changes to the DBM API or DV360 API.
If you have technical questions regarding new versions of the API, please reach out via the support contact form.
Sincerely,
The DoubleClick Bid Manager API Team

The POST works and a table is created with the correct header/files but there is no data inserted into the table.

Have tried this on a few Reach reports with the same issue and confirmed there is indeed data with both single and multiple file flag enabled.

From time to time (≈ 2 time a week) I get an error in my logs, that GaxiosService is unavailable.
As Argon is running multiple times a day, my CM reports get imported eventually.
Is there something I can do to prevent this error or is this simply cause by some 3rd party services which ar not reachable from time to time?

I have deployed the app, however, calls to the service with cloud scheduler keep on failing with the following error.
ERROR: Provide DCM Profile ID and Account Email ID.
I put some logging in place to see what payload is actually received in App Engine, that shows null (mind you I do send the request with body as mentioned in the instructions).
I am pretty certain that all the requirements for this app to run are in place. Has anybody else got this app to work?

Hi All,

Before deploying the function, what should the index.js file contain?
Here is what I indicated for the package.json :

  "name": "argon",
  "version": "0.0.1",
  "description": "DCM Reporting to BigQuery connector",
  "main": "app.js",
  "engines": {
    "node": ">=10"
  },
  "scripts": {
    "start": "node app.js",
    "start-dev": "nodemon app.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Google Inc.",
  "license": "Apache-2.0",
  "dependencies": {
    "@google-cloud/bigquery": "^2.0.6",
    "google-auth-library": "^3.0.1",
    "hapi": "^18.0.0",
    "luxon": "^1.10.0"
  },
  "devDependencies": {
    "eslint": "^5.12.1",
    "eslint-config-google": "^0.11.0",
    "nodemon": "^1.18.9"
  }
}
but I don't know what to write in the index.js
Do you have an idea?

Thanks a lot !

Hi,
I had to add node in the script part, otherwise after deploying, it App Engine will always say
/bin/sh: 1: exec: app.js: not found
So the final working package.json looked like below.

{
  "name": "argon",
  "version": "0.0.1",
  "description": "DCM Reporting to BigQuery connector",
  "main": "app.js",
  "engines": {
    "node": ">=10"
  },
  "scripts": {
    "start": "node app.js",
    "start-dev": "nodemon app.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Google Inc.",
  "license": "Apache-2.0",
  "dependencies": {
    "@google-cloud/bigquery": "^2.0.6",
    "google-auth-library": "^3.0.1",
    "hapi": "^18.0.0",
    "luxon": "^1.10.0"
  },
  "devDependencies": {
    "eslint": "^5.12.1",
    "eslint-config-google": "^0.11.0",
    "nodemon": "^1.18.9"
  }
}

Hi,
Thanks for the sharing !

Do you know which SQL query would work in order to match CM and DV data ?
Taking for example that the Placement Name CM and the Line Item DV would be the same so that the data match.
Thanks

Long time watcher, first time caller!

This project would ease my workload a lot setting up our internal reporting, so hoping to get some pointers for getting it working. All help is appreciated :)

I've tried to set up Argon based on the setup guide, but are hitting a wall when running the function (500 error on API). My goal is to set up some automated DV360 reports to BQ, for use in Data Studio.

Things I've done:

• I've created the service account, set it up with our DV360 account (reporting only), created a report, and shared it with the service account (link).
• Enabled DBM and DCM/DFA API's in the GCP project (The API console reports requests to DBM API)
• Set up the dataset
• Deployed the cloud function without errors.
• Set up at scheduler based on the instructions.
• Tried to trigger the scheduler, but it failed.
• Tried to repeat everything a couple of times.

The result I get
So I went into the testing tab on the cloud function, tried to use the same triggering event to trace what could be wrong. The code seems to go through the steps ok until it hits the API client where it gets a 500 error. See the attached images.

What could I be missing?