singer-io / tap-facebook Goto Github PK

View Code? Open in Web Editor NEW

110.0 110.0 144.0 962 KB

Singer.io tap for Facebook Marketing API

License: GNU Affero General Public License v3.0

Python 100.00%

singer tap

tap-facebook's People

Contributors

Stargazers

Watchers

Forkers

awbauer nareshkorruggk plenadatadave kufton stvhanna domb16 cubedevinc cyan-zheng lightyoruichi ssdzd bradcosine juwce juan-m12i siddharthsudheer vrixyz naeemahaz avinashkadam cambly aiguofer josesaribeiro rechall chisomekweani lullcode databill86 joaopcoelho bachtsevanis rthirumulu kapluni shandyooo joe-heffer-mpe kobakhit ajludwikowski dreamdata-io kayvink vitoravancini tsrivishnu johncarlov06 homeruntech andrearuffini scentbird popsa-hq bonobos fctester9 kartini-tech manavkohli admariner tydo jakedavies tomzouaoui whalyapp kalimuthu123 mohitverma24 tomdxb0004 charbelrseif goes-funky pblankley beatchain adswerve chalenge ivanovzlatan2 auyer yingwang-clare maxlcoder sherifnada aledbf mariapantea rkdev pawan-blotout isabella232 thiriet newguylhl angeldimov sabaum gh-pankaj minhphuc0101 ricardosllm monkidea mauasoliveira ruggable sumoheavy explorermarco explore-ai jhowliu wheelhousedev aimutopia spacecowboy joel-ruggable tonypro aboliv gary-beautypie sourcetable datalized swifilaboroka alexasteiner srush-inc juniorexperience vibeus affcool ks3057 bodd114

tap-facebook's Issues

Object of type 'Decimal' is not JSON serializable.

I'm trying to retrieve "spend" data for stream 'ads_insights', but this error is raised. I've tried to change the data type to string in my properties file, but the result is the same.

Add SKAN conversion windows to ALL_ACTION_ATTRIBUTION_WINDOWS

There are two conversion window options in the Meta Insights API which are not available in this tap: skan_view & skan_click. Without using these options, certain action values no longer seem to be available e.g. mobile_app_install and app custom events.

Deleted/Archived Ads are not included

I believe you have to add something like this:

Apply a filtering for all delivery_info states.
Then you get all campaigns / adsets / ads, even they were deleted or archived.

'filtering': [{
"field": "ad.delivery_info",
"operator": "IN",
"value": ["active", "archived", "completed", "limited", "not_delivering", "not_published", "pending_review", "permanently_deleted", "recently_completed", "recently_rejected", "rejected", "scheduled", "inactive"]
}]
or

'filtering': [{
"field": "adset.delivery_info",
"operator": "IN",
"value": ["active", "archived", "completed", "limited", "not_delivering", "not_published", "pending_review", "permanently_deleted", "recently_completed", "recently_rejected", "rejected", "scheduled", "inactive"]
}]
or

'filtering': [{
"field": "campaign.delivery_info",
"operator": "IN",
"value": ["active", "archived", "completed", "limited", "not_delivering", "not_published", "pending_review", "permanently_deleted", "recently_completed", "recently_rejected", "rejected", "scheduled", "inactive"]
}]

(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.

I am trying to download data from one of the ad accounts and get this error. I tried cloning and updating facebook_business version to 5.0.1 and then reinstalling tap-facebook. It worked and that fixed the issue. Below is the original error. Let me know If I can provide more details.

tap-facebook -c config.json -p properties.json

ERROR 

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v4.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "AXCdZoi0hBpoXGbHE3nXKXb"
      }
    }
Traceback (most recent call last):
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/tap_facebook/__init__.py", line 682, in main
    main_impl()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/tap_facebook/__init__.py", line 663, in main_impl
    accounts = user.get_ad_accounts()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/adobjects/user.py", line 537, in get_ad_accounts
    return request.execute()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 663, in execute
    cursor.load_next_page()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 830, in load_next_page
    params=self.params,
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 336, in call
    raise fb_response.error()
facebook_business.exceptions.FacebookRequestError: 

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v4.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "AXCdZoi0hBpoXGbHE3nXKXb"
      }
    }

CRITICAL 
CRITICAL 
CRITICAL   Message: Call was not successful
CRITICAL   Method:  GET
CRITICAL   Path:    https://graph.facebook.com/v4.0/me/adaccounts
CRITICAL   Params:  {'summary': 'true'}
CRITICAL 
CRITICAL   Status:  400
CRITICAL   Response:
CRITICAL     {
CRITICAL       "error": {
CRITICAL         "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.",
CRITICAL         "type": "OAuthException",
CRITICAL         "code": 2635,
CRITICAL         "fbtrace_id": "AXCdZoi0hBpoXGbHE3nXKXb"
CRITICAL       }
CRITICAL     }
Traceback (most recent call last):
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/bin/tap-facebook", line 8, in <module>
    sys.exit(main())
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/tap_facebook/__init__.py", line 690, in main
    raise e
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/tap_facebook/__init__.py", line 682, in main
    main_impl()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/tap_facebook/__init__.py", line 663, in main_impl
    accounts = user.get_ad_accounts()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/adobjects/user.py", line 537, in get_ad_accounts
    return request.execute()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 663, in execute
    cursor.load_next_page()
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 830, in load_next_page
    params=self.params,
  File "/Users/kobakhitalishvili/.local/share/virtualenvs/facebook_ads_api-DZzLcf3-/lib/python3.6/site-packages/facebook_business/api.py", line 336, in call
    raise fb_response.error()
facebook_business.exceptions.FacebookRequestError: 

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v4.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "AXCdZoi0hBpoXGbHE3nXKXb"
      }
    }

There have been too many calls to this ad-account. Wait a bit and try again.

Hello! I have this error after run the tap for a while:

Status: 400
Response:
{
"error": {
"message": "(#80004) There have been too many calls to this ad-account. Wait a bit and try again. For more info, please refer to https://developers.facebook.com/docs/graph-api/overview/rate-limiting.",

What could I do?

State doesn't work

tap-facebook writes empty state lines into the output but there is a bookmark property - date_start in the schema:

"bookmark_properties": [
    "date_start"
  ]

At present tap-facebook doesn't handle state well. But it should.

INFO No properties were selected

Below is my catalog.json file. I'm trying to run the tap using the below command

tap-facebook -c config.json --catalog catalog.json but I keep getting the log saying
INFO No properties were selected

Here is my catalog.json file content:

{
"streams": [

{
  "stream": "ads_insights",
  "tap_stream_id": "ads_insights",
  "additionalProperties": false,
  "schema": {
    "type": [
      "null",
      "object"
    ],
    "properties": {
      "unique_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "action_values": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "outbound_clicks": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_10_sec_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_30_sec_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_p25_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_p50_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_p75_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "video_p100_watched_actions": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "1d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_click": {
              "type": [
                "null",
                "number"
              ]
            },
            "1d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "7d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "28d_view": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            },
            "value": {
              "type": [
                "null",
                "number"
              ]
            }
          }
        }
      },
      "clicks": {
        "type": [
          "null",
          "integer"
        ]
      },
      "date_stop": {
        "type": [
          "null",
          "string"
        ],
        "format": "date-time"
      },
      "ad_id": {
        "type": [
          "null",
          "string"
        ]
      },
      "website_ctr": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "value": {
              "type": [
                "null",
                "number"
              ]
            },
            "action_destination": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_target_id": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            }
          }
        }
      },
      "unique_inline_link_click_ctr": {
        "type": [
          "null",
          "number"
        ]
      },
      "adset_id": {
        "type": [
          "null",
          "string"
        ]
      },
      "frequency": {
        "type": [
          "null",
          "number"
        ]
      },
      "account_name": {
        "type": [
          "null",
          "string"
        ]
      },
      "canvas_avg_view_time": {
        "type": [
          "null",
          "number"
        ]
      },
      "unique_inline_link_clicks": {
        "type": [
          "null",
          "integer"
        ]
      },
      "cost_per_unique_action_type": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "value": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            }
          }
        }
      },
      "inline_post_engagement": {
        "type": [
          "null",
          "integer"
        ]
      },
      "relevance_score": {
        "type": [
          "null",
          "object"
        ],
        "properties": {
          "score": {
            "type": [
              "null",
              "number"
            ]
          },
          "status": {
            "type": [
              "null",
              "string"
            ]
          }
        }
      },
      "campaign_name": {
        "type": [
          "null",
          "string"
        ]
      },
      "inline_link_clicks": {
        "type": [
          "null",
          "integer"
        ]
      },
      "campaign_id": {
        "type": [
          "null",
          "string"
        ]
      },
      "cpc": {
        "type": [
          "null",
          "number"
        ]
      },
      "ad_name": {
        "type": [
          "null",
          "string"
        ]
      },
      "cost_per_unique_inline_link_click": {
        "type": [
          "null",
          "number"
        ]
      },
      "cpm": {
        "type": [
          "null",
          "number"
        ]
      },
      "cost_per_inline_post_engagement": {
        "type": [
          "null",
          "number"
        ]
      },
      "inline_link_click_ctr": {
        "type": [
          "null",
          "number"
        ]
      },
      "cpp": {
        "type": [
          "null",
          "number"
        ]
      },
      "cost_per_action_type": {
        "type": [
          "null",
          "array"
        ],
        "items": {
          "type": [
            "null",
            "object"
          ],
          "properties": {
            "value": {
              "type": [
                "null",
                "string"
              ]
            },
            "action_type": {
              "type": [
                "null",
                "string"
              ]
            }
          }
        }
      },
      "unique_link_clicks_ctr": {
        "type": [
          "null",
          "number"
        ]
      },
      "spend": {
        "type": [
          "null",
          "number"
        ]
      },
      "cost_per_unique_click": {
        "type": [
          "null",
          "number"
        ]
      },
      "adset_name": {
        "type": [
          "null",
          "string"
        ]
      },
      "unique_clicks": {
        "type": [
          "null",
          "integer"
        ]
      },
      "social_spend": {
        "type": [
          "null",
          "number"
        ]
      },
      "reach": {
        "type": [
          "null",
          "integer"
        ]
      },
      "canvas_avg_view_percent": {
        "type": [
          "null",
          "number"
        ]
      },
      "account_id": {
        "type": [
          "null",
          "string"
        ]
      },
      "date_start": {
        "type": [
          "null",
          "string"
        ],
        "format": "date-time"
      },
      "objective": {
        "type": [
          "null",
          "string"
        ]
      },
      "impressions": {
        "type": [
          "null",
          "integer"
        ]
      },
      "unique_ctr": {
        "type": [
          "null",
          "number"
        ]
      },
      "cost_per_inline_link_click": {
        "type": [
          "null",
          "number"
        ]
      },
      "ctr": {
        "type": [
          "null",
          "number"
        ]
      }
    }
  },
  "metadata": [
    {
      "breadcrumb": [

      ],
      "metadata": {
        "table-key-properties": [
          "campaign_id",
          "adset_id",
          "ad_id",
          "date_start"
        ],
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "action_values"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "outbound_clicks"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_10_sec_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_30_sec_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_p25_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_p50_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_p75_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "video_p100_watched_actions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "clicks"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "date_stop"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "ad_id"
      ],
      "metadata": {
        "inclusion": "automatic"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "website_ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_inline_link_click_ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "adset_id"
      ],
      "metadata": {
        "inclusion": "automatic"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "frequency"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "account_name"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "canvas_avg_view_time"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_inline_link_clicks"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_unique_action_type"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "inline_post_engagement"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "relevance_score"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "campaign_name"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "inline_link_clicks"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "campaign_id"
      ],
      "metadata": {
        "inclusion": "automatic"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cpc"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "ad_name"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_unique_inline_link_click"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cpm"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_inline_post_engagement"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "inline_link_click_ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cpp"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_action_type"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_link_clicks_ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "spend"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_unique_click"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "adset_name"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_clicks"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "social_spend"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "reach"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "canvas_avg_view_percent"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "account_id"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "date_start"
      ],
      "metadata": {
        "inclusion": "automatic"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "objective"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "impressions"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "unique_ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "cost_per_inline_link_click"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    },
    {
      "breadcrumb": [
        "properties",
        "ctr"
      ],
      "metadata": {
        "inclusion": "available",
        "selected": "true"
      }
    }
  ]
}

]
}

I have checked my config.json file and made sure that I add selected: true in catalog.json file as well. Could someone guide me in the right direction here?

Where the heck is tap_tester?

All tap tests use tap_tester module. But I can't find it anywhere, on GitHub, on Google, ...
Where can I pull this dependency from?

You are calling a deprecated version of the Ads API. Please update to the latest version: v2.10.

Facebook TaP NOt working...

Config File :
{"start_date":"2016-02-02",
"account_id":"104104556902392",
"access_token":"EAAKPKW4rDdEBAIv4VK6aoiCjAtWDZBGN46W0ZAjQYNjsTQ9L8aRGsfsVbZBgOyjqzhFjs3ktavpvoKQiSKbAZAtQoborQZCnVuxFCzcZCea30x3oq94Dy8fI8x1gEbFAK2dRo6mlZAHJe7dtgY4a580LOILZAM4BhoNFbVZCNXNZBu6OcP2tV9OH1dHnMGGDfwsVbm9I8IwZBwn9nVwZCS6BkOYsE76BgGsehkcZD"}

Properties File :

[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"properties.json" 44L, 1859C
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "country",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "country",
"action_attribution_windows": ["1d_click",
[{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "country",
"action_attribution_windows": ["1d_click",
"breakdowns": null,
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "age_and_gender",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",
"28d_view"] },
{ "level" : "ad",
"actions_breakdowns": ["action_type",
"action_target_id",
"action_destination"],
"breakdowns": "country",
"action_attribution_windows": ["1d_click",
"7d_click",
"28d_click",
"1d_view",
"7d_view",

Error:

Message: Call was not successful
Method: GET
Path: https://graph.facebook.com/v2.8/me/adaccounts
Params: {'summary': 'true'}

Status: 400
Response:
{
"error": {
"message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v2.10.",
"type": "OAuthException",
"code": 2635,
"fbtrace_id": "HyzA5DGRz3Q"
}
}

Extracting less data by Meltano as compared to data in Facebook UI

Hello Team,
We are working on extracting data from Facebook API using the tap-facebook extractor. We were able to extract and load the data but when we are comparing the spend with the Facebook data, (we have a python file that is extracting the data directly from Facebook UI using the Facebook API) it's not matching with the data we have in the UI.
We are missing lot of data and would love some assistance. If someone has faced or can give suggestions on how to resolve this issue that would be great.
Context:
Before (up until mid-December) our queries were showing ~10,000 rows loaded per day and now we're down to ~250 rows loaded per day.
Going off of the Meltano status, all data should be loaded but we're not seeing that when we query the data.
We have used batch sizes: 50, 100, 1000, 3000, 4000, 6000, 8000, 10000 using the variable: max_batch_rows
The pipeline is currently the pipeline is running at an hourly basis.
Total rows
Facebook UI: 5073620
Meltano: 382000
We are using _sdc_batched_at to count the daily number of total rows.
Could this be causing an issue since _sdc_batched_at gets updated?

LLT4

Can't get account objects

The ability to query the me/adaccounts endpoint is missing.

`Campaigns` Stream doesn't have an `ads` field

WARNING Property campaigns.ads is not defined in the facebook_business library

tap-facebook does not reattempt async report retrieval upon encountering a 400 status response

According to this bug report Facebook devs suggest that "<3% is an acceptable ratio and we recommend retries" when async report retrieval fails.

When this issue is encountered, job's will show as 100% complete but attempts to retrieve them return the following scrubbed error:

  Status:  400
  Response:
    {
      "error": {
        "type": "OAuthException",
        "error_user_title": "Loading Async Ads Report Failed",
        "message": "Error accessing adreport job.",
        "code": 2601,
        "error_subcode": 1815107,
        "fbtrace_id": "xxxxxxxxxxx",
        "error_user_msg": "Sorry, the report cannot be loaded successfully. Please check if your job status is completed instead of failed or running before fetching the data.",
        "is_transient": true

Currently the tap hard-fails on the occurrence of these errors, as the retry logic is distinct from the response for "complete" jobs:

eg:

2018-06-22 03:00:14,969Z    tap - INFO Job Not Started, 0% done
2018-06-22 03:00:14,970Z    tap - INFO sleeping for 10 seconds until job is done
2018-06-22 03:00:25,019Z    tap - INFO Job Completed, 100% done
2018-06-22 03:00:25,020Z    tap - INFO "insights" job took 10.3s, ended with status "succeeded"
2018-06-22 03:00:25,964Z    tap - INFO replicated X records from "ads_insights_region" endpoint
2018-06-22 03:00:25,965Z    tap - ERROR 
2018-06-22 03:00:25,965Z    tap - 
2018-06-22 03:00:25,965Z    tap -   Message: Call was not successful
2018-06-22 03:00:25,965Z    tap -   Method:  GET
2018-06-22 03:00:25,966Z    tap -   Path:    https://graph.facebook.com/v2.11/[redacted]/insights?access_token=...................................................................................................................................................................................&limit=25&after=NDkZD
2018-06-22 03:00:25,966Z    tap -   Params:  {}
2018-06-22 03:00:25,966Z    tap - 
2018-06-22 03:00:25,966Z    tap -   Status:  400

tap-facebook does not include the `outbound_clicks` parameter

According to this page of Facebook's documentation, the top-level outbound_clicks parameter will return an action_stats list containing, "The number of clicks on links that take people off Facebook-owned properties."

The outbound_click action_type does not currently appear to be included in the actions sub-table.

tap-facebook isn't returning any data

Hi,

this is the 1st time I'm using Singer, so please be patient with me :)

I'm trying to do something very simple: just pull my Facebook Ads data and save it as a CSV in my local.
These are the commands I'm running:

pyenv virtualenv test-singer
pyenv activate test-singer
pip install --upgrade pip
pip install tap-facebook target-csv
tap-facebook --config facebook_config.json --properties facebook_properties.json --state facebook_state.json | target-csv --config csv_config.json

After I run my tap-facebook command, nothing happens... no file is created in my current directory, and there's no error message or anything, the command run successfully (check with echo $? command) but no file result was generated

These are my JSON files configured:

facebook_config.json

{
    "start_date":"2020-01-01T00:00:00Z",
    "account_id":"XXX",
    "access_token":"XXX"
}

facebook_state.json

{
    "adcreative": "2020-01-01T00:00:00Z",
    "ads": "2020-01-01T00:00:00Z",
    "adsets": "2020-01-01T00:00:00Z",
    "campaigns": "2020-01-01T00:00:00Z",
    "ads_insights": "2020-01-01T00:00:00Z",
    "ads_insights_age_and_gender": "2020-01-01T00:00:00Z",
    "ads_insights_country": "2020-01-01T00:00:00Z",
    "ads_insights_platform_and_device": "2020-01-01T00:00:00Z",
    "ads_insights_region": "2020-01-01T00:00:00Z",
    "ads_insights_dma": "2020-01-01T00:00:00Z",
    "ads_insights_hourly_advertiser": "2020-01-01T00:00:00Z"
}

facebook_properties.json was generated by the following command: tap-facebook -c facebook_config.json --discover > facebook_properties.json

And here's my pip freeze:

aiohttp==2.3.10
async-timeout==3.0.1
attrs==16.3.0
backoff==1.8.0
backports.zoneinfo==0.2.1
certifi==2021.5.30
chardet==3.0.4
ciso8601==2.2.0
curlify==2.2.1
facebook-business==10.0.0
idna==2.7
idna-ssl==1.1.0
importlib-resources==5.2.2
jsonschema==2.6.0
multidict==5.1.0
pendulum==1.2.0
pycountry==20.7.3
python-dateutil==2.8.2
pytz==2018.4
pytzdata==2020.1
requests==2.20.0
simplejson==3.11.1
singer-python==5.10.0
six==1.16.0
tap-facebook==1.15.0
target-csv==0.2.0
typing-extensions==3.10.0.0
tzlocal==3.0
urllib3==1.24.3
yarl==1.6.3
zipp==3.5.0

Oh, and also I know that this should return data, since I already have a pipeline pulling data from Facebook through their API, so it's not because there's no data available ;)

any ideas?

Facebook Ads API updated to v15

Most recent version is v15
https://developers.facebook.com/docs/graph-api/changelog/

update to API v9

Hi there,

Facebook released v9 of their API on November 10. This means that new Facebook apps cannot use this connector unless it is updated to v9. The tap fails with an error like the following:

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v8.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v9.0.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "APOaj0bkymwIFMmFAWNb-W3"
      }
    }

CRITICAL
CRITICAL
CRITICAL   Message: Call was not successful
CRITICAL   Method:  GET
CRITICAL   Path:    https://graph.facebook.com/v8.0/me/adaccounts
CRITICAL   Params:  {'summary': 'true'}
CRITICAL
CRITICAL   Status:  400
CRITICAL   Response:
CRITICAL     {
CRITICAL       "error": {
CRITICAL         "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v9.0.",
CRITICAL         "type": "OAuthException",
CRITICAL         "code": 2635,
CRITICAL         "fbtrace_id": "APOaj0bkymwIFMmFAWNb-W3"
CRITICAL       }
CRITICAL     }

is updating to v9 on the roadmap for this tap?

Filter Ads/Campaigns by updated_time?

I think this might be possible, so I'm leaving this here to check back at some point for an improvement.

The current logic retrieves all Ads/Campaigns and checks their updated_time against the bookmark value. If they pass the test, they are yielded.

It seems like this document describes that you could simply filter on the request - https://developers.facebook.com/docs/marketing-api/ad-rules-getting-started:

       {
            "field": "Ad.updated_time",
            "operator": "GREATER_THAN",
            "value": <bookmark value in epoch seconds int>
        }

Error: You are calling a deprecated version of the Ads API. Please update to the latest version: v7.0

Same issue as #77 but 6.0 is no longer available in newly created apps.

Support time_increment in config file

Hi,
I submitted a feature request for this with Shaun (via email), but wanted to follow up here. We'd really love to see FB's time_increment parameter supported in this tap.

The reason we're looking for this parameter is to support our use of unique-based data. For non-unique measurements (clicks, impressions, spend, etc), the metrics roll up cleanly into a weekly/monthly/yearly view. However for unique-based measures (reach, unique impressions, frequency, etc), the only usable dimension is the interval pulled by time_increment. For us, this renders the unique-based measures essentially useless; we're never looking at them in a daily view.

Here's a gist that sums up the issue. Take a look at the reach of 258,031 from the time_increment_monthly.txt file, for example. There simply isn't any way to replicate that result from the time_increment_1.txt results. This is a huge issue for us because any reporting we'd be running from this data would be "What was our reach over X period?"

Allowing us to set time_increment to the monthly level would be a massive win for us. Currently, our only alternative is manually pulling the data from the Facebook dashboard.

Thanks for all the work on this integration! We're really thrilled with the potential here.

Facebook Ads API Deprecated

The Facebook Ads API version in the tap is v13.0 which is deprecated now. Update to v14.0

IMPORTANT: Facebook Marketing API v4 sunset

I've noticed that the current version of tap-facebook seems to be using version 4.0.2 of the Facebook Marketing API. This API is being sunset on 31st March 2020.

The next version (v5) is also being sunset at the end of May 2020.

Because of this it surely makes sense to skip v5 and move straight to v6? This will, obviously, need to be done before the end of March.

https://developers.facebook.com/docs/graph-api/changelog#marketing-api-changelog

Support get edge/connection data

For example, I want to get activities data of an ad set as described at https://developers.facebook.com/docs/marketing-api/reference/ad-campaign/activities/#Reading

Does currently tap-facebook support this?

Upgrade to Facebook API v3.3

Changelog: https://developers.facebook.com/docs/graph-api/changelog/version3.3#marketing-api

We do not expect any breaking changes based on the changelog.

Steps:
1 . Update the Facebook SDK version to 3.3 in setup.py
2. Run tap-tester tests

pip install fails when building wheel for cisco8601

Hello,

I am trying to install tap-facebook via pip without any luck. I have tried uninstalling/installing xcode CL tools again. Here is the error message:

  Building wheel for ciso8601 (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/jordanburger/tap-facebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"'; __file__='"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-wheel-mqcus5f1
       cwd: /private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/
  Complete output (22 lines):
  running bdist_wheel
  running build
  running build_py
  package init file 'ciso8601/__init__.py' not found (or not a regular file)
  creating build
  creating build/lib.macosx-11-x86_64-3.9
  creating build/lib.macosx-11-x86_64-3.9/ciso8601
  copying ciso8601/__init__.pyi -> build/lib.macosx-11-x86_64-3.9/ciso8601
  copying ciso8601/py.typed -> build/lib.macosx-11-x86_64-3.9/ciso8601
  running build_ext
  building 'ciso8601' extension
  creating build/temp.macosx-11-x86_64-3.9
  Compiling with an SDK that doesn't seem to exist: /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk
  Please check your Xcode installation
  /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -DCISO8601_VERSION=2.1.3 -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/Users/jordanburger/tap-facebook/include -I/usr/local/Cellar/[email protected]/3.9.0_4/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c module.c -o build/temp.macosx-11-x86_64-3.9/module.o
  clang: warning: no such sysroot directory: '/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk' [-Wmissing-sysroot]
  In file included from module.c:1:
  /usr/local/Cellar/[email protected]/3.9.0_4/Frameworks/Python.framework/Versions/3.9/include/python3.9/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for ciso8601
  Running setup.py clean for ciso8601
Failed to build ciso8601
Installing collected packages: ciso8601, backoff, aiohttp, singer-python, pendulum, facebook-business, attrs, tap-facebook
    Running setup.py install for ciso8601 ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/jordanburger/tap-facebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"'; __file__='"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-record-xf3_auzy/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jordanburger/tap-facebook/include/site/python3.9/ciso8601
         cwd: /private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/
    Complete output (22 lines):
    running install
    running build
    running build_py
    package init file 'ciso8601/__init__.py' not found (or not a regular file)
    creating build
    creating build/lib.macosx-11-x86_64-3.9
    creating build/lib.macosx-11-x86_64-3.9/ciso8601
    copying ciso8601/__init__.pyi -> build/lib.macosx-11-x86_64-3.9/ciso8601
    copying ciso8601/py.typed -> build/lib.macosx-11-x86_64-3.9/ciso8601
    running build_ext
    building 'ciso8601' extension
    creating build/temp.macosx-11-x86_64-3.9
    Compiling with an SDK that doesn't seem to exist: /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk
    Please check your Xcode installation
    /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -DCISO8601_VERSION=2.1.3 -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/Users/jordanburger/tap-facebook/include -I/usr/local/Cellar/[email protected]/3.9.0_4/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c module.c -o build/temp.macosx-11-x86_64-3.9/module.o
    clang: warning: no such sysroot directory: '/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk' [-Wmissing-sysroot]
    In file included from module.c:1:
    /usr/local/Cellar/[email protected]/3.9.0_4/Frameworks/Python.framework/Versions/3.9/include/python3.9/Python.h:25:10: fatal error: 'stdio.h' file not found
    #include <stdio.h>
             ^~~~~~~~~
    1 error generated.
    error: command '/usr/bin/clang' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/jordanburger/tap-facebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"'; __file__='"'"'/private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-install-3ccpmtzu/ciso8601_a143c506d2af4869981f91306c278c8f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/vy/2rtm288x7g9_5fxgf7m7srn80000gn/T/pip-record-xf3_auzy/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jordanburger/tap-facebook/include/site/python3.9/ciso8601 Check the logs for full command output.```

Error: You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0

I am testing this using a sandbox account (therefore don't mind sharing the details here). I have created the following config file called facebook_tap_config.json:

{"start_date":"2019-01-01", "account_id":"1014179965585963", "access_token":"EAAIf967aEYcBAHa01KVZAc0yR8SGxn8VMKkEZCGuhTtqUzV66YCWcIsJdUZC14oBAUStQZAic8GhZArfb3GOMZCjFlMZCKEXuGVylvtaxzJdXuBwQ7azA74k1OGQrjSbN76z7mBViw8iauDsr2zprgHwZCy4DvK43lZCyjkaITk8vS9ZBAjpxb44vdZAPZBgFphKtaYZD"}

I have validated that this token works via the Python SDK. However, when I try and run the tap in discovery mode with the following command:

tap-facebook --config facebook_tap_config.json --discover

I receive the following error:

ERROR 
  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v4.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v5.0.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "Ab_nkbQIV95DvMiJHEA2mGK"
      }
    }

Is there a way of changing the API version/base URL? Thanks!

ERROR SingerConfigurationError GET: 400 Message: (#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v10.0.

Hello there,

I'm getting this error when trying to run the discovery mode of the tap-facebook.
Only v10 of FB's API is now available for new apps.

Can you please help?

Thx,
Raphael

'limit' parameter of graphql endpoints needs to be configurable, or backed-off upon receiving 'Please reduce the amount of data you're asking for, then retry your request'

When pulling insights data from the facebook marketing api, we have started getting the following error:

Message: Call was not successful
Method:  GET
Path:    https://graph.facebook.com/v3.3/<1234567890>/insights?access_token=<hidden_token>&limit=25&after=NzQZD
Params:  {}

Status:  500
Response:
{
  "error": {
    "code": 1,
    "message": "Please reduce the amount of data you're asking for, then retry your request"
  }
}

After getting the error, I used the path specified in the error with Postman, and was able to decrease the limit and retrieve all of the data by following the page.next value

Getting no data returned in sync mode, anyone have some insights?

Hi,

I will be honest I am learning singer to get data into Facebook, and look like I am stuck from the start.

I am trying to get data that has had historical spend but currently, it doesn't have any ads running- is this the problem? if so how do I get the historical insights?
Here is my config.json file

{"start_date":"2015-09-01", "end_date":"2018-01-01", "account_id":"88016------095", "access_token":"-------------aub3Pi66R7SN53ElGbXBZBK1q1PjltKcm0jxkizZAzvlZCoZAfGoLOiI9G7ZCaTGXW$
I added the end data after looking at the code, thought it would help, doesn't work both ways

I followed all the instruction properly created a virtual env and installed tap-facebook
tap-facebook -c config1.json -p catalog.json here is the command i try.
I am getting the access token from here
I am getting errors back for example if I change the account or the access token, I do get the error back
I am using the catalog that I got from the discover mode

Any help is appreciated, thanks for helping in advance, if someone has good blog to follow for this it would be great also.

Error with python 3.6

I install tap-facebook in a clean venv with python 3.6.8 and I get the following error running tap-facebook --help

Traceback (most recent call last):
  File "/blah/venv/bin/tap-facebook", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3126, in <module>
    @_call_aside
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3110, in _call_aside
    f(*args, **kwargs)
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3139, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 596, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/blah/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 789, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (attrs 16.3.0 (/blah/venv/lib/python3.6/site-packages), Requirement.parse('attrs>=17.3.0'), {'aiohttp'})

This is my output of pip freeze

aiohttp==3.6.2
async-timeout==3.0.1
attrs==16.3.0
backoff==1.8.0
certifi==2020.6.20
chardet==3.0.4
ciso8601==2.1.3
curlify==2.2.1
facebook-business==8.0.3
idna==2.7
idna-ssl==1.1.0
jsonschema==2.6.0
mock==4.0.2
multidict==4.7.6
pendulum==1.2.0
pycountry==20.7.3
python-dateutil==2.8.1
pytz==2018.4
pytzdata==2020.1
requests==2.20.0
simplejson==3.11.1
singer-python==5.8.1
six==1.15.0
tap-facebook==1.9.4
typing-extensions==3.7.4.3
tzlocal==2.1
urllib3==1.24.3
yarl==1.5.1

Any insight into this one?

"The access token could not be decrypted" Error on Catalog Request

Hi, when trying to generate a catalog with the current master version of tap-facebook I am encountering this error. The credentials used come from Facebook OAuth using the v16.0 version of the Graph API.

`ERROR SingerConfigurationError
GET: 400 Message: The access token could not be decrypted
Traceback (most recent call last):
File "/Users/alec/.virtualenvs/tap-facebook/lib/python3.9/site-packages/tap_facebook-1.18.5-py3.9.egg/tap_facebook/init.py", line 899, in main_impl
accounts = user.get_ad_accounts()
File "/Users/alec/.virtualenvs/tap-facebook/lib/python3.9/site-packages/facebook_business-16.0.2-py3.9.egg/facebook_business/adobjects/user.py", line 455, in get_ad_accounts
return request.execute()
File "/Users/alec/.virtualenvs/tap-facebook/lib/python3.9/site-packages/facebook_business-16.0.2-py3.9.egg/facebook_business/api.py", line 677, in execute
cursor.load_next_page()
File "/Users/alec/.virtualenvs/tap-facebook/lib/python3.9/site-packages/facebook_business-16.0.2-py3.9.egg/facebook_business/api.py", line 841, in load_next_page
response_obj = self._api.call(
File "/Users/alec/.virtualenvs/tap-facebook/lib/python3.9/site-packages/facebook_business-16.0.2-py3.9.egg/facebook_business/api.py", line 350, in call
raise fb_response.error()
facebook_business.exceptions.FacebookRequestError:

Message: Call was not successful
Method: GET
Path: https://graph.facebook.com/v16.0/me/adaccounts
Params: {'summary': 'true'}

Status: 400
Response:
{
"error": {
"message": "The access token could not be decrypted",
"type": "OAuthException",
"code": 190,
"fbtrace_id": "ASGS9BAk2nagYYhOSc_jM86"
}
}`

Digging around online seems to show that this error comes from invalid tokens however I just generated these OAuth tokens today.

Can anyone offer any insight around solving this error?

[Urgent] Empty response

Hello,

I had tried tap-facebook but get empty response. No error. No data. Please help. Currently, I'm working task ETL. It urgent. Thanks.

It's log:

ancs21_ps@cloudshell:~/hello-singer (mediskin-285304)$ pip3 install "https://github.com/singer-io/tap-facebook/archive/master.zip"
ancs21_ps@cloudshell:~/hello-singer (mediskin-285304)$ tap-facebook -c config.json  -d > catalog.json
INFO Loading schemas
INFO Loading schema for adcreative
INFO Loading schema for ads
INFO Loading schema for adsets
INFO Loading schema for campaigns
WARNING Property campaigns.ads is not defined in the facebook_business library
INFO Loading schema for ads_insights
INFO Loading schema for ads_insights_age_and_gender
WARNING Property ads_insights_age_and_gender.age is not defined in the facebook_business library
WARNING Property ads_insights_age_and_gender.gender is not defined in the facebook_business library
INFO Loading schema for ads_insights_country
WARNING Property ads_insights_country.country is not defined in the facebook_business library
INFO Loading schema for ads_insights_platform_and_device
WARNING Property ads_insights_platform_and_device.impression_device is not defined in the facebook_business library
WARNING Property ads_insights_platform_and_device.platform_position is not defined in the facebook_business library
WARNING Property ads_insights_platform_and_device.publisher_platform is not defined in the facebook_business library
WARNING Property ads_insights_platform_and_device.placement is not defined in the facebook_business library
INFO Loading schema for ads_insights_region
WARNING Property ads_insights_region.region is not defined in the facebook_business library
INFO Loading schema for ads_insights_dma
WARNING Property ads_insights_dma.dma is not defined in the facebook_business library
ancs21_ps@cloudshell:~/hello-singer (mediskin-285304)$ tap-facebook -c config.json -p catalog.json
ancs21_ps@cloudshell:~/hello-singer (mediskin-285304)$

Periodically getting the following error: `SingerSyncError GET: 400 Message: Error accessing adreport job.`

I am periodically getting the following error upon extraction:
SingerSyncError GET: 400 Message: Error accessing adreport job.

After an initial look-through troubleshooting the error I think it may have to do with the while condition being met when the async job is not completely done.

A stackoverflow thread says it better than I can.

This thread implies that status != "Job Completed" needs to be changed to status != "Job Completed" or percent_completion < 100 for the following line.

Again, this is an initial take, but if that change is made, you may consider updating this condition as well.

Would anyone happent to have an example of integrating this with a csv target?

Hi, I'm trying to pull data from fb using taps and targets, and have been unsuccessful so far.
I've tried the following:

~/.virtualenvs/tap-facebook/bin/tap-facebook --config config.json --catalog catalog.json | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

~/.virtualenvs/tap-facebook/bin/tap-facebook -c config.json -s state.json --discover | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

But each time I'm getting a blank output. My state.json file looks like this:
{"ads":"2019-01-01T00:00:00Z",
"adcreative":"2019-01-01T00:00:00Z",
"ads_insights":"2019-01-01T00:00:00Z"}

I also have another version of tap-facebook installed on a different virtualenv, and simply use the cmd:

/venv/bin/tap-facebook -c config.json --discover > tap_output.json | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

I get an output in tap_output.csv, which I'm guessing is the schema. But no output in target-csv, any idea how to get that working?

Would really appreciate any help!

pkg_resources.ContextualVersionConflict: (attrs 16.3.0 (/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages), Requirement.parse('attrs>=17.3.0'), {'aiohttp'})

(.venv)$ tap-facebook

Traceback (most recent call last):
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (attrs 20.3.0 (/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages), Requirement.parse('attrs==16.3.0'), {'tap-facebook'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/bin/tap-facebook", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/geofflangenderfer/work/test_tap_facebook/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'attrs==16.3.0' distribution was not found and is required by tap-facebook

"Unsupported get request. Object with ID 'act_XXXXX' does not exist,..."

Getting this error across multiple Stitch connectors for the last several days. Not sure what to do to resolve. I have reauthorized account and confirmed permissions are still set appropriately. Any suggestions appreciated!

`Message: Call was not successful
Method: GET
Path: https://graph.facebook.com/v6.0/act_4XXXXXXXXXXXX4/ads
Params: {'limit': 100, 'filtering': '[{"field":"ad.delivery_info","operator":"IN","value":["active","archived","completed"]},{"field":"ad.updated_time","operator":"GREATER_THAN","value":1582578140}]', 'summary': 'true', 'fields': 'updated_time,id'}

Status: 400
Response:
{
"error": {
"type": "GraphMethodException",
"message": "Unsupported get request. Object with ID 'act_4XXXXXXXXXXXX4' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api",
"code": 100,
"fbtrace_id": "AkC_QOl81J_K3geIwCnMSj9",
"error_subcode": 33
}
}`

singer.transform.SchemaMismatch: Errors during transform

Hi, im getting the following error when try to run tap-facebook. I followed each step of the Quick start guide
../site-packages/singer/transform.py", line 95, in transform raise SchemaMismatch(self.errors)

singer.transform.SchemaMismatch: Errors during transform updated_time: 2018-07-03T21:45:19-0400 does not match {'type': ['string', 'null'], 'format': 'date-time', 'inclusion': 'automatic'}

Can anyone help me please.

Property ads_insights_age_and_gender.date_{start,stop} missing 'date-time'

The property definition for ads_insights_platform_and_device.date_start contains "format" "date-time" and the resulting BigQuery field has type TIMESTAMP. For a similar schema, ads_insights_age_and_gender.date_start this "format" is missing and the resulting BigQuery field has type STRING. The "format" is also missing for ads_insights_age_and_gender.date_stop .

Documentation does not show how the properties.json is formatted

Allow to sync data from multiple Facebook Marketing Account

Currently, the tap-facebook only allow fetching data from one account_id
Can we change the config so that multiple account_ids can be passed

Getting results/ actions metrics accurately

Hi - I have pulled data from FB ads, to my bigquery, I have removed the multiple batches issue, so the numbers are fairly accurate.

My spend/ impressions counts are similar to FB HOWEVER...

I am trying to get counts for "website leads", and "landing page views" etc. It seems I should be able to get it using a combination of the columns seen below

Attached are the columns I believe will let me get the right numbers, but everytime I do the filtering of actions.1d_click by actions.actions_type = "landing_page_view" the result actions.1d_click is waay off, worse when I use a table and include "spend" the spend result is completely off more than 10x the number.

Would like a simple breakdown of how to get the right result/action counts in a table with other metrics such as spend.

Please note I am accessing the data through data studio and tableau and getting the same issues

Access to "by hour" insights

Hi, I already left a message in the Singer Slack space, but I am reposting here in case it's a better way to reach out.

I've been exploring ways to get Facebook ads data with using the hourly breakdown (didn't find it neither Stitch nor tap-facebook).

I already made a version that works for me, and I was wondering if I can and should create a fork and submit a PR to get that incorporated in the project (maybe there is a rationale why this wasn't done in the past, so before doing anything I wanted to ask what would be the right steps).

In the middle of coming to my solution I had to do a small tweak to the schema files for the existing reports, so I'd like to talk to a maintainer to understand if I should have actually done something different to correct the problem I was having (basically I was getting errors when parsing a particular field that expected a date-time and facebook was sending a date, so had to edit the tap-facebook schemas in my local copy to make it work).

Looking forward to hearing from you.
Best,
juan

Calling a deprecated version of the Ads API

When I ran the tap-facebook today (Sept 13th, 2018), I got the following error:

CRITICAL 
CRITICAL 
CRITICAL   Message: Call was not successful
CRITICAL   Method:  GET
CRITICAL   Path:    https://graph.facebook.com/v3.0/me/adaccounts
CRITICAL   Params:  {'summary': 'true'}
CRITICAL 
CRITICAL   Status:  400
CRITICAL   Response:
CRITICAL     {
CRITICAL       "error": {
CRITICAL         "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v3.1.",
CRITICAL         "type": "OAuthException",
CRITICAL         "code": 2635,
CRITICAL         "fbtrace_id": "DLTgDI8GfSs"
CRITICAL       }
CRITICAL     }
Traceback (most recent call last):
  File "/root/venv/bin/tap-facebook", line 11, in <module>
    sys.exit(main())
  File "/root/venv/lib/python3.6/site-packages/tap_facebook/__init__.py", line 660, in main
    raise e
  File "/root/venv/lib/python3.6/site-packages/tap_facebook/__init__.py", line 652, in main
    main_impl()
  File "/root/venv/lib/python3.6/site-packages/tap_facebook/__init__.py", line 633, in main_impl
    accounts = user.get_ad_accounts()
  File "/root/venv/lib/python3.6/site-packages/facebook_business/adobjects/user.py", line 234, in get_ad_accounts
    return request.execute()
  File "/root/venv/lib/python3.6/site-packages/facebook_business/api.py", line 657, in execute
    cursor.load_next_page()
  File "/root/venv/lib/python3.6/site-packages/facebook_business/api.py", line 801, in load_next_page
    params=self.params,
  File "/root/venv/lib/python3.6/site-packages/facebook_business/api.py", line 330, in call
    raise fb_response.error()
facebook_business.exceptions.FacebookRequestError: 

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v3.0/me/adaccounts
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v3.1.",
        "type": "OAuthException",
        "code": 2635,
        "fbtrace_id": "DLTgDI8GfSs"
      }
    }

Any thoughts?

By the way, I have never run this tap before.

Thank you,
Ales

Return exit code 0 on failure

Description of the bug

Calling CLI tap-facebook returns exit code 0 even when it fails.

Expected behavior

Failed call should return exit code 1.

This is important for instance when calling the tool from airflow: airflow thinks the task succeeded when it did not.

Investigation

This is caused by litl/backoff#72

Proposal

Set on_giveup to a function raising an exception.

TypeError: list indices must be integers or slices, not str

We are getting an error when running this tap:

File "/Users/XXX/Documents/bigquery/venv/lib/python3.6/site-packages/singer/catalog.py", line 87, in from_dict
    for stream in data['streams']:
TypeError: list indices must be integers or slices, not str

Default settings so far but still delivering problems. Does someone know why?

Redesign <singer-io\∆