Python package use to streamline data transfer and credential management
AWS Secret and Client Secret must be added to environment variables before use.
pip install reagan
or
python -m pip install reagan
Manages interactions with SQL Server.
Credentials
Managed in Amazon AWS Parameter Sotre.
/sqlserver/[server alias] = [SQL Server Connection String]
Functions (see documentation within functions)
execute - Executes a DML statement
to_df - Loads results from a sql query to a pandas dataframe
to_list - Loads results from a sql query to a list
to_dict - Executes query and returns results into a dictionary object with user specified key/valuesw
get_scalar - Loads a single result from a sql query
to_sql - Dumps data from a pandas dataframe to a sql table
Examples
from reagan import SQLServer
# establish connection by instanciating the class
ss = SQLServer([server alias])
# forumlate query
query = '''
SELECT TOP (20) *
FROM sys.tables
WHERE Create_Date BETWEEN '[START DATE]' AND '[END DATE]'
'''
# set any query parameters
replacements = {
'[START DATE]' : '2019-01-01',
'[END DATE]' : '2019-02-01',
}
# transfer from SQL Server to memory (pandas DataFrame)
df = ss.to_df(query=query, replacements=replacements)
Manages interactions with PostgreSQL.
Credentials
Managed in AWS Parameter Store
/postgres/[server alias] = [psql connection dict]
Functions (see documentation within functions)
to_df - Loads results from a sql query to a pandas dataframe
to_sql - Dumps data from a pandas dataframe to a sql table
Examples
from reagan import PSQL
# establish connection by instanciating the class
ps = PSQL('102')
# forumlate query
query = '''
SELECT *
FROM carat_gm.dcm_date
WHERE Date BETWEEN '[START DATE]' AND '[END DATE]'
LIMIT 1000
'''
# set any query parameters
replacements = {
'[START DATE]' : '2019-01-01',
'[END DATE]' : '2019-02-01',
}
# transfer from SQL Server to memory (pandas DataFrame)
df = ss.to_df(query=query, replacements=replacements)
Manages interactions with the DCM API using Google's Python libraries.
Credentials
Managed in AWS Parameter Store. Interactions with DCM made via service account.
/dcm/service_account_path = [path to the servie account .json file]
Functions
list - Makes a list call to DCM and returns the object in the form of a list of dictionaries.
update - Makes an update call to DCM.
patch - Makes a patch call to DCM.
to_df - Makes a list call to DCM then converts that to a pandas dataframe.
set_profile_id - Populate the DCM profileId based on the network you select.
report_to_df - Pulls a report to a pandas dataframe.
Examples
from reagan import DCMAPI
# first connect to DCM via service account by instanciating the class
dcm = DCMAPI()
# set the object specifications
obj = "ads"
arguments = {"active": True}
columns = ["id", "name", "placementAssignments_placementId"]
all = False
# Pulling the cities to raw dcm object
ads = dcm.list(obj, arguments=arguments, all=all)
# Pulling a dataframe of the cities
df_ads = dcm.to_df(obj=obj, columns=columns, all=all, arguments=arguments)
Manages interactions with the SA360 API using Google's Python libraries.
Credentials
Managed in AWS Parameter Store. Interactions with DCM made via service account.
/sa360/service_account_path = [path to the servie account .json file]
Functions
reports_to_df - Makes a list call to DCM and returns a generator that yields a pandas dataframe with 1000000 rows with the report specifications
Examples
from reagan import SA360
# first connect to DCM via service account by instanciating the class
sa = SA360()
# set the report specifications
report_type = 'campaign'
agency_id = 123456
columns = ['campaignId','campaign','campaignStartDate','campaignEndDate']
# Loop through the generator to get files
for df in sa.reports_to_df(report_type = report_type, agency_id = agency_id, columns=columns):
print(df.head())
Manages interactions with the Google Drive API using Google's Python libraries.
Credentials
Added in version 1.4.0
Managed in AWS Parameter Store. Interactions with DCM made via service account. The service account path is hardcoded to point to the json file located on 102.
/drive/service_account_path = [path to the servie account .json file]
Functions
retrieve_all_files - Makes a list call to the API and returns all of the files shared with the service account.
download_file - Makes an update call to DCM.
Examples
from reagan import Drive
# first connect to DCM via service account by instanciating the class
dr = Drive()
# Set the parameters
file_id = '16w136pklr3LjDf67PSXSo_GzE4Ey_bYg'
path = "C:\\Users\\Public\\Downloads\\hi.gif"
# Download the file
dr.download_file(file_id, path)
Manages interaction with the Smartsheet API via the smartsheet-python-sdk module
Credentials
Connections to the Smartsheet API are made using a bearer token generated through the smartsheet developer UI.
/smartsheets/bearer_token = [bearer token]
Functions
sheet_to_df - Makes GET call for Sheets and returns the object in a DataFrame. Through some manipulation in the raw json object, each row in the DataFrame will correspond to a cell in the DataFrame. It will have a row number and column number to identify the position on the sheet.
discussions_to_df - Makes GET call for Discussions and returns the object in a DataFrame
get_attachment_url - Makes GET call for Attachments and returns the url of the attachment that was generated. As of this update, per the Smartsheet API documentation the link will expire after two minutes.
attachment_to_df - Currently only supports attachments in Excel. First uses the get_attachments_url method to get the url for the attachment. Then returns a dictionary object with each key as the sheet name in the attachment and the corresponding DataFrame as the value.
Examples
from reagan import SmartsheetAPI
# First instanciate the object to ensure the bearer token is pulled in
ss = SmartsheetAPI()
# Set the sheet id for which you want to pull
sheet_id =
# Pull a dataframe for that sheet
df = ss.sheet_to_df(sheet_id)
Manages interaction with the Google Cloud Platorm (as of now only supports compute instances)
Credentials
Managed in AWS Parameter Store. Interactions with DCM made via service account.
/gcp/service_account_path = [path to the servie account .json file]
/gcp/project = [default project to use]
/gcp/zone = [default zone]
/gcp/region = [default region]
Functions
create_instance - Creates a boots up a compute instance on the GCP. Only supports limited functionality
list_to_df - Lists all of the Compute Instances currently running
delete_instance - Deletes an instances on GCP
Examples
from reagan import GCP
# First instanciate the object to pull in default settings
ss = GCP()
# Set compute instance parameters
name = "test-instance1"
machine_type = "g1-small"
source_disk_image = "https://www.googleapis.com/compute/v1/projects/us-gm-175021/global/images/adstxt-image"
# Make the API call to create the instance
instance = gcp.create_instance(name=name,machine_type=machine_type,source_disk_image=source_disk_image)
Manages interaction with the Microsoft DevOps API
Credentials
Managed in AWS Parameter Store. Interactions with DevOps made via user account.
/devops/personal_access_token = [Generated user access token]
/devops/organization = [DevOps Organization Name]
Functions
to_df - Pulls in objects to a pandas dataframe
Examples
from reagan import DevOpsAPI
# First instanciate the object to pull in default settings
dva = DevOpsAPI()
# Pull in the list of all projects
df = dva.to_df(obj = 'projects')