apigee / devrel

Common solutions and tools developed for Apigee

Apigee DevRel

Common solutions and tools developed by Apigee.

References

This folder contains reference solutions across a variety of Apigee products. It is expected that these solutions are used as an illustration or extended further to fit a particular use case.

• Common Shared Flows - Reusable API Policies including Security, Traffic Management, Error Handling and CORS
• Custom Java Extensions in Apigee - A reference project that includes a Java Extension
• GCP Service Account Shared Flow - A shared flow to obtain access tokens for GCP service accounts
• CI/CD Pipeline - Reference implementation for a CI/CD Pipeline using the Apigee Deploy Maven Plugin and a choice of either Jenkins or Google Cloud Build
• CI/CD Pipeline for Shared Flows - Reference implementation for a CI/CD Pipeline for Sharedflows using the Apigee Deploy Maven Plugin
• Cloud Logging Shared Flow - Reference implementation for a shared flow to log to Google Cloud Logging
• Cloud Run - Reference implementation for using Apigee to expose a Cloud Run Service
• Data API - A reference implementation for using Apigee to facilitate the consumption of data from a data platform.
• Data Converters Shared Flow - A reference shared flow for doing common response data JSON conversions
• Product Recommendations - Smart API to Predict Customer Propensity to buy using Apigee, BigQuery ML and Cloud Spanner
• Proxy Template - An extensible templating tool to bootstrap API proxies containing Security, Traffic Management, Error Handling
• Writing JavaScript in Apigee - Demonstrate best practices in writing JavaScript code in Apigee context
• Southbound mTLS - Reference for using mTLS client authentication for securely connecting Apigee to backend services
• OIDC Mock - A simple OIDC mock identity provider implementation
• Identity Facade - Reference implementation for an Identity Facade proxy in front of an OIDC compliant identity provider
• OAuth Admin API - API proxy to enable the revocation of Apigee-issued access tokens by application or end user id.
• Dutch Healthcare Reference - An API reference to accelerate implementation of Dutch Healthcare standards.
• XML & JSON Threat Protection - A reference for protecting API proxies against XML and JSON threats
• Auth Schemes - Example implementations for various popular API auth schemes
• reCAPTCHA enterprise - A reference for API protection against bot leveraging reCAPTCHA enterprise
• Firestore Facade - Reference implementation for a long term caching/storage solution based on Cloud Firestore

Tools

This folder contains ready-made utilities which simplify and assist the usage of Apigee products.

• Pipeline Runner - A tool to build and test groups of Apigee projects
• Pipeline Linter - A tool to lint groups of Apigee projects
• Apigee hybrid Quickstart GKE - A quickstart setup configuration for Apigee hybrid on GKE
• Decrypt Hybrid Assets - A rescue utility to decrypt proxies and shared flows
• Apigee X Trial Provisioning Reference - A reference provisioning script for Apigee X trial provisioning
• Apigee Sackmesser - A unified proxy deployment utility for Edge, hybrid and X
• Apigee OpenLegacy Kickstart - A kickstart script to integrate OpenLegacy, Apigee and Cloud Run
• OpenAPI to Apigee Mock Proxy - A utility to generate an Apigee Proxy bundle with mock responses from a spec
• Cloud Endpoints OpenAPI Importer - A utility to generate Apigee Proxies based on OAS with Cloud Endpoints OAS extensions
• Generate Shared Flows Dependency List - A tool to generate topologically sorted Shared Flow dependencies.
• Apigee Envoy Quickstart Toolkit - A tool to set up the sample deployments of Apigee Envoy.
• Apigee API Proxy Endpoint Unifier - A tool to unify/split proxy endpoints based on API basepath.
• Apigee Target Server Validator - A tool to validate all targets in Target Servers & Apigee API Proxy Bundles.

Labs

This folder contains raw assets used to generate content to teach a particular technical or non-technical topic.

• Best Practices Hackathon (web) A 300 level lab to learn Apigee Best Practices
• BDD Proxy Development (web) A 200 level lab demonstrating behavior-driven API development by building, deploying and testing a mock API, generated from an Open API Specification.
• Identity facade with Okta (web) A 300 level lab that shows how to configure the Apigee Identity Facade with Okta IDP.
• Eventarc + API Hub integration - A lab for a workflow-based automation to deploy proxies using API hub and Eventarc

Contributing

See the contributing instructions to get started.

License

All solutions within this repository are provided under the Apache 2.0 license. Please see the LICENSE file for more detailed terms and conditions.

Disclaimer

This repository and its contents are not an official Google product.

Contact

Questions, issues and comments should be directed to [email protected].

devrel's Issues

Implement pipeline checks for hybrid quickstart

Open questions:

• How to supply GCP credentials
• Non-expiring hybrid organization

migrate to github actions?

Solidarity app

https://github.com/apps/in-solidarity

Extend proxy template to support Target Servers

./solutions/proxy-template/template-v1/apiproxy/targets/default.xml currently supports <URL>@TargetURL@</URL> (currently only in PR #30)

The template could/should support the use of Target Server definitions either exclusively or as an alternative.

what happens if a project overwrites APIGEE_USER or APIGEE_PASS

can we reinit them?

Move Alpine APK versions to min reference instead of fixed ones

We had several broken builds lately because of patch version increments of the Alpine packages and subsequent removal of the previous versions.

I recommend we move to using minimum versions instead of fixing exact versions.

Alternatively we have to host our own container images for the pipeline to deal with disappearing dependencies.

Pipeline Stability for Java Lint

When running the pipeline without any Java files the current check reports a failure even though no checks are performed.

Steps to reproduce:

Run:

npm run pipeline -- ./references/proxy-template

Produces a Java Lint Failure

Java Lint                                                fail

even though there are no Java files in this folder:

$ find ./references/proxy-template/ -type f -name "*.java"
#no files

"test -f unknown operand"

when running pipeline for a single project, we see

test -f unknown operand

can we do some portal automation?

Analytics is not supported in all regions

When setting the env variable $REGION (and $ZONE), it is worth mentioning that the Analytics are not available in all regions. If a user chooses a $REGION where Analytics are not available, the script fails.

Missing pre-requisite in devrel/tools/hybrid-quickstart/hybrid13/

When a user specifics a DNS_NAME, one pre-requisite to add is that the domain shall be verified and a DNS zone called "apigee-dns-zone" shall be created in advance before starting the installation script. Otherwise, the later will fail with the following error "ERROR: (gcloud.dns.record-sets.transaction.execute) HTTPError 404: The 'parameters.managedZone' resource named 'apigee-dns-zone' does not exist."

consider limitation of 1 hosted target per org in eval

consider the github super linter

https://github.com/github/super-linter

leveraging this could reduce our cost of maintenance

Check README project list for a single project

Currently it doesn't take into consideration the value of $DIR

travis to only run builds after a code review

it would be nice to only run the build after a code review, incase PRs contain malicious code. this may require reviews to manually triggger the build after a review.

pettier broke JS files in CICD reference

commit 42facea reformatted all JS files and caused the CICD reference to break at the eslint step.

Options:

1. Revert the changes to the CICD reference (Airports proxy)
2. Change the eslint preferences in the CICD reference

Also why didn't the Travis build catch this?

Nighly build failure

Pipeline Report

run-pipelines                                                      pass
references//workspace/references/apigee-sandbox-v1 Pipeline        pass
references//workspace/references/apimocker-hostedtargets Pipeline  pass
references//workspace/references/cicd-jenkins Pipeline             pass
references//workspace/references/common-shared-flows Pipeline      pass
references//workspace/references/gcp-sa-auth-shared-flow Pipeline  pass
references//workspace/references/identity-facade Pipeline          fail
references//workspace/references/java-callout Pipeline             pass
references//workspace/references/js-callout Pipeline               pass
references//workspace/references/kvm-admin-api Pipeline            pass
references//workspace/references/oidc-mock Pipeline                pass
references//workspace/references/proxy-template Pipeline           pass
references//workspace/references/southbound-mtls Pipeline          pass
labs//workspace/labs/best-practices-hackathon Pipeline             pass
tools//workspace/tools/another-apigee-client Pipeline              pass
tools//workspace/tools/decrypt-hybrid-assets Pipeline              fail
tools//workspace/tools/hybrid-quickstart Pipeline                  pass
tools//workspace/tools/organization-cleanup Pipeline               pass
tools//workspace/tools/pipeline-linter Pipeline                    pass
tools//workspace/tools/pipeline-runner Pipeline                    pass

view details in Cloud Build (permission required)

Allow Github Packages Publishing

Allow projects to publish artefacts such that casual users do not have to build the entire stack to quickly use a tool/reference.

https://github.com/features/packages

Candidates:

• Docker Image for Jenkins Reference
• Proxy Generator
• ...

Add AKS support for hybrid quickstart

Add postman collections as tests to Hello World project

e.g. https://github.com/ApigeeEMEA/demo-general-circuit-breaker/blob/master/postman/Circuit%20Breaker%20Cache.postman_collection.json

should pipeline.sh be a dotfile?

E.g. If I have an identity-v1 reference project in devrel, as a reader I might think that pipeline.sh is required in my own project. Hiding it could be useful.

Ability to add projects with GCP dependencies

For example https://github.com/ApigeeEMEA/demo-operations-api-analytics-load-generator

Dev Rel will require:

• GCP credentials + project (with billing)
• gcloud sdk
• maven

(maybe DNS is required, or maybe IP address is okay)

Should we suggest a folder template?

Such as here ?

move APIGEE_USER and APIGEE_PASS to repo settings variables instead of in .travis.yml

"Encrypted variables are not available to untrusted builds such as pull requests coming from another repository."

https://docs.travis-ci.com/user/environment-variables/#defining-encrypted-variables-in-travisyml

Nighly build failure

Pipeline Report

run-pipelines                                                      pass
references//workspace/references/apigee-sandbox-v1 Pipeline        pass
references//workspace/references/apimocker-hostedtargets Pipeline  pass
references//workspace/references/cicd-jenkins Pipeline             pass
references//workspace/references/common-shared-flows Pipeline      pass
references//workspace/references/gcp-sa-auth-shared-flow Pipeline  pass
references//workspace/references/identity-facade Pipeline          fail
references//workspace/references/java-callout Pipeline             pass
references//workspace/references/js-callout Pipeline               pass
references//workspace/references/kvm-admin-api Pipeline            pass
references//workspace/references/oidc-mock Pipeline                pass
references//workspace/references/proxy-template Pipeline           pass
references//workspace/references/southbound-mtls Pipeline          pass
labs//workspace/labs/best-practices-hackathon Pipeline             pass
tools//workspace/tools/another-apigee-client Pipeline              pass
tools//workspace/tools/decrypt-hybrid-assets Pipeline              fail
tools//workspace/tools/hybrid-quickstart Pipeline                  pass
tools//workspace/tools/organization-cleanup Pipeline               pass
tools//workspace/tools/pipeline-linter Pipeline                    pass
tools//workspace/tools/pipeline-runner Pipeline                    pass

view details in Cloud Build (permission required)

Every project to have a CODEOWNERS line

Reusable script to read deployment secrets from Google Cloud secret manager

Currently this is done using the approach as suggested here:
https://cloud.google.com/cloud-build/docs/securing-builds/use-encrypted-secrets-credentials#store_credentials

We would like to create a utility to abstract this away and make it reusable for all projects.

Improve test coverage for references/proxy-template to cover all shared flows

.editorconfig for vs code?

Usability Pipeline check for preferred terms

When running the command as per the pipeline we get the following error

$ [ ! `grep -ir "blacklist\|whitelist\|master\|slave" .` ]

sh: [: too many arguments

Also the sections which contain the preferred terms are not printed.

add an npm run command for autofixing licenses, eslint and remark

as above

dont ADD src to image, but mount volume instead

Pipelines sometimes hang

See build: https://github.com/apigee/devrel/actions/runs/271099476

organization cleanup should delete apiproducts

run spellcheck for md files

eslint not picking up correct rc file when devrel opened in vscode

run shellcheck for sh files

https://github.com/koalaman/shellcheck

dynamically create service account for gcp-sa-auth-shared-flow

settting up the variables is a little clunky for new users - can we automate this as part of the pipeline?

Move apigee-crl to devrel

https://github.com/seymen/apigee-crl

improve deploy time for static changes

if the only files changed were markdown/static content, can we reduce pipeline steps?

Apigee NG requires that the cache entry has an expiry time for a cache resource

Apigee NG requires that the cache entry has an expiry time for a cache resource.
When the shared flow does a lookup on the cache entry it exists, but when the set am header policy tries to add the bearer token it cant find the entry

can we push specs via alpha APIs?

resolve pipeline for identity-facade-v1

identity facade pipeline is failing with a curl 404- @JoelGauci lets take a look

execute build-pipeline-runner before docker run

Generate the jenkins image and store the image in ghcr

Something like https://github.com/danistrebel?tab=packages

Fail pipeline if project name is not in the README

So we don't have to remember it all the time.

[Idea] Reference Shared Flow for GCP Service Account Tokens

To use GCP services from Apigee most services require a service account access or identity token to be available from within a flow.

Apigee SaaS provides an extension but has the following short comings:

• not available in hybrid
• no key rotation possible
• no flexibility over where to store the SA key

A shared flow could be added to perform the following tasks

• lookup access tokens in cache
• exchange a SA key for an access / id token
• cache the token for later use

move pipeline-builder-runner to tools

superlinter has broken our build

https://github.com/github/super-linter/issues/938

pipeline broken due to curl dependency in golang:alpine

https://github.com/apigee/devrel/actions/runs/411849328