equinor / api-strategy

Equinor API Strategy

Equinor API Strategy

The Equinor API Strategy outlines the direction for management, design and development of APIs at Equinor. The Equinor REST API Guidelines is an accompanying document containing guidelines and recommendations specifically for REST API design.

Background

Application programming interfaces (APIs) are a core element of any digital business platform. APIs provide the interfaces between apps, data and services. These interfaces enables the connection between people, businesses and things. With proper management, APIs can be an enabler for innovation, faster development of digital products and new business models. This can positively affect the profitability of the company, a concept referred to in the industry as the API Economy.

With the latest version of the governing document TR1621 (v7), Equinor is heading in an "API first" direction; software components should offer APIs to communicate with other components, share data and functionality. To further capitalize on the potentials of the API Economy, CIT via Chief Engineer IT has established the API Strategy.

Documents

• API Strategy
• REST API Guidelines
• Naming conventions and guidelines
• API Management tool decision
• API Management
• Mandate

Contributing

To contribute to this repository, please see the contribution guidelines.

api-strategy's Issues

STID API QA

STID API QA, as agreed on Software Development group on Yammer

API: https://stidapi.statoil.no/

APIM security recommendations

• API backends secured with OAuth2 and OIDC
• Authentication and authorization fully handled in backend
• Set up token validation rules in APIM, as an additional layer of protection
  • Validate token
  • Verify that token is issued by Azure AD
  • Verify token directory = Equinor
• Set up APIM rate limiting
• Production APIs should only be accessible through APIM, to ensure all requests are covered by APIMs features for security, logging and monitoring. This can be set up using one of the following options:
  • Authenticate APIM towards backend using client certificates
  • Authenticate APIM towards backend using OAuth2 with either client credentials or on-behalf-of-flow (note that OBO will prevent the API from doing a second OBO-flow towards another services, as OBO cannot be nested)
  • Use firewall/IP-filtering to limit access to backend, if supported by backend service

API presentation for CIT MC

Questions from EITA gathering/session

• Q: Does the «API first» strategy mainly apply to applications developed in Azure/Omnia, or is this strategy valid for all other runtime environments and platforms?
  Is the main idea that all API’s towards external partners should be made in Azure and registered in Azure API Management, and that all such communications should go through those API’s?

• A: The API strategy is independent of platforms or run-environment used for deploying your application/solution. For the plant area, the strategy has been updated with reference to the OPC UA protocol.

• Q: What about On-Prem applications that need to communicate with external applications? Can they use regular web services running on an internal on-prem web server to communicate, or should a new API in Azure API management be made for this in case of an upgrade? What are the recommendations and guidelines around this from you?

• A: All REST APIs which is intended to be exposed to external partners should be registered in APIM for visibility in our API portal, thus also get access to APIMs security and monitoring features. (It is possible to expose on-prem REST APIs through APIM.) Whether the application is hosted on-prem or in Azure is not a topic related to the API strategy.

• Q: What exactly is considered an API in the API first strategy? Can it be a FTP file share, an internal web service or an ODBC database connection? Or is it mainly an Azure API management thing?

• A: The API strategy is not referring to FTPs files hares or ODBC connection as APIs. The strategy mention that REST is the preferred API style in Equinor. APIs should use approved API-mechanism ref. the tech lifecycle list.

• Q: Many “developers” of PowerBI dashboards are connecting directly to one or more databases to get data. Should they instead use proper API’s for this (re. TR1621 2.3.2)?

• A: Need to revert back on this question, the data access strategy is an important input here. However, it is crucial that business capabilities are exposing access to their data and functions through APIs and thus solutions provided to the business are consuming these APIs. This question also touches up-on how we should govern Citizen Development / No-Code / Lo-Code, which is a topic we still investigate.

• Q: According to the principles of an xOps team, the team should be responsible for the operations and maintenance of whatever they develop. Does this also apply to API’s? If so, what about the API’s created by people in the business who do not have any skills or time for doing O&M?

• A: Yes. You build, operate and maintain your solutions, whether it is an API or another kind of IT solution. Important DevOps principle and in-line with ITs new operating model (Transform IT).

• Q: Should important API’s have technical or functional support? If so, should they be registered in ServiceNow so that ownership and tickets can be made on them, as for applications?

• A: Yes, a consequence of "API as a product" principle. Using ServiceNow for tickets or CIs is an operation question and not related to the API strategy. On a general comment, we should probably revisit the Service Mangement processes to fit a new way of doing SDLC (automation, CI/CD etc).

Data versioning

@Qif-Equinor has proposed including a section on data versioning in the API Strategy. From his PR:

Data versioning

Immutability of persisted objects

• Caching - Get requests for a specific version of an object is idempotent and can be cached at a various layers
• Reproducibility - By ensuing that data used as inputs to a computation can never be modified, the computations can be re-executed and guarantee identical results.
• Concurrency - Reading specific versions is safe without the need for locking.
• Traceability - Modification history is never lost.
• Performance - Inserts are handled better than updates.

Naming conventions

OMNIA teams have asked for naming conventions for some services in Azure, e.g. API Management API names, API Management Product Names, etc.

Clarify scope of strategy

Consider stating explicitly in the API strategy that it applies not only to applications and data, but also IT services like infrastructure and middleware.

Constraints on API Versioning

Perhaps, API Versioning constraints shall be introduced? Like COM "Interfaces must be immutable" or, at least, backward-compatible? In general, guidelines on API lifecycle may be needed.

API security section

Several people have requested the API strategy containing a section on security

License

"Be prepared to externalize" paragraph.

Some "common sense" constraints on generalizing the interfaces may be put here. Often it is difficult to predict usage of the interface outside the current scope (except obvious/trivial cases). Generalized interface may be more difficult to implement and use in the current scope, and still it may require adjustments for the future uses.

State preference for max visibility in API portal

Destinction between public and published API

Consider clarifying the distinction between published and public API

Backlog & TaskBoard on GitHub

API related authentication scenarios

• Consumer of public API via gateway (subscription, OAuth2)
• Consumer of internal API via gateway (OAuth2)
• Gateway to back-end (Certificate, OAuth2, bridge to other authentication protocol)
• Consumer to back-end (OAuth2, legacy protocols)

Review Steffans proposal on rephrase openness

WBS for all things....

Written / expressed mandate

Formulate a written mandate for the team.

Database Views as API in TR1621

Hi!
I have a simple question regarding the use of DATABASE VIEW API instead of a REST API. Now, I have been told that according to governing documents in Equinor we are required to use REST APIs where each table is exposed via separate endpoints instead of database views. I cannot understand this being the intention with the statement being made in TR1621 section 2.3. I think it is a misunderstanding to say that a database view is not an API. A database view can be supported as an API independently of the underlying database, just as a REST API. The difference is that a database view is way more powerful.

Let me illustrate with an example:
Say I have two tables in a remote database: Cars A and Persons B, with millions of rows each, too much data to download. Now, say I want to have all rows from table A with cars, where the owners work at Equinor. BUT, one cannot see from Table A that a person works in Equinor, so first a query in table B must be made to search by employer and find say 8000 people. These are the owners. Now, find the cars. I have to filter the cars table one by one for each of those 8000 people. If this has to be done one by one through a REST API endpoint that takes 151ms to respond. This will take quite some time, actually 20 minutes.

Database joins on a view would solve this. A database join would allow me to join the tables together and return the result in one query. It would perform, less error prone and way easier to maintain.

In my specific case, I am asked to work with the operations and maintenance data REST APIs at api.equinor.com. This interface has endpoints one-to-one with table views in an Omnia COPY of the on premise SAP database with millions of rows in each table. Since it is only a copy of the original database – all operations are limited to read only. However, as I am not working ON the omnia team, I am told that I am not allowed to make database connections from my backend app directly to the database, but must use the API. (I have a backend app that is serving web-clients) In my case I need to join, not just two tables as in the example above, but four, and I have not one, but several such queries.

One solution is to ask a remote team in a separate location to create custom endpoints. However this is going to demand a lot in meetings and coordination. If instead, the existing database view could be supported as an API we would have a flexible, yet powerful integration point. A view is decoupled from database tables and can be versioned just like a REST API. The general structure of SAP does not change either. I talked to colleagues with experience in similar integration patterns and everyone I have asked so far recommends using database views. I have searched the net, and everywhere I have seen a rest API being recommended, it is for a simpler problems with small amounts of data, or different problems altogether.
So, my question is if we can include database views as APIs in TR1621. I really cannot see any reason why not.
regards, K

Guidelines on weak data typing

Ideally - all weakly typed data must be fixed, or, at least, documented. Must conform to eventual immutability/compatibility constraints . That is, not only formal interface, but semantics as well should be immutable.

API Policies

• API Design
• Process
• Architecture
• Versioning
• Life cycle management
• Authentication
• Specification
• Documentation
• Discoverability ("HATEOAS”)

Notification system to announce planned & unplanned downtime

A system should be in place to post and retrieve information about planned and unplanned downtime of APIs.

The value from a user perspective is as follows

User story
As a user of application X
I would like to be informed about planned and unplanned downtime of data providers in the application X
In order to
Plan my usage around planned downtime, and during unplanned downtime have trust that someone knows about the issue and that steps are taken to fix it.

Create an example script for defining an API in APIM

Create script using azure cli tools to verify that APIM is scriptable without too much effort.

API Management Tool, what & why

Describe what we want from an API Management Tool and Why we need it

Comments to API strategy - draft

Ref email "API Strategy - Draft - provide feedback", please find feedback from Omnia Core Data team below;

Suggested sections

• API high level architecture

  • architecture diagram/pattern - the Equinor way
  • Ref; https://www.infolob.com/how-to-plan-your-api-management-strategy/

• API Security

  • Authentication
  • Authorisation
  • Secure communication

Comments
Swap order of following sections?

• API Management tools and roles
• API design

REST; https://github.com/equinor/api-strategy/blob/master/docs/strategy-draft.md#rest

• Include short introduction to REST - or references to
• HTTP as transport protocol
  • should it be: HTTPS as transport protocol ?
• Subsection on standard operations - ref CRUD - guidelines on verbs

API First; https://github.com/equinor/api-strategy/blob/master/docs/strategy-draft.md#api-first

• Consider including this in Goals section; https://github.com/equinor/api-strategy/blob/master/docs/strategy-draft.md#goals

Our ref; https://github.com/equinor/Omnia/issues/1191

DOP Cloud Mobile API QA

https://equinor.slack.com/files/U02JL08ES/FHKQBTZRD/dbrcloudapi-swagger.yaml

Kan også sees i APIM (Log in and subscribe):
https://api-dev.portal.equinor.com/docs/services/version1/operations/getRigs

Plant Timeseries API QA

Do a review of the Plant Timeseries API.

https://github.com/equinor/OmniaPlant/tree/master/Omnia%20Timeseries%20API

API presentation for EITA

API Presentation for GBS IT MC

API presentation for GBS IT MC

Support for other protocols

Support for other protocols than REST, e.g. WebSockets, MQTT, AMQP, etc.

Promote public thinking

Promote public thinking in API design

Alignment with key vendors

Many of the key vendors have their own extensive API style guidlines. Whilst strategy and some details might be specific for Equinor, does it make sense to align with one of these vendors as a general starting point and highlight differences rather than 'reinventing the wheel'?

http://apistylebook.com/design/guidelines/

API visibility

The APIs should be visible for everyone, even for those who do not have access to use the APIs.

It is beneficial for developers to see existing APIs so that they know what they need to apply for.

API Management Architecture

Should the API strategy cover API management architecture?

Preferred date format

@ssorenes has proposed that the API Strategy states a preference for a specific date format, e.g. RFC3339 and UTC

Plan API track on EDC

Plan API "track" on EDC.
Could be a series of API sessions.

API Blog post(s)

Create a first API blog post on developer.equinor.com

Evaluate options for API Mgmt Tool

Evaluate tools for API Management.

Resource naming cardinality

@Qif-Equinor and others have proposed to include guidelines on resource naming cardinality in the API Strategy. From Qiang's PR:

Resource naming

A resource name should clearly indicate the cardinality by using plural nouns for collections, and an instance id for a member of that collection. For example, to get info on well 1234:

GET .../wells/1234

If a relation can only exist within another resource, then append the sub resources to the path. For example: since a well can have multiple well logs, the URL to get a ROHB log for 1234 would appear as follows:

GET .../wells/1234/logs/rohb
```https://github.com/Qif-Equinor/api-strategy.git

Reference to or description of REST in strategy doc

OWASP Input Validation Cheat Sheet - broken link

Hi,

Ref; section https://github.com/equinor/api-strategy/blob/master/docs/strategy.md#input-validation

Link to OWASP Input Validation Cheat Sheet seems to be broken, assume new link is;
https://cheatsheetseries.owasp.org/cheatsheets/Input_Validation_Cheat_Sheet.html

Regards,
-Arild

api.equinor.com

Establish api.equinor.com as a developer portal for APIs.
We should assess if api-portal.equinor.com should be used for the portal, and api.equinor.com for the endpoints

WellCom API QA

Swagger:
https://equinor.slack.com/files/U02JL08ES/FHL27LE95/wellcomapi-swagger.json

Api site:
https://readwebapidbr.azurewebsites.net/swagger/index.html

What is "Standard Specification Language"?

"API Design/Design Principles/API First", first bullet. What is a "standard specification language"? Open API Specification? Isn't it too much REST specific? What about Non-REST APIs?

Guidelines on REST verbs

@arildeikeland has proposed that the API Strategy contains guidelines on rest verbs

Follow up of developer survey

Preferred data formats

@ssorenes has proposed that the API strategy should state that API response should be machine readable
It has also been proposed to standardise on JSON as data format.

Link to TR1621 is outdated

There is a link to previous TR1621 version in the docs/mandte.md, which should be fixed or removed - as externally readers does not have access to this.

Experiment with Exploration and composite logs

Do an experiment with Exploration on composite logs, and the development, deployment, use & workflow adaptations.

Preferred mechanisms in strategy

Take industrial IT into account