Table of Contents generated with DocToc

Introduction

Application suite with Web dashboard service-lifecycle-platform (further SLP) performs building, versioning, running, updating, and monitoring of services.

When building the service, an image file is created with a unique version number. The assembly can run on different SLP servers, which allows building services in different environments.

To install service version on a specific client, a new version image is generated, supplemented with the settings and private files of this client.

After generation, the new version will be automatically installed and launched on the server. The old service version will be stopped and replaced by the new one. It is also possible to roll back to one of the previous versions.

Service logs and fault reports of all services are sent to the SLP distribution server where they are indexed for easy searching.

One of the interesting uses of SLP is installing and updating services on virtual machines (VMs). In this case, the VM is created from the image which requires regenerating of VM image when some service is changed. With SLP we can make base image for VM and install our services while doing cloud init script.
This significantly speeds up the development and debugging of services on VMs.

Required platforms

Currently, SLP works on Linux Ubuntu servers. This is due to the use of bash scripts for starting and updating services. Scripts are a small part of SLP, so support for other platforms will not be difficult.

Service source codes must be in the Git VCS repositories. For example on GitHub.

Microsoft Azure cloud service is supported now. Minimal coding required to support other cloud services.

Agreements

The following is accepted to display the structure of Json files.

Pair name/value:

name : string
name:number : number
name:boolean : boolean

Object

name
- contents

Array

name[] - strings array
name:number[] - numbers array
name[] - objects array
- ... - contents

Optional value:

[name]

Services

A service is an entire application, or a part of it. For example, a service could be an executable file with a set of shared libraries, or only shared libraries.

For each service, the assembly version and installation rules must be described. If the service is an executable program, the startup rules are also described for it, logging and crash reporting.

Service lifecycle

Creating a development version of the service
Creation of a client version of the service with customization of settings for a specific client
Installing/Upgrading a Service Instance
For runnable services:
- Run service instance
- Writing service logs to database
- Write service crash reports to database

Service profiles

Profiles are named groups of services. Profiles are used to designate distributed groups of services.

Service role

The service can perform one of the assigned roles. The role can be specified in the launch options or in the configuration file.

Service versions

When creating a service, an archive of installation files and meta information about this archive are created, marked with a unique version number.

Developer version

Format of developer version is distribution-build:

distribution - the name of the distribution server on which the version was created.
build - build number from numbers separated by dots, for example 4.21.2.

For the distribution server jetto and build 3.5.1, there will be a version jetto-3.5.1.

Client version

The client version is created from the developer version on the distribution server where it will be used. The client version contains customization of the developer's version for a specific client. It can specify specific settings, set keys, certificates, etc.

The initial client version has the same name as the developer version it was created with. If the client settings have changed, the client version is recreated. The new version will end with ._N, where N - client version generation number. For example, created from the developer's version jetto-3.5.1, the client version will be called jetto-3.5.1, next builds will be called jetto-3.5.1_1, jetto-3.5.1_2, etc.

Desired versions

The distribution server stores a number of the latest generated versions of the service.

In order to indicate which version to use at the current moment, is intended the list of desired versions in the format service-version.

There is a list of desired developer versions and desired client versions.

Desired versions are changed in a whole list, since the versions of the services depend on each other.

When generating a new version, by default, the new version number becomes desirable for the given service. The desired version number can be changed manually.

Services settings

Settings for updating and starting services

The file update.json is created in the main service development repository. It describes the rules for building, installing and running.

File format:

update[] - service descriptions
- service : service name
- build - service version assembly rules
  - [buildCommands[]] - version build commands
    - command : build command
    - [args[]] - build options
    - [env[]] - command run environment
      - 'name' : value
    - [directory] : directory to execute, if not specified, current
    - [exitCode:number] : exit code
    - [outputMatch] : matching output to stderr and stdout, regular expression
  - copyFiles[] - files and directories to copy to assembly
    - sourceFile : source file in the service repository
    - destinationFile : file in version image
    - except[] - exclude this files
    - [settings[]] - macro values in source files. When copied to an assembly, the macro is replaced with a value.
      - 'macro name' : macro value
  - install
    - [installCommands] : version install commands, as in buildCommands
    - [postInstallCommands] : commands after installing the version, as in buildCommands
    - [runService] - service start rules
      - command : run command
      - [args[]] - run arguments
      - [writeLogs] - write service logs to local disk
        
        directory : directory to write logs
        
        filePrefix : start of log file name, full name .
        
        maxFileSizeMB : maximum log file size
        
        maxFilesCount : maximum number of log files
      - [uploadLogs:boolean] - uploading service logs to the update server
      - [faultFilesMatch] - regular expression to search for files with crash information when the service fails unexpectedly. The files are included in the crash report.
      - [restartOnFault:boolean] - restart service after fault
      - [restartConditions] - conditions for service restart
        
        [maxMemoryMB:number] : the service used more than the specified number of megabytes of memory
        
        [maxCpu] - the service used more CPU resources than specified
        
        percents:number : percentage of CPU usage
        
        durationSec:number : within the specified time interval
        
        [makeCore:boolean] : for Unix-like systems, service termination with SIGQUIT signal, core dump is created by this signal
        
        checkTimeoutMs:number : interval between check conditions

Macros may be present in the string values of the settings:

%%version%% - Developer version
%%role%% - Service role. Can be specified in parameters of commands: installCommands, postInstallCommands and runService.

When generating the developer version, the install section from the update.json file is written to the install.json file version image root directory.

Service settings by the developer

Common settings for the service, which are determined by the developer, are placed in the source files. In common settings defined default values, or macros %%name%%. Common settings are recorded to developer version image.

Customizing settings for the client

When generating a client version, the settings from the developer may be defined, supplemented and overridden. To do this, a directory with additional settings is created, with the same structure as the version image directory.

The complement of common settings by optional settings done according to the following rules:

If it is required to complete the settings of a file with the extension .conf (typesafe format), .json or .properties, a file is created with the same name and path to this file. This file adds or redefines settings section. The content of the files will be automatically merged.
If you need to define macro values %%name%% in the settings file, a file with the same name and extension .defines is created, which contains values for macro definitions in the format name=value.
For one configuration file, several additional files can be created with adding an index to the name .. Files are applied sequentially, according to the index. This allows you to change the client configuration from different sources.
If you just want to add a settings file, an additional file with a unique name is created.

The install.json file can be completed for the client according to the same rules.

Private files

In addition to source codes, sometimes it is required to include to the image of service version of files with private information, such as access keys. This is often needed for client versions. Such files are not stored in source repositories, but are uploaded to the distribution server directly.

Own Services

SLP itself consists of services. Thus, SLP creates versions of itself and updates itself.

Own services:

scripts - Startup and initializing scripts (Shell, YAML)
distribution - Distribution server
- Backend
  - Distribution Web server Scala Akka HTTP
    - Supports GraphQL requests Sangria GraphQL
  - Data base MongoDB
- Frontend
  - Distribution Dashboard React Js
builder - Version builder (Scala command line application)
updater - Services installer and executor (Scala command line application)

Scripts

Perform initial installation and service upgrades.

.update.sh - base script, updates the scripts themselves, starts and updates the service
- distribution - distribution server scripts
  - .create_distribution_service.sh - installs the distribution server as an operating system service, only Linux systemd is currently supported
  - .make_distribution_config.sh - creates an initial configuration file
  - distribution.sh - start distribution server
- builder - builder scripts
  - builder.sh - run builder
- updater - updater scripts
  - .updater_setup.sh - installs updater
    - creates updater configuration file
    - installs updater as operation system service, currently only Linux systemd is supported
  - instance.yaml - start script of cloud-init
  - updater.sh - runs updater

Distribution Server

The main component of SLP is the distribution server, managed through the Distribution Dashboard. New versions of services are created on the distribution server. To create a new version, distribution starts builder.

Create distribution server

The distribution server can be built and installed from source, or from another distribution server.

Building and installing the distribution server from source codes

Install data base MongoDB.

Clone source repository SLP to distribution server:

git clone [email protected]:jorkey/_SLP_.git

Execute the command in the repository directory:

sbt "project builder; run buildProviderDistribution [cloudProvider=?] distribution=? directory=? host=? port=? [sslKeyStoreFile=?] [sslKeyStorePassword=?] title=? mongoDbConnection=? mongoDbName=? [logsMongoDbConnection=?] [logsMongoDbName=?] [persistent=?]"

Replace ? with values:

cloudProvider
- type of cloud service, if the distribution server is in the cloud:
  - Azure - the only currently supported cloud
distribution
- unique distribution server name, no spaces
directory
- installation directory
host
- DNS name for the service
port
- TCP port
[sslKeyStoreFile]
- file with private key and certificate for HTTPS service
[sslKeyStorePassword]
- password to file with private key and certificate for HTTPS service
title
- title of distribution server
mongoDbConnection
- MongoDB connection URL, for example mongodb://localhost:27017
mongoDbName
- MongoDB database name
[logsMongoDbConnection]
- MongoDB connection URL for store log records, for example mongodb://localhost:27017.
- Log records are stores in the common database if this parameter is not specified.
[logsMongoDbName]
- MongoDB database name for store log records. Must be defined when logsMongoDbConnection is defined.
- Log records are stores in the common database if this parameter is not specified.
persistent
- currently only for installation on Linux
- true, if the distribution server is installed as a Linux systemd service

After the command completes, the distribution server process should start.

Installing a distribution server from another distribution server

Create an account for the new customer on the source distribution server (see Service accounts).

Install the MongoDB database.

Copy the file .update.sh to the temporary directory from the directory of the source distribution server. Create the script clone_distribution.sh in the same place:

#!/bin/bash -e
set -e

serviceToRun=builder

distributionUrl=?
accessToken=?

. .update.sh buildConsumerDistribution providerUrl=? [cloudProvider=?] distribution=? directory=? host=? port=? [sslKeyStoreFile=?] [sslKeyStorePassword=?] title=? mongoDbConnection=? mongoDbName=? [logsMongoDbConnection=?] [logsMongoDbName=?] provider=? consumerAccessToken=? testConsumerMatch=? [persistent=?]

Replace ? with values:

providerUrl
- source distribution server URL
cloudProvider
- type of cloud service, if the distribution server is in the cloud
  - Azure - the only currently supported cloud
distribution
- distribution server unique name
directory
- installation directory
host
- DNS name for the service
port
- TCP port
[sslKeyStoreFile]
- file with private key and certificate for HTTPS service
[sslKeyStorePassword]
- password to file with private key and certificate for HTTPS service
title
- title of distribution server
mongoDbConnection
- MongoDB connection URL, for example mongodb://localhost:27017
mongoDbName
- MongoDB database name
[logsMongoDbConnection]
- MongoDB connection URL for store log records, for example mongodb://localhost:27017.
- Log records are stores in the common database if this parameter is not specified.
[logsMongoDbName]
- MongoDB database name for store log records. Must be defined when logsMongoDbConnection is defined.
- Log records are stores in the common database if this parameter is not specified.
provider
- source distribution server name
consumerAccessToken
- Access Token for consumer account
  - Go to Settings/Accounts/Distribution Consumers and click on the key icon for the relevant entry.
testConsumerMatch
- test installation distribution server name
persistent
- currently only for installation on Linux
- true, if the distribution server is installed as a Linux systemd service

Run script clone_distribution.sh. After the script completes, the distribution server process will start.

Distribution server setup

Configuration file

The distribution.json configuration file is located in the working directory of the distribution server and has the following format:

distribution : distribution server unique name
title : distribution server header
instance : virtual machine instance identifier
jwtSecret : encoding key JWT
mongoDb - setting up a connection to Mongo DB
- connection : URL подключения
- name : base name
[logsMongoDb] - setting up a connection to Mongo DB for store log records. If not defined, log records are stored in the mongoDb.
- connection : URL подключения
- name : base name
network
- port:number : HTTP or HTTPS server port number
- [ssl] - settings for HTTPS service
  - keyStoreFile : JKS file with keys and certificates
  - keyStorePassword : key store access password
versions - version history
- maxHistorySize:number : maximum number of service versions in history
serviceStates - state of services
- expirationTimeout:FiniteDuration : duration of storage of the last state of the service. If the state is not updated during this time, the service instance is considered dead.
logs - logs of services
- serviceLogExpirationTimeout:FiniteDuration : duration of service log record storage in the database
- taskLogExpirationTimeout:FiniteDuration : duration of task log record storage in the database
faultReports - fault reports
- expirationTimeout:FiniteDuration : duration of fault report storage in the database
- maxReportsCount:number : maximum number of fault reports in the database

FiniteDuration format:

unit - unit of time: "SECONDS", "HOURS", "MINUTES", "DAYS"
length:number - number of units

Functions of distribution server

GraphQL queries

The distribution server accepts GraphQL requests from users, other SLP services, and other distribution servers. GraphQL queries serve:

accounts
distribution servers connection settings
source code repositories
profile of services
developer versions
desired developer versions
client versions
desired client versions
store/find state of services
store/find logs of services
store/find fault reports

To build versions, the distribution server runs builder locally or on another distribution server.

Logging

Service logs, including logs of distribution itself, are written to a MongoDB collection. Each entry in the collection contains one log entry and consists of the following fields:

service - service name
instance - instance identifier of the server on which the service is running
directory - current process directory
process - process identifier
[task] - the task identifier, if the entry was received while the task was running
time:Date - time and data of log record
level - debug level
unit - functional unit
message - lgo message
terminationStatus:Boolean - termination status
- true - success
- false - error

The GraphQL schema has a query to search for log entries by different criteria. It is also possible to do a GraphQL subscription on the logs, and then new records in the log will be flow to the GraphQL client in real time.

Tasks

Actions asynchronous to GraphQL queries are framed as tasks. Task information is written to the MongoDB collection in the following format:

Actions asynchronous to GraphQL queries are executed in tasks. Task information is written to a MongoDB collection in the following format:

id - task identifier
type - task type
parameters - set of parameters, in the format:
- name - parameter name
- value - parameter value
creationTime:Date - time and data when task is created

The execution log and task completion status can be retrieved from the logs.

Services state handling

Updater periodically sends to distribution server status of running services. The state of the services is written to the MongoDB collection, and to the local disk.

State record format:

distribution - identified of distribution server
instance - instance identifier
service - service name
directory - service working directory
state:ServiceState - service state
- time - time of state
- [installTime] - installation time
- [startTime] - service start time
- [version] - current version of service
- [updateToVersion] - target version, if the service is currently being updated
- [updateError] - update error
- [failuresCount] - count of service failures after install and run updater
- [lastExitCode] - last exit code

Handling fault reports

Fault reports are store to MongoDB collection, and to local file storage.

Document format:

distribution - identifier of distribution server
id - identifier of fault report
info - last service state
- time - time of state
- instance - identifier of instance
- service - service name
- [serviceRole] - service role
- serviceDirectory - working directory of service
- serviceState:ServiceState - service state
- logTail:LogLine[] - last 500 log records
files[] - files with fault information

Interaction with other distribution servers

Distribution servers can be networked on a provider/consumer basis.

The provider profile defines the services for the consumer. If the provider has new versions, the consumer can download and install them. Thus, a company developing services can create a consumer account on its distribution server and determine the list of services that it wants to give to it.

The consumer periodically sends information to the provider about the provider's installed services:

Installed versions numbers
Instances state
Fault reports

The provider can also execute builder for the consumer. This is necessary if the consumer server does not have sufficient resources to build the version, or the provider has the necessary set of system libraries.

It is possible to automatically update the services of the consumer when appeared new developer versions at the provider. The consumer periodically checks for updates, and when updates are found, runs an update task.

Update of distribution server

The distribution server is a SLP service, which means that it is built and updated by general rules. The server periodically checks its current version against the client version in the database, and if they differ, restarts. Scripts after the completion of the distribution server install and run new version.

The distribution server also terminates if the client version of the scripts has been updated. In this case, the scripts update themselves and start the distribution server again.

Document history in Mongo DB

All SLP documents are stored in collections of a special format.

Each document, except for the content part, has additional fields:

_sequence - sequence number of adding/modifying the document in the collection
- indexed by ascending and descending
_modifyTime - time of adding/modifying the document
_archiveTime - document deletion time
- indexed by ascending with option expire after 7 days

Adding custom fields allows to:

Sort documents in collections by the order they were added/modified
Store the last modification date of a document
Keep a history of deleted documents for 7 days, and restore deleted document if necessary

Builder

Produces:

Build/install distribution server
Build developer versions
Build client versions

Builder install and update

Builder is installed by the distribution server in the /builder/ directory. Here:

distributionDir - distribution server working directory
distribution - distribution server name

Installation and updating is performed by SLP scripts from the distribution server for which the assembly is performed.

Distribution server install

Has been described in the section (см Create distribution server).

Build developer version

The distribution server executes the command:

builder.sh buildDeveloperVersion service=? version=? sourceRepositories=? [macroValues=?] [buildClientVersion=true/false] comment=?

Here:

service - service for which the version is being produced
version - number of new version
sourceRepositories - source repositories configuration, Json format:
- []
  - name - repository name, is the name of the directory where the sources are copied
  - git
    - url - Git repository access URL
    - branch - Git repository branch
    - [cloneSubmodules] - clone submodules
    - [subDirectory] - subdirectory inside repository If specified, sources are taken only from this directory.
privateFiles[] - list of private files to include in version, array in Json format
[macroValues] - macro values in settings in Json format:
- []
  - name - name
  - value - value
comment - comment to the new version

Version build steps:

clone or pull repositories are produced in the <builderDir>/developer/services/<service>/source directory, specified in sourceRepositories. Each repository is copied into a subdirectory called name from the configuration.
The version is built in the first directory described in sourceRepositories. This directory should contain the update.json file described above. A link to another directory from another repository should look like ../<name>.
When building, commands from build/buildCommands are executed. If macros are present in the description of commands, macro substitutions will be made:
- macro %%version%% is replaced with the build version
- other values are taken from the values parameter
If exitCode is given, its value is compared with the command's return code.
If outputMatch is set, the output of the command must match this regular expression.
After executing the build commands, the files specified in copyFiles are copied from directory <builderDir>/developer/services/<service>/source to directory <builderDir>/developer/services/<service>/build
The install section of the update.json file is written to the install.json file of the build directory.
The privateFiles files are downloaded from the distribution server and written to the build directory.
Packing the build directory to ZIP file and downloading the version image and version information to the distribution server.
Then the source file repositories are tagged with the generated version number. Tag format: <service>-<distribution>-<build>.

Build client version

Distribution server runs command:

builder.sh buildClientVersion service=? developerVersion=? clientVersion=? [settingsRepositories=?] [macroValues=?]

Where:

service - service for which the version is being produced
developerVersion - source developer version from which the client version is made
clientVersion - client version
[settingsRepositories] - developer version with which the client version is made:
- []
  - name - repository name, is the name of the directory where the settings are copied
  - git
    - url - Git repository access URL
    - branch - branch of Git repository
    - [cloneSubmodules] - clone with с submodules
  - [subDirectory] - directory name inside the repository If specified, settings are taken from this directory only
privateFiles[] - list of private files to include to the version, array in Json format
[macroValues] - macro values in settings in Json format as in buildDeveloperVersion

Build version steps:

The version image is downloaded from the distribution server and unpacked to directory <builderDir>/client/services/<service>/build.
Produces clone of repositories, specified in settingsRepositories to the directory <builderDir>/client/services/<service>/settings. Each repository is copied into a subdirectory called name from the configuration.
Additional settings are merged with developer version settings (see Customing settings for client)
The privateFiles files are downloaded from the distribution server and written to the build directory.
The build directory is packaged into a ZIP file, which is uploaded to the distribution server along with version information.

Updater

Installs, runs and updates service instances.

Install

Produces by script .updater_setup.sh. Script creates configuration file updater.json and installs updater as service systemd.

.updater_setup.sh <cloudProvider> <name> <services> <distributionUrl> <accessToken> [environment]

Where:

cloudProvider
- Name of cloud service. The only value currently supported is Azure.
name
- Service name part systemd. The full name will be update-${name}.service.
services
- Services to run, separated by commas, can be added to the service name the role it runs as -<role>
distributionUrl
- server distribution URL
accessToken
- Access token for access to distribution server

Configuration file updater.json

instanceId
- VM instance identifier
distributionUrl
- server distribution URL
accessToken
- Access token for access to distribution server

Run updater

updater.sh runServices services=<service1>[-<role>][,...]

Where:

services - services for run, as for .updater_setup.sh

Updater execution

For each of the services specified in the arguments, updater does the following.

Service install

The desired client version for this service is downloaded from the distribution server and unpacked to the build directory <service[-role]>/new.

The install.json file is read from the build directory. In the build directory, installCommands commands are executed sequentially with pre-executed macro extensions:

role - service role
version - developer version

Build directory <service[-role]>/new renamed to work directory <service[-role]>/current.

In the working directory, the postInstallCommands commands are executed sequentially with pre-executed macro expansions:

role - service role
version - developer version

The installation commands are divided into two groups to reduce the time the service is inactive during an upgrade. The installCommands group contains commands that can be executed from any directory. They are executed until the old version is stopped. The postInstallCommands group contains commands that can only be executed from the working directory.

Start service

In the <service[-role]>/current directory, the runService command is executed with pre-made macro expansions:

role - service role
version - developer version

Logging

The service should write the operation log to standard output. The log line format should be: date level unit message

Where:

date - date in format yyyy-MM-dd HH:mm:ss.SSS
level - logging level: TRACE, DEBUG, INFO, WARN или ERROR
unit - functional unit
message - logging message

The service's standard output and error output are grabbed by updater.

If the writeLogs section is defined in the configuration, the received work log lines are written to the <service[-role]>/current/ directory according to the specified rules.

If the uploadLogs parameter is set to true, log lines are also sent to the distribution server in real time.

Sending the status of services

Once every 10 seconds, updater sends the state of services ServiceState to the distribution server (see Services state handling).

Services failures handling

If the service terminates unexpectedly, a crash report is generated.

The regular expression contained in the faultFilesMatch setting describes files with fault information to be placed in fault report. For example, it could be a core dump file, or Java hprof file.

The fault report is written to the directory <service[-role]>/faults/. The fault report is archived and sent to the distribution server.

If the setting value restartOnFault is not defined, or is defined as true, the service is started again.

Service abort by updater

The restartConditions setting describes the conditions for forcibly terminating the service.

To limit the service by the amount of memory used, the maxMemoryMB parameter is specified. If the service process exceeds the specified limit, the service is forcibly terminated.

To limit the service by CPU usage, the maxCpu parameter is specified. If the service process uses more than the specified percents during durationSec, the service is forcibly terminated.

The makeCore parameter indicates that the service should be terminated with a SIGQUIT signal, to form a core dump.

The interval for checking the occurrence of service interruption conditions is set by the checkTimeoutMs parameter.

Service version update

If the desired version of the service changes on the distribution server, updater:

downloads new version
executes commands from installCommands
stops old version
- sends stop signal to service process and its descendants
service logs are copied to the directory <service[-role]>/log.history/-.log>
executes commands from postInstallCommands
runs new version

Self-update

If the desired version of updater or scripts changes on the distribution server, updater downloads new versions, they are not updated immediately, but waits until one of the updater's services is updated.

As soon as the desired version of one of the services changes on the distribution server, updater downloads updates and restarts.

Distribution Dashboard

Settings

Open the link to the distribution server in the browser. Enter Account Name: admin and Password: admin. You are logged in as an administrator.

Change the initial password. To do this, go to Settings/Accounts/Users, select the user admin, and click "Change Password".

Accounts

Administered in Settings/Accounts.

Users

Services

By clicking on the key icon, you can get an Access Key for this service. The Access Key is used in service startup and update scripts.

Distribution Consumers

Accounts of other distribution servers. The account includes, among other things:

Distribution server URL. It is used by builder to build the service remotely.
Services Profile - a list of developed services that are supplied to this distribution server.

Build

Version build settings.

Developer

Settings for the development version build. Includes:

Setting distribution server to run builder
Description of parameters for run builder:
- Сервер дистрибуции для запуска builder
- Environment
- Source code repositories, macro values (see Build developer version)
- Private files
  - Path - the path to the file in the build directory
  - File To Upload - local file to download

Client

Settings for building the client version. Includes:

Setting the distribution server to run builder
Description of parameters for launching builder:
- Distribution server to run builder
- Environment
- Settings repositories, macro values (see Build client version)
- Private files
  - Path - the path to the file in the build directory
  - File To Upload - local file to download

Profiles

Service profiles are subsets of the list of development services. Next, the profile is assigned to distribution consumer in the settings.

To refer to services in development for which client versions are also being created on a given distribution server, the self profile is defined.

Providers

This defines the providers of this distribution server. Among other things, for the provider are determined:

Access Token - Access key for requests to the server
Test Consumer - The name of the consumer, after testing which, services can be installed on this server
Upload State Interval - State load interval per provider
Auto Update - When you change the desired versions at the provider, they are automatically downloaded and installed on the distribution server. The interval for checking is 10 sec.

Build developer version

After the development services are defined, open Build/Developer. A table of services and their versions will appear:

Service - service under development
Last Build - last developer version build
Client Last Build - last clientr version build
Author - author of last developer version build
Time - time of last developer version build
Comment - comment of last developer version build
Status
- In Process - version is currently being built
- Completed - build completed

If the service version is not currently being built, you can run the task of creating a new version by selecting an entry. The new version number will be one more than the old one, or it can be assigned manually. If this service is present in the 'own' client services profile, by default you will be prompted to create a client version as well.

Clicking on "Get Lat Commit Comment" will read the change history of the service repositories. The Comment field will display the comment for the last commit in the service repository.

After starting the new version creation task, the task execution log will be displayed in real time. You can interrupt the version creation task.

If a version is currently being created, by selecting an entry, you can view the creation log in real time. You can interrupt the version creation task.

Build client versions

Open Build/Client if development services are defined, otherwise Build.

If client versions are not currently being built, a build start dialog will appear.

With download from the provider

If a provider is selected, a table of that provider's services will appear, in relation to the consumer's profile:

Service - service for installation
Provider Desired Version - the desired version of the service from the provider
Developer Desired Version - desired developer version on the distribution server
Last Client Build - last client version build
Tested Developer Version - last tested version

There are already selected services for building for which there are no client versions with the desired version from the provider. Other services cannot be selected for assembly.

Without download from the provider

If no provider is selected, the native services described in the 'own' profile are displayed:

Service - service for installation to client
Developer Desired Version - desired developer version on the distribution server
Last Client Build - last client version build

There are already selected services for assembly for which there are no client versions with the desired developer version on the distribution server.

You can also mark for assembly services for which client versions have already been built. This makes sense if the client configuration has changed and you need to rebuild the client version. In this case, a client version will be generated with a new client version build number.

Monitoring the build process

After the build is run, a real-time task execution log will be displayed. If necessary, you can interrupt the build task.

The task execution log is also displayed if, when opening the build client versions dialog assembly is already in progress.

Marking versions as tested

Before installing services on production servers, it is advisable to conduct comprehensive testing. In order for services to be installed on working servers only after they are tested, SLP has a mechanism to mark the list of service versions as tested.

For example, there is a distribution server named production, in the configuration of which the test distribution server test is specified. Installation of new versions of services on the production server will occur only if they have been tested on test and marked as tested by test.

Desired versions

In this page, you can edit the lists of desired developer and desired client versions. For example, you can roll back an update, or several recent updates. This is possible because the distribution server stores the last few installed versions and history changes to desired versions. Editing is done in one of two ways.

Rollback to a previous revision of a list

To move to an older revision, press the 'down' button, to move to a newer revision, press the 'up' button. The list revision time and author are listed at the top. The entries in the table display the desired versions of the selected revision compared to the latest revision:

unmodified records are displayed in standard font
bold - services for which the desired version has changed
red - in this revision there is no service that is in the latest revision
green - in this edition there is a service that is not in the latest edition

To make the list of desired versions current, click Save.

Assigning versions manually

You can also arbitrarily assign versions to services. To do this, click on the service version, after which a list of available versions in the history will appear. When the version is changed, the record with the service is highlighted in bold.

To make the edited list of desired versions the current one, click Save.

Monitoring logs

You can view logs in the context of a task or service execution.

Monitoring task logs

The task table is displayed in reverse chronological order:

Creation Time - task creation time
ID - task identifier
Type - task type
Parameters - task parameters with format: name:value
Active - task is currently running

When a task is selected, entries in the task log are displayed:

Time - logging time
Level - log level
Line - record

You can select logs by:

Level - log level
- when the level is selected, records of this level and above are displayed
Find Text - sample text

When Follow is selected, the end of the log is displayed, new log entries are displayed in real time.

Monitoring service logs

The logs of the selected service are searched with an optional selection by:

Instance - instance identifier
Directory - work directory of service
Process - service process number
Level - log level
- when the level is selected, records of this level and above are displayed
From/To - time range
Find Text - sample text

When Follow is selected, the end of the log is displayed, new log entries are displayed in real time.

Fault reports

The distribution server receives reports about failures of its services, and the services of its consumers. By default, all reports are displayed in descending chronological order. You can narrow search by the following criteria:

Distribution - distribution server
Service - service name
From/To - time range

When you select a report, information about the failure and the latest entries of the service log are displayed.

It is possible to download the ZIP archive of fault report to a file.

Dashboard

The Dashboard page is intended for operational control of SLP. With its, you can see how SLP lives, quickly find possible problems, rollback installed versions.

Panels are used to display information from different angles.

Panel Builds

Designed to monitor active and history of builds.

A table with fields is displayed:

Service - service name
Version - target build version
Author - build author
Time - build time
Comment - comment to build version
Target - build target
- DeveloperVersion
- ClientVersion
Status
- InProcess
- Success
- Failure

Panel Versions

Designed to monitor the distribution and operation of service versions.

A table with fields is displayed:

Service - service name
Developer Desired Version
Client Desired Version
Working Version - client version currently running
Directory - service working directory
Instances - instances identifiers list
Info - service state

Thus, this table entry displays the full distribution path of the service version from developer version to the version running on the instance and its state.

If the distribution path is completed completely, all three versions will be the same. Otherwise, the first version that does not match in the path is displayed in red. When Only Alerts is selected, only services with an incomplete distribution path are displayed.

When a consumer is selected, the versions installed on that consumer are displayed.

jorkey / service-lifecycle-platform Goto Github PK

service-lifecycle-platform's Introduction

Introduction

Required platforms

Agreements

Services

Service lifecycle

Service profiles

Service role

Service versions

Developer version

Client version

Desired versions

Services settings

Settings for updating and starting services

Service settings by the developer

Customizing settings for the client

Private files

Own Services

Scripts

Distribution Server

Create distribution server

Building and installing the distribution server from source codes

Installing a distribution server from another distribution server

Distribution server setup

Configuration file

Functions of distribution server

GraphQL queries

Logging

Tasks

Services state handling

Handling fault reports

Interaction with other distribution servers

Update of distribution server

Document history in Mongo DB

Builder

Builder install and update

Distribution server install

Build developer version

Build client version

Updater

Install

Configuration file updater.json

Run updater

Updater execution

Service install

Start service

Logging

Sending the status of services

Services failures handling

Service abort by updater

Service version update

Self-update

Distribution Dashboard

Settings

Accounts

Users

Services

Distribution Consumers

Build

Developer

Client

Profiles

Providers

Build developer version

Build client versions

With download from the provider

Without download from the provider

Monitoring the build process

Marking versions as tested

Desired versions

Rollback to a previous revision of a list

Assigning versions manually

Monitoring logs

Monitoring task logs

Monitoring service logs

Fault reports

Dashboard

Panel Builds

Panel Versions