qovery / engine

The Orchestration Engine To Deliver Self-Service Infrastructure ⚡️

The simplest way to deploy your apps in the Cloud

Deploy your apps on any Cloud providers in just a few seconds ⚡

Qovery Engine is an open-source abstraction layer library that turns easy application deployment on AWS, GCP, Azure, and other Cloud providers in just a few minutes. The Qovery Engine is written in Rust and takes advantage of Terraform, Helm, Kubectl, and Docker to manage resources.

• Website: https://www.qovery.com
• Qovery documentation: https://hub.qovery.com/docs
• Community: Join us on Discord and on our Q&A forum

Please note: We take Qovery's security and our users' trust very seriously. If you believe you have found a security issue in Qovery, please responsibly disclose by contacting us at [email protected].

✨ Features

• Zero infrastructure management: Qovery Engine initializes, configures, and manages your Cloud account for you.
• Multi Cloud: Qovery Engine is built to work on AWS, GCP, Azure and any Cloud provider.
• On top of Kubernetes: Qovery Engine takes advantage of the power of Kubernetes at a higher level of abstraction.
• Terraform and Helm: Qovery Engine uses Terraform and Helm files to manage the infrastructure and app deployment.
• Powerful CLI: Use the provided Qovery Engine CLI to deploy your app on your Cloud account seamlessly.
• Web Interface: Qovery provides a web interface through qovery.com

🔌 Plugins

Qovery engine supports a number of different plugins to compose your own deployment flow:

• Cloud providers: AWS, Scaleway (in beta), Azure (vote), GCP (vote)
• Build platforms: Qovery CI, Circle CI (vote), Gitlab CI (vote), GitHub Actions (vote)
• Container registries: AWS ECR, DockerHub, ACR, Scaleway Container Registry
• DNS providers: Cloudflare
• Monitoring services: Datadog (vote), Newrelic (vote)

See more on our website.

Demo

Here is a demo from Qovery CLI from where we use the Qovery Engine.

Getting Started

Installation

Use the Qovery Engine as a Cargo dependency.

qovery-engine = { git = "https://github.com/Qovery/engine", branch="main" }

Usage

Rust lib

Initialize EKS (AWS Kubernetes) and ECR (AWS container registry) on AWS

let engine = Engine::new(
    context, // parameters
    local_docker, // initialize Docker as a Build Platform
    ecr, // initialize Elastic Container Registry
    aws, // initialize AWS account
    cloudflare, // initialize Cloudflare as DNS Nameservers
);

let session = match engine.session() {
    Ok(session) => session, // get the session
    Err(config_error) => panic!("configuration error {:?}", config_error),
};

let mut tx = session.transaction();

// create EKS (AWS managed Kubernetes cluster)
tx.create_kubernetes(&eks);

// create the infrastructure and wait for the result
match tx.commit() { 
    TransactionResult::Ok => println!("OK"),
    TransactionResult::Rollback(commit_err) => println!("ERROR but rollback OK"), 
    TransactionResult::UnrecoverableError(commit_err, rollback_err) => println!("FATAL ERROR")
};

Deploy an app from a Github repository on AWS

// create a session before
//------------------------

let mut environment = Environment {...};

let app = Application {
    id: "app-id-1".to_string(),
    name: "app-name-1".to_string(),
    action: Action::Create, // create the application, you can also do other actions
    git_url: "https://github.com/Qovery/node-simple-example.git".to_string(),
    git_credentials: GitCredentials {
        login: "github-login".to_string(), // if the repository is a private one, then use credentials
        access_token: "github-access-token".to_string(),
        expired_at: Utc::now(), // it's provided by the Github API
    },
    branch: "main".to_string(),
    commit_id: "238f7f0454783defa4946613bc17ebbf4ccc514a".to_string(),
    dockerfile_path: "Dockerfile".to_string(),
    private_port: Some(3000),
    total_cpus: "1".to_string(),
    cpu_burst: "1.5".to_string(),
    total_ram_in_mib: 256,
    min_instances: 1,
    max_instances: 4,
    storage: vec![], // you can add persistent storage here
    environment_variables: vec![], // you can include env var here
};

// add the app to the environment that we want to deploy
environment.applications.push(app);

// open a transaction
let mut tx = session.transaction();

// request to deploy the environment
tx.deploy_environment(&EnvironmentAction::Environment(environment));

// commit and deploy the environment
tx.commit();

Note: the repository needs to have a Dockerfile at the root.

Documentation

Full, comprehensive documentation is available on the Qovery website: https://docs.qovery.com

Contributing

Please read our Contributing Guide before submitting a Pull Request to the project.

Community support

For general help to use Qovery Engine, please refer to the official Qovery Engine documentation. For additional help, you can use one of these channels to ask a question:

• Discord (For live discussion with the Community and Qovery team)
• GitHub (Bug reports, Contributions)
• Roadmap (Roadmap, Feature requests)
• Twitter (Get the news fast)

Roadmap

Check out our roadmap to get informed of the latest features released and the upcoming ones. You may also give us insights and vote for a specific feature.

FAQ

Why does Qovery exist?

At Qovery, we believe that the Cloud must be simpler than what it is today. Our goal is to consolidate the Cloud ecosystem and makes it accessible to any developer, DevOps, and company. Qovery helps people to focus on what they build instead of wasting time doing plumbing stuff.

What is the difference between Qovery and Qovery Engine?

Qovery is a Container as a Service platform for developers. It combines the simplicity of Heroku, the reliability of AWS, and the power of Kubernetes. It makes the developer and DevOps life easier to deploy complex applications.

Qovery Engine is the Open Source abstraction layer used by Qovery to abstract the deployment of containers and databases on any Cloud provider.

Why is the Qovery Engine written in Rust?

Rust is underrated in the Cloud industry. At Qovery, we believe that Rust can help in building resilient, efficient, and performant products. Qovery wants to contribute to make Rust being a significant player in the Cloud industry for the next 10 years.

Why do you use Terraform, Helm and Kubectl binaries?

The Qovery Engine is designed to operate as an administrator and takes decisions on the output of binaries, service, API, etc. Qovery uses the most efficient tools available in the market to manage resources.

License

See the LICENSE file for licensing information.

Qovery

Qovery is a CNCF and Linux Foundation silver member.

engine's Issues

Why does the readme say you support certain cloud providers even though you don't?

The readme says that you support AWS, Digital Ocean, Azure, GCP and Scaleway. But as I can see it only AWS and Digital Ocean are supported currently? Why are the others also listed under that heading?
Same with Build Platforms, Container registries and Monitoring services.
This is really confusing when you try to understand if this piece of software is something I'd be interested in using.

[Enchancement] Consider a simple interpreter to reduce scope of shell exec

After reading the code, there is a lot of files that are doing command wrapping by doing shell exec behind the scene.
While doing shell exec is OK as reimplementing everything of what binaries are doing is

• Tedious and error prone
• Does not bring much value
• Risk of breaking with different version

A big chunk of the code is riddled with system exec with its own had hoc way of parsing output, calling convention and doing the same kind of operation.

To centralize, standardize and isolate those system-exec, consider moving from raw Command::new to a simple interpreter.
It will have the benefit of :

• Output is a standardized json
• Command only need to return a result per line and will be automatically turned into an array of string
• Every system exec will happen behind a standard code interface
• System exec will appear only once in the code, inside exec_shell
• Avoid Command::new to creep more into the code
• Ease the test of those function, only need to check script output

Please find below an example of working interpreter. It can still be improved (timeout can be added, be safer) but provide enough flexibility IMHO.

// Operations available for our shell support
pub enum Shell<'a> {
    ExecCmd(&'a str, &'a str),
    ExecCmdWithRetry(&'a str, &'a str, u32),
    EnsureCmd(&'a str),
    SetEnv(&'a str, &'a str),
    SetEnvs(&'a Vec<(&'a str, &'a str)>),

}

// Standardized output for all our shell command, a script output is just a Vec<ShellOutput>
#[derive(Deserialize, Debug)]
pub struct ShellOutput {
    name: String,
    command: String,
    error: String,
    values: Vec<String>
}

// Snippet of shell script that exec a command and format it's output to a json file "ret.json"
pub fn cmd_output_to_json(name: &str, cmd: &str) -> String {
    format!(r#"
##################
# {0}
##################
{1} > ret.stdout 2> ret.stderr
cmd_exit_code=$?
echo '{{ "name": "{0}",' >> ret.json
echo '   "command": "{1}",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1
"#, name, cmd)
}

// Exec a command with nb max retry
pub fn retry(name: &str, cmd: &str, max_retry: u32) -> String {
    format!(r#"
##################
# RETRY {2} - {0}
##################
COUNTER=0
cmd_exit_code=0
while [  $COUNTER -lt {2} ]; do
    let COUNTER=COUNTER+1
    {1} > ret.stdout 2> ret.stderr
    cmd_exit_code=$?
    [[ $cmd_exit_code -eq 0 ]] && break
    sleep 2
done

echo '{{ "name": "{0}",' >> ret.json
echo '   "command": "{1}",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1
"#, name, cmd, max_retry)
}

// Transform operation/opcode of our shell into real sh script
fn eval_opcode(action: &Shell, builder: &mut String) {
    match action {
        Shell::ExecCmd(name, args)=> {
            builder.push_str(cmd_output_to_json(name, args).as_str());
        },
        Shell::SetEnv(name, value) => {
            builder.push_str(format!("export {}=\"{}\"\n", name, value).as_str());
        }
        Shell::SetEnvs(envs ) => {
            for (name, value) in envs.iter() {
                builder.push_str(format!("export {}=\"{}\"\n", name, value).as_str());
            }
        }
        Shell::ExecCmdWithRetry(name, cmd, max_retry) => {
            builder.push_str(retry(name, cmd, *max_retry).as_str());
        }
        Shell::EnsureCmd(cmd) => {
            eval_opcode(&Shell::ExecCmd(format!("ensure command {}", cmd).as_str(),
                                        format!("command -v {}", cmd).as_str()),
                        builder)
        }
    };
}

// Generate a complete/executable shell script from our Shell Opcode
pub fn generate_shell(actions: &Vec<Shell>) -> String {

    let mut builder = String::with_capacity(1024);
    builder.push_str("#!/bin/sh
echo '[' >> ret.json
trap \"rm ret.stdout; rm ret.stderr; sed -i '$ d' ret.json ; echo ']' >> ret.json\" EXIT
    ");

    for cmd in vec![EnsureCmd("cat"), EnsureCmd("sed"), EnsureCmd("paste")] {
        eval_opcode(&cmd, &mut builder);
    }
    for cmd in actions {
        eval_opcode(&cmd, &mut builder);
    }

    builder
}

// Execute our shell script
// 1. Generate the shell script and write it into a temporary folder
// 2. Execute the script with a Command/system exec
// 3. Use serde to parse standard json and extract its value
pub fn exec_shell<P: FromStr>(actions: &Vec<Shell>) -> Result<P, SimpleError> 
{
    use std::fs::File;
    use tempfile::tempdir;

    // Generate and write our shell script into a tmp dir (cleaned when variable is droped)
    let dir = tempdir()?;
    let mut script_path = dir.path().join("run.sh");
    File::create(&script_path)?
        .write_all(generate_shell(&actions).into_bytes().as_slice());

    // Exec our script
    let exit = Command::new("sh")
        .arg(&script_path)
        .current_dir(dir.path())
        .output()?;

    // Parse ret.json to extract value of the commands
    let return_file = File::open(dir.path().join("ret.json"))?;
    let results: Vec<ShellOutput> = serde_json::from_reader(BufReader::new(return_file))?;

  // Check run success when empty json
  // Can be turned into None/Option to simplify
    if exit.status.success() && results.is_empty() {
        return P::from_str("").or_else(|err| {
            Err(SimpleError{
                kind: SimpleErrorKind::Other,
                message: Some(format!("Cannot parse string to correct format"))
            })
        });
    }

    if !exit.status.success() && results.is_empty() {
        return Err(SimpleError{
            kind: SimpleErrorKind::Command(exit.status),
            message: Some(format!("Error executing command: {:?}", exit.stderr))
        });
    }

    // Standard case
    for result in results.iter().rev() {
        if !result.error.is_empty() {
            return Err(SimpleError{
                kind: SimpleErrorKind::Command(exit.status),
                message: Some(format!("Error executing command: {:?}", result))
            });
        }
    }

    P::from_str(results.last().unwrap().values.join("").as_str()).or_else( |err|
        Err(SimpleError{
            kind: SimpleErrorKind::Other,
            message: Some(format!("Cannot parse string to correct format"))
        })
    )

}

#[cfg(test)]
mod tests {
    use crate::cmd::kubectl::Shell::{ExecCmd, ExecCmdWithRetry, EnsureCmd, SetEnv};
    use crate::cmd::kubectl::{generate_shell, exec_shell};

    #[test]
    fn test_generate_shell() {
        let actions = vec![
            EnsureCmd("kubectl"), EnsureCmd("docker"),
            SetEnv("KUBECONFIG", "/home/erebe/.kube/config"),
            ExecCmd("get pod number of restart", "kubectl get pods -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true"),
            ExecCmdWithRetry("list lol", "kubectl get lol", 3),
        ];

        //let script = generate_shell(&actions);
        //println!("{}", script);

        let ret = exec_shell::<String>(&actions);
        println!("{:?}", ret);
    }

}

It generates script like below

#!/bin/sh
echo '[' >> ret.json
trap "rm ret.stdout; rm ret.stderr; sed -i '$ d' ret.json ; echo ']' >> ret.json" EXIT

##################
# ensure command cat
##################
command -v cat > ret.stdout 2> ret.stderr
cmd_exit_code=$?
echo '{ "name": "ensure command cat",' >> ret.json
echo '   "command": "command -v cat",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1

...

With ret.json file looking like

[
{ "name": "ensure command cat",
   "command": "command -v cat",
   "error": "",
   "values": ["/usr/bin/cat"]
}
,
{ "name": "ensure command sed",
   "command": "command -v sed",
   "error": "",
   "values": ["/usr/bin/sed"]
}
,
{ "name": "ensure command paste",
   "command": "command -v paste",
   "error": "",
   "values": ["/usr/bin/paste"]
}
,
{ "name": "ensure command kubectl",
   "command": "command -v kubectl",
   "error": "",
   "values": ["/usr/bin/kubectl"]
}
,
{ "name": "ensure command docker",
   "command": "command -v docker",
   "error": "",
   "values": ["/usr/bin/docker"]
}
,
{ "name": "get pod number of restart",
   "command": "kubectl get pods -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true",
   "error": "",
   "values": ["0","0","0","0","0","1","0","0"]
}
,
{ "name": "list lol",
   "command": "kubectl get lol",
   "error": "error: the server doesn't have a resource type 'lol'",
   "values": []
}
]

As an example, 2 kubectl commands rewrote with this simple interpreter

pub fn kubectl_exec_get_number_of_restart(
    kubernetes_config: &str,
    namespace: &str,
    podname: &str,
    envs: &Vec<(&str, &str)>,
) -> Result<String, SimpleError> {
    let cmd_str = format!("get pods {} -n {} -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true", podname, namespace);
    let cmd = vec![
        EnsureCmd("kubectl"),
        SetEnv(KUBECONFIG, kubernetes_config),
        SetEnvs(&envs),
        ExecCmd("pod nb of restart", cmd_str.as_str())
    ];

   exec_shell(&cmd)
}


pub fn kubectl_exec_get_external_ingress_hostnamel(
    kubernetes_config: &str,
    namespace: &str,
    selector: &str,
    envs: &Vec<(&str, &str)>,
) -> Result<String, SimpleError>
{
    let cmd_str = format!("get svc -o json -n {} -l {}", namespace, selector);
    let cmd = vec![
        EnsureCmd("kubectl"),
        SetEnv(KUBECONFIG, kubernetes_config),
        SetEnvs(&envs),
        ExecCmd("get external ingress hostname", cmd_str.as_str())
    ];

    exec_shell(&cmd)
}

// Do the same for terraform, docker, etc...

Feature request - qovery deploy button

It seems that Qovery is heavily inspired by heroku among other things.

I was wondering if Qovery already has a feature similar to heroku-button?
And if not, that it would be considered as a new feature.

Document huawei cloud support

In my experience Huawei cloud is pretty similar to S3 (though occasionally some things are just ever so slightly different). It would be nice if huawei cloud could be tested and added to the readme if it works. Just because it's probably the most popular cloud provider in china

Migrate away from deprecated `heroku/builder-classic:22` CNB builder image

Hi 👋

This project uses Heroku's heroku/builder-classic:22 CNB builder image:

engine/src/build_platform/local_docker.rs

Lines 30 to 31 in 056c195

	const BUILDPACKS_BUILDERS: [&str; 1] = [
	"heroku/builder-classic:22",

We have deprecated this builder image as of heroku/cnb-builder-images#429.

Please migrate to heroku/builder:22 or heroku/builder:20, to continue to receive security updates, and to avoid disruption when the build time deprecation warning is upgraded to an error in the future.

More information on the difference between the various builder images can be seen here:
https://github.com/heroku/cnb-builder-images#heroku-cnb-builder-images

The repo URL for the builder images has also changed, see #739.

Using native APIs instead of using command line interfaces of Terraform, Helm, Kubectl, and Docker [feature request]

Hi,

I know, it's a goal of the project to get rid of using command line interfaces.

After reading issue #25, I did some research, and I thought it would be useful to start this issue to collect relevant information, and to track progress.

For kubectl u/garypen proposed kube.rs and k8s-openapi.

For the other tools: Bindings most likely also would be useful to the ecosystems of other programming languages as well. Maybe a solution that's based on gRPC, WASM, or similar would make it easier to secure support from the maintainers of these tools?

Basically, there would need to be a definition of the API (i.e. in JSON) and a gRPC server (or similar). New clients then could automatically generate the native API from the protocol definition (which would be almost trivial in Rust with serde).

Go has experimental WASM support.

Terraform already has a gRPC server for plugins, but I'm not sure if it could be used to implement a native API for what the command line tool does.

Another approach would be to use different tools, that have better APIs.

Here is a list of alternatives to Terraform, for example:

https://en.wikipedia.org/wiki/Infrastructure_as_code#Tools

Pulumi looks promising. They have already an SDK for TypeScript, JavaScript, Python, and Go, which they seem to have implemented via a gRPC server.

Currently, no support for Rust is planned, but community contributed support would be welcome.

I found the following resources that show how to implement an SDK for a new language:

• "How can I add support for my favorite language?"
• New Language Bring Up

What do you guys think?

I'd love to use Rust for my Infrastructure as Code needs, but using command line interfaces internally is a dealbreaker for me (and most likely for many other people, as well).

Contributing Guide is empty

this space was intentionally left blank.

Terraform C API

Is there any appetite here to work on a C API for Terraform?

Naturally this would then be easy to wrap in higher level languages like Rust; once it's in C.

Really want to avoid using the CLI or Go interfaces, and properly integrate, e.g., into a single binary or dynamic whatever cloud library files are in search directory will be used.

Just found your project, and I assume this would be of interest to you too.

(Happy to help out)

https://discuss.hashicorp.com/t/c-api-to-terraform-providers/14195

Readme: Azure vote link down

In the readme the vote link for the Azure cloud (https://github.com/Qovery/engine/blob/dev/README.md?plain=1#L45) is down. It leads to https://hub.qovery.com/docs/using-qovery/configuration/cloud-service-provider/azure/ which throws a 404

“Select a repo” don’t list the repository I want to use

Issues information

• OS: Linux
• databases: No
• Programming language and version: Python 3.X
• Link to your project on Github/Gitlab: https://github.com/diagrams-web/diagrams-web

Your issue

The list of repositories fetched are not correct with multiple organizations.

Describe here your issue

To prevent duplication and maybe misunderstood please refer to the following thread:

https://community.qovery.com/t/select-a-repo-dont-list-the-repositorie-i-want-to-use/140

I'm not sure if it's the right place to open this issue so feel free to close it or move it to another repository.

Thanks again for the fast reply and support!

Postgres Connection URL not compatible with Python database drivers.

I wanted to create a database in Qovery. I grabbed the db_url from the UI.

DB_URL = "postgresql://postgres:_Ij=de-6S8B$avgXtFN?@[email protected]:5432/postgres"
(Don't worry, this database does not exist in Qovery anymore.)

I'm getting an error.

File "/home/harshitsinghai/.pyenv/versions/3.9.5/lib/python3.9/urllib/parse.py", line 178, in port
    raise ValueError(message) from None
ValueError: Port could not be cast to integer value as '_Ij=de-6S8B$avgXtFN'

I deleted the database, created a new one, and tried again with the new db_url. But I'm getting the same error.

I don't think using postgres as the default DB username is a good idea. It will be confused by many URL parsers.

postgresql://postgres:U8T%2T#YjO#-ep-MjGBcJ_ this string might be creating confusion for lots of parsers.

Either give the user the option to change the default username or use something more generic.

Code Snippet

QOVERY_URL = "postgresql://postgres:U8T%2T#YjO#-ep-MjGBcJ_@!fgczi!%[email protected]:5432/postgres"
database = databases.Database(QOVERY_URL)
metadata = sqlalchemy.MetaData()

engine = sqlalchemy.create_engine(QOVERY_URL)

Hetzner Cloud support

Currently there is Scaleaway support in Qovery, a similar provider who is gaining popularity is Hetzner. So Hetzner Cloud would be an awesome addition.

`kube` dependency v0.86.0 is yanked

First, congrats for this awesome project. I'm triying to create a pilot project with this tool but I've found that kube dependency seems broken.

Currently, this project depends on kube version 0.86.0 but it seems yanked in crate.io.

If I tried to build it locally or use it as lib, it seems that it is not possible to compile the project:

cargo build output:

error: failed to select a version for the requirement kube = "^0.86.0"
candidate versions found which didn't match: 0.88.1, 0.88.0, 0.87.2, ...
location searched: crates.io index
required by package qovery-engine v0.1.0 (/home/xxx/qovery/engine)
perhaps a crate was updated and forgotten to be re-vendored?

EDIT: it seems that 0.87.x works fine.