memazouni / malloovia

Use linear programming to allocate applications to cloud infrastructure

Malloovia

Use linear programming to allocate applications to cloud infrastructure.

• Free software: MIT license
• Documentation: https://malloovia.readthedocs.io.

Introduction

Malloovia is a Python package to solve virtual machine (VM) allocation problems in Infrastructure as a Service (IaaS) clouds from the point of view of the cloud customer. It was first presented in the paper "Cost Minimization of Virtual Machine Allocation in Public Clouds Considering Multiple Applications" presented at GECON 2017.

The problem to solve is: given a cloud infrastructure composed of different virtual machine types, each one with its own hardware characteristics, and prices, some of them with different pricing schemas, such as discounts for reservation over long periods, and given a set of applications to run on that infrastructure, each one with a different performance on each different VM type, and with a different workload over time, find the number of VMs of each type to activate at each timeslot for each application, so that the expected workload is fulfilled for all applications, the cloud provider limits are not exceeded and the total cost is minimized.

It works in two phases: first, it computes the number of reserved VMs using a Long Term Workload Prediction (LTWP) and then, it computes the number of on-demand for each time slot using a Short Term Workload Prediction (STWP).

Malloovia can be directly used in Python or by a CLI interface. The problems and the solutions can be saved using a YAML format.

This is an example that assumes that the problem definition is in problems.yaml, with problem1 describing the LTWP and problem2 describing the STWP:

$ malloovia solve problems.yaml --phase-i-id=problem1 --phase-ii-id=problem2
Reading problems.yaml...(0.004s)
Solving phase I...(0.020s)
Solving Phase II |███████████████████████████████████| 100.0% - ETA: 0:00:00
(0.101s)
Writing solutions in problems-sol.yaml...(0.006s)

This is an example in Python (explained in more detail in the Usage section of the documentation):

from malloovia import *

# Infrastructure definition
region1 = LimitingSet("r1", name="us.east", max_vms=20)
zone1 =  LimitingSet("r1_z1", name="us.east_a", max_vms=20)
m3large_z1 = InstanceClass(
    "m3large_r1_z1", name="reserved m3.large in us.east_a",
    limiting_sets=(zone1,), is_reserved=True,
    price=7, time_unit="h", max_vms=20)
m4xlarge_r1 = InstanceClass(
    "m4xlarge_r1", name="ondemand m4.xlarge in us.east",
    limiting_sets=(region1,), is_reserved=False,
    price=10, time_unit="h", max_vms=10)

# Performances
app0 = App("a0", "Web server")
app1 = App("a1", "Database")
performances = PerformanceSet(
    id="example_perfs",
    time_unit="h",
    values=PerformanceValues({
        m3large_z1: {app0: 12, app1: 500},
        m4xlarge_r1: {app0: 44, app1: 1800}
        })
)

# Workload

# Long term workload prediction of each app, for Phase I
ltwp_app0 = Workload(
    "ltwp0", description="rph to the web server", app=app0,
    values=(201, 203, 180, 220, 190, 211, 199, 204, 500, 200)
)
ltwp_app1 = Workload(
    "ltwp1", description="rph to the database", app=app1,
    values=(2010, 2035, 1807, 2202, 1910, 2110, 1985, 2033, 5050, 1992)
)

# Building the problem for phase I and solving
problem = Problem(
    id="example1",
    name="Example problem",
    workloads=(ltwp_app0, ltwp_app1),
    instance_classes=(m3large_z1, m4xlarge_r1),
    performances=performances
)

phase_i_solution = PhaseI(problem).solve()

# Building the problem for a timeslot in phase II and solving
phase_ii = PhaseII(problem, phase_i_solution)
timeslot_solution = phase_ii.solve_timeslot(
    workloads=(Workload("stwp0", app=app0, description=None, values=(315,)),
               Workload("stwp1", app=app1, description=None, values=(1950,))
               )
    )
    
# Showing the cost and the allocation
print("Cost:", timeslot_solution.solving_stats.optimal_cost)
print(timeslot_solution.allocation._inspect())

You can find example problems and solutions in YAML format in the test data directory and in the GECON 2017 data repository, where you can find a notebook that shows how to compute the solutions from the problems.

Please, refer to the documentation and the he paper "Cost Minimization of Virtual Machine Allocation in Public Clouds Considering Multiple Applications" for more details.