apache / cloudstack

Apache CloudStack is an opensource Infrastructure as a Service (IaaS) cloud computing platform

Apache CloudStack

Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. CloudStack is used by a number of service providers to offer public cloud services, and by many companies to provide an on-premises (private) cloud offering, or as part of a hybrid cloud solution.

CloudStack is a turnkey solution that includes the entire "stack" of features most organizations want with an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, a full and open native API, resource accounting, and a first-class User Interface (UI).

CloudStack currently supports the most popular hypervisors: VMware vSphere, KVM, XenServer, XenProject and Hyper-V as well as OVM and LXC containers.

Users can manage their cloud with an easy to use Web interface, command line tools, and/or a full-featured query based API.

For more information on Apache CloudStack, please visit the website

Who Uses CloudStack?

• There are more than 150 known organizations using Apache CloudStack (or a commercial distribution of CloudStack). Our users include many major service providers running CloudStack to offer public cloud services, product vendors who incorporate or integrate with CloudStack in their own products, organizations who have used CloudStack to build their own private clouds, and systems integrators that offer CloudStack related services.

• See our case studies highlighting successful deployments of Apache CloudStack.

• See the up-to-date list of current users.

• If you are using CloudStack in your organization and your company is not listed above, please complete our brief adoption survey. We're happy to keep your company name anonymous if you require.

Demo

See the project user-interface QA website that runs CloudStack against simulator hypervisor: https://qa.cloudstack.cloud/simulator/ (admin:password)

Getting Started

• Download a released version
• Build from source with the instructions in the INSTALL.md file.

Getting Source Repository

Apache CloudStack project uses Git. The official Git repository is at:

https://gitbox.apache.org/repos/asf/cloudstack.git

And a mirror is hosted on GitHub:

https://github.com/apache/cloudstack

The GitHub mirror is strictly read only and provides convenience to users and developers to explore the code and for the community to accept contributions via GitHub pull requests.

Documentation

• Project Documentation
• Release notes
• Developer wiki
• Design documents
• API documentation
• How to contribute

News and Events

• Blog
• Twitter
• Events and meetup
• YouTube channel

Getting Involved and Contributing

Interested in helping out with Apache CloudStack? Great! We welcome participation from anybody willing to work The Apache Way and make a contribution. Note that you do not have to be a developer in order to contribute to Apache CloudStack. We need folks to help with documentation, translation, promotion etc. See our contribution page.

If you are a frequent contributors, you can request to be added as collaborators (see https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub) to our GitHub repos. This allows you to use project GitHub with ability to report issue with tags, and be assigned to issues and PRs. This is done via the .asf.yaml file in this repo. You may do so by sharing your GitHub users ID or raise a GitHub issue.

If you're interested in learning more or participating in the Apache CloudStack project, the mailing lists are the best way to do that. While the project has several communications channels, the mailing lists are the most active and the official channels for making decisions about the project itself.

Mailing lists:

• Development Mailing List
• Users Mailing List
• Commits Mailing List
• Issues Mailing List
• Marketing Mailing List

Report and/or check bugs on GitHub and check our developer page for contributing code.

Reporting Security Vulnerabilities

If you've found an issue that you believe is a security vulnerability in a released version of CloudStack, please report it to [email protected] with details about the vulnerability, how it might be exploited, and any additional information that might be useful.

For more details, please visit our security page.

License

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Please see the LICENSE file included in the root directory of the source tree for extended license details.

Notice of Cryptographic Software

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See The Wassenaar Arrangement for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software:

• CloudStack makes use of JaSypt cryptographic libraries.
• CloudStack has a system requirement of MySQL, and uses native database encryption functionality.
• CloudStack makes use of the Bouncy Castle general-purpose encryption library.
• CloudStack can optionally interact with and control OpenSwan-based VPNs.
• CloudStack has a dependency on and makes use of JSch - a java SSH2 implementation.

cloudstack's Issues

dedicatePublicIpRange to domain doesn't work

ISSUE TYPE

• Bug Report

COMPONENT NAME

Network (Public)

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

advanced networking

OS / ENVIRONMENT

Ubuntu 16.04 (Management/Hypervisor)
KVM

SUMMARY

dedicate a public IP range to a domain doesn't work

STEPS TO REPRODUCE

create a new (should not be the first) public IP range  
dedicate this range to a domain
acquire a new public IP in a project which is located in the specified domain

EXPECTED RESULTS

get a IP which is located in the specified IP range for this domain

ACTUAL RESULTS

ip is acquired from an other public pool (first created public IP range - with no assignment for domain)

listVirtualMachines response ostypeid is foreign DB ID, not UUID

ISSUE TYPE

• Bug Report
• Enhancement

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

The response of listVirtualMachines contains the ostypeid returned as foreignkey ID of the DB, instead of UUID

STEPS TO REPRODUCE

cs listVirtualMachines id=d949e443-9bc8-4399-9e09-842bbb395cae --region simulator | jq '.virtualmachine[].ostypeid'

EXPECTED RESULTS

...
"ostypeid": "1ef04fab-4b9a-11e7-8fea-0242ac110003",
...

ACTUAL RESULTS

...
"ostypeid": 142, 
...

CloudStack should have Github issues: Hey we got Issues now!

I guess the issue of not having "issues" is fixed now? Should we close it?

UI LDAP settings

ISSUE TYPE

• Other

COMPONENT NAME

Management server

CLOUDSTACK VERSION

4.11.0.0

Don't no if it's issue or not.. In CS UI when configuring LDAP settings instead of values, which were entered I see this:

New values (if I, for example, edit default value) entered are showing normally until I reboot management server (to apply configuration). After, all values regarding LDAP are showed like on screen

Password Reset does not work with Isolated Networks with redundant routers

ISSUE TYPE

• Bug Report

COMPONENT NAME

VR

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

Advanced Network, Isolated Network with Redundant Router and DNS, DHCP, Passwort, etc. Feateres all on

OS / ENVIRONMENT

SUMMARY

When using the cloudstack-set-guest-password skript, the query is done against http://IP-OF-THE-DHCP-SERVER-THAT-SERVED-THE-LEASE:8080

A quick peek inside the MASTER of a redundant router setup shows, that the service (python) is only listening to the service-IP (the default gateway)

Firewall instead shows that port 8080 is open for both host- and service-IP.

STEPS TO REPRODUCE

Deploy a VM connected to an Isolated Network with redundant routers and use /etc/init.d/cloudstack-set-guest-password inside the VM

wget just silently quits, telnet shows a "connection refused"

SSH keys are not affected since the pubkeys are deliverd by another service.

EXPECTED RESULTS

Getting the password via DHCP-SERVER-HOST-IP:8080

ACTUAL RESULTS

Add AMD FirePro MxGPU assignment to CloudStack guests

ISSUE TYPE

• Feature Idea

COMPONENT NAME

Apache CloudStack UI, Agent, API, etc.

CLOUDSTACK VERSION

Apache CloudStack 4.11.0.0 (LTS)

SUMMARY

With the release of gim (https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization), the AMD FirePro is now supported on the KVM hypervisor. Similar to NVIDIA's GRID or Tesla, the AMD FirePro allows KVM/ESXi/XenServer guests to utilize VF's to enhance graphics performance. The AMD FirePro requires the use of SR-IOV and IOMMU to pass VFs. I would like to see support for the AMD FirePro MxGPU in Apache CloudStack. As time progresses, we are going to possibly develop this ourselves.

EXPECTED RESULTS

Add the AMD FirePro as an option in Service Offering > GPU. When chosen, the user can choose how many VFs they want to pass to the virtual guest. 

CPU cores per socket setting not functional

ISSUE TYPE

• Bug Report
• Enhancement Request

COMPONENT NAME

• Cores per socket setting in user_vm_details / vm_template_details table when using XenServer hypervisor

CLOUDSTACK VERSION

• CloudStack version 4.9.2
• Not attempted reproduced in other versions but assumption is issue is still present in 4.11.x

CONFIGURATION, OS / ENVIRONMENT

• CloudStack 4.9.2
• XenServer 6.2 and 7.0 - fully licensed with Citrix licensing
• Advanced zone

SUMMARY

• Even with cpu.corespersocket (or cpuid.coresPerSocket as suggested in some mailing list threads) specified CloudStack will ignore this setting.
• All ACS specifications for cores and sockets will translate to "sockets" in XenServer
• Overall problem is some VMs when given a higher core count does not see all it's CPU cores/sockets.

STEPS TO REPRODUCE

• Create a VM with cpu count 4 cores
• Add entry for cpu.corespersocket=2 in user_vm_details
• Start VM
• Check allocation in XenCentre

EXPECTED RESULTS

• VM with 2 sockets and 2 cores per socket

ACTUAL RESULTS

• VM with 4 sockets

HTML shown to end-user in UI

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

N/A

OS / ENVIRONMENT

Any OS

SUMMARY

HTML code (<br/><br/>) in the UI

STEPS TO REPRODUCE

Create a zone

EXPECTED RESULTS

non-HTML text

ACTUAL RESULTS

Register ISO image in all zones

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11.0
4.11.1

CONFIGURATION

Two advanced networking zones

OS / ENVIRONMENT

Apache Cloudstack version 4.11.1
Two advanced networking zones
NFS Primary and secondary storage
VMware 5.5u3 and Xenserver 7.0 hosts

SUMMARY

If more than one zone has been setup and you try to register a ISO to all zones you get the following error Please specify a valid zone.

STEPS TO REPRODUCE

1. Setup more that one zone 
2. Enable zones 
3. Register new ISO to all zones 

EXPECTED RESULTS

ISO images gets downloaded to all zones 

ACTUAL RESULTS

Error is given Please specify a valid zone.

API uploadSslCert does not work

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

Advanced Networking, Cloudstack 4.11.0.0 on Ubuntu 14.04 LTS, MySQL 5.5.59-0ubuntu0.14.04.1

OS / ENVIRONMENT

Ubuntu 14.04.5 LTS

SUMMARY

Uploading sslcertificates ( https://cloudstack.apache.org/api/apidocs-4.11/apis/uploadSslCert.html ) fails for two reasons.
a) Including the CertChain, it's very likely that the Request Header Size grows > 8k and results in HTTP Error 414.
b) Leaving the CertChain empty to get a shorter GET Request, the Cloudstack API responds with Error 530: DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@7ba59657

STEPS TO REPRODUCE

Here 's the sample output of a response. For obvious reasons the certificate, key and fingerprint  as well as the name are redacted.

Error 530: DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@7ba59657: INSERT INTO sslcerts (sslcerts.id, sslcerts.uuid, sslcerts.certificate, sslcerts.chain, sslcerts.key, sslcerts.password, sslcerts.account_id, sslcerts.domain_id, sslcerts.fingerprint, sslcerts.name) VALUES (null, _binary'4ed417b3-77b6-42bf-9a82-81ed25aaedc2', _binary'-----BEGIN CERTIFICATE-----\n R E D A C T E D \n-----END CERTIFICATE-----', null, _binary'K E Y R E D A C T E D==', null, 160, 2, _binary'00:11:22:33:44:55:66:77:88:F7:AE:73:81:BA:D9:B7:AF:46:D8:E2:F8:EA:7C:FF:00:B7:DE:F7:D7:CF:9B:C9', _binary'STAR.some.tld')
errorcode,uuidList,cserrorcode,errortext

EXPECTED RESULTS

One would expect a certificate available to select after uploading it.

ACTUAL RESULTS

details not in listVirtualMachines response

ISSUE TYPE

Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11
4.9

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

listVirtualMachines does not return details as doc says.

STEPS TO REPRODUCE

cs listVirtualMachines id=d949e443-9bc8-4399-9e09-842bbb395cae --region simulator    

EXPECTED RESULTS

the resource details as below should be included in listVirtualMachines response, as the documentation of the API says or the API doc should be updated.

cs listResourceDetails resourcetype=uservm resourceid=d949e443-9bc8-4399-9e09-842bbb395cae --region=simulator     
{
  "count": 3, 
  "resourcedetail": [
    {
      "fordisplay": true, 
      "key": "foo", 
      "resourceid": "5", 
      "resourcetype": "UserVm", 
      "value": "bar"
    }, 
    {
      "fordisplay": true, 
      "key": "Message.ReservedCapacityFreed.Flag", 
      "resourceid": "5", 
      "resourcetype": "UserVm", 
      "value": "false"
    }, 
    {
      "fordisplay": true, 
      "key": "SSH.PublicKey", 
      "resourceid": "5", 
      "resourcetype": "UserVm", 
      "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCalaJdHI7bK4qsCMSqJ1iq7K4YuHyjlkahPuSsbMrZInN2Mo1ehjieILocEITo8ck+rFkC4NHc6mvUDF3gdUZmWKpWYfc5eKsymOHgk05Nv1e+1TjvKGWaZprisUFXD+52PkQ6DcAZUDYw1yufxGfufYuK3WWzDHriSBIiaDEcgw== \n"
    }
  ]
}

ACTUAL RESULTS

{
  "count": 1, 
  "virtualmachine": [
    {
      "account": "admin", 
      "affinitygroup": [
        {
          "account": "admin", 
          "id": "2d41d761-6d4e-400a-9f26-cc0d5ef6f884", 
          "name": "ansible-test-titan-10437836-ag"
        }
      ], 
      "cpunumber": 1, 
      "cpuspeed": 500, 
      "cpuused": "10%", 
      "created": "2018-03-31T18:07:13+0000", 
      "diskioread": 0, 
      "diskiowrite": 0, 
      "diskkbsread": 0, 
      "diskkbswrite": 0, 
      "displayname": "ansible-test-titan-10437836-display-1", 
      "displayvm": true, 
      "domain": "ROOT", 
      "domainid": "1f1d6aa1-4b9a-11e7-8fea-0242ac110003", 
      "guestosid": "1ef0010d-4b9a-11e7-8fea-0242ac110003", 
      "haenable": false, 
      "hostid": "41adc5d5-3b6a-4035-9913-7b14fa099c1f", 
      "hostname": "SimulatedAgent.db961dea-4e46-48c6-983e-566712333810", 
      "hypervisor": "Simulator", 
      "id": "d949e443-9bc8-4399-9e09-842bbb395cae", 
      "instancename": "i-2-5-VM", 
      "isdynamicallyscalable": false, 
      "keypair": "ansible-test-titan-10437836-sshkey", 
      "memory": 512, 
      "memoryintfreekbs": 0, 
      "memorykbs": 0, 
      "memorytargetkbs": 0, 
      "name": "ansible-test-titan-10437836-vm-1", 
      "networkkbsread": 148668416, 
      "networkkbswrite": 74334208, 
      "nic": [
        {
          "broadcasturi": "vlan://untagged", 
          "gateway": "60.147.41.1", 
          "id": "8bee0927-6506-4a18-93bc-ccce070714a2", 
          "ipaddress": "60.147.41.246", 
          "isdefault": true, 
          "macaddress": "06:ce:d4:00:01:f2", 
          "netmask": "255.255.255.0", 
          "networkid": "6f4007a0-ddba-4cdf-ac34-c9c0e5f049b8", 
          "networkname": "guestNetworkForBasicZone", 
          "secondaryip": [], 
          "traffictype": "Guest", 
          "type": "Shared"
        }
      ], 
      "ostypeid": 142, 
      "passwordenabled": false, 
      "rootdeviceid": 0, 
      "rootdevicetype": "ROOT", 
      "securitygroup": [
        {
          "account": "admin", 
          "egressrule": [], 
          "id": "abcdb1a7-4795-4aff-82a9-ca4347b017fd", 
          "ingressrule": [], 
          "name": "ansible-test-titan-10437836-sg", 
          "tags": [], 
          "virtualmachineids": []
        }
      ], 
      "serviceofferingid": "a9ad69ae-391d-43ba-97a8-5f33b78a955d", 
      "serviceofferingname": "Small Instance", 
      "state": "Running", 
      "tags": [], 
      "templatedisplaytext": "CentOS 5.6 (64-bit) no GUI (Simulator)", 
      "templateid": "4f4c3063-4b9a-11e7-8fea-0242ac110003", 
      "templatename": "CentOS 5.6 (64-bit) no GUI (Simulator)", 
      "userid": "1f1daf4d-4b9a-11e7-8fea-0242ac110003", 
      "username": "admin", 
      "zoneid": "ab54814a-b8b9-4889-ae76-cae19c0c843f", 
      "zonename": "Sandbox-simulator-basic"
    }
  ]
}

Unable to create VM from ISO

ISSUE TYPE

Bug

COMPONENT NAME

CloudStack Agent

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

• Advanced Networking
• KVM
• CEPH Storage (RBD pool)

OS / ENVIRONMENT

CentOS Linux release 7.4.1708

Summary

Ubuntu 16.04 ISO is registered (ISO link)
ISO status: ready.
When trying to create vm from ISO, getting error: unsupported configuration: only RAW volumes are supported by this storage pool

Management server log

2018-05-14 10:25:14,263 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-2:ctx-37bb2bb5 job-46/job-47 ctx-d1a0c838) (logid:46f24ebc) Unable to create Vol[5|vm=5|ROOT]:com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: unsupported configuration: only RAW volumes are supported by this storage pool
2018-05-14 10:25:14,264 INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-2:ctx-37bb2bb5 job-46/job-47 ctx-d1a0c838) (logid:46f24ebc) Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:3] is unreachable: Unable to create Vol[5|vm=5|ROOT]:com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: unsupported configuration: only RAW volumes are supported by this storage pool
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1336)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1110)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4927)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5090)
	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Agent log

2018-05-14 10:42:56,234 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ab093391) Succesfully refreshed pool 9a2e9bd0-2218-373e-851c-e7c7176a3974 Capacity: 10199249412096 Used: 4794017843 Available: 10155745476608
2018-05-14 10:42:56,235 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ab093391) Attempting to create volume 56df4307-6d19-4832-bcad-a54b840bca04 (RBD) in pool 9a2e9bd0-2218-373e-851c-e7c7176a3974 with size 21474836480
2018-05-14 10:42:56,237 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ab093391) <volume>
<name>56df4307-6d19-4832-bcad-a54b840bca04</name>
<capacity>21474836480</capacity>
<target>
<format type='qcow2'/>
<permissions><mode>0744</mode></permissions></target>
</volume>

2018-05-14 10:42:56,244 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-1:null) (logid:ab093391) Failed to create volume:
com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: unsupported configuration: only RAW volumes are supported by this storage pool
	at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createPhysicalDiskByLibVirt(LibvirtStorageAdaptor.java:705)
	at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createPhysicalDisk(LibvirtStorageAdaptor.java:664)
	at com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.createPhysicalDisk(LibvirtStoragePool.java:115)
	at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createVolume(KVMStorageProcessor.java:1323)
	at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:119)
	at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:58)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:75)
	at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1438)
	at com.cloud.agent.Agent.processRequest(Agent.java:563)
	at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:976)
	at com.cloud.utils.nio.Task.call(Task.java:83)
	at com.cloud.utils.nio.Task.call(Task.java:29)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2018-05-14 10:42:56,247 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:ab093391) Seq 4-4642648265865560075:  { Ans: , MgmtId: 168361987, via: 4, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: unsupported configuration: only RAW volumes are supported by this storage pool","wait":0}}] }

SSVM cannot reconnect after connection disruption if there is an active event.

ISSUE TYPE

• Bug Report

COMPONENT NAME

System VMs
(Maybe also KVM hosts)

CLOUDSTACK VERSION

4.11.0

CONFIGURATION

OS / ENVIRONMENT

4.11.0 environment with VMware

SUMMARY

If there is an interruption to mgmt server <-> agent communications while an action is taking place (such as the mgmt server restarting when the ssvm is performing a snapshot) the SSVM will not be able to reconnect due to following error:
2018-05-09 11:37:09,403 INFO [cloud.agent.Agent] (Agent-Handler-9:null) Lost connection to host: 10.220.136.127. Dealing with the remaining commands...
2018-05-09 11:37:09,404 INFO [cloud.agent.Agent] (Agent-Handler-9:null) Cannot connect because we still have 1 commands in progress.

STEPS TO REPRODUCE

During a volume snapshot exporting the ovf restart the management server.  

EXPECTED RESULTS

SSVM reconnects.

ACTUAL RESULTS

The storage VM does not reconnect to the management server and has an error such as: 
INFO  [cloud.agent.Agent] (Agent-Handler-9:null) Lost connection to host: 10.220.136.127. Dealing with the remaining commands...
INFO  [cloud.agent.Agent] (Agent-Handler-9:null) Cannot connect because we still have 1 commands in progress.

Once the job had finished it will reconnect but until this point all other jobs failed unless there is another secondary storage vm up and running.
The backup job even though it is forced to complete from secondary storage is still left in the db as state backing up forever so it does not make that it even waiting for it to finish.

Infra->Pods does not show which pod from which zone is, admin needs to name them with the zone name to have this info

ISSUE TYPE

• Improvement Request
• Enhancement Request
• Feature Idea

COMPONENT NAME

UI

CLOUDSTACK VERSION

Reported in 4.11, but also seen in previous 

CONFIGURATION

Have more than one zone with multiple pods

OS / ENVIRONMENT

NA

SUMMARY

When opening the Pods in the UI, Cloudstack admin is not able to see which zone a pod is part of, if it's using same naming it could be confusing. It'll be good if we have a fields here to display the zone.

STEPS TO REPRODUCE

Just open Infra->Pods

EXPECTED RESULTS

Pods to have zone displayed 

ACTUAL RESULTS

They dont

The system version supported by the instance

I want to create examples of various systems, but I don't know that cloudstack specifically supports those systems and the version number of the system.

Why do we package final artifacts with developer profile?

This technically is not an issue, but I was just got curios why we package our final artifacts (both rpm and deb) with developer maven profile. in:

• debian/rules#L16
• packaging/centos63/cloud.spec#L209-223
• packaging/centos7/cloud.spec#L192

It doesn't make sense since these are going to be installed in prod or alike, and most probably won't be any deploydb and db.properties.override.

I believe we should remove these.

VPC network modify host IP problem

ISSUE TYPE

• Bug Report

COMPONENT NAME

 VR

CLOUDSTACK VERSION

4.12 master

CONFIGURATION

advanced networking

OS / ENVIRONMENT

Centos 7.4_x64

SUMMARY

VPC network modify host IP problem

STEPS TO REPRODUCE

1.Under the VPC network, stop one virtual host
2.modify the host IP under the NIC tab of the host details,
3.watch the host IP changed on the website
4.Start the virtual host
5.in virtual host,use ifconfig to view host IP

EXPECTED RESULTS

The host IP is a modified IP

ACTUAL RESULTS

The host IP is still the original IP after the actual operation. when this problem occurs, how to modify the host IP through the website in the future will not be successful. This problem does not occur every time, but the occurrence rate is high.

Non-admin User receives an error message when entering disk offering section of create instance wizard

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI - Add instance wizard

CLOUDSTACK VERSION

4.11.1

CONFIGURATION

4.11.1 build

OS / ENVIRONMENT

SUMMARY

An error message pops up for all (non-admin) users saying
"Filter all can be specified by admin only"
when using the add vm instance wizard at the 'Disk Offering' stage.

searched for "Filter can be specified by admin only" and found that QueryManagerImpl.java has that message and that it had been modified by this recent commit:
https://github.com/apache/cloudstack/commit/8b09620d777805bdbb59255ef722e0a6ddbb85b8#commitcomment-29087677

STEPS TO REPRODUCE

Create a non-admin user.  Log in as that user.  try to create a new VM instance using the wizard.

EXPECTED RESULTS

more through wizard without error messages, when entering Disk Offering selection page.

ACTUAL RESULTS

![image](https://user-images.githubusercontent.com/4810220/40418282-c9cdbb0e-5e79-11e8-9ba5-c33253de10d4.png)


Using developer tools we can see the following response:

{"listtemplatesresponse":{"uuidList":[],"errorcode":431,"cserrorcode":4350,"errortext":"Filter all can be specified by admin only"}}

Upgrade to 4.11 and pre-existent saml2 authentication settings

ISSUE TYPE

• Bug Report

COMPONENT NAME

SAML2 Auth plugin

CLOUDSTACK VERSION

4.11

CONFIGURATION

Cloudstack 4.5.2 with SAML2 authentication working well on national federation.

OS / ENVIRONMENT

Clean install of Cloudstack 4.11 on Ubuntu 16.04.4 LTS pointing to a copy of 4.5.2 cloudstack production database.

SUMMARY

After upgrade from 4.5.2 to 4.11 (parallel build process) the http://IP:8080/client show an error (HTTP ERROR 503 -
Problem accessing /client/. Reason: Service Unavailable) and UI does not load.

It appears that pre-existent content of columns "key" and "certificate" of rows with "name" content "SAMLSP_X509CERT" and "SAMLSP_KEYPAIR" need some conversion, but the upgrade procedure did not made it.

If we delete the old saml rows from cloud.keystore table, the /client works but https://IP:8080/client/api?command=getSPMetadata returns an certificate different from that registered on national federation. And so, the authentication fails for our web users.

STEPS TO REPRODUCE

1-) Do a clean install of Cloudstack 4.11.
2-) Point this install to a copy of 4.5.2 production database that has SAML2 authentication enabled and working inside an federation.
3-) Run cloudstack-setup-databases.
4-) Run cloudstack-setup-management and wait for upgrade completion and cloudstack-management service start.
5-) Try to access the UI interface

EXPECTED RESULTS

Can access and use a fully functional Cloudstack UI.

ACTUAL RESULTS

HTTP ERROR 503
Problem accessing /client/. Reason:

    Service Unavailable

No volume name and virtualmachine ID in volume usagerecord

ISSUE TYPE

• Improvement Request

COMPONENT NAME

Usage (Usagerecords)

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

advanced networking

OS / ENVIRONMENT

Ubuntu 16.04 

SUMMARY

If the usagerecords of a volume are queried, the result does not contain the name of the volume or the virtualmachineid of the VM the volume is attached to.
It just contains a description with the internal volumeid.

For processing this data in some 3rd party tools, it would be helpfull to know how the volume is called and on which VM it was attached in the timeframe of the usagerecord.

STEPS TO REPRODUCE

http://<cloudstack>/client/api/?command=listUsageRecords&startdate=<startdate>&enddate=<enddate>&type=6

EXPECTED RESULTS

The data should contain the volume name and the virtualmachine id if the volume is attached to a VM.

ACTUAL RESULTS

No volume name or virtualmachine id is delivered.

Web UI is not responsive

ISSUE TYPE

• Improvement Request

COMPONENT NAME

UI

CLOUDSTACK VERSION

Latest

CONFIGURATION

N/A

OS / ENVIRONMENT

Web browser

SUMMARY

The web UI is not responsive.

STEPS TO REPRODUCE

Open the website UI in a small monitor. I tried in a 1024x768 screen. In smaller screens will be even worse.

EXPECTED RESULTS

A responsive website

ACTUAL RESULTS

It wasn't responsive, and I had to scroll. 

KVM hosts are being reset by a script

ISSUE TYPE

Bug Report

COMPONENT NAME

KVM

CLOUDSTACK VERSION

All 4.x and 2.x versions seem to be affected.

CONFIGURATION

N/A

OS / ENVIRONMENT

This issue is related to KVM hosts. As far as I can see, XEN hosts are affected too.

SUMMARY

KVM hosts are being rebooted by kvmheartbeat.sh when the NFS-server that is being used as a primary storage becomes unreachable.

I'd like to propose change behavior of kvmheartbeat.sh, as it's not necessary to reboot a whole host when only one of VMs is having difficulties while other VMs are working fine with other storages.

STEPS TO REPRODUCE

It's easy to reproduce: just deploy a VM with a volume on an NFS-based storage, break NFS and wait.

EXPECTED RESULTS

The KVM host will be reset.

ACTUAL RESULTS

The kvmscript.sh will reboot the server.

listOsTypes respond returns isuserdefined as string

ISSUE TYPE

• Bug Report
• Enhancement

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

listOsTypes returns isuserdefined as json string, instead of json bool

STEPS TO REPRODUCE

 $ cs listOsTypes id=1ee61d39-4b9a-11e7-8fea-0242ac110003 --region simulator                                          

EXPECTED RESULTS

{
  "count": 1, 
  "ostype": [
    {
      "description": "Windows XP SP3 (32-bit)", 
      "id": "1ee61d39-4b9a-11e7-8fea-0242ac110003", 
      "isuserdefined": false, 
      "oscategoryid": "1ede7e9a-4b9a-11e7-8fea-0242ac110003"
    }
  ]
}

ACTUAL RESULTS

{
  "count": 1, 
  "ostype": [
    {
      "description": "Windows XP SP3 (32-bit)", 
      "id": "1ee61d39-4b9a-11e7-8fea-0242ac110003", 
      "isuserdefined": "false", 
      "oscategoryid": "1ede7e9a-4b9a-11e7-8fea-0242ac110003"
    }
  ]
}

KVM hosts fail to connect if there is more than one IP address on the host

The secure hosts feature appears to detect the IP address that it uses to identify itself. if there is more than one IP address associated with the host, then it often returns the incorrect IP.

The mgmt log would show:

2018-03-14 18:13:10,020 DEBUG [o.a.c.c.p.RootCACustomTrustManager] (pool-77-thread-1:null) (logid:) A client/agent attempting connection from address=10.0.0.19 has presented these certificate(s):
Certificate [1] :
Serial: 7dc845e9c253e9fc
Not Before:Wed Mar 14 06:06:39 UTC 2018
Not After:Thu Mar 14 18:06:39 UTC 2019
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:C=cloudstack, O=cloudstack, OU=cloudstack, CN=PhysHost2
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:[[7, 10.5.2.12], [2, 10.5.2.12]]
Certificate [2] :
Serial: 9498673f271fef0e
Not Before:Wed Mar 14 05:40:42 UTC 2018
Not After:Fri Mar 06 17:40:42 UTC 2048
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:CN=ca.cloudstack.apache.org
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:null
2018-03-14 18:13:10,022 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-77-thread-1:null) (logid:) Certificate ownership verification failed for client: 10.0.0.19
2018-03-14 18:13:10,022 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-1:null) (logid:) SSL error caught during wrap data: General SSLEngine problem, for local address=/10.2.2.96:8250, remote address=/10.0.0.19:45810.

10.0.0.19 is a unrelated monitoring interface

I believe that the client should identify itself using the IP or hostname which was passed when adding the KVM host to CloudStack.

CloudStack-Usage Broken after Upgrade from 4.9 to 4.11

ISSUE TYPE

• Bug Report

COMPONENT NAME

Cloudstack-Usage

CLOUDSTACK VERSION

4.11
4.9

CONFIGURATION

OS / ENVIRONMENT

Management Server Ubuntu 14.04 with repo:
deb http://packages.shapeblue.com/cloudstack/upstream/debian/4.11 /

SUMMARY

After upgrading from CloudStack 4.9 to 4.11 as described in
http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.11.0.0/upgrade/upgrade-4.9.html
cloudstack-usage does no longer startup.

STEPS TO REPRODUCE

Do upgrade as in http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.11.0.0/upgrade/upgrade-4.9.html
service cloudstack-usage start

EXPECTED RESULTS

"service cloudstack-usage status" should show a correctly running service.

ACTUAL RESULTS

"service cloudstack-usage status" indicates that cloudstack-usage is not running.

problem adding new shared network NIC to VM "A NIC with this MAC address exits for network:"

ISSUE TYPE

• Bug Report

COMPONENT NAME

Network

CLOUDSTACK VERSION

4.11

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

We're unable to create more than one NIC on a shared network through the Web-UI.

STEPS TO REPRODUCE

Log in to Web-UI
Create new shared network.
Add NIC on this new network to a VM. (works)
Add NIC on this new network to another VM. Fails with  "A NIC with this MAC address exits for network: (uuid of the current (other NIC) network)"

EXPECTED RESULTS

The second NIC should be added correctly.

ACTUAL RESULTS

Adding the NIC fails with  "A NIC with this MAC address exits for network: (uuid of the current (other NIC) network)"

Unable to add NIC to VM - Insufficient capacity when adding NIC to VM

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

OS / ENVIRONMENT

Advanced Networking, Cloudstack 4.11.0.0 on Ubuntu 14.04 LTS,
MySQL 5.5.59-0ubuntu0.14.04.1

SUMMARY

On one of our two CloudStack instances we have an issue adding NICs to
VMs through the Web-UI after upgrading to 4.11.0.0. On another instance the problem does
not occur after the upgrade. I already asked on users@cloudstack to reproduce the issue, but got no response. So this might be an individual problem.

Adding a NIC with CloudMonkey through a command like the following ist working perfectly:

create network displaytext=deleteme-cloudmonkey
name=deleteme-cloudmonkey networkofferingid= zoneid=
projectid= gateway=172.17.1.1 netmask=255.255.252.0
networkdomain=deleteme-cloudmonkey.heinlein-intern.de

This issue maybe is close to #2533 "Web-UI creates all isolated Nets with IP range 10.1.1.0/24" - in both cases the API call through CloudMonkey works fine, but using the web-UI leads to an error. Maybe it could make sense to investigate those two issues together.

STEPS TO REPRODUCE

Create a new isolated network
Wait until network is "implemented"
Add NIC on this new network to a VM

EXPECTED RESULTS

To have the NIC added to VM and network. ;)

ACTUAL RESULTS

=> Textbox: Insufficient capacity when adding NIC to VM[User|i-<id>-VM]:
com.cloud.exception.InsufficientAddressCapacityException: Insufficient
address capacityScope=interface com.cloud.dc.DataCenter; id=1
=> No new NIC is added to the VM

XCP-ng 7.4 support

Support for XCP-ng 7.4 in Apache CloudStack

The new community version of XenServer - XCP-ng 7.4 was released 31/March 2018:
https://xcp-ng.github.io/news/2018/03/31/first-xcp-ng-release.html

As this will become an important addition to the CloudStack hypervisor ecosystem support for this will be a great feature. Since this version should be close to feature compatible with XenServer 7.4 the existing XenServer code can hopefully be utilised.

Test results

Test environment:

• CloudStack 4.11 on CentOS 7.4
• Hypervisors running first release of XCP-ng version 7.4
• NFS storage

When tested in CloudStack 4.11 this was as suspected not recognised:

2018-04-03 11:34:29,334 INFO [c.c.r.ResourceManagerImpl] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) Trying to add a new host at http://IPremoved in data center 1
2018-04-03 11:34:29,638 DEBUG [c.c.h.x.d.XcpServerDiscoverer] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) host IPremoved doesn't have 996dd2e7-ad95-49cc-a0be-2c9adc4dfb0b Hotfix
2018-04-03 11:34:29,648 DEBUG [c.c.h.x.d.XcpServerDiscoverer] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) host IPremoved doesn't have 0850b186-4d47-11e3-a720-001b2151a503 Hotfix
2018-04-03 11:34:29,659 WARN [c.c.h.x.d.XcpServerDiscoverer] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) Only support XCP 1.0.0, 1.1.0, 1.4.x, 1.5 beta, 1.6.x; XenServer 5.6, XenServer 5.6 FP1, XenServer 5.6 SP2, Xenserver 6.0, 6.0.2, 6.1.0, 6.2.0, >6.4.0 but this one is XCP-ng 7.4.0
2018-04-03 11:34:29,660 DEBUG [c.c.h.x.d.XcpServerDiscoverer] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) other exceptions: java.lang.RuntimeException: Only support XCP 1.0.0, 1.1.0, 1.4.x, 1.5 beta, 1.6.x; XenServer 5.6, XenServer 5.6 FP1, XenServer 5.6 SP2, Xenserver 6.0, 6.0.2, 6.1.0, 6.2.0, >6.4.0 but this one is XCP-ng 7.4.0
java.lang.RuntimeException: Only support XCP 1.0.0, 1.1.0, 1.4.x, 1.5 beta, 1.6.x; XenServer 5.6, XenServer 5.6 FP1, XenServer 5.6 SP2, Xenserver 6.0, 6.0.2, 6.1.0, 6.2.0, >6.4.0 but this one is XCP-ng 7.4.0
at com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer.createServerResource(XcpServerDiscoverer.java:439)
at com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer.createServerResource(XcpServerDiscoverer.java:454)
at com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer.find(XcpServerDiscoverer.java:301)
at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:756)
at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:601)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy178.discoverHosts(Unknown Source)
at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:731)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:555)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:310)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:127)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:89)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:686)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:530)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
2018-04-03 11:34:29,660 WARN [c.c.r.ResourceManagerImpl] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) Unable to find the server resources at http://IPremoved
2018-04-03 11:34:29,661 INFO [c.c.u.e.CSExceptionErrorCode] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
2018-04-03 11:34:29,661 WARN [o.a.c.a.c.a.h.AddHostCmd] (qtp788117692-16:ctx-040da28f ctx-9347d25e ctx-ab04f31e) (logid:a961de98) Exception:
com.cloud.exception.DiscoveryException: Unable to add the host

Templates created under project are not visible

ISSUE TYPE

• Bug Report

COMPONENT NAME

Management server

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

CEPH storage, Advanced networking

OS / ENVIRONMENT

Centos 7.4.1708

SUMMARY

Trying to create a template from VM.
VM was created, prepared and stopped under the project.
Template creation task was completed successfully in terms that I can see notification about successful template creation. I can see job status in logs that everything is ok.
At the same time created template are shown nowhere in UI, also I can't get it through cloudmonkey

list templates templatefilter=all listall=true isrecursive=true

but if I specify template uuid directly

list templates listall=true templatefilter=all id=c350641c-26e6-484d-b0df-058b96ac3911

template is shown

At the same time I can see it in cs db in vm_template table for example.
I tried to investigate a little, what went wrong.. I don't now internals of cs and there are no docs, but what looks strange for me:
account_id is set to value, that belongs to account which was never created by me (manually or automatically).
This account is not showing in UI, but is presented in cs db table account

id: 7
account_name: PrjAcct-spg-infra_srv-3
uuid: 4e3ad0e6-919e-4066-b81a-9ea555f84e4e
type: 5
role_id: NULL
domain_id: 3
state: enabled
removed: NULL
cleanup_needed: 0
network_domain: NULL
default_zone_id: NULL
default: 0

Why role_id is null?
After some testing, I found that for every project account of type 5 is created.
Type 5 is defined, but can't find its role in cs (Account).

So may be the reason, that created template is owned by this special account

STEPS TO REPRODUCE

• Create vm under project
• Stop vm
• Create template from ROOT disk

EXPECTED RESULTS

Template is shown in "Templates" area

ACTUAL RESULTS

Template is not shown anywhere in UI and partly visible through API

Could not connect to SMTP host using STARTTLS

ISSUE TYPE

Other

COMPONENT NAME

Management Node

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

• Advanced Networking
• KVM

OS / ENVIRONMENT

CentOS Linux release 7.4.1708

Summary

Goal: integrate CloudStack with external SMTP relay.
SMTP relay is Office365 relay from Office365 subscription.
Parameters to connect: smtp.office365.com:587

In general there are two cases for secure connection:

• SSL (First SSL handshake, then SMTP HELO)
• TLS (First SMTP HELO, then STARTLS)

CS does not allow to specify method to use and by default use SSL, what leads to errors:

2018-04-25 14:26:15,723 WARN  [c.c.p.ProjectManagerImpl] (API-Job-Executor-5:ctx-0242c153 job-423 ctx-8109c588) (logid:13177452) Failed
 to send project id=Project[2|name=test_project|domainid=2] invitation to the email <removed>; removing the invitation record from the db
javax.mail.MessagingException: Could not connect to SMTP host: smtp.office365.com, port: 587;
  nested exception is:
        javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
        at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1961)
        at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:654)
        at javax.mail.Service.connect(Service.java:295)
        at javax.mail.Service.connect(Service.java:176)
        at javax.mail.Service.connect(Service.java:125)
        at com.cloud.projects.ProjectManagerImpl$EmailInvite.sendInvite(ProjectManagerImpl.java:990)
        at com.cloud.projects.ProjectManagerImpl.generateTokenBasedInvitation(ProjectManagerImpl.java:708)
        at com.cloud.projects.ProjectManagerImpl.inviteAccountToProject(ProjectManagerImpl.java:590)
        at com.cloud.projects.ProjectManagerImpl.addAccountToProject(ProjectManagerImpl.java:563)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
        at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy140.addAccountToProject(Unknown Source)
        at org.apache.cloudstack.api.command.user.account.AddAccountToProjectCmd.execute(AddAccountToProjectCmd.java:91)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
        at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
        at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
        at sun.security.ssl.InputRecord.read(InputRecord.java:527)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
        ... 45 more
2018-04-25 14:26:15,741 WARN  [c.c.p.ProjectManagerImpl] (API-Job-Executor-5:ctx-0242c153 job-423 ctx-8109c588) (logid:13177452) Failed to generate invitation for email <removed> to project id=Project[2|name=test_project|domainid=2]

Option is described here (mail.smtp.starttls.enable): Java Mail

New vmware.ports.per.dvportgroup global setting breaks existing deployments

ISSUE TYPE

• Bug Report

COMPONENT NAME

Hypervisor (VMware)

CLOUDSTACK VERSION

4.11.0

CONFIGURATION

dvSwitch in VMware environment >5.0

OS / ENVIRONMENT

SUMMARY

commit:
https://github.com/sureshanaparti/cloudstack/commit/deb2f530b71c1dc13665c0b8dacfb025c38f58b9#diff-2e10685e0f701ef7fd43267f2879bb8e

sets vmware.ports.per.dvportgroup to a low value allowing autogrow to set the value going forward.  However, If a portgroup already exists and has more than (say) 10 ports in use already then a error such as:

> Cannot set number of ports to 10 because at least 11 out of 18 ports are still in use

will be thrown when creating a VM on said dvSwitch and the VM will fail to be deployed.
Setting ALL dvsportgroups to use a sufficiently high number to cover ALL dvsportgroup values will be expensive in large environments.

STEPS TO REPRODUCE

Create a VMware environment >5.0 (5.5 in this case), which uses dvSwitches - this example manifests itself on the mgmt network.  CloudStack version 4.6
use the environment sufficiently to create more than 10 ports on mgmt dvSwitch (or any other network to create other failures).

Upgrade to ACS 4.11.0

Attempt to destroy CPVM in order for it to be recreated.

EXPECTED RESULTS

CPVM is destroyed and recreated (CloudStack either leaves number of ports attached 'as is' or reduces the number of ports in the portgroup to slightly greater than the number of ports in use.

ACTUAL RESULTS

An error is thrown:

> Cannot set number of ports to 10 because at least 11 out of 18 ports are still in use

CPVM is not deployed.

Unknown parameters : projectid

ISSUE TYPE

Other

COMPONENT NAME

Management Node

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

• Advanced Networking
• KVM

OS / ENVIRONMENT

CentOS Linux release 7.4.1708

SUMMARY

During different UI operations (VPC creation, VM management, Forwarding Rules managements and etc.) I can see such WARNINGS:

2018-05-04 16:45:33,690 WARN  [c.c.a.d.ParamGenericValidationWorker] (qtp788117692-323:ctx-9c5a250d ctx-83e49862) (logid:87d0fc5a) Received unknown parameters for command listNics. Unknown parameters : projectid
2018-05-04 16:45:35,497 WARN  [c.c.a.d.ParamGenericValidationWorker] (qtp788117692-13:ctx-a6e4db2e ctx-06296171) (logid:7cb8a0e1) Received unknown parameters for command createPortForwardingRule. Unknown parameters : projectid

STEPS TO REPRODUCE

I don't now if such messages mean that something will go wrong. But as I understand if you create some entity under project, cs should link somehow this entity to the project

DB Exceptions seen on latest master

On latest master, with only db deployed and no environments I see this:

2018-05-14 16:57:47,617 INFO  [c.c.r.ResourceLimitManagerImpl] (ResourceCountChecker-1:ctx-ed4a764d) (logid:d70d0be5) Started resource counters recalculation periodic task.
2018-05-14 16:57:47,637 WARN  [c.c.r.ResourceLimitManagerImpl] (ResourceCountChecker-1:ctx-ed4a764d) (logid:d70d0be5) Discrepency in the resource count (original count=0 correct count = 13) for type template for account ID 1 is fixed during resource count recalculation.
2018-05-14 16:57:47,657 DEBUG [c.c.u.d.T.Transaction] (ResourceCountChecker-1:ctx-ed4a764d) (logid:d70d0be5) Rolling back the transaction: Time = 2 Name =  ResourceCountChecker-1; called by -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-ReflectiveMethodInvocation.proceed:174-ExposeInvocationInterceptor.invoke:92-ReflectiveMethodInvocation.proceed:185-JdkDynamicAopProxy.invoke:212-$Proxy71.countCpuNumberAllocatedToAccount:-1-ResourceLimitManagerImpl.countCpusForAccount:929-ResourceLimitManagerImpl$4.doInTransaction:902-ResourceLimitManagerImpl$4.doInTransaction:872
2018-05-14 16:57:47,658 WARN  [c.c.u.d.T.Transaction] (ResourceCountChecker-1:ctx-ed4a764d) (logid:d70d0be5) txn: Commit called when it is not a transaction: -Transaction.execute:41-Transaction.execute:47-ResourceLimitManagerImpl.recalculateAccountResourceCount:872-ResourceLimitManagerImpl$ResourceCountCheckTask.runInContext:1065-ManagedContextRunnable$1.run:49-DefaultManagedContext$1.call:56-DefaultManagedContext.callWithContext:103-DefaultManagedContext.runWithContext:53-ManagedContextRunnable.run:46-Executors$RunnableAdapter.call:511-FutureTask.runAndReset:308-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301:180
2018-05-14 16:57:47,661 DEBUG [c.c.u.d.T.Transaction] (ResourceCountChecker-1:ctx-ed4a764d) (logid:d70d0be5) Rolling back the transaction: Time = 3 Name =  tx-320; called by -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-ReflectiveMethodInvocation.proceed:174-ExposeInvocationInterceptor.invoke:92-ReflectiveMethodInvocation.proceed:185-JdkDynamicAopProxy.invoke:212-$Proxy71.countMemoryAllocatedToAccount:-1-ResourceLimitManagerImpl.calculateMemoryForAccount:933-ResourceLimitManagerImpl$4.doInTransaction:904-ResourceLimitManagerImpl$4.doInTransaction:872

ISSUE TYPE

• Bug Report

COMPONENT NAME

DB, API, background tasks

CLOUDSTACK VERSION

master, possibly 4.11 branch

OS / ENVIRONMENT

Basic install with no env deployed

STEPS TO REPRODUCE

Deploy latest master, start mgmt server with no env deployed see logs

EXPECTED RESULTS

Exceptions should not be seen

Templates from XenServer snapshots fail to deploy VM

SUMMARY

Template created from XenServer snapshots fail to deploy VM. A workaround exists to set snapshot.delta.max = 0.

ISSUE TYPE

• Bug Report

COMPONENT NAME

XenServer

CLOUDSTACK VERSION

Affected ACS since 4.9

STEPS TO REPRODUCE

Create two volume snapshots of a VM, create a template from the last snapshot.
Deploy a VM, it will fail to boot. The vhd file size will be much lower than the original volume.

EXPECTED RESULTS

VM deployment should have worked, the template should have been a coalesced vhd file.

ACTUAL RESULTS

VM deployment fails.

Latest Docker container is broken

ISSUE TYPE

• Bug Report

COMPONENT NAME

Docker

CLOUDSTACK VERSION

Master branch (15afc35)

CONFIGURATION

No configuration at all, just ran the docker container with the command specified on:

https://hub.docker.com/r/cloudstack/simulator/

OS / ENVIRONMENT

Docker 17.12.0-ce running on Ubuntu 16.04.3 LTS

SUMMARY

The container stops doing anything after setting up mysqld (and restarting it around 8 times) and the web server on port 8080 returns error 503 Service Unavailable.

STEPS TO REPRODUCE

docker pull cloudstack/simulator
docker run --name cloudstack -d -p 8080:8080 cloudstack/simulator
<open browser at http://localhost:8080/client/>

EXPECTED RESULTS

A functional Cloudstack dashboard

ACTUAL RESULTS

503 Service Unavailable

Powered by Jetty

Fix Jetty configuration file bugs during Upgrade from 4.x to 4.11

ISSUE TYPE

• Bug Report

COMPONENT NAME

packaging

CLOUDSTACK VERSION

4.11
master

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

The packaging/centos7/cloud.spec file line 400 moves the %{_sysconfdir}/sysconfig/%{name}-management file to %{_sysconfdir}/default/%{name}-management.
This is unnecessary because the file contains the deprecated tomcat configurations. Moving it prevents the 4.11 upgrade from overwriting the old file with the new jetty configurations file.

Also systemctl daemon-reload command needs to be added to the end of the %post management step in order to refresh daemon services so that management server can start up seamlessly after the upgrade.

STEPS TO REPRODUCE

To Reproduce: install version 4.6 of cloudstack.
Update Systemvm templates. http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.11.0.0/upgrade/upgrade-4.5.html#update-system-vm-templates
Stop management server and usage server.
Do database backup.
Change the /etc/yum.repos.d/cloudstack.repo file to upgrade to 4.11.
Do sudo yum upgrade cloudstack-management.
After upgrade it fails to start the management server because:

1. the less /etc/default/cloudstack-management file still contains the old tomcat configurations and not the new 4.11 Jetty server configurations.
2. the systemctl daemon-reload needs to be executed to relaod the daemon services

systemctl stop cloudstack-management
systemctl stop cloudstack-usage
mysqldump -u root -p cloud > cloud-backup_`date '+%Y-%m-%d'`.sql
mysqldump -u root -p cloud_usage > cloud_usage-backup_`date '+%Y-%m-%d'`.sql
vi /etc/yum.repos.d/cloudstack.repo
sudo yum upgrade cloudstack-management
sudo yum upgrade cloudstack-usage
systemctl start cloudstack-management
systemctl start cloudstack-usage

EXPECTED RESULTS

after upgrade:
less /etc/default/cloudstack-management should show the new jetty server environment variables
management server should start up successfully without manually executing systemctl daemon-reload
able to log in to the UI  

ACTUAL RESULTS

management server failed to start and returned an error message "Job for cloudstack-management.service failed because the control process exited with error code. ..."

systemctl start cloudstack-management produced the above error output

registered windows10 ISO

I registered windows10 ISO
, creating an instance, unable to create, indicating that Windows system error has been restarted and unable to enter the system. I think it is not compatible with windows10.

listResourceDetails: ID is not UUID, but DB ID

ISSUE TYPE

• Enhancement
• Bug

COMPONENT NAME

API

CLOUDSTACK VERSION

4.11

CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

The respone of listResourceDetails includes an ID of the primary field not the UUID

STEPS TO REPRODUCE

 $ cs listResourceDetails resourcetype=uservm resourceid=d949e443-9bc8-4399-9e09-842bbb395cae --region=simulator     

EXPECTED RESULTS

{
  "count": 3, 
  "resourcedetail": [
    {
      "fordisplay": true, 
      "key": "foo", 
      "resourceid": "d949e443-9bc8-4399-9e09-842bbb395cae", 
      "resourcetype": "UserVm", 
      "value": "bar"
    }, 

ACTUAL RESULTS

{
  "count": 3, 
  "resourcedetail": [
    {
      "fordisplay": true, 
      "key": "foo", 
      "resourceid": "5", 
      "resourcetype": "UserVm", 
      "value": "bar"
    }, 

Using the simulator to run tests

ISSUE TYPE

• Bug Report
• Documentation Report

COMPONENT NAME

simulator, test, marvin

CLOUDSTACK VERSION

master

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

Trying to run the simulator and to perform the integration tests, the maven command to call Marvin to deploy the DC and run the test is failing: http://docs.cloudstack.apache.org/en/latest/developer_guide.html#using-the-simulator

STEPS TO REPRODUCE

Perform the steps in the simulator documentation and the step to perform the integration tests is failing.

mvn -Pdeveloper,marvin.setup -Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test

EXPECTED RESULTS

Tests running successfully

ACTUAL RESULTS

[DEBUG] Executing command line: python deployAndRun.py -c /home/marco/code/cloudstack-public/setup/dev/basic.cfg -d /home/marco/code/cloudstack-public/tools/marvin/marvin/testSetupSuccess.py
Usage: deployAndRun.py [options]

deployAndRun.py: error: no such option: -c
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.641 s
[INFO] Finished at: 2018-04-27T15:00:31+02:00
[INFO] Final Memory: 62M/449M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (pre-integration-test) on project cloud-marvin: Command execution failed.: Process exited with an error: 2 (Exit value: 2) -> [Help 1]

Related to this PR I'm trying to test locally #2428

Add "Lets Encrypt CA" Certpath to SSVM Keystore (for cdimage.debian.org)

ISSUE TYPE

• Improvement Request

COMPONENT NAME

SSVM

CLOUDSTACK VERSION

4.11

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

The Download of ISO Images from https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso fails.

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The site "cdimage.debian.org" uses Certificates from "Lets Encrypt".

Could it be the certification path for "Lets Encrypt" is not included in the Java Keystore on the SSVM? If so, could "Lets Encrypt" be added to the Keystore?

STEPS TO REPRODUCE

Log in to Web-UI
Register ISO
Enter Download URL "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso"

EXPECTED RESULTS

The Debian ISO image should be installed correctly.

ACTUAL RESULTS

Download fails with:

    sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Ceph RBD primary storage fails connection and renders node unusable

ISSUE TYPE

• Bug Report

COMPONENT NAME

Cloudstack agent

CLOUDSTACK VERSION

4.11

CONFIGURATION

KVM cluster with CEPH backed RDB primary storage

OS / ENVIRONMENT

Ubuntu 16.04 / 14.04

SUMMARY

On a perfectly working 4.10 node with KVM hypervisor and Ceph RBD primary storage, after upgrading to 4.11, cloudstack agent is unable to connect the BRD pool in libvirt, giving just a generic "operation not supported" error in its logs:

2018-04-06 16:27:37,650 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:91b4e1df) Attempting to create storage pool be80af6a-7201-3410-8da4-9b3b58c4954f (RBD) in libvirt
2018-04-06 16:27:37,652 WARN [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:91b4e1df) Storage pool be80af6a-7201-3410-8da4-9b3b58c4954f was not found running in libvirt. Need to create it.
2018-04-06 16:27:37,653 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:91b4e1df) Didn't find an existing storage pool be80af6a-7201-3410-8da4-9b3b58c4954f by UUID, checking for pools with duplicate paths
2018-04-06 16:27:37,664 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:91b4e1df) Failed to create RBD storage pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor on: storagepool1:6789,: Operation not supported
2018-04-06 16:27:42,762 INFO [cloud.agent.Agent] (Agent-Handler-4:null) (logid Lost connection to the server. Dealing with the remaining commands...

Exactly the same pool was previously working before upgrade:

2018-04-06 12:53:52,847 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:14dace5e) Attempting to create storage pool be80af6a-7201-3410-8da4-9b3b58c4954f (RBD) in libvirt
2018-04-06 12:53:52,850 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:14dace5e) Found existing defined storage pool be80af6a-7201-3410-8da4-9b3b58c4954f, using it.
2018-04-06 12:53:52,850 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:14dace5e) Trying to fetch storage pool be80af6a-7201-3410-8da4-9b3b58c4954f from libvirt
2018-04-06 12:53:53,171 INFO [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:14dace5e) Proccess agent ready command, agent id = 46

STEPS TO REPRODUCE

Take an existing and working cloudstack cluster with 4.10, with RDB primary storage and Ubuntu 14.04 based agents and upgrade them to version 4.11 of the agent.

EXPECTED RESULTS

The cluster should be working fine, the agents should be connecting and the RDB pool should be correctly opened in libvirt.

ACTUAL RESULTS

Cloudstack agents fails to boot with a generic "Failed to create RBD storage pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor on: storagepool1:6789,: Operation not supported" error and loops in a failed state, rendering the machine unusable.

WORKAROUND

To workaround the issue I have tried to use the following XML config (dumped from another node where it is correctly running) and define the pool directly in libvirt, and it worked as expected:

be80af6a-7201-3410-8da4-9b3b58c4954f be80af6a-7201-3410-8da4-9b3b58c4954f cephstor1

virsh pool-define test.xml
Pool be80af6a-7201-3410-8da4-9b3b58c4954f defined from test.xml

root@compute6:~# virsh pool-start be80af6a-7201-3410-8da4-9b3b58c4954f
Pool be80af6a-7201-3410-8da4-9b3b58c4954f started

root@compute6:~# virsh pool-info be80af6a-7201-3410-8da4-9b3b58c4954f
Name: be80af6a-7201-3410-8da4-9b3b58c4954f
UUID: be80af6a-7201-3410-8da4-9b3b58c4954f
State: running
Persistent: yes
Autostart: no
Capacity: 10.05 TiB
Allocation: 2.22 TiB
Available: 2.71 TiB

And now the cloudstack agent correctly starts:

2018-04-09 10:29:19,989 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:f0021131) Attempting to create storage pool be80af6a-7201-3410-8da4-9b3b58c4954f (RBD) in libvirt
2018-04-09 10:29:19,990 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:f0021131) Found existing defined storage pool be80af6a-7201-3410-8da4-9b3b58c4954f, using it.
2018-04-09 10:29:19,991 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:f0021131) Trying to fetch storage pool be80af6a-7201-3410-8da4-9b3b58c4954f from libvirt
2018-04-09 10:29:20,372 INFO [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:f0021131) Proccess agent ready command, agent id = 56

cloud-early-config detects unknown hypervisor type "xen-domU"

ISSUE TYPE

• Bug Report

COMPONENT NAME

systemVM image 4.11 for XenServer

CLOUDSTACK VERSION

4.11.0 as well as 4.11.1 git from today.
SystemVM: 4.11.0 Sun Jan 14 15:37:28 UTC 2018

CONFIGURATION

XenServer 6.5 SP1

OS / ENVIRONMENT

Management Server on CentOS 7.
SystemVMs running as "Debian 9.3" in HVM mode.

SUMMARY

Newly created system VMs are not configured, cloud.log reports "systemvm type=" (nothing else).

the /opt/cloud/bin/setup/cloud-early-config script tries to detect the hypervisor type, there are cases for "xen-pv" and "xen-hvm". But it uses "virt-what", which returns the unknown type "xen-domU".

After applying these fixes...

• add "xen-domU" to the "xen-hvm" case
• changing the path to xenstore-read from /usr/sbin to /usr/bin

and rebooting, the SystemVM comes up correctly.

Errors in Spanish translation

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

latest

CONFIGURATION

OS / ENVIRONMENT

N/A

SUMMARY

There are some errors in the Spanish translation: "esta" when used as a verb should be written "está", text have two spaces between words, "mio" is incorrect (it should be "mío").

STEPS TO REPRODUCE

EXPECTED RESULTS

I expected a good translation

ACTUAL RESULTS

It had some errors.

Autogenerate marvin wrapper files for external plugins

ISSUE TYPE

• Enhancement Request

COMPONENT NAME

API, plugin, marvin

CLOUDSTACK VERSION

master

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

The Marvin wrapper files are currently only generated for APIs and internal plugins within CloudStack. We wish to have an automated generic way of generating this wrapper files for external plugins that are not part of CloudStack code.

STEPS TO REPRODUCE

Automation for an external plugin

EXPECTED RESULTS

ACTUAL RESULTS

Admin Dashboard System Capacity broken with German Locale

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.11

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

With german locale settings, the graphics in the section "System Capacity" in the Dashboard are displayed incorrectly.

STEPS TO REPRODUCE

echo LANG=\"de_DE.UTF-8\" > /etc/default/locale

service cloudstack-management restart

Login to CloudStack Web-UI

EXPECTED RESULTS

See the graphics for "System Capacity" displayed correctly.

ACTUAL RESULTS

The graphics show all red circles.

configdrive testing needs refactorring

Config drive refactoring has led to an external utility being used in a unittest for it (genisoimage). This test must be done, however but in an integration test.
#2665 deletes the original test
#2651 introduced the change and has validity so we should keep the new way of creating the configdrive and only change the way of testing it.

ISSUE TYPE

• Other (test improvement)

COMPONENT NAME

CLOUDSTACK VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

Ceph RBD Feature flag update

• Improvement Request
  Cloudstack Ceph Integration

When use Ceph >= Jewel / KVM with Cloudstack the RBD feature codes are not the new default values.

SUMMARY

When Cloudstack creates a VM from template it manually specifies the feature set and forces layering only. If you manually clone the volume via the rbd command line tool, the expected features are retained from the parent template.

The features that I require are for these use cases:

1.Using 'rbd du' the get the real disk usage of a volume without fast-diff is very slow. We need to schedule this after hours on large images.
2.We are starting to use rbd mirroring across to our secondary DC and this requires exclusive-lock.
3.Deep flatten can not be turned on after image creation. I am not sure if I require this feature at this time.

STEPS TO REPRODUCE

ACTUAL RESULTS

Example of VM volume from template: - 

user@NAS-AUBUN-RK3-CEPH01:~# rbd info AUBUN-KVM-CLUSTER01-SSD/feeb52ec-f111-4a0d-9785-23aadd7650a5

rbd image 'feeb52ec-f111-4a0d-9785-23aadd7650a5':
        size 150 GB in 38400 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.142926a5ee64
        format: 2
        features: layering
        flags:
        create_timestamp: Fri Apr 27 12:46:21 2018
        parent: AUBUN-KVM-CLUSTER01-SSD/d7dcd9e4-ed55-44ae-9a71-52c9307e53b4@cloudstack-base-snap
        overlap: 150 GB

Note the features are not the same as the parent : -

user@NAS-AUBUN-RK3-CEPH01:~# rbd info AUBUN-KVM-CLUSTER01-SSD/d7dcd9e4-ed55-44ae-9a71-52c9307e53b4
rbd image 'd7dcd9e4-ed55-44ae-9a71-52c9307e53b4':
        size 150 GB in 38400 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.141d274b0dc51
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
        flags:
        create_timestamp: Fri Apr 27 12:37:05 2018

@wido

Libvirt-bin has trouble starting if -d is stated in /etc/default/libvirt-bin (libvirtd_opts)

ISSUE TYPE

• Other

COMPONENT NAME

Apache CloudStack Agent (4.9.3) & libvirt-bin (1.3.1)

CLOUDSTACK VERSION

Apache CloudStack 4.9.3

CONFIGURATION

Basic network zone. Three compute nodes running KVM & libvirt-bin. Ceph is being utilized for Primary Storage.

OS / ENVIRONMENT

Ubuntu 16.04.4 LTS

SUMMARY

* Libvirt-bin has trouble starting if -d is stated in /etc/default/libvirt-bin. Referencing this from Apache CloudStack documentation (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.9/hypervisor/kvm.html#install-and-configure-libvirt). 

STEPS TO REPRODUCE

1.) Add -d in libvirtd_opts="" within /etc/default/libvirt-bin. -l works fine for listening on TCP.

EXPECTED RESULTS

Libvirt-bin should've started with no errors. The parameters -d -l as libvirtd_opts should exist together, per the CloudStack 4.9 documentation.

ACTUAL RESULTS

Libvirt-bin fails to start. This also causes the ACS agent to fail as well. If you take out the -d in libvirtd_opts and leave the -l, libvirt-bin works great. I followed this solution to solve the problem:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1594902
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1594902/comments/10

I thought I would report this, just in case anyone running Ubuntu 16.04 with ACS 4.9.3 has the same problem.

Textbox "Account and projectId can't be specified together"

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.11

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

When I view the "Networks" view after selecting a project in the web-UI a textbox with the message "Account and projectId can't be specified together" has to be confirmed.

STEPS TO REPRODUCE

Log in to the web-UI.
Select a project.
Select the "Network" view

EXPECTED RESULTS

No textbox. ;)

ACTUAL RESULTS

A textbox with the message "Account and projectId can't be specified together" has to be confirmed.

Web-UI creates all isolated Nets with IP range 10.1.1.0/24

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.11.0.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

After upgrading to 4.11 we have the issue, that isolated nets created with the web-UI are always created for the IP range 10.1.1.0/24 - no matter what values are filled in the fields "Guest Gateway" and "Guest Netmask".

Creating an isolated network with CloudMonkey works perfectly using the syntax:

create network displaytext=deleteme-cloudmonkey name=deleteme-cloudmonkey networkofferingid= zoneid= projectid= gateway=172.17.1.1 netmask=255.255.252.0 networkdomain=deleteme-cloudmonkey.heinlein-intern.de

Paul Angus confirmed on [email protected]:

Yes looks like a UI bug. I tried through the UI with the browser in developer mode to see what was sent. The guest netmask is missing.http://192.168.56.11:8080/client/api?command=createNetwork&response=json&zoneId=6d0ebc4e-1f70-4a87-b29b-cb98f9ab9594&name=test_net&displayText=test-sourcenat&networkOfferingId=e6425385-e5d2-457a-b3df-4845b9e79edc&gateway=10.0.0.1&_=1522849857187

STEPS TO REPRODUCE

Login to web-UI with DomainAdmin User.
Create new isolated network.
Specifiy 
"Guest Gateway" other than 10.1.1.1 or
"Guest Netmask" other than 24

EXPECTED RESULTS

Have the "Guest Gateway" and "Guest Netmask" configured according to the values entered.

ACTUAL RESULTS

The network is created for guest IP range 10.1.1.1/24 no matter what values where entered for "Guest Gateway" and "Guest Netmask".