GithubHelp home page GithubHelp logo

eucalyptus / eutester Goto Github PK

View Code? Open in Web Editor NEW
36.0 17.0 44.0 15.01 MB

Automated Test Framework for Eucalyptus

License: Other

Python 61.82% Java 22.50% Groovy 15.06% Erlang 0.39% HTML 0.04% RobotFramework 0.12% Shell 0.07%

eutester's Introduction

Stories in Ready Eutester

Build Status

Intro to Eutester

Video - Eutester Overview: An Introduction to the Functional Testing Framework for Eucalyptus

Getting Setup

Writing Your First Testcase

Guidelines for Contributing Testcases

Creating a Eucalyptus Test Harness: Jenkins, Testlink, and Eutester

Eutester Documentation on packages.python.org

eutester is an attempt to leverage existing test code to make test writing faster and standardized.

Installation

If easy_install is not available in your environment use your package manager to install python-setuptools

yum install python-setuptools gcc python-devel git
apt-get install python-setuptools gcc python-dev git

Installing eutester and its dependencies is as easy as:

easy_install eutester

For development purposes you can then clone the code from github and then reinstall with your changes

git clone https://github.com/eucalyptus/eutester.git
cd eutester
[CHANGE CODE]
python setup.py install

Main Classes

eutester contains the framework pieces like parsing config/creds, setting up connections and providing test primitives. eucaops uses the framework provided by eutester to provide validated higher order operations on those cloud connections. For more information regarding the module structure of eutester, please refer to the Eutester Python Module Documentation Index.

class-diagram

Example test cases written with this library can be found in the testcases/unstable directory of the source tree

Design

Eutester is designed to allow a user to quickly generate automated tests for testing a Eucalyptus or Amazon cloud. In the case of testing a private cloud a configuration file can be used to create cases that require root access to the machines. The config file describes a few things about the clouds configuration including the bare metal machine configuration and IPs.

The eucaops class can be imported in order to use pre-defined routines which validate common operations on the cloud:

from eucaops import Eucaops

Constructor

The basic constructor can be used for 2 different connections:

  1. Private cloud with root access - CLC SSH, Cloud, and Walrus connections
    Purpose - connect to and manipulate Eucalyptus components using boto and command line. Recommended to use Eucaops.
    Required arguments: root password, config file with topology information
    Optional arguments: credential path so that new credentials are not created

     private_cloud = Eucaops( password="my_root_pass",  config_file="cloud.conf")
     private_cloud.sys("euca-describe-availability-zones") ### use local credentials to determine viable availability-zones
    
  2. Public cloud - local SSH, EC2 and S3 connections
    Purpose - can be used to wrap euca2ools commands installed locally on the tester machine or with Eucaops
    Required arguments: credential path

     public_cloud = Eucaops(credpath="~/.eucarc")    
     public_cloud.run_instance(image) ## run an m1.small instance using image
    

Config file

Tab Delimited: clc.mydomain.com CENTOS 6.4 64 REPO [CC00 CLC SC00 WS] nc1.mydomain.com CENTOS 6.4 64 REPO [NC00]

Network

Possible values are MANAGED, MANAGED-NOVLAN, SYSTEM, or STATIC

Columns

IP or hostname of machine  
Distro installed on machine  
Distro version on machine  
Distro base architecture
System built from packages (REPO) or source (BZR), packages assumes path to eucalyptus is /, bzr assumes path to eucalyptus is /opt/eucalyptus
List of components installed on this machine encapsulated in brackets []

These components can be:

CLC - Cloud Controller   
WS - Walrus   
SC00 - Storage controller for cluster 00   
CC00 - Cluster controller for cluster 00    
NC00 - A node controller in cluster 00   

eutester's People

Contributors

dkavanagh avatar dmitrii avatar gregdek avatar hspencer77 avatar jeevanullas avatar kushaldas avatar kwurst avatar mbacchi avatar mmunn1 avatar mspaulding06 avatar nephomaniac avatar paulweiss avatar sebgoa avatar shaon avatar sjones4 avatar tbeckham avatar vasiliykochergin avatar viglesiasce avatar waffle-iron avatar zhill avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eutester's Issues

instance.get_metadata doesn't work with credpath used with Eucaops

Issue

In testcase instancetest.py, we use credpath. Since we require credpath, instance.metadata doesn't execute anymore. The following error is given:

Setup

tester = Eucaops( credpath="/Users/hspencer/Desktop/eutester-dev/creds/epc/euca2-admin-x509/")
.....

Error

In [97]: for instance in reservation.instances:
if re.match("internal", instance.private_dns_name.split('eucalyptus.')[-1]):
instance.get_metadata("local-ipv4")[0]

....:

AttributeError Traceback (most recent call last)
/Users/hspencer/ in ()
1 for instance in reservation.instances:
2 if re.match("internal", instance.private_dns_name.split('eucalyptus.')[-1]):
----> 3 instance.get_metadata("local-ipv4")[0]
4

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/euinstance.pyc in get_metadata(self, element_path)
297 def get_metadata(self, element_path):
298 """Return the lines of metadata from the element path provided"""
--> 299 if self.tester.config:
300 isManaged = re.search("managed", self.tester.get_network_mode())
301 if not isManaged:

AttributeError: 'Eucaops' object has no attribute 'config'

The function get_dev_dir(self, match=None ) in euinstance.py not work.

Hi:
when i do snapshot test, I found the get_dev_dir() can't get the devices. I add some code to print the return value of get_dev_dir(), I saw the return value are "[ ]". print "before attach", print before_attach, print "after attach", print after_attach are added by me. In log you can see there only print two "[ ]". so may be the function get_dev_dir(self, match=None ) is not work. thanks!
my source code:

def createVolume1(self,instance,size=2):
'''
Create a volume and attach it to an instance,crean a txt file in it
'''
assert(instance is not None)
self.volume1 = self.tester.create_volume(instance.placement, size)
device_path = "/dev/" + instance.block_device_prefix +"j"
before_attach = instance.get_dev_dir()
#this line is added by zhangjian for debug
print "before attach"
print before_attach
try:
assert(self.tester.attach_volume(instance, self.volume1, device_path,timeout=180)==True)
except AssertionError, e:
assert(self.__deleteVolume(self.volume1,timeout=180)==True)
self.volome1 = None
return False
after_attach = instance.get_dev_dir()
print "after attach"
print after_attach
new_devices = self.tester.diff(after_attach, before_attach)
if len(new_devices) is 0:
return False
self.volume_device1 = "/dev/" + new_devices[0].strip()
instance.assertFilePresent(self.volume_device1)

    # create filesystem for volum1
    instance.sys("mkfs -t ext2 " +self.volume_device1)        
    # create mnt dir
    instance.sys("mkdir "+self.volume_mnt1)
    # mount volume1
    instance.sys("mount -t ext2 "+self.volume_device1 +" "+ self.volume_mnt1)
    # create a file in the volume
    instance.sys("touch "+self.volume_mnt1+"/"+self.testfilename)
    instance.sys("echo HelloWorld! >" +self.volume_mnt1+"/"+self.testfilename)
    # umount the volume
    instance.sys("umount "+self.volume_device1)
    # detach the volume
    self.tester.detach_volume(self.volume1,timeout=180)
    self.tester.sleep(5)
    return True

my test log:

KEYWORD: ${BOOL_VAR} = snapshotbasic.Create Volume 1 ${INSTANCE} Expand All
Documentation:
Create a volume and attach it to an instance,crean a txt file in it
Start / End / Elapsed: 20121031 14:43:48.634 / 20121031 14:44:32.609 / 00:00:43.975
14:43:48.636 DEBUG Sending create volume request
14:43:48.637 DEBUG Method: POST
14:43:48.637 DEBUG Path: /services/Eucalyptus/
14:43:48.637 DEBUG Data:
14:43:48.637 DEBUG Headers: {}
14:43:48.637 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:43:48.638 DEBUG Token: None
14:43:48.638 DEBUG using _calc_signature_2
14:43:48.639 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=CreateVolume&AvailabilityZone=CC&SignatureMethod=HmacSHA256&SignatureVersion=2&Size=2&Timestamp=2012-10-31T06%3A43%3A48Z&Version=2009-11-30
14:43:48.639 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=CreateVolume&AvailabilityZone=CC&SignatureMethod=HmacSHA256&SignatureVersion=2&Size=2&Timestamp=2012-10-31T06%3A43%3A48Z&Version=2009-11-30
14:43:48.639 DEBUG len(b64)=44
14:43:48.639 DEBUG base64 encoded digest: R0+FgAa8go2LD5qLQFbfYfhHSVel3nWlHIIlYZyAwf4=
14:43:48.640 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=CreateVolume&AvailabilityZone=CC&SignatureMethod=HmacSHA256&SignatureVersion=2&Size=2&Timestamp=2012-10-31T06%3A43%3A48Z&Version=2009-11-30 Signature: R0+FgAa8go2LD5qLQFbfYfhHSVel3nWlHIIlYZyAwf4=
14:43:48.920 DEBUG vol-EEC9417D2CCcreating2012-10-31T06:43:48.673Z
14:43:48.921 DEBUG Polling for volume to become available
14:43:48.922 DEBUG Volume (vol-EEC9417D) State(creating) sleeping 10s
14:43:58.927 DEBUG Method: POST
14:43:58.927 DEBUG Path: /services/Eucalyptus/
14:43:58.927 DEBUG Data:
14:43:58.927 DEBUG Headers: {}
14:43:58.941 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:43:58.942 DEBUG Token: None
14:43:58.942 DEBUG using _calc_signature_2
14:43:58.942 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A43%3A58Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:43:58.943 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A43%3A58Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:43:58.943 DEBUG len(b64)=44
14:43:58.943 DEBUG base64 encoded digest: tcmLZGP0N9D2lNKpRJh2P9HjFzRgNJkhW6IQopvUMXE=
14:43:58.943 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A43%3A58Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: tcmLZGP0N9D2lNKpRJh2P9HjFzRgNJkhW6IQopvUMXE=
14:43:59.428 DEBUG vol-EEC9417D2CCcreating2012-10-31T06:43:48.673Z
14:43:59.429 DEBUG Volume:vol-EEC9417D in creating state
14:43:59.430 DEBUG Volume (vol-EEC9417D) State(creating) sleeping 10s
14:44:09.434 DEBUG Method: POST
14:44:09.434 DEBUG Path: /services/Eucalyptus/
14:44:09.435 DEBUG Data:
14:44:09.435 DEBUG Headers: {}
14:44:09.435 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:09.435 DEBUG Token: None
14:44:09.435 DEBUG using _calc_signature_2
14:44:09.436 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A09Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:09.436 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A09Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:09.436 DEBUG len(b64)=44
14:44:09.437 DEBUG base64 encoded digest: Jdwph70PJ9iJdSaJiO5coOPo2Icwu/ZwJmHrPAx4kn0=
14:44:09.437 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A09Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: Jdwph70PJ9iJdSaJiO5coOPo2Icwu/ZwJmHrPAx4kn0=
14:44:09.912 DEBUG vol-EEC9417D2CCcreating2012-10-31T06:43:48.673Z
14:44:09.913 DEBUG Volume:vol-EEC9417D in creating state
14:44:09.914 DEBUG Volume (vol-EEC9417D) State(creating) sleeping 10s
14:44:19.924 DEBUG Method: POST
14:44:19.925 DEBUG Path: /services/Eucalyptus/
14:44:19.925 DEBUG Data:
14:44:19.925 DEBUG Headers: {}
14:44:19.925 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:19.925 DEBUG Token: None
14:44:19.926 DEBUG using _calc_signature_2
14:44:19.926 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A19Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:19.926 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A19Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:19.926 DEBUG len(b64)=44
14:44:19.927 DEBUG base64 encoded digest: XdNQzt6S/I2pRneKItESl1j5I4TsmsF4d8pGj+lBzGE=
14:44:19.927 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A19Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: XdNQzt6S/I2pRneKItESl1j5I4TsmsF4d8pGj+lBzGE=
14:44:20.409 DEBUG vol-EEC9417D2CCavailable2012-10-31T06:43:48.673Z
14:44:20.410 DEBUG Volume:vol-EEC9417D in available state
14:44:20.411 DEBUG Done. Waited a total of 31 seconds
14:44:20.411 DEBUG [[email protected]]# ls -1 /dev/ | grep '^sd'
14:44:20.412 DEBUG [chan 3] Max packet in: 34816 bytes
14:44:20.468 DEBUG done with exec
14:44:20.469 DEBUG
14:44:20.469 DEBUG Sending attach for Volume:vol-EEC9417D to be attached to Instance:i-DD714141 at requested device /dev/sdj
14:44:20.469 DEBUG Method: POST
14:44:20.469 DEBUG Path: /services/Eucalyptus/
14:44:20.470 DEBUG Data:
14:44:20.470 DEBUG Headers: {}
14:44:20.470 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:20.470 DEBUG Token: None
14:44:20.470 DEBUG using _calc_signature_2
14:44:20.471 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=AttachVolume&Device=%2Fdev%2Fsdj&InstanceId=i-DD714141&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A20Z&Version=2009-11-30&VolumeId=vol-EEC9417D
14:44:20.471 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=AttachVolume&Device=%2Fdev%2Fsdj&InstanceId=i-DD714141&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A20Z&Version=2009-11-30&VolumeId=vol-EEC9417D
14:44:20.471 DEBUG len(b64)=44
14:44:20.471 DEBUG base64 encoded digest: jYsaOZWawS0Rqtx8Jbt3AyqNpas8W3061p7gOMIOl2M=
14:44:20.472 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=AttachVolume&Device=%2Fdev%2Fsdj&InstanceId=i-DD714141&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A20Z&Version=2009-11-30&VolumeId=vol-EEC9417D Signature: jYsaOZWawS0Rqtx8Jbt3AyqNpas8W3061p7gOMIOl2M=
14:44:21.114 DEBUG vol-EEC9417Di-DD714141/dev/sdjattaching2012-10-31T06:44:21.060Z
14:44:21.115 DEBUG Method: POST
14:44:21.115 DEBUG Path: /services/Eucalyptus/
14:44:21.116 DEBUG Data:
14:44:21.116 DEBUG Headers: {}
14:44:21.116 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:21.116 DEBUG Token: None
14:44:21.116 DEBUG using _calc_signature_2
14:44:21.117 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:21.117 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:21.117 DEBUG len(b64)=44
14:44:21.117 DEBUG base64 encoded digest: dfQ1jG4u6fsQuhg7E5ERcwLVLNvBKthiLbrjAS+Ao/Q=
14:44:21.118 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: dfQ1jG4u6fsQuhg7E5ERcwLVLNvBKthiLbrjAS+Ao/Q=
14:44:21.581 DEBUG vol-EEC9417D2CCin-use2012-10-31T06:43:48.673Zvol-EEC9417Di-DD714141/dev/sdjattaching2012-10-31T06:44:21.073Z
14:44:21.582 DEBUG Method: POST
14:44:21.582 DEBUG Path: /services/Eucalyptus/
14:44:21.582 DEBUG Data:
14:44:21.582 DEBUG Headers: {}
14:44:21.583 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:21.583 DEBUG Token: None
14:44:21.583 DEBUG using _calc_signature_2
14:44:21.583 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:21.584 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:21.584 DEBUG len(b64)=44
14:44:21.584 DEBUG base64 encoded digest: dfQ1jG4u6fsQuhg7E5ERcwLVLNvBKthiLbrjAS+Ao/Q=
14:44:21.584 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A21Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: dfQ1jG4u6fsQuhg7E5ERcwLVLNvBKthiLbrjAS+Ao/Q=
14:44:22.019 DEBUG vol-EEC9417D2CCin-use2012-10-31T06:43:48.673Zvol-EEC9417Di-DD714141/dev/sdjattaching2012-10-31T06:44:21.073Z
14:44:22.020 DEBUG Volume:vol-EEC9417D not 'attached', attach_data.status=attaching
14:44:22.021 DEBUG Volume:vol-EEC9417D state:in-use pause:10 elapsed:0
14:44:22.021 DEBUG Sleeping for 10 seconds
14:44:32.031 DEBUG Method: POST
14:44:32.032 DEBUG Path: /services/Eucalyptus/
14:44:32.032 DEBUG Data:
14:44:32.032 DEBUG Headers: {}
14:44:32.032 DEBUG Host: eucalyptus.vaas126.cd.nsn-rdnet.net:8773
14:44:32.033 DEBUG Token: None
14:44:32.033 DEBUG using _calc_signature_2
14:44:32.033 DEBUG query string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A32Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:32.034 DEBUG string_to_sign: POST
eucalyptus.vaas126.cd.nsn-rdnet.net:8773
/services/Eucalyptus/
AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A32Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D
14:44:32.034 DEBUG len(b64)=44
14:44:32.034 DEBUG base64 encoded digest: Zl5O9R/GTOL/ukPGEF4G5l0b8GBmP80r/WXN/+TEF1g=
14:44:32.046 DEBUG query_string: AWSAccessKeyId=UHEN1S3YAIF3ZIXMQYGGX&Action=DescribeVolumes&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-10-31T06%3A44%3A32Z&Version=2009-11-30&VolumeId.1=vol-EEC9417D Signature: Zl5O9R/GTOL/ukPGEF4G5l0b8GBmP80r/WXN/+TEF1g=
14:44:32.550 DEBUG vol-EEC9417D2CCin-use2012-10-31T06:43:48.673Zvol-EEC9417Di-DD714141/dev/sdjattached2012-10-31T06:44:21.073Z
14:44:32.551 DEBUG Volume:vol-EEC9417D, Attached: in-use - attached, elapsed:10
14:44:32.551 DEBUG [[email protected]]# ls -1 /dev/ | grep '^sd'
14:44:32.552 DEBUG [chan 4] Max packet in: 34816 bytes
14:44:32.608 DEBUG done with exec
14:44:32.608 DEBUG
14:44:32.608 INFO before attach
[]
after attach
[]
14:44:32.609 INFO ${BOOL_VAR} = False

Readme improvements

Very minor but figured I'd leave this to more knowledgable python folk. After cloning from git and running 'python setup.py install' I had to also install argparse:

easy_install argparse

Very trivial stuff but this isn't mentioned in the README.md. Maybe we suggest this step or yum install python-argparse (or apt-get whatever) package with yum install?

Perhaps split the getting started for setup on different distros?

get_emi() in eucaops selects de-registered image

Hey Vic,

Looks like get_emi() is selecting de-registered images for me?

Does this works?

--- /root/eutester/eucaops/init.py 2012-01-24 05:25:32.000000000 -0800
+++ /root/eutester/eucaops/init.py.new 2012-01-24 05:28:25.000000000 -0800
@@ -352,7 +352,7 @@
         image_id = self.ec2.register_image(name=name, description=description, kernel_id=kernel, image_location=image_location, ramdisk_id=ramdisk, block_device_map=bdmdev, root_device_name=rdn)
         return image_id
     

  •    def get_emi( self, emi="emi-", root_device_type=None, root_device_name=None, location=None, state=None, arch=None, owner_id=None):
  •    def get_emi( self, emi="emi-", root_device_type=None, root_device_name=None, location=None, state="available", arch=None, owner_id=None):

Cheers,
Deependra

Create a standardized Eutester log/output format

We need to standardize the way we present info about a running test to a user - essentially, output formatting.

Initial thoughts:

  • date
  • file
  • function
  • severity

Eutester currently uses the python logging library (http://docs.python.org/library/logging.html) which seems to have everything we want, but just needs to be customized. Clarksb has offered to take this on -- the first step will be figuring out the format we want (and then the next bit will be implementing it).

SOAP AWS API coverage

As per an IRC discussion with ke4qqq, vic and others. The question was: does eutester cover the SOAP API calls?

The summary is that, eutester uses boto and euca2ools at the moment, and both of them only uses the REST api, thus there is currently no coverage of the SOAP calls. One possibility raises during the meeting is to use ec2-tools, but, unless the license changes, we cannot use them against a non-AWS cloud. Another point (raised by gholms) was that AWS is deprecating the SOAP calls.

In the end the agreement is that we should track this and if possible add coverage for this test.

Restructure Eucaops into multiple modules

Eucaops has grown a little too large for my liking and I'd like to split it out in chunks as follows:

EC2ops
S3ops
IAMops

We would then have a Cloudops class that would inherit from all of these 3 classes.

The current Eucaops would then inherit from Cloudops and include anything that is necessary to use Euca (for example modifying properties) that does not apply to AWS.

Create a Test Directory Wiki

This directory should be the start of what will be the directory for all tests and suites that we want to bundle with eutester itself. Moving forward any testcase that gets submitted into Eutester should be accompanied with a Wiki article that defines what the test does and why it exists. When the testcase and wiki article are accepted into the codebase they will be added to the Test Directory with a link and quick summary of the test.

Allow credpath option with config_file option when instantiating Eucaops

I found when you instantiate Eucaops with the config_file and credpath variables, there is the following error:

In [2]: from eucaops import Eucaops
In [3]: tester = Eucaops(config_file="/eutester-dev/creds/euca3/config_file", credpath="/eutester-dev/creds/euca3", password="********")

[2012-04-21 11:52:15,361] [EUTESTER] [DEBUG]: Reading config file: /Users/hspencer/Desktop/eutester-dev/creds/euca3/config_file
[2012-04-21 11:52:15,409] [192.168.5.100:['clc', 'cc', 'sc', 'ws']] [DEBUG]: SSH connection has hostname:192.168.5.100 user:root password:22dark10man
[2012-04-21 11:52:16,695] [192.168.5.100:['clc', 'cc', 'sc', 'ws']] [DEBUG]: [[email protected]]# . /Users/hspencer/Desktop/eutester-dev/creds/euca3/eucarc && //usr/sbin/euca-describe-services --system-internal | grep SERVICE
[2012-04-21 11:52:16,713] [192.168.5.100:['clc', 'cc', 'sc', 'ws']] [DEBUG]: done with exec
[2012-04-21 11:52:16,713] [192.168.5.100:['clc', 'cc', 'sc', 'ws']] [DEBUG]: bash: /Users/hspencer/Desktop/eutester-dev/creds/euca3/eucarc: No such file or directory


IndexError Traceback (most recent call last)
/Users/hspencer/ in ()
----> 1 tester = Eucaops(config_file="/Users/hspencer/Desktop/eutester-dev/creds/euca3/config_file", credpath="/Users/hspencer/Desktop/eutester-dev/creds/euca3", password="***********")

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eucaops/init.pyc in init(self, config_file, password, keypath, credpath, aws_access_key_id, aws_secret_access_key, account, user, username, region, boto_debug)
17
18 def init(self, config_file=None, password=None, keypath=None, credpath=None, aws_access_key_id=None, aws_secret_access_key = None,account="eucalyptus",user="admin", username="root",region=None, boto_debug=0):
---> 19 super(Eucaops, self).init(config_file,password, keypath, credpath, aws_access_key_id, aws_secret_access_key,account, user,region, boto_debug)
20 self.poll_count = 48
21 self.username = username

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/init.pyc in init(self, config_file, password, keypath, credpath, aws_access_key_id, aws_secret_access_key, account, user, region, boto_debug)
166 raise Exception("Could not get credentials from second CLC and no other to try")
167
--> 168 self.service_manager = EuserviceManager(self)
169 self.clc = self.service_manager.get_enabled_clc().machine
170 self.walrus = self.service_manager.get_enabled_walrus().machine

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/euservice.pyc in init(self, tester)
146 raise AttributeError("Tester object does not have CLC machine to use for SSH")
147
--> 148 self.update()
149
150

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/euservice.pyc in update(self, name)
249 ### Get all services
250 self.reset()
--> 251 services = self.get(name)
252 for current_euservice in services:
253 ### If this is system wide component add it to the base level array

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/euservice.pyc in get(self, type, partition, attempt_both)
173 services = []
174 for service_line in describe_services:
--> 175 services.append(Euservice(service_line, self.tester))
176 return services
177

/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/euservice.pyc in init(self, service_string, tester)
49 self.state = values[4]
50 self.uri = values[6]
---> 51 self.fullname = values[7]
52 self.hostname = self.uri.split(":")[1].split("/")[2]
53 self.running = True

IndexError: list index out of range

If I use just config_file without a credpath, everything works fine (i.e. the credentials of the admin are downloaded via --get-credentails, and euca-describe-services is executed.)

Please let me know if you need any additional information.

Move logging functionality from eutester.py to the machine class

Currently there is an implementation in place in eutester that allows for logs to be gathered during the execution of a testcase. The implementation is simply a prototype and is hard coded in a way that does not expose the features full potential. In order to make the log capturing more flexible it should be implemented in the machine class and allow capturing an arbitrary file.

Can't delete_object or bucket

Hi,
I do below tests, and failed because can not delete object or bucket:

  1. Create a bucket
  2. upload a object to the bucket
    3.delete the object ( Return WARN CRITICAL]: Walrus bucket still exists after delete
  3. delete the bucket ( can not return from this step)

Brs
Thanks.

BTW:
The detail information are below:

from eucaops import Eucaops
import os
tester = Eucaops(credpath="/root/lincx/eucarckey")
[2012-07-06 14:39:40,169] [EUTESTER] [DEBUG]: Extracting keys from /root/lincx/eucarckey
bucketname = "mybucket"
keyname = "mykey"
localfile = os.curdir+"/"+keyname
tester.local("touch "+localfile)
[]
tester.local("echo hello > "+localfile)
[]
bucket = tester.create_bucket(bucketname)
[2012-07-06 14:42:05,995] [EUTESTER] [DEBUG]: Created bucket: mybucket
tester.upload_object(bucket_name=bucketname,key_name=keyname,path_to_file=localfile)
[2012-07-06 14:44:00,730] [EUTESTER] [DEBUG]: Uploaded key: mykey to bucket:mybucket
<Key: mybucket,mykey>
obj = tester.get_objects_by_prefix(bucket_name=bucketname,prefix=keyname)
obj
<Key: mybucket,mykey>
tester.delete_object(obj)
[2012-07-06 14:45:10,222] [EUTESTER] [CRITICAL]: Walrus bucket still exists after delete #### what's mean of the warn?
tester.delete_bucket(bucket) #### it is suspent here and cannot return

ec2.get_all_images gets images regardless of availability, causing test failures

Repro:

  1. Install testing branch of latest eutester.
  2. Pull down creds for greg@eucalyptus on the 3.3 internal test cloud.
  3. (optional) Run cloud_admin/create_user.py to get some new users.
  4. Run cloud_admin/create_resources.py.

Full test report at the following gist: https://gist.github.com/gregdek/5499920

So it seems like what's happening is that a tester is being loaded that does not have access to all the EMIs. create_resources.py calls:

reservation=resource_tester.run_instance(keypair=keypair.name,group=group.name,zone=zone)

...so ec2ops.run_instance basically says "run the EMI given, and if that doesn't work, just run the first EMI you find." Trouble is, this in turn calls the ec2.get_all_images call to get a list of potential images -- and sometimes picks an EMI that the current credentials don't have access to.

There's one public image on this cloud that every account should have access to: emi-66AC3CEB. But run_instance, after pulling all images, instead chooses emi-65CC32E0 -- which is not available to the various users.

I'm guessing this is only showing up now because the new 3.3 clouds have lots of private images for the various services; previously, maybe all images were public by default?

Not sure why boto's get_all_images would even allow this.

Possible remediations:

  1. Put the responsibility on the individual test to pull a valid list of EMIs and deprecate the ability for ec2ops.run_instance to run a random image;
  2. Filter the ec2.get_all_images output so that only properly public EMIs are returned.

instancetest.py does not really accept --instance_user argument

Hi,

I specified following

instancetest.py --credpath /root/cloud33/ --emi emi-E2E53BC8 --config config --password password --instance_user ubuntu --tests BasicInstanceChecks --use_color

While it was running the test

[2013-04-24 02:55:45,902] [i-F1644202] [DEBUG]: SSH connection has hostname:euca-10-104-3-41.mycloud.jeevanullas.org user:root and keypath: /root/eutester/testcases/cloud_user/instances/keypair-1366797302.71.pem

It should have used the user passed, that is, ubuntu.

Cheers!
Deependra

eutester returns ping error for instances launched off cached images

Running eutester testcase instancetest.py => BasicInstanceChecks. Ping will fail and testcase will fail when running against instances launched from a cached image. Looks like ping isn't going through the entire 10 poll count.

$ ./testcases/cloud_user/instances/instancetest.py --credpath=/home/hspencer/admin-creds/ --emi emi-A1BF3721 --tests BasicInstanceChecks --instance-user root --user-data /home/hspencer/cloud-init.config
setuptestname:None
setup_debugmethod:
testcasename:None
log_level:None
logfile:None
logfile_level:debug
Starting setup_debugmethod, name:instancetest
After populating... setup_debugmethod: testcasename:Nonelog_level:debuglogfile:Nonelogfile_level:debug
[2013-07-04 09:07:47,574] [instancetest] [DEBUG]: instancetest: Eulogger init test message. Init complete
[2013-07-04 09:07:47,577] [instancetest] [DEBUG]: (setuptestcase:335): <pre>
[2013-07-04 09:07:47,580] [instancetest] [DEBUG]: (show_self:1300):
-------------------------------------------------------------------------
TESTCASE INFO:
----------
NAME:                     --->:  instancetest
TEST LIST:                --->:  []
CONFIG FILES:             --->:  []
-------------------------------------------------------------------------
[2013-07-04 09:07:47,583] [instancetest] [DEBUG]: (show_args:1321):
-------------------------------------------------------------------------
TEST ARGS:                       VALUE:
----------                      ------
debug_method              --->:  <bound method InstanceBasics.debug of <__main__.InstanceBasics testMethod=instancetest>>
logger                    --->:  <eutester.eulogger.Eulogger object at 0x2b6c5d0>
-------------------------------------------------------------------------
setup_debugmethod:
testcasename:None
log_level:None
logfile:None
logfile_level:None
Starting setup_debugmethod, name:instancetest
After populating... setup_debugmethod: testcasename:Nonelog_level:debuglogfile:Nonelogfile_level:debug
[2013-07-04 09:07:47,585] [instancetest] [DEBUG]: instancetest: Eulogger init test message. Init complete
[2013-07-04 09:07:47,588] [instancetest] [DEBUG]: (show_self:1300):
-------------------------------------------------------------------------
TESTCASE INFO:
----------
NAME:                     --->:  instancetest
TEST LIST:                --->:  []
CONFIG FILES:             --->:  []
-------------------------------------------------------------------------
[2013-07-04 09:07:47,591] [instancetest] [DEBUG]: (show_args:1321):
-------------------------------------------------------------------------
TEST ARGS:                       VALUE:
----------                      ------
args                      --->:  Namespace(config=None, config_file=None, configblocks=[], configfile=None, configsections=['MEMO', 'globals', 'instancetest'], cred_path='/home/hspencer/admin-creds/', credpath='/home/hspencer/admin-creds/', emi='emi-A1BF3721', html_anchors=False, ignoreblocks=[], instance_passsword=None, instance_user='root', keypair=None, log_level='debug', logfile=None, logfile_level='debug', password=None, region=None, tests=['BasicInstanceChecks'], use_color=False, user_data='/home/hspencer/cloud-init.config', vmtype='c1.medium', zone=None)
config                    --->:  None
config_file               --->:  None
configblocks              --->:  []
configfile                --->:  None
configsections            --->:  ['MEMO', 'globals', 'instancetest']
cred_path                 --->:  /home/hspencer/admin-creds/
credpath                  --->:  /home/hspencer/admin-creds/
debug_method              --->:  <bound method InstanceBasics.debug of <__main__.InstanceBasics testMethod=instancetest>>
emi                       --->:  emi-A1BF3721
html_anchors              --->:  False
ignoreblocks              --->:  []
instance_passsword        --->:  None
instance_user             --->:  root
keypair                   --->:  None
log_level                 --->:  debug
logfile                   --->:  None
logfile_level             --->:  debug
logger                    --->:  <eutester.eulogger.Eulogger object at 0x2b6cdd0>
password                  --->:  None
region                    --->:  None
tests                     --->:  ['BasicInstanceChecks']
use_color                 --->:  False
user_data                 --->:  /home/hspencer/cloud-init.config
vmtype                    --->:  c1.medium
zone                      --->:  None
-------------------------------------------------------------------------
[2013-07-04 09:07:47,592] [EUCAOPS] [DEBUG]: EUCAOPS: Eulogger init test message. Init complete
[2013-07-04 09:07:47,592] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:124)Starting method: setup_ec2_connection(self, endpoint=10.104.1.216, aws_access_key_id=L4836KVYWMJCXT4T6Q6B9, aws_secret_access_key=XCJ6sZVFVfFMR4DNVIUL7N7e4cgk8ebvEW0ej5dZ, is_secure=False, host=None, region=None, path=/services/Eucalyptus, port=8773, APIVersion=2011-01-01, boto_debug=0)
[2013-07-04 09:07:47,592] [EUCAOPS] [DEBUG]: Attempting to create ec2 connection to 10.104.1.2168773/services/Eucalyptus
[2013-07-04 09:07:47,593] [EUCAOPS] [DEBUG]: Attempting to create IAM connection to 10.104.1.216:8773/services/Euare
[2013-07-04 09:07:47,594] [EUCAOPS] [DEBUG]: Attempting to create STS connection to 10.104.1.2168773/services/Eucalyptus
[2013-07-04 09:07:47,595] [EUCAOPS] [DEBUG]:
--->(cwops.py:122)Starting method: setup_cw_connection(self, endpoint=10.104.1.216, aws_access_key_id=L4836KVYWMJCXT4T6Q6B9, aws_secret_access_key=XCJ6sZVFVfFMR4DNVIUL7N7e4cgk8ebvEW0ej5dZ, is_secure=False, host=None, region=None, path=/services/CloudWatch, port=8773, boto_debug=0)
[2013-07-04 09:07:47,595] [EUCAOPS] [DEBUG]: Attempting to create cloud watch connection to 10.104.1.2168773/services/CloudWatch
[2013-07-04 09:07:47,595] [EUCAOPS] [DEBUG]: Attempting to create S3 connection to 10.104.1.216:8773/services/Walrus
[2013-07-04 09:07:47,595] [EUCAOPS] [DEBUG]: Unable to create AS connection because of: Unable to find AWS_AUTO_SCALING_URL id in eucarc
[2013-07-04 09:07:47,596] [EUCAOPS] [DEBUG]:
--->(elbops.py:88)Starting method: setup_elb_connection(self, endpoint=10.104.1.216, aws_access_key_id=L4836KVYWMJCXT4T6Q6B9, aws_secret_access_key=XCJ6sZVFVfFMR4DNVIUL7N7e4cgk8ebvEW0ej5dZ, is_secure=False, host=None, region=None, path=/services/LoadBalancing, port=8773, boto_debug=0)
[2013-07-04 09:07:47,596] [EUCAOPS] [DEBUG]: Attempting to create cloud watch connection to 10.104.1.2168773/services/LoadBalancing
[2013-07-04 09:07:47,596] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:358)Starting method: add_group(self, group_name=group-1372954067.6, description=None, fail_if_exists=False)
[2013-07-04 09:07:47,596] [EUCAOPS] [DEBUG]: Looking up group group-1372954067.6
[2013-07-04 09:07:47,648] [EUCAOPS] [DEBUG]: Creating Security Group: group-1372954067.6
[2013-07-04 09:07:47,770] [EUCAOPS] [DEBUG]: Found matching security group for name:group-1372954067.6 and id:None
[2013-07-04 09:07:47,770] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:418)Starting method: authorize_group_by_name(self, group_name=group-1372954067.6, port=22, protocol=tcp, cidr_ip=0.0.0.0/0)
[2013-07-04 09:07:47,771] [EUCAOPS] [DEBUG]: Attempting authorization of group-1372954067.6 on port 22 tcp
[2013-07-04 09:07:47,826] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:418)Starting method: authorize_group_by_name(self, group_name=group-1372954067.6, port=-1, protocol=icmp, cidr_ip=0.0.0.0/0)
[2013-07-04 09:07:47,826] [EUCAOPS] [DEBUG]: Attempting authorization of group-1372954067.6 on port -1 icmp
[2013-07-04 09:07:47,876] [EUCAOPS] [DEBUG]: Looking up keypair keypair-1372954067.88
[2013-07-04 09:07:47,910] [EUCAOPS] [DEBUG]: Creating keypair: keypair-1372954067.88
Creating testunit:BasicInstanceChecks, args:
[2013-07-04 09:07:48,239] [instancetest] [DEBUG]: (populate_testunit_with_args:1339): Attempting to populate testunit:BasicInstanceChecks, with testcase.args...
[2013-07-04 09:07:48,244] [instancetest] [DEBUG]: (populate_testunit_with_args:1347): Testunit keyword args:{}
[2013-07-04 09:07:48,247] [instancetest] [DEBUG]: (populate_testunit_with_args:1352): Got method args:('self', 'zone')
[2013-07-04 09:07:48,250] [instancetest] [DEBUG]: (populate_testunit_with_args:1360): test unit total args:{}
[2013-07-04 09:07:48,253] [instancetest] [DEBUG]: (populate_testunit_with_args:1366): Found matching arg for:zone
[2013-07-04 09:07:48,256] [instancetest] [DEBUG]: (print_test_unit_startmsg:856):
-------------------------------------------------------------------------
STARTING TESTUNIT: BasicInstanceChecks
METHOD:BasicInstanceChecks, TEST DESCRIPTION:

        This Case Was Developed To Run Through A Series Of Basic Instance Tests.
             The Tests Are As Follows:
                   - Execute Run_Instances Command
                   - Make Sure That Public Dns Name And Private Ip Aren'T The Same
                       (This Is For Managed/Managed-Novlan Networking Modes)
                   - Test To See If Instance Is Ping-Able
                   - Test To Make Sure That Instance Is Accessible Via Ssh
                       (Ssh Into Instance And Run Basic Ls Command)
             If Any Of These Tests Fail, The Test Case Will Error Out, Logging The Results.

End on Failure:False
Passing ARGS:
---------------------
zone : None
---------------------
Running list method: "BasicInstanceChecks(zone:None)"
-------------------------------------------------------------------------
KWARG:zone = None
[2013-07-04 09:07:48,256] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:2239)Starting method: run_instance(self, image=emi-A1BF3721, keypair=keypair-1372954067.88, group=group-1372954067.6, type=None, zone=LayinDaSmackDown, min=1, max=1, user_data=/home/hspencer/cloud-init.config, private_addressing=False, username=root, password=None, is_reachable=True, monitoring_enabled=False, timeout=480)
[2013-07-04 09:07:48,257] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:1881)Starting method: get_images(self, emi=emi-A1BF3721, root_device_type=None, root_device_name=None, location=None, state=available, arch=None, owner_id=None, filters=None, not_location=None, max_count=1)
[2013-07-04 09:07:48,438] [EUCAOPS] [DEBUG]: Got 14 images matching prefix: emi-A1BF3721, now filtering...
[2013-07-04 09:07:48,438] [EUCAOPS] [DEBUG]: Returning image:emi-A1BF3721
[2013-07-04 09:07:48,439] [EUCAOPS] [DEBUG]: Attempting to run instance-store image Image:emi-A1BF3721 in group group-1372954067.6
[2013-07-04 09:07:48,703] [EUCAOPS] [DEBUG]: Beginning poll loop for the 1 instance found in [Instance:i-96804019]
[2013-07-04 09:07:48,704] [EUCAOPS] [DEBUG]: Beginning poll loop for instance Instance:i-96804019 to go to running
[2013-07-04 09:07:48,778] [EUCAOPS] [DEBUG]: Instance(i-96804019) State(pending), elapsed:0/480
[2013-07-04 09:07:58,892] [EUCAOPS] [DEBUG]: Instance(i-96804019) State(pending), elapsed:10/480
[2013-07-04 09:08:08,992] [EUCAOPS] [DEBUG]: Instance(i-96804019) State(pending), elapsed:20/480
[2013-07-04 09:08:19,099] [EUCAOPS] [DEBUG]: Instance(i-96804019) State(running) time elapsed (30)
[2013-07-04 09:08:19,099] [EUCAOPS] [DEBUG]: Instance:i-96804019 is now in running
[2013-07-04 09:08:19,099] [EUCAOPS] [DEBUG]: Instance i-96804019 now in running state  in zone: LayinDaSmackDown
[2013-07-04 09:08:19,100] [EUCAOPS] [DEBUG]: Instance:i-96804019 got Public IP: 10.104.7.12 Private IP: 172.17.112.234 Public DNS Name: 10.104.7.12 Private DNS Name: 172.17.112.234
[2013-07-04 09:08:19,100] [EUCAOPS] [DEBUG]:
--->(ec2ops.py:2865)Starting method: wait_for_valid_ip(self, instances=Instance:i-96804019, private_addressing=False, poll_interval=10, timeout=60)
[2013-07-04 09:08:19,193] [EUCAOPS] [DEBUG]: i-96804019: FOUND public ip. Current:10.104.7.12, elapsed:0/60
[2013-07-04 09:08:19,193] [EUCAOPS] [DEBUG]: Check_system_for_dup_ip starting...
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking reservation: r-D61E3E4D
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking instance i-8805459E          , state:running              pubip:10.104.7.10          privip:172.17.116.138
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking reservation: r-557F3DD1
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking instance i-96804019          , state:running              pubip:10.104.7.12          privip:172.17.112.234
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking reservation: r-36533C3E
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking instance i-997E4489          , state:terminated           pubip:172.17.116.137       privip:172.17.116.137
[2013-07-04 09:08:19,362] [EUCAOPS] [DEBUG]: Checking reservation: r-43114410
[2013-07-04 09:08:19,363] [EUCAOPS] [DEBUG]: Checking instance i-83D64095          , state:running              pubip:10.104.7.11          privip:172.17.116.148
[2013-07-04 09:08:19,363] [EUCAOPS] [DEBUG]: Done with check_system_for_dup_ip
[2013-07-04 09:08:19,363] [EUCAOPS] [DEBUG]: Wait_for_valid_ip done
[2013-07-04 09:08:19,363] [EUCAOPS] [DEBUG]: Attempting to ping 10.104.7.12

TESTUNIT FAILED: BasicInstanceChecksTraceback (most recent call last):
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/eutestcase.py", line 277, in run
    ret = self.method(*self.args, **self.kwargs)
  File "./testcases/cloud_user/instances/instancetest.py", line 76, in BasicInstanceChecks
    keypair=self.keypair.name, group=self.group.name, zone=zone, timeout=self.instance_timeout)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 341, in methdecor
    return func(*func_args, **func_kwargs)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eucaops/ec2ops.py", line 2348, in run_instance
    self.ping(instance.ip_address, 20)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 170, in ping
    if self.found("ping -c 1 " + address, "1.*1.*received"):
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 151, in found
    result = self.local(command)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 145, in local
    raise error
CalledProcessError: Command 'ping -c 1 10.104.7.12' returned non-zero exit status 1

Define licensing

Need to figure out what license we're offering Eutester under. I presume GPLv3 to be compatible with the Eucalyptus product itself, but we need to be explicit. Also need to make it clear that it's copyright Eucalyptus.

eucaops does not propogate the correct credentials to the disabled CLC if it is first in the cloud config file

Here is the first few lines of the log.

[2012-09-12 21:56:05,189] [EUTESTER] [DEBUG]: Reading config file: cloud.conf
[2012-09-12 21:56:05,190] [10.104.1.163:['clc', 'ws']] [DEBUG]: SSH connection has hostname:10.104.1.163 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:05,592] [10.104.1.164:['clc', 'ws']] [DEBUG]: SSH connection has hostname:10.104.1.164 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:05,732] [10.104.1.165:['sc00', 'cc00']] [DEBUG]: SSH connection has hostname:10.104.1.165 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:05,866] [10.104.1.166:['cc00']] [DEBUG]: SSH connection has hostname:10.104.1.166 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:06,003] [10.104.1.167:['nc00']] [DEBUG]: SSH connection has hostname:10.104.1.167 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:06,156] [10.104.1.168:['nc00']] [DEBUG]: SSH connection has hostname:10.104.1.168 user:root password:gnuwheelsqadagespiro
[2012-09-12 21:56:06,294] [EUTESTER] [DEBUG]: Attempting to get credentials and setup sftp
[2012-09-12 21:56:06,310] [EUTESTER] [DEBUG]: Starting the process of getting credentials
[2012-09-12 21:56:06,311] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# mkdir eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:06,369] [10.104.1.163:['clc', 'ws']] [DEBUG]: mkdir: cannot create directory `eucarc-10.104.1.163-eucalyptus-admin': File exists

[2012-09-12 21:56:06,369] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:06,369] [10.104.1.163:['clc', 'ws']] [DEBUG]: mkdir: cannot create directory `eucarc-10.104.1.163-eucalyptus-admin': File exists
[2012-09-12 21:56:06,369] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# rm -f eucarc-10.104.1.163-eucalyptus-admin/creds.zip
[2012-09-12 21:56:06,438] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:06,438] [10.104.1.163:['clc', 'ws']] [DEBUG]:
[2012-09-12 21:56:06,439] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# //usr/sbin/euca_conf --get-credentials eucarc-10.104.1.163-eucalyptus-admin/creds.zip --cred-user admin --cred-account eucalyptus
[2012-09-12 21:56:07,860] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:07,860] [10.104.1.163:['clc', 'ws']] [DEBUG]:
[2012-09-12 21:56:07,860] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# unzip -o eucarc-10.104.1.163-eucalyptus-admin/creds.zip -d eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:07,945] [10.104.1.163:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zip
To setup the environment run: source /path/to/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc
inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem

[2012-09-12 21:56:07,946] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:07,946] [10.104.1.163:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zipTo setup the environment run: source /path/to/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem
[2012-09-12 21:56:07,946] [EUTESTER] [DEBUG]: Downloading credentials from 10.104.1.163
Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zip
To setup the environment run: source /path/to/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc
inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem
[2012-09-12 21:56:08,035] [EUTESTER] [DEBUG]: Sending credentials to 10.104.1.163
[2012-09-12 21:56:08,037] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# mkdir eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:08,092] [10.104.1.163:['clc', 'ws']] [DEBUG]: mkdir: cannot create directory `eucarc-10.104.1.163-eucalyptus-admin': File exists

[2012-09-12 21:56:08,092] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,092] [10.104.1.163:['clc', 'ws']] [DEBUG]: mkdir: cannot create directory `eucarc-10.104.1.163-eucalyptus-admin': File exists
[2012-09-12 21:56:08,097] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# unzip -o eucarc-10.104.1.163-eucalyptus-admin/creds.zip -d eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:08,152] [10.104.1.163:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zip
To setup the environment run: source /path/to/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc
inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem

[2012-09-12 21:56:08,153] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,153] [10.104.1.163:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zipTo setup the environment run: source /path/to/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem
[2012-09-12 21:56:08,153] [10.104.1.163:['clc', 'ws']] [DEBUG]: [[email protected]]# sed -i 's/10.104.1.163/10.104.1.163/g' eucarc-10.104.1.163-eucalyptus-admin/eucarc
[2012-09-12 21:56:08,210] [10.104.1.163:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,210] [10.104.1.163:['clc', 'ws']] [DEBUG]:
[2012-09-12 21:56:08,210] [EUTESTER] [DEBUG]: Sending credentials to 10.104.1.164
[2012-09-12 21:56:08,211] [10.104.1.164:['clc', 'ws']] [DEBUG]: [[email protected]]# mkdir eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:08,267] [10.104.1.164:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,267] [10.104.1.164:['clc', 'ws']] [DEBUG]:
[2012-09-12 21:56:08,272] [10.104.1.164:['clc', 'ws']] [DEBUG]: [[email protected]]# unzip -o eucarc-10.104.1.163-eucalyptus-admin/creds.zip -d eucarc-10.104.1.163-eucalyptus-admin
[2012-09-12 21:56:08,327] [10.104.1.164:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zip
To setup the environment run: source /path/to/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc
inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc
inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem
inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem

[2012-09-12 21:56:08,327] [10.104.1.164:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,327] [10.104.1.164:['clc', 'ws']] [DEBUG]: Archive: eucarc-10.104.1.163-eucalyptus-admin/creds.zipTo setup the environment run: source /path/to/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/eucarc inflating: eucarc-10.104.1.163-eucalyptus-admin/iamrc inflating: eucarc-10.104.1.163-eucalyptus-admin/cloud-cert.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/jssecacerts inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-pk.pem inflating: eucarc-10.104.1.163-eucalyptus-admin/euca2-admin-68109805-cert.pem
[2012-09-12 21:56:08,327] [10.104.1.164:['clc', 'ws']] [DEBUG]: [[email protected]]# sed -i 's/10.104.1.163/10.104.1.164/g' eucarc-10.104.1.163-eucalyptus-admin/eucarc
[2012-09-12 21:56:08,384] [10.104.1.164:['clc', 'ws']] [DEBUG]: done with exec
[2012-09-12 21:56:08,384] [10.104.1.164:['clc', 'ws']] [DEBUG]:
[2012-09-12 21:56:08,385] [EUTESTER] [DEBUG]: Successfully downloaded and synced credentials

The function associate_address() in eu2ops.py not work

Hi:
I found these code in associate_address() in eu2ops.py not work normal, in my test case I found that the instance.public_dns_name is DNS name like "euca-10-68-122-141.eucalyptus.vaas126.cd.nsn-rdnet.net" while the address.public_ip is a IP address like "10.68.122.14". so the while loop will always failed.
The follow are the source code my test log.

source code :
while instance.public_dns_name not in address.public_ip:
if poll_count == 0:
raise Exception('Address ' + str(address) + ' never associated with instance')
instance.update()
self.debug('Instance {0} has IP {1} attached instead of {2}'.format(instance.id, instance.public_dns_name, address.public_ip) )
poll_count -= 1
self.sleep(5)
self.debug("Associated IP successfully")

my log:
The code "self.debug('Instance {0} has IP {1} attached instead of {2}'.format(instance.id, instance.public_dns_name, address.public_ip) )" will out put the instance.public_dns_name and address.public_ip, so we can check what are the real value of them in log output.
In my log the real output is:
14:41:58.750 DEBUG Instance i-34B5465F has IP euca-10-68-122-141.eucalyptus.vaas126.cd.nsn-rdnet.net attached instead of 10.68.122.141

so can you help me solve the problems! thanks!

Create a standard set of debug tools available when tests fail

When a eutester test fails, we need to provide a standard set of debugging tools back to the user. This debug info should include as much as is necessary to find the possible cause of the behavior. Currently, the Eutester class provides a very basic framework for this, but we'd like to expand it.

Some feature ideas for the debug tools set (not the final list):

  • grep (already implemented)
  • euca-describe-* (get a listing of all resources in the system)
  • get_all_everything in boto
  • get logs
  • get topology
  • get distro
  • get build information
  • get environment (cpu, mem, etc)
  • euca-describe-everything
  • get test script used
  • self replicating code
  • yum
  • tar
  • graceful error messages (for instance, some tools may only work with root privs -- so we need a nice "you don't have the privs to do that" message for non-root users, instead of spewing gibberish.)

So step 1 is presumably to figure out what the final list is. :) We may need to break this into separate tickets if implementing some of these turn out to be less trivial than others.

sshconnection.py is trying to use '~/.ssh/authorized_keys' as a private key

sshconnection.py contains this bit of fallback code in two different places:

file = self.pub_key_file or os.path.expanduser('~/.ssh/authorized_keys')
if not os.path.isfile(file):
    raise Exception('No way to authenticate, need key, password, or file of keys, etc..')
self.debug("Using local keys, no keypath/password provided, trying:" + str(file), verbose=verbose)
ssh._auth(username, password,None,[file], True, True)

This means that it's doing a couple of things wrong:

  1. it's trying to use authorized_keys as a private key, which will always fail (and in fact, there's some code in the _auth function to handle the exception it's causing).
  2. if a system does not have an authorized keys file, it's raising an exception without attempting useful authentication methods like ssh-agent's stored keys (my preferred method).

You should simply pass an empty key list if a key isn't specified, and _auth will do the right thing, raising an SSHException if no authentication methods are available.

Also, "pub_key_file" as a variable name for a private key file is very confusing. Can we change that to priv_key_file?

eutester should allow the opening of multiple ssh connections

Id like eutester to allow the creation of multiple SSHClient sessions that are kept in a dictionary. This way we dont limit the number of devices we can use in a test without creating a new connection. The syntax could be something like:

eutester.add_ssh(name="new_session", host="foo", password="bar")

then you can run commands against this by saying:

eutester.sys(cmd="ls", session="new_session")

When no session name is provided the default will be used which is the connection passed in when the eutester object was constructed.

Need way for machine access when keys have been previously synced

Currently we rely on the password to be passed in to Eucaops or Eutester in order to determine whether we will be needing to connect to machines (ie create machine objects). There should be a way to say that you want connections to machines without passing in a password.

Goals, purpose, and guidelines for contribution

Submitted by Greg DeKoenigsberg 2 months ago, migrated to Github.

We need a bit more copy in the README to explain what Eucatester is all about. Specifically:

Why it's awesome;
Where we're going with it;
How to contribute some test cases.

create_ssh does not pass correct key file name argument

Hey guys,

I was just trying to check out the code looks like in create_ssh() the following patch would help:

--- /root/eutester/eutester/init.py 2012-01-21 04:45:01.000000000 -0800
+++ /root/eutester/eutester/init.py.new 2012-01-22 08:47:21.000000000 -0800
@@ -129,7 +129,7 @@
if keypath == None:
client.connect(self.get_component_ip("clc"), username="root", password=password)
else:

  •                client.connect(self.get_component_ip("clc"),  username="root", keyfile_name=keypath)
    
  •                client.connect(self.get_component_ip("clc"),  username="root", key_filename=keypath)
             self.ssh = client
             self.sftp = self.ssh.open_sftp()  
             self.credpath = self.get_credentials(account,user)  
    

Was reading through the paramiko api docs and seems like key_filename is the correct argument name. Let me know if it makes sense.

By the way great stuff !

Cheers,
Deependra

get_debug.py => NC's can't be reached in Managed/Managed-NOVLAN mode when CC is router for physical machines

Hey guys,

Here is my setup:

$ cat ../ha-centos-cloud.conf
NETWORK MANAGED
192.168.39.101 CENTOS 6.3 64 REPO [CLC WS]
192.168.39.102 CENTOS 6.3 64 REPO [CLC WS]
192.168.39.103 CENTOS 6.3 64 REPO [CC00 SC00]
192.168.39.104 CENTOS 6.3 64 REPO [CC00]
10.101.39.41 CENTOS 6.3 64 REPO [NC00]
10.101.39.42 CENTOS 6.3 64 REPO [NC00]
10.101.39.43 CENTOS 6.3 64 REPO [NC00]
10.101.39.44 CENTOS 6.3 64 REPO [NC00]
10.101.39.45 CENTOS 6.3 64 REPO [NC00]

When I run get_debug.py, I get the following error:

$ ./testcases/cloud_admin/get_debug.py --config ../ha-centos-cloud.conf --password 123456
[2012-10-01 10:03:44,669] [EUTESTER] [DEBUG]: Reading config file: ../ha-centos-cloud.conf
[2012-10-01 10:03:44,671] [192.168.39.101:['clc', 'ws']] [DEBUG]: SSH connection has hostname:192.168.39.101 user:root password:123456
[2012-10-01 10:03:46,302] [192.168.39.102:['clc', 'ws']] [DEBUG]: SSH connection has hostname:192.168.39.102 user:root password:123456
[2012-10-01 10:03:48,195] [192.168.39.103:['cc00', 'sc00']] [DEBUG]: SSH connection has hostname:192.168.39.103 user:root password:123456
[2012-10-01 10:03:50,690] [192.168.39.104:['cc00']] [DEBUG]: SSH connection has hostname:192.168.39.104 user:root password:123456
[2012-10-01 10:03:52,377] [10.101.39.41:['nc00']] [DEBUG]: SSH connection has hostname:10.101.39.41 user:root password:123456
Traceback (most recent call last):
File "./testcases/cloud_admin/get_debug.py", line 97, in
debugsuite = GatherDebug(config_file=args.config, password = args.password)
File "./testcases/cloud_admin/get_debug.py", line 33, in init
self.tester = Eucaops( config_file=config_file, password=password)
File "/Library/Python/2.7/site-packages/eutester-0.0.3-py2.7.egg/eucaops/init.py", line 77, in init
self.config = self.read_config(config_file)
File "/Library/Python/2.7/site-packages/eutester-0.0.3-py2.7.egg/eucaops/init.py", line 278, in read_config
username = username
File "/Library/Python/2.7/site-packages/eutester-0.0.3-py2.7.egg/eutester/machine.py", line 128, in init
verbose=True)
File "/Library/Python/2.7/site-packages/eutester-0.0.3-py2.7.egg/eutester/sshconnection.py", line 127, in init
self.connection = self.get_ssh_connection(self.host, username=self.username, password=self.password, keypath=self.keypath, timeout=self.timeout)
File "/Library/Python/2.7/site-packages/eutester-0.0.3-py2.7.egg/eutester/sshconnection.py", line 332, in get_ssh_connection
ssh.connect(hostname, username=username, password=password, timeout= timeout)
File "/Library/Python/2.7/site-packages/paramiko-1.7.7.2-py2.7.egg/paramiko/client.py", line 296, in connect
sock.connect(addr)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 60] Operation timed out

Seems like access to the NCs is expected to be accessible from the client where get_debug.py is running. A lot of Managed/Managed-NOVLAN setups use the CC as router for the NCs that I have seen.

I have a couple of suggestions here:

  • Looking at the code, we would need to pull in knowledge of the CC, and then hop from the CC to the NCs.
  • A good idea to design get_debug.py to do the following:
    1. To get to the CC, Walrus, SC, go through the CLC
    2. To get to the NCs, go through the CC

My thought here is that in any networking mode, you will always be able to get debugging information without having to worry about tweaking the code for a specific networking setup.

What you guys think about those suggestions? Any feedback would be greatly appreciated.

Cheers,

Harold Spencer, Jr.

Feature Request: Allow for NC to be an optional value in config_file

Description

When you create a config_file with only the CLC, CC, SC and WS, an error occurs when tests are executed because tests are ran from the CLC. This does not keep the tests from being ran, but this give the impression that there is an error.

euca-prod-1-173:unstable hspencer$ cat /Users/hspencer/Desktop/eutester-dev/creds/euca3/config_file192.168.5.100 UBUNTU 10.04 64 REPO [CLC CC00 SC00 WS]

Exception in thread Thread-10:
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 551, in *bootstrap_inner
self.run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 504, in run
self.__target(_self.__args, _self.__kwargs)
File "/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/__init
.py", line 529, in poll_euca_logs
nc_ssh = self.create_ssh("nc00", password=self.password)
File "/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/init.py", line 499, in create_ssh
hostname = self.swap_component_hostname(hostname)
File "/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/init.py", line 393, in swap_component_hostname
component_hostname = self.get_component_ip(hostname)
File "/Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eutester/init.py", line 368, in get_component_ip
raise Exception("Could not find component " + component + " in list of machines")
Exception: Could not find component nc00 in list of machines

The reason why one would not put the NC in the configuration file, is because in Managed or Managed-NOVLAN mode, the CLC may not have direct access to the NC and/or logs aren't needed.

If the NC is in the config_file, a suggestion would be to have requests for log gathering be done from the CC machine. The CC will always have access to the NC(s).

README.md

Hi,

Looks like the README fails to mention what the 5th column (REPO) in cloud.conf means ?

Cheers,
Deependra

run_instance should still provide a euinstance regardless of reachability

When run_instance is called in Eucaops it chooses to convert the boto instances to euinstances only if the instance is reachable. Instead it should always return the euinstance and just not pass keys to the Euinstance constructor which will ensure that the euinstance will not try to create an SSH connection.

connect to walrus should use s3_url

Hi Vic,

Does this makes sense?

--- init.py 2012-01-25 17:12:47.000000000 +0530
+++ init.py.new 2012-01-31 01:34:59.000000000 +0530
@@ -158,7 +158,7 @@
self.walrus = boto.connect_s3(aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key,
is_secure=False,

  •                                      host=self.get_clc_ip(),
    
  •                                      host=self.get_walrus_ip(),
                                       port=8773,
                                       path="/services/Walrus",
                                       debug=self.boto_debug)
    

Cheers,
Deependra

eutester using subprocess.check_output, which isn't available until python 2.7

I ran the instancetest.py testcase with BasicInstanceChecks and got the following error:

[2013-06-27 18:28:31,986] [EUCAOPS] [DEBUG]: Done with check_system_for_dup_ip
[2013-06-27 18:28:31,987] [EUCAOPS] [DEBUG]: Wait_for_valid_ip done
[2013-06-27 18:28:31,987] [EUCAOPS] [DEBUG]: Attempting to ping 10.104.7.10

TESTUNIT FAILED: BasicInstanceChecksTraceback (most recent call last):
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/eutestcase.py", line 277, in run
    ret = self.method(*self.args, **self.kwargs)
  File "./testcases/cloud_user/instances/instancetest.py", line 76, in BasicInstanceChecks
    keypair=self.keypair.name, group=self.group.name, zone=zone, timeout=self.instance_timeout)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 329, in methdecor
    return func(*func_args, **func_kwargs)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eucaops/ec2ops.py", line 2348, in run_instance
    self.ping(instance.ip_address, 20)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 158, in ping
    if self.found("ping -c 1 " + address, "1.*1.*received"):
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 139, in found
    result = self.local(command)
  File "/home/hspencer/eutester-dev/lib/python2.6/site-packages/eutester-0.0.7-py2.6.egg/eutester/__init__.py", line 133, in local
    std_out = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True)
AttributeError: 'module' object has no attribute 'check_output'

[2013-06-27 18:28:31,993] [instancetest] [DEBUG]: (endsuccess:696):
-------------------------------------------------------------------------
- UNIT ENDED - BasicInstanceChecks
-------------------------------------------------------------------------

Looking at the documentation regarding subprocess, check_output isn't available until python 2.7. Additionally, there is a security warning associated with using this with shell=True.

System Information

$ yum repolist
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
repo id                                                           repo name                                                                                             status
base                                                              CentOS-6 - Base                                                                                       6,381
elrepo                                                            ELRepo.org Community Enterprise Linux Repository - el6                                                  269
epel                                                              Extra Packages for Enterprise Linux 6 - x86_64                                                        8,954
euca2ools-release                                                 euca2ools-release                                                                                         6
eucalyptus-release                                                eucalyptus-release                                                                                       35
extras                                                            CentOS-6 - Extras                                                                                        12
updates                                                           CentOS-6 - Updates
$ python -V
Python 2.6.6
$ cat /etc/*release
CentOS release 6.4 (Final)
CentOS release 6.4 (Final)
CentOS release 6.4 (Final)

Using eutester on a CloudStack ec2 interface

Hi, I would like to see how I could use eutester to test a CloudStack EC2 interface.
CloudStack does not expose S3 or IAM.
We can use boto with something like:

region = boto.ec2.regioninfo.RegionInfo(name="ROOT", endpoint="localhost")
conn = boto.connect_ec2(aws_access_key_id=accesskey, aws_secret_access_key=secretkey, is_secure=False, region=region, port=7080, path="/awsapi", api_version="2012-08-15")

How can I instantiate Eucaops() without S3 and IAM connections ?

Looks like I might have to modify Eucaops() a bit, which I hoped not to have to do.

Any help appreciated

thanks

-sebastien

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.