if submiting a few jobs in job list, the final job may get repeated again and again, which could cause either the storage pool to fill, or the drives to overheat.

The tool must be able to reclaim disk space in under 4 hours between test runs to prevent storage devices from being filled. (Default background garbage collection may take 2hours -10 days or longer… duration varies by system load)

action required:
. Add explanation why disk space is not release even object is deleted.

In the Appendix – the examples do not demonstrate breaking up the tasks to show how you can save the files from being automatically cleaned up at the end of the test run… this was mentioned in previous feedback.

Need improve document & samples to resolve a few confusions:

1. why disk space is not release even object is deleted?
   --> need some explanations in document, and it should cover swift and amplistor.
2. the diagrams in user guide are specific to swift, need generic diagrams for both swift and amplistor.
   --> need some generic diagrams and terms to cover different storage system in document.
3. how can save the files from being automatically cleaned up at the end of the test run? need samples to demostrate them.
   --> need some samples to show how to reuse data across test runs.
4. how to install COSBench and have it start automatically after a reboot?
   --> need FAQ to show how to auto start COSBench.
5. why there are errors at war deployment?
   --> It's an known issue, but doesn't affect real functionality. Actually, it is caused by osgi bundle loading mechanism.
6. in ipv6 env, socket can not bind, how to resolve it?
   --> it's a tomcat related issue, one workaround is to set the ip address in tomcat server configuration file, need FAQ to show it.
7. why the parameter list shown on UI is a bit different from xml file?
   --> need explanation in document.

need users' input to help enrich sample workload list, this is a long-term work.

Check status of drivers if stuck/ crashed before commencement of test

provide a simplified dash board (or file) for test execution summary that shows current execution state of COSBench as well as Object Store performance as observed from traffic across all API interface(s); usability issue

need find more effective message to help people identify issue.

in main stage, if using range selector for delete operation, the work will fail.

: Need a way to put system to sleep between runs (to allow separation of data) eg., IOMeter “idle” - can be combined with earlier request covering job management

even though they are in the archive directory.

the test is failed at login with error "204 No Content"

S3 interface is a popular public cloud interface, we will deliver one S3 adapter in upcoming version 0.3.1.

the "objects=…" is used internally, but not from workload definition, which confuse user.

The RangeIntGenerator is supposed to split the range in to su-ranges for every thread, and return different ints to different threads. So for a range of 100, used by 10 threads, the first thread should get ints 1-10, the second thread should get 11-20, etc…

As there is only one offset pointer (cursor) implemented, this can not work. If threads one, two, three ask in this order, thread one will get "1", thread two will get "12", thread 3 will get "23", etc…

I think the Class can not be implemented properly with the given signature of next(), as one can not be sure that the amount of threads won't change. If the amount of threads given by "int all" changes, all the splitting of the range breaks apart.

Also synchronizing does next() not give best performance. It would be better to use AtomicInt for the pointer (cursor).

All of the diagrams are specific to Swift, need generic digrams, shows swift and amplidata (multipe controller nodes, terms).

if the certificate is self-signed, java doesn't trust it. but at internal evaluation case, self-sign certificate is often used, need support it.

It's an known issue, but doesn't affect real functionality. Actually, it is caused by osgi bundle loading mechanism.

There was no information seen in the User guide about the File Write operation which was added in the latest version of COSBench.

if defining parameter like " cprefix=10M_read_", the created container name is still "mycontainers_" like form.

need investigate.

so far GET/PUT/DELETE.

Some storage system like amplistor requires digest authentication for login.

There was no performance data seen on any of the six metrics when read operation was performed on Amplidata (version 3.1.4).

Cosbench versions tried were (0.3.0.0, 0.3.1.0 and 2.1.0-GA) with multiple drivers (4)
and different workloads (100%read, 90% read, 80% read)

The XML Test Generator needs to support configuring the Amplidata test case and populate the namespace/policy UID's.

The tool should supply real-time performance and progress data that can be displayed during a demonstration… Today you must configure small files for the test to complete during demo time windows. We want to demonstrate large objects.

Cosbench terminates during the test phase - especially when what seems to be putting the object store under stress. This should not happen, it should report errors during test run as failed attempts and should impact completion percentage, but not terminate the the test stage. My tests use timed runs not totalops.

expect to see more adaptors for public cloud will be added. The upcoming one is S3.

Action:
one alpha version is included, need more verification or enhancement.

the generated xml will be named as "workload.xml", user expects to have chance to name it by themselves.

Action:
code enhancement.

I need to route S3 requests to specific S3 regions (i.e. url) and not rely on AWS network maps to route the standard s3 URL. This control is need to ensure controlled environment for specific S3 AWS test configurations.

The tool must have a self-test mode that can identify failures of workers and controllers. If workload gets failed, need identify which part causes the failure.

maintain time synchronization of less than 100mSec with the tool’s NTP Service. This means stimulus to the Object Store can be coordinated across all Generators. It is desirable to sync the COSBench NTP service with the Object Store’s NTP resource if available

for job management:
. No reordering
. No cancel all jobs
. Resumit the last (choose from history)

it's a tomcat related issue, one workaround is to set the ip address in tomcat server configuration file.

action:
will document it.

in one workload with 80% read and 20% write mixed, even ALL writes are dropped, the controller GUI still marks it as valid.

A few areas need improve to help quicker trouble shooting:

1. If workload gets failed, need identify which part causes the failure.
2. while the target Amplistor doesn’t enable the port. There is no warning in log.
3. Check status of drivers if stuck/ crashed before commencement of test

Does the “Config Workloads” really work for more than one authentication type? I did not see it make any changes when I selected different types of authentication in the pull-down.

Hi there,

On my debian squeeze System, cosbench was not able to check if it started properly when calling start-all.sh because in cosbench-start.sh on line 82 netcat would not end.

I fixed it with a simple patch but I am not sure about its implications for other systems… So I am posting here...

my Patch:

diff --git a/0.3.0.0/cosbench-start.sh b/0.3.0.0/cosbench-start.sh
index 016a0ff..28a93ce 100644
--- a/0.3.0.0/cosbench-start.sh
+++ b/0.3.0.0/cosbench-start.sh
@@ -32,6 +32,8 @@
TOMCAT_CONFIG=conf/$SERVICE_NAME-tomcat-server.xml

TOOL="nc"
+TOOL_PARAMS="-q 1"
+

#-------------------------------

MAIN

@@ -79,7 +81,7 @@
attempts=60
while [ $ready -ne 1 ];
do

•            echo "ss -s ACTIVE cosbench" | $TOOL localhost $OSGI_CONSOLE_PORT | grep $module >> /dev/null
  

•            echo "ss -s ACTIVE cosbench" | $TOOL $TOOL_PARAMS localhost $OSGI_CONSOLE_PORT | grep $module >> /dev/null
           if [ $? -ne 0 ];
           then
  

There is no example of how to install COSbench and have it start automatically after a reboot.

. Should be .css file is not approrpiately loaded, need investigate.

while the target Amplistor doesn’t enable the port. There is no warning in log.

so far in one worker, the http request is synchnous, at high latency case, it will bring difficulty to saturate target system.

The tool must have an XML test file checker that report any syntax errors with the configuration files.

. XML validator

• syntax (first step)
• parameter checking (next step)

Enhancement supporting addition/removal of multiple main stages from config.html

CDMI is an emerging cloud storage interface, expect to evaluate it and add corresponding adapter.

. Add explanation why disk space is not release even object is deleted? For amplidata

If testing environment is internet facing, so authentication is necessary.

this is one swift specific issue, which is caused by swift consistency mechanism.

for S3 adapter, if the secret key contains an = character, the adaptor is truncating the value at that character.

it's necessary to define workload with multiple hot object sizes.