S3 testing with COSbench

COSbench with ECSToday I decided to play with COSBench. Will test S3 access to ECS.


The first task to get access to ECS was resolved extremely easy. I just registered on https://portal.ecstestdrive.com/account/register .Voila, I have access to my own ECS.

The next step is to generate credentials.

ECS Test drive credentials

Time to test access. First configure S3 browser for Windows. Storage Type is S3 Compatible Storage.

S3 Browser account

Cyberduck connection on my mac is configured similarly.

Cyberduck account

Let’s check the connection, create the first bucket and upload something. Just uploaded the folder with VMware icons. Check in S3 Browser:

Connection in S3 browser

Check in Cyberduck:

Connection in Cyberduck

Check with http://130790262785219969.public.ecstestdrive.com/mybucket/ in browser:

Test ECS in browser

So far so good ! Time to install COSBench. Download zip from https://github.com/intel-cloud/cosbench/releases, unpack it and run start-all.sh.

start COSbench

Type http://localhost:19088/controller/index.html in your browser and you can play with your tests.

Start COSbench in browser

Take s3-config-sample.xml file in COSBench config and modify

  • accesskey=130790262785219969@ecstestdrive.emc.com
  • secretkey=bJn0Lw13Q7NyfjiIX7NVU9hltHLHxxxj
  • endpoint=object.ecstestdrive.com”
  • Proxy parameters are not needed.


Time to submit the new workload. Select your .xml file as a workload config.

Submit new COSbench workload

After several minutes, your workload is completed.

COSbench workload is completed

You can check the specific stage details and performance results.

COSbench specific stage details and performance results

Please don’t criticize the performance. I’m in mycountryside house now. So I ran this test via wireless network available here. Public ecstestdrive.com resource itself is also not tuned for performance tests.

COSbench performance

Finally, I’d like to discuss briefly the rules specified in the default S3 config sample.

Init stageCOSbench “init” stage

Two buckets are created

2015-06-17 23:55:41,416 [INFO] [NoneStorage] - performing PUT at /s3testqwer1

2015-06-17 23:55:42,111 [INFO] [NoneStorage] - performing PUT at /s3testqwer2

Prepare stage

COSbench “prepare” stage

Ten objects 64K size are created in every bucket

2015-06-17 23:55:49,159 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects1

2015-06-17 23:55:54,839 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects2

2015-06-17 23:56:13,473 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects10

2015-06-17 23:56:18,197 [INFO] [NoneStorage] - performing PUT at /s3testqwer2/myobjects1

2015-06-17 23:56:26,487 [INFO] [NoneStorage] - performing PUT at /s3testqwer2/myobjects2

2015-06-17 23:56:48,569 [INFO] [NoneStorage] - performing PUT at /s3testqwer2/myobjects10

 Main stage

Eight streams simultaneously read existing and write new objects. Read / Write proportion is 80/20. Duration is 30 seconds. In that specific test, ECS system processed 76 requests.

COSbench Main stage

2015-06-17 23:56:56,945 [INFO] [NoneStorage] - performing PUT at /s3testqwer2/myobjects12

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer2/myobjects10

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects8

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects2

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects9

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects9

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer2/myobjects7

2015-06-17 23:56:56,946 [INFO] [NoneStorage] - performing GET at /s3testqwer2/myobjects10

2015-06-17 23:56:59,423 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects7

2015-06-17 23:56:59,425 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects15

2015-06-17 23:56:59,784 [INFO] [NoneStorage] - performing GET at /s3testqwer1/myobjects5

2015-06-17 23:57:26,220 [INFO] [NoneStorage] - performing GET at /s3testqwer2/myobjects1

2015-06-17 23:57:26,523 [INFO] [NoneStorage] - performing GET at /s3testqwer2/myobjects5

 Cleanup and Dispose stages just delete objects and buckets.

COSbench Cleanup and Dispose stages

2015-06-17 23:57:34,678 [INFO] [NoneStorage] - performing DELETE at /s3testqwer1/myobjects1

2015-06-17 23:57:35,570 [INFO] [NoneStorage] - performing DELETE at /s3testqwer1/myobjects2

2015-06-17 23:57:35,899 [INFO] [NoneStorage] - performing DELETE at /s3testqwer1/myobjects3

2015-06-17 23:57:46,849 [INFO] [NoneStorage] - performing DELETE at /s3testqwer2/myobjects20


2015-06-17 23:57:52,414 [INFO] [NoneStorage] - performing DELETE at /s3testqwer1

2015-06-17 23:57:53,596 [INFO] [NoneStorage] - performing DELETE at /s3testqwer2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s