ECS S3 access via s3curl

ECS S3 access via curlIn that post we will discuss how to get access to another ECS protocol S3 with CLI curl tool.

user1 was configured during the ECS implementation. S3 secret is k2rsBLLn4ZiI74c7C67bDSMpBplS+cPvFDxlWF/T

ECS user configured for S3

s3curl is installed on ECS nodes here: /usr/share/s3curl
xmllint is not installed, so xml should be formatted by another way. Here I formatted xml using xmllint for better readability.

  • Modify .s3curl config file in the home directory (dot at the beginning of the file name means that it is hidden ).
  • 600 permissions should be assigned to .s3curl file.
  • Several profiles can be used. Here “ecsid” profile is configured.
ecsnode:/usr/share/s3curl # touch ~/.s3curl
ecsnode:/usr/share/s3curl # chmod 600 ~/.s3curl
ecsnode:/usr/share/s3curl # vi ~/.s3curl
%awsSecretAccessKeys = (
 # ECS account
 ecsid => {
 id => 'user1',
 key => 'k2rsBLLn4ZiI74c7C67bDSMpBplS+cPvFDxlWF/T',
 },
);
  • Check authentication using ECS account nickname you specified by an ecsid profile in .s3curl file.
ecsnode:/usr/share/s3curl # ./s3curl.pl --id=ecsid -- -v -s http://10.10.10.1:9020/ | xmllint --format -
* Hostname was NOT found in DNS cache
* Trying 10.10.10.1...
* Connected to 10.10.10.1 (10.10.10.1) port 9020 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.10.10.1:9020
> Accept: */*
> Date: Wed, 28 Oct 2015 13:58:06 +0000
> Authorization: AWS user1:In0Qpmtlccq7nzQSJY+/dJVDHDU=
>
< HTTP/1.1 200 OK
< Date: Wed, 28 Oct 2015 13:58:06 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a460258:150a4e067fb:2184:0
< x-amz-id-2:
< Content-Type: application/xml
< Content-Length: 624
<
* Connection #0 to host 10.10.10.1 left intact
 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
 <Owner>
<ID>user1</ID>
 <DisplayName>user1</DisplayName>
 </Owner>
 <Buckets>
 <Bucket>
 <Name>bucket1</Name>
<CreationDate>2015-10-27T10:32:23.041Z</CreationDate>
 </Bucket>
 <Bucket>
 <Name>bucket2</Name>
<CreationDate>2015-10-27T10:36:50.492Z</CreationDate>
 </Bucket>
 <Bucket>
 <Name>bucket3</Name>
<CreationDate>2015-10-27T14:39:57.369Z</CreationDate>
 </Bucket>
 <Bucket>
<Name>bucket4</Name>
<CreationDate>2015-10-27T15:00:38.649Z</CreationDate>
 </Bucket>
 </Buckets>
 <IsTruncated>false</IsTruncated>
</ListAllMyBucketsResult>
  • List objects in a bucket
ecsnode:/usr/share/s3curl # ./s3curl.pl --id=ecsid -- -s http://10.10.10.1:9020/bucket1
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name>bucket1</Name>
 <Prefix/>
 <Marker/>
<MaxKeys>1000</MaxKeys>
<IsTruncated>false</IsTruncated>
 <Contents>
 <Key>Cyberduck-Installer-4.7.3.exe</Key>
<LastModified>2015-10-27T11:28:04.874Z</LastModified>
<ETag>"257e14103628475ba1d49c750807bbe7-4"</ETag>
<Size>28092248</Size>
<StorageClass>STANDARD</StorageClass>
 <Owner/>
 </Contents>
 <Contents>
 <Key>ecs-2.0.1.0-502.a2432fc.240-production.tgz</Key>
<LastModified>2015-10-27T11:27:37.876Z</LastModified>
<ETag>"2e5bdebd336b4a648e459d8f37d909d6-196"</ETag>
<Size>1641001622</Size>
<StorageClass>STANDARD</StorageClass>
 <Owner/>
 </Contents>
 <Contents>
</ListBucketResult>
  • Create and upload a file
ecsnode:/usr/share/s3curl # vi myfile.txt
Hello World
 
ecsnode:/usr/share/s3curl # ./s3curl.pl --id=ecsid --put=myfile.txt -- -s http://10.10.10.1:9020/newbucket/myfile.txt
 
ecsnode:/usr/share/s3curl # ./s3curl.pl --id=ecsid -- -s http://10.10.10.1:9020/newbucket/myfile.txt
Hello World
  • Download and check the file
ecsnode:/usr/share/s3curl # rm myfile.txt
 
ecsnode:/usr/share/s3curl # ./s3curl.pl --id=ecsid -- -s http://10.10.10.1:9020/newbucket/myfile.txt -vv -o myfile.txt
 
* Hostname was NOT found in DNS cache
* Trying 10.10.10.1...
* Connected to 10.10.10.1 (10.10.10.1) port 9020 (#0)
> GET /newbucket/myfile.txt HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.10.10.1:9020
> Accept: */*
> Date: Wed, 28 Oct 2015 14:15:04 +0000
> Authorization: AWS user1:N6jhTHJGPJj7oWJasnrmMUI05x8=
>
< HTTP/1.1 200 OK
< Date: Wed, 28 Oct 2015 14:15:04 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a460258:150a4e067fb:232a:0
< x-amz-id-2: 76a632aadd31cea659699f6944cfc46f959525a08a7243f1c89ddf49b7cc7898
< ETag: "e59ff97941044f85df5297e1c302d260"
< Last-Modified: Wed, 28 Oct 2015 14:12:26 GMT
< x-emc-mtime: 1446041546338
< Content-Encoding: identity
< Content-Length: 12
< Content-Type: application/octet-stream
<
{ [data not shown]
* Connection #0 to host 10.10.10.1 left intact
 
ecsnode:/usr/share/s3curl # cat myfile.txt
Hello World

2 thoughts on “ECS S3 access via s3curl

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s