https://aws.amazon.com/

Configure aws CLI

OS X

You need to have homebrew installed

1
2
$ brew install awscli
$ aws configure

then put in your access ID & key (don’t know where to get, read IAM doc)

Reference:

Delete all pending queue items

1
$ aws sqs purge-queue --queue-url https://sqs.ap-southeast-1.amazonaws.com/123456789012/queue_name
Reference:

Use aws command with specific credentials

We can set multiple credentials in ~/.aws/credentials

1
2
3
4
5
6
7
8
9
10
11
[default]
aws_access_key_id=ABCDEFGHIJKLMNOPQRSTUVWXYZ
aws_secret_access_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[pf1]
aws_access_key_id=ZYXWVUTSRQPONMLKJIHGFEDCBA
aws_secret_access_key=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

[pf2]
aws_access_key_id=ABCDEFGHIJKLMESBM7VFCXU
aws_secret_access_key=zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

We can specify with --profile option. e.g.

1
$ aws s3 ls --profile pf2

OR

Run any command with specific aws profile

1
$ AWS_PROFILE=pf2 python3 download_s3.py
Reference:

S3 copy wildcard (*)

let say we want to achieve

1
$ cp /path/to/files/wanted* s3://bucket/new_folder/

The correct way should be

1
$ aws s3 cp /path/to/files/ s3://bucket/new_folder/ --exclude "*" --include "wanted*" --recursive
Reference:

Get the top 10 lines from a large S3 file

1
aws s3api get-object --bucket my-bucket --key path/to/large-file.csv --range bytes=0-10000 /dev/stdout | head -10

S3 bucket policy deny all but allow only a single role

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Deny",
			"NotPrincipal": {
				"AWS": [
					"arn:aws:iam::444455556666:root"
				]
			},
			"Action": "s3:*",
			"Resource": ["arn:aws:s3:::BUCKETNAME", "arn:aws:s3:::BUCKETNAME/*"],
            "Condition": {
                "ArnNotEquals": {
                    "aws:PrincipalArn": "arn:aws:iam::444455556666:role/the-allowed-role-name"
                }
            }
		}
	]
}
Reference: