AWS – S3 Buckets Tag remediation automation

This article will help you to remediate the S3 buckets tags using the AWS CLI script. Amazon’s simple storage service aka s3 is object storage built to retrieve any amount of data from anywhere. It gives a lot of power to the developers and IT support teams to provision and access the storage in no time. Tagging resources in the cloud is one of the most important aspects. Without proper tags, organizations will not have any clue to track the cloud resources cost. The following script will help you to search for the key and update the value.

Even startups will have 100 to 200 s3 buckets and updating the tags for each bucket is a tedious job. Using the following script, we can change the specific tag key’s value.

Here is my S3 bucket with defined tags.

Pre-requesties:

• AWS CLI version > 2.0x
• Bash shell (Linux Machine or AWS CloudShell from portal)

Script:

#!/bin/bash
LOGFILE=Tagupdate_`date +%Y-%m-%d_%H-%M-%S`.log
while IFS=, read -r BUCKETNAME TAGKEY OLDTAGVAL NEWTAGVAL; do
            aws s3api get-bucket-tagging --bucket $BUCKETNAME > bck.json 
            if [[ $? -ne 0 ]]; then
                    echo "unable to fetch the tags for $BUCKETNAME" >> $LOGFILE;
            else 
                    echo "updating tag for $BUCKETNAME"
                    sed -i '/'"$KEY"'/ {N;s/'"$OLDTAG"'/'"$NEWTAG"'/}' bck.json 
                aws s3api put-bucket-tagging --bucket $BUCKETNAME --tagging file://bck.json
                if [[ $? -ne 0 ]]; then
                        echo "unable to update the tags for $BUCKETNAME" >> $LOGFILE;
           else 
                echo "List of tags for $BUCKETNAME after update"
                aws s3api get-bucket-tagging --bucket $BUCKETNAME
                fi
            fi
done < s3_tag_info.csv

How to use the script ?

1. Copy the script and make it executable.

[cloudshell-user@ip-10-0-180-42 UA]$ ls -lrt
total 4
-rwxrwxr-x 1 cloudshell-user cloudshell-user 893 Nov 24 16:50 s3_tag_update.sh
[cloudshell-user@ip-10-0-180-42 UA]$ 

2. Create a CSV file in the following format.

BUCKET NAME, TAG KEY, OLD TAG VALUE, NEW TAG VALUE

Example:

[cloudshell-user@ip-10-0-180-42 UA]$ cat s3_tag_info.csv 
test1ualin,CostCenter,UnixArena001,UA002
test2ualin,CostCenter,UnixArena001,UA003
[cloudshell-user@ip-10-0-180-42 UA]$ 

3. Here is my bucket’s existing tag values.

[cloudshell-user@ip-10-0-180-42 UA]$ aws s3api get-bucket-tagging --bucket test1ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UnixArena001"
        },
        {
            "Key": "Environment",
            "Value": "QA"
        }
    ]
}
[cloudshell-user@ip-10-0-180-42 UA]$ aws s3api get-bucket-tagging --bucket test2ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UnixArena001"
        },
        {
            "Key": "Environment",
            "Value": "DEV"
        }
    ]
}

4. Execute the script to update the new tag value which is defined in the CSV file.

[cloudshell-user@ip-10-0-180-42 UA]$ bash -x s3_tag_update.sh 
++ date +%Y-%m-%d_%H-%M-%S
+ LOGFILE=Tagupdate_2021-11-24_17-29-49.log
+ IFS=,
+ read -r BUCKETNAME TAGKEY OLDTAGVAL NEWTAGVAL
+ aws s3api get-bucket-tagging --bucket test1ualin
+ [[ 0 -ne 0 ]]
+ echo 'updating tag for test1ualin'
updating tag for test1ualin
+ sed -i '/CostCenter/ {N;s/UnixArena001/UA002/}' bck.json
+ aws s3api put-bucket-tagging --bucket test1ualin --tagging file://bck.json
+ [[ 0 -ne 0 ]]
+ echo 'List of tags for test1ualin after update'
List of tags for test1ualin after update
+ aws s3api get-bucket-tagging --bucket test1ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UA002"
        },
        {
            "Key": "Environment",
            "Value": "QA"
        }
    ]
}
+ IFS=,
+ read -r BUCKETNAME TAGKEY OLDTAGVAL NEWTAGVAL
+ aws s3api get-bucket-tagging --bucket test2ualin
+ [[ 0 -ne 0 ]]
+ echo 'updating tag for test2ualin'
updating tag for test2ualin
+ sed -i '/CostCenter/ {N;s/UnixArena001/UA003/}' bck.json
+ aws s3api put-bucket-tagging --bucket test2ualin --tagging file://bck.json
+ [[ 0 -ne 0 ]]
+ echo 'List of tags for test2ualin after update'
List of tags for test2ualin after update
+ aws s3api get-bucket-tagging --bucket test2ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UA003"
        },
        {
            "Key": "Environment",
            "Value": "DEV"
        }
    ]
}
+ IFS=,
+ read -r BUCKETNAME TAGKEY OLDTAGVAL NEWTAGVAL
[cloudshell-user@ip-10-0-180-42 UA]$ 

5. If there are any permission errors, it will get updated in the log file.

6. Script already retried the updated tag in the output. If you would like to check it using aws cli, you could check it.

[cloudshell-user@ip-10-0-180-42 UA]$ aws s3api get-bucket-tagging --bucket test1ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UA002"
        },
        {
            "Key": "Environment",
            "Value": "QA"
        }
    ]
}
[cloudshell-user@ip-10-0-180-42 UA]$ aws s3api get-bucket-tagging --bucket test2ualin
{
    "TagSet": [
        {
            "Key": "CostCenter",
            "Value": "UA003"
        },
        {
            "Key": "Environment",
            "Value": "DEV"
        }
    ]
}
[cloudshell-user@ip-10-0-180-42 UA]$ 

We have successfully updated tags for S3 buckets using s3api API.

This is just one of the ways to update the tag values. There are other methods to update the AWS resource tags. AWS’s tagging API very powerful command-line tool to update the tags and another method is using AWS SSM document.