如何使用shell脚本列出未使用的AWS S3存储桶和空存储桶?

v64noz0r  于 2023-04-21  发布在  Shell
关注(0)|答案(2)|浏览(157)

我正在寻找从过去90天未使用的s3桶列表,也为空桶列表。
为了得到它,我试着写了如下代码:

#/bin/sh
for bucketlist in  $(aws s3api list-buckets --query "Buckets[].Name");
do
  listobjects=$(\
    aws s3api list-objects --bucket $bucketlist \
    --query 'Contents[?contains(LastModified, `2020-08-06`)]')
done

此代码将打印以下输出:[我只添加了一个桶的结果,以供参考]

{
    "Contents": [
        {
            "Key": "test2/image.png",
            "LastModified": "2020-08-06T17:19:10.000Z",
            "ETag": "\"xxxxxx\"",
            "Size": 179008,,
            "StorageClass": "STANDARD",
        }
    ]
}

期望:

1.在上面的代码中,我只想打印在过去90天内没有修改/使用的对象的桶列表。
1.我也在寻找空的遗愿清单
我不擅长编程,有人能指导我吗?提前感谢您的支持。

x8diyxa7

x8diyxa71#

我编写了这个小bash脚本来查找我帐户中的空桶:

#!/bin/zsh
for b in $(aws s3 ls | cut -d" " -f3)
do
  echo -n $b
  if [[ "$(aws s3api list-objects-v2 --bucket $b --max-items 1)" == "" ]]
  then
    echo " BUCKET EMPTY"
  else
    echo ""
  fi
done

我使用list-objects-v2列出了对象,最大项目数为1。如果没有项目,结果为空,我在存储桶名称旁边打印“BUCKET EMPTY”。

  • 注1:您必须具有列出对象的访问权限。
  • 注2:我不确定它将如何工作的版本存储桶与删除的对象(似乎是空的,但实际上包含旧版本的删除对象)

这很好用,但是没有考虑非当前版本的对象。为此,你可以使用“list-object-versions”,然后它也会查找非当前版本的对象。

!/bin/zsh for b in $(aws s3 ls|cut -d”“-f3)do echo -n $B if “$(aws s3api list-object-versions --bucket $b --max-items 1)”==“” then echo“〈----- BUCKET EMPTY”else echo“”fi done

pu3pd22g

pu3pd22g2#

这是我今天写的一个脚本,它不会改变任何东西,但它会给予你命令行来进行更改。

#!/bin/bash
profile="default"
olddate="2020-01-01"
smallbucketsize=10

emptybucketlist=()
oldbucketlist=()
smallbucketlist=()

#for bucketlist in  $(aws s3api list-buckets  --profile $profile  | jq --raw-output '.Buckets[6,7,8,9].Name'); # test this script on just a few buckets
for bucketlist in  $(aws s3api list-buckets  --profile $profile  | jq --raw-output '.Buckets[].Name');
do
  echo "* $bucketlist"
  if [[ ! "$bucketlist" == *"shmr-logs" ]]; then
    listobjects=$(\
      aws s3api list-objects --bucket $bucketlist \
      --query 'Contents[*].Key' \
      --profile $profile)
#echo "==$listobjects=="
    if [[ "$listobjects" == "null" ]]; then
          echo "$bucketlist is empty"
          emptybucketlist+=("$bucketlist")
    else
      # get size
      aws s3 ls --summarize  --human-readable --recursive --profile $profile s3://$bucketlist | tail -n1

      # get number of files
      filecount=$(echo $listobjects | jq length )
      echo "contains $filecount files"
      if [[ $filecount -lt $smallbucketsize ]]; then
          smallbucketlist+=("$bucketlist")
      fi

      # get number of files older than $olddate
      listoldobjects=$(\
        aws s3api list-objects --bucket $bucketlist \
        --query "Contents[?LastModified<=\`$olddate\`]" \
        --profile $profile)
      oldfilecount=$(echo $listoldobjects | jq length )
      echo "contains $oldfilecount old files"

      # check if all files are old
      if [[ $filecount -eq $oldfilecount ]]; then
        echo "all the files are old"
        oldbucketlist+=("$bucketlist")

      fi
    fi
  fi
done
echo -e "\n\n"

echo "check the contents of these buckets which only contain old files"
for oldbuckets in ${oldbucketlist[@]};
do
  echo "$oldbuckets"
done
echo -e "\n\n"

echo "check the contents of these buckets which don't have many files"
for smallbuckets in ${smallbucketlist[@]};
do
  echo "aws s3api list-objects --bucket $smallbuckets --query 'Contents[*].Key' --profile $profile"
done
echo -e "\n\n"

echo "consider deleting these empty buckets"
for emptybuckets in "${emptybucketlist[@]}";
do
  echo "aws s3api delete-bucket --profile $profile --bucket $emptybuckets"
done

相关问题