如何使用适用于GOlang的AWS开发工具包清空S3存储桶?

如何使用适用于GOlang的AWS开发工具包清空S3存储桶?

问题描述:

目标:使用适用于GOlang的AWS开发工具包清空现有S3存储桶。如何使用适用于GOlang的AWS开发工具包清空S3存储桶?

+0

**建议:**你有联系AWS或阅读他们的更新? – Ibo

+0

我创建了这个问题来帮助其他可能面临同样问题的人。我已经在下面提出了我的解决方案。感谢您的提示,但! – SKRelan

NOTE:这些代码片段可能需要您在您的侧面进行更改以使其运行。

您将需要实现以下方法:

//EmptyBucket empties the Amazon S3 bucket 
    func (s awsS3) EmptyBucket(bucket string) error { 
     log.Info("removing objects from S3 bucket : ", bucket) 
     params := &s3.ListObjectsInput{ 
      Bucket: aws.String(bucket), 
     } 
     for { 
      //Requesting for batch of objects from s3 bucket 
      objects, err := s.Client.ListObjects(params) 
      if err != nil { 
       return err 
      } 
      //Checks if the bucket is already empty 
      if len((*objects).Contents) == 0 { 
       log.Info("Bucket is already empty") 
       return nil 
      } 
      log.Info("First object in batch | ", *(objects.Contents[0].Key)) 

      //creating an array of pointers of ObjectIdentifier 
      objectsToDelete := make([]*s3.ObjectIdentifier, 0, 1000) 
      for _, object := range (*objects).Contents { 
       obj := s3.ObjectIdentifier{ 
        Key: object.Key, 
       } 
       objectsToDelete = append(objectsToDelete, &obj) 
      } 
      //Creating JSON payload for bulk delete 
      deleteArray := s3.Delete{Objects: objectsToDelete} 
      deleteParams := &s3.DeleteObjectsInput{ 
       Bucket: aws.String(bucket), 
       Delete: &deleteArray, 
      } 
      //Running the Bulk delete job (limit 1000) 
      _, err = s.Client.DeleteObjects(deleteParams) 
      if err != nil { 
       return err 
      } 
      if *(*objects).IsTruncated { //if there are more objects in the bucket, IsTruncated = true 
       params.Marker = (*deleteParams).Delete.Objects[len((*deleteParams).Delete.Objects)-1].Key 
       log.Info("Requesting next batch | ", *(params.Marker)) 
      } else { //if all objects in the bucket have been cleaned up. 
       break 
      } 
     } 
     log.Info("Emptied S3 bucket : ", bucket) 
     return nil 
    } 

UPDATE:AWS SDK为GO的最新版本已经解决了这个问题之前,我有。

+0

您是否在sdk中提出了恐慌问题?你用'ListObjectsV2'试过了当前的api吗? – JimB

+0

不,我还没有提出恐慌的问题。我计划在这个周末做这件事。是的,这个问题出现在使用当前的API中。 – SKRelan

适用于Go的AWS开发工具包具有Amazon S3批处理抽象。看看here

不要忘记,默认情况下,ListObjects最多只能返回1000个存储桶项目。如果您可能拥有超过1000个,请在返回值上检查IsTruncated属性。如果为true,则使用返回值中的NextMarker属性获取下一个1000项。

见我的例子去开发指南:http://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-basic-bucket-operations.html#s3-examples-bucket-ops-delete-all-bucket-items