如何使用Java列出存储桶中的所有AWS S3对象

使用Java获取S3存储桶中所有项目的列表的最简单方法是什么？

List<S3ObjectSummary> s3objects = s3.listObjects(bucketName,prefix).getObjectSummaries(); 

这个例子只返回1000个项目。

Answer 1

这可能是一个解决方法，但这解决了我的问题：

 ObjectListing listing = s3.listObjects( bucketName, prefix ); List<S3ObjectSummary> summaries = listing.getObjectSummaries(); while (listing.isTruncated()) { listing = s3.listNextBatchOfObjects (listing); summaries.addAll (listing.getObjectSummaries()); } 

Answer 2

我正在处理由我们的系统生成的大量对象集合; 我们改变了存储数据的格式，需要检查每个文件，确定哪些文件是旧格式的，并将其转换。 还有其他方法可以做到这一点，但是这个与你的问题有关。

  ObjectListing list = amazonS3Client.listObjects(contentBucketName, contentKeyPrefix); do { List<S3ObjectSummary> summaries = list.getObjectSummaries(); for (S3ObjectSummary summary : summaries) { String summaryKey = summary.getKey(); /* Retrieve object */ /* Process it */ } list = amazonS3Client.listNextBatchOfObjects(list); }while (list.isTruncated()); 

Answer 3

这是从AWS文档直接：

 AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider()); ListObjectsRequest listObjectsRequest = new ListObjectsRequest() .withBucketName(bucketName) .withPrefix("m"); ObjectListing objectListing; do { objectListing = s3client.listObjects(listObjectsRequest); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } listObjectsRequest.setMarker(objectListing.getNextMarker()); } while (objectListing.isTruncated()); 

Answer 4

作为一个稍微更简洁的解决scheme来列举S3对象时，他们可能会被截断：

 ListObjectsRequest request = new ListObjectsRequest().withBucketName(bucketName); ObjectListing listing = null; while((listing == null) || (request.getMarker() != null)) { listing = s3Client.listObjects(request); // do stuff with listing request.setMarker(listing.getNextMarker()); } 

Answer 5

灰色的解决scheme很奇怪，但你看起来像个好人。

 AmazonS3Client s3Client = new AmazonS3Client(new BasicAWSCredentials( .... ObjectListing images = s3Client.listObjects(bucketName); List<S3ObjectSummary> list = images.getObjectSummaries(); for(S3ObjectSummary image: list) { S3Object obj = s3Client.getObject(bucketName, image.getKey()); writeToFile(obj.getObjectContent()); } 

Answer 6

我知道这是一个旧的post，但是这仍然可能对任何人都有用：版本2.1上的Java / Android SDK提供了一个名为setMaxKeys的方法。 喜欢这个：

 s3objects.setMaxKeys(arg0) 

您现在可能已经find了一个解决scheme，但请检查一个答案是否正确，以便将来可以帮助其他人。

Answer 7

这对我有效。

 Thread thread = new Thread(new Runnable(){ @Override public void run() { try { List<String> listing = getObjectNamesForBucket(bucket, s3Client); Log.e(TAG, "listing "+ listing); } catch (Exception e) { e.printStackTrace(); Log.e(TAG, "Exception found while listing "+ e); } } }); thread.start(); private List<String> getObjectNamesForBucket(String bucket, AmazonS3 s3Client) { ObjectListing objects=s3Client.listObjects(bucket); List<String> objectNames=new ArrayList<String>(objects.getObjectSummaries().size()); Iterator<S3ObjectSummary> oIter=objects.getObjectSummaries().iterator(); while (oIter.hasNext()) { objectNames.add(oIter.next().getKey()); } while (objects.isTruncated()) { objects=s3Client.listNextBatchOfObjects(objects); oIter=objects.getObjectSummaries().iterator(); while (oIter.hasNext()) { objectNames.add(oIter.next().getKey()); } } return objectNames; } 

Answer 8

使用AWS SDK for Java列出密钥

http://docs.aws.amazon.com/AmazonS3/latest/dev/ListingObjectKeysUsingJava.html

 import java.io.IOException; import com.amazonaws.AmazonClientException; import com.amazonaws.AmazonServiceException; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3Client; import com.amazonaws.services.s3.model.ListObjectsRequest; import com.amazonaws.services.s3.model.ListObjectsV2Request; import com.amazonaws.services.s3.model.ListObjectsV2Result; import com.amazonaws.services.s3.model.ObjectListing; import com.amazonaws.services.s3.model.S3ObjectSummary; public class ListKeys { private static String bucketName = "***bucket name***"; public static void main(String[] args) throws IOException { AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider()); try { System.out.println("Listing objects"); final ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucketName).withMaxKeys(2); ListObjectsV2Result result; do { result = s3client.listObjectsV2(req); for (S3ObjectSummary objectSummary : result.getObjectSummaries()) { System.out.println(" - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println("Next Continuation Token : " + result.getNextContinuationToken()); req.setContinuationToken(result.getNextContinuationToken()); } while(result.isTruncated() == true ); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, " + "which means your request made it " + "to Amazon S3, but was rejected with an error response " + "for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, " + "which means the client encountered " + "an internal error while trying to communicate" + " with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } } } 

Answer 9

试试这个

 public void getObjectList(){ System.out.println("Listing objects"); ObjectListing objectListing = s3.listObjects(new ListObjectsRequest() .withBucketName(bucketName) .withPrefix("ads")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println(" - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } } 

您可以使用特定的前缀来存储桶中的所有对象。