Most of the interactions you’ve had with S3 in this course had to do with objects. You didn’t see many bucket-related operations, such as adding policies to the bucket, adding a LifeCycle rule to transition your objects through the storage classes, archive them to Glacier or delete them altogether or enforcing that all objects be encrypted by configuring Bucket Encryption.
Manually managing the state of your buckets via Boto3’s clients or resources becomes increasingly difficult as your application starts adding other services and grows more complex. To monitor your infrastructure in concert with Boto3, consider using an Infrastructure as Code (IaC) tool such as CloudFormation or Terraform to manage your application’s infrastructure. Either one of these tools will maintain the state of your infrastructure and inform you of the changes that you’ve applied.
If you decide to go down this route, keep the following in mind:
- Any bucket related-operation that modifies the bucket in any way should be done via IaC.
- If you want all your objects to act in the same way (all encrypted, or all public, for example), usually there is a way to do this directly using IaC, by adding a Bucket Policy or a specific Bucket property.
- Bucket read operations, such as iterating through the contents of a bucket, should be done using Boto3.
- Object-related operations at an individual object level should be done using Boto3.
Mark Walle on Aug. 24, 2020
There are AWS CDK (Cloud Development Kit) bindings for Python that allow people who prefer to scaffold their IAC in an imperative language rather than as declarative markup. After defining an IAC stack in Python using the CDK, it can then generate the necessary CloudFormation template to build, deploy, or destroy the infrastructure defined in your Python code.
It would be great to see a multipart series on using AWS CDK with Python on Real Python. But in absence of that, here is an AWS Developer blog that provides a good start with the tool: aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-python/