IAM Policies in Depth

Our recommended way to grant Qloudstat access to your Amazon S3 & CloudFront log files is to create an IAM user with a read-only policy attached. This post is derived from the AWS S3 documentation and CloudFront documentation on IAM policies.

When setting up a new AWS configuration in Qloudstat, you are asked to enter a valid Access Key and Secret Key. This could be your main AWS credentials but this is discouraged. Instead we recommend you to login to the IAM console and create a new user with its dedicated access key.

S3
You can attach the IAM Read Only Policy Template which should suit most needs. A further restricted custom policy with the least grants would be edited like

{
    "Statement":[
        {
             "Effect":"Allow",
             "Action":[
                "s3:GetObject",
                "s3:ListBucket"
             ],
             "Resource":"arn:aws:s3:::logging-target-bucket/*",
             "Condition":{
                "Bool":{
                    "aws:SecureTransport":"true"
                }
             }
        },
        {
            "Effect":"Allow",
            "Action":[
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation",
                "s3:GetBucketLogging"
            ],
            "Resource":"arn:aws:s3:::*",
            "Condition":{
                "Bool":{
                   "aws:SecureTransport":"true"
                }
            }
        }
    ]
}
  • To facilitate handling of your buckets in Qloudstat, we recommend to grant the s3:ListAllMyBuckets to the user.
  • Grant reading the logging status and location of every bucket.
  • Grant listing and fetching files in the target logging bucket named logging-target-bucket. You must repeat this statement for all your logging target buckets or use the wildcard resource name arn:aws:s3:::*
  • All communication must be secured using HTTPS.

You can find additional information in the Qloudstat FAQ.

CloudFront
A policy to fetch log files for CloudFront distributions must allow to read your CloudFront distribution status plus fetching the log files from the S3 logging target bucket.

{
    "Statement": [
        {
          "Action": [
            "s3:Get*",
            "s3:List*"
          ],
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::logging-target-bucket/*"
        },
        {
          "Action": [
            "cloudfront:Get*",
            "cloudfront:List*"
          ],
          "Effect": "Allow",
          "Resource": "*"
        }
    ]
}
  • An asterisk (*) is used as the resource when writing a policy to control access to CloudFront distributions. There are no CloudFront resource ARNs (Amazon Resource Names) for you to use in an IAM policy, because IAM cannot control access to specific CloudFront distributions.
  • To facilitate handling of your distributions in Qloudstat, we recommend to grant the cloudfront:ListDistributions to the user. We use a cloudfront:List* wildcard to include both download and streaming (cloudfront:ListStreamingDistributions) API actions.

You can find additional information in the Qloudstat FAQ.

Purge log files

To enable purging log files after analytics you have to setup IAM (Identity and Access Management) in AWS to allow Qloudstat to delete log files in your account. It is best to restrict such write application to the least minimum required.

Below a code snippet with an example policy to attach to the user setup in IAM in addition to the existing read-only policy that you have configured to allow Qloudstat to fetch log files.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:DeleteObject",
      "Resource": "arn:aws:s3:::bucket/loggingprefix*"
    }
  ]
}

Login to your Qloudstat account to edit your Amazon S3 or CloudFront configuration, where we display this policy preconfigured for your bucket names next to the option to toggle purge for log files.

Keep your data safe

We want to emphasize that Qloudstat is engineered to allow access to your log files with dedicated access tokens that are under your control and can be revoked independently from your personal access credentials.

For AWS S3 and AWS CloudFront configurations instead of using your personal access and secret key the best option is to set up an IAM (AWS Identity and Access Management) user in the AWS console. When setting up a new configuration in Qloudstat, you can follow the step-by-step instructions recommended.

Adding a Google Cloud Storage configuration will prompt you for the x-goog-project-id and a token to authorize access for Qloudstat issued with OAuth 2.0 authentication with the permission to read log files from your account without revealing personal access credentials. The token issued is under your control and can always be revoked using your Google Account settings.

We are working together with Rackspace to offer a similar technical solution to access Cloudfiles Akamai CDN access logs. Currently this configuration requires the shared credentials for API access to Cloudfiles.