Query Strings in URIs

It is a common practice to include query strings in URIs pointing to CloudFront or S3 resources to get a better understanding of the popularity of different sources for your content. Qloudstat visualizes these query strings by hits or bandwidth usage over time.

For CloudFront, any query string appended to an URI is logged and then stripped when fetching the origin resource. AWS had a blog post when introducing this feature. Arbitrary query strings are supported. The same applies for Rackspace Cloudfiles with a Akamai CDN setup.

For S3, you must prepend query strings in URIs with “x-“. More details from the S3 documentation:

You can include custom information to be stored in the access log record for a request by adding a custom query-string parameter to the URL for the request. Amazon S3 will ignore query-string parameters that begin with “x-“, but will include those parameters in the access log record for the request, as part of the Request-URI field of the log record. For example, a GET request for “s3.amazonaws.com/mybucket/photos/2006/08/puppy.jpg?x-user=johndoe” will work the same as the same request for “s3.amazonaws.com/mybucket/photos/2006/08/puppy.jpg”, except that the “x-user=johndoe” string will be included in the Request-URI field for the associated log record. This functionality is available in the REST interface only.

Purge log files

To enable purging log files after analytics you have to setup IAM (Identity and Access Management) in AWS to allow Qloudstat to delete log files in your account. It is best to restrict such write application to the least minimum required.

Below a code snippet with an example policy to attach to the user setup in IAM in addition to the existing read-only policy that you have configured to allow Qloudstat to fetch log files.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:DeleteObject",
      "Resource": "arn:aws:s3:::bucket/loggingprefix*"
    }
  ]
}

Login to your Qloudstat account to edit your Amazon S3 or CloudFront configuration, where we display this policy preconfigured for your bucket names next to the option to toggle purge for log files.