New Time Metric for CloudFront Logs

Amazon silently introduced a new time-taken property to the CloudFront access logs for web distributions last week.

The time-taken field shows the number of seconds between the time a CloudFront edge server receives a viewer’s request and the time that CloudFront writes the last byte of the response to the server’s output queue as measured on the server.

We have added support for this additional field which can be evaluated using the Total Time metric in Qloudstat.

Elastic Load Balancer Support

Amazon has just announced support for access logs of their Elastic Load Balancers (ELB). We have already added the configuration option for ELB in Qloudstat. Choose Add Amazon Elastic Load Balancer Configuration on your configuration page making sure the IAM user you create for Qloudstat has the required permissions. Just copy and paste the IAM policy statements that are displayed when entering the user credentials in Qloudstat.

ELB Setup

We support URI, geographical location, service provider, query string, HTTP operation and backend status code dimensions with hits, bandwidth, cost and turnaround time metrics. The turnaround time metric allows you to graph the backend processing time by URI with drill down options by country, HTTP operation, HTTP status code and and query string dimensions.

HTTP Status Code response time metrics

Turnaround time and total request processing time for URIs by HTTP Status code from the backend server.

Turnaround Time by Country

Turnaround time by Country. 

Discovery API

We have just enhanced our API to make it easier to automate scripts to discover endpoints configured in Qloudstat. You can find the documentation on the API page. The API endpoint now returns JSON for various levels of your configuration:

  • Query all enabled endpoints in your account with
    curl --digest --user <user>:<secret>
  • Query all supported dimensions
    curl --digest --user <user>:<secret><endpoint>
  • Query available metrics for dimensions
    curl --digest --user <user>:<secret><endpoint>/uri,user_agent

The responses are JSON formatted suitable for automated processing with scripts or programs.

Configuration change notifications

We are now sending out notification mails for changes in your configurations detected by Qloudstat. When a new S3 bucket or CloudFront distribution is detected or has been changed with logging enabled, a mail is sent to all registered users of your Qloudstat account that allows you to enable the endpoint for analytics by opening an activation link.

This will make it a lot easier to keep analytics in sync with changes to your configurations without the need to manually edit the configuration in Qloudstat.

Retention Options

We now offer new retention options for custom plans. For historical and recurring subscription with a plan quota size above our Enterprise offering one can choose between 12, 24 and 36 months.

Custom plan retention period

A lower retention time gives a substantial discount to our regular pricing as it drives down our storage costs. Try our calculator for an offer based on your current log volume.


Partial Transfer Metric

Qloudstat supports a partial transfer metric for S3 and Rackspace Cloudfiles/Akamai endpoints. We determine partial transfers as follows:

  • S3. Log files for S3 include the actual transmitted number of bytes. If it differs from the content length the request is counted as partial.
  • Rackspace Cloudfiles. Partial transfer counts are based on a feature by Akamai CDN that marks transfers interrupted by the client with a 000 status code in the log file.

These are also taken into account when counting bandwidth and transfer cost metrics.

Integration with Cyberduck

The latest release of Cyberduck, a cloud storage browser to access S3 or Rackspace accounts, simplifies the setup of access log analytics with Qloudstat with a click of a button. If you already use Cyberduck to configure access logging for CloudFront distributions, S3 buckets or containers in Rackspace Cloudfiles, getting reports in Qloudstat just got a lot easier with the new Read Access for Qloudstat checkbox in the Info panel tab for CDN or S3 buckets.

Rackspace Analytics Setup for Qloudstat in CyberduckS3 Analytics Setup for Qloudstat in Cyberduck.png

For Amazon S3 buckets or CloudFront distributions, this will create a dedicated IAM user with a read only IAM policy for Qloudstat to fetch log files. Unchecking the Read Access for Qloudstat checkbox will remove the IAM user again revoking all access for Qloudstat. When enabled, a clickable link is displayed that redirects to Qloudstat to confirm the new setup.

Confirm Rackspace Setup in Qloudstat

Announcing the Qloudstat API


We are happy to announce the beta of the Qloudstat API today. For the impatient, you find all the technical documentation in our API documentation, including samples. The ReSTful API allows you to request data as JSON or CSV depending on the further processing required. You can query your analytics data by values or by time.

  • Queries by value return dimension values (such as the URIs or HTTP status codes) with metrics.
  • With a timeline query, requests that match the query filter (such as 404 in the HTTP status code dimension) are grouped by day.

Authentication is implemented using digest access authentication which is both secure and  is implemented in popular tools such as curl and HTTP client libraries in Python, Ruby or Java or any other popular programming languages. This allows you to quickly wrap up a script solution to integrate the data in Qloudstat with your applications. Multiple access tokens can be created and revoked for your account if you want to grant (temporary) read access to your data to third parties.

The Qloudstat API has the same feature set to allow all queries that you get in the web user interface. This feature parity allows to make a valid API call for every single graph that you see in the web interface.

We think the API is an important step to liberate the data in Qloudstat. While previously data could be downloaded from the web interface as a Excel or CSV, the full blown API allows for much better integration in your existing or new workflows.

To get started, copy the API endpoint URL from your configuration and follow the directions to query by time or values per our API documentation. Please don’t hesitate to let us know how it works for you!

Custom Plan Price Calculator

We have just added a calculator for custom plans based on the estimated log file volume for enterprise customers which need a higher quota than what we offer in the standard plans.

Custom Plan Calculator

To make a educated guess about the expected log file size volume, Qloudstat peeks into the logging targets of your configured endpoints to predict the estimated quota requirement and suggest a plan

Support for Rackspace/Akamai Streaming Logs

Containers in Rackspace Cloudfiles have both a download and streaming CDN URL to distribute content. Log events from the HTTP streaming URL for both US and UK containers are now supported in Qloudstat and give an overview of streaming events triggered by clients (such as connect, play, stop and disconnect) for a given URI with hits and bandwidth metrics. No configuration change is required for your existing endpoints.