Analyze historical log files

The regular subscription plans in Qloudstat cover analytics as of the signup date onward. We now offer additional one-time payment subscriptions to fetch log files from prior months.

From your account settings, choose the months backwards from your signup date with unprocessed log files you want to fetch.

A preview will be displayed summarizing the total byte count to be fetched for the selected period with the corresponding price plan. The calculation is based on all unprocessed log files of all your configured endpoints.

Distribution of metrics per hour

Qloudstat offers a resolution of all metrics by day. To give you an idea how these are distributed over a given day we now additionally plot a graph of accumulated hits per hour to give you a better insight of access characteristics over the day.

Additionally you can now also display the timeline with a resolution of weeks or months if you want a more generic view of time data.

Filtering dimension values

Qloudstat allows you to draw quick conclusions about individual dimension values (such as a single resource in your container) by selecting an entry in the value result list. We now also allow to filter these table results with a custom input supporting the ? and * glob patterns to match dimension values that fit the input pattern.

Glob Pattern for URIs

To give an example consider you want to have the metrics for all files that are stored in a container with a given prefix or suffix only. Navigate to the URI dimension analytics page of your report and enter a glob pattern such as */5902/*.png to only list URIs that contain this pattern in their path. This is quite useful to filter versioned resources or limit the result list to files that are served from a specific location in your bucket or CDN origin.

The dimension values list supports multi-selection which gives you a simple way to bring the selected values in relation. When selecting multiple values a pie chart next to the table shows up with the percentage of each value compared to all selected ones.

Update: The filter with search patterns now also works for enumerated dimension values such as HTTP status code.

Purge log files

To enable purging log files after analytics you have to setup IAM (Identity and Access Management) in AWS to allow Qloudstat to delete log files in your account. It is best to restrict such write application to the least minimum required.

Below a code snippet with an example policy to attach to the user setup in IAM in addition to the existing read-only policy that you have configured to allow Qloudstat to fetch log files.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:DeleteObject",
      "Resource": "arn:aws:s3:::bucket/loggingprefix*"
    }
  ]
}

Login to your Qloudstat account to edit your Amazon S3 or CloudFront configuration, where we display this policy preconfigured for your bucket names next to the option to toggle purge for log files.

Our Price Plans

Qloudstat is available with five different price plans to choose from because one size doesn’t fit all. The free plan allows to use Qloudstat at zero costs for sites with less than 100’000 hits per month. The large plan allows as much as 5GB of log data per month. If you need even more, choose the enterprise plan or contact us for specific needs.

The price plans differ based on the following criteria:

  • Endpoints. An endpoint refers to a bucket (S3), distribution (CloudFront) or container (Rackspace). We have just updated our plans to allow an  unlimited (∞) number of configurable endpoints except the free plan. You can freely organize your data in as many endpoints you want without any restriction.
  • Volume per month. Each plan has an upper limit for the volume (number of bytes) of your log files we process. We sum up the number of uncompressed bytes of each log file processed on a monthly base. After a month we reset our counters. If you hit the limit of your plan we temporarily suspend the fetching of new log files until you upgrade your subscription plan. With that said you can always start with the free plan and align your subscription with your changing needs.
  • Retention. We store all your processed log files for at least 36 months. That gives you the possibility to gain a deep insight on how your files have been accessed  over a long time period. Please contact us if you are interested in an even higher retention.
  • Resolution. Log files are processed with a sampling rate of 1 day with a summed up value for each dimension, metric and day.

Note that there are no long term contracts. Once subscribed to a plan, you can cancel, downgrade and upgrade at any time.

Contact us if you are confused which plan to choose. We will help you figure out the best option for you.

Real Artists Ship

After a private beta phase of three months with intensive testing and refinements, we are pleased to announce Qloudstat is now available to the general public. Sign up today! You will be up and running within minutes. Qloudstat does all the work behind the scenes without any administration, maintenance and infrastructure required by you. We have a zero cost plan to get you started.

Download the Press Kit.

We are pleased to announce the immediate availability of Qloudstat, the premier service for analytics of server side access logs of cloud hosted content.

Qloudstat is doing analytics for not only hits but other crucial metrics such as bandwidth and transfer costs. Data is plotted in an interactive chart, timeline, table or geographical map split up by different dimensions in a user friendly web interface. Gain insight of URIs and filetypes used. List referring sites and search keywords. Analyze HTTP user agents, operations and status codes. Compare CDN edge locations. Visualize requests on a country, region or city map. The analytics provided are not static but queries can be made for for any given daily time period with custom filters applied. Dynamic reports are rendered instantaneously not dependent on the time period chosen to be visualized.

Reports are updated continuously around the clock to give instant and always up to date statistics no matter of access volume or the number and size of log files. Qloudstat copes with the rapidly growing traffic using highly optimized scalable systems without any installation, administration, maintenance and infrastructure required by clients. During the 3 months private beta the service was already operational with customers facing up to 25 million hits per day.

Qloudstat integrates with the market leaders in cloud storage and content delivery networks (CDN) currently supporting AWS S3, AWS CloudFront, Google Cloud Storage and Rackspace CloudFiles (Akamai CDN).

Security is a pivotal requirement for cloud based applications. Qloudstat accesses log files in third party accounts using dedicated security credentials either using OAuth authentication for Google Cloud Storage or a user managed under the Identity and Access Management (IAM) for integration with Amazon Web Services. Log files are fetched using a TLS secured connection and the website to access reports is only available with HTTPS as well.

Pricing is based on raw log data volume with four different monthly subscription plans offered. Additionally a free plan allows to use the service at zero costs for sites with less than 100’000 hits per month. With data liberation in mind, export formats for further external processing are provided.

Qloudstat is Swiss made software.

Fast Results

The aim of Qloudstat is to give you instant and always up to date statistics of your cloud accounts. The update frequency depends on the interval your cloud provider writes log files that we can analyze.

  • Rackspace Cloudfiles CDN writes access logs multiple times per hour depending on the traffic and delivery of log files by Akamai.
  • AWS S3 log files are delivered within a few hours of the time that they were recorded.
  • AWS CloudFront access logs are delivered multiple times per hour depending on the traffic.
  • Google Storage currently only provides access logs in 24 hours intervals as their support is still labeled experimental provides hourly log files as of 2012/06/27.

In the list of active configurations and on the left hand below the navigation menu you always see the last update to your report and if new files are currently fetched and analyzed.

 

Qloudstat copes with the rapidly growing traffic using highly optimized scalable systems. Your report will be updated in no time as soon as log files are delivered by the cloud storage provider.

Keep your data safe

We want to emphasize that Qloudstat is engineered to allow access to your log files with dedicated access tokens that are under your control and can be revoked independently from your personal access credentials.

For AWS S3 and AWS CloudFront configurations instead of using your personal access and secret key the best option is to set up an IAM (AWS Identity and Access Management) user in the AWS console. When setting up a new configuration in Qloudstat, you can follow the step-by-step instructions recommended.

Adding a Google Cloud Storage configuration will prompt you for the x-goog-project-id and a token to authorize access for Qloudstat issued with OAuth 2.0 authentication with the permission to read log files from your account without revealing personal access credentials. The token issued is under your control and can always be revoked using your Google Account settings.

We are working together with Rackspace to offer a similar technical solution to access Cloudfiles Akamai CDN access logs. Currently this configuration requires the shared credentials for API access to Cloudfiles.

Private beta now open!

We are now accepting users to the private beta. Signup and give it a try. Configuration is a snap and you will be ready within minutes with plotted graphs of your access logs.

Update: We are currently not enforcing any quota of the different subscription plans while in beta. You are more than welcome to try Qloudstat with big data. Log files created up to two month prior your account setup are processed.

Update (20/03/2012): Following our announcement of public availability we are now enforcing quotas as per our plans.

What is Qloudstat?

The aim of Qloudstat is to make it simple and cost effective to provide access logs analytics of your content distributed in the cloud. The cloud in numbers. plotted.

Qloudstat gathers statistics from access logs of your cloud storage and CDN accounts. Whether using Amazon S3, Amazon CloudFront, Google Cloud Storage or Rackspace CloudFiles with Akamai CDN, Qloudstat gives you instant access to statistics by days, months or years plotted in graphs and maps.

Analyzing access log statistics from cloud providers is a tedious task due to the vast numbers involved and the raw format of the access logs provided. Qloudstat solves this for you with software as a service working behind the scenes without any administration, maintenance and infrastructure required by you.

A typical use case would be to break down downloads hosted in S3 by country, split up transfer costs by referring sites or map the geographical location of visitors.

Stay tuned and request an invitation for the private beta opening very soon!