After running Qloudstat for almost a decade we have decided to not continue our analytics and monitoring service. As many competitors and Amazon itself have heavily invested in providing analytics similar to Qloudstat we were lacking manpower to keep the pace with extending the platform. Qloudstat will be shut down on February 28, 2021. The service is operating normally till this date and all your logs are still processed. During this time you will continue to be able to export your data using the web interface or our API. All existing subscriptions will automatically be cancelled by the end of February. If you want to continue using an analytics service for your cloud log files we recommend to use any of the Amazon tools such as
Thank you for trusting Qloudstat over all the years!
We now support analytics for AWS Application Load Balancer (ALB) in addition to classic ELB. ALB endpoints configured in Qloudstat offer many dimensions and metrics that you can drilldown to get insights into the usage of your applications hosted on AWS EC2.
Elastic Load Balancing has become a key architectural component for many AWS-powered applications. In conjunction with Auto Scaling, Elastic Load Balancing greatly simplifies the task of building applications that scale up and down while maintaining high availability.
Application Load Balancers support content-based routing, and supports applications that run in containers.
With Qloudstat, it is now much easier to get quickly an overview to access characteristics to your application running behind an Application Load Balancer (ALB) without having to manually parse log files dumped by AWS to your logging container.
We have added support for completed and partial transfer metrics for CloudFront distributions. Per the CloudFront documentation, these requests have a
200 HTTP response status and a value of
Error for the edge result type.
You can also drill down manually by CDN Edge Cache Result → HTTP Status Code.
Amazon Web Services has announced its second region in Europe opened in Frankfurt which is now fully supported in Qloudstat.
Amazon silently introduced a new
time-taken property to the CloudFront access logs for web distributions last week.
The time-taken field shows the number of seconds between the time a CloudFront edge server receives a viewer’s request and the time that CloudFront writes the last byte of the response to the server’s output queue as measured on the server.
We have added support for this additional field which can be evaluated using the Total Time metric in Qloudstat.
Eating our own dog food, the website query interface is now using entirely our public API.
Amazon has just announced support for access logs of their Elastic Load Balancers (ELB). We have already added the configuration option for ELB in Qloudstat. Choose Add Amazon Elastic Load Balancer Configuration on your configuration page making sure the IAM user you create for Qloudstat has the required permissions. Just copy and paste the IAM policy statements that are displayed when entering the user credentials in Qloudstat.
We support URI, geographical location, service provider, query string, HTTP operation and backend status code dimensions with hits, bandwidth, cost and turnaround time metrics. The turnaround time metric allows you to graph the backend processing time by URI with drill down options by country, HTTP operation, HTTP status code and and query string dimensions.
Turnaround time and total request processing time for URIs by HTTP Status code from the backend server.
Turnaround time by Country.
Several business customers asked us for an annual payment option. We are now offering a one year subscription with a fixed quota per month for all plans with a discount of one month for free.
We have just enhanced our API to make it easier to automate scripts to discover endpoints configured in Qloudstat. You can find the documentation on the API page. The API endpoint now returns JSON for various levels of your configuration:
- Query all enabled endpoints in your account with
curl --digest --user <user>:<secret> https://api.qloudstat.com/v12/
- Query all supported dimensions
curl --digest --user <user>:<secret> https://api.qloudstat.com/v12/<endpoint>
- Query available metrics for dimensions
curl --digest --user <user>:<secret> https://api.qloudstat.com/v12/<endpoint>/uri,user_agent
The responses are JSON formatted suitable for automated processing with scripts or programs.
We are now sending out notification mails for changes in your configurations detected by Qloudstat. When a new S3 bucket or CloudFront distribution is detected or has been changed with logging enabled, a mail is sent to all registered users of your Qloudstat account that allows you to enable the endpoint for analytics by opening an activation link.
This will make it a lot easier to keep analytics in sync with changes to your configurations without the need to manually edit the configuration in Qloudstat.
We now offer new retention options for custom plans. For historical and recurring subscription with a plan quota size above our Enterprise offering one can choose between 12, 24 and 36 months.
A lower retention time gives a substantial discount to our regular pricing as it drives down our storage costs. Try our calculator for an offer based on your current log volume.
Qloudstat supports a partial transfer metric for S3 and Rackspace Cloudfiles/Akamai endpoints. We determine partial transfers as follows:
- S3. Log files for S3 include the actual transmitted number of bytes. If it differs from the content length the request is counted as partial.
- Rackspace Cloudfiles. Partial transfer counts are based on a feature by Akamai CDN that marks transfers interrupted by the client with a 000 status code in the log file.
These are also taken into account when counting bandwidth and transfer cost metrics.
The latest release of Cyberduck, a cloud storage browser to access S3 or Rackspace accounts, simplifies the setup of access log analytics with Qloudstat with a click of a button. If you already use Cyberduck to configure access logging for CloudFront distributions, S3 buckets or containers in Rackspace Cloudfiles, getting reports in Qloudstat just got a lot easier with the new Read Access for Qloudstat checkbox in the Info panel tab for CDN or S3 buckets.
For Amazon S3 buckets or CloudFront distributions, this will create a dedicated IAM user with a read only IAM policy for Qloudstat to fetch log files. Unchecking the Read Access for Qloudstat checkbox will remove the IAM user again revoking all access for Qloudstat. When enabled, a clickable link is displayed that redirects to Qloudstat to confirm the new setup.
We are happy to announce the beta of the Qloudstat API today. For the impatient, you find all the technical documentation in our API documentation, including samples. The ReSTful API allows you to request data as JSON or CSV depending on the further processing required. You can query your analytics data by values or by time.
- Queries by value return dimension values (such as the URIs or HTTP status codes) with metrics.
- With a timeline query, requests that match the query filter (such as 404 in the HTTP status code dimension) are grouped by day.
Authentication is implemented using digest access authentication which is both secure and is implemented in popular tools such as curl and HTTP client libraries in Python, Ruby or Java or any other popular programming languages. This allows you to quickly wrap up a script solution to integrate the data in Qloudstat with your applications. Multiple access tokens can be created and revoked for your account if you want to grant (temporary) read access to your data to third parties.
The Qloudstat API has the same feature set to allow all queries that you get in the web user interface. This feature parity allows to make a valid API call for every single graph that you see in the web interface.
We think the API is an important step to liberate the data in Qloudstat. While previously data could be downloaded from the web interface as a Excel or CSV, the full blown API allows for much better integration in your existing or new workflows.
To get started, copy the API endpoint URL from your configuration and follow the directions to query by time or values per our API documentation. Please don’t hesitate to let us know how it works for you!
We have just added a calculator for custom plans based on the estimated log file volume for enterprise customers which need a higher quota than what we offer in the standard plans.
To make a educated guess about the expected log file size volume, Qloudstat peeks into the logging targets of your configured endpoints to predict the estimated quota requirement and suggest a plan