icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading App for AWS Billing
SHA256 checksum (app-for-aws-billing_2011.zip) e2d3994fc64ab4f7d2c622804613a916c7b06edce97e80b3f80e3c14021c1d26 SHA256 checksum (app-for-aws-billing_2010.zip) 315d41ae4c5d03df98311b68b931306d069a3429772da14e81e9e5fef9863a91 SHA256 checksum (app-for-aws-billing_209.zip) 7b7796594773da93ba961ed8766a752864193300d53585c40707f8f00abb6ade SHA256 checksum (app-for-aws-billing_207.zip) 87580e6ca6eacb829c6fd4e8fa58dedf563a0a354d6b8ec1dbce6f516f00cf40 SHA256 checksum (app-for-aws-billing_206.zip) 96551b9f7e69b6390cd3ece75e9254f60d82d7ec29ca4eb19e9decbd42ff0939 SHA256 checksum (app-for-aws-billing_205.tgz) d5617c500ab37824e7241bc9734b950a592acf31c067eef7edbbf6bdb328d881 SHA256 checksum (app-for-aws-billing_204.tgz) bd80d22865fb4cc5ce5b986599039a50504dcf34fb1303d37a6fcbe4db67035f SHA256 checksum (app-for-aws-billing_203.tgz) 59385a6559d319a11f2c34fc7c56acacdf3201cb6abe249cf501be9d056349bb SHA256 checksum (app-for-aws-billing_200.tgz) 5d15be397ae18899393d277ca82812a9124b923d7567d9674dd5cbfbeb2cc48c SHA256 checksum (app-for-aws-billing_122.tgz) ea1f66f0dc989ce7077c4f7e68b262a155405c99f3c1e6b76cc84ccbe86848c9 SHA256 checksum (app-for-aws-billing_121.tgz) 0204952074faa195ba52cdecc2ffd827142de867c7076d35d6626986a8895c8f SHA256 checksum (app-for-aws-billing_12.tgz) 7c6e000b2968d6e679506a61668ee4987df02d2a86754abe038455fa0c85db00 SHA256 checksum (app-for-aws-billing_11.spl) 60deb0db3caa16e41d96d12b9d8fa15b52b1afa6f2a84cf51619c38c8fc4ccb0 SHA256 checksum (app-for-aws-billing_10.spl) b1b9c305d7cbcadb260ef4a98fd436c61849a2d7c6730cb8a2eaff371b9a267e
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

App for AWS Billing

Overview
Details
Splunk App for AWS Billing allows you to collect Detailed Billing data from which in-depth analysis of usage patterns and spending becomes available for Amazon Web Services environment. It provides a base for you to extend and articulate your own spending and usage patterns.

SplunkAppforAWSBilling v2.0.10

Splunk App for AWS Billing allows you to collect Detailed Billing data from which in-depth analysis of usage patterns and spending becomes available for Amazon Web Services environment.
It provides a base for you to extend and articulate your own spending and usage patterns.

Features

Billing Analysis for:

  • AWS Direct Connect
  • AWS Support (Business)
  • Amazon DynamoDB
  • Amazon Elastic Compute Cloud
  • Amazon RDS Service
  • Amazon Simple Email Service
  • Amazon Simple Notification Service
  • Amazon Simple Queue Service
  • Amazon Virtual Private Cloud

Supports:

  • Custom Tags
  • Billing Projections
  • Month-over-Month Billing Comparision
  • Day/Night Billing Comparison
  • Usage Analysis
  • Simple YAML configuration

Graphs

  • Costs by AWS Product
  • Current Spend
  • Billing items
  • Monthly Raw Costs
  • Costs by Availability Zone

Includes:

  • Boto 2.38.0
  • PyYaml 3.10

Installation Instructions

Install as you normally would by unpacking all the files in the SPLUNK_HOME/etc/apps directory or uploading using the app manager.

You will need to fill in the following pieces of information in the local directory aws.yaml file. (an example is provided)

Splunk APP for AWS billing

Overview

Splunk App for AWS Billing allows you to collect Detailed Billing data from which in-depth analysis of usage patterns and spending becomes available for Amazon Web Services environment.

It provides a base for you to extend and articulate your own spending and usage patterns. It converts your billing line items into JSON which is imported and can be processed in SPlunk.

Any feedback, including requests for enhancement are most welcome. Email: magic.monkee.magic@gmail.com

Splunk App for AWS Billing is hosted at GitHub:

Feel free to submit a pull request:
https://github.com/monk-ee/SplunkAppforAWSBilling

Also feel fre to raise any issues here:
https://github.com/monk-ee/SplunkAppforAWSBilling/issues

Cost Models

The application supports the straight Cost model as well as the Blended/Unblended Cost Model. Dashboards for both are provided; use the one that suits your billing settings.

Keep in mind that blended rates apply only to Consolidated Billing customers.

Blended and Unblended Rates:

The detailed billing report enables customers to perform detailed cost analyses on their usage.
AWS meters usage in hourly increments; for each product resource in use, a rate is applied for operations performed by usage type in that hour, with each operation comprising a
line item. The detailed billing report shows both blended and unblended rates for each line item. An unblended rate is the cost per hour for a product, usage type, and the
operation performed. A blended rate is an average rate calculated for identical instance usage in an Availability Zone for members of a Consolidated Billing family.

A Quiet Word about the detailed Billing Reports eg Why no Cost field?

If you have a single AWS account and you don't use Consolidated billing your report only contains the Rate and Cost Fields.

eg NON-CONSOLIDATED: Your search should look like this:

index=aws-bill | timechart sum(Cost) as $ by ItemDescription

If you have have multiple AWS accounts and use Consolidated billing your report contains BlendedRate, BlendedCost, UnBlendedRate and UnBlendedCost fields BUT not the Cost and Rate fields.

eg CONSOLIDATED: Your search should look like this:

index=aws-bill | timechart sum(BlendedCost) as $ by ItemDescription

This means you need different searches and dashboards for the different billing reports eg a Consolidated OR a Single billing account report.

Setup Amazon detailed billing

v2.0 Indexes are not compatible with earlier versions of the app, you will have to rerun all your log data and flush the older indexes - the data has changed from csv items to JSON.

You will need to ensure that you have set your billing preferences correctly.

If you are not sure go to Billing Preferences:

You will then need to make sure that you have subscribed to "Recieve Billing Reports" and have setup a valid S3 bucket for your bills.

Now make sure you have also ticked the Detailed Billing Report (with resources and tags):

It's also a great idea to setup an IAM user account that only has readonly rights to your file bucket.

Setup and Configuration of the Splunk App

The app is provided as an spl an can be installed using the GUI - feel free to unpack it if you want:

It's file system location is $SPLUNK_HOME/etc/apps/SplunkAppforAWSBilling.

You will need to take the aws.yaml.example and put in your own configuration details then save it as aws.yaml in the $SPLUNK_HOME/etc/apps/SplunkAppforAWSBilling/local directory.

For a single account use the following style of aws.yaml:

accounts:
    - account_number    : 123456
      billing_bucket    : company-billing
      aws_access_key    : AAAAAAAAAAAA
      aws_secret_key    : AAAAAAAAAAAABBBBBBBBBBBBBBCC

For multiple accounts use the following style of aws.yaml:

accounts:
    - account_number    : 123456
      billing_bucket    : company-one-billing
      aws_access_key    : AAAAAAAAAAAA
      aws_secret_key    : AAAAAAAAAAAABBBBBBBBBBBBBBCC
    - account_number    : 654321
      billing_bucket    : company-two-billing
      aws_access_key    : AAAAAAAAAAAA
      aws_secret_key    : AAAAAAAAAAAABBBBBBBBBBBBBBCC

S3 Bucket Policy

Your bucket policy for the programmatic user (the user that accesses the bucket) should look something like this; as a minimum:

{

 "Version": "2012-10-17",

 "Statement": [{

     "Effect": "Allow",

     "Action": ["s3:ListBucket", "s3:GetObject", "s3:GetObjectVersion"],

     "Resource": [

         "arn:aws:s3:::billing-bucket",

         "arn:aws:s3:::billing-bucket/*"

     ]

   }]

}

The best practice for this is to create a new policy and user for this, NEVER EVER EVER EVER use your root credentials for this.

The index

The index is called aws-bill

The views

There are three main views:

  • AWS billing
  • Resources Breakdown
  • Example Tag charts

Customizing your Tag Graphs

When the app imports your billing report it adds all of the custom fields from your report.
They appear like this in your report:

You can then use them in a search or to customize a dashboard like so:

index=aws-bill | timechart sum(BlendedCost) as $ by user:Customer

Logs

The error log file system location is $SPLUNK_HOME/var/log/splunk/SplunkAppforAWSBilling.log.

Contributors

Special thanks to Nilesh Khetia who's module I borrowed to make this one http://answers.splunk.com/users/114849/nkhetia

Release Notes

2.0.5

- Supports multiple accounts
- set driven duplicate protection for line items
- events are streamed in json
- file positions are tracked using pickle binary files, to reduce the likelihood of double ups
- all functions have been moved to classes
- boto has been updated to 2.38.0
- old reports are posted to splunk using splunklib index post
- proxy support is introduced - a must for corporate network fun
- pivot tables on LinkedAccountId

2.0.6

- automated history fetching - last 12 months
- automated report processing - last 12 months
- no date fudging
- checksums of s3 billing reports are checked so only changed files are downloaded
- all original zip files are downloaded locally
- the raw data files are unzipped on the system
- fix for UsageStartDate 'fudge' misfiring Issue #5 https://github.com/monk-ee/SplunkAppforAWSBilling/issues/5
- fix for Double Indexed Items - Remove KVMODE  Issue #6 https://github.com/monk-ee/SplunkAppforAWSBilling/issues/6
- be aware the application needs more space as it gets a years worth of data and unzips it
- maintenance script will clean up files older than a year (but it is still disabled by default)

2.0.7

- fix recurring position bug

2.0.8

- clear duplicates from positional set - this a cracker - sorry about that folks

2.0.9

- changed the import to KV_MODE json - this is more flexible than commiting to an index at import time

2.0.10

- remove deprecated files
- add doco abput upgrading

Examples

Here are some example searches, I hope you find them useful:

index=aws-bill | timechart sum(BlendedCost) as $ by ItemDescription
index=aws-bill BlendedCost !="" | timechart span=1day sum(BlendedCost) as $ by ProductName useother=f

Search for Duplicate Record Id

index=aws-bill | stats count by RecordId | where count > 1

Reports

  • Monthly Blended Raw Costs (Bar)
  • Monthly UnBlended Raw Costs (Bar)
  • Last Month Daytime Blended Costs (Bar)
  • Last Month Nighttime Blended Costs (Bar)
  • Blended Costs by AWS Product (Bar)
  • Current $ Spend (Gauge)
  • Billing Items
  • Usage Quantity

  • Account

  • Daily Cost
  • Tag Report
  • This Month vs. Last Month Detail
  • This Week vs. Last Week by Account
  • Total Invoiced Cost
  • Usage Type.

Billing and Cost Management

Dashboard
- spend summary - include subscriptions
- month-to-date spend by service
- consolidated bill charges by service charges
- bill details by account ****

Cost explorer
- monthly spend by service view
- monthly spend by linked account view
- daily spend view

Pivot
- API Operation
- Availability Zone
- Linked Account
- Purchase Option
- Service
- Tag

Upgrading from older versions (especially pre 2.0.9)

The scripts have changed to go back at least 12 months to fecth older reports, this means the old method for getting
reports and processing has been deprecated. DO NOT under any circumstances run the fetch older or process older reports
scripts. At best they will not work at worst they will cause data duplicates.

Currently on upgrade these files will be in your bin directory, delete them for your own safety and sanity.

Versions moving forward will not have these files.

It would be better if you have the time, to flush all your indexes and install the application
"as if for the first time", these will avoid any of the nasty import duplication bugs that were present in the
2.0.5 - 2.0.8 releases.

If you would like older reports than just a year (it defaults to 12), you can change the history value in aws.yaml file:

history: 12

Warning: Danger Will Robinson!!!

  • v2.0 Indexes are not compatible with earlier versions of the app, you will have to rerun all your log data and flush the older indexes - the data has changed from csv items to JSON.
  • GST Items are thrown away - because of the currency conversion complexity. (I am considering adding these but with exclusion flags on the reports)
  • Any LineItem that does not have a RecordId is thrown away - because it is not charged to your bill

Troubleshooting

In the $SPLUNK_HOME/etc/apps/SplunkAppforAWSBilling/local directory are the *.p files that keep track of processed records. If you find that you have got twisted or need to
reimport a file you may need to clear the aws-bill index or delete these files. They are basically sets of imported RecordIds - to avoid duplicates.

Release Notes

Version 2.0.11
Feb. 5, 2016

2.0.11

- harden up history file fetching
- talk about the fetching of aws reports

Issues fixes

- https://github.com/monk-ee/SplunkAppforAWSBilling/issues/11
- https://github.com/monk-ee/SplunkAppforAWSBilling/issues/12

Version 2.0.10
Jan. 13, 2016

- remove deprecated files
- add doco abput upgrading

Version 2.0.9
Dec. 6, 2015

2.0.9

- changed the import to KV_MODE json - this is more flexible than commiting to an index at import time

Version 2.0.7
Dec. 4, 2015

- fix for recurring position bug

Version 2.0.6
Dec. 4, 2015

2.0.6

- automated history fetching - last 12 months
- automated report processing - last 12 months
- no date fudging
- checksums of s3 billing reports are checked so only changed files are downloaded
- all original zip files are downloaded locally
- the raw data files are unzipped on the system
- fix for UsageStartDate 'fudge' misfiring Issue #5 https://github.com/monk-ee/SplunkAppforAWSBilling/issues/5
- fix for Double Indexed Items - Remove KVMODE Issue #6 https://github.com/monk-ee/SplunkAppforAWSBilling/issues/6
- be aware the application needs more space as it gets a years worth of data and unzips it
- maintenance script will clean up files older than a year (but it is still disabled by default)

Version 2.0.5
May 21, 2015

Highlights include
+ more efficient json objects
+ refactored import classes
+ catch blank date fields
+ multiple account set resets
+ new ui pages

Version 2.0.4
May 7, 2015

- Supports multiple accounts
- set driven duplicate protection for line items
- events are streamed in json
- file positions are tracked using pickle binary files, to reduce the likelihood of double ups
- all functions have been moved to classes
- boto has been updated to 2.38.0
- old reports are posted to splunk using splunklib index post
- proxy support is introduced but not implemented - coming soon!

Version 2.0.3
May 7, 2015

Supports multiple accounts
set driven duplicate protection for line items
events are streamed in json
file positions are tracked using yaml files, to reduce the likelihood of double ups
all functions have been moved to classes
boto has been updated to 2.38.0
proxy support is introduced but not implemented - coming soon!

Version 2.0.0
May 5, 2015

- Supports multiple accounts
- events are streamed in json
- file positions are tracked using yaml files, to reduce the likelihood of double ups
- all functions have been moved to classes
- boto has been updated to 2.38.0
- proxy support is introduced but not implemented - coming soon!

Version 1.2.2
May 20, 2014

## Cost Models
The application supports the straight Cost model as well as the Blended/Unblended Cost Model. Dashboards for both are provided; use the one that suits your billing settings.

Version 1.2.1
May 16, 2014

Minor bucket fix - changed method to get_bucket() from create_bucket()
Removed .pyc files

Version 1.2
May 16, 2014

Version 1.1
Aug. 26, 2013

## Version
1.0 Initial Version
1.1 Bug fixes for boto pathing, error handlers and configuration

Version 1.0
Aug. 19, 2013

Initial Release

45
Installs
2,186
Downloads
Share Subscribe LOGIN TO DOWNLOAD

Subscribe Share

AppInspect Tooling

Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 50GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2019 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.