How to meter and bill compute instance hours
There are a few easy options for metering instance hours for compute resources in Amberflo depending on your reporting and latency requirements and your method of integration with Amberflo for event ingestion.
Option 1:
We can use the ‘sum’ meter type and report usage for each instance when that usage concludes. This method is not events-based and instead depends on regularly reporting usage after it has concluded. There are more advanced options such as to automatically report usage every X hours by checking some usage table or database. For example:
06:00am - c5.metal instance activated on us-east-1 08:00am - same c5.metal instance deactivated
Total usage - 2 hours. Send this 2 hours to Amberflo as a meter event the next time usage is updated in the system.
Option 2:
We can use the ‘duration’ meter type, and report the usage for each instance in realtime. This works by sending a start event to Amberflo when the resource usage commences, and a stop event when the usage concludes. A resource ID dimension is used to associate the start and stop events with each other (ie. sending the instance ID and the customer name with each start and stop event allows the system to associate the two events and automatically calculate the difference between start and stop as the total duration of usage. For example:
06:00am - c5.metal instance activated on us-east-1 (start meter sent to Amberflo containing the customer name, timestamp, and instance identifier)
08:00am - same c5.metal instance deactivated (stop meter sent to Amberflo containing customer name, timestamp, and instance ID)
Amberflo automatically calculates the duration between the start and stop events in milliseconds and stores this value.
In practice, compute resources are typically billed based on several different factors or traits. In Amberflo, these are modeled using dimensions (additional metadata as key-value pairs associated with each meter event), and these dimensions can then be used to filter, analyze, and create multidimensional pricing plans.
The pricing generally depends on the number and type of hardware being accessed (instance type), as well as the region the hardware is hosted, the amount of available memory, the availability zone, and other factors.
In Amberflo, when selecting the rate model you can configure 'Per Unit with Dimensions' or 'Tiered with Dimensions' to create these complex, multidimensional pricing arrays. In the screenshot below you can see a basic example that uses two dimensions, the instance type (type of compute resource being used) and the AWS region that hardware is being accessed from.
In practice, when customers consume compute for a job, they often provision multiple different instances as a cluster. Consider the example:
A user may be using 3 server instances (it can be any other number, usually a multiple of 3) of type i3.large in region US East. So in this case, the user should be charged $0.123 3 per hour instead of just $0.123 per hour (which is the current per-unit price in Amberflo). *
There are two options for handling this use case, depending on whether each instance will be metered individually or if you will be squashing events before ingestion.
Option 1:
Group the meters by cluster using a 'cluster_id' dimension. When 3 instances (or some multiple) are activated as a cluster, meter each individual instance (using 'instance_id' dimension), and associate them together using the 'cluster_id' dimension. Then when configuring the pricing plan, be sure that the checkbox is selected on 'For Invoice - Group By', and select 'cluster_id' as the grouping dimension.
See an example meter event data structure below:
On the invoice, the usage will be organized by clusters, with the usage-based charge calculated for each based on the number of instances and the amount of time they ran.
Option 2:
Squash the meters pre-ingestion by multiplying the meter value by the number of instances (sending one meter per cluster). In this case, the cluster size would be a known quantity before the meter was ingested to Amberflo. Using this metadata, you can simply multiply the meter value for a single instance by the cluster size to create a cluster-level meter that shows all usage for that cluster as a single event.
Consider the following example of a meter event and the pre-ingestion multiplication. Here is the original event - notice that the 'meterValue' is 1.0.
Using the example above, the cluster size is 3, so the meter value would be multiplied by 3 to give the cluster-level meter. See below:
This approach involves sending fewer total meter events and may be simpler to orchestrate and maintain, however the trade-off is a slight loss of granularity since the instance-level events are being squashed pre-ingestion to create cluster-level events.
See related from Pricing Plans and Product Items:
