Meter Corrections (undo events)
There are occasions where a meter event source may send or report incorrect or erroneous meters.
For example, if a bug causes some system to send incorrect meter events to Amberflo or if it is simply a specific event you wish to cancel. See additional examples below.
How do you correct an invoice or bill based on inaccurate usage data?
How do you undo (delete) incorrect meters?
Amberflo provides the ability to cancel (undo) one or more meter events as needed.
A Metering system can only claim to be an accurate system of record for usage and consumption data if and only if it provides built-in artifacts (idempotency, data deduplication) and tooling (undo ingested meters) to guarantee accuracy.
Amberflo provides an easy-to-use API that you can invoke to cancel a single usage event or a large set of events. The system will also take care of propagating the changes to the usage aggregations as well as to the invoicing or rewards payment systems if those are affected by the cancellation.
These sorts of errors are basically data problems or bugs in the account's system that caused the account to send false or inaccurate data to Amberflo. These errors are called system errors as their source is related to the origin system that sent the events to Amberflo.
Let's assume that âSmart-MLâ uses Amberflo for usage-based pricing. âSmart-MLâ sends Amberflo a meter called âapi-callsâ which has the following dimensions: âregionâ, âclusterâ, and âhost-typeâ.
Now letâs assume that on 2/2/2022 at 1:00am there was a âbad deploymentâ, and because of some bug, all of the âapi-callsâ events do not have values for the âclusterâ and âhost-typeâ dimensions.
On 2/15/2022 at 4pm, George, a PM with âSmart-MLâ discovers something wrong with the meters sent to Amberflo, and after short investigation, âSmart-MLâ oncall reaches to the conclusion that all of the âapi-callsâ meters sent to Amberflo since 2/2/2022 are invalid.
Unlike systematic errors which are sourced from bad data or a bug in the source system, unintentional or unsuccessful usage is a more natural occurrence that happens in the course of normal use. In this scenario, a customer starts or intends to start using a resource, but then for some reason the action the customer wanted to take by using the resource is canceled, so ultimately, the resource ends up not being used.
âRabbit-IOâ is a company that allows users to develop and run big-data pipelines on its clusters (some sort of PAAS company). âSmart-MLâ is a customer of âRabbit-IOâ.
On 3/3/2022 at 3:21am âSmart-MLâ sends a request to âRabbit-IOâ to start a certain data-pipeline job. âRabbit-IOâ assigned cluster X machines for this job and even sent Amberflo an event stating the âSmart-MLâ started using machines on cluster X on 3/3/2022 at 3:21am.
But then 15 minutes later, at 3:36am, cluster X goes down and the job fails. âRabbit-IOâ doesnât want to charge âSmart-MLâ for failed resource usage, so they want to cancel that usage event since it never successfully took place.
To help cope with âSystem Errorsâ Amberflo exposes a âfiltering-rulesâ API. This API allows you to define rules for filtering out and removing sets of previously ingested meter events.
If we take the âSmart-MLâ example, then in order to filter out bad âapi-callsâ, all âSmart-MLâ needs to do is:
Letâs assume âSmart-MLâ only had a bad deployment for region âus-west-1â, and so there is no need to filter out all of the âapi-callsâ events that happened between 2/2/2022 1am to 2/15/2022 4pm, but instead, only those those with where the âregionâ dimension is âus-west-1â. To define such a filter just populate the âdimensionValuesMapâ property with the dimensions keys and values which you wish to filter out. For example:
For more information, please refer to our filtering rules API: Create or update a filtering ruleďťż.
đ Notice
A few things to keep in mind regarding the filtering tool:
Time limitations
Events can only be canceled within one year of ingestion.
Impact on Payments
While canceling usage and replacing it with a âcorrectedâ usage is possible and easy, canceling events cannot affect invoices which are already locked (past their grace period time).
Locked invoices should be considered to have already been paid. An account can issue a refund or give a future discount in such cases instead.
ďťż
You can use the filtering mechanism mentioned above to filter out specific events according to their unique-id (just add the unique-id of the event you want to the âdimensionValuesMapâ property mentioned above). This provides an accurate way of canceling events, assuming that you have the unique-id of the event you wish to cancel and that you did not reuse the same unique-id. If you do not have the exact unique-id of the event you wish to cancel, yet at the point where you want to cancel an event you do have the âresource related propertiesâ then we can allow you an alternative approach.
First, letâs define âresource related propertiesâ. These related properties depend on the meter type as described below:
As you can see, the âresource related propertiesâ allows you to identify the resource which was utilized to serve the customer.
Now, to cancel the events for a given resource, just send an ingest event that includes the relevant resource-id related properties and the following dimension: âaflo.cancel_previous_resource_eventâ with âtrueâ as its value. For example, for the âRabbit-IOâ example mentioned above, âRabbit-IOâ needs to send the following ingest event in order to cancel the usage:
đ Limitations
A few things to keep in mind regarding the this way of canceling events:
9-hour time limit
From the moment a resource starts being used, you have 9 hours to cancel the usage event.
Cancelation event timestamp
Also make sure the timestamp you assign to the cancelation event is no later than 9 hours after the original event. In fact, try and set the timestamp of the cancelation event to be as close to the original event as possible, or even use the exact same time of the event you want to cancel.
Why these limitations ?
Notice that when defining a cancelation event we send the âresource related propertiesâ and nothing else that identifies the event. Amberflo will assume that the event we want to cancel is some very recent event (in the scope of the last 9 hours) and will go ahead and cancel the most recent event of the specific resource if such an event exists. So for example if we used the same resource twice in the last 9 hours. Then Amberflo will always cancel the later event of the two. This is also why the value of the cancellation event doesnât matter (because the cancellation event instructions is to just drop the most recent event regardless of its value).
ďťż
The "aflo.cancel_previous_resource_event" dynamic cancellation described above cancels events. These events indicate: start using a resource, stop using a resource, or an update to the usage rate.
For 'Max' or 'Total Duration' meters, Amberflo allows to send an secondary flag "aflo.ignore_cancellation_if_no_usage" (this addition flag has a meaning only if it comes together with "aflo.cancel_previous_resource_event"). This flag, if set to 'true', will tell Amberflo to ignore the dynamic cancelation of "stop events".
To see what's the meaning of this secondary flag let's look at an example: Let's assume that we have a total-duration meter and that Amberflo system ingested the 3 following events:
As mentioned, event cancelation cancels the latest event regardless of what it indicates. So in the example above, after applying the dynamic cancellation, we will be left with only the start event:
Now, if we want the dynamic cancellation to cancel existing usage events (as opposed to cancelling any type of events), we will need to add the "aflo.ignore_cancellation_if_no_usage", with "true" as a value, to the 3rd event. So the sequence of events sent to Amberflo will look like:
In this case the result of the system run will be:
This is because there was no usage to cancel for the 3rd event.
đ NOTICE
"aflo.ignore_cancellation_if_no_usage" has a meaning only in the context of "Max" or "Total Duration" meters.