Fingerprint Management
Intro
Fingerprint usage grows when many different combinations of labels are used by logs and metrics.
This is called cardinality. Let's use some example inserts and analyze their cardinality:
Example 1
cpu_seconds{host="host1",cpu="0",type="user"} 1
cpu_seconds{host="host1",cpu="0",type="user"} 2
cpu_seconds{host="host1",cpu="0",type="user"} 3
cpu_seconds{host="host1",cpu="0",type="user"} 2
The label value is constant for all time series of logs.
This insert would use one fingerprint.
Example 2
cpu_seconds{host="host1",cpu="0",type="user"} 1
cpu_seconds{host="host1",cpu="1",type="user"} 1
cpu_seconds{host="host1",cpu="0",type="user"} 2
cpu_seconds{host="host1",cpu="1",type="user"} 1
The label value of cpu is different for each time series even though host and mode are the same.
This insert would use two fingerprints.
Example 3
Labels using 6 different types, 10 different hosts and 4 CPUs they would use 240 fingerprints (6 10 4).
Example 4
Logs and Metrics share fingerprints Using the same labels for both causes no additional load in the fingerprint calculation.
Fingerprint management
Gigapipe allows you stay on top of your fingerprint generation (as this a metric used for billing, it's important to not allow your fingerprints to spiral out of control).
Fingerprint limiter
In the Config section of the qryn cloud integration, you're able to set either a numberical or fiscal limit on how many fingerprints will be ingested into qryn for that month. Generally this feature is used for DEV environments while experimenting with label numbers to avoid too high a cardinality (resulting in high fingerprints).
Fingerprints are charged at $5 per 1000, so you can either set the number you want, or the amount you'd like to set as your max spend, and Gigapipe will calculate the other value.
Once the fingerprint limit is reached, qryn cloud will stop ingesting NEW fingerprints. This means if you ingest logs and metrics with fingerprints that match existing sets, they will still be ingested as they're not new unique fingerprints.
New fingerprint management features
We will soon be releasing a fingerprint management suite which will provide you with which labels are creating the highest number of fingerprints and then allow you to delete unwanted existing fingerprints. You will then be able to re-fill up the fingerprint quota without additional charges!