Skip to main content

qryn cache

Intro

qryn-cache is a mechanism to store the response data into a special table to be able to run the same query much faster in the future (particularly useful for commonly run queries/dashboards).

It dramatically improves the performance and reduces query time on large queries run over long periods of time (more than an hour, and more than 24 hours).

How to use​

1. Enable the qryn-cache feature in the 'config' tab of your qryn integration page on the Gigapipe dashboard:​

qryn retention policy

2. Make a request over a period of more than 24 hours:​

qryn retention policy

This will run the query normally and store it in the qryn-cache.

3. Make the same request again over the same amount of time, or a subperiod of that time:​

qryn retention policy

During the second example above, only the last hour was re-calculated, the rest was taken from the qryn-cache.

You can see the dramatic increase in performance (reduced time taken for the total request).

How to use for very large requests​

qryn retention policy

  1. Make a large request (over a long period of time, 2months, 3 months etc) but set the refresh time to 5 minutes (5m), then wait until you get a response

  2. The first attempt on such a large query could potentially timeout but don't worry. Run the query again and it will then be utilizing the cache and should run much faster while leveraging the cache

Limitations

The cache is desigend to help specifically with large requests over long periods of time. Queries which would usually be slow to run. There are some limitations as to when the cache will be used, and when it wont

1. Cache is for requests over large time periods only

  • If the request is for less than an hour, no cache will be used
  • If the request is over a period of between 1-24 hours, the cache will be used in the read-only mode
  • If the request is over a period of over 24 hours, the cache will be used and stored

2. Cache is not for all data types

  • If a request has 'json', 'logfmt' or 'line_format' pipelines, it can't be cached

3. Cache has a 24h expiration time

  • The cache will only hold the results of a large query over a 24h period. If the query is not run again during that time, the cache will be dropped. However if you want to run this query again another time, simply run it once and it will be re-added to the cache

4. Cache can't be manually dropped

  • If you wish to manually drop/stop using the cache. Simply disable the qryn-cache feature, and after 24 hours the entire cache will be dropped
  • However we are currently working on the ability to manually clear the cache at will. This feature will be released soon
Last updated on