Posted On: Oct 26, 2021
Amazon QuickSight announced the availability of Incremental Refresh, a feature in Amazon QuickSight that supports incrementally loading new data to SPICE data sets without needing to refresh the full set of data. SPICE is QuickSight's Super-fast, Parallel, In-memory Calculation Engine.
Previously, QuickSight customers could only have full refresh of SPICE data sets that can take hours to reload all the data even if only a small portion has changed. With incremental refresh, QuickSight customers can update their SPICE data sets in a fraction of the time a full refresh would take, enabling users to access the most recent insights much sooner. Incremental refresh can be scheduled to run up to every 15 minutes on a data set so they can serve up-to-date insights.
Incremental refresh works for SQL data sources connected to databases. QuickSight customers can select a timestamp column from the database that correlates to event, for example, creation time, update time, or published date, and specify a look-back window, for example, 30 days or 3 hours. QuickSight refreshes data changed within that look-back window and can complete the refresh in minutes instead of hours if the data size in the look-back window is much smaller than the entire data set. By allowing to refresh more frequently with small amount of data, readers get access to fresh data for large dataset in minutes instead of hours.
Incremental Refresh is available in Amazon QuickSight Enterprise Editions in all QuickSight regions - US East (N. Virginia), US East (Ohio), US West (Oregon), US West (Canada), US West (Sau Paulo), Europe (Frankfurt), Europe (Ireland), Europe (London), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), and AWS GovCloud (US). If your dataset uses CustomSQL, Incremental Refresh may not be able to benefit you. For further details, visit here.