Added: Added support for the IBM Db2 LUW database. Added a Locality setting and the ability to upload custom seed files for the chosen locality. The Ruleset Generator uses these locality-specific seed files to generate some masking tasks. Added support for discovery of list, struct and map Parquet column types (including nested columns of those types). Added support for assuming another role when connecting to an S3 bucket, allowing for cross-account use of an S3 bucket. This can be configured in the connection settings for any file masking connections that use S3. The retain_age mask now supports the force_change option. Added several new built-in seed files for: US companies, addresses and states. AU companies. Street Names and Types (e.g. Road, Street, Avenue, etc.). Added a UI option to change the ruleset file's extension (.yml or .yaml) when pushing it to Git. When pulling from Git, DataMasque checks for both extensions. A user can now set their own Git directory path which overrides the instance default path. Database connections can now be set to read-only in the UI (previously this was only available via the API). Added In-Data Discovery rules for gender, street address, and US state names. Added an In-Data Discovery option to force use of In-Data Discovery on columns that already have metadata matches. Added an In-Data Discovery option to flag data matching user-specified regex pattern(s) as non-sensitive. Added a navigation UI to the Settings page. Added documentation for configuring SAML Single Sign-On to DataMasque using the Okta platform.

Changed: DataMasque now logs file masking run history to a file .datamasque_run_history.ndjson. This file is created serving as an indicator that masking has taken place, and can be used for run validation. The validation API supports validating file masking runs using the values from the run history file. The run validation API now uses a random run_hash rather than a deterministic one, and the parameter ruleset_hash has been renamed to ruleset_content_sha256. Values for these fields can be found in the run history table or file. Improved run startup performance by not re-validating a ruleset that was marked as valid when saved in the editor. Improved the performance of the Run Logs page. Long run logs will be truncated for display and can be downloaded to see the full content. Tasks that require temporary modifications to the database (secure_shuffle, from_unique_imitate, from_blob) are no longer executed in dry runs. Tasks that get stuck in the Cancelling state for more than 5 minutes will now be forcibly cancelled. Improved the built-in In-Data Discovery rule for MAC address to minimize false matches. Improved In-Data Discovery to include credit card issuer validation in addition to Luhn checksum. Where a column or field has both a metadata match and an In-Data Discovery match, the UI now displays both matches rather than just the latter. Clicking the Add Ruleset button on the File Masking page now opens the File Ruleset Generator. The My Account page no longer displays usage information. To view usage information, download the usage report. Minor wording change to the EULA: "Target System" now means a physical or virtual machine, rather than a physical or virtual node. Keys other than ROWID can now be used as the key column for Oracle databases. Note that DataMasque still recommends use of ROWID as the key wherever possible, and the Ruleset Generator will select ROWID as the key column.

Fixed: Temporary indexes created by DataMasque are now correctly cleaned up at the end of a masking run. Additionally, any dangling temporary indexes from previous masking runs are now cleaned up before creating a new one. Temporary tables and indexes for from_unique_imitate masks are now correctly cleaned up when created on a schema other than the connection's default schema. run_data_discovery tasks no longer return results from views on MySQL and MariaDB. Masking runs that are cancelled before they can start no longer get stuck in the Cancelling state. The Ruleset Generator now generates the correct masks for NUMBER, NUMERIC, and Oracle LONG columns. The File Ruleset Generator now generates include patterns that accurately match the files to be masked by each task. The File Ruleset Generator now discovers files in subdirectories of the connection's base directory. The File Ruleset Generator page now correctly displays errors for malformed custom In-Data Discovery regex patterns. DECIMAL columns now correctly compare against the value in the seed file when using the column as a table_filter_column in a from_file mask. UI web responses now use the correct browser-side caching options. This should avoid any stale data being displayed in the UI. The UI will no longer erroneously display a warning that DataMasque is taking a long time to start. Masking now works correctly when the destination S3 bucket is empty. Fixed upload and delete of connection filesets (Oracle wallets or MySQL/MariaDB SSL ZIP files). An error is now shown trying to upload two connection filesets with the same name and type. DataMasque displays a more descriptive error when the user tries to mask files in Amazon Glacier storage. from_unique_imitate no longer produces duplicate warnings for dangerous parameters. Improved API documentation. In the documentation, clarified the operation of the glob and regex options for include and skip in file masking rulesets. In the documentation, updated the list of supported key column types for all databases.