...
Practical tests have shown that data compression in the depository is faster if the data is previously reorganized. Therefore, there is a TrezorCompressReorg parameter that is a TrezorCompressReorg parameter is preset to 1 and that activates data reorganization before compression.
...
Compressed depositories on PostgreSQL are subject to the following limitation: the data size of one archive object (or one structured archive item) that is stored in one row of a compressed data table CDATA must be less than 1 GB. Otherwise, the compression ends with an error , and the ID of the object that caused it is listed. Example:
...
TrezorCompressReorg - reorganization of the depository data before compression. This parameter has a default value of 1 - in practice, it turns out that it is more efficient and faster to reorganize the data first (by an SQL command CLUSTER DATA) and then compress them. A value of 0 disables the reorganization of the depository data before compression. Disabling reorganization before compression is not recommended.Kotva trezorcompressreorg trezorcompressreorg
The parameter can be changed online, by the SET_OPTION TrezorCompressReorg command.
TrezorCompressCmt -the parameter specifies the number of archive objects after the compression of which a COMMIT will be executed. The default value is 10. In applications with intensively archived objects (many values of one archive object in the depository ), this parameter can be reduced down to 1. Conversely, in applications with many archive objects that change infrequently, it is possible to increase the value of this parameter.Kotva trezorcompresscmt trezorcompresscmt
The parameter can be changed online, by the SET_OPTION TrezorCompressCmt command.
TrezorCompressOrder - the parameter refers to reading from depositories with compressed data. This parameter indicates whether time sorting is required when reading data from a compressed depository. Since the data is sorted during compression, it is possible to leave the parameter at 0 by default, which speeds up reading from the compressed vault (ORDER BY clause is omitted)Kotva trezorcompressorder trezorcompressorder
The parameter can be changed online, by the SET_OPTION TrezorCompressOrder command.
...
- If the old depositories have already been cleaned (e.g. the backup script contained the "cluster data" command), it is possible to turn off the cleaning with the following command before compression:
SET_OPTION TREZOR_COMPRESS_REORG OFF - The depositories to be compressed must be mounted for writing. Therefore, they must first be disconnected and then reconnected. For example for depositories 1-10:
DISMOUNT_TREZOR 1 10
MOUNT_TREZOR 1 10 WRITE - It is now possible to compress the depositories with the command
TREZOR COMPRESS <trezor_id>
e.g. TREZOR_COMPRESS 1
The depository with a size of about 20 GB is compressed on a standard server in about an hour or two. If it's much more, then the data is probably "cleaned" and cleaning needs to be turned on before compression (SET_OPTION TREZOR_COMPRESS_REORG ON).
By default, the archive has the debug category DBG.ARCHIV.COMPRESS.TREZOR enabled, and debug logs are visible in the D2000 Sysconsole. They talk about the compression of concrete objects (or items of structured archives), in parentheses is the serial number / total number of objects). Once every 10 objects, a commit to the database follows (this can be parameterized with the command SET_OPTION TREZOR_COMPRESS_CMT <number>).
Compress trezor 81 ID $11291( 1048/ 39113)
Compress trezor 81 ID $11291( 1048/ 39113) done
Compress trezor 81 ID $11293( 1049/ 39113)
Compress trezor 81 ID $11293( 1049/ 39113) done
Compress trezor 81 ID $11295( 1050/ 39113)
Compress trezor 81 ID $11295( 1050/ 39113) done (commit)
Compress trezor 81 ID $11297( 1051/ 39113)
Compress trezor 81 ID $11297( 1051/ 39113) done
Compress trezor 81 ID $11299( 1052/ 39113)
Compress trezor 81 ID $11299( 1052/ 39113) done - After compression, it is advisable to disconnect the depositories and connect them for reading. For example for depositories 1-10:
DISMOUNT_TREZOR 1 10
MOUNT_TREZOR 1 10 - With the command LIST_TREZORS it TREZOR, it is possible to check that all required depositories are compressed. The compressed depositories has have a "cps" flag. For example, depository 140:
0140 31-07-2019 22:00 31-08-2019 22:00 mounted read-only cps - If compression before cleaning was turned off, we recommend turning it on with the command:
SET_OPTION TREZOR_COMPRESS_REORG OFF - Compression of newly created depositories is turned on by command:
SET_OPTION TREZOR_COMPRESS ON
...