Our Barman install doesn't seem to be compressing the WAL files. The wal file sizes are the same as they are on the Postgresql server. We've got streaming_archiver turned on, and in barman show-server compression is set to pigz, but wal_compression shows as off. Barman version is 2.12.
Thanks in advance.
barman show-server v2db:
Server v2db:
active: True
archive_timeout: 0
archiver: False
archiver_batch_size: 0
backup_directory: /var/lib/barman/v2db
backup_method: rsync
backup_options: BackupOptions({'concurrent_backup'})
bandwidth_limit: None
barman_home: /var/lib/barman
barman_lock_directory: /var/lib/barman
basebackup_retry_sleep: 30
basebackup_retry_times: 3
basebackups_directory: /var/lib/barman/v2db/base
check_timeout: 30
checkpoint_timeout: 300
compression: pigz
config_file: /etc/postgresql/12/main/postgresql.conf
connection_error: None
conninfo: host=**********
create_slot: manual
current_lsn: 50/35C7FF00
current_size: 49459828350
current_xlog: 000000010000005000000035
custom_compression_filter: None
custom_decompression_filter: None
data_checksums: off
data_directory: /var/lib/postgresql/12/main
description: Backups from DB
disabled: False
errors_directory: /var/lib/barman/v2db/errors
has_backup_privileges: True
hba_file: /etc/postgresql/12/main/pg_hba.conf
hot_standby: on
ident_file: /etc/postgresql/12/main/pg_ident.conf
immediate_checkpoint: False
included_files: ['/etc/postgresql/12/main/conf.d/9999-override.conf']
incoming_wals_directory: /var/lib/barman/v2db/incoming
is_in_recovery: False
is_superuser: True
last_backup_maximum_age: 7 days (latest backup: 2 hours, 52 minutes, 55 seconds )
max_incoming_wals_queue: None
max_replication_slots: 10
max_wal_senders: 10
minimum_redundancy: 3
msg_list: []
name: v2db
network_compression: False
parallel_jobs: 1
passive_node: False
path_prefix: /usr/lib/postgresql/12/bin
pg_receivexlog_compatible: True
pg_receivexlog_installed: True
pg_receivexlog_path: /usr/lib/postgresql/12/bin/pg_receivewal
pg_receivexlog_supports_slots: True
pg_receivexlog_synchronous: False
pg_receivexlog_version: 12.6
pgespresso_installed: False
post_archive_retry_script: None
post_archive_script: None
post_backup_retry_script: None
post_backup_script: None
post_delete_retry_script: None
post_delete_script: None
post_recovery_retry_script: None
post_recovery_script: None
post_wal_delete_retry_script: None
post_wal_delete_script: None
postgres_systemid: 6902948791374441963
pre_archive_retry_script: None
pre_archive_script: None
pre_backup_retry_script: None
pre_backup_script: None
pre_delete_retry_script: None
pre_delete_script: None
pre_recovery_retry_script: None
pre_recovery_script: None
pre_wal_delete_retry_script: None
pre_wal_delete_script: None
primary_ssh_command: None
recovery_options: RecoveryOptions({'get-wal'})
replication_slot: Record(slot_name='backup_pgsql_masterbarman', active=True, restart_lsn='50/35000000')
replication_slot_support: True
retention_policy: RECOVERY WINDOW OF 4 WEEKS
retention_policy_mode: auto
reuse_backup: link
server_txt_version: 12.12
slot_name: backup_pgsql_masterbarman
ssh_command: ssh -q postgres@******
streaming: True
streaming_archiver: True
streaming_archiver_batch_size: 0
streaming_archiver_name: barman_receive_wal
streaming_backup_name: barman_streaming_backup
streaming_conninfo: host=********
streaming_supported: True
streaming_systemid: 6902948791374441963
streaming_wals_directory: /var/lib/barman/v2db/streaming
synchronous_standby_names: ['']
tablespace_bandwidth_limit: None
timeline: 1
wal_compression: off
wal_keep_segments: 0
wal_level: replica
wal_retention_policy: MAIN
wals_directory: /var/lib/barman/v2db/wals
xlog_segment_size: 16777216
xlogpos: 50/35C7FF00
barman.conf:
[barman]
barman_home = /var/lib/barman
barman_user = barman
log_file = /var/log/barman/barman.log
; rsync or postgres
backup_method = rsync
parallel_jobs = 1
; Incremental backup support: possible values are off (default), link or copy
reuse_backup = link
; Identify the standard behavior for backup operations: possible values are
; exclusive_backup (default), concurrent_backup
backup_options = concurrent_backup
; Immediate checkpoint for backup command - default false
immediate_checkpoint = False
; Global bandwidth limit in KBPS - default 0 (meaning no limit)
;bandwidth_limit = 4000
; Default compression level: possible values are None (default), bzip2, gzip, pigz, pygzip or pybzip2
compression = pigz
; Enable network compression for data transfers - default false
network_compression = False
; Number of retries of data copy during base backup after an error - default 0
basebackup_retry_times = 3
; Number of seconds of wait after a failed copy, before retrying - default 30
basebackup_retry_sleep = 30
; Minimum number of required backups (redundancy) - default 0
minimum_redundancy = 3
; Global retention policy (REDUNDANCY or RECOVERY WINDOW) - default empty
retention_policy = RECOVERY WINDOW OF 4 WEEKS
; Recovery options (influences recovery.conf written by barman recover)
recovery_options = 'get-wal'
; Time frame that must contain the latest backup date.
; If the latest backup is older than the time frame, barman check
; command will report an error to the user.
; If empty, the latest backup is always considered valid.
; Syntax for this option is: "i (DAYS | WEEKS | MONTHS)" where i is an
; integer > 0 which identifies the number of days | weeks | months of
; validity of the latest backup for this check. Also known as 'smelly backup'.
last_backup_maximum_age = 1 WEEK
configuration_files_directory = /etc/barman.d
path_prefix = /usr/lib/postgresql/12/bin
server.conf:
[v2db]
description = "Backups from DB"
ssh_command = "ssh -q postgres@******"
conninfo = "********** "
streaming_conninfo = "************** "
streaming_archiver = on
slot_name = "backup_pgsql_masterbarman"
active = true
Parameter wal_compression is fetched from your PostgreSQL server configuration. It means you have configured wal_compression = off on your database server. Check in psql with:
show wal_compression;
You can check more info about this parameter on: https://postgresqlco.nf/doc/en/param/wal_compression/
Anyway, your WAL compression on barman side should work as It seems you have configured compression correctly. Make sure you have installed required OS library for pigz on barman server.
P.S. You will not see compressed WAL size during streaming operation. WAL log will be compressed after complete stream operation. But you can check already compressed WAL logs in wal directory.
Command to check your configured wals_directory:
barman show-server <server_name_here> | grep "wals_directory"