Not a Issue but I wasn't able to create a PR ....

[6-ril]

Hi,
I add this at the beginning of archivist.sh :
echo() {
command echo $(date +%T) ": $@"
}
So the log show the time before each step and I can see how slow is my server when doing backup !!

If you mind, please add it to your script.

Cheers,
Cyril

[6-ril]

Haaaa, I didn't check properly .. "echo" is used to populate files too.
I suppose "printf" may be used instead of echo in this case ....
For logging or for file ... not sure what is better.

[maniackcrudelis]

Hi

I'm not sure to understand what you're really trying to achieve.
The current script is printing the date at the beginning and the end of the execution.

If you want to have a date between each backup, maybe you can try to use the pre backup commands.

In any case, a simple date, or date +%T is enough.

[6-ril]

I have added the time in front of each step of the backup.
I was wondering how many backup I can start in a day. Log give me the time at the beginning of the backup and at the end, so I already know that my backup takes ages.
I was looking for what was taking so much time. Now I know, zipping nextcloud backup is taking lot of time.

My log looks this now:

`jeudi 7 octobre 2021, 10:20:03 (UTC+0200)
10:20:03 : Error: Compression format not recognized for ynh_compression_mode !
10:20:03 : Fall back to gzip.
10:20:03 : > WARNING: Compression format has been modified for YunoHost backups. All backups will be rebuilt
10:20:03 : Error: Compression format not recognized for files_compression_mode !
10:20:03 : Fall back to gzip.
10:20:03 : > WARNING: Compression format has been modified for files and directories. All backups will be rebuilt
10:20:03 : > Build list of files to backup
10:20:03 : > Compress backups

10:20:03 : > Backup YunoHost core and apps

10:20:03 : > Backup hooks used: conf_ldap10:20:03 : >> Make a temporary backup for ynh_core_backup

10:25:51 : >>> This backup is different than the previous one

...

10:52:33 : >> Make a temporary backup for nextcloud_backup

12:28:56 : >>> This backup is different than the previous one

12:28:56 : >>>> Make a real backup for nextcloud_backup

73G /home/yunohost.app/archivist/backup/ynh_backup/nextcloud_backup.tar.gz
15:55:14 : >> Make a temporary backup for opensondage_backup

15:55:21 : >>> This backup is different than the previous one

15:55:21 : >>>> Make a real backup for opensondage_backup

5,3M /home/yunohost.app/archivist/backup/ynh_backup/opensondage_backup.tar.gz
15:55:26 : >> Make a temporary backup for roundcube_backup

15:55:47 : >>> This backup is different than the previous one

15:55:47 : >>>> Make a real backup for roundcube_backup

33M /home/yunohost.app/archivist/backup/ynh_backup/roundcube_backup.tar.gz
15:55:59 : >> Make a temporary backup for shaarli_backup

15:56:09 : >>> This backup is different than the previous one

15:56:09 : >>>> Make a real backup for shaarli_backup

18M /home/yunohost.app/archivist/backup/ynh_backup/shaarli_backup.tar.gz
15:56:15 : >> Make a temporary backup for shellinabox_backup

15:56:17 : >>> This backup is different than the previous one

15:56:17 : >>>> Make a real backup for shellinabox_backup

12K /home/yunohost.app/archivist/backup/ynh_backup/shellinabox_backup.tar.gz
15:56:20 : >> Make a temporary backup for slingcode_backup

15:56:23 : >>> This backup is different than the previous one

15:56:23 : >>>> Make a real backup for slingcode_backup

768K /home/yunohost.app/archivist/backup/ynh_backup/slingcode_backup.tar.gz
15:56:26 : >> Make a temporary backup for unattended_upgrades_backup

15:56:29 : >>> This backup is different than the previous one

15:56:29 : >>>> Make a real backup for unattended_upgrades_backup

24K /home/yunohost.app/archivist/backup/ynh_backup/unattended_upgrades_backup.tar.gz
15:56:33 : > Clean old backup files

15:56:33 : Remove old archive /ynh_backup/searx_backup.tar.gz

15:56:33 : Remove old archive /ynh_backup/zabbix_backup.tar.gz

-> Build the list of files for the recipient local
...
`

[maniackcrudelis]

Oh ok, so you would like to have a timestamp before each line printed by the script ?
I can do that properly indeed.

[maniackcrudelis]

Please do try with the script archivist.sh from the branch timestamp and tell me if it works for you ;)

[6-ril]

I have to do few more test as the script archivist.sh is running fine, and I have timestamp.
But the sender script that use rsync stop while building the list to copy.
I'll let you know.

[6-ril]

With the original archivist.sh script, the sender works fine.
Is there anything to change in the sender file configuration ?

Thanks !

[maniackcrudelis]

I didn't change anything about the sender, so it shouldn't be a problem.
Do you have any log or error that can help understand what's happening ?

[6-ril]

The error come probably from rsync as it stops at this moment.
The last message in archivist.log is "building file list ...". it's a step from rsync, if I'm not wrong.

sudo rsync --archive --copy-links --verbose --human-readable --stats --itemize-changes \ --delete-excluded --prune-empty-dirs --exclude-from="$script_dir/../exclude_list" \ "$backup_source/" "$dest_directory"

Is it possible that some list has been modified by the archiviste.sh script ?

[maniackcrudelis]

Hi @6-ril, sorry for the delay.

I did try on my own with my instance of Archivist.
It worked without any trouble on my side.

All my files have been transferred to the distant server.

But it was the encrypted version of the files that the scrip sent. Is it the same for you ?

[6-ril]

Hi @maniackcrudelis, no pb there's no hurry.

I don't use the encrypted version of the files.
I will take the time next week to "tail -f" all the files used by rsync and see what happens to them during the backup.
Unless you have another idea of what I can do to find the problem?