Hi there Gsrfan Apologies for the delayed response, I had to shelf this passion project to focus on other work So far managed to get the host, static IP, packages, users & their keys/password etc sorted with the init I was working last on lvm partitions, to grow them and that ended up a bit of a mess I might return to this at a later stage but I found your help and advice invaluable to getting the config to where it is now Thank you so much

@olivierlambert said in file restore on large backups ends in print_req_error: I/O error's: Yes, that sounds the next step to do Something is wrong during the mount phase. Also check you have enough RAM in your XOA. Ok, filesystem checks out - as far as I can see having: Done full restore of VM from the same Delta image. Booted VM, logged in and done 'tar cvf /dev/null *' of the filesystem I'm trying to restore from - which ran to completion without error. I upped the RAM for XOA from default 2GiB to 16GiB and re-tried - and same result. I can try yet more RAM if needs be but would hope 16GiB is more than enough? The console error on XOA is more detailed than my install- so don't know if this helps: blk_update_request: I/O error, dev loop0, sector 8912912 op 0x00:(READ) flags 0x80700 phys_seg 1 prio class 0 Looks to be the same issue, just reported differently. You also (on both systems) get kernel chatter about "tasked blocked" (understandable, as it is).

@olivierlambert said in Can't restore VM's from 'read-only' remote?: A remote (now Backup repository) is meant to write backups into it. If you aren't able to write, then, sure, you can restore but you won't be able to backup anything. Yes, understand that - I'll have to re-test here, but I'm sure when it was 'read-only' - I couldn't restore anything from it, as nothing showed up in XO's "Backup / Restore' list of backups... The remote was mounted on two systems - hence why made R/O for one of them (this was just for testing restores).

@Zevgeny Did you succeed to remove the protection with "xe vm-param-clear"? It didn't worked perfectly for me. Before I use xe vm-param-clear, I have the following: [18:28 xcpng02 ~]# xe vm-param-list uuid=37b7a201-eb45-b7de-5ce4-da25b1c7b547 | grep block blocked-operations (MRW): pause: true; hard_shutdown: true; (unknown operation): true; hard_reboot: true; clean_shutdown: true; suspend: true; clean_reboot: true [18:28 xcpng02 ~]# Now I try to clean: [18:28 xcpng02 ~]# xe vm-param-clear uuid=37b7a201-eb45-b7de-5ce4-da25b1c7b547 param-name=blocked-operations The value given is invalid field: blocked_operation value: (unknown operation) [18:28 xcpng02 ~]# After, "blocked-operations" looks like this: [18:28 xcpng02 ~]# xe vm-param-list uuid=37b7a201-eb45-b7de-5ce4-da25b1c7b547 | grep block blocked-operations (MRW): (unknown operation): true; hard_reboot: true; clean_shutdown: true; suspend: true; clean_reboot: true [18:30 xcpng02 ~]# In the GUI of XOA, the "Protect from accidental shutdown" is removed but not on the command line.

I think this is important. When the issue appears the replicated VM that auto-starts has the [IMPORTING] prefix as in the image below. [image: 1689001402269-130e1c45-ad28-4b6c-a4d7-202b552ab723-image.png]

Question for you @florent

@pdonias Hi, On Master, as of yesterday, I see this only when I am logged in using an IP address on the URL instead of the hostname. Either way it is SSL with trusted certificate. There are no console errors Browser is Microsoft Edge on Windows No difference in behaviour between incognito mode and normal. Yes I see all the VMs There is an improvement - now if I am creating a new VM only the selected network is reverting back to default when it refreshes, the other things like number of cores and ram is remaining when I set it, whereas previously these also were wiped back to default every 20 seconds or so. Thanks Alex

@rauly94 Hello everyone. Anyone can help me on this issue. Now it started happening on 2 vm's instead of 1 vm. It is happening on the backup replication. Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error Start: Jul 5, 2023, 09:03:18 AM End: Jul 5, 2023, 09:44:46 AM Duration: 41 minutes Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error Start: Jul 5, 2023, 09:03:09 AM End: Jul 5, 2023, 09:45:12 AM Duration: 42 minutes Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error Type: delta

Yeah for doing a fresh start it's easiest to just nuke the backup directory you were using and start over, have done it before and it's quick and easy vs trying to manually remove them all within XOA.

Not that I am aware of.

@ronivay, I resolved my problem with orchestra which did not want to make backup with NFS. Orchestra wrote errors like: Command failed with exit code 1: mount -o -t nfs 192.168.11.53:/backups/xo /run/xo-server/mounts/0e5e9511-8912-48d4-96ad-532979533332 mount: only root can use "--options" option In general, orchestra tried to use mount option "--options" but mount command rejected this option. When I mount my NFS-shares manually everything was ok - mount works, read-write too, umount also. I did simplest process. I removed at all "--options" from mount command. (As far as I understand, *BSD mount will not work without this option but my Ubuntu works). Background: xcp-ng server (centos 8.2) as NFS server ubuntu server 16.04 where orchestra is going to be placed Orchestra installed 10 days ago from sources according to https://xen-orchestra.com/docs/installation.html#from-the-sources I complete followed document before chapter "Installing dependencies". I made yarn Then, before "yarn build", I manually edited file: /opt/xen-orchestra/@xen-orchestra/fs/src/_mount.js at string 68 I replaced await this.#execa('mount', ['-o', options, '-t', type, device, realPath], { with await this.#execa('mount', ['-t', type, device, realPath], { I then proceeded to follow the instructions: yarn build a so on... Now "patched" orchestra fine makes any kind of backup with NFS. At least, I solved this problem. Yet another question appeared as backup works What kind of Backup should I use to make full copy which enough for restoring (Heaven forbid)? I tried simply "Backup". It SUCCESSFULLY (!) created several files with 56 kbytes (no giga, no mega but kilo). I suppose it abnormal. My VM has got 20 Gb of disk space and about one half is already used (ubuntu server, postgres etc). As far as I understand (https://xcp-ng.org/forum/topic/7034/backup-with-local-usb-hdd-attached-sr_operation_not_supported/5?_=1688120707451) my version of xcp-ng (8.2.1) does not support "Snapshots" with local disks. Now I copy imagine of my VM with cron as temporary option but may be there is some more correct backup option?

@sumansaha Hello Team, The issue has resolved after applying xe-toolstack-restart. Thanks.

@sumansaha that is a good news yes, the replication go through the XO VM, the faster the link the better the performances

@AtaxyaNetwork Thanks

@olivierlambert Thanks

@julien-f @julien-f All I have done is let ronivay's script do the builds daily on schedule if there is a new commit on the master branch. His script has not changed. There have been no new patches to the hosts. I don't have an explanation.

XO saved the day

@Danp Yes but unfortunately before of this incident audit is disabled

No problem! Did you manage to do what you need?