I recently implemented Kopia to replace Duplicati on some systems in my network. Now that I have more than a week of backups, I have gained appreciation for the retention tags they use in the KopiaUI program.

It does a good job of indicating why this version is being retained while also showing how many copies it is maintaining at each retention level. Really handy.

I just noticed the pin to the right. That lets you pin the snapshot so it won't be removed during cleanup. That's kinda cool to be able to single one out to hang on a bit longer, maybe while investigating something I suspect went wrong after that snapshot.

I'm not shilling for Kopia, I haven't used it enough to be sure I trust it yet. I am hoping XO could blatantly steal this kind of idea for backups, especially delta backups. I often get confused about how to get the retention I want when configuring backups and when I search for answers I see other people are also confused.

I guess for the sake of completeness, this is how KopiaUI lets me configure retention:

Amazing timing. I have been having this issue with a VM I just set up and I came here to find out if this is a known problem, only to discover that the 97 patches I have been planning to install tonight are going to fix it.

Thank you @florent. I'll do that.

Update: It worked.

@Forza Sorry, you were correct, I just mixed in another new issue. NFS is currently used only for backups. All my SRs are in local storage. It just happened that I now have backups failing not just because of the NFS issue but because of the VDI issue but I think it's a side-effect of the NFS problem causing the backup to get interrupted so now the VDI is stuck attached to dom0. I should have made that more clear or never mentioned the VDI issue at all.

@Forza Seems you are correct about showmount. On UNRAID running v4 showmount says there are no clients connected. I previously assumed that meant XO only connected during the backup. When I look at /proc/fs/nfsd/clients I see the connections.

On Synology, running v3, showmount does show the XO IP connected. Synology is supposed to support v4 and I have the share set to allow v4 but XO had trouble connecting that way. Synology is pretty limited in what options it lets me set for NFS shares.

Synology doesn't have rcpdebug available. I'll see if I can figure out how to get more logging info about NFS.

@olivierlambert Well, last night the backup completed just fine despite me taking no action.

I updated the XO to the latest commit when I got in this morning so hopefully the issue I had back in June don't come back.

@julien-f Thank you, that's super helpful and even easier than I thought it would be.

@florent Yeah, it's a lot of data, thankfully my other VMs are not nearly as large. I'm still not sure why it failed when none of the virtual drives are 2TB. The largest ones are configured with a 1.82TB max so even the capacity of the drive is less than the max.

I'm moving ahead with a file level sync attempt to see if that works.

To be clear, this post was as much or more about helping you figure out what's wrong so other people don't have the same issue, than it is about making this import work for my VM. With the flood you are getting from VMware refugees, I figure I'm not the only person with large drives to import. In other words, if there's something I can do to help you figure out why it fails then I'm willing to help.

@gskger Yeah, looks like it would be too tight. Ouch, those T4s are an order of magnitude more expensive. I'm definitely not interested in going that route.

@paco Oh, that's great information, thank you. I'd much rather reset that flag instead of constantly adjusting the fans.

I got my R730 working, thanks to @majorp93 giving the tip about dmidecode. Thanks a bunch. It turns out that while this is a Dell R730, it was actually sold by a different vendor for use as video storage so the internal product name doesn't have Dell in it, it's HD NVR. I used that as the vendor field and tweaked a couple other plugin settings and now it gives me data.

I am still missing two of the values in the UI and I'm not sure why, but for different reasons.

On my R630 servers they show the server IP address under the IPMI badge. They also both have the default Dell /^ip address$/i as the regex in the ip setting in the plugin. I used that same regex in the R730 and I can see in the ipmitool listing that it has the same field name for the IP address as the R630, yet it shows IPMI: Unknown.

Also confusing is that the R630 servers have nothing under fanStatus in the plugin settings but they show 14x fans status: ✅ in the UI. The R730 also has nothing under the plugin settings fanStatus field but it has no data in the UI. Looking at the ipmitool output I am not sure what field it would use for fan status. It makes me think there must be a default field name it is trying with the R630 to get that status but that doesn't work with the R730.

Any hints about how to get those last two badges to work on the R730?

I am also not sure what bmcStatus should be set to. I don't know what that means exactly so I don't know if any of the fields shown in ipmitool are applicable.

This plugin is really cool, thank you so much for building it.

@TheNorthernLight Thanks for pointing that out, that totally makes sense, I should have realized to look it up that way. I guess I just think of XCP-ng as its own product so much that I forget it's based on Xen even though I know that.

If anyone can point me to documentation on this, I would greatly appreciate it. I have looked in the docs and searched for an answer and come up empty.

I have two PE R630 and one PE R730xd. The two R630s show the IPMI data in XO but the R730xd does not. If I run ipmitool sensor from an ssh session, I get very similar data back on the 730 and 630.

This isn't critical to me, just letting you know that there are cases that functioning ipmitool on a Dell does not seem to be enough to show it.

There are some differences in the labels. The R730 has "Fan1 RPM", "Fan2 RPM" while the R630 has "Fan1A", "Fan2A", "Fan1B". Most of the other interesting fields seem to have the same names but the order is a bit different and I didn't do an exhaustive comparison.

It's very cool that you are surfacing this data now.

I was rebooting my servers due to the patches released last week and I notice that the BIOS in my server was not set to interleave the memory. Since I have symmetric memory sticks, it seems like it should be set to interleave. However, the BIOS does say that for NUMA aware OSs, that interleaving should be left to the OS. I don't recall now, and my notes don't say, if I intentionally left interleaving off because v8.2 I was installing at the time was NUMA aware.

Memory interleaving on a Dell R630: on or off?

During the last patch I turned on interleaving for my R730xd (without reading the help that says not to do it with NUMA aware OSs) and it feels faster to me, but I didn't do any real testing so that could be wishful thinking bias. It certainly doesn't seem to have hurt performance.

@nikade I did install the 97 XCP patches and that night the VM that was failing to backup was able to successfully backup. It had a 500GB secondary drive that was formatted but totally empty.

I had not yet patched my XO from source to the latest, it was maybe 24 commits behind. On Friday I updated it so I will find out on Monday if those backups continued to work but I assume they did.

Amazing timing. I have been having this issue with a VM I just set up and I came here to find out if this is a known problem, only to discover that the 97 patches I have been planning to install tonight are going to fix it.

@paco Oh, that's great information, thank you. I'd much rather reset that flag instead of constantly adjusting the fans.

I recently implemented Kopia to replace Duplicati on some systems in my network. Now that I have more than a week of backups, I have gained appreciation for the retention tags they use in the KopiaUI program.

It does a good job of indicating why this version is being retained while also showing how many copies it is maintaining at each retention level. Really handy.

I just noticed the pin to the right. That lets you pin the snapshot so it won't be removed during cleanup. That's kinda cool to be able to single one out to hang on a bit longer, maybe while investigating something I suspect went wrong after that snapshot.

I'm not shilling for Kopia, I haven't used it enough to be sure I trust it yet. I am hoping XO could blatantly steal this kind of idea for backups, especially delta backups. I often get confused about how to get the retention I want when configuring backups and when I search for answers I see other people are also confused.

I guess for the sake of completeness, this is how KopiaUI lets me configure retention:

@JamfoFL Thank you for the follow-up. Now I feel like I can install those patches. Bonus that it appears there are 23 patches now rather than the 22 I saw last week. Now I'm glad I waited, though I think the newest patch doesn't apply to my hardware anyway.