random backups failing
-
Probably too long ago, mentioned it on my follow up post while you were posting.
-
@Greg_E Worth remembering that DC backups need to be completed by something that is database aware, as far as I know, XO doesn't currently have this functionality, and if so, you are risking your entire environment....
-
@DustinB Hmmm... They have always completed in the past, but I'll admit that I haven't tried to restore one and see if it really works. too busy to perform the test the last couple years.
but now I know the time out is after the machine boots, and I've seen the console come up and wait longer than 4 minutes, so probably it is OK if it could connect to the other DC and compare the database.
-
@Greg_E Right, the backup operations are occurring outside of the VM, at the hypervisor layer. The guest drivers, don't stop or flush any database writes to disk etc.
So while the VM will likely boot, you might lose records or changes.
Depending on the size of your organization, it may be moot, but still worth keeping an eye on it.
I've seen other people use Veeam Backup and Replication to backup their Domain Controllers outside of the XO backup mechanism, and now that it's officially integrated, it might be a good contender here (and for others). https://www.veeam.com/products/free/backup-recovery.html for things that need to be handled "carefully".
-
Ok, generally I'm doing backups after everyone has gone home for the night, so changes to the DB should be minimal.
I'll have to look into this more deeply and maybe use the Server Backup tool to schedule both DC going forward. Used to use that when bare metal, and had to restore a couple times too.
-
I'm guessing this might be having updates applied, and now I need new Xen Tools. I have a feeling tomorrow will be filled with updating the tools and checking things because I bet all my backups fail tonight. And I bet all fail after they try to start.
-
I'm guessing this might be having updates applied, and now I need new Xen Tools. I have a feeling tomorrow will be filled with updating the tools and checking things because I bet all my backups fail tonight. And I bet all fail after they try to start.
Are you performing nightly healthy checks? Seems obsessive to need to check every system makes it to a login screen, but on the other hand, I understand.
Just curious.
-
@DustinB Everything I care about gets backed up and health checked each night, some things really should be several times a day, but 1 day is tollerable.
I'll have to get things updated after I deal with some php2shell stuff, that will need to come first. And now node.js in my XO machines.
-
P poddingue marked this topic as a question
-
Pretty sure I figured this out. I went to update the Xenserver tools, and the updater said the installer was busy. That meant the machine was in a "dirty condition" and couldn't work right. The dirty condition was the newest Xenserver tools needing a couple more reboots to get fully insatalled.
testing now and will have to wait on the DC until the network use get lighter this afternoon.
Running the second VM right now, but pretty sure it will now pass the health check. I'll update after I check the DC in a few hours.
-
OK, marking this solved... It was the phantom reboots causing issues because the tools were not fully installed, only thing I can't understand is why only 2 out of 3 Windows servers would quit. Any way, got the Xen Tools updated and a couple reboots later, everything is working again.
Also I can now report that a bug that was fixed maybe a year ago is confirmed to work in my system. The bug was backing up a Windows VM that had a lot of empty space in the drive (over 250GB free) and turning on compression. It would time out counting zeros because it thought the process wasn't doing work. Both of these VMs are now compressing with zstd, one of them is substantially smaller now, the other only shrunk about 20GB.
Because of the way Windows uses the drive, these are all full backups. The differential never seemed to be small because Windows does a lot of changes on the drive (there is a post about this somewhere in here from Olivier).
-
G Greg_E has marked this topic as solved
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login