How to do Simple Backup to Local USB Drive?
-
I've been at this for hours and am pretty frustrated at how unnecessarily complicated this is... I've successfully used alot of different backup systems in my career but Xen Orchestra makes me feel like an absolute idiot with this.
I'm starting out with the basics: I have a 4TB external HDD plugged into my XCP-ng host via USB. It is empty and ready to be a dedicated backup disk. I want to backup ALL my host's many VMs to it on a regular basis.
From the CLI, I see the disk is attached at /dev/sdb, 3.7T reflects as the size.
What are the steps do make this available for a backup job??
Whether I mount or don't mount, create an SR or don't create an SR, format it myself or let XO format it, I always end up with a red triangle for the Local Remote saying:
Connection failed {
"errno": -28,
"code": "ENOSPC",
"syscall": "write" }Again I've been at this for hours trying everything... I've looked all over search engines and this forum. I can't imagine what I want to do should ever be this complicated. I should be able to just go into XO, tell it I want /dev/sdb to be my backup drive, and it should just make it ready and available to the backup jobs.
Would someone be able to assist me with the proper steps to accomplish this? I would massively appreciate it. Thank you!
-
Create a local backup remote under Settings > Remotes. Then use it as the target for your backup jobs.
-
@Danp I've done that, many times actually, as I mentioned in my original post.
It isn't that simple, I've tried many variations of this by now. XO keeps thinking the local remote has no space. That's why I'm asking what the prerequisite steps are to ensure XO allows this.
-
Local means local to XO, not to XCP-ng.
You are struggling because you need to understand first how XCP-ng works. A regular VM (like where XO is running) will NEVER have access to the "host" (which is also… another VM, even if special).
XO is never connecting directly to the host, it goes via an API. If you want to backup in a local drive, then it means you need to pass the local drive to XO VM first, and then use local backup repository.
-
@TechGrips said in How to do Simple Backup to Local USB Drive?:
I've done that, many times actually, as I mentioned in my original post.
Hmmm... I must have missed that in my quick review of your OP. Checked again, still don't see where you mentioned trying to create a local BR.
-
@Danp I don't know what your deal is, and I'm not trying to be rude at all, but I literally mentioned:
Whether I mount or don't mount, create an SR or don't create an SR, format it myself or let XO format it, I always end up with a red triangle for the Local Remote saying:
You can't even do anything with a backup job until you have a functioning Remote defined. I thought my post gave enough detail to indicate this is the step I was struggling with.
-
@olivierlambert Ahhh, interesting, that makes sense, especially from a security perspective.
My XO VM resides in VirtualBox on my laptop. So my whole issue was that I need to instead be trying to attach my external storage to my laptop and then pass it thru to my XO VM for setting up as a backup's Local Remote?
Moreover, the method I've been attempting where I have the external storage physically attached to my VM Pool Host (which is NOT where XO is housed) could never work as a Backup Remote because there's no easy way I could ever expose that storage over to my XO VM on a separate system, correct?
-
Yes, all BR are from XO system perspective, not XCP-ng. Backups are never stored in XCP-ng system itself. You can use a network share maybe running on your own PC that XO can reach to store the backup.
-
@olivierlambert Alrighty, I got it now!! Thanks very much, I love what you're doing and appreciate you very much!!
-
@TechGrips I'm simply trying to provide assistance. It easy to misinterpret a forum post, so let's just chalk it up to a miscommunication.
-
@Danp To be frank, I disagree and I'd prefer you refrain from engaging with me in the future, as you quite literally contributed nothing to this thread and also exhibited an unnecessarily condescending tone in your previous comment.
My company is just about to sign-up for the Essentials Pro Support and you certainly haven't made me feel warm & fuzzy about making that decision.
-
EDIT: DISCLAIMER: I'm using the below config in a home lab, using spare parts, and where there are no financial or service-level consequences if it all burns down.
@TechGrips From one of your posts I see your XO is separate from the XCP-ng host (running on VirtualBox). I tried something similar, and went through some of the same pains as you when it came to backup remote (BR) setups. In the end, I used a spare mini-PC running Ubuntu, installed XO from sources, attached a USB drive formatted as EXT4, mounted it (/mnt/USB1 or something) and used that as a 'Local BR'.
I then noticed that I sometimes saw 0 bytes free - turned out that the USB drive was going to sleep. Without properly spending any time on power management, I wrote a cron job to simply write the date to a text file on /mnt/USB1 every 5 minutes... Lazy but it kept the USB drive awake and XO backups worked great.
The biggest risk is that, if the USB drive disconnects / fails / unmounts, the /mnt/USB1 folder still exists on the root filesystem, and could fill up if a backup job consumes all the free space - so definitely look into controls for that (perhaps quotes or something more intelligent than the lazy keepawake method I have)
Side note: please remember that many people on this forum speak languages other than English, so sometimes their written messages don't carry the full context or intention. Sorry to see you've felt condescended to. I've found it helps me keep positive to try assume that people's intentions are to help, even if the person's message seems blunt or odd, or if they're asking questions that seem to blame (they're probably just trying to get more info to help). We're all here to help each other, and I've only had good, helpful service from the Vates team (@danp specifically) to date.
-
@TechGrips While I can understand the desire to use removable USB as a Backup Repo, I would highly discourage it.
Managing and rotating USB drives is a pain, if they go to sleep, it's a pain, if they fail it's a pain, if you forget to rotate your drives, it's a pain.
I personally can understand the desire to do so, it's cheap and relatively affective if you can deal with these risks, however so is just using any NFS or SMB share and then having a replication script that could write to your USB, which you could then rotate. Separating your XCP-ng hosts, XO, and your backups is of critical importance because if you have any sort of server room environmental issues or failure, you're risking loosing everything.
XCP-ng and Xen Orchestra, while they do offer a ton of flexibility, there is obviously trades-offs to using less than ideal components, such as external USB drives as your primary backup repository.
If you really want to insist on using USB drives, you'll have to attach the drives to your host and then pass them through to your XO installation, which when you want to rotate those drives you'll have to update your Backup jobs within XO and confirm that your XO VM has the proper access to the drives. This seems like a lot of complexity for very little financial benefit.
Separately I think you're taking your own frustrations out on the community, because of a lack of understanding in the tooling that you testing in comparison to ESXi where you'd attach a USB drive directly, perform your backup, remove the disk and attach another.
I get that ESXi can make things "simple" but simple isn't always better.
HTH
-
This post is deleted! -
@DustinB You're 110% correct, but more robust backup architecture costs money. Coming from a background working in a datacenter, I completely understand that using external drives creates some risks and inconveniences. However, the risk and inconvenience of having NO backups at all can of course be catastrophic. I'm still early on in building my business hosting infrastructure and money is tight so contrary to what you say, I see tons of financial benefit being able to save months of work onto some USB 4TB HDDs.
And yes you're also right that I was unnecessarily frustrated in my OP, it was late at night, I was in a rush, and was fed up toiling with something I erroneously approached with a false presumption. Yes, I should've taken the time to read the docs and understand the system architecture first.
Also, ESXi is overly expensive proprietary trash!
I really appreciate you taking the time to clarify all this but none of this is new to me, and basic/outdated backup techniques can work perfectly fine if that's all you have access to and fits with the rest of your environment well enough.
-
@TechGrips As a side thought, you could take these 4TB USB drives and put them into any old chassis and create your own NAS/SAN helping to address the issues we're discussing.
You're not going to need a lot of compute resources, though of course more helps, but if you're in a financial pinch, I would still defer to using any other backup repo over USB drives. The challenges posed with USB drives are simply a non-starter for me personally and professionally.
Something like TrueNAS is free and open source, you would just need a motherboard, CPU and RAM.
It would make the backup operation way simpler from a general management standpoint of XCP-ng and XO, though you'd have to manage the TrueNAS environment (some additional overhead).
-
@TS79 Thanks for your experience, I'll definitely keep all that in-mind!
Regarding your sidenote, I understand and 110% agree with you. I recognize this thread is maybe a poor example but I do resonate with your approach to forum etiquette and that's usually my default. I still stand by and feel valid in my responses and reactions to the support member though.
-
@TechGrips said in How to do Simple Backup to Local USB Drive?:
Also, ESXi is overly expensive proprietary trash!
I don't disagree. I'd use Hyper-V over ESXi, as at least Hyper-V is free. The management is utter trash, but at least I'm not paying for simply virtualization.
-
@DustinB Yes, I completely agree and understand everything you're saying. All of my compute is used elsewhere and I have no spares. In my situation where I lack the budget (and time) for something more robust, USB HDD manual backups will work perfectly well for me (atleast temporarily in these early stages) to preserve my investment of months of work on my datacenter. I appreciate your insights though.
