Offline
Figured it out after some digging, appears it's related to Netbox having ipv4 translated into ipv6 for that field, so to use IPv4 you have to have :ffff:x.x.x.x/128 as your IP address (the x's being your octet).
Once you do this it works without issue.
I configured Netbox sync in an environment today, and after I did everything by the book, I got a 403 forbidden error. I figured this was odd since I've done this before without any issues and was really sure I followed it all identically.
Anyway, after some digging, I realized the sync did work it just threw the error anyway.
I am seeing the "Source IP x.x.x. is not permitted to authenticate" but the API key I have setup is indeed set to allow the XOA IP address.
Any idea what would cause this? It's odd to me to see a 403 response when the sync absolutely worked.
Nothing has changed since the initial sync so I can't so for sure if it'll keep syncing OK or not, but it definitely worked the first time and did show the 403 error.
@florent OK gotcha, I figured this was the case.
So best option would be to create a new backup job, encrypt that to a new remote, then go back and delete all the old stuff when ready?
@olivierlambert Oh yeah good point, that makes sense.
Well I think I've got all my answers now, thank you!
@olivierlambert right that makes sense. More so what I meant is, if you have a VM you want to migrate to another host, but that VM has a backup job (and therefore at least one snapshot since the backup jobs require there to be a snapshot at all times), then you can't migrate that VM.
So seems to me like backups and VM migration abilities sort of conflict with each other.
Or maybe there's something I'm misunderstanding here.
@olivierlambert OK this makes sense.
Though it does bring up 2 thoughts:
Thanks for the help here!
@olivierlambert Yes, it does. I should've noticed that, woops.
Is it not possible to migrate a VM with any snapshots at all or?
I'm a little lost on this one, and feel like maybe I'm missing something (went back through some previous blogs posts about CBT but didn't see anything related to this and the error doesn't appear on any google searches).
In my lab, I have a VM I am trying to migrate while powered off and it's giving me the below error whenever I do it.
},
"start": 1762284957343,
"status": "failure",
"updatedAt": 1762284960438,
"end": 1762284960438,
"result": {
"code": "VDI_CBT_ENABLED",
Any clue what this means? CBT is actually NOT enabled for this particular VDI either.
My XO is on the latest commit (58f02), hosts are all on 8.3 but don't have the most recent patches yet.
What GPU do you have? I thought I read something about NVidia's newer stuff having vGPU without the expensive licensing and hardware. But it's possible I'm just conflating that with the increase in concurrent NVENC encodes.
On the bright side, you can always get a cheap GPU and just do passthrough with it. I do that for my Jellyfin server with a 2060, it's very easy to do in XCP-ng now and works super reliably.
Did a little digging and I don't think I've seen anything about this on the forum before so wanted to post.
Does anyone know how Xen Orchestra behaves if you add an encryption key to a remote after said remote has already been used for unencrypted backups?
I'm planning to start encrypting everything I upload in the near future, if it's as simple as adding a key that's great, but I am guessing it's better to create a new remote (and new bucket) and then just restart all the backups with the new remote?
Or should I re-create the backup jobs entirely?
@DustinB Yeah I thought about doing something like this, but then we are getting into overkill costs lol.
My main point behind this post was to get info on what other people are doing, I presumed most are just relying on config backups and will restore it at the other site if needed, which is fine.
I was just also debating about whether or not I wanted the subnets to match between these 2 sites, since that can create some routing headaches, but should still be doable and would make recovery even faster.
@olivierlambert yeah I do use that feature, so that's very helpful! Should make this all doable.
@Andrew Yeah so I hear you on this, and it all makes sense, I've recovered XO plenty of times in my lab.
But, it still feels more complicated than it needs to be and I feel like there has to be a better solution in place. The idea with disaster recovery sites is relatively quick failover (even if manual). Having to restore XO, then reconnect it to all the servers, reconfigure it's IP addresses, etc... feels like it's more work than it should be to do a restore. Not to mention accessing the other cluster assuming something like complete natural disaster destruction at the primary site.
As for backups, yes, I manage a lot of infrastructure like this, not a noob by any means I promise lol, I'm just trying to improve the way I am going to do this in the future, if possible.
Restoring XOA at another site and then reconfiguring it entirely seems like a lot of work.
This is something I've put a lot of time into recently but am still not quite sure the #1 way I would go about doing it.
If XOA is hosted at one physical site, and you're planning for the possibility of that site collapsing completely, how do you go about managing other remote sites with XOA?
Mostly posting because I am curious how others are architecting this.
Thinking for me I will mostly just have XOA itself replicated and can power it on at another site if ever need be. I think the complication comes into IP addressing and the like though, since the other site might not have the same subnet layout.
@Danp I haven't yet, I will do that when I have a free moment and report back.
This VM has 8GB of RAM though, and so far it only seems to happen when viewing the console. I've only tested with 1 VM so far, but will try a few more to see if it's the console view of 1 VM causing it.
It's been about 3 hours of not viewing the console and it's stayed connected, so at least that's good news.
I'll try to get back today after checking those logs, just didn't want to leave this thread empty.
Thanks for the help!!
I have a production environment that I moved to 5.110 yesterday, this resolved the backup errors I was seeing, but now I am seeing the XOA disconnect from my hosts periodically.
So far it seems to primarily be while using the Console on a VM, though I am not 100% sure, it was stable all last night and just started happening while I was consoled into a Win10 VM.
Going into the servers tab and just disconnecting and reconnecting fixes it each time, but it's still odd so I figured I would report it.
@Bastien-Nollet I somehow missed the email about this.
But wanted to come say thanks on this thread anyway. The update fixed the issue, great work!
@Bastien-Nollet Thanks for the suggestion, though I think I am going to avoid doing that in a production setup.
I'll look forward to it being included in a newer version.
Our backups run nightly and it's random which VMs fail with this error, but so far it's almost never been the same one each night, so at worst we are 24hrs behind on a single VM backup which isn't a huge deal.
Our other nightly backup job to S3 works just fine too so we're covered.
Thanks!
@olivierlambert This is a production setup, rather than my lab, so I don't want to use anything that has a chance of being unstable.
Do we know what version this change may end up in? I'm happy to try the Latest channel if/when this fix is in place.
Apologies if I am misunderstanding anything.
@olivierlambert @florent wanted to post an update here. I upgraded to 5.108.1 in this environment and my SMB backups are doing the same thing with the error just being Footer1 !== !Footer2
It seems about half of my backups fail, so it's not all VMs.
I will try to watch it tonight to see if it's possibly a concurrency issue where it's doing all of them at the same time instead of taking my 2 concurrency setting in place.
Let me know if you want any additional info and I can also submit a ticket.