Xen Orchestra Container Storage Interface (CSI) for Kubernetes

jmara

robfox2

Is this in prod yet or is it still considered under development?

Cyrille

Hi @robfox2, a release candidate version is about to be published. After that, we will wait for feedback before releasing a stable version.
Keep an eye out for the release blog post on https://xen-orchestra.com/blog/tag/release/

koushik

@Cyrille

we have RKE@ managing the cluster via its own CCM and seems xenorchestra is not supporting other external CCM providers and it requires XenOrchestra CCM to be mandatory which we cannot do as RKE2 Managing our clusters.

https://github.com/vatesfr/xenorchestra-csi-driver/blob/main/docs/topology.md#is-the-ccm-mandatory

Problem:

AS RKE2 our CCM, we can't run xen CCM and hence below node objects cannot be managed by xen.

topology.k8s.xenorchestra/pool_id = <uuid>
topology.k8s.xenorchestra/host_id = <uuid>
topology.k8s.xenorchestra/pool_name_label = <name>
topology.k8s.xenorchestra/host_name_label = <name>

We tried adding them manually on the nodes, but still the PVC is waiting and complaining

Is there a workaround available for now? or is it something getting released as part of the release cycle?

Cyrille

@koushik
I don't kown RKE2. Is it its own CCM which set the node providerID?

Regarding the CSI, it needs the labels set by the CCM to know the pool topology. Try to enable only the cloud-node-label-sync in the CCM:

--controllers=cloud-node-label-sync

I'm not sure if that's enough. Maybe you'll have an issue with the provider ID, too

koushik

@Cyrille Thanks for the response.

Regarding the --controllers=cloud-node-label-sync flag: RKE2 does not use an external cloud-controller-manager (CCM) by default unless explicitly configured; it relies on its internal node controllers. Because of this, it won't automatically sync the topology.k8s.xenorchestra/* labels required by your CSI topology setup.

The main issue we are running into: Even though we applied the topology labels to the node objects manually, the XenOrchestra CSI driver's ExternalProvisioning loop is failing with a ProvisioningFailed error (error generating accessibility requirements: no topology key found for node).

This happens because the CSI driver expects the specific node providerID format to match XenServer / XenOrchestra constraints so it can look up and validate those manual topology keys.

Because RKE2 sets its own providerID formatting (providerID: rke2://koushik-xen01), the CSI driver cannot map the node to its Xen pool configuration.

Is there any flag or configuration in the CSI driver to completely bypass the strict CCM/Topology validation logic for non-native cloud setups, or do we have to wait for an update that allows an option like is-the-ccm-mandatory=false?

Cyrille

@koushik a dit:

Is there any flag or configuration in the CSI driver to completely bypass the strict CCM/Topology validation logic for non-native cloud setups, or do we have to wait for an update that allows an option like is-the-ccm-mandatory=false?

The CSI is designed to run with the Xen Orchestra provider ID because it enables the CSI node server to identify the VM on which it is running. This also means that the node server itself has no access to the Xen Orchestra API; only the CSI controller server has access.

There are no plans to remove this dependency in the next release. Sorry about your issue, we will investigate and see what we can do about that.

koushik

@Cyrille Appreciate the help! Please keep posted.

Cyrille

Hi @koushik,
Having read the RKE2 documentation, I understand that the only difference between the embedded CCM and the Xo CCM is that the latter does not have the ServiceLB LoadBalancer controller. The Xo CCM does everything else that the embedded CCM does (e.g. clearing the uninitilazed taint and setting the node IP) + it set the right providerID.

Disabling the embedded CCM will "only" result in the loss of the load balancer.

I would suggest disabling the RKE2 embedded CCM in order to use the Xo CCM as an external Cloud Controller Manager, as described in the RKE2 documentation. You need also to edit the kubelet configuration and set --kubelet-arg=cloud-provider=external.

You could then use another service to run the load balancer.

JeremyLARDENOIS

New releases !

CSI v1.0.0-rc.1

This is the first release candidate for v1.0.0. We're looking for feedback and real-world testing before the final release.

What's new:

Split controller/node modes — nodes no longer hold Xen Orchestra credentials
SR selection & live volume migration via VolumeAttributesClass
A proper XO client timeout flag
A brand-new Helm chart, with component toggles and provisioning tests

https://github.com/vatesfr/xenorchestra-csi-driver/releases/tag/v1.0.0-rc.1

CCM v1.1.2

The Helm chart can now be configured to disable Role and RoleBinding creation, making it easier to integrate into environments where RBAC resources are managed separately.

We've also bumped dependencies, including CI tooling and go-xo-sdk.

https://github.com/vatesfr/xenorchestra-cloud-controller-manager/releases/tag/v1.1.2