jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 resource list ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ Node ┊ Port ┊ Usage ┊ Conns ┊ State ┊ CreatedOn ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ ovbh-pprod-xen01 ┊ 7029 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 12:47:18 ┊ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ ovbh-pprod-xen02 ┊ 7029 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 12:47:21 ┊ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ ovbh-pprod-xen03 ┊ 7029 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 12:47:21 ┊ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ ovbh-vtest-k8s01-worker02 ┊ 7029 ┊ InUse ┊ Ok ┊ Diskless ┊ 2024-10-02 11:47:52 ┊ ┊ pvc-826d675c-1efe-4b64-9410-6c913bab1493 ┊ ovbh-pprod-xen01 ┊ 7030 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 13:29:27 ┊ ┊ pvc-826d675c-1efe-4b64-9410-6c913bab1493 ┊ ovbh-pprod-xen02 ┊ 7030 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 13:29:25 ┊ ┊ pvc-826d675c-1efe-4b64-9410-6c913bab1493 ┊ ovbh-pprod-xen03 ┊ 7030 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 13:29:27 ┊ ┊ pvc-826d675c-1efe-4b64-9410-6c913bab1493 ┊ ovbh-vtest-k8s01-worker02 ┊ 7030 ┊ InUse ┊ Ok ┊ Diskless ┊ 2024-10-02 11:02:43 ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ ovbh-pprod-xen01 ┊ 7031 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-02 09:07:36 ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ ovbh-pprod-xen02 ┊ 7031 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-02 09:07:41 ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ ovbh-pprod-xen03 ┊ 7031 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-02 09:07:42 ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ ovbh-vtest-k8s01-worker02 ┊ 7031 ┊ InUse ┊ Ok ┊ Diskless ┊ 2024-10-02 11:02:43 ┊ ┊ pvc-39591cf5-278b-4121-8cb3-108dd38dde64 ┊ ovbh-pprod-xen01 ┊ 7013 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:34:56 ┊ ┊ pvc-39591cf5-278b-4121-8cb3-108dd38dde64 ┊ ovbh-pprod-xen02 ┊ 7013 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:35:01 ┊ ┊ pvc-39591cf5-278b-4121-8cb3-108dd38dde64 ┊ ovbh-pprod-xen03 ┊ 7013 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:35:01 ┊ ┊ pvc-39591cf5-278b-4121-8cb3-108dd38dde64 ┊ ovbh-vprod-rancher02 ┊ 7013 ┊ InUse ┊ Ok ┊ Diskless ┊ 2024-10-02 11:58:23 ┊ ┊ pvc-da7efda8-3391-40f0-90ee-d0602ae5aa78 ┊ ovbh-pprod-xen01 ┊ 7014 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:47:50 ┊ ┊ pvc-da7efda8-3391-40f0-90ee-d0602ae5aa78 ┊ ovbh-pprod-xen02 ┊ 7014 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:47:52 ┊ ┊ pvc-da7efda8-3391-40f0-90ee-d0602ae5aa78 ┊ ovbh-pprod-xen03 ┊ 7014 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-26 12:47:52 ┊ ┊ pvc-da7efda8-3391-40f0-90ee-d0602ae5aa78 ┊ ovbh-vprod-rancher03 ┊ 7014 ┊ InUse ┊ Ok ┊ Diskless ┊ 2024-10-02 11:57:05 ┊ ┊ xcp-persistent-database ┊ ovbh-pprod-xen01 ┊ 7000 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-13 13:23:28 ┊ ┊ xcp-persistent-database ┊ ovbh-pprod-xen02 ┊ 7000 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-13 13:23:28 ┊ ┊ xcp-persistent-database ┊ ovbh-pprod-xen03 ┊ 7000 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-13 13:23:28 ┊ ┊ xcp-volume-0cfd2786-e854-47f3-8cb4-1f32b5787bd3 ┊ ovbh-pprod-xen01 ┊ 7008 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:20 ┊ ┊ xcp-volume-0cfd2786-e854-47f3-8cb4-1f32b5787bd3 ┊ ovbh-pprod-xen02 ┊ 7008 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:20 ┊ ┊ xcp-volume-0cfd2786-e854-47f3-8cb4-1f32b5787bd3 ┊ ovbh-pprod-xen03 ┊ 7008 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:20 ┊ ┊ xcp-volume-1a1ea1d6-a025-4ae1-b152-9c46b0cbd568 ┊ ovbh-pprod-xen01 ┊ 7024 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-27 15:10:35 ┊ ┊ xcp-volume-1a1ea1d6-a025-4ae1-b152-9c46b0cbd568 ┊ ovbh-pprod-xen02 ┊ 7024 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-27 15:10:35 ┊ ┊ xcp-volume-1a1ea1d6-a025-4ae1-b152-9c46b0cbd568 ┊ ovbh-pprod-xen03 ┊ 7024 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-27 15:10:35 ┊ ┊ xcp-volume-1a7700e6-b5a7-45dd-ba84-d25a81ddea7c ┊ ovbh-pprod-xen01 ┊ 7028 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:35 ┊ ┊ xcp-volume-1a7700e6-b5a7-45dd-ba84-d25a81ddea7c ┊ ovbh-pprod-xen02 ┊ 7028 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:34 ┊ ┊ xcp-volume-1a7700e6-b5a7-45dd-ba84-d25a81ddea7c ┊ ovbh-pprod-xen03 ┊ 7028 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:34 ┊ ┊ xcp-volume-3df2d70a-a0b6-4a01-83de-c8a1c216a0bd ┊ ovbh-pprod-xen01 ┊ 7020 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:30 ┊ ┊ xcp-volume-3df2d70a-a0b6-4a01-83de-c8a1c216a0bd ┊ ovbh-pprod-xen02 ┊ 7020 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:30 ┊ ┊ xcp-volume-3df2d70a-a0b6-4a01-83de-c8a1c216a0bd ┊ ovbh-pprod-xen03 ┊ 7020 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:30 ┊ ┊ xcp-volume-4ac6337a-a033-4d33-8469-f41cafdf9c5f ┊ ovbh-pprod-xen01 ┊ 7006 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:28 ┊ ┊ xcp-volume-4ac6337a-a033-4d33-8469-f41cafdf9c5f ┊ ovbh-pprod-xen02 ┊ 7006 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:28 ┊ ┊ xcp-volume-4ac6337a-a033-4d33-8469-f41cafdf9c5f ┊ ovbh-pprod-xen03 ┊ 7006 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:28 ┊ ┊ xcp-volume-6f52f1aa-e181-4a0f-a747-25dd0686cd8a ┊ ovbh-pprod-xen01 ┊ 7010 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:13:27 ┊ ┊ xcp-volume-6f52f1aa-e181-4a0f-a747-25dd0686cd8a ┊ ovbh-pprod-xen02 ┊ 7010 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:13:27 ┊ ┊ xcp-volume-6f52f1aa-e181-4a0f-a747-25dd0686cd8a ┊ ovbh-pprod-xen03 ┊ 7010 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:13:27 ┊ ┊ xcp-volume-9bbee9d8-e481-420e-a3cf-c5be5822207c ┊ ovbh-pprod-xen01 ┊ 7016 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:09 ┊ ┊ xcp-volume-9bbee9d8-e481-420e-a3cf-c5be5822207c ┊ ovbh-pprod-xen02 ┊ 7016 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:09 ┊ ┊ xcp-volume-9bbee9d8-e481-420e-a3cf-c5be5822207c ┊ ovbh-pprod-xen03 ┊ 7016 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:09 ┊ ┊ xcp-volume-10aeac73-506a-490e-8434-457f66c00241 ┊ ovbh-pprod-xen01 ┊ 7012 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:39 ┊ ┊ xcp-volume-10aeac73-506a-490e-8434-457f66c00241 ┊ ovbh-pprod-xen02 ┊ 7012 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:39 ┊ ┊ xcp-volume-10aeac73-506a-490e-8434-457f66c00241 ┊ ovbh-pprod-xen03 ┊ 7012 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:39 ┊ ┊ xcp-volume-84a6d001-f493-4431-b469-92b7650986c5 ┊ ovbh-pprod-xen01 ┊ 7025 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:48 ┊ ┊ xcp-volume-84a6d001-f493-4431-b469-92b7650986c5 ┊ ovbh-pprod-xen02 ┊ 7025 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:48 ┊ ┊ xcp-volume-84a6d001-f493-4431-b469-92b7650986c5 ┊ ovbh-pprod-xen03 ┊ 7025 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:48 ┊ ┊ xcp-volume-86fbe439-38dd-4dcd-86c5-1b38c7a30ebc ┊ ovbh-pprod-xen01 ┊ 7005 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:00 ┊ ┊ xcp-volume-86fbe439-38dd-4dcd-86c5-1b38c7a30ebc ┊ ovbh-pprod-xen02 ┊ 7005 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:00 ┊ ┊ xcp-volume-86fbe439-38dd-4dcd-86c5-1b38c7a30ebc ┊ ovbh-pprod-xen03 ┊ 7005 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:00 ┊ ┊ xcp-volume-87a878f9-6333-4159-9c0b-20a67bf6efb1 ┊ ovbh-pprod-xen01 ┊ 7018 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 09:53:58 ┊ ┊ xcp-volume-87a878f9-6333-4159-9c0b-20a67bf6efb1 ┊ ovbh-pprod-xen02 ┊ 7018 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 09:53:58 ┊ ┊ xcp-volume-87a878f9-6333-4159-9c0b-20a67bf6efb1 ┊ ovbh-pprod-xen03 ┊ 7018 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 09:53:58 ┊ ┊ xcp-volume-87d4c73b-feab-4640-8844-998b85b9f4b5 ┊ ovbh-pprod-xen01 ┊ 7004 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:20 ┊ ┊ xcp-volume-87d4c73b-feab-4640-8844-998b85b9f4b5 ┊ ovbh-pprod-xen02 ┊ 7004 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:20 ┊ ┊ xcp-volume-87d4c73b-feab-4640-8844-998b85b9f4b5 ┊ ovbh-pprod-xen03 ┊ 7004 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:20 ┊ ┊ xcp-volume-929a8254-8d81-4fc9-ace5-29e9860f585d ┊ ovbh-pprod-xen01 ┊ 7027 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:17 ┊ ┊ xcp-volume-929a8254-8d81-4fc9-ace5-29e9860f585d ┊ ovbh-pprod-xen02 ┊ 7027 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:17 ┊ ┊ xcp-volume-929a8254-8d81-4fc9-ace5-29e9860f585d ┊ ovbh-pprod-xen03 ┊ 7027 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:17 ┊ ┊ xcp-volume-38885e62-f5b0-4da8-9e79-8746a7fd58bd ┊ ovbh-pprod-xen01 ┊ 7019 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:20 ┊ ┊ xcp-volume-38885e62-f5b0-4da8-9e79-8746a7fd58bd ┊ ovbh-pprod-xen02 ┊ 7019 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:20 ┊ ┊ xcp-volume-38885e62-f5b0-4da8-9e79-8746a7fd58bd ┊ ovbh-pprod-xen03 ┊ 7019 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:20 ┊ ┊ xcp-volume-169757fc-3021-42c5-ae9e-eff5a846f588 ┊ ovbh-pprod-xen01 ┊ 7011 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:12 ┊ ┊ xcp-volume-169757fc-3021-42c5-ae9e-eff5a846f588 ┊ ovbh-pprod-xen02 ┊ 7011 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:12 ┊ ┊ xcp-volume-169757fc-3021-42c5-ae9e-eff5a846f588 ┊ ovbh-pprod-xen03 ┊ 7011 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:52:12 ┊ ┊ xcp-volume-410345c4-a26d-4087-baac-9717f6a34adf ┊ ovbh-pprod-xen01 ┊ 7003 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:36 ┊ ┊ xcp-volume-410345c4-a26d-4087-baac-9717f6a34adf ┊ ovbh-pprod-xen02 ┊ 7003 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:36 ┊ ┊ xcp-volume-410345c4-a26d-4087-baac-9717f6a34adf ┊ ovbh-pprod-xen03 ┊ 7003 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:36 ┊ ┊ xcp-volume-aba4c22b-d85b-47ce-82d2-942c14ffcb96 ┊ ovbh-pprod-xen01 ┊ 7021 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:56 ┊ ┊ xcp-volume-aba4c22b-d85b-47ce-82d2-942c14ffcb96 ┊ ovbh-pprod-xen02 ┊ 7021 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:56 ┊ ┊ xcp-volume-aba4c22b-d85b-47ce-82d2-942c14ffcb96 ┊ ovbh-pprod-xen03 ┊ 7021 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:56 ┊ ┊ xcp-volume-bbc3f6bd-d934-4231-b800-0beb9e16a63d ┊ ovbh-pprod-xen01 ┊ 7015 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:01 ┊ ┊ xcp-volume-bbc3f6bd-d934-4231-b800-0beb9e16a63d ┊ ovbh-pprod-xen02 ┊ 7015 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:01 ┊ ┊ xcp-volume-bbc3f6bd-d934-4231-b800-0beb9e16a63d ┊ ovbh-pprod-xen03 ┊ 7015 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:01 ┊ ┊ xcp-volume-d30e689e-e140-4b1c-8322-ec7681f6dfc0 ┊ ovbh-pprod-xen01 ┊ 7022 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:28 ┊ ┊ xcp-volume-d30e689e-e140-4b1c-8322-ec7681f6dfc0 ┊ ovbh-pprod-xen02 ┊ 7022 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:28 ┊ ┊ xcp-volume-d30e689e-e140-4b1c-8322-ec7681f6dfc0 ┊ ovbh-pprod-xen03 ┊ 7022 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:28 ┊ ┊ xcp-volume-d43820d0-fc48-43cc-a62d-751d802f25e4 ┊ ovbh-pprod-xen01 ┊ 7023 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:10 ┊ ┊ xcp-volume-d43820d0-fc48-43cc-a62d-751d802f25e4 ┊ ovbh-pprod-xen02 ┊ 7023 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:10 ┊ ┊ xcp-volume-d43820d0-fc48-43cc-a62d-751d802f25e4 ┊ ovbh-pprod-xen03 ┊ 7023 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:17:10 ┊ ┊ xcp-volume-e4ef9d58-334b-4105-8ce9-6ca20e029713 ┊ ovbh-pprod-xen01 ┊ 7009 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:52 ┊ ┊ xcp-volume-e4ef9d58-334b-4105-8ce9-6ca20e029713 ┊ ovbh-pprod-xen02 ┊ 7009 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:52 ┊ ┊ xcp-volume-e4ef9d58-334b-4105-8ce9-6ca20e029713 ┊ ovbh-pprod-xen03 ┊ 7009 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:52 ┊ ┊ xcp-volume-e7ad2ce0-2292-4f83-9a72-5c0088e8231b ┊ ovbh-pprod-xen01 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-13 15:34:05 ┊ ┊ xcp-volume-e7ad2ce0-2292-4f83-9a72-5c0088e8231b ┊ ovbh-pprod-xen02 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-13 15:34:05 ┊ ┊ xcp-volume-e7ad2ce0-2292-4f83-9a72-5c0088e8231b ┊ ovbh-pprod-xen03 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-13 15:34:05 ┊ ┊ xcp-volume-e9fb3127-9ae6-4d27-a5fb-44ebb42c96cd ┊ ovbh-pprod-xen01 ┊ 7002 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 14:35:57 ┊ ┊ xcp-volume-e9fb3127-9ae6-4d27-a5fb-44ebb42c96cd ┊ ovbh-pprod-xen02 ┊ 7002 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 14:35:57 ┊ ┊ xcp-volume-e9fb3127-9ae6-4d27-a5fb-44ebb42c96cd ┊ ovbh-pprod-xen03 ┊ 7002 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 14:35:57 ┊ ┊ xcp-volume-e12f4d54-2756-49b7-879f-3b7ff24cfb64 ┊ ovbh-pprod-xen01 ┊ 7026 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:59 ┊ ┊ xcp-volume-e12f4d54-2756-49b7-879f-3b7ff24cfb64 ┊ ovbh-pprod-xen02 ┊ 7026 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:59 ┊ ┊ xcp-volume-e12f4d54-2756-49b7-879f-3b7ff24cfb64 ┊ ovbh-pprod-xen03 ┊ 7026 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:16:59 ┊ ┊ xcp-volume-e278368b-dafa-4c07-a060-dcf47a15ec31 ┊ ovbh-pprod-xen01 ┊ 7007 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:43 ┊ ┊ xcp-volume-e278368b-dafa-4c07-a060-dcf47a15ec31 ┊ ovbh-pprod-xen02 ┊ 7007 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:43 ┊ ┊ xcp-volume-e278368b-dafa-4c07-a060-dcf47a15ec31 ┊ ovbh-pprod-xen03 ┊ 7007 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-09-23 15:51:43 ┊ ┊ xcp-volume-feae0859-b5ac-431b-87a2-8d5d4b282e29 ┊ ovbh-pprod-xen01 ┊ 7017 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:42 ┊ ┊ xcp-volume-feae0859-b5ac-431b-87a2-8d5d4b282e29 ┊ ovbh-pprod-xen02 ┊ 7017 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:42 ┊ ┊ xcp-volume-feae0859-b5ac-431b-87a2-8d5d4b282e29 ┊ ovbh-pprod-xen03 ┊ 7017 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2024-10-01 10:15:43 ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 snapshot list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ SnapshotName ┊ NodeNames ┊ Volumes ┊ CreatedOn ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241003_173503 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 14:35:04 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 15:12:57 ┊ Successful ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list usage: linstor backup list [-h] [-p] [-r RESOURCE] [-s SNAPSHOT] [-o] [-i] remote_name linstor backup list: error: too few arguments jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 remote l ╭────────────────────────────────────────────────────────────────────────────╮ ┊ Name ┊ Type ┊ Info ┊ ╞════════════════════════════════════════════════════════════════════════════╡ ┊ linbit-velero-backup ┊ S3 ┊ us-east-1.s3.wasabisys.com/velero-preprod ┊ ┊ sourceCluster ┊ Linstor ┊ http://10.2.0.19:3370 ┊ ╰────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭───────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═══════════════════════════════════════════════════════╡ ╰───────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup create linbit-velero-backup pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' for snapshot SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' after snapshot INFO: Generated snapshot name for backup of resourcepvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup INFO: Shipping of resource pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup in progress. jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭───────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═══════════════════════════════════════════════════════╡ ╰───────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 snapshot list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ SnapshotName ┊ NodeNames ┊ Volumes ┊ CreatedOn ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241003_173503 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 14:35:04 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:07:50 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 15:12:57 ┊ Successful ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 err l ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Id ┊ Datetime ┊ Node ┊ Exception ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ 66E49F36-00000-000000 ┊ 2024-09-13 13:23:21 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F43-00000-000000 ┊ 2024-09-13 13:23:33 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F46-00000-000000 ┊ 2024-09-13 13:23:37 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F46-00000-000001 ┊ 2024-09-26 12:30:33 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'ovbh-pprod-xen1' not found. ┊ ┊ 66F5B327-88419-000000 ┊ 2024-09-26 12:43:12 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66E49F46-00000-000002 ┊ 2024-09-26 15:37:13 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'pvc-f531a5c0-0ce6-4b7c-b5d3-eb8439cd... ┊ ┊ 66F5E05F-88419-000000 ┊ 2024-09-26 19:02:03 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-88419-000001 ┊ 2024-09-26 19:02:04 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-88419-000002 ┊ 2024-09-26 19:27:55 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000003 ┊ 2024-09-26 19:27:55 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000000 ┊ 2024-09-26 19:27:57 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000001 ┊ 2024-09-26 19:28:09 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000004 ┊ 2024-09-26 19:28:15 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000005 ┊ 2024-09-26 19:28:16 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000002 ┊ 2024-09-26 19:28:17 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000003 ┊ 2024-09-26 19:28:29 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000000 ┊ 2024-09-26 19:59:17 ┊ S|ovbh-vprod-rancher01 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-95078-000001 ┊ 2024-09-26 20:24:04 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000002 ┊ 2024-09-26 20:24:25 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000003 ┊ 2024-09-26 20:24:45 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000004 ┊ 2024-09-26 20:25:06 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000004 ┊ 2024-09-27 00:58:41 ┊ S|ovbh-vprod-rancher02 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E060-34515-000005 ┊ 2024-09-27 06:57:56 ┊ S|ovbh-vprod-rancher02 ┊ IOException: Connection reset by peer ┊ ┊ 66E49F46-00000-000003 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000004 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000005 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000006 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000007 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000008 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000009 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66FC4F7F-F1B83-000000 ┊ 2024-10-01 19:54:24 ┊ S|ovbh-vtest-k8s01-worker03 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F7F-F1B83-000001 ┊ 2024-10-01 20:02:43 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-F1B83-000002 ┊ 2024-10-01 20:02:53 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F80-ED49B-000000 ┊ 2024-10-01 21:34:04 ┊ S|ovbh-vtest-k8s01-worker02 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F80-ED49B-000001 ┊ 2024-10-01 21:42:24 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F80-ED49B-000002 ┊ 2024-10-01 21:42:34 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-CDC5F-000000 ┊ 2024-10-01 23:21:55 ┊ S|ovbh-vtest-k8s01-worker01 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F7F-CDC5F-000001 ┊ 2024-10-01 23:30:13 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-CDC5F-000002 ┊ 2024-10-01 23:30:23 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66E49F46-00000-000010 ┊ 2024-10-02 08:48:19 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Unable to create encrypt key without having a master key ┊ ┊ 66E49F46-00000-000011 ┊ 2024-10-02 09:40:10 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66FD78AB-3AE91-000000 ┊ 2024-10-02 09:45:44 ┊ S|ovbh-pprod-xen01 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD7C33-A3684-000000 ┊ 2024-10-02 10:00:42 ┊ S|ovbh-pprod-xen02 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD7DA8-00000-000000 ┊ 2024-10-02 10:06:50 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FD7EE6-EF9B7-000000 ┊ 2024-10-02 10:12:21 ┊ S|ovbh-pprod-xen03 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD78AB-3AE91-000001 ┊ 2024-10-02 11:37:58 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000002 ┊ 2024-10-02 11:59:44 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000003 ┊ 2024-10-02 13:39:19 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000004 ┊ 2024-10-02 14:00:37 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000005 ┊ 2024-10-02 14:18:59 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD7DA8-00000-000001 ┊ 2024-10-02 15:30:31 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'linbit-velero-backup' not found. ┊ ┊ 66FD78AB-3AE91-000006 ┊ 2024-10-02 15:32:53 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000007 ┊ 2024-10-02 15:41:34 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000008 ┊ 2024-10-02 15:43:31 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000000 ┊ 2024-10-02 16:05:49 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FDD1AE-3AE91-000000 ┊ 2024-10-02 16:11:47 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000001 ┊ 2024-10-03 08:47:03 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FDD1CA-00000-000002 ┊ 2024-10-03 08:47:14 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: ' (Service: Amazon S3; Status Code: 400; Error Code: Auth... ┊ ┊ 66FDD1CA-00000-000003 ┊ 2024-10-03 08:50:06 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: The remote could not be created because the following req... ┊ ┊ 66FDD1CA-00000-000004 ┊ 2024-10-03 08:50:35 ┊ C|ovbh-pprod-xen01 ┊ SdkClientException: Unable to execute HTTP request: velero-preprod.wasabisys.... ┊ ┊ 66FDD1AE-3AE91-000001 ┊ 2024-10-03 08:52:01 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000005 ┊ 2024-10-03 09:02:32 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: A remote with the name 'linbit-velero-backup' already exi... ┊ ┊ 66FDD1CA-00000-000006 ┊ 2024-10-03 09:02:44 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FEE891-00000-000000 ┊ 2024-10-03 11:55:16 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FEE891-00000-000001 ┊ 2024-10-03 12:02:14 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Unable to create encrypt key without having a master key ┊ ┊ 66FEE891-00000-000002 ┊ 2024-10-03 12:02:59 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FF0404-00000-000000 ┊ 2024-10-03 13:52:23 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0468-3AE91-000000 ┊ 2024-10-03 14:02:26 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FF094E-00000-000000 ┊ 2024-10-03 14:14:57 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0D03-00000-000000 ┊ 2024-10-03 14:30:46 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0D66-3AE91-000000 ┊ 2024-10-03 14:35:11 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FF0E92-00000-000000 ┊ 2024-10-03 14:37:25 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FD94C6-ED49B-000000 ┊ 2024-10-04 01:10:25 ┊ S|ovbh-vtest-k8s01-worker02 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-ED49B-000001 ┊ 2024-10-04 01:19:03 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-ED49B-000002 ┊ 2024-10-04 01:19:13 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-CDC5F-000000 ┊ 2024-10-04 05:26:00 ┊ S|ovbh-vtest-k8s01-worker01 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-CDC5F-000001 ┊ 2024-10-04 05:34:35 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-CDC5F-000002 ┊ 2024-10-04 05:34:45 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-F1B83-000000 ┊ 2024-10-04 07:19:17 ┊ S|ovbh-vtest-k8s01-worker03 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-F1B83-000001 ┊ 2024-10-04 07:27:49 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-F1B83-000002 ┊ 2024-10-04 07:27:59 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FF0E92-00000-000001 ┊ 2024-10-04 15:33:18 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000002 ┊ 2024-10-04 15:38:40 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Invalid property key ┊ ┊ 66FF0E92-00000-000003 ┊ 2024-10-04 15:40:01 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Invalid property key ┊ ┊ 66FF0E92-00000-000004 ┊ 2024-10-04 15:41:26 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000005 ┊ 2024-10-04 15:45:22 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'pvc-086a5817-d813-41fe-86d8-3fac2ae2028f' not found. ┊ ┊ 66FF0E92-00000-000006 ┊ 2024-10-04 15:46:07 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000007 ┊ 2024-10-04 15:46:14 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000008 ┊ 2024-10-07 08:59:24 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 err show 66FF0E92-00000-000008 ERROR REPORT 66FF0E92-00000-000008 ============================================================ Application: LINBIT® LINSTOR Module: Controller Version: 1.26.1 Build ID: 12746ac9c6e7882807972c3df56e9a89eccad4e5 Build time: 2024-02-22T05:27:50+00:00 Error time: 2024-10-07 11:59:24 Node: ovbh-pprod-xen01 Thread: MainWorkerPool-7 ============================================================ Reported error: =============== Category: RuntimeException Class name: ApiRcException Class canonical name: com.linbit.linstor.core.apicallhandler.response.ApiRcException Generated at: Method 'restoreBackupL2LInTransaction', Source file 'CtrlBackupRestoreApiCallHandler.java', Line #1123 Error message: Could not find suitable storage pool to receive backup Asynchronous stage backtrace: Error has been observed at the following site(s): *__checkpoint ⇢ restore backup *__checkpoint ⇢ Backupshipping L2L start receive Original Stack Trace: Call backtrace: Method Native Class:Line number restoreBackupL2LInTransaction N com.linbit.linstor.core.apicallhandler.controller.backup.CtrlBackupRestoreApiCallHandler:1123 Suppressed exception 1 of 1: =============== Category: RuntimeException Class name: OnAssemblyException Class canonical name: reactor.core.publisher.FluxOnAssembly.OnAssemblyException Generated at: Method 'restoreBackupL2LInTransaction', Source file 'CtrlBackupRestoreApiCallHandler.java', Line #1123 Error message: Error has been observed at the following site(s): *__checkpoint ⇢ restore backup *__checkpoint ⇢ Backupshipping L2L start receive Original Stack Trace: Call backtrace: Method Native Class:Line number restoreBackupL2LInTransaction N com.linbit.linstor.core.apicallhandler.controller.backup.CtrlBackupRestoreApiCallHandler:1123 lambda$startReceivingInTransaction$4 N com.linbit.linstor.core.apicallhandler.controller.backup.CtrlBackupL2LDstApiCallHandler:526 doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:149 lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:76 call N reactor.core.publisher.MonoCallable:72 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.Flux:8759 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 request N reactor.core.publisher.Operators$ScalarSubscription:2545 onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141 subscribe N reactor.core.publisher.MonoJust:55 subscribe N reactor.core.publisher.MonoDeferContextual:55 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 onNext N reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber:129 completePossiblyEmpty N reactor.core.publisher.Operators$BaseFluxToMonoOperator:2071 onComplete N reactor.core.publisher.MonoCollect$CollectSubscriber:145 onComplete N reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner:260 checkTerminated N reactor.core.publisher.FluxFlatMap$FlatMapMain:847 drainLoop N reactor.core.publisher.FluxFlatMap$FlatMapMain:609 drain N reactor.core.publisher.FluxFlatMap$FlatMapMain:589 onComplete N reactor.core.publisher.FluxFlatMap$FlatMapMain:466 checkTerminated N reactor.core.publisher.FluxFlatMap$FlatMapMain:847 drainLoop N reactor.core.publisher.FluxFlatMap$FlatMapMain:609 innerComplete N reactor.core.publisher.FluxFlatMap$FlatMapMain:895 onComplete N reactor.core.publisher.FluxFlatMap$FlatMapInner:998 onComplete N reactor.core.publisher.FluxMap$MapSubscriber:144 onComplete N reactor.core.publisher.Operators$MultiSubscriptionSubscriber:2205 onComplete N reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber:85 complete N reactor.core.publisher.FluxCreate$BaseSink:460 drain N reactor.core.publisher.FluxCreate$BufferAsyncSink:805 complete N reactor.core.publisher.FluxCreate$BufferAsyncSink:753 drainLoop N reactor.core.publisher.FluxCreate$SerializedFluxSink:247 drain N reactor.core.publisher.FluxCreate$SerializedFluxSink:213 complete N reactor.core.publisher.FluxCreate$SerializedFluxSink:204 apiCallComplete N com.linbit.linstor.netcom.TcpConnectorPeer:506 handleComplete N com.linbit.linstor.proto.CommonMessageProcessor:372 handleDataMessage N com.linbit.linstor.proto.CommonMessageProcessor:296 doProcessInOrderMessage N com.linbit.linstor.proto.CommonMessageProcessor:244 lambda$doProcessMessage$4 N com.linbit.linstor.proto.CommonMessageProcessor:229 subscribe N reactor.core.publisher.FluxDefer:46 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.FluxFlatMap$FlatMapMain:427 drainAsync N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:453 drain N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:724 onNext N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:256 drainFused N reactor.core.publisher.SinkManyUnicast:319 drain N reactor.core.publisher.SinkManyUnicast:362 tryEmitNext N reactor.core.publisher.SinkManyUnicast:237 tryEmitNext N reactor.core.publisher.SinkManySerialized:100 processInOrder N com.linbit.linstor.netcom.TcpConnectorPeer:419 doProcessMessage N com.linbit.linstor.proto.CommonMessageProcessor:227 lambda$processMessage$2 N com.linbit.linstor.proto.CommonMessageProcessor:164 onNext N reactor.core.publisher.FluxPeek$PeekSubscriber:185 runAsync N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:440 run N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:527 call N reactor.core.scheduler.WorkerTask:84 call N reactor.core.scheduler.WorkerTask:37 run N java.util.concurrent.FutureTask:264 run N java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:304 runWorker N java.util.concurrent.ThreadPoolExecutor:1128 run N java.util.concurrent.ThreadPoolExecutor$Worker:628 run N java.lang.Thread:829 END OF ERROR REPORT. jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup create linbit-velero-backup pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' for snapshot SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' after snapshot INFO: Generated snapshot name for backup of resourcepvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup INFO: Shipping of resource pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup in progress. jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 err l ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Id ┊ Datetime ┊ Node ┊ Exception ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ 66E49F36-00000-000000 ┊ 2024-09-13 13:23:21 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F43-00000-000000 ┊ 2024-09-13 13:23:33 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F46-00000-000000 ┊ 2024-09-13 13:23:37 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66E49F46-00000-000001 ┊ 2024-09-26 12:30:33 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'ovbh-pprod-xen1' not found. ┊ ┊ 66F5B327-88419-000000 ┊ 2024-09-26 12:43:12 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66E49F46-00000-000002 ┊ 2024-09-26 15:37:13 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'pvc-f531a5c0-0ce6-4b7c-b5d3-eb8439cd... ┊ ┊ 66F5E05F-88419-000000 ┊ 2024-09-26 19:02:03 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-88419-000001 ┊ 2024-09-26 19:02:04 ┊ S|ovbh-vprod-rancher03 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-88419-000002 ┊ 2024-09-26 19:27:55 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000003 ┊ 2024-09-26 19:27:55 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000000 ┊ 2024-09-26 19:27:57 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000001 ┊ 2024-09-26 19:28:09 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000004 ┊ 2024-09-26 19:28:15 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-88419-000005 ┊ 2024-09-26 19:28:16 ┊ S|ovbh-vprod-rancher03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000002 ┊ 2024-09-26 19:28:17 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000003 ┊ 2024-09-26 19:28:29 ┊ S|ovbh-vprod-rancher02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000000 ┊ 2024-09-26 19:59:17 ┊ S|ovbh-vprod-rancher01 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E05F-95078-000001 ┊ 2024-09-26 20:24:04 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000002 ┊ 2024-09-26 20:24:25 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000003 ┊ 2024-09-26 20:24:45 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E05F-95078-000004 ┊ 2024-09-26 20:25:06 ┊ S|ovbh-vprod-rancher01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66F5E060-34515-000004 ┊ 2024-09-27 00:58:41 ┊ S|ovbh-vprod-rancher02 ┊ IOException: Connection reset by peer ┊ ┊ 66F5E060-34515-000005 ┊ 2024-09-27 06:57:56 ┊ S|ovbh-vprod-rancher02 ┊ IOException: Connection reset by peer ┊ ┊ 66E49F46-00000-000003 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000004 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000005 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000006 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000007 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000008 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66E49F46-00000-000009 ┊ 2024-09-27 16:06:52 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66FC4F7F-F1B83-000000 ┊ 2024-10-01 19:54:24 ┊ S|ovbh-vtest-k8s01-worker03 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F7F-F1B83-000001 ┊ 2024-10-01 20:02:43 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-F1B83-000002 ┊ 2024-10-01 20:02:53 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F80-ED49B-000000 ┊ 2024-10-01 21:34:04 ┊ S|ovbh-vtest-k8s01-worker02 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F80-ED49B-000001 ┊ 2024-10-01 21:42:24 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F80-ED49B-000002 ┊ 2024-10-01 21:42:34 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-CDC5F-000000 ┊ 2024-10-01 23:21:55 ┊ S|ovbh-vtest-k8s01-worker01 ┊ IOException: Connection reset by peer ┊ ┊ 66FC4F7F-CDC5F-000001 ┊ 2024-10-01 23:30:13 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FC4F7F-CDC5F-000002 ┊ 2024-10-01 23:30:23 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66E49F46-00000-000010 ┊ 2024-10-02 08:48:19 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Unable to create encrypt key without having a master key ┊ ┊ 66E49F46-00000-000011 ┊ 2024-10-02 09:40:10 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Resource definition 'xcp-volume-c2e6d2ae-6e79-49ab-9b4f-e... ┊ ┊ 66FD78AB-3AE91-000000 ┊ 2024-10-02 09:45:44 ┊ S|ovbh-pprod-xen01 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD7C33-A3684-000000 ┊ 2024-10-02 10:00:42 ┊ S|ovbh-pprod-xen02 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD7DA8-00000-000000 ┊ 2024-10-02 10:06:50 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FD7EE6-EF9B7-000000 ┊ 2024-10-02 10:12:21 ┊ S|ovbh-pprod-xen03 ┊ StorageException: Unable to parse free thin sizes ┊ ┊ 66FD78AB-3AE91-000001 ┊ 2024-10-02 11:37:58 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000002 ┊ 2024-10-02 11:59:44 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000003 ┊ 2024-10-02 13:39:19 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000004 ┊ 2024-10-02 14:00:37 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000005 ┊ 2024-10-02 14:18:59 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD7DA8-00000-000001 ┊ 2024-10-02 15:30:31 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'linbit-velero-backup' not found. ┊ ┊ 66FD78AB-3AE91-000006 ┊ 2024-10-02 15:32:53 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000007 ┊ 2024-10-02 15:41:34 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FD78AB-3AE91-000008 ┊ 2024-10-02 15:43:31 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000000 ┊ 2024-10-02 16:05:49 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FDD1AE-3AE91-000000 ┊ 2024-10-02 16:11:47 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000001 ┊ 2024-10-03 08:47:03 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FDD1CA-00000-000002 ┊ 2024-10-03 08:47:14 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: ' (Service: Amazon S3; Status Code: 400; Error Code: Auth... ┊ ┊ 66FDD1CA-00000-000003 ┊ 2024-10-03 08:50:06 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: The remote could not be created because the following req... ┊ ┊ 66FDD1CA-00000-000004 ┊ 2024-10-03 08:50:35 ┊ C|ovbh-pprod-xen01 ┊ SdkClientException: Unable to execute HTTP request: velero-preprod.wasabisys.... ┊ ┊ 66FDD1AE-3AE91-000001 ┊ 2024-10-03 08:52:01 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FDD1CA-00000-000005 ┊ 2024-10-03 09:02:32 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: A remote with the name 'linbit-velero-backup' already exi... ┊ ┊ 66FDD1CA-00000-000006 ┊ 2024-10-03 09:02:44 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FEE891-00000-000000 ┊ 2024-10-03 11:55:16 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FEE891-00000-000001 ┊ 2024-10-03 12:02:14 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Unable to create encrypt key without having a master key ┊ ┊ 66FEE891-00000-000002 ┊ 2024-10-03 12:02:59 ┊ C|ovbh-pprod-xen01 ┊ AmazonS3Exception: The request signature we calculated does not match the si... ┊ ┊ 66FF0404-00000-000000 ┊ 2024-10-03 13:52:23 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0468-3AE91-000000 ┊ 2024-10-03 14:02:26 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FF094E-00000-000000 ┊ 2024-10-03 14:14:57 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0D03-00000-000000 ┊ 2024-10-03 14:30:46 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FF0D66-3AE91-000000 ┊ 2024-10-03 14:35:11 ┊ S|ovbh-pprod-xen01 ┊ AbortedException ┊ ┊ 66FF0E92-00000-000000 ┊ 2024-10-03 14:37:25 ┊ C|ovbh-pprod-xen01 ┊ SocketException: Protocol family unavailable ┊ ┊ 66FD94C6-ED49B-000000 ┊ 2024-10-04 01:10:25 ┊ S|ovbh-vtest-k8s01-worker02 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-ED49B-000001 ┊ 2024-10-04 01:19:03 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-ED49B-000002 ┊ 2024-10-04 01:19:13 ┊ S|ovbh-vtest-k8s01-worker02 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-CDC5F-000000 ┊ 2024-10-04 05:26:00 ┊ S|ovbh-vtest-k8s01-worker01 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-CDC5F-000001 ┊ 2024-10-04 05:34:35 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-CDC5F-000002 ┊ 2024-10-04 05:34:45 ┊ S|ovbh-vtest-k8s01-worker01 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-F1B83-000000 ┊ 2024-10-04 07:19:17 ┊ S|ovbh-vtest-k8s01-worker03 ┊ IOException: Connection reset by peer ┊ ┊ 66FD94C6-F1B83-000001 ┊ 2024-10-04 07:27:49 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FD94C6-F1B83-000002 ┊ 2024-10-04 07:27:59 ┊ S|ovbh-vtest-k8s01-worker03 ┊ LinStorException: Non-existent API '' called by the client ┊ ┊ 66FF0E92-00000-000001 ┊ 2024-10-04 15:33:18 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000002 ┊ 2024-10-04 15:38:40 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Invalid property key ┊ ┊ 66FF0E92-00000-000003 ┊ 2024-10-04 15:40:01 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Invalid property key ┊ ┊ 66FF0E92-00000-000004 ┊ 2024-10-04 15:41:26 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000005 ┊ 2024-10-04 15:45:22 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Node 'pvc-086a5817-d813-41fe-86d8-3fac2ae2028f' not found. ┊ ┊ 66FF0E92-00000-000006 ┊ 2024-10-04 15:46:07 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000007 ┊ 2024-10-04 15:46:14 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ┊ 66FF0E92-00000-000008 ┊ 2024-10-07 08:59:24 ┊ C|ovbh-pprod-xen01 ┊ ApiRcException: Could not find suitable storage pool to receive backup ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 snapshot list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ SnapshotName ┊ NodeNames ┊ Volumes ┊ CreatedOn ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241003_173503 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 14:35:04 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:07:50 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120905 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:09:06 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 15:12:57 ┊ Successful ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-086a5817-d813-41fe-86d8-3fac2ae2028f ┊ back_20241007_121136 ┊ 2024-10-07 09:11:48 ┊ ┊ Success ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭───────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═══════════════════════════════════════════════════════╡ ╰───────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup create linbit-velero-backup pvc-58af89d6-a7fc-43e1-92dd-861c8835115f SUCCESS: Suspended IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-vtest-k8s01-worker02' for snapshot SUCCESS: Suspended IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen03' for snapshot SUCCESS: Suspended IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen02' for snapshot SUCCESS: Suspended IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen01' for snapshot SUCCESS: Took snapshot of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-vtest-k8s01-worker02' SUCCESS: Took snapshot of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen01' SUCCESS: Took snapshot of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen02' SUCCESS: Took snapshot of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen03' SUCCESS: Resumed IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-vtest-k8s01-worker02' after snapshot SUCCESS: Resumed IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen02' after snapshot SUCCESS: Resumed IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen03' after snapshot SUCCESS: Resumed IO of '[pvc-58af89d6-a7fc-43e1-92dd-861c8835115f]' on 'ovbh-pprod-xen01' after snapshot INFO: Generated snapshot name for backup of resourcepvc-58af89d6-a7fc-43e1-92dd-861c8835115f to remote linbit-velero-backup INFO: Shipping of resource pvc-58af89d6-a7fc-43e1-92dd-861c8835115f to remote linbit-velero-backup in progress. SUCCESS: Started shipping of resource 'pvc-58af89d6-a7fc-43e1-92dd-861c8835115f' SUCCESS: Started shipping of resource 'pvc-58af89d6-a7fc-43e1-92dd-861c8835115f' SUCCESS: Started shipping of resource 'pvc-58af89d6-a7fc-43e1-92dd-861c8835115f' jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 snapshot list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ SnapshotName ┊ NodeNames ┊ Volumes ┊ CreatedOn ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:16:09 ┊ Shipping ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241003_173503 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 14:35:04 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:07:50 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120905 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:09:06 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 15:12:57 ┊ Successful ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup create linbit-velero-backup pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' for snapshot SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' after snapshot INFO: Generated snapshot name for backup of resourcepvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup INFO: Shipping of resource pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup in progress. jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup create linbit-velero-backup pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' for snapshot SUCCESS: Suspended IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' for snapshot SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' SUCCESS: Took snapshot of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-vtest-k8s01-worker02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen02' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen03' after snapshot SUCCESS: Resumed IO of '[pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d]' on 'ovbh-pprod-xen01' after snapshot INFO: Generated snapshot name for backup of resourcepvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup INFO: Shipping of resource pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d to remote linbit-velero-backup in progress. jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ┊ ┊ Shipping ┊ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ 2024-10-07 09:33:07 ┊ ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ 2024-10-07 09:33:08 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241003_181256^snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120905 ┊ 2024-10-07 09:33:10 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_120749 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122517 ┊ 2024-10-07 09:33:11 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_120905 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122812 ┊ 2024-10-07 09:33:14 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_122517 ┊ Success ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 backup list linbit-velero-backup ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Resource ┊ Snapshot ┊ Finished at ┊ Based On ┊ Status ┊ ╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ┊ ┊ Shipping ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ 2024-10-07 09:33:07 ┊ ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ 2024-10-07 09:33:08 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241003_181256^snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120905 ┊ 2024-10-07 09:33:10 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_120749 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122517 ┊ 2024-10-07 09:33:11 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_120905 ┊ Success ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122812 ┊ 2024-10-07 09:33:14 ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d_back_20241007_122517 ┊ Success ┊ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ jonathon@jonathon-framework:~$ linstor --controller 10.2.0.10 snapshot list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ SnapshotName ┊ NodeNames ┊ Volumes ┊ CreatedOn ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ pvc-58af89d6-a7fc-43e1-92dd-861c8835115f ┊ back_20241007_121608 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:16:09 ┊ Shipping ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241003_173503 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 14:35:04 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120749 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:07:50 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_120905 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:09:06 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122517 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:25:18 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ back_20241007_122812 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-07 09:28:12 ┊ Successful ┊ ┊ pvc-7746af6f-d37e-4c5d-9f44-9616f2f9b33d ┊ snapshot-e21c614a-d8cf-493c-b95f-eca30bd24a92 ┊ ovbh-pprod-xen01, ovbh-pprod-xen02, ovbh-pprod-xen03 ┊ 0: 50 GiB ┊ 2024-10-03 15:12:57 ┊ Successful ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯