Understood, I'm adding it in our internal "Product improvement/whishlist" 🙂

I have no idea for the legacy Linux tools, they are badly coded and you can get rid of them entirely We've been using them in prod since almost a year without any issue. 1.0 will come soon but it's already very stable and safe (it's written in Rust). Our goal is to get them in all Linux distro upstream ASAP.

@plaidypus Ah gotcha, this makes sense.

I second scaling out instead of up.

If you're getting new hosts, I'd also keep in mind newer CPUs do have much higher per core performance (not sure what your current stuff is), so you also might be able to get away with less vCPUs and lower likelihood of NUMA spanning.

Either way though I think scaling out is the better direction to go.

@Danp

Deleted, thank you. I still have 27 VDIs waiting to coalesce:

Jan 2 13:15:27 svr10533 SM: [2342589] lock: opening lock file /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/sr Jan 2 13:15:27 svr10533 SM: [2342589] Failed to lock /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/sr on first attempt, blocked by PID 2340106 Jan 2 13:15:40 svr10533 SM: [2340106] pread SUCCESS Jan 2 13:15:40 svr10533 SMGC: [2340106] SR eb90 ('BCDR ZFS') (212 VDIs in 17 VHD trees): Jan 2 13:15:40 svr10533 SMGC: [2340106] *d392e41a(450.000G/335.775G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5368079d(450.000G/7.028G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *bb4dd6c9(450.000G/1.320G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *4150078f(450.000G/1.473G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c893141c(450.000G/3.060G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5091ac53(450.000G/2.279G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *b498e3b2(450.000G/1.026G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *72d80edb(450.000G/2.318G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *54939254(450.000G/2.011G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] cc25225e(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *1669002a(450.000G/4.676G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *3d9e0d20(450.000G/1.443G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0bd09d16(450.000G/4.287G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 6c2fd471(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *ff3f6743(450.000G/8.050G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f771c50e(450.000G/2.521G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *2fd6e0ca(450.000G/9.193G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *372ed817(450.000G/3.230G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 91f4def4(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8da33343(450.000G/3.698G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 535118ad(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *fdf924c8(450.000G/2.314G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2cc902a0(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *2f93e3d9(450.000G/2.478G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d1565001(450.000G/4.572G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0df3c67f(450.000G/6.324G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *25aabe84(450.000G/5.118G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 97458549(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *54dced10(450.000G/5.351G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 4356441e(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *bf0bde01(450.000G/11.944G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0192ca17(450.000G/4.881G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 3d57d327(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *9680bc97(450.000G/3.140G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] d8212c94(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c39b8fa1(450.000G/3.116G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] a0f49273(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *b0427242(450.000G/3.504G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *dedf7172(450.000G/3.426G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 29fad3de(450.000G/3.109G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 64569ddc(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] d979c709(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 4447de43(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] c56631bc(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 788f3c92(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2c99371b(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 91d990f1(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 4245abae(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] bbad84cd(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] b1c74906(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 7e4bbd4e(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 601eea79(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0ba17bd5(150.205G/108.311G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *caa4bd30(200.000G/200.113G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] f3676cfa(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5e463b47(200.000G/8.948G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d193286e(200.000G/10.261G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 8a88e7e0(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8245abc8(200.000G/2.670G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 11317ba9(200.000G/10.284G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 35688176(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 46f10e3a(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *9f82a5e9(450.000G/330.550G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2f032201(450.000G/16.224G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *6568c0e7(400.000G/260.262G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] bf1d1461(450.000G/43.394G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f3a1151d(200.000G/200.113G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 5a67a45c(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 89787746(200.000G/14.133G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *27071e21(200.000G/200.113G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *77f76017(200.000G/35.576G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d25f5bd0(200.000G/200.113G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 5a92681c(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *bc94ee2d(200.000G/4.488G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *85354126(200.000G/783.934M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *54feec66(200.000G/757.883M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *dfce1ec6(200.000G/830.024M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *2d62272e(200.000G/745.860M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *20d8dcd4(200.000G/818.000M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *1a31cfc0(200.000G/914.188M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *33fa92c4(200.000G/787.942M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *837e1716(200.000G/948.254M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *9e7ced06(200.000G/769.907M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *be5b966d(200.000G/3.564G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 98da96e0(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *af859d6b(200.000G/12.576G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *bd8ddcf9(200.000G/9.419G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] c869d924(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *500d2ba3(200.000G/3.272G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] dd17434b(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5144ffb1(200.000G/2.973G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *dbac8b21(200.000G/3.139G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *ad5c7f4b(200.000G/2.509G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 18714fd7(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *cc162d59(200.000G/1.777G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 379eef00(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8c765d96(200.000G/1.513G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 36aa88b0(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *93f7e854(200.000G/8.801G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *6a9e4fc9(200.000G/1.370G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 89362443(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *e59a2438(200.000G/5.175G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *47501359(200.000G/7.715G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *7b3bf3ad(200.000G/4.092G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 5a907525(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *b493cbf7(200.000G/1.638G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *17101b86(200.000G/2.368G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f79302fc(200.000G/1.210G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] dcb62c95(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *929d416a(200.000G/1.936G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2a003ce4(200.000G/1.838G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 46b709d1(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] d160100a(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 3690a309(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 25c2e5d2(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] cc0fbcc1(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] a33614b4(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] e73c0abd(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 91e85aef(200.000G/416.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *ab1b3785(450.000G/335.775G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 08f300a1(450.000G/2.222G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *6f311515(400.000G/260.262G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] be6ea092(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *56ff007e(400.000G/86.977M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *930dca33(400.000G/56.918M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *b32a99bc(400.000G/76.958M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8a8a7702(400.000G/64.934M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0aa6aba1(400.000G/78.961M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *036ea895(400.000G/58.922M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *a2d9ad98(400.000G/131.063M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c22844ef(400.000G/86.977M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *04859261(400.000G/66.938M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] a5e0f92c(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8e95d466(400.000G/48.903M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] cdb5b504(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d7bbeab7(400.000G/44.895M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 981b347b(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *7a406ecd(400.000G/82.969M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] b5447aa6(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *6a1e21ab(400.000G/92.989M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 604f9159(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *4f2fbec0(400.000G/103.008M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *1be5b6dc(400.000G/121.043M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0b817a36(400.000G/103.008M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *161692ae(400.000G/76.958M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2dbe1d72(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5558085b(400.000G/66.938M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] f4b3686f(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *a55fda20(400.000G/107.016M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 1d3810b0(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *8b84a288(400.000G/137.075M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 10464d76(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *956e4520(400.000G/66.938M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 389a4b09(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f167ab44(400.000G/26.860M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *e9cf3262(400.000G/68.942M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *806d24c8(400.000G/28.864M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *89195b87(400.000G/30.868M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] f2895634(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *3b172a66(400.000G/34.875M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 6181d25a(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *5f7a446e(400.000G/32.872M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] ad06d915(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *6e61eb74(400.000G/26.860M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 651f15e3(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 07f380f4(400.000G/22.852M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 935c5c1c(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 795db4de(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 99c02ee7(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] cba8403f(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 72668087(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] c5a4e60d(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 86c9fa8e(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *904fb0d1(150.205G/108.311G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *29644a15(150.205G/10.568G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *4b567dde(150.205G/509.297M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *e6e84427(150.205G/517.313M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *43272c07(150.205G/7.380G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *04f46494(150.205G/7.092G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f99e0c83(150.205G/611.497M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *cce513da(150.205G/409.102M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *fcc700fb(150.205G/483.247M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *66fbe925(150.205G/537.352M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *69045ab5(150.205G/898.055M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *23a8167a(150.205G/892.043M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c5ce038a(150.205G/13.354G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f65b7f44(150.205G/791.848M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *11d2d98e(150.205G/567.411M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *893cc829(150.205G/641.555M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c61ec70f(150.205G/555.387M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *227f1d6f(150.205G/585.446M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d02a0ed2(150.205G/547.372M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *23b812d3(150.205G/849.961M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *c4e42d7d(150.205G/495.270M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *a7eda3b5(150.205G/625.524M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *dabc2ec9(150.205G/1.881G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *f750d332(150.205G/11.270G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d961c364(150.205G/619.512M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *d83d65ef(150.205G/741.750M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *25b3992d(150.205G/880.020M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *0df362d8(150.205G/509.297M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *4cb855af(150.205G/533.344M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 9b87ab70(400.000G/260.262G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *28213209(400.000G/260.262G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 047c4ec7(400.000G/829.000K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 308fa0ad(400.000G/147.094M?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 7375989a(400.000G/41.570G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *e7b59332(450.000G/335.775G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 2bb12a6d(450.000G/932.500K?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 17474198(450.000G/12.809G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] *536e0771(200.000G/200.113G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] 4ac70c28(200.000G/9.149G?) Jan 2 13:15:40 svr10533 SMGC: [2340106] Jan 2 13:15:40 svr10533 SMGC: [2340106] *** UNDO LEAF-COALESCE Jan 2 13:15:40 svr10533 SM: [2340106] lock: released /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/sr Jan 2 13:15:40 svr10533 SMGC: [2340106] GC process exiting, no work left Jan 2 13:15:40 svr10533 SM: [2342589] lock: acquired /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/sr Jan 2 13:15:40 svr10533 SM: [2342589] sr_scan {'host_ref': 'OpaqueRef:3e26fadd-48c8-2900-fd65-34780cd7b46b', 'command': 'sr_scan', 'args': [], 'device_config': {'SRmaster': 'true', 'location': '/mnt/zfs'}, 'session_ref': 'OpaqueRef:954fba5b-ff7d-a11d-0383-2608f998c88f', 'sr_ref': 'OpaqueRef:8d501aaf-3f70-ea99-946b-eb8fed144c66', 'sr_uuid': 'eb9076c3-13a5-f9a7-43e3-ff1146e5599d', 'subtask_of': 'DummyRef:|ceb29e66-df6b-a75b-b210-dc331b19edb5|SR.scan'} Jan 2 13:15:40 svr10533 SM: [2342589] ['findmnt', '-o', 'FSTYPE', '-n', '/mnt/zfs'] Jan 2 13:15:40 svr10533 SM: [2340106] lock: released /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/gc_active Jan 2 13:15:40 svr10533 SMGC: [2340106] In cleanup Jan 2 13:15:40 svr10533 SMGC: [2340106] SR eb90 ('BCDR ZFS') (212 VDIs in 17 VHD trees): no changes Jan 2 13:15:40 svr10533 SM: [2342589] pread SUCCESS Jan 2 13:15:40 svr10533 SM: [2342589] ['/usr/bin/vhd-util', 'scan', '-f', '-m', '/var/run/sr-mount/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/*.vhd'] Jan 2 13:15:40 svr10533 SM: [2340106] lock: closed /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/running Jan 2 13:15:40 svr10533 SM: [2340106] lock: closed /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/gc_active Jan 2 13:15:40 svr10533 SM: [2340106] lock: closed /var/lock/sm/eb9076c3-13a5-f9a7-43e3-ff1146e5599d/sr

@jasonnix Yes, a vCPU means a virtual CPU, which is the assignment of a VM to a physical CPU core.
Servers have sockets that contain physical CPUs, so it sounds like your system has four sockets, holding four physical CPUs.
Each physical CPU can have multiple cores and in some cases, one thread per core or in others, two threads per core, but let's stick
to the simpler case here.
A configuration of 4 cores with 1 core per socket means each of the 4 vCPUs will reside on a core on four separate physical CPU sockets,
so all four physical CPUs are accessed. This is in most cases not ldeal as in many servers with 4 physical CPUs, the memory banks are split between pairs of CPUs,
two on one bank, two on the other. Having VMs cross over physical CPU memory bank boundaries is generally inefficient and should
be avoided if possible. This is why NUMA (Non-Uniform Memory Access) and vNUMA become important in the configuration.
And @gskger is correct that licensing can sometimes depend on the configuration.
I should add that under some circumstances, configuring the CPUs for turbo mode can be an advantage.
Suggested reading: my article on the effects of vCPU and GPU allocations in a three-part set of articles. In particular, Part 3 addresses
NUMA and configuration issues and Part 2 discusses turbo mode.
I hope this helps as it is initially quite confusing.
https://community.citrix.com/citrix-community-articles/a-tale-of-two-servers-how-bios-settings-can-affect-your-apps-and-gpu-performance/
https://community.citrix.com/citrix-community-articles/a-tale-of-two-servers-part-2-how-not-only-bios-settings-but-also-gpu-settings-can-affect-your-apps-and-gpu-performance/
https://community.citrix.com/citrix-community-articles/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/

Posted this as I personally found this configuration quite involved, and the permissions earlier in the thread were insufficient to make it work when using AWS KMS for bucket encryption as well as the XO provided encryption secret.

@danieeljosee If your XCP-ng pool is set up correctly, live migration with Xen Orchestra works straight away. Compatibility is key in a pool (CPU manufacturer, CPU architecture, XCP-ng version, network cards/ports). There is a lot of leeway for CPU architecture, but I would recommend not mixing new and older CPU generations.

For the network settings, it depends on your network architecture. You can run all network traffic via one NIC or you can separate the network traffic from management, VM migration, VM and/or storage/backup using more NICs. In a homelab, one NIC is usually sufficient.

While shared storage is very helpful for VM live migration, you can also live migrate the VM with its disks attached on local storage. Just select the destination host and destination storage repository. For local storage, VMs and attached disks must of course be located on the same destination host.

Check out Tom's YT channel as @ph7 suggested.

@olivierlambert If you want me to check anything else, just tell me and I will test for you.

Many Thanks

BR
Marco

@olivierlambert Makes sense! I would schedule the replication to occur a couple hours after the backup runs are complete to ensure its a replica of all data and not a partial replica!

@olivierlambert Will do... that VM is running a massive rsync job at the moment, but once it's finished I'll be able to shut it down.

IIRC, just remove it (small cross) so it should use something default

Q1. Yes. IB & IR share the same principle.
Q2. Key backups are a plus to be on the safe side, but it's not the only way to do it. You also have backup health check, as you suggested. To me, it depends on the criticality of your data. To me, health check after 20 IR isn't bad but it's really a "personal" choice 🙂

@mattnj

Thanks for the update, I didn't realize you could select the kernel in the ipxe boot, this might help me down the road.

The workaround I have found at this point is to rebuild grub after migrating to XCP-NG. Adds a few minutes to each VM but I don't have that many.

Boot to live iso in UEFI mode (I used Debian12 given it was a Debian12 vm)

sudo su - mount /dev/mapper/????? /mnt mount /dev/xvda2 /mnt/boot mount /dev/xvda1 /mnt/boot/efi for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done mount --bind /sys/firmware/efi/efivars /mnt/sys/firmware/efi/efivars chroot /mnt grub-install /dev/xvda update-grub exit shutdown now

I'm using LVM so adjust the first mount to match your root partition.
I shutdown so that I can remove iso and boot when I'm ready.

Hope this saves somebody a bunch of frustration in the future.

@hansve You're welcome.

Hi,

IIRC, the scheduler is smart, if it's using the same time, it will reuse snapshots as possible. @florent could give more details when he's around 🙂

I am running the latest XO. I ended up fixing the issue, it has backed up successfully 2 times now without any errors. What I did was removed ALL snapshots for the specific VM. Once the tasks finished for removing the snapshots, I created a new manual snapshot. Then let the backup happen as normal. What I noticed though is the backup snapshot replaced my manual snapshot so the VM still only has 1 snapshot. But it seems to be working now.

If you are that much confident, then feel free to purchase one and use it: XCP-ng supports SRIOV but for NICs, I don't know how different it could be for a GPU, but without any hardware to test it's impossible to answer. How difficult it was to work on a more simpler hypervisor (KVM) doesn't send encouraging messages though.