@stormi

I sent a PULL request to the fork source.
https://github.com/xapi-project/sm/pull/576

a573net opened this pull request in xapi-project/sm

closed QNAP product attribute is incorrect #576

I noticed that sr could not unplug after the NBD backup failed.

I'll blurt out the solution. There may be others out there, but I hope it helps someone.
I assume this is because when an NBD backup makes a vbd connection and exits with an error, the vbd connection is not broken.

# xe pbd-unplug uuid=79a53996-60d7-8fc0-1f8b-ba1e30b5d0ce
This operation cannot be performed because this VDI is in use by some other operation
vdi: 41b7ff1a-05be-43db-a592-849f9ab30444 (***************)
operation: 071e6f0b-9670-44b6-b455-b8d1319da28b (***************)

# xe vdi-param-list uuid=071e6f0b-9670-44b6-b455-b8d1319da28b | grep vbd-uuids
               vbd-uuids (SRO): 61a50e07-51c4-3476-d47c-45e756f127fe

Unplug the displayed vbd-uuids.

# xe vbd-unplug uuid=61a50e07-51c4-3476-d47c-45e756f127fe

@stormi, @olivierlambert

Thanks to everyone, I was able to complete it.
Thank you very much!!

Hello,

The QNAP setting in multipath.conf is incorrect. The settings cannot be called correctly because the product attribute is different. It seems to work with default values.

I will feed back my experience to XCP-NG project.

product attribute - [Underbar is incorrect. Space is correct]

/etc/multipath.xenserver/multipath.conf.old

device {
                vendor                  "QNAP"
                product                 "iSCSI_Storage"
        }

/etc/multipath.xenserver/multipath.conf

device {
                vendor                  "QNAP"
                product                 "iSCSI Storage"
        }

I was editing multipath.conf to give priority to multipathing sessions.
I started debugging because the settings didn't apply. You have noticed a difference in the call string "iSCSI Storage".

[16:30 xxxxxxxxx etc]# multipath -ll
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx dm-0 QNAP    ,iSCSI Storage   
size=20T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=9 status=active
| `- 9:0:0:0  sdb 8:16 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 10:0:0:0 sdc 8:32 active ready running

By replacing the underscore with a space, the setting is now recognized correctly and the session can be given priority.

device {
                vendor                  "QNAP"
                product                 "iSCSI Storage"
                path_grouping_policy    "failover"
                path_selector           "round-robin 0"
                path_checker            readsector0
                prio                    weightedpath
                prio_args               "hbtl 9:0:0:0 9 10:0:0:0 1"
                uid_attribute           ID_SERIAL
        }

My Storage: TS-432PXU-RP
Version: 5.0.0.1828 Build 20211020

I noticed that sr could not unplug after the NBD backup failed.

I'll blurt out the solution. There may be others out there, but I hope it helps someone.
I assume this is because when an NBD backup makes a vbd connection and exits with an error, the vbd connection is not broken.

# xe pbd-unplug uuid=79a53996-60d7-8fc0-1f8b-ba1e30b5d0ce
This operation cannot be performed because this VDI is in use by some other operation
vdi: 41b7ff1a-05be-43db-a592-849f9ab30444 (***************)
operation: 071e6f0b-9670-44b6-b455-b8d1319da28b (***************)

# xe vdi-param-list uuid=071e6f0b-9670-44b6-b455-b8d1319da28b | grep vbd-uuids
               vbd-uuids (SRO): 61a50e07-51c4-3476-d47c-45e756f127fe

Unplug the displayed vbd-uuids.

# xe vbd-unplug uuid=61a50e07-51c4-3476-d47c-45e756f127fe

I did a cloud backup of 52 VMs from XO to Backlaze.
The number of files in the cloud reached about 620,000 files and the capacity reached 580GB.
After the initial upload was complete, about 30GB/day of Delta was sent.

The bill in August was $5.65.
The XO is likely to provide cost-effective backup.

I hope this helps you.

@olivierlambert said in Change the size of the backup file:

Thank you olivierlambert and florent.

I'm experimenting with direct backups from XO to Backblaze.
I will report the results of the experiment again.

@olivierlambert said in Change the size of the backup file:

My problem is that I get too many "api list file names called" calls to Backblaze.

My structure is a little complicated. Incremental backup from XO to QNAP. Using QNAP's cloud sync function, I'm slowly backing up my XO backup files to Backblaze.

The my goal was to get the backup done fast locally, and the cloud transfer done slowly with limited internet bandwidth.

At this time, we noticed that the number of file list calls was abnormally high and Backblaze charges a lot during the cloud transfer.

I think this is because QNAP's cloud sync function keeps checking the number of files on the cloud side. I thought it would be solved by changing the block size that XO outputs.

If you back up to the cloud using an S3-compatible backup as a remote server on the XO, does that significantly reduce the number of file references? I haven't experimented yet.

Hi ALL,

How do I change the size of the XO backup file?

I want to increase the size of the backup file that XO creates from 2MB to about 200MB.

I back up from XO to QNAP. Replicating from QNAP to cloud at low frequency.
Because of the large number of files, the number of API calls is wasted. The purpose is to increase the file size and reduce the number of files.

ls -plah fe96dbe9-0d78-50a9-4624-7ea0f9ccb1aa/vdis/19217d2e-d234-4f2c-85c1-ecf49cb2389f/045c97cf-ab7f-4915-ab3e-a79165674dbc/data/83945dbd-9580-4d01-92c6-a1cf6e12891a.vhd/blocks/0
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 649
-rw-r--r--  1 admin administrators 1.8K 2024-06-19 22:54 65
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 650
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 651
-rw-r--r--  1 admin administrators 2.0M 2024-06-19 22:54 652
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 653
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 654
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 655
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 656
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 657
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 658
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 659
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 660
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 661
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 662
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 663
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 664
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 665
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 666
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 667
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 668
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 669
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 670
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 671
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 672

@stormi, @olivierlambert

Thanks to everyone, I was able to complete it.
Thank you very much!!

@stormi

I sent a PULL request to the fork source.
https://github.com/xapi-project/sm/pull/576

a573net opened this pull request in xapi-project/sm

closed QNAP product attribute is incorrect #576

I wrote two blocks of underscores and spaces in my QNAP environment.
A forward match was detected and two blocks were read as separate configurations.
This method lists both old and new, so I don't think there will be any problems if old QNAPs use underscores.

I will try to contact github.

I don't know if the QNAP specifications have changed.
I searched past information.

I found another post of 2019 year. In this post, it was "iSCSI Storage". (Spaced)
https://www.reddit.com/r/qnap/comments/fv514j/iscsi_vendor_and_product_for_multipathd_config/

I found official QNAP report of 2018 year. it was "iSCSI Storage". (Spaced)
https://www.qnap.com/ja-jp/how-to/tutorial/article/configuring-linux-iscsi-storage-with-qnap-es-nas

I found Citrix forum of 2018 year. it was "iSCSI Storage". (Spaced)
https://discussions.citrix.com/topic/394780-how-to-use-iscsi-target-directly-in-xenserver/page/2/

It's not smart, but you might be able to follow both old and new by listing two blocks of underbars and spaces. In my debugging, I made a distinction between underbars and spaces, so I thought it would be solved by writing two blocks.

I put two blocks in my server's multipath.conf, check it and reply. Please give me some time.
Are there any commands or logs you would like me to check?

Hello,

The QNAP setting in multipath.conf is incorrect. The settings cannot be called correctly because the product attribute is different. It seems to work with default values.

I will feed back my experience to XCP-NG project.

product attribute - [Underbar is incorrect. Space is correct]

/etc/multipath.xenserver/multipath.conf.old

device {
                vendor                  "QNAP"
                product                 "iSCSI_Storage"
        }

/etc/multipath.xenserver/multipath.conf

device {
                vendor                  "QNAP"
                product                 "iSCSI Storage"
        }

I was editing multipath.conf to give priority to multipathing sessions.
I started debugging because the settings didn't apply. You have noticed a difference in the call string "iSCSI Storage".

[16:30 xxxxxxxxx etc]# multipath -ll
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx dm-0 QNAP    ,iSCSI Storage   
size=20T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=9 status=active
| `- 9:0:0:0  sdb 8:16 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 10:0:0:0 sdc 8:32 active ready running

By replacing the underscore with a space, the setting is now recognized correctly and the session can be given priority.

device {
                vendor                  "QNAP"
                product                 "iSCSI Storage"
                path_grouping_policy    "failover"
                path_selector           "round-robin 0"
                path_checker            readsector0
                prio                    weightedpath
                prio_args               "hbtl 9:0:0:0 9 10:0:0:0 1"
                uid_attribute           ID_SERIAL
        }

My Storage: TS-432PXU-RP
Version: 5.0.0.1828 Build 20211020