NBD setup - No such item

[florent]

@Tristis-Oris I think it is built in in the xapi .
ss -tulpn on my host :

there is a xapi-nbd service

[Tristis Oris]

@florent looks fine.

also i add extra iptables rules, nothing changed.

iptables -A INPUT -p tcp --dport 10809 -m comment --comment "NBD" -j ACCEPT
iptables -A OUTPUT -p tcp --dport 10809 -m comment --comment "NBD" -j ACCEPT

[florent]

@Tristis-Oris my iptable are

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N RH-Firewall-1-INPUT
-N xapi_nbd_input_chain
-N xapi_nbd_output_chain
-A INPUT -p tcp -m tcp --dport 10809 -j xapi_nbd_input_chain
-A INPUT -p gre -j ACCEPT
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A OUTPUT -p tcp -m tcp --sport 10809 -j xapi_nbd_output_chain
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -i xenapi -p udp -m udp --dport 67 -j ACCEPT
-A RH-Firewall-1-INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m conntrack --ctstate NEW -m udp --dport 694 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 21064 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m multiport --dports 5404,5405 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
-A xapi_nbd_input_chain -i xenbr0 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A xapi_nbd_input_chain -j REJECT --reject-with icmp-port-unreachable
-A xapi_nbd_output_chain -o xenbr0 -j RETURN
-A xapi_nbd_output_chain -j REJECT --reject-with icmp-port-unreachable

there is already a line for nbd

[Tristis Oris]

@florent everything is default except NBD rule. looks similar.

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
   16   960 xapi_nbd_input_chain  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:10809
    0     0 ACCEPT     47   --  *      *       0.0.0.0/0            0.0.0.0/0
9872M  155T RH-Firewall-1-INPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:10809 /* NBD */
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:10809 /* NBD */

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RH-Firewall-1-INPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 22M packets, 34G bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 xapi_nbd_output_chain  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp spt:10809
    4   240 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:10809 /* NBD */
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:10809 /* NBD */

Chain RH-Firewall-1-INPUT (2 references)
 pkts bytes target     prot opt in     out     source               destination
 341M 9622G ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
   29  5104 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 255
    0     0 ACCEPT     udp  --  xenapi *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
9530M  146T ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW udp dpt:694
    1    52 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW tcp dpt:22
65232 3914K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW tcp dpt:80
 488K   29M ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW tcp dpt:443
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:21064
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 5404,5405
 588K 4800M REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain xapi_nbd_input_chain (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  xenbr0 *       0.0.0.0/0            0.0.0.0/0            ctstate NEW,ESTABLISHED
   16   960 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      xenbr0  0.0.0.0/0            0.0.0.0/0
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

[Tristis Oris]

after telnet request to 10809, rejected packages increased.

Chain xapi_nbd_input_chain (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  xenbr0 *       0.0.0.0/0            0.0.0.0/0            ctstate NEW,ESTABLISHED
   17  1020 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

[Tristis Oris]

if i have no errors during backup, it means XO didn't try to use NBD?

but at begining of each NBD backup i got +1 to iptables rejected list.

[florent]

@Tristis-Oris said in NBD setup - No such item:

but at begining of each NBD backup i got +1 to iptables rejected list.

I think there is something with the network configuration, but I am far out of my element here .

[Tristis Oris]

@florent is it any requirements for network\switch etc. Any layers outside of Xen.

[florent]

@Tristis-Oris it hae to let the encrypted traffic flow through the 10809 port. XO will connect directly to the hosts through NBD, not only to the master

[Tristis Oris]

@florent sounds pretty simple. Nothing at my network should block that traffic.

Anyway, if i got rejected packets at iptables - backup task try to connect NBD server. Then it can't do something, so i should get an error at XO, but it not happens.

[Tristis Oris]

No such item task happens when host interface at status none without IP.

[Tristis Oris]

@olivierlambert i hope your vacation WAS good, be we are stuck a bit here

[Tristis Oris]

bump

[olivierlambert]

Re Ping @florent or @julien-f

[Tristis Oris]

once again i configured it on fresh, test cluster, and it working.

so it some problem with iptables on production. But it default, same 8.2.1 clean installation.

Both this hosts have only 1 connected link, so it both backup and management.
On prod i try empty default backup network, or NBD link network - no effect. So maybe it still going through managemnt link without NBD?

aaaaand enabled NBD on Mng link - now it working. SO yes, it ignore specified backup link.
Or this option didn't work, because NBD now enabled only on Mng link, so it can't go through backup link.

[Tristis Oris]

next weird thing.
i enabled NBD on all manage interfaces at all pools.

it working with CR backup, but shouldn't, at least because it no option to enable NBD.

it working with half of other delta backup tasks, but not for all. Main backup is still without nbd.

only idea, it probably because of 1 small pool with few vm, where i forget to enable NBD. so 1 VM without nbd, force it to work in usual mode?

[florent]

@Tristis-Oris said in NBD setup - No such item:

next weird thing.
i enabled NBD on all manage interfaces at all pools.

it working with CR backup, but shouldn't, at least because it no option to enable NBD.

it working with half of other delta backup tasks, but not for all. Main backup is still without nbd.

only idea, it probably because of 1 small pool with few vm, where i forget to enable NBD. so 1 VM without nbd, force it to work in usual mode?

that is really puzzling. I don't forget this issue, but I don't have much idea of how to fix this

Did you add the preferNbd flag to the config files ?

[Tristis Oris]

@florent lets a brief summary.

nbd work only via management interface. If nbd enabled on any another interface and specified as pool backup interface, it don't work (the issue with blocked packets at iptables). That was a main reason of my problem.

For now i made it work at all my pools. Max single thread speed increased 80-90 > 140-150. At usual backup task (when daily delta very small) it show same 10-40Mb, because it done too fast. Average task time decreased fo about twice.

[florent]

@Tristis-Oris said in NBD setup - No such item:

@florent lets a brief summary.

nbd work only via management interface. If nbd enabled on any another interface and specified as pool backup interface, it don't work (the issue with blocked packets at iptables). That was a main reason of my problem.

For now i made it work at all my pools. Max single thread speed increased 80-90 > 140-150. At usual backup task (when daily delta very small) it show same 10-40Mb, because it done too fast. Average task time decreased fo about twice.

That is really strange, but it is also useful. We'll try to see why , and update the doc accordingly

[Tristis Oris]

@florent also it works too good.
i don' used advanced mode for CR backup, so NBD is disabled for job.

but task is working with nbd)