Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046388] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: PingAck did not arrive in time. Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046406] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046411] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-2: pdsk( Diskless -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046527] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Preparing cluster-wide state change 1177275017: 0->all empty Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046656] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:02:40 xencc-hp03 kernel: [1010131.046693] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.078920] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:02:40 xencc-hp03 kernel: [1010131.078929] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:40 xencc-hp03 kernel: [1010131.083910] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:40 xencc-hp03 kernel: [1010131.083951] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:40 xencc-hp03 kernel: [1010131.083962] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Restarting receiver thread Feb 6 23:02:40 xencc-hp03 kernel: [1010131.536607] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Rejecting concurrent remote state change 1094984595 because of state change 1177275017 Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558393] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: PingAck did not arrive in time. Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558407] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558410] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005: quorum( yes -> no ) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558412] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: pdsk( UpToDate -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558441] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Retrying cluster-wide state change after 0ms Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558771] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Terminating sender thread Feb 6 23:02:40 xencc-hp03 kernel: [1010131.558803] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.559005] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005: helper command: /sbin/drbdadm quorum-lost Feb 6 23:02:40 xencc-hp03 kernel: [1010131.562533] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Preparing cluster-wide state change 1511458820: 0->all empty Feb 6 23:02:40 xencc-hp03 kernel: [1010131.562535] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Committing cluster-wide state change 1511458820 (0ms) Feb 6 23:02:40 xencc-hp03 kernel: [1010131.562622] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:40 xencc-hp03 kernel: [1010131.563548] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005: helper command: /sbin/drbdadm quorum-lost exit code 0 Feb 6 23:02:40 xencc-hp03 kernel: [1010131.594897] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Connection closed Feb 6 23:02:40 xencc-hp03 kernel: [1010131.594905] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:40 xencc-hp03 kernel: [1010131.600457] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:40 xencc-hp03 kernel: [1010131.600496] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:40 xencc-hp03 kernel: [1010131.600501] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Restarting receiver thread Feb 6 23:02:40 xencc-hp03 kernel: [1010131.600505] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350408] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: PingAck did not arrive in time. Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350426] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350430] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-2: pdsk( Diskless -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350462] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 2231655647: 0->all empty Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350657] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:02:42 xencc-hp03 kernel: [1010133.350692] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.390957] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:02:42 xencc-hp03 kernel: [1010133.390966] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:42 xencc-hp03 kernel: [1010133.395107] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:42 xencc-hp03 kernel: [1010133.395147] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:42 xencc-hp03 kernel: [1010133.395157] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Restarting receiver thread Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866465] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: PingAck did not arrive in time. Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866482] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866485] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004: quorum( yes -> no ) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866487] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: pdsk( UpToDate -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866518] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Retrying cluster-wide state change after 0ms Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866814] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:02:42 xencc-hp03 kernel: [1010133.866855] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.867199] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004: helper command: /sbin/drbdadm quorum-lost Feb 6 23:02:42 xencc-hp03 kernel: [1010133.871663] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004: helper command: /sbin/drbdadm quorum-lost exit code 0 Feb 6 23:02:42 xencc-hp03 kernel: [1010133.874443] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 2962952287: 0->all empty Feb 6 23:02:42 xencc-hp03 kernel: [1010133.874446] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Committing cluster-wide state change 2962952287 (0ms) Feb 6 23:02:42 xencc-hp03 kernel: [1010133.874495] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:42 xencc-hp03 kernel: [1010133.907174] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:02:42 xencc-hp03 kernel: [1010133.907182] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:42 xencc-hp03 kernel: [1010133.911582] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:42 xencc-hp03 kernel: [1010133.911619] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:42 xencc-hp03 kernel: [1010133.911625] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Restarting receiver thread Feb 6 23:02:42 xencc-hp03 kernel: [1010133.911629] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:43 xencc-hp03 kernel: [1010134.794582] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:02:43 xencc-hp03 kernel: [1010134.794584] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614478] drbd xcp-persistent-database xenrt-1: sock_recvmsg returned -110 Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614488] drbd xcp-persistent-database/0 drbd1000: Would lose quorum, but using tiebreaker logic to keep Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614496] drbd xcp-persistent-database xenrt-1: conn( Connected -> BrokenPipe ) peer( Secondary -> Unknown ) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614498] drbd xcp-persistent-database/0 drbd1000 xenrt-1: pdsk( UpToDate -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614543] drbd xcp-persistent-database: Preparing cluster-wide state change 817396942: 1->all empty Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614763] drbd xcp-persistent-database xenrt-1: Terminating sender thread Feb 6 23:02:45 xencc-hp03 kernel: [1010136.614828] drbd xcp-persistent-database xenrt-1: Starting sender thread (peer-node-id 0) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.646888] drbd xcp-persistent-database xenrt-1: Connection closed Feb 6 23:02:45 xencc-hp03 kernel: [1010136.646897] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:45 xencc-hp03 kernel: [1010136.650956] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:45 xencc-hp03 kernel: [1010136.651037] drbd xcp-persistent-database xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:45 xencc-hp03 kernel: [1010136.651048] drbd xcp-persistent-database xenrt-1: Restarting receiver thread Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806489] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: sock_recvmsg returned -11 Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806511] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806521] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: short read (expected size 16) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806524] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Authentication of peer failed, trying again. Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806633] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:02:45 xencc-hp03 kernel: [1010136.806694] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:45 xencc-hp03 kernel: [1010136.816318] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:02:45 xencc-hp03 kernel: [1010136.816321] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:45 xencc-hp03 kernel: [1010136.842936] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:02:45 xencc-hp03 kernel: [1010136.842945] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:45 xencc-hp03 kernel: [1010136.847397] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:45 xencc-hp03 kernel: [1010136.847436] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:46 xencc-hp03 kernel: [1010137.112047] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:02:46 xencc-hp03 kernel: [1010137.114499] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Preparing cluster-wide state change 2539246606: 0->1 role( Primary ) conn( Connected ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126496] drbd xcp-persistent-database xenrt-2: PingAck did not arrive in time. Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126511] drbd xcp-persistent-database xenrt-2: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126513] drbd xcp-persistent-database/0 drbd1000: quorum( yes -> no ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126515] drbd xcp-persistent-database/0 drbd1000 xenrt-2: pdsk( Diskless -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126592] drbd xcp-persistent-database: Retrying cluster-wide state change after 0ms Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126731] drbd xcp-persistent-database xenrt-2: Terminating sender thread Feb 6 23:02:46 xencc-hp03 kernel: [1010137.126755] drbd xcp-persistent-database xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.127161] drbd xcp-persistent-database/0 drbd1000: helper command: /sbin/drbdadm quorum-lost Feb 6 23:02:46 xencc-hp03 kernel: [1010137.131513] drbd xcp-persistent-database/0 drbd1000: helper command: /sbin/drbdadm quorum-lost exit code 0 Feb 6 23:02:46 xencc-hp03 kernel: [1010137.134457] drbd xcp-persistent-database: Preparing cluster-wide state change 123194053: 1->all empty Feb 6 23:02:46 xencc-hp03 kernel: [1010137.134458] drbd xcp-persistent-database: Committing cluster-wide state change 123194053 (0ms) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.134474] drbd xcp-persistent-database xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:46 xencc-hp03 kernel: [1010137.163024] drbd xcp-persistent-database xenrt-2: Connection closed Feb 6 23:02:46 xencc-hp03 kernel: [1010137.163034] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:46 xencc-hp03 kernel: [1010137.168077] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:46 xencc-hp03 kernel: [1010137.168125] drbd xcp-persistent-database xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:46 xencc-hp03 kernel: [1010137.168133] drbd xcp-persistent-database xenrt-2: Restarting receiver thread Feb 6 23:02:46 xencc-hp03 kernel: [1010137.168148] drbd xcp-persistent-database xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446449] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: PingAck did not arrive in time. Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446462] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: Would lose quorum, but using tiebreaker logic to keep Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446471] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446473] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: pdsk( UpToDate -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446513] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 2473106665: 0->all empty Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446661] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Terminating sender thread Feb 6 23:02:46 xencc-hp03 kernel: [1010137.446692] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:46 xencc-hp03 kernel: [1010137.742384] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: new current UUID: 7EE5A75822D9908B weak: FFFFFFFFFFFFFFFE Feb 6 23:02:46 xencc-hp03 kernel: [1010137.742970] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Connection closed Feb 6 23:02:46 xencc-hp03 kernel: [1010137.742979] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:46 xencc-hp03 kernel: [1010137.747072] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:46 xencc-hp03 kernel: [1010137.747119] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:46 xencc-hp03 kernel: [1010137.747132] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Restarting receiver thread Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474544] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: PingAck did not arrive in time. Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474556] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007: Would lose quorum, but using tiebreaker logic to keep Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474563] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474566] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: pdsk( UpToDate -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474596] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Preparing cluster-wide state change 1458190404: 0->all empty Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474870] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:02:47 xencc-hp03 kernel: [1010138.474917] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:47 xencc-hp03 kernel: [1010138.507200] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:02:47 xencc-hp03 kernel: [1010138.507212] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:47 xencc-hp03 kernel: [1010138.512737] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:47 xencc-hp03 kernel: [1010138.512779] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:47 xencc-hp03 kernel: [1010138.512790] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Restarting receiver thread Feb 6 23:02:48 xencc-hp03 kernel: [1010139.090763] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: State change 1458190404: primary_nodes=0, weak_nodes=0 Feb 6 23:02:48 xencc-hp03 kernel: [1010139.090765] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Committing cluster-wide state change 1458190404 (616ms) Feb 6 23:02:48 xencc-hp03 kernel: [1010139.090818] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:48 xencc-hp03 kernel: [1010139.843567] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:02:48 xencc-hp03 kernel: [1010139.843571] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:49 xencc-hp03 kernel: [1010140.040739] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:02:49 xencc-hp03 kernel: [1010140.040743] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:49 xencc-hp03 kernel: [1010140.139128] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:02:49 xencc-hp03 kernel: [1010140.146496] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 1722619325: 0->2 role( Primary ) conn( Connected ) Feb 6 23:02:49 xencc-hp03 kernel: [1010140.507130] drbd xcp-persistent-database: Preparing cluster-wide state change 3834580557: 1->all role( Secondary ) Feb 6 23:02:49 xencc-hp03 kernel: [1010140.507133] drbd xcp-persistent-database: Committing cluster-wide state change 3834580557 (0ms) Feb 6 23:02:49 xencc-hp03 kernel: [1010140.507137] drbd xcp-persistent-database: role( Primary -> Secondary ) [secondary] Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678489] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> NetworkFailure ) Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678534] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -4 Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678644] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Aborting cluster-wide state change 1722619325 (532ms) rv = -21 Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678648] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Failure to connect: Interrupted state change (-21); retrying Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678653] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:02:49 xencc-hp03 kernel: [1010140.678705] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:49 xencc-hp03 kernel: [1010140.710974] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:02:49 xencc-hp03 kernel: [1010140.710983] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:49 xencc-hp03 kernel: [1010140.715302] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:49 xencc-hp03 kernel: [1010140.715358] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:49 xencc-hp03 kernel: [1010140.715365] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Restarting receiver thread Feb 6 23:02:49 xencc-hp03 kernel: [1010140.715369] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:49 xencc-hp03 kernel: [1010140.726329] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Preparing remote state change 4072674928: 1->all empty Feb 6 23:02:50 xencc-hp03 kernel: [1010141.608833] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:02:50 xencc-hp03 kernel: [1010141.610535] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 2526117818: 0->1 role( Primary ) conn( Connected ) Feb 6 23:02:51 xencc-hp03 kernel: [1010142.051664] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Committing remote state change 4072674928 (primary_nodes=0) Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310515] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: sock_recvmsg returned -11 Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310527] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310704] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Aborting cluster-wide state change 2539246606 (5196ms) rv = -21 Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310708] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Failure to connect: Interrupted state change (-21); retrying Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310712] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Terminating sender thread Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310735] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:51 xencc-hp03 kernel: [1010142.310860] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:51 xencc-hp03 kernel: [1010142.339095] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Connection closed Feb 6 23:02:51 xencc-hp03 kernel: [1010142.339130] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:51 xencc-hp03 kernel: [1010142.343426] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:51 xencc-hp03 kernel: [1010142.343462] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:51 xencc-hp03 kernel: [1010142.343468] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Restarting receiver thread Feb 6 23:02:51 xencc-hp03 kernel: [1010142.343471] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550532] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550546] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550559] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: short read (expected size 8) Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550561] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: timeout while waiting for feature packet Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550680] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:02:53 xencc-hp03 kernel: [1010144.550711] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:53 xencc-hp03 kernel: [1010144.591089] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:02:53 xencc-hp03 kernel: [1010144.591102] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:53 xencc-hp03 kernel: [1010144.595294] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:53 xencc-hp03 kernel: [1010144.595335] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:55 xencc-hp03 kernel: [1010146.184956] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:02:55 xencc-hp03 kernel: [1010146.184959] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662567] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: sock_recvmsg returned -11 Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662578] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662651] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Aborting cluster-wide state change 2526117818 (5052ms) rv = -21 Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662657] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Failure to connect: Interrupted state change (-21); retrying Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662663] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662901] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:55 xencc-hp03 kernel: [1010146.662972] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:55 xencc-hp03 kernel: [1010146.707123] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:02:55 xencc-hp03 kernel: [1010146.707135] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:55 xencc-hp03 kernel: [1010146.711560] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:55 xencc-hp03 kernel: [1010146.711606] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:55 xencc-hp03 kernel: [1010146.711613] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Restarting receiver thread Feb 6 23:02:55 xencc-hp03 kernel: [1010146.711617] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198593] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: sock_recvmsg returned -11 Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198606] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198617] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: short read (expected size 16) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198620] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Authentication of peer failed, trying again. Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198746] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Terminating sender thread Feb 6 23:02:57 xencc-hp03 kernel: [1010148.198780] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.247107] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Connection closed Feb 6 23:02:57 xencc-hp03 kernel: [1010148.247116] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:02:57 xencc-hp03 kernel: [1010148.252628] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:57 xencc-hp03 kernel: [1010148.252667] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454550] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: PingAck did not arrive in time. Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454566] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454568] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: quorum( yes -> no ) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454571] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-2: pdsk( Diskless -> DUnknown ) repl( Established -> Off ) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454598] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Retrying cluster-wide state change after 0ms Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454801] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Terminating sender thread Feb 6 23:02:57 xencc-hp03 kernel: [1010148.454848] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.455047] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: helper command: /sbin/drbdadm quorum-lost Feb 6 23:02:57 xencc-hp03 kernel: [1010148.458594] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 1543818795: 0->all empty Feb 6 23:02:57 xencc-hp03 kernel: [1010148.458601] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Committing cluster-wide state change 1543818795 (0ms) Feb 6 23:02:57 xencc-hp03 kernel: [1010148.458637] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:57 xencc-hp03 kernel: [1010148.459564] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: helper command: /sbin/drbdadm quorum-lost exit code 0 Feb 6 23:02:57 xencc-hp03 kernel: [1010148.503360] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Connection closed Feb 6 23:02:57 xencc-hp03 kernel: [1010148.503369] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:57 xencc-hp03 kernel: [1010148.507566] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:57 xencc-hp03 kernel: [1010148.507608] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:57 xencc-hp03 kernel: [1010148.507615] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Restarting receiver thread Feb 6 23:02:57 xencc-hp03 kernel: [1010148.507618] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:58 xencc-hp03 kernel: [1010149.254598] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:58 xencc-hp03 kernel: [1010149.446669] drbd xcp-persistent-database xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:02:58 xencc-hp03 kernel: [1010149.446677] drbd xcp-persistent-database xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:58 xencc-hp03 kernel: [1010149.742295] drbd xcp-persistent-database xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:02:58 xencc-hp03 kernel: [1010149.742604] drbd xcp-persistent-database: Preparing cluster-wide state change 3267980735: 1->2 role( Secondary ) conn( Connected ) Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246585] drbd xcp-persistent-database xenrt-2: conn( Connecting -> NetworkFailure ) Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246630] drbd xcp-persistent-database xenrt-2: sock_recvmsg returned -4 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246776] drbd xcp-persistent-database: Aborting cluster-wide state change 3267980735 (504ms) rv = -21 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246781] drbd xcp-persistent-database xenrt-2: Failure to connect: Interrupted state change (-21); retrying Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246788] drbd xcp-persistent-database xenrt-2: Terminating sender thread Feb 6 23:02:59 xencc-hp03 kernel: [1010150.246812] drbd xcp-persistent-database xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:59 xencc-hp03 kernel: [1010150.275050] drbd xcp-persistent-database xenrt-2: Connection closed Feb 6 23:02:59 xencc-hp03 kernel: [1010150.275059] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:59 xencc-hp03 kernel: [1010150.278981] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.279025] drbd xcp-persistent-database xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:59 xencc-hp03 kernel: [1010150.279031] drbd xcp-persistent-database xenrt-2: Restarting receiver thread Feb 6 23:02:59 xencc-hp03 kernel: [1010150.279035] drbd xcp-persistent-database xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:02:59 xencc-hp03 kernel: [1010150.467789] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.467792] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:59 xencc-hp03 kernel: [1010150.602695] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.602698] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:02:59 xencc-hp03 kernel: [1010150.763382] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:02:59 xencc-hp03 kernel: [1010150.763499] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:02:59 xencc-hp03 kernel: [1010150.763609] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:02:59 xencc-hp03 kernel: [1010150.763637] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:02:59 xencc-hp03 kernel: [1010150.799073] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:02:59 xencc-hp03 kernel: [1010150.799082] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:02:59 xencc-hp03 kernel: [1010150.803260] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:02:59 xencc-hp03 kernel: [1010150.803313] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:02:59 xencc-hp03 kernel: [1010150.803320] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Restarting receiver thread Feb 6 23:02:59 xencc-hp03 kernel: [1010150.803323] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038636] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038644] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038654] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038656] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038668] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: short read (expected size 8) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038669] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038671] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: short read (expected size 8) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038676] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038786] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038818] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038892] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:03:01 xencc-hp03 kernel: [1010152.038925] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.083314] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:03:01 xencc-hp03 kernel: [1010152.083316] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:01 xencc-hp03 kernel: [1010152.083324] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:01 xencc-hp03 kernel: [1010152.083326] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:01 xencc-hp03 kernel: [1010152.087995] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.088039] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:01 xencc-hp03 kernel: [1010152.088108] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.088168] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614620] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614632] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614642] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: short read (expected size 16) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614645] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Authentication of peer failed, trying again. Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614753] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Terminating sender thread Feb 6 23:03:01 xencc-hp03 kernel: [1010152.614787] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:01 xencc-hp03 kernel: [1010152.631098] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Connection closed Feb 6 23:03:01 xencc-hp03 kernel: [1010152.631107] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:01 xencc-hp03 kernel: [1010152.635474] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:01 xencc-hp03 kernel: [1010152.635513] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:02 xencc-hp03 kernel: [1010153.094631] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:02 xencc-hp03 kernel: [1010153.094637] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:02 xencc-hp03 kernel: [1010153.638637] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:03 xencc-hp03 kernel: [1010154.054694] drbd xcp-persistent-database tcp:xenrt-1: initial packet M crossed Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147084] drbd xcp-persistent-database xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147098] drbd xcp-persistent-database xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147109] drbd xcp-persistent-database xenrt-2: short read (expected size 8) Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147110] drbd xcp-persistent-database xenrt-2: timeout while waiting for feature packet Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147235] drbd xcp-persistent-database xenrt-2: Terminating sender thread Feb 6 23:03:04 xencc-hp03 kernel: [1010155.147274] drbd xcp-persistent-database xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:04 xencc-hp03 kernel: [1010155.171121] drbd xcp-persistent-database xenrt-2: Connection closed Feb 6 23:03:04 xencc-hp03 kernel: [1010155.171130] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:04 xencc-hp03 kernel: [1010155.175221] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:04 xencc-hp03 kernel: [1010155.175264] drbd xcp-persistent-database xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:05 xencc-hp03 kernel: [1010156.070782] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:05 xencc-hp03 kernel: [1010156.070785] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:05 xencc-hp03 kernel: [1010156.159752] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: meta connection shut down by peer. Feb 6 23:03:05 xencc-hp03 kernel: [1010156.159794] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> NetworkFailure ) Feb 6 23:03:05 xencc-hp03 kernel: [1010156.198641] drbd xcp-persistent-database xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:05 xencc-hp03 kernel: [1010156.966648] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:05 xencc-hp03 kernel: [1010156.966659] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: short read (expected size 8) Feb 6 23:03:05 xencc-hp03 kernel: [1010156.966660] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: timeout while waiting for feature packet Feb 6 23:03:05 xencc-hp03 kernel: [1010156.966714] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:05 xencc-hp03 kernel: [1010156.966778] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:05 xencc-hp03 kernel: [1010157.007129] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:05 xencc-hp03 kernel: [1010157.007140] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:05 xencc-hp03 kernel: [1010157.011352] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:05 xencc-hp03 kernel: [1010157.011392] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:06 xencc-hp03 kernel: [1010157.865321] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:06 xencc-hp03 kernel: [1010157.865480] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:06 xencc-hp03 kernel: [1010157.865630] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:06 xencc-hp03 kernel: [1010157.865664] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:06 xencc-hp03 kernel: [1010157.887175] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:06 xencc-hp03 kernel: [1010157.887183] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:06 xencc-hp03 kernel: [1010157.891346] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:06 xencc-hp03 kernel: [1010157.891396] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:06 xencc-hp03 kernel: [1010157.891401] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Restarting receiver thread Feb 6 23:03:06 xencc-hp03 kernel: [1010157.891405] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:06 xencc-hp03 kernel: [1010158.026661] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:07 xencc-hp03 kernel: [1010158.310656] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 tcp:xenrt-2: Error receiving initial packet: -11 Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950674] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950691] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950708] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: short read (expected size 8) Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950710] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950869] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:03:07 xencc-hp03 kernel: [1010158.950902] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:07 xencc-hp03 kernel: [1010158.995189] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:03:07 xencc-hp03 kernel: [1010158.995200] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:07 xencc-hp03 kernel: [1010159.000393] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:07 xencc-hp03 kernel: [1010159.000431] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:08 xencc-hp03 kernel: [1010160.006685] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198687] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198699] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198710] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: short read (expected size 8) Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198711] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198836] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Terminating sender thread Feb 6 23:03:09 xencc-hp03 kernel: [1010160.198869] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:09 xencc-hp03 kernel: [1010160.231137] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Connection closed Feb 6 23:03:09 xencc-hp03 kernel: [1010160.231147] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:09 xencc-hp03 kernel: [1010160.235026] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:09 xencc-hp03 kernel: [1010160.235109] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:10 xencc-hp03 kernel: [1010161.254742] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766706] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: PingAck did not arrive in time. Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766723] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766727] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007: quorum( yes -> no ) Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766729] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-2: pdsk( Diskless -> DUnknown ) repl( Established -> Off ) Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766762] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Preparing cluster-wide state change 583983657: 0->all empty Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766765] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Committing cluster-wide state change 583983657 (0ms) Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766955] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Terminating sender thread Feb 6 23:03:10 xencc-hp03 kernel: [1010161.766983] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:10 xencc-hp03 kernel: [1010161.819153] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Connection closed Feb 6 23:03:10 xencc-hp03 kernel: [1010161.819163] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:10 xencc-hp03 kernel: [1010161.823461] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:10 xencc-hp03 kernel: [1010161.823500] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:10 xencc-hp03 kernel: [1010161.823506] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Restarting receiver thread Feb 6 23:03:10 xencc-hp03 kernel: [1010161.823510] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:12 xencc-hp03 kernel: [1010163.275207] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:12 xencc-hp03 kernel: [1010163.275210] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:12 xencc-hp03 kernel: [1010163.686704] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 tcp:xenrt-1: Error receiving initial packet: -11 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286737] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286749] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286759] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: short read (expected size 16) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286761] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Authentication of peer failed, trying again. Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286859] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:14 xencc-hp03 kernel: [1010165.286879] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.327250] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:14 xencc-hp03 kernel: [1010165.327260] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:14 xencc-hp03 kernel: [1010165.331298] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.331350] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:14 xencc-hp03 kernel: [1010165.542760] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f tcp:xenrt-2: Error receiving initial packet: -11 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770736] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770749] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770760] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: short read (expected size 8) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770761] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770885] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:14 xencc-hp03 kernel: [1010165.770932] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:14 xencc-hp03 kernel: [1010165.811223] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:14 xencc-hp03 kernel: [1010165.811233] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:14 xencc-hp03 kernel: [1010165.816140] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:14 xencc-hp03 kernel: [1010165.816178] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:15 xencc-hp03 kernel: [1010166.342763] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:15 xencc-hp03 kernel: [1010166.822737] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:15 xencc-hp03 kernel: [1010166.886851] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:15 xencc-hp03 kernel: [1010166.886854] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:15 xencc-hp03 kernel: [1010166.925949] drbd xcp-persistent-database xenrt-2: sock was reset by peer Feb 6 23:03:15 xencc-hp03 kernel: [1010166.925964] drbd xcp-persistent-database xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:15 xencc-hp03 kernel: [1010166.925975] drbd xcp-persistent-database xenrt-2: short read (expected size 8) Feb 6 23:03:15 xencc-hp03 kernel: [1010166.926059] drbd xcp-persistent-database xenrt-2: Terminating sender thread Feb 6 23:03:15 xencc-hp03 kernel: [1010166.926095] drbd xcp-persistent-database xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:15 xencc-hp03 kernel: [1010166.967336] drbd xcp-persistent-database xenrt-2: Connection closed Feb 6 23:03:15 xencc-hp03 kernel: [1010166.967346] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:15 xencc-hp03 kernel: [1010166.971279] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:15 xencc-hp03 kernel: [1010166.971322] drbd xcp-persistent-database xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182502] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: sock was reset by peer Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182519] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182532] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: short read (expected size 16) Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182534] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Authentication of peer failed, trying again. Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182640] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:03:16 xencc-hp03 kernel: [1010167.182666] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:16 xencc-hp03 kernel: [1010167.219247] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:03:16 xencc-hp03 kernel: [1010167.219257] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:16 xencc-hp03 kernel: [1010167.223258] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:16 xencc-hp03 kernel: [1010167.223299] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:16 xencc-hp03 kernel: [1010167.974746] drbd xcp-persistent-database xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:17 xencc-hp03 kernel: [1010168.234752] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182803] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182823] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182835] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: short read (expected size 8) Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182836] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: timeout while waiting for feature packet Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182969] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:19 xencc-hp03 kernel: [1010170.182999] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:19 xencc-hp03 kernel: [1010170.207294] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:19 xencc-hp03 kernel: [1010170.207303] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:19 xencc-hp03 kernel: [1010170.212359] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:19 xencc-hp03 kernel: [1010170.212412] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:19 xencc-hp03 kernel: [1010170.630912] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:19 xencc-hp03 kernel: [1010170.630920] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:20 xencc-hp03 kernel: [1010171.238798] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:21 xencc-hp03 kernel: [1010172.458956] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.458959] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550800] drbd xcp-persistent-database xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550812] drbd xcp-persistent-database xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550824] drbd xcp-persistent-database xenrt-1: short read (expected size 8) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550825] drbd xcp-persistent-database xenrt-1: timeout while waiting for feature packet Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550942] drbd xcp-persistent-database xenrt-1: Terminating sender thread Feb 6 23:03:21 xencc-hp03 kernel: [1010172.550975] drbd xcp-persistent-database xenrt-1: Starting sender thread (peer-node-id 0) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.591303] drbd xcp-persistent-database xenrt-1: Connection closed Feb 6 23:03:21 xencc-hp03 kernel: [1010172.591314] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:21 xencc-hp03 kernel: [1010172.595448] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.595490] drbd xcp-persistent-database xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:21 xencc-hp03 kernel: [1010172.646810] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.646820] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.646829] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: short read (expected size 16) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.646832] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Authentication of peer failed, trying again. Feb 6 23:03:21 xencc-hp03 kernel: [1010172.646935] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:21 xencc-hp03 kernel: [1010172.647018] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:21 xencc-hp03 kernel: [1010172.679219] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:21 xencc-hp03 kernel: [1010172.679229] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:21 xencc-hp03 kernel: [1010172.683279] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.683319] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:21 xencc-hp03 kernel: [1010172.874944] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:21 xencc-hp03 kernel: [1010172.874947] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510805] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510818] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510829] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: short read (expected size 8) Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510831] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: timeout while waiting for feature packet Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510937] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Terminating sender thread Feb 6 23:03:22 xencc-hp03 kernel: [1010173.510967] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:22 xencc-hp03 kernel: [1010173.555331] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Connection closed Feb 6 23:03:22 xencc-hp03 kernel: [1010173.555340] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:22 xencc-hp03 kernel: [1010173.559583] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:22 xencc-hp03 kernel: [1010173.559626] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:22 xencc-hp03 kernel: [1010173.606805] drbd xcp-persistent-database xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:22 xencc-hp03 kernel: [1010173.706825] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.470844] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:23 xencc-hp03 kernel: [1010174.470857] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:23 xencc-hp03 kernel: [1010174.470869] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: short read (expected size 16) Feb 6 23:03:23 xencc-hp03 kernel: [1010174.470871] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Authentication of peer failed, trying again. Feb 6 23:03:23 xencc-hp03 kernel: [1010174.470985] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:03:23 xencc-hp03 kernel: [1010174.471024] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:23 xencc-hp03 kernel: [1010174.499331] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:03:23 xencc-hp03 kernel: [1010174.499344] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:23 xencc-hp03 kernel: [1010174.504272] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:23 xencc-hp03 kernel: [1010174.504313] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.566808] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.697030] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:23 xencc-hp03 kernel: [1010174.697194] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.697317] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Terminating sender thread Feb 6 23:03:23 xencc-hp03 kernel: [1010174.697345] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:23 xencc-hp03 kernel: [1010174.715327] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Connection closed Feb 6 23:03:23 xencc-hp03 kernel: [1010174.715336] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:23 xencc-hp03 kernel: [1010174.719419] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:23 xencc-hp03 kernel: [1010174.719455] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.719460] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Restarting receiver thread Feb 6 23:03:23 xencc-hp03 kernel: [1010174.719464] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:23 xencc-hp03 kernel: [1010174.891409] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:23 xencc-hp03 kernel: [1010174.891416] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:24 xencc-hp03 kernel: [1010175.142836] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:24 xencc-hp03 kernel: [1010175.142853] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:24 xencc-hp03 kernel: [1010175.142870] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: short read (expected size 8) Feb 6 23:03:24 xencc-hp03 kernel: [1010175.142871] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: timeout while waiting for feature packet Feb 6 23:03:24 xencc-hp03 kernel: [1010175.143014] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:24 xencc-hp03 kernel: [1010175.143049] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:24 xencc-hp03 kernel: [1010175.171336] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:24 xencc-hp03 kernel: [1010175.171345] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:24 xencc-hp03 kernel: [1010175.176029] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:24 xencc-hp03 kernel: [1010175.176069] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:24 xencc-hp03 kernel: [1010175.186699] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:24 xencc-hp03 kernel: [1010175.186939] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:24 xencc-hp03 kernel: [1010175.187003] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Terminating sender thread Feb 6 23:03:24 xencc-hp03 kernel: [1010175.187021] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:24 xencc-hp03 kernel: [1010175.215233] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Connection closed Feb 6 23:03:24 xencc-hp03 kernel: [1010175.215241] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:24 xencc-hp03 kernel: [1010175.219295] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:24 xencc-hp03 kernel: [1010175.219330] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:24 xencc-hp03 kernel: [1010175.219336] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Restarting receiver thread Feb 6 23:03:24 xencc-hp03 kernel: [1010175.219340] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:24 xencc-hp03 kernel: [1010175.526940] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:25 xencc-hp03 kernel: [1010176.198845] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:25 xencc-hp03 kernel: [1010176.314176] drbd xcp-persistent-database xenrt-1: Handshake to peer 0 successful: Agreed network protocol version 122 Feb 6 23:03:25 xencc-hp03 kernel: [1010176.314179] drbd xcp-persistent-database xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:25 xencc-hp03 kernel: [1010176.359002] drbd xcp-persistent-database xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:25 xencc-hp03 kernel: [1010176.359005] drbd xcp-persistent-database xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:25 xencc-hp03 kernel: [1010176.654565] drbd xcp-persistent-database xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:25 xencc-hp03 kernel: [1010176.654706] drbd xcp-persistent-database xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:25 xencc-hp03 kernel: [1010176.654846] drbd xcp-persistent-database xenrt-2: Terminating sender thread Feb 6 23:03:25 xencc-hp03 kernel: [1010176.654872] drbd xcp-persistent-database xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:25 xencc-hp03 kernel: [1010176.695326] drbd xcp-persistent-database xenrt-2: Connection closed Feb 6 23:03:25 xencc-hp03 kernel: [1010176.695336] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:25 xencc-hp03 kernel: [1010176.699516] drbd xcp-persistent-database xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:25 xencc-hp03 kernel: [1010176.699562] drbd xcp-persistent-database xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:25 xencc-hp03 kernel: [1010176.699569] drbd xcp-persistent-database xenrt-2: Restarting receiver thread Feb 6 23:03:25 xencc-hp03 kernel: [1010176.699572] drbd xcp-persistent-database xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:25 xencc-hp03 kernel: [1010176.806853] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:25 xencc-hp03 kernel: [1010176.806864] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:25 xencc-hp03 kernel: [1010176.806875] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: short read (expected size 8) Feb 6 23:03:25 xencc-hp03 kernel: [1010176.806877] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: timeout while waiting for feature packet Feb 6 23:03:25 xencc-hp03 kernel: [1010176.806986] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:25 xencc-hp03 kernel: [1010176.807026] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:25 xencc-hp03 kernel: [1010176.851518] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:25 xencc-hp03 kernel: [1010176.851528] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:25 xencc-hp03 kernel: [1010176.855789] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:25 xencc-hp03 kernel: [1010176.855835] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:26 xencc-hp03 kernel: [1010177.862841] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342860] drbd xcp-persistent-database xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342874] drbd xcp-persistent-database xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342885] drbd xcp-persistent-database xenrt-1: short read (expected size 16) Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342887] drbd xcp-persistent-database xenrt-1: Authentication of peer failed, trying again. Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342968] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.342976] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:27 xencc-hp03 kernel: [1010178.343010] drbd xcp-persistent-database xenrt-1: Terminating sender thread Feb 6 23:03:27 xencc-hp03 kernel: [1010178.343046] drbd xcp-persistent-database xenrt-1: Starting sender thread (peer-node-id 0) Feb 6 23:03:27 xencc-hp03 kernel: [1010178.377412] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.377414] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:27 xencc-hp03 kernel: [1010178.387400] drbd xcp-persistent-database xenrt-1: Connection closed Feb 6 23:03:27 xencc-hp03 kernel: [1010178.387409] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:27 xencc-hp03 kernel: [1010178.391533] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.391577] drbd xcp-persistent-database xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:27 xencc-hp03 kernel: [1010178.638653] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:27 xencc-hp03 kernel: [1010178.638829] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:27 xencc-hp03 kernel: [1010178.639021] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Terminating sender thread Feb 6 23:03:27 xencc-hp03 kernel: [1010178.639041] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:27 xencc-hp03 kernel: [1010178.672961] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:27 xencc-hp03 kernel: [1010178.675379] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Connection closed Feb 6 23:03:27 xencc-hp03 kernel: [1010178.675393] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:27 xencc-hp03 kernel: [1010178.678852] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 674641720: 0->1 role( Primary ) conn( Connected ) Feb 6 23:03:27 xencc-hp03 kernel: [1010178.679659] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.679723] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:27 xencc-hp03 kernel: [1010178.679731] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Restarting receiver thread Feb 6 23:03:27 xencc-hp03 kernel: [1010178.679735] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:27 xencc-hp03 kernel: [1010178.854991] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:27 xencc-hp03 kernel: [1010178.854999] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:28 xencc-hp03 kernel: [1010179.398887] drbd xcp-persistent-database xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:29 xencc-hp03 kernel: [1010180.131922] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:29 xencc-hp03 kernel: [1010180.131925] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:29 xencc-hp03 kernel: [1010180.327975] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:29 xencc-hp03 kernel: [1010180.327979] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:29 xencc-hp03 kernel: [1010180.874862] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:29 xencc-hp03 kernel: [1010180.874873] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:29 xencc-hp03 kernel: [1010180.874883] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: short read (expected size 16) Feb 6 23:03:29 xencc-hp03 kernel: [1010180.874885] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Authentication of peer failed, trying again. Feb 6 23:03:29 xencc-hp03 kernel: [1010180.875011] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Terminating sender thread Feb 6 23:03:29 xencc-hp03 kernel: [1010180.875037] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:29 xencc-hp03 kernel: [1010180.919441] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Connection closed Feb 6 23:03:29 xencc-hp03 kernel: [1010180.919451] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:29 xencc-hp03 kernel: [1010180.923844] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:29 xencc-hp03 kernel: [1010180.923886] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:30 xencc-hp03 kernel: [1010181.129708] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:30 xencc-hp03 kernel: [1010181.129711] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:30 xencc-hp03 kernel: [1010181.924169] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:30 xencc-hp03 kernel: [1010181.924325] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:30 xencc-hp03 kernel: [1010181.924436] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:30 xencc-hp03 kernel: [1010181.924471] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:30 xencc-hp03 kernel: [1010181.926925] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:30 xencc-hp03 kernel: [1010181.947317] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:30 xencc-hp03 kernel: [1010181.947326] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:30 xencc-hp03 kernel: [1010181.951714] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:30 xencc-hp03 kernel: [1010181.951754] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:30 xencc-hp03 kernel: [1010181.951760] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Restarting receiver thread Feb 6 23:03:30 xencc-hp03 kernel: [1010181.951763] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146972] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: drbd_sync_handshake: Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146976] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: self 7EE5A75822D9908B:66C12C45F223EBF2:0000000000000000:0000000000000000 bits:5 flags:520 Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146978] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: peer 66C12C45F223EBF2:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:1120 Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146981] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: uuid_compare()=source-use-bitmap by rule=bitmap-self Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146995] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: State change 674641720: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFFC Feb 6 23:03:31 xencc-hp03 kernel: [1010182.146997] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Committing cluster-wide state change 674641720 (3468ms) Feb 6 23:03:31 xencc-hp03 kernel: [1010182.147031] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:31 xencc-hp03 kernel: [1010182.147033] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006: quorum( no -> yes ) [connected] Feb 6 23:03:31 xencc-hp03 kernel: [1010182.147036] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS ) [connected] Feb 6 23:03:31 xencc-hp03 kernel: [1010182.147697] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 30(1), total 30; compression: 100.0% Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150904] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150913] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150934] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: short read (expected size 16) Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150936] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Authentication of peer failed, trying again. Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150964] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Terminating sender thread Feb 6 23:03:31 xencc-hp03 kernel: [1010182.150991] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:31 xencc-hp03 kernel: [1010182.199522] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Connection closed Feb 6 23:03:31 xencc-hp03 kernel: [1010182.199534] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:31 xencc-hp03 kernel: [1010182.204037] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:31 xencc-hp03 kernel: [1010182.204081] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:32 xencc-hp03 kernel: [1010183.146886] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: sock_recvmsg returned -11 Feb 6 23:03:32 xencc-hp03 kernel: [1010183.146898] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> BrokenPipe ) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.146909] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: short read (expected size 16) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.146911] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Authentication of peer failed, trying again. Feb 6 23:03:32 xencc-hp03 kernel: [1010183.147028] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:03:32 xencc-hp03 kernel: [1010183.147056] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.191401] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:03:32 xencc-hp03 kernel: [1010183.191410] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:32 xencc-hp03 kernel: [1010183.195609] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:32 xencc-hp03 kernel: [1010183.195648] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:32 xencc-hp03 kernel: [1010183.206921] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:32 xencc-hp03 kernel: [1010183.750904] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:32 xencc-hp03 kernel: [1010183.750916] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.750928] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: short read (expected size 8) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.750930] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: timeout while waiting for feature packet Feb 6 23:03:32 xencc-hp03 kernel: [1010183.751042] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Terminating sender thread Feb 6 23:03:32 xencc-hp03 kernel: [1010183.751070] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:32 xencc-hp03 kernel: [1010183.791416] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Connection closed Feb 6 23:03:32 xencc-hp03 kernel: [1010183.791427] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:32 xencc-hp03 kernel: [1010183.795464] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:32 xencc-hp03 kernel: [1010183.795504] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:33 xencc-hp03 kernel: [1010184.198933] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:33 xencc-hp03 kernel: [1010184.810924] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:34 xencc-hp03 kernel: [1010185.092016] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:34 xencc-hp03 kernel: [1010185.092019] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:34 xencc-hp03 kernel: [1010185.545598] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: pdsk( Consistent -> Outdated ) [peer-state] Feb 6 23:03:34 xencc-hp03 kernel: [1010185.546416] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 30(1), total 30; compression: 100.0% Feb 6 23:03:34 xencc-hp03 kernel: [1010185.546425] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: helper command: /sbin/drbdadm before-resync-source Feb 6 23:03:34 xencc-hp03 kernel: [1010185.550551] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: helper command: /sbin/drbdadm before-resync-source exit code 0 Feb 6 23:03:34 xencc-hp03 kernel: [1010185.550590] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource ) [receive-bitmap] Feb 6 23:03:34 xencc-hp03 kernel: [1010185.550837] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: Began resync as SyncSource (will sync 20 KB [5 bits set]). Feb 6 23:03:34 xencc-hp03 kernel: [1010185.683127] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:34 xencc-hp03 kernel: [1010185.686926] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 4099367983: 0->2 role( Primary ) conn( Connected ) Feb 6 23:03:35 xencc-hp03 kernel: [1010186.569290] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:35 xencc-hp03 kernel: [1010186.569293] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:36 xencc-hp03 kernel: [1010187.337560] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:36 xencc-hp03 kernel: [1010187.337570] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:36 xencc-hp03 kernel: [1010187.847036] drbd xcp-persistent-database xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:36 xencc-hp03 kernel: [1010187.847039] drbd xcp-persistent-database xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:37 xencc-hp03 kernel: [1010188.392940] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:37 xencc-hp03 kernel: [1010188.393083] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:37 xencc-hp03 kernel: [1010188.393201] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Terminating sender thread Feb 6 23:03:37 xencc-hp03 kernel: [1010188.393219] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:37 xencc-hp03 kernel: [1010188.419440] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Connection closed Feb 6 23:03:37 xencc-hp03 kernel: [1010188.419449] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:37 xencc-hp03 kernel: [1010188.425052] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:37 xencc-hp03 kernel: [1010188.425114] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:37 xencc-hp03 kernel: [1010188.425124] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Restarting receiver thread Feb 6 23:03:37 xencc-hp03 kernel: [1010188.425129] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:37 xencc-hp03 kernel: [1010188.827660] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:37 xencc-hp03 kernel: [1010188.827817] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:37 xencc-hp03 kernel: [1010188.827974] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Terminating sender thread Feb 6 23:03:37 xencc-hp03 kernel: [1010188.828023] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:37 xencc-hp03 kernel: [1010188.855486] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Connection closed Feb 6 23:03:37 xencc-hp03 kernel: [1010188.855495] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:37 xencc-hp03 kernel: [1010188.859766] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:37 xencc-hp03 kernel: [1010188.859899] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:37 xencc-hp03 kernel: [1010188.859912] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Restarting receiver thread Feb 6 23:03:37 xencc-hp03 kernel: [1010188.967064] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:37 xencc-hp03 kernel: [1010188.967069] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:38 xencc-hp03 kernel: [1010189.418411] drbd xcp-persistent-database xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:38 xencc-hp03 kernel: [1010189.422963] drbd xcp-persistent-database: Preparing cluster-wide state change 288177520: 1->2 role( Secondary ) conn( Connected ) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.534962] drbd xcp-persistent-database/0 drbd1000 xenrt-2: self 07AD6A6892BC65BA:0000000000000000:065AC82552A65674:0000000000000000 bits:0 flags:0 Feb 6 23:03:38 xencc-hp03 kernel: [1010189.534964] drbd xcp-persistent-database/0 drbd1000 xenrt-2: peer's exposed UUID: 8A57F94B44CB2D10 Feb 6 23:03:38 xencc-hp03 kernel: [1010189.534991] drbd xcp-persistent-database: State change 288177520: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFF8 Feb 6 23:03:38 xencc-hp03 kernel: [1010189.534993] drbd xcp-persistent-database: Committing cluster-wide state change 288177520 (112ms) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.535016] drbd xcp-persistent-database xenrt-2: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:38 xencc-hp03 kernel: [1010189.535018] drbd xcp-persistent-database/0 drbd1000: disk( UpToDate -> Outdated ) [connected] Feb 6 23:03:38 xencc-hp03 kernel: [1010189.535020] drbd xcp-persistent-database/0 drbd1000 xenrt-2: pdsk( DUnknown -> Diskless ) repl( Off -> Established ) [connected] Feb 6 23:03:38 xencc-hp03 kernel: [1010189.670939] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: PingAck did not arrive in time. Feb 6 23:03:38 xencc-hp03 kernel: [1010189.670956] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown ) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.670959] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: repl( SyncSource -> Off ) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.671015] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 1036112468: 0->all empty Feb 6 23:03:38 xencc-hp03 kernel: [1010189.671022] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Committing cluster-wide state change 1036112468 (0ms) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.671326] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Terminating sender thread Feb 6 23:03:38 xencc-hp03 kernel: [1010189.671365] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:38 xencc-hp03 kernel: [1010189.700758] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Connection closed Feb 6 23:03:38 xencc-hp03 kernel: [1010189.700768] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:38 xencc-hp03 kernel: [1010189.705183] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:38 xencc-hp03 kernel: [1010189.705225] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:38 xencc-hp03 kernel: [1010189.705232] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Restarting receiver thread Feb 6 23:03:38 xencc-hp03 kernel: [1010189.705236] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.151059] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.151062] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:39 xencc-hp03 kernel: [1010190.151838] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.154985] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Preparing cluster-wide state change 1902533795: 0->1 role( Primary ) conn( Connected ) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.219106] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.219110] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:39 xencc-hp03 kernel: [1010190.220049] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.223044] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 3471716900: 0->1 role( Primary ) conn( Connected ) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231052] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: drbd_sync_handshake: Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231055] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: self EDD43057DEFA1C60:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:520 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231058] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: peer EDD43057DEFA1C60:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:1120 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231061] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: uuid_compare()=no-sync by rule=reconnected Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231072] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: State change 1902533795: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFFC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231074] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Committing cluster-wide state change 1902533795 (76ms) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231100] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231102] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005: quorum( no -> yes ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231104] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.231269] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-1: cleared bm UUID and bitmap EDD43057DEFA1C60:0000000000000000:0000000000000000:0000000000000000 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.299078] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: drbd_sync_handshake: Feb 6 23:03:39 xencc-hp03 kernel: [1010190.299082] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: self 7EE5A75822D9908B:66C12C45F223EBF2:0000000000000000:0000000000000000 bits:5 flags:120 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.299084] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: peer 66C12C45F223EBF2:0000000000000000:0000000000000000:0000000000000000 bits:2 flags:1124 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.299087] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: uuid_compare()=source-use-bitmap by rule=bitmap-self Feb 6 23:03:39 xencc-hp03 kernel: [1010190.302722] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: State change 3471716900: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFFC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.302725] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Committing cluster-wide state change 3471716900 (76ms) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.302775] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.302777] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: repl( Off -> WFBitMapS ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.303785] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 30(1), total 30; compression: 100.0% Feb 6 23:03:39 xencc-hp03 kernel: [1010190.305579] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 30(1), total 30; compression: 100.0% Feb 6 23:03:39 xencc-hp03 kernel: [1010190.305589] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: helper command: /sbin/drbdadm before-resync-source Feb 6 23:03:39 xencc-hp03 kernel: [1010190.310400] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: helper command: /sbin/drbdadm before-resync-source exit code 0 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.310469] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: repl( WFBitMapS -> SyncSource ) [receive-bitmap] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.310587] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: Began resync as SyncSource (will sync 20 KB [5 bits set]). Feb 6 23:03:39 xencc-hp03 kernel: [1010190.322140] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: updated UUIDs 7EE5A75822D9908B:0000000000000000:66C12C45F223EBF2:0000000000000000 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.324611] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: Resync done (total 1 sec; paused 0 sec; 20 K/sec) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.324617] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-1: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established ) [resync-finished] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.347021] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.347031] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:39 xencc-hp03 kernel: [1010190.347917] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.350965] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Preparing cluster-wide state change 38411930: 0->2 role( Primary ) conn( Connected ) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.426987] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-2: self EDD43057DEFA1C60:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:0 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.426990] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-2: peer's exposed UUID: EDD43057DEFA1C60 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.427002] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: State change 38411930: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFF8 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.427004] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f: Committing cluster-wide state change 38411930 (76ms) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.427048] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f xenrt-2: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.427050] drbd xcp-volume-e9428d9d-97a7-4a37-a2bb-630f8b5f3f0f/0 drbd1005 xenrt-2: pdsk( DUnknown -> Diskless ) repl( Off -> Established ) [connected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.461659] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.461789] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.461906] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Terminating sender thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.461936] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.490008] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 tcp:xenrt-1: Closing unexpected connection from 172.27.18.162 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.503480] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Connection closed Feb 6 23:03:39 xencc-hp03 kernel: [1010190.503490] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:39 xencc-hp03 kernel: [1010190.507636] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.507679] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.507686] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Restarting receiver thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.507690] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.694985] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: sock_recvmsg returned -11 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.694997] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> BrokenPipe ) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.695221] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Aborting cluster-wide state change 4099367983 (5008ms) rv = -21 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.695226] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Failure to connect: Interrupted state change (-21); retrying Feb 6 23:03:39 xencc-hp03 kernel: [1010190.695230] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Terminating sender thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.695287] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.695373] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.727463] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Connection closed Feb 6 23:03:39 xencc-hp03 kernel: [1010190.727489] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:39 xencc-hp03 kernel: [1010190.732469] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.732516] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( BrokenPipe -> Unconnected ) [disconnected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.732522] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Restarting receiver thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.732526] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.855126] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.855130] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:39 xencc-hp03 kernel: [1010190.856054] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:39 xencc-hp03 kernel: [1010190.856193] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.856322] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Terminating sender thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.856347] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Starting sender thread (peer-node-id 2) Feb 6 23:03:39 xencc-hp03 kernel: [1010190.895618] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Connection closed Feb 6 23:03:39 xencc-hp03 kernel: [1010190.895628] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected Feb 6 23:03:39 xencc-hp03 kernel: [1010190.900168] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:39 xencc-hp03 kernel: [1010190.900208] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:39 xencc-hp03 kernel: [1010190.900215] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Restarting receiver thread Feb 6 23:03:39 xencc-hp03 kernel: [1010190.900218] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.041516] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.041524] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.042653] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.043002] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Preparing cluster-wide state change 3771283314: 0->2 role( Secondary ) conn( Connected ) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.118985] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-2: self 03D475EC201461F8:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.118988] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-2: peer's exposed UUID: 03D475EC201461F8 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.119002] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: State change 3771283314: primary_nodes=0, weak_nodes=0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.119004] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Committing cluster-wide state change 3771283314 (76ms) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.119037] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-2: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.119040] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-2: pdsk( DUnknown -> Diskless ) repl( Off -> Established ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.211079] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.211082] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.211960] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.214977] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 1169357274: 0->1 role( Primary ) conn( Connected ) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.266807] drbd xcp-persistent-database xenrt-1: Handshake to peer 0 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.266810] drbd xcp-persistent-database xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.267696] drbd xcp-persistent-database xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.267785] drbd xcp-persistent-database xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.267902] drbd xcp-persistent-database xenrt-1: Terminating sender thread Feb 6 23:03:40 xencc-hp03 kernel: [1010191.267922] drbd xcp-persistent-database xenrt-1: Starting sender thread (peer-node-id 0) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.275030] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.275033] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.276006] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295048] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: drbd_sync_handshake: Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295051] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: self 6E134E436749B322:0000000000000000:C652E222C96B3A8A:0000000000000000 bits:0 flags:520 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295053] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: peer 6E134E436749B322:0000000000000000:C652E222C96B3A8A:0000000000000000 bits:0 flags:1120 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295055] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: uuid_compare()=no-sync by rule=reconnected Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295082] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: State change 1169357274: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFFC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295085] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Committing cluster-wide state change 1169357274 (80ms) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295116] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295121] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004: quorum( no -> yes ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295124] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.295260] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-1: cleared bm UUID and bitmap 6E134E436749B322:0000000000000000:C652E222C96B3A8A:0000000000000000 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.315468] drbd xcp-persistent-database xenrt-1: Connection closed Feb 6 23:03:40 xencc-hp03 kernel: [1010191.315477] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:40 xencc-hp03 kernel: [1010191.319291] drbd xcp-persistent-database xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.319333] drbd xcp-persistent-database xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.319340] drbd xcp-persistent-database xenrt-1: Restarting receiver thread Feb 6 23:03:40 xencc-hp03 kernel: [1010191.319343] drbd xcp-persistent-database xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.338985] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Preparing cluster-wide state change 175407658: 0->2 role( Primary ) conn( Connected ) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.378984] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-2: self 6E134E436749B322:0000000000000000:C652E222C96B3A8A:0000000000000000 bits:0 flags:0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.378987] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-2: peer's exposed UUID: 6E134E436749B323 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.378999] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: State change 175407658: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFF8 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.379001] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4: Committing cluster-wide state change 175407658 (40ms) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.379041] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4 xenrt-2: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.379043] drbd xcp-volume-623a917e-614f-4176-8e58-505248ee9db4/0 drbd1004 xenrt-2: pdsk( DUnknown -> Diskless ) repl( Off -> Established ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.431193] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Handshake to peer 2 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.431196] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.432088] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.434979] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Preparing cluster-wide state change 3142675959: 0->2 role( Primary ) conn( Connected ) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523379] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-2: self 7EE5A75822D9908B:0000000000000000:66C12C45F223EBF2:0000000000000000 bits:0 flags:0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523382] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-2: peer's exposed UUID: 7EE5A75822D9908A Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523395] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: State change 3142675959: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFF8 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523397] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907: Committing cluster-wide state change 3142675959 (88ms) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523440] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907 xenrt-2: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.523444] drbd xcp-volume-f4093dba-0a5a-4c28-a31b-a844c81fc907/0 drbd1006 xenrt-2: pdsk( DUnknown -> Diskless ) repl( Off -> Established ) [connected] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.879082] drbd xcp-persistent-database xenrt-1: Handshake to peer 0 successful: Agreed network protocol version 122 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.879084] drbd xcp-persistent-database xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:40 xencc-hp03 kernel: [1010191.879939] drbd xcp-persistent-database xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:40 xencc-hp03 kernel: [1010191.922998] drbd xcp-persistent-database: Preparing remote state change 3820879929: 0->1 role( Primary ) conn( Connected ) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.955164] drbd xcp-persistent-database/0 drbd1000 xenrt-1: drbd_sync_handshake: Feb 6 23:03:40 xencc-hp03 kernel: [1010191.955168] drbd xcp-persistent-database/0 drbd1000 xenrt-1: self 07AD6A6892BC65BA:0000000000000000:065AC82552A65674:0000000000000000 bits:0 flags:520 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.955170] drbd xcp-persistent-database/0 drbd1000 xenrt-1: peer 8A57F94B44CB2D11:07AD6A6892BC65BA:065AC82552A65674:0000000000000000 bits:16 flags:1120 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.955172] drbd xcp-persistent-database/0 drbd1000 xenrt-1: uuid_compare()=target-use-bitmap by rule=bitmap-peer Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962717] drbd xcp-persistent-database xenrt-1: Committing remote state change 3820879929 (primary_nodes=1) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962729] drbd xcp-persistent-database xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Primary ) [remote] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962731] drbd xcp-persistent-database/0 drbd1000: quorum( no -> yes ) [remote] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962733] drbd xcp-persistent-database/0 drbd1000 xenrt-1: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT ) [remote] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962740] drbd xcp-persistent-database/0 drbd1000: Disabling local AL-updates (optimization) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.962745] drbd xcp-persistent-database/0 drbd1000: Setting exposed data uuid: 8A57F94B44CB2D10 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.963652] drbd xcp-persistent-database/0 drbd1000 xenrt-1: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 33(1), total 33; compression: 99.9% Feb 6 23:03:40 xencc-hp03 kernel: [1010191.963689] drbd xcp-persistent-database/0 drbd1000 xenrt-1: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 33(1), total 33; compression: 99.9% Feb 6 23:03:40 xencc-hp03 kernel: [1010191.963698] drbd xcp-persistent-database/0 drbd1000 xenrt-1: helper command: /sbin/drbdadm before-resync-target Feb 6 23:03:40 xencc-hp03 kernel: [1010191.968463] drbd xcp-persistent-database/0 drbd1000 xenrt-1: helper command: /sbin/drbdadm before-resync-target exit code 0 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.968521] drbd xcp-persistent-database/0 drbd1000: disk( Outdated -> Inconsistent ) [receive-bitmap] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.968523] drbd xcp-persistent-database/0 drbd1000 xenrt-2: resync-susp( no -> connection dependency ) [receive-bitmap] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.968525] drbd xcp-persistent-database/0 drbd1000 xenrt-1: repl( WFBitMapT -> SyncTarget ) [receive-bitmap] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.968722] drbd xcp-persistent-database/0 drbd1000 xenrt-1: Began resync as SyncTarget (will sync 64 KB [16 bits set]). Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976593] drbd xcp-persistent-database/0 drbd1000 xenrt-1: Resync done (total 1 sec; paused 0 sec; 64 K/sec) Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976600] drbd xcp-persistent-database/0 drbd1000 xenrt-1: updated UUIDs 8A57F94B44CB2D10:0000000000000000:065AC82552A65674:0000000000000000 Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976607] drbd xcp-persistent-database/0 drbd1000: disk( Inconsistent -> UpToDate ) [resync-finished] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976610] drbd xcp-persistent-database/0 drbd1000 xenrt-2: resync-susp( connection dependency -> no ) [resync-finished] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976612] drbd xcp-persistent-database/0 drbd1000 xenrt-1: repl( SyncTarget -> Established ) [resync-finished] Feb 6 23:03:40 xencc-hp03 kernel: [1010191.976814] drbd xcp-persistent-database/0 drbd1000 xenrt-1: helper command: /sbin/drbdadm after-resync-target Feb 6 23:03:40 xencc-hp03 kernel: [1010191.981429] drbd xcp-persistent-database/0 drbd1000 xenrt-1: helper command: /sbin/drbdadm after-resync-target exit code 0 Feb 6 23:03:41 xencc-hp03 kernel: [1010192.551104] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:41 xencc-hp03 kernel: [1010192.551109] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:41 xencc-hp03 kernel: [1010192.552027] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:41 xencc-hp03 kernel: [1010192.552214] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> NetworkFailure ) [disconnected] Feb 6 23:03:41 xencc-hp03 kernel: [1010192.552294] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Terminating sender thread Feb 6 23:03:41 xencc-hp03 kernel: [1010192.552326] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Starting sender thread (peer-node-id 1) Feb 6 23:03:41 xencc-hp03 kernel: [1010192.595463] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Connection closed Feb 6 23:03:41 xencc-hp03 kernel: [1010192.595473] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected Feb 6 23:03:41 xencc-hp03 kernel: [1010192.600128] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: helper command: /sbin/drbdadm disconnected exit code 0 Feb 6 23:03:41 xencc-hp03 kernel: [1010192.600171] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( NetworkFailure -> Unconnected ) [disconnected] Feb 6 23:03:41 xencc-hp03 kernel: [1010192.600180] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Restarting receiver thread Feb 6 23:03:41 xencc-hp03 kernel: [1010192.600184] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Unconnected -> Connecting ) [connecting] Feb 6 23:03:42 xencc-hp03 kernel: [1010193.127097] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Handshake to peer 1 successful: Agreed network protocol version 122 Feb 6 23:03:42 xencc-hp03 kernel: [1010193.127100] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG Feb 6 23:03:42 xencc-hp03 kernel: [1010193.127968] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: Peer authenticated using 20 bytes HMAC Feb 6 23:03:42 xencc-hp03 kernel: [1010193.131004] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Preparing cluster-wide state change 1182010644: 0->1 role( Secondary ) conn( Connected ) Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215075] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: drbd_sync_handshake: Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215080] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: self 03D475EC201461F8:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:120 Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215082] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: peer 03D475EC201461F8:0000000000000000:0000000000000000:0000000000000000 bits:0 flags:1120 Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215085] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: uuid_compare()=no-sync by rule=reconnected Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215098] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: State change 1182010644: primary_nodes=0, weak_nodes=0 Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215100] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278: Committing cluster-wide state change 1182010644 (84ms) Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215138] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278 xenrt-1: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected] Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215140] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007: quorum( no -> yes ) [connected] Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215142] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [connected] Feb 6 23:03:42 xencc-hp03 kernel: [1010193.215242] drbd xcp-volume-9dd3dc66-aa58-40f2-aa56-14b8846a4278/0 drbd1007 xenrt-1: cleared bm UUID and bitmap 03D475EC201461F8:0000000000000000:0000000000000000:0000000000000000