Andrew C

Andrew C

Good afternoon,

I have a small Pool of 2 hosts. I have a PureStorage device that I use for SR's.
I have a production SR that works 100% on both hosts.
I have created a new LUN on that PS device (LUN 6) I create the SR, Format it, and it attaches to the the first host without issue.
The 2nd host it will not connect to:
If I build the SR in XenCenter I see this error >

If I build the SR in XOA i see this error in the Log>
SR_BACKEND_FAILURE_47(, The SR is not available [opterr=[Errno 2] No such file or directory: '/dev/disk/by-scsid/3624a9370ab52430403ee4bb800011bce'], )
The information looks correct

Any idea's why this is happening?!
Any pointers would be very valuable.

Thanks,
Andrew

Andrew C

@nraynaud Any update?

Andrew C

OK no problem, i look forward to hearing from you.

Andrew C

I have tried to use a SR that is used for ISO:

xo-upload-ova --upload Spiceworks-Production.ova 1fee8757-644e-2bce-beb0-9f85380a5842 --override 'nameLabel=test' descriptionLabel='Spiceworks' memory=json:671088640 disks.vmdisk1.descriptionLabel='disk description' disks.vmdisk1.capacity=json:1342177280

The issue of course is, how do i get the settings for disk size and memory. When i try to:

 xo-upload-ova --inspect Spiceworks-Production.ova 1fee8757-644e-2bce-beb0-9f85380a5842

It doesn't know where that file is, does it have to be on a Local SR to XOA?

Andrew C

The VM is 138GB. I read those instructions, what i am failing to understand is where and how I point it to my local disk? Do you have an example of using a file in C:\Export (on a windows box)?

The example the OVA appears to be local to XOA?

Please advise,
Thank you

Andrew C

After making those changes, it got further, but still halted at about 10%. (I upped to 8GB RAM)

Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.411Z xo:xapi DEBUG Creating VM Spiceworks-Production
Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.482Z xo:xapi DEBUG Creating VDI Hard disk 1 on Local storage
Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.483Z xo:xapi DEBUG Creating VIF for VM Spiceworks-Production on network MGMT-LAN
Apr  8 16:38:23 xoa xo-server[467]: 2020-04-08T20:38:23.294Z xo:xapi DEBUG Creating VBD for VDI Hard disk 1 on VM [Importing...] Spiceworks-Production
Apr  8 16:38:27 xoa kernel: [   66.534931] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2660 PROTO=2 
Apr  8 16:40:37 xoa kernel: [  196.531337] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2665 PROTO=2 
Apr  8 16:42:47 xoa kernel: [  326.527746] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2671 PROTO=2 
Apr  8 16:44:16 xoa xoa-updater[472]: 2020-04-08T20:44:14.045Z xoa-updater WARN onMessage Error: Command failed: xenstore-read vm
Apr  8 16:44:16 xoa xoa-updater[472]: fatal error: runtime: out of memory
Apr  8 16:44:16 xoa xoa-updater[472]: runtime stack:
Apr  8 16:44:16 xoa xoa-updater[472]: runtime.throw(0x8108647, 0x16)

Here is my xo-server.service

ExecStart=/usr/local/bin/node --max-old-space-size=7680 /usr/local/bin/xo-server

Anything else we can try?

Andrew C

Ok the file syslog has alot of errors in it, looks like unhandled promise chains. (.then errors) The file is 3MB, how can i get this file to you?

Here is a snippet near the area of import:

Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.657Z xo:xapi DEBUG Creating VM Spiceworks-Production
Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.728Z xo:xapi DEBUG Creating VDI Hard disk 1 on Local storage
Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.729Z xo:xapi DEBUG Creating VIF for VM Spiceworks-Production on network MGMT-LAN
Apr  8 01:09:52 xoa xo-server[115382]: 2020-04-08T05:09:52.538Z xo:xapi DEBUG Creating VBD for VDI Hard disk 1 on VM [Importing...] Spiceworks-Production
Apr  8 01:11:07 xoa kernel: [3494938.746302] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65109 PROTO=2 
Apr  8 01:11:07 xoa kernel: [3494938.746397] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65110 PROTO=2 
Apr  8 01:13:17 xoa kernel: [3495068.743182] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65116 PROTO=2 
Apr  8 01:13:17 xoa kernel: [3495068.743286] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65117 PROTO=2 
Apr  8 01:15:27 xoa kernel: [3495198.739905] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65122 PROTO=2 
Apr  8 01:15:27 xoa kernel: [3495198.739971] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65123 PROTO=2 
Apr  8 01:15:54 xoa kernel: [3495225.770190] node invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
Apr  8 01:15:54 xoa kernel: [3495225.770191] node cpuset=/ mems_allowed=0
Apr  8 01:15:54 xoa kernel: [3495225.770196] CPU: 1 PID: 115382 Comm: node Not tainted 4.19.0-8-amd64 #1 Debian 4.19.98-1
Apr  8 01:15:54 xoa kernel: [3495225.770196] Hardware name: Xen HVM domU, BIOS 4.11.1-7.8.xcpng8.0 12/12/2019
Apr  8 01:15:54 xoa kernel: [3495225.770197] Call Trace:
Apr  8 01:15:54 xoa kernel: [3495225.770206]  dump_stack+0x66/0x90
Apr  8 01:15:54 xoa kernel: [3495225.770211]  dump_header+0x6b/0x283
Apr  8 01:15:54 xoa kernel: [3495225.770214]  ? do_try_to_free_pages+0x2ec/0x370
Apr  8 01:15:54 xoa kernel: [3495225.770215]  oom_kill_process.cold.30+0xb/0x1cf
Apr  8 01:15:54 xoa kernel: [3495225.770217]  ? oom_badness+0xe9/0x140
Apr  8 01:15:54 xoa kernel: [3495225.770219]  out_of_memory+0x1a5/0x450
Apr  8 01:15:54 xoa kernel: [3495225.770221]  __alloc_pages_slowpath+0xbd8/0xcb0
Apr  8 01:15:54 xoa kernel: [3495225.770223]  __alloc_pages_nodemask+0x28b/0x2b0
Apr  8 01:15:54 xoa kernel: [3495225.770225]  filemap_fault+0x3bd/0x780
Apr  8 01:15:54 xoa kernel: [3495225.770228]  ? alloc_set_pte+0xf2/0x560
Apr  8 01:15:54 xoa kernel: [3495225.770229]  ? filemap_map_pages+0x1ed/0x3a0
Apr  8 01:15:54 xoa kernel: [3495225.770282]  ext4_filemap_fault+0x2c/0x40 [ext4]
Apr  8 01:15:54 xoa kernel: [3495225.770285]  __do_fault+0x36/0x130
Apr  8 01:15:54 xoa kernel: [3495225.770286]  __handle_mm_fault+0xe6c/0x1270
Apr  8 01:15:54 xoa kernel: [3495225.770288]  handle_mm_fault+0xd6/0x200
Apr  8 01:15:54 xoa kernel: [3495225.770291]  __do_page_fault+0x249/0x4f0
Apr  8 01:15:54 xoa kernel: [3495225.770294]  ? page_fault+0x8/0x30
Apr  8 01:15:54 xoa kernel: [3495225.770295]  page_fault+0x1e/0x30
Apr  8 01:15:54 xoa kernel: [3495225.770297] RIP: 0033:0x13a72b9
Apr  8 01:15:54 xoa kernel: [3495225.770303] Code: Bad RIP value.
Apr  8 01:15:54 xoa kernel: [3495225.770304] RSP: 002b:00007ffcf8ff0490 EFLAGS: 00010246
Apr  8 01:15:54 xoa kernel: [3495225.770305] RAX: 0000055c072656b9 RBX: 0000000000c0f1f0 RCX: 00000000030be010
Apr  8 01:15:54 xoa kernel: [3495225.770306] RDX: 000000003ab98c20 RSI: 0000000000000007 RDI: 0000000000000001
Apr  8 01:15:54 xoa kernel: [3495225.770307] RBP: 00007ffcf8ff04a0 R08: 000000000000001f R09: 0000000008bf8810
Apr  8 01:15:54 xoa kernel: [3495225.770307] R10: 0000000000000006 R11: 0000000000000127 R12: 00000f8be5127ad9
Apr  8 01:15:54 xoa kernel: [3495225.770308] R13: 00000000030fc400 R14: 0000000000000008 R15: 00007ffcf8ff04e8
Apr  8 01:15:54 xoa kernel: [3495225.770309] Mem-Info:
Apr  8 01:15:54 xoa kernel: [3495225.770313] active_anon:330003 inactive_anon:114458 isolated_anon:0
Apr  8 01:15:54 xoa kernel: [3495225.770313]  active_file:23 inactive_file:43 isolated_file:0
Apr  8 01:15:54 xoa kernel: [3495225.770313]  unevictable:0 dirty:0 writeback:0 unstable:0
Apr  8 01:15:54 xoa kernel: [3495225.770313]  slab_reclaimable:8284 slab_unreclaimable:12775
Apr  8 01:15:54 xoa kernel: [3495225.770313]  mapped:35 shmem:13 pagetables:2766 bounce:0
Apr  8 01:15:54 xoa kernel: [3495225.770313]  free:13146 free_pcp:413 free_cma:0
Apr  8 01:15:54 xoa kernel: [3495225.770315] Node 0 active_anon:1320012kB inactive_anon:457832kB active_file:92kB inactive_file:172kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:140kB dirty:0kB writeback:0kB shmem:52kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Apr  8 01:15:54 xoa kernel: [3495225.770316] Node 0 DMA free:8008kB min:360kB low:448kB high:536kB active_anon:2988kB inactive_anon:4688kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15988kB managed:15904kB mlocked:0kB kernel_stack:0kB pagetables:32kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Apr  8 01:15:54 xoa kernel: [3495225.770318] lowmem_reserve[]: 0 1916 1916 1916 1916
Apr  8 01:15:54 xoa kernel: [3495225.770319] Node 0 DMA32 free:44576kB min:44692kB low:55864kB high:67036kB active_anon:1317024kB inactive_anon:453144kB active_file:376kB inactive_file:524kB unevictable:0kB writepending:0kB present:2072576kB managed:1995344kB mlocked:0kB kernel_stack:4624kB pagetables:11032kB bounce:0kB free_pcp:1652kB local_pcp:300kB free_cma:0kB
Apr  8 01:15:54 xoa kernel: [3495225.770322] lowmem_reserve[]: 0 0 0 0 0
Apr  8 01:15:54 xoa kernel: [3495225.770323] Node 0 DMA: 12*4kB (UM) 9*8kB (UM) 5*16kB (UME) 6*32kB (UME) 5*64kB (UE) 5*128kB (UME) 4*256kB (UME) 3*512kB (UE) 0*1024kB 2*2048kB (UM) 0*4096kB = 8008kB
Apr  8 01:15:54 xoa kernel: [3495225.770342] Node 0 DMA32: 308*4kB (UMEH) 188*8kB (UMEH) 187*16kB (UMEH) 238*32kB (UMEH) 91*64kB (UEH) 54*128kB (UEH) 27*256kB (UMEH) 4*512kB (UME) 2*1024kB (UM) 4*2048kB (UM) 0*4096kB = 45280kB
Apr  8 01:15:54 xoa kernel: [3495225.770351] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Apr  8 01:15:54 xoa kernel: [3495225.770352] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Apr  8 01:15:54 xoa kernel: [3495225.770352] 99 total pagecache pages
Apr  8 01:15:54 xoa kernel: [3495225.770353] 31 pages in swap cache
Apr  8 01:15:54 xoa kernel: [3495225.770354] Swap cache stats: add 2889276, delete 2889244, find 929577/1183567
Apr  8 01:15:54 xoa kernel: [3495225.770354] Free swap  = 0kB
Apr  8 01:15:54 xoa kernel: [3495225.770355] Total swap = 2086908kB
Apr  8 01:15:54 xoa kernel: [3495225.770356] 522141 pages RAM
Apr  8 01:15:54 xoa kernel: [3495225.770356] 0 pages HighMem/MovableOnly
Apr  8 01:15:54 xoa kernel: [3495225.770356] 19329 pages reserved
Apr  8 01:15:54 xoa kernel: [3495225.770357] 0 pages hwpoisoned

Andrew C

OK , i updated to lastest, and it has the same effect. Where do i find CLI documentation for importing?

Andrew C

@Danp

I have tried from XOA (Enterprise) as well from XenCenter.
Current version: 5.44.1
What would I look for in these lots, there is alot in here.

Thank you,

Andrew C

Ah found something after opening xen center:

They appear to have stopped VMs! Any ideas?

Andrew C

@Andrew C

Latest posts made by Andrew C