Import OVA silently fails



  • There's a newer version that includes improvements to the OVA import process. You can access it by switching to the latest release channel in XOA.



  • OK , i updated to lastest, and it has the same effect. Where do i find CLI documentation for importing?


  • XCP-ng Team



  • Ok the file syslog has alot of errors in it, looks like unhandled promise chains. (.then errors) The file is 3MB, how can i get this file to you?

    Here is a snippet near the area of import:

    Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.657Z xo:xapi DEBUG Creating VM Spiceworks-Production
    Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.728Z xo:xapi DEBUG Creating VDI Hard disk 1 on Local storage
    Apr  8 01:09:51 xoa xo-server[115382]: 2020-04-08T05:09:51.729Z xo:xapi DEBUG Creating VIF for VM Spiceworks-Production on network MGMT-LAN
    Apr  8 01:09:52 xoa xo-server[115382]: 2020-04-08T05:09:52.538Z xo:xapi DEBUG Creating VBD for VDI Hard disk 1 on VM [Importing...] Spiceworks-Production
    Apr  8 01:11:07 xoa kernel: [3494938.746302] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65109 PROTO=2 
    Apr  8 01:11:07 xoa kernel: [3494938.746397] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65110 PROTO=2 
    Apr  8 01:13:17 xoa kernel: [3495068.743182] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65116 PROTO=2 
    Apr  8 01:13:17 xoa kernel: [3495068.743286] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65117 PROTO=2 
    Apr  8 01:15:27 xoa kernel: [3495198.739905] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65122 PROTO=2 
    Apr  8 01:15:27 xoa kernel: [3495198.739971] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=65123 PROTO=2 
    Apr  8 01:15:54 xoa kernel: [3495225.770190] node invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
    Apr  8 01:15:54 xoa kernel: [3495225.770191] node cpuset=/ mems_allowed=0
    Apr  8 01:15:54 xoa kernel: [3495225.770196] CPU: 1 PID: 115382 Comm: node Not tainted 4.19.0-8-amd64 #1 Debian 4.19.98-1
    Apr  8 01:15:54 xoa kernel: [3495225.770196] Hardware name: Xen HVM domU, BIOS 4.11.1-7.8.xcpng8.0 12/12/2019
    Apr  8 01:15:54 xoa kernel: [3495225.770197] Call Trace:
    Apr  8 01:15:54 xoa kernel: [3495225.770206]  dump_stack+0x66/0x90
    Apr  8 01:15:54 xoa kernel: [3495225.770211]  dump_header+0x6b/0x283
    Apr  8 01:15:54 xoa kernel: [3495225.770214]  ? do_try_to_free_pages+0x2ec/0x370
    Apr  8 01:15:54 xoa kernel: [3495225.770215]  oom_kill_process.cold.30+0xb/0x1cf
    Apr  8 01:15:54 xoa kernel: [3495225.770217]  ? oom_badness+0xe9/0x140
    Apr  8 01:15:54 xoa kernel: [3495225.770219]  out_of_memory+0x1a5/0x450
    Apr  8 01:15:54 xoa kernel: [3495225.770221]  __alloc_pages_slowpath+0xbd8/0xcb0
    Apr  8 01:15:54 xoa kernel: [3495225.770223]  __alloc_pages_nodemask+0x28b/0x2b0
    Apr  8 01:15:54 xoa kernel: [3495225.770225]  filemap_fault+0x3bd/0x780
    Apr  8 01:15:54 xoa kernel: [3495225.770228]  ? alloc_set_pte+0xf2/0x560
    Apr  8 01:15:54 xoa kernel: [3495225.770229]  ? filemap_map_pages+0x1ed/0x3a0
    Apr  8 01:15:54 xoa kernel: [3495225.770282]  ext4_filemap_fault+0x2c/0x40 [ext4]
    Apr  8 01:15:54 xoa kernel: [3495225.770285]  __do_fault+0x36/0x130
    Apr  8 01:15:54 xoa kernel: [3495225.770286]  __handle_mm_fault+0xe6c/0x1270
    Apr  8 01:15:54 xoa kernel: [3495225.770288]  handle_mm_fault+0xd6/0x200
    Apr  8 01:15:54 xoa kernel: [3495225.770291]  __do_page_fault+0x249/0x4f0
    Apr  8 01:15:54 xoa kernel: [3495225.770294]  ? page_fault+0x8/0x30
    Apr  8 01:15:54 xoa kernel: [3495225.770295]  page_fault+0x1e/0x30
    Apr  8 01:15:54 xoa kernel: [3495225.770297] RIP: 0033:0x13a72b9
    Apr  8 01:15:54 xoa kernel: [3495225.770303] Code: Bad RIP value.
    Apr  8 01:15:54 xoa kernel: [3495225.770304] RSP: 002b:00007ffcf8ff0490 EFLAGS: 00010246
    Apr  8 01:15:54 xoa kernel: [3495225.770305] RAX: 0000055c072656b9 RBX: 0000000000c0f1f0 RCX: 00000000030be010
    Apr  8 01:15:54 xoa kernel: [3495225.770306] RDX: 000000003ab98c20 RSI: 0000000000000007 RDI: 0000000000000001
    Apr  8 01:15:54 xoa kernel: [3495225.770307] RBP: 00007ffcf8ff04a0 R08: 000000000000001f R09: 0000000008bf8810
    Apr  8 01:15:54 xoa kernel: [3495225.770307] R10: 0000000000000006 R11: 0000000000000127 R12: 00000f8be5127ad9
    Apr  8 01:15:54 xoa kernel: [3495225.770308] R13: 00000000030fc400 R14: 0000000000000008 R15: 00007ffcf8ff04e8
    Apr  8 01:15:54 xoa kernel: [3495225.770309] Mem-Info:
    Apr  8 01:15:54 xoa kernel: [3495225.770313] active_anon:330003 inactive_anon:114458 isolated_anon:0
    Apr  8 01:15:54 xoa kernel: [3495225.770313]  active_file:23 inactive_file:43 isolated_file:0
    Apr  8 01:15:54 xoa kernel: [3495225.770313]  unevictable:0 dirty:0 writeback:0 unstable:0
    Apr  8 01:15:54 xoa kernel: [3495225.770313]  slab_reclaimable:8284 slab_unreclaimable:12775
    Apr  8 01:15:54 xoa kernel: [3495225.770313]  mapped:35 shmem:13 pagetables:2766 bounce:0
    Apr  8 01:15:54 xoa kernel: [3495225.770313]  free:13146 free_pcp:413 free_cma:0
    Apr  8 01:15:54 xoa kernel: [3495225.770315] Node 0 active_anon:1320012kB inactive_anon:457832kB active_file:92kB inactive_file:172kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:140kB dirty:0kB writeback:0kB shmem:52kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
    Apr  8 01:15:54 xoa kernel: [3495225.770316] Node 0 DMA free:8008kB min:360kB low:448kB high:536kB active_anon:2988kB inactive_anon:4688kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15988kB managed:15904kB mlocked:0kB kernel_stack:0kB pagetables:32kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
    Apr  8 01:15:54 xoa kernel: [3495225.770318] lowmem_reserve[]: 0 1916 1916 1916 1916
    Apr  8 01:15:54 xoa kernel: [3495225.770319] Node 0 DMA32 free:44576kB min:44692kB low:55864kB high:67036kB active_anon:1317024kB inactive_anon:453144kB active_file:376kB inactive_file:524kB unevictable:0kB writepending:0kB present:2072576kB managed:1995344kB mlocked:0kB kernel_stack:4624kB pagetables:11032kB bounce:0kB free_pcp:1652kB local_pcp:300kB free_cma:0kB
    Apr  8 01:15:54 xoa kernel: [3495225.770322] lowmem_reserve[]: 0 0 0 0 0
    Apr  8 01:15:54 xoa kernel: [3495225.770323] Node 0 DMA: 12*4kB (UM) 9*8kB (UM) 5*16kB (UME) 6*32kB (UME) 5*64kB (UE) 5*128kB (UME) 4*256kB (UME) 3*512kB (UE) 0*1024kB 2*2048kB (UM) 0*4096kB = 8008kB
    Apr  8 01:15:54 xoa kernel: [3495225.770342] Node 0 DMA32: 308*4kB (UMEH) 188*8kB (UMEH) 187*16kB (UMEH) 238*32kB (UMEH) 91*64kB (UEH) 54*128kB (UEH) 27*256kB (UMEH) 4*512kB (UME) 2*1024kB (UM) 4*2048kB (UM) 0*4096kB = 45280kB
    Apr  8 01:15:54 xoa kernel: [3495225.770351] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
    Apr  8 01:15:54 xoa kernel: [3495225.770352] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
    Apr  8 01:15:54 xoa kernel: [3495225.770352] 99 total pagecache pages
    Apr  8 01:15:54 xoa kernel: [3495225.770353] 31 pages in swap cache
    Apr  8 01:15:54 xoa kernel: [3495225.770354] Swap cache stats: add 2889276, delete 2889244, find 929577/1183567
    Apr  8 01:15:54 xoa kernel: [3495225.770354] Free swap  = 0kB
    Apr  8 01:15:54 xoa kernel: [3495225.770355] Total swap = 2086908kB
    Apr  8 01:15:54 xoa kernel: [3495225.770356] 522141 pages RAM
    Apr  8 01:15:54 xoa kernel: [3495225.770356] 0 pages HighMem/MovableOnly
    Apr  8 01:15:54 xoa kernel: [3495225.770356] 19329 pages reserved
    Apr  8 01:15:54 xoa kernel: [3495225.770357] 0 pages hwpoisoned
    

  • XCP-ng Team

    This is a RAM issue. Raise your XOA RAM to 4GiB, if it continues, follow: https://xen-orchestra.com/docs/troubleshooting.html#memory



  • After making those changes, it got further, but still halted at about 10%. (I upped to 8GB RAM)

    Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.411Z xo:xapi DEBUG Creating VM Spiceworks-Production
    Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.482Z xo:xapi DEBUG Creating VDI Hard disk 1 on Local storage
    Apr  8 16:38:22 xoa xo-server[467]: 2020-04-08T20:38:22.483Z xo:xapi DEBUG Creating VIF for VM Spiceworks-Production on network MGMT-LAN
    Apr  8 16:38:23 xoa xo-server[467]: 2020-04-08T20:38:23.294Z xo:xapi DEBUG Creating VBD for VDI Hard disk 1 on VM [Importing...] Spiceworks-Production
    Apr  8 16:38:27 xoa kernel: [   66.534931] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2660 PROTO=2 
    Apr  8 16:40:37 xoa kernel: [  196.531337] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2665 PROTO=2 
    Apr  8 16:42:47 xoa kernel: [  326.527746] [UFW BLOCK] IN=eth0 OUT= MAC=01:00:5e:00:00:01:00:18:0a:86:53:c4:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0xC0 TTL=1 ID=2671 PROTO=2 
    Apr  8 16:44:16 xoa xoa-updater[472]: 2020-04-08T20:44:14.045Z xoa-updater WARN onMessage Error: Command failed: xenstore-read vm
    Apr  8 16:44:16 xoa xoa-updater[472]: fatal error: runtime: out of memory
    Apr  8 16:44:16 xoa xoa-updater[472]: runtime stack:
    Apr  8 16:44:16 xoa xoa-updater[472]: runtime.throw(0x8108647, 0x16)
    

    Here is my xo-server.service

    ExecStart=/usr/local/bin/node --max-old-space-size=7680 /usr/local/bin/xo-server
    

    Anything else we can try?


  • XCP-ng Team

    So there's a leak when you import 😕 How big is the OVA?

    Can you try with the OVA CLI?



  • The VM is 138GB. I read those instructions, what i am failing to understand is where and how I point it to my local disk? Do you have an example of using a file in C:\Export (on a windows box)?

    The example the OVA appears to be local to XOA?

    Please advise,
    Thank you



  • I have tried to use a SR that is used for ISO:

    xo-upload-ova --upload Spiceworks-Production.ova 1fee8757-644e-2bce-beb0-9f85380a5842 --override 'nameLabel=test' descriptionLabel='Spiceworks' memory=json:671088640 disks.vmdisk1.descriptionLabel='disk description' disks.vmdisk1.capacity=json:1342177280
    

    The issue of course is, how do i get the settings for disk size and memory. When i try to:

     xo-upload-ova --inspect Spiceworks-Production.ova 1fee8757-644e-2bce-beb0-9f85380a5842
    

    It doesn't know where that file is, does it have to be on a Local SR to XOA?


  • XCP-ng Team

    Ping @nraynaud who did that


  • XCP-ng Team

    I am looking into it on my side. I might come back with questions.



  • OK no problem, i look forward to hearing from you. 😊



  • @nraynaud Any update?


  • XCP-ng Team

    I'm still on it, I think I found an unrelated bug whose effect would be to upload a VM without any disk.


  • XCP-ng Team

    I think I got the bug, there is some cleanup and QA to be done before releasing a fix.



  • @nraynaud Thanks for working on this. Is the fix pushed to the latest stream or still in QA?


  • XCP-ng Team

    It will be released very soon 🙂 (for this month release)



  • @olivierlambert Thanks!


Log in to reply
 

XCP-ng Pro Support

XCP-ng Pro Support