VDI_IO_ERROR Continuous Replication on clean install.
-
{"code":"VDI_IO_ERROR","params":["Device I/O errors"],"url":"https://192.168.0.100/import_raw_vdi/?format=vhd&vdi=OpaqueRef%3Af4a60bad-36da-46cb-a555-8e76ff2f3dd6&session_id=OpaqueRef%3Aba2fcb1d-bc46-445b-9540-b0e9e51236ee&task_id=OpaqueRef%3A8ca30f05-cc13-4c04-907f-6b4468a6ace2","task":{"uuid":"fa99f516-2352-3b92-7419-4950559458d8","name_label":"[XO] Importing content into VDI 0","name_description":"","allowed_operations":[],"current_operations":{},"created":"20230204T06:49:36Z","finished":"20230204T06:54:36Z","status":"failure","resident_on":"OpaqueRef:dba25009-ad21-4aad-a7d6-0b03af29a906","progress":1,"type":"<none/>","result":"","error_info":["VDI_IO_ERROR","Device I/O errors"],"other_config":{},"subtask_of":"OpaqueRef:NULL","subtasks":[],"backtrace":"(((process xapi)(filename ocaml/xapi/vhd_tool_wrapper.ml)(line 77))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/import_raw_vdi.ml)(line 170)))"},"pool_master":{"uuid":"e16ba86e-9915-446d-824e-f2246a2711ad","name_label":"blade7.quape.com","name_description":"Default install","memory_overhead":2050928640,"allowed_operations":["vm_migrate","provision","vm_resume","evacuate","vm_start"],"current_operations":{},"API_version_major":2,"API_version_minor":16,"API_version_vendor":"XenSource","API_version_vendor_implementation":{},"enabled":true,"software_version":{"product_version":"8.2.1","product_version_text":"8.2","product_version_text_short":"8.2","platform_name":"XCP","platform_version":"3.2.1","product_brand":"XCP-ng","build_number":"release/yangtze/master/58","hostname":"localhost","date":"2022-10-12","dbv":"0.0.1","xapi":"1.20","xen":"4.13.4-9.19.1","linux":"4.19.0+1","xencenter_min":"2.16","xencenter_max":"2.16","network_backend":"openvswitch","db_schema":"5.602"},"other_config":{"agent_start_time":"1670522053.","boot_time":"1670465798.","rpm_patch_installation_time":"1670522042.976","iscsi_iqn":"iqn.2022-10.com.quape:88df7156"},"capabilities":["xen-3.0-x86_64","hvm-3.0-x86_32","hvm-3.0-x86_32p","hvm-3.0-x86_64",""],"cpu_configuration":{},"sched_policy":"credit","supported_bootloaders":["pygrub","eliloader"],"resident_VMs":["OpaqueRef:64055ef1-c538-42e9-a880-e6803fe02003"],"logging":{},"PIFs":["OpaqueRef:bb16676d-1c45-4363-9550-ffc6f2323239","OpaqueRef:ae69076b-b4c6-4711-9ab2-95b135daf2d7","OpaqueRef:171c0df8-5b44-459e-8756-01b70115586d","OpaqueRef:12f18eda-5b70-4735-b95e-316fc901d635"],"suspend_image_sr":"OpaqueRef:NULL","crash_dump_sr":"OpaqueRef:NULL","crashdumps":[],"patches":[],"updates":[],"PBDs":["OpaqueRef:08cf61cc-234c-4f62-b006-05fc9f14d27b","OpaqueRef:4dae1d57-a359-40d2-a80d-08acb311c2d3","OpaqueRef:064440da-c058-491d-85b6-e9faa1166878"],"host_CPUs":["OpaqueRef:060a9fa7-302b-4f3e-9125-c51c649cc080","OpaqueRef:13839264-af8f-47dc-b235-533aa28810c7","OpaqueRef:bea9bc59-c762-447c-8145-961af6bf404b","OpaqueRef:ee0ab5bc-3d46-4edc-84b2-cf6f33ce060d","OpaqueRef:8b7690a2-7ac0-4b6e-955c-ede13892942a","OpaqueRef:ed322770-a47f-499e-8c6f-68dbca0627cf","OpaqueRef:4e02a573-5038-4667-b89f-aeef10f64b57","OpaqueRef:f86550ec-15f0-4d85-96e9-e2da1f9fdde3","OpaqueRef:282ccd71-14f2-49ed-b5e3-ad0ad2ea4182","OpaqueRef:02cc7f55-a453-40e9-8285-45a8eb97c4eb","OpaqueRef:a4af195f-2530-42a9-a684-8882097558ab","OpaqueRef:b1ec191e-a995-4c1b-8bac-dde94e355bb9","OpaqueRef:e4a5b5b9-4263-4061-b1a6-2241ff9df33e","OpaqueRef:15a31b91-5daa-4fdd-a128-64222be18599","OpaqueRef:a63dbbc2-43d2-4a17-81de-deef98c4ff70","OpaqueRef:865c00c1-ab16-4c36-83e0-26058bfc9d6f","OpaqueRef:f6c5abf4-b772-427c-b533-9e1dac402dd7","OpaqueRef:e4870fbc-9263-41a4-84fc-bcf747ce6326","OpaqueRef:ac916ff7-e936-4ae4-9841-9e07743881f9","OpaqueRef:029a2790-2795-444f-b0cf-3db6bd0195c2","OpaqueRef:33f83ad1-d849-48e5-995c-7f9b4a696ccd","OpaqueRef:deb97b8b-fff4-4192-ad8f-dfb26f1f8c88","OpaqueRef:dd32e3ce-e147-479b-b0ac-487d25db1384","OpaqueRef:1b9d353c-77bd-4caf-9ae0-64eb2887c06a","OpaqueRef:97fe2a88-fa69-48f6-9f52-a6b3eb39ba10","OpaqueRef:fa1ec86c-2f3a-49d5-b1b0-52787894153e","OpaqueRef:5d5e4dbe-a11f-4d10-aac6-34f806b52317","OpaqueRef:f65eccc8-5fd2-4439-8817-5736ef0c7c08","OpaqueRef:20755a4b-2ce8-4f29-bce8-5e5f058ddc71","OpaqueRef:da6f5842-1388-435c-a741-df424fe3216d","OpaqueRef:504cce7a-66a7-49bd-8c2a-76d093aa5328","OpaqueRef:3801bf53-3bc3-4de9-bdcc-51142bbfcd41"],"cpu_info":{"cpu_count":"32","socket_count":"2","vendor":"GenuineIntel","speed":"3200.329","modelname":"Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz","family":"6","model":"63","stepping":"2","flags":"fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid pni pclmulqdq monitor est ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 bmi2 erms xsaveopt","features_pv":"1fc9cbf5-f6f83203-2991cbf5-00000023-00000001-00000329-00000000-00000000-00001000-8c000400-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000","features_hvm":"1fcbfbff-f7fa3223-2d93fbff-00000423-00000001-000007ab-00000000-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000","features_hvm_host":"1fcbfbff-f7fa3223-2c100800-00000021-00000001-000007ab-00000000-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000","features_pv_host":"1fc9cbf5-f6f83203-28100800-00000021-00000001-00000329-00000000-00000000-00001000-8c000400-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000"},"hostname":"blade7.quape.com","address":"","metrics":"OpaqueRef:1434bd31-37b0-416e-93f1-bdcbddafd324","license_params":{"restrict_vswitch_controller":"false","restrict_lab":"false","restrict_stage":"false","restrict_storagelink":"false","restrict_storagelink_site_recovery":"false","restrict_web_selfservice":"false","restrict_web_selfservice_manager":"false","restrict_hotfix_apply":"false","restrict_export_resource_data":"false","restrict_read_caching":"false","restrict_cifs":"false","restrict_health_check":"false","restrict_xcm":"false","restrict_vm_memory_introspection":"false","restrict_batch_hotfix_apply":"false","restrict_management_on_vlan":"false","restrict_ws_proxy":"false","restrict_vlan":"false","restrict_qos":"false","restrict_pool_attached_storage":"false","restrict_netapp":"false","restrict_equalogic":"false","restrict_pooling":"false","enable_xha":"true","restrict_marathon":"false","restrict_email_alerting":"false","restrict_historical_performance":"false","restrict_wlb":"false","restrict_rbac":"false","restrict_dmc":"false","restrict_checkpoint":"false","restrict_cpu_masking":"false","restrict_connection":"false","platform_filter":"false","regular_nag_dialog":"false","restrict_vmpr":"false","restrict_vmss":"false","restrict_intellicache":"false","restrict_gpu":"false","restrict_dr":"false","restrict_vif_locking":"false","restrict_storage_xen_motion":"false","restrict_vgpu":"false","restrict_integrated_gpu_passthrough":"false","restrict_vss":"false","restrict_guest_agent_auto_update":"false","restrict_pci_device_for_auto_update":"false","restrict_xen_motion":"false","restrict_guest_ip_setting":"false","restrict_ad":"false","restrict_nested_virt":"false","restrict_live_patching":"false","restrict_set_vcpus_number_live":"false","restrict_pvs_proxy":"false","restrict_igmp_snooping":"false","restrict_rpu":"false","restrict_pool_size":"false","restrict_cbt":"false","restrict_usb_passthrough":"false","restrict_network_sriov":"false","restrict_corosync":"true","restrict_zstd_export":"false","restrict_pool_secret_rotation":"false"},"ha_statefiles":[],"ha_network_peers":[],"blobs":{},"tags":[],"external_auth_type":"","external_auth_service_name":"","external_auth_configuration":{},"edition":"xcp-ng","license_server":{"address":"localhost","port":"27000"},"bios_strings":{"bios-vendor":"Dell Inc.","bios-version":"2.15.0","system-manufacturer":"Dell Inc.","system-product-name":"PowerEdge FC630","system-version":"","system-serial-number":"F94H182","baseboard-manufacturer":"Dell Inc.","baseboard-product-name":"0JXJPT","baseboard-version":"A02","baseboard-serial-number":".F94H182.CN7016354O001T.03.","oem-1":"Xen","oem-2":"MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d","oem-3":"Dell System","oem-4":"5[0000]","oem-5":"14[1]","oem-6":"17[6D0BB6C5CA6B4566]","oem-7":"17[6D0C10A7A9419261]","oem-8":"18[0]","oem-9":"19[1]","oem-10":"19[1]","hp-rombios":""},"power_on_mode":"","power_on_config":{},"local_cache_sr":"OpaqueRef:NULL","chipset_info":{"iommu":"true"},"PCIs":["OpaqueRef:d0f241a8-4daa-4501-b3cf-adb0892cc89b","OpaqueRef:a04776fb-5068-4ac7-8bd2-9d5771e1f852","OpaqueRef:92c5c165-8350-4b59-abf5-faed53c18098","OpaqueRef:8aec8abe-10c7-4927-804b-6cbb34e36b0b","OpaqueRef:6b676c66-16f3-484d-a091-ea0dee4bf8a8","OpaqueRef:644d2d29-5dbf-40e7-8416-73005db02084","OpaqueRef:34d72682-8f23-4236-a2c4-d4b2b5d32f26","OpaqueRef:29a4690d-bda1-4d5c-b4d1-17b62bb93df3","OpaqueRef:0779a461-8555-4f99-b967-b2edb5384afd"],"PGPUs":["OpaqueRef:d4dfdda0-964f-4421-9612-8019572b5e80"],"PUSBs":["OpaqueRef:4a3f6f0c-1ba1-4bc2-9d10-eb058a68268f"],"ssl_legacy":false,"guest_VCPUs_params":{},"display":"enabled","virtual_hardware_platform_versions":[0,1,2],"control_domain":"OpaqueRef:64055ef1-c538-42e9-a880-e6803fe02003","updates_requiring_reboot":[],"features":[],"iscsi_iqn":"iqn.2022-10.com.quape:88df7156","multipathing":false,"uefi_certificates":"","certificates":[],"editions":["xcp-ng"]},"SR":{"uuid":"73bf933d-050d-453c-5b43-1f464221678c","name_label":"SSD1","name_description":"","allowed_operations":["vdi_enable_cbt","vdi_list_changed_blocks","unplug","plug","pbd_create","vdi_disable_cbt","update","pbd_destroy","vdi_resize","vdi_clone","vdi_data_destroy","scan","vdi_snapshot","vdi_mirror","vdi_create","vdi_destroy","vdi_set_on_boot"],"current_operations":{},"VDIs":["OpaqueRef:f4a60bad-36da-46cb-a555-8e76ff2f3dd6"],"PBDs":["OpaqueRef:4dae1d57-a359-40d2-a80d-08acb311c2d3"],"virtual_allocation":322122547200,"physical_utilisation":80355328,"physical_size":1007984996352,"type":"ext","content_type":"user","shared":false,"other_config":{},"tags":[],"sm_config":{"devserial":"scsi-357c35481e852d688"},"blobs":{},"local_cache_enabled":false,"introduced_by":"OpaqueRef:NULL","clustered":false,"is_tools_sr":false},"VDI":{"uuid":"ab00cbc9-7da2-443e-b145-b8a907abed3e","name_label":"quape.net 0","name_description":"Created by template provisioner","allowed_operations":["generate_config","update","forget","destroy","snapshot","resize","copy","clone"],"current_operations":{},"SR":"OpaqueRef:fed4ee9c-d397-4087-a4f8-902da574d30c","VBDs":["OpaqueRef:6ef02723-5995-4699-993e-2e0c8bbff868"],"crash_dumps":[],"virtual_size":322122547200,"physical_utilisation":636416,"type":"system","sharable":false,"read_only":false,"other_config":{"content_id":"d111595a-6f0d-53fa-07a9-97ac599cc436","xo:copy_of":"f8004138-6815-4c17-b31c-c00083542337"},"storage_lock":false,"location":"ab00cbc9-7da2-443e-b145-b8a907abed3e","managed":true,"missing":false,"parent":"OpaqueRef:NULL","xenstore_data":{},"sm_config":{},"is_a_snapshot":false,"snapshot_of":"OpaqueRef:NULL","snapshots":[],"snapshot_time":"19700101T00:00:00Z","tags":[],"allow_caching":false,"on_boot":"persist","metadata_of_pool":"","metadata_latest":false,"is_tools_iso":false,"cbt_enabled":false},"message":"VDI_IO_ERROR(Device I/O errors)","name":"XapiError","stack":"XapiError: VDI_IO_ERROR(Device I/O errors)\n at Function.wrap (/opt/xen-orchestra/packages/xen-api/src/_XapiError.js:16:12)\n at _default (/opt/xen-orchestra/packages/xen-api/src/_getTaskResult.js:11:29)\n at Xapi._addRecordToCache (/opt/xen-orchestra/packages/xen-api/src/index.js:954:37)\n at forEach (/opt/xen-orchestra/packages/xen-api/src/index.js:988:14)\n at Array.forEach (<anonymous>)\n at Xapi._processEvents (/opt/xen-orchestra/packages/xen-api/src/index.js:978:12)\n at Xapi._watchEvents (/opt/xen-orchestra/packages/xen-api/src/index.js:1144:14)\n at runNextTicks (node:internal/process/task_queues:60:5)\n at processImmediate (node:internal/timers:447:9)\n at process.callbackTrampoline (node:internal/async_hooks:130:17)"} -
I did quite a number of test and these are all fresh install and the codes for node js fix is there. I'm using the restore function to local SR ( ext4 ) Is there some other codes that needs to be updated ? Anyone have similar issue ?
-
@EddieCh08666741 Hope my above logs can help team to see if there's a bug in the script.
-
I want to share my findings. I did a clean install XOCE on several distros ubuntu,debian and the 5mins restore still happens. I've tried both commits one with the requesttimeout=0 and the other requesttimeout=86xxxx running on node 18. I've installed XOA stable version which is running lower version of node runs perfectly fine always 100% on every restore.
-
want to share that I fix this issue by upgrading NPM to latest stable version.
-
Okay thanks for your feedback. Do you remember the previous npm version and what's your current one?
-
seems i spoke too soon lol. Tested another VM and got this same 5mins A "socket" was not created for HTTP request before 300000ms
-
So. theare is a fix, i have this issue also, clean install
-
I got the same error "Error: A "socket" was not created for HTTP request before 300000ms". My two XCP-ng hosts are 8.2.1 with a clean install & applied all patches. I still don't know how to fix it.
-
i tryed eveything
1 downgrade orchestra nodejs to v16 - same
2 reinstaled xcp-ng clean - same
-
I've been testing with many variations all clean install from XO from sources all stopped at 5mins. installed 3-4 fresh copy. The only one which work consistently is the node 16 xoa stable version.
-
@EddieCh08666741
what branch in git ? -
@arsenieciprian the official supported XOA.
-
Hello,
I'm having the same issue in Delta Backups.
Opened here a topic here https://xcp-ng.org/forum/topic/7007/a-socket-was-not-created-for-http-request-before-300000ms -
@BrunoMorais I had the same error yesterday, I rolled back to a previous commit and it started working again. I don't know the exact commit right now, sorry
(Note that this is not the same error as "vdi_io_error" and the cause it's probably also not the same) -
Are you sure you are on the latest commit on
masterand did a rebuild? -
We'll be happy to know if you can have the exact commit causing the issue, this will tremendously help
-
I did another test on fresh new copy of debian 11. Install XOCE. And run some backup to test.
M packages/xo-server/config.toml
Your branch is up to date with 'origin/master'.
-
@EddieCh08666741 we still have issues to reproduce the issue sadly. So we count on affected users to point to the commit where it started to fail
-
@olivierlambert i'm doing the git bisect . Will update the commit which cause this issue from restore backup and CR issue.
