VM Creation Failure | "Expected int64, got 'N'\"
-
kagbasi-ngc said in VM Creation Failure | "Expected int64, got 'N'\":
INTERNAL_ERROR(Failure("Expected int64, got 'N'"))
This is weird, it doesn't ring any bell
I would have a relative confidence on a such unstable setup, the first thing I'd like to rule out is hardware (at least CPU & RAM). Can you run a memtest on this?
-
olivierlambert You mean run a memtest within the OS or via the BIOS? I can certainly do that tonight, I'm in the lab now and will be here for 3-4 hours.
-
olivierlambert So, while troubleshooting this issue, I restarted the Xen Toolstack on the pool master (VMH01).
So I tried creating the VM just to see if perhaps the toolstack restart would've had an effect, and it worked (using fast clone). I then tried without fast clone and it worked as well. Looks like restarting the toolstack on the pool master did the trick - though very concerning why this happened in the first place without any warnings of any underlying issues.
-
Yeah that's weird, but I'm still concerned about the hardware. -
olivierlambert Do you want the memtest on all three hosts, or just the master (VMH01)?
-
olivierlambert So, the memtest of the master host is still ongoing. 1 of 4 passes has completed and no errors have been found. Do you want me to continue and let it run all 4 passes?
One thing that struck me as odd is that even though I have 160GB of RAM installed on this host, when I run
top, it's showing only 2.41GB of total RAM. Is this normal?
-
kagbasi-ngc Yes, top is executed from your Dom0, which is... a VM
When you run memtest, you run it on the entire host. And yes, ideally let it run entirely. -
olivierlambert Awesome, thanks for the explanation - I suspected that. Cool, I'm gonna let it run all 4 passes. Also, should I run the same on the other two hosts?
-
If it's a lab and you can afford the time, at least we'll be on the safe side
-
olivierlambert Cool, I'll run the other two when I get back. I'll kick them off simultaneously.
-
olivierlambert So, as luck would have it, the lab lost partial power around 10pm last night - right when the memtest had reached 98% complete on the 4th pass. Unfortunately, I'm gonna have to run the tests again and let it run over the weekend.
It took about 10 hours to run, so that should be enough hours over the weekend for all three hosts to finish.
-
Ouch
-
olivierlambert yeah, I almost cried when it happened.
-
olivierlambert Memory tests on all three hosts completed successfully; results below.
VMH01 (Master)
VMH02
VMH03
-
Great, so at least we can rule out a memory or CPU issue with memory
-
olivierlambert Yes we can, however, it's unsettling not knowing the root cause. But seems that restarting the Xen Toolstack may have resolved this issue.
I haven't attempted creating a VM yet, so I'll do that when I get in tomorrow.
