Updated all my hosts but ended up with a bunch of stuck tasks for API host calls, didn't seem too healthy! It looks like they were stuck, kept seeing a host unhealthy power state repeatedly pop up and disappear. I opted to select all tasks and delete, same with my logs (I monitor externally anyway) which appears to have resolved this for the moment. I no longer see these mdadm logs being generated and everything appears normal.

An update, if anyone ever comes across this via search engine. Turns out it was my container's timezone. The image was set to pure UTC, no timezone, by default, so I believe when it was writing files to my network storage it introduced a discrepancy. My network share was recording the file metadata accurately to real-time, so I assume when it came time to do another backup, the file time XO expected was different, making it think it was "stale" or still being "held". Have now run both scheduled metadata and VM backups without any errors . In summary: make sure your time, date and timezone are set correctly!

@DustyArmstrong Got it got it! Makes total sense. I do think making sure you somehow are backing things up in a way that covers large natural disasters is important, not quite sure what the ideal solution here would be though.

@DustyArmstrong Thanks for your report

@DustyArmstrong Perfect, thanks for your feedback

I fixed it by running the following: yarn cache clean yarn --network-timeout 10000000