XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. norpan
    N
    Offline
    • Profile
    • Following 0
    • Followers 0
    • Topics 2
    • Posts 4
    • Groups 0

    norpan

    @norpan

    1
    Reputation
    1
    Profile views
    4
    Posts
    0
    Followers
    0
    Following
    Joined
    Last Online

    norpan Unfollow Follow

    Best posts made by norpan

    • RE: Delta backup stuck on "Clean VM Directory" for a long time

      That sounds interesting, the backups are on a new remote on a new Minio-instance so the issue is handled for now. I guess we will see over time if the backup times increases.

      posted in Backup
      N
      norpan

    Latest posts made by norpan

    • RE: Delta backup stuck on "Clean VM Directory" for a long time

      That sounds interesting, the backups are on a new remote on a new Minio-instance so the issue is handled for now. I guess we will see over time if the backup times increases.

      posted in Backup
      N
      norpan
    • RE: Delta backup stuck on "Clean VM Directory" for a long time

      One thing I've notices is that if I do a trace in Minio I see no S3-requests but suddenly it shows a couple of s3.list_objects_v2 witch gives status 499 and the duration is just about 10 minutes. But this gives no timeoutin the backuplog.

      Are you thinking about test a public provider such as AWS?
      I'm not sure we are able to do so. One thing I'm about to do is to test a newer version of Minio in docker on another server running Ubuntu 24.04
      As of now it's an older version running native on a OmniOS server ontop of ZFS-storage.

      Is there a way to get more verbose logging from the backup job?

      posted in Backup
      N
      norpan
    • Delta backup stuck on "Clean VM Directory" for a long time

      Hi,

      I have problems with delta backup which take a long time to finish.
      The remote is in a local Minio instance to buckets with a retention-policy set to get immutable backups.
      I has been working fine for about 4 month but it's after that it got bad.

      First of all, I'm not sure it's related to the long backup times but I get some of these in the logs and my guess it's because with retention enabled Minio keeps empty directories/prefixes until the deleted files exceeds than the retention period:

      no alias references VHD
      

      Transfer and Merge seems quite ok and I think it is "Clean VM Directory" which is the culprit, it can go on for hours.
      At first I thought the long backup times could be related to performance on Minio but I can't see any S3 requests at that time.
      During my troubleshooting I start to think it's the backup-process which is idle or waiting and will continue after a timeout but I don't know how to debug this further.
      There have been a few timeout logged:

                  "message": "Connection timed out after 600000 ms",
                  "stack": "TimeoutError: Connection timed out after 600000 ms\n    at ClientRequest.<anonymous> (/opt/xen-orchestra/node_modules/@aws-sdk/node-http-handler/node_modules/@smithy/node-http-handler/dist-cjs/set-socket-timeout.js:7:30)\n    at Object.onceWrapper (node:events:632:28)\n    at ClientRequest.emit (node:events:518:28)\n    at ClientRequest.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)\n    at TLSSocket.emitRequestTimeout (node:_http_client:863:9)\n    at Object.onceWrapper (node:events:632:28)\n    at TLSSocket.emit (node:events:530:35)\n    at TLSSocket.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)\n    at Socket._onTimeout (node:net:604:8)\n    at listOnTimeout (node:internal/timers:588:17)"
      

      I have adjusted Concurrency both up and down, with a lower value the timeout seems to be avoided, but even without there errors the backup times is just as large.

      Some assistance would be much appreciated.

      posted in Backup
      N
      norpan
    • OmniOS won't boot on xcp-ng 8.1 and 8.2

      I just posted this in omnios-discuss but I'll try here as well, not sure if I'm picked the correct category.

      OmniOS guest fails to boot in xcp-ng on version 8.1 and 8.2.

      Installation is fine: create a new vm, set viridian=false and installation progress smoothly. I've noticed that installation seem to lack PV-drivers.

      When booting from freshly installed disk it panics with:
      panic[cpu0]/thread=ffffffffffbc490c0: unable to configure /xpvd nexus

      I've tried several version of OmniOS (and latest OpenIndiana), tried disabling apix with apix_enable=0 in /etc/system and all without success.
      The same setup in xcp-ng 8.0 works like a charm with working PV-drivers (but no tools).

      Anyone with an idea of a workaround or even better, a resolution to this.

      Best regards
      Niclas

      posted in Compute
      N
      norpan