Timestamp lost in Continuous Replication

kratos

@florent
Yes, that is absolutely correct. I have a pool with two members without shared storage. Some VMs run on the master, and some on the second pool member. I replicate between the pool members so that, if necessary, I can start the VMs on the other member. This may not be best practice.

florent

@kratos you probably heard the sound of my head hitting my desk when I found the cause
the fix is in review, you will be able to use it in a few hours

kratos

@florent
I’m a developer myself, so I can totally relate—just when you think everything is working perfectly, someone like me comes along
I’m really glad I could help contribute to finding a solution, and I’ll report back once I’ve tested the new commit. Thanks a lot for your work.

However, this does raise the question for me: is my use case for continuous replication really that unusual?

florent

@kratos no, it's not that rare. I even saw in the wild replication on the same storage (wouldn't recommend it , though )

the cross pool replication is a little harder since the objects are each split on their own xen api, so the calls must be routed to the right one
We tested the harder part, not the mono xapi case

Pilow

@kratos said:

This may not be best practice.

in a two hosts pool, if your replicated VMs live on the Master, and it's gone, you won't be able to start the replicated VMs

you will first need to transition slave to master

indeed CR is better to another pool

ph7

updated to f5468 and it seems to work fine in my home lab lab
I will update my homelab "production" later

I have retention of 2
In XO I only see 1 VM and I think this is intended
I get 2 snaps and I can restore 2 different VMs from them

I think You nailed it

edit: And delta is back

florent

@ph7 that is a good news
thank you for your patience and help

joeymorin

I observed similar behaviour.

Two pools. Pool A composed of two hosts. Pool B is single-host. B runs a VM with XO from source. Two VMs on host A1 (on local SR), one VM on host B1 A2 (on local SR).

Host A2 has a second local SR (separate physical disc) used as the target for a CR job.

CR job would back up all four VMs to the second local SR on host A2.

The behaviour observed was that, although the VM on B would be backed up (as expected) as a single VM with multiple snapshots (up to the 'replication retention'), the three other VMs on the same pool as the target SR would see a new full VM created for each run of the CR job. That rather quickly filled up the target SR.

I noticed the situation was corrected by a commit on or about the same date reported by @ph7.

Incidentally, whatever broke this, and subsequently corrected it, appears to have corrected another issue I reported here. I never got a satisfactory answer regarding that question. Questions were raised about the stability of my test environment, even though I could easily reproduce it with a completely fresh install.

Thanks for the work!

edit: Correction B1 A2

florent

@joeymorin said:

I observed similar behaviour.

Two pools. Pool A composed of two hosts. Pool B is single-host. B runs a VM with XO from source. Two VMs on host A1 (on local SR), one VM on host B1 A2 (on local SR).

Host A2 has a second local SR (separate physical disc) used as the target for a CR job.

CR job would back up all four VMs to the second local SR on host A2.

The behaviour observed was that, although the VM on B would be backed up (as expected) as a single VM with multiple snapshots (up to the 'replication retention'), the three other VMs on the same pool as the target SR would see a new full VM created for each run of the CR job. That rather quickly filled up the target SR.

I noticed the situation was corrected by a commit on or about the same date reported by @ph7.

Incidentally, whatever broke this, and subsequently corrected it, appears to have corrected another issue I reported here. I never got a satisfactory answer regarding that question. Questions were raised about the stability of my test environment, even though I could easily reproduce it with a completely fresh install.

Thanks for the work!

edit: Correction B1 A2

sometimes it's hard to find a n complete explanation without connecting to the hosts and xo, and going through a lot of logs , which is out of the scope of community support

I am glad the continuous improvement of the code base fixed the issue . We will release today a new patch, because migrating from 6.2.2 to 6.3 for a full replication ( source user that updated to the intermediate version are not affected )

kratos

Please excuse my late reply - I was on a fishing trip in Spain in the meantime. However, I did have time to test extensively and could no longer identify any issues.

Thank you for all your work.

P.S.:
@Pilow Thanks for the explanation — I’ll give this some more thought.

olivierlambert

Thank you for your feedback @kratos !