I just tried to install Ubuntu 24.04 to test it out, and I experienced the same problem with it not recognizing the IP address. I was first using the Ubuntu-provided package (xe-guest-utilities=7.20.2-0ubuntu1), which was failing. I then tried the package I had been using with my Ubuntu 22.04 servers that used to be part of the XCP-ng guest-tools.iso (xe-guest-utilities_7.20.0-9_amd64.deb) and had the same results. I mounted my current guest-tools.iso, which now has xe-guest-utilities_7.30.0-11_amd64.deb, and installed it. Now it was retrieving the IP address correctly. I'm not sure why the OP was still having trouble with that version (I'm using UEFI instead of BIOS, but I wouldn't think that would matter).
I went ahead and tried out the Rust-based tools mentioned (xen-guest-agent_0.4.0_amd64.deb), and it was properly getting the IP address as well. I'm guessing there's some incompatibility (probably with the 6.x kernel) that was fixed between 7.20 and 7.30 (intentionally or accidentally).
Given how much the Linux tools have changed over the years and the fact that they're not used for PV drivers anymore, is there a particular reason to use one over the other (legacy vs Rust)? What features do they really provide now? Is it just CPU/memory/disk/network status?