XO Rest API Supported Queries

DustyArmstrong

I'm trying to use the REST API for XO to pull backup info to display on my Grafana dash, but I'm having some trouble formatting the query to get the result I'm after. I see the documentation says it supports filter and limit but I can't get them to work and it doesn't really specify how to use.

Is this correctly formatted? It just returns all results (500~ or so):

http://xo.localdomain/rest/v0/backups/logs?fields=id,status,jobName?limit=10

Equally, if I want to filter on job name, how can I do that? The documentation gives an example like so home?s=power_state%3Arunning+ but I don't see how I should translate that for jobName.

The query I am trying to construct should filter for job name i.e. Pool Metadata and limit to 10 (newest first).

olivierlambert

Question for @julien-f

julien-f

@DustyArmstrong No, your URL is not correctly formatted, the parameters should be passed in a standard query string which uses ampersand (&) to separate parameters:

http://xo.localdomain/rest/v0/backups/logs?fields=id,status,jobName&limit=10

Regarding your second question, filter is the parameter used for search, ie filtering results. The value is a string that uses the XO filter syntax, and because it is part of an URL, should be properly percent encoded. In JavaScript, you can use the encodeURIComponent() function to achieve that:

encodeURIComponent('power_state:running jobName:"My job"')
// power_state%3Arunning%20jobName%3A%22My%20job%22

olivierlambert

@DustyArmstrong also feel free to share all your work when it works, we'll be happy to see it!

DustyArmstrong

@julien-f Thanks, I did also try with & but it still returns every result (553 rows) so I wasn't sure. The exact query:

http://xo.localdomain/rest/v0/backups/logs?fields=id,status,jobName&limit=10

Is it possible I have an older version of the REST API that doesn't support filter and limit?

Also still unsure how to format the query to filter, I understand it uses XO syntax but how is that actually constructed? I think most of my issue is that I can't troubleshoot as it just returns every row regardless so I don't know what should work.

Are either of these correct? They both still return all 553 results.

http://xo.localdomain/rest/v0/backups/logs?fields=id,status&s=jobName%3A%22Pool%20Metadata%22

http://xo.localdomain/rest/v0/backups/logs?fields=id,status?s=jobName%3A%22Pool%20Metadata%22

julien-f

@DustyArmstrong

The parameter used for filtering is filter not s, if you saw the last one in the documentation, it's an error, please let me know and I'll fix it
Filtering and limiting is not implemented for backup logs yet, I'll do it now.

DustyArmstrong

@julien-f Thanks Julien.

I am just going based off of the REST API docs https://xen-orchestra.com/docs/restapi.html. These refer to https://xen-orchestra.com/docs/manage_infrastructure.html#live-filter-search which is what I've been basing it off as the API docs don't say what formatting to use, I assumed it was the same.

Edit: to be fair I think I probably just misunderstood, the REST docs do say filter but it linked to the syntax page which says use s maybe mixed me up. Ideally if the docs could give quick examples of how to use the parameters that would be great, but obviously that's just a "nice to have" for the inexperienced people like me, not a demand.

DustyArmstrong

@olivierlambert It's very basic for now! I have just created a dash with VM Deltas colored status success/fail by date (the id is of course the epoch so that's handy), a LAST GOOD BACKUP for deltas by hours since i.e. 9 hours ago, same again for Pool Metadata and finally a list of all VMs and their state i.e. Running/Halted.

I am using the Infinity plugin to achieve it. Even without limit and filter it's still very useful to me as I had no good way to map XO backups into Grafana til I discovered the API, so many thanks for implementing! My main reason for wanting limit and filter is so the query returns less overall data (loading faster on the dash hopefully) and so I can separate out each backup job/component a bit better for manipulation (like the last good backup for that specific job).

julien-f

@DustyArmstrong filter and limit are now supported for backups/logs: https://github.com/vatesfr/xen-orchestra/commit/8a26e081027c5410757dcc7ae4ece6ffa7bd81a9

0

julien-f committed to vatesfr/xen-orchestra

feat(xo-server/rest-api): `filter`/`limit` support for `{backups/restore}/logs`

Fixes https://xcp-ng.org/forum/post/64789

DustyArmstrong

@julien-f That's awesome, thanks!

DustyArmstrong

@julien-f Sorry to bring up again, have now updated and the limit is working, but it returns the data in reverse order (oldest backup is shown first). Is there any functionality present in the API at the moment to order it with newest first? If not no issue, I can just go back to doing it how I was.

olivierlambert

Good question for @julien-f
He's not available right now, but I will remind him to take a look when he's around

DustyArmstrong

@olivierlambert Thanks! I really do appreciate the quick responses and implementations. The filter works perfectly - so does the limit of course, it's just the ordering the way it is restricts its usefulness as I need to see the most recent backup info rather than the oldest.

I did try playing around with the web UI, you can sort by ID in Backup - /backup/overview?s_logs=1_0_desc- - but didn't know how/if it translated to the API.

julien-f

@DustyArmstrong I'm reluctant to add a sort feature, but I think it makes sense for the limit to be applied at the end of the sorted collection

DustyArmstrong

@julien-f EDIT: I have just seen your commit from yesterday that looks to resolve my below post! Sorry I read your comment about sorting as the limit ordering ("sorting") wouldn't be implemented. My query around "sorting" was in reference to exactly what you have implemented so, many thanks!

@DustyArmstrong said in XO Rest API Supported Queries:

@julien-f Sorry to bring up again, have now updated and the limit is working, but it returns the data in reverse order (oldest backup is shown first). Is there any functionality present in the API at the moment to order it with newest first? If not no issue, I can just go back to doing it how I was.

julien-f

@DustyArmstrong Perfect, thanks for your feedback