XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XOA json-rpc call basic exemple

    Scheduled Pinned Locked Moved Xen Orchestra
    12 Posts 4 Posters 1.9k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • mco-systemM Offline
      mco-system
      last edited by

      Hello,

      This is my first post here, I apologize for its length.

      I'm trying to understand how to use xo-api call to perform some actions programmatically.

      I'm aware of how to use a ReST API but I don't know much about JSON-RPC.

      So I looked into xen-orchestra code base (expecially xo-cli, xo-lib, xo-server) and also the xen-orchestra terraform provider which I already use with success.

      Unfortunately I'm not enough well versed in JS nor Go to really understand how to correctly request the JSON-RPC API, not even even to sucesfully connect to the XOA (from source) instance via JSON-RPC.

      In fact, I'm not even sure of the exact exposed URL of the JSON-RPC API !

      I tried with my favorite language, Python3, using "jsonrpcclient" to connect passing module without success.

      # Very basic json-rpc auth attempt
      
      from jsonrpcclient import request
      
      # url = 'wss://xoa.local.example.com'
      # url = 'wss://xoa.local.example.com/api'
      # url = 'https://xoa.local.example.com'
      url = 'https://xoa.local.example.com/api'
      credentials = {'email': 'test@example.com', 'password': 'testme'}
      
      response = request(url, "signin", credentials=json.dumps(credentials)
      

      The above code fails on a misinterpretation of the XO server response. I guess the response is not JSON formated but more something like HTTP 302 response.

      I probably miss some very basic knowledge but can't find any direction.
      I need to know:

      • What precise URL to call
      • What form and content should have the given parameters

      Does someone could be kind enough to post a basic code snippet in another language than JS or Go that use the XOA JSON-RPC, even with plain cURL ?

      A sample like the following one would help me a lot to understand (the following code snippet is not working):

      curl --include \
           --no-buffer \
           -H "Host: xoa.local.example.com:443" \
           -H "Origin: https://xoa.local.example.com:443" \
           -H "Upgrade: websocket" \
           -H "Connection: Upgrade" \
           -H "content-type: application/json" \
           -H "Accept: application/json" \
           -X POST https://xoa.local.example.com/api \
           -d '{"method":"signin","credentials":{'email': 'test@example.com', 'password': 'testme'}}'
      
      
      julien-fJ 1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Pinging @julien-f

        1 Reply Last reply Reply Quote 0
        • julien-fJ Offline
          julien-f Vates 🪐 Co-Founder XO Team @mco-system
          last edited by

          @mco-system I don't know how to do a WebSocket call with cURL, it's probably not a simple HTTP request.

          1 Reply Last reply Reply Quote 0
          • julien-fJ Offline
            julien-f Vates 🪐 Co-Founder XO Team
            last edited by

            Maybe try something like https://github.com/websockets/wscat, though I have no experience with it.

            mco-systemM 1 Reply Last reply Reply Quote 0
            • mco-systemM Offline
              mco-system @julien-f
              last edited by

              @julien-f Thank you for your reply.

              I tried wscat and I can see I'm successfully connected at TCP level as I would with a basic telnet client but then I have no response from the server at all. until timeout occurs.

              Though, the xo terraform provider is working without problem, so I guess I'm doing something wrong or miss a very basic step at protocol level (i'm very new at using json-rpc).

              I also don't get how does the authentication mechanism work.
              Where and when send credentials to the API ?

              /me is currently reading json-rpc specification

              mco-systemM 1 Reply Last reply Reply Quote 0
              • mco-systemM Offline
                mco-system @mco-system
                last edited by

                I successfully sniffed traffic between the XOA Terraform provider and I have now a better understanding of what a JSON-RPC requests looks like.

                However, I'm still unable to connect to the XO host with wscat which systematically timeouts .
                I can't get the Connected (press CTRL+C to quit) prompt from wscat as I would if the connection was successful.

                However, on XO host, I can see traffic using tcpdump, the problem does not seem network, nor SSL related.

                xoa:/opt/xo/xo-server# tcpdump -nni eth0 host 192.168.2.1 and not port 22
                tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
                listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
                
                
                00:28:35.456615 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [S], seq 3784699052, win 64240, options [mss 1460,sackOK,TS val 3147372683 ecr 0,nop,wscale 7], length 0
                00:28:35.456660 IP 192.168.2.251.443 > 192.168.2.1.49074: Flags [S.], seq 962165778, ack 3784699053, win 65160, options [mss 1460,sackOK,TS val 4217115831 ecr 3147372683,nop,wscale 7], length 0
                00:28:35.456904 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [.], ack 1, win 502, options [nop,nop,TS val 3147372684 ecr 4217115831], length 0
                00:28:35.457104 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [P.], seq 1:372, ack 1, win 502, options [nop,nop,TS val 3147372684 ecr 4217115831], length 371
                00:28:35.457120 IP 192.168.2.251.443 > 192.168.2.1.49074: Flags [.], ack 372, win 507, options [nop,nop,TS val 4217115831 ecr 3147372684], length 0
                00:28:35.460902 IP 192.168.2.251.443 > 192.168.2.1.49074: Flags [P.], seq 1:1672, ack 372, win 507, options [nop,nop,TS val 4217115835 ecr 3147372684], length 1671
                00:28:35.461216 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [.], ack 1672, win 495, options [nop,nop,TS val 3147372688 ecr 4217115835], length 0
                00:28:35.462613 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [P.], seq 372:700, ack 1672, win 501, options [nop,nop,TS val 3147372690 ecr 4217115835], length 328
                00:28:35.463727 IP 192.168.2.251.443 > 192.168.2.1.49074: Flags [P.], seq 1672:2214, ack 700, win 505, options [nop,nop,TS val 4217115838 ecr 3147372690], length 542
                00:28:35.464040 IP 192.168.2.1.49074 > 192.168.2.251.443: Flags [.], ack 2214, win 501, options [nop,nop,TS val 3147372691 ecr 4217115838], length 0
                

                In xo-server config.toml, I turned verboseApiLogsOnErrors to true and restarted the xo-server service but the log does not show any connection attempt.

                It's just like the connection is lost by the xo-server... I'm very confused...

                1 Reply Last reply Reply Quote 0
                • mco-systemM Offline
                  mco-system
                  last edited by

                  Using opensll s_client I can connect and upgrade to WebSocket without any difficulties:

                  openssl s_client -crlf -connect xoa.local.example.com:443 -servername xoa.local.exampe.com
                  CONNECTED(00000003)
                  depth=1 C = NC, ST = Somewhere, L = Shell-city, O = Example Co, OU = Lab, CN = Root CA example.com
                  verify return:1
                  depth=0 C = NC, ST = Somewhere, O = Example Co, CN = xoa.example.com
                  verify return:1
                  [...]
                  
                  GET /api/ HTTP/1.1
                  Host: xoa.local.example.com:443
                  User-Agent: Go-http-client/1.1
                  Connection: Upgrade
                  Sec-WebSocket-Key: 7lcEMScTa7sTRzwW8jebLQ==
                  Sec-WebSocket-Version: 13
                  Upgrade: websocket
                  
                  HTTP/1.1 101 Switching Protocols
                  Upgrade: websocket
                  Connection: Upgrade
                  Sec-WebSocket-Accept: 5pKEtEmZTJbNiFx/gZ9VIMVy2B8=
                  

                  And the corresponding line in syslog

                  Apr 30 02:22:04 xoa xo-server[31752]: 2021-04-30T02:22:04.785Z xo:main INFO + WebSocket connection (192.168.2.1)
                  

                  So everything should work but I'm still missing something that I'm pretty sure is very simple. Some basic stuff that I can not see by myself....
                  Very frustrating...

                  julien-fJ 1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Don't worry @mco-system @julien-f will answer when he's around šŸ™‚

                    1 Reply Last reply Reply Quote 0
                    • julien-fJ Offline
                      julien-f Vates 🪐 Co-Founder XO Team @mco-system
                      last edited by

                      @mco-system I don't know wcat, I just mentioned it because it looked interesting.

                      I cannot explain how the WebSocket protocol works, all I can tell you is that XO API is JSON-RPC inside a WebSocket connection, and that before being able to use methods, you need to sign in via the session.signIn method:

                      # Open WebSocket connection to http://xoa/api/
                      → {"id":0,"jsonrpc":"2.0","method":"session.signIn","params":{"username":"jsmith","password":"passw0rd"}}
                      ← {"id":0,"jsonrpc":"2.0","result":{"id":"ad2593b0-97c7-446f-bc0e-f5558c013d52","email":"jsmith","groups":[],"permission":"admin","preferences":{}}}
                      

                      If all you need is a CLI, you can use xo-cli.

                      mco-systemM 1 Reply Last reply Reply Quote 1
                      • mco-systemM Offline
                        mco-system @julien-f
                        last edited by

                        @julien-f Thank you for your help.

                        I finally figured out that I missed the API endpoint trailing / which seems mandatory.

                        My test with openssl s_client worked fine because the trailing / was in place:

                        GET /api/ HTTP/1.1
                        

                        To answer your advice about xo-cli, I don't need a CLI tool and xo-cli is fine when I need one.

                        My goal is to determine what would be the needed efforts to code something like an xo-api Python connector that would allow requesting xo-server from Python apps and scripts.

                        Such a connector would be useful to me but also to all Python coders.
                        I'm seriously considering it.

                        Thank you again for your time, it could have take me days to figure out this missing trailing /.

                        M 1 Reply Last reply Reply Quote 0
                        • M Offline
                          mbaron @mco-system
                          last edited by

                          Hi @mco-system,

                          I propose to you a solution to communicate from a Python program to xo-server api.

                          import json
                          import aiohttp
                          import asyncio
                          
                          from jsonrpc_websocket import Server
                          
                          async def routine():
                              async with aiohttp.ClientSession() as client:
                                  server = Server('ws://XO_SERVER_IP/api/', client)
                          
                                  await server.ws_connect()
                          
                                  # No signIn required
                                  methodsInfoResult = await server.system.getMethodsInfo()
                                  print('\n'.join([str(e) for e in methodsInfoResult.keys()]))
                          
                                  # signIn required
                                  result = await server.session.signIn(username='YOUR_LOGIN', password='YOUR_PASSWORD') # email attribute is working in place of username
                                  result = await server.xo.getAllObjects(filter={"type": "VIF"}, limit=10)
                          
                                  print('[')
                                  print(', \n'.join([str(json.dumps(e, indent=4)) for e in result.values()]))
                                  print(']')
                          
                          asyncio.get_event_loop().run_until_complete(routine())
                          

                          This code is using the jsonrpc_websocket library.

                          A full description is available on my blog => https://mickael-baron.fr/blog/2021/05/28/xo-server-websocket-jsonrcp (in french). A Java version is also available.

                          1 Reply Last reply Reply Quote 1
                          • mco-systemM Offline
                            mco-system
                            last edited by

                            Hello @mbaron,

                            Your blog helped me to get a better understanding of the different steps involved in jsonrpc.

                            Thank you for sharing your code samples.

                            1 Reply Last reply Reply Quote 1
                            • B Butcat referenced this topic on
                            • G Gurve referenced this topic on
                            • First post
                              Last post