XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Reverse Proxy Configuration - HAProxy

    Scheduled Pinned Locked Moved Solved Xen Orchestra
    30 Posts 6 Posters 5.8k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • GheppyG Offline
      Gheppy @olivierlambert
      last edited by Gheppy

      @olivierlambert
      It is HTTPS ( not HTTP ) and proxy-protocol-v2, and as far as I know the "Forwarded" directive is not supported on HTTPS, only on HTTP

      delafD 1 Reply Last reply Reply Quote 0
      • delafD Offline
        delaf @Gheppy
        last edited by

        @Gheppy The RFC says that Forwarded must (when active) be set on HTTP requets but not HTTP response. It use HTTP for HTTP and HTTPS in this context, that is confirmed by :

           Information passed in this header field can be, for example, the source IP
           address of the request, the IP address of the incoming interface on
           the proxy, or whether HTTP or HTTPS was used
        

        The syntax describe by mozilla (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded) :

        Syntax: Forwarded: by=<identifier>;for=<identifier>;host=<host>;proto=<http|https>
        
        GheppyG 1 Reply Last reply Reply Quote 0
        • GheppyG Offline
          Gheppy @delaf
          last edited by Gheppy

          @delaf
          Forwarded it seams that is supported with HAProxy 2.8, I use 2.6. I'll install 2.8 from source to see if is ok

          delafD 1 Reply Last reply Reply Quote 0
          • delafD Offline
            delaf @Gheppy
            last edited by

            @Gheppy ok, here even with the Forwarded header (set by an HAProxy 2.8), XO does not log in right IP (in the audit log).

            J 1 Reply Last reply Reply Quote 0
            • J Offline
              john.c @delaf
              last edited by john.c

              @delaf @olivierlambert Maybe worth investigating this to get this corrected so it will log the right IP address in the audit log.

              delafD 1 Reply Last reply Reply Quote 0
              • delafD Offline
                delaf @john.c
                last edited by delaf

                @olivierlambert any idea on what can I do to help to debug this issue?

                Step to reproduce the issue :

                1. Install xo and make it listening on 127.0.0.1:8080
                2. Install HAProxy 2.8 on Debian 12 (check https://haproxy.debian.net/) :
                # curl https://haproxy.debian.net/bernat.debian.org.gpg | gpg --dearmor > /usr/share/keyrings/haproxy.debian.net.gpg
                # echo "deb [signed-by=/usr/share/keyrings/haproxy.debian.net.gpg] http://haproxy.debian.net bookworm-backports-2.8 main" > /etc/apt/sources.list.d/haproxy.list
                # apt-get update
                # apt-get install haproxy=2.8.\*
                # systemctl stop haproxy
                
                1. Configure HAProxy
                  Very small HAProxy config (/etc/haproxy/haproxy.cfg), update the bind line to listen on the IP:PORT you want. XO is listening on 127.0.0.1:8080.
                global
                  log /dev/log local0
                  log /dev/log local1 notice
                  chroot /var/lib/haproxy
                  stats socket /run/haproxy/admin.sock mode 660 level admin
                  stats timeout 30s
                
                  user haproxy
                  group haproxy
                  daemon
                
                defaults
                  mode http
                  log global
                  log-format "%ci:%cp [%t] %ft %b/%s %Th/%Ti/%TR/%Tw/%Tc/%Tr/%Td=%Tt %ST %U %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %sslv %sslc %[last_rule_file]:%[last_rule_line] %ID"
                  error-log-format "%ci:%cp [%tr] %ft %ac/%fc %[fc_err]/%[ssl_fc_err,hex]/%[ssl_c_err]/%[ssl_c_ca_err]/%[ssl_fc_is_resumed] %[ssl_fc_sni]/%sslv/%sslc %{+Q}[fc_err_str]"
                  option dontlognull
                  option redispatch
                  timeout connect 5s
                  timeout client 50s
                  timeout server 50s
                  unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid
                  unique-id-header X-Unique-ID
                
                frontend ft_public
                  bind IP:PORT
                  mode http
                  default_backend bk_xo
                
                backend bk_xo
                  mode http
                  option forwarded
                  option forwardfor
                  option http-server-close
                  http-request add-header HAProxy yes
                  server xo 127.0.0.1:8080 check
                
                1. Start HAProxy: systemctl start haproxy

                2. Test it http://IP:PORT 😉

                3. Check on the network the headers sent by HAProxy to XO tcpdump -ni lo port 8080

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  What's happening on step 5 and what's an example of sent stuff on 6?

                  delafD 1 Reply Last reply Reply Quote 0
                  • delafD Offline
                    delaf @olivierlambert
                    last edited by delaf

                    @olivierlambert

                    • step 5, you get access to XO.
                    • step 6 (I did a curl on http://X.X.X.X:PORT/signin)

                    Request sent by HAProxy:

                    09:34:03.294200 IP 127.0.0.1.34134 > 127.0.0.1.8080: Flags [P.], seq 1:266, ack 1, win 512, options [nop,nop,TS val 2919287515 ecr 2919287515], length 265: HTTP: GET /signin HTTP/1.1
                    E..=^.@.@............V.PR,.5.........1.....
                    ........GET /signin HTTP/1.1
                    host: xoau.ivy1.aquaray.com:8080
                    user-agent: curl/7.74.0
                    accept: */*
                    haproxy: yes
                    x-unique-id: AC1014F0:BCD6_AC1014F0:1F90_6576C97B_000A:1783DC
                    forwarded: proto=http;for=X.X.X.X
                    x-forwarded-for: X.XX.X
                    connection: close
                    

                    Response from XO:

                    09:34:03.296545 IP 127.0.0.1.8080 > 127.0.0.1.34134: Flags [P.], seq 1:2009, ack 266, win 512, options [nop,nop,TS val 2919287517 ecr 2919287515], length 2008: HTTP: HTTP/1.1 200 OK
                    E...t.@.@..`.........P.V....R,.>...........
                    ........HTTP/1.1 200 OK
                    X-DNS-Prefetch-Control: off
                    X-Frame-Options: SAMEORIGIN
                    Strict-Transport-Security: max-age=15552000; includeSubDomains
                    X-Download-Options: noopen
                    X-Content-Type-Options: nosniff
                    X-XSS-Protection: 1; mode=block
                    Content-Type: text/html; charset=utf-8
                    Content-Length: 1464
                    ETag: W/"5b8-OqUsZViW2KwDMOq1IfmEYkCzkN0"
                    Set-Cookie: connect.sid=s%3A3imvxT96Uq9L224R8NpArAHuW5Ho7jOS.ArGj0Ms2cIXEalxqdYSk95JS7J0ihftv%2FcURH53p07A; Path=/; HttpOnly
                    Vary: Accept-Encoding
                    Date: Mon, 11 Dec 2023 08:34:03 GMT
                    Connection: close
                    
                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      Does it ring a bell @julien-f ?

                      julien-fJ 1 Reply Last reply Reply Quote 0
                      • julien-fJ Offline
                        julien-f Vates 🪐 Co-Founder XO Team @olivierlambert
                        last edited by

                        @delaf Can you test the PR https://github.com/vatesfr/xen-orchestra/pull/7233?

                        You will need to set http.useForwardedHeaders to true in your xo-server's config.

                        Let me know if that helps 🙂

                        julien-f opened this pull request in vatesfr/xen-orchestra

                        closed feat(xo-server): http.useForwardedHeaders #7233

                        delafD 1 Reply Last reply Reply Quote 0
                        • delafD Offline
                          delaf @julien-f
                          last edited by

                          @julien-f it works 🙂

                          As it is said in the doc, relying on x-forwarded-* should be done carefuly in a controlled network.
                          @julien-f FYI, some software (like mod_remoteip for apache) use another variable to know from which proxy it is safe to use x-forwarded-* (https://httpd.apache.org/docs/2.4/mod/mod_remoteip.html#remoteiptrustedproxy).

                          BWT thank you!

                          julien-fJ 1 Reply Last reply Reply Quote 0
                          • julien-fJ Offline
                            julien-f Vates 🪐 Co-Founder XO Team @delaf
                            last edited by

                            @delaf We could add the support for an array of IP addresses for useForwardedHeaders if necessary in the future.

                            Do you think it's required to do this now?

                            julien-fJ 1 Reply Last reply Reply Quote 0
                            • julien-fJ Offline
                              julien-f Vates 🪐 Co-Founder XO Team @julien-f
                              last edited by

                              @delaf I've added the support for trusted addresses, if you can test it that would be great 🤗

                              delafD 1 Reply Last reply Reply Quote 0
                              • delafD Offline
                                delaf @julien-f
                                last edited by

                                @julien-f I have both case with a forwarding proxy on 127.0.0.1: useForwardedHeaders = ['127.0.0.1'] and useForwardedHeaders = ['10.0.0.0/8'] and all seems OK 🙂

                                julien-fJ 1 Reply Last reply Reply Quote 0
                                • julien-fJ Offline
                                  julien-f Vates 🪐 Co-Founder XO Team @delaf
                                  last edited by

                                  @delaf Great! Thank you so much! 🙂

                                  delafD 1 Reply Last reply Reply Quote 1
                                  • olivierlambertO olivierlambert marked this topic as a question on
                                  • olivierlambertO olivierlambert has marked this topic as solved on
                                  • delafD Offline
                                    delaf @julien-f
                                    last edited by

                                    @julien-f I think we should add this in the reverse proxy documentaion (https://xen-orchestra.com/docs/configuration.html#reverse-proxy). I have created https://github.com/vatesfr/xen-orchestra/pull/7289

                                    gdelafond opened this pull request in vatesfr/xen-orchestra

                                    closed docs(configuration): useForwardedHeaders #7289

                                    julien-fJ 1 Reply Last reply Reply Quote 0
                                    • julien-fJ Offline
                                      julien-f Vates 🪐 Co-Founder XO Team @delaf
                                      last edited by

                                      @delaf Thank you!

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post