Our architecture:

  • Two cells
  • One load balancer (HAProxy)
  • Two vCenters

After updating one of our vCloud Director installation to 9.5.0.1 we found an issue in editing rights and roles.
Selecting a specific role to edit, an error occurred: "JSON.parse: unexpected character at line 1 column 1 ".

A 502 error was logged in HAProxy.

In the “known issues” section of vCloud Director 9.5 for Service Providers Release Notes our error was reported: “vCloud Director UI and API users might encounter HTTP 502 errors if the load balancer does not permit large payloads”
This document suggested some workarounds:

  • “Set a higher value for vcloud.http.responseHeaderMax on both cellscell-management-tool manage-config -n vcloud.http.responseHeaderMax -v new-value”
    We tried it without success.
  • “Set a lower value for restapi.queryservice.maxPageSize on both cells
    cell-management-tool manage-config -n restapi.queryservice.maxPageSize -v new-value”
    We tried it without success.
  • “Configure the load balancer with larger buffers,Configure the load balancer to either not enforce the maximum HTTP header limit or set the limit to a large number. Refer to the documentation for the load balancer that you use.”
    Third option did the trick!As we used a HAProxy load balancer, we added these two performance tuning to HAProxy “global” section:

    • tune.bufsize
    • tune.http.maxhdr

    We set the first one to 49152 and the second one to 150, but you should adjust them based on your enviroment.

After restarting HAProxy, the issue disappeared!