- Two cells
- One load balancer (HAProxy)
- Two vCenters
After updating one of our vCloud Director installation to 9.5.0.1 we found an issue in editing rights and roles.
Selecting a specific role to edit, an error occurred: "JSON.parse: unexpected character at line 1 column 1 ".
A 502 error was logged in HAProxy.
In the “known issues” section of vCloud Director 9.5 for Service Providers Release Notes our error was reported: “vCloud Director UI and API users might encounter HTTP 502 errors if the load balancer does not permit large payloads”
This document suggested some workarounds:
- “Set a higher value for vcloud.http.responseHeaderMax on both cells
cell-management-tool manage-config -n vcloud.http.responseHeaderMax -v new-value”
We tried it without success. -
“Set a lower value for restapi.queryservice.maxPageSize on both cells
cell-management-tool manage-config -n restapi.queryservice.maxPageSize -v new-value”
We tried it without success. -
“Configure the load balancer with larger buffers,Configure the load balancer to either not enforce the maximum HTTP header limit or set the limit to a large number. Refer to the documentation for the load balancer that you use.”
Third option did the trick!As we used a HAProxy load balancer, we added these two performance tuning to HAProxy “global” section:tune.bufsizetune.http.maxhdr
We set the first one to 49152 and the second one to 150, but you should adjust them based on your enviroment.
After restarting HAProxy, the issue disappeared!