CentOS 8.x Cloudstack 4.15 Troubleshooting

<yambe:breadcrumb self="Cloudstack 4.15 Troubleshooting">CentOS 8.x Cloudstack 4.15|Cloudstack 4.15</yambe:breadcrumb>

CentOS 8.x Cloudstack 4.15 Troubleshooting

Generic troubleshooting suggestions

Agent troubleshooting steps

In general validate:

1. Check status of cloudstack-agent or libivrtd etc. services

   systemctl status cloudstack-agent
   systemctl status libvirtd

2. If the above services are stopped / failed then look at their logs

   journalctl -xe -u cloudstack-agent
   journalctl -xe -u libvirtd

3. Look at agent log files

   tail -f /var/log/cloudstack/agent/agent.log | grep -i 'warn\|error'

4. Validate that required primary storages are mounted and writable / accessible etc.

   df -y

5. Validate that you are able to list VMs using virsh

   virsh list --all

6. If this is new setup validate general setup steps related to disable selinux, disable firewall, installation of packages including java, are done correctly.

   Especially validate network configuration of bridges cloudbr0, cloudbr1 etc. is correct.

7. If host was removed from a running cloudstack-manager and is planned to be readded look at CentOS 8.x Cloudstack 4.15 Re-add existing cloudstack KVM host to cloudstack manager

Manager troubleshooting steps

In general validate:

1. Check status of cloudstack-management services

   systemctl status cloudstack-agent
   systemctl status libvirtd

2. If the above services are stopped / failed then look at their logs

   journalctl -xe -u cloudstack-agent
   journalctl -xe -u libvirtd

3. Look at agent log files

   tail -f /var/log/cloudstack/agent/agent.log | grep -i 'warn\|error'

4. Validate that secondary storage is automatically mounted

   df -y

internal error: unknown feature amd-sev-es error on libvirtd

If you see following error in "systemctl status libvirtd" output:

Jun 11 15:32:57 IN-HYD-STACK01 libvirtd[11346]: internal error: unknown feature amd-sev-es
Jun 11 15:32:57 IN-HYD-STACK01 libvirtd[11346]: Failed to connect socket to '/var/run/libvirt/virtlxcd-sock': No such file or directory
Jun 11 15:32:57 IN-HYD-STACK01 libvirtd[11346]: End of file while reading data: Input/output error

Other error lines:

error: failed to get emulator capabilities
error: internal error: unknown feature amd-sev-es

Then solve it using:

mkdir -p /etc/qemu/firmware
touch /etc/qemu/firmware/50-edk2-ovmf-cc.json
systemctl restart libvirtd
systemctl status libvirtd
systemctl restart cloudstack-agent

Refer:

• https://serverfault.com/questions/1065246/virsh-reporting-unknown-feature-amd-sev-es

Cloudstack system router fails to start

We found that router for a shared network failed to start when the gateway of that shared network was not available on the specified VLAN. After creating the gateway in VLAN we could start the router using:

1. Go to Network -> Guest networks
2. Select network for which router is not starting.
3. Select three dots and choose option to Restart network. Hover mouse over mutiple icons to identify which one of them is for restart.
4. In the popup dialog Enable Clean up
5. Click ok to destroy current router and deply a fresh router
6. This time the router should pass health check. To see router health check status go to
   1. Network -> Guest networks
   2. Click on the name of guest network.
   3. Go to Virtual routers tab
   4. Click on the name of router
   5. Go to Health checks tab

<yambe:breadcrumb self="Cloudstack 4.15 Troubleshooting">CentOS 8.x Cloudstack 4.15|Cloudstack 4.15</yambe:breadcrumb>