Start Outage: 11:12 CET
End Outage: 14:35 - 16:00 CET depending on machine
11:25 One of our hypervisors is unable to start. Ceph-based machines have already been migrated to other hypervisors and are up and running again. SSD-based machines are currently locked to the affected host. Our technicians are on their way to the data-center to carry out tests directly on the hypervisor. We will inform you immediately if progress can be foreseen.
12:00 The on-site analysis showed that the host is dead; we are currently trying to rescue the SSD images and copy them to other hypervisors.
13:15 In the meantime the SSDs have been removed from the defective hypervisor, we are now trying to install them in another server and migrate the data.
14:30 We found a way to access the images. We are currently copying the images to other hypervisors and trying to start the affected machines
15:30 Most of the machines are already running on other hypervisors. The remaining machines are still being copied and started. Customers with faulty machines will be contacted again separately. No snapshots were imported, but the images were restored from earlier today (latest data on machines).
16:06 All machines have been restored
