I've recently written about using the Cloud for Flexiscale had that took their Cloud storage offline., as well as the potential for data loss on Cloud based services should things go wrong. Case in point, service provider
However, thanks to some good planning and well-managed processes, customer data is getting back online and the company hasn't seen any loss of data.
The update email from CEO Tony Lucas below:
This is an update in regards to the FlexiScale service problems and the progress we have made since my last e-mail.
As I mentioned in my previous e-mail we were awaiting the delivery of new storage before we could start the restoration of services. This arrived yesterday lunchtime and was configured and installed as soon as possible with the help of the staff at the datacentre.
Once this was complete the storage was configured and tested both by ourselves and the storage manufacturer to ensure it was all working correctly. We then performed a number of test restores of servers (using internal test servers, not customer data). This went successfully but took longer than anticipated.
In the early hours of this morning testing was completed on the system to restore all customer data and around 7am this morning the first customer data was restored and servers put online. It was observed at this time that although Linux servers were working correctly, Windows servers wouldn't boot (even though all the data was intact). Unfortunately we had to slow down the restore of servers while we diagnosed these problems.
We have now successfully diagnosed and solved these problems and successfully started to restore all configurations of server. This is now ongoing and a number of customers are fully up and running again. Unfortunately due to the pure amount of data this will still take time to get everyone up and running, but all the problems have now been resolved in making this happen. I should reaffirm at this point that we have not seen any permanent data loss for the customers we have restored so far.
Staff members will be monitoring this closely around the clock until it is complete, although functionality to control servers directly will be disabled to avoid confusing the process.
I sincerely apologise for the time this has taken, and I will be publishing a full report based on the investigation we will undertake next week of this weeks events and the changes we will be making in the future. I will also be making a further announcement in regards to compensation for the problems.
I would like to thank you for your patience during this time, and a sincere thanks to all our staff who have worked throughout the days and nights to get things working again. I'm very proud to have them as fellow colleagues.
Chief Executive Officer