Peter Molyneux and his team of six programmers at 22Cans scrambled to cope when their first game, Curiosity, arrived on Apple's App Store a day early and high demand walloped the company's servers.
Here's a blow-by-blow look at how the English startup struggled through the launch and eventually recovered its poise. For a look at Molyneux' plans for the cube-tapping game, check our related.
October 31: 22Cans, a startup closely watched in gaming circles based on Molyneux's decades-long track record, posts on its Facebook page that its first game, Curiosity, will arrive at 22 minutes past midnight Pacific time on November 7. "Joining hopefully thousands of people on the journey to the center of the cube is going to be a fascinating experiment that we will share with the world. Remember you could be the single person to find out what TRULY amazing thing lies in the center," the company says.
November 6: at 2 a.m., a day earlier than expected, Apple publishes Curiosity. "That was a surprise, Molyneux tweets as he scrambles to get back to England from a conference in Israel. "Little surprised at early iOS release BUT team are pushing as hard as they can to get Android out today," he adds, and indeed the programmers release Curiosity for Android, too.
November 6: Early signs of popularity emerge. "We are watching the sun rise with registrations and installs of Curiosity. We didn't dare hope there'd be this many curious people," 22Cans tweets. Within three hours of launch, there are 100,000 players, much more than 22Cans expected.
November 6: Early problems rear their heads as players struggle to reach 22Cans' servers and their apps lose track of how many gold coins they've accumulated and people have trouble logging in using Facebook. "Coins won't be lost, our servers are just playing catch up with all the curious people. Facebook delay happens for same reason," 22Cans tweets.
November 6: Molyneux's mind boggles more: "I have just been glancing at the stats, amazing results. Can't believe 625 people have purchased the 1 million chisel. And 2bn coins spent," he tweets. Players earn coins by tapping tiny "cubelets" that make up the six faces of a single large virtual cube; tapping a single cubelet gets a single coin, but there are ways to multiply earnings, and buying tools with the coins can accelerate people's cubelet destruction abilities.
November 7: 22Cans starts trying to work out problems. "I now understand what's going on, basically we and our server are overwhelmed by the number of people trying out the experiment," Molyneux tweets. "
November 7: The all-nighters begin. "After a little cry I am immensely grateful to everyone for their patience, the six of us are working overnight to keep the service going," he says. "Everyone please remember something as concurrent as this has never been attempted before, and we're just a tiny company," he adds. Concurrency refers to the fact that thousands of people are tapping into the central servers at the same time, and those servers collectively must maintain and share a single state of Curiosity's cube.
November 7: 22Cans tries to rework the Curiosity servers as people are playing the game. "What makes this so scary, because we want to keep the cube active we are having to make changes on the server live. Praying to the gods," tweets Molyneux.
November 7: There's no shortage of backlash by frustrated players. "WHAT THE HECK!? I want my coins back! I was at 980k and it turned back to 420k !!!!!!!!" tweets Stefano Bettega.
November 8: The first layer of the cube is chipped fully away. It consisted of 100,614,152 cubelets.
November 9: 22Cans offers a more elaborate apology on Facebook. "We are learning so much from our first experiment Curiosity but are extremely disappointed with ourselves and the connection/coin issues caused by the overwhelming number of curious people. The team shares your frustration but please hear it from them directly...What has blown us away is the number of taps each player wants to do, we estimated it would be hundreds, but it is actually thousands, and in a few cases millions. We simply did not anticipate this level of curiosity but are doing everything we can to have this resolved over the weekend," 22Cans writes.
November 9: 22Cans offers a video tour on YouTube of its bleary-eyed programming team, an abjectly apologizing Molyneux, and a glimpse at a recrafted server approach designed to scale better.
November 10: The second layer of the cube is removed.
November 13: Signs of progress emerge, though many players still have problems. 22Cans' new approach involves a single server balancing workload across a group of 14 other servers. "The coders here at 22cans did another 36-hour marathon. We now have hugely improved servers. #curiosity is starting to behave like it should," Molyneux tweets. "People should now see the cube update smoother, and our priority is to look at the coins," 22Cans adds.
November 13: Faced with rising server costs, 22Cans adds a PayPal donation button to its home page, raising about $900. It later scraps the idea, deciding it's not in keeping with the company's approach.
November 13: The release of Curiosity 1.1 strips out some features in an effort to improve reliability.
November 14: Eight days after launch, players obliterate the sixth layer of cubelets.
November 16: 22 Cans reports that more than 400,000 people have logged in to Curiosity.
November 17: Eleven days in, the 16th layer of Curiosity's cube is removed.
November 20: Another apology on Facebook. "The point of this communication is to take responsibility and admit that we messed up by misjudging the performance of the system under the load when we went live," 22Cans says. They pared back every optional service, and one server thrown under the bus synchronized coin totals across multiple devices and linked with Facebook. "The new servers have now greatly improved the connection issue and you will be able to join in when you want to. However, the coin reconciliation is not online which means that if you upgrade, uninstall, or press the Facebook button you will lose coins," 22Cans warns.
November 23: Curiosity 1.2 fixes the coin-loss issue. 22Cans considers ways to restore coin balances, including scouring its multi-terabyte log of tap events and a special layer later in the cube.
November 25: Twenty days in, 43 layers have been removed.
November 27: Curiosity is getting back on an even keel, but Molyneux can't catch a break. "Food poisoning wiped me out over the weekend, good for weight loss, bad for everything else," he tweets.
November 28: The 53rd layer is removed just after 4 p.m. PT. More than 95 million cubelets remain on the 54th layer. Molyneux says there are hundreds more layers to go to completely demolish all 64 billion cubelets that make up the cube.