Taking a cue from Google, Aster Data Systems has come up with an massively parallel processing analytical engine and cluster of commodity hardware for extracting insight from hundreds of terabytes of data. MySpace has deployed 100 nodes of the Aster "nCluster" to load millions of rows per second to surface trends that can help the company fine-tune its services.
Aster nCluster nodes consist of 16GB of RAM, four 250GB SATA disks, and dual-processor quad-core Intel Xeon systems interconnected via 24-port 1Gb Ethernet switches. It works with the popular business intelligence and ETL tools, and it can talk to the standard relational databases.
The secret sauce, according to the company, is patent-pending algorithms and processes for partitioning, balancing, replication, and querying across nCluster. Pricing is based on the amount of customer data processed.
In the tradition of Google, Yahoo, and other Silicon Valley start-ups, Aster was co-founded by three Stanford computer science Ph.D. students and funded by Silicon Valley VCs and angel investors.
Aster is entering a crowded field (see below), but its Google-like approach to data warehousing could reset expectations.