Shopping in data warehouse market
Toolmakers can use their technology to load up the increasingly popular, yet labor intensive, data warehouses and data marts.
The companies are starting to eye the data migration market as a new frontier for their niche products. After all, data mining and report writing are nothing more than pulling data from various sources and then translating and formatting it for a specific use.
So why not use that same technology to load up the increasingly popular, yet labor-intensive, data warehouses and data marts that are popping up in nearly every corporate division and department?
"There is a real opportunity for a product like this. It solves the real problem of getting data out of legacy systems, which is time-consuming and expensive," said Marc Peterson, senior vice president of North American operations at DataWatch of Wilmington, Massachusetts.
Next week, DataWatch plans to announce a new migration tool called Monarch Data Pump, a tweaked version of its data-reporting product.
Peterson said many data-migration jobs require users to understand the underlying technology of the various sources from which the data is being pulled. But the technology behind reporting tools does that work for a user, easing the process.
"These reporting tools already contain metadata and business rules, and there is value and knowledge inherent in them," he said. "What our product does is leverage that aspect of reporting tools."
But companies may have much of this type of technology already in their hands, according to Brian Murphy, analyst at the Yankee Group in Boston. Murphy said packaged data marts and warehouses come with migration tools. He also said that, to some degree, reporting toolmakers market their products to this segment.
However, Murphy added that DataWatch seems to be the first among reporting toolmakers to get out of the gate with a product specifically for data migration. He said there may be an advantage in using a tool that starts with reporting and backtracks to the data warehouse rather than the other way around, because it gives more platform independence.
"There's probably standard utilities out there that achieve the same thing, but the advantage here is it's packaged and it's integrated--a one-stop shop," Murphy said. "It makes perfect sense. It's not unusual for people to find new uses for preexisting products."
The tweaks involved in transforming a report-writing tool to a data migration tool include a scripting facility, which tells a system, among other things, how to map to a database. Scheduling functionality was also added so that jobs could be triggered based on different events as well as the addition of logging and error handling functionality.
Many companies are now attacking the problem of gathering data from the multitude of systems in their organization to set up data warehouses and data marts. The data often resides in old mainframe systems or a plethora of client-server applications spread around the company, each with its own unique way of formatting and storing data.
The need for migration tools is also being fueled by the ever-deepening database wars as Oracle, Microsoft, IBM, Sybase, and Informix Software fight for a piece of the market.
Dataquest in San Jose, California, estimates the data warehousing industry grew 34 percent in 1997, with revenue reaching $1.47 billion, with the fastest-growing segment in the data warehousing market being data marts, which grew 205 percent in 1997 with revenue reaching $61.6 million, up from $20.2 million in 1996.
Dataquest predicts the market will reach $1.88 billion this year as growth is spurred by the entrance in the market of relational database vendors, such as Oracle, IBM, and Microsoft, as well as new offerings from enterprise resource planning vendors like SAP.
Monarch Data Pump is to debut Monday and will sell for $29,995 for the server-based software. It runs on Windows NT but extracts and pulls data from most platforms.