Microsoft unpacks details of Longhorn storage

The Longhorn edition of Microsoft's Windows operating system is at least two years away--but the company is revealing some details on how it intends to create a smooth transition from today's Windows PCs.

One of the most significant enhancements to Longhorn is a data storage system called WinFS, technology designed to make information easier to find and view. Clearing up long-standing confusion, a Microsoft senior vice president said that WinFS will work with--not replace--the existing file system in Windows, called NTFS, when WinFS debuts in late 2005 or 2006.

WinFS "uses NTFS," Bob Muglia, senior vice president of Microsoft's enterprise storage and enterprise management divisions, told CNET News.com. "We built on top. NTFS does what it does incredibly well."

Successful co-existence of different file systems is important to ensuring a clean--and potentially quicker--transition to Longhorn, analysts say. A new file system that breaks with the storage system in Windows PCs today could be disruptive to end users. Also, Longhorn applications could encounter compatibility problems with older Windows applications, causing problems for commercial software providers.

NTFS is only one component of the revamped storage system in WinFS. Another key building block is the querying capabilities of Microsoft's SQL Server relational database, according to Microsoft. WinFS also will incorporate the data labeling capabilities of Extensible Markup Language (XML), Muglia said.

"Think of WinFS as pulling together relational database technology, XML database technology, and file streaming that a file system has," he said. "It's a (storage) format that is agnostic, that is independent of the application."

With Longhorn and WinFS, Microsoft is tackling a nagging problem the company has long sought to address. For nearly a decade, the company has touted the vision of a single storage system that would break down barriers between applications and serve up stored information quickly and accurately.

"The desire has been around forever. It's almost like the Holy Grail of data storage," said Michael Cherry, an analyst at market researcher Directions on Microsoft.

The software giant later this month will disclose additional details on the company's ambitious plans with WinFS. At the Professional Developers Conference in Los Angeles at the end of October, Microsoft will describe how application developers and end users can take advantage of the revamped file system.

Roping in Longhorn
WinFS will be one of several new initiatives Microsoft will discuss at the conference, most of which are designed to build interest in Longhorn, the desktop edition of Windows due in 2005 or 2006.

Microsoft will give attendees an early version of Longhorn, as well as editions of its Visual Studio.Net and SQL Server databases, which are both due to be completed in the second half of 2004. The company also will sketch out a Web services initiative called Indigo and a graphics and presentation system named Avalon, both of which will be integral to Longhorn.

Microsoft calls WinFS "the next-generation storage platform for Windows (that) manages data for organizing, searching and sharing." With WinFS, the company seeks to create a common system for finding and storing data across all types of Windows applications.

Right now, the kind of application dictates how data is stored. Databases are typically used for more numerically oriented applications, such as storing bank account information, while file systems are usually used for document-centric applications with unstructured data types. The problem is that retrieving information from different storage systems is cumbersome, at best.

WinFS seeks to bridge the worlds of unstructured documents and data stored in relational databases with a common storage and look-up mechanism. If Microsoft is successful, the net result will be greater data interoperability and much improved viewing and searching.

"Today, applications encapsulate data. In the future, applications will be able to read and write data created with multiple applications," Muglia said. "Information opens up dramatically."

WinFS will be a significant change for application developers as well. The software is designed to give programmers more powerful tools to build applications that can fetch information from different data stores, such as e-mail servers, databases and desktop applications, at the same time.

One of the ways that WinFS will improve searching across Windows machines is a mechanism to store related "metadata" with a given file. The metadata will provide more context and keyword information than Windows applications do today. For example, currently a person can use Windows to view when photos were taken. WinFS is being designed to allow a person to search with more specific information, such as who is in the picture.

WinFS also is designed to improve search across several applications on a corporate network. If WinFS is used as the underlying storage system in many applications, a person could search for all documents and data related to a particular topic.

The metadata will be described in "schemas," or an XML data format. These schemas will define common objects, such as documents, music files and e-mail messages, which the operating system can find and store, Muglia said. Customers can define their own schemas for identifying stored data as well.

The need for WinFS is being driven by the explosion of data and ever-growing hard drives, according to Microsoft. The company has recognized that search, or querying, is fast becoming a more practical means of retrieving information than navigating through a hierarchy of folders and subfolders.

"With 1-terabyte hard drives in the not-too-distant future, it is starting to seem reasonable to store pretty much any piece of information you own on a PC. But if you can't organize it, what's the point?" Jeremy Mazner, a Longhorn evangelist at Microsoft, wrote in a recent blog posting.

In quest of improved search
Microsoft competitors are hot on the trail of improved information retrieval. IBM plans to incorporate search capabilities from a research project called WebFountain in a forthcoming version of its DB2 Information Integrator. Oracle several years ago introduced into its Oracle 9i database the Internet File System, which was designed to let people search across databases and data that are typically stored in file systems such as documents and Web pages.

Microsoft also is investing in Web search technologies independent of WinFS to enhance its MSN portal and take on Web search leader Google.

Directions on Microsoft's Cherry said the technical difficulty of creating a new file system could well be one of the toughest nuts for Microsoft to crack in Longhorn--and a likely culprit for any future delays.

"This is one of the harder things they have to implement," Cherry said. "It's not a small task. They have to come up with a great design, implement the design, and a lot of little pieces have to come into place."

One piece that's crucial to adoption of WinFS is the support of Windows software developers. People will get the benefits of the expanded search capabilities only if third-party software companies use WinFS, analysts note.

At the Professional Developers Conference, Microsoft is expected to go into greater technical detail on how developers can make use of the planned features in WinFS. For example, WinFS will have synchronization capabilities, which will allow a person to synchronize data stored between WinFS and other data stores. The data storage system will allow developers to build security into applications.

WinFS will address concerns in how developers create applications, Muglia said. Whereas programmers had to deal with a separate data format for each type of application, WinFS provides developers with a new storage option designed to bridge all application types.

WinFS's enhanced searching capabilities will give developers another option other than relational databases for storing information, said Mike Sax, president of Sax Software.

"Having 'rich' files that have searchable attributes would allow you to build apps in such a way that you no longer need a database, but you still have all the advantages of doing fast searches," Sax said.

Even with the planned improvements, Microsoft has made it clear that it intends to create a smooth transition for application developers as well. A conference description of WinFS notes that the storage system supports the new application programming interfaces (API) built into Longhorn as well as the existing Win32 APIs for tapping into Windows' functions.

For end users, the introduction of Longhorn and the new file system could mean a clearer view of widely scattered information. Rather than navigating through a hierarchy of different directories, people can rely more on queries.

"With WinFS, the concept of hierarchy becomes a secondary concept," Muglia said. "It still exists, but it's not necessarily the only, not necessarily the primary, way users find information."