Software to cut the digital fat

Charles Simonyi has spent his professional life building software applications.

While at Xerox Palo Alto Research Center, he developed the first WYSIWYG (what you see is what you get) word processor, called Bravo. He joined Microsoft in 1981 and headed up the development of the company's major software applications, including Word and Excel.

Now, Simonyi is focusing on building software for building software. His company, Intentional Software, plans to deliver programming tools that reduce the discontinuities among the various stakeholders (domain experts, designers, IT staff members, lawyers, programmers and others), which cause software projects to lose the design intent in the development process.

Simonyi describes software as the "bottleneck on the digital horn of plenty." He doesn't believe that training subject matter experts to program will reduce the bottleneck.

"We should not build the future by piling on more tasks. Moore's Law improves because greater portions of the process are mechanized. We just need to push it upstream and preserve the design intent of the subject matter experts," Simonyi said.

Simonyi's notion of pushing programming upstream is a PowerPoint-like design tool that allows stakeholders to describe an application in their own terms and then hand it off to the programmers to write a "generator" to produce the machine-readable code. His company will develop tools, taking advantage of recent innovations in aspect-oriented development, design patterns, model-integrated computing and other programming methods. We caught up with Simonyi to learn about the concepts behind Intentional Software development and how he believes it will pave the way for more adaptable, reliable programs.

What is the problem you are trying to solve with your work at Intentional Software?

I always say that software is the bottleneck on the digital horn of plenty.

My conclusion is that there is a structural, systemic problem with software. Of course, it's very frustrating because our brothers in hardware celebrate Moore's Law. Now, we have three Moore's Laws--one in processors, one in memory and one in bandwidth. Software is the opposite. We don't celebrate Moore's law; we commiserate over the continuing software crisis. That's why I always say that software is the bottleneck on the digital horn of plenty. It's so obvious that there is a giant gap between the processing capabilities of the machine compared to actual services it provides.

You are tackling the software development bottleneck problem. Can you elaborate on the problem and how you plan to solve it?
The goal is to do something about the bottleneck, to analyze the systemic problem and redeploy the resources in a way that helps resolve the problem. Tools have to be involved--that's the business proposition for our company--but they have to operate in a new relationship between subject matter experts and the programmers.

Currently, the key element to a killer app is what the application does for people. In health care, for example, helping doctors with patient care is a tremendous opportunity. You need subject matter experts, like doctors and health care administrators, who understand the issues of their domain. The biggest problem is that what a subject matter expert is trying to accomplish is not expressed in the code. The code is really the first truly precise description of the problem. The intent of the subject matter expert, however, is not apparent in the code.

What do you mean by the 'intent is not apparent in the code'?
I make a semijoke that programmers are stenographers; they take the wonderful statements of subject matter experts and then they create thousand of lines of code that effectively contain the information, just like a famous picture of a flower might contain a secret message of one sentence. That is the structural problem; the intent has been lost or obscured, and once it has been lost, then the problems start. To do anything to it, the intent has to be recovered, which is what programmers do all day--constantly struggle to recover the design from the code.

These are industrywide structural problems. Quality software is achieved at a cost that is not proportional to the size of the problem. Any change (in an application) must involve the programmer and software knowledge.

Does the cross-training of subject-matter experts and programmers result in better software?
When people talk about end-user programming, even Smalltalk or Logo, you are attempting to teach programming semantics, the idea of variables, state, and all those things you learn in programming 101, 102, 103 and 104 to the subject matter expert. How is that going to help anyone? The process will still be the same, and you are trying to get a subject matter expert motivated to become a programmer.

It brings us to this absurd process of people proposing new subjects that programmers or subject matter experts should learn. Where is this going to stop? Is the solution to burden the curriculum even further? Should programmers be taught technical writing? We shouldn't build our future by piling on more tasks--that's not how Moore's Law has been accomplished. Semiconductors didn't improve their reliability and performance by exhorting workers to do a better job, wash their hands more often or put in longer hours. It was accomplished by mechanizing a greater and greater portion of the process.

How then do you narrow the gap or increase the fidelity between the design intent and the actual programming?

		Get Up to Speed on... Web services Get the latest headlines and company-specific news in our expanded GUTS section.

First, we need to push the boundary of computer activity further upstream, and second, we mustn't lose the design intent. The other stuff of end-user programming is not the goal. This gets us to the mission of Intentional Software: We want to improve software development by making the code look like the design. We don't lose the intent and move the processing upstream.

Are you talking about business process modeling and XML (Extensible Markup Language) Web services?
The business model is not code--it has no variable or objects. It has business concepts, abstractions that the subject matter experts are familiar and have created. Basically, it's the sketch on the napkin. It's marketing speech. It's not false or ineffective, but it's not scientific.

XML is not a language in the sense of a programming language any more than sketches on a napkin are a language. XML is basically a computer-exchange syntax. We might as well call it ASCII (American Standard Code for Information Interchange). If I said we would encode your business rule in ASCII would that help the discussion? It doesn't matter if it's encoded in ASCII or XML--the issue is what are the business rules and what do we do with them. Without ASCII or XML, which is a big deal, we couldn't do this (create applications), just like we couldn't do it without silicon, but it's not the essence of the idea.

XML is important, but it's not the underlying concept. What is the essence of the idea?
The code necessarily includes software-programming ideas, which the stakeholders should not be burdened with or are unwilling to bear. As meta-analysts of the process, we (Intentional Software) don't want to burden them. Then what language should they use? The answer is that they should use their own language and build their own model. But what's wrong with that model? The only thing wrong with that model is that it doesn't run. That's where the programmers come in.

If stakeholders, other than programmers, are using their own language to describe the goal of an application, how then do programmers take that input and turn it into machine-readable code that preserves the original intent?
For the sake of argument, assume that we would ask the subject matter experts to write a nice PowerPoint presentation and give it to programmers so they can write the software. That would be a very modest improvement over current practices, which insist that the contributions of subject matter experts are organized in a decent way.

Our proposal goes further. We don't ask the programmers just to read the presentation and write a program. We'll ask the programmer to write a program that reads the presentation and writes the program. We are making a little twist to our request to the programmer: Don't convert the design into a program by hand, write a generator to write the program. We will be actively supporting the process by giving the subject matter experts a CAD (computer-aided design)-like program that the generator can read its input from and process it easily and without loss of information.

Thirty years from now nobody will remember Java and everyone will remember Microsoft.

Another way to think about it is that the programmers you would have employed anyway to solve your problem are now creating a domain-specific language for your problem. Programmers will admit that every big program is a language on its own. Microsoft Word is written in C++, for example, but if you want to work on it your colleagues won't just ask if you know C++. Knowing C++ will get you 1 percent of the way toward learning about how Microsoft Word works. The other 99 percent is to learn all about Word's procedures, services and data structures, which all have names, relationships, internal logic and a kind of syntax.

The programmers are subject matter experts in how to turn designs that are not computer specific into a software program. Value semantics, variables, states or decision tables, sequential and parallel logic--all of those computer science ideas are part of their expertise. The design has to be expressed in those ideas to run on a computer.

How do aspect-oriented programming and design patterns fit into what you are doing?
Aspect and design patterns are another validation of the value of raising the level of intentionality and abstraction. They describe an incremental path to where we want to go. Incrementally, with aspects and design patterns, you can look at an existing code base and by refactoring it, raise the level of intentionality to a higher level. The implications are tremendous. More and more programming activity can be moved into the machine realm, increasing productivity and quality accordingly.

In another interview, you said that Intentional programming techniques could reduce software bugs to one in a million compared with an estimated 150 in a 1,000 today. How does what you are doing make that possible?
I was saying the obverse. If we want to reduce failure rates to one in a million, we have to use mechanical means. How do I know? Look at the failure rate of compilers--it's much less than one in a million. In terms of the number of bad instructions generated compared to the total number of instructions, it's probably in the 1015 range. If anything goes wrong, it goes wrong very badly in a grander, royal way. In manufacturing, it's exactly the same. Turbine blades are made completely automatically, but not just to reduce the price (they are still a pricey item). We could afford the craftsmen, who must carefully file and hand fit the turbine blades, but we wouldn't reach the required reliability. We were forced to automate the manufacturing, and software is going down the same path.

Domain bugs aren't reduced, however; an administrator can put in a bad rule. But you have quick turnaround for fixes because you have domain experts who can evaluate the output. We are much more comfortable as a society with domain bugs rather than implementation bugs. Historically, the bottleneck has to do with implementation bugs.

In 1997 you said in an interview in Forbes that Java would be totally forgotten. It would be significant only to vendors with tiny market share. That hasn't turned out to be the case.
I was talking in the very long term. Thirty years from now nobody will remember Java and everyone will remember Microsoft.

Will Intentional programming concepts improve software development within existing programming, such as Java or C++?
They will benefit from having more design intent showing up explicitly in the programs, such as expressing cross-cutting aspects of the design as with AspectJ. The structure of popular design patterns will be graphically more explicit. Today, the structure of design patterns is implicit, or at best, expressed in informal comments.

Are you designing your own tools with the concept?
We are doing it in moderation. Bootstrapping a two-edged sword, we are dancing on one of those edges.