Spaghetti Data and a Whine List

blogger: Lyn Robison

I am working on the outline for my upcoming overview on data services, data integration, and the renaissance of data management. Working on the topic of data integration reminded me of an old diagram I saw once that showed the data flows between various enterprise IT systems at a big company. This particular diagram was called the "spaghetti diagram", and with good reason.

Spaghetti can be satisfying and delicious, but not when it shows data flowing between applications. Each of those lines of spaghetti on the diagram that I saw represented custom code that developers had to write to handle a particular point-to-point integration. That code is expensive to create, and over time, the spaghetti becomes brittle. In fact, this type of spaghetti calcifies and cements obsolete, legacy applications into the very foundation of an enterprise.

Lots of people have realized that there has to be a better way to integrate enterprise applications. (It is important to realize that there are three kinds of integration: application, process, and data integration.) Over the years, the software industry has tried several different approaches to data integration. After lots of trial and error, I think we now might finally have some approaches that work. I will talk more about these approaches to data integration in my upcoming overview.

At many enterprises, there can be an astounding degree of complacency about problems with enterprise data and the use of bad integration practices. One approach to overcoming complacency about using a better approach to data integration is to illustrate the problems with the status quo. A good spaghetti diagram might be a key ingredient for driving away complacency about bad data integration.

So if your enterprise has data integration problems, but you don’t have a spaghetti diagram, now might be a good time to put one on the menu. If you combine a spaghetti diagram with a list of whines from businesspeople about data they can’t get, you might find that the IT and business leadership are hungry for a better way.

Comments

[Prof. Florian Matthes]

You might be interested in the results of our research project "Softwarecartography"(look for English Papers on the page http://wwwmatthes.in.tum.de/file/Projekte/Softwarekartographie/Softwarekartographie.htm).

Where can I learn more about your ideas on data integration and data integration diagrams?

[Lyn Robison]

Hello Prof. Matthes. Your software cartography research looks fascinating. The report I am working on currently does not deal directly with diagrams. Rather, my report will attempt to set the context for data integration in SOA. Stay tuned...