A day in the life of a piece of corporate data will see that data move through many different perspectives of use and handling. First, the data is captured, and then checked for errors in entry, format, and other attributes according to its validation rules. Next, it travels to the systems that utilize the data for primary business contact, processing transactions, and supporting primary business functions, such as sales, order-fulfillment, and services. After being cleaned up for corporate standards, the data moves on to be shared with other systems that need it to complete their picture of the company's business, and is manipulated and stored in decision-support holding areas. Finally, the data is distributed for selective viewing of both company insiders and external users, such as customers and suppliers.

Data management needs to recognize the cycle around which data stewardship revolves. At each step of the process, different priorities will surface. They will require different technical design and development approaches employing defined terms and agreed-upon structures. Figure 1 shows the steps of the cycle and some of the supporting technical structures. It adopts a starting point where customer and supplier interactions initiate business transactions that must be processed. Keep in mind that although it's a common starting point, it is a little arbitrary because activities in any step of this model can act as a starting point.

The following are the steps of the Cycle of Data Management:

1. Process business transactions.

2. Transform data for sharing.

3. Support business decisions.

4. Create internal portals.

5. Create external portals.

6. Interact with customers and suppliers.

Business events, such as a customer contact resulting in a sales order being placed, or a customer request that is filled by customer service personnel, produce data that must be managed. They involve relatively small amounts of data that must be maintained with a high level of accuracy at the time of the transaction and access of the data. Business transactions are typically simple transactions involving very detailed data. For example, a customer order includes specific mailing address, product information, and pricing details.

The systems that support such business transactions must handle many simultaneous updates, reads, and insertions of data. They must often be available for processing on a continual basis, 24 hours a day, 7 days a week. They have low requirements for storage of historical information, usually needing only that information required for supporting the flow of work.

The data model for supporting business transactions is usually highly normalized, which means that data components are structured so that they are stored only once for convenient control of updates. The data model also attempts to achieve a balance between design for state modification (insert, update, delete) and speed of retrieval or data access. Because the data usage is predictable, the data model can be optimized for these performance concerns.

Data management in business transactions usually prioritizes data persistency and state management. It will be utilized by down-stream systems (subscribers to the information) that are usually unrelated to the systems producing the data.

The data produced in business transactions must often be transformed before being shared with other applications. Sometimes sensitive or irrelevant information must be removed before sharing. At other times, information must be made generic so that it matches the definitions of other application views. Some data will be summarized and used to support analytical processing or decision support. Other information will be used internally by other departments such as operations or financial reporting, whereas some data is selected for sharing with customers and suppliers, depending on the requirements of their interactions with the company.

Data extraction, transformation, and middleware tools are all means of transforming data to make is suitable for sharing. Extraction and transformation tools usually help to convert, cleanse, and standardize data. Middleware and messaging technologies facilitate the physical aspects of sharing data, through the use of data delivery mechanisms and subscription maintenance machinery.

The priorities at this stage are for credible data, verified, cleansed, and transformed in a streamlined process. Speed of data availability for sharing with mission-critical applications is also an important factor. In some cases, speed of data availability is considered more important than one hundred-percent accuracy of the data. For example, data that is summarized and used for forecasting might be accurate to within a five-percent margin of error, a margin that would not be acceptable for the processing of business transactions.

The data that is delivered to decision-support applications is used to help companies measure performance, manipulate revenue, yield ratios, make market decisions, and monitor operational statistics. It is earmarked for business management and strategic functions, and used in determining competitive advantage. It can be used to identify opportunities for improvement and growth of the company.

Decision-support data is often derived through summarization and calculations applied to detailed data taken from transaction processing systems. It often involves large volumes of data, with significant amounts of historical data used for trend analysis and reporting of regulatory and markets information. Analytical processing usually involves complex transactions or queries against the data, utilized in unpredictable ways by processes characterized by discovery. A fairly small user community (managers, executives, and analysts) needs actual access to the data.

Data models for analytical processing are optimized for rapid access through non-repetitive queries, producing unpredictable workloads. The priority is on efficient data retrieval, which requires that data be heavily indexed and de-normalized (more than one copy of a data component stored) for convenient access rather than normalized for convenient update. Integrity constraints, such as those utilized for transaction processing (primary keys, foreign keys, and column constraints), are generally relaxed in decision-support systems because the source systems can be expected to enforce the referential integrity that is required.

Internal portals allow parties within a company to access data according to the needs of their particular business or application viewpoint. Data warehouses spawn data marts, which house application-specific views of data replicated from the central repository. Intranet portals provide windows on data that can be needed by operational personnel or marketing analysts.

Internal portals represent multiple views against the company's information and are generally accessed with low security requirements, within the confines of a company firewall.

External portals provide access to customers and suppliers for carefully selected portions of a company's information. Portions of the business process in which customers or suppliers participate can be detailed on public Web sites. Financial performance can be provided in annual reports online or investor information packets. Product and service specifications and pricing can also be made available through an external portal on company information.

Characterized by selective viewing, external portals usually reside outside a company's firewall, providing a highly secured environment for customer and supplier interactions.

Interactions with customers and suppliers can occur through the window of an external portal, or they can be facilitated by business transaction processing systems. Sometimes they occur with replicated versions of company data, carried by a sales or marketing representative in a remote device such as a laptop computer configured to provide price quotes and service specifications.

Distribution of data becomes important in the interaction step, with an emphasis on convenient access and accuracy to certain limits established within a window of time. Partitioning and data replication are schemes that provide the desired distributed deployment of components of data.

Beyond the stewardship and management of data as an asset, other disciplines combine to forge corporate information into valuable business intelligence. Data warehousing converges with business process re-engineering and Internet disciplines to deliver an enterprise-wide data strategy. The successful data warehouse often drives the reengineering of business data and processes by surfacing the contradictions and confusion in current application systems. Once information is verified, organized, and available, Internet technologies are utilized both internally and externally to provide increasingly important data portals, which act as windows on the world of business information.

(Reprint of an article by Laura Brown, first published on InformIT's guest expert page.  For more information, contact lbrown@systeminnovations.net.  All rights reserved.)