Ascending the Data Infrastructure Hierarchy
Maslow’s groundbreaking work Hierarchy of Needs, published in 1954, reflected the continuum of individuals and societies from dependency and disorder toward accomplishment and fulfillment. At the bottom of the pyramid Maslow describes, individuals are consumed with simply making it through the day–foraging for the staples of life, such as food and water. There’s little time for higher-level personal development.
When an individual meets these very basic needs in a more systematic fashion, that person is free to address the next level of needs, including shelter and health. Eventually, as these needs are all met on an ongoing and automatic basis, the individual reaches the highest stage, at the top of the pyramid, which Maslow described as “self-actualization.”
An analogy can be drawn to the way organizations manage data. The more effectively an organization meets rudimentary data management needs, the more likely it is able to move up the continuum to become a well-developed, peak performer. While many companies remain mired in reactive and idiosyncratic practices, high performers rely on disciplined, proactive, and predictive approaches to data infrastructure management (Data IM). In return, a peak-performing organization can devote all its resources to advancing the business – be it stock trading, manufacturing, or retailing.
How does an organization advance from a relative state of immediacy to a higher-value operation? By first understanding and recognizing the five essential stages that an organization moves through as it becomes increasingly proficient at Data IM.
Level 1: Tribal
It used to be common to have database administrators or developers who lived with the system from the day it was installed. They almost had a sixth sense for anomalies as soon as they emerged. Such is how things are done at the tribal level, the most basic form of Data IM. An organization may function from day to day but without any formal processes or systematic management. Instead, Data IM is left to the whims of individuals, who carry all relevant knowledge and knowhow in their heads.
When a company remains at the tribal level, it has no clear understanding of what data professionals are doing and how they do it. Documentation of individual duties does not exist, and professionals simply go about their duties, whether it’s backing up data, monitoring networks, or administering patches to servers. Many such individuals may be highly proficient at what they do; it’s just that there is no formal system assigning the tasks at hand. There is no formal method of tracking and documenting how well they succeeded in those tasks.
Of course, today’s business requirements simply do not allow for this approach. The tribal management level allows no ability to comply with mandates such as Sarbanes-Oxley in an organized way. Plus, having so few people with so much company knowledge may even pose a security or business continuity risk.
The informal network of professionals that is typically seen at the tribal level also produces inconsistent performance across various systems. Different individuals have knowledge on different systems, and the quality of support across systems varies as a result.
Or, as Maslow put it, “When the only tool you own is a hammer, every problem begins to resemble a nail.”
Level 2: Enforced
As a company advances to the next level of the Data IM hierarchy, the Enforced stage, it adopts some semblance of governance, which includes writing down standard operating procedures and putting controls in place to better manage workflow within its data environment. This may include deployment of a repository, in which data professionals are required to document changes or processes they have completed.
At the Enforced stage, stronger management takes hold, and data professionals gain a greater focus on performing tasks in a more systemized fashion – doing what is needed versus what they want to do at that moment. A greater sense of teamwork begins to evolve at this stage.
Level 3: Standardized
As organizations advance beyond the Enforced stage, they reach the Standardized stage, in which processes are established to handle various aspects of Data IM, and automation is introduced. At this stage, Data IM depends less on individual discretion, and more likely takes advantage of some automated or systematic approaches. In addition, standards begin to propogate across the various data environments, providing a more cohesive approach for assuring the viability of enterprise data sources. The organization may even establish a data warehouse or metadata repository at this stage to more effectively collect data from across the enterprise in a more consistent fashion.
There are change management practices in place, and many of these processes may be automated. The company will have established standard operating procedures for maintaining data libraries, for doing backups, for performance and tuning, upgrades, and all other aspects of Data IM.
The organization will also have formal procedures in place for its operational workflow. For example, this may include a formal system for vetting and testing new systems and applications. The organization may emulate the best practices of other companies in its own workflows and processes.
When an organization reaches the Standardized level in its Data IM evolution, it has reached an important transition point. No longer do overworked staff members manage data management operations on an ad-hoc basis. All work is now thoroughly documented and replicable, and many mundane or routine tasks are automated.
Level 4: Actualized
In the Actualized stage, organizations begin getting more creative with their data. Data can be extracted from across the enterprise – as well as from outside sources – and combined to create useful information and applied to create new knowledge and therefore new value to the business. No longer are data confined to silos, and all data management issues are addressed on a systematic basis.
As an organization moves into the actualized stage, it can explore using data assets as leverage in new and higher-value ways, as its underlying Data IM is managed in a systematic and highly automated way. Data IM delegates workflow to the best resources available, freeing up data management professionals to engage with the business as consultants. As a result, greater alignment with the business is achieved.
The organization will have a standard operating procedure for handling changes that are logged into the system. The system automatically determines who in the organization is assigned the work and what is going to happen. Service-level agreements (SLAs) are also automatically tracked, along with the length of time to complete tasks and whether the SLA is met.
Because the organization is freed from the constraints of Data IM, it has achieved the flexibility and agility to compete on analytics, since it has access to a wealth of enterprise data that are reliable and up-to-date. The management and maintenance of data are highly automated, and decision-makers are assured that they are using the right information to steer the company, delivered at the right time. A company in the Actualized stage can even put automated decisionmaking into place, in which the system can deliver decisions almost instantaneously based on data received and mapped against rules and patterns.
At last, the organization reaches the pinnacle of development, where it can devote all its resources to high-level strategic initiatives, rather than administrative issues. Once the organization has attained Peak Performance, it no longer needs to spend inordinate resources managing data. Data IM is self-managing, automatic, and embedded into every system and process of the organization. In essence, the data infrastructure manages itself.
Personnel turnover, which often creates so much chaos in lower-level organizations, is almost a non-event at Peak Performing organizations. When a member of the Data IM team leaves, new personnel can be quickly inserted and brought up to speed on standard operation procedures. In addition, these standard operating procedures are continually upgraded or re-evaluated as new circumstances arise.
At this stage, IT leaders actively look for ways to capitalize on their data. They even compete on the analytics that are derived from their data foundations. They focused fully on their competitive differentiators – the parts of the business that bring unique value to the marketplace. And they are actively engaged in efforts to drive collaboration with partners, suppliers and customers. They are thinking far beyond the boundaries of the organization to continuously drive entry into new markets.
Where Are You?
Many companies may be still in the lower stages of Data IM, as they struggle with better ways to develop more streamlined and systematic ways to manage their data assets. With today’s increasing competition, along with government mandates, it’s not too soon to make the move up the continuum. A company with an IT or data management staff consumed with administration issues – performing fixes, patches, or various unplanned activities on a daily or weekly basis – is not in a position to effectively compete in today’s data-driven marketplace. By contrast, a well-managed organization that has attained a “Peak Performance” state of Data IM is able to devote its full attention and resources to high-value activities.