Typology of Evidence

Typology of Evidence

Although the failure of once high-flying Circuit City cannot be attributed to a single cause, analysts tend to agree that the shareholders’ push for stock buyback that drained the company’s cash reserves heading into a severe economic downturn (the Great Recession of 2007–2008) was a significant contributor to its eventual demise. When considering their strategic options, should Circuit City’s managers have factored in the company shareholders’ desires into their decision-making process? Or stated more generally, should organizational stakeholders’ subjective values and concerns be considered ‘evidence’, in the same way that data analysis or empirical research generated insights are considered evidence?

There are not many situations where a decision-maker is free of preexisting beliefs, expectations, or ‘gut feelings’ regarding the most appropriate course of action, primarily because organizational decisions tend to be made by those who are at least somewhat invested, interested, or otherwise connected to the decision at hand. Subjective beliefs and expectations, however, can stem from incomplete or skewed fact base, not to mention the reason-bending impact of numerous cognitive biases. And yet the primary decision-making value of evidence lies in the infusion of perception- or recall-untainted information that can enhance the rationality of one’s sensemaking processes. To arrive at what will indeed turn out to be the best decision, the decision-maker needs external-to-self and dependable insights that can either shed new light on current knowledge or introduce an altogether new knowledge. It thus follows that the definition of evidence should be limited to objective and unbiased insights.

Those considerations were captured in the general evidence tree typology discussed in Chapter 5, which categorized organizational decision-pertinent evidence into two broad categories of ‘empirical’ and ‘experiential’, with the former further subdivided into ‘operational data’ and ‘empirical research’, and the latter broken down into ‘expert judgment’ and ‘norms & standards’. Those still very coarse groupings are further refined into a three-tiered typology of evidence, summarized in Figure 7.2.

The overall evidence grouping logic is built around a nested three-tiered schema, where two broad meta-categories are each comprised of two narrower categories, each of which is, in turn, made up of several distinct subcategories. More specifically, the two meta-categories of evidence, empirical and experiential, are each comprised of two more narrowly scoped categories of operational data and theoretical research (empirical), and norms & standards and expert judgment (experiential).

FIGURE 7.2 General Evidence Typology.

Within the empirical meta-category, the operational data category is comprised of three distinct subcategories of transactional details, communications, and descriptive attributes, which are all still relatively broad types of data that are either directly or indirectly captured by business organizations.4 For instance, as there are numerous types of transactions that business organizations engage in, there are numerous sources of transactional data, such as product shipments, product purchases, promotional offer redemptions, and many others. Similarly, each of the distinct and different communication channels and modalities produces a unique communication data source and type, and there are copious sources and types of descriptive attributes, which are transactional entity characteristics, as exemplified by demographics or firmographics. Overall, the defining characteristic of operational data as a source of organizational decision-making inputs is that it encompasses a typically enormous varieties and volumes of raw data, which require a combination of vision, data analytical skill, and organizational commitment before it can offer decision-guiding value.

The second of the two categories nested within the empirical meta-category, theoretical research, contributes very different types of informational inputs – in contrast to raw materials supplied by operational data sources, theoretical research offers ready-to-use insights. It is important to note that while the domain of theoretical research is very broad, only those efforts that make use of explicit hypotheses tests or other confirmatory or exploratory analyses of observational or experimental data are considered here because those types of studies demonstrate the requisite external-to-self validation. As implied earlier, theoretical research studies can be observational or experimental – in addition, summarization-focused research summaries, which consolidate and reconcile similar types of outcomes from similar studies, are also taken into account. Overall, the defining characteristic of theoretical research as a source of organizational decision-making inputs is that it offers consumption-ready, but not necessarily application- or situation-specific insights.

Turning to the experiential meta-category, the norms & standards category encompasses several distinct, but similarly minded subsets, most notably industry benchmarks, which typically take the form of numeric yardsticks capturing activity or outcome-specific peer averages, organizational norms, which are internal (to the organization) expected activity or outcome-specific levels, and best practices, which usually describe tried and tested processes and means of accomplishing specific goals. Jointly, those sources of decision inputs can be considered a product of organization- or group-level practical experience, or insights that represent highly aggregate generalizations.

The second of the two experiential categories, expert judgment, captures the accumulated tacit knowledge of properly credentialed and practiced individuals. It can manifest itself as application-specific knowledge in the form of select tasks, as exemplified by one’s knowledge and experience in a specific domain of practice, such as risk modeling, knowledge of general processes, as illustrated by familiarity with the process of commercial insurance procurement, and future projections, best exemplified by demonstrated expertise in forecasting outcomes of interest, such as frequency and severity of windstorms. In contrast to groupthink-sourced norms and standards, expert judgment is person-specific and thus those individual-level inputs can vary, quite considerably at times, across experts. For that reason, making use of judgment pooling techniques (see Chapter 6), such as the Delphi method, is often recommended.