A Richer Concept of Ground Truth

In a previous post, I explored the role people play as “social sensors” in the generation of “ground truth.” A colleague of mine, Ross MacDonald, who is co-authoring a book with me about collective leadership in the non-profit sector, suggested that ground truth is a far richer concept than what I had explained in my posting. He offered to write a series of “articles” about ground truth that would provide different ways of seeing how this concept could be understood and applied in diverse social settings. Because ground truth is so essential to healthy adaptation in social systems and is integral to the design of successful frameworks for change, his offer could not have been more timely and appreciated. What follows is the posting of his first article. I thank you in advance for giving his ideas careful consideration. We both look forward to your comments! And now, Ross…

In his aforementioned blog posting, Steve Bosserman provides a brief but thoughtful definition of ground truth in a predominantly social context. This response is the first in a series of short articles exploring more fully the concept of ground truth. The entries are based on my work creating sustainable links within and among community groups, government entities, businesses, education, and individual – especially those people who have been historically ignored or poorly served. This first entry looks closely at the term “ground truth” in order to further sensitize our understanding of its applications to adaptive learning in social systems.

Ground truth refers to the information provided by people and instruments on site which assesses the accuracy and value of information and inferences derived from more removed sources, such as aerial photographs and satellite imagery. Consider for example that the French seem to be the first to use balloons for military aerial reconnaissance during a conflict with Austria in 1794. We will assume for the moment that a person making observations from a balloon constitutes remote sensing and imagine as well that civilian informants, reconnaissance patrols, and secret agents provided additional data on the ground. Taken together, observations from the scared soul in the balloon and additional reports from those on the ground enabled more effective strategic and tactical adjustments. This deliberate triangulation of information is the fundamental process for seeking ground truth.

Uses of the term ground truth are easily found today in a wide range of fields including agriculture, anthropology, biology, earth sciences, geography, landscape design, library sciences, medicine, music, physics, and zoology. A team of geologists, for example, might physically examine the ground at a specific site to confirm the nature and possible causes of temperature changes detected from satellite imaging. Should the team document a change in underground geothermal activity on site, then the remotely-detected temperature shift is better understood. As remote sensing technology has advanced, it has became less dependent on human operators / observers, leading to an increased demand for humans to do something on site to authenticate data and sharpen inferences. As a result the term ground truth is entering the public parlance.

So the term “ground truth” is both a label for a process and a label for a product. The process of ground truth is to have a person or persons at a given site with instruments so as to verify and sensitize remotely sensed phenomena. The product of ground truth is more accurate and reliable information.

The very label ground truth has appeal – but if understood superficially it is more of an appeal to shallow hubris than to good thinking. The term lays claims to “truth,” and truth, after all, connotes a definitive and irrefutable reality. How satisfying it is to capture the high ground by positioning one’s belief as truth! A famous scene from Woody Allen’s movie “Annie Hall” illustrates. In this scene, Alvy (played by Allen) and his charming girlfriend Annie (Diane Keaton) are waiting in line for a movie. Alvy becomes increasingly annoyed with what he believes are the empty and inaccurate pontifications of a pseudo-intellectual also in line. Alvie, convinced that the pedantic snob is completely mis-representing Marshall McLuhan, enacts an everyman fantasy when he pulls the real Marshall McLuhan (playing himself) from out of a billboard. McLuhan, at Alvie’s direction, beratingly refutes the veracity of the annoying snob, stating “You know nothing of my work!” – much to the satisfaction of Alvie and everyone else who has wished for such a moment.

Ahh, ground truth! However, as subsequent blog entries will explore, this level of truth is indeed elusive. As tempting as Alvie’s triumph is to all of us, we should be skeptical of such absolute truths whether seen in the fiction of a movie, proclaimed from intolerant political or religious pulpits, or announced in scientific press conferences. It is important to distinguish multiple truths and their small “t’s” from “The Truth” and its capital “T.” A future blog entry will reveal the higher value of multiple truths to understanding and improving human social systems over the misleading value of a claim to a single Truth.

The “ground” in the label ground truth also explicitly lays claim to a tempting position: the false pride of being on solid turf in one’s assertions. Being grounded stands in implied contrast to an ethereal disconnection from practical matters. As in Truth with a capital “T,” there is a temptation in the label of “ground” to validate one’s belief by simply planting a sign claiming one owns the Truth. Very familiar but counterproductive language often accompanies such hollow claims: “I’ve been doing this for thirty years and believe you me . . .” In this kind of claim, a person is essentially saying, “because I am more grounded than anyone else” (“thirty years doing this”), “my truth is The Truth” (“believe you me”). By falsely laying claim to the most solid ground, this kind of approach attempts to gain validity by rendering other truths invalid. Such a mixture of brute power with self-aggrandizing opinions is directly antithetical to the search for multiple perspectives and more sophisticated reasoning about them which characterizes ground truth inquiry.

An interesting case is Lloyd Bentson’s famous retort to Dan Quayle’s self-comparison to John F. Kennedy during the October 1988 vice presidential debate. Bentsen’s ground truth about the comparison? “Senator, I served with Jack Kennedy. I knew Jack Kennedy. Jack Kennedy was a friend of mine. Senator, you’re no Jack Kennedy.” Bentsen grounds his position by reminding us that he “served with”, “knew” and was friends with Kennedy. Although the Dukakis-Bentsen ticket suffered a landslide loss, Bentsen struck a public chord. Many people and certainly many democrats believed as well that Quayle paled in comparison to JFK. Bentsen’s remark aligned with the less dramatically staged opinions of many others, profoundly stigmatizing Quayle’s vice presidency. But we must remember that regardless of what one thinks of Quayle and regardless of how well Bentsen positioned his analysis; Bentsen’s statement was only one observation from one source. That so many agreed with it and that it had such a profound effect derives from the alignment of all those ground truths, not just from Bentsen’s moment on stage.

Ground truth refers to both a product and a process which, despite potential for prideful misappropriation, is of tremendous value for improving organized human activities.

At its best, ground truth labels a set of practices by which multiple sources yield better knowledge and so empower humans to adapt and improve their practices. This is the very work we do in my company, ground truth consulting. In my next entry, I will discuss particular ground truth processes and underlying ethical principle by discussing two specific projects: a Himalayan eco-tourism project in the north of India and an upcoming, collaboratively produced book on academic leadership.

Originally posted to New Media Explorer by Steve Bosserman on Tuesday, January 10, 2006

Ground Truth and Social Sensors

Ground truth is the unfettered and unfiltered relating of people’s experiences within a human social system. It can be associated with a specific event at a particular point in time, such as personal interviews with survivors on December 26, 2004 in Sri Lanka shortly after the devastating tsunami struck. It can be the review of a series of experiences across a period of time such as follow-up interviews with those same Sri Lankans who were interviewed originally to understand how their circumstances are changing. It can be the stories told about how members of a social system, who experienced a catastrophe like the tsunami, adapted their social, political, and economic structures in response; namely, what worked, what didn’t, and what changes to make. Ground truth is given when people speak for themselves. It is the ONLY way a human social system knows what is REALLY going on.

People who deliver their ground truth are acting as well-functioning “social sensors” in a broader social system. They draw upon a wide range of information and communication technologies (ICT) e.g., websites, email, blogs, wikis, with land-line, cellular, and satellite connections from locations throughout the world, no matter how remote, to relate their experiences instantaneously and continuously. Social sensing parallels similar functions within mechanical and biological systems. In fact, there is a point of convergence between scientific and social sensor development paths that establishes the possibility of two working together in a highly interrelated manner that enables large, complex systems to be better managed.

Sensor technology, too, has it roots in ICT. The earliest application of ICT during its commercial development was data collection. People would make manual entries into databases of data they and others collected utilizing various measuring devices at certain points within factory or office operations. The computer would use programs to analyze those data and put them into an informational format that could help interpret what was occurring and develop responses to improve the processes or procedures.

One of the first areas where ICT quickly developed in the 1970s and 1980s was in sensor technology. With increased capability and reliability, sensor technology contributed significantly to the replacement of humans as the means through which data was collected and entered into databases. Further developments over the last 15-20 years greatly reduced the size and power requirements for sensors, and increased the sophistication and range of type and capability of sensor technology. Now, sensors are pervasive; they influence almost every aspect of our lives and endeavors. And they continue to displace people from those activities where consistency, repetitiveness, quality, and reliability are essential for effective and efficient operations.

Of particular significance in these developments and displacement is the degree of integration and compatibility between what is being sensed and what is doing the sensing. Today, extensive sensor networks are carefully nested within all manner of systems: mechanical, chemical, optical, biological, and social. Regardless of application, these sensor networks monitor and evaluate conditions which become feedback in larger, adaptive systems that devise corrective strategies and take appropriate actions in response. The key to their success is the accuracy and timeliness of their input as well as the pervasiveness and comprehensiveness of their coverage.

In many ways, the human body is a complex web of sensor networks. Millions of nerve receptors of different types and functions are distributed throughout the body and send continuous signals through the central nervous system to the brain where they are processed and given responses. And like any sensor network, the quality of the response is tied directly to the quality of the input.

Despite highly evolved and elaborate redundancies that function effectively the vast majority of the time, our senses can be fooled: hot can feel cold and vice-versa; we see mirages we believe are real; we hear sounds when there is silence; odors we smell and taste evoke memories that do not accurately reflect what we are experiencing in the moment. And as in the interplay between sensor networks and the larger systems they help regulate, there are different ways of analyzing and processing input with each eliciting different responses. Furthermore, we can ignore sensory input or respond in ways that override evidence suggesting a more appropriate course of action. So, regardless of how well-designed the system and how well-refined the processes, the arbitrariness and irrationality of our decision-making have the potential to bring it to naught.

Like the human body, human social systems are vast sensor networks. Each member of the system is a “sensor” who “reports” on conditions as they are experienced. The system – comprised of hierarchical political, economic, and social structures that operate according to sets of self-serving rules – sorts, aggregates, and analyzes data entries from sensory members in an effort to understand, interpret, determine response possibilities, consider alternatives, and decide on a course of action. Of course when considered on a global scale there are myriad social systems in play simultaneously. Members of one social system can concurrently be members of others. Interpretation of sensory input in one social system can elicit a different response compared to what happens in response by another social system. The key determinants are rank, status, and position in the formal structure and presence, voice, and passion in the informal structures.

Human social systems are analogous to the human body in other ways. There are over 6 million people in the world. The human body consists of billions of cells. Thousands of people die and thousands more are born every day. Millions of cells in the human body die daily and millions more are regenerated. Of the thousands who are born, live, and die each day, I have the opportunity to know only a handful. I know my body, in general, through its organization by function, role, and relationship of one part or system to another. Most of it I will never see and I don’t have to; I trust that it will do what it should without my deliberate attention if I follow simple rules of good health in terms of diet, nutrition, exercise, and rest. Similarly, most people in the world obey the rules of the social system to which they belong. These rules present choices and people decide in ways that permit them to adapt to current circumstances, but preserve the integrity of the system. Behavior is managed and people stick by the intent of their roles, responsibilities, and relationships.

What happens when taking care and following the rules is not enough? Even when we do our best to prevent it, inevitably, our bodies get sick. Sensory cells we seldom hear from send messages that indicate they or the systems to which they belong are in trouble. Depending on the nature of the condition they are signaling there is a wide array of prescriptive treatments from which we can select. These can be non-invasive wherein normal functions of the cells and systems are restored through medications; or invasive through the repair, removal, or replacement of tissue. The same phenomenon occurs in social systems. People in their “sensory roles” relate experiences wherein the system – no matter how well-designed the rules and how noble the principles and ideals that frame them – fails to respond within an acceptable range. Functions break down; remedies are required. In some instances a simple reinterpretation of an existing rule is all that is needed. Other times, though, more radical steps are in order such as rescinding laws and enacting new ones, closing operations and opening others, and eliminating products or canceling services and offering of others.

Oftentimes, we do not heed the early warning signals from our bodies indicating something is amiss and what was once easily restored must now be repaired, removed, or replaced. The sensory networks did not fail, but we chose through our heads or hearts to ignore the input, e.g., “I don’t feel any pain” or “I don’t see any bruises” or to not give the input appropriate attention, e.g., “It will go away” or It’s nothing.” Because the human body is marvelously adaptive, this approach works to some degree, but the performance of the whole and the cellular arrangement and functioning that comprise it are compromised. We live with it in a compromised state or we take more radical steps to correct or reverse the damage.

Again, there is a clear parallel within human social systems. Billions of people in the world have a nearly infinite variety of experiences daily. How do these experiences fit the frameworks of the social systems to which people are members? Where are there anomalies between expectations and experiences? Do these differentials drive responses? Is the system stretched beyond its limits to adequately respond and more deliberate and protracted strategies are needed to spur deeper adaptation? To know the answers requires being attentive to the “sensors.” It means getting to “ground truth” with people in the system about their circumstances.

Establishing ground truth is a three-step process:

  1. Ask people for the truth about their realities and encourage them to tell their stories openly
  2. Hear their truth, once offered, understand it; and commit to respond with appropriate action
  3. Follow-up afterward to confirm that the responses were, indeed, appropriate and that the current situation is corrected and steps are underway for longer term changes in the system preventing recurrence of the problems

Just as we do not heed messages within our bodies we do the same in social systems. To know what is really going on requires ground truth. To not ask, listen, comprehend, and take action are just as effective in shutting down responsiveness and adaptation in the social system as it is with our bodies. Much of time it is for the same reasons: “don’t confuse me with the facts” and “if I am ignorant I cannot be held accountable.” Typically, we do not like change even though circumstances warrant it. In addition, we do not like to know about circumstances where change is needed because we will be challenged to take action – in other words, make it happen. Either way we claim we will lose focus and be distracted from the mission we are locked into at the time.

Just as the health of our bodies is compromised when warning signals are ignored or overridden, social systems become corrupted when the ground truth of members is not heard or heeded. Social systems can continue to function, in general, despite certain levels of corruption, albeit their effectiveness and efficiency are significantly reduced depending on the type, degree, and pervasiveness of the corruption. Change is particularly problematic in established social systems. Power concentrates in the tops of the ruling hierarchies, corruption increases, and along with it an aversion to change that might disrupt the structure, grows. As a result, these hierarchies uphold tenets and “rules” that support the dominant culture remaining dominant.

Ruling minorities become increasingly distant from their ruled majorities. In so doing they become increasingly cut-off from what is really happening within the social systems they are charged to “protect and serve.” Ground truth exists in the heads, hearts, and souls of social system members whether it is sought after and cared for or not. People and their truth, like life itself, will find a way to express itself, even it means setting in motion disruptive patterns of behavior that threaten to totally transform the system in which they exist. David Brooks, in his editorial entitled, “Trade, Oppression, Revenge,” published in the NY Times on December 25, 2005, illustrates this point through a very recent example. The native Indian people of Bolivia, who comprise 65% of the population, dominated for years by a ruling white elite representing 3% of the population and controlling almost all of the resources in the country, used the democratic process to elect an Indian president. What is in store for the ruling minority of Bolivia and their repressive, exploitative policies? Something not nearly as pleasant as it could have been had the ground truth been spoken, heard, understood, heeded, and the outcomes confirmed.

Therein lays the challenge with respect to ground truth: some have to want to hear, some have to be willing to say, and others still have to respond to what the truth means about the design of the system and make changes accordingly. For any sensor to work effectively, regardless of type or application, its input signal must be captured, processed, and acted upon. This certainly pertains to people as social sensors in human social systems: their input is in the form of valuable stories to tell and their experiences constitute important feedback in regulating the function and adaptation of these same systems. Are you asking…and listening?

Originally posted to New Media Explorer by Steve Bosserman on Monday, December 26, 2005