What Does Ecological Validity Have to Do with Business?
October 4, 2019 ~ Written by: W.B. “Bud” Kirchner
Approx. Read Time: 10 Minutes
Ecological Validity
“In the business world the rearview mirror is always clearer than the windshield.” – Warren Buffet
“For the unexamined life is not worth living.” – Plato, The Trial and Death of Socrates
Several of the topics contained in the Business Brain Model writings touch on topics that fall squarely in the realm of “popular psychology”. The implication of this (per Wikipedia) is that “the concepts and theories about human mental life and behavior that are purportedly based on psychology find credence among and pass muster with the populace.”
There are, however, numerous complications in this context; we summarize Kenneth S. Pope’s series of articles identifying some of the following categories of pitfalls:
- 26 types of logical fallacies
- 8 common language patterns that hide or confuse issues
- 21 ethical fallacies
- 10 fallacies in psychological assessment
With this in mind, there are a number of ‘tests’ that the results should meet including the gold standard of medical research: randomized controlled trial.
One category of validation is known as ecological validity.
Definition
“Beware of false knowledge; it is more dangerous than ignorance.” – George Bernard
Perhaps, the easiest concept for a non-scientist to relate to is the following from Rethink: The Surprising History of New Ideas:
- Ecological validity refers to the extent to which the findings of a research study are able to be generalized to real-life settings that are realistic – that is, how well do the effects translate from the artificial simplicity of the lab to the ungovernable messiness of real life?
- “We know that it has striking effects under highly simplified and controlled laboratory conditions, where the subjects are exposed only to the stimuli that the experimenters provide. But it is very difficult to know how significant priming stimuli (thinking about money, large numbers, abstract questions) would be in real-life, uncontrolled environment, where all sorts of stimuli might be conflicting with one another.”
Additional perspective comes from Ecological Validity in Psychological Research:
- “Ability to generalize the findings of a study beyond the situation in which the behavior was tested and relate it back to real life.”
- “If a piece of research is conducted in a setting that lacks mundane realism and has little (if any) relation to real life it will then be argued to lack in ecological validity.”
- “A key strength for a study having high ecological validity is that it allows psychologists to generalize the findings.”
- “For example, studies with low ecological validity are often easier to replicate and therefore will have increased reliability.”
- “A major weakness of a study that has low ecological validity is that it is much harder to generalize the findings from the study as they are not representative of the real world or tasks.”
Just to be clear, ecological validity is also a controversial concept in psychological research based on challenges such as lack of control subjects, etc.
Case Study
A real-world example of concerns of this nature have risen from the work related to ‘Grit’, the model built by Angela Duckworth. Duckworth has achieved notoriety from her books, Ted Talks and such prestigious awards as the MacArthur Genius.
The very popular term ‘grit’ expresses determination with which one pursues their goal.
By way of full disclosure, we have written about ‘grit’ in the following:
The salient points and counterpoints of the controversy are captured in an article by Anya Kamenetz (MacArthur ‘Genius’ Angela Duckworth Responds to A New Critique of Grit). The author presents two sides via statements from Duckworth and Marcus Crede et al. (Much Ado about Grit: A Meta-Analytic Synthesis of the Grit Literature).
The conclusion of the latter: “My overall assessment is that grit is far less important than has commonly been assumed and claimed,” says the lead author, Marcus Crede. By way of rebuttal, Duckworth has responded: “I can’t see, exactly, how the author goes from these findings to the rather bold claims in his press release.”
The salient points of the back/forth as documented by Kamenetz:
- Effect sizes in one of Duckworth’s major papers on grit were described incorrectly to sound misleadingly large.
- Duckworth concedes she “badly describes” the size of her outcomes.
- The impact of grit is exaggerated, especially when looking at broader populations of people — not just the high achievers in Duckworth’s initial studies.
- Duckworth seems to agree with this comment.
- Grit is nearly identical to conscientiousness, which has been known to psychologists for decades as a major dimension of personality.
- Duckworth has debated this in literal sense but is reportedly revising her ‘grit scale’.
So – there you have it – at one level two academics duking it out. At another level, a collective focus on what helps people achieve their goals. No harm? No foul?
Implications to the Business Brain Model
In closing, I summarize in the context of intellectual integrity and review our approach to these articles:
- Personal
- I identify what I believe to be relevant to business based on more than a 50-year business career. I am not a scientist or an academic, but I have had diversified exposure to neuroscience, psychology and related cognitive sciences. So – generally speaking, the ideas I express are business-angled expansions or simply restatements of other people’s ideas. When possible, I link to the original reference.
- Sources
- I verify sources in a number of ways.
- Results relate to the work of dignified scientists.
- The majority of the scientific community has embraced the findings.
- The findings are consistent with my empirical experience.
- I verify sources in a number of ways.
- Process
- The process is designed to flag issues – raise questions – provide occasional editorial comments – inject personal experience and its inherent basis.
After 60 years on the street, we have pretty good radar for charlatans and snake oil salesmen. However, legitimate differences of opinion and interpreting are part/parcel of acquiring knowledge.
“Plato says that the unexamined life is not worth living. But what if the examined life turns out to be a clunker as well?” ― Kurt Vonnegut Jr., Wampeters, Foma and Granfalloons
Bibliography
- Jill Barshay – Research scholars to air problems with using ‘grit’ at school
- Marcus Crede, Michael C. Tynan and Peter D. Harris – Much Ado about Grit: A Meta-Analytic Synthesis of the Grit Literature
- Encyclopedia Britannica – Ecological validity
- JRWPSCHOLOGY – Ecological Validity in Psychological Research
- Anya Kamenetz – MacArthur ‘Genius’ Angela Duckworth Responds To A New Critique Of Grit
- Chathurika S. Kannangara, Rosie E. Allen, Gill Waugh, Nurun Nahar, Samia Zahraa Noor Khan, Suzanne Rogersona and Jerome Carson – All That Glitters Is Not Grit: Three Studies of Grit in University Students
- Susan O’Brien – The Only Time It’s Okay To ‘Fake It Till You Make It’
- Steven Poole – Rethink: The Surprising History of New Ideas
- Kenneth S. Pope, Ph.D., ABPP – Logical Fallacies in Psychology: 26 Types
- Kenneth S. Pope, Ph.D., ABPP – Ethics, Language, & Critical Thinking: Using Words to Deceive
- Kenneth S. Pope, Ph.D., ABPP – 21 Ethical Fallacies: Cognitive Strategies To Justify Any Unethical Behavior
- Kenneth S. Pope, Ph.D., ABPP – 10 Fallacies in Psychological Assessment
- Alexander T. Vazsonyi, Albert J. Ksinan, Gabriela Ksinan Jiskrova, Jakub Mikuška, Magda Javakhishvili and GuangyiCui – To grit or not to grit, that is the question!
Business Brain Model articles
About the Author: W.B. “Bud” Kirchner is a serial entrepreneur and philanthropist with more than 50 years of business success. He is not a scientist or an academic but he does have a diversified exposure to neuroscience, psychology and related cognitive sciences. Generally speaking, the ideas he expresses here are business-angled expansions of other people’s ideas, so when possible, he will link to the original reference.