Lesson Idea: Generalizability (External Validity)

Travis DixonResearch Methodology

External validity refers to the extent to which the results can be generalized to other contexts (e.g. people, places, situations). For example, if one study had Thai girls as participants, could we expect the same effect in a population of Peruvian males?

The Task

  • Read the following fictional summaries of how people have applied results from psychological studies.
  • Identify one or more reasons why the application is misguided and might not work.

#1 Social Media and Anxiety: One study in the USA showed a correlation between social media and anxiety so a school in Japan has banned the use of social media.

#2 Music Class and GPA: A study in the USA found that elementary school students who took music lessons after school had higher GPAs on average than other students, so a high school in Malaysia made music lessons mandatory for all students.

#3: Hotel Towels: Because of Cialdini’s study on using social influence to increase the rate of hotel towel usage, a hotel chain in India is spending thousands of dollars to change the cards they use in their own hotels.

#4: Rat Disneyland: A mom in Canada read about the Rosenzweig and Bennet study on enriched and deprived environments and the effect this has on rats’ brains, so she redecorated her kids’ bedrooms so they are filled with toys, posters, a TV, video games and lots of stimulation. The kids desks where they do their homework are covered in toys, gadgets and other stimulating things.

#5: Mindfulness in Prisons: Based on studies on healthy participants that show mindfulness can reduce amygdala activity and increase PFC activity when confronted with social threat (e.g. images of angry faces in fMRI studies), prisons across Russia are making mindfulness sessions mandatory for all prisoners.

#6: Chocolate and PTSD: Studies have shown that low tryptophan can decrease activity in the PFC when viewing emotional images of faces while in an fMRI machine. Low PFC function has been correlated with PTSD symptoms. Because chocolate contains tryptophan, a psychiatrist in South Africa is prescribing a bar of chocolate per day for her patients.

bigstock--194065177

Based on studies about tryptophan and the PFC, would you prescribe a piece of chocolate cake for sufferers of PTSD?

Teachers: Some answers are included in the teacher workbook in the Quantitative Methods Support Pack. AVAILABLE HERE


Extension Task #1

There are some key terms that can be used to identify specific issues with generalizability. Read the definitions below (and there’s more detail in the textbook) and see if any of these terms are relevant to your generalizability issues you identified in the above study.

  • Ecological validity: generalizability based on the environment of the original study (most commonly applied to laboratory experiments).
  • Population validity: generalizability based on the extent to which the results can be generalized to a wider (or different) population.
  • Mundane realism: generalizability based on the extent to which the procedures in the study reflect real life scenarios.
  • Historical validity: generalizability based on the era in which the study was conducted (e.g. applying studies from the 1950s to today’s society).
  • Correlation/Causation: not a type of validity of methodology, but it is relevant when assessing the use of evidence to support an argument.

Exam Tip: If you’re not sure which type of validity you are referring to, it’s fine to use the broad term “generalizability.” 

Extension Task #2

The key to scoring high marks for critical thinking is to have well-developed explanations of issues of generalizability. This includes giving reasons why the results might not be applicable to other contexts. Choose one of the study/application summaries above and write a detailed explanation (about 100+ words) of why there is an issue of generalizability.

These posts might help as they have examples:


Feel free to leave questions or comments.