There continues to be a debate over the value of psychological research, the validity of particular research methods and the analysis of data. I came across a study published in “Advances in Methods and Practices in Psychological Science”. To quote from the abstract,
“The 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions”
And, further down,
“The process of certifying a particular result on the basis of an idiosyncratic analytic strategy might be fraught with unrecognized uncertainty (Gelman & Loken, 2014), and research findings might be less trustworthy than they at first appear to be (Cumming, 2014). Had the authors made different assumptions, an entirely different result might have been observed (Babtie, Kirk, & Stumpf, 2014). Silberzahn, R et all (2018)”
The question I want to raise in the light of so many arguments about data, analysis and measurement is that the “science” behind so much of psychological research is little more than ideology wrapped up in blindingly clever statistical analyses.
Despite this the ongoing debate over the value of psychological evidence with regard to policy-making makes for great reading. From my point of view, it highlights that wonderful observation that academic research trails behind the reality by about 10 years. The idea that we should be using the current accepted evidence base to inform policy seems well beyond its sell by date.
Currently, the psychological evidence that is informing current government policy is that coming from Improving Access to Psychological Therapies. The evidence that is supporting IAPT interventions now seems to be about 10-15 years out of date. This is so simply because the population has moved on from the kind of individualistic and paternalistic treatment intervention that epitomises current psychological offerings.
Psychological “truths” are not dictated by psychological researchers but by the social milieu. One only needs to read a history book mapping the development psychology and in particular that of psychological therapies to realise that there is a journey in which the nature of interventions changes over time to coincide with the changes in society and the understanding and education of the population as a whole.
I note that the authors of the paper “overrated: our capacity to impact policy” make the following claim,
“Could a focus on practices – rather than on experts, immutable clinical categories, technologies and fixed knowledges – allow us to appreciate the ways that knowledge, status, relief, atmosphere and solidarity come together effectively in informal practices? If we want to recognise the fluid and innovative nature of the many informal care practices then a future course for a psychology of distress could be to develop and celebrate methods that are sensitive to this.”
I think it is important to contrast that with the early history of caring interventions which were strictly the realm of the expert in the form of the psychoanalyst and his esoteric knowledge and just how much the laws of psychology have changed in the last hundred years.
Psychological care, psychological language is no longer the exclusive realm of the expert but is in common parlance and as the authors of the above quoted paper clearly indicate there is much opportunity for the public to become engaged in the process of psychological care whether it is at the level of the individual, the group, the community or a population.
Virchow was a pathologist in the 1800s and once said, “Medicine is a social science, and politics is nothing else but medicine on a large scale. Medicine, as a social science, as the science of human beings, has the obligation to point out problems and to attempt their theoretical solution: the politician, the practical anthropologist, must find the means for their actual solution. The physicians are the natural attorneys of the poor, and social problems fall to a large extent within their jurisdiction.” Rudolf Virchow http://www.azquotes.com/author/29202-Rudolf_Virchow
Psychology as a discipline has yet to come of age as a science. This can be clearly illustrated by question I found in an online survey being carried out by a PhD student. The question was, “what is your theoretical approach?” This is a question that if you asked an engineer, physicist or a chemist would be met with a blank look, as apart from minor variations, their theoretical orientation is based upon a single set of principles. This is far from true for psychology. In the case of therapy alone there are over 360 different forms of “therapeutic intervention”. Each with their own theoretical underpinning and often at odds with each other.
A good science can also make predictions based upon hypotheses and yet while there are some areas in which there is little disagreement in psychology, there are vast areas where hypotheses have been made, interventions been carried out that have worked initially but then not been replicable by other researchers. Richard Feynman said, “It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong.” https://www.brainyquote.com/authors/richard_p_feynman Unfortunately far too many academics have far too much invested to be able to let go of their pet hypotheses and as Max Planck once said, “science progresses one funeral at a time”.
Part of this problem it seems to me is somewhat analogous to a discovery made during the Second World War ”Wald (found in Syed) carried out many investigations during the Second World War in an attempt to make Allied planes safer and the most fundamental thing his investigations revealed was “that in order to learn from failure, you have to take into account not merely the data you can see, but also the data you can’t” (Syed 2015). This statement came from observing and analysing bullet holes in the planes that returned from bombing raids over Germany. Initially there was a good deal of effort put into armouring the places where the bullet holes were. Then it dawned on Wald that these were the planes that came back; he realised that they should be looking to armour the places where the planes that didn’t come back got hit!
Recently a significant paper was published that says,
“We were surprised that genetic factors of some neurological diseases, normally associated with the elderly, were negatively linked to genetic factors affecting early cognitive measures. It was also surprising that the genetic factors related to many psychiatric disorders were positively correlated with educational attainment,” says Anttila (2018). “We’ll need more work and even larger sample sizes to understand these connections.” https://www.sciencedaily.com/releases/2018/06/180621141059.htm
This is the kind of data that illustrates the point I was trying to make above that currently psychology in the field of therapy is very focused, particularly clinical psychology, on trying to identify and resolve problems rather than looking at potential wider solutions. Antillia’s study gives a clear steer to looking at environmental and social factors as well as genetic factors (genetic factors are only weakly correlated with psychological/psychiatric disorder according to the study). My particular observation in this situation would be that we should be looking for environmental and social conditions in which psychological and psychiatric disorders occur less often and be exploring why that is rather than simply trying to solve the problems and reduce the distress that occur in other less benign environments. In other words, creating the conditions for good mental health rather than trying to reduce the bad conditions that produce bad mental health.
Despite a hundred years of research the holy Grail of psychology, a single unified theory, remains as elusive as ever. Perhaps this is because the human brain is evolving and changing in response to the changes in environmental conditions (and who can doubt that societal and environmental conditions have not changed in the hundred years so since Sigmund Freud) at a phenomenal rate. This ever-changing environment laying waste, in a decade or two, to each last “great theory” produced to explain human behaviour.
Perhaps it is time that psychologists started to work with what is already in place and what is wanted, rather than working with what was and what they would like it to be! There are steps in this direction, but they are being met with fierce resistance from entrenched opinions. Letting go of treasured beliefs and ideologies is not easy but at least heretics don’t suffer capital punishment any longer!
Anttila V. et al. Analysis of shared heritability in common disorders of the brain. Science, 2018; 360 (6395): eaap8757 DOI: 10.1126/science.aap8757
Syed, M. (2015), Black Box Thinking, The Surprising Truth about Success, John Murray publishers, Great Britain
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., . . . Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1, 337–356. doi:10.1177/2515245917747646