Negative Findings

Negative findings are helpful in in program development but they can be overlooked in the at time overwhelming flow of ACT data. That is why it is often recommended in the various sections on ACT research evidence on the website to prioritize well done meta-analysis and reviews by high quality scientific agencies; it's very hard nowadays for any one person to pick through thousands of studies to find negative findings. The list below is meant more as a set of examples. Readers can help by sending items they have stumbled across in the Randomized Controlled Trials list or elsewhere to missingstudies at gmail dot com (the email address is written like that to avoid spam). If you have them organized by topic and you are an ACBS member who has logged in you can easily add a daughter page below and describe the study or list of studies. If you have actual data about the population or treatment characteristics that predict poor response, that would be especially important. We know moderation is key and there is no "one size fits all" solution so finding you when ACT is less effective and when it is not is key to our progress. Thus it is good to list ideas about why it might not have worked, but do also consider that the other alternative is that it did not work because it does not work! It is especially important if the process moved significantly but the outcome did not. That is a model failure and it is a type of failure that demands careful attention. If the process did not move, or the outcome did not move, or both, it could be a technology failure or an interaction with other features and so on so. In either case, it needs to be pursued and this list is thus a resource for progress. In general below we are trying to emphasize studies where the primary outcome was moved significantly less by ACT than other methods -- mere mixed finding are quite common as any good meta-analysis will reveal but if it is unclear (e.g.. the primary outcome is not spelled out) those too can be included here. We hope this helps you find the times and places where ACT is second best or maybe even downright unhelpful.

Studies with negative findings or mixed findings (note, this is not a comprehensive list! See above):

Van Wasshenova, E., Boardley, D., Geers, A., Tull, M. & Steiner, V. (2023) A Brief Value-based Randomized Intervention to Promote Physical Activity in Patients Attending Cardiac Rehabilitation. American Journal of Health Education, 54(1), 10-19, DOI: 10.1080/19325037.2022.2142334 The results indicate that the intervention did not change positive affective associations or increase physical activity behavior.
Hansen, K.E., Brandsborg, B., Kesmodel, U.S. et al. (2023). Psychological interventions improve quality of life despite persistent pain in endometriosis: results of a 3-armed randomized controlled trial. Quality of Life Research, 32, 1727–1744. DOI: 10.1007/s11136-023-03346-9 Compared to WL, psychological intervention (MY-ENDO + Non-specific) did not significantly reduce pain. However, psychological intervention did significantly improve three QoL-subscales as well as the endometriosis-related symptoms ‘dyschezia’ and ‘constipation’.
Behnamfar, K., Rajaei, A., Teymouri, S. (2023 in press). Comparing the effectiveness of cognitive-behavioral therapy and acceptance and commitment therapy on resilience of mothers with autistic children. Razavi International Journal of Medicine. DOI: 10.30483/rijm.2023.254412.1234 The results confirmed that CBT is more effective than ACT on the resilience of the mothers of children with autism. It is suggested that CBT be used to promote the mental well-being of these mothers.
Myin-Germeys, I., van Aubel, E., Vaessen, T., Steinhart, H., Klippel, A., Lafit, G., Viechtbauer, W., Batink, T., van Winkel, R., van der Gaag, M., van Amelsvoort, T., Marcelis, M., Schirmbeck, F., de Haan, L., & Reininghaus, U. (2022). Efficacy of Acceptance and Commitment Therapy in Daily Life in Early Psychosis: Results from the Multi-Center INTERACT Randomized Controlled Trial. Psychotherapy and Psychosomatics, 91, 411–423. DOI: 10.1159/000522274 INTERACT did not support a significant effect of ACT-DL over TAU on the primary outcome measure of psychotic distress.
Mueller, J., Richards, R., Jones, R. A., Whittle, F., Woolston, J., Stubbings, M., Sharp, S. J., Griffin, S. J., Bostock, J., Hughes, C. A., Hill, A. J., & Ahern, A. L. (2022). Supporting Weight Management during COVID-19 (SWiM-C): A randomised controlled trial of a web-based, ACT-based, guided self-help intervention. Obesity Facts, 15(4), 550–559. DOI: 10.1159/000524031 Differences in weight and some other outcomes were compatible with no effect of the intervention.
Levin, M. E., Krafft, J., Seifert, S. & Lillis, J. (2022). Tracking valued and avoidant functions with health behaviors: A randomized controlled trial of the acceptance and commitment therapy matrix mobile app. Behavior Modification, 46(1), 63–89. DOI: 10.1177/0145445520913987 Neither the HBT or HBT+ACT app improved other health behavior outcome measures or values processes relative to the waitlist. Overall, findings suggest some benefits of the ACT Matrix app for addressing physical activity by tracking valued/avoidant functions, but mixed findings on acceptability, outcomes, and processes of change suggests impact may be relatively limited.
Braun, L., Terhorst, Y., Titzler, I., Freund, J., Thielecke, J., Ebert, D.D., & Baumeister, H. (2022). Lessons learned from an attempted pragmatic randomized controlled trial for improvement of chronic pain-sssociated disability in green professions: Long-term effectiveness of guided online-based Acceptance and Commitment Therapy (PACT-A). International Journal of Environmental Research and Public Health, 19(21), 13858. DOI: 10.3390/ijerph192113858 No treatment effect for reduction of pain interference was found.
Kianpour Barjoee, L., Amini, N., Keykhosrovani, M., & Shafiabadi, A. (2022). The Effect of Positive Thinking Skills Training and Acceptance and Commitment Therapy on Perceived Stress among Women with Breast Cancer. Women’s Health Bulletin, 9(1), 9-16. DOI: 10.30476/whb.2022.93905.1159 According to the results, positive thinking training and ACT were effective in reducing perceived stress in women with breast cancer. Moreover, positive thinking training was more effective in declining stress in women with breast cancer.
Ebrahimi, A., Nasre Esfahan, E., Akuchekian, S., Izadi, R., Shaneh, E., & Mahaki, B. (2022). A randomized clinical trial: Comparison of group acceptance and commitment therapy with drug on quality of life and depression in patients with obsessive–compulsive disorder. Journal of Research in Medical Scienes, 27(1), 9. SSRIs are more effective than ACT in treating depression in obsessive–compulsive patients.
Behnamfar, K., Rajaei, A.R., & Teymouri, S. (2022). Comparison of cognitive-behavioral therapy and acceptance and commitment therapy on cognitive flexibility in mothers with autistic children. Social Determinants of Health, 8(1), 1-9. DOI: 10.22037/sdh.v8i1.37302 The results indicated that both training methods of CBT and ACT significantly increased the cognitive flexibility of the mothers with autistic children. Based on the results of Tukey's test, CBT training was more effective on cognitive flexibility of the mothers with autistic children compared to ACT training.
Faghihi, A., Zanjani, Z., Omidi, A. & Fakharian, E. (2022). A comparison of cognitive behavioral therapy and acceptance and commitment therapy received by patients with major depressive disorder following traumatic brain injury for emotional status and quality of life of their caregivers: A randomized controlled trial. Asian Journal of Social Health and Behavior, 5(1), 24-32. DOI: 10.4103/shb.shb_19_22 CBT is more effective than ACT.
Laula, T. (2022). Psykologisk utvärdering av Acceptance and Commitment Therapy vid diabetes typ 2 : En behandlingsstudie med SCED-design inom primärvård. Dissertation, Mittuniversitetet, Sweden. The conclusion the author makes is that the present study as well as
previous studies does not support the continued use of this treatment without further studies of its efficacy. Retrieved from https://urn.kb.se/resolve?urn=urn:nbn:se:miun:diva-45625
Sabouri, H., Zarei, E., Samavi, A., & Amir Fakhraei, A. (2020). Comparison the effectiveness of Acceptance and Commitment Therapy and Compassion Focused Therapy on the quality of life and tendency to betray in married students with tendency to infidelity. Iranian Evolutionary and Educational Psychology, 2(3),184-199. DOI:10.29252/ieepj.2.3.184 CFT was more effective than ACT in terms of physical health and, conversely, ACT was more effective than CFT in terms of mental health and social communication.
Rahimi, A., Amiri, H., Afsharriniya, K., & Arefi, M. (2020). Comparing the Effectiveness of Cognitive Behavioral Therapy (CBT) with Acceptance and Commitment Therapy (ACT) in the Enhancement of Marital Satisfaction and Sexual Intimacy in Couples Referred to Counseling Centers. Avicenna Journal of Neuro Psycho Physiology, 7(2), 126-132. DOI: 10.32592/ajnpp.2020.7.2.107 It was found that CBT exerted a more profound effect on marital satisfaction and sexual intimacy, compared to ACT.
Levin, M.E., Hayes, S.C., Pistorello, J. & Seeley, J. (2016). Web-based self-help for preventing mental health problems in universities: Comparing acceptance and commitment training to mental health education. Journal of Clinical Psychology, 72, 207-225. doi: 10.1002/jclp.22254.
Herbert, J. D., Forman, E. M., Kaye, J. L., Gershkovich, M., Goetter, E., Yuen, E. K., Glassman, L., Goldstein, S., Hitchcock, P., Tronieri, J. S., Berkowitz, S., & Marando-Blanck, S. (2018). Randomized controlled trial of acceptance and commitment therapy versus traditional cognitive behavior therapy for social anxiety disorder: Symptomatic and behavioral outcomes. Journal of Contextual Behavioral Science, 9, 88-96. DOI: 10.1016/j.jcbs.2018.07.008 Those who received CBT evidenced greater improvements in self-reported social anxiety symptoms and overall functioning than those who received ACT. Medium effect sizes, while not statistically significant, indicate that ACT participants may have had greater improvements in observer-rated social behavior than CBT participants.
Montazeri, S., Aghaei Jashoghani, A., & Golparvar, M. (2018). Comparison of the Effectiveness of Cognitive-Behavioral Management Based on Mindfulness and Treatment Based on Acceptance and Commitment on Psychological Well-being in Multiple Sclerosis Patients. Sadra Medical Journal, 7(1), 59-74. DOI: 10.30476/smsj.2018.44755 The results of Bonferroni's post hoc test for the comparison of couples in groups showed that cognitive-behavioral management stress based on mindfulness rather than the one based on acceptance and commitment has had a greater impact on both post-test and follow-up on the total score of psychological well-being and autonomy components, environmental mastery, purposeful life, and personal growth.
Masjedi-Araani, A., & Khanaliloo, R. (2018). Comparison of the Efficacy of Cognitive-Behavioral Therapy (CBT) and ACT (ACT) in Reducing Depression in Women with Multiple Sclerosis (MS). The International Journal of Body, Mind and Culture, 5(2), 112-121. DOI: 10.22122/ijbmc.v5i2.117 CBT was more effective than ACT.
Jung, M. J., Na, M. O., & Son, C. N. (2017). Effects of Acceptance and Commitment Therapy (ACT) on premenstrual symptoms, attitudes about menstruation, and perceived stress of women with premenstrual syndrome. Journal of Digital Convergence, 15(1), 485-495. DOI : 10.14400/JDC.2017.15.1.485 In regards to attitudes toward menstruation, there were no statistical significance in the acceptance and commitment therapy (ACT) group
Ko, Y-J., & Kim, J. M. (2015). The effect of cognitive behavioral therapy and acceptance and commitment therapy on female university students with social anxiety disorder. Korean Journal of Emotional & Behavioral Disorders, 31(2), 175-197. In the case of the ACT group, emotional characteristics and social interaction anxiety did not show significant changes.
Craske, M. G., Niles, A. N., Burklund, L. J., Wolitzky-Taylor, K. B., Vilardaga, J. C. P., Arch, J. J., Saxbe, D. E., & Lieberman, M. D. (2014). Randomized controlled trial of cognitive behavioral therapy and acceptance and commitment therapy for social phobia: Outcomes and moderators. Journal of Consulting and Clinical Psychology, 82(6), 1034–1048. DOI: 10.1037/a0037212 Equivalent findings overall for ACT and CBT but better findings for CBT with participants with lower self-reported psychological flexibility at baseline
and either very low or very high self-reported fear of negative evaluation.
Luciano, J. V., Gullar, J. A., Aguado, J., López-del-Hoyo, Y., Olivan, B., Magallón, R., Alda, M., Serrano-Blanco, A., Gilli, M., & Garcia-Campayo, J. (2014). Effectiveness of group acceptance and commitment therapy for fibromyalgia: A 6-month randomized controlled trial (EFFIGACT study). Pain, 155, 693-702. DOI: 10.1016/j.pain.2013.12.029 Group based ACT was statistically superior to both recommended pharmacological treatment and waitlist immediately after treatment, and improvements were maintained at 6 months with medium effect sizes in most cases. Unexpectedly, 4 of the 5 tested path analyses did not show a mediation effect.
Lanza, P. V., García, P. F., Lamelas, F. R., & González-Menéndez, A. (2014). Acceptance and commitment therapy versus cognitive behavioural therapy in the treatment of substance use disorder with incarcerated women. Journal of Clinical Psychology, 70, 644-657. DOI: 10.1002/jclp.22060 At posttreatment, CBT was more effective than ACT in reducing anxiety sensitivity.

Steven Hayes January 29, 2006 - 10:27pm

ACT for Math Anxiety

Zettle, R. D. (2003). Acceptance and commitment therapy (ACT) versus systematic desensitization in treatment of mathematics anxiety. The Psychological Record, 53, 197-215.

This study compared ACT to systematic desensitization. The recent meta-analysis of the ACT literature included it and you can download the study from this website. Rob found that ACT worked better than a control condition, as did desensitization. On measures of math anxiety, ACT and desensitization were not statistically significantly different, but the d was in the wrong direction, and on general levels of overall anxiety, desensitization was better.

In 1999 ACT book we said this about fitting a fairly challenging intervention like ACT to the problem: "Before ACT begins, the client must be prepared for it. It can be an intensive intervention and the clients should not be subjected to such interventions lightly." In line with that, it seems possible that Math Anxiety is too low level of a problem to warrant ACT. Interestingly, in Zettle's study highly experientially avoidant participants did better in ACT than those who were not avoidant -- while the same did not hold true for desensitization.

This study has since been replicated and the results were a bit different:

Brown, L. A., Forman, E. M., Herbert, J. D., Hoffman, K. L., Yuen, E. K. and Goetter, E. M. (2011). A randomized controlled trial of acceptance-based behavior therapy and cognitive therapy for test anxiety: A pilot study. Behavior Modification, 35, 31-53.

It was a very small RCT (N = 16) for test anxiety comparing ACT (with mindfulness elements) and Beck’s cognitive therapy. Much like the Zettle study, there were similar outcomes on self-reports. In this study however ACT participants did better on actual test scores in school. So perhaps the Zettle result was not just due to a focus on a more minor population.

Steven Hayes February 3, 2006 - 12:45am

ACT for Pain

I have had many failures replicating the Hayes et al (1999) pain tolerance study. Over the years I have answered criticisms of reviewers in my attempt to have negative results published. As a result I have begun to video record my interventions for analysis by anyone who is interested. I now take a whole host of adherence measures and subjective reports. I have even examined acceptance over the long run in the lab. But still no effects on pain tolerance for ACT-derived protocols over placebos or alternative treatments (relaxation and education or supression). often no effects for acceptance-based interventions at all.

I would love to offer practical suggestions but as you will see from my list I have five years of research here - so I am trying! I am all out of suggestions. I also think that if the procedure has to be contrived so much that effects are only measureable on such measures as the AAQ (which measures what ACT teaches the client - ipso facto we will see changes in scores) or if the research requires a clincian as experimenter or highly elaborate and exhaustive subjective ratings and statistical techniques - its not a very powerful effect! So I hesitate to contrive procedures much mroe complex than what I have got. (The details of which I hope we will discuss as a community in reposne to these postings).

So here is my list of failures....

Failure 1
Examining the effectiveness of acceptance and control – based interventions on pain tolerance.

This study compared the effectiveness of an acceptance-based and control-based intervention on pain tolerance using a cold pressor task, and is a part-replication and extension of the Hayes et al., (1999) study. Twenty college students were exposed to the cold pressor task before, immediately after, and 20 minutes subsequent, to an 8 minute acceptance-based or control-based therapeutic intervention, including the use of physical and abstract metaphors. Half of the participants were also assigned to a high demand characteristic condition in which the experimenter purposely placed subtle social pressure on them to please the experimenter. The results showed that the most significant factor influencing performance on the cold pressor task was the effect of placing social pressure on participants, with no significant overall effect for Acceptance or Control interventions.

Failure 2
A Systematic Analysis of the Role of Demand Characteristics in an Acceptance Based Approach to Pain Tolerance.

This study compared the role of demand characteristics in an acceptance-based approach to pain tolerance and both the long and short-term effects of the acceptance-based versus the no therapy interventions. Forty participants were exposed to a cold pressor task before and immediately after a short intervention. Twenty-eight participants also completed a follow up task three months later. Half of the participants receiving each intervention were also subject to high levels of demand characteristics. In this high demand condition the experimenter placed subtle social pressure on the participants to perform well on the second cold pressor task. The findings showed that participants in the acceptance condition improved more, but not significantly more, than those in the no therapy condition. Participants in the high demand condition performed significantly better than those in the low demand condition. Interaction effects for therapy x demand were also found between the experimental groups.

Failure 3
Comparing the Effectiveness of Acceptance and Control Strategies for Pain Tolerance with a Sub-clinical Population

This study used an experienced psycho synthesis therapist and cognitive behavior therapy postgraduate as an experimenter who had studied ACT and taken the full ACT weekend workshop. This study was a part replication and extension of the Hayes, Bissett, Korn, Zettle, Rosenfarb, Cooper and Grunt (1999) study. Four sub-clinical volunteers (two smokers, one drinker and a tantrum thrower) were exposed to a cold pressor task before, immediately after, and several weeks subsequent to a 90 minute acceptance-based therapeutic intervention. Baseline rates of idiosyncratic problem behaviours were also recorded prior to, and for several weeks subsequent to, the initial intervention. The acceptance-based intervention was then administered weekly for up to 25 weeks by a qualified cognitive-behavior therapist to asses its impact on pain tolerance and target problem behaviour rates in the longer term. The acceptance-based intervention showed weak effects on pain tolerance during all phases of the study and no discernible effects on problem behaviors were observed (in fact they got worse!).

Failure 4
Examining the effectiveness of an acceptance and relaxation-based intervention on pain tolerance

This study attempted to compare the effectiveness of an acceptance and relaxation-based intervention on pain tolerance to a cold pressor task and is a partial replication and extension of the Hayes et al., (1999) study. Forty college students were exposed to a cold pressor task before and immediately after an eight minute acceptance-based and relaxation-based intervention. Half of the participants in each group were also assigned to a high demand condition, in which subtle social pressure was placed on the participants to please the experimenter and do well in the task. The results confirmed that the most significant factor influencing performance on the cold pressor task was placing social pressure on participants to do well. There was no significant overall effect for either the acceptance or relaxation-based intervention although both produced mild improvements in pain tolerance.

Failure 5
The effectiveness of an acceptance and control-based interventions on pain tolerance at two different levels of pain in a cold pressor task.

This study employed the now standard procedure of using 40 subjects – half get an acceptance protocol and half get a control-based protocol following a baseline cold pressor task and before a post-intervention cold pressor task. Half of each group get cold pressors at 0 degrees centigrade – the remainder at 3 degrees. Acceptance has a mild effect – not significant – and does not interact with temperature on an ANOVA. This dashed our hopes that maybe acceptance was more useful for intense pain over mild pain – no lab data to support that idea yet.

Failure 6
So maybe it’s the subject’s fault! Assessing the effectiveness of acceptance and control based interventions with anxious and non-anxious subjects.

We recruited 20 high and 20 low trait anxiety subjects by screening with the STAI. We defined high and low as one SD above and below the mean score as outlined in the standardized distribution scores. Half of each got a control intervention and half got acceptance. Anxious subjects did not benefit more than non anxious from either the acceptance or the control intervention on a cold pressor task. Overall no effect for acceptance. No effects were found on any subjective reports.

btroche February 10, 2006 - 8:19pm

ACT for weight control

We did a study in the mid 1980's comparing an ACT protocol for weight control with a "best of breed" CBT protocol in a 12 session group format. The results were pretty good for both of the interventions, but not different. This was early in the research program. Self as context, contact with the present moment, and values were all less emphasized than they are now.

Jason Lillis' dissertation in 2007 showed that a brief ACT workshop was able to reduce weight related self-stigma and increase quality of life. Even though the intervention did not target weight loss directly, there was an average of 5 pounds lost and maintained at follow up. Check out the publication for the main study and for the weight-specific AAQ.

Steven Hayes January 29, 2006 - 10:28pm