How can I tell what a program's success figures actually measure?

Why the question is worth asking

NIAAA's treatment booklet includes “Is treatment success measured?” among the questions it suggests asking a program. It says you may be able to compare options better by finding out whether and how a program measures success. NIDA's guide Seeking Drug Abuse Treatment: Know What to Ask says the best programs measure progress.

Asking is not an accusation. A program that tracks its results should be able to explain them.

Who was counted

The same program can produce very different percentages depending on who is in the group. A figure based on everyone who was admitted includes people who left early. A figure based only on people who completed the program leaves them out, and usually looks better. A figure based only on people who answered a follow-up survey leaves out everyone who could not be reached or chose not to reply.

Ask: “Out of how many people? Everyone admitted, everyone who finished, or everyone who answered?” If the answer is a percentage with no number of people behind it, ask for the number.

What “success” meant

Programs define success in different ways. It might mean completing the program, not drinking or using at all since discharge, not drinking or using in the past month, drinking or using less, or reporting a better quality of life. These are all worth knowing about, but they are not the same.

NIDA's Principles of Drug Addiction Treatment describes the goal of treatment as more than stopping drug use: it includes returning to productive functioning in family, work and community. NIAAA's booklet notes that definitions of recovery vary, and that many emphasize improvements in quality of life. A program that measures only one narrow outcome may be telling you something true and still not the thing you most want to know.

Ask the program to say exactly what outcome the figure describes, in one sentence.

When it was measured

A result measured on the day of discharge tells you something different from a result measured six months or a year later. Many people are not drinking or using on the day they leave a program. The more useful question is what happens over the following months.

Ask how long after discharge the figure was measured, and whether people were followed up more than once.

How the information was collected

Some programs contact former patients by phone or survey and record what they say. Some use drug tests or information from other providers. Self-reported results can be honest and still incomplete, because people who are struggling may be less likely to reply.

Ask who collected the information and how. Ask whether the figures were checked or reported by anyone outside the program, and when they were last updated.

Reading a figure against what you know

Even a carefully measured figure describes a group of people, not you. NIDA's principles say individual outcomes depend on the nature of a person's problems, how appropriate the treatment is, what other services are available and the quality of the relationship between the person and their care team. A program's results cannot predict what will happen to one person.

Relapse is also part of the picture. NIDA and NIAAA both describe a return to use as something that can happen with a long-term condition. A figure that suggests almost no one ever drinks or uses again after treatment should prompt more questions, not fewer.

Promises that go too far

Some states restrict how treatment programs advertise. Texas law, for example, prohibits treatment facilities from making promises of cure or guarantees of treatment results that cannot be substantiated. California law bars licensed and certified programs from making false or misleading statements about their services. Rules differ between states.

You do not need to know the law to be cautious. Words such as “guaranteed,” “cure” or a success rate with no explanation are reasons to slow down and ask what stands behind the claim.

What a useful answer sounds like

A useful answer might say: “We surveyed everyone discharged last year at six months. We reached about half of them. Of those we reached, this many said they had not used since leaving.” That tells you who was counted, what was measured, when and how, and it is honest about who is missing.

Write the answer down and compare programs on the same terms. The goal is not to find the highest number. It is to understand what each number means, so you can weigh it alongside everything else you learn about a program.