How Do You Judge a Scheduling Study?

Every claim about scheduling rests on a comparison, and the credibility of the claim is set by how the two sides of that comparison came to exist. In the Gap study, stores were randomized into treatment and control, so the groups were alike on average before anything changed and the difference that followed can be attributed to the treatment. That is the reason its 5.1% productivity result supports causal language such as raised or increased (Kesavan et al., 2022).

The Shift Project studies ask closely related questions and earn a very different warrant. They survey workers at one point in time on a non-probability sample recruited through social media advertising, so nobody assigned anyone to a stable or an unstable schedule (Schneider & Harknett, 2019, 2021). Workers reporting unpredictable schedules differ from workers reporting predictable ones in ways a snapshot cannot fully separate, including which employers hire them and which jobs they are able to accept, so the findings support association and nothing stronger.

The practical rule follows directly. If a write-up never tells you how people ended up in each group, treat the finding as a pattern worth knowing rather than a lever worth pulling. Associations remain useful, because they tell you where to look and what to measure in your own operation, but they do not tell you what will happen when you change the schedule.

A single experiment usually produces several headline numbers, and the biggest one is rarely the one you should carry into a plan. The Gap experiment reports 5.1% as its intent-to-treat estimate and 16.4% as the adherence-adjusted treatment-on-treated estimate (Kesavan et al., 2022). Intent-to-treat counts everyone assigned to treatment whether or not managers followed the practices evenly, so it is often the more policy-relevant estimate for a real rollout. It remains one result from one retailer, not a lower bound or promised return for your operation.

Survey work carries a parallel distinction between what was observed and what was modeled. The hardship gradient in the Shift Project research, where hunger runs from 23% to 50% across the range of schedule instability, is a predicted probability, adjusting for wages, income, and hours, not the observed share of workers in each group (Schneider & Harknett, 2021). The adjustment is a strength, since it answers the objection that the pattern is really about pay, but the resulting figure describes statistically comparable workers rather than a headcount anyone counted on a shop floor.

The strictest version of this check applies to modeled counterfactuals. The 2019 authors model what eliminating on-call shifts would be associated with, and they state plainly that these estimates are not causal (Schneider & Harknett, 2019). A projection like that extends an observed pattern into a hypothetical, and it is not a measured effect of a policy anyone actually changed. When a deck quotes such a figure as an expected result, the number has been promoted a rung it did not earn.

Even a clean randomized result gets misread if you skip what was actually delivered. The Gap experiment assigned stores to a bundle of six practices that moved as one package, so the 5.1% productivity gain belongs to the package rather than to any member of it (Kesavan et al., 2022). Our write-up of the Gap experiment sets out what the bundle contained. The reading lesson is narrower than the contents: an experiment can only speak to the unit it randomized, and here the unit was the whole set.

The companion question is what the comparison group already had, because an effect size measures the distance between two arms and not the distance from zero. Gap's control stores were not sitting at an untouched status quo, as the Gap write-up explains. The habit generalizes further than that one case, so before porting any published number into a plan, ask what the study's comparison group was already doing on the day measurement started.

Diagnosing your own baseline is concrete work rather than a caveat you nod at. Write down how much notice your sites actually give, how often published shifts get canceled or moved, how many people work variable start times from week to week, and how much of the schedule is set by one manager's habit. If that inventory sits below where the study's comparison arm stood, the published estimate is not measuring the move you are about to make. Your early gains could run larger, because the cheapest fixes tend to come first, or smaller, because your operation differs in ways the design never tested.

Two widely circulated talking points did not survive our verification and are deliberately absent. The first is the claim that unstable schedules matter more than low wages. That ranking did not verify against the sources it is usually attributed to, and we do not repeat it.

The second is a bundled set of prevalence statistics about how many workers get short notice or work clopenings. The bundle failed verification as it circulates, though a couple of its components do appear individually in verified sources elsewhere in this library. Statistics that travel as a bundle deserve to be split apart, because bundles pick up a stray figure or a mismatched year as they pass through blog posts and decks.

Apply the same test to anything quoted at you, including by us. Ask for the study rather than the slide, then confirm that the figure on the slide is the figure the study reports, that it is attached to the design that supports it, and that the comparison group is what you assumed. Most failed scheduling claims break at one of those checks rather than because anyone set out to mislead.