DVOS — Draft Value Over Slot

The scouting-director analog of WAB: how much fWAR a director's picks produced above what their draft slots historically return. Plus the surprising structure of where draft skill is — and isn't — real.

What it measures

Wins and rings get credited to the GM, but the amateur draft is run by the scouting director, and his raw output is dominated by where he picks — a team drafting 1st overall has far more value available than one drafting 28th. DVOS strips that out. Each pick is valued at the player's career fWAR and benchmarked against what that overall draft slot has historically returned:

Value is credited only to the draft where a player actually signed (so an unsigned-then-redrafted player counts once), and recent classes are maturity-adjusted so a 2021 pick isn't punished for not having debuted. A perfectly average drafter lands at 0 by construction; +108 means his picks out-produced their slots by 108 career wins. bref WAR is never used.

The part that surprised us — what's actually a skill

At the single-draft level DVOS is pure noise (year-over-year stickiness ≈ 0): one star swamps a whole class. So the real test is split-half reliability — does a director's value in half his drafts (odd years) predict the other half (even years)? Across directors with ≥6 drafts, Spearman-Brown adjusted:

This breaks into two independent skills (an SD's early-round value is uncorrelated, ≈ 0, with his late-round value — being good at one says nothing about the other):

• Early-Round Selection (rounds 1–10, cost-controlled value): reliability +0.49. A broad skill — most directors rank meaningfully, and it survives removing the top few (right panel, blue). Premium, scoutable picks; shared information, executed competently by many.
• Late-Round Value (rounds 11+, total-career value): reliability +0.60, the single stickiest scouting signal. But it is rare and concentrated in a handful of individuals — Brian Sabean (+134), Mickey White (+83), Ron Hopkins (+63), Sandy Johnson (+63), Dan O'Brien Jr (+55). Drop those ~5 and the +0.60 collapses to ~0 (right panel, red): for everyone else, the deep rounds are a coin flip. It also travels with the person across teams (cross-franchise corr +0.51): the Yankees' late-round value was Sabean, not the Yankees — it left when he did.

So scouting is real skill, but in the deep rounds it's scarce, individual, alpha-like: most directors earn the "index return" (slot value) because finding a star at pick 300 is a luck-dominated game where a repeatable edge is genuinely rare — exactly like persistent alpha among active fund managers. It is not post-2012 bonus-pool exploitation (the edge is stronger pre-2012), and it's not survivorship in a handful of orgs — it's a handful of people.

Position: hitters yes, pitchers no

Splitting DVOS by the drafted player's position, only one cut is reliable: drafting hitters (C/IF/OF) above slot repeats (split-half +0.54), while drafting pitchers above slot does not (≈ 0). The two are nearly independent (corr +0.28), so this isn't one "drafting" skill — it's specifically a hitter-evaluation skill.

Why pitchers don't repeat — the realized fWAR of a drafted arm is mostly decided after the pick, by forces the scout doesn't control:

• Injury attrition. Pitchers break at far higher rates than hitters — a perfectly-scouted arm can need Tommy John surgery and come back a fraction of himself, or never. A big chunk of a pitcher's career outcome is a health lottery that no draft-day look predicts, so it lands as noise rather than scouting signal.
• Development- and timing-dependence. Pitcher outcomes hinge on what happens post-draft: pitch design, mechanical overhauls, velocity gains, the org's pitching-development infrastructure, and when the arm gets pushed vs. shut down. Two clubs can draft the same projectable high-schooler and end up with wildly different players depending on their pitching dev — so realized fWAR credits (or blames) the development system and timing as much as the original evaluation. Hitters are more projectable, and their value accrues more directly from traits visible at the draft.

The upshot: pitcher value-over-slot is dominated by factors downstream of the draft decision, so it doesn't repeat for a given evaluator — the classic "there is no such thing as a pitching prospect." That's why the leaderboard ranks Hit DVOS but shows Pitcher DVOS only descriptively. (Finer buckets like SS or C are too thin per director to be anything but noise.)

How to read it

• Total DVOS is the headline, but it blends two uncorrelated skills — judge a director on the column that matches the job: a top-10-picking rebuilder on Early Sel, a contender or farm-depth org on Late Val.
• Never on a single draft (noise), and below the top few names Late Val is coin-flip variance, not a ranking.
• Coverage is incomplete. We credit only draft-years where SABR records a single, unambiguous scouting director (~79% of franchise draft-years, 1985–2024); years with multiple names or none are left out. Some clubs have missing eras — notably the Cardinals' 1985–2009 is unrecorded, so Albert Pujols (1999) is excluded — meaning a director's absence or short record can be a data gap, not a verdict.
• Selection vs. player-development is entangled (a great drafting org may also develop well).

Fit on full team-seasons (≥100 games), 1985–2024/25. All figures are franchise-level outcomes credited to the decision-maker in the relevant year — see the GM profiles for per-executive numbers.