Teaching Case (VI): When Incremental Wins Become Catastrophic Loss

The Software Touchdown: When Incremental Wins Become Catastrophic Loss

Case Type

Decision-forcing institutional analogue (engineering, safety, corporate governance)

Core Themes

• Incrementalism as a cover for risk
• Narrative competition vs. system integrity
• Safety as an organizational property
• Diffuse responsibility in technical systems
• When “not stopping” is the decision

Case Part I: The Situation (Boeing, mid-2010s)

The Context

Timeframe: 2011–2018
Actors:

• Boeing senior leadership
• Engineering teams
• FAA (regulator)
• Airline customers

Boeing faces a competitive threat:

• Airbus’s A320neo offers better fuel efficiency.
• Airlines demand rapid response.
• Market share and reputation are at stake.

The strategic constraint:

Boeing wants to respond without designing a new aircraft, to save time, cost, and certification effort.

This is the opening condition.

Case Part II: The Decision Frame (As It Appeared Internally)

Boeing leadership confronts a choice:

1. Design a new aircraft
   • expensive,
   • slow,
   • uncertain,
   • requires full certification.
2. Update the existing 737 platform
   • faster to market,
   • cheaper,
   • preserves pilot commonality,
   • framed as “incremental.”

Option 2 is chosen.

This is not reckless on its face.
It is rational under competitive pressure.

Case Part III: Where the Touchdown Bias Enters

As the 737 MAX evolves, successive decisions accumulate:

• Larger engines alter flight characteristics.
• Software (MCAS) is introduced to compensate.
• MCAS authority increases incrementally.
• Training requirements are minimized.
• Documentation is simplified.

Each decision:

• appears narrow,
• is justified locally,
• avoids triggering broader review.

Crucially:

No single decision looks like a hail mary.

This is touchdown bias by accretion.

Case Part IV: The Checkdowns That Existed (And Were Rejected)

At multiple points, the system could have slowed:

• redesigning the airframe,
• limiting MCAS authority,
• requiring additional pilot training,
• escalating certification review,
• pausing deployment.

Each option:

• increased cost,
• delayed delivery,
• weakened competitive narrative,
• risked appearing unresponsive to Airbus.

The checkdowns were visible.
They were simply not rewarded.

Case Part V: Ego Without Villains (For the Third Time)

This case resists moral simplification.

No single engineer wanted harm.
No executive set out to endanger passengers.
No regulator consciously abdicated duty.

Instead, the system rewarded:

• schedule adherence,
• cost containment,
• competitive parity,
• narrative continuity (“still a 737”).

Ego here is not arrogance.
It is identity preservation:

“We can do this without changing who we are.”

That belief replaced restraint.

Case Part VI: The Failure (The Interception That Couldn’t Be Recovered)

Two crashes follow:

• Lion Air Flight 610 (2018)
• Ethiopian Airlines Flight 302 (2019)

Loss of life is immediate and irreversible.

Post-crisis revelations show:

• MCAS behavior misunderstood,
• pilot training inadequate,
• certification fragmented,
• accountability diffused.

This is the key analogy:

The system kept scoring small points until the field collapsed.

Case Part VII: Aftermath and Responsibility Diffusion

After the crashes:

• programs are grounded,
• executives resign,
• reports are issued,
• reforms are promised.

But note the pattern:

• no single “play” is owned,
• decisions are spread across years,
• blame is distributed thinly.

This is touchdown bias’s final move:

When everyone scored, no one threw the ball.

Case Part VIII: Compare to Prior Cases

Different domains.
Same structure.

Case Part IX: Governance Insight

The failure was not technical alone.
It was governance exhaustion.

The institution lacked:

• a hard stop mechanism,
• permission to redesign,
• authority to say “this is no longer incremental.”

In football terms:

The team never switched to kneel-down mode.

Case Part X: Bridging to ACP

ACP addresses exactly this failure mode:

• Scope locks → prevent silent expansion of authority
• Phase awareness → distinguish iteration from production
• Auditability → preserve decision lineage
• Human ownership → no “the system decided”
• Refusal legitimacy → stopping is success

ACP would not have guaranteed safety.
But it would have forced the question earlier:

“Are we still checking down, or are we pretending we are?”

Teaching Notes (Facilitator)

What this case teaches

• Incrementalism can be more dangerous than boldness.
• Safety failures often begin as efficiency wins.
• Institutions fail when stopping becomes socially impossible.

Common discussion traps

• Blaming engineers alone
• Treating regulation as the fix
• Assuming better documentation would have sufficed

Success outcome

Participants articulate:

“The problem wasn’t that Boeing acted.
It was that it never re-asked whether acting was still safe.”

Optional Extensions

• Compare to medical device recalls
• Compare to infrastructure maintenance deferrals
• Use as a prelude to AI deployment governance