The Rocket That Could
Lose an Engine

Why redundancy in spacecraft is a decision about where failure is allowed to happen


I. Engine 1Cape Canaveral, October 7, 2012

Falcon 9 lifted off at 8:35 on a Sunday night with Dragon, SpaceX’s cargo capsule, on top. It was the first official resupply run to the International Space Station under SpaceX’s contract with NASA. About a minute and nineteen seconds into the climb, one of the nine Merlin engines at the base of the first stage, Engine 1, lost pressure suddenly. The flight computer commanded it to shut down at once. The fairing that shielded the engine from the airflow ruptured as the pressure escaped, but none of the other eight engines were affected.

What happened next was mostly nothing, and that was the point. The flight computer recomputed the ascent in real time. The eight remaining engines burned longer to make up for the lost thrust, and Dragon reached its orbit. Three days later the station crew grappled it and berthed it. “Falcon 9 did exactly what it was designed to do,” SpaceX said the next morning, comparing it to the Saturn V, which had lost engines on two flights.

Redundancy in space engineering means duplicating the systems that matter so that if one fails, another takes over and the mission goes on. It sits near the center of spacecraft design for a simple reason. Once a vehicle is climbing to orbit, or on its way to another planet, nobody can send a repair crew. Whatever the machine will need in order to survive its own failures has to be built in before it leaves the ground.

II. The Only AbortKennedy Space Center, July 29, 1985

The Space Shuttle carried the same idea in a different shape. It rode on three main engines and two solid rocket boosters, and it was designed to survive the loss of one main engine during ascent. On this afternoon, with Challenger and its Spacelab 2 payload, the design was tested. Five minutes and forty-five seconds after liftoff, the center engine shut down. Challenger flew an “abort to orbit,” the first in-flight abort of the program and, as it turned out, the only one. The crew reached a lower orbit than planned, and NASA declared the mission a success.

The details carry a second lesson. According to NASA’s mission report, the engine was shut down because the two temperature sensors watching its fuel turbopump had both failed, and the reading from one of them drifted past the redline. The engine’s other parameters looked normal. About eight minutes into the flight, a sensor on another engine began to drift the same way, and automatic shutdown of the two remaining engines was inhibited so that the orbiter could reach an acceptable orbit.

The machinery meant to protect the engines had become a way for them to fail. Every part added for safety is still a part, with its own ways of breaking.

III. Spares for the SparesVesta and Ceres, 2010–2018

Engineers usually sort redundancy into three kinds. Component-level redundancy doubles individual parts, like two radios or two power supplies. System-level redundancy duplicates an entire subsystem. Functional redundancy is the subtlest: a different system does the same job by different means.

NASA’s Dawn probe, which orbited the protoplanet Vesta and then the dwarf planet Ceres, carried all three. It had three ion thrusters but ran only one at a time, because each was qualified to process only part of the xenon the mission would need. Its hydrazine thrusters were arranged in two independent strings of six. And it pointed itself with four reaction wheels, spinning masses that turn a spacecraft without spending propellant.

The wheels failed one by one, in 2010, 2012 and 2017. As they did, more of the work passed to the hydrazine thrusters, which could do the same job by firing small jets, and after the third failure the thrusters took over. That was functional redundancy working as planned, and it had a price that the wheels did not. Every correction spent propellant. On October 31, 2018, Dawn ran out of hydrazine and could no longer point its solar arrays at the Sun or its antenna at Earth. The backup had kept the mission alive for years, but it was a backup with a fuel gauge.

IV. Five WheelsColumbia Hills, Mars, March 2006

Not every failure needs a spare. Sometimes the better answer is a system that keeps working, only less well. Engineers call this graceful degradation.

Spirit, one of NASA’s two Mars Exploration Rovers, had six wheels. Its right-front wheel had been troublesome since 2004, drawing about twice the current of the others, and the team had already taken to driving the rover backward to spare it. Driving that way, one rover engineer said at the time, was “like dragging an anchor.” In March 2006 the wheel stopped turning for good. Controllers gave up on its motor and went on with five wheels, dragging the sixth, slower and with more strain on the rest. The rover was not as capable as before, but it was still moving.

V. The BillCape Canaveral, October 2012

None of this is free. Every extra component adds weight, and weight costs fuel. More parts also mean more ways to fail, as Challenger’s sensors showed. So engineers spend redundancy where a failure would be catastrophic, in propulsion, power and communications, and accept single points of failure in places where losing one part would hurt less.

The CRS-1 flight showed what that choice looks like in practice. Falcon 9 had a second passenger that night, a prototype communications satellite for the company Orbcomm, meant to be released into a higher orbit after a second burn of the upper stage. The engine-out had used extra propellant. To protect the station, NASA had required that the restart happen only if there was more than a 99 percent chance of completing it. The liquid oxygen left on board was enough for roughly 95 percent. The rocket did not try. The satellite was left in a low orbit, re-entered within days, and Orbcomm declared it a total loss.

The rocket did not decide that in the moment. The rule had been written in advance, and it said which mission would be protected and which one could be given up.

⁂

Redundancy is often described as insurance, or as waste you tolerate because you are afraid. The record of these machines suggests something more precise. It is a design decision about where failure is allowed to happen. Falcon 9 could lose an engine. Dawn could lose its wheels until it ran out of hydrazine. Spirit could lose a wheel and keep driving. In each case someone had decided beforehand what the machine could afford to lose, and had paid for that decision in weight, propellant and complexity, knowingly.

Sources

SpaceX, “SpaceX CRS-1 Mission Update,” October 8, 2012 (via SpaceRef).

SpaceX, “CRS-1 Mission,” spacex.com/launches/crs1.

Stephen Clark, “Orbcomm craft falls to Earth, company claims total loss,” Spaceflight Now, October 11, 2012.

NASA, “STS-51F,” nasa.gov/mission/sts-51f; NASA, STS 51-F National Space Transportation System Mission Report, 1985.

NASA PDS, Dawn instrument host description (ion propulsion and reaction control systems).

NASA JPL, “Dawn Observing Ceres; 3rd Reaction Wheel Malfunctions,” April 25, 2017; NASA Science, Dawn mission status, November 2018.

NASA JPL, “NASA’s Mars Rovers Roll Into Martian Winter,” July 2004; “Spirit Team Gives Up On Front Wheel,” SpaceDaily/Mars Daily, April 2, 2006; New Scientist, “Mars rover’s wheel breaks as winter beckons,” March 2006.

Quotations appear only where they are attributed in these sources.