Apollo 13 and acceptance testing

From The Space Review:

Let’s look back again at the Apollo 13 “missed opportunities”.

During the routine countdown rehearsals, oxygen tank #2 was filled with oxygen but could not be emptied. The ground crew thought a loose nozzle fitting was the source of the difficulty, but there was no thorough investigation. The loose fitting problem was not fixed since gaseous oxygen still passed through the nozzle as needed. Instead of thoroughly investigating the problem, they worked around the problem.

When the normal procedure to empty the tank failed to work, ground crews improvised a procedure and used heaters and fans to empty the tank. (Please notice a similarity of living with and working around an Apollo program unknown problem is similar to the Space Shuttle program accepting the unknown problem of foam strikes.)

The improvised detanking procedures had never been used before, and the tank had not been qualified for the conditions experienced. (Notice a similarity between this and the Challenger launching in cold temperatures for which the vehicle had not been qualified.)

In reviewing the improvised procedures, officials at NASA, North American, Beach, and even the flight crew did not recognize the hazard of overheating.

Many of the managers were not even aware of the extended heater operation.

Neither qualification nor acceptance testing required switch cycling under load as should have been done. This was a missed opportunity.

The problem could have gone completely unnoticed and the Apollo 13 flight completed without the anomaly if the special detanking improvised procedure had not been done, because the switch remained cool and closed during flight and could take a momentary or short 65 V DC charge and probably not fail. Imagine if the Columbia foam strike had just been slightly smaller; perhaps the incident never would have occurred and would have gone unnoticed.

The thermostatic switches failures probably would have been captured if the heater current readings had been checked during the detanking operation.

The oxygen #2 tank had been dropped during installation at North American Aviation, which caused the fitting to become loose, but there was no investigation.

The tank heaters were equipped with 28-volt thermostatic switches supplied by the spacecraft fuel cells. But during the countdown rehearsal they were powered by 65-volt ground power supply. The 65-volt load likely caused the switches to fail. The ground crew kept the heaters on assuming the thermostatic switch would trigger if the tank temperature exceeded 80° Fahrenheit, but the heaters did not shut off and temperatures reached 1000° F. This heat burned the Teflon insulation off the fan motor wiring, leaving bare wires that would short circuit during the mission. The ground crew should have noticed this high temperature or burning smell. Apparently nobody was aware the temperature had reached such a high reading, or else they just did not report the anomaly. Maybe they were in a hurry to complete the task. After all, the heaters had been on for six hours! The electrical parts were damaged and the stage set for potential disaster.

The thermostatic switch 28-volt specification, dating to 1962, was revised in 1965 to carry the 65-volt Kennedy Space Center ground supply. However, Beach Aircraft Corporation, which manufactured the switches, did not make the needed change to the switches. This opportunity was missed by Beach, either intentionally or as an oversight, and also missed by North American and NASA in all of the design, documentation, and flight review systems.

The Apollo 13 problem was right in front of everyone, including the astronauts, just as the foam strike problems had been known and accepted since the very first space shuttle flight.