My last stop on ATom-3 was Punta Arenas, Chile. The flight around the Southern Ocean from New Zealand is long, and produces the worst on the whole mission for jet-lag. We left New Zealand early in the morning, and arrived after about 11 hours flying in Chile, a few hours earlier the same morning (things get weird across the international date line!). Despite the dark, you always know you’re in Punta Arenas when you step off that plane as the wind hits you and threatens to blow you off the steps. Because there are no continents to get in the way around the Southern Ocean the wind picks up tremendous speed and it circles around the globe unencumbered, making this one of the windiest places on earth.
It was close to 4am when we arrived at the hotel, so I got a few hours sleep. When my alarm rang at 9am tI felt the full-on grogginess of jet-lag, but food and a brisk walk along the beach looking over the Magellan Straights helped keep me awake. Agnieszka (Aga), my team-mate, met me here as she is now taking over for the second half of the mission. This starts with an out-and-back flight over Antartica, which I was slightly jealous about missing.
The morning of the antarctic flight, I went with Aga to prepare the instruments in pre-flight, just to introduce her to any changes in the system since we’d parted back in California and help with settling in. After doors closed I went back to our operations room inside the terminal and loaded up our online chat where we can communicate between the plane and the ground.
Almost immediately after take-off I received an urgent message from Aga that there was a major problem with our whole flow system. The flow we use to make sure that the air we pull into the aircraft to analyse is flowing at the same speed as the air flowing past the aircraft (this reduces losses of particles in the air as it enters the tubing) was uncontrolled – the valve had just shot to fully open! This meant we had undefined losses, as did 3 other instruments sampling off our inlet – a major problem!
Aga diagnosed that it was in-fact the flow meter that tells the valve how much to open and close that was broken. We were not carrying a spare (not that it could be replaced during flight anyway) and nothing she could try in flight was fixing it. We realized the only way to start getting usable data was to override the software and manually set the valve position. While Aga continued managing things on the plane, I needed to figure out the right settings from the ground for each pressure level.
Of course, at this moment, the last shuttle was leaving to take us back to the hotel. My colleague Karl (who’s instrument was also affected by this flow problem) and I leapt in, but he kept chatting with Aga from his cell phone, whilst I sat with my laptop on my knees over the bumpy Patagonian roads, trawling passed data to get sensible valve settings for the plane altitudes for Karl to relay to Aga, who then set the values.
At this point, it is useful to remember that the ATom flights scan constantly up and down to get the vertical structure of the atmosphere. This means Aga didn’t need to set just one value, but to constantly adjust the values as the plane changed altitude! And we needed to be ahead of her, guessing what altitude would be most useful. It took a while to get the rhythm of how fast the plane was ascending vs. how fast we could get the values, relay them to Aga and she could set them.
As we drew up to the hotel, Karl and I leapt out and continued our patten of calculating and sending numbers up to Aga until the plane reached a level leg and the three of us could catch our breath and reorder our thoughts. We re-established our base of operations in the hotel breakfast room, fortified with much needed coffee and croissants.
Now we had a moment to think, we could devise a more systematic approach. If I could produce a fit of altitude to valve position from data from past flights, I could then produce a look-up table for Aga to read-off during the rest of the flight and adjust to. I could also figure out how frequently she would need to change the valve position to stay within our tolerances on how accurate that flow needed to be.
As I started amalgamating data from the last few flights and plotting it out a terrible realization came over me. The flow meter had failed on a previous flight, not this morning! I had simply missed the signs in the data that something was amiss, as I’d been too focused on those optical problems from the last few flights. I’d simply not noticed the problem Aga had seen straight after take-off! I felt hollow in the pit of my stomach, knowing that our data and that of 3 other instruments would be adversely affect by my oversight. The instinct to berate myself for mistake and wallow in the problems it might cause was strong, but there was limited time before the plane started down again, and Aga needed new numbers. Self-pity and chastisement would have to wait, there was serious maths to be done, and fast!
I figured out a fit and our tolerances, and produced a look up table of altitude to valve position, which I relayed line by line up to Aga on the plane. She transcribed this into the log book, and, for the remainder of the 11 hour flight, manually set the valve positions every few minutes on ascent and descent, getting breaks to do the rest of the monitoring and management of the instruments, eat, and go to the rest-room only when the plane leveled out on high or low legs.
Back at the hotel, I contacted a colleague back in Colorado who could help us quickly write a computer program to manage this in future flights, and looked into how we might get a replacement flow meter delivered to somewhere on the flight route. With the shipping and ordering time, plus the problem that a lot of the stops on this leg of the mission are incredibly remote and hard to get mail to more frequently than once per week (if that), the only stop we could get it to was Anchorage, Alaska, our penultimate stop! We decided this was worth it, since we could operate the new manual programming of the value and the new flow meter together, and get a measure of how accurate my fit was and what corrections would need to be applied.
To ease my guilt and try and make-up for my error, I figured out the corrections needed for the data taken when the flow meter was broken and I hadn’t noticed, and the cost to our precision and accuracy this presented. I was somewhat relieved to see the loss wasn’t as big as my horrified mind had been imagining in shock over breakfast, but couldn’t lift the disappointment I felt in myself for missing the problem.
Over the next few days, discussing with Aga, my advisor Chuck back in Colorado, and some other trusted colleagues on the mission, I came to the conclusion that while part of Aga spotting that problem where I had missed could be excused as coming to the equipment with fresh eyes, a big part of it was also a difference in our approaches as instrument operators. I have a tendency to laser-focus in on problem areas. While this was helpful for spotting and diagnosing the optics problems that had affected us back in Fiji and New Zealand, and managing other parts of the system that can be problematic at times, it meant that I had less of a systematic approach to monitoring the system as a whole. Aga, on the other hand, is very systematic in her approach to everything, including the instruments in flight, meaning that she has a better picture in her mind at any one moment in the flight of how the system is performing as a whole.
Chuck, as a ex-aircraft pilot himself, suggested a helpful concept known as ‘the scan’, which pilots use to regularly go over the entire dashboard in the cockpit to check that everything is ok. They have a specific pattern in which to go over each meter and dial, and a fixed frequency of doing it, which means, whatever else comes up, they always maintain the same base-level cognizance of the health of the system as a whole. I have since developed such a scan for our instrument system, a routine to instill this systematic approach into my flights, and ensure that whatever minor crisis I’m dealing with, or optimization I’m focused on, I will never lose sight of the health of the system as a whole again.
