South Pole Weekly Report, February 01, 2009

The current IceCube population stands at 35. The Seasonal Equipment Site (Drill Camp) was relocated to its 2009/2010 season location. All systems were drained/flushed, dismantled, and winterized. Equipment repair and inventory has been completed. Drillers leave the station early next week.

A train of 5 MDS containers is pulled by a D6 to next season's SES pad.

Photo by T. Hutchings

The rest of the camp was winterized after the weekend break and the whole SES moved to next year's position on Wednesday, January 28. The move went very smoothly and was completed in one morning. The cargo was then packed between the buildings and the whole process was completed in one day. The two TOSs and the DCC have been moved to their winter positions and powered up. The hose reel has also been relocated to its winter position and was covered with the electric blankets. The winterover crew has been given a tour of the buildings and the reel and shown what to keep an eye on to make sure they remain warm. The rest of the firn holes were completed this week, making a total of 3 of next year's holes plus the RodWell. The RodWell was drilled down as deep as we could get it (52m) in order to test how well the firn drill can drill in a fluid filled hole. The only activity left at the camp is in the IceTop tent. All the drillers, but one, will be leaving Pole this coming Wednesday.

Deployment equipment has been winterized and stored within the SES and on the winter berms. A detailed inventory has been completed. The Robertson winch had a damaged section of cable removed; the connector will be reinstalled next season. The logger probe was moved to the ICL for winter storage.

This week recorded zero reportable injuries and no new incident reports. This marks more than 92 days on the ice without a reportable injury.

Closed and groomed IceTop Station.

Photo by S. Amandusson

It has been a busy week for IceTop. In addition to keeping the snow cleared from the open tanks, we closed the tanks in 13 stations, and removed the freeze controllers from 8 tanks. There are another 8 tanks that have frozen and are ready to have the freeze controllers removed. Refurbishing and packing the shades and frames is progressing in the WeatherPort, which has been moved to the new drill site, outfitted with a new plywood floor, and powered by a portable generator.

DOM testing is complete for the season. The Optical Module Lab (OML) has been winterized and shut down.

The upgraded HSM is in production and performing well, but not perfectly. We identified one bug in the software, fixed it with a newer release from the manufacturer, and immediately identified a different problem with that release which we're working around.

Iridium OpenPort mounted on the roof of the ICL, separated from the well-populated red antenna box to prevent mutual interference.

Photo by J. Richards

Three of four Promise disk storage enclosures have upgraded firmware and more stable redundant controller configurations, and the fourth is scheduled for upgrade on Tuesday. The majority of the summer tapes shipped to Madison in two large boxes (the remainder of summer tapes will be hand carried), and a few hours later the Iridium Openport array, two spare servers, and a lot of cables arrived. Many thanks to Tilo and Paul who carried hundreds of pounds of delicate equipment up and down stairs, despite the thin Antarctic air. The Iridium OpenPort array has been mounted and tested. It performs significantly better here than in Wisconsin—better exposure to uninterrupted sky and almost no competition for satellite time. Network configuration and connectivity are the next challenges.

Last winter the antenna box had a hard time maintaining its temperature, so we replaced the inadequate heaters with two newly refurbished ones from the drill camp. Remote monitoring shows the interior to be a steady 55 degrees F, with one of the heaters running and the other available in case the first one fails. Later in the day the ICL server room could not maintain its temperature either (see chart below).

At 9:00am Friday the temporary insulation around the string cable entry into the server room was replaced with much nicer custom fitted insulation, and shortly thereafter temperatures in the server room climbed ten degrees. A significant amount of cold air must have been leaking in and we were using it to keep the temperature down.

We worked to rebalance the air handling in the server room and with some redirecting of air flow we cooled down the hottest spots. Also, by swapping a resistor in the control panel we reduced the overall air temperature about five degrees, enough that systems no longer produced high temperature alarms. Ideally the system would run cooler than this, but our attempts to go lower were foiled by an automatic override intended to keep the system from freezing. Station support already started an upgrade to the monitoring and control for the air handling in the ICL, and this latest temperature problem adds immediacy to the project. In the next two weeks we'll work on how much can be done before station close.

Two new strings (5, 11) were handed off by Low Level Commissioning to pDAQ on January 28 and were run in a successful 4hr IC49 run controlled by IceCube Live (the run had to be redone to a failed DOM power supply on String 74). The strings passed through Verification flasher runs on the same day. Four more new strings may be delivered for pDAQ commissioning as soon as the next 24 hrs. A fourth release candidate for Betelnut was run for most of the tests run this week. The release candidate seems stable. Analysis of Betelnut data continues. There was some suggestion that Betelnut may drop DOMs at a higher rate, though this remains to be confirmed and we have limited statistics. It is expected that Betelnut will be run for the next full week to facilitate upcoming commissioning runs. Some thought and energy is going into the dropped DOM issue again. It would be nice to address this long-standing problem at the lowest level (likely DOM and/or DOR firmware). Efforts are underway to create a test case which would allow us to capture information on a scope attached to an individual wire pair. The problem is somewhat resistant to systematic repetition or even characterization, as it is very intermittent and seems to happen at various points in the startup sequence. The most likely path to success may be in the North at LBNL, where an effort is being made to prepare a test stand to use pDAQ to try and trigger and study the problem.

A fuels technician signals an arriving LC-130 carrying the last IceCubers to arrive this season.

Photo by J. Jacobsen

Bug fixes and improvements to IceCube Live are ongoing. Operator documentation is being written so that winter-overs can receive training during the last week of the summer. Their feedback, both at this time and throughout the winter, is important in order to continue to improve usability and reliability of the system.

Six more strings were commissioned this week : Strings 02, 03, 04, 05, 10 and 11; 05 and 11 on Tuesday, January 27. No major issues were encountered with these 2 strings; however, we decided to split Dom 05-07 "Snare-Drum" and its pair companion for further studies. This DOM seems to have a higher and fluctuating rate, which might account for noisy Local Coincidence. We unplugged 1 quad on each of strings: 02, 03, and 04, because each had one DOM showing a high rate and/or high temperature which might mean a DOM is not frozen yet. Commissioning has gone smoothly and will be finished this Sunday evening, and no issues have been encountered so far. We are now waiting for the remaining 6 strings to freeze.