The findings of the HIFI investigation team and the proposed plan for bringing HIFI back in operation were presented to the ESA Director of Science and Robotic Exploration and others in a briefing meeting held in ESTEC on 25 November 2009. The plan presented was endorsed, and is now the adopted way forward.
[G. Pilbratt, Herschel Project Scientist, posted 27 November 2009]
19 November 2009 (L+189, DOY 323)
The plan for bringing HIFI back in operation is now on the table, and subject to formal approval in the coming week. The plan consists of performing a number of software updates in the coming weeks, followed by a full instrument switch-on in January 2010, after which HIFI operations will resume. See information provided on the SRON website.
[G. Pilbratt, Herschel Project Scientist, posted 20 November 2009]
26 October 2009 (L+165, DOY 299)
The HIFI failure investigation team set-up at ESA in support of the Principal Investigator effort in SRON is finalising its work and a draft report is currently under review within the team. The content of this report will be presented to the ESA Director General and the Director of Science and Robotic Exploration this week. Due to the capability to perform extended tests on fully representative hardware and software on the ground, and to the excellent cooperation between the PI team and ESA specialists, the investigation has arrived at a complete and consistent failure scenario which can explain the observable evidence. Some further consolidating investigations on this scenario are still on-going but the picture that has emerged is the following:
A Single Event Upset (SEU) in the Random Access Memory (RAM) of the Local Oscillator Control Unit (LCU) microcontroller would be at the origin of a chain reaction that eventually results in an unplanned emergency switch off of sensitive instrument components. Designed to protect the local oscillators against damage from undervoltage on the spacecraft 28 V bus, this switch was thrown while the 28 V bus was alive. The resulting load transient in the internal power system of the LCU associated with this mode change created a stress in the power converters, leading to a permanent failure of one of the diodes.
Before the restart of the instrument on the redundant signal chain can be performed, changes in the operation and protection logic of the instrument must be implemented and validated in on-board software to prevent re-occurrence of such a sequence of events.
Although the detailed investigations have uncovered some marginalities in the stress applied to certain diodes in the internal power system of the LCU, laboratory tests have shown these diodes to be quite resilient against the short overvoltage spikes they are subjected to during nominal instrument operations. The investigation team is confident that HIFI can perform nominally for the remainder of the mission if the required corrective actions are implemented.
[J. Riedinger, Herschel Mission Manager and co-chair of the ESA investigation team]
5 October 2009 (L+144, DOY 278)
Progress towards re-enabling HIFI
Today, SRON have added an entry to their top level web page (www.sron.nl), in which they report about the progress that is being made towards re-enabling HIFI operations. Despite the progress which is undoubtedly made by the joint HIFI/ESA investigation team, the HIFI Project Manager cautions that switching the instrument back on may still be a few weeks in the future: We must ensure that we have taken all conceivable measures to minimise the risk of such a chain of events from happening again. One of these measures may require changes in on-board software, the validation of which would have to be performed with the utmost care.
[J. Riedinger, Herschel Mission Manager]
29 September 2009 (L+138, DOY 272)
First science data distributed to the users community
Yesterday 28 September Herschel reached another important milestone: the first set of observations corresponding to the so-called 'Science Demonstration Phase" were made available to their owners. This delivery took place several weeks ahead of the originally planned schedule and marks the start of the transition from the Performance Verification Phase to the Routine Phase of operations.
The observations (six so far only) were conducted on 12 September 2009 and successfully processed with the latest version of the pipeline as implemented in a dedicated version of HIPE that was also distributed to the observers. They are SPIRE scan maps of a variety of astronomical sources including a galactic HII region, a proto-planetary nebula, a supernova remnant, and a couple of galaxy clusters. This is the first of the observing modes released to the users community following its early validation during the Performance Verification Phase.
If there are no major contingencies, we expect to be running the Science Demonstration Phase at full speed during the second half of October and the whole month of November, as initially planned, first with PACS and SPIRE AORs, and then with HIFI as well, once the instrument resumes operations and completes its own performance and verification phase, delayed because of the problems already reported in this (B)log.
[P. Garcia-Lario, HSC ESAC, posted 30 September 2009]