The immense volume of data generated by the suite of instruments on NASA’s Solar Dynamics Observatory (SDO) requires new tools for efficient identifying and accessing data that is most relevant to research investigations. Scientists at the Solar and Astrophysics Laboratory of the Lockheed Martin [NYSE: LMT] Advanced Technology Center (ATC) in Palo Alto have developed the Heliophysics Events Knowledgebase (HEK) to fill this need.

The HEK system combines automated data mining using feature detection methods and high-performance visualization systems for data markup. In addition, web services and clients are provided for searching the resulting metadata, reviewing results and efficiently accessing the data. The HEK is designed to catalog interesting solar events and features and to present them to members of the solar physics community in such a way that guides them to the most relevant data for their purposes. This is a problem of data markup that is arising in many scientific and other fields. For example, the Monterrey Bay Aquarium Research Institute developed a similar system for annotating and cataloging video sequences of ocean fauna and activities, and various sports leagues have systems for cataloging clips of athletic events. The distinguishing factor for SDO is the large image format and complex event types.

HEK consists of registries to store metadata pertaining to observational sequences (the Heliophysics Coverage Registry, or HCR), to heliophysical events (Heliophysics Events Registry, or HER), as well as browse products such as movies. Interfaces for communications and querying between the different registries are also provided as web services. The design and implementation of the HEK and its interfaces allow expansion beyond its primary function as a searchable database that contains metadata on solar events and on the observation sets from which these are derived. It can track temporary data products that users may wish to have online for a limited period. It is also possible to include numerical data sets based on assimilated observations, or even such data sets and events within them. In addition, it can also contain information on papers published based on certain data sets and events within them, so that for any given event users can be pointed to such publications, or likewise for a given publication users can be pointed to one or more data sets from one or more observatories.

Lockheed Martin developed the HEK system to address the immediate needs of SDO. However, the underlying motivation of devising a means for coping with enormous data sets resonates with many other missions and projects throughout modern science, and HEK could serve as a model. Two of the SDO instruments were built at the Solar and Astrophysics Laboratory of the Lockheed Martin Advanced Technology Center (ATC) in Palo Alto. The Atmospheric Imaging Assembly (AIA), a suite of four telescopes, provides an unprecedented view of the solar corona, taking images that span at least 1.3 solar diameters in multiple wavelengths nearly simultaneously, at a resolution of 0.6 arc-seconds and at a cadence of 10 seconds or better. The Helioseismic and Magnetic Imager (HMI), designed in collaboration with Professor Philip Scherrer, HMI Principal Investigator, and other scientists at Stanford University, studies the origin of solar variability and attempts to characterize and understand the Sun’s interior and magnetic activity. The third SDO instrument, the Extreme Ultraviolet Variability Experiment (EVE), measures fluctuations in the Sun’s ultraviolet output. EVE was built by the Laboratory for Atmospheric and Space Physics at the University of Colorado.