Status of LOFAR Data in HDF5 Format
Résumé
The Hierarchical Data Format, version 5 (HDF5) is a data model, library, and file format for storing and managing data. It is designed for flexible and efficient I/O and for high volume, complex data. The Low Frequency Array (LOFAR) project is solving the challenge of data size and complexity using HDF5. Most of LOFAR's standard data products will be stored using HDF5; the beam-formed time-series data and transient buffer board data have already transitioned from project-specific binary format to HDF5. We report on our effort to pave the way towards new astronomical data encapsulation using HDF5, which can be used by future ground and space projects. The LOFAR project has formed a collaboration with NRAO, the Virtual Astronomical Observatory (VAO) and the HDF Group to obtain funding for a full-time staff member to work on documenting and developing standards for astronomical data written in HDF5. We hope our effort will enhance HDF5 visibility and usage within the community, specifically for LSST, the SKA pathfinders (ASKAP, MeerKAT, MWA, LWA), and other major new radio telescopes such as EVLA, ALMA, and eMERLIN.