1981 | OriginalPaper | Chapter
A Self-Describing Data File Structure for Large Data Sets
Author : Robert A. Burnett
Published in: Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface
Publisher: Springer US
Included in: Professional Book Archive
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
A major goal of the Analysis of Large Data Sets (ALDS) research project at Pacific Northwest Laboratory (PNL) is to provide efficient data organization, storage, and access capabilities for statistical applications involving large amounts of data. As part of the effort to achieve this goal, a self-describing binary (SDB) data file structure has been designed and implemented together with a set of data manipulation functions and supporting SDB data access routines. Logical and physical data descriptors are stored in SDB files preceding the data values. SDB files thus provide a common data representation for interfacing diverse software components. This paper describes the data descriptors and data structures permitted by the file design. Data buffering, file segmentation and a segment overflow handler are also discussed.