Abstract
For faster processing and higher efficiency, columnar data storage is preferred for many applications. However, data stored in columnar format may be required in row-oriented format in certain use cases. For large analytical databases, transforming data from column to row format and storing the transformed data can be expensive. This disclosure describes techniques to efficiently access columnar data in a database without transforming the data into row-oriented format. An iterator facade layer with multiple pointers to columnar data is used to present data to applications that require data in a row-oriented format. Columnar data is retrieved in batches that are read into memory. When reading data, a first pointer identifies the batch, a second pointer points to the current row, and a third pointer points to the current column. When a column name is queried, table schema metadata is used to translate the column name into a column index. With the column index known, the specific cell can be looked up inside the columnar data block to fetch and return the data as a response to the query.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Mishra, Prashant; Jain, Prateek; and Shevek ., "Iterator Facade Layer with Multiple Pointers to Facilitate Access to Columnar Data", Technical Disclosure Commons, (March 08, 2024)
https://www.tdcommons.org/dpubs_series/6773