Database Overview

DataPaths is an in-memory database specifically crafted for analytical applications. Developed in low-level C/C++ language, DataPaths aims to utilize minimal system resources while offering maximum data storage capacity and superior data retrieval performance. Our database is engineered to compress low-entropy columns within a table to reduce memory consumption, all while indexing these columns to enhance performance. This innovative data storage approach significantly reduces overall data storage requirements.

While most databases store relational data in either row or columnar format, our database only stores high entropy data in data-nodes while the rest of the data is compressed into very lightweight structures which lend themselves naturally to sorting and indexing operations. Consequently, select queries on these structures can be optimized for very efficient data retrieval.

DataPaths has been developed primarily for analytical application. A variety of options are available for integrating DataPaths into reporting/analytical applications and software projects. A subset of SQL language has been implemented for interacting with the database.

Data Structure

Efficient data storage impementation in DataPaths translates into lower hardware requirements compared to other databases. In most cases the resulting data structures takes siginificantly less memory than corresponging flat file. This approach not only allows siginificantly more data to be loaded than any other in memory database but also dramatically improves data retrieval capabilities - data retrieved along data paths can be near constant time complexity.

Due to the unique nature of our data storage schema it is also possible to enrich existing tables with additional low-information columns while using little additional memory. This creates opportunities to create denormalized tables thus eliminating any need for complex join operations.

DataPaths Applications

DataPaths had been designed to provide optimized in-memory data storage primarily for analytical applications. By integrating database functionality directly into application software, analytical data can be retrieved directly from computer memory for superior performance. Currently DataPaths can be integrated in C++ projects as a stanalone library or into Python projects as a Python module.

Multiple applications in Risk Management and Business Analytics were developed based on DataPaths functionality. Additionally, we can build custom analytical reports and dashboards based on existing reporting components.