A MULTISTORE ENVIRONMENT IMPLEMENTATION
Project no. 1329 · Status: Kapanmış · 2018
Summary
Big data is a difficult phenomenon to manage by having velocity, variety, volume and veracity dimensions. In order to manage big data, it is necessary to take into account all these dimensions in a certain manner. In the solution to be presented, large-scale data should be persisted in the store(s) suitable for its structure, it should be queried quickly, and it should also be obtained from accurate and reliable sources. At this point, multistore / polystore systems emerge as a solution to manage the big data. The system basically consists of a multistore environment and a query engine that queries this environment as simple or federated. From this point of view, it is aimed to create a multistore environment within the scope of the project in order to provide an exemplary big data management. First of all, the medical area has been determined for the big data to be managed, and MIMIC-III has been chosen as the dataset which contains large amount of data in different structures. Thus, after proving the correctness and structure of the data, its the models have been analyzed and determined according to the field, size and structure. Storage technologies suitable for these data models have been selected and MIMIC-III has been got ready for importing into stores. All studies that reference MIMIC-III have been scanned and critical use case scenarios and the sub-dataset contained in these scenarios have been determined. This sub-dataset has been uploaded to the specified stores and the the multi-store environment has been set up. In order to demonstrate the effectiveness of this established environment, the first version of a query engine was implemented on the basis of the actor model by taking into account the query tracking capability and scalability. The system has been evaluated, its results have been measured, and has been turned into a scientific journal paper study by presenting its effectiveness.