The Ophidia framework and EUBrazil Cloud Connect: towards cloud-based big data analytics for climate change

Posted on

In many domains such as life sciences, climate, and astrophysics, scientific data is often “multi-dimensional” and require tools that support specialized data types (like arrays) as well as data analytics and mining functionalities to properly store, access, analyze and visualize data.
Current On-Line Analytical Processing (OLAP) systems provide datacube oriented support but unfortunately they fail when large scale (multi-terabyte order) data management is needed. Different storage models and data management strategies are needed to fully address scalability, as well as parallel/distributed solutions to meet the demand of complex data analytics processes.
The Ophidia project, a research effort on big data analytics, is facing these scientific data analysis challenges in the climate change domain. It provides the Parallel Data Analysis Service (PDAS), an internal storage model, a parallel I/O sub-system and a hierarchical data organization to manage large amount of n-dimensional scientific data.
The entire PDAS software stack has been deployed at CMCC.
One of the most relevant data analytics use cases to test all the features provided by Ophidia targets the interactions between climate change and biodiversity within the framework of the EUBrazil Cloud Connect project.
The Ophidia framework is being extended in order to integrate dynamic and scalable VM-based solutions for the management of large volumes of scientific data (both climate and satellite data) in a cloud-based environment to study how climate change affects biodiversity.

References
RP0194 – A big data analytics framework for scientific data management
RP0193 – Support for “bit” data type in the Ophidia framework
RP0192 – Data analytics operators in the Ophidia framework
RP0191 – Metadata management operators in the Ophidia framework
RP0151 – The Implementation of the Ophidia Desktop GUI
RP0150 – The design of the Ophidia Desktop GUI
RP0149 – Data flow in the Ophidia system

The official EUBrazil Cloud Connect project website.

Start typing and press Enter to search

Shopping Cart