The National Centre for Scientific Research “Demokritos” (Climate domain) Consortium organised a webinar (online hangout) and presented the first SC5 Pilot Use Case on Tuesday, July 12, 2016 from 11:00 to 12:00 CET.
The goal of the Pilot is to serve as a test bed for the evaluation of the BDE Aggregator Platform according to the requirements previously expressed by the societal domain (SC5). The pilot aims to demonstrate the usefulness and flexibility of Big Data technologies in managing and working with climate data.
The hangout was initiated by Diamando Vlachogiannis who presented the agenda and the objectives of the hangout. The hangout consisted of two parts: i) an introduction and general description of the 1st Climate pilot use case and ii) a step-by-step technical description including a live, hands-on demonstration giving the opportunity to the hangout participants to test some functions of the pilot. Apart from demonstrating the functions of the Pilot through a hands-on testing, the hangout aimed also to obtain feedback from the participants through the completion of a short questionnaire for the evaluation of the functions of the pilot and the provision of ideas about the possible expansions of the pilot.
Then, Spyros Andronopoulos gave a general description of the 1st Pilot. He emphasised that the basic aim of the Pilot was to demonstrate what can be achieved using the tools of the BigDataEurope platform in managing large volumes of climatic and / or weather numerical data. The pilot application is concerned with the process of dynamical downscaling. Dynamical downscaling of climatic and / or meteorological data is the process where output from a “large-scale” model (such as a GCM) is used to drive a regional or local model in higher spatial and temporal resolution, which is able to simulate local conditions in greater detail. It is an essential first step for any further analysis, assessment or processing in climate and related domains, such as environmental impact assessment, air pollution etc. The downscaling procedure can consist of several successive steps with nested (geographical) computational domains. The current features of the 1st Climate pilot were shown to be the following:
- Data Ingestion from NetCDF files
- Data export to NetCDF files
- It can start and monitor WRF-based downscaling on institutional (i.e., non-BDE platform) resources, depending on whether the requested results already exist in BDE data base
- It maintains provenance record of results on the BDE platform, for monitoring and further analysis
- It supports basic analytics on the BDE platform (Hive queries)
- It has a console-based user interface, targeted on climate / atmospheric scientists( Python/Jupyter interface for demonstration)
Following that, Iraklis Klampanos presented the Pilot architecture. It was shown that the added value from this Pilot lies in the following aspects:
- Increased speed and ease in managing large data sets (with the functions of ingestion, export and analytics)
- Efficient use of institutional resources in performing the computations for dynamical downscaling
- Data lineage, to trace how certain results have been obtained, facilitating their analysis) or how they can be reproduced
Planned activities for further Pilot development include the realization of a transparent transformation of climate / weather NetCDF files tailored to WRF, using the BDE integrator and further testing and development regarding data lineage and downscaling parameterization and execution.
The online hangout was concluded with a hands-on demonstration of the pilot. The system configuration allowed the participation of 6 users. The users were guided to apply the steps and run the pilot. Finally, they were asked to complete the pilot evaluation form for feedback information.