Free, Open Source, Comprehensive Big Data Toolkit Launched!


PDF version

European project makes big data technologies easy to use.

After more than 2 years’ development, the Big Data Europe project is proud to announce the release of its free, open source Big Data Integrator Platform. Through a simple user interface, most of the widely used big data tools can be selected and installed as an entirely customised and extremely powerful computing platform able to process arbitrarily large volumes of data across any number of physical and virtual machines. This is achieved using the popular Docker system that wraps each software component inside a container and automatically handles the necessary dependencies that it needs to operate.

With BDE we have a platform that is intrinsically extensible… we wanted to make it easy to integrate your own stuff and easy to get started with new stuff.

Aad Versteden, Innovation Manager at TenForce

There are more than 15 different tools available, including Flink and Spark for data processing at scale, Kafka for message handling, HDFS for storage, triple stores like 4Store and Strabon and many data stores like Hive, MongoDB or Cassandra. Installing each of these natively can take a lot of time and effort but the Big Data Europe Integrator Platform (BDI) takes care of everything to get you up and running quickly and easily, so that you can directly concentrate on processing and analysis tasks without being overwhelmed by tedious and error prone cluster installations.

And we know it works.

The project includes 7 distinct pilots, each of which has used the BDI with a different configuration to carry out a different task, each related to one of the European Commission’s 7 Societal Challenges.

We have a lot of heterogeneous data: phenotypic data, genetic data, analytic chemistry. In Big Data Europe we were able to combine and make sense out of all this data.

Maritina Stavrikaki, Viticulturalist and Oenologist at Agroknow/Agricultural University of Athens

Semantic Big Data

The 3 Vs of big data are well established: volume, velocity and variety. In all 7 of the pilots, it was the variety that posed the biggest problem, so one of the outstanding aspects of Big Data Europe is that it tackles this head on with the development of the ‘Semantic Data Lake.’ Data is stored in whatever format it arrives in but can be queried as if it were stored as an RDF graph. SPARQL queries are rewritten as necessary at run time so that data is extracted from wherever it is in the data lake and then the results combined into a single result.

Users have a unified view of all the data in the platform, even though the actual data is heterogeneous, making it much easier to work with.

Mohamed Nadjib Mami, researcher, Fraunhofer IAIS


Project Coordinator, Prof. Dr. Sören Auer of Fraunhofer IAIS, explained that the release of the Big Data Europe Integrator Platform today marks a significant milestone, making big data technologies much more readily available in ways that suit the user, not the vendor. Work on the platform will continue, in particular the development of the Semantic Data Lake and of the analytics engine, SANSA, that uses machine learning to provide real insights into the available data, but already, through the pilots, we know the platform works and is ready for others to use.

Creating the virtual RDF graph from any data source and being able to use machine learning to infer information from it are real innovations within BDE that we’re most proud of.

Dr Hajira Jabeen, Senior Researcher, University of Bonn

Live Webinar Today

Join the webinar at Wed, May 3, 2017 3:00 PM – 4:30 PM CEST by visiting and providing the Webinar ID 891-511-459 and your email address.

Try It

All code and documentation is freely available on GitHub

About Big Data Europe

Big Data Europe is a project within the EU’s “Horizon 2020” framework programme that aims to build a knowledge- and innovation-based society so as to strengthen the competitiveness of Europe’s economy. Together with Fraunhofer IAIS, the following institutions and companies also participate in the project: AgroKnow (GR), CESSDA (NO), National Center for Scientific Research DEMOKRITOS (GR), ERCIM (FR), ERTICO – ITS Europe (BE), European Union Satellite Centre (ES), Food and Agriculture Organization of the United Nations FAO (IT), Institute for Applied Informatics (InfAI) e.V. (DE), Kentro Ananeosimon Pigon Ke Exikonomisis Energeias – Center for Renewable Energy Sources (GR), National and Kapodistrian University of Athens (GR), CERTH – Centre for Research and Technology Hellas (GR), The Open PHACTS Foundation (GB), Tenforce (BE), Semantic Web Company (AT), VU University (NL) and W3C (FR).

For more information, please visit:
The platform can be downloaded from GitHub
See also the installation instructions and video.

Press contacts:

Alexandra Garatzogianni
Fraunhofer IAIS
Phone +49 2241 14-2974
Phil Archer
Phone +44 7887 767755