We’re really excited about our upcoming Webinar featuring one of Apache Flink’s biggest committers, Kostas Tzoumas, CEO of dataArtisans. Apache Flink is an open source framework for distributed stream and batch data processing that provides an execution environment for transforming unbounded data sets where the time dimension is critical. That makes it ideal for working in areas like transport and Internet of Things where data comes as streams from sensors and where the value of that data is highly time-dependent.
During this (free) Webinar you’ll discover:
More about Apache Flink and the problems is it designed to solve.
Some of the applications that re using Flink.
How you can include Apache Flink in your comprehensive workflow using the Big Data Europe platform.
How it’s being used in the Big Data Europe pilot around transport.
We are pleased to announce the first public release of the open source BigDataEurope Platform. This platform is designed to help communities to solve societal challenges and problems by accelerating the process of getting started with the big data technology-zoo.
The BDE platform released today, is an easy-to-deploy, easy-to-use and adaptable (cluster-based and standalone) platform for running big data frameworks and tools. We have selected these latter based upon the requirements gathered from the seven different societal challenges. Thus, the platform allows to execute a variety of big dataflow tasks, starting from storage (Hive, Cassandra), message passing (Kafka, Flume), multi-purpose data processing and analysis (Apache Hadoop, Apache Spark, Apache Flink), to publishing (Geotriples).
The user can follow a simple set of instructions to install, or follow the installation video for getting started with the platform.
Users can then pull ready-to-use images from the BDE repository or create a dataflow pipeline to realize a full data value chain using the components available at the BDE repository. This platform will be refined and expanded collaboratively with the users and the societal challenge (use case) partners. This public release of our platform represents the beginning of the operational phase, with which the use cases will be materialized and tested on the developed platform.
In the future, the BDE platform will introduce the concept of smart data to support a range of processing tools for the semantic web and knowledge graphs. This includes the concept of semantic data lake, a semantic analytics stack engine and a semantic platform for logging and integration for cluster resilience.
Overall, the platform has lowered the barrier to entry for new big data users and scientists from different domains to experiment with a variety of big data tools in a plug-and-play fashion.
Crowdsourcing is one of the best things that happened to humanity in the past years. Because two heads are better than one, hearing more voices triggers new perspectives, and taking ownership is a good thing. It’s no wonder Airbnb, Waze, Wikipedia or Kickstarter are such a hit that even countries are tapping in with crowdsourced policymaking. Empowering communities works in the most diverse industries, locations, and business fields.
Transport in particular has a lot of potential in the area. It affects a lot of people, on a regular basis, and has the power to trigger sense of ownership among users. Using the power of the crowd to gather and disseminate data to achieve hassle-free travel is becoming more and more widespread in cities especially.