This last blog post on EDF2015 focuses exclusively on the impacts of big data on the labour market. Day two of the conference offered a session on “Educating Data Scientists & Data Skills” which provided some valuable insight into the impact that big data is having on the labour market and whether the observed skills gap in Europe in terms of data technologies can be overcome.
We learnt that data work required multiple skills, from technical to business market, meaning that they are rarely in the same person and that teamwork was therefore very important in a company to design and implement technologies. Gabriella Cattaneo, from IDC’s European Government Consulting unit explained that there were different data worker profiles. “Data workers” were defined as people who collect, store, manage and/or analyze, interpret and visualize data as their primary or as a relevant part of their activity”.
According to her estimates, there were about 6.1 million data workers in the EU, which represents 3% of total EU employment, most being high profile professionals, managers in the field of data innovation. She then revealed that there was a skill gap of 7.5 percent of total demand, in other words vacancies in 2014, between the supply of data workers in Europe and the demand for data workers, and that that represents a threat to the European data industry. It became apparent that the most relevant gap concerned data scientist skills. “Data scientists” were defined as people with special technical, mathematical tools specialised for designing tools and applications for data and there are about 100 000 -150 000 in Europe of these highly specialised scientists.
The next speaker, Frank Bensberg, from Osnabrück University of Applied Sciences, continued the session by focusing on the impact of big data on the job market and presented a report by IDC and Open Concept that looked into which skills and skill profiles are searched for in the labour market for the domain of big data. To do this, they carried out an analysis of job advertisements from publicly available job websites and from special companies including Microsoft (in combination with text mining methods) and then looked at companies active in the big data sector.
Bensberg also reported a skills gap which had the effect of reducing the capacity to innovate and to develop new markets as well as the fact that the evolution of new information technologies and would lead to erosion of knowledge. Programming expertise became apparent as a skill during their research and they found that new skills obliterate as-is skills and redefine competency requirements. This led to the conclusion that in order to avoid IT skill gaps, there was a need to address educational institutions such as university to see if education being offered fit the job market. The important question which Bensberg put the audience was: which skills and skill profiles are searched for in the labour market for the domain of big data?
In the framework of this report, they looked at about 4 million job ads and selected about 80 000 which concerned data, data science and big data positions. A major finding was that 70 percent of the job titles represented five job families: big data developer or software engineer (28%), data scientist (25%), big data architecture, in other words large infrastructure (20%), big data analyst (11%), big data analyst (9%), data engineer (7%).
Bensberg then went into the job profile of the data scientist, data analyst and data visualisation engineer in more detail, listing their first competencies (see slides):
- concepts for data scientists
- programming languages (python, java, SQL, R, MATLAB, etc.)
- software programmes and products (Hadoop, SAS, Hive, SPSS, Excel, etc.)
- soft skills (communication skills, working experience, team skills, etc.; these were not striking for these profiles)
He concluded that big data contributes to the evolution of new job profiles in the labour market, that educational programmes and ICT curricula should consider this evolving skill demand and that established frameworks should be adapted to support HR management.
You can access the programme, presentations and photographs from EDF2015 as well as see their press coverage on the conference website.
Our previous blog posts on EDF2015: