How to address the talent shortage in big data

The current search for people with deep analytical skills is just the tip of the iceberg – more valuable is those who can make business sense of the data.

One of the key consequences of our increasingly interconnected and technology-driven world is that we are creating new information 50 times faster than we did ten years ago. Some 2.5 quintillion bytes of data (or 2.5 exabytes) are created every day and the volume is growing.

Whilst the computing technologies needed to support these data are keeping pace, the human skills and talents business leaders required to leverage big data are lagging behind.

Little wonder therefore that where data mining, advanced analytics and other specialist skills are required, companies are facing increasingly tough competition for the people they need. For once, the sector is less relevant than the skill base of the individual.

Few will doubt that big data is becoming the driver of innovation, competition and productivity. But is big data just digital-era hype or is there more to it? Is the problem with big data that it is just too big? After all, businesses have had access to meaningful market research data for more than 80 years.

In the early days of market research, customer intelligence was derived largely from focus groups and surveys. The information collected was then used by marketers to fine-tune advertising and promotion programmes to reach their target audience effectively. The data gathered would also be used to ensure the products or services they produced matched the requirements of the target market.

Today, that has all changed. Big data has become a holistic research tool for the entire enterprise, not just marketing and manufacturing. The data come from many sources, including conventional surveys, focus groups, sensors, cookies and GPS, and they are often referred to as internal and external data or structured and unstructured data.

It may be seen as hype but that hype is certainly justified. Big data and predictive analytics are seen as critical tools for making fast and accurate business decisions and are a huge source of competitive advantage. Businesses should now be able to create value in ways they haven’t been able to previously.

In the past, corporate success was based largely on the effective use of tangible assets such as plant and equipment. Whilst they still have a bearing, measurement of success now includes intangible assets such as intellectual capital, brand loyalty and, of course, data.

As a trending buzzword increasingly being used in the commercial and academic worlds, the definitive description of big data remains conceptually vague. However, it is usually defined as data sets so large and complex, created at enormous velocity and variety, that traditional data processing applications are inadequate to capture, manage and process the information in an acceptable timeframe.

Furthermore, to provide useful insights these data sets need to be processed with advanced analytic and algorithmic tools that are able to generate meaningful information and outcomes.

Capturing opportunities to enhance customer experience, improve process efficiency and launch new products and business models using big data is creating a new challenge for companies. Not only is investment in information infrastructure required, but also an investment in new talent and training-development programmes.

Key issues business leaders need to address regarding big data include defining the business outcome that they want from the data and identifying who is going to take responsibility for the analysis of the information provided.

From that an implementation team needs to be set up to make the relevant decisions as they will affect the entire organisation and the team will need to be drawn from all disciplines.

Looking at the human capital requirement, let’s start with the internal function of human resources. Certainly in major corporations, HR sits on a massive amount of internal data. When that material is overlaid with external data – information gathered not only from within an industry but also outside of the specific industry – HR personnel have at their fingertips invaluable data on remuneration, rewards, deployment of skills, tenure and much more.

This information is vital to the effective running of the organisation, particularly in an era where technology is advancing so rapidly and a new digitally-savvy generation of employee is entering the workforce.

Yet in a recent study by professional services firm PwC, less than half (43%) of the CEOs interviewed (1322 across 77 countries) stated they were making any significant use of data analytics to provide (amongst other things) better insights into how effectively skills are being deployed within their organisation.

Advanced talent analytics not only enhance HR capabilities to identify performance issues, they can also provide greater proactive targeting of candidates for recruitment and succession. Having the hard data to back up their decisions provides HR executives with the ability to act quickly and decisively.

However, appropriate knowledge of data analytics and leadership capacity constraints are undermining many companies’ efforts to achieve that.

In the future, HR directors will need to ensure that their teams have the skills and culture to support effective analytics. The development of relevant measurement parameters and the embedding of analytics tools and methods in processes will be vital. Generating reams of extraneous data is pointless if they don’t provide the basis for actionable insights and solutions.

Whilst technology plays an important role in all companies, and there will be IT issues to deal with such as information architecture, data flow, data quality, optimisation of network traffic and data storage, the principal focus of data analytics must be on the use of big data in the corporate decision making process

The future for those business leaders wanting (and being able) to grab the opportunities that big data will provide is very bright indeed. Big data contributes toward a total impression of how customers behave and why. And by using multiple layers of data and predictive analytics, companies will gain the ability to deliver the right product to the right person at the right time. But overlaying all this must be the human element, providing the “sense check” on the consequential actions of the analytics.

Since big data spans a wide range of corporate functions, including IT, marketing, sales, HR, customer services, risk and operations, whoever is appointed to take charge of big data needs to have multidisciplinary skills and strong leadership experience to influence and inspire appropriate action, as well as the business nous to be able to make effective decisions, based upon, but not slavishly relying on, the analytics.

In response to the changing business environment over the past three decades, businesses have added new positions in the C-Suite in finance, marketing and strategy (CFOs, CMOs and CSOs). With data and analytics profoundly altering the business landscape, capturing data-related opportunities to improve revenues, boost productivity and possibly creating entirely new businesses puts increased demands on companies.

Today, businesses are considering a new role at the top table, that of chief analytics officer. But along with this top appointment there is a requirement for new talent and investments in information infrastructure, as well as significant changes in mindsets and frontline training.

Whilst extra executive firepower is needed to continue the momentum of data analytics, the key focus now needs to be on solving the issue of the shortage of appropriate talent. Otherwise the potential value of data risks being leeched away.