Thriving Data Culture Starts with Data Literacy Assessments

What is data literacy?

To understand the value of data literacy assessments you must first understand the nature of what is being assessed. Data literacy describes an individual’s ability to identify and understand data sources, analyze them to generate insights, and use those insights to make better decisions that increase organizational value.

What are the levels of data literacy?

All employees do not need the same level of data literacy.  In addition, the core competencies of data literacy cover a wide range of subsets of skills including numeracy, communication, analysis, critical thinking, culture, etc. (Read more about the ten most common data literacy skills here). Data literacy requirements vary depending on factors such as where a company is on its digital transformation journey and its data maturity model, as well as its industry and business functions. The variation in data literacy skills poses a challenge to many organizations because not all employees and teams have or need the same exposure to and knowledge of data. Just because a Marketing Manager has a high level of education doesn’t mean that they understand data visualization techniques or can perform at a high-level using data. Yet, to be most effective in their role, they should be able to do so.To address these challenges, it is helpful to think of data literacy at different levels–individual, team/group, and organization.

At the individual level, it is helpful to further break data literacy down into what we call “personas” that describe the level of data skills an employee possesses. At QuantHub, we identify someone who has the most basic level of data literacy as a “Data Citizen”. Thereafter, we define six additional personas, each of which represents progressively more sophisticated skill levels up to data scientist.

Individual data skill profiles can then be rolled up into a team or group literacy profile. At the team/group level, it’s helpful to think in terms of business functions such as Finance or Sales or a subset of those, and the collection of data skills represented by the employee personas in that group. 

Understanding the skills already represented in the team/group and what skills are still needed to accomplish team objectives provides insights used for both training and hiring purposes. These diagnosed skill gaps inform targeted learning and development goals and the ability to be more precise in human resource requirements.

Finally, at the highest organizational level, the collection of group skills (or lack thereof) forms the basis for the level of corporate-wide data literacy and its measurement.

What is a data literacy assessment?

Leaders need to define and track relevant learning and development metrics in order to build a data-literate workforce. Data literacy assessments are the first step in the process. This type of assessment tool is a relatively new talent management and learning and development tool that diagnoses and measures gaps in a (current or future) employee’s data-related skills. The assessment can test an individual’s numeracy skills, ability to identify sources of data and their purpose, understanding of data visualization tools and techniques, ability to generate insights and make data-driven decisions, and more. In addition to identifying technical aspects of data literacy, assessments can also be used to identify non-technical aspects like employee attitudes towards data, their willingness to communicate data insights and support data-based decisions, and overall data culture.

How does a data literacy assessment work?

Who should be assessed? Although it may seem simple to suggest all employees should take a data literacy test, the reality is a bit more complicated and requires advanced preparation and decision-making. In light of our suggestion to examine individual, team/group, and organization-wide profiles, it makes sense to review various business units, functional teams, and product teams and determine which employees should be assessed and what their individual skill levels should be to fulfill the needs of the group.

What skills should be assessed?

As we mentioned there are numerous categories of data and analytics skills, and not all are required for all employees. Remember the personas? Skills assessments ideally will be tailored to the individual and their role, in the context of their group’s needs. That said, here are some general questions a data literacy assessment should answer:

• Basic skills:
  • To what degree can employees understand statistical operations such as correlations or averages?
  • Can they interpret data correctly?
  • Do they have the ability to read graphs?
  • Are employees able to choose the best graphs for presenting data to others?
• Intermediate skills:
  • Can managers develop a proposal based on concrete, accurate and relevant numerical analysis and trends supported by data?
• Advanced skills:
  • Can a Data Scientist present the output of their machine learning algorithms in a way that fosters data-driven decisions throughout the organization?
  • Can product managers propose new features and functions by analyzing customer segments and usage data?

The answers to such questions serve early on to define the foundation of an organizational data literacy program and a roadmap for a data literacy program.

What is the format of assessments?

Data literacy assessments generally test individual knowledge at the lowest level of topic hierarchy, then roll those topics up into higher categories of skills. At a high level, an assessment will show how an individual has varying degrees of skill mastery. For example, an individual may have higher than average competencies in data visualization understanding or analysis but only a basic ability to understand time series analysis or communicate data trends. This is why it is important to test at the most granular level of skills and topics.In practice, most literacy assessments return a quantitative score calculated for each subtopic or sub-skill. These are then rolled up to higher-level topics and skills. 

With respect to format, QuantHub data literacy assessment questions are divided into three types to address different types of knowledge and learners:

• Exercise questions – Items directing a user to complete a task.
• Concept questions – Items where a user is questioned about the underlying theory of a particular topic.
• Keyword questions – Items where the user must define a particular term.

How can data literacy assessments be used as a learning and development tool?

Continuous data skills measurement and promotion of more advanced data skills over time is necessary to progress towards a data literate workforce. An initial data literacy assessment provides a foundational view of an organization’s data literacy through a gap analysis that identifies learning paths for individuals and groups across the organization. Thereafter, as employees progress through their learning paths, assessments serve to measure and track progress on data literacy at the organizational level. 

At QuantHub, we view data literacy assessments as part of a continuous learning and development cycle composed of:

1. Assess for skills
2. Recommend learning paths
3. Learn new skills

Identifying development opportunities through assessment

The range of possible skills an employee should develop goes from basic data citizen skills such as identifying different data types up to advanced data professional skills. Most employees will start as data citizens. After further learning, they might progress to the data storyteller level capable of presenting complex data visualizations and advising management how to apply data to decisions. Ultimately, through training and educational paths supported by assessment results, a business can take a functional employee who is using Excel periodically and advance their skills to the level of an analyst. No matter the learner’s functional role, a data literacy learning journey leads to the ability to make data-driven decisions. 

Discovering data literacy strengths through assessment

Data literacy assessments are not just about identifying skill development needs. They should also be incorporated into career planning and retention. Individuals who want to advance their data skills and career opportunities find assessing their data literacy useful and informative. When assessments identify strengths and knowledge that can be applied in new ways within an individual’s role or demonstrate that an individual is performing above their target learning level, it’s important to recognize this and take action.  Managers should ask what growth and career opportunities they could offer a talented individual who understands business intelligence or other big data concepts.  For instance, managers using a data literacy assessment to identify possible mentors or champions find that leading their team through data transformation is much more achievable. In this way, assessments can be used to retain valuable employees.

Data literacy assessments as knowledge checks

Skills such as interpreting data, data mining, creating data visuals, and data analysis methods are too difficult to master during special training once a year and too hard to track with no testing. Data literacy assessments should therefore be part of a continuous process of learning and working towards becoming data literate. As employees continue to work towards data literacy, periodic self-assessments can serve as knowledge checks, enabling employees and their managers to focus training on specific areas. The occasional data literacy knowledge check also reminds employees of skills they can use and inhibit knowledge decay. Continually monitoring and measuring learning and skill development outcomes will inform leaders about overall progress towards developing a data literacy culture and data maturity.

What are other benefits of data literacy assessments?

It’s worth noting that assessments provide some less obvious but arguably important organizational benefits. Improved workforce communication is one. In many cases, data and lack of understanding data presents communication barriers between individuals and groups. The result is frustration and a failure to use data to improve performance and achieve business outcomes. With data literacy, employees learn the common language of data, and that starts with the individual assessment. Assessments also help to build employee confidence. One data literacy survey by Accenture found that just 21% of employees were confident in their data literacy skills. The process of self-assessment and tracking of progress can help improve confidence with data concepts and usage. Finally, a data literacy assessment can be used as an overall talent management tool.  Knowledge of organizational and team skill gaps can indicate whether internal promotion is possible to fill a skills gap or external hiring may be needed. The results can not only help to identify appropriate job candidates and their potential for data literacy, but also identify strengths and provide a baseline for future learning and assessments.  

You’ve assessed individual data literacy, now what?

Developing organizational literacy requires regular assessment of the individual, group, and organizational literacy, followed by planning to fill any knowledge gaps identified. Individual assessment is the foundation for organizational assessment. Organizational assessment is essential to planning and building a culture of data literacy. An organization can obtain the desired level of data literacy by mapping the current literacy levels; implementing learning and development paths; and integrating assessments into training, career growth, and recruitment. The data literacy assessment is the first step in an organization’s journey toward adopting data analytics, data strategy, data-driven decision making, and ultimately a data culture.