building an assessment TO MEASURE digital maturity
Role: Digital Product Design, Product Management, Project Management
Tools: Benchmark, Desk Research, Surveys, Testing, Psychometric Tests, Prototypes & Pilot, Visual dashboard
Date: 2018
THE CONTEXT
-
Digital transformation in Peru was becoming increasingly relevant. However, just digitizing the processes was not enough. It was key to adapt the way in which employees should approach work as well.
-
As an initial tool to begin working on the human pillar for digital transformation, I developed a methodology to diagnose the digital maturity level of current employees and upcoming applicants in companies.
-
I took full ownership for this project; it was a “1-person team” where my role involved being the Designer, Product Owner and Project Lead.
Building a psychometric test
The test consisted of 2 dimensions: mindset (soft skills) and knowledge (hard skills).
To define the constructs of the digital mindset dimension, I reviewed primary and secondary sources, and clustered themes to define the 4 constructs. I built a sample test of 150 items, which was validated through the judgment of 10 experts to adjust or eliminate items. Only the 75 items that met the defined standards were added to the final test.
On the other hand, the digital knowledge dimension measured 4 areas, each with 3 levels of difficulty, and with 3 questions per level to validate knowledge.
Dashboard to visualize results
I automated the processing of results and created an online interactive dashboard that allows to discover insights on employees potential to perform and promote digital transformation. All test can be visualized individually or filtered by area, job rank and period and compared with other companies.
The results obtained per dimension are captured on a matrix with 4 quadrants of digital maturity. The final score per employee is accompanied by the profile obtained and an automated conclusion that indicates the areas with low items in which the employee should focus, and actions to improve.
Wireframes created for the dashboard
Dashboard built for the pilot showing general, detailed results and individual location on the 4-quadrant matrix
Final dashboard used with clients showing general results, per area, per role and employee’s individual results
Carrying out a Pilot
The items and constructs were subjected to a field test. For this purpose, I configured the Digital Assessment test on an integrated platform with a cloud database to automate processing of results. The results were captured in an automated cloud dashboard.
I administered a preliminary version of the test with 118 items to a sample of 150 employees from LatAm companies specialized on digital consulting and digital marketing. Based on their answers, the internal consistency of each construct was evaluated through an item-test correlation and Cronbach's alpha, reducing the number of items from 118 to 74.
Implementing the assessment
The Digital Assessment was implemented on 8 enterprises from the financial, education, consumer goods and advertising industries as part of their digital transformation strategy.
I proposed a 3-step process:
-
Evaluate: Apply the Digital Assessment to measure the digital level of employees.
-
Analyze: Identify training opportunities and topics for current employees, or assess if job candidates meet the position’s digital requirements.
-
Re-skilling & Up-skilling: Improve current employee capabilities to promote digital culture and innovation by conducting training.
see other projects
How might we enhance the day-to-day journey of the bank's branch employees through the tools provided for them?
(before and during the COVID-19 pandemic)
MAKING REMOTE CHANNELS THE FIRST CHOICE
How might we turn remote advisors into the main interaction channel, leaving the bank branches as the last option?