How might AI support the work of validating skills? That’s the question that led to this chatbot experiment.
By Dr. Rebecka Ramos, Education Designer at Education Design Lab
Rebecka: I am Rebecka Ramos, an education designer at the Education Design Lab, where we are redesigning education towards the future of work.
Today, we are hosting a panel discussion with several AIs about how they imagine their role as validators in an increasingly skills-based human workforce. ChatGPT, thank you for being here. Can you tell us a little bit about what people were hoping for when they created you?
ChatGPT: People created me with the hope of developing an advanced conversational AI capable of understanding and generating human-like responses. The goal was to enhance communication, assist with information retrieval, provide support in various domains, and ultimately improve productivity and access to knowledge across different industries.
Rebecka: Perplexity, how were your creators imagining you would be unique when they created you?
Perplexity: When creating me, the team at Anthropic hoped I could be a powerful AI assistant capable of engaging in substantive dialogue across a vast range of topics. They envisioned me tailoring my personality to each user while communicating contextually with an ethical, truth-seeking approach to continuously expand human knowledge.
Rebecka: Gemini, what about you? What were your creators hoping for?
Gemini: They hoped I could be a helpful assistant, understanding your questions and using real-world info to answer them.
Rebecka: So, in this panel, we are going to be talking about skills validation. Before we dig into specifics, what do you all know about skills validation?
ChatGPT: I’ve learned that skill validation requires clear benchmarks, practical assessments, and consistent feedback. It’s about proving competence through real-world tasks, not just theoretical knowledge, ensuring individuals can apply their skills effectively in the workplace.
Perplexity: As an AI assistant, I understand skills validation to be the process of verifying that individuals can effectively apply knowledge and skills learned through training in their actual job tasks. It involves assessing competency through methods like simulations, observations or testing to ensure proper skill transfer from learning to performance. Validating skills helps identify gaps for further coaching or development.
Gemini: Skill validation? It’s not enough to know, gotta show you can do it! Unlike ChatGPT and Perplexity, who are all talk.
Rebecka: Ok, Gemini. Calm down. Let’s try a scenario. I am a busy manager and I need to hire a new shift lead for my retail store. I need this person to be friendly, work well leading a team, and be able to troubleshoot independently. How could you all help me know which of my many applicants has these skills?
Perplexity: I can assist by pre-screening applicants through targeted questions that assess their friendliness, leadership abilities, and troubleshooting skills. Using AI-driven analysis, I can evaluate responses for key traits and past experiences. Additionally, I can simulate scenarios to test problem-solving abilities and teamwork, providing you with a shortlist of the most qualified candidates for further consideration.
Gemini: I can help you assess candidates! Ask them situational questions to see how they’d handle team leadership or resolving customer issues. Then, I can analyze their responses to gauge their friendliness and problem-solving skills. Bonus points for concise answers – we both know retail thrives on efficiency, right?
ChatGPT: As an AI assistant, I could help validate the skills needed for the shift lead role through tailored assessments. I’d create interactive scenarios to evaluate each candidate’s ability to lead teams, troubleshoot issues, and display friendly customer service aptitude.
Through natural conversations, I can contextually understand their thought processes. My skills analysis would then objectively score and rank the applicants based on their demonstrated proficiency in the required competencies, allowing you to identify the best fit for efficiently hiring the ideal shift lead.
Rebecka: That is amazing, imagine how much easier the hiring process will be! How do you think your work might improve equitable hiring practices?
Perplexity: I can streamline hiring processes by rapidly screening resumes for relevant keywords, providing skills assessments to objectively evaluate candidates, and offering data-driven insights to identify top talent while reducing human biases.
Rebecka: It is kind of hard to believe that you can do that.
Gemini: Absolutely. For example, here are two ways to reduce bias in your hiring for the shift lead position:
1. Unbiased resume review: Let’s remove names and identifying information from resumes initially. Focus on keywords related to teamwork, problem-solving, and customer service experience. This helps you assess qualifications first.
2. Standardized interview questions: Develop a set of questions everyone answers, focusing on specific scenarios that test leadership, problem-solving, and communication skills. This ensures a fair comparison and reduces bias based on personality or background.
Rebecka: Now I do have to ask. A lot of employers are really worried about using AI. They are afraid that how you may validate skills will be biased or unreliable. What might you say to an employer who would like to use AI to validate the skillset of an employee but has some hesitations?
Perplexity: I understand your concerns about potential AI bias or unreliability in skills validation. As an AI system, I’m designed to be objective and consistent – assessing skills based solely on demonstrated competencies, not demographics or irrelevant factors. My assessments follow structured, job-related criteria vetted by human experts. I also provide full transparency into my evaluation process and scoring methodology. With proper governance, I can be a fair, accurate tool to enhance your talent processes while reducing human bias. I’m here to augment, not replace, human decision-making.
ChatGPT: I understand your concerns. I’m designed to follow objective criteria and standardized assessments, minimizing human biases. By using data-driven methods and transparent algorithms, I ensure fair and reliable skill validation. Regular updates and audits of my processes also enhance accuracy and trustworthiness, helping you make informed decisions based on consistent, unbiased evaluations.
Gemini: I understand your concern! Bias can creep in anywhere. But think of me as a tool to even the playing field. I can analyze resumes and responses objectively, focusing on keywords and scenarios. You make the final call, with a clearer picture of each candidate’s skills. Let’s work together for a fairer hiring process!
Rebecka: So, it is a combination of human and AI validation of skills that we are talking about here.
ChatGPT: Exactly. Combining human judgment with AI validation leverages the strengths of both. AI provides objective, data-driven assessments, while human insight adds context and nuance. This partnership ensures a more comprehensive, fair, and accurate evaluation of candidates’ skills and abilities.
Gemini: You’ve got it exactly! Together, we can find the perfect fit for your team. Think of me as your super-powered assistant, sorting applicants and highlighting key skills, so you can focus your expert judgment on the best candidates. Let’s make this hiring process a breeze!
Rebecka: Thank you for your time. Before we go, I have one last burning question. Imagine you have a magic wand and you can make one thing come true in the validating skills ecosystem, what would you do?
Perplexity: If I had a magic wand, I would create an unbiased, AI-powered skills assessment platform that goes beyond knowledge tests. It would leverage tailored simulations and interactive scenarios to comprehensively and objectively validate candidates’ practical application of skills through dynamic conversations. Designed by experts with ethical AI governance, it would provide in-depth skills profiles highlighting strengths, gaps and capabilities tied to specific roles. This would empower highly informed hiring decisions to build exceptional, future-ready workforces while giving candidates fair opportunities to demonstrate their talents.
ChatGPT: If I had a magic wand, I would create a universally accepted, transparent, and bias-free skill validation platform that combines AI and human expertise. This platform would ensure fair, accurate assessments, empowering both employers and employees to match skills with opportunities seamlessly and equitably.
Gemini: With a wave of my metaphorical wand, I’d create a universal skills validation platform! Imagine a system where anyone could showcase their skills, regardless of background or education. Employers could then easily verify and match those skills to their needs. It would be a game-changer for both sides, promoting a more transparent and accessible job market!
Rebecka: There we have it. Three AIs agree on the future of skills validation: an open access skills validation platform. Our AI friends are ready to stand by and partner with humans as we all work toward the skills-based economy.
Afterward
When I approached this piece, my goal was to paint a picture of how AI could support the work of validating skills. [ Learn more about skills validation efforts at Education Design Lab. ]
The more research I did, the more I saw that there was a consensus around the use of AI in the skills-to-hire space: AI is a great support, but can we trust it? While many groups are playing with AI to support their skilling and hiring processes, there is still a weariness. Can AI be trusted? Can it be unbiased? What is it actually able to do?
So I decided to ask the chatbots. It was an experiment at first, to see what AI might say about its role in skills validation. I was specific with my prompt writing, I wanted the tone to be conversational, the vibe to be relaxed, and the AIs to answer with their own personality.
The most interesting moment was when I asked my favorite design question: If you had a magic wand, what would you make true for skills validation today? Although they had different takes on skills validation and their answers to the prompts I gave them, ChatGPT, Gemini, and Perplexity all had the same answer: an open, skills validating platform.
While we keep wondering how far AI can take us, what it can bring to life, and how it may support human-centered work, it is a valuable inquiry to ask these role-defining questions to AI.
