15.Feb.2018 Babies and Bathwater and Bad Interview Questions


The Big Bad Competency Wolf

I’ve noticed a few articles recently criticising the use of competency based interview questions. These questions explore past examples of a specific, job-related behaviour. The rationale is that they’re the best predictor of future behaviour. They’ve become extremely popular in recruitment, to the extent they’re now a kind of status quo. I think that’s why some people want to challenge this and make space for new ideas. Like the idea that ‘character-based queries’ in interviews are more effective.

Where there’s a market for something that bucks the trend, somebody will always fill that niche, consciously or otherwise. That’s great when the facts back the ideas—that’s progress. An outdated orthodoxy needs to be challenged. It’s less good when the ideas ignore proven wisdom in favour of novelty or grabbing attention.

Out of the Frying Pan into the Fire

As an example, an article on TED Ideas recently pushed the view that competency-based interviews ignore important traits like compassion and wisdom, and that the most important thing is to work with people of good character, rather than merely skilled people.

On the surface this sounds great, and some of the questions suggested may seem refreshing at first glance. The trouble is that here, the new suggestions ignore the evidence backing competency based questions—the many years of study and proven effectiveness at getting past some dangerous hiring and decision-making traps. Traps that these questions fall straight into. Let’s take a look at three of them.

‘What are the one or two traits from your parents that you most want to ensure you and your kids have for the rest of your life?’

Really a rephrase of the traditional ‘what are your strengths/weaknesses’ question, long ago discredited. It plays into the hands of people who talk a good talk. A personable interviewee, or who’s spent five minutes with Google, can reel off a slick answer that sounds good but has little to do with reality.

Also, it guarantees no link with the criteria you’re recruiting against—will the traits they choose link to what you’re looking for? And you’re opening the door to similarity bias—with questions like this one, traits similar to the interviewer’s own will carry undue weight.

‘What is 25 times 25?’

This is designed to see how you handle pressure, but this idea doesn’t stand up to scrutiny. The test affects different candidates differently—those who are good at maths won’t be under pressure. And whether they’re good at maths or not, this question may well make them wonder about relevance and undermine the relationship-building aspect of good interviews.

If handling pressure is a key criterion for the role or team, it’s better to test it openly rather than by ambush, and in a situation related to their job. Unless the job is being interviewed, how they handle interview pressure (or maths pressure) is less important than how they handle relevant pressure in context. How well they do on this question may also affect how well they do on other, unrelated questions—if they feel like they handled this question badly they may be thinking about it afterwards.

‘After an interview, ask yourself (and other team members, if relevant) “Can I imagine taking this person home with me for the holidays?”’

Even a passing knowledge of the extent to which humans are affected by bias—particularly, in this case, similarity bias—suggests that this is a poor way to recruit. You’re recruiting somebody to fulfil a role, not to join your family. You want the best person for the job and the team, not a new drinking buddy or a date.

And if you do use this, how will you work it into the record of your decision-making (‘4/5 on Compatibility With My Home Environment?’), and how would you defend your decision (‘The way this question helped me objectively assess their fit with justifiable criteria is…?’)

Okay, So What Then?

Character is important. But no less important is a robust process that starts with fair criteria, and then, rather than coming up with nice-sounding ways to ask about them, uses a solid foundation to devise a way to test the candidate against each criterion.

This is easier with skills than it is with attitudes and character traits. But these can be tested, once they’ve been clearly identified, in a variety of more effective—and proven—ways, such as:

  • Separate assessment exercises and job-related tasks
  • A competency-based question that asks for an example of a time when they showed that trait, with skilful probing
  • Tests or questions where two conflicting good traits have to be prioritised, e.g. following procedure versus caring for the customer, or fostering good team spirit versus meeting targets

Keeping the Baby

So, just to drive home the analogy, in this case I guess the bathwater is an under-focus on character in interviews, and the upshot is that if this is happening, you can address it without throwing out the baby of proven, structured predictors of job performance.

The more general point is for everyone concerned with ongoing improvement and progress: existing ideas needs to be well-understood first, and if they’re to be taken down, the new idea needs to have better evidence going for it, or at the very least something more than novelty alone.

There are 2 Comments to "Babies and Bathwater and Bad Interview Questions"

  • If I thought an interviewer was trying to decide if they would want to take me on holiday even if just hypothetically, I’d be so freaked out.

    • admin says:

      Agreed. I’m trying to imagine my reaction if I was on a panel with somebody and they suggested this. And yet TED are giving the idea profile; the Facebook comments section when they published the article there had a lot of outrage, but a lot of people saying: ‘hey, I might try these’.

Write a Comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>