I think that's a good approach. I can write FizzBuzz in 4 different programming styles, in at least 3 different programming languages, and I'd be happy to talk about all 12 permutations of them.
It's important to avoid well-known questions where the candidate can present a memorized answer. When the candidate has memorized the answer in advance, it's not a conversation, it's a presentation.
Likewise, I'm not looking for the candidate to show off. To be very specific, I'm trying to gauge two things:
1: The candidate's ability to problem solve around a set of immutable constraints.
2: The candidate's ability to listen.
The above pretty much go hand-in-hand: Someone might be able memorize "all 12 permutations" of Fizzbuzz; but can they comprehend the constraints in a discussion and intelligently discuss tradeoffs?
A different way to say it: It might take me 2 minutes to explain a problem. If the candidate only listens to the first 10 seconds, it shows, and demonstrates that they will be difficult, if not impossible, to work with.
(There's usually very predictable mistakes that I've anticipated. For example, one question I used in the past I would explicitly say, "do not worry about multithreading," and some candidates would still including locking.)
> Live interviews are generally excellent at revealing how individuals perform when placed outside their comfort zone.
I suppose if by "outside their comfort zone," you mean in a potentially literal life or death situation (because, face it, most people can't live long without a job in the US), under an arbitrary and very short deadline, possibly using unfamiliar tools, and simultaneously having to explain every little thing they do, then yes, I would agree with this. I don't believe the sentence after this follows logically from it, though.
No. More like the ability to be around new people, have their work evaluated, getting frank feedback, being honest about their mistakes, defending their positions.
If interview feels like life and death, maybe you are not the right hire.
Why do you need arbitrary (and very short) deadlines, and for someone to stand up at a whiteboard while simultaneously trying to solve a problem and "walk you through their thought process" to filter out people who can't write code on the job?
The short deadlines are because neither the company nor the candidate wants to spend a month on an extended interview. Solving a problem and walking through the thought process are because that's what "coding" is.
I don't know about you, but I've never had to live code a PR and explain to my reviewer what I was thinking while writing the code. By "deadlines" I'm referring to the length of the interview. Take home problems theoretically solve both these issues, but they need to be properly scoped and specified to be valid assessments.
I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it.
Like I said the deadlines work for both sides. If a company wants to give homework instead of having their own senior engineers spend time talking to me, that tells me what I need to know about how they value my time.
> I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it.
That's not equivalent to what I said, nor is it live coding.
Again, those deadlines are artificially short compared to real world scenarios, and completely arbitrary. They are so short, in fact, that they render the interview an invalid test of real working ability. A work sample has been proven time and again to be the most valid measure of whether a candidate can actually perform the job, but the conditions under which a live coded "work sample" is performed in an interview render it invalid.
It's not artificial: the company has a day of my time. I have a day of their time. We both want me to meet several people on the team to see if it's a good fit. Because of the constraint, we keep it to relatively simple discussions around toy problems that can be solved in an hour.
Yes, it is artificial. Everything about a live coding interview is artificial. Code doesn't get written in 1 hour blocks while someone's watching over one's shoulder, all the while asking questions to interrupt one's thought process, in any company I've ever worked for.
Like I said, this is literally a thing I do all the time. I have standing 1 hour blocks for each of my team members every week and it's not uncommon for us to build out the skeleton of a problem solution together. I literally did what you said on Wednesday for someone for a gitlab change because I don't expect they know how secret injection works, but I want them to know. And absolutely I've encouraged them to ask questions, and I ask them questions to check their understanding.
In most of the western world, firing employees is a high risk, high cost task. Ideally companies would hire quickly and fire poor matches just as quickly once they've been evaluated in the real world environment of the company. For this to work, on the employee side there needs to be knowledge that this is the company's process, financial depth to deal with the job not being stable, and a savviness to not relocate into a job that's risky. On the employer side, there needs to be a legal and social environment that doesn't punish removing non-productive employees.
The legal environment is what it is and unlikely to change. The social environment is fickle and trend driven. Workers can't always fully evaluate their odds of success or the entirety of risk of leaving a job that's valuable for the employee and employer for one that might end up as a poor match, even if both sides have been transparent and honest. It's a difficult matchmaking problem with lots of external factors imposed and short term thinking on all sides.
Ideally young workers would have an early career period that involves a small number of short lived jobs, followed up by a good match that lasts decades, providing value to both the employee and employer. Much like finding a spouse used to be a period of dating followed by making a choice and sticking with it so a life could be built together, employment ideally should result in both sides making the other better. Today however everyone seems focused on maximizing an assortment of short term gains in search for the best local timescale deal at the expense of the long term. It's interesting how the broken job market and broken family formation process in the western world mirror each other so much.
There was a lot of social pressure in the past for permanent marriages. That doesn't mean they were happy marriages. With the social changes in the west in the 1960s, divorce became more socially acceptable. Legal changes meant women had the ability to join the workforce and support themselves. People in unhappy marriages had options to seek happiness elsewhere. Those options didn't exist before.
For job retention, the problem is that changing jobs is often the only way to advance. I lost my best worker because the suits wouldn't give him a raise. He now makes more than I do at a different company. He liked his job with us, but he tripled his pay by leaving. My coworkers all tell the same story. I'm one of the lucky ones that managed to move up in the company, and that's only because I had the boss over a barrel.
> We can’t change the fact that live coding is a common practice in tech interviews. But we can try to mitigate the stress it causes.
Yes, we can. Don't do them. But, we have to replace them with something that works. That means none of these poorly constructed take home projects that are almost universally either drastically over scoped, criminally under specified, or both.
> we have to replace them with something that works
We don't. The simple solution is to stop maintaining the illusion that 100% perfect hire is possible.
Design your post-hire process around imperfect hiring rate / quick feedback loop, and accept the losses (they will happen anyway despite any perfectionistic illusions you choose to maintain).
These are few questions that really matter:
- Is there a track record of delivering meaningful results?
- Does their past experience seem credible?
- Are their soft skills and communication skills up to your expectations?
- Do they demonstrate adequate hard skills for the role?
- What interests and motivates them on professional and personal level?
Your interview process will always be just an attempt at answering these somewhat accurately, with diminishing returns after a certain point. Getting actual accurate answer to these is only possible through collaborative work in real environment.
I was about to suggest the problem with that is that applicants may think they meet your standards, and then be fired, but then realised that of course that very few coding interviews measure their skills to a sufficient standard to prevent that anyway.
Take-homes suck on both sides nowadays - if you're the interviewer, how do you know they didn't just plug it into chatGPT and spend the rest of the afternoon studying the solution?
I just don't know what a better proxy of coding ability even is, when we exclude options that can't be gamed/cheated.
> how do you know they didn't just plug it into chatGPT
If someone can present a good solution and talk about the reasoning behind it with enough detail & savvy to convince you that they actually wrote it, does it matter if they didn't?
That presumes that someone using a chatbot would necessarily generate a high quality solution and be able to explain the underlying reasoning.
And that is my point: there's a difference between someone pushing forth slop, and someone who is simply not doing the actual labor but could if they wanted to do so.
What’s wrong with it if they studied the codeGPT solution well enough to explain it, answer the questions about corner cases and suggest improvements? Won’t it be a good indicator of the candidates skills? ChatGPT is one of the daily tools nowadays, we should not ban it, but the one using it should be able to understand and explain the code, and his logic, and explain how he architected the solution and how the LlM assisted him, and where it worked well and where not so good.
If they can produce working code that solves the problem, and explain how it works, that is more than “just com[ing] back with the ChatGPT answer.” I'm not saying ChatGPT doesn't have its own issues, but this is not one of them.
I've had candidates who successfully did this to explain how a SQL JOIN works. But I'm not looking for candidates who can read a GPT prompt; I'm looking for people who deeply understand how a join works.
I used to be a fan of take-homes, but they have gotten ridiculous. Most importantly, many companies don't even follow up on them! It used to be fairly common etiquette that if you asked somebody to spend a day writing code for you, you at least had the decency to give them feed back.
Well, and as a candidate, while you're going to do some homework on the company in any case--any real take-home you know you're competing against people who are going to take a weekend or more with it whatever the instructions are.
Seems like now adays theyd just hook you up to whatever AI they claim to use and your job is to tell them why the AIs code would fail real world tests.
The catch would be not knowing whether the interviewee has the AI cargo cult PRIORS
The Soviets literally beat the US to every single major milestone in the space program, up through the 60's, except for literally landing men on the moon.
That was deliberate for propaganda purposes. They rushed many of their programs (and got a number of people killed by doing so) and simply never told anyone about the failures. Besides which Sputnik 1 wasn't very useful, but it was what they could rush out the door once they knew the US launch schedule.
Not to say the US Apollo program was fundamentally different... it was just much much bigger. And unlike the Soviets the US published their failures (see the Apollo 1 fire).
Odd to think about how much progress was generated thanks to national pride and propaganda. What a strange time in the history of the world.
