Context:

AI tools are increasingly becoming part of our daily workflows. To some extent, the ability to use these tools reflects the ability to effectively do your job. But it isn't that simple.

As a hiring manager, I find myself frustrated that I'm not getting to know candidates anymore. As a candidate, you worry that your fate is left in the hands of a tool that may or may not make the right call. It's a standoff I don't think the industry is going to solve anytime soon, but I have some thoughts to share now, and recommendations to make to candidates going through the SOE hiring process.

AI Usage by the SOE Recruiting Team:

There are two main concerns we have with using AI in the recruiting process: candidate experience, and tool validity.

For candidate experience, there are obvious ways AI can improve the process. By automating certain parts of the process, candidates can hear back faster and move through the process more quickly and with more transparency. On the flip side, it is not a positive candidate experience to only interface with an AI interviewer or to have to participate in automated screenings before earning the right to talk to a human.

Validity is more black and white. We find AI assessments of candidates have a pretty poor correlation with candidate quality. When our ATS tries to suggest qualified applications (this is a common feature in new tools but still requires a human to review), the correlation between "match percentages" and who we actually want to talk to is around 50%. I've interviewed plenty of people the ATS says is a 30% match and passed on plenty the ATS says are an 80% match. When I let AI tools parse interview notes, they're spectacularly bad at identifying good responses vs mediocre responses. So we won't be outsourcing to an AI interviewer anytime soon.

All that said, there are some things that we believe are worth looping AI into. If you're curious, you can check out those categories below:

Job description drafting:

We write up the key concepts as a bullet list, have AI turn that into a first draft based on our JD structure and role/company context, then make any necessary edits before sending it to the engineering team to pick apart.

Keyword highlighting on resumes:

The screening process is expedited when we have our ATS get familiar with a resume first. For each role, I indicate key experiences I'm looking for (working on a team, experience with HITL testing, etc.), and the tool will show me where on the page to start reading. Since resumes often have wildly differing formats, this makes the process more efficient.

Interview notes:

I use a transcription tool so I can focus on listening to you instead of typing through the conversation. I have the settings such that I'm not getting any "analysis" or commentary from the model included (as previously noted, I find this to be way off). What I want is the raw record of what you said. I'll ask at the start of the call whether that's okay with you, and if it isn't, I'll take notes by hand.

Scheduling:

Nobody on the team should end up with eight interviews in one week. Using AI tools to balance load and manage communications on that front means that you get an interviewer who isn't on their fourth call of the day and dying to get back to technical work. If you run into trouble with scheduling, we're around to help problem solve.

Feedback on me:

Every so often I feed my own transcripts back in and ask whether my questions, style, or framing is introducing any bias or noise into the process. Most of the time it tells me something blatantly incorrect, or something I already knew. Occasionally it's useful enough that I keep doing it.

Recommended AI Usage by Candidates:

To understand where it does and doesn't make sense to use AI in our process, it's helpful to understand what we're looking for in our interviews. We care about how you think, reason, and problem solve; we don't particularly care about polish or grammatical correctness.

When candidates use an AI assistant during interviews, they tend to generate decent answers to most of our initial questions (e.g. "tell me about yourself" style prompts). The assistants do not generate excellent answers, and they do not help us understand how you think and problem solve in real time. What will happen if you use an assistant in our conversation is that you will not be able to generate compelling answers to follow up questions, demonstrate logical consistency, or show that you can pivot in real time. As such, I'd recommend against these tools as it will waste both our time and yours.

There are a few places I'd recommend you spend your tokens if you want to actually increase your odds of moving through the process:

Preparing for interviews in general:

Mock interview tools are decent for getting your bearings and building enough confidence that you're not fumbling the first two minutes of an interview. It's more expensive than talking to your mirror, but it can help it feel real before the actual call. Don't lean on them too heavily though - every process is different, and just because you can fit a prepped "STAR" response to a question doesn't mean it's going to be what the recruiter is looking for.

Understanding what we build:

If you're coming from outside of industry, or just want to streamline your research on us, this is a good place to leverage AI. Use your preferred model to explain topics such as free-space optical communications or why nuclear companies need high signal data acquisition systems.

Targeting your resume

One of my favorite job searching tips is to keep a file of (true) bullet points about your work. List out all of them, going back years, even if it's far more than you'd ever put on one page. Then, you can feed AI a specific job description or title and let it pull out the ones that best communicate your value. Double check it afterward, but this can be a great way to tailor your resume quickly.

We're working this out as we go. If you have questions or concerns about anything on here, let us know!