Interview them where they are: Master the art of conducting technical interviews

[00:00] (upbeat music)
[00:21] - So it think we'll get started.
[00:23] And I'd like to start by telling you a story.
[00:29] This story, I've anonymized, or rather left out
[00:33] some names so as to protect companies
[00:36] that have interesting interview practices.
[00:40] But I do want to let you know ahead of time
[00:45] this story, everything that happened in it is true,
[00:47] but I've stitched together multiple interview loops
[00:50] into one for the sake of the story.
[00:52] So don't feel bad for me.
[00:53] I did not have one super awful day
[00:55] where everything went terribly wrong.
[00:57] But these are all things that did happen to me
[01:00] in the course of interviewing at small companies,
[01:02] large companies, companies who's names
[01:04] you know and read about in the news,
[01:06] and then companies that maybe you don't.
[01:10] So I drove to the company and I was very excited.
[01:14] And I started my interview loop
[01:17] by talking to the coordinator who was running it.
[01:19] Everyone was super nice, friendly.
[01:21] Restroom, water, coffee, all that.
[01:24] And I go into the interview room and
[01:26] my first interviewer comes in and introduces himself.
[01:29] Pulls out a piece of paper
[01:33] and starts asking me things like, "What is a pipe?"
[01:37] Not how do they work, not why are they interesting,
[01:39] not what are some Unix running utilities
[01:42] that you'd think to use.
[01:44] But literally what is that character right there,
[01:48] (laughter) what does it do.
[01:50] So I said, 'Okay.'
[01:51] And I gave them the Wikipedia definition
[01:54] and they said "Okay great."
[01:56] And then they said "Let's move on."
[02:00] "Does Ajax return things other than X amount?"
[02:04] And I said 'Sometimes, I think.'
[02:08] And I talked about Json and I talked about
[02:11] web standards and XMS script (chuckles),
[02:13] XML script, XML script, anyway.
[02:15] (audience laughs)
[02:17] And they said "Okay, it sounds good."
[02:20] And continued on in this way for 35 minutes
[02:24] asking just, kind of, trivia questions
[02:27] that ranged all over the place.
[02:28] Back-end, front-end, command run utilities things like that.
[02:32] And at the end they said, "Thanks for your time."
[02:34] And they left.
[02:36] That was interesting.
[02:38] And then I had another person come in
[02:41] and they introduced themselves and said,
[02:43] "Let me ask you a problem."
[02:44] And they asked me, you always kinda know, right?
[02:46] When someone asks you a question
[02:47] that's something like, "How would you validate
[02:49] "that this data structure is correct?"
[02:50] You know there's like a secret follow-up question
[02:52] that they really wanna ask but
[02:53] they want to ask the easy one first.
[02:55] And then ask you something like,
[02:56] "How would you generate a data structure?"
[02:59] So I started doing all the things
[03:00] you're supposed to do, you know?
[03:01] Writing on the white boards,
[03:02] saying, 'Let me make sure I understand the question.
[03:05] 'Let me make sure that I've got this right.'
[03:09] And they interrupted me and said,
[03:11] "This isn't hard, just write the code."
[03:15] And I don't know what that would do to you,
[03:17] but that is very affective at throwing
[03:18] me off my game entirely.
[03:20] So I then spent the next 35 minutes
[03:23] mumbling and sweating and freaking out,
[03:25] and producing a very poor answer
[03:26] to not a very hard question.
[03:28] Although I would ask things like,
[03:30] 'Does it seem reasonable, is this right?'
[03:31] And they would say "No."
[03:32] And after a lot of poking and prodding,
[03:35] I would find out I had missed a semicolon,
[03:36] or I had swapped two indices.
[03:38] Is and Js look similar on the white board.
[03:43] So we didn't get to their favorite part
[03:45] and it became abundantly clear that
[03:46] they were very irritated that we
[03:48] didn't get to their fun question.
[03:49] But the nice thing about interviews
[03:50] is they are eventually over.
[03:52] (laughter)
[03:53] And thanked me for my time, took a picture
[03:55] of the whiteboard and left.
[03:57] And I felt pretty bad but it was okay
[03:59] because now this was the lunch interview, right?
[04:02] This is the one where you go and you have food
[04:05] in the company cafeteria and somebody
[04:06] tells you all about what it's like to work there,
[04:09] the culture, other people, things like that.
[04:12] And this person was very nice, a little bit older,
[04:14] had been with the company for sometime.
[04:16] Told me about the culture,
[04:17] what they liked working on, what they didn't.
[04:20] And I don't (chuckles) know if HR told him
[04:22] to say this or if this was just a thing.
[04:24] All they could think of to tout to me,
[04:26] on the one end who's interested in things like inclusivity.
[04:29] But this is a direct quote how we've
[04:30] heard all about the unisexual bathrooms.
[04:33] (laughter)
[04:34] The unisexual bathrooms.
[04:36] I think they meant non, like, gender-nonspecific bathrooms.
[04:38] But that was nice, like it was well-meaning.
[04:40] So I finished my meal, I took my unisexual bathroom break,
[04:45] and I came back. (laughter)
[04:45] And I said, 'Okay, let's continue.'
[04:51] So, and this one threw me the most.
[04:52] This person came in, they were super friendly,
[04:54] really energetic, very warm.
[04:56] And they gave me a reasonably good,
[04:57] you know, the well-scoped interview question.
[05:00] And they said, "Does that make sense?"
[05:01] And I said, 'It does.'
[05:02] And then they sat down and they
[05:04] opened their laptop and I thought
[05:05] maybe they'd be taking notes,
[05:06] maybe there was like some interactive component.
[05:09] And I started writing on the whiteboard
[05:11] and they proceeded to ignore me, entirely.
[05:15] As though I were not there for 40 minutes.
[05:18] (laughter)
[05:18] And I would say things like,
[05:19] 'Does this make sense, is this reasonable?
[05:21] 'Is this the right approach?'
[05:22] And sometimes they would look at the
[05:23] whiteboard and say, "Yeah."
[05:25] And sometimes they would just say "Yeah."
[05:27] Or nothing.
[05:28] And then they thanked me for my time,
[05:29] and took a picture of the whiteboard and they left.
[05:32] This is the theme. (laughter)
[05:35] And finally I'm getting to the end of the day,
[05:37] I'm very tired but I know there's
[05:39] only one interview left.
[05:40] And so this is okay.
[05:42] And the interviewer comes in,
[05:43] also very nice, very friendly.
[05:45] And he asks me a question but it
[05:47] turns out you cannot meaningfully answer,
[05:49] unless you know what a Derange sequence is.
[05:54] Which I did not.
[05:56] And what made me really mad about this is that
[05:58] Derange sequences are super interesting mathematically.
[06:02] And so I went home and looked up
[06:03] the events there on Google and Wikipedia,
[06:06] and I was super mad because I loved
[06:08] the idea but I was mad that it came up
[06:10] in a interview and I didn't know the answer.
[06:13] So that was, I think, the most trickiest one for me.
[06:15] Whereas this would otherwise be super cool
[06:16] and I kind of see why this person asked the question.
[06:20] But anyway, so that was the end of my interview loop.
[06:23] And I said, 'Thank you.'
[06:23] And I went home.
[06:25] And you'd think that they would've said no but they didn't.
[06:29] And you'd think that they would've said yes but they didn't,
[06:31] they said, "Would you like to come
[06:32] "in for some more interviews?"
[06:33] (laughter)
[06:35] And I said, 'No, uh uh.' (laughter)
[06:37] So I don't like the idea of comparing
[06:42] software developers to medical professionals
[06:44] or doctors, it is knowledge work like medicine.
[06:48] We are on call sometimes.
[06:50] But I think that comparison is super fraught.
[06:52] I don't like pretending that whether or not
[06:55] the webpage is up is equivalent to,
[06:58] you know, life or death situations.
[07:00] But I have friends who are doctors
[07:01] and this part resonates with me.
[07:02] Where I realized, imagine if you like go in
[07:05] and you're interviewing as a doctor,
[07:06] and they say, "You're references all check out.
[07:08] "You're a well-known professional.
[07:11] "Clearly you have a track record of success.
[07:13] "If you wouldn't mind just baring with me
[07:16] "for just a brief exercise. (laughter)
[07:19] "Just to demonstrate that you really
[07:21] "do know how to do surgery."
[07:22] (laughter)
[07:24] This is what it makes me think of and I don't like that.
[07:26] I don't like that interviewing
[07:28] makes me think of an operation (laughter).
[07:30] So interviewing is broken.
[07:32] I don't say that lightly.
[07:34] I don't like to say broken unless I am willing to continue.
[07:37] Because when someone says something is broken
[07:38] that immediately invites the question,
[07:40] "How is it broken, what's wrong?"
[07:42] So I'm gonna go through that and
[07:44] I'm gonna talk about how we can fix it.
[07:47] And that's the whole point of this talk.
[07:50] So this talk is called Interview Them Where They Are.
[07:53] And hello RailsConf, hello Minneapolis.
[07:55] I'm delighted to be here, I've never been to Minnesota,
[07:57] I've never been to Minneapolis before.
[07:59] So this has been fantastic.
[08:01] My name is Eric.
[08:02] I am a Software Consultant for the
[08:05] company called Test Double.
[08:06] If you're not familiar with Test Double,
[08:08] we are a distributed remote consultancy
[08:11] that pairs with client teams to not only deliver
[08:14] great software but insure that the teams themselves
[08:17] are better as a result of having worked with us.
[08:20] So if you are thinking about your current projects,
[08:24] your current environment, and you think
[08:26] there's something that we could help with,
[08:27] please don't hesitate to reach out to me.
[08:29] I'm Eric Q Weinstein.
[08:30] I'm on most things, Twitter, GitHub,
[08:32] you can reach out to me at Eric@TestDouble.com.
[08:36] If you are looking for your next adventure we are hiring.
[08:39] So, and I'm happy to talk a little bit to you
[08:41] about the interview process at Test Double.
[08:43] And finally, if you do want to talk more
[08:46] about interviewing, and diversity, and inclusion,
[08:48] these are topics that are very important to me,
[08:49] and I'm always happy to chat.
[08:51] So please do come find me after the show.
[08:54] And a few years ago I wrote a book
[08:56] to teach Ruby to eight, nine, and 10 year olds.
[08:59] Published by No Starch Press.
[09:00] It's called Ruby Wizardry.
[09:02] Let me know if you're interested,
[09:03] if for some reason you'd like a copy
[09:04] and just can't afford it let me know
[09:06] and we'll work something out.
[09:09] So the points of interviewing in my mind are to
[09:12] find the people that we need to grow our teams,
[09:15] which I think is somewhat non-controversial.
[09:18] But the one thing that I think is not
[09:20] always talked about and one that I think is critical
[09:21] is we're optimizing for demonstrating competencies.
[09:26] Too often, someone, an interviewer
[09:29] intentionally or inadvertently views the
[09:30] interview process as sort of a challenge.
[09:31] Kind of, a way to find something
[09:33] that the candidate doesn't know.
[09:35] To probe for weaknesses and say
[09:36] this person really knows Javascript,
[09:39] really knows React, doesn't understand Active Record,
[09:41] doesn't understand databases, etcetera, etcetera.
[09:44] Without really thinking too much about
[09:46] is that what you're looking for.
[09:52] So I really like running tests.
[09:53] I test a lot, I usually do test-driven development.
[09:58] And this has happened to me a lot in my career
[10:00] but this first time it happened it was very frustrating.
[10:02] I remember writing some code and I was like
[10:04] I'm gonna sit down, I'm gonna write the test.
[10:06] So I wrote a test and it was red,
[10:08] which is great because that is the
[10:09] first step in writing your tests.
[10:11] In red, green refactor.
[10:13] Which nobody tells you initially is
[10:14] shorthand for red, red, red, red, green,
[10:19] red, red, red, green refactor. (laughter)
[10:23] So I'm writing my test and I'm excited
[10:24] 'cause it's failing and I write the
[10:25] production code and my test stays red.
[10:28] And so I start digging around, I'm trying to figure it out.
[10:31] I'm writing down the execution paths on paper,
[10:33] staring at my code, back to the paper, back to the editor.
[10:37] Finally I looked back at the test after way too long,
[10:40] and I realized that my test was wrong.
[10:42] And this is a thing that I think we don't think
[10:45] about often outside the context of writing code.
[10:48] If you're interviewing someone
[10:50] and they have years of experience,
[10:51] and all these great open source contributions.
[10:52] And they're very smart and very sharp,
[10:54] and very empathetic, and they don't know
[10:56] the answer to some trivia question.
[10:58] Or they don't do well on the whiteboard,
[11:00] and you decide well the test is the test,
[11:03] so this person is not qualified to work here,
[11:05] maybe the test is wrong.
[11:10] One thing I think a lot about is
[11:11] the way that we construct interviews.
[11:15] And a lot of people think interviews
[11:16] start when you walk into the room and the candidates there,
[11:19] or maybe a couple weeks before when you get
[11:20] that calender invite with the attached resume.
[11:23] But it actually starts much much earlier,
[11:25] months ahead of time.
[11:26] And this is a job description.
[11:29] I'm sure many of you have seen these,
[11:30] we call them JDs in the biz.
[11:32] Having been a Engineering Manager,
[11:33] I've written several of them,
[11:35] and written some really bad ones.
[11:37] I wrote this one but for the talk,
[11:38] this is not one that I wrote for work.
[11:41] I won't read it to you but this
[11:42] all seems familiar, right?
[11:43] We're looking for some credential or some equivalent,
[11:46] we want a certain amount of experience,
[11:48] certain technologies.
[11:50] And bonus points, whatever that means for
[11:52] our upper particular stack.
[11:54] I don't think this is a very good job description.
[11:56] And I think we can kind of take our
[11:58] inclination to refactor out of the realm of
[12:02] writing tests and bring it to our job descriptions.
[12:04] So that's what we're gonna do.
[12:06] So I'm gonna break this up into the
[12:08] four kind of pieces, you know?
[12:10] Credential, experience, particular languages, bonus points.
[12:15] And let's look at this first one.
[12:16] "We're looking for someone with a
[12:18] "bachelor's degree in computer science or equivalent."
[12:22] I don't think this is a valuable
[12:23] thing to put in a job description
[12:25] because most of the time you don't need it.
[12:28] Now there is some roles where you really,
[12:30] a full grounding in computer science
[12:32] is necessary or is important.
[12:33] And you may work at a place where, you know,
[12:36] for whatever reason they try and say,
[12:36] "Hey we do need people to have
[12:38] "four year degrees for X, Y, Z reason."
[12:41] That's fine.
[12:42] But I don't think a degree in
[12:43] computer science should be necessary
[12:44] for most of the work that we do.
[12:47] And I don't anyone has ever explained
[12:48] to me what equivalent means.
[12:50] It's sort of a content free statement, right?
[12:52] (laughter)
[12:52] It's like equivalent experience.
[12:53] And so what is that?
[12:54] Like is there a secret bachelors afterwards?
[12:56] I don't know what that is supposed to be.
[12:57] (laughter)
[12:58] So I recommend we get rid of it.
[13:01] So when we refactor our JDs in dead code
[13:04] basically we're not gonna put this in.
[13:07] "One to three years of experience."
[13:09] This says to me that you're trying to
[13:12] find someone with a certain level of seniority.
[13:15] But what happens when you say one to three years
[13:17] of experience is, one, it's extremely broad.
[13:20] I was different in my third year of writing
[13:22] software professional then in my first.
[13:25] But also there's such a broad range of experience.
[13:30] If you start at a startup that's always on fire,
[13:32] and you're wearing in multiple hats,
[13:33] and do all these different things.
[13:35] You will have a much different set of skills
[13:37] than if you go for a big bank and
[13:38] you write Java for three years.
[13:40] Not to knock big banks.
[13:42] Maybe a little bit.
[13:44] So I don't think this is really very valuable.
[13:48] What this is really asking is
[13:49] well what does this person know how to do?
[13:52] And so what I would say is,
[13:53] 'Well we're looking for someone who is
[13:55] 'comfortable writing features and is
[13:57] 'looking to own entire services,
[13:58] 'or large swats of the single application.
[14:01] 'If you manage one application.'
[14:04] And that'll get you closer to what
[14:05] you actually are looking for,
[14:06] and not some arbitrary number of years.
[14:09] Especially because we probably have worked
[14:10] with people who have three years of experience,
[14:11] and we've probably have worked with people
[14:13] who have one year of experience three times.
[14:15] That is very different. (laughter)
[14:18] So moving on.
[14:19] And so we want someone who knows
[14:20] Javascript, and React, and Go.
[14:22] And I think this is reasonable,
[14:23] this is the piece that's gonna change the least.
[14:25] But I think it's kind of unclear where those
[14:28] weights lie and what you're actually looking for.
[14:31] So I would say something more like,
[14:33] 'We prefer the candidates know Javascript,
[14:34] 'and React, or Go.'
[14:36] But maybe for more experienced or
[14:38] senior candidates it's fine if they don't.
[14:40] It's not a hard requirement.
[14:41] Kind of acknowledging that people
[14:42] with more experience have sort of
[14:44] learned how to learn in some capacity
[14:45] and can come up to speed faster.
[14:46] Then someone who's brand new to maybe
[14:49] Git, and editor, and all these other things.
[14:51] And finally bonus points for
[14:53] Postgre, microservices, Kubernetes.
[14:56] I also think this is kind of content-free,
[14:58] it's unclear to me what this means.
[15:00] But I think what it's driving at is
[15:01] well here's what we do.
[15:03] And my opinion is that you should just say that.
[15:05] Our stack is Javascript/React with Go on the back end.
[15:09] Those Go services are organized as
[15:10] microservices orchestrated by Kubernetes,
[15:12] and the data services post grads.
[15:15] So now we can actually put these
[15:16] back together into a job description
[15:18] that I feel much better about.
[15:19] This feels really like an improvement,
[15:21] it's not ideal, I don't think.
[15:22] But I think it's really nice to say,
[15:23] 'Here's what we're looking for.
[15:25] 'Do you know these things?
[15:26] 'Or have you done these things?
[15:27] 'Are you looking for these things?'
[15:29] We can have a conversation.
[15:31] One thing people sometimes say
[15:34] in response to this is, "Well I don't have
[15:37] "specifics anymore this too vague.
[15:38] "I don't have a degree requirement.
[15:39] "I don't have information on how many years of experience.
[15:43] "I'm gonna get unqualified people."
[15:45] And I think the opposite is true.
[15:47] I think when you have gatekeeping language
[15:49] like degree, number of years,
[15:51] what happens is people self-select out
[15:52] even when they are qualified for the role.
[15:55] And this is disproportionately true
[15:57] of under-represented minorities.
[15:59] Groups in tech, or like women in tech,
[16:01] and folks who don't have medium-white-man-syndrome.
[16:05] Which I have.
[16:06] As a medium white man, both in terms
[16:09] of my average capacity and my propensity
[16:12] to write things on medium.
[16:13] (laughter)
[16:15] (chuckles) I feel like I should have a lot of things,
[16:18] like here's what happened to my startup.
[16:19] Anyway. (laughter)
[16:22] The thing that doesn't happen is I'll say,
[16:24] 'Well I got three of these five checkboxes.
[16:26] 'I'll just send in my resume and see what happens.'
[16:29] This is not something that happens
[16:31] necessarily outside of my sphere of privilege.
[16:35] And so you have to be mindful of the fact
[16:36] that there are people who will select themselves out
[16:38] if you've inadvertently put in all this,
[16:40] or intentionally put in all this
[16:41] gatekeeping lanugage around years and credentials.
[16:44] So I think this is a substantial improvement.
[16:45] 'Cause now, you know, you're gonna
[16:48] start finding who you're looking for.
[16:51] And there's a few key takeaways.
[16:52] One of the big ones from this talk is know
[16:54] what you're looking for before you interview.
[16:57] And critically you have to know
[16:59] how you're gonna measure success.
[17:01] Now, I said that job description wasn't ideal.
[17:03] I would love job descriptions that
[17:04] include how we measure success,
[17:05] how the company thinks about reviews,
[17:08] and what we want to see from you as a Software Developer.
[17:12] But if you don't include in the JD,
[17:13] you at least need to think about it.
[17:15] When you write it, before you interview
[17:17] because you're setting people up for failure
[17:18] if you don't know how you'll evaluate
[17:20] them once they join your team.
[17:22] So no we have people on our pipeline.
[17:25] Imagine no people like me who don't listen
[17:27] and don't know how to self-assess.
[17:29] (chuckles) We have people who theoretically
[17:30] are who we're looking for, right?
[17:32] Which is, we know popular space, which is nice.
[17:36] And so now the question is how do we evaluate them, right?
[17:39] Historically this has been done
[17:41] on the whiteboard which is a blackboard,
[17:42] because I'm limited by the available emojis.
[17:45] (laughter)
[17:46] And what we're interviewing,
[17:48] I honestly think really was valuable at one point.
[17:51] Back in the day you had a certain amount of compute time,
[17:53] you would schedule time to use the computer.
[17:55] In the meantime you'd write down all
[17:57] your code on paper, make sure it was right.
[17:59] And then type it in as fast as you could (chuckles),
[18:01] and compile it and see if it worked.
[18:03] That is not the world that we live in anymore.
[18:06] And I think that whiteboarding interviews
[18:08] can have a place, sort of, and I'll expand on that.
[18:12] But probably it's not a good proxy
[18:14] for writing code with other humans,
[18:15] which is what we do. (laughter)
[18:16] We work on teams, we write code collaboratively,
[18:18] we communicate and then spend time together.
[18:20] And whiteboarding by yourself on a board
[18:22] while somebody says, "Yeah that's right."
[18:23] Or "No that's not."
[18:24] Even if they're more engaged, I think
[18:26] it's not a good way of doing it.
[18:29] So there are some obscure programmers
[18:32] on the internet who also agree with me.
[18:34] I hope DHH is, no he's not here.
[18:37] But he said the same thing and
[18:39] I agree with DHH obviously.
[18:41] I would fail to do this, I have failed to do this.
[18:44] I don't fear whiteboarding anymore, I used to,
[18:47] I used to be super afraid of whiteboarding.
[18:50] And now I just don't like it.
[18:51] (laughter)
[18:53] It's because the medium doesn't really make sense,
[18:56] and again the content is not correct
[18:59] for what we're evaluating.
[19:00] We're tryna figure out how do you communicate,
[19:02] how do you take ambiguous requirements
[19:04] and turn them into increments of work.
[19:05] How do you write code (chuckles) with other people?
[19:09] And this has been noted in texts like
[19:12] Programming Interviews Exposed.
[19:14] Where they kind of said, "Well based on
[19:15] "these constraints you're not gonna be
[19:18] "asked any real-world problems."
[19:20] Unfortunately, real-world problems are all that we solve.
[19:25] And so we're back here.
[19:27] So, what are some ways that we can actually
[19:30] get a signal and interview people
[19:32] in a way that's more inclusive.
[19:34] Gets us better, like a higher fidelity signal
[19:36] around what they can do.
[19:38] So this is Ada.
[19:41] Ada is a fresh computer science graduate,
[19:43] she went to college.
[19:45] Maybe has an internship or two
[19:47] but has not really written production code
[19:51] in a group, in a company for very long.
[19:53] So the first question is well what
[19:54] is Ada going to be good at?
[19:57] Now, I would assume her strengths are algorithms right?
[20:01] Sorting, searching, things like that.
[20:03] Things that are taught in a traditional
[20:04] computer science program, data structures,
[20:08] and maybe also stack overflow.
[20:09] But I think everyone is well-versed in stack overflow.
[20:13] And, kind of, well-defined tightly scoped providence.
[20:16] Thing that are, sort of, there's a right answer,
[20:18] there's an input, there's an output.
[20:20] And this is maybe the only time
[20:23] I would still be okay with whiteboarding.
[20:25] And this is absence, you know, projects,
[20:28] a data profile, internships.
[20:30] Things that demonstrate that collaborative
[20:32] nature of writing going together.
[20:34] You also want to make sure you're
[20:36] asking for things that are going to
[20:38] be relevant to their experience.
[20:40] So, if Ada has spent months and months
[20:41] preparing for whiteboard interviews,
[20:42] and I say 'Hey, let's pair on something.'
[20:44] And she has never paired before.
[20:45] That's is kinda like me going into a room
[20:47] and expecting to pair and getting whiteboarding.
[20:50] So again, it's flexing the interview
[20:52] and sort of meeting people where they are that's critical.
[20:55] I call these made-to-measure interviews,
[20:57] you can construct modules around
[20:59] what you're looking to assess based on
[21:01] who you're looking for in the world.
[21:04] And you can mix and match them a little bit
[21:07] or even allow our interviewees to select
[21:08] candidates to select what they want.
[21:10] And say, "Well you know, I'd really like to do
[21:12] "a take-home and then pair on it."
[21:13] Right, that's one of the options?
[21:14] It's almost like when you go and have
[21:16] dinner at a wedding, you know?
[21:17] It's like the fish or the chicken.
[21:18] No one yells at you if you pick the wrong one.
[21:20] (laughter)
[21:21] No one's like you didn't go to dinner
[21:21] if you had a different dinner than they had.
[21:24] No one gives you just a plate of garbage
[21:27] and says here you go, everyone gets the
[21:28] same plate of garbage so this is fine.
[21:29] (laughter)
[21:30] Fairness is, I think, a toxic idea sometimes.
[21:33] So this is what I would recommend for you.
[21:37] Like I said, there are projects,
[21:38] opportunities to pair, and they really simulate the worker.
[21:40] We prefer that.
[21:43] So this is Ben.
[21:44] Ben graduated from a bootcamp maybe
[21:46] three for four years ago.
[21:47] So he does not have a traditional CS background,
[21:49] but has worked in the industry for a few years.
[21:52] And has attended a bootcamp.
[21:53] So he spends a lot of time thinking about
[21:54] code quality, writing modular,
[21:56] easy to change, and test code.
[21:58] So I would expect Ben could potentially
[22:00] be really good at pairing.
[22:01] We do a lot of pair at Test Double.
[22:03] I know there're other companies that do a lot of pairing.
[22:05] Bootcamps that emphasize pairing.
[22:08] Probably good at testing, refactoring,
[22:11] changing code, preserving behavior
[22:13] but making the code base nicer.
[22:15] I would expect him to be able to
[22:17] talk about the trade-offs involved in
[22:20] his language or framework of choice, okay?
[22:24] To have that sort of deeper conversation.
[22:27] So one thing I might ask Ben to do is,
[22:30] 'Hey, let's pair on a small window-downed
[22:34] 'version of a real production thing.
[22:35] 'Let's maybe if you have the time,
[22:37] 'do a take-home and then we can pair on either
[22:40] 'refactoring it or adding new functionality afterwards.'
[22:43] These are pieces that are actually
[22:44] used in the Test Double interview process.
[22:47] But here I'm advocating we often
[22:49] give them the choice and say,
[22:51] 'Hey, could you do this and could you do that?'
[22:52] And being respectful of other people's time
[22:54] because there are people who are new
[22:55] parents or have other obligations.
[22:57] Maybe they don't have time for a full
[22:58] interview loop but they can do a take-home.
[23:00] Maybe they don't have time for a take-home
[23:01] but they're happy to pair.
[23:02] And I think they key here is having
[23:04] that flexibility and looking for strengths
[23:06] rather than prodding for weaknesses.
[23:11] Finally this is Charlie.
[23:14] So Charlie did do a tradition computer science degree
[23:18] but they graduated 10/12 years ago.
[23:21] So they they have not spent a lot of time
[23:22] thinking about algorithms or data structures.
[23:25] They've been thinking about production problems,
[23:28] they've been thinking about keeping the lights on,
[23:30] they've been thinking about fires,
[23:31] management, dealing with product stakeholders.
[23:34] All the things that we do everyday, above and beyond.
[23:37] There's sort of, like underlying competitions.
[23:40] So, again I would expect Charlie
[23:42] to be able to do programming, pairing.
[23:45] Maybe I might give them a thornier pairing task,
[23:48] then I might give to them something
[23:49] that's gonna have more edge cases.
[23:51] I would expect Charlie to have evolved
[23:53] to the point where it's not if when,
[23:55] you know, if things go wrong but when things go wrong.
[23:58] And having that mindset of like this will
[24:00] eventually fail so what is the failure mode.
[24:04] I would expect Charlie to have a lot of strength
[24:07] in terms of one-to-many communication, right?
[24:09] Maybe Ben I would expected to master
[24:11] maybe one-to-one communication but not to
[24:13] have mastered getting by in front of a large group,
[24:16] or giving a conference talk and getting
[24:19] consensus from a large number of people.
[24:22] So I would expect to be able to
[24:24] test that out and say things like,
[24:26] 'Hey tell me about a time you had to
[24:27] 'get consensus or get by in front of a group,
[24:30] 'especially a group that didn't report to you
[24:31] 'or didn't have an obligation to you.'
[24:34] I would look for service and system's level thinking
[24:36] and say 'Tell me about this big project
[24:39] 'you did on your resume.'
[24:40] Or, 'Hey we have a service XYZ at our company
[24:43] 'how would you go about developing that service?'
[24:46] Right?
[24:46] And really, again, not prod their weaknesses
[24:49] but ask for clarification and say, 'What was the trade off?
[24:52] 'What happened when you did this?'
[24:54] Or 'What were you trying to avoid
[24:55] 'by having this architectural pattern?'
[24:59] And that's kind of like, I don't like whiteboarding
[25:02] code necessarily on the whiteboard
[25:03] but I do like drawing systems diagram.
[25:05] It is super fun.
[25:06] And so just having them write on the whiteboard,
[25:08] hey you know this service talks to this one,
[25:10] this service uses this database.
[25:11] Here's where was cache, here's where we don't,
[25:13] here's the trade-offs involved, this like that.
[25:14] That's what I would look for.
[25:18] Now I will take a minute to talk
[25:20] a little bit about bias, right?
[25:23] People have asked me, "Doesn't this introduce
[25:25] "a lot bias now because not everyone
[25:28] "gets the same interview.
[25:29] "Some people will be interviewed in
[25:30] "this particular way and these people
[25:32] "will be interviewed in this other way."
[25:35] And that's a good question.
[25:37] There's two aspects to it.
[25:38] One, this already happens anyway.
[25:40] Even though we say everyone gets the same interview,
[25:42] not everyone gets the same interview.
[25:44] People are asked different interview questions,
[25:46] people have different standards,
[25:47] people have different experiences.
[25:48] So you're already getting a lot of human volatility
[25:52] depending on who is on your interview loop.
[25:55] And further, I think, just because
[25:57] everything is the same, doesn't mean it's correct, right?
[26:01] Again, the test can be wrong.
[26:03] And I think that understanding that uniformity
[26:06] is not the same thing as being unbiased.
[26:07] You can have systemic bias and we do.
[26:11] This is, I think, present in all parts of the
[26:13] tech community and in human organizations at large.
[26:16] So this notion that just because you
[26:20] don't give everyone the same interview,
[26:22] you're somehow privileging some people and not others.
[26:26] And again, I think it's the opposite.
[26:28] If you don't flex to someone's interview style,
[26:30] if you don't help them to shine and to make clear
[26:36] where they're really good and where they're not.
[26:37] I think this is where we do disservices.
[26:39] This is where we exclude people by not
[26:41] allowing them to demonstrate what they know.
[26:43] Because everyone gets the same test
[26:44] and you have to pass this test,
[26:46] it has nothing to do with the work.
[26:48] It's like the SAT, right?
[26:49] The SAT measures absolutely nothing,
[26:51] other than are you good at the SAT, right?
[26:54] But it's a proxy for college admissions.
[26:55] Which then if you get your four year degree
[26:57] and then you go, you can say,
[26:58] "Hey I have a four year degree in computer science."
[27:00] And really all that means is you
[27:01] did well on the standardized test
[27:02] and maybe your parents have enough
[27:03] money to send you somewhere nice, right?
[27:05] It has nothing to do with your ability.
[27:08] So, the major takeaways, I think,
[27:12] are knowing who you want and what you're
[27:15] looking for months before your interview,
[27:18] when you get to that job description.
[27:21] Trying out what I call made-to-measure interviews.
[27:23] Again, it's not bespoke.
[27:24] Not everyone gets their own completely
[27:27] personalized interview but they get
[27:29] to select or you can help flex, you know?
[27:31] And give them the modules that
[27:33] are gonna be valuable to them.
[27:35] And again, allow them to say,
[27:37] "Hey, I'm really good at pairing.
[27:38] "I'm really good at refactoring.
[27:39] "These are the things I'd like to be tested on."
[27:43] Looking for strengths and not for weaknesses.
[27:45] We don't spend a lot of time with our own peers
[27:48] saying, "Hey I think that you're
[27:50] "super bad at XYZ and that bothers me."
[27:51] Right?
[27:53] We usually say, "I know that we need to do XYZ
[27:55] "and this person on this team has
[27:56] "a deep expertise and I'd like to find him."
[27:58] So, starting from the get-go and saying,
[28:00] this is what we're looking for
[28:03] and allowing people to demonstrate
[28:04] that competency is critical to having
[28:06] an inclusive and a positive interview pipeline.
[28:10] Finally, the interview fails the interviewee
[28:13] when that interview does not anticipate people like them.
[28:17] When someone has designed a loop and it's for
[28:20] people with a certain amount of privilege,
[28:21] who went through a certain set of programs,
[28:23] with a certain amount of experience.
[28:25] And this is just the way things are
[28:26] and everyone else is kind of tough luck,
[28:27] you have to figure out some way to learn enough of this.
[28:31] And you go and you buy a Cracking the Code Interview,
[28:32] you buy Programming Interviews Exposed,
[28:34] or things like that.
[28:36] This is truly, I think, toxic.
[28:39] I think it gets us to a place,
[28:41] not only where interviews feel bad.
[28:43] As I mentioned in my earlier story,
[28:45] but you don't get a good signal, right?
[28:47] I have told stories where people
[28:50] didn't do super well in the interview loop,
[28:51] but I knew they were capable, I'd worked with them before.
[28:53] Or they came back and did a second round
[28:55] and were asked different things,
[28:56] and they did completely differently.
[28:58] I know people who have been hired
[29:00] in an organization who crushed the interview loop,
[29:02] and were unsuccessful in the role.
[29:05] And so that tells us that again,
[29:07] the test is not meaningful if the
[29:09] test is not testing what it's supposed to be.
[29:12] Anyway, I am very happy to talk for
[29:14] 40 minutes forever, and ever, and ever.
[29:17] I have an unlimited amount of language
[29:20] but a limited amount of wisdom.
[29:22] So, that is all I've got.
[29:24] Thank you so much for coming to this talk.
[29:27] (audience applauding)
[29:33] (upbeat cheerful music)
[29:38] (squeaking)
[29:39] (upbeat cheerful music)