Software Engineering Interviewing

Foreword

In August 2020, and for the first time in my career, I made the decision to leave a job without having another one lined up. This was a highly unusual decision for me: all my previous moves had been planned and I always waited to have an offer in hand before giving notice. I’d previously seen friends and coworkers make the same decision and I felt uneasy about it, primarily for material reasons. I tend to be conservative when it comes to managing my personal finances and I don’t like the idea of suddenly cutting my main source of income without knowing when –or how– it would come back.

But for the most part, I’d seen this approach work, occasionally leading to great results. From experience, I knew interviewing with other companies while still on the job can quickly lead to feeling like you’re living a double life (and that the second life feels like no less work than the first one). The idea of having an open schedule to research, prep and to interview was alluring. Over time, I had warmed up to the idea that this might be something I could do myself.

Considering the uncertainty caused by the pandemic and by the upcoming US election, having to look for a new gig in the second half of 2020 wasn’t really a position I wanted to find myself in. Along with a few others, two factors principally contributed to my decision to quit before figuring out what would be next:

  1. I was simply not happy with my position at Lyft and the environment that the company fostered: the pandemic hit the ride-sharing industry very hard, morale was low and my team suffered from it
  2. My team was on the verge of embarking in a large long-term project and interviewing in parallel of that would mean not being able to properly honor these commitments

Once I was out of a job, I first took a bit of time off and then worked to get myself back into interviewing shape. This post unpacks my experience going through the interviewing wringer with about a dozen companies over the course of two months. I can’t say I was too surprised by what I found, but being able to compare a handful of companies over a relatively short time span painted an informative picture of what interviewing is like these days.

Acknowledgments

I have to acknowledge that the decision to quit my job bore less risk for me than for most. I have enough savings to stop working for a few months without being hit too hard, the employment market for software engineers looked strong despite the pandemic, and my spouse’s benefits allowed me to retain medical coverage. I don’t think I would have been in the position to do so had any of these conditions not been met.

I would like to credit Carlin Eng for inspiring me to write such a post. He went through a short halt in his career after leaving Strava and wrote an excellent article that unpacks how the experience went for him. I hope my post can serve as a slightly different take on the same topic: I wanted to go a little deeper than Carlin did on the topics of preparing for interviewing and what the final stages of the process can look like.

I also want to clarify that in spite of explaining my approach for engaging with interviews, this post is not an endorsement of the interviewing process itself. Our industry has, in my opinion, consistently failed at developing a method to vet candidates which is both effective and respectful of people. What I mean by “effective” here is: scalable in the moment and able to accurately predict the actual performance of engineers on the job. Google itself acknowledged that, in spite of spending tens of millions of dollars every year on moving candidates through the pipeline, stellar candidates can turn into very poor engineers. And we don’t hear of poor candidates that would have made great engineers—they were by definition left behind.

What I’m getting to here is that companies cynically focus on the process piece ("Does this candidate meet $BAR?") while candidates care about outcomes ("I’m a good engineer, I should be given an offer"). So much frustration stems from that disconnect. It took me about a decade to wrap my head around it and this post contains some of what I learned along the way. But I want to be clear: I’m playing this game in spite of its core premise being broken to some degree.

As a final disclaimer: what is written below is my personal take and experience on the interviewing and hiring process. One size doesn’t fit all and your mileage may vary. There is no single way to prepare for coding interviews, or to navigate a design a question, or to consider an offer. The approaches and points of view I’m sharing here are the ones I hold true, either because they stem from core beliefs or because I’ve seen them work as a candidate or interviewer. I believe there’s value in making the process less opaque, but please keep a healthy skepticism of what is written below.

On Hiring

By the time it is actively hiring, a company has entered a failure mode: there are needs in the face of which the current staff is unable to scale. In that light, I reason about the hiring process as a method to fail as gracefully as possible: just like a normal incident response, the process needs to be tight, coherent, snappy, and transparent. I began realizing the importance of building a good hiring process since I worked in a startup whose throughput depended on getting the right people. I carried it with me in my following gigs, and that’s something I pay a good amount of attention to, both as an interviewer and as a candidate.

For software engineering candidates, interviewing is vastly different task than what they are expected to do on their day job. You’d never be expected to have to code yourself out of a never-seen-before problem and ship it without writing any tests in under 20 minutes. You also wouldn’t crack out a full-fledged design for a scalable distributed system without doing research or experimentation to inform your proposal. Yet somehow, these are the skills you are expected to demonstrate during interviews.

What I’m getting to here is that neither side of the hiring process is effortless: each party should put some work into preparing for it. Part of it is theater, part of it is contrived, but being good at it doesn’t come naturally—at least not to me.

Preparation

Coding

Language

An important aspect of going into code interviews is to have some proficiency in a programming language. From my experience conducting interviews, I would actively discourage anyone from using a compiled language. In coding interviews, time is of the essence: running your code is something you will want to do very early and very often.

Compiled languages not only tend to be more verbose, time is also lost with the various type checking and assembly steps that the compiler needs to do every time you want to run the code. Scripting languages are definitely the way to go. A usual favorite is Python, but I myself have developed a strong personal preference for Ruby.

Ruby is full of one-liners and idioms that, once mastered, become very useful shortcuts to use during an interview:

['fig', 'pear', 'apple', 'flea', 'kea', 'rhea'].reject do |word|
    word.length <= 3
end.sort do |a, b| 
    a.length != b.length ? a.length <=> b.length : a <=> b
end.first

I wouldn’t recommend learning a new language just for the sake of interviewing; my own proficiency in Ruby stems from 4+ years writing production-grade code at Strava. But if you are on the fence, I do believe Ruby to be a great language for coding interviews.

On brainteasers

As Carlin pointed out in his post, Cracking the Coding Interview(CtCI) is a useful book to refer yourself to ahead of the actual interviews. But I also do not believe there is a ton of value trying to implement or memorize the solution to the hundreds of problems contained in the book. Some trivial problems found in the early pages can be useful to get yourself back into a mode where you have to deliver a working solution quickly and then fiddle with it with an extra constraint.

Other more complex ones may be getting frankly into the brainteaser territory, which, as far as I’m concerned, are not worth preparing for. What I qualify as a brainteaser is a problem that meets the following criteria:

  1. there is an optimal solution that doesn’t lend itself to practical trade-offs
  2. getting to the solution requires an insight or directed hints from the interviewer

The fewer hints you get, the better engineer you’re supposed to be. But if you Googled the question, you would likely find the one true solution in the first page of results.

It’s a bad sign if an interviewer is assessing you using problems that can be solved through Google — maybe it means candidates that passed are brilliant problem solvers who can think on their feet, or they might have memorized the solutions when they inevitably ran into the problem on Leetcode. What I’m looking for in a good coding problem is one of two things:

Brainteasers just don’t make the cut.

What I’m getting to is that if you pass a brainteaser interview, you’re going to end up working with other people who have met the same bar. It might work out in the end — and historically, you might argue that it has worked for companies the likes of FAANGs. My point here is that the very questions you are asked during your interview offer an insight into how the company selects candidates, and what is valued in terms of skills.

Good programming interview questions are few and far apart — companies that spend time crafting a good bank of questions have an edge over those that don’t. Going off of stuff you find on Leetcode is lazy and provides a weak signal. For candidates, this should be a flag that company isn’t taking hiring seriously.

Algorithms and data structures

As far as preparing to answer coding problems, I forced myself to re-implement a number of commonly used data structures: hash tables, binary search trees, tries, linked lists, heaps, etc… This is the kind of stuff that you would normally always pull out of a standard library but going through those basics provided a playground to brush up my Ruby programming and a refresher on complexities. Once you have these, you can then work on higher-level structures and algorithms: expiring caches, traversal algorithms, bloom filters, tree re-balancing.

I then spent time working through abstractions such as iterators, comparators, streams, and, lastly, sorting algorithms. I did go through some Leetcode-like problems —brainteasers, to call them what they are— but I felt like they were less important to my preparation than most other things I decided to do on my own.

Non-technical skills

One value of CtCI is the preparation it forces you to do for non-technical questions. I’m not going to disclose the exact contents of the book but it fundamentally parallels the STAR framework:

I came ready to discuss these things for 3 to 4 distinct projects or programs I personally worked on. To this, I also added a few follow-up questions that I’d be asking myself if I was the interviewer:

Design

100% of the design questions I’ve worked on during my month of interviewing were distributed systems problems, which naturally lend themselves to trade-offs between performance, reliability and correctness. The way I went about preparing for this was two-fold:

  1. Going over my own previous designs documents and working through the architecture and the decisions I made at the time. It’s helpful to look at those with a critical eye and attempt to find faults with certain approaches I took.
  2. Reading up articles / explainers on the implementation of key pieces of software:
    • MySQL indexing and replication
    • Kafka (1, 2)
    • Envoy’s life of a request and support for load balancing
    • Redis
    • Cassandra
    • TCP — which might seem like a peculiar recommendation, but this is the base protocol that powers the Internet. It is not only fundamental but also packs a lot of good features that are often overlooked and re-implemented at a higher level of the stack (flow control! congestion control!).

Keep in mind that, ultimately, very, very few engineers in the world actually design and develop scalable distributed systems from scratch. Part of an engineer’s job is to know how to piece things together in order to meet the stated goals of the system being built. Knowing what those pieces are, and knowing how they work and what their limits are, that’s the thing you want to be able to demonstrate during a design interview.

Basically, it’s unlikely that an interviewer will grill a candidate on their full understanding of Paxos, but it’s good for the candidate to know how a leader election happens. A design interview is just as much an assessment of communication skills as it is of technical skills.

Process

My personal approach to answering design questions is to let my stream of consciousness take over: I openly share the thoughts that come to me, or the struggles I have about an approach. This gives opportunities for the interviewer to intervene, object, assist, or clarify a requirement. I’ve generally followed a 5-or-so step process:

  1. Ask clarifying questions, or, alternatively, make statements that clarify the problem at hand: “it seems to be a read-heavy system, so I’ll make sure to to center my design around that”. This is especially important for parts that appear trivial to you, since it’ll help the interviewer know that you are thinking this through and not glossing over things.
  2. Top-down first: decompose the problem in large chunks that each have a coherent role, e.g. “I’ll need a queue, a load balancer and some storage”. Establish the scale for each of those, the interfaces and the invariants that need to remain true between them. If no obvious architectural choices come to you, just talk out loud about your thinking process.
  3. Bottom-up second: run through the set of components you have identified and make implementation proposals for each of them. Explain out loud what your thinking is and the rationale for those decisions.
  4. Do a run through of the entire system once and see if the goals are met. Identify bottlenecks or failure modes. Discuss how you would counter them or solve for them. Sometimes, the interviewer will challenge the design or change a parameter that feels like the rug is being pulled from under your feet. This can make you feel like you design suddenly has to cover more ground but it is generally an invitation to discuss trade-offs: no real-world system is perfect, and your design cannot be perfect either.
  5. Open things up: if you were writing a true technical specification, there would be a ton of other considerations you’d need to cover: ease of operations, observability, configurability, security, resilience, deployment, etc… If you have the time, talking about those can yield a solid conversation to wrap things up and convey that you care about more than the base design.

Company Selection

I didn’t really have a positive filter for the profile of companies I considered. I did however prune out companies and domains early on: this is ultimately a fairly personal choice but it might explain how I ended up with the final slate of companies.

Other than that, I tend to favor companies whose product I use every day, e.g. the ones whose app has a place on my phone’s home screen. I made myself available on LinkedIn and began looking for postings on the websites of said companies.

Timeline

I overall engaged with a total of 12 companies, across a 10-week period.

For the companies I directly applied to, I was intent to write a cover letter — even though I had the intuition they would be dismissed or discarded by screeners, it was a forcing function for me to investigate the company, to dig into job description and write down why I think I’d be a match for the role. This was more for my benefit than for theirs.

In total, I was concurrently pursuing 5 leads throughout the month of October. Since I don’t feel like there is value in naming them, I will from this point on refer to them using greek letters: α, β, γ, δ, ε. And since my initial selection was fairly open, I ended up with a fairly varied set of company profiles: headcounts ranging from a couple dozen of employees to many thousands, valuations going from $100M to many tens of billions, and a healthy mix of private and public companies.

SepOctNovApplicationIntroductory call(s)Phone screenOnsite(s)Final / OfferGithubMon Aug 24 2020 - AppliedMon Aug 31 2020 - RejectedNetlifyFri Aug 28 2020 - AppliedThu Sep 17 2020 - Followed-upδFri Aug 28 2020 - Sourcer outreachFri Sep 04 2020 - Second Sourcer outreachMon Sep 14 2020 - ResponseTue Sep 15 2020 - First intro callTue Sep 29 2020 - Phone screenTue Oct 13 2020 - OnsiteThu Oct 15 2020 - FeedbackMon Oct 19 2020 - Team fitFri Oct 30 2020 - Team fit follow-upMon Nov 02 2020 - OfferTue Nov 03 2020 - Offer follow-upγMon Sep 14 2020 - AppliedFri Oct 02 2020 - Email ResponseMon Oct 05 2020 - First CallThu Oct 15 2020 - Phone ScreenFri Oct 16 2020 - FeedbackTue Oct 27 2020 - OnsiteFri Oct 30 2020 - FeedbackMon Nov 02 2020 - Reference CheckTue Nov 03 2020 - Team fit callsWed Nov 04 2020 - OfferGitLabTue Sep 15 2020 - AppliedαThu Sep 17 2020 - Sourcer outreachWed Sep 23 2020 - Response from meMon Sep 28 2020 - Intro call with recruiterThu Oct 01 2020 - Intro call with hiring managerWed Oct 07 2020 - Coding challengeTue Oct 13 2020 - Feedback on coding challengeFri Oct 16 2020 - Onsite prep callWed Oct 21 2020 - Day of onsite interviewsThu Oct 22 2020 - Feddback / RejectedLocal LaboratoryFri Sep 18 2020 - AppliedThu Oct 01 2020 - ResponseThu Oct 08 2020 - CallFri Oct 09 2020 - No fit1PasswordTue Sep 22 2020 - AppliedεWed Sep 23 2020 - AppliedThu Oct 08 2020 - Email ResponseWed Oct 14 2020 - First CallTue Oct 20 2020 - Phone Screen + FeedbackMon Oct 26 2020 - OnsiteWed Oct 28 2020 - OnsiteThu Oct 29 2020 - FeedbackFri Oct 30 2020 - Team fit & OfferSubstackFri Sep 25 2020 - AppliedMon Oct 26 2020 - ResponseThu Oct 29 2020 - Intro callβFri Sep 25 2020 - Sourcer outreachTue Sep 29 2020 - First intro callMon Oct 05 2020 - OnsiteWed Oct 07 2020 - FeedbackFri Oct 09 2020 - Follow-upFri Oct 23 2020 - Pair programmingMon Oct 26 2020 - Reference sentFri Oct 30 2020 - OfferTue Nov 03 2020 - CEO callSlackSun Oct 11 2020 - ReferredTue Oct 13 2020 - First intro call

Beyond searching for a job, I spent the rest of my days working on personal projects, playing games, running errands, going on runs or on bike rides. Following Carlin’s example, I tried to set aside a day a week to go outside on a ride and empty my head. In spite of this, I didn’t feel quite enough in control of the whole process to really be at ease.

Some companies I was excited about were entirely non-responsive and I had to make peace with that. The process dragged on with certain companies while others were really snappy. Offers that came with deadlines meant having to rush things — which is good to an extent, but also incurs a higher level of stress.

The end-to-end timeline ran from the tail-end of August to the beginning of November (actually, election week 😬). I kicked it off with a couple of companies which I applied to in the couple of weeks right after leaving Lyft. I resumed applying in mid-September and sent applications or responded to a handful of companies each week.

In addition to sending my resume around, I started going through my interview training program. There wasn’t a ton of responses at first but things warmed up once October came around.

Candidate Experience

Overall, the interviewing process went pretty well across the board, with the notable exception of one company where the day of onsite went poorly: company α.

Interview formats

I was happily surprised to see new interview formats introduced since the last time I went though the process:

Diversity

I identify as the absolute dominant profile in the north american software industry: white, cisgender, straight, and male. I’ve also been on the organizing side of DEI initiatives in companies and products I worked on. I wanted to pay attention to this as I was going through my interview process and it’s no surprise: no company is really doing this well.

As I went through the “onsites”, I tried to keep a running tally of the profiles of people who were interviewing me: their role and seniority, but also their apparent gender and ethnicity. A major caveat here is that I never explicitly asked my interviewers how they identify in terms of gender identity or racial background, which means the data presented below is tainted by my own perception and bias.

I was interviewed by 35 people in total, and the panels were all between 6 and 8 people. I’ve opted against presenting a per-company breakdown in this post because I believe the samples are too small to be meaningful. As a whole, however, the data paints a relevant picture:

  1. Bar for one, all my panels had at least one female interviewer. Two thirds of my interviewers were male.
  2. All panels had at at least one non-white person, sometimes just a single one. Two of the five companies had people of color in positions of leadership. Only one black person interviewed me.
  3. One interviewer shared that they were involved in their LGBT ERG

Two caveats are worth mentioning: in some cases, a single interviewer was wearing multiple of those hats, and sometimes those interviewers were in non-technical roles (e.g. designer or product manager). The numbers would be grimmer if I focused solely on engineering roles.

Similarly to the questions they chose to ask, the makeup of an interview panel can clue candidates as to what is valued at the company. Three of the five companies had me interview with non-engineers, and another one had three of my onsite interviews grill me extensively about cross-functional collaboration and project management. This is great! Companies should absolutely vet an engineer’s experience working with less technical partners.

Notably, company α’s panel was solely comprised of men, all of them engineers. I was only asked technical questions, a few of these being brainteasers. In the moment, I was honestly frustrated that I didn’t do better. In hindsight, and despite the hiring manager’s reassurance that the company fostered a very healthy culture, I didn’t have a ton of regret about flunking.

Logistics

The pandemic threw a wrench in many processes, and the interviewing one is no exception. By the time I started to interview, I had worked from home since the sheltering in place had begun almost six months prior. For companies, onsites are an opportunity to showcase their offices and perks. Meeting some prospective teammates in person and casually chatting over lunch is part of the experience as well.

All of that is gone in 2020. It makes for a less appealing day, but also requires a bit more preparation for candidates — things that coordinators would normally be in charge of. Here’s what was helpful for me:

Companies also need to change their ways — notably, it becomes harder for candidates and interviewers to exchange non-verbal cues. This means communication skills become increasingly important for both. The one notable negative experience I had was a design interview where both interviewers kept their webcams turned off, which obviously made the interview a lot harder for me. Putting candidates at ease by doing proper introduction also becomes more important, and increases the chances that candidates will perform their best. Interviewers that cut the introduction short without expressing the slightest interest in what I had to say were a clear red flag for me.

Offers

Comparison

It’s already hard enough to precisely know what lies within a single offer, and the fit for the role and with the folks also plays a huge part in a decision that is deeply personal. In my case, 4 of the 5 companies I interviewed with ended up extending an offer. The basic components are still the same: base pay, equity, bonus and benefits. There’s no tried and true method for comparing those uniformly, especially as the cost of equity can differ significantly:

Equity compensation is a complex, potentially thorny problem which warrants knowing what you get into before you sign a contract. The Holloway guide to equity compensation is a free, comprehensive document about equity compensation that I cannot recommend enough.

Below is the framework I came up with to compare offers:

Label Yearly Base Equity Bonus Benefits
Value Type Cost Sign-on Annual Parental Leave 401K Matching PTO
β Bβ = .802 ⨉ B̅ 5.615 ⨉ Bβ ISO .071 ⨉ Bβ No No N/A No Untracked
γ Bγ = 1.054 ⨉ B̅ 4.671 ⨉ Bγ RSU 0 .144 ⨉ Bγ .200 ⨉ Bγ 16 weeks 50% of $2,000 21 days
δ Bδ = .997 ⨉ B̅ 2.401 ⨉ Bδ RSU 0 .174 ⨉ Bδ 🚧 16 weeks No Untracked
ε Bε = 1.148 ⨉ B̅ 2.755 ⨉ Bε ISO .187 ⨉ Bε .094 ⨉ Bε No 16 weeks 100% of $3,000,
then 25%
Untracked

Equity

My philosophy in terms of equity is two-fold:

Equipped with this framework, let’s look at company ε: the base pay is the highest of the lot but 18.7% of it needs to go into purchasing equity each year, which is also quite high. In the case of company γ, the equity more than doubles the base pay year after year, which comes at no direct cost except for the associated tax burden, which is high because RSUs are taxed as income.

One offer should clearly stand out in the table above: company β is a small, early-stage company. The quoted base (which was presented as non-negotiable) was lower than my previous one at Lyft, and it significantly drags down the average presented in the table. To make up for it, the cost of acquiring the equity is fairly low (7.1% of the base) and the stock has a lot of room to grow since the company is so young. This company also has no sign-on, no bonus structure and the parental leave is unspecified at this time.

Benefits

Benefits are where comparing offers gets a lot harder, primarily because there’s so much ground to cover and what matters to people is so personal. Some companies offer stipends for gym memberships, internet access reimbursement, lump sums for buying office equipment, etc… Health coverage can be such a personal decision that I’m not even going to begin trying to pitch policies against one another. I focused on three aspects of benefit packages:

Role fit

While the compensation package is definitely a factor, I’ve primarily made my decision based on how excited I was about the role. As can be seen in the timeline, company δ took well over two weeks to extend an offer after giving me the thumbs up. It was apparent that it took time for the hiring team to come up with a compensation package, which was not a good sign: compensation packages should be an easy thing for a company to figure out and companies should strive to work on the candidate’s schedule.

The other reason for this delay is that I had several follow-up conversations with prospective managers in order to determine team fit. Even though I originally applied for a TLM position, it was clear that the expected people management aspect of that role was going to be too heavy for my taste. I was oriented towards a different team and talked with another manager, then with an engineer on that team, until another switch was made to have me report to a higher-level manager at the very last minute of the process.

It also became evident that the organization and reporting structure was a work in progress, which made it hard to have a consistent conversation about the role, the context of the work and expectations. In contrast, the other companies I interviewed with were clearer as to what they were looking for, and they crafted the process around that goal. From my perspective, the exchanges with those companies were more consistent and productive.

Parting Notes

Upon making the decision to quit, I knew I would need to keep an open mind about the path this job search would lead me on. I’d never been in a situation where I interviewed with so many companies at once, which turned out to be a fairly involved process. It required some level of work pretty much every day over the course of two months. This is about the average workload I originally anticipated, but I don’t think I expected how fragmented this time would be, and how unevenly distributed it would be.

I try not to let it get to me too much, but I was raised with the idea that work —and, by extension, employment— serves not just a social purpose but also a personal one. In other words, labor is expected to fulfill individuals and idleness is good only in small quantities. As such, the decision to go into unemployment voluntarily and for an unknown length of time didn’t only have financial or logistical consequences. To an extent, having a day job sort of alleviates the pressure of having to answer the age-old question: “Am I doing the right thing with my time?”.

I felt varying degrees of guilt from the moment I last closed my Lyft laptop and until my signature was at the bottom of an offer letter. This feeling was only reinforced by the fact that my spouse was in the room next door, working her ass off every day.

There was also a broader context to this: on top of a pandemic that had brutally changed our day-to-day for over half a year, the US election was set to be a very disputed one, with more in the balance than any other in my lifetime. Working through a deeply personal process at a time where so much was at stake on a global scale felt quite odd — a feeling that was sometimes openly shared by my interviewers.

Outcome

🎉 I joined Stripe’s Payments API team in January of 2021 🎉

Based on this outcome, I can’t look back and say that quitting and making room for job searching was a bad decision: Stripe was on the shortlist of companies I was excited about when I began my search.

But it’s also clear that the whole process didn’t feel great and I can’t say precisely how I would change things if I had do to it again. I’m happy about where I landed but I came out of it spent, and I’m wary that this isn’t avoidable. The guilt I experienced is also something that’s too personal for me to just shake off.

The two takeaways I have from this experience are:

  1. Working with several companies in parallel made me more empowered to ask questions that got me a better grasp of what the company really needed: knowing what will be expected out of my work made me a lot more at ease when signing an offer
  2. Anyone who’s gone through the tech interviewing wringer knows too well that this is a nerve-racking process: multiplying this five-ways makes only makes sense when your time is clear from other distractions

For me, in hindsight, part of the value was in exploring the variety of roles I could see myself in: TLM, staff engineering or founding engineer of an org. It’s one thing to daydream about making such moves and another to have real people trusting you to act in that capacity in real, critical projects they want to ship. I came out of it with a refined sense of what I like to do and a chance to match that with a company’s needs. Which is about as graceful as a failure can be.