In 2018, I've have spent more time interviewing than I have ever done since 2004.
I went through the entire interview loop at five companies in total. At two of those places, I had two cycles of interviews as I got referred sideways by the original interviewers. Including the recon visits, lunches and phone screens, I've spent about two whole work weeks of my 2018 talking to recruiters, managers, engineers, architects and directors.
One half-hearted offer (pay-cut included) and one golden one (wow) later, I'm still working for Hortonworks till it doesn't. To make sense of this, I'm trying to distill those two weeks into something that can be bottled for the top shelf.
First up, the Bay Area is special and you are not.
Silicon valley pulls in technical talent from across the planet. It might be the most expensive place to hire people, but the companies I interviewed at can afford it. If you are looking for a low-level systems engineer working on performance problems who understands distributed systems, you might be looking a pool of high hundreds. Rejecting an almost perfect candidate isn't as much of a problem here, because give it a couple of weeks and the recruiter will dig up another prospect locally or at least, find someone who wants to move here.
From that perspective, there's no reason to hire someone like me to work on a CDN route optimization or large scale object store, when they can just snipe people burning out at Amazon or tired of being not promoted at Google. There's no need to find someone who will learn things quickly or grow - you can pick out people who've spent years completely conquering their niche and employ them for two years, four tops.
In short, if you want to do something new, amazing and interesting, find a startup, cut your pay to nothing and unbalance work for life - don't come looking at big company to take a bet on you, go to a VC or someone beholden to them. And there's nothing wrong with that approach, just that it is very different from the tech bubble over in India.
Second, I've got "advanced impostor syndrome". I've got enough knowledge to make myself dangerous, but not enough to be a renaissance man.
Performance and debugging are really wide fields where you spread yourself thin, except where you go way too deep. There's nobody out there who can know all of what's necessary to do that work and if they do, they're out of date in six months anyway. Actually you don't need to know it all, but you have to know enough to guide your search to narrow down a symptom to a problem. The real skill set is using your intuition to ask better questions and finding ways to test for those questions, temporarily hold mental models to work out what's happening in the time dimension across multiple layers of user code, virtual machines, system libraries, kernels and hardware.
At this point in my life, I probably know three things about everything anyone could ask me in an interview, but to the real expert in the field, that is good but never enough. I'm full of anecdotes about how a particular standard solution to a problem doesn't actually work, because there are other considerations which mess with some assumptions hiding in it. And then a lot of anecdotes about how theoretically impractical problems do have actual solutions, but only at the scale of the current use-case. And then some more about how approximations do work better, because they work well enough to be an answer in the real world (yeah, who care about a 0.001 pixel difference?).
My point being that the war story chest doesn't make me more employable, but to me, those stories represent instances where I learn new things. A small hint that learning didn't stop the day I graduated and I walked out with a clear recollection of all binary tree algorithms anyone would ever need for leetcode. If next week someone threw a bunch of Rust code at me and asked me to fix it, I'll find a way, but not in 45 minutes and probably not with code written on a whiteboard. However, as I learned that is not relevant to finding a job in Silicon Valley, at least not at places I interviewed.
In that perspective, my choices have turned me into an inveterate beginner. Being able to learn, absorb and get to answers on something completely new to me is exactly what my current job demands from me. And that means my skill sets are starting to leave the realm of classification or specialization. For anyone trying to box me in, I don't quite fill a box and I straddle too many.
That makes me feel like I don't belong - an impostor syndrome, but an advanced one.
--A company of wolves, is better than a company of wolves in sheep's clothing.
-- Tony Liccione