Almost exactly to a year ago the merger between Cloudera and Hortonworks went through.
I've had a fairly inside field view of the process and the rollercoaster has been fun or at least never boring. Along the way, I kept having these four questions.
Four questions which are like an organizational rorschach test. But they all look like butterflies to me.continue reading "Four Questions and No Answers"....
“I would rather have questions that can't be answered than answers that can't be questioned.”
― Richard Feynman
In 2018, I've have spent more time interviewing than I have ever done since 2004.
I went through the entire interview loop at five companies in total. At two of those places, I had two cycles of interviews as I got referred sideways by the original interviewers. Including the recon visits, lunches and phone screens, I've spent about two whole work weeks of my 2018 talking to recruiters, managers, engineers, architects and directors.
One half-hearted offer (pay-cut included) and one golden one (wow) later, I'm still working for Hortonworks till it doesn't. To make sense of this, I'm trying to distill those two weeks into something that can be bottled for the top shelf.
First up, the Bay Area is special and you are not.continue reading "Advanced Impostor Syndrome"....
A company of wolves, is better than a company of wolves in sheep's clothing.
-- Tony Liccione
Often Software (with a capital 'S') is not built, it is grown. And a lot of green digits are involved, not always thumbs.
When I got an education which taught me how to build software, the process emphasized on sitting down in front of my computer, pushing my will upon a barely explicable machine which was built to resemble advanced technology on some days and complete magic on others.
In fact, when I think about it, every part of my computer science education was about imperative commands to a system which, if you give it adequate commands always throws up unsurprising results everytime (and all surprises were bugs). Everything I was taught about this fit that model, where when I had a "Software Management" class it was full of Gantt charts and PERT charts, going over how to schedule time for different engineers to work on the same project in parallel. The recommended reading was The Mythical Man Month and that was hilarious to read, at least. Mostly that class was taught as a way to have an engineer understand what a manager might have to do, not exactly explaining the operational realities.
This post is about the third time I realized I was ill-equipped to deal with the actual demands of being an engineer with the industry. Because none of what I describe works the way I've been taught - at least, when it comes to these skills I'm entirely self-taught.continue reading "A Software Gardener"....
Plants do not grow merely to satisfy ambitions or to fulfill good intentions.
They thrive because someone expended effort on them.
-- Liberty Hyde Bailey
This isn't about leadership, but the events leading up to it. There's nothing new for me to say here, except to walk you through the path I took reading Socrates, Plato and Aristotle.
Though this truly started because I read Men At Arms again and wondered what Corporal Carrot is made of (& Vetinari and Vimes too).
Because reading it made me pick up H2G2 and look at Zaphod Beeblebrox & the man who rules the universe. But that's for a different episode, about presidents with a penchant for orange sashes.
In the Republic, Plato records Socrates's argument about the nature of leadership. That guardians of the state are not necessarily the leaders, but leaders employ themselves where they can show the way and to do that may abdicate power when they observe a well functioning, if not perfect state of affairs.
Weighing between the potential advances they could make from mastery of their art against the heavy burden of leadership, they choose to advance their mastery instead and indicate to others that their mastery is advancing the world.
The world changes shape, right at the moment where those have ignored politics realize that the state of affairs is regressing, despite their progress through life. In those moments of chaos, they consider leaving their avowed profession and take up the unthankful jobs that they had left to others before - leadership. There is no longer a question about comparing the rewards of leadership against anything else, since the alternative is to lose out on the most relevant of common goods - peace.
And I quote
And this is the reason, my dear Thrasymachus, why, as I was just now saying, no one is willing to govern; because no one likes to take in hand the reformation of evils which are not his concern without remuneration. For, in the execution of his work, and in giving his orders to another, the true artist does not regard his own interest, but always that of his subjects; and therefore in order that rulers may be willing to rule, they must be paid in one of three modes of payment: money, or honour, or a penalty for refusing. What do you mean, Socrates? said Glaucon. The first two modes of payment are intelligible enough, but what the penalty is I do not understand, or how a penalty can be a payment. You mean that you do not understand the nature of this payment which to the best men is the great inducement to rule? Of course you know that ambition and avarice are held to be, as indeed they are, a disgrace?
And then Socrates explains.
And for this reason, I said, money and honour have no attraction for them; good men do not wish to be openly demanding payment for governing and so to get the name of hirelings, nor by secretly helping themselves out of the public revenues to get the name of thieves. And not being ambitious they do not care about honour. Wherefore necessity must be laid upon them, and they must be induced to serve from the fear of punishment. And this, as I imagine, is the reason why the forwardness to take office, instead of waiting to be compelled, has been deemed dishonourable.
And Socrates goes onto explain the punishment that awaits a good person who sees a leadership vacuum and does not step up to it.
Now the worst part of the punishment is that he who refuses to rule is liable to be ruled by one who is worse than himself. And the fear of this, as I conceive, induces the good to take office, not because they would, but because they cannot help --not under the idea that they are going to have any benefit or enjoyment themselves, but as a necessity, and because they are not able to commit the task of ruling to any one who is better than themselves, or indeed as good.
And that's how Socrates concludes that leaders are forged out of a crisis, not out of peace or prosperity - not by intention, but by choice and circumstance.
That does paint the progression as sort of inevitable, but it is entirely rational to observe the choice ahead and just leave.
PS: Republic talks about doctors getting paid for good health, women being educated equally and rulers being enlightened - it's hard to think of it being written two millenia ago, while most of that is still fought out. Read the whole thing.--
There is no harm in repeating a good thing.
Money is just misplaced effort.
Most of my waking hours are spent working on things which are not part of my long term goals in life. There is a implied social contract that if I turn my hours into money, there's some way to use it to accomplish those goals much more efficiently than if I worked on the goals on my own in those hours.
I've been thinking about where all I misplace my efforts, but for extremely good reasons.continue reading "Misplaced Efforts"....
The most pitiful among men is he who turns his dreams into silver and gold.
-- Khalil Gibran
Tech interviews are very strange. The whole process makes no sense in general, when it actually comes to being hired to do actual work. This gets worse when you're dealing with a specialization that is all about identifying trade-offs and fitting solutions to them.
But let me explain this with a car analogy.continue reading "The Paradox of Tech Interviews"....
New ideas pass through three periods:
1) It can't be done.
2) It probably can be done, but it's not worth doing.
3) I knew it was a good idea all along!
-- Arthur C Clarke
For the last decade, I feel inconsolably sad about my dad's death and its circumstances, as Dec 2nd draws nearer.
For some reason, this year was different. Perhaps, it is because my mother is here with me - it is the first time in a decade that I've talked to her on Dec 2nd. But it is not just that, there's been a shift, which I barely understood till I sat down to write this.
My sadness has changed over the years. I no longer feel that sadness the same way - it is not a soul sapping feeling of despair, but a different and new emotion, that is still sadness.
I feel sorry that my dad couldn't be here to participate in our lives. He would've loved to sit around and do literally nothing, on this other side of the planet he never saw. I'd have enjoyed sitting with him, drinking tea every few hours and arguing about the legal drama that surrounds today's political news.
For years, I imagined how my life would be better if not his death ... for a brief moment today, it dawned on me how his life would've been better, if he were still here with us.
The burden of my loss has suddenly has turned into empathy for his loss. Because I'm still here, a decade later, finding light in unexpected places.
Because for once, as a parent, I know what I'd lose if I wasn't on this planet when I'm done with my working years. I'm getting ready to spend my final years observing and rambling incessantly.
Actually, I feel old enough start already - the good old days when things were really uphill both ways to school, but we did it anyway - you kids won't understand it anyway, but we had to ask people who was calling when the phone rang and all that, except these days you all ask "hey, where are you ..." instead.
Just smile, listen and nod, because that's what I'm doing for my dad, in memoriam.--
If you don't know where you are going, any road will get you there.
-- Lewis Carroll. "Alice in Wonderland"
I have started to remember more details about problems than I do about code.
Code used to occupy a large amount of my work-time thinking roots - algorithms, data structures and architectural concepts. Over the last two years, the organization of my thoughts has shifted onto understanding parallels with the real life structure of the problems. I'm looking through my attic for my brain and finding new uses for all that I've already done, which makes a better part II.
So for a while I've been putting thought into revisiting ideas, to feed them through this new lens. Here are some of them which I'd like to work towards thinking more clearly about them over 2018. If any of you have one of these problems or would like to enlighten me about how you are solving them, I'd like to buy you a beverage of your choice.continue reading "Problems I Want to Precipitate"....
If you're not part of the solution ...
August is the single month every year, where I go looking for a better job.
Because everytime August comes around, I'm reminded of a very specific day of my life - Aug 11th 2003.
Very clearly a bad day to recall, but first impressions are often very deep.continue reading "Looking for a better job - part I"....
I don't know who you are.
I don't know what you want.
I can tell you I don't have money.
But what I do have are a very particular set of skills,
Skills I have acquired over a very long career.
-- Paraphrasing "Taken" (2008)
We're into the second act, people!
This is a moment of despair, but you know what the second act is. It is a moment of character, where you get to find out if you're part of the narrative or not. Some of you reading are definitely looking away, but this is really for those who are staring out at the world in despair, watching it stare back into you.
Remember what the second act is - it is the rising action. The heroes' attempts to fix the crisis leads to ever worsening situations. And they can't win, because they haven't got the skills to deal with the forces of the other side. They must not only learn how to wield the light saber in attack, but also to believe in the force that flows through.
There's a part of me which wants to unplug the news and go back to where I have a vote. I could've done a runner with a clean conscience, refusing to fund the process with my taxes. But I've got a tiny american with me, who never got to decide what his future looks like. I cannot remain idle in his defence, but I didn't sign up to die trying.
So, I must think of the second act and what it demands.
Bassiano: And I beseech you, Wrest once the law to your authority: To do a great right, do a little wrong, And curb this cruel devil of his will. Portia: It must not be; there is no power in Venice Can alter a decree established: 'Twill be recorded for a precedent, And many an error by the same example Will rush into the state: it cannot be.
In this fight for essential liberties, the rule of law is overturned, we are lost. I hear the words of the lovely Portia from the Merchant of Venice. Thus we must look to the lawyers, to ride in like the cavalry. In Shakespeare's words of "The Butcher" in Henry V, lawyers for sure remember the laws and not easily forget that all animals are created equal.
JACK CADE: I thank you, good people:- there shall be no money; all shall eat and drink on my score; and I will apparel them all in one livery, that they may agree like brothers, and worship me their lord. DICK: The first thing we do, let's kill all the lawyers.
The soap box and ballot box have not helped, let's just hope the next box has got enough stopping power. I'd hate to get to the last box, because down that way madness lies. To learn politics and law, so that my children may learn literature and art is no sacrifice, but the last box leaves us without civilization.
I don't know how this will all end. I'm not anxious anymore though, because there's one thing I worried about - that world would turn away in apathy. Because in the end what we will remember most often will not be the words of your enemies, but the silence of your friends. Do not imagine all is lost, do not mistake being out of power with being in the minority - there are more of us, when the numbers are in. This plot arc is long and we're just into the second act.
And though wise men at their end know dark is right, do not go gently into the good night!--
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.
I'm off facebook, after 9 years of ignoring friend requests.
This was not some voluntary choice to spend less time on useless things.
Facebook in its infinite wisdom, locked me out and demanded my driver's license (or green card, that will also do) to unlock it.
I really wish they'd just delete everything instead of holding it in some deep archive limbo.--
Nobody exists on purpose, nobody belongs anywhere, everybody's gonna die. Come watch TV.
This is perhaps the darkest week of the year - every year since 2006.
A decade later, once again, I feel like nothing about me will be the same again.
This new sense of purpose I feel is a few months old, sleeping a few feet away.
The baton is passed, not quite yet and I haven't let go.--
"Life is effort, and I’ll stop when I die."
-- Rick & Morty
Today's the first day of year 5 for me in Hortonworks.
I've now spent more than a decade getting paid to do open source work. But until Hortonworks, all work I did was tangential to the business arm of the company. All my customers were internal, which meant mostly I got privileged access to the problems. The bug resolution for complex issues never had to deal with walls of customer solutions obscuring my insights into the system. There was never a day when I couldn't tell someone from Yahoo mail that I was working on another issue with Frontpage.
During my Zynga period, I learnt how to sell your ideas to a product focus team looking for solutions - to explain better, to justify my arguments with numbers, to convince and to persuade. To keep my sanity, I also learnt to pick my battles, drop issues and walk away when the friction gets personal.
I never had to sell to a customer direct - even at my best, on the books of the company I was a cost, an engineering expense. My work at Zynga or Yahoo, was helping the company bring down its bottom line by a fraction without ever touching the top line, except by accident (yay, faster load times).
This is generally an area of work with no real conflicts of interest built into it - use of APC for instance was mandated and in most cases, my code was the only available solution to the problem. Giving up on APC meant switching to HipHop, which also I worked on (never got the CLA signed by Zynga, which was sad though). The pattern followed me through out the period even through ZCloud build-out, through ZPerfmon, ZProf and pretty much everything I touched.
Hortonworks has been a completely different experience. I'm still a developer, but my contribution to the company has turned into a top-line enlarging investment in most scenarios. The work I was doing with Stinger was actively bringing in more customers and indirectly, more revenue in. This brings in some interesting conflicts for a developer, when talking to a customer about your features, because that particular transaction starts off as a zero sum exchange of money with the implicit declaration that the "solution" is a non-zero sum win-win.
This shift in the nature of my job has been interesting. The role now requires a completely different social skill set which I didn't need in any job before this. The goal is nearly the same, but conflicts fall onto my lap instead of being able to just say "Zynga first" to end a discussion.
Honestly, it's been a bit of a culture shock.continue reading "Hortonworks: Day 1, Year 5"....
Boredom is the feeling that everything is a waste of time; serenity, that nothing is.
-- Thomas Szasz
My grandfather passed away last week.
When I was at one of life's cross-roads, he rescued me with some advice I hold onto as one of my life's truths.
Sacrifice. It's a word that gets thrown around in the social context without any inherent meaning. Just like other things caused by peer pressure, the sacrificial act is a social demonstration of emotion, at the detriment of nearly everyone intimately involved in it. Mothers give up their careers to stay at home. fathers end up working longer and so on.
After my dad's untimely end, there was a choice left to me - envelope pushed under the door of my mourning, if you will. Move back to Trivandrum and try to support my mother & sister while they mourn the inconsolable loss. Social signals told me clearly that it's what's expected from a "good son".
I thought otherwise. I was wracked with guilt at the thought of staying in Bangalore, chasing my own life's goals which had been on hold for the couple of years my father was ill. And I couldn't bring myself to ignore the social pressures and was caving in to them, when I talked to my muthachan.
He got to right to the point.
"Is this something your mother asked you to do?" / "No". "Do you think leaving Yahoo and moving to Trivandrum is good for you?" / "No". "Is moving back home likely to be a disappointing thing for you?" / "Yes".
"Then don't do it. We're here to take care of your mother. She is my child." / "Ok, but ..."
"Your mother will need you in a few years, your sister will need you. Be there. But you will do this one crazy gesture they never asked for, because people pressure you into making it. When times are bad and your life isn't moving forward, you won't think of those pressures. You will blame the ones you really did it all for."
"And one day, you will get angry about something and tell your mother that you did it all for her. That your life's frustrations are due to her. Maybe it won't be your mother and it will be your sister - she will tell you that she didn't ask for it, that you did it all on your own. That your sacrifice wasn't necessary and then you'll feel hollow, because it was all for nothing, how you turned away from your self, silenced your wants and died a little for them. You will be truly angry with them for your thoughtless sacrifice and never let it go."
"If what I said sounds inevitable. Don't do it. Go do your thing now and whenever your mother or your sister need help they will not hesitate to ask. Be there for them without question. When it's the right decision to take, you won't feel yourself tearing apart."
That is how I got to today, half a world away mourning his death, knowing fully well I couldn't go & be by my mother's side.
And this time, I wasn't tearing myself apart.--
Much of what was said did not matter, and that much of what mattered could not be said.
We'll see what happens. Yes, but ... OK, I will try.
There are so many ways to say No. Some of you might've realized that's what gets said. But I've seen very few who understand why it gets said in circuitous soft words, other than attributing it to race or upbringing.
Because of what happens when you say No. There's always someone else who's waiting to say Yes. Time after time, each No you say shrinks the number of times you get asked and life's opportunities show up as questions. The fewer times you're approached with a question, the fewer times will you get to say Yes!. Each No shrinks the funnel through which other people interact with you.
Being reasonable produces the opposite result you want. Those that say "Yes, let's do this!" and flake out are often asked more often than those who can offer a negative in time. In fact that those who say "No, I'm meeting a friend tomorrow" only to have their friend flake out is worse off than someone with double bookings.
Whether the opportunities are social, personal or professional, those that a "Yes, we can try" are preferred to the respectful No. Of course, we prefer the "Can Do" people here.
The definite negative implies intention, while the loosely held hope is always treated better. The negative is interpreted as sub-textual while the other is taken as a circumstantial "stuff happens" even when the eventual end result is the same - probably worse for the "maybe" because it steals away the ability to plan an alternative.
In such an environment where all Yes-es are Maybe, being "Definitely Maybe" is understood well - and saying "No" becomes a "No-No".
In short, when it comes to saying No - I'll try.--
The right word may be effective, but no word was ever as effective as a rightly timed pause.
-- Mark Twain
Must be a strange coincidence, but I think nearly all of my farewell letters have been written in August.
As the days get a tiny bit shorter, all the energy that the summer brings turns into a certain impatience. Sitting in front of a desk, when the brightness outside is getting wasted.
Perhaps you don't feel it, but there's a general ebb of life that makes me acutely conscious of my mortality. A certain knowledge that summer is passing and I'm still indoors, to wait for another year perhaps to enjoy the summer sunshine.
And yet, it's not that I feel sad about the passing of days - those days aren't here. It's a call to freedom. A primal urge to make most of the days in the sun, without concern, because the winter that will follow is inevitable. To plant all the energy of the summer that's half-gone and reap the rest.
Need to just head out into the sunshine & high mountains, leaving work in the valleys.--
Your career is like a box of chocolates - you never know what you're going to get.