Almost exactly to a year ago the merger between Cloudera and Hortonworks went through.

I've had a fairly inside field view of the process and the rollercoaster has been fun or at least never boring. Along the way, I kept having these four questions.

Four questions which are like an organizational rorschach test. But they all look like butterflies to me.

Do you want to release on schedule or by bug count?: There are fundamentally good reasons to ship software only when its bug count drops below a certain margin, but that usually circles around a single development line and not an entire ecosystem distribution. The customers aren't all the same and they don't care about problems in the same way across the components. Shipping a release with bugs is so much worse when you are planning a big bang release with another six months before the next attempt at fixing the problems. However, there are two assumptions hidden in this - new adopters will wait for a release instead of exploring alternatives during the delay and that the internal bug count is a good proxy for the issues that customers will encounter.

There are immediate organizational downsides to this as well. Any team that meets its bug counts for the expected due date through personal heroics of team members will feel disappointed that either they worked too hard or that they took up tech debt for workarounds they didn't need to. And the team that is currently holding up the release will get pressure from all sides and possibly a bad review in the future.

Of course, there's still no good answer because you can't just ship whatever you have because it is the 2nd Tuesday of the month. There's quite a lot of balancing between hitting the dates and closing all the release blockers. In general, most of the quality of code discussions circle around this particular trade-off point.

If there's an argument, is it better to build or discuss?: Decision making is always full of conflict. Technical discussions tend to be easier to tie-break since measurements are possible without involving external entities, unlike say advertising or marketing ones. That said, often specifications which are left on the table without implementations tend to grow outwards either to add more scope to the project or to tackle specific conditions which are imagined during a meeting. A design discussion is not the right place to shift the scope of a project or to add customer scenarios into the mix, but several arguments are discussed out to expand the implementation routinely. These are easily recognizable when applied to cross-cutting features like authentication or authorization which every teams to gets to provide input on.

The main reason why these arguments end up in discussions rather than arriving at an implementation which can be criticized more constructively is because the discussion happens between non-implementors, either being architects or senior engineers. The people who would be responsible for putting together a prototype are usually never in the meeting and even if they were, they are happier to evade being targeted by the engineers in charge of oversight.

Discussions are useful to clarify disagreement. And in this note Chesterton's fence is very much applicable. The implementation is often surprise-heavy and ends up having to bypass several decisions made in dicussions. Empowering the implementor to communicate disagreement with the designer is the most important communication pathway I've observed.

Do you organize teams by skills or involvement?: Before I get into it, let's talk about specialists versus generalists. For a total team conflict reduction, having specialists is better than generalists, since each person has clear responsibility for their own area of expertise with no involvement with others in the team - either to approve or disagree. Naturally, this results in teams getting fragmented into niche skill specialists and leaves the organization to manage staffing by either over-staffing specialists or under-staffing the team when someone is on vacation or quits. From a skip-level up the ladder, that's where the utility of folks like me come in, since I don't mind being thrown into a problem which requires reskilling (at the expense of some conflict with established patterns with questions like "why do we always do it this way?").

Assuming you have a team of specialists and a single manager, then the immediate problem comes up when any of the specialists has a "career" conversation with you. The next level up is management, at which point the specialist will be tackling a team where they are familiar with a fraction of the skills. Having organizations where specialists occupy their own organization structure and float between projects is a way to bypass that issue, since the size of the project is not related to the location of the specialist in the org chart. But that reduces the involvement of the specialist in the team, where the success of the project is only indirectly tied to the future prospects. There is some middle ground here, but that needs to be found for each growth stage of the organization.

Is it better to have big plans or small plans?: Project planning isn't quite war, but it still holds that the map isn't the same as territory. The size and scale of the ideal plan varies as you move between approval of said plan and the execution of it. Big plans tend to motivate leadership, while they tend to overwhelm the foot soldier who can't quite see the map. However, the difference in software engineering is that a big plan can come from the other direction - engineers wanting to do complete rewrites to improve productivity and implement features faster. Because the rates of distruption over a month remains roughly the same, a big plan therefore is likely to fail, since priorities can change over two quarters more than it does over a single one.

It is easy for organizational panic when objectives can be further away than your plans. The comment about "I don't see how your plan is going to get us there" isn't the end of discussion, it is merely an opportunity to admit that from the hole you are currently in, a better plan can come only after you climb out of it. Again, there isn't a good size for a plan - there are plans within plans and all that. And there are ways to meaningfully go ahead with feature flags and fall back mechanisms where the big plan can roll out in stages, where an unexpected event is merely a pause in the process.

And then: These questions are important to me, not because they have right or wrong answers, but because your answers tell me what your experiences and perspective on software development are. And then perhaps, more questions to ask me.

posted at: 15:11 | path: /observations | permalink | Tags: public, work, valley

Often Software (with a capital 'S') is not built, it is grown. And a lot of green digits are involved, not always thumbs.

When I got an education which taught me how to build software, the process emphasized on sitting down in front of my computer, pushing my will upon a barely explicable machine which was built to resemble advanced technology on some days and complete magic on others.

In fact, when I think about it, every part of my computer science education was about imperative commands to a system which, if you give it adequate commands always throws up unsurprising results everytime (and all surprises were bugs). Everything I was taught about this fit that model, where when I had a "Software Management" class it was full of Gantt charts and PERT charts, going over how to schedule time for different engineers to work on the same project in parallel. The recommended reading was The Mythical Man Month and that was hilarious to read, at least. Mostly that class was taught as a way to have an engineer understand what a manager might have to do, not exactly explaining the operational realities.

This post is about the third time I realized I was ill-equipped to deal with the actual demands of being an engineer with the industry. Because none of what I describe works the way I've been taught - at least, when it comes to these skills I'm entirely self-taught.

The first time I figured out that this was all wrong was when I was bumped to a tech lead, one rung below turning full manager. Humans are pretty hard to instruct and they respond to imperative commands by being confused about the goals, slowing down till they understand it and planning around your well laid plans. This was the first time I figured out that breaking down a complex project into a set of sprint tasks, going over burn down charts and in general replacing motivation with process does not actually work for humans. Instead it sets up a bunch of perverse incentives which builds up technical debt and kicks the can on potential problems until one day they can pawn off that problem as "above my pay grade" to a special tiger team which drops in out of nowhere. People don't work like machines, but they do work - and they move much faster when they can take decisions that they can reverse on their own (turns out machines do that too, see SPECTRE).

The second time I figured out that the world where I'm employed doesn't look like my classes is when I started talking to customers. I wrote about my Zynga tenure that what I learned was to persuade people, not direct them. The interaction between a customer and a software engineer is one that is fraught with many traps which primarily rise from tossing bugs over with a blame thrower. Feature discussions are yet another scenario where being a "can do" person brings its own challenges. The crucial requirements for a customer are often expressed as potential solutions, where they ask for something very specific which worked for them in a different generation of tech or just ask for a feature which they read about in a press release by some competitor. Technology persuasion is a gentle process similar to managing reports, more art than science - which only works if you really listen, head back and build what approximates the need, instead of desire.

However, today I'm faced with a slightly different challenge - I'm overflowing out of my current role as a "Principal Engineer - II". That isn't a destination for an engineer as much as it is a reserved parking spot.

There's a slightly different track adjacent to it, which will use most of what I already know, but I'm contemplating what I have to give up to move over to a "Software Architect" in role, if not title. I find that title somewhat archaic, because it implies very strongly that software is designed by someone as a measured blueprint and then put together by bricklayers, overseen by foremen and all that. However, the reality disagrees vehemently, at least from my vantage point.

And I realize it is hard for an executive to look at me through an architect cookie cutter, because I'm already something else. There are literally dozens, DOZENS of us in this industry.

My open source software work is sort of like a slightly overgrown garden. I don't make plans with measurements, I make grids and boundaries for sure. The bugs creep in, but I try to make it easier to debug. The feature branches are trimmed to stay manageable and the features which pop up like weeds need to be taken care of early, before they start to creep over. But most often what I really point to is that the garden just needs time and attention to keep it up, however any week it is always unfinished with a bunch of flower beds yet to bloom or bulbs to repot out of the alpha bed into the beta. Gardeners do get credit though - but in a garden it is obvious that the plants all grow by themselves, but it is still all work of the gardener to fertilize the soil, regularly water and put all the effort into giving it the opportunity to thrive. The significant and hard requirement is being constantly present, to rotate through your skills over the lifecycle of your crop - including calling in & giving up specialized tasks to the right people.

The frustrations are extremely similar too - a drought in the market, needing to redo everything because the plumbers need to dig it up to fix something fundamental with the way the funding irrigation comes in, to literaly turn on the tap. A worker in a garden is not entirely in control of the process, as much as they take responsibility for the plants, there are no miracles they can perform beyond what nature allows them to. And in a round about way, everything that grows in this garden grows up on the dead remains of a previous generation without giving it any credit for laying down all this top-soil. Every few years, literally turning-over old projects to prepare the ground to plant new seeds.

And sometimes we do need to plant a few trees whose shade you will never sit under.

As history of the Santa Clara valley rhymes through the bracero years, I've turned into a gardener of open source software in fertile soil, brought in from a different land, taught again how to do this at an industrial scale, but organically and sustainably.

posted at: 11:26 | path: /observations | permalink | Tags: public, me, work

This isn't about leadership, but the events leading up to it. There's nothing new for me to say here, except to walk you through the path I took reading Socrates, Plato and Aristotle.

Though this truly started because I read Men At Arms again and wondered what Corporal Carrot is made of (& Vetinari and Vimes too).

Because reading it made me pick up H2G2 and look at Zaphod Beeblebrox & the man who rules the universe. But that's for a different episode, about presidents with a penchant for orange sashes.

In the Republic, Plato records Socrates's argument about the nature of leadership. That guardians of the state are not necessarily the leaders, but leaders employ themselves where they can show the way and to do that may abdicate power when they observe a well functioning, if not perfect state of affairs.

Weighing between the potential advances they could make from mastery of their art against the heavy burden of leadership, they choose to advance their mastery instead and indicate to others that their mastery is advancing the world.

The world changes shape, right at the moment where those have ignored politics realize that the state of affairs is regressing, despite their progress through life. In those moments of chaos, they consider leaving their avowed profession and take up the unthankful jobs that they had left to others before - leadership. There is no longer a question about comparing the rewards of leadership against anything else, since the alternative is to lose out on the most relevant of common goods - peace.

And I quote

And this is the reason, my dear Thrasymachus, why, as I was just now
saying, no one is willing to govern; because no one likes to take
in hand the reformation of evils which are not his concern without
remuneration. 

For, in the execution of his work, and in giving his orders to another,
the true artist does not regard his own interest, but always that of
his subjects; and therefore in order that rulers may be willing to rule,
they must be paid in one of three modes of payment: money, or honour, 
or a penalty for refusing. 

What do you mean, Socrates? said Glaucon. The first two modes of payment
are intelligible enough, but what the penalty is I do not understand,
or how a penalty can be a payment. 

You mean that you do not understand the nature of this payment which
to the best men is the great inducement to rule? 

Of course you know that ambition and avarice are held to be, as indeed
they are, a disgrace?

And then Socrates explains.

And for this reason, I said, money and honour have no attraction for
them; good men do not wish to be openly demanding payment for governing
and so to get the name of hirelings, nor by secretly helping themselves
out of the public revenues to get the name of thieves. And not being
ambitious they do not care about honour.

Wherefore necessity must be laid upon them, and they must be induced to 
serve from the fear of punishment.

And this, as I imagine, is the reason why the forwardness to take office,
instead of waiting to be compelled, has been deemed dishonourable.

And Socrates goes onto explain the punishment that awaits a good person who sees a leadership vacuum and does not step up to it.

Now the worst part of the punishment is that he who refuses to rule is liable
to be ruled by one who is worse than himself.

And the fear of this, as I conceive, induces the good to take office,
not because they would, but because they cannot help --not under the
idea that they are going to have any benefit or enjoyment themselves,
but as a necessity, and because they are not able to commit the task
of ruling to any one who is better than themselves, or indeed as good.

And that's how Socrates concludes that leaders are forged out of a crisis, not out of peace or prosperity - not by intention, but by choice and circumstance.

That does paint the progression as sort of inevitable, but it is entirely rational to observe the choice ahead and just leave.

PS: Republic talks about doctors getting paid for good health, women being educated equally and rulers being enlightened - it's hard to think of it being written two millenia ago, while most of that is still fought out. Read the whole thing.

posted at: 13:52 | path: /philosophy | permalink | Tags: public, work, philosophy

Tech interviews are very strange. The whole process makes no sense in general, when it actually comes to being hired to do actual work. This gets worse when you're dealing with a specialization that is all about identifying trade-offs and fitting solutions to them.

But let me explain this with a car analogy.

Ok - I want you to design the architecture for something that can get from A to B very fast, in a city. Make sure that it can go around corners and also stop when required.

(starts to draw a motorcycle, with a handlebar and brakes ...)

Sure that would work, but we're mostly have to have four wheels in this one, because that's necessary for stability when going around corners.

(Starts to describe how in the hands of a good driver, motorcycles do go around corners without toppling over, like all over every city)

Alright, but let's take it as a given that the vehicle needs four wheels when this company makes one. Perhaps a fifth one which has to be carried around all the time, because we have four wheels.

(Ok, so the best design to go around corners really fast involves a really low car with a wing at the back, for downforce at high speed)

Yes, but that's not very practical - it rains sometimes and you need to put a roof over it, so that you can drive around in the rain without getting wet.

(That's reasonable, so there's this nice Miata that I once drove, it had a cloth roof that would come up whenever the car is parked & you could drive it pretty fast around corners)

Yes, but capacity is a big problem with a car like that so something a bit bigger would be more scalable as the company grows in size.

(Sure, the Tesla Model X is a great example of a bigger and more ecofriendly car, which is still pretty fast and has all the weight down below ... it does go around corners and stops where you want it, with a lot of on board capacity for growth)

That's good, but that's an exotic solution - in general, we prefer simpler solutions over things that need new and unproven technologies. The goal is to use as much off-the-shelf tech as possible, mostly because we're still limited cost concerns.

(Sighs, starts drawing a volvo minivan ... but stops to check)

That's a good approach, can you go into a little more detail about how you'd make this go faster, particularly when it comes to acceleration.

(You can put a V8 in it, maybe go as high as 400 Horsepower and it will be okay - I've done that before, but it's an ugly kludge).

See, this is the point where I quickly realize that the conversation was explicitly tailored to come to the same conclusion as the other side of the table - that the company has a minivan they designed for growth, but they want it to go faster and they're hiring me because I know how to put a V8 in a giant car, because I've done this before.

Being different is not exactly a good thing, but it is the necessary step towards innovation, in various degrees (improvement is change, duh). And yet, in an interview you run up against hidden limitations which completely up-end your ideas for change, because everything is about trade-offs, not absolute right answers.

And that almost never happens when you work on something for a month.

posted at: 17:11 | path: /rants | permalink | Tags: public, work, me

I have started to remember more details about problems than I do about code.

Code used to occupy a large amount of my work-time thinking roots - algorithms, data structures and architectural concepts. Over the last two years, the organization of my thoughts has shifted onto understanding parallels with the real life structure of the problems. I'm looking through my attic for my brain and finding new uses for all that I've already done, which makes a better part II.

So for a while I've been putting thought into revisiting ideas, to feed them through this new lens. Here are some of them which I'd like to work towards thinking more clearly about them over 2018. If any of you have one of these problems or would like to enlighten me about how you are solving them, I'd like to buy you a beverage of your choice.

First up, large corpus compression. At a previous job, we had a few million users' game worlds in blob form, compressed with SNAPPY. This of course was incredibly inefficient to store every user's world's stereotypical artifacts again and again. The fundamental problem was to split apart the common structures between multiple blobs into a common blob and implement a LZ77 variant which can entirely remove large chunks of the input data from the compressed form and merely reference the common blob range (i.e SNAPPY COPY instructions with -ve offsets). This is not very revolutionary, because you can find something similar for english text in a new compression algorithm like Brotli (see RFC7932 Appendix A). It is already possible to do something similar, when it comes to Druid segment files, to compress URL columns by a large fraction within the segment, while still being able to decode any random row out of the column.

But what I want to do with that core idea is to apply it for another data-set which has a large amount of repetition between records - DNA and RNA. The idea is to apply this sort of lossless compression models to FASTQ data, but to seed it with known sequences as a database for long-range LZ77. The core idea is to compress the million genomes (like the one VA is building with the Million Veterans Project) while making it much easier to lookup by a known (or relevant) sequence. The compression format offers a fast way to do 1000 base comparisons, if you could build a pre-set dictionary and map them while they are being ingested. This is not a trivial operation at scale, but if you can cross-cut through the compression algorithm for your search implementation, this offers a massive improvement on the IO requirement on a cloud system. In a very vague way of saving, the more data you have the less data you'll have per-person.

The basic search optimization problems are also interesting for this, since nobody is going to download a large warehouse of data to match a single chunk of ctDNA that they pulled out of somebody's blood, but it is a batch problem with interesting parallels to other plain alphabet problems. My scrabble word finding cheat scripts do prime-hashing, which would work for fragments shuffles (& also off-by 1 identifications), which compresses 8 letter combos into 1 long without order (i.e all combinations of 8 letters into the same long). As a performance engineer (& and not a scientist), finding patterns with a prime multiply Rabin Karp in an alphabet soup is exactly the as complex as an alighnment problem. But the difference between the toy problem and the real one is literally life and death.

Second up, fleeting value capture problems. There are some problems where the value of a data entry decays as more time passes, if nothing reacts to it. These show up for a large number of streaming real-time systems, though the real life problems tend to tolerate a fractional loss of information while the streaming platforms are built to be feeding a system of record without loss. The demand-supply curve in a number of systems tend to work in similar fashion, where the historical data is not particularly relevant except in aggregate, but the current values are exceedingly pointy problems. Reactions to discounts and surge pricing are extremely individual, if you could map the actual demand to the demand against 68719476736a price point, in real-time, that can drive decisions at a finer granularity while keeping the maximum number of people happy.

This might look like a new idea, but I've seen this used to great effect in Facebook games - the random rewards within that skinner box is not exactly random. However, the difference is that in a value-cost model, the supply-demand curve does not meet at the same place for all people who are making financial decisions. The micro-economics in the single party model (with the "house" monopoly handing out infinite inventory and the players putting money down, there is no per-unit-profitability, only non-recoverable-engineering costs from the artwork & code) is much more easy to model than the delayed elasticity of the supply curve in a two-party model like the new gig economy.

This brings up another point, when you have weakly interacting processor models, they do not support strong failure tolerance characteristics. Failure tolerance is one of those "can't do without" concepts in distributed systems. This is sort of the "UDP for audio streams" discussion, because failure tolerance often demands waiting for the failure to be corrected before moving on - in a number of these processing models, delaying a future operation to make sure you have processed the current one is a complete waste of time. If you had an ETA estimation system for cars, then there is absolutely no value in processing a five minute old record when a user has already left a car. These problems routinely crop up in geo-fencing problems, when sending location updates over a cell phone network - when position changes, cancel the previous queued request and send a new one.

68719476736 The weakly interacting processors tend to be implemented as reliable queues and retried/stateless micro-services, both of which are imperfect abstractions to the actual problem at hand, but very clearly look like what MPI with buffering would look like. The data flow model is easier to delegate to different teams to build, but it does not match the characteristics of a single system which can handle geographical data as a sort of cellular network handing off cars and rides to the next cell as a vehicle moves through the city. If San Francisco has 45,000 drivers operating in it and 150,000 passengers with their apps open, you might realize that the state distribution for those + all the people in a vehicle right now is not exactly a giant scale problem, even if it looks like a 50k HTTP calls/sec of traffic going over the wire when it rains.

Geo-spatial data is another crucial problem for me to reapply some optimization work I did with NPCs in a video game - basically, if you ask them to get coffee, they need to find the nearest coffee shop and walk there. This meant dividing up the city into QuadTrees and build an A* path routing over that space. However, after I started to poke about that class of problems, I ran into Hilbert curve numbers and Hex bins. A Hilbert curve converts a 2-dimensional space into a single number and there are a large number of ways to handle single dimensional data in indexes, which suddenly become useful when you dump your data into Druid or another SQL DB which handles integers much faster than a complex square root function. There are space filling curves for hexagons which form a much prettier picture, but the real advantage with Hex bins is that for a QuadTree for a given position, there are 8 adjacent grids to check, while the Hexagon only ends up with six adjacent blocks to check.

To put it together, Geo-spatial data mapped onto a weakly handed-off cellular processor model (i.e the fuzzy boundaries) tends to work very well in a distributed systems, without a significant cost of the failure tolerance required to perform a strong hand-off. This is something left over from my first job writing a call-handling application for Ericsson, where the cellular paging channels and how they handle a user who merely drives through cells over someone who's on active call driving through, while being able to get GSM audio packets correctly (also it is encrypted). The interesting part of that is using the first derivative of position computed for each bogey in motion & the accidental utility of knowing collision/occlusion math from old video games.

Over the last 15 years, I've gathered up a bunch of eclectic experience, which seem to be useful in several other places across the industry in different scenarios with their own additional trade-offs and complexities. Complexities I haven't considered and trade-offs I haven't explored.

So, if you're working on something like this, I'd like to pick your brain and understand what I don't know - expand my ignorance, if you will.

posted at: 18:17 | path: /fun | permalink | Tags: public, me, work

Today's the first day of year 5 for me in Hortonworks.

I've now spent more than a decade getting paid to do open source work. But until Hortonworks, all work I did was tangential to the business arm of the company. All my customers were internal, which meant mostly I got privileged access to the problems. The bug resolution for complex issues never had to deal with walls of customer solutions obscuring my insights into the system. There was never a day when I couldn't tell someone from Yahoo mail that I was working on another issue with Frontpage.

During my Zynga period, I learnt how to sell your ideas to a product focus team looking for solutions - to explain better, to justify my arguments with numbers, to convince and to persuade. To keep my sanity, I also learnt to pick my battles, drop issues and walk away when the friction gets personal.

I never had to sell to a customer direct - even at my best, on the books of the company I was a cost, an engineering expense. My work at Zynga or Yahoo, was helping the company bring down its bottom line by a fraction without ever touching the top line, except by accident (yay, faster load times).

This is generally an area of work with no real conflicts of interest built into it - use of APC for instance was mandated and in most cases, my code was the only available solution to the problem. Giving up on APC meant switching to HipHop, which also I worked on (never got the CLA signed by Zynga, which was sad though). The pattern followed me through out the period even through ZCloud build-out, through ZPerfmon, ZProf and pretty much everything I touched.

Hortonworks has been a completely different experience. I'm still a developer, but my contribution to the company has turned into a top-line enlarging investment in most scenarios. The work I was doing with Stinger was actively bringing in more customers and indirectly, more revenue in. This brings in some interesting conflicts for a developer, when talking to a customer about your features, because that particular transaction starts off as a zero sum exchange of money with the implicit declaration that the "solution" is a non-zero sum win-win.

This shift in the nature of my job has been interesting. The role now requires a completely different social skill set which I didn't need in any job before this. The goal is nearly the same, but conflicts fall onto my lap instead of being able to just say "Zynga first" to end a discussion.

Honestly, it's been a bit of a culture shock.

I feel like I'm astride an elephant, with my toes pressing suggestions of direction. The benchmark numbers are just numbers, unless someone else decides that those need some action. I cannot actually move the beast, but the beast does move, never to stand still.

As always, I've stayed on top of that beast here for the last four years to learn and cope with that process. The learnings are rather harsh, that while in a single company you get to deal with your users first, the actual users in this mode don't run the systems, those who run the system don't make purchasing decisions and those who write the checks don't really deal with either of those folks directly. And in the end, the check they write is eventually only fractionally related to my paycheck anyway.

In that world, it feels ultra depressing to sit in on a call where the dreaded phrase is said - "what the hell do we pay you for?" . There's the sub-text in your head going - uh, you don't ... I'm here because I am the only one who can help with this and this is not even my job, neither is it my turn on the on-call roster, so get off my back. But you learn not to lash out in knee-jerk fashion and close down the conversation, but continue debugging which is upto your discipline, a thick skin and the knowledge that your self-worth isn't tied up in that guy's opinion. I still don't know how to communicate that Insulting my commitment to your problem isn't helping - making it unpleasant might feel better, but you're burning good will here. Looking back, some of what I did never made any sense in the big picture, to feel the pressure from the sales cycle to deliver something over the christmas weekend and then get back on it through the new year weekend too. However hard those weeks turn out to be, the rewards for such heroics diffuses away as the company gets larger and as the customer base expands outwards - no deal ends up being the end-all of your work.

I still love solving those tough problems and solving them are their own reward, but the process friction is draining for your attention spans and your ability to take criticism for no fault of your own wears thin (I'm rarely the person who wrote the slow bit of code I'm fixing and I'm unlikely to be the one in the next case either, *sigh*). Collective responsibility is hard with open source, when this issue was because of someone back in 2009 who thought this was good enough, way before the YARN and Hive teams sat under the same roof. It's nobody's fault, but remains your problem.

There are some upsides to being thrown about into the rough and tumble of that process, you develop a lot of breadth in your skill sets out of sheer necessity - whether it's about something esoteric like Kerberos or the specifics of file listings in Azure or S3. There are technical savings you build up over the years, which work exactly the opposite of how technical debt works - readymade solutions which you refine through one customer aftet the other, to feed back into the product as much as you can. This is what a front-line engineer does in a small company, to carry a huge memory bank of all the problems cross-referenced to their solutions. A bigger company divides away expertise until no one person has the entire picture and start operating in managable fractions of expertise.

I came into Hortonworks, expecting to fix MapReduce and HDFS, possibly to dig deeper into HBase latencies - it is odd to say that right because instead of polishing old code, I got to work on brand new ideas with Stinger on Hive, ORC and Tez. But unlike my work at Zynga, I've never had to bury any of work (and to have the company salt the earth behind it) - my patents from Zynga are clearly dead-ends which I cannot pursue somewhere else, original ideas which I had and lost posession of. Putting up every little thing I build onto a github repo has been a liberating experience, being able to talk about my work in great detail to everyone including business competitors gives me a sense of autonomy which I have not experienced since college.

There have been so many mishaps along the way too. It was a horrible idea to work as a contractor in India when Hortonworks failed to hire me in time for the H1B cap - every bit of my immigration that followed has been a pain because of the fact that I was an Indian contractor moving to the US. Unfortunately it's too late now that I'm already here and is probably beyond recourse, I would probably have to quit or move out of America to reset that, in the near future. And that's sort of been the only regret of mine in joining Hortonworks the way I did.

But this is the second longest I've worked in my career and this journey has been very different from the Gumball years of the Yahoo. With Hortonworks, I joined a small company in Maude & Matilda four years ago and I now work for a company that is so big it has offices around the planet. When I interviewed at Hortonworks in February of 2012, that was way before the this company was - back before there was a packaged product or an installer, before the "do or die" phase of working for an underdog company.

And an underdog might be down, but always gets to punch up.

posted at: 22:40 | path: /me | permalink | Tags: public, work

Must be a strange coincidence, but I think nearly all of my farewell letters have been written in August.

As the days get a tiny bit shorter, all the energy that the summer brings turns into a certain impatience. Sitting in front of a desk, when the brightness outside is getting wasted.

Perhaps you don't feel it, but there's a general ebb of life that makes me acutely conscious of my mortality. A certain knowledge that summer is passing and I'm still indoors, to wait for another year perhaps to enjoy the summer sunshine.

And yet, it's not that I feel sad about the passing of days - those days aren't here. It's a call to freedom. A primal urge to make most of the days in the sun, without concern, because the winter that will follow is inevitable. To plant all the energy of the summer that's half-gone and reap the rest.

Need to just head out into the sunshine & high mountains, leaving work in the valleys.

posted at: 16:27 | path: /me | permalink | Tags: me, work, public

Here's a short quick set of questions.

Listen, I'm not here to run your life. What I came here to do was to ask those questions and get a response from you. I can't really tell you more than what you already know about those boxes you checked off.

I started to play with computers as a hobby. I'm sure a lot of you did and that's probably the passionate drive behind your talent. But at some point in my career, that fun got intermixed with a professional atmosphere and even then that passion was a huge advantage, because I could keep going even after other people gave up on a problem. But work is not play, particularly when play involves walking towards failures as a learning experience. Doing something hard is still joyful, but it's not something that's necessary in an enterprise workplace full of constant low-skill challenges, with-in a framework of burn down charts, delivery schedules for back-ported fixes, day to do day management and daily standups. And bit by bit, I have realized that what I do at work is never going to fill in for a hobby - it's work.

I want to feel the joy, of a purposefully hard problem solved, by sheer persistence, despite multiple dead-ends.

So, I wrote this list because I'm going through a period of self-reflection after failing to wrest back my hobby back from my profession.

posted at: 21:48 | path: /observations | permalink | Tags: public, me, work

I work from home and that causes more trouble at home than at work.

Today Kavi asked me if I wanted to come along for a 4 hour round-trip to the airport to drop off her sister. Now those four hours of that has to come out of my work hours for the day & is a distraction I hadn't planned on having, but that is really my problem, not hers.

But if I were in an office building approximately two kilometres away, doing the exact same thing today I wouldn't have been asked that question. There would've been no expectation that I can spare four hours from the middle of my work day for a car ride.

I didn't go, of course - I'm sitting here ranting about it while my cluster is rebuilding ORC tables. But I didn't really have a pleasant time saying "NO" either. Just having to say *NO* and having her drive back from the airport alone seems like a choice I'm implicitly making.

In reality, I'm spending more time with both of them today, while my cluster rebuilds. But the fact that the question is pursued implies that the people involved do not consider me "at work" and therefore unavailable for anything else. And that question hurts my commitment towards both parts of my life.

The real trouble is that occasionally I do take time out of my work hours to do things with her. It is no sacrifice, because we do things that can only be done in daylight - shopping for things, brunches which last till 2 PM and evening runs where I spend an hour in the middle of the work day clocking up some mileage. I did that yesterday, with my cluster churning profiles out.

I probably really only work 4 hours a day. But those 4 hours come in two sessions of 2 hours of uninterrupted work, when I do not reply to emails, my phone is in another room, I'm not reading HackerNews or tweeting anything interesting (people, now you can see how often I work). I keep every activity which involves staring at progress bars out of those hours - my maven repos will be upto-date, my unit tests are run later and my work setup is all done somewhere in the middle, preferably during a meal or a run.

For those hours I'll be in a state of mind that is fragile enough to be destroyed by someone just asking me a rhetorical question like "XYZ is in town next week, we'll meet them on Wed?".

The trouble is that people who inhabit my life can't differentiate between those four hours and the other four hours. You know, the non-work work hours that go into reading JIRAs, replying to threads, attending calls or setting up EC2 nodes. Some of those leaks into the weekends, but I deal with it if it means having more quality time to spend on a Monday. In those hours, I will not snap back when interrupted, even drop what I'm doing to look at something or get a coffee.

I do try to make it all fit in, usually with some careful & invisible planning. It is the invisibility that is its undoing - it is effort that rarely goes appreciated by people at home (people at work don't even know I do this). It is really hard to say yes to new plans when they keep coming up all the time, without sharing my carefully timed plans - "Yes, I need 10 minutes to setup something that will take 4500 seconds to run. Let's go in a bit, but can we back by 9?".

I love being able to say "Yes!" to a plan, but I rarely say out the rest of the above sentence. And that has repurcussions that affect my happiness over-all. If unexpectedly I have to spend two hours at CPK eating dinner (enjoyable time, of course), I have to catch up by staying up till 3 AM to get my four hours in that day. Perhaps people will come home and I will offer to cook pasta instead of going out. A part of it is selfish even, because if I stick at home, I might get to stare at a monitor while the pasta gets to al dente and get some of the scaffolding work in. Those are not really work hours - if I can cook pasta while I work, I'm not working.

Now, you might wonder about my deadlines when I talk about my time and work. I'm not slogging to meet some arbitrary deadline set by Hortonworks. I'm working because I get paid to do something I actually like. And I do like to take my time off and do other things I like. I want to spend about 9 days somewhere in the Himalayas next month, without cellphones, computers and with my camera, tripod and a stack of well chosen filters. To balance my work and life, I need to put in my time at work doing productive things and I do that without anyone forcing me to "SHIP CODE!" - what else counts as work? :)

But the trouble is that I can't seem to balance my work & life, particularly living with someone who's always up for new things by the hour (which is why I fell in love with her). And it is a question of expectations from the people in my life, because I work where I live, most of the time. Perhaps if I walk out of the home for 8 hours a day and go sit in Yoga House, the assumptions of my availability for plans will drop out of her mind.

I still love working from home. I do get to head out for an hour or so, eat lunch, drink coffee and spend quality time with people. I just wish they'd understand that I have a work schedule between those pleasant punctuations and make my life a little easier with fewer questions I have to say "*NO*" to.

Because saying "NO, I HAVE WORK" makes me sad. Very sad, indeed.

posted at: 19:13 | path: /me | permalink | Tags: public, work, me

Friday was my last day in Zynga.

Zynga has been a very different place for me to work in. Coming from Yahoo, where the tech folks had sway over the company, to a place where the product managers were the core of the company was jarring to say the least. But adapting to thrive, indeed succeed there has polished off a few rough edges that I had.

I've sat at the cliff's edge between what's management and the frontline. I've stared into the abyss and stared it down. I've seen exactly how incentives work and more importantly, when incentives tend to work against you. I've learnt to temper my competitive spirit to skip the usual pitfalls of meritocracy. I've understood how to work with people who are not engineers.

I've known the difference between talking & persuading. To effectively bring people over to your side of the debate - whether it is with statistics or merely by running experiments. In some sense this is more diplomacy than being technically right every time - but there's no value in actually being wrong. And I got better at saying No.

I've learnt what risk looks like. Very early in this "game", I learnt to play offense. To gauge a risk, take it and occasionally clean up afterwards if it blew up in my face. And in some sense that's the general motto around there. I became quicker at churning out a solution, battle testing it and minor setbacks weren't really a deterrant anymore. And I was brave.

I got to treat crisises differently. I was on the on-call rota. Instead of panicking and hyperventilating, I started treating them as merely opportunities to offer solutions. And in my spare time, I prepared for these crisises which turned out to be massively productive moments of innovation. I tried to get some of it opensourced, but a lot of it is locked up in the paperwork that I never finished.

I've had my moments of frustration too. There have been scenarios which were charged with egos and people politics, which are probably no different in any other workplace - people trying to play schedule chicken, folks trying to delay others with scope creep, territorial engineers telling me to keep off their turf & avoidable meetings where people duel with power point slides full of promises they can't keep. I've occasionally gotten demoralized about all that, but I've managed to move beyond that and build out a stronghold around me.

And finally, the best thing that's happened to me was working with the senior technical folks in the Bangalore office. People like Binu, Prashun, Prakash & Jayesh have been a big part of my development at Zynga. And not just them actually, the whole team is full of fun folks who don't really consider work as the end-all of their daily life.

It felt like I was part of an uber startup!

posted at: 15:05 | path: /misc | permalink | Tags: public, work, zynga

In Memoriam - Monday, August 11th 2003.

And I mourn for a loss I cannot explain.

posted at: 17:30 | path: /me | permalink | Tags: public, work, experiences

Occasionally, as I flip back the pages of my life, I find myself in conversation with a younger me from a much older time. As if caught in a flipbook time machine, I see myself change, grow and in some sense, stay the same. Once in a while though, I turn up a page with which I disagree with enough to need revisiting.

In the mid-summer of 2007, out of my frustrations with work was born an unadulterated rant of pure cynicism.

There's some sort of misplaced humility that is injected into us by our educational system. Or maybe it is some sort social stigma attached to the braggart or overacheiver. Must've been what was going through Lennon's mind as he penned down "they'll hate you if you're clever and they despise a fool". Eventually, the struggle to stand out and the pressures to blend in, find some sort of balance in your inner selves. You'll be happy to be the best at something everybody is doing. And even if you aren't, it's somewhat a partially ordered set. Life makes sense and the years roll on.

So you are the real deal, the bee's knees. To start off your career, you dive in and start pulling your weight. Even a moron in a hurry can see that what you're doing valuable, nay essential to the company. Your management wants to know that, it's exactly the kind of information they crave. When it's handed to them in a platter, they love it. But, you keep working, in your little corner. Nobody notices anything and if they do, it's when you fail. You complain about not being noticed to your peers, you write out long rants on your blog about how your life sucks.

Most people at this point in their careers blame the management for everything that's wrong at their job. And treat every peer who chooses to move into management as a blood traitor. I'm not denying that there are bad managers, just the same as there are bad people. But most managers promoted out of rank & file end up being good people with a job which looks like herding cats, except without any catnip. The people working under ordinary managers go passive agressive in their rebellion, complicating the situation further. Eventhough 'tis a betrayal every which way, it happens because nobody trusts anybody.

You are doing something very important and valuable to the company. Then why don't they trust you? Because they have had people work under them in past who were poor communicators because they weren't getting anything done. They even had good people work under them who secretly didn't like the plans, but kept their traps shut and worked towards a fait accompli. So if you communicate poorly, they are not going to give you the benefit of the doubt. No matter how many poor communicators actually end up getting work done, the managers will always remember the times they've been burned.

You don't need to be a 'Yes Man' or a atrocious sychophant to get your manager to treat you well. All you need to do is to make his job easier ... after all, managing you is pretty hard work. Though, there's such a thing as overdoing it. But that's yet another story altogether.

Watching your future with much interest,
  — future me.

posted at: 22:01 | path: /observations | permalink | Tags: public, philosophy, work

After much thought and inaction, I've compiled a quick list of basic things you can do at work to reduce your tangible value to the company (coming to a bank account near you, soon !).

• Answer questions on devel forums
• Write a technical blog
• Live the company values
• Write cool hacks
• Have a serious hobby
• Contribute to an open source project

All of these things shout out that you have too much free time. If you send detailed, well thought out answers to a question on a devel forum - you aren't working hard enough. If you've got time to write a decent blog about technology - you aren't working hard enough. If you actually live the company values (for instance, like I do) - your irreverance and sense of humour do not suit a professional in this business. So is going around your product managers to actually build something that you personally like, without any input from the product strategy team - that's not what you get paid for. And hobbies - they waste your time and cost you money. Without a hobby to spend your disposable income on, you won't need as much money as you're asking for now.

But the last one takes the cake. Assume your full time job involves working on an open source project. Now answer me this, "what *competitive* advantage does your work bring to this company ?". After all the code that you write automatically becomes available to everybody - irrespective of who paid for the development costs. Code thus released drops to near zero value and ergo, the process of creating it ...

These should just about work, but YMMV.

PS: I'm being seriously *sarcastic* (or tersely ironic) here - these things are a potential investment in your betterment, do them, be a better person and as the Bhagavat Gita said - Karmanye vadhikarasthe, ma phaleshu kadachara (do your duty and expect no reward).

posted at: 23:45 | path: /rants | permalink | Tags: public, rants, work

Repeat after me, three times - C++ is not C. This is a fact which has to be hammered into every programmer who claims to know C/C++, with a nice clue bat if necessary. But in this case, it was more of g++ isn't gcc and only for those who use RHEL4.

Here's a bit of working code in C99 which is totally different from C89 (otherwise known as your dad's C standard) - which is technically speaking, legal C++ code as well.

/* compile with gcc -std=c99 */

#include <limits.h>
#include <stdio.h>

int main()
{
  printf("Maximum value for unsigned long long: %llu\n", ULLONG_MAX);
}

But the exact same code was not working when treated as C++ code. For a few versions of glibc, no matter which C++ standard you used, ULLONG_MAX wouldn't be defined. Not even if the code segment is enveloped in an extern "C" block.

As it turns out, this was a quirk of the glibc's extension to the C pre-processor - include_next. Rather than include the standard /usr/include/limits.h, what the first include statement does is pull the limits.h file from the compiler include files - from /usr/lib/gcc/.... You can figure this out by running g++ -M limit.cpp, which dumps a pre-order traversal of the include hierarchy.

And the definition of ULLONG_MAX was probably written by someone who never expected a compiler include file to be included directly from a user program - and rightly so. Except, there is no real way to fix the include order for such similarly named files.

Eventually, the fix was to use ULONG_LONG_MAX instead of the slightly shorted ULLONG_MAX. But the glibc bug has been fixed for a while - was just not critical enough to be pushed to all machines.

posted at: 07:12 | path: /hacks | permalink | Tags: public, hacks, work

Jargon is the refuge of the elitist, but I will stoop to it - to hide behind a word, my ideas. Performance Inversion is that dark fate that awaits a "Can Do" person in the hands of an unscrupulous [1] taskmaster. Something dark and deadly which destroys your work ethic and pushes you ever closer to a burn out, while still under-utilizing your real talents. In relatively mild quantities, the problem solves itself, thanks to that great human ability to forget. But for some more unfortunate souls, it throws them into a bottomless pit of effort, in a spiral of work and more work.

Performance: You are stepping into an existing team, filling someone else's boots. Invariably, your first few weeks will be typical bootcamp. You are pushed to deliver and most of us, do deliver in that initial phase putting in extra effort in return for the manager's confidence. But having gained the confidence and trust, the work cycle settles down into a more sustainable load, by mutual consent from both parties.

Distribution: Not everybody in the team works at the same levels of productivity. There are a few who have bursts of extreme results, while some of the more settled folks deliver a constant, steady stream of work. Most managers would like a decent mixture, so they have enough afterburner fuel in the former while the latter keeps chugging. Work comes in and the work load sort of averages out for everyone, a few extra bits given & taken, in general everyone ending up with their fair share - no more, no less.

"Can Do" XPloited: The former category of "in reserve" people are those who are more commonly known as "Can Do" people. Nobody sane (or at least has heard of "Mythical Man Month") will dare to plan for these people to work at full throttle. But a few still break this cardinal principle to keep the wolf off the door, for a few months more. After all, what good are developer resources, if not for working.

'Tis an act of betrayal, worthy of Macbeth and more. But unlike the lady of the play, the blood seems to wash right off the hands - maybe she should've tried the sweat and tears of engineers, instead of perfumes from arabia. Anyway, if someone had yelled out "abort mission!" at that point, I'd shake them by the hand and buy them lunch. For otherwise they have to be eternal optimists who never learn (incompetent) or of the feudal weasel family, whose concern for the serfs is legendary - either way, let them expense lunch.

Blame Spread Thick: Having been told to do the impossible, the developer has a faint idea that he/she is going to be pushed to the very limits of his/her ability. In some sense, the "Can Do" people enjoy that experience. But then something goes wrong, some dependency fails to deliver, the whole plan's a pile of toilet paper as far as the project's concerned. And a landslide of backlogs land on the developer's shoulder and the difficult task becomes mission impossible (where work should've been a walk in the park instead). To compound insult to injury, he/she is called in by senior management to explain the delays. Watching the man responsible for the mess (in your opinion) sit on the other side of the table with a reproachful look, doubting your commitment, is too much for anyone with a straight backbone to bear.

Character Shows: But you still eat crow, swallow the remnants of your self-respect and buckle down. You work insane hours, on weekends, skip meals, eat junk food and work work work. By an amazing coincidence, you manage to drag the entire module back on schedule, mainly by doing the dependencies yourself and doing QA's job when you're done coding. Having got your workload to a manageable level, you wait for the release, the associated kudos and a general pat in the back.

Inversion: And when you think it can't get any worse, it does. So the boss shows up at your cube, and says "Ummm,yeah ... $_nameless_ is quite behind schedule. You're the only one ahead of your tasks, so I'm moving a bit of work off to you". Not only are you not getting credit for finishing your work, but you are getting more work just because you keep doing it. All this while, $_nameless_ has been shirking work, lazing about in the food court and generally enjoying life. So, faced with the basic injustice of the situation and remembering the last lecture about "communication", you walk back up to your boss's office and enunciate - "No, I'm not doing that". Boss takes offense to your "Not my job" reply and lectures for half-hour on team work and "one for all, all for one" philosophy.

Final Straw: For an external observer (such as middle management), in a matter of months, you've gone from being a "Can Do" whiz kid to becoming a whiner who says "No" to work. They just can't comprehend why or how of the scenario and decide to take strong steps to discourage this behavior. In response to your glowing achievements on your self appraisal, they write out a "negative attitude to work" and give you an average hike. Your complaints to HR fall on deaf ears, used to listening to employees bitching about pay, quote salary parity to ignore you. Of course, the boss probably got a good hike for managing with (*sic*) employees like you. The inversion discontent spiral is now complete.

In a few short steps, one person has gone from being an excellent employee to someone who's polishing up their resumes. But the really sad part is that as long as the real offender is not punished, this developer is only one among a long line of people whose work ethic will be destroyed totally by mismanagement. There are a few other branches and variants to the above scenario, some worse, some better, but they all end up in the same gutter anyway.

All this only goes on to prove why I think of management as a dark art of sorts. It requires a lot of finesse and panache to pull it off, rather than mere authority behind you. You can't convert any average joe from a tech lead into a manager, at least not into a good one. There is a certain Je ne sais quoi which sets apart the good ones from the average ones. So if you have a good manager, you don't know what you're missing

And for the record - I only observe, I don't interfere.

[1] - Hanlon's Razor:
                Never attribute to malice that which is adequately explained by stupidity.

posted at: 17:42 | path: /observations | permalink | Tags: public, life, work

One of the rarest of the rare species that inhabit cubicles is that creature of myth and legend - the Team Player. Often naturalists hiding behind indoor plants near the watering holes of employees claim to have encountered the creature in the wild. But they are probably mistaken or lying, because it is widely accepted that no fossil evidence has been found past the piled carcasses of the Y2K mass extinction event. Ever since the job atmosphere has lost its ozone layer of job security, most of these magnificient creatures have succumbed to cancerous career growths or perished in hibernation between jobs.

Jokes aside, I'm here to debunk the myth of the team player. To study and expose the nature, being, migration patterns and if time permits, the mating habits of the common team player which is endemic to air conditioned cubicles of software companies.

For a borrowed concept from professional sports, the team player label has undergone a sea change before it has been used to describe a software engineer. Rather than deride the noble concept, which embiggens the smallest man, the target is the namesake euphemism which has replaced it in corporate vocabulary.

Managing a software project is no walk in the park and if Mythical Man Month is any indication, is often contrary to conventional wisdom. Essentially, most of the problems stem from one single assumption - that software is an industrial product. Building software is unlike any other industrial process and is more of a research & development activity than an assembly line of coders assembling components. Allowances have to be made for dead end attempts, work done to maintain status quo (aka Backward Compat), regressions and other anomalies.

Programmers, by definition, are only human. And humans have good days, off days and then those days when it doesn't pay to get out of bed. The productivity of a programmer is bursty and unpredictable. But predicting that is exactly what all the money in management is all about - creating schedules, timelines, plans and bullet points. And they'd rather have their task made easier.

So it is obvious that a steady, yet low, throughput would be considered more suitable to the management principles adopted from the industrial revolution, rather than the odd week of lucidity separated by a fortnight of stupor. The brilliant programmer who works in bursts falls out of favour, while the predictable programmer is pushed forward. The moral of the Tortoise and the Hare is vindicated in this modern race, where the hare is caught napping, though the jury is still out on whether 'twas the management's inaction which let the hare sleep.

The team player has come to be a euphemism for such a slow and steady worker - predictable and absolutely devoid of hidden reserves & surprises. Someone who would rather move with the team rather than run ahead and look back at others. To make no exceptions and just keep on working, despite lack of motivation or support from above is the clinching quality of the newly defined team player.

To put it more bluntly, being a team player (in context) is not for the benefit of the team of peers, but functions primarily as yet another variable removed from the game of uncertainity that is management. The ease of control is why this group is encouraged to form and survive, even re-evolve in new circumstances. By whom ? Take a guess. And where the species is missing, others are dressed up in the robes of team play-dom, under the flag with the device SCRUM.

The moment the system favours such team-players over strong contributors, the team will quickly lose its edge and motivation. The balance has to be maintained to ensure that the average output of the whole group exceeds the sum of the parts.

Finally, all the above discussions treat team players from the cynical point of view. But just because you are not a team player in the eyes of your management does not make you not-a-team-player in the eyes of your peers. As long as you can have fun, while working with them, help those who struggle and in general, not let your ego rule your decisions, you'll be one of the team - truly.

Otherwise, you're just a team player.

posted at: 01:46 | path: /observations | permalink | Tags: public, life, work

This one's for all the times I saw an eskimo on a sled with the banner North Pole or Bust in Miami. Actually that was just a cartoon - but that's the point. Ever wonder why it's funny ? Actually I have a pageful below that will do nothing explain it. But I think you'll understand why I hold back no smiles. This is actually a page from my life, carefully bookmarked (disclaimer: some names changed for more hilarity).

The problem with opportunities is that when they do knock, they're likely to press alarm bells rather than door bells. As articulated so clearly by Wally - there are no problems, only issues, challenges and opportunities. So, what does that have to do with my life ? Well, I was the neighborhood Wally when I used to work in Initech.

I used to show up at work a bit after nine. Yes, 9 AM and that's late by Initech standards. I'd walk upto to my desk with a cup of coffee as if I just nipped downstairs for a coffee. I never carried a bag and hardly ever an umbrella. I'd imagine somebody would be hard pressed to figure out whether I just walked in to work or whether I'm relaxing after my early morning bit of work. My job involved taking a mobile phone, flashing the latest build and literally key mash my way into the bugzilla records. I was a QE and they were wasting a really quality engineer by making him poke a few buttons. The managers knew it, but were really powerless to pull in a kid out of college into writing embedded software (*oooh*).

So the problems started in the April of 2004, when 25-odd people from the 34 member development team gave in their resignations. What I was doing at the moment was having an argument with my technical manager about my leave, without knowing about all the hush hush resignations. I basically had put in more than a month's salary into my round-trip (non-refundable) plane tickets to Trivandrum and wouldn't stand to any level of bullying about the leave which I'd applied for two months ago. So I left office to spend an enjoyable week at home.

I returned to a chaotic office. Almost every good developer in the team is on notice. At that point hardly anyone cares about the project to waste their time on it. Here's where the management decisions kick in with true force :- We need to hire more people. Since they've already sold the IP for the project, management is now being paid for time rather than code output - there's no motivation to hire excellent developers. Solution, throw the bunch of freshers who've been sitting around on to the code. The more time they take to do things, the more money the account pays. And I was one among them.

That, my friends, is how I became a developer in profession. The loss of a significant amount of technical talent in one place, means a significant opportunity to people sitting in the benches elsewhere. That might sound too simplified, but there are two unspoken assumptions. The new guys on the bench have to be really good and the management has to have enough confidence in them (or shouldn't care which way) to throw these kids at the hardcore work. On a normal day, that'd be a big risk - but during these few desperate hours, this could be the last gambit.

It might just be my big empty head, but I keep hearing echoes.

posted at: 08:46 | path: /observations | permalink | Tags: public, life, work