Must be a strange coincidence, but I think nearly all of my farewell letters have been written in August.
As the days get a tiny bit shorter, all the energy that the summer brings turns into a certain impatience. Sitting in front of a desk, when the brightness outside is getting wasted.
Perhaps you don't feel it, but there's a general ebb of life that makes me acutely conscious of my mortality. A certain knowledge that summer is passing and I'm still indoors, to wait for another year perhaps to enjoy the summer sunshine.
And yet, it's not that I feel sad about the passing of days - those days aren't here. It's a call to freedom. A primal urge to make most of the days in the sun, without concern, because the winter that will follow is inevitable. To plant all the energy of the summer that's half-gone and reap the rest.
Need to just head out into the sunshine & high mountains, leaving work in the valleys.--
Your career is like a box of chocolates - you never know what you're going to get.
Here's a short quick set of questions.
Listen, I'm not here to run your life. What I came here to do was to ask those questions and get a response from you. I can't really tell you more than what you already know about those boxes you checked off.
I started to play with computers as a hobby. I'm sure a lot of you did and that's probably the passionate drive behind your talent. But at some point in my career, that fun got intermixed with a professional atmosphere and even then that passion was a huge advantage, because I could keep going even after other people gave up on a problem. But work is not play, particularly when play involves walking towards failures as a learning experience. Doing something hard is still joyful, but it's not something that's necessary in an enterprise workplace full of constant low-skill challenges, with-in a framework of burn down charts, delivery schedules for back-ported fixes, day to do day management and daily standups. And bit by bit, I have realized that what I do at work is never going to fill in for a hobby - it's work.
I want to feel the joy, of a purposefully hard problem solved, by sheer persistence, despite multiple dead-ends.
So, I wrote this list because I'm going through a period of self-reflection after failing to wrest back my hobby back from my profession.
If this myth is tragic, that is because its hero is conscious.
Where would his torture be, indeed, if at every step the hope of succeeding upheld him?
The workman of today works everyday in his life at the same tasks, and his fate is no less absurd.
But it is tragic only at the rare moments when it becomes conscious.
-- Albert Camus, "The Myth of Sisyphus"
Nobody commits suicide.At least, nobody commits suicide any more than they commit a diabetic coma or a cardiac arrest.
It isn't a crime to commit, it is the unfortunate demise brought on by years of accumulated damage - perhaps a less acceptable one than a surfeit of cheeseburgers.
If you're the one playing the devil's advocate for sinners, please rank it with gluttony and sloth.continue reading "A Rush to Final Judgement"....
It is often said that before you die your life passes before your eyes. It is in fact true. It's called living.
-- Terry Pratchett
It has been 7 years since my last conversation with him. Maybe it's only been six since I missed him.
Maybe I didn't say anything to anyone when he died, but with all the lack of orginality I posess, let me repeat something. If only as l'esprit de l'escalier, 8 years later if you will.
MY FATHER WAS AN EXCEPTIONAL MAN! He had his... shortcomings, but he took care of his family. He loved his family. He loved this house. He worked hard. All I wanted today was to show him how much we all loved and admired him. To give him the respect that he deserved... Is that really so much to ask? We all get so distracted by the... little things in life. We forget about the important things. Like the fact that we lost a great man. We don't know why a man makes the choices he makes. But I do know that my father made the best choices he could make. Life is complicated. We're just thrown here together in a world filled with chaos and confusion... and we do our best. He taught us to go for what we wanted in life, because you never know how long you're going to be here. So when you all leave here today, I want you to remember him for who he really was. A decent, loving man who never condemned anyone for how they lived. Who never cast disparaging remarks or held prejudices against race, gender... height. If only we could all be as giving, as generous, as understanding as my father. If I am half the man my father was, my child will be incredibly blessed. -- (paraphrased from) Death at a Funeral, 2007
My father was an exceptional man. He was exceptional in every way, in his strength and his weakness.
My mother taught me how to endure, to abide, to suffer the endless slings & arrows - my father showed me how to fight, how to lose battles and win wars. To take invisible swords upto invisible monsters and slay them. If my father taught me to fight, my mother taught me to never give up. She passed onto me, her endless source of hope. She taught, if not by words but example, that everything passes, including waves of troubles & sadness. Just hold your breath and dive, she showed - find the bottom and push back all the way up.
I learnt strategy from him - I very rarely played chess with him and he would reluctantly destroy me in calculated moves. I rarely put myself in front of that juggernaut because of what came after I lost, a clear description of why I lost, step by step of when I had an advantage & how I squandered it. I learnt patience from watching him - that patience isn't waiting, patience is observing, watching for the right moment - with a finger on the trigger, eye on the scope.
I learnt to listen and remember. Of all of my life's conversations (yet), he's the person I've talked the most to. I missed the conversations with him about fundamental human nature, sitting on a Bajaj Chetak, with him explaining to me in great detail the importance of understanding. Nothing more, just understanding everything. Perhaps, I learnt to be pedantic too - to understand what's different about wealth, value, cost, price. Understand that money is important, but never a goal. Understand that you should always engineer all-winner scenarios and never work with people who will still want you to lose. Understand altruisim & co-operation is just another word for long-term selfishness. Understand that help you give that is nothing to you, but everything to someone else is the best form of it. Understand that gender is meaningful, but not discriminatory (between me & my sister). That laws we have to obey are never entirely fair - that both the rich & poor are prohibited from sleeping on park benches in the night.
He didn't just talk about it, I saw his principles moving him - some of those came up because he was the Director of Social Welfare in Kerala government.
Not all his lessons were imparted kindly. I watched over my father for a year. I felt like the parent - making sure he ate food on time, took his medication, to make sure he slept, to run to his bed-side when he had nightmares. To stay up next to him when he slept in his hospital bed, reading, while the sedatives kicked in and kept him from tossing about in his bed. I had to deal with him when he threw tantrums and I know it hurt him more when I couldn't understand him or calm him down. I was filled with a homesickness that only comes to those who are already home and I wanted him to go back to being the same awesome father I always had. Deep inside his heart, he knew he could never be that person again. And he tried to tell me, but I didn't believe that - to believe him seemed like letting go of all hope. Never wanted to pass through the door inscribed "Abandon all hope, ye who enter here".
Years after, in mid-November I get angry about all that again, I remember that he was the most considerate of human beings even to the end - I found his paperwork sorted and filed in the order of use after his death.
I wonder about all our conversations - was he was leaving a part of him to live, even after he takes the rest to an early grave?
And just for that, I will always love him.--
All that we love deeply becomes a part of us.
-- Helen Keller
This is very definitely not about computer monitors. It is a story about data hashing, distribution and indeed re-distribution - about divisors, primes and more than all of that, downtime & costs.
In all honesty, I'm just posting a conversation from a coffee break almost two years ago (and some context to frame it). I'm writing this down now because it has reached the edge of my memory and is about to fall off.
One day, Prakash in the middle of coffee started to talk about the pains that powers of two cause. I assume, it was because one of the membase clusters (now called zBase) was running out of capacity and needed to be doubled in size.
Doubling in size was the only way our clusters grew, which was all well & good when the product was on its way up. It means you can double in size rather rapidly and without any real downtime. But it does cost a lot of money to keep doing after the first or second rehashing. Doubling made even more sense in a master-slave setup because the old cluster became the new masters and dropped half the data each, which would then be caught up by a new array of slaves which meant it would happen without any downtime for data copying and was a nearly no-risk operation.
This is relatively straight-forward, because the masters only have key invalidations happening during the re-shard. No data actually moves between masters during the reshard. This is brilliant because a few seconds after you flip the switch, the master array is ready to update and the slaves have a few minutes to catch up. But this is very expensive. Say, you had a 4 node cluster serving out data and eventually node1 runs out of space, then you need to add 4 more extra servers where otherwise only 2 would have been sufficient.
But this is primarily a one-way street. You cannot shrink a server cluster to half its size by this method.continue reading "1440 > 1024"....
There is no safety in numbers, or in anything else.
-- James Thurber
I work from home and that causes more trouble at home than at work.
Today Kavi asked me if I wanted to come along for a 4 hour round-trip to the airport to drop off her sister. Now those four hours of that has to come out of my work hours for the day & is a distraction I hadn't planned on having, but that is really my problem, not hers.
But if I were in an office building approximately two kilometres away, doing the exact same thing today I wouldn't have been asked that question. There would've been no expectation that I can spare four hours from the middle of my work day for a car ride.
I didn't go, of course - I'm sitting here ranting about it while my cluster is rebuilding ORC tables. But I didn't really have a pleasant time saying "NO" either. Just having to say *NO* and having her drive back from the airport alone seems like a choice I'm implicitly making.
In reality, I'm spending more time with both of them today, while my cluster rebuilds. But the fact that the question is pursued implies that the people involved do not consider me "at work" and therefore unavailable for anything else. And that question hurts my commitment towards both parts of my life.
The real trouble is that occasionally I do take time out of my work hours to do things with her. It is no sacrifice, because we do things that can only be done in daylight - shopping for things, brunches which last till 2 PM and evening runs where I spend an hour in the middle of the work day clocking up some mileage. I did that yesterday, with my cluster churning profiles out.
I probably really only work 4 hours a day. But those 4 hours come in two sessions of 2 hours of uninterrupted work, when I do not reply to emails, my phone is in another room, I'm not reading HackerNews or tweeting anything interesting (people, now you can see how often I work). I keep every activity which involves staring at progress bars out of those hours - my maven repos will be upto-date, my unit tests are run later and my work setup is all done somewhere in the middle, preferably during a meal or a run.
For those hours I'll be in a state of mind that is fragile enough to be destroyed by someone just asking me a rhetorical question like "XYZ is in town next week, we'll meet them on Wed?".
The trouble is that people who inhabit my life can't differentiate between those four hours and the other four hours. You know, the non-work work hours that go into reading JIRAs, replying to threads, attending calls or setting up EC2 nodes. Some of those leaks into the weekends, but I deal with it if it means having more quality time to spend on a Monday. In those hours, I will not snap back when interrupted, even drop what I'm doing to look at something or get a coffee.
I do try to make it all fit in, usually with some careful & invisible planning. It is the invisibility that is its undoing - it is effort that rarely goes appreciated by people at home (people at work don't even know I do this). It is really hard to say yes to new plans when they keep coming up all the time, without sharing my carefully timed plans - "Yes, I need 10 minutes to setup something that will take 4500 seconds to run. Let's go in a bit, but can we back by 9?".
I love being able to say "Yes!" to a plan, but I rarely say out the rest of the above sentence. And that has repurcussions that affect my happiness over-all. If unexpectedly I have to spend two hours at CPK eating dinner (enjoyable time, of course), I have to catch up by staying up till 3 AM to get my four hours in that day. Perhaps people will come home and I will offer to cook pasta instead of going out. A part of it is selfish even, because if I stick at home, I might get to stare at a monitor while the pasta gets to al dente and get some of the scaffolding work in. Those are not really work hours - if I can cook pasta while I work, I'm not working.
Now, you might wonder about my deadlines when I talk about my time and work. I'm not slogging to meet some arbitrary deadline set by Hortonworks. I'm working because I get paid to do something I actually like. And I do like to take my time off and do other things I like. I want to spend about 9 days somewhere in the Himalayas next month, without cellphones, computers and with my camera, tripod and a stack of well chosen filters. To balance my work and life, I need to put in my time at work doing productive things and I do that without anyone forcing me to "SHIP CODE!" - what else counts as work? :)
But the trouble is that I can't seem to balance my work & life, particularly living with someone who's always up for new things by the hour (which is why I fell in love with her). And it is a question of expectations from the people in my life, because I work where I live, most of the time. Perhaps if I walk out of the home for 8 hours a day and go sit in Yoga House, the assumptions of my availability for plans will drop out of her mind.
I still love working from home. I do get to head out for an hour or so, eat lunch, drink coffee and spend quality time with people. I just wish they'd understand that I have a work schedule between those pleasant punctuations and make my life a little easier with fewer questions I have to say "*NO*" to.
Because saying "NO, I HAVE WORK" makes me sad. Very sad, indeed.--
Any idiot can face a crisis - it's day to day living that wears you out.
-- Anton Chekhov
This is a written ovation to Tahatto's, "Romeo & Juliet: No Strings Attached".
I'm sure a teenager from the 90s set the score to this. The 90s teenager in me watched this and knew every sound bite was aimed right at me. To hear Pehla Nasha for Juliet's intro (and other Karan Johar references) and laugh needed someone who was 14 in the 90s - and still is.
If there is one line I will truly remember this play by, it is the monologue of Tybalt. He laments that the it is upto the villain to keep the plot moving.
And it is true, to let go of the plot takes perhaps a few good natured gestures from the villain. Perhaps to to walk away from Mercutio when the swords were drawn or perhaps even devious cunning to treat the Montague-Capulet romance as the foundation to eventually raise his own relevance in Verona. It would fall far from the material but if you need an example of what happens if the Capulets decide to lead Romeo on, watch malayalam's Godfather.
And in fact, Juliet too. Her rant about her wishes stands out in the play. Even removing the tinges of modern feminism from that monologue, it has to be admitted the men run the show. Neither Tybalt nor Romeo, well nobody at all, actually listens to her wishes for peace & love. Perhaps her suicide plot was a last ditch measure to have her thoughts heard, just like several other calls of attention which have ended in tragedy.
Even Mercutio haranguing the audience, breaking the fourth wall after his death. Only to ask if the audience is so jaded and detached, that they laugh at a man's untimely demise.
But the funniest of all are the multiple characters played by the two puppets, who despite switching characters often break to reveal that they play others as well. In particular, the death of Mercutio, only to be followed by Romeo's confusion over Benvolio played by the same. It is self-aware in a rather direct way that this is a play within the stage, performed by different hats rather than different actors.
Of course, for me the most meta reference was the play being run by the players themselves, with a very large & obvious presence of the lack of a director.
After the play is over, they decide to do it again, because to follow the script is the only way to get the girl, even if only for a little while. And therein lies the rub.--
O happy dagger! [Snatches Romeo's dagger.]
This is thy sheath; there rest, and let me die.
-- "Romeo & Juliet - Act V, Scene iii"
I made a lot of fuss when I was turning 30. It seemed like a big deal.
There's something special about completing a revolution around the yellow star.
It is special to know that you spent the last year doing the important things - writing code, climbing mountains, running distances and sometimes just doing nothing at all.
But what overwhelms me is a realization that the more important anniversaries are elsewhere on my calendar now.--
Like most women my age, I am 28.
-- Mary Schmich.
We begin this story in the middle.
Six of us have just clambered over last rocks onto the crater rim of the planet's highest free standing mountain. We felt like we were on top of the world, standing there with a cup of tea, with the sun rising behind us with Mawenze in sillhouette, a full moon on the western horizon and a cutting wind blowing from glaciers the next generation might not get to see. And for me & Kavi it was one hell of an anniversary!
The night seemed like a distant dream, from where we were. We'd started walking sometime before midnight, from School hut down in the east of the mountain. At first progress was slow and measured, legs freezing up as we took stops. Over the night we had left Navdha behind at Hans Meyer cave because she couldn't keep pace with the rest of us. She was going to make it too, but that was no place for waiting. We had to push on towards the peak on the west.
At Stella Point, we dropped off all our backpacks and headed up. Sleep deprivation was competing with the oxygen withdrawal at this point and with unsteady legs we got to the very top. Not that there's anything to see there, just a board and tickmark on your list. But it did mark a very important point in the whole trip. From this moment now, everything would be downhill.continue reading "Snows of Kilimanjaro"....
"Because it's there."
-- George Mallory (on climbing Everest)
Friday was my last day in Zynga.
Zynga has been a very different place for me to work in. Coming from Yahoo, where the tech folks had sway over the company, to a place where the product managers were the core of the company was jarring to say the least. But adapting to thrive, indeed succeed there has polished off a few rough edges that I had.
I've sat at the cliff's edge between what's management and the frontline. I've stared into the abyss and stared it down. I've seen exactly how incentives work and more importantly, when incentives tend to work against you. I've learnt to temper my competitive spirit to skip the usual pitfalls of meritocracy. I've understood how to work with people who are not engineers.
I've known the difference between talking & persuading. To effectively bring people over to your side of the debate - whether it is with statistics or merely by running experiments. In some sense this is more diplomacy than being technically right every time - but there's no value in actually being wrong. And I got better at saying No.
I've learnt what risk looks like. Very early in this "game", I learnt to play offense. To gauge a risk, take it and occasionally clean up afterwards if it blew up in my face. And in some sense that's the general motto around there. I became quicker at churning out a solution, battle testing it and minor setbacks weren't really a deterrant anymore. And I was brave.
I got to treat crisises differently. I was on the on-call rota. Instead of panicking and hyperventilating, I started treating them as merely opportunities to offer solutions. And in my spare time, I prepared for these crisises which turned out to be massively productive moments of innovation. I tried to get some of it opensourced, but a lot of it is locked up in the paperwork that I never finished.
I've had my moments of frustration too. There have been scenarios which were charged with egos and people politics, which are probably no different in any other workplace - people trying to play schedule chicken, folks trying to delay others with scope creep, territorial engineers telling me to keep off their turf & avoidable meetings where people duel with power point slides full of promises they can't keep. I've occasionally gotten demoralized about all that, but I've managed to move beyond that and build out a stronghold around me.
And finally, the best thing that's happened to me was working with the senior technical folks in the Bangalore office. People like Binu, Prashun, Prakash & Jayesh have been a big part of my development at Zynga. And not just them actually, the whole team is full of fun folks who don't really consider work as the end-all of their daily life.
It felt like I was part of an uber startup!--
The world hates change, yet it is the only thing that has brought progress.
-- Charles Kettering
I ain't trying to preach, I believe I can reach But your mind ain't prepared, I'll c u when u get there. I'd be a fool to surrender when I know I can be a contender If everybody's a sinner then everybody can be a winner. I'ma scuffle and struggle until I'm breathless and weak I done strived my whole life to make it to the mountain peak. Always keep reaching sure to grab on to something I'll be there when you get there with muted sounds bumping. We prefer to keep our eyes shut 'til it's right When there's something involved that we desire. So hold your head up high if you're poor and righteous I know times seem strife and problems seem endless. But at the times of despair we gotta put ourselves together And if you feel you're out of the game. Then you need to get back in it 'Cause nothing worse than a quitter. You gotta face responsibility one day, my brother So wrap up your pity and turn it to ambition. And put your vehicle in 'Drive' and stop by my side. As we walk down the road of our destiny And the time comes to choose which shall it be The wide and crooked or the straight and narrow. We got one voice to give and one life to live. Stand up for something or lie down in your game. Listen to the song that we sing. It's up to you to make it big I guess, I'll c u when u see me. I'll c u when u get there.
Lyrics from Coolio's C U When U Get There.--
It is not what we read, but what we remember that makes us learned.
It is not what we intend but what we do that makes us useful.
And, it is not a few faint wishes but the struggle that makes us valiant.
--Henry Ward Beecher
Nothing is ever perfect: Perfection is a quest, not a goal. Sometimes it feels good to be done with something, put it aside and move onto the next thing on your mind. It might be half-assed, but it is not just a thought anymore, it is out there. Bear in mind, it's also out there for people to criticize - but that's not a thing to fear. Carry on the quest though, but look for it in your life not in others.
There are shortcuts in life: Sure there are shortcuts, they rarely look like one. I'm lazy and that has made me look for these a bit more often than others. It took a great teacher to explain to me that education was what got him out of a hard life. Made the whole process look like liberation instead of the enslavement I used to blame the school bell for.
Rewards are not for effort or patience: You can't argue with results. The only time to talk about the effort is when it goes wasted.
Money is important: I wouldn't blame everything on money, but the lack of money is the root of some evils. Growing up in the wide middle class of Kerala, almost every limit in life I saw was about money. I know it can't buy everything, but removing that limit lets me focus on the things that can't be bought or sold - also known as the important things in life.
Luck is overrated: I think so. But who can really tell?
Never let fear stop you & never let greed drive you: I'm not exceptionally brave. But I've maintained the little inner strength I have by trying to reflect on my motives. And in cutting down the knee-jerk reactions to fear & greed, I've torn out the insecurities that fed my self doubt. No, I'm not afraid she will leave me and I'm not waiting for my next raise - I care that she's happy and I need the money, but I'm here to kick ass & chew gum. And I'm all out of gum.
Sometimes the right thing to do is to walk away: You don't have to win every fight. Trust me, you don't have the time to fight every loser who steps out. So don't start a fight you don't want to finish. And when it comes to true confrontation, be as Teddy "Bear" Roosevelt said, speak softly & carry a big stick. Whack him good.
You've been wrong before & you'll be wrong again: This bears repeating - being wrong is important. Just one advice - eating your own words is painful. Even more reason to make them sweet in the first place.
When you're wrong, change your opinion: What's the alternative?
Don't be a rebel, change something instead: I probably had my teenage rebellion way late. And it went about this by turning my world upside down and shaking it till everything that wasn't nailed down fell out. I thought I was watching the world fall down on me, but something else happened. And I came out of it armoured with purpose.
Intelligence and skill doesn't scale: Eventually there's just 24 hours in everybody's day. And I'm not going to put in all 16 hours into work - I have a life. But more importantly, I've learnt to work away from my loner approach to doing things (instilled by the student-beat-student education system) into doing things together. Cooperation scales way better and more easily. I wouldn't say this if I was Roger Federer, but I'm not.
You can't be anything you want: Nothing's stopping you from being all you can be. It'll take a while to figure out what limits we see are real and which ones are self imposed. Ambition will let you find that out, but that's when you need to take a reality check & cash it.
Everything's harder than you realize: Our society respects people who are "brilliant". In honesty, I can't really blame society for it when there's an easier target out there - The Karate Kid. Anyway, effortless was another word for brilliant. And I wanted some of that - so I tried to get that tag for myself. But in the process I learnt something, the things I love are easier for me. The sacrifices are easier and time just flies when I'm having fun.
Failure is OK, Defeat isn't: Sometimes you give up. Life's going to keep bringing up this topic, but the day I can't get up when life knocks me down, I'll be truly defeated. I plan to get out of this life undefeated.
Do not live in the present - tomorrow does come: It's a hellish momentary existence to live in the present forever. Hopes and dreams do not have a place in it, because there is no tomorrow. I've nearly killed myself to escape it & swore never to return there. I forever live in the boundary between days, sometimes wallowing in the past and sometimes pushing towards the future. Today is merely tomorrow turning into yesterday.
May not be much, but this took 10,000 days to come into being. And perhaps, just perhaps, I'm writing this down to read someday later.--
22,000 days, it's not a lot,
it's all you've got
-- Moody Blues
To put a price on a thing is an odd thing to do.
I've never sold my photographs. I'd dreamt of doing a photo exhibition. I wanted to do more than show them only to take them home afterwards. I wanted to do something good along with it - to auction them and raise money. I didn't know what putting a price on one would feel.
But before I wade into my moral dilemma, let me thank the people without whom this wouldn't have been possible - Kavita, Tharak, Kritika, Pooja & Vivek. These five people were the hands, feet, brains, heart and occasionally filled in for all the other organs that were needed. Kavi was the mastermind of this project. I might've taken the photographs but she put on this show. Tharak for all the posters & the artwork in the coffee table book. But my gratitude really goes out to the last three, who flew in from Delhi & stayed the whole week to help!
Back to my dilemma. It wasn't as if people were paying for the photographs. They were paying for a visually impaired student's future & education - that this framed photo was merely a glorified thank you card, from me to you, all paid-for by me. But it did feel like I was making a sale.
Until that point in my life, I'd marked a large number of my photos online as CC-BY. All anyone needed to do was connect my photo to the original upload & use it as they please. They ended up on song cover art, safari brochures, treehugger articles, wikipedia and god knows where else. And if I cared, it was only to know that they were of use to someone.
As an auctioneer, I was conflicted. I wanted to give everyone the picture they liked, but I had to push the price up for the cause. Bidding wars may push the price up, but they do so by denying someone their wish to take the photo home. I wasn't happy to call up someone and tell them they weren't getting their photo of choice - I wanted to prevent people from outbidding others. At the same time, I wanted to hit my fund raising target.
I felt like I was the bad person here, keeping people from having what they want - for money. Generating artificial scarcity for something that was still available at no cost - I felt like I had sold out. Felt like I was doing something morally wrong, cheapening the sensation of having raised a whopping 1.3 lakhs for mitrajyothi. I might be feeling a bit low about the auction, but that money is going to go do a lot more good to a lot more people. But I do have something to re-iterate about my photos and your right to enjoy them.
My Creative Commons photos have always been available for anyone to download - use one as your wallpaper, take a print & frame it or paint a copy.
Just leave a comment or something, so that I have a reason to keep uploading more.--
The essence of all art is to have pleasure in giving pleasure.
-- Dale Carnegie
Divide and conquer. That's how the web's scaling problems have always been solved.
And the tier scales out horizontally for a while. You scale the tiers and everything works. But sooner or later you end up with a different problem - latency. The system gets choked out with the interconnects, but the magnitude of the problem is just mind boggling.
Let's take a random example - imagine about 500 memcached servers, 1000 web nodes and 64 processes on each node. Simple back of the envelope math turns it into 32 million persistent connections going on at any given time. But I'm assuming the worst-case scenario - only because that's what's in production in most places.
The real problem is that the preferred scale-out approach for a web tier is a round-robin or least-connection based fair distribution of requests. That works great for read-heavy throughputs where the key distribution is not tied to a user session. But if you ended up with a scenario where you are operating on only one user's data per-request, the wastefulness of this scenario starts to become evident.
What I really want is to pick a web-node which is currently optimal for this particular user's request. The traditional approach is to pick a node and route all subsequent requests to the particular node and hope that I can do at least a few stale reads off the local cache there. We want to go further and pick an optimal web node (network-wise) for this user session. Since the data layer gets dynamically rebalanced and failed nodes get replaced, the mapping is by no means static. Not only is that an issue, strict pinning might cause a hotspot of web activity might bring down a web server.
The solution is to repurpose stateless user pinning as used by HAProxy to let the web tier rebalance requests as it pleases. We plan on hijacking the cookie mechanisms in haproxy and setting the cookies from the webservers themselves instead of injecting it from the proxy.
Here's how my haproxy.cfg looks at the moment
backend app balance roundrobin cookie SERVERID indirect preserve server app1 127.0.0.1:80 cookie app1 maxconn 32 server app2 127.0.0.2:80 cookie app2 maxconn 32 server app3 127.0.0.3:80 cookie app2 maxconn 32
That's pretty much the complicated part. What remains to be done is merely the php code to set the cookie. The following code is on each app node (so that "app2" can set-cookie as "app3" if needed).
$h = crc32("$uid:blob"); # same used for memcache key $servers = $optimal_server[($h % $shard_count)]; shuffle($servers); $s = $servers; header("Set-Cookie: SERVERID=$s");
As long the optimal_server (i.e ping-time < threshold or well, EC2 availability zone) is kept up-to-date for each user data shard, this will send all requests according to server id till the maxconn is reached. And it fails over to round-robin when no cookie provided or the machine is down/out-of-capacity. HAproxy even holds a connection to reroute data to a different node for failover instead of erroring out.
And nobody said you had to do this for every user - do it only for a fraction that you care about :)--
“Many are stubborn in pursuit of the path they have chosen, few in pursuit of the goal.”
-- Friedrich Nietzsche
And I'm concerned.
I'm concerned that she does not understand the difference between marriage and love. That she does not understand the difference between love and sacrifice. That she does not understand the difference between loving someone and living with them. That she does not understand that marriage is not the finish line for love. That she does not understand marriage takes more than love.
And the more I think about it - I'm even more concerned.
I'm concerned that her apathy towards her career will be the death of it. That she will end up being a homemaker, after having a gold medal from NLS. That she will end up succumbing to the social pressures of being a Mrs Somebody. That one day somewhere in her middle ages she would regret her simple surrenders. That if she's wilfully ignoring the springboard of opportunity that so many of her peers don't have. That she might be decieving herself into becoming the role model wife and bahu - and fail. That she'll lose something precious and ephemeral - the fire to burn as bright as you can.
And in some way, I'm saddened.
I'm saddened that whenever she explains her idea of relationships it is to suffer for the sake of her partner, to be with him whatever it takes at whatever cost to her potential. That I couldn't pass off my own ephiphany about love and relationships - that it took me further ahead than I could've done on my own. That my efforts to fulfil my dad's hopes for her weren't enough - simple enough as they were, to see her "breaking through the limitations society might impose". That she won't see that in some way or the other, I've been on her side for twenty odd years in the case of Sister vs The World, 1986-present.
I'd don't think I'll ever stop being concerned about her - she's my baby (sister) too.
But in more than one way, I'm happy for her. And proud of her too.
She's marrying someone of her own choice - after six years of knowing each other. And it's happening out of co-operation between both the families. She's doing this completely out of her own volition, dreams and desires - which is what will truly give her the strength to push through when push comes to shove.
Also, she's been known for making me look very stupid - which this blog post might do in a couple of decades.--
Marriage has some thorns, but the alternative has no roses.
-- Vernon K. McLellan
The last time I drove my bike fast was in June.
No, that's not true. I drove it fast today. Too fast, perhaps. But that leads me to another question, how fast is too fast? I'm not talking about the speed limits or the consequences of speeding. I find that my sense of speed is completely off depending on what I'm driving - I've hit 65 on a bicycle downhill, on a 75cc two wheeler, on a 350cc cruiser and of course, in various cars. The sensation is very different, but in reality the speed is the same. So, in a rather objective sense, can I tell my brain what my driving speeds really mean?
I've always been wary of heights - though not exactly when I've been climbing. I've jumped off enough tall things to have a sense of how it feels. Maybe that's the way I can explain this. Let's take 3 different speeds and work out how they feel - in "jumping out of the window" terms.
The calculations are rather trivial.
def height(v): g = 9.80665 # m/s^2 # convert to m/s vm=v*(1000/3600.0) # v^2 = u^2 + 2as s = (vm*vm)/(2*g) print "Height to fall from (for %d kmph) = %d metres" % (v,s) height(40) # = 6 m height(60) # = 14 m height(80) # = 25 m
Now, the numbers clearly indicate the height goes up to the square of the speed you're driving in, rather than than linearly (duh, kinetic energy is proportional to the square of velocity). But what does scare me is what I've added for perspective is an olympic high dive, which sort of looks like this. Not to mention, you aren't going to be hitting a nice deep pool of water.
And I still drive fast.--
It is impossible to travel faster than the speed of light, and certainly not desirable, as one's hat keeps blowing off.
-- Woody Allen