< January 2017
1 2 3 4 5 6 7
8 91011121314
Mon, 30 Jan 2017:

We're into the second act, people!

This is a moment of despair, but you know what the second act is. It is a moment of character, where you get to find out if you're part of the narrative or not. Some of you reading are definitely looking away, but this is really for those who are staring out at the world in despair, watching it stare back into you.

Remember what the second act is - it is the rising action. The heroes' attempts to fix the crisis leads to ever worsening situations. And they can't win, because they haven't got the skills to deal with the forces of the other side. They must not only learn how to wield the light saber in attack, but also to believe in the force that flows through.

There's a part of me which wants to unplug the news and go back to where I have a vote. I could've done a runner with a clean conscience, refusing to fund the process with my taxes. But I've got a tiny american with me, who never got to decide what his future looks like. I cannot remain idle in his defence, but I didn't sign up to die trying.

So, I must think of the second act and what it demands.

Bassiano: And I beseech you,
          Wrest once the law to your authority:
          To do a great right, do a little wrong,
          And curb this cruel devil of his will.

Portia: It must not be; there is no power in Venice
        Can alter a decree established:
        'Twill be recorded for a precedent,
        And many an error by the same example
        Will rush into the state: it cannot be.

In this fight for essential liberties, the rule of law is overturned, we are lost. I hear the words of the lovely Portia from the Merchant of Venice. Thus we must look to the lawyers, to ride in like the cavalry. In Shakespeare's words of "The Butcher" in Henry V, lawyers for sure remember the laws and not easily forget that all animals are created equal.

JACK CADE: I thank you, good people:- there shall be no money; 
           all shall eat and drink on my score; 
           and I will apparel them all in one livery, 
           that they may agree like brothers, and worship me their lord.

DICK:      The first thing we do, let's kill all the lawyers.

The soap box and ballot box have not helped, let's just hope the next box has got enough stopping power. I'd hate to get to the last box, because down that way madness lies. To learn politics and law, so that my children may learn literature and art is no sacrifice, but the last box leaves us without civilization.

I don't know how this will all end. I'm not anxious anymore though, because there's one thing I worried about - that world would turn away in apathy. Because in the end what we will remember most often will not be the words of your enemies, but the silence of your friends. Do not imagine all is lost, do not mistake being out of power with being in the minority - there are more of us, when the numbers are in. This plot arc is long and we're just into the second act.

And though wise men at their end know dark is right, do not go gently into the good night!

Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.

posted at: 03:00 | path: /observations | permalink | Tags: ,

Fri, 02 Dec 2016:

This is perhaps the darkest week of the year - every year since 2006.

A decade later, once again, I feel like nothing about me will be the same again.

This new sense of purpose I feel is a few months old, sleeping a few feet away.

The baton is passed, not quite yet and I haven't let go.

"Life is effort, and I’ll stop when I die."
      -- Rick & Morty

posted at: 00:34 | path: /me | permalink | Tags: , ,

Thu, 08 Sep 2016:

Today's the first day of year 5 for me in Hortonworks.

I've now spent more than a decade getting paid to do open source work. But until Hortonworks, all work I did was tangential to the business arm of the company. All my customers were internal, which meant mostly I got privileged access to the problems. The bug resolution for complex issues never had to deal with walls of customer solutions obscuring my insights into the system. There was never a day when I couldn't tell someone from Yahoo mail that I was working on another issue with Frontpage.

During my Zynga period, I learnt how to sell your ideas to a product focus team looking for solutions - to explain better, to justify my arguments with numbers, to convince and to persuade. To keep my sanity, I also learnt to pick my battles, drop issues and walk away when the friction gets personal.

I never had to sell to a customer direct - even at my best, on the books of the company I was a cost, an engineering expense. My work at Zynga or Yahoo, was helping the company bring down its bottom line by a fraction without ever touching the top line, except by accident (yay, faster load times).

This is generally an area of work with no real conflicts of interest built into it - use of APC for instance was mandated and in most cases, my code was the only available solution to the problem. Giving up on APC meant switching to HipHop, which also I worked on (never got the CLA signed by Zynga, which was sad though). The pattern followed me through out the period even through ZCloud build-out, through ZPerfmon, ZProf and pretty much everything I touched.

Hortonworks has been a completely different experience. I'm still a developer, but my contribution to the company has turned into a top-line enlarging investment in most scenarios. The work I was doing with Stinger was actively bringing in more customers and indirectly, more revenue in. This brings in some interesting conflicts for a developer, when talking to a customer about your features, because that particular transaction starts off as a zero sum exchange of money with the implicit declaration that the "solution" is a non-zero sum win-win.

This shift in the nature of my job has been interesting. The role now requires a completely different social skill set which I didn't need in any job before this. The goal is nearly the same, but conflicts fall onto my lap instead of being able to just say "Zynga first" to end a discussion.

Honestly, it's been a bit of a culture shock.

I feel like I'm astride an elephant, with my toes pressing suggestions of direction. The benchmark numbers are just numbers, unless someone else decides that those need some action. I cannot actually move the beast, but the beast does move, never to stand still.

As always, I've stayed on top of that beast here for the last four years to learn and cope with that process. The learnings are rather harsh, that while in a single company you get to deal with your users first, the actual users in this mode don't run the systems, those who run the system don't make purchasing decisions and those who write the checks don't really deal with either of those folks directly. And in the end, the check they write is eventually only fractionally related to my paycheck anyway.

In that world, it feels ultra depressing to sit in on a call where the dreaded phrase is said - "what the hell do we pay you for?" . There's the sub-text in your head going - uh, you don't ... I'm here because I am the only one who can help with this and this is not even my job, neither is it my turn on the on-call roster, so get off my back. But you learn not to lash out in knee-jerk fashion and close down the conversation, but continue debugging which is upto your discipline, a thick skin and the knowledge that your self-worth isn't tied up in that guy's opinion. I still don't know how to communicate that Insulting my commitment to your problem isn't helping - making it unpleasant might feel better, but you're burning good will here. Looking back, some of what I did never made any sense in the big picture, to feel the pressure from the sales cycle to deliver something over the christmas weekend and then get back on it through the new year weekend too. However hard those weeks turn out to be, the rewards for such heroics diffuses away as the company gets larger and as the customer base expands outwards - no deal ends up being the end-all of your work.

I still love solving those tough problems and solving them are their own reward, but the process friction is draining for your attention spans and your ability to take criticism for no fault of your own wears thin (I'm rarely the person who wrote the slow bit of code I'm fixing and I'm unlikely to be the one in the next case either, *sigh*). Collective responsibility is hard with open source, when this issue was because of someone back in 2009 who thought this was good enough, way before the YARN and Hive teams sat under the same roof. It's nobody's fault, but remains your problem.

There are some upsides to being thrown about into the rough and tumble of that process, you develop a lot of breadth in your skill sets out of sheer necessity - whether it's about something esoteric like Kerberos or the specifics of file listings in Azure or S3. There are technical savings you build up over the years, which work exactly the opposite of how technical debt works - readymade solutions which you refine through one customer aftet the other, to feed back into the product as much as you can. This is what a front-line engineer does in a small company, to carry a huge memory bank of all the problems cross-referenced to their solutions. A bigger company divides away expertise until no one person has the entire picture and start operating in managable fractions of expertise.

I came into Hortonworks, expecting to fix MapReduce and HDFS, possibly to dig deeper into HBase latencies - it is odd to say that right because instead of polishing old code, I got to work on brand new ideas with Stinger on Hive, ORC and Tez. But unlike my work at Zynga, I've never had to bury any of work (and to have the company salt the earth behind it) - my patents from Zynga are clearly dead-ends which I cannot pursue somewhere else, original ideas which I had and lost posession of. Putting up every little thing I build onto a github repo has been a liberating experience, being able to talk about my work in great detail to everyone including business competitors gives me a sense of autonomy which I have not experienced since college.

There have been so many mishaps along the way too. It was a horrible idea to work as a contractor in India when Hortonworks failed to hire me in time for the H1B cap - every bit of my immigration that followed has been a pain because of the fact that I was an Indian contractor moving to the US. Unfortunately it's too late now that I'm already here and is probably beyond recourse, I would probably have to quit or move out of America to reset that, in the near future. And that's sort of been the only regret of mine in joining Hortonworks the way I did.

But this is the second longest I've worked in my career and this journey has been very different from the Gumball years of the Yahoo. With Hortonworks, I joined a small company in Maude & Matilda four years ago and I now work for a company that is so big it has offices around the planet. When I interviewed at Hortonworks in February of 2012, that was way before the this company was - back before there was a packaged product or an installer, before the "do or die" phase of working for an underdog company.

And an underdog might be down, but always gets to punch up.

Boredom is the feeling that everything is a waste of time; serenity, that nothing is.
          -- Thomas Szasz

posted at: 22:40 | path: /me | permalink | Tags: ,

Wed, 20 Apr 2016:

My grandfather passed away last week.

When I was at one of life's cross-roads, he rescued me with some advice I hold onto as one of my life's truths.

Sacrifice. It's a word that gets thrown around in the social context without any inherent meaning. Just like other things caused by peer pressure, the sacrificial act is a social demonstration of emotion, at the detriment of nearly everyone intimately involved in it. Mothers give up their careers to stay at home. fathers end up working longer and so on.

After my dad's untimely end, there was a choice left to me - envelope pushed under the door of my mourning, if you will. Move back to Trivandrum and try to support my mother & sister while they mourn the inconsolable loss. Social signals told me clearly that it's what's expected from a "good son".

I thought otherwise. I was wracked with guilt at the thought of staying in Bangalore, chasing my own life's goals which had been on hold for the couple of years my father was ill. And I couldn't bring myself to ignore the social pressures and was caving in to them, when I talked to my muthachan.

He got to right to the point.

"Is this something your mother asked you to do?" / "No". "Do you think leaving Yahoo and moving to Trivandrum is good for you?" / "No". "Is moving back home likely to be a disappointing thing for you?" / "Yes".

"Then don't do it. We're here to take care of your mother. She is my child." / "Ok, but ..."

"Your mother will need you in a few years, your sister will need you. Be there. But you will do this one crazy gesture they never asked for, because people pressure you into making it. When times are bad and your life isn't moving forward, you won't think of those pressures. You will blame the ones you really did it all for."

"And one day, you will get angry about something and tell your mother that you did it all for her. That your life's frustrations are due to her. Maybe it won't be your mother and it will be your sister - she will tell you that she didn't ask for it, that you did it all on your own. That your sacrifice wasn't necessary and then you'll feel hollow, because it was all for nothing, how you turned away from your self, silenced your wants and died a little for them. You will be truly angry with them for your thoughtless sacrifice and never let it go."

"If what I said sounds inevitable. Don't do it. Go do your thing now and whenever your mother or your sister need help they will not hesitate to ask. Be there for them without question. When it's the right decision to take, you won't feel yourself tearing apart."

That is how I got to today, half a world away mourning his death, knowing fully well I couldn't go & be by my mother's side.

And this time, I wasn't tearing myself apart.

Much of what was said did not matter, and that much of what mattered could not be said.

posted at: 02:51 | path: /me | permalink | Tags: , ,

Sat, 06 Feb 2016:

We'll see what happens. Yes, but ... OK, I will try.

There are so many ways to say No. Some of you might've realized that's what gets said. But I've seen very few who understand why it gets said in circuitous soft words, other than attributing it to race or upbringing.

Because of what happens when you say No. There's always someone else who's waiting to say Yes. Time after time, each No you say shrinks the number of times you get asked and life's opportunities show up as questions. The fewer times you're approached with a question, the fewer times will you get to say Yes!. Each No shrinks the funnel through which other people interact with you.

Being reasonable produces the opposite result you want. Those that say "Yes, let's do this!" and flake out are often asked more often than those who can offer a negative in time. In fact that those who say "No, I'm meeting a friend tomorrow" only to have their friend flake out is worse off than someone with double bookings.

Whether the opportunities are social, personal or professional, those that a "Yes, we can try" are preferred to the respectful No. Of course, we prefer the "Can Do" people here.

The definite negative implies intention, while the loosely held hope is always treated better. The negative is interpreted as sub-textual while the other is taken as a circumstantial "stuff happens" even when the eventual end result is the same - probably worse for the "maybe" because it steals away the ability to plan an alternative.

In such an environment where all Yes-es are Maybe, being "Definitely Maybe" is understood well - and saying "No" becomes a "No-No".

In short, when it comes to saying No - I'll try.

The right word may be effective, but no word was ever as effective as a rightly timed pause.
            -- Mark Twain

posted at: 23:35 | path: /rants | permalink | Tags: ,

Tue, 25 Aug 2015:

Must be a strange coincidence, but I think nearly all of my farewell letters have been written in August.

As the days get a tiny bit shorter, all the energy that the summer brings turns into a certain impatience. Sitting in front of a desk, when the brightness outside is getting wasted.

Perhaps you don't feel it, but there's a general ebb of life that makes me acutely conscious of my mortality. A certain knowledge that summer is passing and I'm still indoors, to wait for another year perhaps to enjoy the summer sunshine.

And yet, it's not that I feel sad about the passing of days - those days aren't here. It's a call to freedom. A primal urge to make most of the days in the sun, without concern, because the winter that will follow is inevitable. To plant all the energy of the summer that's half-gone and reap the rest.

Need to just head out into the sunshine & high mountains, leaving work in the valleys.

Your career is like a box of chocolates - you never know what you're going to get.

posted at: 16:27 | path: /me | permalink | Tags: , ,

Mon, 06 Jul 2015:

Here's a short quick set of questions.

Do you work heavily whenever a project causes disappointment, when you are under pressure or have had a quarrel with someone?

Have you built up a tolerance to your life style and developed strategies to cope with lack of sleep when driving?

Have you gone back to work in the morning only to not understand what you were thinking when you wrote some code last night?

When you work in a team, do you find yourself taking up an unfair share of work automatically?

Do you ever feel uncomfortable when you are too far away from work stress, particularly if you've turned off work email?

Are you in a hurry to get some part of your work done, before you have to meet the rest of the team for a standup?

Do you find yourself picking up some work to do, after you make sure your wife and kids are already asleep?

Has a family member or friend complained about you preferring to work instead of going to a social occasion?

Do you ever promise to meet for an event (kid's recital, date night) and missed it because you were caught up in work?

Have you noticed that despite spending more time at work, you get less and less done these days?

Do you eat very little or irregularly, during periods of overworking?

Do you ever feel depressed or anxious during or after periods of heavy working?

Have you tried to change your work habits for several attempts without achieving any real change?

Do you ever find yourself making excuses that this is entirely temporary, while consistently overworking for months?

Do you try to cram in as many lifestyle activites as possible into a single weekend, to make the break from work count?

Do you ever find yourself wondering how other people find time for their hobbies?

Do you motivate yourself with thoughts that when one day you're rich from all this work, you will be able to retire and pursue hobbies?

If this myth is tragic, that is because its hero is conscious.
Where would his torture be, indeed, if at every step the hope of succeeding upheld him?
The workman of today works everyday in his life at the same tasks, and his fate is no less absurd.
But it is tragic only at the rare moments when it becomes conscious.
        -- Albert Camus, "The Myth of Sisyphus"

posted at: 21:48 | path: /observations | permalink | Tags: , ,

Tue, 18 Nov 2014:

It has been 7 years since my last conversation with him. Maybe it's only been six since I missed him.

Maybe I didn't say anything to anyone when he died, but with all the lack of orginality I posess, let me repeat something. If only as l'esprit de l'escalier, 8 years later if you will.

He had his... shortcomings, but he took care of his family. 
He loved his family. He loved this house. He worked hard. 

All I wanted today was to show him how much we all loved and admired him.

To give him the respect that he deserved... Is that really so much to ask?

We all get so distracted by the... little things in life.

We forget about the important things. Like the fact that we lost a great man.

We don't know why a man makes the choices he makes.

But I do know that my father made the best choices he could make. 

Life is complicated. 

We're just thrown here together in a world filled with chaos and confusion... and we do our best.

He taught us to go for what we wanted in life, because you never know how long you're going to be here.

So when you all leave here today, I want you to remember him for who he really was. 

A decent, loving man who never condemned anyone for how they lived. Who never cast disparaging remarks 
or held prejudices against race, gender... height. If only we could all be as giving, as generous, 
as understanding as my father. 

If I am half the man my father was, my child will be incredibly blessed.
            -- (paraphrased from) Death at a Funeral, 2007 

My father was an exceptional man. He was exceptional in every way, in his strength and his weakness.

I am my mother in weakeness and my father in strength. He's the part of me who can plot paths to wins, she taught me how to fail & be undefeated.

My mother taught me how to endure, to abide, to suffer the endless slings & arrows - my father showed me how to fight, how to lose battles and win wars. To take invisible swords upto invisible monsters and slay them. If my father taught me to fight, my mother taught me to never give up. She passed onto me, her endless source of hope. She taught, if not by words but example, that everything passes, including waves of troubles & sadness. Just hold your breath and dive, she showed - find the bottom and push back all the way up.

I learnt strategy from him - I very rarely played chess with him and he would reluctantly destroy me in calculated moves. I rarely put myself in front of that juggernaut because of what came after I lost, a clear description of why I lost, step by step of when I had an advantage & how I squandered it. I learnt patience from watching him - that patience isn't waiting, patience is observing, watching for the right moment - with a finger on the trigger, eye on the scope.

I learnt to listen and remember. Of all of my life's conversations (yet), he's the person I've talked the most to. I missed the conversations with him about fundamental human nature, sitting on a Bajaj Chetak, with him explaining to me in great detail the importance of understanding. Nothing more, just understanding everything. Perhaps, I learnt to be pedantic too - to understand what's different about wealth, value, cost, price. Understand that money is important, but never a goal. Understand that you should always engineer all-winner scenarios and never work with people who will still want you to lose. Understand altruisim & co-operation is just another word for long-term selfishness. Understand that help you give that is nothing to you, but everything to someone else is the best form of it. Understand that gender is meaningful, but not discriminatory (between me & my sister). That laws we have to obey are never entirely fair - that both the rich & poor are prohibited from sleeping on park benches in the night.

He didn't just talk about it, I saw his principles moving him - some of those came up because he was the Director of Social Welfare in Kerala government.

Not all his lessons were imparted kindly. I watched over my father for a year. I felt like the parent - making sure he ate food on time, took his medication, to make sure he slept, to run to his bed-side when he had nightmares. To stay up next to him when he slept in his hospital bed, reading, while the sedatives kicked in and kept him from tossing about in his bed. I had to deal with him when he threw tantrums and I know it hurt him more when I couldn't understand him or calm him down. I was filled with a homesickness that only comes to those who are already home and I wanted him to go back to being the same awesome father I always had. Deep inside his heart, he knew he could never be that person again. And he tried to tell me, but I didn't believe that - to believe him seemed like letting go of all hope. Never wanted to pass through the door inscribed "Abandon all hope, ye who enter here".

Years after, in mid-November I get angry about all that again, I remember that he was the most considerate of human beings even to the end - I found his paperwork sorted and filed in the order of use after his death.

I wonder about all our conversations - was he was leaving a part of him to live, even after he takes the rest to an early grave?

And just for that, I will always love him.

All that we love deeply becomes a part of us.
      -- Helen Keller

posted at: 11:47 | path: /me | permalink | Tags: ,

Sun, 20 Oct 2013:

This is very definitely not about computer monitors. It is a story about data hashing, distribution and indeed re-distribution - about divisors, primes and more than all of that, downtime & costs.

In all honesty, I'm just posting a conversation from a coffee break almost two years ago (and some context to frame it). I'm writing this down now because it has reached the edge of my memory and is about to fall off.

One day, Prakash in the middle of coffee started to talk about the pains that powers of two cause. I assume, it was because one of the membase clusters (now called zBase) was running out of capacity and needed to be doubled in size.

Doubling in size was the only way our clusters grew, which was all well & good when the product was on its way up. It means you can double in size rather rapidly and without any real downtime. But it does cost a lot of money to keep doing after the first or second rehashing. Doubling made even more sense in a master-slave setup because the old cluster became the new masters and dropped half the data each, which would then be caught up by a new array of slaves which meant it would happen without any downtime for data copying and was a nearly no-risk operation.

This is relatively straight-forward, because the masters only have key invalidations happening during the re-shard. No data actually moves between masters during the reshard. This is brilliant because a few seconds after you flip the switch, the master array is ready to update and the slaves have a few minutes to catch up. But this is very expensive. Say, you had a 4 node cluster serving out data and eventually node1 runs out of space, then you need to add 4 more extra servers where otherwise only 2 would have been sufficient.

But this is primarily a one-way street. You cannot shrink a server cluster to half its size by this method.

Most startups never need to do that - but this kind of trickery needs to be done to optimize costs in a more mature product. And more relevantly, doubling a cluster to go from 8 to 16 to fix one node that's going out of capacity is so wasteful. If you are prepared to move a lot of data around, you don't necessarily need to double it - but that's the question, the downtime involved in moving the data is usually too much to justify that approach - but at least some of the data moving problems can be bypassed by more intelligent hash partitioning.

The most common hash partitioning method used is a modulo of the hash. Where you take the hash value and % by the number of servers to end up with a server where the data resides. But this has catastrophic consequences when rehashing data across multiple machines. Let us run through the simple scenario of data distributed across 10 machines and adding a 11th machine. So if we follow the hashing algorithms' approach, the only data that would sit still would be items with hash keys which are a multiple of both 10 and 11. For a uniform distribution of data that would approx 0.9% of the data.

Or in other words, with such a stupid hash partitioning mechanism, you'll end up moving 99.1% of data around to bring the cluster to order. So someone was bound to invent a method to work around the doubling problem.

When I was in Yahoo!, I ran into one of the most brilliant solutions to this problem. A mathematical, clean solution - but one which I notice hasn't even been patented. This is somewhat sad, but it shouldn't be lost to the world because it hasn't been published. Basically, it is a hash based solution that outputs a partition id instead of a hash value - basically, when you move up from a 10 node to 11 node cluster, the data only gets moved from all 10 nodes to the 11th node. And with a uniform distribution, each node only loses 10% of data.

There are tons of examples of this approach of consistent hashing, the easiest to use would be the Ketama hash. Or if you care about its origin story, you can see the genesis of this idea elsewhere. But the trouble really is that even with a consistent hash, you'll be moving around individual keys and value pairs to balance your data out. And there was no real clean way to drop data from the machine quickly without going through everything.

As with the revolution pallets caused in warehousing and containers did for shipping (or Quantum theory for physics), the idea of putting data into indivisble containers was the solution. On top of that, having partitioning be independent of machines was a rather easy way out of this particular problem in membase. Basically, bucket the data into a fixed number of partitions independently of actual machine count and then have a second order lookup map which actually tells you where that particular bucket is residing. This would be catastrophic if the clients went out of sync in this mechanism - but all a server had to do was say "Not Mine" when it sees an update or get to the wrong bucket. The client could then refresh the 2nd order mapping lazily without having to get a push update for each reconfiguration of the servers.

Since you move buckets between machines instead of keys and values, it is easy to proxy all updates to a given server into another with a filter on the bucket id (instead of failing - the failure is triggered after the move is finalized), avoiding downtime during the rehash. And you can just drop off the hash for the bucket once it has moved all data from the current machine to another. The locking, deallocation and movement of data now happens in a larger chunk as single transaction - a break in transaction does not leave either side in an inconsistent state. Just drop that vbucket in destination, pick another destination and serialize the in-mem hash for that vbucket for streaming it over.

In some human way, knowing which of those buckets have data/key-space/traffic allows a fairly involved user to actually micro-manage the resharding. And ideally, you can reshard down into a smaller cluster with this approach faster by hand than you can with the ketama, which is likely to make you go nuts since it is a spectrum of keys instead of being wrapped bundles. It is far easier to write a packing algorithm in 2 dimensions with such "rectangular" blocks - x being data size and y being traffic.

Now is when things start to get really interesting, from the Prakash angle. Almost everybody uses a power of 2 as number of these buckets. I remember that membase used to use 1024 by default and I talked to people at Aerospike recently, where they use the last 12 bits of the hash to determine virtual buckets (for the innumerate that is 4096 buckets). Now there is no fair way to distribute these numbers fairly across server counts, naively. According to Prakash, a number like 1440 would've been far nicer. And I did some grade school math to explore that. Basically, the number of fair distribution values below 1000.

1440 : [1, 2, 3, 4, 5, 6, 8, 9, 10, 12, 15, 16, 18, 20, 24, 30, 32, 36, 40, 45, 48, 60, 72, 80, 90, 96, 120, 144, 160, 180, 240, 288, 360, 480, 720]
1024 : [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] 

In essence, 1024 has 10 divisors and 1440 has 35. In the 1440 case there are 3x number of possible fair distributions of uniform data than in the 1024 case. The scenario is even better when you look at the small digit numbers in the distribution. The first 10 numbers for 1440 are 1, 2, 3, 4, 5, 6, 8, 9, 10, 12. All of which are fair configurations of bucket distributions, where as the 1024 case jumps straight from 8 to 16.

The real commentary here is on the lack of magic in 1024 (and 4096). Also slightly about people who pick prime numbers in this scenario - they work better for some key distributions, but lack the divisibility (duh!) that simplifies the distribution. This number choice is something you have to live with once you pick it, because changing it requires as massive a data movement as the original modulo distribution (because let's face it, the vbucket is a modulo distribution, with modulo picking the vbucket). And for those who picked a power of 2 already, I have to say that it is possible to acheive an ideal distribution of data/traffic irrespective of the actual number of vbuckets. When a server in a group runs out of capacity, just move the highest capacity vbucket in it out onto a new node. It is that simple - the default number only sucks in scenarios of uniform distribution and uniform traffic.

1024's just an imperfect number which will work for you in an imperfect world. Which I am told is often the case.

There is no safety in numbers, or in anything else.
  -- James Thurber

posted at: 23:20 | path: /hacks | permalink | Tags: , ,

Tue, 20 Aug 2013:

I work from home and that causes more trouble at home than at work.

Today Kavi asked me if I wanted to come along for a 4 hour round-trip to the airport to drop off her sister. Now those four hours of that has to come out of my work hours for the day & is a distraction I hadn't planned on having, but that is really my problem, not hers.

But if I were in an office building approximately two kilometres away, doing the exact same thing today I wouldn't have been asked that question. There would've been no expectation that I can spare four hours from the middle of my work day for a car ride.

I didn't go, of course - I'm sitting here ranting about it while my cluster is rebuilding ORC tables. But I didn't really have a pleasant time saying "NO" either. Just having to say *NO* and having her drive back from the airport alone seems like a choice I'm implicitly making.

In reality, I'm spending more time with both of them today, while my cluster rebuilds. But the fact that the question is pursued implies that the people involved do not consider me "at work" and therefore unavailable for anything else. And that question hurts my commitment towards both parts of my life.

The real trouble is that occasionally I do take time out of my work hours to do things with her. It is no sacrifice, because we do things that can only be done in daylight - shopping for things, brunches which last till 2 PM and evening runs where I spend an hour in the middle of the work day clocking up some mileage. I did that yesterday, with my cluster churning profiles out.

I probably really only work 4 hours a day. But those 4 hours come in two sessions of 2 hours of uninterrupted work, when I do not reply to emails, my phone is in another room, I'm not reading HackerNews or tweeting anything interesting (people, now you can see how often I work). I keep every activity which involves staring at progress bars out of those hours - my maven repos will be upto-date, my unit tests are run later and my work setup is all done somewhere in the middle, preferably during a meal or a run.

For those hours I'll be in a state of mind that is fragile enough to be destroyed by someone just asking me a rhetorical question like "XYZ is in town next week, we'll meet them on Wed?".

The trouble is that people who inhabit my life can't differentiate between those four hours and the other four hours. You know, the non-work work hours that go into reading JIRAs, replying to threads, attending calls or setting up EC2 nodes. Some of those leaks into the weekends, but I deal with it if it means having more quality time to spend on a Monday. In those hours, I will not snap back when interrupted, even drop what I'm doing to look at something or get a coffee.

I do try to make it all fit in, usually with some careful & invisible planning. It is the invisibility that is its undoing - it is effort that rarely goes appreciated by people at home (people at work don't even know I do this). It is really hard to say yes to new plans when they keep coming up all the time, without sharing my carefully timed plans - "Yes, I need 10 minutes to setup something that will take 4500 seconds to run. Let's go in a bit, but can we back by 9?".

I love being able to say "Yes!" to a plan, but I rarely say out the rest of the above sentence. And that has repurcussions that affect my happiness over-all. If unexpectedly I have to spend two hours at CPK eating dinner (enjoyable time, of course), I have to catch up by staying up till 3 AM to get my four hours in that day. Perhaps people will come home and I will offer to cook pasta instead of going out. A part of it is selfish even, because if I stick at home, I might get to stare at a monitor while the pasta gets to al dente and get some of the scaffolding work in. Those are not really work hours - if I can cook pasta while I work, I'm not working.

Now, you might wonder about my deadlines when I talk about my time and work. I'm not slogging to meet some arbitrary deadline set by Hortonworks. I'm working because I get paid to do something I actually like. And I do like to take my time off and do other things I like. I want to spend about 9 days somewhere in the Himalayas next month, without cellphones, computers and with my camera, tripod and a stack of well chosen filters. To balance my work and life, I need to put in my time at work doing productive things and I do that without anyone forcing me to "SHIP CODE!" - what else counts as work? :)

But the trouble is that I can't seem to balance my work & life, particularly living with someone who's always up for new things by the hour (which is why I fell in love with her). And it is a question of expectations from the people in my life, because I work where I live, most of the time. Perhaps if I walk out of the home for 8 hours a day and go sit in Yoga House, the assumptions of my availability for plans will drop out of her mind.

I still love working from home. I do get to head out for an hour or so, eat lunch, drink coffee and spend quality time with people. I just wish they'd understand that I have a work schedule between those pleasant punctuations and make my life a little easier with fewer questions I have to say "*NO*" to.

Because saying "NO, I HAVE WORK" makes me sad. Very sad, indeed.

Any idiot can face a crisis - it's day to day living that wears you out.
        -- Anton Chekhov

posted at: 19:13 | path: /me | permalink | Tags: , ,

Tue, 04 Jun 2013:

I made a lot of fuss when I was turning 30. It seemed like a big deal.

There's something special about completing a revolution around the yellow star.

It is special to know that you spent the last year doing the important things - writing code, climbing mountains, running distances and sometimes just doing nothing at all.

But what overwhelms me is a realization that the more important anniversaries are elsewhere on my calendar now.

Like most women my age, I am 28.
        -- Mary Schmich.

posted at: 16:56 | path: /me | permalink | Tags: , ,

Mon, 12 Nov 2012:

We begin this story in the middle.

Six of us have just clambered over last rocks onto the crater rim of the planet's highest free standing mountain. We felt like we were on top of the world, standing there with a cup of tea, with the sun rising behind us with Mawenze in sillhouette, a full moon on the western horizon and a cutting wind blowing from glaciers the next generation might not get to see. And for me & Kavi it was one hell of an anniversary!

The night seemed like a distant dream, from where we were. We'd started walking sometime before midnight, from School hut down in the east of the mountain. At first progress was slow and measured, legs freezing up as we took stops. Over the night we had left Navdha behind at Hans Meyer cave because she couldn't keep pace with the rest of us. She was going to make it too, but that was no place for waiting. We had to push on towards the peak on the west.

At Stella Point, we dropped off all our backpacks and headed up. Sleep deprivation was competing with the oxygen withdrawal at this point and with unsteady legs we got to the very top. Not that there's anything to see there, just a board and tickmark on your list. But it did mark a very important point in the whole trip. From this moment now, everything would be downhill.

As much as I could feel proud about that moment, we were there thanks to the efforts of 32 people who were behind us and often ahead of us to set up camp & welcome us with hot food in every camp. To them, I'd like to say Ahsante Sana!

Look back six nights back and you'd find a different sentiment among as we settled down in our first night at the Big Tree camp. The comforts of the city was still fresh on our minds, everything seemed uphill. Sure, we were full of popcorn, warm chai and not too worse for wear. But in a crowded campsite, we had trouble sleeping and the Diamox ensured I was waking up every two hours to pee. And in the near zero temperatures, we were thankful for the zippered toilet tents.

The second day was a jaunt through the rainforest too. But this time the mountain decided to rain on our parade. Never before have I regretted not carrying a poncho so much. After moving my camera under Partha's poncho, I decided to ignore the damp pants and walk it off. By the time we reached our camp, the Sun had come out of the clouds, but the rain had dampened more than our spirits. Our sleeping bag liners were wet, as were my waterproof pants in my bag. After drying some of that in the cooks' tent, we settled in for what turned out to be a really cold night. We were already on the smallest mountain of the trio that makes up Kilimanjaro - Shira. And for the very first time, we could see the giant we were out to climb - Kibo.

The night at Shira 1 camp, the temperatures dropped below freezing. I knew this because my stomach decided to void itself at 2 AM, leaving little choice but to follow the call of nature. By early morning, the tents had started to sweat on the inside from the moisture in our breath. A thin sheet of ice had started to form on the roof, which would melt once the Sun came out and drip back onto our faces as we got dressed up.

The walk to Shira 2 camp across the plateau was rather uneventful. We split up into groups, with Partha (the fittest among us, also the oldest) leading in, with Navdha in the rear and the rest of us distributed in the middle. This was a rest day of sorts, so we landed up at the camp around lunch time and settled in for a nap. We were out of the clouds and we had entered the heather.

The walk from Shira 2 to Moir hut was perhaps the easiest of the trip. We had all acclimatized well. Given a flat approach to the next camp, I took off and tried to catch up with Partha who had moved ahead. But Kavi was having trouble and had to grab her inhaler. The rest of the section was covered comfortably, but in rather good time. This was the last part we had in common with the Lemosho route, so we we took the evening to do some more climbing to try and breathe some air at 4000 metres. Navdha had trouble climbing, so we sent her back and headed up the rock to watch a brilliant sunset. Unfortunately, that meant that we had to do our climb down using headlamps - but we were in a good mood all of us.

The switch over from Moir hut to the Northern Circuit was what followed the next day. A bright sunny day for walking, but the guides had convinced us that it was shorter than it was. After a number of "camp is beyond that ridge", we finally got to the Pofu camp. I think we lost trust in guides a bit that day. Eventually even after we saw the cample, the walk took us an hour to finish even after because of the ridges and switchbacks slowing us down a bit. This seemed like a temporary campsite, because we were the only ones there. And being out of the wind, I took this opportunity to actually use my camera in the night. And the D300 doesn't feel like it was meant to be used with gloves on - I had to leave my gloves off to twist the aperture dials. But the small amount of skin off my fingers was a price worth paying for those pictures.

By now we had settled into a nice rhythm & perhaps had forgotten how the world down below the clouds were. The air had dried up a lot, but the sparks that flew when we used our silk sheets inside the sleeping bag was just something normal that we had come to expect. The tent had become our home, the morning wake up routine had kicked in, energy bars and quick snacks were the norm, headlamps were enough light to live with and we were all comfortable to some extent. To reach camp every evening was coming home.

We stopped over at the Third Cave campsite next. The camp was near a dry riverbed (or flood path). The night was brilliant and full of moon light. Most of my long exposures ended up looking as if they were shot in daylight. By now Kilimanjaro was looking like a small hill to our right. If not for the altitude, we felt like we could just run up the mountain in half a day. But without anything to provide scale, the distances were not apparent and we had more than a vertical km left to go up before we got up there.

School hut was the last camp we were in. By now we could really taste the victory. The mountain was right there - but the cold was starting to really get to me. The final ascent was to be done in the darkness of night. The gloves weren't helping and neither did the lack of sleep. We headed out with Navdha leading in front and setting the pace. The boys stuck to the back of the group. We had wrapped all our water bottles in socks, but I had an excessively gatoraded pouch of water hidden under my jackets. The water in the bottles froze up pretty quickly - the sipper was kept free of ice by blowing air back into the tube to push the water away from the valve. But the cold wind was hard to bear as the windchill dropped below -20.

We walked single file up the hill, but too slowly for us to actually get warmed up. Frequent stops didn't help that either - I kept running into the backpacks in front. The guides picked up the girls' backpacks pretty soon. The moon was high in the sky as we slowly started to turn off our headlamps. We started to really sleepwalk uphill in some pretty steep trails leading up to the Hans Meyer cave. There we stopped for a short break out of the wind, when the decision was made by Jimmy, our lead guide, to split the group up into six and two.

So the six of us trudged on, picking up a fair amount of speed and a couple of hour's lead on the stragglers. And then the sun rose behind us as we made a final push to reach the lip of the crater.

Downhill was hard challenge for my knees. The lack of water was turning into a big problem, the heat was picking up in the morning, but the ice in our bottles remained frozen. We pretty much slid down entire hillsides full of scree to the bottom. In the middle, we ducked into a cave, got a few minutes of sleep. Even though we could see the Barafu camp at 8 AM, it took us till 1 PM to actually get to the spot. Dehydration, lack of sleep and the effort of the previous night was starting to catch up to me - my camera was cutting into my shoulder muscles which were starting to seize up and turned into a glowing ball of pain. I had to get one of the guides to carry it downhill.

Barafu wasn't our final destination still. We had just bivouacked at the camp, waiting for the other two. Lunch was being prepared and the tents were up so that we could get an hour's worth of sleep. We heard over the radio that Navdha is coming down the rescue path instead of the regular trail, with two porters who will carry her if she can't walk. Finally they caught up with us and joined us for lunch.

Somewhere around 4 PM, we headed down towards the Millenium camp which was our resting place for the night. By now Navdha had completely lost the ability and the will to walk. She had to be carried downhill on a stretcher from Barafu to the final campsite at Millenium camp. The walk was pleasant beyond belief, the lower altitudes were filling our lungs with oxygen and the brain was on a pleasure trip after days of being oxygen starved. It was unbelievable to see the mountain above the clouds that day and realize that a few hours ago we were on top of that.

We had walked for about 16 hours out of the last 24 and 20+ out of the last 48.

The rest of the journey was made in some pain by me. My knees were gone, the downhill steps were hurting every bit of the way and unlike uphill there was no reward at the end of this trail. Sullenly, I somehow got to the bottom to find folks selling coca cola for all those who associated it with their normal life. For Navdha, they had arranged an ambulance pickup from somewhere up on a 4x4 trail, so we headed for the lodge as soon as she got dropped off. The moment we got back to the lodge, I took a hot & unnecessarily long shower and slept off. The crazy people I travel with ended up going to Moshi town to shop instead, with their sunburns covering their noses glowing pink.

It has been an incredible experience to leave civilization behind, leave all the comforts behind in the quest of this one singular goal. Somehwere in the back of my mind, I want to go back and do it all over again. Perhaps in 20 years, after Kilimanjaro erupts again.

"Because it's there."
    -- George Mallory (on climbing Everest)

posted at: 18:30 | path: /travels | permalink | Tags: , ,

Sun, 19 Aug 2012:

Friday was my last day in Zynga.

Zynga has been a very different place for me to work in. Coming from Yahoo, where the tech folks had sway over the company, to a place where the product managers were the core of the company was jarring to say the least. But adapting to thrive, indeed succeed there has polished off a few rough edges that I had.

I've sat at the cliff's edge between what's management and the frontline. I've stared into the abyss and stared it down. I've seen exactly how incentives work and more importantly, when incentives tend to work against you. I've learnt to temper my competitive spirit to skip the usual pitfalls of meritocracy. I've understood how to work with people who are not engineers.

I've known the difference between talking & persuading. To effectively bring people over to your side of the debate - whether it is with statistics or merely by running experiments. In some sense this is more diplomacy than being technically right every time - but there's no value in actually being wrong. And I got better at saying No.

I've learnt what risk looks like. Very early in this "game", I learnt to play offense. To gauge a risk, take it and occasionally clean up afterwards if it blew up in my face. And in some sense that's the general motto around there. I became quicker at churning out a solution, battle testing it and minor setbacks weren't really a deterrant anymore. And I was brave.

I got to treat crisises differently. I was on the on-call rota. Instead of panicking and hyperventilating, I started treating them as merely opportunities to offer solutions. And in my spare time, I prepared for these crisises which turned out to be massively productive moments of innovation. I tried to get some of it opensourced, but a lot of it is locked up in the paperwork that I never finished.

I've had my moments of frustration too. There have been scenarios which were charged with egos and people politics, which are probably no different in any other workplace - people trying to play schedule chicken, folks trying to delay others with scope creep, territorial engineers telling me to keep off their turf & avoidable meetings where people duel with power point slides full of promises they can't keep. I've occasionally gotten demoralized about all that, but I've managed to move beyond that and build out a stronghold around me.

And finally, the best thing that's happened to me was working with the senior technical folks in the Bangalore office. People like Binu, Prashun, Prakash & Jayesh have been a big part of my development at Zynga. And not just them actually, the whole team is full of fun folks who don't really consider work as the end-all of their daily life.

It felt like I was part of an uber startup!

The world hates change, yet it is the only thing that has brought progress.
          -- Charles Kettering

posted at: 15:05 | path: /misc | permalink | Tags: , ,

Tue, 17 Jul 2012:

I ain't trying to preach, I believe I can reach
But your mind ain't prepared, I'll c u when u get there.

I'd be a fool to surrender when I know I can be a contender
If everybody's a sinner then everybody can be a winner.

I'ma scuffle and struggle until I'm breathless and weak
I done strived my whole life to make it to the mountain peak.
Always keep reaching sure to grab on to something 
I'll be there when you get there with muted sounds bumping.

We prefer to keep our eyes shut 'til it's right
When there's something involved that we desire.
So hold your head up high if you're poor and righteous 
I know times seem strife and problems seem endless.

But at the times of despair we gotta put ourselves together 
And if you feel you're out of the game.
Then you need to get back in it 
'Cause nothing worse than a quitter.
You gotta face responsibility one day, my brother 
So wrap up your pity and turn it to ambition.
And put your vehicle in 'Drive' and stop by my side.

As we walk down the road of our destiny 
And the time comes to choose which shall it be
The wide and crooked or the straight and narrow. 
We got one voice to give and one life to live.

Stand up for something or lie down in your game.
Listen to the song that we sing.
It's up to you to make it big I guess, 
I'll c u when u see me. 

I'll c u when u get there. 

Lyrics from Coolio's C U When U Get There.

It is not what we read, but what we remember that makes us learned.
It is not what we intend but what we do that makes us useful.
And, it is not a few faint wishes but the struggle that makes us valiant.
        --Henry Ward Beecher

posted at: 15:17 | path: /observations | permalink | Tags: , ,

Sat, 30 Jun 2012:
Life isn't fair: Justice isn't some inherent property of the universe. Chaos is. And no matter what any batman villain says, a coin toss isn't fair. But it is no reason to complain, because somedays it works in your favour. Good it might be, but fair it isn't.

Nothing is ever perfect: Perfection is a quest, not a goal. Sometimes it feels good to be done with something, put it aside and move onto the next thing on your mind. It might be half-assed, but it is not just a thought anymore, it is out there. Bear in mind, it's also out there for people to criticize - but that's not a thing to fear. Carry on the quest though, but look for it in your life not in others.

There are shortcuts in life: Sure there are shortcuts, they rarely look like one. I'm lazy and that has made me look for these a bit more often than others. It took a great teacher to explain to me that education was what got him out of a hard life. Made the whole process look like liberation instead of the enslavement I used to blame the school bell for.

Rewards are not for effort or patience: You can't argue with results. The only time to talk about the effort is when it goes wasted.

Money is important: I wouldn't blame everything on money, but the lack of money is the root of some evils. Growing up in the wide middle class of Kerala, almost every limit in life I saw was about money. I know it can't buy everything, but removing that limit lets me focus on the things that can't be bought or sold - also known as the important things in life.

Luck is overrated: I think so. But who can really tell?

People die (unfortunately): I thought I knew that, but every year tells me more about it than I ever wanted to know. My only tribute to the dead is to keep on living.

Never let fear stop you & never let greed drive you: I'm not exceptionally brave. But I've maintained the little inner strength I have by trying to reflect on my motives. And in cutting down the knee-jerk reactions to fear & greed, I've torn out the insecurities that fed my self doubt. No, I'm not afraid she will leave me and I'm not waiting for my next raise - I care that she's happy and I need the money, but I'm here to kick ass & chew gum. And I'm all out of gum.

Sometimes the right thing to do is to walk away: You don't have to win every fight. Trust me, you don't have the time to fight every loser who steps out. So don't start a fight you don't want to finish. And when it comes to true confrontation, be as Teddy "Bear" Roosevelt said, speak softly & carry a big stick. Whack him good.

You've been wrong before & you'll be wrong again: This bears repeating - being wrong is important. Just one advice - eating your own words is painful. Even more reason to make them sweet in the first place.

When you're wrong, change your opinion: What's the alternative?

Don't be a rebel, change something instead: I probably had my teenage rebellion way late. And it went about this by turning my world upside down and shaking it till everything that wasn't nailed down fell out. I thought I was watching the world fall down on me, but something else happened. And I came out of it armoured with purpose.

Intelligence and skill doesn't scale: Eventually there's just 24 hours in everybody's day. And I'm not going to put in all 16 hours into work - I have a life. But more importantly, I've learnt to work away from my loner approach to doing things (instilled by the student-beat-student education system) into doing things together. Cooperation scales way better and more easily. I wouldn't say this if I was Roger Federer, but I'm not.

You can't be anything you want: Nothing's stopping you from being all you can be. It'll take a while to figure out what limits we see are real and which ones are self imposed. Ambition will let you find that out, but that's when you need to take a reality check & cash it.

Everything's harder than you realize: Our society respects people who are "brilliant". In honesty, I can't really blame society for it when there's an easier target out there - The Karate Kid. Anyway, effortless was another word for brilliant. And I wanted some of that - so I tried to get that tag for myself. But in the process I learnt something, the things I love are easier for me. The sacrifices are easier and time just flies when I'm having fun.

Failure is OK, Defeat isn't: Sometimes you give up. Life's going to keep bringing up this topic, but the day I can't get up when life knocks me down, I'll be truly defeated. I plan to get out of this life undefeated.

Do not live in the present - tomorrow does come: It's a hellish momentary existence to live in the present forever. Hopes and dreams do not have a place in it, because there is no tomorrow. I've nearly killed myself to escape it & swore never to return there. I forever live in the boundary between days, sometimes wallowing in the past and sometimes pushing towards the future. Today is merely tomorrow turning into yesterday.

May not be much, but this took 10,000 days to come into being. And perhaps, just perhaps, I'm writing this down to read someday later.

22,000 days,
22,000 days, it's not a lot,
it's all you've got
22,000 days
      -- Moody Blues

posted at: 23:27 | path: /observations | permalink | Tags: , ,

Mon, 04 Jun 2012:

To put a price on a thing is an odd thing to do.

I've never sold my photographs. I'd dreamt of doing a photo exhibition. I wanted to do more than show them only to take them home afterwards. I wanted to do something good along with it - to auction them and raise money. I didn't know what putting a price on one would feel.

But before I wade into my moral dilemma, let me thank the people without whom this wouldn't have been possible - Kavita, Tharak, Kritika, Pooja & Vivek. These five people were the hands, feet, brains, heart and occasionally filled in for all the other organs that were needed. Kavi was the mastermind of this project. I might've taken the photographs but she put on this show. Tharak for all the posters & the artwork in the coffee table book. But my gratitude really goes out to the last three, who flew in from Delhi & stayed the whole week to help!

Back to my dilemma. It wasn't as if people were paying for the photographs. They were paying for a visually impaired student's future & education - that this framed photo was merely a glorified thank you card, from me to you, all paid-for by me. But it did feel like I was making a sale.

Until that point in my life, I'd marked a large number of my photos online as CC-BY. All anyone needed to do was connect my photo to the original upload & use it as they please. They ended up on song cover art, safari brochures, treehugger articles, wikipedia and god knows where else. And if I cared, it was only to know that they were of use to someone.

As an auctioneer, I was conflicted. I wanted to give everyone the picture they liked, but I had to push the price up for the cause. Bidding wars may push the price up, but they do so by denying someone their wish to take the photo home. I wasn't happy to call up someone and tell them they weren't getting their photo of choice - I wanted to prevent people from outbidding others. At the same time, I wanted to hit my fund raising target.

I felt like I was the bad person here, keeping people from having what they want - for money. Generating artificial scarcity for something that was still available at no cost - I felt like I had sold out. Felt like I was doing something morally wrong, cheapening the sensation of having raised a whopping 1.3 lakhs for mitrajyothi. I might be feeling a bit low about the auction, but that money is going to go do a lot more good to a lot more people. But I do have something to re-iterate about my photos and your right to enjoy them.

My Creative Commons photos have always been available for anyone to download - use one as your wallpaper, take a print & frame it or paint a copy.

Just leave a comment or something, so that I have a reason to keep uploading more.

The essence of all art is to have pleasure in giving pleasure.
          -- Dale Carnegie

posted at: 10:55 | path: /me | permalink | Tags: , ,

Thu, 22 Dec 2011:

Divide and conquer. That's how the web's scaling problems have always been solved.

And the tier scales out horizontally for a while. You scale the tiers and everything works. But sooner or later you end up with a different problem - latency. The system gets choked out with the interconnects, but the magnitude of the problem is just mind boggling.

Let's take a random example - imagine about 500 memcached servers, 1000 web nodes and 64 processes on each node. Simple back of the envelope math turns it into 32 million persistent connections going on at any given time. But I'm assuming the worst-case scenario - only because that's what's in production in most places.

The real problem is that the preferred scale-out approach for a web tier is a round-robin or least-connection based fair distribution of requests. That works great for read-heavy throughputs where the key distribution is not tied to a user session. But if you ended up with a scenario where you are operating on only one user's data per-request, the wastefulness of this scenario starts to become evident.

What I really want is to pick a web-node which is currently optimal for this particular user's request. The traditional approach is to pick a node and route all subsequent requests to the particular node and hope that I can do at least a few stale reads off the local cache there. We want to go further and pick an optimal web node (network-wise) for this user session. Since the data layer gets dynamically rebalanced and failed nodes get replaced, the mapping is by no means static. Not only is that an issue, strict pinning might cause a hotspot of web activity might bring down a web server.

The solution is to repurpose stateless user pinning as used by HAProxy to let the web tier rebalance requests as it pleases. We plan on hijacking the cookie mechanisms in haproxy and setting the cookies from the webservers themselves instead of injecting it from the proxy.

Here's how my haproxy.cfg looks at the moment

backend app
	balance roundrobin
	cookie SERVERID indirect preserve
	server app1 cookie app1 maxconn 32
	server app2 cookie app2 maxconn 32
	server app3 cookie app2 maxconn 32

That's pretty much the complicated part. What remains to be done is merely the php code to set the cookie. The following code is on each app node (so that "app2" can set-cookie as "app3" if needed).

$h = crc32("$uid:blob"); # same used for memcache key

$servers = $optimal_server[($h % $shard_count)];
$s = $servers[0];

header("Set-Cookie: SERVERID=$s");

As long the optimal_server (i.e ping-time < threshold or well, EC2 availability zone) is kept up-to-date for each user data shard, this will send all requests according to server id till the maxconn is reached. And it fails over to round-robin when no cookie provided or the machine is down/out-of-capacity. HAproxy even holds a connection to reroute data to a different node for failover instead of erroring out.

And nobody said you had to do this for every user - do it only for a fraction that you care about :)

“Many are stubborn in pursuit of the path they have chosen, few in pursuit of the goal.”
            -- Friedrich Nietzsche

posted at: 10:55 | path: /hacks | permalink | Tags: , ,

Sat, 22 Oct 2011:

And I'm concerned.

I'm concerned that she does not understand the difference between marriage and love. That she does not understand the difference between love and sacrifice. That she does not understand the difference between loving someone and living with them. That she does not understand that marriage is not the finish line for love. That she does not understand marriage takes more than love.

And the more I think about it - I'm even more concerned.

I'm concerned that her apathy towards her career will be the death of it. That she will end up being a homemaker, after having a gold medal from NLS. That she will end up succumbing to the social pressures of being a Mrs Somebody. That one day somewhere in her middle ages she would regret her simple surrenders. That if she's wilfully ignoring the springboard of opportunity that so many of her peers don't have. That she might be decieving herself into becoming the role model wife and bahu - and fail. That she'll lose something precious and ephemeral - the fire to burn as bright as you can.

And in some way, I'm saddened.

I'm saddened that whenever she explains her idea of relationships it is to suffer for the sake of her partner, to be with him whatever it takes at whatever cost to her potential. That I couldn't pass off my own ephiphany about love and relationships - that it took me further ahead than I could've done on my own. That my efforts to fulfil my dad's hopes for her weren't enough - simple enough as they were, to see her "breaking through the limitations society might impose". That she won't see that in some way or the other, I've been on her side for twenty odd years in the case of Sister vs The World, 1986-present.

I'd don't think I'll ever stop being concerned about her - she's my baby (sister) too.

But in more than one way, I'm happy for her. And proud of her too.

She's marrying someone of her own choice - after six years of knowing each other. And it's happening out of co-operation between both the families. She's doing this completely out of her own volition, dreams and desires - which is what will truly give her the strength to push through when push comes to shove.

Also, she's been known for making me look very stupid - which this blog post might do in a couple of decades.

Marriage has some thorns, but the alternative has no roses.
     -- Vernon K. McLellan

posted at: 23:34 | path: /me | permalink | Tags: , ,

Sat, 08 Oct 2011:

The last time I drove my bike fast was in June.

No, that's not true. I drove it fast today. Too fast, perhaps. But that leads me to another question, how fast is too fast? I'm not talking about the speed limits or the consequences of speeding. I find that my sense of speed is completely off depending on what I'm driving - I've hit 65 on a bicycle downhill, on a 75cc two wheeler, on a 350cc cruiser and of course, in various cars. The sensation is very different, but in reality the speed is the same. So, in a rather objective sense, can I tell my brain what my driving speeds really mean?

I've always been wary of heights - though not exactly when I've been climbing. I've jumped off enough tall things to have a sense of how it feels. Maybe that's the way I can explain this. Let's take 3 different speeds and work out how they feel - in "jumping out of the window" terms.

The calculations are rather trivial.

def height(v):
    g = 9.80665 # m/s^2
    # convert to m/s
    # v^2 = u^2 + 2as 
    s = (vm*vm)/(2*g)
    print "Height to fall from (for %d kmph) = %d metres" % (v,s)

height(40) # = 6 m
height(60) # = 14 m
height(80) # = 25 m

Now, the numbers clearly indicate the height goes up to the square of the speed you're driving in, rather than than linearly (duh, kinetic energy is proportional to the square of velocity). But what does scare me is what I've added for perspective is an olympic high dive, which sort of looks like this. Not to mention, you aren't going to be hitting a nice deep pool of water.

And I still drive fast.

It is impossible to travel faster than the speed of light, and certainly not desirable, as one's hat keeps blowing off.
    -- Woody Allen

posted at: 15:15 | path: /observations | permalink | Tags: ,

Wed, 14 Sep 2011:

Anna Hazare has proved me wrong. Maybe I'm bitter about that.

To begin with, I never thought the entire country could be distracted away from Sonia Gandhi's cancer & hospitalization. But it turns out that a Gandhian (*sic*) fasting can actually do that. I wasn't spared either - I was debating about the protests and its impact on future democracy to notice this. But in retrospect, Soniaji failing to survive that surgery was an actual national crisis than anything the Team Anna could cause or prevent. It would've been an all-out civil war to fill that particular power vacuum.

And I feel like a sucker. But that is merely the conspiracy theorist in me talking.

Now, let me tackle the bonafide followers in said "Team Anna". I would like to welcome you to the land of unintended consequences.

Just because I approve of your goals does not mean I approve how you go about it - hunger strike or protests. Netaji & Gandhi had the same vision of a unified India. And I'll say with no doubt that they never saw eye-to-eye on the means. If you are trying to devise a legislative way to solve corruption in India, you are barking up the wrong tree. A social majority movement would work, but then you have a luxury Gandhi never had - elections. That is, if indeed you believe enough in democracy to go through with it. But more on that later, let's first assume that the legislations go through.

Making something already illegal, more illegal does not work. The problem was never the legislation, but more in terms of enforcement. Honestly, Kanimozhi never even suspected she would be in jail (I'm glad she is) - deterrence is a non-factor for career politicians. The issue is one completely of enforcement and the problem right now is that the bosses of enforcers are the criminals involved. Producing a different authority body would solve this, but it's a solution with a limited shelf life - until that authority is corrupted or worse, neutered.

Upping the severity of crimes has an even more negative effect. When vigilance raids on police stations in Kerala picked up, my dad was in the home department as one of the good guys. He used to lament that the extra enforcement merely upped the risk involved in taking bribes, pushing up the "rate" as a sort of risk-offset. Ironically, during the raids, people suffered more - paid more or had their needs ignored.

I don't mean to drag down the new-age revolutionaries here, but this is a democracy. Every uninformed voter in this country dilutes my personal vote. If you can bring that majority over to your fold and fight this battle against corruption, I'd stand with you and vote - the only other way would leave bloodstains across history. But the media based guilt fest that you staged was shameful. If this was the best attempt at fixing corruption in this country, I don't want to really support this movement.

Let's assume you want to start a social revolution and bring aboard the majority. The problems start from the top - I don't think very many would want to live in an Anna Hazare world. To begin with I don't eat meat, don't drink, and have never smoked. I don't gamble and I've *never* paid a bribe. But those are my values and they work for me. I might not stand to lose much in an Anna Hazare world, but I wouldn't let him take away my freedom to eat meat, if I wish to. He might be an incorruptible man, but he's still a fascist who values his written laws over the real legislature's. And even worse, he's not open for negotiations - I get to do what I'm told (yeah, we all clap when the politicians are in his cross-hairs).

A "cure": Maybe there will be a more moderate leader who's policies on my private life are more sane? No, the story gets even worse. If Anna Hazare fails to make any impact on indian politics and its kleptocracy, would you support a new leader next year? Could it be that this is a ploy to disillusion the vocal intelligentisa into not supporting revolutionary leaders? Something to convince the indian middle class that the only people the protests affect are the plebs, while the politicos sit in some farmhouse upstate. A political vaccine if you will, to build up some antibodies to clip off the next cycle of protests in the bud. Even if they didn't mean it, would it work that way?

All in all, the whole episode illustrates the failure of democracy. Its weaknesses exploited by both the politicians and protestors - but two wrongs make a right, so it's okay.

“The first sign of corruption in a society that is still alive is that the end justifies the means”
        -- Georges Bernanos

posted at: 19:18 | path: /rants | permalink | Tags: , ,

Thu, 11 Aug 2011:

Monday, August 11th 2003.

I will remember.

I can still feel it in my gut. The pointless despair, the rising anger, panic and the frustration of being at the recieving end of someone else's ego trip. Knowing that I was forever changed from that moment forth.

It was a valuable lesson - something I don't plan on forgetting.

Experience is the most brutal of all teachers.
     -- C.S Lewis

posted at: 08:30 | path: /me | permalink | Tags: ,

Mon, 11 Jul 2011:

I laughed till it hurt and then some more.

The movie is all entertainment. There is no deeper message, no underlying moral - just plain all out entertainment. But that's why I go to the movies, to laugh, not to see a poignant portrayal of urban india (even about its lack of hygiene or its lack of running water). Totaly worth everything I paid for, despite the fact that Lido chose to charge me extra because they don't have an intermission to fleece me with overpriced junk.

Now, there's nothing new about the plot or the premise. Let's get that out of the way - I saw this movie before in 1992 when it was called Mimics Parade. But for anyone who's seen a Priyadarshan re-warmed-over movie, this movie wasn't stolen. This was reborn afresh, just like an idea whose time had come, yet again.

This is a director's movie. This is a script writer's movie more than adequately carried by the actors. The presentation is absolutely original and so is its framing sequence in 2011's Delhi. Whoever though up the name "Suntara" needs a pat-on-the-back (same for the "Mill on the Floss" reference). All in all, they've managed to turn what was probably a shitty (*sic*) joke into a running gag throughout the movie.

And then they went crazy with the idea. Guns, car chases and robberies. Every tiny coincidence, probable as it may be, building up to ridiculous proportions. Like a Coupling episode, but with more fart jokes. As a metaphor for the whole movie there's the bad guys with a paper bag over Shenaz Treasurywalla's head - in short, all the good stuff, with all the bad stuff papered over. Entertaining and full of laughs, without any of stuff that made romantic comedies unbearable.

And the songs aren't half bad either.

"Our comedies are not to be laughed at."
    -- Samuel Goldwyn

posted at: 19:11 | path: /movies | permalink | Tags: ,

Thu, 02 Dec 2010:

Not for me, of course.

Today's the day when the burdens of all the what-ifs of my life becomes unbearable. The day I work up a year full of courage to face. A day which I devote to despair, for I leave no room in my life for it otherwise. To lay myself bare to all the demons of my mind. And survive.

Four years ago today, I lost a father. And today, I will remember the greatest lesson he ever taught me.


It seems to me most strange that men should fear;
Seeing that death, a necessary end,
Will come when it will come.
      -- William Shakespare, "Julius Caesar"

posted at: 16:30 | path: /me | permalink | Tags: , ,

Thu, 16 Sep 2010:

This sunday, there will only be a finish line. There are no easy ways out of it.

42 long kilometres between me and that line. My tired feet pounding out fifty thousand footsteps to get to that line in the sand.

There's no quitting, there's no giving up, hell ... even death is not an option.

We won he said, and immediately he fell down and died.
        -- "Battle of Marathon", Herodotus.

posted at: 22:40 | path: /me | permalink | Tags: , ,

Wed, 11 Aug 2010:

In Memoriam - Monday, August 11th 2003.

And I mourn for a loss I cannot explain.

I'm not upset that you lied to me, I'm upset that from now on I can't believe you.
        -- Nietzsche

posted at: 17:30 | path: /me | permalink | Tags: , ,

Fri, 16 Jul 2010:

Religious books are works of fiction.

At least, they were meant to be (yes, Mr. Tom Cruise).

I've read every religious book in English I could find, before I was twenty. Partly because I wanted to get a hang of religion, but mostly because some of the books made entertaining reading. I'm not going to repeat my issues with religion here, but if you do care - I agree a lot with George Carlin. But I didn't come here to dig a shallow grave for religion & its fan club, I came here to praise the writings it has preserved over the ages.

There is a passage of the Bible, I read often. Hidden in a history book of who begat who, is a passage of pure undiluted philosophical gold. Ecclesiastes - The book of the Preacher. The words, said to be those of King Solomon, inherited from the Talmud.


  And I gave my heart to know wisdom, and to know madness and folly: 
  I perceived that this also is vexation of spirit. 
  For in much wisdom is much grief: 
  and he that increaseth knowledge increaseth sorrow.

As I read on, the world made more sense.


  Then I saw that wisdom excelleth folly, as far as light excelleth darkness. 

  The wise man's eyes are in his head; but the fool walketh in darkness: 
  and I myself perceived also that one event happeneth to them all. 
  Then said I in my heart, As it happeneth to the fool, so it happeneth even to me; 
  and why was I then more wise? 
  Then I said in my heart, that this also is vanity. 
  For there is no remembrance of the wise more than of the fool for ever; 
  seeing that which now is in the days to come shall all be forgotten. 
  And how dieth the wise man? as the fool.

  Therefore I hated life; 
  because the work that is wrought under the sun is grievous unto me:
  for all is vanity and vexation of spirit. 
  Yea, I hated all my labour which I had taken under the sun: 
  because I should leave it unto the man that shall be after me.

  And who knoweth whether he shall be a wise man or a fool? 
  Yet shall he have rule over all my labour wherein I have laboured, 
  and wherein I have shewed myself wise under the sun. 
  This is also vanity. 
  Therefore I went about to cause my heart to despair
  of all the labour which I took under the sun. 
  For there is a man whose labour is in wisdom, and 
  in knowledge, and in equity; 
  Yet to a man that hath not laboured therein shall he leave it for his portion.
  This also is vanity and a great evil. 
  Therefore I hated life; 

For all the days I questioned my purpose, here was a man from ages beyond who was at the very same threshold of questioning his. To hate life because it takes away from you what you possess, to what end? To know that the world that gets your precious wisdom and labour, does not deserve it. That I'm creating a better world to only leave it behind to someone who never deserved it, because he was not as wise, as hard working, as worthy of it. But in my vanity, I choose to think all my wisdom was mine. And there in lies my pain and vexation of spirit (sic).

Very rarely, does the Bible not talk about human beings as being the pinnacle of creation. This book deviates from that course and does indeed ask the question whether it is vanity to assume our superiority over the beast. Indeed does ask you to enjoy your time on earth and your good deeds, instead of a promise of a glorious afterlife. There'd be a better world today

Ecc 3-19
  For that which befalleth the sons of men befalleth beasts; 
  even one thing befalleth them: as the one dieth, so dieth the other; 
  yea, they have all one breath; 
  so that a man hath no preeminence above a beast: for all is vanity. 
  All go unto one place; all are of the dust, and all turn to dust again. 
  Who knoweth the spirit of man that goeth upward, 
  and the spirit of the beast that goeth downward to the earth? 
  Wherefore I perceive that there is nothing better, than that a man should rejoice in his own works;
  for that is his portion: for who shall bring him to see what shall be after him?

In a very definite departure from the general tone of religious texts, the book of the preacher is very pragamatic and rarely invokes hopes or promises of a better life once you are through this one. And reciprocal altruism finds its way into an ancient book (move over Robert Trivers, Talmud just took your case).

Ecc 11-1:
  Cast thy bread upon the waters: for thou shalt find it after many days. 
  Give a portion to seven, and also to eight; 
  for thou knowest not what evil shall be upon the earth.

  He that observeth the wind shall not sow;
  and he that regardeth the clouds shall not reap.

  In the morning sow thy seed, 
  and in the evening withhold not thine hand.

For those who might recognize, this is the same sentiment of action that the Gita begins with. To hold the courage to do, without heed to circumstance.

  Truly the light is sweet, and a pleasant thing it is for the eyes to behold the sun: 

  But if a man live many years, and rejoice in them all; 
  yet let him remember the days of darkness; 
  for they shall be many.

And I shall.

Only as a work of religion would this text have survived millenia, without change and through thoughtful translations. And for that, I'm thankful.

Vanity is so secure in the heart of man that everyone wants to be admired: even I who write this, and you who read this.
      -- Blaise Pascal.

posted at: 20:01 | path: /philosophy | permalink | Tags: , ,

Mon, 05 Apr 2010:

I own an iPod.

Actually, I paid for it. I have never felt like I own it. It has a life of its own. It's something of a snob when it comes to playing nice with everything else I own. It looks down up on me and my software.

And the attitude is somewhat an embodiment of its maker. Apple is absolutely capable of making a device that is simple to its inner core. The layers of obfuscation and intrigue that they bury the workings of it is intentional and not in my interest. To use SQLite and then hash72/hash58 to prevent you from reading it, it is as blatant as it gets to a smoking gun.

The reasons are rather obvious. Apple wants people to use iTunes and eventually click on one of those "Buy Now" links in it. Now, to ensure that I (as an iPod "user") uses iTunes, they don't have any scruples about twisting my arm.

And the pain will go away the moment I use Apple iSomething. Everything would work and it would be nirvana. It is not my fault that it is painful for me to use an iPod without using the rest of the apple cult-products. Stop bullshitting me about that. I know what's causing this pain.

Apple wanted me inconvenienced. It was intentional on their part.

I'm not even asking for them to build a simpler device. I'm just angry at them for intentionally breaking their already undocumented interfaces with every generation of the iPod. I'm not angry at them for not making a linux iTunes. I'm angry at them for springing a surprise every release and hanging onto the file layout specs as if it was their Hello Kitty diary. I'm angry at them because they don't care about me or my money. There are enough dollars to be made with people with iTunes & its Click-to-Buy convenience.

Last of all, I'm angry with myself. I'm angry about paying Apple good money to sell me something that they are going to use as a lever on my choices at other things. I'm angry about giving them more money to screw someone else over, come the next gen of the iWhatEver.

Indeed, one bite of that poisoned apple ...

Happy is he who has the pure truth in him.
He will regret no sacrifice that keeps it.
        -- Faust

posted at: 17:55 | path: /rants | permalink | Tags: , ,

Wed, 10 Mar 2010:

Don't trust me on this ... I've been wrong before.

The difference between good friends and bad shows itself when you're wrong - when you're wrong and you don't know it. Sure, misfortune is a true test, but it hardly comes around every day and I'm glad it doesn't. But try being wrong about something. And you'll notice a strange fact.

These days friendships are too shallow. We're too independent to really need them. No, I'm not decrying the current times from the chair of age. I'm talking about the way my life's taken. And I notice that I've stopped being wrong - there was no wrong way to live my life. For a while, I thought it was because I finally had life figured out.

And I was wrong. Wrong on both counts. And people have noticed. I've been insulted. Told off by people that I was an idiot. But I didn't care for insults. They've never been a way to make me comply with anything. People have tried shame on me for years and failed. I've rarely got anything to prove to anyone but me.

A friend would've told me why. Felt comfortable enough to sit me down and outline the flaws. Because I'm not my mistakes, I'm more. Friends have got stuff to salvage, the snipers from afar don't seem to. There's a world of a difference between "He's such an idiot!" and "Don't be an idiot". And I react very differently to both.

Like I said, I've been wrong before. And I see no reason to stop now.

Well, you know what to do. Also, bring popcorn.

The need to be right is the sign of a vulgar mind.
              -- Albert Camus

posted at: 03:19 | path: /philosophy | permalink | Tags: , ,

Mon, 08 Mar 2010:

You & I both know there's no We here.

As I write down blog entry after the other, I've come to realize that there's only one person I can talk about with any sort of clarity - me. I can't speak for anyone else. Everything is as I observe, as I experience and as I feel - all mixed up into a general pile of nothing. To draw out a clear & coherent thread of thought out of that requires me to unravel a bit of myself in the process.

Self reflection leaves its own smudges in my thoughts. The searchlight of my mind leaves shadows, of contrasts & comparisons with itself. The similarities just merge into the backdrop, the differences stick out like a sore thumb. The edges & cracks appear, just like on a lake in winter, when the fluidity of thought is frozen into something solid.

And the words, like charcoal rubbing on paper, merely picks up what stands out. Everything in black & white, clearly marked out. Makes for a pretty picture, but is hardly what really exists.

Frame it up, hang it up and sign my name. And call it a blog.

My mind not only wanders, sometimes it leaves completely.

posted at: 03:32 | path: /observations | permalink | Tags: , ,

Fri, 05 Mar 2010:

It all started with the petty battles, with a collective comedown on a cultural philistine like me - one without taste in music, art, literature or topic du jour.

What they looked in each other was not for glimmers of intelligence, but for a bit of something shared. Something to set them apart from the rest, the secret handshakes, the shibboleth to exclude those of a lower culture. Respect doled out for abstract obscurity, while clarity was despised. The vaguer it got, the easier could everyone trot out their pet ideas without stepping on each others' toes.

I could've been a mute spectator to all that. But then the challenges appeared on my table.

I never did define my identity with things external. Rarely was it propped up with books, music or art. Something reprehensible to embellish yourself with someone else's creativity. To listen, read, collect it. And dole it out instead of your own. There I was, with life's experiences and I thought that was all that remained to be said about me. What I do, that is where my fount of self is rooted on and with some sort of gratitude, I pour myself back into it.

Never felt the urge to defend my choices, in anything that fed my mind. But I almost fanatically defend the choices I make, when it comes to actions. What went in seemed far insignificant to what eventually came out. My principles, ethics and the path I tread in life, those are up for criticism - always have been.

There was no point in responding to those challenges - to be beaten down just for someone's pleasure. For them in their world to feel superior. Maybe that's what gets them through their day, but I've got no time or energy to fight these petty battles. I've got things to do.

Culture intrigues me. I'd rather learn than fight about my personal opinions. Bizarrely, the same people fighting for their opinions object to others sharing theirs. What they always craved I guess was smug superiority, not to convince. Popularity of their niche seems to be their enemy rather than a sign of success.

The world of high culture is full of people who'd love something, yet dissuade the world from sharing it. In a sort of self destructive selfishness, they cordon off their niches. Watching them over the years, I've seen these hypocrites slink away from the bright sunlight of popular attention. Not revel in the new found wisdom of the world or applaud at its good taste.

I've tried to learn what it is about these ideas that make it special. Read Hegel & Kant, Foucault & Derida. Listened to Mozart (ooh, the 5th!), gone to Chopin recitals (thankee hyacie), observed the layered randomness of Coltrane. I've liked some, I haven't others. Perhaps arbitrarily, I don't know.

But equally arbitrarily, I've followed popular culture. I've liked some, I couldn't care less for others. I dig down into Simpsons or Futurama, I play Lady Gaga in a loop for days. Not mindlessly, I notice the nuances of timing & melody of the Gaga, the college level literature references littered in Simpsons, the secret messages written in alienese in Futurama. I notice, I enjoy and I'm not ashamed of it.

I'm overcome by an urge to share & enjoy. I think the fact that more people enjoy it, the better it was. Perhaps it takes more talent to make something the whole world can enjoy. A deeper understanding of all humanity perhaps. And I'll do my part. I can't understand how someone can enjoy something so much, but dissuade someone else from exploring. Even more puzzlingly, only seem to enjoy things that nobody else around seem to be capable of appreciating. Are you that special or is that all a facade put up?

After much thought, I've come to a shocking, but inescapable conclusion.

Show me an elitist, and I'll show you a loser.
          -- Tom Clancy

posted at: 07:52 | path: /rants | permalink | Tags: ,

Thu, 04 Mar 2010:

I have an ego. A nice, cheap and refurbished one in good condition.

And yes, I'm proud to have one. I've been without one, lost nearly all traces of it. Killed, choked it, sacrificed it at the altar of love & togetherness. Apologized for what wasn't my fault, forgave without apologies, silenced my self respect and cut off my ego from my life.

And that nearly was the end of me.

I couldn't survive. Because here's the thing - the world isn't always fair. Life's a bitch and it shows its true colours. It criticizes without reason and often without gain. To keep your course through that minefield of criticism requires a tough skin and a crumple zone. An ego is the crumple zone for your real self. It stands up to the world, in your stead. Takes a few dents, but nothing permanent.

Building myself back up from nearly nothing, there was my ego, leading the charge. Driving me, pushing me to do things I'd never done before, channeling my Id into the useful. Everything accomplished was an ego boost. Every failure hurt, but every failure challenged.

An ego strong enough to repel the slings and arrows of outrageous fortune, from the inner sanctum of your self and spirit. Something to keep the material world out of the spiritual, something with an edge to cut my path through the world.

There's that bright light within my eyes again. A smile on my lips and a spring in my step. And screams that it is here to stay, till death do us part. But there's balance. Between me, my ego and my Id, I'm ambitious, curious and cautious all at the same time.

In a mirror, I see me. And I smile.

The ego is not master in its own house.
        -- Sigmund Freud

posted at: 04:12 | path: /philosophy | permalink | Tags: ,

Wed, 03 Mar 2010:

So bluesmoon wrote a blog entry on function currying in javascript. Read it first, if you've got no idea what I'm talking about.

But the example given there is hardly the *nice* one - you don't need a makeAdder(), you can sprinkle a little bit more magical pixie dust to make a maker. I remembered that I had a better sample lying around from early 2005, but unfortunately it wasn't quoted in my journal entry.

I couldn't find the exact code I wrote back then, but here's a re-do of the same idea.

function curried(f, args, arity)

  return function() {
    var fullargs = args.concat(toArray(arguments));
    if(fullargs.length < arity) 
      /* recurse */
      return curry(f).apply(null, fullargs);
      return f.apply(null, fullargs);


function curry(f, arity) 
  if(!arity) arity = f.length;

  return function() {
    var args = toArray(arguments);
    if(args.length < arity) 
      return curried(f, args, arity);
      /* boring */
      return f.apply(null, args);


Basically with the help of two closures (the two function() calls without names), I created a generic currying mechanism which can be used as follows.

function add(a,b) { return a+b;}

add = curry(add);

var add1 = add(1);
var c = add1(2);

Now, the hack works because of the arguments object available for use in every javascript function. Also every function, being an object as well, lets you look up the number of arguments (arity) it accepts by default. You can even make a full-class decorator, if you pay more attention to the scope (null, in my examples) passed to the function apply().

Here's the full code example.

Things are are rarely simple. The function of good software is to make the complex appear to be simple.
            -- Grady Booch.

posted at: 21:45 | path: /hacks | permalink | Tags: , ,

Sun, 28 Feb 2010:

Five years is a long time, no matter how I look at it. I've been in Yahoo! for half a decade now or to put it in perspective, half my adult life.

If I had to describe the last year of here, I'd use almost the same words as I'd used to describe my first year here (with fairy godmothers and unicorns inked in). And it almost feels like all that was yesterday as I stare at that gumball machine at my desk.

No matter how I look at it, working here has been a huge personal event in my life. Not just professionally, but with everything else that I've managed to accomplish over the years. I guess that's what's kept me here, year after year.

The best way to appreciate your job is to imagine yourself without one.
          -- Oscar Wilde

posted at: 23:03 | path: /yblr | permalink | Tags: , ,

Fri, 26 Feb 2010:

I love Bokeh. Nothing stands out more in a portrait or a macro photo than the bokeh and the shallow DoF you can get out of a wide aperture lens. Here's a quick tutorial on how I managed to add to the effect of bokeh with some cheap carboard, masking tape and a bit of math.

The idea is to mask out the light from distant sources, without masking out the close up objects at all. The lens is designed such that the distant object light rays hit the lens and form large circles of light, instead of points as the beams focus before the sensor and diverge out into blurs. The math involved in designing the lens hood is to actually cut off some of the distant beams while retaining all the close object beams.

This is drawn roughly from a 50mm f/1.8 Nikkor. Anything covering the internal 15mm would block closer objects , but everything within the 20mm (approx) ring would only block distant objects. So anything you could punch out between those, would form a neat and clean bokeh image.

After trying to remember enough of 1st year engineering drawing, in my attempt to draw a heart with a 5mm tolerance, I gave up. Instead, I just took a print-out (A4) of something I could easily draw on a computer. Here's an easily printable PDF, in case you want to try it out on your own.

Here's how my Mk1 version looked like. I eventually ended up making a more collapsible version nearly completely out of duct-tape, which is far uglier, but has a slot in the top to slide in different filters. The tube collapses, making it slightly squarer and the pdf has the pull-tab version that the Mk2 uses.

Update: Someone pointed out that today's flickrblog covers this exact topic ... *ugh*, I'm an hour late. But at least, the PDF should come of some use to the lazier of you :).

It was fun because it's something we normally wouldn't do.
      -- Misty May

posted at: 03:11 | path: /tutorials | permalink | Tags: , ,

Fri, 22 Jan 2010:

Censorship is bad, mmkay?

Free speech is not any different just because there's the internet involved. And any country attempting to close its internet borders in the name of censorship should be defeated, for the fear of setting a precedent. Censorship anywhere is a threat to freedom everywhere.

Otherwise very soon, the internet will turn into a series of █████ which will only be used by █████████████, unless ██ take action and stop the █████████.

So, visit ███████████████████ to learn more about what's going on.

Censorship reflects a society's lack of confidence in itself.
    -- Potter Stewart

posted at: 22:20 | path: /misc | permalink | Tags: ,

Tue, 12 Jan 2010:

I run. Out on the road, as each stride drags me ahead, my mind is truly free. Free of this world, free of troubles and free from everything else, but the next footfall. Distances blur out, time stretches out, pain becomes a companion and your body falls into a rhythm which you dare not break.

found somewhere on tumblr, not mine

About three years ago, I started to run. I never really had a reason for it. I ran everywhere and ran back as well, if I could. But then people started to pop up in my life who couldn't keep up. And I stopped ... for a while.

But just like everything important, it came about from a random conversation over coffee. There we were, me & @teemus, sitting in Java city, checking out all the upcoming concerts. Right in the middle of everything from jazz to house, was a bright red poster telling us to run (and eat sunfeast biscuits).

For the good part of two weeks, I lived a very disciplined life. Ate right, slept right and spent an hour or more in the gym. Racing each other on the treadmill, steadily upping the distance, speed. Finally one morning, stretching up in Kanteerva stadium, drinking redbull mixed with orange juice, I knew I'd always wanted to run long distance.

10 Km: Ran the Sunfeast 10k. Running slowly became more of a mental challenge than a physical challenge. To actually cross the line of discomfort to actually hit the limit of pain was more of a mental barrier than I thought. It was just too easy to just quit, stop by the side and take a breather.

The physical barriers were there. They were definitely a huge challenge all by themselves. But even when I could run 5km without falling over, the urge to give up does not fade. To actually gag and silence that part of your mind which keeps whispering "quit now, there will be cake".

heh, was that too obvious?

It was all in your head.

You could stop anytime you wanted. But it took all my vanity and ego, to keep me running. To see others vanish ahead, lit a fire that would burn me through the miles and miles ahead of me.

faster: The Nike+ Human Race was the next one. The last 400 metres were run at blistering pace with me and @balajijegan running together. Adrenaline pumping through my veins, Nicotine blasting in my headphones and out of breath, I crossed that finish line. 10 Km was turning into a race than a test.

25 km: Finally, it was time to play with the big boys. The Bangalore Ultra was looking like too big a challenge to actually pull off. The track was all ups and downs. I ran the first 18km in complete zen, in under two hours. The safety pins holding my bib to my chest had started to cut into my chest and I had cuts on my chest from the friction. I had to stop after the 20th km to get medical attention. After barely six or seven minutes of standing still, I had cramps.

last mile: The pain was getting to me. 21st to the 23rd was a sheer exercise in masochism, all the way uphill and through slippery mud. After that, it was just a test of sheer willpower. I knew that I would just fall over if I stopped running. Running along on empty, in pain, with my face showing it - you'd wonder why I was doing it. You have to be a runner to know why - there was no way out of that agony other than that finish line. I don't think I left myself any options.

21 more: The next half-marathon had significantly less training behind it. The weeks preceding it was mostly spent on FOSS.in and almost everything going wrong with that conf was my fault. The Times of India midnight marathon was a lot of fun to run, because it was flat, late in the night and I had my personal cheerleader waiting at the finish line. I ran that at my best time, ever.

Running will change your life. I'm running out of reasons to explain why I run, but it's changed my life around and made me a happier person overall.

One day your life will flash before your eyes. Make sure its worth watching.

posted at: 00:09 | path: /me | permalink | Tags: , ,

Mon, 11 Jan 2010:

Avatar is Pocahontas set in Fern Gully, except with Smurfs who are Thundercats. And hero's plugged in from somewhere, with a smattering of a messiah complex thrown in (and a pity, they'll never make any sequels ... *ssh*).

But now that I've got the cliched bits out of the way, let me rave about the visual beauty of the movie. The reason this movie takes so much flak about the plot is that the CG does not jarr the suspension of disbelief required. To watch Neytri wail and definitely do the damsel in distress routine does bypass the fact that she's a giant cat-alien, into my emotional awareness. Somehow that disrupts the dehumanization of the enemies that the Colonel is under (if he had a cigar on him, I'd have had Quake3 flashbacks). The fight sequences are hardly overdone and they haven't gone bullet-time or john woo freeze-frames on it. Perhaps the brain-stem connections all animals share is probably the only thing that really stretches the imagination. But the movie goes to great lengths to illustrate that it's not really a form of direct control. There's hardly anything wrong with the movie as such - as long as you're only watching it.

There's EPIC FAIL and some glimmers of brilliance in the fauna of Pandora. Most of the animals on the planet have six limbs. This is perfectly acceptable, but if only they'd kept it consistent and extended it to the Na'avi. A bipedal/tetrapod Na'avi co-existing in parallel with hexapodal large animals sort of suggests a really un-bottlenecked evolutionary history (think of the Cambrian explosion, followed by millenia without an extinction event). Which would almost make the nerve fibre connections into a miraculous act of convergent evolution. But not all of it is bad. The lung openings on the direhorses, on their chests with a large volume intake, would've been a brilliant evolutionary jump away from a narrow trachea and probably an easier jump from book lungs & gills. Somehow the fauna is vaguely reminiscent of Nemo Ramjet's Snaiad universe. Last but not least, Turok - the flying giant in butterfly colours. I just can't get over the fact that its name is "last shadow" and the import of that.

All in all, it's a mish-mash of the noble savage, the greed of man (ah, Rousseau vs Hobbes), the modern industrial military complex and an imperfect romance. The movie is watchable, though not by any means an instant classic.

If you want a happy ending, that depends, of course, on where you stop your story.
    -- Orson Welles

posted at: 20:03 | path: /movies | permalink | Tags: ,

Mon, 04 Jan 2010:

I've left too many things behind with 2009. And I'm better off for it.

It wasn't exactly a pleasant year to begin with. Betrayals seemed the theme from the start. Which I could've dealt with, except my inability to deal with betrayal was treated as some sort of personal failing on my end. As if my disappointment was some hint of immaturity I had to grow myself out of.

Anger boiled, seethed in my veins. But I held my hand, because I still believed that people would do the right thing, albeit eventually. Patience lasted exactly a month. Then it was time to blast off, blow everyone off, leave the liars and traitors behind, to cleanse myself of the grime and corruption of their world. Yes, I still don't think as my trust being misplaced ... it was betrayed.

But perhaps, adversity is a forge of character. I revved up my thunderbird, cruised along the hills and valleys of south india, in a quest for my soul, self and maybe something more. The wandering took me to places of my self where I've been loath to shine a light into, hardly tread.

20/20: Twenty meals alone. Once you're through that barrier, being alone hardly holds any terrors. Life flows around you, every face a stranger's, every smile an accident. And you're in no hurry, there's nowhere to be, no one to meet, nothing really left to do. Sitting there, as you watch the world rush about, you feel content to just be a spectator to the human race.

Rock bottom is a pretty productive place to be. Wrapping myself in a cocoon of solitude, I spent hours, nay days scribbling bits of myself into my little diary. An outpouring of self, a vignette of modern urban life, my only story, a fictional biography of someone I used to be.

Perhaps it was cathartic. Perhaps it was inspiring almost to examine myself through an honest mirror, instead of relying on the judgement of others. Brutal honesty cut through the threads of self pity tying me down. I may have had a crappy time for half a decade, but I've come out of it scarred, but smiling. And again.

Resurrection. Life's a little less serious now, I think I have most of it figured. Simple rules to live by - smile, don't give a damn about what "they" think and do what you really want. If you're good, you'll float ... else, you sink. But there's no room for pretending.

And that was 2009[1].

[1] - Wait, that was only four months? The rest of the year? A blur of fun, parties, travel and stuff that'll eventually be a footnote to such a page.

Strike me down now and I shall be more AWESOME than ever!

posted at: 03:03 | path: /me | permalink | Tags: , ,

Mon, 21 Dec 2009:
e + 1 = 0

Euler's Identity. 'tis a thing of pure beauty.

Three very suspicious numbers in a menage-trois, creating something real. How can two irrational numbers and an imaginary number work together to make a very real integer? It boggles the mind entirely. Somewhere in a past left behind, this was the first equation to make me sit up and consider imaginary numbers as something more than a trick.

The standard wuss way of explaining this (as happened to me) was that of pre-cooked trigonometry.

e = cos(ϑ) + i sin(ϑ)

But that is just a completely arbitrary equation, when you really think about it. And I'm an incorrigible skeptic. But that's where that lesson ended and Math is not taught as much as lectured on. But somewhere during my engineering, I learnt about the Taylor series, for approximating sine and cosine values. Except, it's not really an approximation, but an infinite series and the partial sum, is used for approximation.

             ϑ3  ϑ5  ϑ7
sin(ϑ) = ϑ - — + — - — + ...
             3!  5!  7!

             ϑ2  ϑ4  ϑ6
cos(ϑ) = 1 - — + — - — + ...
             2!  4!  6!

Remember, that works on radians, not regular 'ol degrees. So ironically, when you throw the magic number in there and spend an eternity calculating it, the sin(π) works out to be one huge zero. And it has to, because looking at it from pure geometry and sine as a pure fraction.

Now, I never understood how an infinite number of operations could ever result in a finite number. Well, it's the ghost of something familiar - Zeno's Paradox. And well, Archimedes debunked it, way before I could even attempt it.

Now if you shift a little from geometry of lengths into the world of co-ordinate geometry, you suddenly realize that imaginary math is literally co-ordinate geometry in disguise, except with imaginary numbers (woooo ...). Pull that very Eucledian right triangle into a unit circle on the imaginary plane, the boundaries between the disciplines start to disappear.

The imaginary pixie dust sprinkled on e results in another taylor series expansion, which ironically just shows to go how you can really go mad learning mathematics. Now, the taylor series expansion for just plain 'ol ex goes like this.

              x2  x3  x4
ex =  1 + x + — + — + — + ...
              2!  3!  4!

Now, here's the clincher. If x just happened to be i, the alternate coefficents would be negative. Oh, yes ... that's pure imaginary pixie dust, but once you get hooked on it, there's no getting off it :)

Now, we get to the final and crucial equation all over again.

e = cos(π) + i sin(&pi);

e = -1 + 0 i

e + 1 = 0

Time to run out on the streets and yell out that ... "they were right, e is REAL!".

That's not right! Heck, that's not even wrong!
    -- Wolfgang Pauli

posted at: 20:19 | path: /misc | permalink | Tags: ,

Tue, 15 Dec 2009:

Inner beauty is overrated.

Perhaps the greatest handicap my parents ever provided for me was the concept of inner beauty. In their attempts to prevent me from turning into a flake, they emphasized that it was probably the most important thing to develop. Their efforts bore fruit. They taught me to look deep into the heart of others, judge them by their intentions and to know them by their actions.

But they also taught me to avoid the shallow. I learnt that the shallow, do not linger to explore another. Like butterflies, from flower to flower, they pass on from one to the other, having known no one, but calling all friends.

Slowly, but subtly, I started to wrap myself up in myself. Layer, by layer, everything that was good in me, was only there for those who lingered long enough to peel back enough. And I thought that only fair, that only those who cared enough to know me, got me. There was precious little of me to go around and I kept it for those special people.

But as it turns out, *that* was a very stupid thing to do.

Sometime over the last year, reading the Bible at some hotel room somewhere in the country, I ran into something that clicked. Something that made sense and shone a light on the errors of my ways (uhh... no, I'm not going Born Again on you folks ... keep reading).

Mathew 5:15

	Neither do men light a candle, and put it under a bushel, 
	but on a candlestick; and it gives light to all that are in the house. 
	Let your light so shine before men, that they may see your good works.

There's a certain arrogance of self that is required to be an introvert. That it seems worthwhile for someone who's met you once to dig through all your issues, fears and eccentricities, to know the really awesome person you are. That the end result, i.e You, is something enticing enough for a stranger to actually embark on that quest. As if there is some secret sauce, essence of pure self, that makes you unique among all others. That it doesn't really matter how you appear to be, that all that matters is how you really are. Pfft, maybe in an Apatow World.

Perhaps it is humbling to know that what you are isn't worth someone's time - at first glance. Perhaps there are so many who are boring on the inside & outside, that the odds are against you, all the way. Perhaps they are indeed shallow people who judge others by appearances or by popular opinion/reputation. I don't exactly know why people don't bother to look twice, but they don't (actually, some do ... which is how I got by for years). But it's not their problem that they don't, it's yours.

So, pull the covers off the true You. Shovel out a path through the icy reaches of your outer surface, put a window on your soul. It'll change your life.

In short, SUIT UP!

Be yourself; everyone else is already taken.
       -- Oscar Wilde

posted at: 01:01 | path: /philosophy | permalink | Tags: , ,

Mon, 26 Oct 2009:

People don't make decisions. Decisions make people.

That can't be true. I know to the exact dot, dash and every crossed tee to why I made the decision to write this blog. Or am I just writing out a rational framework as an after-thought to a pithy cliche? Maybe it's because I picked up an Orwell book and read Shooting an Elephant again?

I will never know. Not for sure. The sequence of thoughts that precede and follow an action are often so mixed up in retrospect. They get even more muddled up when I introspect deeper. I can't use my mind to understand itself. Going third-person collective on this stuff!

We strive to maintain a certain rational self-integrity as a survival trait. In some sense, our self images involve a picture of a conscious, self-evolved and rational person. We cling to it, however transparently false it might be to everyone else around. We are proud of it.

Impulsive decisions prompt a certain cognitive dissonance in deep dark of your sub-conscious. You know you aren't that kind of a person, but the act is behind you and there's no rewriting your actions. But perhaps there's other things you can change to make it all fit. Most of us fight it by becoming a new self, to whom the actions are a natural consequence of who they are. I understand, even have grown to respect that it's inevitable. But rather than admit that the change of heart was after the action, we'd rather revise our history a bit to recover a bit of internal coherence. Because in the disordered and confused world they live in, the coherence of self is perhaps the only thing they've got left to hold onto.

You've become a different person and it surprises everyone around you. The most convenient lie to trot out to mask all this internal turmoil is the ever cliched "I've always been like this, you didn't know me well enough!". I can't really read minds, but I've learnt to read people. Observing people will themselves into believing this - that they haven't changed due to their decisions and that causality flowed the other way around - has brought me some insight into the ways change has creeped into me.

I've come to embrace it. My decisions have changed me, some for better & some for worse. I'm a product of my decisions, not of my dreams or desires - of my decisions & actions. I live out my own punctuated equilibrium of personal evolution. And not everything that changed me came from within. I'm not taking anything away from myself with that admission. It's the truth.

But I've come to despise the impulsive pretenders of later rationality.

Perhaps despise is too strong a word. But it'll do for now.

One could laugh at the world better if it didn't mix tender kindliness with its brutality.
          -- D. H. Lawrence

posted at: 06:01 | path: /observations | permalink | Tags: , ,

Wed, 14 Oct 2009:

Inglourious Basterds. I almost had to be dragged to this movie. I'd like to thank that person before I actually put down anything here.

Now, I'm not a big fan of gore. I mean, superflous, gratuitous gore that Tarantino has almost made into an art. I'm not against realistic, in-context gore, but the sauce and ketchup show that was the restaurant scene in Kill Bill is exactly the kind of scene I never want to see again. I shouldn't have worried about that. I really shouldn't have, because I was in for a treat.

Tarantino had it spot-on with the Basterds. For what, from a quick glance, promises to be a jewish cowboy Western mashed into a World War II universe, the movie really revolves round the brilliant performances of two characters. It turns into a drama of unexpected events and odd coincidences, instead of the grit & gumption of the war hero. The script really winds around them and are perhaps the most real characters I've ever encountered in a Tarantino movie. So, move over Brad Pitt and the rest of the Basterds off the posters and let Shoshanna & Landa take their well deserved privileges. They're the real deal. It's their movie.

Shoshanna. I'm undeniably smitten by this woman. She is not a woman of violence, but neither is she driven to it. Violence lay in her path and she accepted it within herself. Her character is beautifully portrayed as woman whose war walked in through her front door & sat down to watch a movie. As the unwittingly chosen angel of death, she brings vengence back into style! Played by the lovely Melanie Laurent, her eyes speaking better than her words, Shoshanna comes to life right before our eyes. And dies.

But without a Hans Landa, there wouldn't be a Shoshanna. The multi-lingual SS officer is probably the key character to the entire plotline. The first few minutes of the movie are entirely his. But as the movie rolls on, it becomes clear that what he does is just what his job is. He is portrayed as a man who is given a dirty job because he's the best at what he does. By no means is he a positive character and eventually painted as a traitor, but he's not a demon posessed with a single-minded quest. Indeed, the way he deals with von Hammersmark, is frightening at a level beyond the impending violence. He's an intelligent opportunist on the wrong side of the line. And he plays that role with a smile and straight faced perfection.

The attention paid to details in the movie is amazing. The German three fingers versus the british three, the fancy shoe left behind, the autograph with the kiss and even the intial scenes where the conversation in French & English is intended to lull the hiding children into a false sense of security. Though I'm not quite happy with the way Hitler has been portrayed, he can be demonized without making him a comical misfit.

But perhaps, just perhaps the movie should've ended with Shoshanna's message and her laugh echoing through the flaming wreckage.

In a war of ideas, it is people who get killed.
    -- Stanislaw Jerzy Lec

posted at: 01:32 | path: /movies | permalink | Tags: ,

Tue, 06 Oct 2009:

Occasionally, as I flip back the pages of my life, I find myself in conversation with a younger me from a much older time. As if caught in a flipbook time machine, I see myself change, grow and in some sense, stay the same. Once in a while though, I turn up a page with which I disagree with enough to need revisiting.

In the mid-summer of 2007, out of my frustrations with work was born an unadulterated rant of pure cynicism.

There's some sort of misplaced humility that is injected into us by our educational system. Or maybe it is some sort social stigma attached to the braggart or overacheiver. Must've been what was going through Lennon's mind as he penned down "they'll hate you if you're clever and they despise a fool". Eventually, the struggle to stand out and the pressures to blend in, find some sort of balance in your inner selves. You'll be happy to be the best at something everybody is doing. And even if you aren't, it's somewhat a partially ordered set. Life makes sense and the years roll on.

So you are the real deal, the bee's knees. To start off your career, you dive in and start pulling your weight. Even a moron in a hurry can see that what you're doing valuable, nay essential to the company. Your management wants to know that, it's exactly the kind of information they crave. When it's handed to them in a platter, they love it. But, you keep working, in your little corner. Nobody notices anything and if they do, it's when you fail. You complain about not being noticed to your peers, you write out long rants on your blog about how your life sucks.

Most people at this point in their careers blame the management for everything that's wrong at their job. And treat every peer who chooses to move into management as a blood traitor. I'm not denying that there are bad managers, just the same as there are bad people. But most managers promoted out of rank & file end up being good people with a job which looks like herding cats, except without any catnip. The people working under ordinary managers go passive agressive in their rebellion, complicating the situation further. Eventhough 'tis a betrayal every which way, it happens because nobody trusts anybody.

You are doing something very important and valuable to the company. Then why don't they trust you? Because they have had people work under them in past who were poor communicators because they weren't getting anything done. They even had good people work under them who secretly didn't like the plans, but kept their traps shut and worked towards a fait accompli. So if you communicate poorly, they are not going to give you the benefit of the doubt. No matter how many poor communicators actually end up getting work done, the managers will always remember the times they've been burned.

You don't need to be a 'Yes Man' or a atrocious sychophant to get your manager to treat you well. All you need to do is to make his job easier ... after all, managing you is pretty hard work. Though, there's such a thing as overdoing it. But that's yet another story altogether.

Watching your future with much interest,
  — future me.

Those who cannot change their minds cannot change anything.
           -- George Bernard Shaw

posted at: 22:01 | path: /observations | permalink | Tags: , ,

Sat, 19 Sep 2009:

This was the trip that almost didn't happen. Decided to go to Africa on the 24th of August. The complications were everything from ICICI causing a fuss about a wire transfer, the yellow fever vaccines, to just plain bad timing of a bank holiday tying up my local funds & foreign exchange.

But it all came together in the end and it was AWESOME!

This trip was a goldmine of interesting sights (and interesting stories). I'm slowly going through the pics and uploading them to flickr .

Nobody succeeds beyond their wildest expectations unless he or she begins with some wild expectations.
        -- Ralph Charell

posted at: 01:45 | path: /travels | permalink | Tags: , , ,

Fri, 04 Sep 2009:

Occasionally in life, I have a blinding flash of the obvious. An idea which has been hiding out there in plain sight, just jumps out and catches my attention. I don't even claim to be original about this, but at least I hope that here's the first time you've seen this in writing. Here's one of those ideas I had when I was 14... looking back at the all the years.

The world has a plan for you. The day you were born, you've watched it unroll in front of you. The system tries to coerce you to its plan with its checks, balances, pains and rewards. The system is all around us in our culture, environment and upbringing. It's implemented by the powers that be, to turn you into a well oiled cog in society.

If you're reading this, you've probably already dismissed the plan. You, the individualist, is determined to make your own way in this world. You've already recognized its shortcomings, pitfalls, weaknesses and in fact, you know you're cut out for bigger things. You are the person you are because you chose to branch out from it, rather than conform to any expected norm.

This is not a bad thing in itself. But the powers that be, they fear the change you represent. They will resist you, they will force you into their systems of indoctrination. You will fight, you will fail. As your idealism tends to wear out, as it grinds incessantly against the real world, you become frustrated with your impotence in this world. The rebellion becomes destructive - to yourself and others around. As you're thrown out of society's inner circle labelled as an outsider and a troublemaker, you're reaching an end which you do not deserve.

There are only two basic rules of survival for the individual:

  • Work the system
  • Fuck with the system

It doesn't get any more contradictory than that.

I listened to the world carefully as it whispered its rules in my ear. I didn't agree at first, but I still listened with as much care as I could muster. Because the day I stood up and disagreed with it, I didn't want to disagree because I didn't understand. I wanted to dismiss their plan because I understood and understood all too clearly.

The System is not your friend or your enemy. Being caught in it is like being caught in a raging torrent. You don't escape it by swimming against it, the only way out is downstream. You need to know the currents, the way the water flows to let the stream take you where you want to go.

Fight the System head on and you will most certainly fail. It is not fair, but that's how it works out. You need to focus your efforts on what you want to do rather than in wasting it on the system's clampdowns. Don't reject it outright, but instead ride the river - use its power to your advantage. But don't let it change who you are, understand that you are making the system work for you.

But you will need patience. Almost infinite quantities of patience, because the Machine will never move at your pace. The right moves at the right time, holding your breath waiting for the right moment and you can move nearly anywhere you want to get to. Without giving up an inch of your inner self, at complete harmony, but in complete control - in surfer cool fashion, you'll be able to move about.

But how do you not become a drone in the process? Society, as powerful in mainstream life as it may be, is not omnipresent. Sooner or later you'll have people around you who do not belong to it. By reaching out and connecting to such people, you'll build yourself a little sub-culture where you are truly truly free. You'll be able to disconnect from the hum-drum of the rest of the world and truly enjoy human interaction sans rules built by others. And those moments will reasure you that you haven't lost it, yet.

Just like any surfer on a wave, now comes your time to stand up & shine. Once the machinations of society are second nature to you, the threads pulling each human being around you start to pop out of the background noise. Even as a non-conformist, you'll be able to manipulate the world around you to your own ends. As you delve deeper into the systems within systems and wheels within wheels, you'll start to "see the code" to the world.

You'll be surprised about how much of your blatant individualism will be tolerated by the powers that be, if they can't detect a threat from your existence.

Don't be an idiot. Don't complain into the ears of your peers. Don't fight the system and self destruct. Know in your heart that you have something to offer to this world, whether it wants it or not. That is not worth risking for any rebel posturing or meteoric martyrdom.

But in the end, they won't call me a rebel. Because I wasn't ... I was just being myself.

So be you.

Every act of rebellion expresses a nostalgia for innocence and an appeal to the essence of being.
          -- Albert Camus

posted at: 16:03 | path: /philosophy | permalink | Tags: , , ,

Tue, 11 Aug 2009:

In memoriam - Monday, August 11th 2003.

They said that time heals all wounds - they didn't know me at all.

Blessed are the forgetful; for they get the better even of their blunders.
        -- Nietzsche

posted at: 17:30 | path: /me | permalink | Tags: ,

Wed, 08 Apr 2009:

We are haunted by our mistakes.

Our successes, they will leave us while we're dulled by that afterglow of satisfaction. Your mistakes, however - they follow you around, as you drag your feet through the journey of your life. Like a shadow in the darkness, but their footfalls ever so muffled for you to suspect your sanity. They linger on, to provide that tinge of regret, which unfortunately is the mark of a life lived, instead of slept through.

As I headed out to watch Dev D for a second time, I wasn't expecting to feel any different from what I felt the first time. But I was to be surprised at what I felt for the protagonist - the disdain I had felt for him vanished into thin air, to be replaced with a sense of despair if only by empathy. I felt for his dillemma, the internal conflict that is the core of it all. Maybe it's a twist of perspective, but the experience was different.

To anyone who's made a mistake, there's an irrational urge to run away from everything that's headed your way - to leave consequences to others and withdraw. That the solution to everything was just to be not there. Not like it hasn't worked before, it just wont work this time - because you didn't wrong someone else, you wronged yourself.

But as you watch Dev and Chanda both go through their lives running away from the consequences of their actions - the realization hits home that you can run, but then that's all you'll ever do in life. Run and run. Run into the arms of drugs, alcohol or any other crutch that could make your mind just stop. Saving you from the monsters that inhabit you, dulling the pain and make you smile, all in ignorance. Anything to stop thinking, stop everything and keep running.

Consequences are strange beasts. They give chase when you run, but wait for you when take your time to walk to them. No one else can make you walk upto your consequences and introduce you to them. The last mile is yours alone to walk. It's not an easy one to walk back, knowing what awaits you at the end. But there comes a point when you're tired of running. When running away is not getting you where you want to be.

There was that moment of epiphany for Dev, standing next to that phone booth picking up the coins. The moment when you realize that this life is just too short to take the long way around. In perhaps what's the most unappreciated twist in the movie, you see the protagonist actually stop and walk back. Back to Delhi, back to Chanda and back to the police station.

On his own feet, to meet his past & make way for his future.

I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind;
yet, strange, I am ungrateful to those teachers.
              -- Kahlil Gibran

posted at: 01:12 | path: /movies | permalink | Tags: , ,

Mon, 23 Mar 2009:

I hate Slumdog Millionaire.

I don't hate it for artistic or aesthetic reasons. I hate it out of principle - because it is an act of third world emotional blackmail for anyone who's seen it for real - meant to repel and disgust. And it turns me off, pushes me into denial and just turns me into someone numb about reality. Freeze over my emotions when I saw the child being blinded with red hot irons. Maybe there are those who seek numbness over pain.

I've had one too many conversations with people from elsewhere about this movie. In response to their curiousity about whether this is how India is, I can only answer "Some parts of it, yes". It's almost like they want to believe what they see and won't settle for an India where people go to work at 8 and drive autorickshaws everyday.

To draw an analogy, when you think of Paris, the image that pops into your head immediately is the lit skyline with the Eiffel Tower. But imagine a quintessential paree movie of your preference and throw in all the moroccan ghettos which have sprung up around. That stark, depressing but real face of the city wouldn't be something I'd enjoy.

Art is about selecting what to represent and what is irrelevant. And the movie makers made a choice, that was completely theirs to make. Now that I think about it, what I'm hating is not the movie. It's the condescension that the movie invites on my bit of the world. Not because it's dishonest or that I'm patriotic, but the intentions behind it are suspect. It's pandering to the needs of someone filled with Schadenfreude.

What I'm hating is the tourist who says "I want to go see a slum".

Nothing ever becomes real until it is experienced.
          -- John Keats

posted at: 11:45 | path: /rants | permalink | Tags: , ,

Wed, 18 Mar 2009:

Yet again, I find myself in a strange position. I observe, I learn and I contemplate. Spend much time on those, travel by thought and arrive at conclusions. Only to find out that I'm a century too late with them; That in fact, I was a century late when I started. Like once before, I'm not ashamed to borrow the unforgettable words of a soul long passed by.

No, we never sicken with love twice. Cupid spends no second arrow on the 
same heart. Love's handmaids are our life-long friends. Respect, and 
admiration, and affection, our doors may always be left open for, but 
their great celestial master, in his royal progress, pays but one visit 
and departs. We like, we cherish, we are very, very fond of—but we never 
love again. 

Love is too pure a light to burn long among the noisome gases that we 
breathe, but before it is choked out we may use it as a torch to ignite 
the cozy fire of affection.

But of the fire we all know, but let him speak of the embers left behind - the death of love and the journey ahead.

I am afraid, dear Edwin and Angelina, you expect too much from love. You 
think there is enough of your little hearts to feed this fierce, devouring
passion for all your long lives. Ah, young folk! don't rely too much upon 
that unsteady flicker. It will dwindle and dwindle as the months roll on, 
and there is no replenishing the fuel. You will watch it die out in anger 
and disappointment. To each it will seem that it is the other who is growing

Both are astonished at the falling off in the other one, but neither sees 
their own change. If they did they would not suffer as they do. They would 
look for the cause in the right quarter—in the littleness of poor human 
nature—join hands over their common failing, and start building their 
house anew on a more earthly and enduring foundation. 

But we are so blind to our own shortcomings, so wide awake to those of 
others. Everything that happens to us is always the other person's fault.

It is a cheerless hour for you both when the lamp of love has gone out and 
the fire of affection is not yet lit, and you have to grope about in the 
cold, raw dawn of life to kindle it. God grant it catches light before the 
day is too far spent. Many sit shivering by the dead coals till night come.

And from a page penned more than a century ago, the man stabs at the heart of our modern lives - only to go unheard again & again. But reader, to you I repeat, words that will make sense of this world - the creed of its people, of self and nothing more.

Ah, those foolish days, those foolish days when we were unselfish and 
pure-minded; those foolish days when our simple hearts were full of 
truth, and faith, and reverence! Ah, those foolish days of noble longings 
and of noble strivings! 

And oh, these wise, clever days when we know that money is the only prize 
worth striving for, when we believe in nothing else but meanness and lies, 
when we care for no living creature but ourselves!

If you enjoyed reading the paragraphs above, this is all quoted verbatim from Jerome K Jerome's "Idle Thoughts of an Idle Fellow". And at least in my ears, it rings absolutely true - scarily so, for something written in 1886 - a different era altogether. In some strange sense, for all the progress we've made, we haven't changed at all. And if you think so, read the whole thing.

Men love in haste, but they detest at leisure.
                -- George Gordon, Lord Byron, "Don Juan"

posted at: 01:12 | path: /philosophy | permalink | Tags: , ,

Sat, 14 Mar 2009:

She's perhaps the most important person in your life - and you'll never acknowledge it.

Perhaps the only constant relationship you maintain over the years, without realizing it. It won't be hard - You'll do nothing more for each other than just listen - hear each other out. And it'll be enough, in fact it'll be more than enough.

She's a friend - a friend you never reached out to. A friendship born out of coincidence and chance. Not someone you want or desire, but someone who's always seemed to be just background to your life. All that makes her special is that your soul is naked to her - she's the one with whom you have never put up on your pretensions. She's the one who's never judged you for it. She's just this girl in your life, never platonic with a hint of impossibility to it. She's just there, somewhere.

But, when shit happens; She's the first one you go to.

All my friends and I are crazy. That's the only thing that keeps us sane.

posted at: 01:46 | path: /observations | permalink | Tags: , ,

Fri, 06 Mar 2009:

Spend every day watching your life pass you by. The days, they go on interminably without any identity of their own. Your thoughts become memories, turn into dreams and fade away like breath on a window pane.

Freeze a moment in time, store them away and chase them back down memory lane. Lay down the breadcrumbs marking your way, mementos held close to your heart, of a day that you'll never forget.

Click, *click* ... and that's all it took.

The moving finger writes; And having writ, moves on.
The writing, it stays written.

posted at: 17:20 | path: /fun | permalink | Tags: , ,

Tue, 03 Mar 2009:

One Monday morning morning four years ago, I walked into an office.

In retrospect, it was a very important moment in my life. But all I did that day was to dump my bags and go to the cafeteria.

But it was going to be my home for years to come. Lived in that little sheltered world for four years. Made friends, made enemies, got a life, then gave it up, discovered myself, travelled the world, grew my hair & shaved it, found love, even lost it, lost a father and nearly killed myself too - picked my life up out of a rut, ran alongside my career and didn't have to give up either.

Ah, good times ...

We are so busy watching out for what's just ahead of us that we don't take the time to enjoy where we are.
                  -- Calvin

posted at: 17:03 | path: /yblr | permalink | Tags: , ,

Thu, 19 Feb 2009:

The word intellectual has been hijacked. That epithet was reserved for those who held views of their own on this world. For those who used their intellect and abilities to percieve the world in their own light. For the renegades of thought, not restricted by the dogmas of their time and life.

Somewhere along the way, it became impossible to distinguish between the true leaders of ideas from those who have squandered their potential as independent thinkers. It's not like the fundamental roots of philosophy have changed over the years, but the world of print is seducing those that wish to futher their development onto the beaten paths of the last century. I do not deny that on shoulders of giants we should stand, but not on their toes.

I meet people like that all the time. I in fact, enjoy the crossfire of ideas that results in. I'm growing, I'm learning without being taught. But I often feel like what I've wandered into is not the melting pot of ideas, the half-bakery of them - but the McDonalds of ideas. Ingredients sourced from all over the world, made with a recipe, packed and tastes the same all over the world. There's a pervading sense of sameness about it - the names dropped, the vocabulary and the people I meet.

The ends remain the same, but the means have lost their meaning. Existential angst is the source of all my philosophy. The contemplation of suicide is perhaps the only source of truth for me - to veer off that path, I've looked far and wide for meaning in my life, till I realized that ironically, that very same quest is the fundamental meaning of sentient existence. To find an answer would be a death unto itself - cogito ergo sum. There's only one certainty and depressing as it might be, the evasion of which is what's so uplifting about every day lived.

I'm not an intellectual. But I like to pretend to be one, because of the social acceptance it provides for my quirks. But beyond that, I'm merely a student of human nature - mine first and everybody's later. And that's just my arrogance claiming how unique I am.

The error of youth is to believe that intelligence is a substitute for experience.
            -- Lyman Bryson

posted at: 17:31 | path: /observations | permalink | Tags: , ,

Wed, 18 Feb 2009:

The really hard part of APC is the internal locking code it has - it's not that hard to do, just hard to figure out if you've done it wrong. And I'm just about to really mess around with the assembly spin locks and pthread mutex locks to make them cross-process locks which live in shared memory (remember that "volatile" keyword in C?). The other couple of lock modes are already cross-process and slow (because of the syscall). If these work right, I won't really have to cripple the fast part of the code to implement the features I have in mind.

But before I start to go MIA into the locking code, I'd like to get my testing in place. So I've written a small and tiny test app called lockhammer - read the makefile and please run it on every platform you want APC to work. (make APC_DIR=~/apc link; make)

The code in lockhammer.c should be easily understood - basically it allocates some shared memory, creates a lock in it, forks, re-attaches the memory in each process. Every process is a loop of lock, write PID into shm, sleep, check the PID. In case someone has a better idea of how to test locks, I'll also like modifications to it, in case any of you think there's some corner case I missed (yes, random sleep & random fork-order is also on my list of TODOs).

Fundamentally, the information about locks is privately held within the lock type code in APC. The information needs to be moved into a shared mode (or at least, transparent) for multiple un-related processes to be able to share the cache without collisions. Eventually, you should be able to use APC in a standard FastCGI deployment without allocating a cache per-process.

And if you're a user, I'd like read something other than a bugreport, occasionaly.

They're gonna lock me up and throw away the key!

posted at: 19:07 | path: /php | permalink | Tags: , ,

Tue, 17 Feb 2009:

We condemn the most in others that which we hate in ourselves. For it is our nature to be honest and judge oneself so harshly, so much so that your judgement passes onto the actions of others. It's never their deed that you despise, but the bitter taste of your own which rises like bile from your gut. And you can never forgive, never forget.

At some fundamental level, we are incapable of being with people like us.

I'm a man, you're a woman. We're just too different.

posted at: 19:12 | path: /philosophy | permalink | Tags: , ,

Freedom isn't degeneracy. Oppression is never constructive.

Rebellion isn't the answer, it's merely a step. Freedom is a state of mind, reality checks are moot.

The ability to be whoever you want to be is no excuse to slip & be somebody you aren't, someone you don't recognize anymore. You are not truly free inside your mind, until you know your Ego & your Id - in your decisions, reasons and emotions. Until that day, freedom from the comfortable clasp of social norms is meaningless. Free from others, but still a captive of your desires and whims. Merely an illusion of isolation, holding onto it; Changing everything, seeking instability, perpetually emptying out your soul. Hoping to be free in solitude, but still not succeeding.

Perhaps I'm wrong and freedom's just another word for nothing left to lose, nothing left to keep and nothing left to fight for.

But how would I know? I'm not free, at least not yet, not for a long time to come.

Man is free the moment he wishes to be.
        -- Voltaire

posted at: 18:12 | path: /philosophy | permalink | Tags: ,

Sat, 14 Feb 2009:

It was always the story of a loser. A loser who lost it all and held onto his sense of loss, because here's the kicker, he's afraid to lose that too. A man who fails to own up to the loss, but keeps it alive, hoping to fill it one day with a replacement. A man so intent on keeping his loss, that what we see of him is a man with a hole where his heart was - wearing it proudly, but in disgust of a world which doesn't care.

What sets him apart from the regular losers of this world is his arrogance, which blinds him. His loss is something the world will never understand, because the world will never understand him. With this utter conviction, he embarks upon a demonstration of his loss - a path of self destruction and deterioration.

The original Devdas was the story of the deterioration of a man in love - unrequited love. A man of the world, taken out of society by himself, unable to rejoin the living in a land of fake love & real affection. A complete loser by any definition but a romantic's. But that's where any comparison to the original story ends.

Perhaps I'm biased. Or perhaps my emotions are lending perspective to the movie that just isn't there for others to draw upon. But I love the movie, the soundtrack, the screenplay and even the casting - half the movie is awesome thanks to Abhay Deol (If I wasn't a fan after Oye Lucky, I'm one, now). I'm not a big fan of Anurag Kashyap - No Smoking looked like the work of someone on some really cheap dope and John Abhraham didn't really help its cause either.

The movie is brilliantly placed in the midst of the day and age of the present. The events picked out from the media of today - the MMS case & the BMW case. The characters instantly recognizable. In fact, you could look at the players and see a little bit of yourself in all of them. The episodes chronicling the arrogance and downfall of Dev can be found in any dive bar in any city in this country - except for the brilliant background score. The events commonplace and the consequences natural - there's no pretension for the sake of the plot, no suspense of disbelief required.

Dev is an arrogant two-faced asshole. He loves no one but himself, but the loss of Paro means more to him than she would have ever meant to him by his side. Every step of the way, we can see a bit of his arrogance being eaten away by the results of it. The inner turmoil of someone who can't understand why Paro cares, but does not love, anymore. A man who's willing to love, but cannot find anyone worthy of such an honour. In fact, someone who's completely lost it - to the point of not remembering what "it" was.

Chanda has also been given a thorough exploration (that she deserves) in the movie, from her origins to the basic humanity of someone in that situation. She's been treated with her history in perspective, to actually lend some substance to a character who's technically a skin-deep style child. Again the casting choice has been excellent, including the accents, the appearance & the fluent tamil diction - all put to use fully in the character portrait.

To go watch that movie, only to notice the drugs, alcohol and prostitution is like looking at the Venus de Milo from the navel down. To me, at least, that was merely a plot device to illustrate the retrogression of Dev. And the soundtrack is simply brilliant - I'm still stuck in a loop between Pardesi, Emotional Atyachar (Rock version) and Nayan Tarse.

The ending is the redemption of the movie and the lead character. The arrogance washed out of him by his experiences, Dev finally arrives at that critical cross-road of life, revisiting his decisions, but without regret - to build a new him out of what's remaining. In a rather inverse-moralistic fashion, the people who gave it all up and tried to find themselves afresh are the ones who have found a measure of happiness. While the pragmatic Paro is stuck in a marriage burdened by an old husband and his other children, Dev & Chanda have actually got a real smile on the final fadeout.

Maybe it's not Happyily Ever After - but is there ever?

It is better to have loved and lost than just to have lost.

posted at: 06:13 | path: /movies | permalink | Tags: ,

Thu, 08 Jan 2009:

About six years ago, in a classroom somewhere I was daydreaming while some out-of-work CS grad was trying to earn his rent, babbling about operating systems to kids who didn't want to be there. This is not about that classroom or what happened there. This is not about the memories I treasure from those days or the tumultous years that were to follow. This is about the here, now and virtual memory.

The essential problem writing code that is suppossed to be transparent is that you can't enforce your will on other code. For instance, when dealing with shared memory, it would be impossible to walk through & prevent all code using it from stomping all over the memory returned from the functions. The rather obvious solution is to deal with this is to make a copy for every case and hand it out. But doing that for all the data basically hammers over memory of the process and basically chews out the entire memory space. For a long time, I've been repeatedly debugging my entire code base with a hacked up valgrind with watchpoints. But eventually, there's that once-in-six-million errors which just won't be reproducible no matter what.

It's Not Real: The thing about memory addresses is that they're merely an abstraction. Once that idea settles down, things like fork(), CoW and mmap() start to make more and more sense. The translation from an address to a real page in memory is fairly complicated - but the actual details are irrelevant. Just like an address gets re-mapped to a different physical location when a process forks, it is possible to engineer a situation where multiple addresses point to the same location.

But why?: Sure, there's no real reason to have two addresses point to the same page, if they behave identically. But what if they behaved differently? With the help of mmap() or shmat() it's possible to create multiple virtual address spaces for the same real memory with different permissions. I discovered this particular trick thanks to someone pointing me to the Xcache Wiki. But it bears more explanation than there was. Here's how it's done.

  char template[] = "/tmp/rowr.XXXXXX";
  int fd = mkstemp(template);

  ftruncate(fd, size);

  void *rw = (void*)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
  void *ro = (void*)mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0);

By mmap'ing the same fd twice, the same pages are available as RDONLY and RDWR addresses. These two addresses are different, but modifying the RW space will be reflected in the RO space - sort of like a read-only mirror of the data. A nearly identical example is possible with the use of shmat() with different permissions.

protect/unprotect: Since these are different address spaces, it is easy to distinguish between the two. Converting a RW ptr to RO, can also be accomplished simply with a little pointer arithmetic. The code would look somewhat like the version below without any bound checking. But ideally, some sort of check should be put in place to ensure double conversions don't cause errors.

  int *p = (int*) rw;

  #define RO(p) (p) += (ro - rw);
  #define RW(p) (p) += (rw - ro);


  *p = 42; /* causes segv */

  *p = 42; /* safe to do */

The essential task left is that all the pointers stored should be RO pointers. After storing the data, the pointers have to be flipped RO. After which, any memory walking would essentially walking over the RO "mirror" and cannot corrupt memory. All the unprotect operations would have to be inside a lock to ensure safe operation. And I do really have to thank Xcache for the idea - I'll finally have to stop hiding behind the "Zend did it!" excuse soon.

For those intrigued by my code fragments, but too lazy to fill in the blanks, here's a fully functional example.

Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats.
       -- Howard Aiken

posted at: 21:43 | path: /hacks | permalink | Tags: , ,

Wed, 07 Jan 2009:

I had a very simple problem. Actually, I didn't have a problem, I had a solution. But let's assume for the sake of conversation that I had a problem to begin with. I wanted one of my favourite hacked up tools to update in-place and live. The sane way to accomplish is called SUP or Simple Update Protocol. But that was too simple, you can skip ahead and poke my sample code or just read on.

I decided to go away from such a pull model and write a custom http server to suit my needs. Basically, I wanted to serve out a page feed using a server content push rather than clients checking back. A basic idea of what I want looks like this.

twisted.web: Twisted web makes it really easy to write an http webserver without ever wondering how an HTTP protocol line looks like. With a sub-class of twisted.web.resource.Resource, a class gets plugged into the server in-place & ready to serve a request. So, I implement a simple resource like this.

from twisted.internet import reactor, threads, defer, task
from twisted.web import resource, server

class WebChannel(resource.Resource):
        isLeaf = 1
        def __init__(self):
                self.children = {}

        def render_GET(self, request):
                return "Hello World"

site = server.Site(WebChannel())
reactor.listenTCP(4071, site)

Controller: The issue with the controller is mainly with the fact that it needs to be locked down to work as a choke point for data. Thankfully twisted provides twisted.python.threadable which basically lets you sync up a class with the following snippet.

class Controller(object):
	synchronized = ["add", "remove", "update"]


loop & sleep: The loop and sleep is the standard way to run commands with a delay, but twisted provides an easier way than handling it with inefficient sleep calls. The twisted.internet.task.LoopingCall is designed for that exact purpose. To run a command in a timed loop, the following code "Just Works" in the background.

	updater = task.LoopingCall(update)
	updater.start(1) # run every second

Combine a controller, the http output view and the trigger as a model pushing updates to the controller, we have a scalable concurrent server which can hold nearly as many connections as I have FDs per-process.

You can install twisted.web and try out my sample code which is a dynamic time server. Hit the url with a curl request to actually see the traffic or with firefox to see how it behaves. If you're bored with that sample, pull the svgclock version, which is a trigger which generates a dynamic SVG clock instead of boring plain-text. I've got the triggers working with Active-MQ and Mysql tables in my actual working code-base.

Success just means you've solved the wrong problem. Nearly always.

posted at: 19:45 | path: /hacks | permalink | Tags: , ,

Tue, 16 Dec 2008:

Finally, after nearly a year of work, it's into a release. Some new stuff has sneaked into it undocumented, that people might find interesting - apc.preload_path would be one of them. The backend memory allocation has been re-done - the api part by me and the internals by shire. There's a hell of a lot of new code in there, both rewritten and added. Tons of php4 cruft removed, php5 stuff optimized, made more stable, then less stable, made faster, then applied brakes. Made leak-proof, quake-proof and in general, idiot-proof. So, on & so forth.

 apc/ $ cvs diff -u -N -r HEAD -r RELEASE_3_0_19 | diffstat /dev/stdin 
 68 files changed, 3255 insertions(+), 5545 deletions(-)

Sorry about the b0rked 3.1.1 release, so please test this one! :)

Each new user of a new system uncovers a new class of bugs.
                -- Kernighan

posted at: 15:27 | path: /php | permalink | Tags: , ,

Tue, 11 Nov 2008:

I'm a very untidy tagger. I tag randomly and according to whim. Over the last year, thanks to my static tags plugin for pyblosxom, I have accumulated 150+ tags. Most of those are applied to hardly one or two entries and increasingly my tag cloud has been cluttered with such one-offs.

I spent an hour today checking out how to represent so much information into a single screen-ful. And I found a brilliant example in the penny packer extension.

So bye bye to the plaintext tags. Say guten tag to the Ishihara version.

price tag on other side.

posted at: 10:45 | path: /hacks | permalink | Tags: , , ,

Sat, 01 Nov 2008:

Next time you walk past a mirror, stop & reflect.

Look deep into those eyes for a glimmer of something real. Break through that facade that has taken several precious years to build. Through all the limits that the world has put on you. Take a good look at yourself.

The most interesting thing about you isn't the people you have surrounded yourself with. It's not the assortment of gadgets that adorn your fingertips, pockets or desk. Neither is it the long list of books you've read and several others you own, merely for others to comment upon. Nor is the rather eclectic taste in music that the pinball machine of life have bounced into you. The sips and dips you've taken, places you've gotten lost in or even whom you hooked up with & when.

It should've been you - just you. But you've lost yourself in incidentals. Thrown away yourself only to replace it with the byproducts of your existence. You became a showcase, instead of a person - with whatever you've got, perpetually on display. What you are suddenly was skin-deep and then it got ugly.

Maybe you were ashamed. Ashamed of who you were. It was fun to pretend you weren't anymore, but what truly changed? You or just a new mask? Wearing it, hoping that nobody ever comes into your life to whom you'd have to bare all? Keep it on forever?

Maybe you'll try. Maybe it was better than being nothing in the eyes of others. I said, maybe.

But just remember not lose all you've collected - at any cost. Because once you've lost yourself, that's all you got. That's all you got.

What ever you can give up, that's all you have. The rest is what you are.

posted at: 02:13 | path: /rants | permalink | Tags: , ,

Sat, 18 Oct 2008:

In the development of things, there comes a point when it escapes the vision and control of one man/one mind. PHP frameworks are such ... beasts. But the simplicity a machine took away can be made to return. And such an attempt at zooming out of the complex file structure bureaucracy of most php projects was inclued.

When I hacked up that extension, nearly a year back, I wished that it would shame at least some php programmers into writing better code. And slowly, thanks to a few slides from Rasmus, people are actually slowly realizing how messy their include hierarchy really is. And here's an example of what I'm talking about.

That was the Zend Framework 1.5.2, as blogged by phpimpact - download the big one and look at it. The joomla CMS has also got its very own pretty picture elsewhere. Rasmus has a bunch of inclued traces from various frameworks - CakePHP, Symfony, Drupal, and perhaps the cleanest of them all, CodeIgniter.

Now, all that remains is a php-graphviz + svg mode which renders these in-browser as an iframe - or maybe someone can help me with the graph reduction to take a collection of the inclued dumps & create a "package". There's none an end to the bells and whistles I want to tack onto this.

But as long as people are scrambling head over heels to reduce the number of includes & include_onces, I think I've done my part here.

Must I hold a candle to my shames?
                -- William Shakespeare, "The Merchant of Venice"

posted at: 01:27 | path: /php | permalink | Tags: , ,

Mon, 29 Sep 2008:

But that's actually okay, because I will not satisfy a woman, either". And that's how Sex Drugs & Cocoa Puffs kicks into action, to embark upon a Low Culture manifesto, exploring the deeply superficial phenomenon that modern culture has turned into. Exactly the kind of trash literature you can only find in the otherwise bare shelves of an airport bookshop. And I guess it was that phrase that made me pick it up from LAX.

There are two ways of look at this book. Actually there are thousands of ways, but the preface said there were only two that me & the author might have in common. The first and of course, the less popular view is that everything's a product of chaos and inherently independent of everything else. On the other hand, everything pretty much stays the same and everything's bound in a chain of causality in an inherently holistic big-picture. And yes, reading each essay of the book somehow emphasizes both the views in some strange paradoxical sense. Completely random thoughs & events, but from a single man's perspective.

There's no semblance of order or even pretence of continuity about this book. But there is a certain domino effect to the essays as they dove-tail into a impressionist masterpeice, but one which exists merely to demonstrate the ever broadening palette of culture.

Porn: Nothing's more representative of the dark side of modern humanity than porn. He's done to porn what Andy Warhol did to a humble soup can. He has framed the entire trend of internet porn, which has acquired a certain Marxism to it, into - "when you don't have to be Lenny Kravitz to know how Lisa Bonet looks like in a shower. You don't even need hemp pants.". And the comfortable knowledge that your wife and Gwent Stefani have more in common than you would admit in the first place. Essentially stealing back the power we lent to celebrities, their privacy (or lack there of) & their exclusivity, as a sort of a price to be paid for fame.

There's more to it than just that. More on that later. My thoughts, which are inextricably weaved into his ideas, probably deserve blog posts of their own.

Do not write below this line.

posted at: 01:01 | path: /books | permalink | Tags: ,

Mon, 15 Sep 2008:

It's a protest. A protest against all the gags the establishment has put on php functions - big, small and useless alike. No more shall they remain ignored and voiceless. Hear me now, as the day has come for them to shake off their silence and SCREAM!

@Error: Php uses the @ operator to silence errors from functions, so that they fail silently. But while tracing through code which uses it, it becomes nearly impossible to properly figure out what is going wrong. The band-aid that is '@' makes it a complete pain to debug code.

Introducing the SCREAM 0.0.1, which has come out of someone else's frustrations with some pear modules which are liberally peppered with such gag instructions. Essentially, it uses the user opcode functionality to override the silence functions into NOPs (literally).

php -dscream.enabled=1 -r '@foo();'

  Fatal error: Call to undefined function foo() in Command line code on line 1

Dump it into php and hopefully debugging sloppily coded libraries should become much easier. This message was brought to you by the dread of Mondays. It's all over now - End Transmission.

You have not convinced a man because you have silenced him.

posted at: 21:09 | path: /php | permalink | Tags: , ,

Mon, 11 Aug 2008:

In memoriam - Monday, August 11th 2003.

Half a decade later, I still remember like it was yesterday.

It will be a memorable time -- no matter how hard you try to forget it.

posted at: 12:30 | path: /me | permalink | Tags: ,

Tue, 05 Aug 2008:

Money is what money does.

Living in the suburbs with an economic major for a dad, I was subjected to long drives filled with words like Price, Cost, Wealth and Money. Slightly different, shadow images of the same thing - Value. Transactions, hoards and transformations there-of. But this system of Value, has no absolutes - it's a general theory of economic relativity. The essential backbone of which is the equation of effort to value and in some essence has been as revolutionary to me as Einstein's has been to 19th century physics.

$$$: Money is worth what it buys you. The only reason it exists is because other people accept it and provide you services/goods. That'd be why it is infact called currency, because that is the most important property for money. But the exchange of money look like a zero-sum game - you gain only as much as I give. But transactions of value have to be a non-zero sum game. For a stable economic system, the buyer & seller should both go home with more than what they started with.

!Zero: The nonzero sum aspect of trade, is what set that juggernaut rolling in the first place. But to acquire more value than the other gives up, requires for an inequity of value in exchange of money. What I give up in return for money is more valuable to the person paying for it than what she pays for it. And thus the ship sails into the next port.

The Pin: The pin maker is a skilled man. All day long, he makes pins. It would take you an entire day's work to make a pin. He makes ten pins a day. If he demanded half-a-day's worth to make a pin for you, would you pay? I assume you would, after cribbing a lot about the rising cost of living, how they don't make them like they used to and how your children no longer behave. But you'd pay, take the pin & spend the rest of the day doing something fun.

And here's where the inequity of effort and value kick in. It is an equation, for sure, but one of the multipliers in it is skill at a task. The price you pay is fair for something if it takes more effort to do it yourself - DIY solutions excepted. The foundation is laid for the segregation of society into guilds, based on skill & strata based on the multiplier of effort to value. Productivity is no longer a measure of effort, but a combination of various other things.

The world we see around us is the effect of such a run-away economic chain reaction - where all's fair in the price war. There's nothing you'd buy at a loss, whether you profit in fluid currency is another matter altogether. You're built to be a winner at the game of value, but transmuting it into gold is another skill altogether. But you still got what you paid for.

On the other hand, there's probably one born every minute, who hasn't.

PS: this entry initially was penned in my notebook, when people protested about Apple dropping the iPhone price

The use of money is all the advantage there is to having money.
                -- B. Franklin

posted at: 09:12 | path: /observations | permalink | Tags: , ,

Frustration is my fuel. I spent an all nighter re-doing up one of my old valgrind patches to work with valgrind-3.3.1. This one was a doozy to patch up the first time (stealing rwalsh's code), but not quite very hard to keep up with the releases. The patch needs to be applied to the 3.3.1 source tree and memcheck rebuilt. It also requires the target code to be instrumented.

#include "valgrind/memcheck.h"

static int foobar = 1;

int main()
	int *x = malloc(sizeof(int));
	int wpid = VALGRIND_SET_WATCHPOINT(x, sizeof(int), &foobar);
	*x = 10;
	foobar = 0;
	*x = 10;

What has been added anew is the foobar conditional (you could just pass in NULL, if you always want an error). In this case the error is thrown only in first line modifying x. Setting the conditional to zero turns off the error reporting.

With the new APC-3.1 branch, I'm again hitting new race conditions whenever I change around stuff. I have no real way of debugging it in a controlled environment. But this patch will let me protect the entire shared memory space and turn on error flag as soon as control exits APC. Just being able to log all unlocked writes from Zend should go a long way in being able to track down race conditions.

Yup, frustration is my fuel.

An intellectual is someone whose mind watches itself.
                -- Albert Camus

posted at: 05:44 | path: /hacks | permalink | Tags: , ,

Wed, 30 Jul 2008:

I thought I'd done this once already. I guess I have no recourse other than to follow the crowds wherever they go - follow the herd and keep up.

So, twitter gets a new t3rmin4t0r. And the countdown into obsolescence begins anew.

It would be illogical to assume that all conditions remain stable.
                -- Spock

posted at: 18:20 | path: /misc | permalink | Tags: , ,

Everyone has one story to tell.

But blessed are those who have the ability to articulate it in words. For the struggle of a mind to tell all and reveal; It's to its detriment perhaps, to give up a bit of itself. But perhaps it is the escape of a mind from the confines of a body, of thoughts from a mind. And of course, ideas from the thought behind them. This is perhaps the greatest leap ideas have ever taken - from mind to mind, over time and distance.

Free to move between the minds of others, the thoughts have transcended the original boundaries of man, to enter the realm of the plural. Man became men, me became we, you becomes us. No longer does it have a single pale shade of white, but the tinges of grey that everyone's dirty fingers have added to the paperback. Context & perspective, shift and shear the words' import and the meaning escapes the control of the author.

My story becomes yours to repeat, if not to experience. Or perhaps not. The true magic of the written word was never in its production, but in the re-production of emotions that such pithy squiggles can carry within them. The imaginative machinings involved in reading, as mundane as it may seem, is no less a daily miracle.

And the rest is left as an exercise for the reader.

The trouble with telling a good story is that it invariably reminds the other of a dull one.

posted at: 18:06 | path: /observations | permalink | Tags: , ,

You are disconnected, you live in your own shell. The world exists for you only when you wish it to be. Your world is one of your own, independent from those around you. And you learn to enjoy the silences, they carry their own sense of meaning, peace and serenity.

And one day you forget how to feel. Caught up as a spectator to your own life, watching, taking apart and doing what's rational. Being successful, but completely unsatisfied with your own success. It's not that you don't crave for meaning, but it's like a tango with the world in lead. You realize that you've lost purpose, drive and emotion - but it's been taken care of.

Of all the things you miss in life, that's what you miss the most - emotion. Pure indiluted irrationality has leaked out of your mind, leaving a completely rational automaton behind. Every action you take is followed by a Why? and one day the answer "just because" stops answering it. And it feels like it's never been enough of a reason, anyway.

Living in the present. A pitiable existence, a slave to your senses. Never nostalgic about the good times of past, never looking towards anything in particular. You eyes, ears and intellect guiding your hands. Doing what's needed - no more, no less. Somehow rationalizing to yourself, that you're just waiting for something. Something that signals the end, turns the page. You know you need something, but not knowing what you need. Spending your days, waiting for your real life to begin in full earnest, with a happily ever after.

Comfortable. Sane. Numb.

Sun rises one day and something in you awakes. Trapped in a perfectly logical box of thought, it tries to break out. Illogically, it seeks to renounce everything that makes you comfortable. Seek pain, not enjoy it - but need it to break out, start feeling alive again. The veil that is laid over your emotions is torn to bits in a fit of anger and desperation. You hurt yourself, you want pain, for the lack of anything better. You hurt others, perhaps you run away from those whom you might. But most of all, you cut yourself open & bleed.

Understanding it was hard. Pain - it's only catharsis. The puff of smoke as you cauterize your endless haemorrage of reason, to be human again. Deal with it, learn to live again, love again; laugh again.

Wait, did I say "again"?

Never look back, the view is never as good.

posted at: 07:47 | path: /philosophy | permalink | Tags: , ,

Tue, 22 Jul 2008:

What a month! First week spent far far away from my worries in Ladakh, a week of mourning my grandmother's death in Cochin, a week (hardly enough) in Bangalore and now I'm in Portland. Trying to adjust my sleep-cycle with the help of coffee and the lack-there-of. Wasn't too bad though, the CX flight had episodes of Life in Cold Blood, Scrubs and The Ali G Show. A potent combination which mixes all that my mind craves - fun, information and philosophy, though not necessarily in that order.

So, I guess it's time I stopped saying that "I hate travel". I don't quite mind the results.

How often I found where I should be going only by setting out for somewhere else.
                -- R. Buckminster Fuller

posted at: 03:10 | path: /travels | permalink | Tags: ,

Fri, 11 Jul 2008:

I'm back.

Had an awesome time in Ladakh - rented bullets to drive around, went white water rafting, camped out near Tso Moriri, rode 100-odd km in a car without brakes, night vigils on a bridge over the roaring Indus and last but not least bicycling down at 55 kmph from the highest paved road on the planet.

But now, I'm back in b'lore and in process of getting myself back into the same 'ol rut. Already got side swiped on the road by an idiot (or whatever the female equivalent is) and now limping along with a toenail torn off.

Trust me on this, Bangalore roads are more dangerous than anything the himalayas could throw up. More on this later.

It is the business of the future to be dangerous.
                -- Hawkwind

posted at: 17:10 | path: /travels | permalink | Tags: , ,

Sat, 28 Jun 2008:

The Yeti, actually. By the time you're reading this, I'll probably be in the high himalayas, gasping for breath.

Planning to mountain bike around the place, will be back soon. After all I've seen all the sights before.

The road to Hades is easy to travel.

posted at: 08:14 | path: /travels | permalink | Tags: ,

Sat, 14 Jun 2008:

Ashamed I'm not to borrow the words of te inimitable Jerome K Jerome.

We are but the veriest, sorriest slaves of our stomach.  Reach not after 
morality and righteousness, my friends; watch vigilantly your stomach, 
and diet it with care and judgment.  Then virtue and contentment will 
come and reign within your heart, unsought by any effort of your own; and 
you will be a good citizen, a loving husband, and a tender father - a 
noble, pious man.

But as someone remarked "nobody starves anyone else".

It is very strange, this domination of our intellect by our digestive 
organs.  We cannot work, we cannot think, unless our stomach wills so.  
It dictates to us our emotions, our passions.  

How good one feels when one is full - how satisfied with ourselves and 
with the world!  People who have tried it, tell me that a clear 
conscience makes you very happy and contented; but a full stomach does 
the business quite as well, and is cheaper, and more easily obtained.  
One feels so forgiving and generous after a substantial and well-digested 
meal - so noble-minded, so kindly-hearted.

I always thought it was just me...

Man doth not live on bread alone;
but philosophy baketh no bread.

posted at: 02:47 | path: /philosophy | permalink | Tags: , ,

Mon, 26 May 2008:

That annoying file descriptor leak that snuck into 3.0.17 has finally been laid to rest. A few double free issues were fixed as I spent quite a long time staring at the same code, till enlightenment hit me like a clue bat. Along with that, there are a bunch of quickfixes for 5.3 quirks. I'm not happy with those, but this is the 3_0 stable branch and by the time 5.3 is popular enough the HEAD should be taking care of those problems. The build is broken in VC++ in this release, but excluding apc_pool.c from the build should work.

Expect more changes... as soon as I get back home.

Delay always breeds danger and to protract a great design is often to ruin it.
              -- Miguel De Cervantes

posted at: 04:27 | path: /php | permalink | Tags: , ,

Thu, 22 May 2008:

Another day passes, so what if it was my birthday?

Maybe tomorrow's yours and the world won't care either way.

^Z ^z^z^z ...

Who will take care of the world after you're gone?

posted at: 08:31 | path: /me | permalink | Tags: ,

Tue, 06 May 2008:

For most of April, I've been on the road. Late night rides all around town, through inner ring road and cubbon, chugging along on my new bike. The weather's been brilliant and I was out, just cruising along the roads, like in a dream.

Yup, I went and splurged on a brand spanking new Royal Enfield Thunderbird. Full 350 cc, 18 BHP of raw petrol power, purring along like a kitten at full throttle, but whisper quiet in idle. For such a heavy bike, it is surprisingly easy to handle, taking bottom heavy curves on turns and excellent stability at speeds. The bike's behaved pretty well so far, the first full tank of petrol lasted me for around 600 Kms. And for once, I'm actually riding a bike made for someone six feet plus. In short, I love it.

The nights are long and the roads are empty
The world's sleeping and nobody's watching
And the wind whispered to me; keep on driving.

If you worried about falling off the bike, you’d never get on.
         -- Lance Armstrong

posted at: 20:20 | path: /hardware | permalink | Tags: ,

There's a certain cultural bankruptcy which shows itself in sequels. It indicates, that you're reduced to imitating yourself. But this isn't that kind of a sequel. No, not the kind where there are T Rexes in the city, trying to make a living drawing cartoons or Arnie switching from ammo boxes to ballots. This is the kind which gives a New Hope.

Yesterday, I had an outpouring of hate against the linux capability model. But the problem turned out to be that setuid resets all the capabilites. In hindsight that makes a lot of sense, but didn't even strike until the kernel people (y! has those too) got involved (and I didn't RTFM).

Enter Prctl: The solution was to use the prctl() call with PR_SET_KEEPCAPS to ensure that the capabilities are not discarded when the effective user-id of a process is changed. But, even then, only the CAP_PERMITTED flags are retained and the CAP_EFFECTIVE are masked to zeros.

So, with the prctl call and another cap_set_proc to reset CAP_EFFECTIVE, it was on a roll. Here's the patch on top of unnice.c.

 #include <sys/resource.h>
+#include <sys/prctl.h>;
@@ -26,12 +27,14 @@

+       prctl(PR_SET_KEEPCAPS, 1, 0, 0, 0);

        /* child */
        if(setuid(nobody_uid) < 0)
+       cap_set_proc(lcap);

        if(setpriority(PRIO_PROCESS, 0, getpriority(PRIO_PROCESS, 0) - 1) < 0)

Thus concludes this adventure and hope that this blog entry serves as warning of things to come. Watch this space for more Tales! Of! INTEREST!.

Only great masters of style can succeed in being obtuse.

posted at: 18:34 | path: /php | permalink | Tags: , ,

Mon, 05 May 2008:

Running infinte loops is a tricky challenge. What happens to a process when a programmer writes an infinite loop, should be familiar to all. But the challenge is to not let that affect the *other* processes. There seemed to be a perfect solution to it - setrlimit().

The function lets you set soft and hard limits on CPU, so that if a process does exceed the soft limit CPU usage, a SIGXCPU is raised. The process can catch the signal and do something sensible. Basically, all that was required was for the process to call setpriority and let the linux process scheduler slow it down to a trickle.

But a process can lower its priority, but not raise it - if it is a non-privileged process. But linux capabilities allows you to grant CAP_SYS_NICE to the process which essentially lets a non-privileged process muck around with priority - down and up.

To begin with /proc/sys/kernel/cap-bound is unbelievably confusing to use. It is a 32 wide bit-mask on which the 23rd bit apparently seems to be the CAP_SYS_NICE value. After much mucking around, I came to the conclusion that "-257" would be 0xFFFFFEFF which only disables CAP_SETPCAP. But even then the setpriority call kept failing. Here's my test code.

cap_t lcap;
const unsigned cap_size = 1;
cap_value_t cap_list[] = {CAP_SYS_NICE};

cap_set_flag(lcap, CAP_EFFECTIVE, cap_size, cap_list, CAP_SET);
cap_set_flag(lcap, CAP_PERMITTED, cap_size, cap_list, CAP_SET);


if(setuid(nobody_uid) < 0) 

if(setpriority(PRIO_PROCESS, 0, getpriority(PRIO_PROCESS, 0) - 1) < 0) 

Here's a link to the test case in a more compileable condition. Build it with gcc -lcap and run with sudo to test it. Right now, my ubuntu (2.6.22) errors out with this message.

bash$ gcc -lcap -o unnice unnice.c
bash$ sudo ./unnice 
0: =ep cap_setpcap-ep
setpriority: Permission denied

The core issue has to do with apache child-process lifetimes. The only recourse for me is to kill the errant process after the bad infinite loop and have the parent process spawn a new process with a normal priority. But which means blowing off nearly all the local process cache, causing memory churn and more than that, the annoyance of a documented feature not working.

This story currently has no ending, but if any kernel hackers are reading this and should happen to know an answer, please email gopalv shift+2 php noshift+> net. And thus we prepare for a sequel (hopefully).

I use technology in order to hate it more properly.
                -- Nam June Paik

posted at: 22:03 | path: /php | permalink | Tags: , ,

Sun, 04 May 2008:

Flaws are inherent in the makings of a hero. For a perfect canditate for heroic tasks would be a man made out of stone - cruel, rational and unemotional. But only cruel to be kind, rational to the core and unstirred by emotional pulls. But who would want to call him a hero?

Over the last two days, I slowly trawled my way through the twelve issue paperback of Watchmen. It has proven to be an interesting comic to peruse.

The book is a continous dismantling of the word "hero". More appropriately, of the super-hero genre - the take-no-lives, nobody-got-hurt-but-the-criminals world of the original heroes. But having accepted collateral damage as a way of life, for the super hero, the question remains - how far will you go to save humanity?.

For your own good: More interesting is the dissection of the purpose of authority. The question hangs in the air - "Who are we protecting them from?". And only the comedian dares answer, but with mirth - "From themselves". Is a nice concept, that few will suffer for the good of many. But that line having been crossed, decisions get mired in a subjective quagmire of the value of a life against another. And it evades the other question - what happens to those who dispense with justice?

quis custodiet ipsos custodes: Ah, but who will watch the watchmen. And even if benevolent they be, what if they take the path piled with corpses. The entire book is a critque on the hypocrisy of authority, the few who are not accountable to any. After all, as I've noticed, in a fair world intentions do not matter.

The book leaves you doubtful how to interpret the shades of grey in people. But at least that's why Rorschach has only black or white - never any gray, on his face. Perhaps to him, it's all black'nwhite.

Perhaps, the real hero is a monster who's cruel to be kind. Amen.

God instructs the heart, not by ideas, but by pains and contradictions.
                -- De Caussade

posted at: 00:01 | path: /books | permalink | Tags: ,

Wed, 30 Apr 2008:

It's a very liberating sensation. But for those who have become accustomed to seeing me with long and curly hair, this must come as a shock. The summer was getting to me and my other reasons to keep my hair uncut have long since disappeared. And so has my hair.

After having survived countless threats by several (near & dear to me) to have it cut during my sleep, my current crop of hair has been reaped. I've come to realize how much of a hassle it had been. Except for the bright April sun beating down on my head or the newly acquired intimate knowledge of A/C vent locations in office, everything else seems to be going according to plan.

And after all, it'll grow back. It's not like this is the first time I've doen this.

Worry about your hair too much and soon you won't need to.

posted at: 20:39 | path: /me | permalink | Tags: , ,

Mon, 14 Apr 2008:

Conflict is the essence of drama. Conflict between people is easy enough to construct - the villain, the hero and the heroine. Put a face on each and the characters play themselves out. Evil shall rule, but it shall be short-lived. Good shall win at the end and order shall be restored.

But occasionally something comes along which distills drama out of nothing but circumstances. No evil, merely ordinary people, their thoughts and cognitive dissonances echoing off each other. That's how I felt at the end of watching Juno.

Unlike other movies I write about, I have no spoilers for this one. But the soundtrack haunts me. Definitely something to go back and listen to - not just for the melody, but for the lyrics.

You're such a good friend   
I haveta break your heart.  
Tell you that I love you    
then tear your world apart.

Listen - and hum along ... ~~ you're a part time lover and a full time friend ~~.

I thought drama was when actors cried.
But drama is when the audience cries.
       -- Frank Capra

posted at: 15:12 | path: /movies | permalink | Tags: ,

Wed, 26 Mar 2008:

In response to CVE-2008-1488, APC 3.0.17 has just been pushed out with the requisite security fixes. But in the process of producing a php4 compatible release, a significant amount of code has been reverted in the merge into an APC_3_0 branch for future bugfixes.

I've spent a couple of hours unmerging my "bye bye php4" cleanups with the help of Kompare. And my sanity is simply due to the fact that I can "cvs diff -u | kompare -" to look at the resulting huge patch. But it is not unpossible that the new code merged from HEAD has regressions, so you could also apply the unofficial patch onto 3.0.16.

I took your advice and did my own thing. Now I've got to undo it.

posted at: 22:03 | path: /php | permalink | Tags: , ,

Fri, 21 Mar 2008:

I'm sorry, I just received a late update (at great cost) from Anthony Baxter.

The last keynote from Linux.conf.au was about the upcoming Python "We'll break all your code" 3k. Perhaps not quite keynote material, but it covered a gamut of issues which will break old code when the move happens - and he did work for a colorful company. The set & dict comprehensions, function annotations and dictionary views are probably worth the terrible loss of reduce(), my favourite companion to map(). And obviously old style classes & string exceptions were excess fat to be trimmed anyway. But there was more interesting code to test.

>>  from __future__ import braces
  File "<stdin>", line 1
SyntaxError: not a chance

>>  import this 

And check the output twice.

I believe a little incompatibility is the spice of life, particularly if he has income and she is pattable.
                -- Ogden Nash

posted at: 04:13 | path: /conferences | permalink | Tags: , , ,

Mon, 17 Mar 2008:

I was there, but I didn't talk. Didn't sit around either.

Yes... I stood behind people.

And then, I came back. With a somewhat odd feeling at the back of my mind. Somehow, unlike all the other years, this year it felt more like a JNU event, rather than an ILUG-D event. What with the focus shifting off software into the rather vague (for me) realms of freedom. Maybe it was the conclave or the dinners, it didn't scream out ILUG-D like it used to - but that may not be a bad thing, per se.

Shall just have to wait and watch.

In this country we enjoy many freedoms; freedom of speech, freedom of thought and indeed, the freedom to do neither.

posted at: 03:13 | path: /conferences | permalink | Tags: , ,

Wed, 12 Mar 2008:

Occasionally, I do wonder why I do what I do, especially with software. The answers I've got with might not make sense to everyone - actually, it no longer makes too much sense to *me* today. But one thing's remained the same - it wasn't the money then and it isn't money now.

Everyone admits that more money helps (talks, walks you home and even buys you dinner). But the question at hand was whether money, more accurately, the presence thereof, hurts open source projects. And Stormy Peters began with that question, into her LCA '08 keynote.

The internal motivations to write F/OSS usually are weakened when there's greenbacks flying. Not to mention money brings in the people who herd it, from the clan of MBA (not to be confused with N.W.A). The processes and rest of the beasts of communication, reports and roadmaps which follow aren't necessarily bad, but they aren't "fun". But even when hacking hasn't been instutionalized, money is a selective reward, which splits the community into those who are paid and the rest. Dunc-Tanc is a classic example of this divide in action. And I'd personally lost that rush the moment I start equating efforts to money - there's a stress level which you can't pay me to take up, but in which I used to live day in & day out.

Stormy Peters takes on the topic from the point of view of someone actively involved in funding open source projects & developers. Occasionally, I run into clearer insights than mine, but conclusions I gathered from the talk are something to a closer look at.

Once a f/oss developer, always a f/oss dev. Even when you've gotten off the rush for some project, pushing it along as a good engineer, the hacker in you doesn't sit around. So, even if you don't do the same for free, it's very likely you'll do something else for the same reasons you did it first - look at the ex-Eazel folks. Anyway, history might just rhyme or pun, or maybe people just go do the same again ( *cough* Gnome Online Desktop), but whatever they already did - is not lost.

Anyway, to the objective reader, I'd suggest that you ignore my blatant bias in the above post and go watch the video.

It's classic percolate-up economics, recognizing that money is like manure: It works best if you spread it around.

posted at: 21:54 | path: /conferences | permalink | Tags: , ,

Thu, 28 Feb 2008:

In the last three years, very few things have stayed the same for me. Hardly did I think when I walked into my cube in Feb 2005, that I'd still be sitting in the same exact cube when 2008 dawned. My team changed, my entire upward hierarchy changed, even the company name changed - but hidden behind the walls of 3A-14 I sat. Slept there, ate there and sometimes even did some work.

I've come a long way in those three years, but I hardly moved at all. But perhaps this world did move a lot and all's relative.

This Universe is populated by stable things.
                -- Richard Dawkins

posted at: 18:52 | path: /yblr | permalink | Tags: , ,

Tue, 19 Feb 2008:

After all the dust from the miniconfs settled down, the real conference began with a real treat - Bruce Schneier talking on Reconceptualizing Security [video (~150 MB)]. The keynote was about dealing with security as an intangible feeling as well as the reality of security.

Lemons: But before I head off into what that talk was about, let me introduce a new concept - Market for Lemons. The idea is that when the quality of a product (like a used car) cannot be assessed easily, the bad car (i.e a lemon) would sell for a profit, while the good car wouldn't have any customers because it is expensive. The scenario is exactly tilted towards the dishonest seller, in the absence of proper information and/or blacklists for lemon peddlers.

The security software market right now is such a one (I'd say that the job market is too). Products which are pure snakeoil are completely indistinguishable from real products from a brochure or presentation. As was clearly illustrated today. The company which invests more efforts in creating the illusion of security would potentially win over someone who has been actually spending time & effort to secure their product. The price war cannot be fought between good & bad without enough in-depth information about stuff under-the-hood.

There is probably a sweet spot between real security and the illusion of security, which would be profitable. It wouldn't be bullet-proof security by any stretch of imagination, but the sense of security wouldn't necessarily be misplaced. Which is what the profit-oriented capitalist economy would gravitate to, provided there was enough awareness & information about what is actually in the box.

Q&A: I actually got to ask Bruce Schneier a question - about the meta-stable state of this balance, where every new development/marketing tactic would again upset the market off this point. He admitted that it was an interesting question, but suggested that it would probably dampen into insignificance (think about soaps & detergent brands).

Anyway, there I was, communicating to Bruce Schneier without encyrption.

There's small choice in rotten apples.
                -- William Shakespeare, "The Taming of the Shrew"

posted at: 20:13 | path: /conferences | permalink | Tags: , ,

Mon, 18 Feb 2008:

Looks like I'll be able to make it to freed.in. Except, in a departure from the usual, I'll not be presenting there. I wasn't online or in any state to prepare a decent proposal for the conference. I'll be attending it as a vanilla delegate to the conference, possibly talking about something during the conclaves.

Small Banner - 430x60

I wouldn't miss it if I were you (even if I were me, which I am).

Nothing will dispel enthusiasm like a small admission fee.
                -- Kim Hubbard

posted at: 23:42 | path: /conferences | permalink | Tags: ,

Wed, 30 Jan 2008:

DHCP makes for bad routing. My original problems with DHCP (i.e name resolution) has been solved by nss-mdns, completely replacing my hacky dns server - ssh'ing into hostname.local names work just fine.

But sadly, my WiFi router does not understand mdns hostnames. Setting up a tunnel into my desktop at home, so that I could access it from office (or australia for that matter), becomes nearly impossible with DHCP changing around the IP for the host.

UPnP: Enter UPnP, which has a feature called NAT Traversal. The nat traversal allows for opening up arbitrary ports dynamically, without any authentication whatsoever. Unfortunately, there doesn't seem to be any easily usable client I could use to send UPnP requests. But nothing stops me from brute-hacking a nat b0rker in raw sockets. And for my Linksys, this is how my POST data looks like.

<?xml version="1.0" ?>
<s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:s=
        <u:AddPortMapping xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
            <NewPortMappingDescription>SSH Tunnel</NewPortMappingDescription>

And here's the quick script which sends off that request to the router.

Air is just water with a lot of holes in it.

posted at: 14:45 | path: /hacks | permalink | Tags: , , ,

Monday: I had this illusion that I had no jet lag after reaching melbourne. So, I headed out to the security miniconf. The WiFi wasn't up yet (it never is for the first day). The couple of talks I was really focussing on was the OpenSSH one and the 'aggressive' Network defense one.

The latter talk had some interesting 'facts' about ICMP. The ICMP host redirect used in the sense of "talk to the hand" packet dumping. Also suggestions for an ICMP quench message to be un-deprecated as well as a lot of ire directed at sysadmins who just drop ICMP at fire-walls.

The OpenSSH talk had a fair amount of detail about writing secure code and more importantly retro-fitting security into existing code. Ironically, it began by quoting a Microsoft research paper as inspiration for a lot of work. Moving on, it shows how different is sshd compared to the standard unix daemon. Rather than move ahead accept-fork model followed very successfully by apache, sshd uses accept-fork-exec to run a master and slave process. The master handles all the higher privilege functions while the latter handles the protocol/crypto/book-keeping functionality.

Tuesday: I slept off till late afternoon. Woke up in time to peek into the gaming miniconf and walk around between Lygon st. & Swanston st.

Still not adjusted to local time yet, but the extra coffee seems to be helping today.

So little time, so little to do.
                -- Oscar Levant

posted at: 12:13 | path: /conferences | permalink | Tags: , ,

Sat, 26 Jan 2008:

LCA '08 beckons and so does a fairly long vacation down under. The blanks left behind have been filled up and I'm sitting in Singapore right now and as usual this trip was also run quite close to the wire in terms of scheduling. But I'm here and on my way.

I'll be reaching MEL by Sunday night (yes, sleepover at the SG transit hotel) and then the week goes into the conference. After which, I plan to hop onto a kangaroo and head off along the coast to Brisbane (that's about as clear my plans are).

If any of you feel like putting up this low-budget traveller somewhere on that coastal route, ping net dot php at gopalv (in reverse).

Gosh that takes me back... or is it forward? That's the trouble with time travel, you never can tell.
                -- Doctor Who, "Androids of Tara

posted at: 20:13 | path: /conferences | permalink | Tags: , ,

Sun, 30 Dec 2007:

I got a nice little present for Christmas.

It had +4 lines, a huge mail explaining why and made me feel happy & stupid at the same time (there's some correlation, I think).

The patch fixes a one-off error in the APC shm allocator (read my mail for a shorter paraphrasing) and has triggered the 3.0.16 release. Now, APC should be stable even when running under cache full/heavy load conditions. And I've been barking up the wrong tree of race conditions for months & months. But the important thing is that this is fixed now.

Merry XMas and a happy New Year! [ citation needed ]

Patience is a minor form of despair, disguised as virtue.
                -- Ambrose Bierce

posted at: 06:27 | path: /php | permalink | Tags: , ,

Mon, 17 Dec 2007:

It's hard to let go of people.

Easier it is to push them away, drive them out of your world and pretend that your paths never ran together. To ignore them, avoid them, separate out their world from yours.

But sometimes the hard way's the easier, the short way goes on longer and life's too short to take the long way around.

A man can be happy with any woman so long as he doesn't love her.
          -- Oscar Wilde

posted at: 21:18 | path: /me | permalink | Tags: , ,

While working towards setting up the Hack Centre in FOSS.in, I had a few good ideas on what to do with my "free" time in there. The conference has come and gone, I haven't even gotten started and it looks like I'll need 36 hour days to finish these.

apt-share: I generally end up copying out my /var/cache/apt/archives into someone else's laptop before updating their OS (*duh*, ubuntu). I was looking for some way to automatically achieve it with a combination of an HTTP proxy and mDNS Zeroconf.

Here's how I *envision* it. The primary challenge is locating a similar machine on the network. That's where Zeroconf kicks in - Avahi service-publish and the OS details in the TXT fields sounds simple enough to implement. Having done that, it should be fairly easy to locate other servers running the same daemon (including their IP, port and OS details).

Next step would be to write a quick HTTP psuedo-cache server. The HTTP interface should provide enough means to read out the apt archive listing to other machines. It could be built on top of BaseHTTPServer module or with twisted. Simply put, all it does is send 302 responses to requests to .deb files (AFAIK, they are all uniquely named), with appropriate ".local" avahi hostnames. And assuming it can't find it in the local LAN, it works exactly like a transparent proxy and goes upstream.

Now, that's P2P.

WifiMapper: Unlike the standard wardriving toolkit, I wanted something to map out and record the signal strengths of a single SSID & associate it with GPS co-ordinates. A rather primitive way to do it would be to change the wireless card to run in monitor mode, iwlist scan your way through the place & map it using a bluetooth/USB gps device (gora had one handy).

But the data was hardly the point of the exercise. The real task was presenting it on top of a real topological map as a set of contour lines, connecting points of similar access. In my opinion, it would be fairly easy to print out an SVG file of the data using nothing more complicated than printf. But the visualization is the really cool part, especially with altitude measurements from the GPS. Makes it much simpler to survey out a conference space like foss.in than just walking around with a laptop, looking at the signal bars.

To put it simply, I've got all the peices and the assembly instructions, but not the time or the patience to go deep hack. If someone else finds all this interesting, I'll be happy to help.

And if not, there's always next year.

Nothing is impossible for the man who doesn't have to do it himself.
                -- A. H. Weiler

posted at: 19:45 | path: /hacks | permalink | Tags: , ,

Sat, 15 Dec 2007:

Futurama, it just won't stay dead.

If you're a Futurama fan, you *have* to watch Bender's Big Score. It's a movie made for the hardcore fan, unlike the Simpsons Movie or other carry-overs from tv into the big screen. Read on for the spoilers.

Since Futurama is going to be running on Comedy Central from now on, they've spent the first four minutes of the movie dissing Fox (oops, I mean the BOX Network). The FOX BOX executives have been fired, beaten up, killed and made into a fine pink powder (TORGO'S Executive Powder), which appears anywhere where any ingredient is required (*bam*).

The movie does a nice take on the information centric world of ours, with the main villains being the spammers/scammers who manage to take over the world with a nice parody of nigerian 419 spam, popup malware and just plain old lottery scams. Of course, from that point onwards the movie has no real plot at all.

Honestly, the movie is just an attempt to make a happier past for Jurassic Bark. Considering how Nibbler blows his cover, Robot Santa turns good, Bender apologizes to all and the Professor never invents anything - the plot is not following canon for any character (except perhaps Zoidberg).

But the role of Fry fills up into someone who'd sacrifice his own happiness to let Leela be happy. Not typical either, but that's a very touching transformation for someone who's been portrayed as immature for all the episodes.

And not even a can of Slurm to be seen.

But I loved it, nonetheless.

Fry: Why would a robot need to drink?
Bender: I don't need to drink, I can quit anytime I want.
          -- Futurama, "Space Pilot 3000"

posted at: 06:51 | path: /movies | permalink | Tags: , ,

A sad day it is for all Terry Pratchett fans (like yours truly).

The master of discworld has gone a bit Bursar - more accurately, he's just been diagnosed with Alzheimer's. As someone who's enjoyed his books immensely, I'm glad he wrote them down as he did, in such profuse quantities.

I just finished reading Wintersmith a few weeks ago - didn't like it enough to write a review about it though. But on reading this news, the only thought was to boycott the series if some greedy publisher tries to push it forward with a ghost writer. Considering what happened to Pern (Dragon's Kin? Gimme a break) or Dune (*bleh* on Chapterhouse Dune) - I'd hate to see that happen to Discworld.

Truly, Alzheimer's is a sad sad way to lose a mind - to fade away, while the body remains.


Personal sn't the same as important.
        -- Terry Pratchett, "Men At Arms"

posted at: 06:20 | path: /books | permalink | Tags: , ,

Tue, 11 Dec 2007:

As fox2mike told me, "We're missing a kick-ass conference" (sitting behind the scenes).

Conferences are unnecessarily cruel on the organizers. After five days of FOSS.in, I was left drained & empty rather than re-energized. Compared to the last year, when I said it was Ardeo Redux, this year my emotions are completely different. Was no fun, this conference - show up early, baby-sit the hack center, feeling sick in the infernal cold snap. Anyway, I suppose that's what responsibility steals from all of us.

Kudos to everyone who's worked tirelessly behind the scenes all these years.

Some people grow with responsibility -- others merely swell.

posted at: 22:43 | path: /conferences | permalink | Tags: , ,

Tue, 04 Dec 2007:

It's been a year.

A year, a really long year.

It is a sobering thought that when Mozart was my age, he had been dead for two years.
                -- Tom Lehrer

posted at: 12:42 | path: /me | permalink | Tags: , ,

Wed, 28 Nov 2007:

Your stress level should be a measure of the work you're doing/done, rather than the work left.

At least, that's the official policy - but it's not quite working out per policy these days.

Of course there's no reason for it, it's just our policy.

posted at: 18:44 | path: /me | permalink | Tags: , ,

Mon, 26 Nov 2007:

Unlike last year, this year I had a pl_n.

It's a little sketchy now, but looks like I've got cash for the tickets, without doing anything too extreme .

Post conference, I've got about a week to get from Melbourne to Brisbane via some sort of ground transport, along the gold coast. Hopefully, I'll be able to find some company to drive and share gas money with. Otherwise, I need to figure out how public transport can take me to point B from point A possibly passing through point Sydney.

But before all that, foss.in beckons.

Life is what happens to you while you're busy making other plans.
                -- John Lennon, "Beautiful Boy"

posted at: 19:13 | path: /conferences | permalink | Tags: , ,

Ruby is now ludicrously fast.

Ludicrious JIT is a Ruby 1.8 JIT which uses the libjit library that was written for dotgnu pnet's JIT core, very similar to what rubydium was trying to do. But libjit's come a long way in the last 3 (!) years, so if you're a Ruboi I suggest you try this out anyway.

It is a testament to the efforts of Rhys, Aleskey and Klaus to keep the library truly modular, reusable and independent of pnet's internals.

The biggest difference between time and space is that you can't reuse time.
                -- Merrick Furst

posted at: 15:14 | path: /dotgnu | permalink | Tags: , , ,

Sat, 17 Nov 2007:

The FOSS.in posters are out. Our local graphics ninja has done an amazing job.

Shreyas, Karunakar, Pradeepto, Tejas, Suparna, Kartik and Naba are on the other posters. The elusive Karunakar had been captured by the able hands of spo0nman - during Freedel '06 (danke for the Creative Commons). And a rockstar-like Pradeepto from one of my FOSS.NITC '06 pics (but for some other pics from that trip ... *cough*).

Anyway, if you want to get on one of these posters - just follow Sankarshan's advice.

The trouble with doing something right the first time is that nobody appreciates how difficult it was.

posted at: 04:53 | path: /conferences | permalink | Tags: , , ,

Sun, 11 Nov 2007:

Deepavali was yesterday.

And I'm sure I haven't lost my hearing (or what I'm hearing are just echoes).

Anyway, Happy Shabdavali !!

'I like the sound of that,' said Mrs Palm.
'I like the echoes,' said Dr Downey.
          -- Terry Pratchett, "Feet of Clay"

posted at: 12:43 | path: /rants | permalink | Tags: ,

Fri, 02 Nov 2007:

It's about a movie and I watched it.

But there's nothing you're going to take away from this movie - whatever you think you took away, you brought it with you when you walked into the theatre.

Just a like a mirror holds no content, the movie too is fairly bereft of a consistent plotline (albeit what's borrowed from Quitters, Inc), but merely reflections of reality appearing and disappearing from a multitude of mirrors.

The camerawork and the general feel of the movie is pretty interesting, (albeit the cabaret song) for a bollywood production, but beyond that the movie has nothing to offer but completely unconnected sequences of plot.

But it is our nature to see patterns where are none, built out of our own biases in perception, in a nonpareil effect. The attempts to make sense out of the confused jumble of thoughts presented results in interesting results indeed.

To make things a lot more blurred than they are right now, let me first touch upon my thoughts - and remember that these are my personal thoughts and not the director's. The movie is about consequences of our actions, as we move through life - the ripples adding up to a tidal wave in the ocean that is life.

The movie starts with the standard caricature of a wannabe rebel yuppie (and smoking is his rebellion) with his socialite wife and their marital issues. Starting from a clear-cut real world scenario, the movie diverges further and further into down-the-rabbit-hole land (even literally) without any perceptible movement out of reality.

Then the social contract is presented and there is no escape. And society steers people with consequences - but here the hidden, mild consequences are merely exaggerated. The basic "How much could one more hurt?" excuse of the habit has been tackled and short-circuited by providing rather harsh penalties indeed. To see someone pass out in a gas chamber full of second-hand smoke is somehow more real than the same happening over a couple of years of constant exposure - somehow the quality of effect alters rather than merely the quantity. And then there's the underline of the paranoia put down by erosion of privacy due to constant surveillance.

To me, the antagonist of the whole sequence is the protagonist himself - not the Baba Bengali or the reality altering effects he seems to have. The struggle between good and evil has been very conveniently left out of the entire script - undergoing a metamorphosis into a mere squabble between your needs and wants. To face himself after realizing the consequences his acts have, learning the lessons from the rough teacher experience is, to be offered the choice again in the end - real life is not so kind (and this was being cruel to be kind).

The basic mode of coercion of an individual to be a building block of a society with rewards and consequences comes out of the seemingly random treatment of the character(s) and the plotline...

The jigsaw puzzle that the movie is perhaps unappealing to a critic who expects a coherent movie. But what you enjoy out of the movie is synthesising the story out of the jigsaw peices you are handed, not analyzing the plot-holes in depth - and that takes effort. The Hindustan times review of the movie was acidly critical and downright mean (and the director failed to show any maturity with a completely Ad Hominem response).

Perhaps you'll find another reviewer talk about this movie as one of individuality or about smoking as peer-induced (Abbas), yet another talking about the religious slants on the concept of the soul & its journey independent from the body's journey (to quote "Athma hai tho shareer ishwar hai, athma nahin hai tho nashwar hai"). And then there'd be the ones who carry an open mind into the movie and carry it rather empty out of it - walk in with your preconceptions & prejudices and you'll find that there's a movie playing.

In short, if you're hoping to be entertained, don't show up till the Bipasha Basu dance sequence :)

Beauty is eternity gazing at itself in a mirror.
But you are eternity and you are the mirror.
                  -- Kahlil Gibran

posted at: 10:24 | path: /movies | permalink | Tags: ,

Fri, 19 Oct 2007:

APC 3.0.15 has been released - read the release announcement. Not too many changes since 3.0.14, but there's a reason it took this long to make so few changes.

To begin with, I've just been lazy. Just kidding! This release was actually delayed to make sure this could be the very last PHP4 release of APC. And with the amount of major changes coming in, the next release is definitely going to be 3.1.0 rather than a 3.0.16 release. Now I can start working on making some fairly big changes.

Bye Bye PHP4, it was a nice ride while it lasted.

While most peoples' opinions change, the conviction of their correctness never does.

posted at: 03:11 | path: /php | permalink | Tags: , ,

Thu, 18 Oct 2007:

It's too late to write anything about it now. Enough good things have been said of how it has been run and the whole vibe of the event, but I'll add my two cents anyway (and my mug made it to the newspaper article).

WiFi The WiFi coverage was awesome. Except a lot of people who had borrowed laptops for the event had no clue how to get it working, not to mention requiring kernel upgrades to get it working. Just for the sake of recording it, this is how you set up a franken-net NAT quickly.

iptables --flush
iptables --table nat --flush
iptables --table nat --append POSTROUTING --out-interface eth1 -j MASQUERADE
iptables --append FORWARD --in-interface eth0 -j ACCEPT

echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward

ifconfig eth0

Then everyone else joining up on that hub can pick a 10.0.0.xx IP and set my machine as a gateway. But after nearly everyone got their WiFi working, the environment got a bit more interesting.

There were a bunch of decent hacks - some of them functional, but not technically brilliant and vice versa. For example, the flickr offliner was an interesting hack - but without offline writes and conflict resolution, it wasn't really cool enough. Or the first prize winning Map canvas hack, which never worked with zoom or pans - which was sad because being able to convert from YCoordPoint to YGeoPoint would have made a lot of cool hacks possible (and was just an overlay of canvas painter on a map div). But I wasn't a judge, merely tech-support.

And when the hacking was all done, we partied out the night, to the rhythms of TAAQ.

When a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully.
                -- Samuel Johnson

posted at: 22:52 | path: /yblr | permalink | Tags: , ,

Sun, 14 Oct 2007:

Yes, the whole trip was full of stupid memes, mostly thanks to Gora. The trip to Manali took a good 20 hours and we spent most of the evening & night talking shop, making jokes and discussing "stuff". But there were times when the conversation was more interesting.

gora: you like someone because of, you *love* someone in spite of.

Somehow in those few words, the distinction between those two words is glaring and obvious (oh, yeah "love is blind"). After the flow of reason stopped, I went to sleep in in the aisle of the bus, as usual. Except this time when the bus braked hard downhill, I slid down nearly half the length of the aisle. Anyway, I had an otherwise uneventful journey to reach our hotel in Naggar, about 20-odd kms away from Manali proper.

The hills were really beautful this time of the year. The sunsets were brilliant and the weather stayed pleasant all through out. It was perhaps a tad too cold, with temperatures slightly below 10 C in the nights.

We went up to Rohtang pass to see some snow and the whole hillside had a mild smattering of snow. It was dark and gloomy, but that didn't dampen anybody's spirits during the snowball fight. But having run out of breath and energy, everyone pretty much settled down to quiet contemplation soon. After a protracted lunch, we headed downhill back into valleys.

And then the jokes started in earnest.

B: Uh... my nose hurts
G: Oh, noes! 


*dyslexia discussion*
*me thinks "On the internet, nobody knows you're a goD"
G: Oh snoe, I'm dyslexic!

But there's more - we invented the concept of CVS Suppositories. And by we, I mean OldMonk, Dr. Gora (*heh*) and plain old me. We'd discovered the ideal version control system for bad code - CVS.

Me: *laughing* ... and if you need your own copy, cvs up yours!

After several jokes about upstreams, branching, merging and of course, the possibility of a flush command, we all retired into bed. And that was the end of that.

To be rather brief about the rest of the trip - we went, we saw, we had fun.

The writing of history is largely a process of diversion.
          -- Chapterhouse Dune

posted at: 06:14 | path: /travels | permalink | Tags: , ,

I had gone to Freed.in to talk on something slightly unusual. My talk was on "!0 Things You didn't know about Python" - and as usual, I overdid the technical bits of the talk. But at least someone will remember memoization can improve your factorial code. So, here are the slides and let's leave it at that.

The whole conference schedule was packed with familiars from #linux-india - but the absence of Lap_64 and Lawgon was felt rather heavily. Nearly all of ILUG-Bom was present at the conference, even the Panvelkar.

There were some interesting things which came up during the conference. One of them was the SME Panel discussion where the adoption of F/OSS in the small & medium enterprise was discussed. The hurdles to most of these arise from the lack of support and surprisingly, vice versa.

As one of the volunteers commented, the paid-for support structures in place are leaning precariously on the seniors in the community. Where there is an OldMonk or BigBeard, there isn't really a gradation into a lower echelon of cheaper support for simpler tasks which a newcomer can take up without incurring much risk. The vicious cycle of establishing a reputation versus requiring one is present in the system.

Repurposing the LUG to handle the responsibility as a contact point was discussed and IMHO was a waste of time. A community organization with democractically inclined operation cannot do that job effectively. Involving it in monetary transactions brings in a large number of problems which are best solved by incorporation elsewhere.

Secondly, there was f3ew's talk on Building Online Communities, which was mind-mapped unconference of sorts. I'm still working on my Breaking & Entering Communities talk, but this sort of served as a stage to throw out some of my material. I personally think that we need help in contributing to communities rather than going about building them.

And as the conference wound down, we all packed our bags and headed out to Manali.

I don't believe in sweeping social change being manifested by one person, unless he has an atomic weapon.
                  -- Howard Chaykin

posted at: 00:42 | path: /conferences | permalink | Tags: ,

Mon, 24 Sep 2007:

PHP programmers don't really understand PHP.

They know how to use PHP - but they hardly know how it works, mainly because it Just Works most of the time. But such wilful ignorance (otherwise known as abstraction) often runs them aground on some issues when their code meets the stupidity that is APC. Bear with me while I explain how something very simple about PHP - how includes work.

Every single include that you do in PHP is evaluated at runtime. This is necessary so that you could technically write an include inside an if condition or a while loop and have it behave as you would expect. But executing PHP in Zend is actually a two step process - compile *and* execute, of which APC gets to handle only the first.

Compilation: Compiling a php file gives a single opcode stream, a list of functions & yet another list of classes. The includes in that file are only processed when you actually execute the code compiled. To simplify things a bit, take a look at how the following code would be executed.


include_once "a.php";

The PHP compiler does generate an instruction to include file "a.php", but since the engine never executed it, no error is thrown for the absence of a.php. Having understood how includes work, classes & OOP face a unique problem during compilation.

include_once "parent.php";

class Child extends Parent

Even though the class Child is created at compile time, its parent class is not available in the class table until the include instruction is actually executed & the parent.php compiled up. So, php generates a runtime class declaration which is an actual pair of opcodes.

ZEND_FETCH_CLASS              :1, 'Parent'
ZEND_DECLARE_INHERITED_CLASS  null, '<mangled>', 'child

But what if the class parent was already in the class table when the file was being compiled? Like the following index.php

include_once "parent.php";
include_once "child.php";

$a = new Child();

Since obviously the parent class is already compiled & ready, Zend does something intelligent by removing the two instructions and replacing them by NOPs. That makes for fewer opcodes and therefore faster execution.

Here's the kicker of the problem. Which of these versions should APC cache? Obviously, the dynamically inherited version is valid for both cases - but APC caches whatever it encounters initially. The static version is obviously incompatible in a dynamic scenario.

So whenever APC detects that it has cached a static version, but this case actually requires a dynamic version, it decides to not cache that file *at* all from that point onwards. That's what the APC autofiltering does.

Now, you ask - how could it appear in perfectly normal code?

Assume child1 and child2 inherits from parent class. And here is how the first hit on index.php looks like from an inclusion perspective. Now, it is obvious that the child2 in this case is actually compiled with the faster static inheritance (marked in orange) while child1 suffers the performance hit of not having Parent available till execution time.

Then we have a profile.php which only requires the child2 class. But while executing this file, APC fetches the copy of child2.php which was in cache - which is the statically inherited one.

As you could've guessed, the cached version is not usuable for this case - and APC drops it out of cache. And for all requests henceforth, even for the index.php case, APC actually ignores the cached version and insists on compiling the file with Zend. If you enable apc.report_autofilter, this information will be printed out into the server error log.

Part of the culprit here is the conditional inclusion using include_once. With mere includes, you get an error whenever parent.php is included multiple times - but that can be annoying too. Where include_once/require_once can be debugged with Inclued, userspace hacks like the rinclude_once or !class_exists() checks make it really hard for me to figure out what's going wrong.

So, if you write One File per Class PHP and use such methods of inclusion, be prepared to sacrifice a certain amount of performance by doing so.

Doubt is not a pleasant condition, but certainty is absurd.
              -- Voltaire

posted at: 13:55 | path: /php | permalink | Tags: , ,

Sun, 16 Sep 2007:

I'm officially starting to refer to Microsoft Office Open XML by its new acronym - MooXML. Its progress through the ISO process looks like a salmon rush - but it looks like a fish, moves like a slippery slippery fish and steers like a cow (with apologies to the late Douglas Adams).

Superb Standard: And then Miguel goes and calls it a "Superb Standard". Basing most of his justification on the fact that a large quantity of existing OO.org code can be re-used to implement MooXML - take a guess why that is so. And just because the current code can handle "autospace like Word95" specifications (IIRC,the 'sz' node) is no reason to call MooXML a good standard. And multiple implementations only add to the mess, with everyone including the creator interpreting the standard as they please. Implementing it would be yet another game of 'chase the lamespec' (well, grep -i lamespec mono-svn/).

Most people who have complained about the specification has been struck down with the "let the implementors speak". But somehow it seems odd that such a specification which has been sped through ISO got a 'YES' vote from *Cuba* - a country where Microsoft (and any other US company) is prohibited from exporting any software to. A good, proper standard doesn't really need a ballot stuffing to get approved.

I mean, I've previously commented about Miguel losing relevance in the world of Free software. But this time, this is not about his opinion or Novell's patent protections - in fact, not even about the format, but about the standard for an editable document, which renders the same everywhere.

So, don't say MooXML, say NoOoxml.

Mediocrity finds safety in standardization.
                -- Frederick Crane

posted at: 01:42 | path: /rants | permalink | Tags: ,

Mon, 10 Sep 2007:

For once, I'm not the one being cryptic (the revenge of the apostrophes?).

Yahoo! Bangalore is throwing a hack fest on the 5th of October and all through the night. If you've ever done something with pipes, YUI, y! mail or flickr apis, this is the place to be for you. And it doesn't even matter if you like writing RoR based backends or greasemonkey funk - what matters is how it works and what it solves.

The hack session is by invite only and we've got a submission queue for you to put in your geek credentials, so make sure to put all your cool hacks in the list. And take a good look at what happened at the previous hack days for inspiration.

It is going to be a "Bring your Own Laptop" hackfest of the kind you've never seen before. To keep the tempo, we'll be trying to put up a strong team of hackers from Yahoo! to help you hack (and more importantly, appreciate the uber-coolness of your hack-in-construction).

Watch hackday.org for more announcements. Hope to see you there ... so start planning!

To whom the mornings are like nights,
What must the midnights be!
                -- Emily Dickinson

posted at: 06:32 | path: /yblr | permalink | Tags: , ,

Fri, 07 Sep 2007:

After procrastinating for nearly two weeks with the code nearly done, I've managed to find the energy (and some caramel coffee) required to fix it up for the public to use - and here it is. In the process, I also threw out all the ZendEngine2 hacks and started to use zend_user_opcode_set_handler, which should let people use this with the faster CGOTO vm core, though I would advise against using that just yet.

The new & improved inclued can dump out class inheritance dependencies (though not the interfaces, as of now). This gives a slightly bigger picture view of what files depend on what other files and provide a tree of the classes clustered into their own files. For example, this is the graph pulled out from the relatively minimal PEAR::HTML_QuickForm2 library.

The usage is as before, the gengraph.php script now has a -t option which will accept either "classes" or "includes". At the very least, it should help people documenting OOP php code. Next up are interface implementations, the data is already in the dumped files, but not output in any human readable format.

It is a very sad thing that nowadays there is so little useless information.
        -- Oscar Wilde

posted at: 06:07 | path: /php | permalink | Tags: , ,

Thu, 06 Sep 2007:

There are moments in my life when I'm at a loss for words.

And something like that happens.

Take my advice, I don't use it anyway.

posted at: 20:52 | path: /yblr | permalink | Tags: , , ,

Mon, 03 Sep 2007:

Now that I think, it was all like a dream - not a particularly good one, but the one you wake up & forget all about, on a monday morning. Roaming around San Francisco, Monterey and Palo Alto in Premshree's Pontiac, with Sachin snoring in the back seat, despite Chris Cornell's shrieks from the CD player. It takes an occasional look back at my photos to remind myself that no dream it was. I'll let the photographs do the talking.

I See Red Golden Gate Bridge Red Muscle
Golden .... Parking Lot with a View I Am Not Alone
Alone. Do you feel Zen ? SF - Green light
Man & Machine San Francisco Cityhouse Gray Pelicans

And in a fleeting glimpse, I saw the clouds blow a heart to the city.

I ♥

And then came the climb down and then I saw the city. And the blue, blue sea.

Fort Pt.

Sat down on the green green grass and let the world pass me by. And pass by it did.

World. pass me by ... Three Stripes ! Girl on a Trike
Beachcombing ... Green, Gold and Blue

Walked along the beach, spent perhaps the most peaceful hour of my visit watching the aerial acrobatics of the kite surfers - jumping twenty feet up and sliding along the water, masters of the wind and water.

Wind beneath my Wings ! Wicked Slide !

And then next week, spent hours staring at the Jellyfish tanks at Monterey.

Jellyfish ! Mushroom Cloud Lion's Mane Jellyfish

And that's all there is.

The trouble with telling a good story is that it invariably reminds the listener of a dull one.
                -- Sid Caesar

posted at: 07:14 | path: /travels | permalink | Tags: , ,

Sun, 02 Sep 2007:

It's finally happening. My friends are slowly drifting away in different directions. There aren't going to be any more sessions of midnight pool, no more late night balcony conversations, no more weekend hack sessions - 'change' happens, but I can live with that. But to actually lose such friends in the milling crowd is a sad thought.

These people seem to have fallen off the planet. They've stopped blogging, have unsubscribed off many of the mailing lists and even their flickr streams are devoid of actual life, tending towards art & technique.

And reluctantly, I turned to facebook to 'bookmark' them. Despite using an online social networking site, I'm trying to keep in touch with people whom I used to meet every day.

Ironic, but completely Web 2.0.

Friendship needs a certain parallelism of life, a community of thought, a rivalry of aim.
                -- Henry Brook Adams

posted at: 18:20 | path: /misc | permalink | Tags: , ,

Sun, 26 Aug 2007:

Ratatouille was a funny movie, no doubt. But what set the tone to the hilarity that was to ensue was the Pixar short 'Lifted'. If you've watched Taken & E.T, take five minutes off and watch it on youtube.

Exquisitely animated, with amazingly clear graphics. the movie is a masterpeice of picture perfect pixels. But the story sort of drags in the middle, after a rather interesting few minutes in the beginning. The whole premise of the movie is somehow flawed at some level, but still it is peppered with a few one liners to remember (like the "what dad doesn't know could fill a few books; And they did, which is why I read them").

Still, the voice of Anton Ego (the food critic) deserves special mention. Somehow it redeems the entire movie by giving it a villain where there really isn't one - Skinner (the short chef) just doesn't cut it. If he had to have a nickname it would be the 'Grim Eater'. Ego's final article about being a critic is a masterpeice in itself.

But this movie doesn't even compare to Monster's Inc, Finding Nemo or Toy Story and has a more forgettable Disney quality rather than pure Pixar magic. Still, there are moments to remember - though maybe not a full movie.

Being a food critic is all about dishing it out - literally

posted at: 22:54 | path: /movies | permalink | Tags: , ,

Fri, 24 Aug 2007:

Nearly every year, I'm usually with my entire joint family for nearly the entire half of the ten day Onam celebrations. But this year, since I'm in mourning, I hadn't thought much about going home and by the time I had gotten back from the US, it was too late to do anything. Thankfully, the office has a significant enough malayalee population to actually make a good friday out of it.

Kudos to all the folks who did all the fantastic floral work ... makes me feel right at home here :)

The worst feeling in the world is the homesickness that comes over a man occasionally when he is at home.
        -- Edgar Watson

posted at: 14:52 | path: /yblr | permalink | Tags: , ,

Mon, 20 Aug 2007:

I recently ran into a fairly generic XSS vector which didn't seem to be on the XSS cheatsheet. It seems to be a quirk of the way the DOM parser handles <script> tags and well, a broken JSON encoder. The vulnerability (or at least gotcha) exists when properly quoted strings are printed out inside a <script> node. A simplified exaple looks somewhat like this.

var a = "</script> <script> alert('XSS !'); </script> <script>";

For some strange reason, Firefox picks up the script closing tag in the quoted string and then proceeds to process the remaining script tags as code. Try it for yourself.

I discovered this problem inside a large/deep block of JSON. Thankfully, the php JSON encoder escapes the forward slash, but the JSON spec doesn't require that explicitly AFAIK. For instance, the standard python-json module which came with Ubuntu generates bad code to embed in a script segment (so, start using "import simplejson as json").

Quoting strings anywhere is complicated enough and if you ever need to dump stuff into a javascript segment, use a json encoder - a good one & not reinvent your own.

If I do not want others to quote me, I do not speak.
          -- Phil Wayne

posted at: 01:12 | path: /insecurity | permalink | Tags: , ,

Thu, 16 Aug 2007:

The "internet" is a series of tubes. So I decided to play plumber and hook up a few pipes the wrong way. What has been generally stopping me from writing too many web mashups has been the simple hurdle of making cross-domain data requests. While poking around pipes, I discovered that I could do cross-domain sourcing of data after massaging it into shape in pipes.yahoo.com.

After that blinding flash of the obvious, I picked on the latest web nitpick I've been having. Since I'm already hooked onto "The Daily Show", I've been watching (or trying to) it online from the Comedy Central website. But it is a very slow application, which shows a very small video surrounded by a lot of blank space - not to mention navigation in flash. A bit of poking around in HTTP headers showed a very simple backend API as well as an rss feed with the daily episodes. Having done a simple implementation as a shell script, I started on a Y! pipes version of it. The task was fairly intutive, though the UI takes some getting used to. Eventually, I got a javascript feed that I could just pull from any webpage, without requiring XMLHttpRequest or running into cross-domain restrictions.

You can poke around my pipe which has been used to create J002ube (say YooToobe ... so that j00z r l33t) to play the Daily Show videos. The player has zero lines of server side code and uses the Y! hosted pipes or client side code to accomplish everything.

More stuff coming up these pipes ...

Whoever pays the piper calls the tune.

posted at: 16:00 | path: /hacks | permalink | Tags: , , ,

The hustle and bustle of real life is a killer. But there's no reason for it to be a silent killer, so here's one of those mental kicking & screaming thoughts which was dragged through my head as I was wasting time in Hongkong Airport. My flight called for me before I finished this, but the time has come to acheive some closure.

To: y-blr <...>
Subject: Cubicle for Rent (rates negotiable)

One premium corner cubicle in M G Road available for rent. The cube is 
surrounded by conference rooms, fully furnished with a laptop dock,
a comfortable chair. Extensive table space, entire sixteen foot carpet
area with easy access to sofas, coffee and the pool table. 24x7 
internet access enabled, fully packed bookshelf and with neighbours used
to loud music after 7 PM.

Available for occupancy for a month and at negotiable rates.

Since it is too late to actually send that mail, I guess this is its home now. But sing with me - My cubicle ... it doesn't have a view.

If money can't buy happiness, I guess I'll just have to rent it.

posted at: 08:20 | path: /fun | permalink | Tags: , ,

Tue, 14 Aug 2007:

Ever notice how hard it is to get something when you exactly know what you want ? Especially when you had it once and lost it ? Ever notice how hard it is to settle for whatever's available and you keep looking & looking, getting more depressed about the lack of choices ?

I'm not talking about true love, as much as these things might be be applicabl,e but just about an mp3 player. My needs were simple. It had to have at least 4 gigs of space, an enqueue feature, decent battery life, a standard USB cable and for less than 150 USD. My last mp3 player had all that and looked cool while doing it - a black Creative Zen Neeon2. It survived several long walks on rainy days, had no front controllers which meant I could operate while it was still in my pocket. But one weekend I left it at my office desk and come monday it was missing.

So, four months later, I started looking for a replacement - but they don't sell Zen Neeon2 in the US anymore. And every other player I found had a proprietary adapter - the iPod, the new Sansa e2xx, even the new Creative vision stuff. Nearly after a week of crawling through specs & window shopping, I finally discovered that the Creative Zen V plus had nearly everything I wanted (looks cheaper and *is* cheaper than an equivalent iRiver).

Bought one, plugged into my laptop and zilch happened. As it turns out, it uses a crappy MSFT protocol called MTP - so much for open interfaces. But that's where Amarok came to the rescue ! Amarok incorporates libmtp5, which lets me just drag & drop between Amarok collections and the mp3 player. Except, I had to apply the following patch to make it plug'n'play (launchpad bug #132392).

But the effort is worth it. Even Album art transfer from Amarok is completely taken care of. Mtp sort of sucks because it does the file transfer in user mode (freezing amarok) rather than kernel buffering that usb-storage gives. But on the flip side is that that unlike the usb-storage players, the usb charging will work even if the player is playing (as long as it is not mounted on Amarok). I'm sort of building a hack to submit my play counts to my last.fm account from the statistics returned by the mtp-tracks commandline client, but it doesn't give last played time to sort of fake an offline update for the correct time - the spam filter in last.fm blocks a bulk update of tracks at the same time.

Now that it's working, time to play with my next new peice of hardware ...

The real trouble with reality is that there's no background music.

posted at: 14:34 | path: /hardware | permalink | Tags: , ,

Sat, 11 Aug 2007:

Yak Shaving: So you start out with that simple problem. But half-way through fixing it, it explodes into this whole exercise in pointless dependencies. It is a rather recent wordification (never heard of that word ? it's a perfectly cromulent word). But considering the fate of the "pre-shaved yaks" guy, who ended up saying "It's a band.", I'd say it is not quite popular enough ... yet.

Now. before I start onto the real topic - let me first say that the next release of APC will be the last release compatible with PHP 4.x. Now, what is wrong with just letting the #ifdefs stay ? That's where this snippet of code comes into play.

  apc_store("a", array(new stdclass()));

It doesn't work. Now, the problem is very simple - the original patch by Marcus only checks for objects in a very shallow way. It will detect & serialize objects which are passed to apc_store - but the check does not extend deeper into the recursive copy functions.

Symmetry: But the zval* copy functions were written to be beautifully symmetric. A copy into cache is nearly the same as copying out of it. And when I say "nearly", I actually mean that until the *_copy_for_execution() optimisations were thrown in, they were actually symmetric - in & out. But objects don't play nicely with that - because they are much more than just data.

In & Out: Objects require assymmetric caching. Storing into cache is a serialize operation, while retrieving from storage is a deserialize. This ensures that they end up with the right kind of pointers, class object initialization and that the resources they hold in their opaque boxes are properly handled. The objects have to implement their appropriate magic persistance methods.

And thus begins the Yak Shaving. I need to rewrite most of the cache copy-in and copy-out functions to handle the basic assymetry. But consider this, most of the code in there has been limited for months because of the fact that I cannot optimize on PHP data structures without breaking the symmetry.

A couple of years ago, I sat through a full hour talk by Rusty Russell about talloc(). Built on top of the trusty old malloc() calls, it simplifies memory management a lot for Samba4. So bear with me as I take a brain dump of my idea - for my very intelligent reader to poke holes in (gopalv shift+2 php net).

APC's allocation strategy is a little brain dead. To allocate 4 bytes of data, it actually requires 24 bytes of space. But much more than the space wastage, I'm more concerned about the number of lock() calls required to cache a single php file - a hello world program takes about 22 lock operations (11 locks, 11 unlocks). Yes, that's actually 22 syscalls just to cache echo "hello world";.

I've previously tried to fix it with partitioned locks. The problem with that was actually cleaning up the locks, because the extension code would have to have special cases for every SAPI - because of some bugs in PHP 5.x. So, the "if you don't succeed, destroy all evidence" principle made me throw out that idea. But the cache-copy, zend-copy separation should help me revive another approach to this.

Pools: So, now that I'm officially b0rking up APC, I could as well slap on a new pool allocator, right on top of sma_allocate - ala, talloc(). The allocation speed would skyrocket, because the in-pool allocs are sequential and do not have any fragmentation issues due to blocks in the middle being free'd. As much as allocates are important, the real advantage of this would be that I could basically speed up cache expunges by a magnitude or more. The 22 syscall cache expunge for hello world would be reduced to a potential pair of syscalls - because it would be a single free of the entire pool space.

Right now the pool is actually built up to be of the following structure.

struct apc_pool_t {
	int capacity;
	int avail;
	void *head;
	apc_pool_t *overflow;
	unsigned char data[0];

I've yet to run this through an x86_64 build, but an even multiple of int/void* should align data area right into a wordsize. And I think nearly every pool should be around 4k (i.e 4096 - sizeof(apc_pool_t)) for opcode cache and 1k for data cache. I might make the latter a runtime tuneable, just to pad the APC manual up into an entire book (just in case someone asks me to write one .. *heh*).

None of this is included in APC 3.0.15, which will exit out of the gates as soon I'm sure I'm happy with its stability. The new code will probably be an APC 3.1 release, marking the end of php4 compat & opening up the door for php6 compat.

A two line bug report which exploded into nearly two thousand lines of C code - that's just classic yak shaving.

10 If it ain't broke, break it;
20 Fix it.
30 Goto 10

posted at: 09:27 | path: /php | permalink | Tags: , ,

In memoriam - Monday, August 11th 2003.

That makes it four years now, but I remember like it was yesterday.

If we could sell our experiences for what they cost us, we would all be millionaires.
                -- Abigail Van Buren

posted at: 08:30 | path: /me | permalink | Tags: ,

Fri, 10 Aug 2007:

There are some movies so cool that they can exist only as trailers. While trawling the depths of the internets, I ran into a bunch of them - well collected & catalogued. Here are a few interesting clips from collegehumor.com.

First up is an epic tale of a young man's struggle against the burdens of his conscience. Of someone who's entire life is about not getting blown up. Experience the safety of being a minesweeper, with all the danger of playing it on your own computer in - The Minesweeper Movie.

If you are more into crime thrillers, rather than the war kind, you might enjoy Facebook / Off. An excellent combination of the classic Face/Off with online identity theft. As you might have guessed, this won't end well.

And for those of us who grew up playing "Where in the World is Carmen Santiago?" (smuggled into school on multiple floppies), here's a Quentin Tarantino-esque adaptation, aptly titled Where the F*ck is Carmen Santiago. After all, you'd have a tough time finding out even with the help of google maps.

Well, this statistical anomaly is from the Daily Show, as presented by the special correspondent Aasif Mandvi. Somewhat in the same lines as most environmental conservation ads - hopefully the comedy central web-player works for you. It didn't for me - but I found a workable workaround (so to speak).

Apparently, youtube isn't the only place to waste time on the intarweb.

The human race has one really effective weapon, and that is laughter.
                -- Mark Twain

posted at: 12:48 | path: /movies | permalink | Tags: , ,

Driving a car in the US seems to be a complete peice of cake. There's none of the hassles of a stick-shift/clutch system with an automatic shift, the roads are wide and nearly all parking lots are easy to pull a car out of. The new-fangled GPS thingys ensure that I'm never lost and the most reassuring phrase you'll ever hear is "recalculating ...", when you realize you've missed your exit.

So far, I've driven once to San Francisco and a fair bit of the freeway to Mt Diablo. In both of the trips, the hardest thing I had to do was drive up there in the seventy MPH speeds. Driving in India has given me a sort of "expect the unexpected" sixth sense which makes me slow down as I expect merging traffic to cut across my path. Just pure speed does not bother me that much, as I have driven at higher speeds on Indian roads. But that was mostly a dare, combined with suicidal insanity, on home turf - but maybe I'm not that young anymore ?

And then in downtown, I suddenly started driving on the left side of the road. Half a second and screams from premshree & bluesmoon later, I swerved back onto the right. Still have to unlearn a bit of muscle memory that serves as my driving. To actively apply my attention on driving just takes too much effort. The task of getting from point A to point B, possibly passing through point C, can be accomplished with a combination of VTA, caltrain and BArt. And that's the plan, going ahead.

In short - been there, drove that and nearly got a ticket.

Left is right and right is wrong.
          -- Drive-in India

posted at: 11:57 | path: /misc | permalink | Tags: , ,

Thu, 09 Aug 2007:

No, this is totally different from sticking a crayon up your nose, but the results seem frighteningly similar. If I seem a little less than well disposed to people who flaunt an iPhone, want an iPhone or even worse "need" an iPhone, it all begins from the first few minutes I had an iPhone in my hand.

Sometime during OSCON, I dropped onto a sofa in the lounge. And suddenly, the cushion started to vibrate. I put my hand into between the cushions and pulled out a bright shiny iPhone. Despite multiple cries of "that's mine" from various unlikely sources, I was just holding it up for whoever to pick it up.

And then this guy in a suit walks up and says "I'll take that and thanks for not stealing it". I mean, with a self-satisfied tone, assuming I would want it ? And I wasn't the only one indignant about that, in fact, one of the women (rrichards's S.O ?) around actually took offense and pretty much demanded an apology.

And then the guy turns on his heels & walks away.

I mean, I've found phones in all kinds of strange places before. I've returned phones (yes, in plural) which have cost over a thousand dollars (in return for a coffee). I don't ask for any reward, but at least, I don't need to be insulted while I do someone a good deed.

I think one of the php dudes started referring to that particular person from that point onwards as "the guy who had an iPhone up his butt". And the name stuck.

As much as I adore the iPhone's multi-touch zoom UI, the early adopters who have slammed down six hundred big ones for the phone seem to have a large proportion of shallow hipsters, which automatically triggers a smug alert for me.

So, if you have an iPhone & consider this unfair portrayal, blame the guy who "introduced" me to this particular stereotype.

Only the shallow know themselves;
The rest of us have depths to plumb.
            -- Oscar Wilde

posted at: 12:42 | path: /rants | permalink | Tags: ,

I'm re-evaluating my current hairstyle. No, this is not out of any newly acquired fashion sense, but more due to image-mismatch reasons. I've never had a hair style - I've just had hair PERIOD. But leaving my hair as it is has of late become a slight problem. It apparently is a very misleading signpost.

So, a snippet from a portland street car experience. As I sat in there day dreaming, a young punk girl came and sat next to me. With all dyed hair and multiple ear peircings, she looks about 17 or 16. Hardly the type to skip a window seat to sit next to me. And then the conversation follows.

punk chic: what's up ? (*chin up*)
me: uh... just ... *mumble* ... stuff

punk chic: got any cigarettes ? can I lift a smoke ?
me: sorry, I don't have any. I don't smoke.

she: Oh, you don't smoke *tobacco* ?

And this is not the first time this has happened. Maybe the time is ripe for yet another transformation ?

May all your weeds be wild flowers.

posted at: 12:03 | path: /me | permalink | Tags: , ,

Wed, 08 Aug 2007:

Once I had recovered from my jet-lag with the aid of Red Bull, double-shot espressos and a bit of napping, Portland nights were starting to become fun. Just like every other conference I've been to, the fun to be had is after 6 PM in some pub nearby. So I tagged along with the knowledgable crowd, who always know a nice place right around the corner, anywhere in the civilized world.

And then, there was the Oregon beer festival. Despite the fact that I don't drink, I headed out there to lose a bit of my inhibitions, purely by kindred spirit rather than the liquid version. The festival on the waterfront park was an amazing place to just sit down on the grass, chill out with a mug of free "designated driver" root beer.

After the dust settled at OSCON, I had nearly three full days to explore Portland. What really impressed me about the city was the very efficent public transport system. The light rail was my primary mode of transportation and it got me everywhere I wanted to. After wandering around the saturday market, I found myself gravitating towards the waterfront, to just spend the late evening sunshine in.

I spend Sunday exploring the north west and south west of the city, from the Pearl District to Washington park, on foot. Eventually, I ended up at the Rose gardens in the late evening. The roses were in nearly full bloom and spring was literally in the air (unfortunately, so was the pollen).

And then while walking around the Washington Park station, I saw something interesting carved into the wall. Maybe a yellow Post-It could've done the job ?

Portland ... hmm, if it were a house, it would have a doormat with "Welcome" on it.

There is nothing stranger in a strange land than the stranger who comes to visit.

posted at: 06:47 | path: /travels | permalink | Tags: , ,

Finally, I got bored enough to update my inclued extension (as promised at OSCON). The extension now comes with a nearly completely non-intrusive data dumping mode. The new inclued.dumpdir can be used to dump the inclued data onto a temporary file without ever modifying any of your php scripts. Also included is some php code to transform the dump data into graphviz formatted .dot files.

Pick up your free & complementary copy of the source code on your way out. And stay clued-in about your includes.

This quote intentionally not included.

posted at: 04:27 | path: /php | permalink | Tags: , ,

Tue, 07 Aug 2007:

FOSS.in is back and it is bigger than ever, stretching for five full days from tuesday to saturday (04-Dec to 08). But as I said last year, there are sort-of miniconfs this year. Instead of doing ordinary mini-conf style, this one is going to be project specific rather than topic-specific. This works out nicely for the big projects to get a little momentum going, but might be slightly bad for the small projects or platform coders (like me). I guess that's a chicken & egg problem which a conference cannot solve by itself.

there's a break in the wall

Since my previous post was about freed.in, I think there's something that needs to be said. While I was at OSCON, trying to convince people to submit talks to foss.in, I was asked an interesting question.

Not enough speakers ?: Sure, India does have enough qualified speakers to run a 5 day conference with the people we have. But foss.in, IMHO, has the resources to actually bring some really interesting people to India. Somehow the assumption that bringing in foreign speakers is an admission of weakness has to be debunked. I consider the fact that the conference can afford that as a strength. Actually being able to be in ad-hoc discussions with these folks itself brings about interesting results. No, I'm not talking about b-school style "networking". I'm talking about ideas getting thrown around, bounced off and suggestions being made by a group who probably would not have met if not for the conference. After all, the aim is to encourage communication, collaboration and contribution from India - not to merely showcase last year's work.

The new avatar of the conference itself has some modifications. The conference dinner is one of the things which has been borrowed from some of the more successful conferences. You can read the full announcement for more details.

You're not an alcoholic unless you go to AA meetings.

posted at: 03:53 | path: /conferences | permalink | Tags: ,

Mon, 06 Aug 2007:

The old Freedel you knew is no more. The new & improved incarnation of the conference is to be named Freed.in and to be held this September in Delhi. It is a community conference and run by the ILUG-D community (*ahem*, the usual culprits). The Call for papers is open, so if you happen to be India & working for a cool F/OSS project, put in a paper now.

I've been to the conference last couple of years and it has been an awesome experience.

No group of professionals meets except to conspire against the public at large.
                -- Mark Twain

posted at: 23:23 | path: /conferences | permalink | Tags: ,

Fri, 03 Aug 2007:

Some of you might have seen me wearing this tshirt. But it was one of those things which me and mojo came up with. After a couple of nearly non-starts, we finally got a half-decent t-shirt design for the Y! Bangalore frontend engineering conference. Nearly completely borrowed the style and attitude of xkcd, threw in a bit of self-deprecating humour (It's so uncool, that even I don't do it).

Somehow the more catty punchline, "When you're *this* pretty, you don't have to do anything" (as said by the machine to the ex-(*heh*)-asperated girlfriend), wouldn't fit into the speech bubble. But this one still is pretty kick-ass.

Oh what a tangled web we weave, when first we practice to deceive.
                -- Shakespeare

posted at: 06:27 | path: /fun | permalink | Tags: , , ,

There's no way to satisfy a Simpsons fan. I mean, for a tv show which has technically reached the age of consent ? I didn't think so and I was right. But before anyone can say "Worst. Simpsons. Ever", I have to say that the movie does cater to the average movie goer. I mean, it is no coincidence that the movie begins with Homer asking "Why would anyone pay to see something they could watch on TV for free ?".

Because they make me laugh, that's why. Actually, I started laughing way before the movie began. As I sat through the previews of movies, there was one uber hilarious preview - Balls of Fury. Well, imagine a table tennis deathmatch ala Bloodsport, but with a straight-faced Christopher Walken (well, watch that trailer). Anyway, onto the Simpsons movie spoilers we go.

The movie opens up pretty decently with enough straight up gags. The Green Day concert accident, their funeral being held with American Idiot played on a church organ. But some of those were repetitive. Lisa and the cute environmentalist kid, for instance (anyone remember the "Dirt First" episode?). Or the Bart nude on a skateboard which was faintly reminiscent of the Austin Powers movies, except the part where he goes through the hedge and there's a glimpse of his ... y'know. And well, you *have* to hear "Spider Pig" to find it hilarious.

Then there's the environmentalists, except instead of being green peace style sabotage - there's real authority in the shape of EPA. The moment I heard the voice of the EPA chief Russ Cargil, I already knew they were the villains - they could've gotten a new voice instead of reminding me of Hank Scorpio. And he goes mad with power, as he himself says - "Yes, I'm mad with power. Have you ever seen someone go mad without power ? It's boring."

They seal off Springfield in a transparent dome to prevent the pollution from leaking (Marge knits "Dome, Sweet Dome"). And Kent Brockman calling it the "Trappuchino" (cue a starbucks cup with a transparent cover). And then they find that Homer is responsible for the last sludge tank which brings out a whole lynch mob for the Simpson family. And we see Homer go back to being a jerkass as he escapes through the sinkhole in maggie's sandbox.

And then they move to Alaska - but without the cliched "The Simpsons are going to _____" (Marge re-knits "Nome, sweet Nome"). Still, the Alaska sequence was perhaps the most funny part of the movie. A Disney movie parody of Snow White, Bambi and Cindrella - except, they're in a bedroom helping Homer & Marge get undressed. And Homer saying "I'm staying in Alaska, nothing can make me go back to America".

Eventually, we have Cargill talking about the "new" Grand Canyon between Shelbyville and Capitol City. Marge realizes that they are going to destroy Springfield, but Homer doesn't want to help the town. They split up and we have Homer trying to save his family.

But it had to have a happy ending, right ? And the movie closes with Maggie saying her second ever (heh) word - sequel and a La Marseillaise parody for the credits.

But to the fan inside me, there were some things glaringly missing. I mean, two hours of a movie and the only one getting drunk is Bart ? Where is Moe's Tavern, Barney or Homer hanging out with Lenny/Carl ? Flanders is a cool normal person, rather than being a complete example of square-hood. There's no Apu at all and I don't know why we didn't get to see Patty & Selma blame it all on Homer. On the other hand, there's Lisa knocking out Bart, which has never happened so far in the series either. And instead of Rainer Wolfcastle we have Ahnold Schwarzenegger, who was elected to lead & not to read.

But where are the flourishes of George Meyer or John Schwartzwelder ? If I wanted to draw parallels, compare an average Southpark episode with the movie itself to get an idea of how to drag three storylines together into a two hour movie - with the odd song thrown in. The Simpsons movie, as funny as the scattered gags were, hardly has anything which makes me really want to watch it again.

Memorable or not, it was funny while it lasted.

Oh, well, of course, everything looks bad if you remember it.
                    -- Homer Simpson ("El Viaje Misterioso de Nuestro Homer")

posted at: 03:41 | path: /movies | permalink | Tags: , ,

Thu, 02 Aug 2007:

In the last episode of Unicode Fun, we met the non-trademark infringing xoferiF. But this time I bring you English from beyond the equator - do not attempt to adjust your screen.

¿ ʇı ǝsnqɐ ʇ,uɐɔ noʎ ɟı ǝpoɔıun sı unɟ ʇɐɥʍ 'llɐ ɹǝʇɟɐ

˙ƃuoɹʍ uǝǝɹɔs ǝɥʇ ʇɐ ƃuıʞool ǝɹ,noʎ sıɥʇ pɐǝɹ uɐɔ noʎ ɟı

In conclusion, unicode ftw.

A smile is but a frown;
Turned upside down.

posted at: 01:30 | path: /fun | permalink | Tags: , ,

Mon, 30 Jul 2007:

Browsing through my list of unread blog entries, I ran into one interesting gripe about lisp. The argument essentially went that the lisp parenthesis structure requires you to think from inside out of a structure - something which raises an extra barrier to understanding functional programming. And as I was reading the suggested syntax, something went click in my brain.

System Overload !!: I'm not a great fan of operator overloading. Sure, I love the "syntax is semantics" effect I can achieve with them, but I've run into far too many recursive gotchas in the process. Except for a couple of places, scapy packet assembly would be one place, I generally don't like code that uses it heavily. Still, perversity has its place.

So, I wrote a 4-line class which lets me use the kind of shell pipe syntax - as long as I don't break any python operator precedence rules (Aha!, gotcha land). The class relies on the python __ror__ operator overload. It seems to be one of the few languages that I know of which distinguishes the RHS and LHS versions of bitwise-OR.

class Pype(object):
  def __init__(self, op, func):
    self.op = op
    self.func = func
  def __ror__(self, lhs):
    return self.op(self.func, lhs)

That was simple. And it is pretty simple to use as well. Here's a quick sample I came up with (lambdas; I can't live without them now).

double = Pype(map, lambda x : x * 2)
ucase = Pype(map, lambda x: string.upper(x))
join = sum = Pype(reduce, lambda x,y: x+y)

x = [1,2,3,4,5,6] | double | sum
y = "is the answer" | ucase | join 

print x,y

And quite unlike the shell mode of pipes, this one is full of lists. Where in shell land, you'd end up with all operations talking in plain strings of characters (*cough* bytes), here the system talks in lists. For instance, the ucase pype actually gets a list with ['i','s' ...]. Keep that in mind and you're all set to go.

Oh, and run my sample code. Maybe there's a question it answers.

Those who do not understand Unix are condemned to reinvent it, poorly.
                -- Henry Spencer

posted at: 13:22 | path: /hacks | permalink | Tags: , ,

Fri, 27 Jul 2007:

For those who've not been keeping track of what I've been upto, I did manage to make it to OSCON. A couple of days too late, but in time to get there, revamp my slides (after I sat through a couple of php sessions) and give a kickass talk. The last minute jet-lagged efforts to tailor the slides to the crowd was very much worth the effort.

But the fun part of the talk was about four minutes into the talk when Amarok decided to splash up an OSD message which said in loud bold letters - I Wanna Have Your Babies. Panic ... brain skips a beat. Well, there was nothing to do but close Amarok and really make people forget they ever saw it. I think I succeeded in that task - nobody's reminded me about that yet, but enough people have said that it was entertaining. And that's a new experience for me :)

In conclusion, I came, I saw ... I talked.

PS: slides and bazookatooth on IRC reminded me that Amarok said "Girlfriend/Avril Lavigne".

Confidence is simply that quiet, assured feeling you have before you fall flat on your face.
                -- Dr. L. Binder

posted at: 22:53 | path: /conferences | permalink | Tags: ,

Mon, 23 Jul 2007:

I just spent the last eight hours reading Harry Potter and The Deathly Hallows. Feeling totally underwhelmed about the book in itself - even though it is not as disappointing as the Order of the Phoenix was, it still has too many rough edges. The story is perhaps a bit too predictable and the other characters beyond Harry are marginalized into conversation peices. Not to mention the multiple dead ends in narration which do not serve as credible plot twists. Read on if you like spoilers.

What's the deal with the whole fuzzing the wand thingy ? I can totally understand the one true wand (*cough*) deathstick theory, but the wandlore rule-bending requires some extra suspension of disbelief. The list of characters acting out of character is also irritating - I mean, why would Kreacher do an about-face on the mudblood attitude or Percy drop back into the Weasleys ? Seems too convenient to me. And then there were the people dying pointlessly. Except for Mad-eye (and maybe Dobby), nobody had a death which was even a blip in the plot line. Maybe there's a certain realism in killing off random characters.

There are even bigger plot holes. But it would have been more compelling to not put words into Dumbledore's mouth after his death. Especially the part about Snape being on Dumbledore's side, shows him up as a puppet master extraordinare rather than the benevolent wizard that we saw in the previous books. And what's with Gryffindor's sword ? First it gets stolen by Griphook and suddenly it falls out of the sorting hat again ? And it is not exactly Gryffindor's sword to begin with ? As someone who liked Harry Potter in his initial adventures, this book feels too weak and watered down.

My advice ? This is a book to be borrowed rather than bought.

The covers of this book are too far apart.
                -- Ambrose Bierce

posted at: 07:53 | path: /books | permalink | Tags: ,

Sun, 22 Jul 2007:

I thought I'd invented this term. I'd thought it was my small bit of original jargon to contribute. I even had about seven pages of scribbled over notepaper dedicated to the topic - several hours worth of cogitation on the topic from airport lounges, cramped airplane seats and other places of extreme boredom. Pages and pages of attempts to distill out the idea from an amorphous concept, which the two words which by themselves cannot contain.

But everything that's worth saying has already been said. Despite being original, I discovered, to my surprise, that I'm not the first one to use "honorary guy". Well, rather than waste effort explaining how I feel, let me point you to this this cartoon (oh, the irony). But before I rain on my own parade, let me try to unload my mental baggage.

Being the new kid on the block isn't easy. Especially when you are entering an insular psuedo-meritocracy. And by psuedo-meritocracy, I mean that the pecking order is regulated by assumptions of your merit before you get to prove yourself. Somehow anybody testing such waters is likely to find it cold and assume a rather defensive toe-in first approach to exploring the community. And that makes for uncomfortable beginnings everywhere - and beginnings are such delicate times.

People who are thrown into such situations broadcast very strong vibes, which are there for any intelligent human to pick up easily. Perhaps this is my personal bias, but the strongest of those vibes is that of a girl entering a primarily male dominated community, feeling defensive and wary. The message sent is probably a very tentative "Hi, here I am", but due to an outgroup homogeneity bias, the message comes across as a self-entitled demand to adapt to the presence of the newcomer.

Being the nice guy that I am, I usually comply. The effects aren't pretty. Instead of being myself, I revert to some fallback stereotype male persona. This ranges from the shy guy, the shuffler or the patronizing alpha male. The first two finds me as a stoic blinker (oh, yeah ... smile at me) and those girls who are relieved to find me the latter are generally marginalized to footnotes in my eyes. And those who actually hate me for patronizing them actually fills me with sadness & hope at the same time.

And then there's the minority who do not broadcast these messages, the ones who are comfortable being themselves and in turn just "let me be", without ignoring me altogether. These are people I generally treasure as friends - men and women, both. These are people whose opinion I take for face value, for that's not driven by any facade I present to them. And due to a lack of vocabulary, I picked "honorary guy" to refer to women of this group.

But rather than stereotype these people into a new bracket, what the term "honorary guy" really does is to suspend judgement based on stereotypes I've accumulated over the years. Sort of short-circuit out the homogeneity biases and treat them like the individuals they are. I guess I need a better word.

And as for the rest of them, I'm just being as little of myself as I can. Shut up, shuffle or condescend - pick one. After all, you asked for it.

PS: Umm... Dorothea had mentioned it on her blog, last year. Same arguments, different conclusion ... .

The surest way to corrupt a youth is to instruct him to hold in higher esteem those who think alike
than those who think differently.
                                                   -- Nietzsche

posted at: 07:46 | path: /observations | permalink | Tags: , ,

Sat, 21 Jul 2007:

In a development which is becoming depressingly familiar, my travel plans are in a mess. As of right now, I have no flight tickets, not hotel booking and not even an ETA on when both these things will be fixed up - and the conference starts on Monday. It's not like this is the first time or even the the second time this has happened.

But what's really different this time from those two is that finally for once Yahoo! is paying for my travel. For the previous two journeys, I had booked my tickets myself (and blowing up a year's savings each time), which ensured that I had a travel agent to call up and bug. And both times I did manage to get where I wanted and everything did work out nicely. But ever since I heard the "We'll check again on monday" on Friday afternoon, I've sort of given up.

But two out of three ain't so bad ...

One possible reason that things aren't going according to plan is that there never was a plan in the first place.

posted at: 07:53 | path: /conferences | permalink | Tags: , ,

Sun, 15 Jul 2007:

A few months back I bought myself a cycle - a Firefox ATB. For nearly two months before heading out to Ladakh, I cycled to work. One of those days, I carried yathin's GPS along with me. So yesterday night, I dug up the GPX files, out of sheer boredom (and inspired by one of shivku's tech talks). After merging the tracks and waypoints, I managed to plot the track on a map with the help of some javascript. Here's how it looks.

I have similar tracklogs from Ladakh, but they have upwards of 2000 points in each day, which do not play nicely with the maps rendering - not to mention the lack of maps at that zoom level. I need to probably try the Google maps api to see if they automatically remove nodes which resolve to the same pixel position at a zoom level.

I've put up the working code as well as the gpx parser. To massage my data into the way I want it to be, I also have a python gpx parser. And just for the record, I'm addicted to map/reduce/filter, lambdas and bisect.

If you want to put yourself on the map, publish your own map.

posted at: 10:45 | path: /hacks | permalink | Tags: , , ,

Mon, 09 Jul 2007:

Recently, one of the php lists I'm on was asked how to implement a stable sort using php's sort functions. But since all of php's sort functions eventually seem to land up in zend_qsort, the default sort is not stable. The query on list had this simple example which illustrates the problem clearly.

bash$ php -r  '$a= array(1,1,1); asort($a); print_r($a);'

    [2] => 1
    [1] => 1
    [0] => 1

The basic problem here is to produce a stable sort which still operates in quicksort O(n*lg(n)) time. Essentially, falling back onto a bubble sort is ... well ... giving up :)

Schwartzian transform: The programming idiom, named after Randal L. Schwartz, is a carry-over of the decorate-sort-undecorate lisp memoization into perl. The real problem here however was putting it into a php syntactic form which was clean and as similar to the original as possible. For example, here's how the python version of the code would look like (despite the fact that the current python sort is stable).

a = [1,1,1]
b = zip(a, range(len(a)))    # decorate
b.sort()                     # sort
a = map(lambda x : x[0], b)  # undecorate

array_walk() magic: Coming from a world of constant iterators, I had read the array_walk documentation and sort of read the "Users may not change array itself ..." as boilerplate. But as it turns out, the callback function is allowed to change the current value *in place* and for that purpose it gets a reference to the value. With that in mind, array_walk becomes a faux in-place map/transform function.

$a = array(1,1,1);

function dec(&$v, $k) { $v = array($v, $k);}
function undec(&$v, $k) { $v = $v[0]; }

array_walk($a, dec);   // decorate
asort($a);             // sort
array_walk($a, undec); // undecorate

And there you have it, a lispism made famous by perl, implemented nearly exactly in php.

Whoever knows he is deep, strives for clarity;
Whoever would like to appear deep to the crowd, strives for obscurity.
                        -- Nietzsche

posted at: 08:06 | path: /php | permalink | Tags: ,

Been a while since I got back, but somehow it feels like a let-down to be back home. Add to that four interminably long hops, after which I've travelled from the northern most state of India, to nearly the southern most tip (tinymap). That doesn't make for a good mood. But still, as I sit here, most of what happened seems like a distant blurry day dream.

So, in short, I did a bunch of things, took a lot of interesting pictures (more to come) and had a lot of fun.

Still haven't gotten rid of the "your work is worthless" back-of-the-mind "why bother" voice (yes, the one which keeps saying "you'll get paid anyway" and "they don't pay you nearly enough for that") - eventually I'll have to shake off that coders' block & make work fun again.

Except for that extra bit, the rest of me is back, as expected :)

Removing the straw that broke the camel's back does not necessarily allow the camel to walk again.

posted at: 02:14 | path: /travels | permalink | Tags: , ,

Fri, 22 Jun 2007:

By the time you're reading this, I'll probably be in the high himalayas, gasping for breath.

Will be back soon, with a lot of pictures, memories and a nice travelogue - just like last year.

Travel gives you the illusion that you're getting somewhere.
But half the time, you're just coming back

posted at: 08:14 | path: /travels | permalink | Tags: ,

Mon, 18 Jun 2007:

As much as I'd like to pretend to be one, I'm not a real photographer. Photographs are secondary to the experience - travel, laughter, people and memories. But sometimes I do give in and take a photo which requires patience, time and often, post-processing. Saturday night, I decided to prepare, polish and publish one of those.

Basic Tools: For stitching images the workflow involves three tools Hugin, autopano-sift and Enblend. All these tools are available with most popular distributions - apt-get install should do just fine.

The input to the tools are six photos taken with my trusty SD450. They have been taken with nearly identical modes with a Manual setting to prevent the camera from adjusting between pictures (forgot to turn off auto white-balance for this set). The camera has a panorama mode which allows me to fairly accurately align multiple pictures with enough overlap, but that's not really as useful as you might think.

Autopano: First step in creating panoramas is to identify common points. Autopano is an automatic image analysis tool which can identify common areas between photos. The command autopanog should get you the UI required to load all the photos. Other than checking absolute paths, there is nothing else that I found useful to tweak there. Autopano after computation throws out a .pto file which is then pushed onto the next tool.

Hugin: Hugin does all the further magic. After loading in the PTO file, select the image required to be the anchor or reference image of the panorama and anchor that image for position. It does really help if the horizon is horizontal for that picture. Thankfully, Hugin can actually read the EXIF data from the images to obtain information about the lens, field of view and other details which I wouldn't know how to provide. The next tab shows the data gathered by autopano in an editable view, to adjust the control points if required.

Rather than waste much time there, head over to the next tab and click on Optimize now!. It should result in Hugin working most of its miracles, in mysterious ways to perform, by realigning shearing and doing all the other important things required. About time, we got a preview of whatever is going to be the result. One of the curious behavioural defects of the tool is that the preview does not update unless manually initiated. I've wondered a couple of times about why the picture looks exactly the same whatever I do. But here's how it looks right now.

As it is very obvious, the camera's vignetting comes into play for the corners of all the images. Merely placing the layers above each other will not produce a satisfactory panorama. Therefore, rather than combine images inside Hugin, the output from this phase has to be the correspondingly distorted seperate images. So in the last tab, calculate the field of view, the optimal size and select the "Multiple TIFF" (cropped) output option. This gives us individual images which look somewhat like these (hmm... my ICC settings are off in these screenshots).

Enblend: Enblend does exactly what its name suggests. It can blend between images, performing a smooth transition between the frames to create a good looking panorama. Merely pass in all the images in order via the command like to enblend and it does the job neatly (but watch your CPU load). The blended image is not equirectilinear and requires cropping to create a rectangular image. Mine threw out a whoppingly large 161 MB uncompressed TIFF (12335x3438) file. After scaling it down, converting to JPG and re-inserting EXIF data, the image is ready for the world to see.

And that's the way it's done - only five months too late :)

Just because you can explain it doesn't mean it's not still a miracle.
                          -- Terry Pratchett, Small Gods

posted at: 09:11 | path: /tutorials | permalink | Tags: , ,

As Friday wound down, my sore body was resigning itself to a weekend to be spent mainly in bed. But what to feed the beast that roams within ? Out I went to Strand, to hunt for something cerebral, to stimulate thought and induce sleep (when consumed in excessive doses). An unsuccessful hour later, I emerged carrying a copy of Games Indians Play.

Somehow in the back of my mind, I was expecting an Indian re-run of Games People Play. But maybe I was setting the bar too high. The book begins rather uncomfortably, by stating a few prerequisites, right from the depths of disillusionment.

There are three points that the readers must be ready to acknowledge
as prerequisites: 

one, that there is indeed something wrong with us;

two, that there are aspects that do not seem to sit well in modern
civilized society; 

and three, that merely because other modern and civilized societies, 
also suffer from many ills, it does not make our own ills more 

Indeed, the book does tend to critique at a rather personal level than would be necessary. The following four chapters were a visible re-hash of nearly all literature on the subject in the last forty years. The basic systems by which selfish individuals coalesce into a mesh of cooperation explained by game theory principles. For someone who's been on Axelrod for two years, it hardly holds any interest. Except for one particular analogy.

Both India and Pakistan argue exactly as the two prisoners cited in the 
prisoner's dillemma, and arrive the 'rational and well reasoned' decision
to go nuclear - to defect - resulting in the two countries forgoing
the opportunity to save millions of rupees which could have been used

The book somewhat redeems itself, by then attacking the concept of fairness. Reading through it, I somehow seemed to recall the famed lines from Animal Farm - All animals are equal, but some animals are more equal than others. But it rather rambles along into the lack of self-regulation and the inherent disregard for others doing the same (as lunatech keeps saying: lesson #1: the universe isn't fair). Eventually to come to rest at the way Indians measure the importance of a person - purely by how many rules he can bend.

But to fulfil its initial promises, it hasn't gone far enough into the depth of the issue. Rather than discuss selection pressures which reward the behaviour that we observe, it merely presents the facts and leaves a rather dissapointed reader behind. And while the book doesn't even try to tackle the vast agrarian belt of India - it still does not understand that there's more of sub-urban India outside the big cities and inside houses. For something like the state of Kerala - a middle class majority, left-leaning socialist governments and high literacy rates seem to have alleviated at least some of the ills mentioned.

Rather than a cutting root cause analysis of the systemic problems that exist, the book seems to be written in half-disgust about the myopia of present day urban India, with game theory to back it up. Not to disagree with some of his conclusions, but the linking logic seems to from real life into game theory rather than the other way around - leaving me nothing to test his theories with. And there is just a subtle hint of silent reverence towards occidental cultures - a completely apples to oranges comparison (you eat both, right ?). The author desperately needs to read The Human Zoo, before judging the Indian concrete jungles by theories transplanted from university towns (Ithaca or wherever).

In one short phrase, the book talks more about "How/What we are ?" than rationalizing it with Why ? - an outsider's objective analysis which puts you (if you were Indian) in the same frame of mind as the emperor with the new clothes.

Along with this, I'd also picked up Just Wait Till You Have Children of Your Own!. Drains me of all bitterness, that book does. :)

The end of the human race will be that it will eventually die of civilization.
                -- Ralph Waldo Emerson

posted at: 06:48 | path: /books | permalink | Tags: ,

Sun, 10 Jun 2007:

Last monday afternoon, I tore myself from the mundane grind of office to go listen to Eben Moglen talk about GPLv3 and Beyond. The evening that was to follow was as brilliant as the golden monsoon sunset. I happened to miss the GPLv3 conference that went on last year, mostly due to personal reasons and I'd been out of the loop from the whole license process.

A Benevolent GPL: Most people assume GPL version 3 is going to get tougher on those who violate the license. But I was surprised to learn about the amnesty period for bonafide violations, to give the errant a chance to redeem themselves. The termination clause allows for a sixty day period to get back to compliance and avoid the penalties associated with copyright infringement.

Patents: I didn't hear this from Eben Moglen, but it bears repeating - "Where they were supposed to patent new inventions, companies have taken to inventing new patents". And with all due respect, that is exactly what is wrong with the current patent regimen.

The whole Microsoft deal was discussed in the meeting, where the failure of them to act up on the violations would automatically bring in Laches into effect. The FUD might actually work against Microsoft, even if they come up with the list of 253 patents because they will be deprived of all royalty claims for infringements wilfully ignored. Essentially, they're eating into their own war chest of patents with the bluffs.

The Process: There were several comments made about the GPLv3 process, which raises the bar for the creation of new licenses. If someone like OSI decides to ask more questions about community involvement in the development of the license, the proliferation of open source licenses will come to a quick grinding halt. Even moreso, the LGPL is going to become truly GPL+LE (for instance, dotgnu's class libraries and GNU classpath are GPL+LE rather than LGPL).

There will always be people who write BSD licensed code, but it lacks the unifying nature of GPL. The proprietary forks for most successful projects are doomed to die, due to the lack of the very things which make their mainline a success. It is a living tree which only sprouts dead branches - an enormous waste of manpower thanks to short term myopia.

Adoption: In this context, it would be good to remember that something as popular as Firefox is dual-licensed under GPL version 2 or *later*. As far as the world of code is concerned nearly all of that is as good as GPL v3'd, after the first commit which says "version 3 or later".

Freedoms: There have been too many comments of companies like Tivo and others who abide by the legalese, but violate the spirit of the license. But it serves some purpose to point out that the intention of the license upgrade was to plug some of the legal loopholes which have come up during the last half-decade. Those who shall suffer are those who have been exploiting those loopholes.

Essentially, the license makes no pretence about the freedoms it tries to protect - the users'. In turn, it does protect the developers who in conjunction with the former, makes up the community - from underhanded attacks by for-profit redistributors. Basically, there are people who will lose out due to the changes, but it somehow looks as though the license is slanted the right way for Joe Hacker - both for individual gain and altruism.

The difference between the Mafia and an insurance agency is that the latter doesn't burn down your house if you don't pay.

posted at: 18:03 | path: /conferences | permalink | Tags: , ,

Wed, 30 May 2007:

Last night, I picked up a book. And when I say a book, I mean two books. One was new and the other old; one's silver and other gold. After nearly a decade, I read Shooting an Elephant again. The essential pathos of someone trapped in a system not of their making, whose identity and behaviour is dictated by unspoken rules, is palpable through out the story. I never got the message with the clarity that I do now - but I was a teenager then.

To see Orwell as merely the author of 1984 and Animal Farm is to do him injustice. Reading through his essays, especially the Lion and the Unicorn throws new light at his ability to communicate adeptly and adequately. Without turning any flowery phrases, he manages to paint a vivid picture of England at war. The part one starts off very directly with "As I write, highly civilized human beings are flying overhead, trying to kill me.", before attacking the essential rationalization of patriotism. But even more cynically, the second part is titled Shopkeepers at War, talking about the private economy's reaction to war (Lend to Defend; Drink Guinness).

I was half-way through "Prevention of Literature", when I switched to the other book. In one of those lapses of judgement (you know the kind), I had bought a copy of Salmon of Doubt. If not for the essays liberally sprinkled throughout the book, I would have regretted that decision. Unfinished books are not meant to be read - for instance, I have read every P.G Wodehouse novel/short-story except for Sunset at Blandings. Hidden in the pages of Salmon of Doubt, between articles for boys' weeklies and random musings about technology is a transcript of an extempore speech.

The speech is titled Is there an Artificial God?. The speech, just like all of his works, is splattered with inconsequential references. For instance, it gets on its way with the following phrase.

I just want to mention one thing, which is completely meaningless,
but I am terribly proud of - I was born in Cambridge in 1952 and 
my initials are D N A!

But it soon settles down into an exploration of man's progress and perhaps for a bit, the lack of it. The ages of sand sequence points out the cornerstones of the last four centuries of human progress - telescopes, microscopes, silicon chips and fibre optics. A quick dip into a quantum universe and back to challenging religious dogmas. To climb out of that argument only to throw in a strawman of money being a completely fictious entity. Eventually concluding that we humans live in a world mainly of our creation - inside our heads or outside.

Reading somehow does not seem to be quite the one-way process most people imagine it to be.

The possession of a book becomes a substitute for reading it.
                -- Anthony Burgess

posted at: 16:02 | path: /books | permalink | Tags: ,

Sat, 26 May 2007:

Brian Shire has put up his slides (835k PDF) of his php|tek talk.

Quite interesting procedures followed to prevent the very obvious cache slam issues by firewalling the apache while restarting it, as well as the priming sub-system they use. Also the cross-server (aka site-vars) seem like a good idea as well - a basic curl POST request moving around json data could potentially serve as a half-reliable cross-server config propogator.

Seeing my code used makes happy ... very happy, indeed.

I'm willing to make the mistakes if someone else is willing to learn from them.

posted at: 22:27 | path: /php | permalink | Tags: , ,

Thu, 24 May 2007:

Most wireless routers come without a DNS server to complement their DHCP servers. The ad-hoc nature of the network, keeps me guessing on what IP address each machine has. Even more annoying was the ssh man-in-the-middle attack warnings which kept popping up right and left. After one prolonged game of Guess which IP I am ?, I had a brainwave.

MAC Address: Each machine has a unique address on the network - namely the hardware MAC Address. The simplest solution to my problem was a simple and easy way to bind a DNS name to a particular MAC address. Even if DHCP hands out a different IP after a reboot, the MAC address remains the same.

Easier said than done: I've been hacking up variants of the twisted.names server, for my other dns hacks. To write a completely custom resolver for twisted was something which turned out to be trivial once I figured out how (most things are), except the dog ate all the documentation from the looks of it.

class CustomResolver(common.ResolverBase):
  def _lookup(self, name, cls, type, timeout):
    print "resolve(%s)" % name
    return defer.succeed([
      (dns.RRHeader(name, dns.A, dns.IN, 600,
          dns.Record_A("", 600)),), (), ()

Deferred Abstraction: The defer model of asynchronous execution is pretty interesting. A quick read through of the twisted deferred documentation explains exactly why it came into being and how it works. It compresses callback based design patterns into a neat, clean object which can then be passed around in lieu of a result.

But what is more interesting is how the defer pattern has been converted into a behind-the-scenes decorator. The following code has a synchronous function converted into an async defer.

from twisted.internet.threads import deferToThread
deferred = deferToThread.__get__

def syncFunction():
    return "Hi !";

The value a returned from the function is a Deferred object which can then have callbacks or errbacks attached to it. This simplifies using the pattern as well as near complete ignorance of how the threaded worker/pool works in the background.

53: But before I even got to running a server, I ran into my second practical problem. A DNS server has to listen at port 53. I run an instance of pdnsd which combines all the various dns sources I use and caches it locally. The server I was writing obviously couldn't replace it, but would have to listen in port 53 to work. Very soon I discovered that the two servers can listen at port 53 on the same machine. There are 255 different IPs available to every machine - is the same as localhost, but the different IP means that pdnsd listening on does not conflict with this. Having reconfigured the two servers to play nice with each other, the really hard problem was still left.

RARP: The correct protocol for converting MAC addresses into IP addresses is called RARP. But it is obsolete and most modern OSes do not respond to RARP requests. One of the other solutions was to put a broadcast ping with the wanted MAC address. Only the target machine will recieve the packet and respond. Unfortunately, even that does not work with modern linux machines which ignore broadcast pings.

ARPing: The only fool-proof way of actually locating a machine is using ARP requests. This is required for the subnet to function and therefore does work very well. But the ARP scan is a scatter scan which actually sends out packets to all IPs in the subnet. The real question then was to do it within the limitations of python.

import scapy: Let me introduce scapy. Scapy is an unbelievable peice of code which makes it possible to assemble Layer 2 and Layer 3 packets in python. It is truly a toolkit for the network researcher to generate, analyze and handle packets from all layers of the protocol stack. For example, here's how I build an ICMP packet.

eth = Ether(dst='ff:ff:ff:ff:ff:ff')
ip = IP(dst='')
icmp = ICMP()

pkt = eth/ip/icmp


The above code very simply sends out an ICMP ping packet to every host on the network (10.0.0.*) and waits for answers. The corresponding C framework code required to do something similar would run into a couple of hundred lines. Scapy is truly amazing.

Defer cache: The problem with flooding a network with ARP packets for every dns request is that it simply is a bad idea. The defer mechanism gives an amazing way to slipstream multiple DNS requests for the same host into the first MAC address lookup. Using a class based decorator ensures that I can hook in the cache with yet another decorator. The base code for the decorator implementation itself is stolen from the twisted mailing list.

Nested Lambdas: But before the decorator code itself, here's some really hairy python code which allows decorators to have named arguments. Basically using a lambda as a closure, inside another lambda, allowing for some really funky syntax for the decorator (yeah, that's chained too).

cached = lambda **kwargs: lambda *args, **kwarg2: \
         ((kwarg2.update(kwargs)), DeferCache(*args, **(kwarg2)))[1]

def lookupMAC(name, mac):

The initial lambda (cached) accepts the **kwargs given (cachefor=420) which is then merged into the keyword arguments to the second lambda's args eventually passing it to the DeferCache constructor. It is insane, I know - but it is more commonly known as the curry pattern for python. But I've gotten a liking for lambdas ever since I've started using map/reduce/filter combinations to fake algorithm parallelization.

After assembling all the peices I got the following dnsmac.py. It requires root to run it (port 53 is privileged) and a simple configuration file in python. Fill in the MAC addresses of the hosts which need to be mapped and edit the interface for wired or wireless networks.

hosts = {
'epsilon': '00:19:d2:ff:ff:ff'
'sirius' : '00:16:d4:ff:ff:ff'
iface = "eth1"
server_address = ""
ttl = 600

But it does not quite work yet. Scapy uses an asynchronous select() call which does not handle EINTR errors. The workaround is easy and I've submitted a patch to the upstream dev. With that patch merged into the scapy.py and python-ipy, the dns server finally works based on MAC address. I've probably taken more time to write the script and this blog entry than I'll ever spend looking for the correct IP manually.

But that is hardly the point .

What's in a name? that which we call a rose
By any other name would smell as sweet;
        -- Shakespeare, "Romeo and Juliet"

posted at: 08:15 | path: /hacks | permalink | Tags: , , ,

Mon, 21 May 2007:

Previously while talking about inclusion checks I had included a few helpful digraphs of php includes. Those were drawn with the help of a gdb macro and a bit of sed/awk. But that becomes a real hassle to actually use very quickly while inspecting fairly large php applications.

The solution to repeatable include graphs from php came from the way the include_once override hack was implemented. By overriding the INCLUDE_OR_EVAL opcode in Zend, I could insert something which could potentially log all the various calls being made to it.

That is exactly what inclued does. The extension provides a single function which provides a copy of the collected data, aptly named inclued_get_data().


The above peice of code returns an array with too much information.

  [includes] => Array
      [0] => Array
          [operation] => include
          [op_type] => 2
          [filename] => x.php
          [opened_path] => /tmp/x.php
          [fromfile] => /tmp/z.php
          [fromline] => 2

Overriding the opcode implies that more information can be collected, like whether the include parameter was a constant (i.e APC can optimize it to a full path), the function the include was in (__autoload *cough*) and the stream wrapper (phar: for instance). The information also includes info about whether the include_once was ignored because of duplication.

Only Data: The extension however does not make any judgements on the code. I have resisted the temptation to put an error log saying "If eval() is the answer, you're almost certainly asking the wrong question" . But there are interesting ways to massage this data into something useful. For example, I just put the following lines into the Wordpress index.php.

$fp = fopen("/tmp/wp.json", "w");
fwrite($fp, json_encode(inclued_get_data());

Loading the JSON data and pushing out a graphviz layout was almost too easy. Here's a clipped section of how my graph ended up looking.

The graph shows about 40-odd includes being hit for a single request, but isn't messy at all. Hopefully, someone finds an actual peice of sphagetti includes which shows off the beauty of this ext (remember, this shows include_once misses & hits).

Hope this helps people debug "Where is this include coming from ?" question I've run into so many times :)

It was the kind of mental picture you tried to forget. Unsuccessfully.
            -- Terry Pratchett, "The Light Fantastic"

posted at: 19:43 | path: /php | permalink | Tags: , ,

Wed, 16 May 2007:

Every year, I dread this day. A day when I take stock of my achievements, attempts and failures. But I guess this year too is different.

So many things happened last year. I did the right thing for some people - put family before my career, principles ahead of money, comfort of others ahead of mine. The priorities did change, but I still did some of the things I wanted - travelled a bit, read a pile of books and just hung out with friends for most of the time. Some things caused me to sit up and evaluate my life - they weren't nice things, but I'm better for having weathered them. In short, it does feel good to be twenty five !

And I got some interesting advice today: "Act your age. Drop the 'been there, done that' attitude and stop acting like you're 35".

I will, oh ... I will.

Old age is too high a price to pay for maturity.

posted at: 05:31 | path: /me | permalink | Tags: ,

Sun, 13 May 2007:

I wouldn't have paid to watch this movie. Instead of the dark, deep conflict between power and losing control, the movie is peppered with a self-obsessed Peter Parker and a depressed MJ. Neither of which are wrong or bad, but the emotional drama is also not used to any effect.

No Villains: Topher Grace is the only saving grace in the whole movie. His transformation into Venom gives a decent bad guy. The sandman puts me to sleep and Harry flips-flops in & out of villainhood. The small tugs & pulls of conflict in the former was brought out in the wrong light - less of dark desperation than I'd imagine.

The scenes looked peiced together from clips inspired from movies. The comedy just didn't fit in and neither did the cool Peter Parker doing a Saturday Night Fever sequence down the boulevard. And the tower spire with the dark spiderman on top looked nearly identical to the scenes from Underworld, including the jump off the top. It attempts to do a fair bit of moralizing, but the message falls far short of the original With great power comes great responsibility, even after they got Stan Lee to say it ("I guess one *Superhero* can make a difference.", 'nuff said). But there is one quote off the movie which I'm still going to take to heart.

Aunt May: You start by doing the hardest thing: you forgive yourself.

All in all, I think I forgive myself for wasting those hours of my life.

Children begin by loving their parents.
After a time they judge them.
Rarely, if ever, do they forgive them.
                -- Oscar Wilde

posted at: 10:12 | path: /movies | permalink | Tags: ,

Thu, 10 May 2007:

APC user cache is cool. It provides an easy way to cache data, in the convenient form of hashes & more hashes within them, to share the data across processes. But it does lend itself to some abuses of shared memory which will leave your pager batteries dead, disk full of cores and your users unhappy. Eventually, the blame trickles down to splatter onto APC land. Maybe people using it didn't quite understand how the system works - but this blog entry is me washing my hands clean of this particular eff-up.

apc_fetch/_store: The source of the problem is how apc_fetch and apc_store are used in combination. For example, take a look at countries.inc from php.net. The simplified version of code looks like this.

if(!($data = apc_fetch('data'))) {
  $data = array( .... );
  apc_store('data', $data);

There is something slightly wrong with the above code, but the window for the race condition is too small to be even relevant. But all that changes the moment a TTL is associated with the user cache entry. For a system under heavy load, all hell breaks loose when a user cache entry expires due to TTL. Let me explain that with some pretty pictures and furious hand-waving.

Each of the horizontal lines represent an individual apache process. The whole chain reaction is kicked off by a cache entry disappearing off the user cache. Every single process which hits the apc_fetch line (as above), now falls back into the corresponding apc_store. The apc_store operation does not lock the cache while copying data into shared memory. So all processes are actually allowed to proceed with the copy into shared memory (yellow block) in parallel. The actual insertion into the cache, however is locked. The lock is hit nearly simultaneously by all processes and sort of cascades into blocking the next process waiting on the lock.

Lock, lock, b0rk !: The cascade effect of waiting on the same lock eventually results in one process locking for so long that it hits the PHP execution timeout Or the user get bored and just presses disconnect from the browser. In apache prefork land, these generate a SIGPROF or SIGPIPE respectively. If for some reason that happens to be while code inside the locked section is being executed, apache might kill PHP before the corresponding unlock is called. And that's when it all goes south into a lockup.

So, when I ran into this for the first time, I did what every engineer should - damage limitation. The signals were by-passed by installing dummy signal handlers and deferring the signals while in locked sections. Somebody needs to rewrite that completely clean-room, before it is going to show up in the pecl CVS. The corresponding cache slam in the opcode cache is controlled by checking for cache busy and falling back to zend_compile - but the user cache has no such fallbacks.

I wish that was the only thing that was wrong with this. But I was slightly misleading when I said the copy into shared memory was parallel. The shared memory allocator still has locks and the actual allocation looks somewhat like this for 3 processes.

The allocations are interleaved both in time and in actual layout in memory (the bottom bar). So adjacent blocks actually belong to different processes, which is not exactly a very bad thing in particular. But as the previous picture illustrates, every single apc_store() call removes the previous entry and free's the space it occupies. Assuming there are only three processes, the free operation happens as follows.

The process results in very heavy fragmentation, due to the large amount of overlap between the shared memory copy (apc_cache_store_zval) across processes. The problem neatly dove-tails into the previous one as the allocate & deallocate cost/time increases with fragmentation. Sooner or later, your apache, php and everything just gives up and starts dying.

There are a couple of solutions to this. Since APC 3.0.12, there is a new function apc_add which reduces the window for race conditions - after the first successful insertion of the entry, the execution time of the locked section is significantly reduced. But this still does not fix the allocation churn that happens before entering the locked section. The only safe solution is to never call an apc_store() from a user request context. A cron-job which hits a local URL to refresh cache data out-of-band is perfectly safe from such race conditions and memory churn associated.

But who's going to do all that ?

In the Beginning there was nothing, which exploded.
                -- Terry Pratchett, Lords and Ladies

posted at: 13:27 | path: /php | permalink | Tags: , ,

The great and awesome teemus has come up with a theme song for APC. Set to the background score of Akon's Smack That !, it goes something like this.

Cache that, give me some more,
Cache that, don't dump some core,
Cache that, don't hit the _store,
Cache that, ~ oohoooh ~

In a sort of related note, I might be singing this somewhere.

Everything in nature is lyrical in its ideal essence, tragic in its fate, and comic in its existence.
                      -- George Santayana

posted at: 01:44 | path: /fun | permalink | Tags: , ,

Fri, 04 May 2007:
Man's unique agony as a species consists in his perpetual 
conflict between the desire to stand out and the need to 
blend in.
                -- Sydney J. Harris

It is a trade-off. Everybody needs their own personal rebellion and to be welcomed back into the herd - like the prodigal son of the fables.

Some people get stuck half-way. Some people don't even start.

Not to be a socialist at twenty is proof of want of heart;
to be one at thirty is proof of want of head.
              -- Georges Clemenceau

posted at: 04:05 | path: /philosophy | permalink | Tags: ,

Tue, 01 May 2007:

Apparently the median for my office persona is a straight faced automaton. But about a fortnight ago, all that changed. I realized the obvious solution in one of those moments of lucidity - "don't worry". It might sound simple, but it takes a lot of effort to side-swipe the society's influence to actually say don't sweat the small stuff to yourself. And it takes great tragedy too, to separate the small things from the large and the large from the inconceivable. So here's how the book would look like if I wrote a book about it - which I won't, but I still love the cover (click image for bigger version).

I've generally stopped being upset by small setbacks. Is only life, as they say it. But it took a quite large learning curve before I dropped into this chilled out world, where everything's cool (eventhough it is summer) and I get the important things done right. I guess I'm not the first one to actually go down this path. Rather than say it out myself, let me pull out those words from the pen of Bill Watterson and the mouth of that cheeky six-year-old.

But could that really be ? I suppose so ... it works at this end.

"That depends a good deal on where you want to get to," said the Cat.
"I don't care much where--" said Alice.
"Then it doesn't matter which way you go," said the Cheshire Cat.

posted at: 05:00 | path: /me | permalink | Tags: , ,

Mon, 30 Apr 2007:

I've sort of been toying around with AttackAPI over the weekend and interestingly, I figured out that I was vulnerable to a lot of the attack vectors that were present. But what javascript causes, javascript can fix too. So here's a quick fix to prevent an intruder from portscanning your local network for ftp or other ports.

FindProxyForURL: The proxy autoconfig file can be used to filter out requests to the naughty port scanners. The aim is to detect a bad URL, redirect it to a proxy which will log the referrer and raise the corresponding alarms. And to parse out the URL properly, I use the following regex.

var urlRegex = new RegExp(
    "^(([^:/?#]+):)?"+ // scheme
    "(" +
        "(//)?"+ // delim
        "(([^@]+)@)?" + // userinfo
        "([^/?#:]*)?" + // host
        "(:([^/?#]+))?"+ // port
    "([^?#]*)"+  // path
    "(\\?([^#]*))?"+ // query
    "(#(.*))?$" // fragment

After parsing out the URL, my very simple filter checks for a port value which is suspicious (i.e below 1024 and not 80) and redirects it to my local logging proxy which eats the request. I still haven't figured out a way of leaving a honeypot for attackers to ensure there's a human at the other end.

function FindProxyForURL(url, host)
  var m = url.match(urlRegex);
  	var port = m[7];
	if(port != "" && port != "80" && port.length < 4)
	  return "PROXY";
  	return "PROXY";

  return "DIRECT";

It should be perfectly possible for this technique to be extended to prevent XSS exploits with javascript land checking for common XSS attack vectors. But the essential problem is that this particular function cannot maintain context easily. The previous request cannot affect the current one, nor the current one affect the next request. Having minimal ability to learn does cause a large number of problems for a really practical solution - but if there is already such a hook, it might be possible to hook an extension into it.

Too many people are thinking of security instead of opportunity.
                            -- James F. Byrnes

posted at: 01:23 | path: /insecurity | permalink | Tags: , ,

Thu, 26 Apr 2007:

For any php performance freak, include_once has been a pain in the neck for a long time. In a previous post I had talked about how the common workarounds affect something like APC. With the release of APC 3.0.14, there is a decent workaround which doesn't require any changes to the php code. But first, let me drag out another almost-workaround to the whole include_once problem.

rinclude_once: So we create a new function for the purpose. Let me just call it rinclude_once and use that everywhere. The function takes in the filenames, pushes it into a hash before including it.

function rinclude_once($file) 
  global $rinclude_files;
  if($file{0} != '/') return include_once($file);
  if(isset($rinclude_files[$file])) return;

  $rinclude_files[$file] = true;


This bit of code works - but only for absolute filenames. I put it through its paces without apc against include_once. Surprisingly, without APC, this code is slower than the php engine's include_once checks. That somewhat makes sense because the extra include for the rinc.php makes a slight dent into the compile and execution time, overshadowing the cycles wasted in the include_once syscall land.

Excluding the slight blip in performance, both the bits of code are nearly neck-to-neck. The real cost of include_once is only evident when you throw in APC. For every file which was included, include_once opens the file before checking for multiple inclusions. The extra system call shows up in the graph below. But the rinclude_once does not work at all for relative path includes (the second pair of bars) and therefore trails badly in this race for performance.

include_once_override: The solution to this problem is freakish. APC meddles with the brains of the Zend interpreter and inserts its own version of ZEND_INCLUDE_OR_EVAL opcode handler. The new handler does not indulge in the gratuitous fopen idiocy present in the default handler (but it does fopen once) and checks for the filename in EG(included_files) hash before doing a normal include() (thank pollita for that). And it should come as no surprise that the C module outperforms the php land equivalent.

But all of these only work for absolute paths as the pathetic numbers on the relative path includes shows. That is where the path canonicalization kicks in. In APC's nostat mode, the filenames have to be absolute paths for the mode to be useful. But rather than force everyone to modify their include lines, APC rewrites the constant strings into the corresponding path names, after lookup. Essentially it converts all relative path includes, such as those from the pear paths, into absolute path includes. This works well with the stat=0 mode because file modifications are ignored after caching.

At last, we see both the relative and absolute includes touching the same level of performance - because they are no different from each other in opcode land. But as you can clearly see, that does not improve the performance of the relative include for the rinclude_once because it does dynamic includes. The opcode cache cannot determine what the value of $file will be for the include_once($file); line and cannot optimise that. The performance actually takes a dive because the relative path name passed in has to be full resolved for every request.

But having said all this, the same benchmark with plain includes is faster than any of these. I think there is a fair bit of optimisation left in this beast, but what is needed for that is the rest of the world to disappear while I code. Deep hack is hard to achieve when you're a ....

You can't second-guess ineffability, I always say.
              -- Good Omens

posted at: 02:27 | path: /php | permalink | Tags: , ,

Mon, 23 Apr 2007:

Goodbyes are hard to say. The heart still hopes that in this small world, our paths shall again cross.

My voyage ran aground into one of those moments today - sabiokap's last day in Yahoo! Bangalore. He's been the joker in our pack, inventor of crazy games and a very good friend. We all were nightbirds who used to hangout together and in general formed the resident population of the office nearly 24x7. Sitting up in the balcony at 2 AM in the morning was actually fun when everybody thought there was a point to working hard & long - and we liked it.

But more than just at work, he's been someone for me to lean on when my life was shaky. When I was penniless and homeless after booking my LCA 07 tickets, I crashed out in his guest room for four months. I managed to get out of that financial hole thanks to premshree spotting me some dough and accomodation at Hell's Kitchen (that's what we call sabiokap's place, for good reason).

There was a certain bit of company ethos that is oozing out slowly to various other places these days. The day (or well night) of the workaholic hacker is gone, obsoleted by the eary morning meetings. The concept of working with each other replaced by working as team, under the bidding of a manager. The days don't pass so easily and to pour the company into a new mould needs a cultural genoicide of sorts. But well, the good thing is that I was here for those good old days - work hard, play harder and don't care what else happens.

I don't hold any grudges against anyone, but all this takes away the thing I loved about this company - interesting people and good friends. For whatever reasons, good or bad, people I like working with are leaving, those with attitude, talent and a sense of humour. All I can do is wish them Godspeed.

That's how it ends, not with a bang but a whimper.

        The ability to bear with calmness the misfortunes of our friends.

posted at: 19:52 | path: /yblr | permalink | Tags: , ,

Poetry is not my forte. But even my prosaic soul was stirred up the caustic wit and wordplay of Dorothy Parker's poetry. My first run-in with her poetry is thanks to spo0nman, who showed a single verse about three months back - that one verse remains scrawled on the corner of my whiteboard to this day.

Drink and dance and laugh and lie,
 Love, the reeling midnight through,
For tomorrow we shall die!
 (But, alas, we never do.)

Ah, poetry like such has to be read again. The flash of brilliance on the last line, twisting everything said before into its opposite. To lead you down one path of thought, only to yell - Ha, you didn't see that coming is hard to do with poetry. Yet she does, but not without sacrificing brevity or any of the other flourishes that poets are wont to do.

There is a certain cynical poke at the foibles of mankind, of the ephemeral pains of the heart, of the dissolution of the soul, which says more between lines than the average page of a novel, yet compressed into verse. There is a certain bitterness in her work, but one not borne out of sadness, but of defiance at a world which disappoints. Yet somewhere, we see a hopeless paradise carved out of this world with knifes and daggers of cynicsim. And still, to give up sadness is not an easy sacrifice to make.

When I am old, and comforted,
 And done with this desire,
With Memory to share my bed
 And Peace to share my fire,


I'll forget the way of tears,
 And rock, and stir my tea.
But oh, I wish those blessed years
 Were further than they be.

But there is humour and fun in this hidden - but only for those whom it comes unbidden. It is not quite verse that I penned, but in my life worse I have sinned. Inspired me she did with her sorrow, but with an ounce of talent of hers to borrow (for it is a poor poet who does not inspire).

The themes are slightly odd for a contemporary of Wodehouse, verging on the suicidal, bittersweet and mordant. But the soul searching gives way to girlish glee in such twists that this reader is left speechless. The gift of repartee is unmatched except perhaps by Nancy Astor on the other side of the Atlantic pond.

Thankfully, a fair bit of her work was done before 1923 which means that the copyright restrictions no longer apply to the work. You can get a fairly large collection of her poems from Poem Hunter as a very convenient pdf file [650k]. Take a good close look at some poems in there like the Burned Child, Recurrence, The Evening Primrose or Surprise.

And it still echoes back But alas, we never do ...

Four be the things I am wiser to know:
Idleness, sorrow, a friend, and a foe.
          -- Dorothy Parker

posted at: 04:45 | path: /books | permalink | Tags: , ,

Thu, 19 Apr 2007:

Over the last month, I've been poking around OpenMoko. The real reason was because toolz had a prototype phone with him. But the real reason I got anything done on the platform is because of the software emulation mode, with qemu-arm. The openmoko wiki has a fair bit of detail on running under QEMU - but if you just want pre-packaged ready to run QEMU images, take a look at jebba's pre-built images. All I've added to that is the qemu tun-tap network adapter ( -net nic -net tap ) so that I can scp stuff in & out of the phone image. Here's how the applications actually look on the emulator phone (it is *very* CPU heavy - if you have a Core2Duo or something, this would be a nice time to take a look at man taskset(1))

pnet on moko: Back in 2005, krokas had built the OE based packages for pnet. So essentially, building pnet ipks for OpenMoko is no different from building it for any other OE platform, especially because pnet hsa nearly no dependencies on anything beyong libc and libX11.

But the register asm() trick pnet uses to ensure that values like program counter and frame pointer are stored in the correct registers does not work on arm gcc-4.1.1. Aleksey has implemented a couple of solutions like the __asm__ barriers. But as of now, the engine is running in pure interpreter mode, which is not fast enough.

The emulator mode is pretty decent - even with the stock qemu-arm. If my interest keeps up, I'll probably try the OpenMoko patched qemu. I did build the whole toolchain and rootfs from scratch with MokoMakefile - but monotone is really painful to set up and the entire build takes a whopping 14 gigs of space on my disk. So if you're thinking of playing around with moko, don't try that first up :)

Telephone, n.:
An invention of the devil which abrogates some of the advantages of making a disagreeable person keep his distance.
                -- Ambrose Bierce

posted at: 15:43 | path: /hacks | permalink | Tags: , ,

Recently AfC wrote a HOW post on linux & open source, but the question I'm more often asked is WHY.

Most of the content in this entry comes from the similarly titled GNUnify 2006 BoF session with inputs from spo0nman, premshree, lunatech, pradeepto, G0SUB and the students at the conf. And it attacks the topic from the other side of the problem - what more does a F/OSS programmer bring to the table at a job than the other guy.

Not Technical Talent: For a long time, I had assumed that it was the proven technical competency which has been tested in the real world. But in the recent past, I've met enough technically adept folks from both sides of the divide to take that assumption to peices. People from the proprietary code land are equally capable and just because your code is open does not make it any better by default. Having your code out in the open does make it easier to judge your ability for a third party - but that'd be end of this blog entry if a programmer was merely a code producing machine.

Co-operation: The transition from college to the workplace is rather jarring. Having spent the last fifteen-odd years in constant competition with your peers, suddenly you are thrown into a world where you need to co-operate with, rather than screw over, the next guy. Most people who work in a successful open source project with multiple contributors have gotten past that particular hurdle much more earlier and the transition into a workplace where the focus is on getting things done rather than merely doing your own part is much more easier.

You got Bugs ! (and users): I've often been shocked by the way people deal with bugs and criticism. The immediate 'full power to shields' reaction is probably understandable, but rather unpleasant. But for someone who has worked with other people in a serious project, criticism from your peers is easier to handle or at least something they have handled in the past (or you'd think so). Also bugs from end users gives a developer some level of user focus which is totally absent in the college graduate. Seeing the user and his problems as one of the factors while coding is hard to acquire if you've written code for a college professor to run once.

Communication Skills: Most f/oss teams are spread across the world. Their communication happens mostly through filtered channels such as mailing lists, irc or bugzilla. It does take some effort to involve yourself in such a global environment when communication can be easily misinterpreted for tone and context. Working in such an environment easily carries across into the modern world of distributed development required for global product development.

Consensus: Have you ever been in a technical argument at work ? There are always people who have a hard time accepting someone else's point. If you've worked on a real peice of code long enough with a group, you've had one of these hard-to-swallow decisions to deal with. It does come as a nasty surprise to most graduates out of college when they run into one of those. Having gone through the standard sulk phase for the first few such run-ins, most f/oss developers are more understanding and less obnoxious about accepting someone else's idea.

Oh the humanity !: Somehow, getting involved with a project, working with different people and enjoying the experience does result in a more rounded work persona. The whole community effect can easily seperate the assholes from the good guys as easily as it seperates the men from the boys (uh... women from the girls too). In general, it also selects for a person of the community rather than the brilliant loner and most employers prefer the former.

There are many more qualities which are quintessential to the f/oss hacker ethos - passion, commitment and plain old curiosity. But they are not unique to the group - anybody who has run into a mac fanboi would agree on the passion part at least :)

I'm by no means the first guy to say that F/OSS hackers are better hires - virtual referral and loudthinking do reinforce my conclusion - but someone needs to tell the students.

Yacc owes much to a most stimulating collection of users. Their irritating unwillingness to learn how to
do things my way has usually led to my doing things their way; most of the time, they have been right.
          -- S.C Johnson.

posted at: 02:54 | path: /misc | permalink | Tags: ,

Sun, 15 Apr 2007:

Happy Vishu to everyone ! For those of us that follow the Malayalam calendar, this marks the birth of the new year. Vishu has like, totally been my favourite festival for so many reasons - it is bang in the middle of the summer vacation, being a kid means you cash in with Vishukkaineetam from all your elders and lastly *FIRECRACKERS* !!.

Because I'm in mourning, I'm not actually celebrating Vishu this year, but that doesn't really stop me from being carried away by the old sounds and smells remembered from the days of my childhood. Ah, nostalgia ... the consolation prize of a loss. For the last 15 years, I've never missed a year in spending this particular day with most of my extended family.

Well, I guess this makes it sixteen in a row ... :)

We used to call him April Showers, because he brought May flowers.
                            -- P.G Wodehouse, The Small Bachelor

posted at: 17:46 | path: /fun | permalink | Tags: ,

Wed, 11 Apr 2007:

After much thought and inaction, I've compiled a quick list of basic things you can do at work to reduce your tangible value to the company (coming to a bank account near you, soon !).

  • Answer questions on devel forums
  • Write a technical blog
  • Live the company values
  • Write cool hacks
  • Have a serious hobby
  • Contribute to an open source project

All of these things shout out that you have too much free time. If you send detailed, well thought out answers to a question on a devel forum - you aren't working hard enough. If you've got time to write a decent blog about technology - you aren't working hard enough. If you actually live the company values (for instance, like I do) - your irreverance and sense of humour do not suit a professional in this business. So is going around your product managers to actually build something that you personally like, without any input from the product strategy team - that's not what you get paid for. And hobbies - they waste your time and cost you money. Without a hobby to spend your disposable income on, you won't need as much money as you're asking for now.

But the last one takes the cake. Assume your full time job involves working on an open source project. Now answer me this, "what *competitive* advantage does your work bring to this company ?". After all the code that you write automatically becomes available to everybody - irrespective of who paid for the development costs. Code thus released drops to near zero value and ergo, the process of creating it ...

These should just about work, but YMMV.

PS: I'm being seriously *sarcastic* (or tersely ironic) here - these things are a potential investment in your betterment, do them, be a better person and as the Bhagavat Gita said - Karmanye vadhikarasthe, ma phaleshu kadachara (do your duty and expect no reward).

The term investment (the basis of all capital) is pretty much forgotten.
Instead, investing money is considered spending it.
                -- slashdot #18632805

posted at: 23:45 | path: /rants | permalink | Tags: , ,

Mon, 09 Apr 2007:

While on my way back from Sydney on Singapore airlines, I ran out of things to do. Now, rather than play Mario on their onboard entertainment, I took a look at their comedy section. That's how I saw my first episode of Futurama - Roswell that ends well. And I was a fan in a few minutes. Thanks to set of borrowed DVDs, I've finished watching every single episode - yup, all 72 of them. I love the series for the the very same reasons I love the other Matt Groening creation - the Simpsons. The referential coherence of both series shows a nearly impossible amount of research put into the script and story line.

Good News, Everyone: Just like every other good cartoon, the jokes are at every level of the series - the characters, situations and even tag lines. In fact, the whole plotline is one giant inside joke about how things change, yet stay the same. For instance, Fry who was a delivery boy in the year 1999, ends up as a delivery boy in the year 3000 - except the deliveries are now to far-off planets, rather than to city blocks.

The Characters: The character set for the show is literally identical to my standard comedy situation. A befuddled hero, a self-confident heroine, one weirdo side-kick and then a couple of total reverse stereotypes. This is literally identical to say, h2g2 (Arthur, Trillian, Zaphod) or Coupling (Steve, Susan, Jeff) - we find Fry, Leela and Bender in each of those slots. Of course, Futurama would be a non-starter without Bender (aka Bender Bending Rodríguez). Rather than being a logical minded machine, he's the ultimate personification of the human vices - drinking, smoking, swearing, gambling, stealing ... in short, the whole deal. Or take the professor for instance, who is actually Fry's nephew by a long way, but is older than Fry. Even Dr Zoidberg, who sounds like a Jewish doctor but is not a doctor (heh, the intro sequence is amazing) and of course, a shellfish (i.e not Kosher). Even Leela, whose one eye gives her no depth perception which is actually essential to flying an aircraft. Add a rich intern, a rastafarian accountant, a shatner clone in short underpants, a robot devil (Beezelbot), a killer Santa - stop laughing !

Cultural Leakages: Bender's "bite my shiny metal ass" has moved out the world of cartoons into being a real cult phrase. For instance, if you do a reverse dns lookup of the pirate bay servers you'll get bite.my.shiny.metal.ass in the domain name. I occasionally run into even more obscure Futurama references such as "What ? My mother was a saint !" in response to utter gibberish. Even the "Why worry about this planet ? It's not like this is the only one we got" makes me smile, although wryly.

Episodes: If I had to pick a couple of favourite episodes. I'd definitely pick Spanish Fry as the one I liked best. The concept of poaching humans for their human horn truly draws parallels to the african rhino situation. The Godfellas episode is also thought provoking - especially the last few minutes. And the section about greenhouse effect in the Crimes of the Hot truly belongs in an educational film - especially the silencing of the questioning child. There are touching episodes as well, like The Sting or The Luck of the Fryrish - both of which have heart-tugging endings.

Language: Every good Sci-Fi sequence requires its own language. Futurama doesn't just stop at using new words like Blernsball or Slurm, but also goes onto redefine some old words like XMas which is pronounced as it is written and nobody remembers the Christ part of it. They even went further to invent a whole new alphabet to put alien messages in. Here's how it actually looks - I wonder if I can get a t-shirt printed (but nobody except Futurama fans will get it).

All in all, a thoroughly enjoyable but pleasantly thought-provoking animated series for educated adults.

You can't go faster than the speed of light.
That's why scientists increased the speed of light in 2208.
                -- Futurama, A Clone of My Own

posted at: 23:27 | path: /movies | permalink | Tags: ,

X11 programming is a b*tch. The little code I've written for dotgnu using libX11 must've damaged my brain more than second-hand smoke and caffeine overdoses put together. So, when someone asked for a quick program to look at the X11 window and report pixel modifications my immediate response was "Don't do X11". But saying that without offering a solution didn't sound too appealing, so I digged around a bit with other ways to hook into display code.

RFB: Remote Frame Buffer is the client-server protocol for VNC. So, to steal some code, I pulled down pyvnc2swf. But while reading that I had a slight revelation - inserting my own listeners into its event-loop looked nearly trivial. The library is very well written and has very little code in the protocol layer which assumes the original intention (i.e making screencasts). Here's how my code looks.

class VideoStream:
    def paint_frame(self, (images, othertags, cursor_info)):
    def next_frame(self):

class VideoInfo:
    def set_defaults(self, w, h):

converter = RFBStreamConverter(VideoInfo(), VideoStream(), debug=1)
client = RFBNetworkClient("", 5900, converter)


Listening to X11 updates from a real display is that simple. The updates are periodic and the fps can be set to something sane like 2 or 3. The image data is raw ARGB with region info, which makes it really simple to watch a particular window. The VNC server (like x11vnc) takes care of all the XDamage detection and polling the screen for incremental updates with that - none of that cruft needs to live in your code.

Take a look at rfbdump.py for the complete implementation - it is hardcoded to work with a localhost vnc server, but it should work across the network just fine.

You can observe a lot just by watching.
                -- Yogi Berra

posted at: 19:45 | path: /hacks | permalink | Tags: , ,

Thu, 05 Apr 2007:

In one of my previous posts, I'd commented off-hand that the Indian F/OSS community doesn't have enough rockstars. But that by itself blossomed out into a rather heated debate on #linux-india, mainly because we never got to hear lawgon's wisdom on what actually is lacking in India. The debate was more tilted towards the effects of the so called "rockstars" than their origins, causes and well ... mating habits (Freud made me do it !).

But before you actually read this blog entry, I'd advise you to watch Kathy Sierra's talk from LCA '07. You can download it from here [101 Mb]. I'll be borrowing some of her terms and ideas because they talk about how people get involved and become passionate users.

Burn Through the Zone: Success is often a matter of persistance. Most people hit the "Why Bother" phase in the first few weeks of trying something new. But what keeps the persistant folks going is the knowledge of an attainable goal, a sort of beautiful picture of "what could be". The so-called "rockstar" of the community is on such a pedestal of achievement. Bereft of such an example, there will be hundreds who hit their first snag and quit - people who are capable, but don't see it worth their effort. Or maybe they just ran into the "I suck" zone, in Kathy Sierra's words (or well, pictures).

The rewards for being good at something have to be obvious and evident for people to try their best - therefore they work hard and succeed - which is a circular argument from the outside. After all only an idiot would set himself on a mission with no goal and idiots aren't our target audience here.

Emulation Mode: The biggest problem people have with this concept is that a rockstar lends hiself to emulation, producing fad-followers rather than future leaders. But the whole basis of human culture and learning has been mankind's ability to recognize a good thing when it sees it and of course, to imitate by whatever means available. You learn more by doing than seeing and the obvious way to gap that bridge is to attempt what the other idiot/genius is doing.

The right people will split out of pure emulation mode very quickly, as they realize their innate urge to do their own thing. And in any case, people who can follow in a clear (albeit beaten) path are still valuable to any community. I personally prefer them over the self-propelled idiots :)

"Coolness" factor: During the formative years of your life, for a large number of reasons, you do what's cool. The urge to stand out or blend in, as the need be, is something which primeval and probably the conflict of which is the essential misery of man. The effect of the rockstar (who by definiton is cool), is to add an aura of coolness to the act of contributing to something. But for some strange reason, coolness is unacheivable in a group of peers.

The rockstar himself is part of the coolness ying-yang as well. In the real world, without a suitable audience to shower admiration, the hacker has nothing to aspire to but some obscure achievement in a world of peers who would rather play down your work compared to theirs. I think jace had called it the Great Wall of 'So What ?' - where anything you did can be dismissed by these two magic words.

The hard part of being a rockstar is not to put up such walls when someone new comes into the community. Trivializing someone's work is hardly a great way to welcome someone into a group and can be perceived as an outright dismissal of someone's hard work. And indeed it does happen to every other developer, at some point or the other.

Honestly, half the "Because I can" people are into it because it's cool ;)

Beacons: In a community with an insanely large number of potential contributors, it is nearly idiotic to try to seperate the chaff from the grain by brute-force. A rockstar by this definition is an evangelist by action and a touch-point by reaction. A prominent figure outside his or her area of action attracts a lot of potential talent who can then be nudged towards potential mentors who have the time & talent, but not the visibility.

Such rockstars, who inspire/guide/find contributors are required for any community. They are the glue that holds together the gears that drive the community (oh, I kid ... I kid). They are like tiger in the jungle, their visibility & influence indirectly showcases the community - to those outside the community.

Communities grow anchored to such people - their visibility and the ease with which they can handle that is a valuable asset to the community. But it is possible to overdo it as well - you know the examples ...

The key to building a superstar is to keep their mouth shut.
                              -- - Bob Ezrin, rock music producer

posted at: 06:40 | path: /observations | permalink | Tags: , , ,

APC 3.0.14 (code named "A bigger boy made me do it, sir") went out a couple of days ago - read the release announcements. The major things in the release is a fair bit of performance improvements for those don't use threads. Also I've figured out a quick way to limit memory fragmentation when APC user cache (apc_fetch/_store) is heavily used - the new fraglimit fixes should solve all the small fragment issues with 3.0.13. And following my recent obsession with drawing pretty graphs for everything, here's how the old version looks compared to the latest code (requests per second for an include_once benchmark).

To get to such levels of performance, the code has some configuration parameters that can be set. The apc.localcache creates a process (yes, not thread) specific lockless cache which is basically a layered shadow cache ontop of the same shm data. The apc.include_override_once is also now usable because of the appropriate checks put in to reduce the overhead of include_once. And now, when you enable apc.stat there's a bit of code which pre-computes the path of the included file so that it can be effective for includes with relative paths or from include_path dirs.

The release is hopefully stable enough to provide someone with enough ramp-up time to get started, if I stop working full-time on APC. I've spent a fair bit of time stabilizing basic functionality and have kept most of these optimisations optional, to be able to look at other work for a while.

Periods of productive stability, interrupted by bursts of test-bed change is much less disruptive than constant ripples of change.
              -- Fred Brooks Jr, "The Mythical Man Month"

posted at: 04:11 | path: /php | permalink | Tags: , ,

Wed, 04 Apr 2007:

I don't have flash on my machines. More than a mere security and convenience measure, it is one of those things enforced by Adobe themselves - by refusing to ship an EM64T/AMD64 build of its mozilla plugins. So when the flickr organizr went Javascript I was happy. But they took away a bit of code which made it really easy to rotate images - because you couldn't do it in Javascript.

But you can. I don't mean with some memory hogging clientside bit twiddling but with the now popular HTML 5 Canvas. So, with a few lines of Greasemonkey code, which you can pull from here, I can now push in image rotate previews back into flickr's organizr. The code has to be run outside greasemonkey land to get full access to the dom data, which I accomplish with the following script insertion.

var _s = document.createElement("script");
_s.appendChild(document.createTextNode(window.myFun.toSource() + "();"));

And just in case you happen to be an IE user, you might want to see if EXCanvas can make my canvas code work decently there.

enhance, v.:
   To tamper with an image, usually to its detriment

posted at: 02:15 | path: /hacks | permalink | Tags: , , ,

Fri, 30 Mar 2007:

So, there I was debugging what looked like a memory leak in APC - a perfectly straightforward bug at first glance. APC's internal allocator was leaving around a bunch of 40 byte fragments all over the place. The fragments were literally killing APC allocate and deallocate performance - with nearly 85k fragments lying around in the 128Mb cache that www.php.net uses. Even though the allocator is a first-fit based system, it still has to traverse a large number of blocks to locate the previous free block to free any particular allocated block.

Basically, it was having serious issues with memory performance. This had something to do with one of the changes I'd put into APC-3.0.13 - canary checks. The canary essentially increased the memory header size by one size_t exactly. This broke the default word alignment on x86, but I thought I had all bases covered when I put in the approriate word aligns.

24 Bytes: Now, the default allocation size in APC is 24 bytes on x86. That is 12 bytes (3 x sizeof(size_t)) plus padding to make it a multiple of 8 coming to a total of 16 bytes. Then put in the data area (say, 1 byte), which is padded up to 8 bytes. Add all of it together and the smallest block APC can allocate is 24 bytes.

Due to some strange quirk of code, 40 bytes seems to be a very unpopular size to allocate. The allocations for 17-24 bytes of data goes into the 40 byte block and for some strange reason that seems really rare. I ran through a bunch of the standard tests I run with APC to get some sane statistics out of it. After running through hundred odd random tests from the standard phpt files, I got a pile of data. Twenty minutes later I had pulled that data together into a rough histogram (which is nearly the same thing as a bar chart for discrete data, I suppose) by printing out SVG and applying styling in inkscape.

Maybe I'm just hooked into drawing pretty graphs. But it clearly indicates what is going wrong. There are not enough 40 byte allocations to consume all the spare chunks being created. But, is this not true for the 80 or 96 blocks, you might ask. Unlike the 40 byte block both 80 (32 + 48) and 96 (48 + 48) byte blocks are easily consumed by requirements for smaller blocks. The 40 byte block on the other hand cannot be split into any smaller block because it is smaller than 24x2.

Thus due to the lack of demand and the inabilty to compromise (*heh*), the 40 byte blocks remain unwilling to accept any commitments. Until a memory block nearby is free'd the block will sit around waiting for someone to allocate 40 bytes - which as the pretty graph shows, is not a popular choice.

Now to sleep on this problem and hope I wake up with a solution - clear and perfect.

If it breaks then you get to keep both pieces.
          -- Warranty disclaimer for the chat program.

posted at: 07:27 | path: /php | permalink | Tags: , ,

Thu, 29 Mar 2007:

Immediately after announcing the Y! mail unlimited storage, the webservices api for Y! mail has also been announced. The API is loosely modelled over NNTP and IMAP (and when I say loosely, I mean the designers read both specs *heh*) and has some really interesting features. But more importantly, now you can do cool things with it.

About 4 months back, one of the mail backend developers, Ryan Kennedy, visited Bangalore to talk about the internal workings of this awesome API. I'd gotten slightly interested because there was talk about a JSON based API which looked a lot easier to use from Javascript land. And when the hack day came around, I had managed to hack up a pretty decent Y! mail reader interface using XUL, which I named Tapestry.

Most of the XUL code is pulled out of Thunderbird code and a large amout of the UI is controlled by CSS. The XUL css selectors are really funky - take a closer look at my css for how the different styles for messages (read, unread, replied) is css based rather than with code in Javascript. Also I played around with image slicing with CSS to put all my toolbar images into a single image and using rectangle clips to use them in appropriate buttons. In short, I had a lot of fun learning stuff to write it. But the problem was that having done it, I couldn't really show it to anyone outside the company - but now I can.

But before the demo, let me quote my bloody stupid threading code which I wrote in under twenty minutes, which unlike jwz's threading algorithm, mine handles only In-Reply-To based mail threads. But the cool part is that this function is sort of "re-entrant", so calling it multiple times from async response code manages to simulate threading as an when a message is fetched - not having to wait for all the messages to load up.

Folder.prototype.sort = function() {
    for (var i=0; i<this.msglist.length; i++) {
        var msg = this.msglist[i];
        var parent = null;
            if(msg.parentid && (parent = this.msgidmap[msg.parentid]))
                msg.parent = parent;

I don't want to attract too much attention to the hack, because of some hosting issues. So if you'd really like to see it in action and have a Y! mail beta account & run firefox 1.5/2.0, keep reading.

Ryan had hosted an in-colo mirror of my hack - it might be slow to load the images because those are on-demand and not JS pre-fetched. It is my initial release and a lot of buttons and menus don't work there. Not much has been done on top of this, but the minimum functionality works and you should probably scroll through with the keyboard which is something I *really* need. I'm sure the layout code could do with a bit of work, especially on widescreen monitors - but it was something I did for fun. The code should prove interesting to anybody who wants to read it, because I've tried a few new things with javascript and generally that has come out really well.

Always do it right. This will gratify some people and astonish the rest.
                -- Mark Twain

posted at: 18:45 | path: /hacks | permalink | Tags: , , ,

Wed, 28 Mar 2007:

Really, I need a word for this concept. ESR might have gotten his just desserts with such efforts as Everybody loves Eric Raymond, but I still need a word to describe ex-programmers who have looped into their own brain-stem for feedback.

The urge to find a suitable word (in the spirit of The Meaning of Liff) came from reading Miguel De Icaza talking about Mono. I've been watching that project from its very beginnings and I've not been impressed by the way Miguel deals with anyone who doesn't toe the party line. Slashdot comments put it a bit more bluntly than I'll care to be.

A Sharp# Divide: I was quite saddened when I read jdub's blog post today about the gtk# ribbon SoC submission. Building new composite controls in C# basically makes the widget useless for other languages as it cannot be wrapped easily into C, python, java or $insert_language. But it does make sense for Miguel to suggest it as because it makes people want to use Gtk# over something like java-gnome or php-gtk (*cough*).

Wrapper Hell: Now, if you look back at the history of mono, you'll see that avoiding the multitude of hand-coded wrappers was the so called rationale behind mono in the first place. Read his 2001 interview and the part about "Making Programming Interfaces Language-Independent". Now pull yourself back to today and look at Jdub's blog post. I can see why Miguel is losing traction with the rest of the community, albeit after 5 years or so of work on Mono.

A GPL'd Java: With the emergence of Java as a proper f/oss VM subsystem, I don't think there is any need to introduce an unacceptable risk in the form of mono. When you combine a decent fork-mode java VM implementation (i.e start VM, JIT compile classpath, listen on domain socket, fork for each application to reduce memory footprint with multiple VMs), with a bunch of decent wrappers for C-based GTK widgets - you can create a non-sucky Java user experience. I don't see the point in pulling along mono in a world with Java, python and whatnot to develop gnome applications (read last three words again till the point is taken).

Honestly, I do have a huge axe to grind about Mono in general (because of my brushes with lupus and miguel while working on dotgnu) - but the project is doing some decent work in re-implementing .NET system libraries, which is in fact a good thing. My only beef is with pushing mono into gnome and peddling it as a target platform for F/OSS applications ( tomboy, f-spot, muine etc...).

So, like I said ... I need a new word.

You'd better smile when they watch you, smile like you're in control.
                -- Smile, "Was (Not Was)"

posted at: 12:12 | path: /rants | permalink | Tags: ,

Mon, 26 Mar 2007:

This had to be documented, like the other offtopic masterpeices. It all starts from a mail which had a typo - "Vacancy for 1/2 girls" which evolved into a discussion about dinosaurs (don't ask). But it eventually ended up with Dinoman and spo0nman in discussion.

I know Kung-Fu: Now throw yourself back to the Matrix, the first ever episode [1] - spo0nman as Neo. dinoman as Tank and me as Morpheus.

spo0nman  : Reg-ex? I'm going to learn Reg-ex?... Holy shit.
dinoman   : Hey Mikey, I think he likes it. How about some more?
spo0nman  : Hell yes. Hell yeah.
t3rmin4t0r: How is he?
dinoman   : Ten hours straight. He's a machine.
spo0nman  : I know Regular expressions.
t3rmin4t0r: Show me.

Volley #2: Now, that alone wouldn't classify as an offtopic masterpeice. So here's the second wave with even more - in the matrix gym.

Sometime later, after t3 beats the cwap out of spo0nman,

dinoman   : There is no spoon, really.
t3rmin4t0r: How did I beat you?
spo0nman  : You were too fast.
t3rmin4t0r: Do you think my beating you has anything to do with perl
            in this place?
spo0nman  : ....

t3rmin4t0r: Again... 

And I for one, welcome our matrix spoofing overlords ... *bows*

I suppose if we couldn't laugh at things that don't make sense, we couldn't react to a lot of life.
                    -- Hobbes

posted at: 17:54 | path: /yblr | permalink | Tags: , ,

Fri, 23 Mar 2007:

Every company has its own unique attitude. Now, I lament a bit about some stuff in Yahoo! I never got to experience - its fun side. I did catch the tail end of this phenomenon, but here's a blast from the past about how different the times were in the last millenium.

This is one of the cards from the Yahoo!-o-poly which was given out as the year end gifts in '99. There is a fair bit of company history hidden inside those images like the Say 'No' to AOL card or the reply to all card.

Somewhere along the way, that sense of humour seems to have been replaced with suit & coat seriousness. Maybe not with the old folks, but every hire you make you dilutes the culture if you fail to inculcate the same in him - which requires slow growth and mainly of fresh talent. A crowd of new joinees in their thirties is impossible to subvert to local cultural standards.

But let me say this - mourn not for what some never had.

Some changes are so slow, you don't notice them;
Others are so fast, they don't notice you.

posted at: 04:14 | path: /yblr | permalink | Tags: , , ,

So it has finally happened - a stable release of dotgnu's Portable.NET and Libjit, after nearly a year and a bit more. The critical thing about this release is that Rhys doesn't yet know about this yet. This is the first release we've done without rhysw's involvement - he's been last sighted in Greephone land. So rather than the usual place, you can get the packages from the savannah download area.

This is by far the most revolutionary release dotgnu has had. The entire x86 engine is now fully native JITted. The windows.forms has taken quite a lot of work. My special congratulations to Aleksey Demakov and Klaus Triechel - who have worked especially hard of late to get libJIT into the shape it is today.

And in other news, we've put up a couple of Google SoC ideas at gnu.org. I'm very much interested in people picking up the AMD64/ARMv4 libjit core, either individually or as a team (does GOOG allow that ?). Also great strides being made in WinForms land, Radek has just managed to get the SharpDevelop editor to run.

The SD code uses a fair bit of PInvoke code bits to draw the caret and such, which is totally disabled in this rebuild. But it is quite an achievement at this point to be able to run a GUI component of this complexity.

Awesome work all around folks !

Time is the most valuable thing a man can spend.
                -- Theophrastus

posted at: 03:58 | path: /dotgnu | permalink | Tags: , ,

Tue, 20 Mar 2007:

One of the first thoughts which crossed my mind when I was writing Migration Patterns of Codemonkeys was how ill-prepared I was for such a migration, at that point. I had decided that I would fix that deficiency as soon as things settled down. But during the tumultous times that followed, that TODO item has lain un-noticed in the pile of higher priorities. But no more.

So yesterday, my resume has risen on stepping-stones of its dead self to higher things. In keeping with the whole semantic markup fad, I've tried to keep to the hResume microformat while peicing together this. The presentation - both print and screen - is all done in css.

The content has been slightly trimmed to fit into a single A4 page - but it does indeed provide a big picture view of my non-achievements over the last 6 years or so.

The closest to perfection a person ever comes is when he fills out a job application form.
                -- Stanley J. Randall

posted at: 14:44 | path: /misc | permalink | Tags: ,

Thu, 15 Mar 2007:

For every other php programmer who reads Rasmus's no framework mvc, these following lines are what they often finally remember.

3. Fast
    * Avoid include_once and require_once
    * Use APC and apc_store/apc_fetch for caching data that rarely changes

Eventhough include_once has its performance hit, some people avoid it by some rather simple code borrowed from their C experience. Here is how the code looks in general.

if(defined(__FILE__)) return;

define(__FILE__, true);

This is nearly identical to what you would use in a C header to prevent inclusion checks. But as the emergence of precompiled headers shows, even those folks are trying to reduce the expenditure of including & pre-processing the same file multiple times.

I do not deny that the include check above works. But it checks for double inclusion during execution, which is exactly what was wrong with include_once as well. Even worse, it hits APC really badly. But the situation takes a bit of context to understand - let us pick a 'real' library fubar (name changed to keep my job) which has been avoiding include_once. Here is how the logical dependency graph looks like :

In a moment of madness, you decide to make all includes properly for design coherence, especially for that doxygen output to look purty. But instead of using include_once as a sane man should, you remember the wisdom of elders and proceed to do includes. And then kick it up a notch with inclusion checks as illustrated above. But this what Zend (and by design APC, too) actually compiles up.

The nodes marked in red are actually never used because of the inclusion checks, but they are compiled and installed. Zend pollutes the function table and class table for such with a bunch of mangled names for each function - APC serves up a local copy of the same cached file for multiple inclusions, which all have the same mangled name - by ignoring redeclaration errors.

If you were using include_once, these files would have never been compiled. But the above solution *seems* to work in APC land, but in reality does not play very nice at high cache loads. And while debugging *cough* fubar, I ran into a very corner case mismatch issue.

During a cache slam or expunge - when the cache is being written to by one process, other processes do not hit the cache and fall back to zend compile calls. Now imagine such a cache fail happening mid-way in one request.

Now the executor has two types of opcode streams to deal with, one which is Zend fresh ! and one which is from the APC (Opcodes in a Can) freezer. Even though only a couple of opcodes in the normal opcodes stream is executed, the pre-execution phase of installing classes and functions in their respective tables runs into issues unknown thanks to early binding and late binding combinations, which was behind that bug from hell in class inhertiance. But more annoyingly, I cannot reproduce them in ideal testing conditions - wasting about two nights of my sleep in the process.

So I implore, beg and plead - please do not write code like this to avoid include_once, it just makes it slower, heavier on your memory footprint and increases cache lock contention. At least don't do it in the name of performance - I wrote this blog entry just because the guy who wrote fubar said "I didn't know it worked this way". There are a bunch of other such gotchas, which is currently my talk proposal for OSCON '07.

And just out of curiousity, I'm wondering whether an apc.always_include_once might help such code. But on the other hand I hate optimising for bad code, much cleaner to drop such files from cache - after all "they don't the deserve the performance".

So, trust me when I say this ... leave include checks to the experts !

Too much is more than enough by definition.

posted at: 18:03 | path: /php | permalink | Tags: , ,

Fri, 09 Mar 2007:

In the dim dark past of October 2005, spo0nman and teemus did a hack in office. It was a tiny bit of javascript magic combined with a lot of heavy PNG maps and individually marking each cube's position on the map. It was the beginning of so many other nights when we did hacks - even weekends. But that's irrelevant. What is relevant is that to avoid slashdotting my puny little desktop, we had to distribute our load across all the machines - someone asked me to explain how we did that, quite recently and it needs a quick mention outside at least.

RewriteMap: Introducing to you, a small bit of hidden magic inside the apache mod_rewrite module - something which gets just a passing mention in the manual. You can define a RewriteMap which is a program - a full fledged script in any language you please.

RewriteMap    lb      prg:/usr/local/bin/lb.pl

RewriteRule   ^/floors/(\?.*)?$ ${lb:$1}  [R]

The script is not invoked for every request, but an instance of the script is kept alive and feed one line at a time. And because spo0nman wrote it, it ended up being a perl script. Here's the entire perl code.

#!/usr/bin/perl -w

$| = 1;

my @urls = qw (http://m1 http://m2 ...)

$cnt = 0;
while () {
        $cnt = (($cnt+1) % 3);
        my $mirror = $urls[$cnt];
        print "$mirror\n";

Now, the beauty of this is not in this code. The perl real beauty that came out of this was something which generates and rewrites dynamically without restarting apache - with a db or just about any data source you can code up.

Imagine a script which watches your access log to accumulate statistics. Now combine that with a script which wget's frequently hit URLs into a local file. And then imagine a perl script which does a stat() on that file and does an internal redirect to static pages if the file is recent enough (blog archives *cough*).

Implementing something like this into S9Y would make a lot of sense - the gain of hitting a static file would be a LOT better than using something like APC. Would make sense for someone like lunatech who still gets heavy hits on his comment disallowed archives but still uses php to serve out the pages (at least, they seem static, yet send out a X-Blog: Serendipity header). And rather than statically rendering all the pages (like I do), a hack like this could let you do only pages which get n+ requests per day or something and clean up on a cron with stat.atime values.

Hope someone reads this blog and saves me all that work ;)

Nothing is impossible for the man who doesn't have to do it himself.
                -- A.H. Weiler

posted at: 07:44 | path: /hacks | permalink | Tags: , ,

Being a psuedo-security guy of sorts, I'd decided to jump back into insecurity land a couple of weeks back. I haven't really been into security-tech for quite a long time, having hung up my script kiddie slingshot a long time back. But of late, it has again started to look attractive - but more than mere implementation issues, I've been looking for true blue design issues.

Recently on IRC, dumbhead was defending his default password on his router, which is conveniently firewalled off from the WAN. In my attempts to prove that setup insecure, I discovered DNS Pinning. It has been truly enlightening to perform a cursory attack on a home router with a faked up nameserver (re-used my twisted.names code).

The first request immediately does an iframe with a made-up hostname to ensure that no dns caches interfere. The resolution of that host (say "mordor") looks somewhat like this.

;mordor.                     IN      A

mordor.              284     IN      A       xxx.xxx.xx.xx
mordor.              284     IN      A

Now there is a good probability that the first IP will be hit nearly immediately by the browser. The server is running a script which tails the access log as soon as that vhost is hit (for dynamic vhosts, install lighttpd-mod-mysql-vhost), marking the vhost in the table as "hit". A sudo'd python script hooks into the mysql table, flips that flag to "block" after running an iptables packet drop on dst port 80, src ip of the victim.

Thirty seconds after loading the first iframe, the code in there creates another iframe with src=document.location+"?xyz". Very soon, that frame loads up in the same domain as the attacking website. I've got a default exploit sequence, which opens up port 22 for the Huawei WA1003A router which BSNL is distributing these days - but this requires the default password to be unchanged.

But the default password might not be required with the more expensive routers. If I could run my first evil iframe on port 2869 to commit the b0rkage, I would essentially be able to access the UPnP which takes a bit of SOAP to reduce the NAT into swiss cheese. But I'm a bit too lazy to actually write out those SOAP calls using XmlHttp (hah, same domain). And all that *without* a single password.

Most people have dismissed the DNS feature as unusable for hacking most websites because it sends invalid Host: headers and without cookies. But none of the routers I've looked at so far care about that or even have javascript checks for iframes (not that it will affect XHR much).

Amazingly simple, elegant. And DNS has been around for only 20 years ...

PS: thanks for providing the domain - you know who you are :)

The Domain Name Server (DNS) is the Achilles heel of the Web.
                      -- Sir Tim Berners-Lee

posted at: 04:23 | path: /insecurity | permalink | Tags: , ,

Sun, 04 Mar 2007:

The time I've been at Yahoo! has been the best couple of years of my life so far. Sure, I've had my share of problems - some with work and some otherwise - but who hasn't ? The thing I truly treasure about these years are the sweet sweet memories. There's a lot that happened which I'll remember for a long long time to come - because I've changed from that other guy who joined-up two years ago and that's no accident.

It wasn't the smooth ride it looks like in hindsight. I admit it, there were all sorts of days in there - good and bad. But most of what I remember is time spent with friends - premshree, spo0nman, teemus, sabiokap, hitesh, sid and aathitude. Hanging out in the cafeteria, playing pool, the 4th floor balcony at 3 AM`.

In fact, you could totally blame my current sanity on pool. That pool table has been a place where I could go to just to get away from the computer. More than just playing the game, there was a point when I got more than usually good at it. The friday evening pool sessions were my time to shine and I generally came back upto level for every game I lost over the week. There is something about sending in your status report that makes you really kick ass in pool.

And then there were the crazy hours. Coming in to office just in time to complain about the buffet lunch and leaving just in time to complain about the early morning traffic. But these days, when I feel like working, I just stay at home and concentrate on working rather than head out somewhere where there's an near infinite supply of friends and coffee.

But after two years, it doesn't quite have the same small company feel it had. I had decided that I'd stick around for at least two years - no matter what. That is done.

As the walrus said, "The time has come to talk of many things" ...

PS: Happy 12th birthday Yahoo! ...

How do you expect me to remember your birthday when you never look any older?
                                          -- The Diplomatic Husband

posted at: 06:52 | path: /yblr | permalink | Tags: , ,

Fri, 02 Mar 2007:

Kerala's IT Mission is sponsoring FOSS Meet @ NITC, which is going to be kicked off tomorrow by the honourable Chief Minister of Kerala. I'd talked there last year and was planning to just drop by there on my way back to Bangalore.

Had to cancel that trip, mainly out of laziness - travelling 10 hours in a train to reach a strange town with nobody to pick me up and no accomodation confirmed isn't my idea of a fun weekend. Was planning to catch up with some folks from around India, who have talks at this event - doing that once a year at FOSS.IN isn't quite enough. But as Mr Pradeepto said, this isn't the only conference - and it isn't as if I've got something useful to add to this conference.

There were a couple of talks I wanted to attend - especially lawgon's "What went wrong with FOSS movement in INDIA ?". I suspect that this has something to do with the what is stopping indian contributors thread which hit nearly all LUG lists in India.

I think the real question asked in the mail is Why are there are no FOSS *Rockstars* from India - people who are larger than life (taj might qualify). And there are FOSS contributors to be found India, but as shres put it bluntly - "contributors are hard to find" - the whiteboard at foss.in is proof that there are enough.

If there is indeed a drought of new comers to the FOSS world, the general argument is that "You're not looking at the right place" to invest time and resources. For example, second tier colleges, with lack-lusture cookie cutter graduate programmes, produce more FOSS contributors than premier institutions (like the IITs). But I've had my own theories about why that happens and why it happened a lot during the post-Y2K bust. So I think that investing into these colleges in a big way might in fact upset the process that is producing hackers today.

So, I've got a feeling that this talk would be well worth attending - to throw more light on what's going wrong and to look at the solutions suggested. I'd be really interested in helping with any practical solution to the contribute-or-mentor problem that I've been fighting for a while (do you mentor or do you contribute directly ?). So if anybody attends it, please blog about it.

I've got fond memories of last year - preparing the slides at the last minute, walking about in that big campus, spending time at Calicut beach. We had a lot of fun and I mean, A LOT. And the conference is really good, the kids do use Fedora on the desktop in their labs. The XGL demos were a particular attraction last year.

Maybe next year.

The best way to avoid responsibility is to say, "I've got responsibilities."

posted at: 02:44 | path: /conferences | permalink | Tags: ,

Thu, 01 Mar 2007:

Some people are born managers, some others acquire that talent over the years and then there are those who have it thrust upon them. But everybody's still gotta manage, play the game with the hand they've been dealt with and not all of them will make good poker players. From my shuffled set of manager cards, let me therefore deal out a card which has been played so often that, it is pointless to attempt a bluff. But first, some setting and scenario - from the receiving end of the card.

Case of the Mondays: Imagine working in a team of ten odd people. For some strange reason your manager seems to insist on the entire team showing up at 9 'O clock sharp, just like factory workers everywhere - except without the benefit of a siren to warn them. Except there are two free-thinking hippies who still show up at work at 11, with bleary eyes as if they've been working all night long - and maybe they have. The first clue that the management (no, it is no longer in singular) is displeased comes from an email, similar to the following with appropriate padding.

From: manager@company
To: team-world-wide@ 
Subject: Punctuality and Official Timings

We as a company ... blah ... blah ... customer ... blah
time is money ... waste no time ... read this long mail 
... with care and precision ... key aspects ... morale
and motivation ... blah blah

Of late, it has been noticed that people ... you know who
you are ... yada yada ... engineers show up late ... as 
late as lunch ... must encourage team work ... on mondays

99 bugs on the product ... Take one for the team ... 
pass it all around ... make sacrifices for the team ... 
ask not what the team can do for you but, ...

So, please ensure sync ... and conf-calls with onsite ... 
by coming in time every day.

Thanking you,
Your Neighbourhood PHB

No Names: A politely worded mail, which in the manager's opinion conveys the essence of his complaint in clarity. But such mails accomplish two things. First it fails to totally point out who's wrong and who's not. This indicates to those in the wrong that the manager is non-confrontational and is more likely to snipe from afar than come out in a melee. And by not naming any names, the manager assumes that the people responsible will know and take action. But by denying personal criticism, they are blocking off the employee's response in advance. There is no way for the employees who have incurred the displeasure to broach this topic and explain in person - without appearing petty. They might have a very good reason for coming in late every day - a conf call at 10 PM every night for instance.

Authority Erosion: Secondly and more importantly, such a public announcement erodes into your authority and trust from the other employees. When they see your orders disobeyed with impunity (yes, when ... not if), you are literally letting your targets eat into your authority, while building their own pseudo-authority as a rebel - especially if they are still good at their job. To give an appropriate analogy, it is indeed hard to keep faith in this world, when certain people aren't instantaneously hit by lightning, even out of stormy skies. In short, it pisses off more people who keep to the lines, when you send such a mail which gets an unconscious "Yeah, but what can you do ?" response.

I call this manoeuvre the Group Therapist, where someone having run out of his power and authority, tries to turn the peer pressure screws and essentially try to shame people into obeying. And sometimes it works, mainly because people are sheep. Nothing bad happens if it works, but as mentioned above, problems get worse if it doesn't work - especially over a prolonged period.

Grapevine: But private direct criticism can still work as a deterrant for others. The office grapevine is strong and long enough to actually leak what was said in that sound proof conference room - and the weird thing about people is that they believe hearsay more than an official memo. Gossip in general giving no advantage to the producer other than the thrill of being "in the know", while official communiques aren't viewed with such pink-tinted glasses - Making the water cooler conversations more effective at communication than any office memo ever sent. Strange, but logical.

If you've read Migration Patterns of Codemonkeys or Performance Inversion, you'd already know that my my bitter well of cynicism holds no answers. But they present some facts, pose a couple of questions and the rest has been left as an exercise to the reader :)

Long gone are the days when personal shame caused you to take personal responsibility.
              -- Linus Torvalds

posted at: 06:46 | path: /observations | permalink | Tags: , ,

Tue, 27 Feb 2007:

After a roller coaster career, my first php extension has hit php cvs, with all the memory leaks settled, nearly all the features done and all promises kept. Hidef, as it was originally known acquires a new and improved slogan "Constants for real" as well as a place in the PECL packages. If you want to install the module, just run this simple command and have the binary built & dropped into your php install - except for the extension=hidef.so line.

bash# pecl install "channel://pecl.php.net/hidef-0.1.0"

If this actually makes a significant difference in your code's performance, I'd say that you've done a wonderful job otherwise. Most of the code is stolen off a template Rasmus had and packaging was thanks to Pierre ... which leaves me with a distinct feeling of having put a few legos together - but to my credit, at least the peices fit.

We demand rigidly defined areas of doubt and uncertainty!
                -- Vroomfondel, H2G2

posted at: 01:27 | path: /php | permalink | Tags: , ,

Mon, 26 Feb 2007:

APC released version 3.0.13. The last couple of months haven't produced too much code from me, so most of the changes in there are due to the efforts of shire & rasmus. But I've left a couple of booby traps in there for invalid free() calls, which should reduce a decent number of those random memory corruptions into a more decent error report.

I've been unsuccessful in making the Real World go away for long enough to actually rewrite the shm allocator - not even a patch job with a linked list, rather than the mythical lockless allocator I've been promising for three months.

I feel guilty, but there's so much to be done and I'm only ... *counts* ... one man.

After months of careful refrigeration, Debian 2.0 is finally cool enough to release.
          -- topic on #debian

posted at: 02:27 | path: /php | permalink | Tags: , ,

Wed, 14 Feb 2007:

*DRUMROLL* I'm proud to announce the first commercial product to use dotgnu Portable.NET in a big way. Trumpf.de have brought out a pulse laser cutting tool which uses dotgnu. The embedded control device runs linux and uses dotgnu to power its control UI, running a Windows.Forms application on the touch screen. Here's how the embedded appliance looks like - isn't it purty ?

It is one of those days that you live for - when you can stand back and marvel at what has become of bits you twiddled. And congratulations to klausT, avd and radekp for supporting Trumpf. Of course, much heartier thanks to Trumpf for believing in our code, being a good customer and investing a good ten thousand dollars or more into the project, even more when you consider the number of patches from them directly. That kind of customer focus has essentially revived pnet from the ashes of its past into a new good groove of embedded goodness.

If want, you can take a look at the official brochure, but which doesn't mention anything about linux or dotgnu, sadly. There's more related stuff upcoming - I'm trying to get ARM support resurrected with the help of Pengutronix folks.

A solid year's worth of work has gone into pnet thanks to this and the next stable release is only four days away. This is the first release to include libjit support - adding a full x86 JIT to the engine core. The twin team of Klaus and Alesky have reworked the engine so much that I've been having trouble jumping back into the project without asking too many stupid questions. So, look out world, here comes 0.8.0.

Been a good five years - looking back.

You know, I have one simple request. And that is to have sharks with frickin' laser beams attached to their heads!
                -- Dr Evil, Austin Powers

posted at: 09:14 | path: /dotgnu | permalink | Tags: , ,

Tue, 13 Feb 2007:

You've got to hand it to Steve Jobs. Moves he in mysterious ways, his wonders to perform. I've ranked him as a master of mystery and hype for years, but never realizing what other depths of talent he might posses. I'm still digesting the lessons I learned off Machiavelli's classics (Art of War, lies at my desk, bookmark a few pages in), but it doesn't take a genius to marvel at the Machiavellian motions of this spin doctor.

Let's get our facts straight first. Steve Jobs is the CEO of one Apple "fruit company" Inc. His company makes and sells the iPod, which nearly monopolizes the digital audio player market. Now, in a very loose-laced attempt to follow the Gillette Razor Blade approach to customers, they have a nice little store which sells songs to any customer who might wish buy them.

Now, to curb piracy and pacify the rabid record industry watch-dogs (popularly known as the MAFIAA), Apple introduced a DRM solution named FairPlay. The system works and is lenient enough to keep people from breaking it accidentally, which lets the average joe ignore the fact that it exists. The iTunes Music Store however, has protections clauses in its contracts with the recording industry, which render the contract null & void if the DRM is broken and Apple can't fix it within a specified period (a fortnight).

So, as of date, iTMS cannot sell songs which would play on one of my MP3 players. Nor, can they relicense their FairPlay codecs to other vendors - because they are liable for all disclosures - accidental or not. So let us sprinkle some iMagic sparkles and see what happens.

The Pledge: Steve Jobs goes out there and proclaims - I wish for a world without DRM - really, that's what most reports about his open letter convey. Now, why would a man who has benifited the most from the status quo advocate such a step which might break his own monopoly ? The cynic in me, suggests - because he could profit more and in the short term, too. But selling un-DRM'd songs to users of other mp3 players doesn't look like something worth risking your credibility, like this - especially not when the iPod is the king.

The Turn: The Music Industry will NOT say yes. Warner music has already said that Mr Jobs' proposal is "without logic and merit". It would be idiotic of His Steveness to assume the industry would suddenly develop a conscience overnight. But they too want iTunes to sell more songs, even to people who don't have iPods - as it turns out this thought was spelled out in the open letter - because that's what Steve really wants too. Mysteriously, suddenly all the executives smell money in the air and they want it !

The Prestige: The open letter details another alternative - licensing FairPlay to current and future competitors, which isn't technically impossible but rendered practically impossible due to the protection clauses in the contracts with the music publishers. Having led their greed to this obvious alternative, when they suggest it (as their choice) I suspect Apple is about to ask them to rewrite that bit of the contract.

If Steve Jobs had asked the MAFIAA to reconsider their contracts in January, they'd have definitely smiled (like the smile that follows seals and has a fin attached) and asked for a cut off every iPod sold. But February is much warmer for Apple - where they are in a win-win situation. If the industry doesn't let Apple go drm-less (a near impossibility, that) - Apple come crying back to us, "They never let us have any fun !". If they rewrite the contract, letting Apple hawk their DRM, more money for Steve - oh, much much much more money than the extra DRM-free iTMS purchases would fetch. And just in case, they manage to go DRM-free, they'll have a new crowd knocking at their door - not to mention all the kudos for fighting those evil corporations for our rights.

All those options are good for Apple - but for anybody who buys from iTMS, only the last option is any good - Apple: 3, you: 1. But you've got to appreciate style, precision and direction of this so called "attack on DRM". After all, a best defence is a good offence.

Some lesson in negotiation that, woohoo. *But* - if it was that transparent to me - could it still work with RIAA ?

"No" is often the first word in every negotiation.

posted at: 06:46 | path: /observations | permalink | Tags: , ,

Sun, 11 Feb 2007:

Oh, how often has that tired old line been repeated. But now, there are a lucky few in Yahoo! who can back it up with a certificate, which looks something like this.

The card is given out once you take a trust fall, where the other guy/gal catches you while you fall and push you onto the soft padding. The card, on the flip side, has more content and it goes like this.

I've got the expertise to make the right decision here. So let me. 

Once you accept this card, you agree to avoid second-guessing me, 
escalating the situation, or switching on the blamethrower. 

If it works out, great. If not, I agree to take a fall.

Oh, and it says on the bottom - may not be transferred, sold or substituted for U.S dollars - maybe I should try paying it with some real geek money, like pizzas, beers or t-shirts.

Cheesy, cheap and corporate - but the designers had their hearts in the right place. And If you liked it, take a look at Yahoo! Bangalore's campus brochure.

Who does not trust enough will not be trusted.
                -- Lao Tsu

posted at: 09:52 | path: /yblr | permalink | Tags: , ,

Divisiveness is never something to be encouraged in any community - online or offline. Regionalism is one of those toys for those first crude attempts of Salami Tactics. The introduction of imaginary lines, dividing where no division existed, pulling in foreign boundaries into an otherwise level ground is one of the first seeds of discord that you can seed a community with - especially where physical meetups occur out of band. People who've met each other physically generally interact better and this can be easily misinterpreted as a regional or language bias.

God knows there have been wars fought over much less. And there was one on #linux-india because I got involved - with me getting most of the abuse. But here's some history behind the story - some idiot was spamming #linux-india in 2005 (*some names changed to protect the guilty (and/or innocents)).

**** Dec 13th 2005

*   vIkSiT has changed the topic to: crazy day at #linux-india the ban 
    list doubled in a matter of minutes || /bin/flame > /dev/chill

<lunatech>    vIkSiT: some people thought that the ops (i.e me) 
               were too leniant
<lunatech>    in not kicking people out at slightest provocation
<t3rmin4t0r>  slightest ?
<lunatech>    that was a major reason why I did not give ops to $culprit
<lunatech>    t3rmin4t0r: after the things spiralled out - he was 
              kicked out
<vIkSiT>      why?
<lunatech>    vIkSiT: why to which statement ?
<vIkSiT>      he was kicking people because they were flooding? :)
<lunatech>    no, he was asking for ops, but I did nit give it --
              he started ranting about how the channel was own3d 
              by north indians
<vIkSiT>      he actually SAID that?
<lunatech>    hehe - yes
<vIkSiT>      good god

Now, fast forward to present day - when things have not changed much, as the events would prove. But right now, it is quiet out in the west frontier.

** Chapter #1 - where tazz is a power hungry maniac ... NOT !

<tazz>         Lap_64, i would suggest you do a '/msg chanserv access 
               #linux-india list'
<techno_freak> tazz, for ?
<tazz>         for the good of the channel
<$culprit>     50 members - vow
<tazz>         so that you can see that my name is not in there and
               when the ops here ask woh wantes to be an op here 
               you can vote for me.
<tazz>         comeon Lap_64 i am just a power hungry maniac
<$culprit>     tazz are you from north of the vindhyas?
<tazz>         no i am from the west of the mumbais
<$culprit>     tazz: then you have no chance of becoming an op
<tazz>         hey i have dreams and hopes you know

Now, I've got a good memory - a very good memory. It is not like I've never been sour grapes about being denied anything - but sooner of later you've got to stop throwing tantrums and grow up. I'm nobody's personal shrink, but the above conversation drew a picture of someone pouring in tiny blue drops of bitterness (everyone knows jealousy comes in green) into the common well of wisdom (*narf*) that is #linux-india.

** Chapter #2 - t3rmin4t0r acquires magical op powers

*   ChanServ gives channel operator status to t3rmin4t0r
<tazz>        i wanna be op!, i wanna be op! :-S
*   t3rmin4t0r has kicked $culprit from #linux-india (I need to be born 
    exactly where ... ?)
*   t3rmin4t0r removes channel operator status from t3rmin4t0r
<brainless>   what did $culprit do ? !
<t3rmin4t0r>  heh, he said that you need to be born north of 
              the vindhyas to be an op here

There are a bunch of people in this world, whom you could respect, if only they'd shut up for a while - ESR comes to mind, immediately. These are people who are not only senior to you in age, but in terms of age in the community - people whom you expect to be calm, mature and mellowed. But there are often exceptions which prove the rule (or compensate for it) - even people twice my age.

** Chapter #3 - $culprit unleases his insults 

<techno_freak> omg, $culprit is saying he wont return back
<segphault>    techno_freak: I am damn sure he will

*   $culprit (n=$culprit@xxx.xxx.xxx.xxx) has joined #linux-india
<techno_freak> ahh
*   techno_freak grins at segphault

<$culprit>     t3rmin4t0r: in my opinion you are an immature rascal 
               unfit to be in this channel
<$culprit>     or any channel
<$culprit>     op is a priveldge given to people deemed to be mature 
               enough to protect a channel

*   segphault feels like nostradamus

<$culprit>     unfortunately at times ops are given to petty minded
               people with a chip on their shoulders who like to 
               flaunt their authority
<tazz>         t3rmin4t0r, was harsh in kicking $culprit out.
<$culprit>     not  harsh - a juvenile display of his unfitness to 
               be an op

<$culprit>     why the hell did they make a madrassi an op?
<segphault>    $culprit: t3rmin4t0r is a veg mallu - madrassis come in 
               different flavours -- like sambar
<brainless>    $culprit, well maybe they are not well represented in 
               the op clan ?
<segphault>    I propose a quota system to fix this issue of ops 
<$culprit>     but no more madrassis - and t3rmin4t0r should be de-opped

Funny, isn't it that the same man who was b*tching about people from south india not getting ops is complaining about some madrassi being an op. Yes, the thing I like about these conversations is their consistency of content. But either way, I'm at the pool table (@office) for the period of that conversation - which leaves enough room for other conversations.

** Chapter #4 - "I am codier than thou" quoth the raven

<$culprit>    that woman is a girl who has not contributed a line of code 
              to foss - like her friend empty_mind
<empty_mind>  $culprit, she has contributed far more to Creativecommons 
              than anyone else
*   floyd_n_milan doesn't understand why only `code' is considered as 

<empty_mind>  $culprit, <blank> is not a computer geek she is a writer
<$culprit>    empty_mind: tell her to stick to writing
<empty_mind>  $culprit, it is stupid people like you who piss away serious 
<empty_mind>  $culprit, she is sticking to writig and contributing more 
              than you have contributed
<$culprit>    what happened to that mallu  clown t3rmin4t0r ?

After the random quarter of the continent insults, it starts to get personal - so I'm putting that part off the RSS feed. Let those who want to dig into the steamy underside of IRC life in #linux-india click the link.

** Chapter #5 - insults get personal

<t3rmin4t0r>    what else can I talk about @ a college ?
<empty_mind>    t3rmin4t0r, loads of things
<t3rmin4t0r>    "Do as I say, not as I do" is so hypocritical
<$culprit>      $culprit kicks t3rmin4t0r
<$culprit>      t3rmin4t0r: is the ultimate hypocripte
<t3rmin4t0r>    $culprit: yeah, I took the Hypocrite's Oath
<t3rmin4t0r>    no run of the mill hypocrite, I am
<$culprit>      t3rmin4t0r: you are a cheap asshole
<t3rmin4t0r>    $culprit: exactly how am i an asshole ?
*   empty_mind watches $culprit's cheap rant again

<t3rmin4t0r>    for good, return good
<$culprit>      in every way
<t3rmin4t0r>    for evil, return justice
<t3rmin4t0r>    so if you think I'm a cheap asshole ... let me not be
                the one to judge
<empty_mind>    t3rmin4t0r, "return NULL " to $culprit
<t3rmin4t0r>    $culprit: I still don't understand what you've got against me 
                or anyone else in this channel
<t3rmin4t0r>    but what goes around, comes around 
<t3rmin4t0r>    if you can't deal with it, don't dish out and so on

*   HereBeDragon didn't get the feeling that $culprit has got anything 
    against anyone
<$culprit>      except that mallu asshhole
 *   empty_mind thinks $culprit forgets he is not in a court room
<$culprit>      t3rmin4t0r: why did you kick me from the channel?
<t3rmin4t0r>    because you said something idiotic about people's birth 
                places and channel ops
<t3rmin4t0r>    now, if you can't press /rejoin and take a joke ... I 
                don't know 
<t3rmin4t0r>    is your problem, I guess
<t3rmin4t0r>    drag politics into this channel and start saying 
                regionalistic things about "above the vindhyas"
<$culprit>      t3rmin4t0r: has licked bihari ass and got power and 
                showing it off
<t3rmin4t0r>    just to stir things up, I don't know what a decent ops 
                guy might do
<$culprit>      t3rmin4t0r: you are a sick son of a bitch
<HereBeDragon>  whoa $culprit, let's not be regionalistic
<t3rmin4t0r>    or personal ad hominem attacks
<Lap_64>        $culprit, may be he licked OldMonks arse

<HereBeDragon>  gawd

<HereBeDragon>  folks, no personal attacks or regionalism. Period. 
                That should be the rule.
<$culprit>      anyway i am leaving this channel - until that asshole 
                t3rmin4t0r  is kicked out
<empty_mind>    t3rmin4t0r, kick $culprit again for personal abuse
<t3rmin4t0r>    empty_mind: nah, I don't really take irc that seriously
<Lap_64>        $culprit, chill out 
*   $culprit (n=$culprit@ has left #linux-india

The insults were so bad that we actually suspected whether it was $culprit himself or just someone else pretending to be him. The IP turned out the be a Bangalore IP, while last known location of $culprit was Chennai. Anyway, I don't really care whether it was the real one or an impostor, that kick was warranted as some conversation later proved - people pickup these biases quite easily.

** Chapter #6 - the influence remains 

<$kid>           Oldmonk is true bihari
<HereBeDragon>   $kid: what about Biharis?
<$kid>           HereBeDragon, lots of things
<HereBeDragon>   $kid: "no regionalism" you get it. Or else I'll 
                recommend you be banned.
<$kid>           HereBeDragon, can list them all here for a real example
                see any bihari talk to him for 5 mins and you will know

So that was friday on #linux-india. I hope I haven't made an enemy out of $culprit, he doesn't quite deserve my enimity - but just as a hint, mellow out, old timer. There are so many better things in life to do than mud sling my teflon coated soul - maybe get yourself a copy of the 70's Show DvD and watch The Cat Fight Club episode.

Why don't those stupid idiots let me in their crappy club for jerks?
            -- Homer Simpson (#2F09, "Homer The Great")

posted at: 06:42 | path: /rants | permalink | Tags: , ,

So as it turns out Linux Asia '07 was sponsored by Microsoft and Novell. Even before this was announced, the local LUG - ILUG-D decided that they would boycott the event, and they went through with that decision. But what was to follow is probably competing for the longest single flamefest ILUG-D has ever seen - no holds barred. But the IRC conversations are getting more and more interesting.

<OldMonk> spo0nman: i have very high regard for his intellect
<OldMonk> didn't you see the subtle and suave way in which he insulted me in this channel?
<spo0nman> OldMonk: you stink of sarcasm.

<sm|CPU> Feel the power, know the wisdom
<t3rmin4t0r> "sudo intellectual" (n):
<t3rmin4t0r>       intellecutal whose facts are true because of who he is (or his uid)

<OldMonk> t3rmin4t0r: it strenghthens your argument if you blast your own side first
<OldMonk> after that you can be as rude as you like and no once can take offense

IMHO, except for the lone exception of bluesmoon, there were no other Indian speakers of note amongs the speaker panel. I think bluesmoon could've made a stand in the situation - but LinuxAsia paid for his flights (heh, more miles) and there are enough people in Delhi to meet. Amidst all the flame wars, Sudev seems to keeping his head - being clear about his worries.

The thread has spread to other mailing lists - where opinions differ from Freedom (capital F) of participation to Freedom to boycott.

I'd given LinuxAsia a wide berth because it has a definite CxO trend to it and is really a media circus, run by real media companies, with a definite slant towards appeasing sponsors (alright, explain away the Google Maps Api talk on day 2).

I really wish Freedel was happening this year in Feb, like it was suggested amidst the heat of September, last year.

All diplomacy is a continuation of war by other means.
                -- Chou En Lai

posted at: 03:42 | path: /rants | permalink | Tags: , ,

Fri, 09 Feb 2007:

Consider the past year. Now consider UserFriendly.org's blatant MS bashing of past. And then take a look at the following cartoon.

In the past the Orbital Mind Control Ray could only have been an obvious reference to MSFT (as seen on slashdot). But today, we're making fun of Google - is the attitude towards GOOG changing as the company grows bigger and starts eating small companies for breakfast, lunch and dinner ? Which reminds of this conversation between Bart and Fat Tony from Simpsons 8F03.

Bart: Are you guys crooks?
Fat Tony: Bart.. uhm. Is it it wrong to steal a loaf of bread to feed 
          your starving family?
Bart:     No..
Fat Tony: Well suppose you got a large starving family. Is it wrong to 
          steal a truckload of bread to feed them?
Bart:     Nuh-uh
Fat Tony: And what if your family don't like bread. They like.. cigarettes.
Bart:     I guess that's okay.
Fat Tony: Now, what if instead of giving them away.. you sold them at a 
          price that was practically giving them away. Would that be a 
          crime, Bart?
Bart:     Hell no!

As someone else pointed out, when Microsoft came out, they were the hungry rebels who were freeing the public from the iron fisted data processing overlords, without the cutsey "Ribbons and Ponies" approach that Apple was taking. The big blue of old, which has now become a savior and hero for Linux, was the evil monopoly ?

How often does sides get switched, old rebels become new masters and acquire new allies from old enemies ?

No permanent friends or allies, only permanent interests.
          -- Lord Palmerston

posted at: 11:45 | path: /fun | permalink | Tags: , ,

I suppose it is my own damn fault for not participating. But unlike last year, the hackfest was during daytime, which severly limits my coding abilities. Like so many other people who started hacking during college, my peak hours of coding lie between 7 PM and 11 PM, with the extended version running upto 3 AM - and the muse of coding cannot just turn on & off, on demand. Not to mention my laptop was showing more bad sectors than a minefield in Cambodia.

But what *really* saddens me is the first question titled jail break. I'd discovered this design flaw in chroot() quite a while back and broken out of a real production chroot. On the other hand, I'd have never really made it with the image processing example.

Maybe there was a point - but Hindsight is always 20/20.

Given the choice between accomplishing something and just lying around, I'd rather lie around.
                              -- Eric Clapton

posted at: 10:53 | path: /conferences | permalink | Tags: , ,

Mon, 05 Feb 2007:

Finally got around to getting a debug build of libgphoto2. After a couple of hours of debugging, the problem turned to be one of design rather a real bug. I had to try a fair bit to trace the original error down to the data structure code. This is code from gphoto2-list.h.

#define MAX_ENTRIES 1024

struct _CameraList {
	int  count;
	struct {
		char name  [128];
		char value [128];
	} entry [MAX_ENTRIES];
	int ref_count;

And in the function gp_list_append(), there is no code which can handle possible spills. As it turns out, I had too many photos on my SD card - in one directory. The assumption that a directory contains only 1024 photos was proven to be untrue - for my SD450.

Breakpoint 3, file_list_func (fs=0x522a60, 
    folder=0x5a3660 "/store_00010001/DCIM/190CANON", list=0x2b11e6c38010, 
    data=0x521770, context=0x523d90) at library.c:3933

(gdb) p params->deviceinfo->Model
$2 = "Canon PowerShot SD450"
(gdb) p params->handles
$3 = {n = 1160, Handler = 0x528c90}

So, the code was exiting with a memory error because it ran out of 1024 slots in the folder listing code. When I explained my problems on the #gphoto channel, _Marcus_ immediately told me that I could probably rebuild my gphoto2 after changing MAX_ENTRIES to 2048 - I had already tried and failed with that. As it turns out there are two places which have MAX_ENTRIES defined and even otherwise, the libraries which use gphoto2 have various places which allocate CameraList on the stack with a struct CameraList list;, which introduces a large number of binary compatibility issues with this. But after I rebuilt libgphoto2 and gphoto2, I was able to successfully download all my photos onto my disk using the command line client, though in the process I completed b0rked gthumb.

And you've definitely gotta love the gphoto2 devs - look at this check-in about 15 minutes after my bug report.

The capacity to learn is a gift;
The ability to learn is a skill;
The willingness to learn is a choice.
              -- Swordmasters of Ginaz

posted at: 03:12 | path: /hacks | permalink | Tags: , ,

Sat, 03 Feb 2007:

After nearly a year of messing around with php extensions, I've finally sat down and written a full extension from scratch. I've used all the skeletons and ext_skel scripts, in the proper way to end up with a half-decent extension. It took me around 4 hours from an empty directory to end up with an extension which basically did what I wanted.

hidef: The define() call in php is slow. Previously the workaround to define a large chunk of constants was to use apc_load_constants, which pulled out stuff from the cache, but still had to define all constants for every one of the requests. Even beyond that the value replacement is at runtime, nearly as expensive as a $global['X']. A quick look with vld indicates the problem very clearly.

define('ANSWER', 42);
echo "The answer is ".ANSWER;

	line     #  op           operands
   2     0  SEND_VAL        'ANSWER'
         1  SEND_VAL        42
         2  DO_FCALL        'define', 0
   3     3  FETCH_CONSTANT  ~1, 'ANSWER'
         4  CONCAT          ~2, 'The+answer+is+', ~1
         5  ECHO            ~2

For a lot of code with a lot of defines(), this is a hell of a lot of CPU wasted just putting data in & reading it out, where a substitution would be much better. But first things first, I got a basic extension which would parse a .ini file and define the constant with some magic flags - this is what you'd put into the ini file.

float PIE = 3.14159;
int ANSWER = 42;

The extension reads this once when apache starts up and puts into the php's constants section. The constant is pushed in with the CONST_PERSISTENT flag which means that the constant lives across requests. Recently, Dmitry had put in a new bit into this mix - CONST_CT_SUBST which marks constants as canditates for compile time substitution.

After adding compile-time substitution into the extension code, the code generator replaces constants as & when it runs into them. And here's what the bytecode looks like.

echo "The answer is ".ANSWER;

line     #  op      operands
   2     0  CONCAT  ~0, 'The+answer+is+', 42
         1  ECHO    ~0

You don't need to be a genius to figure out which one would be faster. But the other gopal had done some benchmarks which didn't seem to show enough difference between constants and literals. So, I wrote a quick & dirty benchmark with 320 defines and adding them all up in the next line. Here is the before and after numbers.

Before After
380.785 fetches/sec 930.783 fetches/sec
14.2647 mean msecs/first-response 6.30279 mean msecs/first-response

But the true significance of these few hundred lines of code fades a bit when you pull in APC into the mix. With APC enabled I was still expecting a significant difference in performance and here it is.

Before After
976.29 fetches/sec 1519.38 fetches/sec
4.95603 mean msecs/first-response 3.15688 mean msecs/first-response

The numbers are seriously biased, because for most code the major bottleneck is their DB and therefore I/O bound. But if this small bit of code helps shave off a few microseconds of CPU time, for a few hours of my hacking time, it is pretty good when you consider the scale factor.

So, without further ado - here's hidef 0.0.1 - should build fine for both php5 and php4. And if you feel the urge to fix something in there or write documentation, go for it ! :)

If you don't know what procrastination is just look up the definition tomorrow.

posted at: 02:45 | path: /php | permalink | Tags: , ,

Fri, 02 Feb 2007:

Unlike my trip to LCA last year, which was blessed with some awesome photos, this year I've not put up too many photos yet. But this is not due to the lack of photos taken. I place the entire blame on one little library called libgphoto2 which is refusing to read my photos off my camera.

[gopalv@sirius photodump]$ gphoto2 -L

There are no files in folder '/'.                                              
There are no files in folder '/store_00010001'.
There are no files in folder '/store_00010001/DCIM'.
There are 99 files in folder '/store_00010001/DCIM/189CANON':

#1     IMG_9445.JPG  1982 KB 2592x1944 image/jpeg
#98    IMG_9981.JPG  1704 KB 2592x1944 image/jpeg
#99    IMG_9982.JPG  1705 KB 2592x1944 image/jpeg
*** Error (-3: 'Out of memory') ***       

For debugging messages, please use the --debug option.

From a quick glance, I think my photo collection has hit a unique boundary condition. The photo counter on my camera (which has been heavily used) has touched 10,000 and rolled back to img_0000.jpg into another folder, which is confusing gphoto2 totally.

Philosophy vs Pragmatism: I remember one of those common room conversations about laptops. There was a distinct proliferation of Macs in the room and this particular dilemma tradeoff was thrown into discussion. I think it was Arjen who commented that running Linux shouldn't be an apology for poor performance of hardware - while we were on the topic of OpenWRT and how the conference APs weren't auto-switching off congested RF channels.

I'm at the doorsteps of such a dilemma. One of my friends commented that, if something doesn't work, just drop it and use whatever works. But as a conscentious developer, I'm conflicted between fixing gphoto2 and plugging my camera into someone's Mac (windows ? what do you mean windows ?). I'm pretty sure that one of these weekends I'll pull gphoto2 off SVN and build myself a debug version. But I'm not likely to do that if I actually plug my camera into somebody's Mac and pull my photos out.

But I did manage to upload a few photos, grabbed off the other SD card. Here're the only ones I've managed to upload, so far.

Hmm... this may take a while.

The makers may make
and the users may use,
but the fixers must fix
with but minimal clues.

posted at: 09:29 | path: /hardware | permalink | Tags: , ,

Mon, 29 Jan 2007:

Back in boring old Bangalore. And it is a mess - a bigger mess than usual, when placed against the clinical sterility of Singapore or the warmth of coastal Sydney. Due to the power cut in Bangalore last week, my machine had been powered down and I suspect a couple of the misbehaving machines have had data losses. And APC seems to be acting up on some dual CPU machines. There seems to be a whole Heisenberg & Schrodinger effect to the whole bug - waiting for me to be on vacation before actually exploding into tiny bits. The last time APC had serious issues, I was somewhere in Ladakh.

But generally put, I'm back ! Got a lot of things to blog about, an even larger number of photos to upload. But before all that, I got me some code to write !

The whole object of travel is not to set foot on foreign land;
it is at last to set foot on one's own country as a foreign land.
                      -- G.K. Chesterton

posted at: 21:51 | path: /travels | permalink | Tags: ,

Fri, 19 Jan 2007:

As I sat around in the keynote today, listening to Kathy Sierra go on and on about users, bits of Jono Bacon's and Zonker's talks popped up. You could classify projects as belonging to three categories - scratch your own itch, benign neglect and user centric. Most library and system developers tend to belong to the middle category, where their users are actually developers and the project waddles along according to the core dev's plans.

What that keynote actually did for me was to rationalize the use of some emotional play into a proper strategy targetting users rather than consider that an insult to their intelligence (or rationality). Well, I'd rather term it more generally as acceptance of their sensitivity and essential humanity, now. The three talks combined have given me a bit of things to think about, about the mistakes I've made over the past couple of years in handling dotgnu. Until we threw up the Southern Summer of Code, the project was literally stuck in a rut.

The talk dealt with the suck zone of doing anything, which makes a lot of sense to me. The hardest part to do is the first few bits, which everybody knows. But what was hammered in was the message of hope and of course, pride. The concept of levels and a set path upward for someone who comes in is pretty good, especially for a slightly large community - sort of mirroring Jono's breaking and entering a community comments (well, he didn't say that - but that's sort of what he meant, I think).

Happiness is a compile away !: The other thing that she talked about was about flow and meaning. I can pretty much relate to the first, which I prefer to call deep hack. And to conclude, a bunch of jokes - ending it with a high note after the heart pullers (like the puppy and the baby photos) in the middle. I absolutely loved Chris Blizzard as Volverine - a good movie that was ... oh, wait.

PS: Btw, as for "Goldilocks likes it just right", that's what she said (watch the video, if you don't believe me).

"This isn't right. This isn't even wrong."
        -- Wolfgang Pauli

posted at: 17:54 | path: /conferences | permalink | Tags: , ,

Thu, 18 Jan 2007:

I've said this before and let me say this again - we need to let people play around with stuff. The freedom, stability and all the other advantages of F/OSS do not appeal to the layman - you've got to get him interested by other means, if only to start with. And IMHO that's exactly what LCA Open Day did.

There were people of all ages, sizes and interests around. And quite a lot of interesting stuff, even for seen'em all folks like me. For instance, there was this cool motion detection and tracking tool, projected onto a huge screen. I realized what it was when I walked in and saw a few stars move exactly like I did. As I stepped back, those exploded and were replaced by others tracking the other folks coming in. Perfect eye-candy which can keep people interested in the show (which it really was). But what really caught my eye was the MythTV demos, running on some sw33t spankin' hardware.

I think it can be called a total success in terms of attendance and crowd pulling ability. But it sort of pokes fun of my previous post about LCA not being an evangelism event - I don't mind, really.

The problem of leadership is inevitably: Who will play God?
                                              -- Maud'Dib, Dune

posted at: 17:44 | path: /conferences | permalink | Tags: , ,

I've been asked this question a couple of times today. Whenever I mention that I've flown in from Bangalore to attend this conference, someone does ask again - What brings you here ?. Considering the loopy hoops I've had to jump through to get here - in fact, not having an apartment to go back to in Bangalore - I've asked myself that question more than a few times.

Why LCA ?: There are a lot of conferences around the world. Some of them are commercial, or rather of a more corporate flavour - which basically covers a fair bit of them. Then there are conferences focussed on evangelism, which I'm not really into, can't sit around teaching people stuff. And then there are those organized for users, like our grand old install fests. But in between all these, right there in the sweet spot of developer heaven is LCA - developer stuff and literally nothing else.

Why this year ?: Now, that's a really interesting question. I had to think a lot about that. First up, last year had been awesome - I was totally transformed when I went back to India. I was sort of in a pit, in a lot of ways, somewhat like this year. But that whole week at Dunedin sort of put the spirit back in me and I came back raring to code out all 8"sorts of things - which I did, even slayed that bug from hell.

But three days into the conference, I'm still not even at level zero. It is the little things isn't it ? The laptop I had borrowed from yahoo has a crap hdd and is randomly destroying bits of my data - photos mostly. Basically a b0rked bit of hardware is an irresistable challenge and wasted a fair bit of my time.

And the other fun thing was meeting new people. Except this year, half the people I met were people I already knew from last year. Being split up into multiple hostels doesn't help that and neither does the lack of a common room in New College.

I've still got two days left to go and probably the cricket match on saturday. All of which and the rest of the week of not coding should push me out of this coding lull (no code since mid-nov, zOMG).

*fingers crossed* ...

When in doubt, mumble;
When in trouble, delegate;
When in charge, ponder.
                -- James H. Boren

posted at: 13:44 | path: /conferences | permalink | Tags: , ,

Wed, 17 Jan 2007:

If you wanted to get high in Sydney, there's no other place to go other than the Sydney Tower - and I did on monday evening. The tower is located in downtown Sydney and we took a bus there. The tower itself is bang in the middle of a lot of buildings, which sort of destroy the effect from the ground, but after we got to the top, the tower dwarfs them and clears up the entire skyline for 360 degrees. Since we ended up a little late, we were just in time to see the city lights go up all around us.

After coming down from the tower, we headed out to Hyde park which is just below the tower. Hyde park is named after the one in London and has a beautiful fountain and full of greenery. There were opposums roaming the grounds and had a sanctuary feel to it in the midst of the city.

Eventually, I ended up at the hostel, continuing to bitch about the lack of WiFi there.

I used to get high on life but lately I've built up a resistance.

posted at: 16:14 | path: /travels | permalink | Tags: , ,

Tue, 16 Jan 2007:

First up, this blog has been delayed. The wifi network at the conference hostels weren't up yesterday and I had better things to do than write blog entries in the pavilion.

GOODMORNING FREEDOM LOVERS: Every morning, for all of last week, sometime around morning sydney time (which is 3:30 AM for me), jdub arrives on #linux.conf.au with this message of freedom, love and well, morning.

Only Speakers can heckle: Jdub went into the introduction full on. Starting from the photos of the team - who have managed to juggle with a colossal budget of 500,000 AUD. And with some embarassing pictures, we headed on to tackle the other special things about this LCA - which have nothing to do with each other.

What if distros were sofas ?: And then it got into a bit of distro bashing - with debian and gentoo taking the brunt of it. Debian got a couch without a back, which doesn't give (*wait for it*) much support, but if we as a community all sit together and lean back, we can support each other. And well, gentoo got an IKEA chair.

Embedded miniconf: I attended the Qtopia greephone talk, firstly because Rhysw works on that and partly because I wanted to see how QCop stuff works. The talk was interesting and they actually managed to get the demo off coding live.

Slacked off most of afternoon and headed off downtown in the evening - more on that later.

The clash of ideas is the sound of freedom.

posted at: 14:22 | path: /conferences | permalink | Tags: , ,

Mon, 15 Jan 2007:

I ended up in Sydney on saturday night, with the flight arriving on time. Shehjar and Hal picked me up from the airport and for the night I crashed out at Shejhar's place. I got myself a good night's sleep and woke up late on Sunday.

First up, I went to the New College hostel to dump all my stuff. The hostel has a prison cell feel to it, with white brick walls and my room had a window facing Anzac Parade. Was quite eager to get out of there and headed out immediately to the nearby Coogee beach.

And it was a sunny day, a very very sunny day. After reaching the beach, found it to be really crowded (being a sunday afternoon, not surprised). Went up to the cliffs for a few moments of solitude, sat around up there in the shade till about 6 PM. Went down to the beach, for a quick dip (can never resist beaches).

Went back to New College in the hopes of finding someone headed to the Rocks. But the hostel is not quite geared up to be a social gathering place, unlike the sofa pit we had in Unicol hostel last year. Eventually, got on a bus to Circular Quay, alone.

After nearly an hour of being lost in George Street, I ended up at the restaurant - but being vegetarian and a non-drinker, German food is not quite my thing. And I showed up there quite late in the evening, eventually got back to New College.

But my room was too near the road and the vehicles woke me up way earlier than I would have otherwise - so LCA was a few naps away.

Love the sea? I dote upon it -- from the beach.

posted at: 17:44 | path: /travels | permalink | Tags: , , ,

Sat, 13 Jan 2007:

I got my tickets to Sydney confirmed on Thursday evening, for an extra 200 USD, making it a round 1400 USD. Right now I'm sitting in the lobby of Changi airport, literally chained to the power plug by an office laptop with a b0rked battery. It must've been the lack of sleep, jet lag and the location in combination, but I found myself reminscing about my trip last year (without the aid of my blog).

No, it isn't the talks that I remember about the conference - well, with the possible exception of the Damian Conway keynote. Other than the thrill of my first foreign excursion, the conference didn't quite have the sterility of the average technical conference - wasn't a single lecture in what I attended.

But the people were more interesting than the talks. And I think the 24x5 nature of the conference, because of the hostels, plays a big part in encouraging interaction. And when such a large number of interesting people are thrown together, stuff's bound to happen.

And then there are the conversations - I remember one which started off about Women & FOSS (remember D00d3tt3z on ILUG-d ?), which turned into a discussion comparing unicycle hockey with unicycle polo.

Not to forget the pranks, jokes and cricket.

As someone said - couple more sleeps till lca '07.

A conclusion is the place where you got tired of thinking.

posted at: 05:44 | path: /conferences | permalink | Tags: , , ,

Fri, 12 Jan 2007:

As I headed back from the dentist in an auto, I was surprised to feel that my purse (in my backpocket) was vibrating. Totally puzzled, I took out my purse and just couldn't explain why an auto should vibrate. Somewhere nearer to Brigade Road, I realized that the Hello Moto which kept repeating was actually from a couple of cellphones lying between the seats.

When I flipped open the phone, there were 31 missed calls from someone. I was still not able to speak clearly and eventually picked up the phone after reaching office. First line off the phone - can you come to Forum ?. Then, apparently the girl realized that she should probably try to get the phone without inconvenincing me.

To cut a long story short, She came to my office. I gave her the phone and asked her to buy me coffee for all my trouble - after all, the two phones together were well worth over 40k.

Good deed for the day - done and Done !

I hope you're not pretending to be evil while secretly being good.
That would be dishonest.

posted at: 03:40 | path: /misc | permalink | Tags: ,

Wed, 10 Jan 2007:

Linux.conf.au 2007 kicks off on monday, but even now, I'm not exactly sure whether I'll be attending or not. Initially, I'd written off the conference as out of my grasp. But some recent events have convinced me that life's just too short to say maybe next year and give up.

But having decided I was going, my economic woes remained - I just couldn't afford to do the trip, not on my salary. The flight ticket and conference fee together would eatup this entire quarter's take home pay and then there's that simple need of mortality - food (whether I travelled or not).

No Homers Club: So, I vacated my house in Bangalore. The house advance refund and a bit more would pay for my tickets to Sydney and back. For a couple of weeks, I slept in office. For the first couple of days, crashed on the sofas - I'm used to doing that. There is an air-conditioned room with two beds upstairs and pretty soon, I was a resident Yahoo!. Except for the early mornings, the office had everything I needed - hot water showers, coffee, bandwidth and a pool table. And most of December was spent in Kerala. And recently, thanks to some very good friends, I've been sleeping in real beds.

Credit where 'tis due: Credit might be the axle grease of the economy, but it feels really bad to borrow from friends. But I owe about 40k INR to my friends - which is what is really keeping me afloat right now. In the past, I'd have borrowed from my parents - but now, I cannot bring myself to ask. The long and short of it is that, I'm running on empty.

And then visas. I got my visas without any hassles and they're with me right now. But, tickets - that's another story altogether. My Bangalore to Sydney tickets are waitlisted for the last nine days and there is about 48 hours left before I've got to report for the flight.

Last year's trip also went down to the wire - with that visa arriving around 4+ hours before the check-in. But I wasn't truly in control last time and I'd have dismissed it as fate, if I'd missed that flight.

But this time, if I don't end up going ... I'm not going to sit around.

Lack of money is the root of all evil.
                -- George Bernard Shaw

posted at: 18:42 | path: /conferences | permalink | Tags: , ,

Tue, 09 Jan 2007:

As I walked into my room on Friday, I was planning to sit down and rewrite a fair bit of code in apc codebase. But my hard-disks had other plans for me. Both my 80 gig boot drive and my 200 gig sata drive had conked out, overheated in the middle of winter. The smaller drive had my /home and uncomitted work of about four months. The codebase had its cvs repository on the sata disk, which was thankfully working, though freezing up randomly. After a few hours of trying to copy out the cvsroot with a live CD, I tried to debug with smartctl on the dying drive.

bash# smartctl -d ata -l selftest /dev/sda

SMART Self-test log structure revision number 1
Num  Test_Description  Status            hours   LBA
# 1  Short offline     read failure 90%   2160   288074266
# 2  Extended offline  read failure 90%   2155   290472493
# 3  Short offline     read failure 90%   2155   290472493

bash# smartctl -d ata -A /dev/sda

5 Reallocated_Sector_Ct  Pre-fail  Always  41253

Basically a large number of sectors were bad and even an e2fsck couldn't complete on the disk without hitting the damaged sectors. After a lot of patient fiddling with debugfs, I managed to recover about 2/3rds of my cvsroot - but a fat lot of good that does. But in the process, I learned about a large number of tools available for data recovery, like badblock_guess, which reminds me of my b0rk-copy hacks. But as I said, I got a fair amount of data out of the second disk - but the first one was a total goner, not even spinning up on power-on.

Eventually, I went out, bought a couple of disks to replace these - 160 and 320 gigs, which are both SMART compatible. But to prevent the overheating from recurring, I tried to write my own alerts using inotify to warn me whenever the temperature exceeds limits. But as it turns out somebody has already done it and done it well - sensors-applet + hddtemp.

And this time, I'll be burning backups - I swear !

The primary cause of failure in electrical appliances is an expired warranty.
                     -- Dave Barry

posted at: 02:45 | path: /hardware | permalink | Tags: , ,

Mon, 25 Dec 2006:

Repeat after me, three times - C++ is not C. This is a fact which has to be hammered into every programmer who claims to know C/C++, with a nice clue bat if necessary. But in this case, it was more of g++ isn't gcc and only for those who use RHEL4.

Here's a bit of working code in C99 which is totally different from C89 (otherwise known as your dad's C standard) - which is technically speaking, legal C++ code as well.

/* compile with gcc -std=c99 */

#include <limits.h>
#include <stdio.h>

int main()
  printf("Maximum value for unsigned long long: %llu\n", ULLONG_MAX);

But the exact same code was not working when treated as C++ code. For a few versions of glibc, no matter which C++ standard you used, ULLONG_MAX wouldn't be defined. Not even if the code segment is enveloped in an extern "C" block.

As it turns out, this was a quirk of the glibc's extension to the C pre-processor - include_next. Rather than include the standard /usr/include/limits.h, what the first include statement does is pull the limits.h file from the compiler include files - from /usr/lib/gcc/.... You can figure this out by running g++ -M limit.cpp, which dumps a pre-order traversal of the include hierarchy.

And the definition of ULLONG_MAX was probably written by someone who never expected a compiler include file to be included directly from a user program - and rightly so. Except, there is no real way to fix the include order for such similarly named files.

Eventually, the fix was to use ULONG_LONG_MAX instead of the slightly shorted ULLONG_MAX. But the glibc bug has been fixed for a while - was just not critical enough to be pushed to all machines.

The strictest limits are self-imposed.
            -- House Harkonnen, Frank Herbert.

posted at: 07:12 | path: /hacks | permalink | Tags: , ,

Thu, 07 Dec 2006:

Yeah, I had a father.

And then last saturday, I didn't.

[1] , [2] , [3] , [4] , [5] .

May his soul rest in peace.

The future is interesting because I am going to spend the rest of my life there.

posted at: 21:33 | path: /me | permalink | Tags: , ,

Fri, 01 Dec 2006:

Somebody has attempting a reality distortion by showing off how shiny smart folders are in Mail.App. As novel as that may seem, what I hadn't realized was that a lot of people don't know that the nearly the same functionality has been available in thunderbird for a while. So, here's a cool feature of Thunderbird 1.5 I've been using for quite a few months now (got me a nightly build, thanks to one bug-fix).

If you look at the thunderbird message pane, you'll notice a combo box named View. You can create custom views with its customize option. For example, here's how one of my custom views look like.

Now, there are a lot of cool things about filters and views. One of my favourite options in there is the Age in Days option. Combine that with a List-Id or [Bug filter, you get interesting sources of information for status reports.

But views are a true bitch to use. The view isn't reset when you switch folders and a view set for a bugs folder makes no sense when you switch back to your Inbox. But that's where this killer feature comes in. The second last option in the View drop-down is titled Save view as a folder - and that's exactly what it does.

Voila ! There you have your smart folder - no swapping views or anything, just visit the folder and you've got everything you need. Well, it does more than just filter a folder - you can even aggregate multiple folders, with this feature. And I hear that there is tagging in thunderbird 2.x - making it truly web two point OH! compatible.

And before you ask, I've become a GUI weenie :)

How do I type "for i in *.dvi do xdvi i done" in a GUI?
                -- comp.os.linux.misc

posted at: 19:19 | path: /misc | permalink | Tags: , ,

Thu, 30 Nov 2006:

Eventually having connected my amd64 desktop upto a decent internet connection, I decided to wipe the FC3 install on the box and replace it with ubuntu. I left my i686 gentoo install alive and started out on an Ubuntu 6.06 LTS (aka Dapper Drake) install, sometime around 11 PM.

My hardware setup is not what I would call conventional. Inside my monster tower I have four SATA ports - two of them on the motherboard and a couple more on my RAID controller. The first port is occupied by my 10k RPM 80gig drive which is basically my boot drive. The other onboard port has a 120 gig drive, which is my secure (AES256) storage and is hardly ever mounted. There are two 200 gig SATA drives connected to the RAID controller (Promise), which after a couple of abortive attempts with dmaraid, have ended up as discrete SATA disks instead of in RAID-0.

Now, as soon as the Dapper liveCD boots, the disks on the RAID subsystem show up as sda and sdb. The onboard SATA controller gets relegated to sdc and sdd. I thought it was strange, but I picked the renamed partitions and went through a complete install with this setup. The first reboot just froze the machine with a pretty GRUB on the left corner. Modifications of device.map, replacing the hd1 entries in the boot menu did nothing either. Whatever I tried, the machine just threw up a blank GRUB stage1.

As it turns out the BIOS and grub still recognize the onboard SATA channels as the first pair of hard disks, but half-way through stage2 of the boot the kernel takes a flip and then it all goes south for the summer. Faced with the renaming, I tried to connect the boot drive to the RAID controller, just to bring it back to sda. But as it turns out my BIOS can only boot off onboard primaries. Which all left me up shit creek without a paddle.

Now, dapper is not without its advantages. The live CD installer meant that I could do more than just watch the lines scroll while the installer did its magic (anyone remember the caldera tetris games ?). So I still had a functional internet browser even though my machine wasn't booting (grub ensured that I couldn't boot gentoo).

Thanks to the internet, I came to know that I wasn't alone (*play x-files music*). Turns out somebody had run into this Grub of Death before. Also there lay open a launchpad bug in the helpful status called CONFIRMED, which basically meant I had run into something real (like an iceberg). As luck would have it, someone had already sent a patch to work-around the problem in kernel land - and it had been rejected. All this was no help at all, till I discovered magic beyond mere device names.

UUID mounts: /etc/fstab could use other identifiers than raw device names to uniquely identify drives. And the first patch I discovered dates from 2001 - this must've been the *best kept secret* in linux kernel land. As it turns out, both grub and /etc/fstab will accept volume UUIDs, completely transparently. But still the installer couldn't figure out where to put grub or what to put in it. At around 3 AM, I managed to pry open my machine, pull out the off-board SATA and attempt a reinstall.

The system reinstall went almost perfectly. All that was left to do was to replace the device names in the relevant sections with the correct UUID volume names. The unique identifiers can be read easily using the blkid command. A couple of reboots later, I plugged the RAID controller back in and added those partitions similarly.

It all worked out, eventually. But the amount of effort it took to get it working reminded me of those nights in 1999, spent struggling with an X configuration, with some magazine's help section open.

"First things first -- but not necessarily in that order."
                -- Doctor Who

posted at: 21:45 | path: /hardware | permalink | Tags: , ,

Wed, 29 Nov 2006:

For the past weekend, I've been roaming the corridors of FOSS.in. I've never attended a talk about something I could figure out for myself, but still there were four talks which I sat through - a grand achievement, compared to the meager two talks last year.

But I wasn't in Bangalore for most of last week. I'd gone home to tie up some loose ends and cleanup my act before I dived into the conference. I reached back in Bangalore only early morning of Thursday, just in time to catch up to some sleep. I took a preliminary peek at the venue on thursday itself. The place was hum of activity with the volunteers stuffing delegate kits, falling lampshades and power mayhem.

And then on friday, it all began. As I watched the lines for registration grow, I noticed a general paucity of the number of delegates this year. Since the conference was on a weekend, I had hoped for more students in the crowd, but that was not to be (as far as I could make out). But thanks to the weekend, there was a larger crowd from outside Bangalore and that's always a good thing.

Since I was a care-free delegate, I had planned to do a security BoF at the conference. But when that didn't quite materialize, I gave up on the idea. Backporting a dbus-send shared connection patch was trivial, but was wasted because we couldn't get beryl to flip the windows for the projector display. Except for that random hack, I didn't touch a single line of code during the conference.

The inaugural keynote was by Suparna Bhattacharya, a kernel developer from IBM. As much a fan I'm of minimalistic development, I skipped out early to find a decent cup of coffee instead. My abominable behavior notwithstanding, it is a proud day for the Indian FOSS community to have an Indian on the podium, for a change.

And then I ran into a surprise. One of the dotgnu libjit developers, krokas, had flown in to India and met me in one of the corridors. Even though I'm not a great fan of his business ideas on professional opensource, it was nice to actually meet one of your fellow developers face to face. Met a couple of people from Microsoft too, discussed a bit of C#/.NET technical quirks (GC-able assemblies), dodged the "what'll it take to get you to work at Microsoft" question and kept wading through the crowds.

One of the talks I actually attended, by Andrew Cowie touched up on a few problems the ubquity of mono brings along, especially considering the latest Novell developments. Initially voiced by Seth Nickell, I find it is a valid concern. For instance, I've never suggested dotgnu to anyone who's wanted to write an application from scratch. Anyway, that talk covered all the normal obstacles that developers place in the way of external contributors and in general was interesting enough to keep me from wandering out.

Another one of the talks I dropped into was Get Rich with Php5 talk by Rasmus. I knew from the slides that it featured my work for the last year - APC. But the content of the talk was still interesting, especially the benchmark numbers for each increment.

The BoF tents looked more interesting than last year. But the initial lack of chairs killed a bit of the enthusiasm. Also moving the FOSS Expo area out of the main lobby area made it almost pointless. Before long we had converted the project expo into a live demo for ioquake3 - sweet GPL love from Id. Though Dalfry ruled the roost for the first fragfest, I eventually climbed the levels due to the basic advantage of knowing the maps by heart.

The other couple of talks I dropped into were the panel discussions about 10 years of Linux India and spo0nman's monitoring with nagios talks. The panel discussion was slightly boring, mainly because the panel wasn't split on most of the issues and the crowd lacked the trolls required to make the discussion interesting. And the only question I held in my hands was about the history of PCQLinux, without which half the FOSS folks in India today wouldn't have ever used it - probably the action of greatest consequence in the last 10 years of FOSS in India. But that was skimmed over with a passing reference.

And then there was the cool stuff left - the jokes, the corridor conversations, the conclusions and the arguments. I'll let a few pictures speak for themselves. And maybe this video (mp4, 12.1 Mb).

But let me calm some fears expressed in the discussions. The growth of FOSS in India has been fairly organic for the hobbyist category, while the explosion of user mass worldwide has pulled up the number of programmers getting paid to write FOSS. The community entries in this game has still retained single digit significance in the last year as it has for the past years - but that is not something to get alarmed about, because we've now got employees contributing to FOSS. Things could be a little better - but that's always the case.

All in all, a good conference. But I've got more plans for next year. Rather than mere BoFs, we should be able to run a mini-conf in the middle of a conference with the speakers present. I suspect that it might be possible with a bit of fine-tuning in the event in real time, rather than organizing a whole parallel set of talks. But next year is nearly a year away ;)

People who go to conferences to talk are the ones who shouldn't.

posted at: 04:12 | path: /conferences | permalink | Tags: , ,

Mon, 27 Nov 2006:

Today afternoon, Yahoo! will pay for your coffee.

Just make sure you land up at Taj Westend, to listen to eminent Yahoo! speakers like Sumeet B. Mulani, Rasmus Lerdorf, Philip Tellis and Gopalrathnam Venkateshan (not me, the other gopal). The talks will cover the breadth of frontend technology Yahoo! uses, from PHP into the YUI libs, JSON and other fnuk.

Make sure you show up and drink in the talks ... uhhh.

When we talk of tomorrow, the gods laugh.

posted at: 12:13 | path: /yblr | permalink | Tags: ,

Thu, 23 Nov 2006:

T minus 20-odd hours left for FOSS.IN and if you aren't going to show up, you don't know what you're missing. This isn't just one technical conference by itself; it is sort of an excuse to meet up for a couple of un-conferences.

While talks dazzle the audience in the big halls, the crowds milling around in the corridors run their own version of foss.in, swapping stories, discussing the last year's developments. Largely unorganized, a large majority of the indian foss community meet up, touch bases and in general have a lot of fun. And eventually something like this gets discussed.

So prepare ...

I'm prepared for all emergencies but totally unprepared for everyday life.

posted at: 14:42 | path: /conferences | permalink | Tags: ,

Mon, 20 Nov 2006:

I've been playing around with twisted for a while. It is an excellent framework to write protocol servers in python. I was mostly interested in writing a homebrew DNS server with the framework, something which could run plugin modules to add features like statistical analysis of common typos in domain names and eventually writing up something which would fix typos, like what opendns does.

To my surprise, twisted already came with a DNS server - twisted.names. And apparently, this was feature compatible with what I wanted to do - except that there was a distinct lack of documentation to go with it.

7 hours and a few coffees later, I had myself a decent solution. Shouldn't have taken that long, really - but I was lost in all that dynamically typed polymorphism.

from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
from twisted.protocols import dns
from twisted.names import client, server

class SpelDnsReolver(client.Resolver):
    def filterAnswers(self, message):
        if message.trunc:
            return self.queryTCP(message.queries).addCallback(self.filterAnswers)
            if(len(message.answers) == 0):
                query = message.queries[0]
                # code to do a dns rewrite
                return self.queryUDP(<alternative>).addCallback(self.filterAnswers)
        return (message.answers, message.authority, message.additional)

verbosity = 0
resolver = SpelDnsReolver(servers=[('', 53)])
f = server.DNSServerFactory(clients=[resolver], verbose=verbosity)
p = dns.DNSDatagramProtocol(f)
f.noisy = p.noisy = verbosity

reactor.listenUDP(53, p)
reactor.listenTCP(53, f)

That's the entire code (well, excluding the rewrite sections). Should I even bother to explain how the code works ? It turned out to be so childishly simple, that I feel beaten to the punch by the twisted framework. To actually run it in server mode, you can start it with twistd -y speldns.py and you have your own DNS server !

In conclusion, I hope I have grossed a few of you out by trying to do soundex checks on dns sub-domains.

DNS is not a directory service.
         -- Paul Vixie

posted at: 04:45 | path: /hacks | permalink | Tags: , ,

First, you should go read The Peanut Butter Manifesto. Also get a copy of Jerry Maguire and refresh your memory of the first half-hour of the movie.

The manifesto ostensibly wants to roll back the clock and go back into the past. I'm doubtful whether it is possible to achieve this, mainly because of a few laws of thermopeople-dynamics. Reducing the head count by 20% is not going to improve performance and all you might end up doing is to give the rest a persecution complex and possibly increase their workload. And all that after admitting to bad decisions in investment distribution.

Dilbert Oct 1994 - buy the book !

Then the wage strawman is throw in, always available as a convenient reason for low productivity.

Moreover, our compensation systems don't align to our overall success. 
Weak performers that have been around for years are rewarded. And 
many of our top performers aren't adequately recognized for their 

Pay packages are generally a function of the years of experience. A 10 year experienced employee churning out run-of-the-mill code will get paid more than a 2+ guy writing top of the line code. The pay parity that the company keeps is indirectly unfair and enocurages you to be as stupid & worthless as you can be, yet earn more than the hard-working junior.

Those are the rules of the game and some of us have come to accept that as inevitable. But when you hear rumours (in Economic Times, no less) that fresh campus recruits are going to be paid at par, maybe a bit more than what you are earning, the quoted paragraph makes no sense.

As a result, the employees that we really need to stay (leaders, 
risk-takers, innovators, passionate) become discouraged and leave.

Well, duh ! If they didn't leave, they'd hardly qualify as people with initiative. I've put down my take on that, a while back. That bit of wisdom from a senior VP, vindicates my personal feelings (or does it prove that I'm not a risk-taking passionate innovator, because I'm still sticking around ?). And maybe ... just maybe toolz had a point.

dilbert 1996 - buy the book !

All in all, I assume that the author of this Manifesto wrote it out of pure good-will, frustration and naivete, as a last call to turn the company around. For once, I'd like to think the dilbert cartoon above is completely fictional and bears no resemblance to any company, outside of a cartoon.

Either way, que sera, sera.

Give me the strength to change what I can, the inability to accept what I can't and the incapacity to tell the difference.
                      -- Calvin prays,

posted at: 04:20 | path: /yblr | permalink | Tags: , ,

Sat, 18 Nov 2006:

There are no innocents in this war. To not have an opinion is treachery to a greater cause - for it needs nothing more for evil to thrive than good men to stay out of the battle. But when the battlelines are a line in sand, unclear & transient, only a vocal minority survives a twinge dissent in a dystopia which rewards disloyalty with a jingling bag.

The squeaking wheel always gets the grease, but does a wheel squeak for all others bereft of lubricant ? But there are wheels within wheels, connected cogs running this juggernaut that fulfils our needs, wishes and aspirations. This isn't a zero sum game, where everyone else has to lose.

But silence is golden. In fact, it will be bought with gold, spices and precious stones. The turncoats reap the profits of their new found discretion and the immorally inept, failing to curb their conscience, experience a re-run of Mr Carrot meets Mr Stick.

Like in the game in its simplest form, the winners always defect.

It occurred to me that my speech or my silence, indeed any action of mine, would be a mere futility.
                -- Joseph Conrad, "Heart of Darkness"

posted at: 02:17 | path: /rants | permalink | Tags: , ,

Fri, 17 Nov 2006:

I've been a great fan of Terry Pratchett ever since the day I started reading the Discworld novels. Rather than raw fiction, it is philosophy, social commentary and a sprinkling of cynicism, which makes his books so enjoyable. The last in the line of books is Thud! (read it to get the Him diamond quote) started to read the Science of Discworld and sort of looking around for Wintersmith. So just out of curiousity, I started counting the Pratchetts I've read and as it turns out, I've read 42 of them. Here's a quick list of what I've read (in no particular order). If you liked Douglas Adams and wanted him to be more Tolkien, while making fun of the modern world - pick up a Pratchett, any one.

Ach, crivens !, wish'e could squiggle a neece ending to all yen tales - ye ken ?.

[A Human] ... is merely a means for a book to create more books.
          -- Daniel Dennet

posted at: 04:42 | path: /books | permalink | Tags: , ,

Wed, 08 Nov 2006:

Recently, I've seen a lot of serious photographers start to watermark their images. I'm not one of them (yet), but I hacked up a quick script to watermark a photo in gimp without much fuss. Basically the script lets you paste a transparent image on your image and without actually using a UI.

To install, copy the watermark.py to your ~/gimp/plug-ins/ and chmod +x it. And to use in batch mode, you'd probably do something like this :-

gimp -i -b \
   '(python-fu-batch-watermark 1 
	  "x.jpg" "watermark.png" "y.jpg" "rb" 33.0)' \
'(gimp-quit 0)'

A more sophisticated invocation would avoid spawning a new gimp instance for every image edit and would truly operate in batch mode.

(for each in images/*.jpg; do
    NEWNAME="`echo $each | sed "s/\.jpg$/_wm&/"`"
    echo "(python-fu-batch-watermark 1 \"$each\"" \
            "\"watermark.png\"" \
            "\"$NEWNAME\" \"rb\" 33.0)"
echo "(gimp-quit 0)") | tee /dev/stderr | gimp -i -b -

Now that truly shows the power of small bits put together, with the odd bug thrown in for good measure.

Great acts are made up of small deeds.

posted at: 05:23 | path: /hacks | permalink | Tags: ,

Sat, 04 Nov 2006:

Very little is known about the Life and Mating Habits of the common Code Monkey. But that should come as no surprise to anyone who has observed a specimen in the cubicles of Asia. But observations from cubesville have often been indicative of a certain flux in the population - a trend to migrate over longer distances. Unlike the famed lemming of the north artic tundra, which takes a downhill (to say the least) approach, this migration is more often in search for higher ground. Is there some herd mentality to it or is it merely an individual moving on ? Join me, as we dig deeper into the mysterious world of the code monkey.

Of late, I've been feeling the urge to leave Yahoo! and go do something else. I couldn't explain exactly why, because I probably have the best job imaginable - work on what I want, from wherever I want, a couple of meetings a month and play pool all afternoon. But the urge was still as strong as ever. It needed a rational explanation and I started to itemize and categorize the possible reasons as objectively as I could. A couple of recent discussions on slashdot and india-gii have added fuel to that fire and then I read this.

Career Phases: In general, the company you work for is really really important for your first couple of jobs. It should come as no surprise that junior engineers want to work with a strong brand. This sets them up to move onto be senior engineers in places which pay better. Experienced engineers are not too desperate to seek out things which look good on their resumes - they're interested in other aspects of the job than how good it will look on their resume in a couple of years.

There's a flip side to the career phases argument too. If you've read the Discworld series by Terry Pratchett, you might remember a class of wizards known as the Sourcerors. Now that nearly all magical spells have been formed out of the raw magic, the discworld needs no more sourcerors.

Now, software companies are like miniature discworlds. There is a phase in the company's life cycle (it is a cycle, it repeats) when the ground is fertile for new ideas. And this age of miracles, attracts the brilliant miracle worker who can shape reality around such ideas. But such proto-geniuses have little to do in the adolescence of the company. As the work force muscle builds up, the concept of a superstar engineer dies and slowly but surely, the emphasis shifts to overall throughput of a team than individual brilliance. When a manager (and his team) can outperform any individual engineer at the same task, the company needs the best managers they can get, rather than a couple more brilliant code monkeys.

People outgrow companies and vice versa.

Wages: Money is like air. A little bit more doesn't do much when you have enough, but you'll know it when you are running short. But there does exist a certain stress level beyond which people do not think it is worth their pay to work - but a large percentage of salaried workers never approach that limit. That is where an interesting economic hypothesis pops up - Efficiency wage hypothesis. Even when you get paid enough, merely the fact that you are paid does not induce any sort of gratitude or loyalty towards your employer - it is money in return for services rendered. But as the shirking model in the theory indicates, often you do get what you pay for.

Pay hikes are often nominal and are significant only when you are promoted. Meanwhile people being freshly hired are being hired at pay scales more in terms with the market demands. Over a period of two or three years, the difference between your hiked pay and your peers being freshly hired climbs to a significant value to prompt you to get back to level. Most companies to refuse to raise the pay of a long standing employee to the levels of a freshly hired of similar level, giving various excuses - the most general of which go - who told you that ? That's not true. And in this world of salary confidentiality, rather than counter that argument, you'd probably try to get a better pay package elsewhere - and you probably will.

Raises don't match lateral entry pay packages.

Vertical Space: Most indian companies don't have a good technical ladder. In an industry where company half lives are measured in years, waiting around for those above to retire is hardly any option. Generally the way to climb the ladder is to move somewhere where you're closer to the top and work towards building the rest of the ladder downwards. In other words, the easiest way to get promoted is to be somewhere small and grow along with it - after all companies don't need to be promoted (*sic*) to grow. But some are unfortunate enough to end up pushed down as the management brings in fresh talent to supplement the growth. This is somewhat in line with the career phases argument where fast growth, high risk approaches are suitable after the initial few years of establishing a brand pedigree.

Last one out is a...: Community is a usually disregarded factor when considering job hopping. But the web of friends can outweigh some of the advantages a job hop might bring. More important would be your relationship with your direct boss. Having to work under a new person and develop the same rapport takes quite some effort and the average asocial code monkey dreads the thought of having to go through that *again*. On the other hand, this explains how much more precious a manager can be, because when he leaves he disturbs the general inertia of his direct reportees.

The moment the community at work starts breaking up, be it due to overwork or attrition, the downsides of leaving start to diminish. Eventually, you'll be able to point out a single departure which snowballed into a mass attrition throughout the company - even if everyone went in their own directions. At some point, a $n people can't be wrong correlation turns into a causation going into a spiral of departures.

Attrition turns into super-attrition when a strong community breaks up.

I've sort of started to understand why people need to switch jobs, about every couple of years - not out of greed or disloyalty - but as suggested by pure common sense. But a closer inspection of the scenario does indeed suggest that status quo is an assumption - a valid assumption for vast majority. That could change, I suppose (or rather, hope).

There's a corollary to all these observations, but I'll leave that as an exercise for the reader.

The biggest mistake you can make is to believe that you are working for someone else.

posted at: 14:21 | path: /observations | permalink | Tags: , ,

Fri, 27 Oct 2006:

Once upon a time, in the cold December of 2004, there was a blog post. I read it, thought about it a lot and eventually made a decision.

All that's now history - friday was swaroop's last day at Yahoo! bangalore.

With that, it looks like I know more ex-Yahoos than those in office ... [-).

A good compromise leaves everyone mad.
                  -- Calvin

posted at: 01:52 | path: /yblr | permalink | Tags: , ,

Wed, 25 Oct 2006:

Weak symbols are a poor man's version of linker land polymorphism. A weak symbol can be overriden by a strong symbol when a linker loads an executable or shared object. But in the absence of a strong sym, the weak version is used without any errors. And that's how it has worked for a long long time in ELF binary land.

But then dlopen() went and changed the rules. When you load a shared library with RTLD_GLOBAL, the symbols became available to all the other shared objects. And the libc rtld.c had the runtime magic required to make this happen (and the unloading was even harder).

Then one fine day, Weak Symbols were empowered. In dynamic shared objects (DSO), there was no difference between weak and strong. It has been so since the glibc-2.1.91 release.

Now let me backtrack to my original problem. Once upon a time, there was a php extension which used an apache function, ap_table_set() to be precise. But for the same php extension to be loadable (though not necessarily useful) inside a php command line executable - the external symbol had to be resolved. That's where a weak symbol proves itself invaluable - a libapstubs.a could be created with a weak ap_table_set, so that as long as the extension is run outside apache, the stub function will get called.

But it wasn't working on linux (works on FreeBSD). And except for the extension, we weren't able to write a single test case which would show the problem. And then I ran into a neat little env variable - LD_DYNAMIC_WEAK. Just set it to 1 and the rtld relegates the weak symbols back into the shadows of the strong ones. But that raised a few other problems elsewhere and I personally was lost.

But now I know where exactly this went wrong. Php was using a glibc dl flag called RTLD_DEEPBIND (introduced in zend.h,1.270). This seems to be a glibc feature and the flag bit is ignored by the FreeBSD libc - which was running the php module happily. As you can read in that mail, it looks up the local shared object before it starts looking in the global scope. Since libapstubs was a static .a object, the local scope of the ext .so did contain the dummy ap_table_set() and since glibc rltd was ignoring the weak flags, that function was called instead of the real apache one.

I'm perfectly aware that RTLD_DEEPBIND can save a large amount of lookup time for shared objects built with -fPIC, because of the PLT (we've met before). But if you are trying to use it to load random binaries (like extension modules), here's a gotcha you need to remember.

Now to get back to doing some real work :)

A wise person makes his own decisions, a weak one obeys public opinion.

posted at: 17:12 | path: /php | permalink | Tags: , ,

Sat, 21 Oct 2006:

A Buffet Intellectual is a less disparaging term than the more cynical pseudo intellectual, which has seen far more use than required. The term does not make any kicks about the actual intellectual's ability, but in the dilettante attitude of the individual. The random samplings of life that such a person goes through leaves him (or her) a veritable gold mine of random conversations but very little to seek them out for.

To identify a buffet intellectual is very easy - just start talking about something in the abstract and watch the classical allusions roll by. But the easiest way is, of course, to peek into the bookshelf. In the midst of all the professional books, you'll find books about Evolution and Dinosaurs. Or half a shelf of books about Quantum Physics and the Universe. Look more closely and you'll even find books about Economics and Social orders. Heck, you may even find a couple of books about Human Psychology.

A nibble here, a nibble there and not a mouthful anywhere. Such intellectual butterflys are curiousities who rarely achieve anything by such half-hearted dabbling. What is needed is focus and perseverance - not two days' worth of everything.

Curiosity killed the cat.

posted at: 16:04 | path: /me | permalink | Tags: ,

Wed, 18 Oct 2006:

Flockr generates digraphs out of your flickr contacts. This is what has triggered me to go off the straight & narrow path of low level programming into hacks with Graph networks and x-mixed-replace. Anyway, what is done is done - take a look at a canned demo if you are on Firefox 1.5 and above (can't host cgi scripts).

The really hard part of the above code comes out of the simple fact that HTML Canvas is totally pixel based. You cannot attach an event to a small circle drawn as lines & figures. The code inside graph.js has a class called NodeHandler which sort of implements event masks and handlers for a node I've drawn on canvas. The code uses a hashed bucket to determine which all nodes it needs to iterate over, rather than loop over every node in the graph for every mouse move - harsh lessons learnt from dotgnu's winforms. It works better than I'd expected of Javascript and certainly seems snappy enough.

Now, I'm off hacks for a while ... time to vent some energy on real code.

An empty canvas has more potential for greatness than any painting.

posted at: 11:12 | path: /hacks | permalink | Tags: , ,

The LCA submissions have been rated, reviewed and the letters sent out. And it shouldn't come as a very big surprise that my talk hasn't been selected - here's to all those who said "Yeah right, save your talks for real conferences". I can't honestly say that I'm not disappointed, mainly because there is no point in talking about libjit in 2008 - it would have crossed over the bump into old & busted. And I'm definitely not going to submit a talk to conference which I haven't attended - I'm too fragile to go through an audience mismatch again.

But on the other hand, it is a testament to the quality of conference. if my talks aren't good enough, that sure is a conference which I shouldn't miss - which brings me to the real reason why I'm depressed right now.

My LCA 2006 trip in total cost around 1.4 Lakhs (~3200 USD) - of which dotgnu chipped in 1700 USD, Yahoo! chipped in 600 USD and I pulled the rest of the ~1000 USD. But this year, I see no reason for DotGNU to pay anything, because most of my efforts have gone into APC rather than the usual creative energy vented out on dotgnu. Since I have switched teams & manager, I think Yahoo! will wash hands clean, especially after what happened to spo0nman.

Since I'm not getting a raise this year, despite complaints, I don't think I can build a slush fund to fulfil the extra ~2000 USD (7 months' savings) - but even for the 1000 USD part, I'll probably have to give up a few luxuries. There's a bit of me that says I should be more optimistic, but deep inside me I know why I'm not.

Disappointment leaves a scar which ultimate fulfillment never entirely removes.
                      -- Thomas Hardy

posted at: 08:42 | path: /conferences | permalink | Tags: , ,

Tue, 17 Oct 2006:

For the entire weekend and a bit of Monday, I've been tweaking my di-graphs to represent flickr entries and in general, that has produced some amazing results. For example, I have 700+ people in a Contact of a Contact relationship and nearly 13,000 people in the next level of connectivity. But in particular, I was analyzing for cliques in the graph - a completely connected subgraph in which every node is connected to every other. For example, me, spo0nman, premshree and teemus was the first clique the system identified (*duh!*).

Initially, I had dug my trusty Sedgewick to lookup graph algorithms and quickly lost myself in boost::graph land. STL is not something I enjoy doing - this was getting more and more about my lack of const somewhere rather than real algorithms. And then I ran into NetworkX in python.

NetworkX is an amazingly library - very efficient and very well written. The library uses raw python structures as input and output and is not wrapped over with classes or anything else like that. The real reasons for this came up as I started using python properly rather than rewrite my C++ code in python syntax. When I was done, I was much more than impressed with the language than the library itself. Here are a few snippets of code which any C programmer should read twice :)

def fill_nodes(graph, id, contacts):
    nodes = [v.id for v in contacts]
    edges = [(id, v.id) for v in contacts]

def color_node(graph):
    global cmap
    node_colours = map(
                lambda x: cmap[graph.degree(x) % cmap.length], 

Or one of the more difficult operations, deleting unwanted nodes from a graph.

# trim stray nodes
def one_way_ride(graph):
    deleted_nodes = filter(
                 lambda x: graph.degree(x) == 1,
    deleted_edges = filter(
                 lambda x: graph.degree(x[0]) == 1 or
                 graph.degree(x[1]) == 1, 

The sheer fluidity of the lambda forms are amazing and I'm getting a hang of this style of thinking. And because I was doing it in python, it was relatively easy to create a cache for the ws requests with cPickle. Anyway, after fetching all the data and all this computation, I managed to layout the graph and represent it interactively, in the process forgetting about clique analysis, but that's a whole different blog entry anyway.

The worst cliques are those which consist of one man.
                -- G. B. Shaw

posted at: 07:44 | path: /hacks | permalink | Tags: , ,

Mon, 16 Oct 2006:

I've been playing around with some stuff over the weekend, which eventhough runs in a browser, needs continous updates of data while maintaining state. So I tried to a socket-hungry version of server push which has been called COMET. Now, this technique has come into my notice because of a hack which bluesmoon did, except I had to reinvent for python what CGI.pm did for bluesmoon by default.

But first, I mixed over a couple of the client side bits. Instead of relying on XHR requests, which are all bound & gagged by the security model, I switched to a simpler cross-domain IFRAME. But what's really cool here is that I use a single request to push all my data through, in stages, maintaining state. So here's a bit of code, with technical monstrosity hidden away.


import sys,time

from comet import MixedReplaceResponse

content = "<html><body><h1>Entry %d</h1></body></html>";

req = new MixedReplaceResponse()

for i in range(0,10):
	req.write(content % i)


The MixedReplaceResponse is a small python class which you can download - comet.py. But the true beauty of this comes into picture only when you put some scripting code in what you send. For example here's a snippet from my iframe cgi code.

wrapper = """
   <html><body> <script>  if(window.parent) {
	} </script></body></html>

script = ("window.parent.updateView(%s);" % json.write(data))

req.write(wrapper % script)

As you can clearly see, this is only a minor modification of the json requests which I'd been playing with. But underneath the hood, on the server side, this is a totally different beast, totally socket hungry and does not scale in the apache cgi model. Interesting experiment nonetheless.

Now, if only I could actually host a cgi somewhere ...

To see a need and wait to be asked, is to already refuse.

posted at: 09:12 | path: /hacks | permalink | Tags: , ,

The first shortlist of talks for FOSS.in/2006 is out. Let me be the first to say that there are a couple of surprises there. First and very surprising one would be a talk by none other than spo0nman himself. Second was botsie talking about Gentoo - not about XFCE. Yet another, sort of surprise, with bluesmoon about libyahoo2 - I suppose he can indeed talk about it, though not commit stuff ?

And then, a few names were obvious by their absence. I don't see a talk by KingDiamond or BigBeard - And kryptic too is missing, which was probably not as much of a surprise (*click* *click*).

The talks list seems a bit biased towards the kernel/library/OS land and a little left of the eyecandy section, but I'm sure that is probably not a real problem. Just make sure one of the computers around is running Beryl with the keyboard on a recorded cycle with xsendkey.

I like the word "indolence", makes my laziness seem classy.
            -- Bern Williams

posted at: 01:53 | path: /conferences | permalink | Tags: ,

Sat, 14 Oct 2006:

After climber over enough mountains, I'm now planning to take a dip & see some corals. Current plan involves a week in Lakshadweep in mid/early November - seems to be a good time to visit, the sunshine shouldn't be too hot and the weather mild. I am planning a single week trip with the weekends to take the slack of the trip. December is actually when the circuit gets really really expensive - which puts this in a nice bracket.

The current budget is about 15-17k INR (which should basically take care of November's paycheck). Current plan is to take a helicopter one-way from Lakshadweep and the other way on the boat - as far as possible on the budget. Current plans involve taking the MV Tipu Sultan which plies between Cochin and the islands (3,400 INR). The helicopter ride is a lot more expensive and costs (7,400 INR).

The only airport in Lakshadweep is on the Agatti island, which is a pretty awesome coral formation. Here's how it looks from Google Earth at maximum magnification - the straight dark strip is the airport. But as far as I can understand, there is only room for one hotel on the island - the Agatti Island Beach Resort which looks to be more than slightly on the expensive side.

After the point-to-point travel, there's a maximum 6,000 INR to do the sightseeing + stay in the islands. So that obviously rules out the expensive hotels (2000+) for more than a single night.

Totally I have 5 islands to hit: Agatti, Kadmat, Bangaram, Minicoy and Kavaratti. Three of them come up in the standard "Coral Reef" tour package and now I've got to find out how to visit the other two (one of them is uninhabited wilderness).

So far the trip plan is not totally set in stone - I'm still debating the possibility of taking a tour package instead of planning out my own trip. Either way, I want that helicopter trip over to Agatti, if not from Cochin, then from somewhere else.

So, if anybody's interested in pooling along with me, please let me know. A group of 4 will be ideal to get at least a few discounts along the way (heh, did I mention that people speak Malayalam in Lakshadweep ?).

And if someone had any advice, that'd be even more than welcome. Leave a comment here, I'll be watching.

A good traveler has no fixed plans, and is not intent on arriving.
                -- Lao Tzu

posted at: 05:55 | path: /misc | permalink | Tags: ,

Fri, 13 Oct 2006:

I was battling with some code that used instanceof() in python. I don't where that's a built-in (I suspect jython), but it doesn't work on my python 2.4.3. While I was wondering about this problem, I tried something outlandish and it worked. But before I actually explain what I did, take a closer look at this module (x.py).

def identity(h):
    return h

def coolfunc(p):
    return identity(p)

That was pretty standard code, right ? Now, to use/abuse this module in y.py.

import x

print x.coolfunc("Hello World !")

x.identity = lambda x : 42;

print x.coolfunc("What is 21 + 21 ?")

As you can probably guess, I can override (uhh.., pollute) x.identity with my own functions, which opens up a solution to my original problem. Pretty simple, now that I think of it.

othermodule.instanceof = lambda x,y : type(x) == y

I hope this has made your friday a little bit more surreal :)

Talent to endure stems from the ignorance of alternatives.

posted at: 10:42 | path: /hacks | permalink | Tags: ,

Found this in one of the mailing lists - but this is total fnuk. Please feel free to click on any of the following links - yahoo shit and google shit. Please take a look at your title bar of your browser to understand the true beauty of bi-directional font-rendering :)

I'm sure xoferiF wouldn't violate any trademarks by the Mozilla foundation.

Drawing on my fine command of language, I said nothing.

posted at: 05:30 | path: /fun | permalink | Tags: , ,

I remember the last one very vividly.

No amount of careful planning will ever replace dumb luck.

posted at: 04:20 | path: /fun | permalink | Tags: ,

Thu, 12 Oct 2006:

I finally got my camera repaired. It cost a whopping 6,800 INR to replace the cracked LCD and backlight-panel - I could've bought a brand new camera for a little more, but I liked to get back my original camera. Some of the pictures I've taken with it while it was broken were good, but I saw the difference when I had the full LCD screen and all the visual controls set. Here's a sampler from my train ride on monday.

Amazing photographs and some of my best pictures of Kerala so far. The weather is more to credit than my camera though - a calm, quiet morning after a stormy night, blue skies and all.

There is one advantage to having nothing, it never needs repair.
                                -- Frank A. Clark

posted at: 00:34 | path: /hardware | permalink | Tags: ,

Wed, 11 Oct 2006:

Last year, Atul had coined a phrase, "The Guru Shishya Syndrome", in a mail to the list. The year has passed us by and now in one of the recent posts, he has had to re-hash it, only with a little bit more sting in its tail (Crikey !). And I say that probably because it touches a few raw nerves - something which was probably more my fault than Atul's. But just so that you know what I'm talking about, let me re-quote that entry with my inherent biases and emphasis added.

For example, if you are involved in a FOSS project and have solved
a problem in that area that helped the project improve its performance,

stop believing that your contribution was too small to talk about, 
stop believing that your "overlords" in the headoffice in the USA
are the only ones who can think. 

I don't have a talk proposal for FOSS.in/2006. I had been vacillating about giving a podium talk for a while - I'd rather talk with a 10-15 member group than "to" 45. But eventually, I didn't have to make the decision. On Sunday as the deadline approached, I was sitting at a close relative's bedside, in Cochin hospital. But even without that twist of fate, I wouldn't have been talking about APC or giving a generic dotgnu talk. But before I go into the reasons for such unreasonable behavior, let me digress into some other territory.

'Jool Noret' effect: To give you some context, this is a character from one my favourite fantasy books - Dune. An excellent fighter who spent a life time achieving perfection with melee weapons. But what sticks in your mind, while you read about this character, lost in all that drama, is his philosophy - "I have so much left to learn, to start teaching".

And as I've previously blogged about, shining a light into the darkness of your ignorance only serves to highlight the shadows cast. In essence, what makes me good (IMHO) is what prevents me from stopping the journey to evangelize or pick laurels.

And even if I had actually gotten time on saturday or sunday - I wouldn't have submitted an APC: Maximizing Throughputness talk, I would've talked about something new, fresh & risky - something like Scripting your Application or probably about using lua or python to script your events and build plugins in (like this). Last year, I did a talk on Firefox: Plug it in - and I'd never written a single firefox extension till the talk schedule was out. But that was a good talk, because I understood and tackled a lot of the problems a newbie faces while writing their first extension. Compared to that, my contributor talk on dotgnu had a definite So On & So Forth feel to it.

I can't afford to experiment with topics or technology at say, LCA (here's why). For my LCA talk proposal, I'm actually planning to write a minimal basic-block JIT for PHP to show off libjit, but that's not quite pushing the boundaries for my personal knowledge - it is more of use than explore. FOSS.in, on the other hand, is as close to a home ground as it gets. That gives me a bit more confidence to do something which I'm not already an expert in. Rather than rehash what I already know, I'd like to pick up something new and run with it.

I'm quite aware that this is completely at cross-purposes with what a conference organizer would obviously want. Rather than have me skim the surface of a new topic, they'd want me to go deep and technical in something I'm really good at. But that sort of expert speak works only with a lot of audience involvement which just does not happen in a group of 50+ (hardly even in groups of 20). Bouncing ideas off others requires that you shouldn't need to wait for someone to shut up before you speak.

So all in all, I'd rather do a BoF with 20-odd people and end up with a new idea, than go the Attention Class! route that the podium talks force me to. If I didn't submit a talk, it had nothing to do with any bigshot from the US headoffice, but with fate keeping me offline on the weekend.

I didn't decide. It was decided for me.

If you understand what you're doing, you're not learning anything.
                -- A. L.

posted at: 11:11 | path: /rants | permalink | Tags: , ,

Sat, 07 Oct 2006:

One of the first cool things I saw in flickr was notes. They're those small boxes which you can drag across a picture to mark off a region or add some more context to something. When I recently started linking in flickr photos to my blog, these were some things I missed. Without a small box saying "That thin line is the road", a few of the more impressive photographs I'd got were quite ordinary landscapes.

While looking at the new flickr apis - especially the JSON apis, something clicked somewhere. Finally, there seemed to be a very nice way to do cross-domain requests from my blog (or literally any web-page) to flickr to read notes, tags and other information. Minimilastically, this is what my code does :

function myMethod(data) 

var photos_get_info = "http://api.flickr.com/services/rest/?"
        + "method=flickr.photos.getInfo&api_key="+api_key
        + "&format=json&jsoncallback=myMethod"
        + "&photo_id=" + photo_id + "&secret="+secret;

/* cross-domain request */
(function(url) {
	var _s = document.createElement("script");
	_s.src = url;

The photo_id and secret are the two parts in a flickr image url, the first part is the id and the second the secret. Provided you've given valid data there, flickr will respond to your data load with something like the following data.

myMethod({"photo" : .... , "stat" : "ok" });

Which when interpreted by the browser, magically seems to call your callback myMethod. Isn't that beautiful or what ? Of course, this workaround is not necessarily new information for me - but pretty damn convenient. Either way, once you've got the cross-domain data, it is only a few steps away from taking that data and displaying it.

Take a closer look at the ugly javascript if you want, or just look at the pretty pictures or funny pictures or even your own pictures.

Actually, more than being able to embed notes in my blog, this has brought along an unexpected advantage. With this script, the flickr notes are scaled according to the picture, so that you can have notes for a large or original size picture. Maybe I should make this a GM script so that I can do the same in flickr's zoom.gne pages.

Either way, the fun ends here.

It's difficult to see the picture when you are inside the frame.

posted at: 04:20 | path: /hacks | permalink | Tags: , , ,

Fri, 06 Oct 2006:

On a cold dark day in November 2005, Trumpf took a risk and bet 9000 USD on dotgnu. These were two grants of 4500 USD each, one for libjit (i686) and the other for the corresponding JIT coder in Portable.net, which I've jokingly referred to as the Southern "Summer of Code" (update: skwashd reminded me that he coined the phrase).

As of October 1st 2006, the work is considered to be complete. Congratulations to klausT, aleksey and kirill, who will all receive an equal cut of the pie. But it is not really about the money here, it is about the true spirit of FOSS - how people stand up to be counted, are put to the test and come out winners. I've been there, done that and still miss the rush.

Moving onto the bad news section of things, I'd like to take advantage of this occasion to formalize my sabbatical in DotGNU land. I've been promising to huff and puff for a while, but nothing's been blow away so far, other than the odd one-liner somewhere. So at least for a couple of months, I'll just stop promising stuff and go into "do & commit" mode of operations.

If that doesn't work out, I'll just have to be promoted to management or taken out back & shot.

The reward of a thing well done is to have done it.
               -- Emerson

posted at: 19:39 | path: /dotgnu | permalink | Tags: ,

Thu, 05 Oct 2006:

This came out of recent conversation -

Failure is not defeat; Defeat is when you stop trying.

I can be lucid on occasions.

I often quote myself; it adds spice to my conversation.
               -- G. B. Shaw

posted at: 06:10 | path: /misc | permalink | Tags: , ,

Tue, 03 Oct 2006:

Ever since I've told my parents that I'm buying a bike, I seem to have moved into a different class of "accident probability", without actually having to own one. Over the last 6 weeks, I've had 3 accidents on the road. I was driving in only of those and the fault was entirely the bus driver's who pushed me off the road, though some might say going off the road at 60 KMPH was unlucky. But thanks to my jacket, I have only a couple of bruises, instead of the standard scraped arms (or worse).

All this has translated into a "Don't buy a bike" and "Driving is dangerous" sentiments in the centre. Which basically means - if I want a bike, I can buy it with my own money - A task, which is getting harder and harder.

All said and done, I spent monday with my right elbow in bandages and wincing everytime I had to use the mouse. I'm thinking seriously about writing to a couple of interface design teams about what they were smoking.

As far as driving is concerned - Buckle up Dorothy, 'coz Kansas is going bye bye.

No, Officer, that tree came out of nowhere and hit me.

posted at: 17:12 | path: /rants | permalink | Tags: ,

Never before in my life have I managed to run through my entire salary before lunch on the 1st. Not even when I got my first 6,600 INR paycheck from Wipro did I actually spend it all, least of it in one day.

This would've made sense if I was paying for last month's credit card purchases, but I don't have a credit card. Except for 4,500 spent on books, none of the other expenses are what I'd consider as luxuries.

What gives ?

We're living in a golden age. All you need is gold.
               -- D. W. Robertson.

posted at: 16:02 | path: /misc | permalink | Tags: ,

Tue, 26 Sep 2006:

I was feeling really bored, so I wrote this greasemonkey script - nocomments.user.js.

Should be enough, I suppose.

This game lends itself to certain abuses.
             --- Calvin

posted at: 18:06 | path: /hacks | permalink | Tags: ,

Jargon is the refuge of the elitist, but I will stoop to it - to hide behind a word, my ideas. Performance Inversion is that dark fate that awaits a "Can Do" person in the hands of an unscrupulous [1] taskmaster. Something dark and deadly which destroys your work ethic and pushes you ever closer to a burn out, while still under-utilizing your real talents. In relatively mild quantities, the problem solves itself, thanks to that great human ability to forget. But for some more unfortunate souls, it throws them into a bottomless pit of effort, in a spiral of work and more work.

Performance: You are stepping into an existing team, filling someone else's boots. Invariably, your first few weeks will be typical bootcamp. You are pushed to deliver and most of us, do deliver in that initial phase putting in extra effort in return for the manager's confidence. But having gained the confidence and trust, the work cycle settles down into a more sustainable load, by mutual consent from both parties.

Distribution: Not everybody in the team works at the same levels of productivity. There are a few who have bursts of extreme results, while some of the more settled folks deliver a constant, steady stream of work. Most managers would like a decent mixture, so they have enough afterburner fuel in the former while the latter keeps chugging. Work comes in and the work load sort of averages out for everyone, a few extra bits given & taken, in general everyone ending up with their fair share - no more, no less.

"Can Do" XPloited: The former category of "in reserve" people are those who are more commonly known as "Can Do" people. Nobody sane (or at least has heard of "Mythical Man Month") will dare to plan for these people to work at full throttle. But a few still break this cardinal principle to keep the wolf off the door, for a few months more. After all, what good are developer resources, if not for working.

'Tis an act of betrayal, worthy of Macbeth and more. But unlike the lady of the play, the blood seems to wash right off the hands - maybe she should've tried the sweat and tears of engineers, instead of perfumes from arabia. Anyway, if someone had yelled out "abort mission!" at that point, I'd shake them by the hand and buy them lunch. For otherwise they have to be eternal optimists who never learn (incompetent) or of the feudal weasel family, whose concern for the serfs is legendary - either way, let them expense lunch.

Blame Spread Thick: Having been told to do the impossible, the developer has a faint idea that he/she is going to be pushed to the very limits of his/her ability. In some sense, the "Can Do" people enjoy that experience. But then something goes wrong, some dependency fails to deliver, the whole plan's a pile of toilet paper as far as the project's concerned. And a landslide of backlogs land on the developer's shoulder and the difficult task becomes mission impossible (where work should've been a walk in the park instead). To compound insult to injury, he/she is called in by senior management to explain the delays. Watching the man responsible for the mess (in your opinion) sit on the other side of the table with a reproachful look, doubting your commitment, is too much for anyone with a straight backbone to bear.

Character Shows: But you still eat crow, swallow the remnants of your self-respect and buckle down. You work insane hours, on weekends, skip meals, eat junk food and work work work. By an amazing coincidence, you manage to drag the entire module back on schedule, mainly by doing the dependencies yourself and doing QA's job when you're done coding. Having got your workload to a manageable level, you wait for the release, the associated kudos and a general pat in the back.

Inversion: And when you think it can't get any worse, it does. So the boss shows up at your cube, and says "Ummm,yeah ... $_nameless_ is quite behind schedule. You're the only one ahead of your tasks, so I'm moving a bit of work off to you". Not only are you not getting credit for finishing your work, but you are getting more work just because you keep doing it. All this while, $_nameless_ has been shirking work, lazing about in the food court and generally enjoying life. So, faced with the basic injustice of the situation and remembering the last lecture about "communication", you walk back up to your boss's office and enunciate - "No, I'm not doing that". Boss takes offense to your "Not my job" reply and lectures for half-hour on team work and "one for all, all for one" philosophy.

Final Straw: For an external observer (such as middle management), in a matter of months, you've gone from being a "Can Do" whiz kid to becoming a whiner who says "No" to work. They just can't comprehend why or how of the scenario and decide to take strong steps to discourage this behavior. In response to your glowing achievements on your self appraisal, they write out a "negative attitude to work" and give you an average hike. Your complaints to HR fall on deaf ears, used to listening to employees bitching about pay, quote salary parity to ignore you. Of course, the boss probably got a good hike for managing with (*sic*) employees like you. The inversion discontent spiral is now complete.

In a few short steps, one person has gone from being an excellent employee to someone who's polishing up their resumes. But the really sad part is that as long as the real offender is not punished, this developer is only one among a long line of people whose work ethic will be destroyed totally by mismanagement. There are a few other branches and variants to the above scenario, some worse, some better, but they all end up in the same gutter anyway.

All this only goes on to prove why I think of management as a dark art of sorts. It requires a lot of finesse and panache to pull it off, rather than mere authority behind you. You can't convert any average joe from a tech lead into a manager, at least not into a good one. There is a certain Je ne sais quoi which sets apart the good ones from the average ones. So if you have a good manager, you don't know what you're missing

And for the record - I only observe, I don't interfere.

[1] - Hanlon's Razor:
                Never attribute to malice that which is adequately explained by stupidity.

Lots of folks confuse bad management with destiny.
               -- Frank Hubbard

posted at: 17:42 | path: /observations | permalink | Tags: , ,

Mon, 25 Sep 2006:

'Tis time I ended journalling this trip.

The trip to Nubra Valley had taken a day less than we had expected, so the previous night had been mostly wasted in bed. So we were up early on 28th morning and were ready before 8:15. And Janina, who'd taken the same jeep with us from Manali, joined us on this trip, along with another girl. She turned out to be scottish and was doing a thesis on attitude differences across the generations in ladakh. We headed towards likir, only pausing to visit Pathar Sahib and a couple of stops where the Zanskar meets Indus.

Likir proved to be a tiny village on the Indus valley, all green and gold with fields of Alfa-Alfa. The gonpa is on a hilltop, with a golden buddha statue. The statue is known as Buddha Coming or something like that, because the statue's feet touch the ground instead of being the usual cross-legged position.

Post Likir, we headed along the Indus river bank to Thingmosgang, passing through but not stopping at Bazgo. Thingmosgang is a small picturesque village, nestled in a valley protected by the elements and fed by meltwater from the glaciers above. The gonpa was deserted, but the view from the top was beyond words. You are struck speechless by beauty of the green fields and terraces, from the vantage point of the monastery.

After we passed out of Thingmosgang, passing through Saspol, we stopped over at Khalstse, which is where our driver was from. Except for the three kids running around, the lunch was completely uneventful. We reached Lamayuru, in the late evening. The monastery is very visibly alive, with kids playing cricket in their saffron robes of monkhood.

We all climbed to the top of the hill on which Lamayuru stands and decided to watch the sunset from the summit. A couple of officers on an evening walk approached us, while we were up there. The officer, of the Sikhla (?) regiment advised us to go back through the Kashmir valley instead of back via Leh. The sunsets so far have been hardly anything to write home about and this one was no different.

We headed back to our rooms before it got too dark and tried to find us some grub. But after a bit of wandering, we decided that the best option we had was to eat at the guesthouse we were in.

I woke up early, though not early enough to attend the morning puja. Since we had plans to visit Bazgo and Alchi on the way back, we hurried out of Lamayuru. But we were stopped mid-way by some blasting work that was going on. After nearly an hour of dilly-dallying, we were finally let through and we proceeded to Alchi gonpa.

Alchi is probably one of the oldest gonpas we visited, so far. Photography was generally discouraged, which was perfectly OK with me there. The road to Alchi, however was quite interesting, it was almost absolute ruler straight, with a nice fork in the road to the monastery. And there was green barley all around the place. We found a nice restaurant with a view and sat down to eat a late lunch.

After heading out of Alchi, we retraced our steps to Bazgo. Bazgo looks like a mud fort in ruins, but when you look closer you can see the solid blocks of stone over which centuries of plastering has created the appearance of soft mud. It is being restored and most of it was closed for us. But the structure did leave a sense of awe about people who'd build such a structure in such an inaccessible place.

After all our adventures in Kargil district, we headed back to Leh to civilization, warm beds and honey lemon ginger tea. But the trip back was no less enchanting in the fading evening light.

After some butter tea, darpu and a bit of shopping, headed back to the hotel for dinner. I'd had bought some Yak cheese to bring back to Bangalore, dried apricots and a jar of apricot jam. Our dinner was hot maggi noodles, with a lot of vegetables. We bid adieu to our fellow travellers and hit the sacks early.

And then we took the flight to Delhi. End of journey.

What makes the universe so hard to comprehend is that there's nothing to compare it with.

posted at: 12:19 | path: /travels | permalink | Tags: , ,

We started off late with me waking up first, at 9 AM. But by ten O'clock we'd had breakfast and were ready to roll. The cab came to us after picking up everybody else. After leaving Leh at around 10:30, we spent the next two hours steadily climbing uphill to 18,380 feetto the highest motorable road in the world - Khardung La. Unfortunately, it has been converted into a touristy place with families and grandmothers climbing up melting snow or sliding down pulling down two sons, one holding each hand.

I did climb up to the top of the peak, nearby. The snow was slippery and I walked up by digging my heel in with each step into the snow. The way up was the easy part, as I would discover later. The view from the top wasn't that different, but I'm easy to please when mountains are concerned. I was feeling euphoric and was slowly making my way down when I realized that there was a faster path down than walking. I went down that hill sitting on my left foot and steering with my right heel. After all that excitement and a cup of cinnamon tea later, we headed downhill into Nubra valley.

The word Nubra means green and the valley was only slowly turning green under the bright summer sun. The glimpses of green in the valley was amazing after the dry desolation on the other side of the mountain range. We kept moving till we got to the village of Khalsar, which was our designated lunch spot.

In the village of Khalsar, I had the most amazing momos I have ever had. They were soya momos and with a simple chutney. I don't know if it was just because we were hungry, but we gobbled up as many momos as the lady could bake. After lunch, we headed out to the Shyok and Nubra confluence, which we would cross to reach Diskit. The bridge across the river is small, yet controlled by army personell.

Finally, we ended up in Diskit and dumped out bags and stuff at a slightly upmarket place called Olthang. While Vinayak & Co were visiting the gonpa nearby, I went walking along the streams which passed near our hotel. Rishi , on the other hand, was watching kids throw water at passing vehicles. One german lady who got wet, also got indignant. Probably needed a camera for the moment when she asked Rishi - are you a monk ?. After all this fun, we headed out for the dunes of Hundar before it became too dark.

Hundar is a stretch of sand dunes, with a stream (or river) flowing through the middle. After the standard shoes off operations, we had a few chases up the sheer slope of a dune. The sand was hard rock sand, quite unlike the beach sand I'm familiar with. Walking barefoot was probably a bad idea as I ended up hobbling across the thorny bushes on the way back. But more than compensated for that by just letting myself go into the warm (not cold) water. After a bit of splashing around, we headed back to the warm beds of the hotel.

27th June: At around 7 AM, I was woken up by the driver. He wasn't upto date with our plans and I was too sleepy to explain. Two hours later, all of us slowly started waking up. After a slush bowl of corn flakes with hot milk, we all pulled ourselves together and moved onto to the other side of the valley. We had to go halfway till Khalsar before we'd take the other fork in the road which would take us to Sumur and Pangmik.

After going down the Khalsar loops down to the valley floor, we passed through Sumur towards Panamik. Panamik is probably the northernmost point in Indian soil that I can put foot without a military permit and escort. The hot springs of Panamik, though were a disappointment. The springs were literally trickles down the hill with the smell of ammonia and algae covering the entire stream bed. But the top of the hill gave us an excellent view of the valley below.

After visiting the Sumur gonpa, we bid adieu to Nubra valley and began our return to Khardung La. We stopped over at Khalsar yet again to have lunch, but the momos had run out. As evening set in, the air was shimmering with the fading light of the summer day. We climbed up into the upper reaches of the mountain, racing against the clock. Since our plans had included a third day in Nubra valley intially, we were sort of cutting it close by shrinking it to a two day trip.

We went up Khardung la, just minutes before 5 'O clock. The entire snow cap was bright with the late evening light. As we watched the mountain grow bigger and bigger, we could see the thin line the hand of man had scratched on this mighty mountain - the scar line of a surfaced road. And up we went.

K-Top was nearly deserted when we arrived. There wasn't any of the swarms of tourists we had seen on the way up. After the customary cup of chai, we took our trip downwards and onto the warm comfy beds in Leh. Our trip to Nubra valley was a couple of hours short of completion. Not here the red and orange glows of the sea shore, but the the golden sun was turning the mountainside into a veritable feast for the eyes.

A dinner that went wrong, though the food was ok when it eventually arrived, and the day ended as it began - with me in a bed.

If the grass is greener on the other side, there's obviously more shit there.

posted at: 12:14 | path: /travels | permalink | Tags: , ,

Wed, 20 Sep 2006:

Currently, I am totally struggling with APC's shm memory. Usually with most memory issues, the system cleanly segfaults. But when the memory is part of a 128 MB mmap() area, a couple of bytes here or there show up way too late in the debug operations to detect and fix.

Now, a lot of the allocator code has things which allocate n bytes + sizeof(header) and return the allocated area - sizeof(header). The problem is that my previous watchpoints code cannot differentiate between these two, unless I put an explicit watchpoint on the location. Not to mention, it has no concept of free operation in conjuction with the original block.

Digging around in valgrind code, I found an elegant answer to the problem - VALGRIND_MALLOCLIKE_BLOCK and VALGRIND_MAKE_NOACCESS. Here's a mocked up version of my code, which seems to work out.

#include <valgrind/valgrind.h>
#include <valgrind/memcheck.h>

void *  alloc(size_t n) 
	void* x = malloc(n + 42);
	x = (unsigned char *)x + 42;
	return x;
void dealloc(void * ptr)
	free(ptr - 42);
int main() 
	size_t n = 200;
	char * a = alloc(n);
	a[0] = '1';
	a[-1] = 'x';

Now, the valgrind has red-zones, which are like canaries or sentinels for memory over-writes. I haven't figured out quite how to use them, but this should be enough right now, I think.

==28706== Invalid write of size 1
==28706==    at 0x80484ED: main (x.c:23)
==28706==  Address 0x4025051 is 41 bytes inside a block of size 242 alloc'd

And the line 23 is the a[-1]. Valgrind is just amazing. Ever since I've hit up on this tool, I've found that my debugging life is a lot easier. Now, to reproduce original bug and nail that son of a b*tch :)

They separate the right from the left, the man from the woman, the plant from the animal,
the sun from the moon. They only want to count to two.
                -- Emma Bull, "Bone Dance

posted at: 15:44 | path: /hacks | permalink | Tags: ,

Last weekend, I was holed up in Delhi attending (and giving a talk) at Freedel. I had given a talk at Freedel 2005 , which generally swooshed over everybody's head and left me disappointed. So this time, I'd picked a dead easy topic which I couldn't miss with - gimp plugins.

Me and premshree arrived there in an evening flight. I had picked the 4 PM flight, so that I could see the sunset from the flight and it was just to die for. To see the clouds outlined by the red borders of golden sunlight, was quite a sight. We were missing our mini cameras, unable to load up the big Nikon while flying. But let us forget the trip and focus on the take-homes from the conferences. Here are a few.

* inside airport
Premshree: Man, Delhi women are *hot*

* few minutes later, outside
Premshree: Shit, Delhi is *hot*

And yes, it was hot & humid.

FUNK: And a new keyword was born, been a while since Awesomeness. Now the latest cool (I mean, funk) word to say is - Funk. It has become literally the new F-word. All thanks to Premshree's diligent effort at shoe horning this word into any sentence in need for wordless words. So the whole funk conference was the total funk.

Then we had a couple of memorable quotes from cray3. The first one being the polite motto ("STFU, please") on her laptop as well as the "The daemons are no longer just in my head". But the "We will not be serving snakes on this plane" deserves first prize for the hinglish accent jokes.

Me and Premshree were staying at Kishore's place. And during a l33tness war during the party at OldMonk's place, Kishore won a few brownie points by asking - "And tcpdump is your mail client ?". Short of pissing off the organizers, we'd pulled off a GNUnify, topping the "Our sysadmin is a cron-job" by a few heads. OldMonk had a couple of quotes to his name as well, but I'll desist from mentioning them in such polite circles - very apt, but truly OldMonk.

And for the very first time, the audience clapped when my demo worked, rather than the respectful silence I'd been used to. So all in all, I'm glad to have talked ... I mean, funked out at this conference. Next year, it is going to be in Feb, which should prove to be cooler and even more funkier :).


Take my advice, take the next flight,
And grow some funk, grow your funk at home.
                -- Elton John, "Grow Some Funk of Your Own"

posted at: 09:22 | path: /conferences | permalink | Tags: , ,

One of the lamest hacks, we've ever done is something called Debt-o-Matic. I know spo0nman has already blogged about this, but there's more to this hack than meets the eye. To start with, this was the first hack which went from decision to action in around twenty minutes. And then we *designed*.

Through out this period, I was being active and generally bouncing around. Sad to say, but I was the product manager for the hack - doing little and suggesting a lot. But I redeemed myself by producing a kick-ass logo for for our hack. And after all that, we finally had ourselves a product before the morning dawned and an idea left to finish.

Remember the good old days in college where we used to adjust debts by transferring debts around ? We wrote up something which would find circles of debt and remove the smallest amount from all of them, by virtually circulating it. Not rocket science, but useful as this system can do it automatically. But stating privacy concerns, we cut down that to a A -> B -> C scenario, which is trimmed into an A -> C scenario.

We never got around to building the audit trail and details view stuff because it was getting late for our flight to Delhi. And off we went, leaving teemus to submit the hack, but there still was nobody to present it.

In short, we had fun.

Forgetfulness, n.:
        A gift of God bestowed upon debtors in compensation for their destitution of conscience.

posted at: 00:42 | path: /yblr | permalink | Tags: , ,

Tue, 19 Sep 2006:

Maybe I was a bit too hasty to conclude, about spo0man's fate on saturday. The system does indeed work, though only on weekdays, which is evident in the prompt action on monday. Spo0nman is currently sitting in Belgium, re-working his slides. And true to form, he's bitching about something.

<spo0nman> t3rmin4t0r: you'd think that EuroOSCON would atleast have bandwidth
<t3rmin4t0r> spo0nman, you just validated my comment
<t3rmin4t0r> "Get the best and then bitch about it"
<spo0nman> t3rmin4t0r, its grey and cold.

Now to pretend that my blog had something do with it ... NOT.

I personally think we developed language because of our deep inner need to complain.

posted at: 18:11 | path: /yblr | permalink | Tags: ,

Mon, 18 Sep 2006:

Rob Levin of freenode is no more. But lilo still lives on over the wires of irc for us.

[freenode] /who lilo
* * i=levin freenode/staff/pdpc.levin irc.freenode.net lilo G :0 Rob Levin
A man’s not dead while his name is still spoken.
                          -- Terry Pratchett (Going Postal)

posted at: 14:51 | path: /misc | permalink | Tags: ,

Sat, 16 Sep 2006:

We all sat down a couple of nights back to work on a hack - the Debt-o-Matic. But for spo0nman to have not written "I am going out of the country" in that entry, he would have had to be clairvoyant. He was headed for Euro-OSCON 2006 to talk about Nagios and about monitoring servers easily. Something he's thoroughly qualified to talk on, having done it day in & day out in the trenches, the on-call hard-hat in place, with a pager in one hand and laptop in other.

But right now, instead of packing his bags for his flight, he's in Delhi sweating it out (yeah, it is hot today) at Freedel with the rest of ILUG-D. With a visa, tickets blocked, hotels booked and yet he's not going to be on a flight tonight, polishing up his slides for the talk.

Nobody can force anybody else to pay up, but when an offer to pay for the tickets out of his own pocket is rejected, it repels me to even speculate about the reasons behind the rejection. If there is a financial crunch and the budget is trim, I can understand denying sponsorship for an employee going overseas. But if your own hard earned money isn't good enough to fix this, I don't know what is.

All said and done, the flight starts in 6 odd hours from now and he still hasn't got any tickets, despite the offer to pay for it from his own pocket. Personally I'd be happy to add my month's savings to that kitty as well, mainly because it is a matter of principle. A direct NO for the proposal at the outset would've been acceptable, but such a tease hurts like nothing else.

In all probability, his blocked tickets have expired by now and cannot be issued. And any attempt to escalate the issue will also probably have to wait till monday morning, which is when Euro-OSCON kicks off. All of which leaves hardly any point in the standard "Patience, my friend" from the-system-works-but-slowly department. Truly, sometimes justice delayed can be justice denied.

But the corollary to all this is even scarier. After all, if it could happen to him, it can happen to me. Go forward a few hypothetical months, substitute me for him and LCA for Euro-OSCON ... you get the point.

In conclusion - WTF !!!! ?.

The average nutritional value of promises is roughly zero.

posted at: 18:52 | path: /rants | permalink | Tags: , , ,

Tue, 05 Sep 2006:

APC released version 3.0.12. Because I've been sitting at home, close to a well stocked refrigerator and with no pool table in walkable distance, I've got a fair bit of work done on APC [1].

And some out of this world patches in the pipeline too.

Is it better to abide by the rules until they're changed or help speed the change by breaking them ?

posted at: 10:11 | path: /php | permalink | Tags: , ,

Nobody expected Steve Irwin would be killed by anything other than an aussie salt water croc [1]. I mean, this was the man who'd put down Eaten By Croc down on his epitaph by choice - struggling with crocdilians in all four continents. And it was quite a surprise when I woke up to this news that he was dead, from a sting ray barb through his heart [2] - a very painful way to die, indeed.

For someone who'd maintained a general irreverance to nature's deadliest creatures (most of which live in Australia, by some weird coincidence), I personally thought him to have aura of invincibility - a firm confidence that these animals mean him no harm, even when they were capable of snapping him into two bits without even trying. Crocodile Hunter wasn't just a nature documentary, it was reality tv with stunts and a fair bit of nature thrown in.

His antics might not have won him any respect from other wildlife experts, but he did manage to get a whole new generation interested in wildlife. If you watched a David Attenborough documentary about spiders and saw a spider in your backyard, you probably wouldn't hesistate to smush it out of existence. But if you'd watched a Steve Irwin special, you'd go looking for someone to throw it out. This guy put animals in your backyard, robbed them of their dignity and in some strange way, let us empathise with the animals. To the less ecologically sensitive masses, he put animals in a different perspective - one that made sense to them.

I'm sure he died doing what he loved, talking about things which could kill you, and him. I mean, I think that Douglas Adams is probably sitting up there wondering why he had to die while on the way to gym. Steve's got it much better, in comparison. Though it's Crocodiles:0 Stingrays:1 right now.

With condolences to Terri & the kids. Rest in Peace, mate.

[1] - /private/gator-baiter.avi

When I die, I want to be hit by a meteorite while I'm coding.

posted at: 01:21 | path: /misc | permalink | Tags: ,

Mon, 04 Sep 2006:

One of the rarest of the rare species that inhabit cubicles is that creature of myth and legend - the Team Player. Often naturalists hiding behind indoor plants near the watering holes of employees claim to have encountered the creature in the wild. But they are probably mistaken or lying, because it is widely accepted that no fossil evidence has been found past the piled carcasses of the Y2K mass extinction event. Ever since the job atmosphere has lost its ozone layer of job security, most of these magnificient creatures have succumbed to cancerous career growths or perished in hibernation between jobs.

Jokes aside, I'm here to debunk the myth of the team player. To study and expose the nature, being, migration patterns and if time permits, the mating habits of the common team player which is endemic to air conditioned cubicles of software companies.

For a borrowed concept from professional sports, the team player label has undergone a sea change before it has been used to describe a software engineer. Rather than deride the noble concept, which embiggens the smallest man, the target is the namesake euphemism which has replaced it in corporate vocabulary.

Managing a software project is no walk in the park and if Mythical Man Month is any indication, is often contrary to conventional wisdom. Essentially, most of the problems stem from one single assumption - that software is an industrial product. Building software is unlike any other industrial process and is more of a research & development activity than an assembly line of coders assembling components. Allowances have to be made for dead end attempts, work done to maintain status quo (aka Backward Compat), regressions and other anomalies.

Programmers, by definition, are only human. And humans have good days, off days and then those days when it doesn't pay to get out of bed. The productivity of a programmer is bursty and unpredictable. But predicting that is exactly what all the money in management is all about - creating schedules, timelines, plans and bullet points. And they'd rather have their task made easier.

So it is obvious that a steady, yet low, throughput would be considered more suitable to the management principles adopted from the industrial revolution, rather than the odd week of lucidity separated by a fortnight of stupor. The brilliant programmer who works in bursts falls out of favour, while the predictable programmer is pushed forward. The moral of the Tortoise and the Hare is vindicated in this modern race, where the hare is caught napping, though the jury is still out on whether 'twas the management's inaction which let the hare sleep.

The team player has come to be a euphemism for such a slow and steady worker - predictable and absolutely devoid of hidden reserves & surprises. Someone who would rather move with the team rather than run ahead and look back at others. To make no exceptions and just keep on working, despite lack of motivation or support from above is the clinching quality of the newly defined team player.

To put it more bluntly, being a team player (in context) is not for the benefit of the team of peers, but functions primarily as yet another variable removed from the game of uncertainity that is management. The ease of control is why this group is encouraged to form and survive, even re-evolve in new circumstances. By whom ? Take a guess. And where the species is missing, others are dressed up in the robes of team play-dom, under the flag with the device SCRUM.

The moment the system favours such team-players over strong contributors, the team will quickly lose its edge and motivation. The balance has to be maintained to ensure that the average output of the whole group exceeds the sum of the parts.

Finally, all the above discussions treat team players from the cynical point of view. But just because you are not a team player in the eyes of your management does not make you not-a-team-player in the eyes of your peers. As long as you can have fun, while working with them, help those who struggle and in general, not let your ego rule your decisions, you'll be one of the team - truly.

Otherwise, you're just a team player.

Captain, a starship also runs on loyalty to one man.
And nothing can replace it or him.
                         -- Spock, "The Ultimate Computer"

posted at: 01:46 | path: /observations | permalink | Tags: , ,

Fri, 01 Sep 2006:

As I type out these words, I'm sitting in Trivandrum, a few minutes' drive away from Kerala's democractic power centre - the AKG centre. And if I read the weather right, everything is in place for Free Software to take educational and government institutions by storm. But this wind of changes wasn't a communist movement or empty promises in the wake of RMS' visit. Months before RMS visited or even the current government came to power, there were undercurrents in the administration towards using Linux (*sic*). Now, they might have been invisible to a lot of the readers of The New York Times, but I've certainly talked about it quite a few months back.

As you can see from the date of the Kairali Linux project, that it pre-dates the current communist government. The good news is actually, that the work done over the past year and a half is actually going into use, even though the power centres have shifted during the last election. The official, face of the whole exercise is CDIT's Open Source Technology Team and they seem like a sensible lot. The following excerpt is from their website :-

As a responsible Government organization, C-DIT does not
wish to involve in tirades against any particular company
nor wish to practice free software like a cult.

Take that, you commie blaming NYT news reporters, confusing a good decision with political opposition to big business. And there's yet a personal angle to it too - in November, I had in my hands a pair of installation CDs. Look at the pretty shrink wrapped software with a big GPL disclaimer on the back panel, price tagged 99 INR.

It also came with a manual titled (translated) - Swathanthra Software and GNU/Linux in malayalam by the Kerala Bhasha Institute and C-DIT. The document is in pretty good detail, explaining how to do simple wordprocessing to playing music or setting up dialup.

This is all on top of the educational reforms removing product specific tutorials from the syllabus and making them electives. Yes, they teach about Word Processors not Microsoft Word. Like somebody recently said, it is about the same as Home Appliance science - but with Oo.org, it at least doesn't come with an extra license fee per copy. Anyway, it's not like there's any engineering college here that doesn't use linux as part of the course curriculum.

Now, at lease one concerned mallu has expressed his concern about the government's policy on F/OSS tools. But I'm quite unconcerned [1] - the whole operation looks to be in very good hands indeed.

[1] Whenever I run into a strawman, I think about the Wizard of Oz.

I'm free -- and freedom tastes of reality.
               -- The Who

posted at: 23:54 | path: /rants | permalink | Tags: , ,

Wed, 30 Aug 2006:

Thanks to all my firefox proxy.pac DNS irritations, I finally decided to ditch my ssh -D socks proxy for a tunnel into a squid. While I set up the firewall and enough protection for the proxy, I wanted to enable password protection on it. But basic authentication is not much of a protection and I didn't want to create a dummy user in the systtem to use this. Basically, I wrote my own squid authenticator - a simple enough task in hindsight.

If you inspect your default squid.conf, you'll find a line somewhat like this. This is your authenticator hook, which is a program which reads a single line in and outputs either "OK" or "ERR".

auth_param basic program  <uncomment and complete this line>

Now, after I know how the authentication works, it was as easy as pi. A simple enough script in whatever language you're comfortable in will do - I prefer python over perl and this sample's in py.

import os, sys,re

LINE_PAT = re.compile("([a-z_]*) (.*)\n")

u = sys.stdin.readline()
while u:
    m = LINE_PAT.match(u)
    if m:
        (user,pw) = m.groups()
        if authenticate(user,pw):
            print "OK"
            print "ERR"
        print "ERR"
    u = sys.stdin.readline()


Define your own version of authenticate, for example mine accepts a password that is "<fixed>.<OTP>" and the OTP is regenerated every 4 hours (not a very secure channel for transmitting that, but it works). You could probably build something similar to what RSA keycards use, which is basically the same principle.

auth_param basic program  /usr/local/bin/sq_custom_auth
acl password proxy_auth REQUIRED
# password protected
http_access allow password

Voila, you have a squid authentication that doesn't need a system account. Of course, there are more proper ways of doing this - like backing it with Mysql, LDAP or even RADIUS. But for a non-sysadmin like me, it needn't scale or be absolutely bulletproof. Probably took me much less time to do this, than write out this blog entry. But I wrote this so that sometime later, I can come back and look at this instead of remembering how to do this.

Always think of something new; this helps you forget your last rotten idea.
               -- Seth Frankel

posted at: 08:23 | path: /hacks | permalink | Tags: , ,

Tue, 29 Aug 2006:

I got irritated of getting kicked off irc around 20 times and the associated ghosting and renicking. So I sat down and wrote something that would keep me t3rmin4t0r on freenode.

import re,xchat

nick_pat = re.compile(":(?P<nick>[^!]*)!(?P<user>[^@]*)@(?P<host>.*)")

def renick(*args):
    nickhost = args[0][0]
    m = nick_pat.match(nickhost)
    if(m.groups()[0] == "t3rmin4t0r"):
        xchat.command("nick t3rmin4t0r");

xchat.hook_server('QUIT', renick)

Load that up with /py load ~/.xchat2/pylugins/re-t3.py and now I can stop worrying about being stuck as a mere gopal__ on irc.

Who steals my purse steals trash; ’tis something, nothing;

posted at: 21:06 | path: /hacks | permalink | Tags: , ,

Mon, 28 Aug 2006:

I watch a lot of cartoons, always have, always will. It might not have improved my grasp of physics, but it's always made me laugh. But as I sat in front of the idiot box today, I didn't want to watch any toon they were showing. Ever since Pokemon pointed out the huge merchandising opportunities, the recent trend of toons are mainly intended at making kids buy useless stuff. Sure, there were G. I. Joes and Skeletor toys in the cupboards when I was a kid as well, but watching Beyblade made me sick. Where Pokemon at least redeemed itself by emphasising evolution (for the US bible thumpers), this one seems to be pure merchandising claptrap.

I mean, I'm not asking for a strong story plot here. But it should have something, at least something that stimulates your brain. Most roadrunner cartoons are stupid, but I still ROFL at the Wyle E. Coyote, Genius business cards or anything that's named Acme (with apologies to Leon Brocard). Or to take another example, Tom and Jerry - perfectly predictable, yet funny in some excellent episodes. Even Scooby Doo has its moments of mirth, especially the Let's split up cliches. None of these needed a story to make it funny, it was merely funny because they were.

But there are still some which are for kids of all ages. They go beyond mere physical humour, into word play and referential humour. The moment, Bugs Bunny said "It's baseball season" [1] or the classic What did you expect, a happy ending ? [2] were landmark events for any cartoon to follow. Even the background music was borrowed from operatic greats [3] and a Casablanca spoof with carrots. Not that Bugs and Daffy toons were lacking in the low brow humour either. Those cartoons were loaded with jokes at all levels possible in six minutes.

But that era has passed and passed on the baton to the new overlords of cartoons. I probably won't consider Simpsons or Southpark as cartoons, but as merely animated series. But there were a few glimmers in the pile of shit that got served to me in the late nineties, by Cartoon Network.

First on the list would be Dexter's Laboratory [4] - the classical mad scientist story, only the mad scientist is just 8 years old. Having a secret laboratory, while living the life of a normal kid in front of the parents makes for some moments which involve the audience in some conspiratorial laughs at the expense of Dexter. Not to mention Dee Dee's meddling of the "Ooooooo. What does this button do?" kind. Nobody with a sister can stop smiling at that. Now, you might laugh at the standard jokes it sets up, but there are a few lines from today morning's episode ( * Figure Not Included ).

Major Glory : So what have you learned today ?
Dexter      : I learnt the important lesson that you cannot buy
              friendship with gifts.
Major Glory : No, not that. 
Dexter      : Then what ?
Major Glory : You're going to learn that you can't get away with 
              Copyright Infringment
Dexter      : Oh ?
Major Glory : Now you'll have to face someone much more 
              powerful than me.
Dexter      : Who's that ?
Major Glory : My attorney.

Or even the referential Mock 5 where Dexter (btw, the name means "Right") races against Racer-D who is actually Dee Dee (remember Speed Racer ?). Yeah, Genddy Tartakovsky is a genius. His other works, such as Samurai Jack or Powerpuff Girls, were excellent as well. In fact, PPG was far more involved than the name would suggest, though it is a slightly acquired taste (you need to watch Mighty Morphin Power Rangers to get some of the jokes).

And then there was Johnny Bravo. For a blonde Elvis clone, who picked up enough Fonzie cliches, comedy comes easy. I for one, love the Kirk Tingblad episodes where Johnny's narcissim is brought to the forefront. Sure I know enough folks who think that it is a stupid show written for stupid people, but the flourishes are there in the details. For example, in Aunt Katie's Farm, Johnny is a pig in the sketch. And after destroying the set, Johnny rolls around in the mud and starts to yell "Four feet good! Two feet bad!". Or even the Prince and Pauper version, except in this one Mark Twain ends up thrown into prison for calling the plot an old chestnut. But my best Johnny line was from the "Panic in Jerky Town" where he comes out of the factory yelling "It's people! Jerky Jake's Beef Jerky is made of people!" [5], which goes whooshing above most viewers' heads.

What I saw on TV today didn't even come close to any of these. I have a glimpse of the future Bill Waterson saw when he said no to selling cuddly stuffed Hobbes to his fans. I'm sure enough younglings will complain that Beyblade is the coolest, but from what I see, it is all about buying tops. I used to love DBZ, but it was never about buying Dragon balls from the nearest shops. These toons seem somehow different and alien to me.

I suppose, every generation survives on nostalgia. Maybe I'm wrong - all these kids will outgrow all the stupid toons and bitch about the next set of twelve year olds when they're twenty four. I mean, I'll really be scared if they don't

Oh, and to relieve you from the suspense about the deer uncles. That's from a Dee Dee quote - "Deers don't have uncles, they have antlers". Laugh if you can ...

This is what entertainment is all about ... Idiots, explosives and falling anvils.
               -- Calvin

posted at: 15:46 | path: /fun | permalink | Tags: ,

Wed, 23 Aug 2006:

I've always used -fprofile-arcs to instrument and profile my code and often been quite unimpressed by the output verbosity or presentation clarity. My opnion about profilers were down in the dumps, till I ran into kcachegrind. That is really a killer application in terms of profiling code.

KCacheGrind has everything I need. It has call graphs, call counters, callee counters and even lets me view code with profiling data in place. The latter is quite a kick ass thing, really. Now, I could actually see what branch was taken what percentage of time, pre-compute branch conditions and basically understand how the code was being used.

But what are words when a picture could do more justice to the tool. The following picture is a source annotation of a bit of apc code, the numbers might scare a few, but that's what it deals with on an average web server. So this is how I start callgrind :-

bash# valgrind --tool=callgrind --dump-instr=yes --trace-jump=yes -v \
      /usr/sbin/apache2  -f /opt/php5/etc/apache2/apache2.conf -X

bash# chmod 0666 callgrind.*  /* so that www-data can write to it */

bash# kill -USR2 <callgrind-pid> /* to kill apache2 gracefully */

After running my code, which is actually a bit of php non-code (you know, the ruby-on-rails or J2EE flavour of non-code), generated from symfony, I get a callgrind dump, which I can proceed to analyze with KCacheGrind.

I can then see the call graph, which looks something like this.

The really cool part is when you add the source dir annotations to the corresponding binary files in the configuration. Then I can get something that actually makes sense for a programmer.

I guess the screenshots speak for themselves. Thanks to this and after moving code pre-computations in place, APC is about 3-10% faster for php 5.1.x .

You cannot have a science without measurement.
               -- R. W. Hamming

posted at: 23:44 | path: /hacks | permalink | Tags: ,

Tue, 22 Aug 2006:

A lot of people have been complaining about APC's stability issues. In fact, they get angrier when I mention that it works for Yahoo!. During the FIFA slams on the servers, we put in a few extra things in APC to make it withstand the hammering. Now, a couple of these protections were borrowed from code that Y! already had lying around and a few more of them were BSD specific. But the short story is that I can never push those changes to the open source version. Nor can I even rewrite the same features after reading Y! © code which does the same, at least not while I'm here.

*But*, one feature that we borrowed was discussed quite a while back on the php-internals mailing list. If someone among you think that they know enough to understand what this means and implement it under the php license, maybe it might be accepted as a patch to php.

All it needs is some elbow grease and a bit of unix magic :)

signal(i, SIG_DFL); /* crunch, crunch, crunch */
               -- Larry Wall in doarg.c from the perl source code

posted at: 05:27 | path: /php | permalink | Tags: , ,

Sun, 20 Aug 2006:

Now, this isn't new, but it just had to be given its due respect. Sung to James Blunt's - You're Beautiful.

My cubicle, My cubicle
It's one of sixty two
It's my small space
In a crowded place
Just a six-by-six foot booth
And I hate it, that's the truth

Well, I give a sigh
As the boss walks by
No one ever talks to me
Or looks me in the eye

And I really should work
But instead I just
Sit here and surf the Internet

My cubicle, My cubicle
It doesn't have a view
It's my small space
In a crowded place
I sit in solitude

I haven't seen anything that has more truth in so few phrases. Kudos to Keith Hughes and Jym Britton for pull this one off, with style. Get the song from morningsidekick and play it loud in office.

How can I "Think Outside the Box" when I'm in the @#$%? box all day!

posted at: 16:51 | path: /fun | permalink | Tags: ,

Thu, 17 Aug 2006:

APC released version 3.0.11. I've been hunting the entire codebase for memory issues and even laid to rest the bug from hell.

Now, all that remains is for all new bug reports to come in.

Your parity check is overdrawn and you're out of cache.

posted at: 18:27 | path: /php | permalink | Tags: , ,

Wed, 16 Aug 2006:

The first lines of this was written around seven months ago, when I was itching to get a proper article published with the stuff I've learned over the last four years. Working on the dotgnu unroller, APC and debugging random bits of code had turned me into a gdb power user. Sure, there's lots I don't know about gdb and most of what I talk about is stuff you can read right off the man page. But, even these I had to learn the hard way and I wanted to get this out to the world in one compact vehicle. Here's the part1 or the so-called duh! component of the four-part tutorial.

Segfaults: No, this is not for developers. Any and everytime I've reported a segfault to a developer, the first thing they ask for is a backtrace. To put it simply, a backtrace is a log of all the calls leading up to the segfault and is invaluable when dealing with most segfaults, be it bad arguments, corner case inputs or plain stupidity (of the user, developer or even his counterpart on the server side). Getting a backtrace out is the step #1 for any bug report. But before that, you've got to figure out how to run the errant program inside gdb.

bash$ gdb ./sig11

(gdb) run argument1=0 argument2=1 file1 file2 ...

That starts off the program in gdb. Now, you can continue to make the program crash by doing whatever you were doing before.

Program received signal SIGSEGV, Segmentation fault.
0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6

To get a backtrace out of it is just the matter of a single command bt

(gdb) bt
#0  0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6
#1  0x000000000040054d in foo (a=0x0) at test.c:5
#2  0x000000000040058a in main () at test.c:9

But often, distros like ubuntu and fedora, ship binaries without any debug info, to save on install space. If you are debugging something real, better look for a package that says -debuginfo. Or if you were building your own programs, compile them with the -g flag (gcc -g test.c).

There are variants of backtraces. You can provide backtrace with a count or even can ask it to be more verbose about the induvidual frames (every method call occupies a frame, unless it is a recursive tail call ... *blink* *blink*). For example, I could run bt full to print out the locals in the ancestor functions directly.

(gdb) bt full
#0  0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6
No symbol table info available.
#1  0x000000000040054d in foo (a=0x0) at test.c:5
No locals.
#2  0x000000000040059f in main () at test.c:11
        xyz = 0x4006b3 "abc"
        template = "abc"

Frames: Frames need more exploring in most cases. The recent debug symbol version, embed the entire code in the function. The key commands to play around with are frame or f for short and the list command. Both of these combined, empowers the user to see the code that went into the binary.

(gdb) f 2
#2  0x000000000040059f in main () at test.c:11
11              foo(NULL);

You can move up and down the frame using the up and down commands.

Now, just seeing the obvious culprit might not have been enough. You might want to see code around the actual call to decide the error condition. That's where list really kicks in.

(gdb) list
6       }
7       int main()
8       {
9               const char * xyz = "abc";
10              const char template[] = "abc";
11              foo(NULL);
12      }

Actually, you can list arbitrary lines of code too. This is just too good for debugging a huge file of code. For example, if I wanted to see what's at line 1 of the file.

(gdb) list test.c:1
1       #include &glt;stdio.h>
3       int foo(char * a)
4       {

Frame info: The frame info commands let you look at the arguments and local variables quickly. Unfortunately, if your code modifies the arguments along the way, you'll just see the new value, especially on AMD64 and other places where there are enough registers to pass around arguments.

(gdb) info locals
xyz = 0x4006b3 "abc"
template = "abc"
(gdb) info args
argc = 1
argv = (char **) 0x7fbffff858

Print: these are my favourite type of commands. Mainly because they print out information based on the inferred types and dump full structures. For example, here's a tidbit of print commands used on some ZendEngine2 (php) code.

(gdb) p h
$2 = (zend_file_handle *) 0x2a9a433540
(gdb) p *h
$1 = {type = 0 '\0', filename = 0x0, opened_path = 0x0, 
      handle = {fd = 2, fp = 0x2, stream = {
      handle = 0x2, reader = 0x2a9a433548 <executor_globals+8>, closer = 0, 
      fteller = 0, interactive = 1}}, free_filename = 104 'h'}

Now, this tells me way more than the average printf does. Especially, if I don't know what I'm looking for. But sometimes, you want some "processed" data out. Maybe the hex version of something or someone's mangled the name with a NUL prepend. So there's still a printf around.

(gdb) p src
$1 = (zend_function *) 0x7fbffff260
(gdb) printf "0x%02x\n", src->common.function_name[0]
(gdb) printf "%s\n", src->common.function_name+1

These are the basic commands I use to start off inspecting a coredump or other misbehaving programs. But there are a lot more you can do if you are debugging a live program. My next tutorial is about attaching a debugger to a program and the extra stuff you can do with the process. Breaking, watching and the works.

Watch this space.

"The debugger is akin to giving the _rabbits_ a bazooka. The poor wolf doesn't need any sharper teeth."
                      -- Linus Torvalds, about NT's new debugger

posted at: 18:11 | path: /tutorials | permalink | Tags: , ,

Sun, 13 Aug 2006:

On this August 11th, I completed the third year I've been working in the software industry. My first day shall forever be burned in my memory, for that was day my ideals died. If I seem too cynical, too pessimistic about the software industry, it all started from day one. For that was one day that caused all five of us to leave eventually and destroyed any vestiges of loyalty we might have had. I don't regret having gone through all that, because I learned a few valuable lessons, just a few hours into the job.

So, all of us were picked up from campus on the very first day of campus interviews. We were all satisified and never attended any of the off-campus interviews that our friends who had got into Infy and CTS were going to. I even lost interest in my GRE score and that pacakge from Georgia Tech is still lying on my shelf unopened. So all idealistic and eager, we arrived in their Electronic City office, around 8:15 AM on that fine monday morning.

We were part of a batch of 68, who were all joining on the same date. There were students from all over - Pune, Coimbatore, Allahabad, Mangalore and Bangalore. So our exams ended by late June and the lab results for us weren't tabulated till date. Of the group, all eight of us and three others from Pune didn't have our final year marklists. After the employee IDs for four of us were generated, some guy realized this fact and literally threw us out of the hall.

"You didn't inform us that your marklists weren't out. We will call you back when you have a degree" - those were his exact words. We were told to go home and that we would be called back along with the October batch of new joinees. We were more scared than pissed off. Fresh out of college and with no job, we were pond scum in the job market mill pond.

First thing we did was call up our parents. We even called up our principal and placement officer. The latter earned our eternal scorn by washing his hands of the affair. And to compound the issue, there was an ongoing student protest in the university college.

In the midst of the rioting students, policemen with tear gas, my father and Joe's father managed to find that our marklists were ready to be tabulated. They managed to find the clerk in charge of this, drive him to the uni, get it tabulated, got him to take it to the VC's house and get it signed & stamped. And all this, they did before 2 PM.

After making 800 rs/- worth of phone calls and such heroics from our parent's, the marklists were faxed to the HR by 3 PM. We were very reluctantly admitted and our employee IDs generated. We had gotten in and we assumed that it would be a smooth ride from then on.

So the training began. Most of us spent the days on the back-row playing virtual pool, copter and other flash games. And it really pissed off our trainers when me and Sreekrishna walked out of the half-hour C++ exam about two minutes after it began, with near perfect scores. Hardly did I know that we would pay dearly for those stunts.

We were informed about a week into september that we were going to be posted in Hyderabad for a Telecom & Internet reqs. And I mean, just the three back-bench wise-asses. And when I asked about openings in their embedded wing, I was told that "In bangalore, there are only testing openings" (for freshers).

Off to Hyderabad, I went. Only to discover that after all this drama, I had been posted to a testing job. My job involved filling in an excel sheet with PASS/FAIL depending on whether my button mashing on the mobile phone caused it to dump core. And all three of us started working on our resumes rather than our day jobs. And then came the rejections. Sometimes outright rejections when I was passed over for people from known colleges - IITs, RECs, BITS and elsewhere. Talent just didn't seem to matter and mine was very hard to measure in an interview (I am useless on paper).

After six interviews and no job, especially that Unix & C rejection, I was disillusioned about a career in software. I decided that day and then that if I manage to escape, I'll never again work in a services firm as a cheap brain for clients somewhere.

But ...

Nothing motivates a man more than to see his boss put in an honest day's work.

posted at: 21:19 | path: /me | permalink | Tags: ,

Sun, 06 Aug 2006:

I have undergone a simple and very dramatic transformation. If you saw me tomorrow somewhere, you'd probably not even recognize me. Ever since december 2005, I've been growing my hair. No haircuts at all. I've grown my hair nearly six inches long in this period, but the hair grew all curly and complicated. It has been a pain to maintain and more often than not, got in my way than becoming part of who I am.

I managed to lose all my hair in one single day and feel the soft caress of the wind on my scalp. Also I rediscovered that feeling of freedom that can only come from doing something stupid. Maybe it does make me look like a terrorist out from jail, but I don't give a damn.

It is a custom in our family to do this on the sixteenth day after the death of a close relative. But this has nothing to do with that, it has everything to do with an august five years ago.

Hair now; Gone tomorrow

posted at: 22:11 | path: /me | permalink | Tags: ,

Tue, 01 Aug 2006:

I do not write this entry for myself. I write it on behalf of the soul departed. But when I climbed onto that bus to Cochin on saturday I was headed for a quiet sunday at home. But before I set foot in Cochin, I received the news. My uncle - V.M Venugopala Menon is no more. After nearly a decade of struggling with diabetes and its complications, he finally succumbed to the silent embrace of death sometime during the early hours of sunday.

For nearly eight years, he had a daily dose of human insulin to keep his blood sugar under check. During my final year in college, he lost a toe and would have nearly lost the foot as well. Ever since mid-2003, his kidneys had started to fail. But he was a man of iron will and still went ahead to enroll as an advocate, travelling nearly the length of Kerala to attend the ceremony in Kasargode, lugging a peritonial dialysis kit. To have that tube jammed into his lower stomach and have the liquid pumped into him was something he hated. But the moment the peritonium lost the osmotic qualities, the horrors began.

When I saw him month after month, he seemed to grow shorter and weaker. The haemodialysis which started off on a weekly basis, slowly became a necessity every other day. A complete kidney dysfunction, combined with the diabetic's slow healing, totally ran riot in his system. To see him on that wheelchair with his swollen feet, bitter about the world in general, yet prepared to fight till the end, if only for his children. His resolve only grew stronger as his health deteriorated.

We've always had our differences. But the blame is often quite evenly distributed in that. Of course, in our last conversation he probably forsaw this day. He was talking about the things he had left to see - my cousin's marriage, her brother's job. And then he said "Marikkenda samayam varumbayuthakum, ethokke nadakanam". And yet, didn't.

Maybe it was all for the good, to not let a man suffer so on this God's earth.

What we have done for ourselves alone dies with us;
What we have done for others and the world remains and is immortal.

posted at: 02:22 | path: /me | permalink | Tags: , ,

Thu, 27 Jul 2006:

Words fail me. To explain the grandeur of this lake, I find my vocabulary lacking and my prose insufficent. To compare it with anything else I have seen would be in vain. But to do justice to this travelogue, I should at least make an attempt. The Pangong Tsu is one of the must-see sights in all of Ladakh.

All six of us, headed out of Leh towards Pangong Tsu sometime around nine. After a good hours's drive out, we passed Shey palace and through the familiar roads we had passed yesterday. We stopped at Karu to buy provisions and stock up on food. After leaving behind Karu, we speeded up and made good time on the valley floor while the going was good. We passed through some interesting country, past a few military camps (Chusul Warriors and Eagles), on our way up to the high passes. This region, being so close to China, is a highly militarized zone. The army presence however is hardly obstrusive and caused no hassles for us. We, of course, had an inner line permit for travel this close to the border.

Sometime near noon, we started climbing up to Chang La, which after Khardung La and Taglang la, is the third highest motorable road in the world. But this time around, we were properly acclimatized, properly clothed and the sun was shining bright. After a couple of half-hearted snow fights (yes, I had bought gloves for this trip) and a hot cup of cinnamon tea, we descended into the greener valleys beyond.

The valley was beautiful, green and had wild horses grazing all over. We even saw a small herd of domesticated yaks. After passing through a few streams and past muddy plains, we reached a valley unlike any other we'd seen so far. The valley floor was covered in sand, looking not quite unlike a beach, but without an ocean to match. We really wanted to go down and investigate, but we were eager to reach the lake and find accomodation.

But we soon found the missing ocean. The Pangong lake is salty, though not in a sea salt way. This lake tasted more of potassium salts than the usual sodium chloride. The lake has no drainage and has fresh glacial water flowing into it. There are hardly any fish in this lake and even fewer algae. All this results in a crystal clear, blue lake of picture perfection.

The place we had landed up was called Lukung and there was a guesthouse there with three rooms. And due to all our hurrying on the way (oh, I kid) we were the first ones to get there. Our guesthouse was hardly half a kilometre away from the lake, though you wouldn't feel the distance because there are no trees to give you a sense of scale. Basically it felt like you were next door to the lake.

To complete the illusion of the sea, there were sea gulls here, which feed on the crustaceans who seem to be thriving in the shallows. We wasted some time skimming stones on the flat surface of the lake. After sending enough stones skimming through the lake's flat surface, three of us took off our shoes and stepped into the water. There is a bay formation around the corner where we were and the water was shallow for as far as I could see. The first few minutes felt cold, then it was as if my body had adjusted to the fact that I was going to go ankle deep into cold water.

Having wet our feet and generally enjoying the water, we climed out and decided to get higher before the sunset. We climbed the nearby hillock and got to a higher view point to watch the entire curve of the bay from above. As we watched the sun set behind us, what we were seeing turned into a moonscape of sorts. The salt deposited by the lake on the flood plain turned the ground into this bright white surface while the dark blue of the lake made for contrast.

There are no power lines in Lukung. We couldn't charge our cameras, nor could we transfer the photos out of our cameras. The dinner consisted of maggi instant noodles, cooked watery and omlettes. We at that in the dim candle light and decided to call it a day. But one step outside and we were stunned by what we saw in the sky.

Whole sky was lit and in the clear air of the high altitude, I could see more stars than I have ever done before. As we tried to identify the stars, we were unable to find the Orion in the night sky. There were too many stars to actually spot the characteristic three star belt of orion. But as we were hunting for Orion, we saw something move very quickly across the sky. The reddish spot moving from South east to North west was probably a sattelite. And in a few minutes, we spotted a couple more of the sattelites moving in polar orbits. As we were scanning the skies for more, a shooting star passed by in the N.W sky. We spent the good part of the next thirty minutes watching the meteors expend themselves in the atmosphere. It was a star gazer's dream and I shall not forget that sky for a while.

The guesthouse was sheltered behind a hillock and safely out of the cold wind. Without any heating of any sort, by midnight all of us were feeling hot inside our blankets. But generally I had a good night's sleep. We had planned to wake up really early to see the sunrise and I had my alarms set for quarter past five.

I woke up sometime past five and discovered that the sky was too cloudy for a good sunrise. Vinayak and Rishi were already out on the bay, scaring birds (*heh*) and trying to photograph the rising sun. After a few more minutes of procrastination, I walked up to the bay and watched the light reflect off the lake surface. It was brilliant - to watch a lake turn into pure silver and slowly darken into a brilliant azure.

I took a walk along the length of the bay. It was a really windy day and the lake ripples were becoming nearly full-fledged waves. I took an hour long walk towards Pangmik, though turned back because I had no idea of when we would be leaving camp. My only lasting regret is that I didn't climb the zig zag trail up on the mountain nearby.

After a bit more time of non-commital wandering, I had breakfast and finished catching up on my journal. We headed back to Tang-Tse and passed by those sand drifts. But this time around, we went down and I even took a small bag back for Premshree. At Tang-Tse, we ran into an archery competition. After watching the archers for a while and having had lunch, we headed back up Chang La. Except for running into a Himalayan Marmot sunning itself on a rock, the rest of the journey was uneventful.

On the way to Leh, we had an amazing idea. Why not stop at Thiksey and enjoy the hospitality of the Chamba restaurant. I had honey pancakes and they were just awesome. After all that, we finally got back to the guesthouse. And we had a few laughs on the way thanks to the B.R.O's sense of humour.

We had gotten to the hotel a bit early and were not quite ready to turn in yet. So we sat out there in the courtyard talking to the other guests. There was a french dude and a british dude playing the guitar and an irish couple. There were some quite interesting conversations about a lot of things. Then something weird happened. The irish couple were smoking something and quite out of nowhere, started passing around the cigarrete. The blonde girl sitting next me offered me a drag and I said NO. The french guy shrugged, the brit said good for you ! and *then* the girl said "When I saw your hair, I thought you smoked pot". And somebody said that right now I looked like a pot plant. Anyway, the dinner had arrived, which I promptly bolted and called it a day (because a day it had been).

You can't judge a book by the way it wears its hair.

posted at: 18:14 | path: /travels | permalink | Tags: , ,

How many times have you run into a CD which would let you play a file off the disk, but wouldn't let you copy it out ? It is one of those irritating problems, which should really have a simple solution. Most AVI files are fairly error tolerant, they wouldn't really care if a few KB are missing from the video stream, as long as they can locate the next key-frame, it will keep playing forward. So playing an AVI off scratched media will often work without much disturbance, but it would be near impossible for a normal copy program to create a copy of the file.

Even though you can accomplish the same task with dd noerror, it tries to recover every bit of data possible which is not something that you require when you are trying to copy a large number of files off a DVD. So I basically hacked up a small program which would leave zero-padded 4k holes wherever the disk area wasn't readable. The program accomplishes this by the easiest way possible - by running the following in a loop.

if(i < len) 
        i += BLOCK_SIZE;
        copyblock(dest, src, i, BLOCK_SIZE, len);

Now, that'd obviously result in a very near fork-bomb of your machine while copying a 700Mb file. So I added code so that a fixed number of processes are spawned to start off while the next process is started off only after one of the processes die off. Also, to improve the copying efficency I increased the BLOCK_SIZE to 64k but still wanted to make sure that the size of the errored blocks weren't more than 4k. So I added a retry section, which would spawn 16 processes each handling a 4k block of the errored 64k.

deadproc = wait(&status);

if(WIFSIGNALED(status) && WTERMSIG(status) == SIGBUS)
    /* retry */

Now the awesome part of this program is not how it works. The really cool thing is to attach strace to this and watch the fork() in operation as well as the SIGCHILD return back. Also it is a very simple example of how something like APC shares memory between all the apache children with mmap (though without any locks). There might just be a better approach for this particular problem, using signal(SIGBUS), but that is left as an exercise to the reader.

Here's the code for the curious. And remember, if you are still running on x86 32 bit, memory mapping big files might cause your OS to run out of address space.

(1) Never draw what you can copy.
(2) Never copy what you can trace.
(3) Never trace what you can cut out and paste down.

posted at: 11:44 | path: /hacks | permalink | Tags: ,

Wed, 19 Jul 2006:

By the 22nd, we had our plans straightened out and fully booked for. Our local monastery (gonpa) tour started out an hour late, by about 9 in the morning. After picking up Amol and Vinod, we headed out of Leh. The sightseeing circuit in Leh is heavily controlled by a sort of guild which operates on a fixed rate and is completely protectionist. So all operators basically charge the same rate and there are unofficial checkposts to prevent other taxis from doing tours. Anyway, we had got a pretty decent Toyota Qualis to travel in, but the driver only spoke broken hindi.

Shey Palace: Our first stop was Shey palace which sits on a hill facing a green valley. The green of the valley contrasted heavily against the barren hills that surround them. Where there is water and shelter from the north winds, mankind has made its niche in ladakh. The palace had a buddha statue, but I was more interested in the view outside.

Thiksey Gonpa: Next up, we went to visit the gonpa at Thiksey. An amazing monastery and school, bang on the leeward side of a hill almost covering the entire hill. We stopped for breakfast (a second one) at the foot of the hill, in the Chamba restaurant, owned and operated by the monastery itself. The food was good and the ambience was calming. Thanks to a customer endorsement for metrogyl, we were all in good spirits before we went up to the monastery.

We went uphill by the road. The monastery was a relatively big one, with an entire wall full of murals. The view from the monastery was amazing and so was the monastery itself. The central courtyard walls were covered in murals and so were the inner walls. I was quite surprised to see a candy bowl near the deities, full of chocolates and biscuits. We hung around there for a while, looking at the various message scrolls hung up there. After nearly a good hour spent there, we headed out.

Hemis Gonpa: After Thiksey, we headed towards the hidden monastery of Hemis. The monastery is completely hidden away amidst the hills and is not visible till you take the last turn. The monastery was relatively inactive and cold like a fridge. You can literally sense the past being preserved in this half-frozen temple.

Matho: We moved on from Hemis on to the village of Matho and into the monastery. The monastery stands as before, on top of a hill overlooking the village. The gonpa is famous for its oracles, who are possessed by the spirits of good and demonstrate their other sight by running on the walls blind folded. But by the time we arrived, we were possessed by the spirits of photography and all of us were more interested in clicking away than actually seeing the sights.

Stok Palace: The last stop was Stok palace, which is still inhabited by the royal family. Only five rooms were open for visitors in the palace and photography is prohibited inside the palace. The display cases are full of preserved jewelery, especially turquoise and jade. The one thing that caught my attention was a Damascus steel sword, tied into a neat knot. Legend has it that some king grew so strong and powerful that he had no regard for the holy men of the land, so one of them put the king in his place by performing the simple task of tying the king's sharp steel sword into the knot. But other than that curio, I found the palace quite boring.

Nevertheless, it was a pleasant day's outing. The roads were good, flat and generally straight, the sky clear and we passed through some very interesting countryside. And I learned my first word of ladakhi - Julley !.

Everyone has a photographic memory. Some don't have film.

posted at: 00:47 | path: /travels | permalink | Tags: , ,

Tue, 18 Jul 2006:

The problem with taking photos in a moving car is that it is very hard to get the baseline of the photo on the horizontal. So eventually to get a decent pic, you need to rotate the photo. But after rotation, you need to crop it back to a square. All operations which take time and is relatively annoying. So, I wrote a gimp script.

  r = math.sqrt(x*x + y*y)/2
  # as calculated for top left corner
  theta = math.radians(45)
  phi = math.radians(45 + angle)
  dx = r * math.cos(phi) - r * math.cos(theta)
  # -1 * because as the angle comes down, our y decreases
  dy = -1 * (r * math.sin(phi) - r * math.cos(theta))

Basically, this python-fu script (rot-crop.py) does both operations together and gives you a cropped rectangular image.

rotated 6 odd degrees

Script-Fu is esoteric stuff, but the python plugins can actually be read, understood and written by mere mortals like me.

My geometry teacher was sometimes acute, and sometimes obtuse, but always,
always, he was right.
       [That's an interesting angle.  I wonder if there are any parallels?]

posted at: 02:22 | path: /hacks | permalink | Tags: , ,

'Ti's a tale of original sin. Is mankind selfish and evil by nature and that civilization has tamed them or did the noble savage become corrupted by the evil influence of society ? The debate is not a new one, with Hobbes holding fort for one side and Rousseau cheering on the other. Matt Ridley rides both paths deep into the subject, but to realize that neither can explain our actions completely. For the fans of Dawkins and Selfish Gene, this book will serve as a good second volume of the story, devoted to the human animal.

In the first chapter of the book, the concept of the Greater Good is dissected and laid bare. The selfishness of the individual is not a direct consequence of selfish genes and yet an individual becomes less of a well oiled machine aimed at survival and reproduction. Then we examine the myth of self-sufficiency, and the advantages division of labour brings, which can serve for the maintenance of societies. More accurately, trade and skilled labour.

But before we get too much into the territory of hawks and doves, the semantics of benevolence is put to the test. Sympathetic responses carry with them an obvious emotional reward on satisfaction. If helping somebody did make you feel better, was that an act of true altruism ? But before we destroy the whole concept of altruism, we head back towards understanding why it feels good to do good.

Duty and the Feast: Why would a hunter hunt when most of his catch is shared out to males who didn't participate in the hunt ? The sharing of meat is prevalent while plant produce is hardly ever shared out. The answer seems to be that mankind were poor hunters to start with. We were completely unequipped to be carnivores and meat was mostly a matter of luck and providence. So sharing the meat meant that the risk of going hungry was spread out among more people. It is a very good way of reducing the risk without reducing overall supply. But what's to stop the idler from exploiting the hunter and not having to work for the meat ? In other words, who'll pay for the lighthouse when everybody gets light for free ?

There's the point to redefine yet another human concept - Prestige. No man can eat a whole mammoth, but by giving it out he can exchange the perishable good into something more durable - social status. The hunter who regularly brings in meat for the tribe is likely to end up with the best women (directly or indirectly) than the idler who sat by the trail picking his nose. The social marketplace repays meat with other favours, in some sort of indirect reciprocity.

But there are other more compelling reasons to give. Hunter gatherer societies seem to have achieved the zen of comfortable living. To live comfortably, aspire for moderate means and achieve them. Rather than go further into luxury and risk jealousy, they stop. In most of these societies, to be rich is to refuse to share. An eskimo caught hoarding can be killed. The taboo paves way for equality and eliminates the class struggle, which has been the cause of economic strife for most of the modern era (count the revolutions).

Morality is drawn into the equation in the next chapter, with the rationality of most of our behaviour questioned. Often we are turned away from the rational alternative of say, taking a bribe, embezzling money or murdering your neighbour, by simple fact that we have evolved a sense of Right and Wrong. Unconsciously or even sub-consciously, we are aware that trust in us by the rest is worth preserving. Trust is the coin of trade and we are eager to prove our capacity for altruism in order to gain it.

Hypocrite's Oath: We admire selflessness. We love the Mother Theresas of the world. But we do not let altruism rule our life directly. We simply do not practise what we preach. The more other people people practise altruism, the better for us. The more we follow self-interest, the better for us. So realistically, we should follow self-interest in private, yet encourage altruism in public. And that we do.

You and what army ? : Ridley, then goes on into the root of human conflict. The group warfare of mankind is not unique among species, but only we take it to a scale of complete destruction. The extreme nature of the armies of mankind rise from the built-in conformism which lends itself well to the organization of relatively homogeneous armies of vast numbers. Face it, we're not very strong individualists.

There were wars, trade and laws before civilization took root. So what did exactly civilization bring in ? The first man who fenced off a part of pasture and said 'Mine' and found people simple enough to accept it was its true founder. If only someone had realized how much misery and horror mankind would have been spared if not for that ...

Property is one of the corner stones of civilization. Without the concept of ownership, the common turns into what is known as the Tragedy of Commons. Give a man a desert to own, it shall become a garden in nine years, but give him lease of a garden for nine, it shall become a desert. Arthur Young was spot on in saying that the magic of property turns sand into gold. But as in the fable of the man running, property meant that you could always want more and indeed own more. With that crucial step, we had broken through the taboo of hoarding and the first steps into civilization had taken mankind away from the true and narrow path of contentment.

So all in all, we were built by selfish genes who found it more convenient to make us social, trustworthy and co-operative beings. The essential paradox contained in Man remains unresolved. We are not the blank slate on which civilization has written and yet we are no more our past selves than you were the kid in your baby pictures. We have changed, but we are yet the same.

We do good and we do bad, but truly we are neither.

In Nature there are neither rewards nor punishments, there are consequences.
               -- R.G. Ingersoll

posted at: 02:13 | path: /books | permalink | Tags: ,

Thu, 13 Jul 2006:

It has always been Us vs Them, only the definitions of either group have varied for mankind. The fact that we are "We" isn't due to "Me", it has always been because of "Them". The phenomena is not restricted to the world outside, but into the air conditioned world of the software industry. Rather than just poke the sordid underbelly of my chosen profession, I wanted to understand how exactly this came to be.

The polarization evolves from a geographical split. An indian wing of a US MNC or an offshore ODC of one could serve as a sufficient model. The cultural and timezone differences aside, the system functions smoothly when the wing is small, yet doing really important things. But somewhere during the latter phase of its growth, things start to go wrong. Work that could have been done by "Us" (yes, it begins...) is kept for "Them", even though they are understaffed or overworked. Suddenly a log jam starts to appear in the smooth flow and you can literally feel the disparity in treatment.

The problem has its roots in communication. While the communication channels remain person to person, as is common with a small outpost of a company elsewhere, the problems, if any, remain purely personal and has none of the communal hatred of the Us vs Them. Then arrives the manager who, for lack of other tasks, takes up the job of communication. Well intentioned maybe, but the moment an engineer has to send a mail to a manager who has to communicate it elsewhere, the seeds of disparity are sown. Information shouldn't need to go up levels to traverse teams. Hierarchies get in the way of information and the barriers start to rise as growth pushes the pyramid upwards.

But there are some managers who seem to have avoided this problem by being so transparent that they seem to be hardly there. My career so far has been short and uneventful (mostly), but it has been my pleasure to work with two individuals who, in my opinion, have discovered the zen of management. To be completely frank with their reportees and extremely diplomatic with their superiors is how they function. Compared to the fold like umbrellas attitudes others have shown towards authority, I have found the other strategy to be better at motivating me. Transparency means that you know why you have to work harder and what's on the line here. Consequences of a boss's displeasure isn't enough to make me work hard, but a real failure of the project is.

But back to the original problem, beyond communication and onto competition. The moment a manager starts collating his bug reports and TPS sheets, it becomes a game of numbers. Now, if you haven't read How to lie with statistics, you should. So some team will have done a bit more or a bit less and the obvious comparisons between the two teams will be made. Unwanted parallels drawn to the disadvantage of the team lagging behind. Once this has been dragged out into the open by somebody, it becomes an open contest for next year.

Competition works. Margaret Thatcher was a great fan of the concept. But a software firm is more of a non-zero sum game than the free market, especially since the customer (who wins) isn't playing the game at all. Like the old game of prisoner's dilemma, the teams draw up their perceived benefits of co-operation or defection (more accurately, delaying or denying help). But as the game suggests, it should easily fall into a cycle of co-operation, if communication was open and clear.

That is where the next tragedy of the software industry comes in. Attrition rates and job hopping essentially means that the shadow of the future is rather short. For the average employee, he'll not be here to play the next round. So as in the one-off version of the game, his obvious choice is to always defect.

In this race to be the one first up with a product, to make the most money, to get the most kudos for their product, a silo of thinking develops. People start reinventing the wheels merely because the other team doing the same thing elsewhere is not co-operating (also because they stand to lose a chunk of the credit). Time and money wasted. Important products slip merely because a manager doesn't want to ask for help.

The lack of communication explodes into a cycle of non-productivity and credit grabbing which are short term benefits paid for by somebody else in the long term. But in reality, you merely end up paying for somebody else's mistakes of the past and merely passing on new burdens to your successors. Unburdened by any past sins, the first generation which indulges in these seem to come out winning.

What could have been a peaceful working environment is now wracked by underground politics which cuts through the basic helplines of the employees - other employees. The command structures demand no creativity, a task which is solidly obeyed in levels below ("the easiest thing to do is to never have good ideas"). Thus a hierarchy has divided and ruled for its own benefit and paved way for its ultimate downfall, merely by replacing the individual with a collective Us in constant conflict with Them ("We'd always been at war with Eurasia").

I've seen the disease and I haven't found a cure yet. Prevention yes, but a cure no.

"Don't worry about the mule. Just load the wagon."
                                -- Project Management Simplified

posted at: 04:01 | path: /observations | permalink | Tags: ,

Wed, 12 Jul 2006:

June 20th: The day long journey to Leh was really tiring and I had planned to sleep till it was late afternoon to compensate. But 'twas not to be, Vinayak's cellphone woke me up around ten. None of our phones were working in Leh. Only BSNL and Airtel have a presence in the valley, but even Airtel allows only post-paid phones to roam inside Jammu & Kashmir. So I asked the hotel guy for directions and headed out towards the town to make some calls.

Our guesthouse was called Tse-Mo View and somehow seemed stuck between a pun for See More and Same 'Ol. It was stuck on the westward side of the palace, in between the Tse-Mo monastery and the Santhi Gonpa. The market was hardly a kilometre away, but the getting used to the alitude is more of a gradual thing.

But it was a bright day, with blue skies like I've never seen before. We spent most of the day just sitting around in the hotel. There is an SBI ATM in Leh which is a boon to travellers and I headed there to withdraw some much needed cash. After an attempted lunch at the La Terrase, we stuck around in the guesthouse, waiting for Amol and planning the trip ahead.

After meeting up with Amol, sometime around 7:45, we went to a pizzeria where a TV set was attracting football fans. After a pleasant half-hour of discussions, we got back to the guesthouse. My dinner consisted of a flattened noodle like preparation called the Thupka.

June 21st: We had plans to do some sightseeing today, to visit some nearby monasteries and local sights. But the obvious communication gap caused by the lack of cellphones prevented us from co-ordinating this with the other group. But we woke up early, nonetheless. Eventually we decided to take it as another day off to acclimatise.

But we managed to meet up with the rest of the gang by lunch and planned to trek up to the Tse Mo monastery and the Namgyal palace. The palace and the monastery sit up on a crumbling gravel sided hill. There is a motorable road which goes up to the palace, but you could see a shorter foot trail on the north face of the hill. We decided to get our body into gear with a climb up. Rishi however was experiencing a full download, if you know what I mean, and did not accompany us.

So, equipped with a water bottle full of honey lemon water and morale high, we ended up behind the masjid trying to find our way out of the twisting lanes. The lanes finally gave way to steps up to the palace.

The palace overlooks the entire town and is under restoration. It has the smell of fresh varnish and drying soft-woods. Considering the traditional architecture, the palace does indeed look like it is fit for a king. But to the naive traveller, it looks like a dive suitable for somebody with an aversion for windows. The central courtyard on the roof was quite interesting, as was the wood carvings on the pillars. But we still had the foot trail up to climb and did not waste too much time inside the palace.

The trail up was really scary. Until I got some confidence and my second wind, I was really not sure I'd get to the top. Our shoes weren't made for the terrain and neither was our minds. Every slight slip you made, every time you looked downwards on what awaits on a fall, the message was hammered in by your brain. We took about forty full minutes to climb the three kilometres of the gravel trail.

We got to the bottom of the monastery and then the next challenge awaited. The monastery was on top of a few more rocks which seemed like a bad idea to try. But given the success we'd so far had, we went for it and climbed to the very top. And the view was definitely worth the effort.

After spending twenty odd minutes on the top, we started to walk down by the road. Soon we realized that the road would take us far out of Leh, before it took us home. Despite my objections on climbing down in the dark, we took the first downhill path we found. The path got a little tricky further down, but we literally crawled on all limbs to get across to the safe path down. The it was all downhill and we literally ran down the slope.

Took a long look backwards and ended up at the hotel. But instead of heading to the town for dinner, Vinayak decided to eat at the guesthouse. I headed out to the Ibex restaurant for a completely horrible dinner. Thanks to a crowd of american tourists, our orders got pushed back for nearly an hour and eventually we got something we didn't order. Anyway, after a few complaints to the manager and a 20% discount, we headed back unsatisfied with the food.

I was extremely tired, but still slightly excited from the trip up. I slept like a log.

Fain would I climb, yet fear I to fall.
               -- Sir Walter Raleigh

posted at: 21:09 | path: /travels | permalink | Tags: , ,

Mon, 10 Jul 2006:

There are times when all things come together. Yesterday was one of those days, when we all piled onto spo0nman's balcony to watch the football on a big screen. After pulling together sabiokap's TV, a cable connection and a projector on rent, we had our own party at midnight.

Despite premshree preparing various cocktails (ok, it had coffee, kiwi fruit, watermelons, vodka, rum, wine and beer in it), we all managed to have a lot of fun. We watched the match, into extra time and till Cannavaro lifted up the cup. The absolute shocker was of course Zi Zou's head butt, but apparently Materrazi should've seen it coming. What ever goes around, comes around.

But lots of fun and sugar (*giggle*).

There are no winners, only survivors.

posted at: 19:16 | path: /parties | permalink | Tags: ,

Sat, 08 Jul 2006:

We had hired a Sumo to travel from Manil to Leh. The travels guy had said that the journey would get us to Leh by late evening on the same day (16 hours). But when we were woken up by the driver at 2 AM, we realized that the journey according to him would take 22-odd hours instead of the barely bearable sixteen. After just two hours sleep, three of us packed ourselves into the back of the Sumo.

The initial two hours of the journey was in nearly complete darkness to reach Rohtang La by day break. During the descent from the pass, we could see the east skies slowly brighten up. After stopping in Khoksar for tea and snacks, we headed towards the chenab valley.

Past Keylong, the roads got really bad. This can hardly be called a highway. After passing keylong, we ended up at Darcha, near a bridge across some river. We stopped there for breakfast and had an excellent stuffed paranthas. I'd been eating the cheese slices with anything I get and this was no exception.

At Baralacha la, we all got out and had our first contact with snow. It was probably a mistake to pick up snow without wearing gloves, but I didn't have any gloves at all. But I did pick up some snow and ended up with my hands looking rather blue.

The whole journey was made even rougher because of the climbs and the descents. The various passes we went up and the further climb down was playing havoc with my breathing. To climb, six thousand feet in less than two hours and to climb down immediately is not my idea of fun. Somewhere in the descent to Keylong, I had thrown up.

Sometime near two, we passed on from Himachal into Jammu, into what passes for plains in this country of hills. The Sarchu plains would be more approriately called a plateau and is one of the first high alitude camps we ran into. We stopped there for lunch and as usual had the parathas, dal & rice. While we were in Sarchu, the wind picked up speed and it started to snow. As much as I'd have loved to see the snow flakes float down, I was more afraid of the road ahead.

We had to make it to Pang before five or we'd be forced to stay out of Taglang La till the next morning. The pace picked up and the driver literally drove through the road and the potholes at break neck speed. We eventually made it to the TCP at Pang minutes before five and were let through ahead. And we finally entered Ladakh district.

At this point, I was so high on adrenaline that I could hardly sit still inside the jeep. But Vinayak was looking very sick and tired. While we passed near all the eroded mountains of Pang, which looked very much like somebody had carved out a temple on the hills and up into the plateau (yet again plains) of Moray. Since I couldn't open a window without Vinayak complaining, there are no photos of that brilliant thousand foot (or more) canyon cutting through the edges of the plateau.

We went off-road from this point to cut across the plains directly instead of the taking the winding road that goes around the highest points. The experience was like something completely out of a hummer ad. That stretch of around forty kilometres was the most exhilirating drive of the trip, driven with the pedal to the metal.

Then we slowly descended to a much milder pasture country and joined back into the road. All those chocolates were working their magic and I was still in the best of my moods, though filtering out water through my body like a sieve. But we were faced with the final climb of the trip, the heights of Taglang La.

Taglang la is the world's third highest pass in the world. More accurately it is the 3rd highest motorable road at 17,582 feet up. We reached the top of the pass sometime nearer to 7 P.M with the sun setting in the background. I ran through the snow which turned out to be quite thick and soft. I broke through the top and ended up ankle deep in freezing slush. My other companions preferred to sit around in the shelter of the jeep, a wise choice as I was soon to find out.

While on the descent, I kept drinking water out of Vinayak's water bottle. But the constant stream of cold water into my body core, finally made me throw up (again). So here's a lesson for you - carry insulated water bottles. Anyway, I remember asking Rishi chocolates !!, to which he replied that it is in his bag. Drowsy as I was, I thought he was holding his bag when he was only sitting there with his hands in the jacket pocket. After stopping at Upshi for dinner, where I just had as much hot tea as I could, we headed on to Leh.

Closer to midnight and much closer to Leh, I had started having day dreams. I mean it was totally weird, I had my eyes open and I was seeing what was happening, but my interpretations were radically different from what was. I mistook the starry sky for some skyscraper and even tried to find a remote to change the channel (no, that's not a TV ... that's the windscreen of the jeep). Calorie deprivation, sleep deprivation and oxygen deprivation - is a very bad combination.

But by midnight, we were at Leh and driver dumped us at a guesthouse (previously arranged, I suppose). We pulled all our junk into the room, got an extra bed and were asleep before our heads touched the pillow. We wouldn't wake up till the next day for lunch and that plan, we stuck to.

Thus ends our journey uphill to Leh and what a journey !

The longest part of the journey is said to be the passing of the gate.
               -- Marcus Terentius Varro

posted at: 21:09 | path: /travels | permalink | Tags: , ,

Fri, 07 Jul 2006:
We have bigger houses but smaller families;
more conveniences, but less time.

We have more degrees but less sense;
more knowledge but less judgment;
more experts, but more problems;
more medicines but less healthiness.

We’ve been all the way to the moon and back,
but have trouble in crossing the street to meet our new neighbour.

We built more computers to hold more copies than ever,
But have less real communication;

We have become long on quantity,
but short on quality.

These are times of fast foods but slow digestion;
tall man, but short character;
steep profits but shallow relationships.

It’s a time when there is much in the window,
but nothing in the room.

              -- The X IVth Dalai Lama

There's much on the mind, but little in the heart.

A paradox is merely a wedge into reality's fractures

posted at: 20:12 | path: /philosophy | permalink | Tags: ,

Wed, 05 Jul 2006:

Whenever you run into a pessimist claiming "It can't get any worse", you'll find three optimisits correcting - "Oh, yes it can. The journey to Delhi and further up to Manali belonged to the optimists. After reaching Delhi really early, we managed to find the travel agent and camp there for most of the day. Despite the rise in temperature and the associated philosophical discussions (of the psuedointellectual variety), we managed to pass time very easily.

The bus ride wasn't very comfortable. I find it difficult to find proper leg room in a good bus and this was beyond awful with the couple in front (quote un-quote) making it a little harder than normal. Eventually, I was sleeping on the aisle of the bus (does somebody remember the return from calicut ?). But the first views of Himachal Pradesh was of a peaceful, green and unpleasantly hilly (opinions of non-bus travellers differ). The weather was amazing, with a bright blue cloudless sky and bright sunshine.

After reaching manali, we were able to get in touch with the others (who were already in Leh). And finally in the evening, in an attempt to get used to the altitude, we took a long walk to the river Beas and then up till the Hidimba temple. The weather had turned a bit chilly, but we didn't mind it so much.

Finally, nearly at midnight, we all turned in to sleep. The car to Leh would pick us up sometime around two in the night.

It gets better, but only after getting a lot worse.

posted at: 19:14 | path: /travels | permalink | Tags: , ,

Mon, 03 Jul 2006:

Back from ladakh. Nearly two thousand photos to sort and upload. More updates on the trip as soon as I settle down.

Got back in one peice. Thank God :)

The world is a small place.
But travelling makes it bigger.

posted at: 21:14 | path: /travels | permalink | Tags: ,

Fri, 16 Jun 2006:

So I'm travelling to Ladakh for a two week trip. Ladakh literally means land of high passes and is sandwitched between the Karokaram and Himalayan ranges. The area is respite with lakes, steep passes and rugged beauty. It is almost akin to visiting Tibet, without actually leaving Indian territory. The J&K state is not known for its political stability and I don't expect a smooth passage through Ladakh, though I do plan to arrive back home in one peice and alive.

Leh is probably going to be the base camp for all of our expeditions outwards. The current route traces itself through Rohtang La (13,000+ ft) and upto the passes at 17,500 feet. The passes are open only between July and September, but we are hoping that the mid-summer Sun has cleared up enough snow to take this route rather than the Srinagar-Kargil (NH1) road which was the scene for the 1999 exchange of fire with Pakistan. Anyway, first stop on the route is the hill station of Manali which is a pleasant enough place and lies on the ancient Silk Route towards the middle east.

This trip is so cool that I had to buy extra sweaters for it. Jokes aside, I have never seen snow in my life. I have always lived in the tropical climes and am accustomed to 20-35 Celsius days. So I expect the mountains to be a challenge to get used to, in addition to the normal altitude sickness. All in all, I think it will be one of those experiences which you can think back to and draw strength from, not just a fun fortnight out on the hills. I can already hear myself say "I can do this".

Wish me luck.

What did Mohammed do when the mountain didn't come to him ?

posted at: 15:53 | path: /travels | permalink | Tags: ,

Thu, 15 Jun 2006:

For almost two years, I've been trying to explore and understand the social situations which have slowly built up the strict roles assigned to the sexes. From the husband trying to bring home the bacon to the housewife packing lunches for their children, it has somehow always struck me as a social conformity rather than a natural state of affairs. But with every bit of new knowledge I gather, I have come to understand that this is merely a snapshot in the life of mankind (womankind too). With all that in context, I bought yet another book on genetics - Adam's Curse.

'To be born woman is to know
Although they do not talk of it at school
That we must labour to be beautiful.'
I said, "It's certain there is no fine thing
Since Adam's fall but needs much labouring.

                     -- Adam's Curse, Yeats

We are all born women. Until the sixth week, a foetus is developing identically for both sexes. But why do we men need to exist in the first place ? As it turns out, there are other genes that exist in our cells than the nuclear DNA that we commonly exchange and intermix during reproduction. The reason men exist is because when two normal haploid cells combine, the two cells' mitochondrial DNA end up fighting to their death. Somewhere in the dark distant past, the cells discovered that they could fix this match by depriving one cell of most of the cytoplasm. Those crippled cells, with a few mitochondria churning out O2 free radicals and literally burning themselves up ended up as representative of the male. And they basically fought the system by making up for that in numbers.

But that can't be the only reason to have all the rampaging testosterone fuelled men ? Well, it turns out that the mutation rate is much higher than in the other gamete and it all explodes into randomness. The Y-Chromosome is the only record of this chaos, since it does not recombine with any other and reduce the mutation damage done to it. It is on a slow but sure path to ending up as completely damaged junk DNA.

With the current population and extrapolating the mutation rate over the next 5000 years, the Y-chromosome would have to be exceptionally lucky to survive that many cell divisions. Unless our non-Junk segments of the gene pattern managed to cross over onto some other chromosome,the Adonis Chromosome according to Bryan Skyes, the future of men is a dead end.

We're merely a passing fad.

<he> Was it good for you ?
<she> Read my blog.

posted at: 16:47 | path: /books | permalink | Tags: ,

When the month of may dawned, I had promised myself I'd walk 221 kms before the month was through. But 'twas not to be, when I got called home for more than a week and all that energy got channeled out for a different use. But in the last thirty days I've been in Bangalore, I've walked 261 KM. Approximately that is equivalent to walking in to work every day. Over the period, I have gained 4 1/2 kilograms, reintroduced breakfast into my diet and cut down my coffee consumption to just six cups a day.

Take a walk. It will change your life.

I love walking in the rain, 'cause then no-one knows I'm crying.

posted at: 15:57 | path: /me | permalink | Tags: ,

Tue, 13 Jun 2006:

As authors both George Orwell and Aldous Huxley were masters at the task. But as visionaries (yes, for the last time 1984 is a warning, not a guidebook), they differed in a very fundamental way. Orwell has always rooted for an external oppressor who shall conquer us and rule our thoughts, lives and the world in total - the infamous Big Brother. On the other hand, Huxley had portrayed an even more outlandish concept, where the people accept and in fact, love the thing that incapacitates them from rational thought. In the Brave New World, there is no necessity for an oppressor to deprive us of our autonomy, individuality or maturity, we would gladly give that up for the security and convenience the oppression offers.

Orwell dreamed of a future where information would be denied, kept hidden from the masses and handed out in small enough parcels. A totalitarian regime where information is the currency and control was achieved by denying it. Huxley feared the opposite, where the important information is drowned in a mass of irrelevance. Where nobody picks up a book because there are far more convenient distractions to choose from.

1984 controlled people by pain, hurt when you try to enter the forbidden corridors of knowledge, while the Brave New World enslaved you with pleasure. Given you so much that you have no desire for anything more, perfectly content to watch the feelies and drink soma. Reduced to passivity and egotism, ever fearful of any disruption which would destroy the comforts that were traded in for free speech and thought, yet oblivious to their own slavery.

Yet, when the year 1984 came, there were those who rejoiced that the world hadn't fallen to a Big Brother. But the Brave New World couldn't be denied, the quest for a happy living strays too close to the ultimate paradise of ignorant bliss. As you watch an everyman sit in front a TV, sipping whatever gets him high and wondering about what exactly is happening with Paris Hilton's latest boyfriend, you do have to wonder is this a Brave New World ? The critique on the free press and its role in oppression is contrary to common belief, but some corner of my mind it is happening today (ok, pickup a Times Of India).

Huxley's message is chilling in its content and cynical in its perception. The fact that people will sacrifice essential liberties of free speech and thought to enjoy a comfortable life sits in opposite to the loss of paradise that the Adam & Eve suffered. In our deepest psyche this is a holy grail we yearn for, even at the cost of our individuality, history or autonomy.

1984 ends the same way the Brave New World began, love instead of hate.

He gazed up at the enormous face. Forty years it had taken him to learn what kind of smile was
hidden beneath the dark moustache. O cruel, needless misunderstanding! O stubborn, self-willed 
exile from the loving breast! Two gin-scented tears trickled down the sides of his nose. But 
it was all right, everything was all right, the struggle was finished. He had won the victory 
over himself. 

He loved Big Brother.

The golden cage that we built for ourselves, the only way these two differ is how we got into the cage. So who is right ? I'd say Huxley, but 1984 is yet to pass.

If society fits you comfortably enough, you call it freedom.
               -- Robert Frost

posted at: 18:12 | path: /philosophy | permalink | Tags: , ,

Been pissed all day due to the stupid advertisements on Yahoo groups emails. Every mail I read has a very irritating sidebar which I find no use for. So I added the following 3 lines to my ~/.thunderbird/*default/chrome/userContent.css .

#ygrp-sponsor, #ygrp-ft, #ygrp-actbar, #ygrp-vitnav {
	display: none;

Much better. Ad block is one of those things where being the minority is sometimes an advantage.

If we don't watch the advertisments it's like we're stealing TV.
                 -- Homer Simpson

posted at: 15:31 | path: /hacks | permalink | Tags: , , ,

Mon, 12 Jun 2006:

Since I do not run any server side code, I'm always playing around with new client side tricks. For example, the XSL in my RSS or the sudoku solver. But recently I was playing with the HTML Canvas and was wondering whether there was some other way I could generate images client side with javascript. And it turns out that you can.

Tool of choice for moving image data from a javascript variable into a real image is a data: URI. And it also helps that windows bitmaps are so easy to generate, after all it was designed with simplicity in mind. Here's a chunk of code which should in most ways be self explanatory.

function Bitmap(width, height, background) 
  this.height = height;
  this.width = width;
  this.frame = new Array(height * width);

Bitmap.prototype.setPixel = function setPixel(x,y, c) {
  var offset = (y * this.width) + x;
  /* remember that they are integers and not bytes :) */
  this.frame[offset] = c;

Bitmap.prototype.render = function render() {
  var s = new StringStream();
  s.writeInt32(14 + 40 + (this.height * this.width * 3)); /* 24bpp */
  /* 14 bytes done, now writing the 40 byte BITMAPINFOHEADER */
  s.writeInt32(40); /* biSize == sizeof(BITMAPINFOHEADER) */
  s.writeUInt16(1); /* biPlanes */
  s.writeUInt16(24); /* bitcount 24 bpp RGB */
  s.writeInt32(0); /* biCompression */
  s.writeInt32(this.width * this.height * 3); /* size */
  s.writeInt32(3780); /* biXPelsPerMeter */
  s.writeInt32(3780); /* biYPelsPerMeter */
  s.writeInt32(0); /* biClrUsed */
  s.writeInt32(0); /* biClrImportant */
  /* 54 bytes done and we can start writing the data */
  for(var y = this.height - 1; y >=0 ; y--)
    for(var x = 0; x < this.width; x++)
      var offset = (y * this.width) + x;
      s.writePixel(this.frame[offset] ? this.frame[offset] : 0);
  return s;

Well, that was easy. Now all you have to do is generate a base64 stream from the string and put in a data: URL. All in all it took a few hours of coding to get Javascript to churn out proper Endian binary data for int32 and uint16s. And then it takes a huge chunk of memory while running because I concatenate a large number of strings. Ideally StringStream should have just kept an array of strings and finally concatenated them all into one string to avoid the few hundred allocs the code currently does. But why optimize something when you could sleep instead.

Anyway, if you want a closer look at the complete code, here's a pretty decent demo.

Curiousity is pointless.

posted at: 14:44 | path: /hacks | permalink | Tags: , ,

Fri, 09 Jun 2006:

After much trials and tribulations, Wikipedia is finally using APC. They've been playing around with Turk MMcache and other accelerators for a while. But currently APC is the only one with the ball as far as caching is concerned. Recently, somebody did a benchmark on the common accelerators used in php land - read it here. But at that point APC just wins hands down, though my commit last night probably must've pushed APC below eAccelerator, it is required to run properly on a multi-CPU apache on high loads.

hw.php deserialize.php include-pma.php
eAccelerator 1093 160 86
apc 1100 163 83
PHP alone 886 157 28

Now, the next heavy user of PHP around is sourceforge.net who is apparently still using eAccelerator. Apc still has a few chinks in its armour, but it is still *my* work. And much more importantly it, for once, doesn't appear doomed :)

Real programs don't eat cache.

posted at: 22:14 | path: /php | permalink | Tags: , ,

Thu, 08 Jun 2006:

I recently discovered an easy way to inspect php files. So xdebug has a cvs module in their cvs called vle. This prints out the bytecode generated for php code. This lets me actually look at the bytecode generated for a particular php data or control structure. This extension is shining a bright light into an otherwise dark world of the ZendEngine2.

Let me pick on my favourite example of mis-optimisation that people use in php land - the HereDoc. People use heredoc very heavily and over the more mundane ways of putting strings in a file, like the double quoted world of the common man. Some php programmers even take special pride in the fact that they use heredocs rather than use quoted strings. Most C programmers use it to represent multi-line strings, not realizing php quoted strings can span lines.


echo <<<EOF
	Hello World


Generates some real ugly, underoptimised and really bad bytecode. Don't believe me, just look at what the vle dump looks like.

line     #  op            ext  operands
   3     0  INIT_STRING        ~0
         1  ADD_STRING         ~0, ~0, '%09'
         2  ADD_STRING         ~0, ~0, 'Hello'
         3  ADD_STRING         ~0, ~0, '+'
         4  ADD_STRING         ~0, ~0, 'World'
   4     5  ADD_STRING         ~0, ~0, ''
         6  ECHO                   ~0

That's right, every single word is seperately appended to a new string and after all the appends with their corresponding reallocs, the string is echoed and thrown away. A really wasteful operation, right ? Well, it is unless you run it through APC's peephole add_string optimizer.

Or the other misleading item in the arsenal, constant arrays. I see hundreds of people use php arrays in include files to speed up the code, which does indeed work. But a closer look at the array code shows a few chinks which can actually be fixed in APC land.


$a = array("x" => "z", 
		"a" => "b",
		"b" => "c",
		"c"	=> "d");

Generating the following highly obvious result. Though it must be said that these are hardly different from what most other VMs store in bytecode, they are limited by the fact that they have to actually write the code (minus pointers) to a file. But Zend is completely in memory and could've had a memory organization for these arrays (which would've segv'd apc months before I ran into the default array args issue).

line     #  op                      ext  operands
   2     0  INIT_ARRAY                   ~0, 'z', 'x'
   3     1  ADD_ARRAY_ELEMENT            ~0, 'b', 'a'
   4     2  ADD_ARRAY_ELEMENT            ~0, 'c', 'b'
   5     3  ADD_ARRAY_ELEMENT            ~0, 'd', 'c'
         4  ASSIGN                           !0, ~0
   7     5  RETURN                           1
         6  ZEND_HANDLE_EXCEPTION            

This still isn't optimized by APC and I think I'll do it sometime soon. After all, I just need to virtually execute the array additions and cache the resulting hash as the operand of the assign instead of going through this stupidity everytime it is executed.

Like rhysw said, "Make it work, then make it work better".

Organizations can grow faster than their brains can manage them.
                    -- The Brontosaurus Principle

posted at: 16:22 | path: /php | permalink | Tags: , ,

Wed, 07 Jun 2006:

The question of which came first, the chicken or the egg, exists because an egg is clearly not a chicken.

Simple, indubitable and unequivocal.

A chicken is an egg's way of producing more eggs.

posted at: 18:37 | path: /philosophy | permalink | Tags: , ,

Mon, 05 Jun 2006:

Most of the last weekend, I spent reading books in my bed. My latest acquisition is a hardbound by Jasper Fforde. Now, I first encountered this author in Rhys's posession, in a book called the The Eyre Affair and was basically recommended with 'It's absolutely crazy'. And it turned out to be exactly that, surreal to the extreme though you are left with a lingering doubt whether all this could really happen. There is just enough reality mixed in to make you wonder, just like the time you saw The Truman Show. The current tome under inspection is titled The Big Over Easy which does nothing to diminish my opinion.

The book tries to transcend reality by introducing nursery rhyme characters including the anthropomorphic animals and the usual fare of animated pastries, girls with 28 feet of hair and Solomon Grundy as an old man. But the beginning of the book starts with a trial of Three Little Pigs for pre-meditated murder of one Mr Wolff. The case hinged around the fact that the pot of water into which the 'Big Bad Wolf' fell in would have taken six hours to reach boiling which indicated premeditation. Since the pigs were tried by a jury of peers, that is a baker's dozen of pigs, they walked scot free. Then there are the detectives whose guild treats public approval as its currency and traces its history back to Sherlock Holmes (as if he's real). Each guild detective has an assigned Official Sidekick whose duties include writing out a passable entry for the mystery hungry magazines. So there's the grungy and bitter man stuck in a rut - Jack Spratt and the ambitious career detective Friedland Chymes. Both of whom started from the same humble beginning of the Nursery Crimes Division, but while Friedland was ranked #2 in popularity, Spratt wasn't even on the list.

And then the case comes up even more nuttier than a christmas fruit cake. Humpty Dumpty had a great fall, but was he pushed ? Well, as it turns out his ex-wife shot someone else thinking it was Humpty, his current lover put poison in his coffee, his previous flame's (who happened to be Rapunzel) husband ordered him killed and then there was the part about him hatching. Anyway, we ended up with a conniving chiropodist, a golden goose, a geek who's obsessed with spelling (re: Unsfzpxkable) , an alien who loves filing and then there's the Jellyman. Not to mention that Jack has killed 4 giants before and cuts down a beanstalk to kill the monster.

The first thought to enter my head at the end of the book was What was this guy smoking ?.

Humpty Dumpty didn't fall, he was pushed.

posted at: 15:17 | path: /books | permalink | Tags: , ,

You can't keep a rain-forest in your backyard, but sometimes a tape of David Attenborough's productions will serve as a sufficient substitute. Not only did he produce groundbreaking nature documentaries, but he assembled an amazing team who were equally passionate about the unrivaled beauty of nature. The Bristol division of BBC turned out a series of expensive nature documentaries which were setting a standard in the field.

My attraction to his particular brand of nature documentaries come from the frequent change of scenery. Where most of the "modern" documentaries reserve an entire half-hour to a particular location or activity, Attenborough documentaries generally travel across the world in the hour. Also the documentaries hardly have any people and does not track people as characters in the story being played out. I am pretty sure they could've made a few more hours of documentaries with the reels they cut out per hour.

Private Life of Plants is my favorite series by David Attenborough. Now, Blue Planet and other works have taken me far, wide and deep throughout the biosphere. But those leave me with a distant yearning to see for myself these wonders of the world - Christmas Island while the crabs migrate, Palau to see the jellyfish swarms, Fjords of Norway where the whales sound, Great Barrier Reef when the corals are spawning. Deep in my heart, I know that I'll probably die before I see these desires fulfilled. Private Life of Plants, on the other hand takes me somewhere which doesn't exist - the world of plants where months seem to pass in minutes and in 30 seconds we've moved from a frigid winter to a warm spring, through the thaws and amidst the flowers that declare the arrival of spring.

That by itself may seem like a magical trip across time. But there's magic even in the simple things. The background music for instance is very appropriate and literally blends into the actions on screen, as if they had been scripted to the music. There are visits to the inaccessible islands in the pacific, which are preceded by the aerial views of them which add to the environmental setup for some exotic flowers. Even something as simple as a himalayan balsam seed falling into the nearby water is dramatic and accentuated by the plink of the drop, pulled into focus in slow motion.

Amidst all these scenery switches, there are no pictures of an elderly (well, in his early forties) English gentleman lugging his luggage, sweating it out in a 4x4 jeep (no Steve Ervin, he is). Even when we rarely see him, it is merely to emphasize the extremity of the environment and how exactly we (or he) just don't fit in.

But as I said earlier, the truly remarkable thing is its fast forwarded view of the plant world, documenting the fights, battles and conquests of these outwardly immobile flora. There is beauty in a flower opening, blackberries ripening and a mushroom slowly poking its head through the dead leaves. There is nothing beyond just sitting and watching wind hit a dandelion patch. Watching and wondering, you couldn't get tired of the magic on screen.

This is the ultimate documentary. *THE* Ultimate.

Art is Nature speeded up and God slowed down.
                -- Chazal

posted at: 10:57 | path: /movies | permalink | Tags: , ,

Fri, 02 Jun 2006:

Something happened tonight that made me pick up my employment contract and read it again. As I was reading the whole document, it suddenly hit me that this document specifies restrictions applicable to me, as an employee of Yahoo!, but would not automatically transfer to either the executor of my estate or anybody whom I assign full power of attorney.

So assuming that I had already assigned full power of attorney to my father (or mother) before I had signed this particular contract, would a civil violation by them automatically make me liable ?

Well, I don't know ... but I intend to find out.

I'm not a lawyer. I don't even play one on TV.
                  -- Linus Torvalds, gcc lists

posted at: 02:39 | path: /me | permalink | Tags: ,

Thu, 01 Jun 2006:

Valgrind is one of the most common tools people use to debug memory. Recently while I was debugging APC, the primary problem I have is of php Zend code writing into shared memory without acquiring the locks required. I had been debugging that with gdb for a while, but gdb is just dead slow for watching writes to 16 Mb of memory and generating backtraces.

The result of all that pain was a quick patch on valgrind 3.1.1. The patch would log all writes to a memory block with backtraces. But valgrind does not have a terminal to type into midway, unlike gdb. So the question was how to indicate a watchpoint. Valgrind magic functions were the answer. The magic functions can pass a parameter to valgrind while in execution. This is a source hack and is a hell of a lot easier to do than actually breaking in gdb and marking a breakpoint everytime you run it. So here's how the code looks like :-

#include "valgrind/memcheck.h"

int main()
	int * k = malloc(sizeof(int));
	int x = VALGRIND_SET_WATCHPOINT(k, sizeof(int));

That is marked out in the normal code with the following assembly fragment.

    movl    $1296236555, -56(%ebp)
    movl    8(%ebp), %eax
    movl    %eax, -52(%ebp)
    movl    $4, -48(%ebp)
    movl    $0, -44(%ebp)
    movl    $0, -40(%ebp)
    leal    -56(%ebp), %eax
    movl    $0, %edx
    roll $29, %eax ; roll $3, %eax
    rorl $27, %eax ; rorl $5, %eax
    roll $13, %eax ; roll $19, %eax
    movl    %edx, %eax
    movl    %eax, -12(%ebp)

This doesn't do anything at all on a normal x86 cpu but inside the valgrind executor, it is picked up and delivered to mc_handle_client_request where I handle the case and add the address and size, to the watch points list.

So whenever a helperc_STOREV* function is called, the address passed in is checked against the watchpoints list, which is stored in the corresponding primary map of access bits. All of these bright ideas were completely stolen from Richard Walsh patch for valgrind 2.x. But of course, if it weren't for the giants on whose shoulders I stand ...

bash$ valgrind a.out

==6493== Watchpoint 0 event: write
==6493==    at 0x804845E: modify (in /home/gopalv/hacks/valgrind-tests/a.out)
==6493==    by 0x80484EA: main (in /home/gopalv/hacks/valgrind-tests/a.out)
==6493== This watchpoint has been triggered 1 time
==6493== This watchpoint was set at:
==6493==    at 0x80484DB: main (in /home/gopalv/hacks/valgrind-tests/a.out)

Now, I can actually run a huge ass set of tests on php5 after marking the APC shared memory as watched and see all the writes, filter out all the APC writes and continue to copy out the other written segments into local memory for Zend's pleasure.

Writing software gives you that high of creating something out of nearly nothing. Since I am neither a poet nor a painter, there's no other easy way to get that high (unless ... *ahem*).

Mathemeticians stand on each other's shoulders while computer scientists stand on each other's toes.
                -- Richard Hamming

posted at: 23:44 | path: /hacks | permalink | Tags: , ,

Some time in late 2002, I got to see a clear picture of what interpreter optimisation is all about. While I only wrote a small paragraph of the Design of the Portable.net Interpreter, I got a good look at some of the design decisions that went into pnet. The history of the CVM engine aside, more recently I started looking into the Php5 engine core interpreter loop. And believe me, it wasn't written with raw performance in mind.

The VM doesn't go either the register VM or stack VM way, there by throwing away years of optimisations which have gone into either. The opcode parameters are passed between opcodes in the ->result entry in each opcode and which are used as the op1 or op2 of the next opcode. You can literally see the tree of operations in this data structure. As much as it is good for data clarity, it means that every time I add two numbers, I write to a memory location somewhere. For example, I cannot persist some data in a register and have it picked up by the latter opcode - which is pretty easy to do with a stack VM.

Neither did I see any concept called verifiability, which means that I cannot predict output types or make any assumptions about them either. For example, the following is code for the add operation.

    zend_op *opline = EX(opline);
    zend_free_op free_op1, free_op2;

        _get_zval_ptr_var(&opline->op1, EX(Ts), &free_op1 TSRMLS_CC),
        _get_zval_ptr_var(&opline->op2, EX(Ts), &free_op2 TSRMLS_CC) TSRMLS_CC);
    if (free_op1.var) {zval_ptr_dtor(&free_op1.var);};
    if (free_op2.var) {zval_ptr_dtor(&free_op2.var);};

Since we have no idea what type of zval is contained in the operators, the code has to do a set of conversion to number. All these operations involve basically a conditional jump somewhere (aka if) which are what we're supposed to be avoiding to speed up.

Neither could I registerify variables easily, because there was a stupid CALL based VM (which is flexible enough to do some weird hacks by replacing opcodes) which throws away all variables in every scope. That's some serious stack space churn, which I can't force anyone to re-do. At least, not yet. So inspite of having a CGOTO core, there was hardly anything I could do without breaking the CALL core codebase.

Basically, after I'd exhausted all my usual bag of tricks I looked a little closer at the assembly thrown out by the compiler. Maybe there was something that wasn't quite obvious happening in the engine.

    leal    -72(%ebp), %eax
    addl    $76, (%eax)
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    movl    %eax, -2748(%ebp)
    jmp .L1194
    leal    -72(%ebp), %eax
    addl    $76, (%eax)
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    movl    %eax, -2748(%ebp)
    jmp .L1194
    jmp *-2748(%ebp)

As you can clearly see, the jump target is yet another jump instruction. For a pipelined CPU that's really bad news, especially when the jump is so long off. So I wrote up some assembly to remove the double jump and convert into a single one.

#ifdef __i386__
#define ZEND_VM_CONTINUE() do { __asm__ __volatile__ (\
        "jmp *%0" \
        :: "r" (EX(opline)->handler) ); \
    /* just to fool the compiler */ \
    goto * ((void **)(EX(opline)->handler)); } while(0)
#define ZEND_VM_CONTINUE() goto *(void**)(EX(opline)->handler)

So in i386 land the jump is assembly code and marked volatile so that it will not be optimised or rearranged to be more "efficent".

    leal    -72(%ebp), %eax
    addl    $76, (%eax)
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    jmp *%eax
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    movl    %eax, -2748(%ebp)
    jmp .L1194
    leal    -72(%ebp), %eax
    addl    $76, (%eax)
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    jmp *%eax
    movl    -72(%ebp), %eax
    movl    (%eax), %eax
    movl    %eax, -2748(%ebp)
    jmp .L1194

The compiler requires a goto to actually realize it has to flush all the stack params inside the scope. I've learnt that fact a long time ago trying to do the same for dotgnu's amd64 unroller. Anyway, let's look at the numbers.

Before:                     After:
simple             0.579    simple             0.482
simplecall         0.759    simplecall         0.692
simpleucall        1.193    simpleucall        1.111
simpleudcall       1.409    simpleudcall       1.320
mandel             2.034    mandel             1.830
mandel2            2.551    mandel2            2.227
ackermann(7)       1.438    ackermann(7)       1.638
ary(50000)         0.100    ary(50000)         0.097
ary2(50000)        0.080    ary2(50000)        0.080
ary3(2000)         1.051    ary3(2000)         1.024
fibo(30)           3.914    fibo(30)           3.383
hash1(50000)       0.185    hash1(50000)       0.182
hash2(500)         0.209    hash2(500)         0.198
heapsort(20000)    0.616    heapsort(20000)    0.580
matrix(20)         0.500    matrix(20)         0.481
nestedloop(12)     0.953    nestedloop(12)     0.855
sieve(30)          0.499    sieve(30)          0.494
strcat(200000)     0.079    strcat(200000)     0.074
------------------------    ------------------------
Total             18.149    Total             16.750

This is in comparison to the default php5 core which takes a pathetic 23.583 to complete the tests. But there's more to the story. If you look carefully, you'll notice that there's a register indirection just before the move. But x86 does support an indirect indexed jump with a zero index.

   __asm__ __volatile__ ("jmp *(%0)",:: "r" (&(EX(opline)->handler))); 

That generates a nice jmp *(%eax); which is perfect enough for my purpose. Except for the fact that I can see in the assembly, the above fix didn't really do much for performance. For example, look at the following code :-

    leal    -72(%ebp), %eax
    addl    $76, (%eax)
    movl    -72(%ebp), %eax
    jmp *(%eax)

The EAX loader between the two custom asm statements is what I was trying to avoid. But the variable is re-loaded again from stack because there is no register variable cache for the handler pointer. One way around that is to do what pnet did, keep your PC (eqiv of handler var) in a register, preferably EBX and use it directly. The seperation between operands (stack) and operator (handler) makes it hard to optimize both in one go. The opline contains both together making it really really hard to properly speed up.

But there's this thing about me - I'm lazy.

Captain, we have lost entire left hamster section.
Now, pedal faster.

posted at: 17:55 | path: /php | permalink | Tags: , ,

Mon, 29 May 2006:

People are sheep. They move in herds, believe in numbers (look at democracy) and avoid solving anything in person. This is, contrary to common sense, perfectly normal and how the world should be - read it through and then let me know, if you think otherwise. Before I can explain to you exactly why people are so timid, I'd have to explain why the Original Hero in our particular story wasn't the Go-Getter Hollywood crafts its dreams around. And it all starts out pretty normally [1].

So, I'm sitting around at my parents', munching on some easily munchables in front of the idiot box. Tuned in at the moment is one of the new cartoon channels which is reinventing Tarzan for the youngster of today. So there you see Tarzan telling his ape friends that the new evil leader (complete with silver back, huge fangs and red eyes) is weaker than the entire family combined. After hearing that motivational speech from somebody hanging upside down from the arms of the giant ape, the entire family drives the bullying big male out. And they all lived happily ever after, at least until the next episode.

Now, to pick out where this particular story diverges from stated 'real' world into a more fictional human world. But first, I want you to marvel at the sophistication of the tool at my disposal now - language. Sure, a cat call in the night or a howl in the distance is communication too. But our chatter possesses something unique in itself. We talk about things that could be - we talk about the future as if it were real. The first time your mother told you don't do that, or else $bad_thing_could_happen.... was when you were introduced to the very possibility (that word itself speaks volumes) of things that could be. In other words, the animals could understand what could be done if they joined forces.

Now, human society is very peaceful compared a lot of other social mammals. In almost every other primate society there is a lot of bullying and fighting at lower levels than the alpha males. Even in a stable society there are always some murmurs of disapproval and fights happen in the background. So what's different in our world that stops this from happening ? While I ponder about such important questions an advertisement for clean teeth (uhmm... I mean toothpaste) pops up.

*CLICK*. Yet another soap on some other channel. *CLICK*. Same story of a family broken by something. *CLICK*. Ah, it's a veritable cat fight between two women with bindis large enough to cover Switzerland (you *know* that I stole that analogy). Oh wait, they're just stopping at name calling. I guess language comes up as a winner yet again in this story - so where are the sheep people that I started talking about ?

Language is merely the tool. You might think the real clincher in the deal is how language helped organize things (like the Tower of Babel for instance). Well, before mankind was big enough to start dividing ourselves over language, religion, caste and creed, we lived in tribes. The organization there needed to kick some ass more than the mot juste and language was hardly (yet) the way to get your average cave-woman interested.

Even with philosophy running in parallel, daytime soaps are boring. *CLICK*. Oh, its some mafia gangster flick and some guy's getting shot because he ratted out some 'brother' to de cops. And he did that because the other guy made eyes at his sister. Seems fair, I think. If I were in the same posish, I'd be wiping the blood off my knuckles too. Wait, he didn't stand up and fight, he merely went to the Big Brother and squealed like the family pig (George Orwell is a genius). Language has its advantages for the weak and oppressed.

But didn't evolution stick up for the 'Fittest' or something ? So, if you are weak and oppressed, you'd do good to the species to stay out of the gene pool. Then why does human society discourage bullying and stick up for a very unnatural concept called fair play. Because long back during the dark ages of human evolution, language helped the weak to team up and beat up the bullies. That's just a prediction and this movie's getting way too bloody to have a happy ending.

*CLICK*. It's one of the 24x7 news channels and it's showing a strike in some factory plant somewhere. The union is demanding special protective masks and compensations to the families of its employees who have succumbed to halitosis. The basic co-operative skills mankind developed in the distant past for hunting large animals of the last ice age have been subverted to bring a factory to a grinding halt. On the other hand, the weak worker class have no way to fight the system other than uniting. And it looks like they're coming out winners.

The critical combination of language and co-operation form a very sharp tool in the arsenal of the weak and the timid. The call goes out - All for one and one for all. Such coalitions and brotherhoods must have picked off every one of the stereotypical aggressive alpha males when the species was passed through an evolutionary pressure like a famine or disease. Of course, you can't blame the mob - dying out was hardly a worthy choice. The badass alpha male just didn't Fit in with the times of crisis.

Even today, our society runs on the basis that its combined might is significantly bigger than is in possession of any one individual. This is why democracy is so popular, because aggressive individuals do not survive in a majority of their own - this town isn't big enough for the both of us. But we still need the risk taker, adventurer and explorer - they deny society and are labeled mavericks, become recluses in old age. You know the examples. We're not like them, as much as we admire them.

Still, somewhere in our primitive brains, we crave for an absolute leader to settle our disputes, punish the wicked and reward the good deeds. The all powerful, ape lord of our own tribe, that we used to see and obey everyday in those dark and distant days when we had hardly climbed down from the trees. We worship him, live our days in awe of his awesome power, sleep nights under his protection and follow him across hades if necessary. Oh, my God, what am I talking about ? (*heh).

Us humans, we are such a coalition of the timid, where decisions are by consensus and where politeness overrides correctness. That's the way it is and short of mass genocide there's nothing you can do to fix it. So quit cribbing.

  [1] - Nothing except loud music or talking women seem to snap my brain out of overdrive.

Let the meek inherit the earth -- they had it coming.

posted at: 12:09 | path: /philosophy | permalink | Tags: , ,

Last night, I was attempting to build Flubox-0.1.14 on my home amd64 box. After struggling with a couple of errors in src/Resource.hh I managed to get it built with gcc-3.4.2. Now the real bug hunting started in earnest. So consider the following code.

void va_arg_test(const char * pattern, ...) 
    va_list va;
    int i = 0;
    const char *p;

    while((p = va_arg(va, char *)) != 0) {
        printf("%02d) '%s'\n", ++i, p);

int main(int argc, char * argv[])
    va_arg_test("wvZX", "hello", "world", 0);

Now, to the inexperienced eye it might look like perfect working code. But the code above is buggy. If I wanted to really fix it, I'd have to rewrite it so.

    va_arg_test("wvZX", "hello", "world", NULL);

The old code pushes 2 pointers and an integer into the stack as variable args, while it reads out 3 pointers inside the loop. In normal x86 land that's not a big deal because pointers are integers and vice versa. But my box is an amd64, there LLP64 holds sway. A pointer is a long long, not just a long. So it reads 32 bits of junk along with the last 0 and goes on reading junk off the stack.

If you'd run my so called buggy code on an amd64, you'd have found that it doesn't actually crash at all. That's where the plot thickens. To understand why it doesn't crash, you have to peer deep into the AMD64 ABI for function calls. As far as I remember, the ABI says that the first 6 arguments can be passed to a function using registers. So the current assembly listing for my code shows up as

    movl    $0, %ecx
    movl    $.LC1, %edx
    movl    $.LC2, %esi
    movl    $.LC3, %edi
    movl    $0, %eax
    call    va_arg_test

But if I increase the arguments to 8 parameters, then the data has to be pushed into the stack to passed around and then you'll note the critical difference in the opcodes between a pointer and integer handling.

    movl    $0, 16(%rsp)
    movq    $.LC7, 8(%rsp)
    movq    $.LC8, (%rsp)
    movl    $.LC4, %r9d
    movl    $0, %eax
    call    va_arg_test

As you can see the integer 0 is moved into the stack using the movl while the pointers were moved in using the movq viz long word and quad word. Doing this for varargs on amd64 leaves the rest of the quad word in that stack slot unitialized. Therefore you are not guarunteed a NULL pointer if you read that data out as a char *.

After that was fixed in XmbFontImp.cc, fluxbox started working. God knows how many other places has similar code that will break similarly.

At this rate you'd be dead and buried before that counter rolls over back to zero.
Better get some exercise if you want to fix it when it happens.

posted at: 10:11 | path: /hacks | permalink | Tags: , ,

Fri, 26 May 2006:

I spent half of today hunting around for a 1 BHK house in Koramangala. The results were disappointing. I couldn't find a house that wasn't a bachelor's dive, which in a normal situation is nice, but not when you are expecting your parents to make long visits. So if you know of a good house somewhere in the vicinity of Forum, please email me at the address below.


And for the curious trying to copy paste that, take a look at cssplay and my pixeltext script.

Earth is like a big big house in some ways.
Nobody wants to be around when it's time to pay the rent.

posted at: 20:11 | path: /rants | permalink | Tags: ,

Thu, 25 May 2006:

Just got pissed with man pages. The idiotic tool has been pissing me off with a complete inability to scroll to the top or wrap around searches to the top. So I did some digging on how man actually displays the data. Here's a inside view on how the gzipped man pages are displayed on screen and it's an excellent example of the unix way of doing things.

bash$ ps x | grep "/usr/share/man"
....    sh -c (cd /usr/share/man && 
		(echo ".pl 1100i"; /usr/bin/gunzip -c '/usr/share/man/man1/seq.1.gz'; 
		echo ".\\\""; echo ".pl \n(nlu+10") | /usr/bin/gtbl |
		nroff --legacy ISO-8859-1 -man -rLL=77n -rLT=77n 2>/dev/null | 
		/usr/bin/less -iRs)

The man page for man provide yet another peice of the puzzle, where exactly this particular command is pulled from. There's a file called /etc/man.config which specifies where the pager (i.e less, more) command is pulled out of. And the default value was less. Instead of replacing it for the entire system, every user can override it by setting $MANPAGER in their environment. And so I did.

export MANPAGER="col -b | vim -R -c 'set ft=man nomod nolist' -"

Now, everytime I type "man something" I can read the manpage in vim.

blithwapping (v): Using anything but a hammer to drive a nail into a wall

posted at: 20:44 | path: /hacks | permalink | Tags: , ,

I don't know if you are even aware of Chris Shiflett's latest injection bug. It was reported in Mysql 5.0 and suddenly every other DB engine writer realized that it was present in almost every one of those. Postgres even churned out an immediate release to fix this particular issue. The blind spot of the whole fiasco has been multibyte encodings whereas the add_slashes is not binary safe. I was lurking in one of the php developer channels listening to exactly how this could've been exploited and it sounded really serious.

So 0x955c is a single SJIS character, when locale is taken into account, it's just a single character. But when you ignore locale and treat it as latin-1 it looks like 0x95 followed by a backslash. So in the php land, instead of treating it as a single character, escapes the backslash giving you a two character string that looks like 0x955c \ . Now you've got a stray slash which can be used as part of a user supplied escape sequence to inject whatever you require into the query data.

After Sara had explained all that, we turned to the quick fixes. Now, let me introduce UTF-8. A closer inspection of the UTF-8 code table and rationale behind contains the following pearl of wisdom - "The remaining bytes in a multi-byte sequence have 10 as their two most significant bits". So if you were using UTF-8 everywhere to handle unicode, it is not possible for a multi-byte character to end with the ordinal value of \ (0x5c). So convert a string to UTF-8 before escaping the backslashes and you're safe from this bug. So practise safe hex and always use UTF-8.

But for me the whole bug had a more hilarious side to it. Yesterday, I got two php server admins to take down talks.php.net (cached). And the reason it was taken down was due to a set of security vulnerabilities in a set of examples attached to a presentation. Authored by, you guessed it, Chris Shiflett.

<g0pz>    Derick: but I want to know what exactly is bugging the system
<Derick>  the apache process don't want to die either
<Derick>  all are dead now
<Derick>  let's start again
<g0pz>    it was probably the one I had attached with gdb
<Derick>  atleast the site works again
<g0pz>    Derick: shall I kill it again ? :)
<Derick>  g0pz: you know how to?

<g0pz>    edink: http://talks.php.net/presentations/slides/shiflett/
<>        oscon2004/php-security/code/shared-filesystem.php?file=/etc/passwd
<g0pz>    I found this in the access_log !!!
<g0pz>    Derick: that script is very very scary
<g0pz>    *please* *please* take it offline ?

<edink>   bloody hell
<Derick>  i now wiped all *.php files in shiff's dir
<edink>   and he now works for omniti?
<Derick>  yeah :)
<g0pz>    well, I'd have just put a .htaccess Deny all there
<edink>   g0pz: rm -rf is more effenctive :)
<g0pz>    more permanent ? :)
<johann__>slides/acc_php/tmp_table.php - sql (union) injection
<edink>   g0pz: yeah, security risks like that need to be dealt with permanently
<g0pz>    edink: IMHO, he probably wrote up that demo during the lunch break 
<>        before his presentation
<edink>   but how did that bring the site down?
<Derick>  some loop i think
<Derick>  .htaccess is disabled
*   Sebastian .oO( And he is an security expert, right? )
<Derick>  i don't have the time to deal with this either
<edink>   yeah ;)
<Derick>  i'll just turn off talks for now so that somebody can fix it

Somehow it stuck me as ironic that a security expert's own code should help someone read out the /etc/passwd from a publically visible, high traffic server. As Bart Simpson put it - The ironing is delicious.

The function of the expert is not to be more right than other people,
but to be wrong for more sophisticated reasons.
                      --  Dr. David Butler

posted at: 03:22 | path: /php | permalink | Tags: , ,

Wed, 24 May 2006:

I hate being the bad cop. For all that cliched formula attached to this particular idea of Good Cop, Bad Cop the sad fact is that it really works. The very idea that a person would often agree to a moderate in the presence of an extremist picking the seemingly lesser evil.

Sometimes to do what's good, you have to make others hate you.

God instructs the heart, not by ideas, but by pains and contradictions.
               -- De Caussade

posted at: 20:46 | path: /observations | permalink | Tags: , ,

Thu, 18 May 2006:

In php5 static variables in functions behave a little differently when the functions are member functions of a class. The problems start when the inheritance starts copying out functions into the child function table hashes. For instance, consider the following bit of code :

    class A {
        function f()
            static $a = 10;
            echo __CLASS__.": $a\n";
    class B extends A { }
    $a = new A();
    $b = new B();

Now, I'd assumed that it would obviously produce 11 12 as the output. Sort of ran into this while messing around with the zend_op_array reference counting code. The static members are killed off before the reference is checked.

gopal@knockturn:/tmp$ /opt/php5/bin/php -q test.php 
A: 11
A: 11

I definitely was slightly freaked and wrote up an almost identical bit of C++ code to just test out my preconceptions about static variables.

#include <stdio.h>

class A 
    void f() {
        static int a = 10;
        printf("A: %d\n", a);

class B : public A {};

int main()
    A a;
    B b;

But static variables inside a function in C++ behave identical to whether it was declared inside a class or outside. There seems to be no special handling for member functions unlike what php shows.

gopal@knockturn:/tmp$ ./static-test 
A: 11
A: 12

I am not enough of a dynamic language geek to decide which one's the right way to do it. Actually if I really had my way, there wouldn't be any static variables in functions at all. They're actually too much like global variables in terms of life-time.

Anyway, using a class static should make it behave like C++.

In theory, there is no difference between theory and practice.
In practice, there is.

posted at: 02:01 | path: /php | permalink | Tags: , ,

Tue, 16 May 2006:

Every year, I dread this day. Usually it is a day to take stock of my life and in general wonder whether living one day at a time is working out or not. But this year, it's different.

I did a lot of things I wanted to. I made that pilgrimage to New Zealand to meet Rhys, I grew my hair long and went to a lot of colleges to talk. I met a lot of people and made a lot friends. I listened to a lot of music, watched a lot of serials and read a shoveful of books. I went home every month, I even made a point to see everybody everytime I go. I did what I wanted.

As bright as that sounded, I did have my moments of darkness. The days when you feel like you've been cheated, used and abused. Days when it hardly paid to get out of bed, bouncing from one problem to another. 'T was but a small price to pay (in hindsight).

Now, I'm 24. Somewhere inside it is still 13 but *that* kid won't come out and play till he's done his homework.

If I learn from my mistakes, pretty soon I'll know everything.

posted at: 05:46 | path: /me | permalink | Tags: ,

Sun, 14 May 2006:

Being the bookworm, I couldn't resist this particular meme. The last one I'd indulged was the superhero one. This one on the other hand, makes a lot more sense.

You're Alice's Adventures in Wonderland!
by Lewis Carroll
After stumbling down the wrong turn in life, you've had your mind opened to a number of strange and curious things. As life grows curiouser and curiouser, you have to ask yourself what's real and what's the picture of illusion. Little is coming to your aid in discerning fantasy from fact, but the line between them is so blurry that it's starting not to matter. Be careful around rabbit holes and those who smile to much, and just avoid hat shops altogether.
Take the Book Quiz at the Blue Pyramid.

I'd hoped for Through the Looking glass or at least H2G2. Anyway, there it is though, full of smiling cats and queens playing crocquet with flamingos.


The real purpose of books is to trap the mind into doing its own thinking.
                -- Christopher Morley

posted at: 03:46 | path: /fun | permalink | Tags: , ,

A long long time ago, one of my classmates happened to read one of my blog entries about taxation. He commented that my English has come a long way but I should under no circumstances write about economics or money. But recently I've started to watch the US stock market, more out of curiosity than actual expectations of profit.

Since I'm neither an investor nor a trader there, unlike most ticker watchers, I was interested in watching the ebb and flow of the stock prices rather than scramble to buy or sell. Considering the (rise and) fall of the petro dollars ever since the new millennium dawned, the dollar value is merely a transactional discrepancy for an outsider. Gold is rather more stable to compare against, especially considering the unique situation India is in terms of gold demand and consumption.

The frontrunner of the Web 2.0 stocks has been Google's IPO. The stock which was priced (possibly overvalued) at 300+ USD was and still is riding on the investor confidence thanks to the recent products (albeit Beta) which have come out of the Google stable. Then, suddenly Google insiders decide to get a little cash out of the market and into their pockets. The sell was more massive than all the other 199 companies put together and basically was a shot in the arm for California's treasury in tax dollars. It could be said that the sales wouldn't affect the economy as the money never really left the market but merely got converted into scrips. But the more important question to ask is whether this is a precursor to something more dramatic (always a good idea to watch CFOs selling stocks).

On the other hand, we see another trend marked up against Gold versus the stock market. Technically speaking, the stock market should be ahead of the curve on Gold in the growth sector mainly because of the risk involved (which should be compensated by increased returns and vice versa). But very recently, that has stopped being true for GOOG stock. This is probably more a reflection of the falling value of the dollar rather than the intrinsic value of the stock in particular but the trend in itself is disturbing.

I' m not the smartest dude in the world, but even I would be hedging my bets in tech stocks if I held a few million of these. So it is probable that these two events are related in some way, but to speculate whether one event was in anticipation of another would be completely irresponsible. In fact, if I were a Wall Street mogul I'd be convincing you to buy while I sold off all of mine slowly (just like the old times in '99).

In conclusion, something's cooking.

** disclaimer ** : these are very much my own opinions and I am a code monkey @ Yahoo!, so if you do something stupid based on these, Y'all can go to this excellent village ["Yee Haw !"] that's missing its idiot.

The only thing necessary for the triumph of evil is for good men to do nothing.
        -- Edmund Burke

posted at: 00:31 | path: /observations | permalink | Tags: ,

Fri, 12 May 2006:

Hospitals are such boring places to sit around. Sitting around in the lobby watching a bald seven year old tug on her father's hand is just too painful. Except for a few minutes sometime around 11 when all the cured pass out through the doors, the atmosphere is sterile and sounds like what a library should be. So I finally gave in and started staring at the TV showing the current events in Kerala.

Yesterday was the election results day. It is hardly a week after the elections, but thanks to all the voting machines the counting is quick and painless. You'd have hardly needed an election to predict the results. LDF (Left Democratic Front) won out 98 seats out of the 140 in offer. Basically, CPI (M) won out an entire majority this time around.

Communism, just like economics, has an entirely new model in Kerala altogether. If Kerala is a nursery for educated labour, it is in no small way indebted to the reforms by the communist regimes. Quite ironic that the Kerala's booming tourist industry and its main export product, viz people, has been due to communist non-developmental strategies. Kerala under the left has always invested in the people and seen its best being drained out into the Gulf, Bangalore and the U.S. Incidentally, that wasn't such a bad thing considering the prevalence of the joint family system.

All in all, the incumbents lost. Or more accurately, the incumbents have never won in Kerala. Now, we'll be introduced to the same old brand of Left corruption - party donations which trickle down instead of the Right's upwards kickbacks.

I've said this before and I'll say this again - democracy dilutes the power of the individual and rationalizes the actions of the mob rulers.

Hospitals are depressing. Facts for the day, there.

Health nuts are going to feel stupid someday, lying in hospitals dying of nothing.

posted at: 02:30 | path: /misc | permalink | Tags: , ,

Tue, 09 May 2006:

Last october, Radek Polak suddenly announced that now dotgnu has a debugger. As much as that was good news, it still didn't fit in easily with the current design where the debugger works over the wire. On wire debug protocols are all the rage these days and for good reason, except that a few too many exist in some places. The new debugger backend is checked into Portable Studio SVN for those interested in the nitty gritty details. Here's a sample debug session.

The server is running at port 4571
Waiting for a connection...
Connection accepted
Shortcuts for recently used commands
0. watch_method PrintFn
1. unwatch_method PrintFn
2. list_threads
3. unwatch_all
4. stack_trace
5. print_locals
6. dasm
7. watch_location simple.cs 2
8. watch_all
9. watch_method Main
> l
 <Breakpoint Offset="-1">
   <Location Linenum="0" Col="0"><SourceFile Filename="" /></Location>
   <Member MemberName=".ctor" Owner="$Synthetic.$164">
     <MemberSignature Language="ILASM" 
        Value="public hidebysig specialname rtspecialname
         instance void .ctor(int32) runtime managed "/>

> print_locals
   <LocalVariable Value="10" />
   <LocalVariable Value="0" />
   <LocalVariable Value="0" />

That is the raw debugger protocol dump, now I'll just wait for Radek (or someone else) to slap a pretty UI on top of that. In all probabilty, Portable Studio will let me debug dotgnu code without all the hassles we've been going through for the past few years.

Developer tools are the crack cocaine of the software world. One try and you're hooked.

Debugging is over when people get tired of doing it.

posted at: 02:14 | path: /dotgnu | permalink | Tags: , ,

Mon, 08 May 2006:

This one's for all the times I saw an eskimo on a sled with the banner North Pole or Bust in Miami. Actually that was just a cartoon - but that's the point. Ever wonder why it's funny ? Actually I have a pageful below that will do nothing explain it. But I think you'll understand why I hold back no smiles. This is actually a page from my life, carefully bookmarked (disclaimer: some names changed for more hilarity).

The problem with opportunities is that when they do knock, they're likely to press alarm bells rather than door bells. As articulated so clearly by Wally - there are no problems, only issues, challenges and opportunities. So, what does that have to do with my life ? Well, I was the neighborhood Wally when I used to work in Initech.

I used to show up at work a bit after nine. Yes, 9 AM and that's late by Initech standards. I'd walk upto to my desk with a cup of coffee as if I just nipped downstairs for a coffee. I never carried a bag and hardly ever an umbrella. I'd imagine somebody would be hard pressed to figure out whether I just walked in to work or whether I'm relaxing after my early morning bit of work. My job involved taking a mobile phone, flashing the latest build and literally key mash my way into the bugzilla records. I was a QE and they were wasting a really quality engineer by making him poke a few buttons. The managers knew it, but were really powerless to pull in a kid out of college into writing embedded software (*oooh*).

So the problems started in the April of 2004, when 25-odd people from the 34 member development team gave in their resignations. What I was doing at the moment was having an argument with my technical manager about my leave, without knowing about all the hush hush resignations. I basically had put in more than a month's salary into my round-trip (non-refundable) plane tickets to Trivandrum and wouldn't stand to any level of bullying about the leave which I'd applied for two months ago. So I left office to spend an enjoyable week at home.

I returned to a chaotic office. Almost every good developer in the team is on notice. At that point hardly anyone cares about the project to waste their time on it. Here's where the management decisions kick in with true force :- We need to hire more people. Since they've already sold the IP for the project, management is now being paid for time rather than code output - there's no motivation to hire excellent developers. Solution, throw the bunch of freshers who've been sitting around on to the code. The more time they take to do things, the more money the account pays. And I was one among them.

That, my friends, is how I became a developer in profession. The loss of a significant amount of technical talent in one place, means a significant opportunity to people sitting in the benches elsewhere. That might sound too simplified, but there are two unspoken assumptions. The new guys on the bench have to be really good and the management has to have enough confidence in them (or shouldn't care which way) to throw these kids at the hardcore work. On a normal day, that'd be a big risk - but during these few desperate hours, this could be the last gambit.

It might just be my big empty head, but I keep hearing echoes.

The solution of problems is the most characteristic and peculiar sort of voluntary thinking.
                -- William James

posted at: 08:46 | path: /observations | permalink | Tags: , ,

Looking at this world is like seeing a movie. The events are dramatic, the villains evil and the heroes just. But recently the script's gone awry - villains are missing. I think we need a Goliath for all the Davids this century has given birth to. So let me introduce to you the third part of the trilogy - Gulf War III. And this time it isn't personal.

Basically, the political actions of the last decade can be reduced to a single line. If you have nukes, the world's policeman would negotiate and embargo you rather walk in with their marines. But assuming you forgot to buy the old russian nukes and those are hardly the stuff that could go off without lightnink and mad scientist included, you'd get stomped over by U N inspectors and then by the afore said marines.

Now that the oil-rich lands of Iraq are ruled by Freedom, we have basically run out of targets which are non-nuclear. Maybe we could try to nail that monster behind 9/11, hunt him out of home, bust every bolt hole and shoot him like a dog ? Or maybe we could bring freedom and peace to the war ravaged lands of Africa but where's the oil, bauxite or natural gas to actually require such a peace keeping effort ?

Anyway, there's enough artillery sitting somewhere in the Persian Gulf to turn Iran into dust. And it is even parked right next to the border that it only takes a hand wave to get the motorcade moving. But whose hand would do the waving and exactly who has his (even her) hands up this particular sock puppet ?

Somebody (yes, a US president himself) said that "Speak softly and carry a big stick". He was right about not hitting anybody in particular. The world will sit idly by watching one getting cut out of the herd and killed, just like all the buffalo on the veldt. Except when you destroy rather than defeat will this world be truly afraid enough to team up and send a message back.

Twice in this world's history have nuclear weaponry been used on innocent civilians. Let this jaded generation wake up to the truth and I don't think anything would do but the glowing horror of the mushroom cloud. So, I say - go ahead and nuke Iran.

Yes, this script's missing a villain.

"I say we take off; nuke the site from orbit. It's the only way to be sure."
      - Corporal Hicks, in "Aliens"

posted at: 08:11 | path: /rants | permalink | Tags: , ,

When we last left off, we were talking about matchpot and the soon to be world championships. But what matchpot lacks in cerberal and social subtlety, Mafia brings out in potfuls. Basically the game is about killing innocent villagers, whether you are the mafia or one of the lynch mob yourself. I was introduced to this game when we were all sitting around in our hotel rooms in Thrissur. The real interesting part is not the game in itself, but how it lets (or in fact forces) you to study other people under a microscope.

After the first few games where Mafia won hands down, slowly the villagers started to pick up on the non-visual cues as well. It was quite interesting to see people trying to be overclever and bluff with poker faces. Also several interesting observations, some particularly personal, were made by a lot of people. I did get a quite inside picture of a couple of people's minds and it is terrifying what some people are actually capable of, compared to your mental estimate of their trick quotient. On top of that, it is also a measure of how successfully you can con other people into changing their opinions. On the receiving side, mafia has a way of exposing your gullibility in a painfully obvious way.

An important lesson the game has taught me is about myself. I found out to my surprise that I think a lot more clearly when I am not formulating a point to present. Being dead in the game gives you a totally different perspective which you are unlikely to get while you are talking. Sometimes just having to sit and watch the entire crowd ignore the clinical quality of the strategy is just way too frustrating. Masterpeices of strategy are completely lost to the villagers who're more concerned about staying alive to the next round rather than bringing the mafia down. Exposed are the simplified versions of our daily grind, where the evil go un-punished and good are targeted. Religions have been based on much less than fixing this (later, much later).

Nobody has seen me as the Mafia yet, but I'm better at finding things out than hiding them.

Fanatics are often blinded in their thoughts.
Leaders are often blinded in their hearts.
          -- Dune

posted at: 07:46 | path: /fun | permalink | Tags: , ,

Fri, 05 May 2006:

When we planned to help Jayadev make the plunge into married life, one of the ideas was to land up at Cochin and head off to somewhere nice for the weekend. So we headed directly for Cochin on the Island Express on friday night. After considering the fact that it's summer (so, no backwaters) and everybody was enchanted by photos of munnar - we decided to head off to the hills in Idukki and see if we can have some fun there. Since I was the only malayalee in the group, I was supposed to organize the trip. But Shivku and Sudhir, with no thanks to my incompetence, managed to book a cottage as well as get two cars to climb the hills with.

We started at 4 PM from Cochin, towards Munnar by following the NH 47 and cutting across to NH 49 before Aluva. We had hired two cars, an Ambassador and a Qualis - I was in the Qualis along with seven of the gang. We had quite a blast on the way up, even though it was slightly dark. I think the highlights of that leg of the journey was the Nadeem Saravanan songs and Shreek's insightful comment. There were minor discussions about Innovation and what happened during Hack Day and all that. I've read too much of the Art of the Start to stay silent in that discussion. Anyway, in case you forgot, here's Shreek's awesome perception.

"Dekha Hai Pehli Baar |         (I've seen for the first time)
 Saajan Ki Aankhon Mein Pyaar"  (love in my lover's eyes)

<Shreek> if she never saw it before, how did she know he 
         was the one ?

After reaching Munnar in about 5 hours, we located the DTPC office to collect the keys. The cottage we would be staying in was about 8 Kms away, so we decided to have food at a hotel in the town. After pointing out all the mistakes in the menu and making me blush (I'll not explain how) - we headed towards our cottage for the night. Also we discovered that Rahim couldn't take a photo the way we normal people take them from eye level.

The cottage was hidden away in the middle of a tea estate. We hadn't realized the exact surroundings in the night when we arrived there. It has the typical horror movie feel to it which was made fun of the moment we got there. But the cottage was very spacious and had hot water (which interestingly used an on-demand gas based heater). We started then to arrange for a campfire to sit and sing around.

The campfire was way too much fun. We sat around the fire and sang all kinds of songs. The really funny part was the A R Rahman songs with people singing the same song in different languages. Telugu, Tamil, Kannada and Hindi versions sung in sequence. Basically the anthakshari went in all directions with us just going nuts with our voices. My vocal chords have still not recovered from all that singing in the cold night. Also there was that incident of Shivku getting a mouthful of Kerosene by blindly putting the nearest bottle to his mouth.

After all that we all basically slept before we touched our pillows. I woke up the next day to a glorious morning. The sun was shining bright and the blue sky promised an excellent day of sight seeing. We checked out of the cottage sometime around 9 and basically headed for the hills.

We went around to the Mattupetty Dam and the aritificial lake behind it. Idukki district is the hydroelectric powerhouse of Kerala with a number of dams on its rocky gorges pumping out megawatts for the coastal towns. After that we crossed the border into Tamil Nadu to visit Top Station. At top station, we were hardly 60 kms away from Kodai if we go straight up the hill and go down. The top station observation point was where we had range for Airtel and a lot of us made phone calls while we were up on that particular hill.

The mist had started to become fog and a dense bank of fog was slowly rolling in towards the hills by about 4 PM. The woods acquired a magical look when the fog started blocking out visibility, though it meant hell for the driver. So we headed straight down the hill. I had a slight mishap while trying to take a picture of the cheeyapara falls and fell into the knee deep water. So I ended up taking a bath in the cold water in the waterfall. By around 10'o clock we had reached Thrissur right back on flat country.

I didn't just sit around in a hotel in Thrissur, but that's a completely different story altogether.

All trails have more uphill sections than they have downhill sections.

posted at: 12:44 | path: /travels | permalink | Tags: ,

As gleaned from the php irc channels on wednesday :-

<helly_>    need to show a table with >2k columns, it can but not show all column headers
<SaraMG>    funny
<Rasmus_>   You need Dell's new 8192" wide monitor for that table
<Davey|Job> *with* the built in HDD
<Rasmus_>   and a bicycle
<g0pz>      and it blinks "pedal faster !!"
<Derick>    8192 inch
<Derick>    wow
<Derick>    that's 208.0768 meters..
<scoates>   Derick: NVGA
<scoates>   (Neighborhood)
<scoates>   it requires 16GB of video ram, though..
<andrei_>   it's not the size that matters
<andrei_>   it's the font

The humor doesn't stop with IRC channels, though. Consider this very awesome groklaw comment, inspired by the Cheese Shop sketch. Also making the rounds is more interesting things like this firefox gurl.

Life's never boring if you watch this world carefully enough.