Thanks to all my firefox proxy.pac DNS irritations, I finally decided to ditch my ssh -D socks proxy for a tunnel into a squid. While I set up the firewall and enough protection for the proxy, I wanted to enable password protection on it. But basic authentication is not much of a protection and I didn't want to create a dummy user in the systtem to use this. Basically, I wrote my own squid authenticator - a simple enough task in hindsight.
If you inspect your default squid.conf, you'll find a line somewhat like this. This is your authenticator hook, which is a program which reads a single line in and outputs either "OK" or "ERR".
auth_param basic program <uncomment and complete this line>
Now, after I know how the authentication works, it was as easy as pi. A simple enough script in whatever language you're comfortable in will do - I prefer python over perl and this sample's in py.
#!/usr/bin/python
import os, sys,re
LINE_PAT = re.compile("([a-z_]*) (.*)\n")
u = sys.stdin.readline()
while u:
m = LINE_PAT.match(u)
if m:
(user,pw) = m.groups()
if authenticate(user,pw):
print "OK"
else:
print "ERR"
else:
print "ERR"
sys.stdout.flush()
u = sys.stdin.readline()
sys.exit(0)
Define your own version of authenticate, for example mine accepts a password that is "<fixed>.<OTP>" and the OTP is regenerated every 4 hours (not a very secure channel for transmitting that, but it works). You could probably build something similar to what RSA keycards use, which is basically the same principle.
auth_param basic program /usr/local/bin/sq_custom_auth acl password proxy_auth REQUIRED # password protected http_access allow password
Voila, you have a squid authentication that doesn't need a system account. Of course, there are more proper ways of doing this - like backing it with Mysql, LDAP or even RADIUS. But for a non-sysadmin like me, it needn't scale or be absolutely bulletproof. Probably took me much less time to do this, than write out this blog entry. But I wrote this so that sometime later, I can come back and look at this instead of remembering how to do this.
--Always think of something new; this helps you forget your last rotten idea.
-- Seth Frankel
I got irritated of getting kicked off irc around 20 times and the associated ghosting and renicking. So I sat down and wrote something that would keep me t3rmin4t0r on freenode.
import re,xchat
nick_pat = re.compile(":(?P<nick>[^!]*)!(?P<user>[^@]*)@(?P<host>.*)")
def renick(*args):
nickhost = args[0][0]
m = nick_pat.match(nickhost)
if(m.groups()[0] == "t3rmin4t0r"):
xchat.command("nick t3rmin4t0r");
xchat.hook_server('QUIT', renick)
Load that up with /py load ~/.xchat2/pylugins/re-t3.py and now I can stop worrying about being stuck as a mere gopal__ on irc.
--Who steals my purse steals trash; ’tis something, nothing;
I watch a lot of cartoons, always have, always will. It might not have improved my grasp of physics, but it's always made me laugh. But as I sat in front of the idiot box today, I didn't want to watch any toon they were showing. Ever since Pokemon pointed out the huge merchandising opportunities, the recent trend of toons are mainly intended at making kids buy useless stuff. Sure, there were G. I. Joes and Skeletor toys in the cupboards when I was a kid as well, but watching Beyblade made me sick. Where Pokemon at least redeemed itself by emphasising evolution (for the US bible thumpers), this one seems to be pure merchandising claptrap.
I mean, I'm not asking for a strong story plot here. But it should have something, at least something that stimulates your brain. Most roadrunner cartoons are stupid, but I still ROFL at the Wyle E. Coyote, Genius business cards or anything that's named Acme (with apologies to Leon Brocard). Or to take another example, Tom and Jerry - perfectly predictable, yet funny in some excellent episodes. Even Scooby Doo has its moments of mirth, especially the Let's split up cliches. None of these needed a story to make it funny, it was merely funny because they were.
But there are still some which are for kids of all ages. They go beyond mere physical humour, into word play and referential humour. The moment, Bugs Bunny said "It's baseball season" [1] or the classic What did you expect, a happy ending ? [2] were landmark events for any cartoon to follow. Even the background music was borrowed from operatic greats [3] and a Casablanca spoof with carrots. Not that Bugs and Daffy toons were lacking in the low brow humour either. Those cartoons were loaded with jokes at all levels possible in six minutes.
But that era has passed and passed on the baton to the new overlords of cartoons. I probably won't consider Simpsons or Southpark as cartoons, but as merely animated series. But there were a few glimmers in the pile of shit that got served to me in the late nineties, by Cartoon Network.
First on the list would be Dexter's Laboratory [4] - the classical mad scientist story, only the mad scientist is just 8 years old. Having a secret laboratory, while living the life of a normal kid in front of the parents makes for some moments which involve the audience in some conspiratorial laughs at the expense of Dexter. Not to mention Dee Dee's meddling of the "Ooooooo. What does this button do?" kind. Nobody with a sister can stop smiling at that. Now, you might laugh at the standard jokes it sets up, but there are a few lines from today morning's episode ( * Figure Not Included ).
Major Glory : So what have you learned today ?
Dexter : I learnt the important lesson that you cannot buy
friendship with gifts.
Major Glory : No, not that.
Dexter : Then what ?
Major Glory : You're going to learn that you can't get away with
Copyright Infringment
Dexter : Oh ?
Major Glory : Now you'll have to face someone much more
powerful than me.
Dexter : Who's that ?
Major Glory : My attorney.
Or even the referential Mock 5 where Dexter (btw, the name means "Right") races against Racer-D who is actually Dee Dee (remember Speed Racer ?). Yeah, Genddy Tartakovsky is a genius. His other works, such as Samurai Jack or Powerpuff Girls, were excellent as well. In fact, PPG was far more involved than the name would suggest, though it is a slightly acquired taste (you need to watch Mighty Morphin Power Rangers to get some of the jokes).
And then there was Johnny Bravo. For a blonde Elvis clone, who picked up enough Fonzie cliches, comedy comes easy. I for one, love the Kirk Tingblad episodes where Johnny's narcissim is brought to the forefront. Sure I know enough folks who think that it is a stupid show written for stupid people, but the flourishes are there in the details. For example, in Aunt Katie's Farm, Johnny is a pig in the sketch. And after destroying the set, Johnny rolls around in the mud and starts to yell "Four feet good! Two feet bad!". Or even the Prince and Pauper version, except in this one Mark Twain ends up thrown into prison for calling the plot an old chestnut. But my best Johnny line was from the "Panic in Jerky Town" where he comes out of the factory yelling "It's people! Jerky Jake's Beef Jerky is made of people!" [5], which goes whooshing above most viewers' heads.
What I saw on TV today didn't even come close to any of these. I have a glimpse of the future Bill Waterson saw when he said no to selling cuddly stuffed Hobbes to his fans. I'm sure enough younglings will complain that Beyblade is the coolest, but from what I see, it is all about buying tops. I used to love DBZ, but it was never about buying Dragon balls from the nearest shops. These toons seem somehow different and alien to me.
I suppose, every generation survives on nostalgia. Maybe I'm wrong -
all these kids will outgrow all the stupid toons and bitch about
the next set of twelve year olds when they're twenty four. I mean,
I'll really be scared if they don't 
Oh, and to relieve you from the suspense about the deer uncles. That's from a Dee Dee quote - "Deers don't have uncles, they have antlers". Laugh if you can ...
--This is what entertainment is all about ... Idiots, explosives and falling anvils.
-- Calvin
I've always used -fprofile-arcs to instrument and profile my code and often been quite unimpressed by the output verbosity or presentation clarity. My opnion about profilers were down in the dumps, till I ran into kcachegrind. That is really a killer application in terms of profiling code.
KCacheGrind has everything I need. It has call graphs, call counters, callee counters and even lets me view code with profiling data in place. The latter is quite a kick ass thing, really. Now, I could actually see what branch was taken what percentage of time, pre-compute branch conditions and basically understand how the code was being used.
But what are words when a picture could do more justice to the tool. The following picture is a source annotation of a bit of apc code, the numbers might scare a few, but that's what it deals with on an average web server. So this is how I start callgrind :-
bash# valgrind --tool=callgrind --dump-instr=yes --trace-jump=yes -v \
/usr/sbin/apache2 -f /opt/php5/etc/apache2/apache2.conf -X
bash# chmod 0666 callgrind.* /* so that www-data can write to it */
bash# kill -USR2 <callgrind-pid> /* to kill apache2 gracefully */
After running my code, which is actually a bit of php non-code (you know, the ruby-on-rails or J2EE flavour of non-code), generated from symfony, I get a callgrind dump, which I can proceed to analyze with KCacheGrind.
I can then see the call graph, which looks something like this.
The really cool part is when you add the source dir annotations to the corresponding binary files in the configuration. Then I can get something that actually makes sense for a programmer.
I guess the screenshots speak for themselves. Thanks to this and after moving code pre-computations in place, APC is about 3-10% faster for php 5.1.x .
--You cannot have a science without measurement.
-- R. W. Hamming
A lot of people have been complaining about APC's stability issues. In fact, they get angrier when I mention that it works for Yahoo!. During the FIFA slams on the servers, we put in a few extra things in APC to make it withstand the hammering. Now, a couple of these protections were borrowed from code that Y! already had lying around and a few more of them were BSD specific. But the short story is that I can never push those changes to the open source version. Nor can I even rewrite the same features after reading Y! © code which does the same, at least not while I'm here.
*But*, one feature that we borrowed was discussed quite a while back on the php-internals mailing list. If someone among you think that they know enough to understand what this means and implement it under the php license, maybe it might be accepted as a patch to php.
All it needs is some elbow grease and a bit of unix magic :)
--signal(i, SIG_DFL); /* crunch, crunch, crunch */
-- Larry Wall in doarg.c from the perl source code
Now, this isn't new, but it just had to be given its due respect. Sung to James Blunt's - You're Beautiful.
My cubicle, My cubicle It's one of sixty two It's my small space In a crowded place Just a six-by-six foot booth And I hate it, that's the truth Well, I give a sigh As the boss walks by No one ever talks to me Or looks me in the eye And I really should work But instead I just Sit here and surf the Internet In My cubicle, My cubicle It doesn't have a view It's my small space In a crowded place I sit in solitude
I haven't seen anything that has more truth in so few phrases. Kudos to Keith Hughes and Jym Britton for pull this one off, with style. Get the song from morningsidekick and play it loud in office.
--How can I "Think Outside the Box" when I'm in the @#$%? box all day!
APC released version 3.0.11. I've been hunting the entire codebase for memory issues and even laid to rest the bug from hell.
Now, all that remains is for all new bug reports to come in.
--Your parity check is overdrawn and you're out of cache.
The first lines of this was written around seven months ago, when I was itching to get a proper article published with the stuff I've learned over the last four years. Working on the dotgnu unroller, APC and debugging random bits of code had turned me into a gdb power user. Sure, there's lots I don't know about gdb and most of what I talk about is stuff you can read right off the man page. But, even these I had to learn the hard way and I wanted to get this out to the world in one compact vehicle. Here's the part1 or the so-called duh! component of the four-part tutorial.
Segfaults: No, this is not for developers. Any and everytime I've reported a segfault to a developer, the first thing they ask for is a backtrace. To put it simply, a backtrace is a log of all the calls leading up to the segfault and is invaluable when dealing with most segfaults, be it bad arguments, corner case inputs or plain stupidity (of the user, developer or even his counterpart on the server side). Getting a backtrace out is the step #1 for any bug report. But before that, you've got to figure out how to run the errant program inside gdb.
bash$ gdb ./sig11 (gdb) run argument1=0 argument2=1 file1 file2 ...
That starts off the program in gdb. Now, you can continue to make the program crash by doing whatever you were doing before.
Program received signal SIGSEGV, Segmentation fault. 0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6 (gdb)
To get a backtrace out of it is just the matter of a single command bt
(gdb) bt #0 0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6 #1 0x000000000040054d in foo (a=0x0) at test.c:5 #2 0x000000000040058a in main () at test.c:9
But often, distros like ubuntu and fedora, ship binaries without any debug info, to save on install space. If you are debugging something real, better look for a package that says -debuginfo. Or if you were building your own programs, compile them with the -g flag (gcc -g test.c).
There are variants of backtraces. You can provide backtrace with a count or even can ask it to be more verbose about the induvidual frames (every method call occupies a frame, unless it is a recursive tail call ... *blink* *blink*). For example, I could run bt full to print out the locals in the ancestor functions directly.
(gdb) bt full
#0 0x0000003c2fe6f200 in strlen () from /lib64/tls/libc.so.6
No symbol table info available.
#1 0x000000000040054d in foo (a=0x0) at test.c:5
No locals.
#2 0x000000000040059f in main () at test.c:11
xyz = 0x4006b3 "abc"
template = "abc"
Frames: Frames need more exploring in most cases. The recent debug symbol version, embed the entire code in the function. The key commands to play around with are frame or f for short and the list command. Both of these combined, empowers the user to see the code that went into the binary.
(gdb) f 2 #2 0x000000000040059f in main () at test.c:11 11 foo(NULL);
You can move up and down the frame using the up and down commands.
Now, just seeing the obvious culprit might not have been enough. You might want to see code around the actual call to decide the error condition. That's where list really kicks in.
(gdb) list
6 }
7 int main()
8 {
9 const char * xyz = "abc";
10 const char template[] = "abc";
11 foo(NULL);
12 }
Actually, you can list arbitrary lines of code too. This is just too good for debugging a huge file of code. For example, if I wanted to see what's at line 1 of the file.
(gdb) list test.c:1
1 #include &glt;stdio.h>
2
3 int foo(char * a)
4 {
Frame info: The frame info commands let you look at the arguments and local variables quickly. Unfortunately, if your code modifies the arguments along the way, you'll just see the new value, especially on AMD64 and other places where there are enough registers to pass around arguments.
(gdb) info locals xyz = 0x4006b3 "abc" template = "abc" (gdb) info args argc = 1 argv = (char **) 0x7fbffff858
Print: these are my favourite type of commands. Mainly because they print out information based on the inferred types and dump full structures. For example, here's a tidbit of print commands used on some ZendEngine2 (php) code.
(gdb) p h
$2 = (zend_file_handle *) 0x2a9a433540
(gdb) p *h
$1 = {type = 0 '\0', filename = 0x0, opened_path = 0x0,
handle = {fd = 2, fp = 0x2, stream = {
handle = 0x2, reader = 0x2a9a433548 <executor_globals+8>, closer = 0,
fteller = 0, interactive = 1}}, free_filename = 104 'h'}
Now, this tells me way more than the average printf does. Especially, if I don't know what I'm looking for. But sometimes, you want some "processed" data out. Maybe the hex version of something or someone's mangled the name with a NUL prepend. So there's still a printf around.
(gdb) p src $1 = (zend_function *) 0x7fbffff260 (gdb) printf "0x%02x\n", src->common.function_name[0] 0x00 (gdb) printf "%s\n", src->common.function_name+1 /opt/apc_test/var/www/html/apc.php (gdb)
These are the basic commands I use to start off inspecting a coredump or other misbehaving programs. But there are a lot more you can do if you are debugging a live program. My next tutorial is about attaching a debugger to a program and the extra stuff you can do with the process. Breaking, watching and the works.
Watch this space.
--"The debugger is akin to giving the _rabbits_ a bazooka. The poor wolf doesn't need any sharper teeth."
-- Linus Torvalds, about NT's new debugger
posted at: 18:11 | path: /tutorials | permalink |
On this August 11th, I completed the third year I've been working in the software industry. My first day shall forever be burned in my memory, for that was day my ideals died. If I seem too cynical, too pessimistic about the software industry, it all started from day one. For that was one day that caused all five of us to leave eventually and destroyed any vestiges of loyalty we might have had. I don't regret having gone through all that, because I learned a few valuable lessons, just a few hours into the job.
So, all of us were picked up from campus on the very first day of campus interviews. We were all satisified and never attended any of the off-campus interviews that our friends who had got into Infy and CTS were going to. I even lost interest in my GRE score and that pacakge from Georgia Tech is still lying on my shelf unopened. So all idealistic and eager, we arrived in their Electronic City office, around 8:15 AM on that fine monday morning.
We were part of a batch of 68, who were all joining on the same date. There were students from all over - Pune, Coimbatore, Allahabad, Mangalore and Bangalore. So our exams ended by late June and the lab results for us weren't tabulated till date. Of the group, all eight of us and three others from Pune didn't have our final year marklists. After the employee IDs for four of us were generated, some guy realized this fact and literally threw us out of the hall.
"You didn't inform us that your marklists weren't out. We will call you back when you have a degree" - those were his exact words. We were told to go home and that we would be called back along with the October batch of new joinees. We were more scared than pissed off. Fresh out of college and with no job, we were pond scum in the job market mill pond.
First thing we did was call up our parents. We even called up our principal and placement officer. The latter earned our eternal scorn by washing his hands of the affair. And to compound the issue, there was an ongoing student protest in the university college.
In the midst of the rioting students, policemen with tear gas, my father and Joe's father managed to find that our marklists were ready to be tabulated. They managed to find the clerk in charge of this, drive him to the uni, get it tabulated, got him to take it to the VC's house and get it signed & stamped. And all this, they did before 2 PM.
After making 800 rs/- worth of phone calls and such heroics from our parent's, the marklists were faxed to the HR by 3 PM. We were very reluctantly admitted and our employee IDs generated. We had gotten in and we assumed that it would be a smooth ride from then on.
So the training began. Most of us spent the days on the back-row playing virtual pool, copter and other flash games. And it really pissed off our trainers when me and Sreekrishna walked out of the half-hour C++ exam about two minutes after it began, with near perfect scores. Hardly did I know that we would pay dearly for those stunts.
We were informed about a week into september that we were going to be posted in Hyderabad for a Telecom & Internet reqs. And I mean, just the three back-bench wise-asses. And when I asked about openings in their embedded wing, I was told that "In bangalore, there are only testing openings" (for freshers).
Off to Hyderabad, I went. Only to discover that after all this drama, I had been posted to a testing job. My job involved filling in an excel sheet with PASS/FAIL depending on whether my button mashing on the mobile phone caused it to dump core. And all three of us started working on our resumes rather than our day jobs. And then came the rejections. Sometimes outright rejections when I was passed over for people from known colleges - IITs, RECs, BITS and elsewhere. Talent just didn't seem to matter and mine was very hard to measure in an interview (I am useless on paper).
After six interviews and no job, especially that Unix & C rejection, I was disillusioned about a career in software. I decided that day and then that if I manage to escape, I'll never again work in a services firm as a cheap brain for clients somewhere.
But ...
--Nothing motivates a man more than to see his boss put in an honest day's work.
I have undergone a simple and very dramatic transformation. If you saw me tomorrow somewhere, you'd probably not even recognize me. Ever since december 2005, I've been growing my hair. No haircuts at all. I've grown my hair nearly six inches long in this period, but the hair grew all curly and complicated. It has been a pain to maintain and more often than not, got in my way than becoming part of who I am.
I managed to lose all my hair in one single day and feel the soft caress of the wind on my scalp. Also I rediscovered that feeling of freedom that can only come from doing something stupid. Maybe it does make me look like a terrorist out from jail, but I don't give a damn.
It is a custom in our family to do this on the sixteenth day after the death of a close relative. But this has nothing to do with that, it has everything to do with an august five years ago.
--Hair now; Gone tomorrow
For the last few days, I've been extremely busy. I've been running around to do so many things. The unemotional machine that I have become, had to organize the memorial service, take my dad to the doctor, pick up relatives from wherever they land up in Trivandrum and essentially be there for everyone. But all that's done.
Now that I can actually sit down, the reality of the situation is starting to seep in.
--It's not reality that's important, but how you perceive things.
I do not write this entry for myself. I write it on behalf of the soul departed. But when I climbed onto that bus to Cochin on saturday I was headed for a quiet sunday at home. But before I set foot in Cochin, I received the news. My uncle - V.M Venugopala Menon is no more. After nearly a decade of struggling with diabetes and its complications, he finally succumbed to the silent embrace of death sometime during the early hours of sunday.
For nearly eight years, he had a daily dose of human insulin to keep his blood sugar under check. During my final year in college, he lost a toe and would have nearly lost the foot as well. Ever since mid-2003, his kidneys had started to fail. But he was a man of iron will and still went ahead to enroll as an advocate, travelling nearly the length of Kerala to attend the ceremony in Kasargode, lugging a peritonial dialysis kit. To have that tube jammed into his lower stomach and have the liquid pumped into him was something he hated. But the moment the peritonium lost the osmotic qualities, the horrors began.
When I saw him month after month, he seemed to grow shorter and weaker. The haemodialysis which started off on a weekly basis, slowly became a necessity every other day. A complete kidney dysfunction, combined with the diabetic's slow healing, totally ran riot in his system. To see him on that wheelchair with his swollen feet, bitter about the world in general, yet prepared to fight till the end, if only for his children. His resolve only grew stronger as his health deteriorated.
We've always had our differences. But the blame is often quite evenly distributed in that. Of course, in our last conversation he probably forsaw this day. He was talking about the things he had left to see - my cousin's marriage, her brother's job. And then he said "Marikkenda samayam varumbayuthakum, ethokke nadakanam". And yet, didn't.
Maybe it was all for the good, to not let a man suffer so on this God's earth.
--What we have done for ourselves alone dies with us;
What we have done for others and the world remains and is immortal.