Often Software (with a capital 'S') is not built, it is grown. And a lot of green digits are involved, not always thumbs.
When I got an education which taught me how to build software, the process emphasized on sitting down in front of my computer, pushing my will upon a barely explicable machine which was built to resemble advanced technology on some days and complete magic on others.
In fact, when I think about it, every part of my computer science education was about imperative commands to a system which, if you give it adequate commands always throws up unsurprising results everytime (and all surprises were bugs). Everything I was taught about this fit that model, where when I had a "Software Management" class it was full of Gantt charts and PERT charts, going over how to schedule time for different engineers to work on the same project in parallel. The recommended reading was The Mythical Man Month and that was hilarious to read, at least. Mostly that class was taught as a way to have an engineer understand what a manager might have to do, not exactly explaining the operational realities.
This post is about the third time I realized I was ill-equipped to deal with the actual demands of being an engineer with the industry. Because none of what I describe works the way I've been taught - at least, when it comes to these skills I'm entirely self-taught.
The first time I figured out that this was all wrong was when I was bumped to a tech lead, one rung below turning full manager. Humans are pretty hard to instruct and they respond to imperative commands by being confused about the goals, slowing down till they understand it and planning around your well laid plans. This was the first time I figured out that breaking down a complex project into a set of sprint tasks, going over burn down charts and in general replacing motivation with process does not actually work for humans. Instead it sets up a bunch of perverse incentives which builds up technical debt and kicks the can on potential problems until one day they can pawn off that problem as "above my pay grade" to a special tiger team which drops in out of nowhere. People don't work like machines, but they do work - and they move much faster when they can take decisions that they can reverse on their own (turns out machines do that too, see SPECTRE).
The second time I figured out that the world where I'm employed doesn't look like my classes is when I started talking to customers. I wrote about my Zynga tenure that what I learned was to persuade people, not direct them. The interaction between a customer and a software engineer is one that is fraught with many traps which primarily rise from tossing bugs over with a blame thrower. Feature discussions are yet another scenario where being a "can do" person brings its own challenges. The crucial requirements for a customer are often expressed as potential solutions, where they ask for something very specific which worked for them in a different generation of tech or just ask for a feature which they read about in a press release by some competitor. Technology persuasion is a gentle process similar to managing reports, more art than science - which only works if you really listen, head back and build what approximates the need, instead of desire.
However, today I'm faced with a slightly different challenge - I'm overflowing out of my current role as a "Principal Engineer - II". That isn't a destination for an engineer as much as it is a reserved parking spot.
There's a slightly different track adjacent to it, which will use most of what I already know, but I'm contemplating what I have to give up to move over to a "Software Architect" in role, if not title. I find that title somewhat archaic, because it implies very strongly that software is designed by someone as a measured blueprint and then put together by bricklayers, overseen by foremen and all that. However, the reality disagrees vehemently, at least from my vantage point.
And I realize it is hard for an executive to look at me through an architect cookie cutter, because I'm already something else. There are literally dozens, DOZENS of us in this industry.
My open source software work is sort of like a slightly overgrown garden. I don't make plans with measurements, I make grids and boundaries for sure. The bugs creep in, but I try to make it easier to debug. The feature branches are trimmed to stay manageable and the features which pop up like weeds need to be taken care of early, before they start to creep over. But most often what I really point to is that the garden just needs time and attention to keep it up, however any week it is always unfinished with a bunch of flower beds yet to bloom or bulbs to repot out of the alpha bed into the beta. Gardeners do get credit though - but in a garden it is obvious that the plants all grow by themselves, but it is still all work of the gardener to fertilize the soil, regularly water and put all the effort into giving it the opportunity to thrive. The significant and hard requirement is being constantly present, to rotate through your skills over the lifecycle of your crop - including calling in & giving up specialized tasks to the right people.
The frustrations are extremely similar too - a drought in the market, needing to redo everything because the plumbers need to dig it up to fix something fundamental with the way the funding irrigation comes in, to literaly turn on the tap. A worker in a garden is not entirely in control of the process, as much as they take responsibility for the plants, there are no miracles they can perform beyond what nature allows them to. And in a round about way, everything that grows in this garden grows up on the dead remains of a previous generation without giving it any credit for laying down all this top-soil. Every few years, literally turning-over old projects to prepare the ground to plant new seeds.
And sometimes we do need to plant a few trees whose shade you will never sit under.
As history of the Santa Clara valley rhymes through the bracero years, I've turned into a gardener of open source software in fertile soil, brought in from a different land, taught again how to do this at an industrial scale, but organically and sustainably.
--Plants do not grow merely to satisfy ambitions or to fulfill good intentions.
They thrive because someone expended effort on them.
-- Liberty Hyde Bailey
posted at: 11:26 | path: /observations | permalink |