being more than programmers

There is a recurrent theme that I see in blogs and books where it is stated that programmers have to open up, broaden their horizons and become more interested in the business perspective of software. Kent Beck dedicating Extreme Programming Explained goes: "Without you, this book would still be about programmers hiding in a corner. Without you, I would still be one of those programmers."

Dave Thomas, with usually very utilitarian views on our careers as software developers, also consistently encourages his audience to take into account the big picture and invest in our personal education and knowledge.

I'm sure this trend is nothing new but it's been very visible recently in the agile movement in general. A recent book along this trend is Chad Fowler's My Job Went To India. Its second chapter bears the title "Coding Don't Cut It Anymore". This idea of opening up is a good one and is usually well received. However, I feel that the direction in which we need to open up is often implicit in most texts. This is unfortunate as I believe this direction should be the result of a conscious decision.

There is no question that we need to be more than programmers. The mere will to write useful software needs us to be more than coders. However, I need more than the proposed, and dare I say obvious, solution. I would paraphrase this solution like so: we need to build a bigger role for ourselves by learning a few things from the way our organizations work and pay more attention to the customer's needs and the business value of our product: software.

Chad Fowler wants us to ask ourselves these questions:

"How much value did you produce last year? What was your positive impact on the company's bottom line? What did you give back? How much money did you cause the company to save? How much more in revenues did you contribute?"

or to follow the wave in search of the next big thing:

"Ten years ago, you would have been surprised to find out just how big object-oriented languages with garbage collection would become. But, there were definitely signs if you were watching. Ten years from now, who knows what the Next Big Thing will be? You've got to keep your eyes and ears open."

The shortcoming of these propositions is that their benefits are not distributed equally. Really, those solutions turn out to be very good for your company and your personal gain is only marginal. How to align the employees' incentives so that their actions benefit the company is a special case of the principal-agent problem. This is the one big problem we should be working on inside our organizations. Here is the spoiler though: this problem is only truly solved if you start your own company.

Short of doing that, there are lots of ways to pick up the slack and "open up" outside of our organizations. Open-source projects are an easy solution, you're not an "agent" when you labor out of love. This point is actually discussed in Chad's book so I will skip it. Community-oriented services are another solution: projects like the OLPC are using technology to respond to social problems and engaging programmers in their communities. Other than that, lots of issues are being publicly debated and it is our duty to understand them, if not take part in them. The copyright laws are completely out of touch with our everyday behaviors, how can we convince the politicians to work with us? The creative commons and copyleft licenses are trying to address problems with the usual licenses, how are they succeeding? How are they failing? Network neutrality boosts each citizen's right to unbiased and unrestricted information, that's a good thing but how do we do that without stifling investment and availability? Kiva.com offers microfinanced loans to people in need, how else can we harness massive audiences for the good of the poorest?

Another way to open things up and become more well-rounded is to mix disciplines. Common sense and general resourcefulness dramatically increases with the number of fields we can master or even just be interested in. This approach has known a lot of successes and will know many more. For instance, applying concepts from biology to programming problems led to genetic algorithms, ant colony optimization, agent-based modeling, etc. What else can we mix and match to increase the value of the whole?

If we do value individuals over processes and collaboration over contracts, I believe we should pay more attention to how software affects our communities. I believe we should pay more attention to how we can transform our organizations to the employee's benefits, not the other way around. Over the span of our careers, we will have many jobs and many employers. I'm only 25 and I've had 5 employers -- double that number if we are counting jobs outside the IT industry. This kind of track record is now the norm. As software experts and citizens, the only innate responsibility we have is to people and software.

• Lawrence Lessig on the state of copyright law (Ted Talk)
• Dave Thomas on developing expertise

for frequent commits

When I code, I always commit as often as possible. I try to bring myself to check code in at least every hour. Here is the case for frequent commits.

The psychological effects on me are just amazing. The rhythmic process becomes a dragonboat's drum that pushes me forward. If I fail to commit after an hour, I feel like I dropped the ball and end up working harder. The following points are more technical, but this one alone is enough to justify frequent commits.

Breaking down the tasks into small chunks becomes essential to make the next commit on time. When a task is hard to explode, I get creative and check in tiny bits of functionality at a time. Write a small test, make it pass, check it in. Refactor, check it in. It helps me to always stay in control, I can justify every step I take toward the goal.

Fewer integration conflicts result since less code is checked in at a time. This one is well understood. Conflicts are bad.

Flexibility. It makes it easier if I want to pair, go solo, or switch pair. I can "change state" every hour if my heart so desires. Also, if someone needs a piece of code from the story I'm working on, it might already be checked in. Sharing and integration cannot happen soon enough.

Thinner diffs between commits makes the problem of finding where a bug appeared/re-appeared easier. Once I found the build that "broke the feature", it's a lot easier to dig through a small change and pinpoint the dubious line. As a bonus, the log message for each check-in usually ends up very precise.

Backups. Every time I commit, I am also making a backup of my code. Losing my local work is never a problem.

Visibility. The commit log becomes a pulse on the state of each pair or each developer. It becomes another tool for the team or the manager to assess progress. Frequent commits become a sign of good health. How hard the current task is can now be estimated by the interval between commits. A slowdown in the commit rate might raise some questions during the stand-up meeting or the retrospective...

Frequent commits are a good thing.

smart shuffle

Here's a feature I've always wanted when using an audio player: shuffle the playlist so that all the songs from the same artist are as spread out as possible. You can try it out with the amarok player (apt-get users can click this link), after you've installed this script. Just right-click the playlist and choose Smart Shuffle -> Shuffle.

Here's the Ruby code that does the job, as long as the parameter to #smart_shuffle is an array containing elements that respond to #artist.

require 'utils'

def smart_shuffle playlist
  playlist.shuffle!
  groups = playlist.group_by {|item| item.artist}

  groups.inject([]) {|decorated_playlist, group|
    decorated_playlist + group.decorate {|item, index|
      every_n = playlist.length.to_f / group.length.to_f
      rank = every_n * (index + 1) - (every_n / 2.to_f)
    }
  }.sort_by_decoration.undecorate
end

Here's the file it depends on, utils.rb:

class Array
  def shuffle
    sort_by { rand }
  end

  def shuffle!
    self.replace shuffle
  end
end

module Enumerable
  def group_by &block
    groups = Hash.new(){|hash, key| hash[key] = []}
    each {|item|
      retval = block.call(item)
      retval = "Unknown" if retval == "" || retval.nil?
      groups[retval.to_sym] << item
    }
    groups.values
  end

  def decorate &block
    decorated = []
    each_with_index {|item, index|
      decorated << [block.call(item, index), item]
    }
    decorated
  end

  def sort_by_decoration
    sort {|this, that|
      this[0] <=> that[0]
    }
  end

  def undecorate
    map {|item| item[1]}
  end
end

I'm using a simple DSU algorithm where the decoration is given in those two lines:

every_n = playlist.length.to_f / group.length.to_f
rank = every_n * (index + 1) - (every_n / 2.to_f)

The variable every_n is the interval between two songs by the same artist -- a song from this artist should come back every n songs. I calculate the rank based on that, adding one to the index to prevent useless cancellation when it is zero. I subtracted (every_n / 2.to_f ) simply because it was giving me better results.