treme Programming, with due credit to its founding fathers, bases its processes and practices on elements of simple and sound common sense. The focus in XP is on the things that matter in the development of software (a detail that seems sadly lacking in the exercise of prevailing methodologies). There are whole volumes available on XP that discuss what "really matters" in software development, and I won't dwell on it here because XP is not what this article is about. This article is about Open Source software development. Given the unrelenting ideal of better software that remains the guiding principle of the Open Source world, it is inevitable that it crosses paths with XP, in philosophy to a large extent and in practice to some. In the rest of this piece, I'll examine some "Xtreme" influences on Open Source software development -- what these are and what these can be.
The customer
XP recognizes that the single most important factor in software development is (or at least should be) the customer. Most key practices in XP revolve around communication with the customer, "listening" to his requirements and "responding" to his feedback. XP even advises that you have a representative of the customer available at the development site throughout the development lifecycle. Open Source projects have a quirky and largely unintentional similarity to this approach. Open Source projects, almost without exception, are born as tools to serve some personal need of the developer, who, eventually, comes to believe that the tool may benefit the rest of mankind, and publishes it. Others then try it, improve it, hack it, rewrite it, or supplement it to address some missing feature or to fix a bug. So, at least in the beginning stages of the development, the developers are the customers. And when the package is actually released to the community, rich feedback in the forms of reviews, bug reports, usage experiences, and comparisons, the like of which developers of proprietary software can only dream of, show up on the ubiquitous mailing lists. All in all, the customer factor in Open Source development is handled in a manner that even XP can learn from.
Releases
Most conventional software development methodologies work on a single release schedule. The development lifecycle spans years and is essentially blackbox from the point-of-view of the customer, who has to step out of the way and patiently await the day the developers transform his requirements into usable software. This approach offers no scope for change, for feedback from the customer on what is being churned out in the development process, or for a cut-and-run. For customers who need some degree of control on the development process, XP is often the answer. XP proposes an incremental release schedule that incorporates a finite subset of product features (sorted by business value) in each release. Customers feed requirement changes as well as their comments on the current release into a system that adopts these into the release cycle. And, importantly, each release is throughly tested, and the customer is assured of flawless execution within the scope of the limited feature set.
Iterative development of this sort is the norm in Open Source development. In fact, it's so much so that the Open Source SDLC has been referred to disparagingly as "release-first". To be fair, some of this criticism is deserved. Open Source package releases are often buggy and full of untested code, thanks to authors who have come to depend on bug reports and feedback from users to such an extent that they do not even perform elementary sanity checks on their code before publishing it. To be fair some more (to the developers this time), it must be mentioned that such releases are tagged appropriately and advertised as unstable. However, that's no excuse. One lesson that Open Source developers can take from XP here is to ensure that any release that they make is thoroughly tested in scenarios that the developers are familiar with. Feedback from the field can then be relied on for scenarios that are untestable in the development setup or which have been forgivably forgotten.
In the same context, XP assures its customers strict adherence to a pre-planned release schedule. Though the scope of each release may be varied (in consultation with the customer), the deadlines remain immovable. On the contrary, the release "schedule" in Open Source projects is, to say the least, haphazard. Planning is sometimes based on a TODO list, but is more often governed by essential bugfixes and the "triviality factor" of feature additions. Since most development endeavors in the Open Source world happen in the off hours when the developers are not at their regular day jobs, we sometimes see releases that follow each other with scarcely a day in between followed by lulls that may span years. Exceptions to this are the large projects with roadmaps that sketch the broad outline for a release schedule.
Formalism
The verbiage and unnecessary generalization of formal SDLC documents often end up cloaking the essentials of software development, what the customer wants his software to do (requirements) and how the developer satisfies these needs (design and implementation). When documentation (of this sort) becomes the mainstay of the development process, the only other supplicant in the process, the software itself, suffers. XP is decidedly against such formalism. XP uses user stories as alternatives to bulky software requirements specifications. These describe usage scenarios drafted by the customer, in his language and as he understands them. And instead of UML-based design documents, XP suggests simple CRC cards that capture the essence of the design in the developers' lexicon on a medium that is easy to retrieve and refer to.
The Open Source community, in its turn, never did believe in the idea of tracking the development process in formal documents. Design documents are usually written to serve as aids to future developers or maintainers. But the denizens of the Open Source world are well-versed in the art of browsing code to understand design and have never needed reams of paper describing it. How about a requirements document, then? Most projects are proposed as ideas on a mailing list. Developers thresh out the design through mail exchanges and posts. Eventually, a skeleton forms that is fleshed out in versions. Often, the bulwark of project documents is the set of user guides. In large projects with several modules and intricate cohesion, documents describing the interactions between the various modules and control flows are, however, essential. To their credit, Open Source developers have been quick to realize this. Most big Open Source development efforts today are backed with the necessary design descriptions.
KISS
Code the simplest thing that could possibly work. This simple idiom is another of XP's cornerstones. An ideal that could be one of XP's most valuable contributions to software development, this still remains one of the most difficult to adhere to. Most developers have a tendency, well-intentioned no doubt, to ensure right from the beginning that their code will cater to the wildest whims of 0.02% percent of their target users. Thanks to this, they end up cluttering their code with so many options that the usage info tends to span 6 screen pages. Or maybe they have chosen to incorporate into their code the best possible implementation of one of the more obscure components, thereby ensuring that the glue that attaches that component with the rest of the stuff is fragile. It is not the catering or the choice of the best implementation that XP looks askance at. It is the timing. It is not necessary that your first release be a study in optimization. Instead, let it be a study in simplicity, in functionality. Let it do all it set out to do and do it in the simplest manner possible. Let it give users a feel for what they can expect of their software. Chances are, they will be happy with the functionality. Optimization without user-perceptible benefits is one of the most unrewarding of activities in software development. Though this particular tenet is something that anybody, no matter how accomplished, can learn something from, Open Source practitioners are fairly adept at making this work. The manner in which they go about doing this, however, may leave something to be desired. Most initial releases don't even do half of what they are supposed to, leaving subsequent releases to implement the missing functionality. Substantially down the version chain, one finds releases that focus on optimization. Though admittedly a corruption of the concept, this does help avoid the pitfalls discussed above.
Refactoring
Refactoring is not easy. It requires the developer to fight one of his greatest fears, that of change, and indulge in an activity that would cause apoplexies over at "upper management" -- complete and merciless redesign of "perfectly good code". It requires that he have enough objectivity to perceive that what was once a beautiful piece of code or design is now archaic, bloated, or simply bad. It requires an understanding that times change and software must keep pace. An understanding that holding on to your brainchild is a sure way to stunt its growth and deny it opportunity to mature and evolve. Open Source developers are getting better at this. And "better", when you consider that the concept has yet to sink in with the proprietary software crowd, is actually pretty good. The Open Source world has long been the breeding ground of hacks and patches. But eventually, there comes a time when developers (or maintainers) grow weary of code that is so chock-a-block full of hacks and workarounds that even new features are implemented as hacks. This is when they bring out not their customary erasers, but a fresh whiteboard. This is when they sit down to rewriting their code, incorporating changes and assimilating workarounds and hacks right into the design, with little regard for the previous work. Didn't we coin the term "from-scratch rewrite"?
This-is-your-code
In the archetypal software development scenario, the software system is split into a set of modules (sub-systems), each of which is assigned to an individual or team. Though this divide-and-conquer strategy works well in practice, it has the inherent shortcoming of insulation. The teams remain isolated in their domains of expertise with regards to the system, and nobody from a team working on module A can even venture an answer to a query on module B, far from being qualified to answer it. XP proposes the strategy of Collective Code Ownership to tackle this fly in the otherwise effective ointment of module-based development. Collective Code Ownership makes every developer working on a project responsible for all the code in the project. Anybody can propose and make changes to any section of the code or design. The doyens of distributed development, the savants of version control, and patch gurus, Open Source developers are completely at home with this strategy. This is understandable because they had this one nailed long before it became fashionable. Most development in the Open Source world happens across continents, across timezones, and often across projects. Nearly everyone who uses the code is eligible to participate in its evolution. He can contribute to the design, write code, fix bugs, document, and influence the contents of the next release. This open-arms, this-is-your-software attitude towards users that is typical of Open Source software development is, if you come to think of it, its very foundation.
A related notion is that of continuous integration. Changes made to a shared codebase by multiple developers are merged, and conflicts (if any) are resolved. Regression tests are conducted to ensure that changes introduced by one developer do not corrupt the working of another's. If all is well, these changes are assimilated into the release cycle. A natural fallout of community development, this is handled effortlessly in the Open Source world. Patches may be submitted for consideration by all and sundry to the maintainers and primary developers. If a patch is adjudged worthy of integration (criteria for which is advertised well in advance), tests are run to verify that it can coexist with other accepted patches and will not break anything in the current codebase. Following this, it is incorporated into the base.
System Metaphor
A system metaphor is essential when a wide cross-section of developers are working on a shared code base. It brings consistency in nomenclature and language so that anyone who looks at a section of code understands what the code is basically about, even if he doesn't understand it all. This ensures that all developers are on the same page with regard to the system, and encourages collective code ownership. Coding standards of this sort are practically non-existent in the Open Source world. Developers tend to use their own vocabularies and their own ideas of how code ought to be written, effectively rendering their produce incomprehensible to most other developers. Of course, it must be noted here that Open Source developers are remarkably proficient at understanding code written by someone else, so lack of system metaphors is not (generally) a significant problem. Still, coding standards are being developed and advocated by several organizations and forums now. These, if adopted, will only serve to enhance the productivity of the Open Source community.
Other ideas
There are other maxims in XP that have yet to make an appearance in the Open Source world. These include interesting techniques such as pair programming and test-driven development. Though their usefulness and general good sense cannot be disputed, the fact that they require rather sharp breaks from what developers identify as "normal" development is to blame for their constrained adoption when compared to other key practices in XP.
Conclusion
Given that this article only sought to examine and comment on (rather than judge) influences of Xtreme Programming on Open Source software development, this conclusion (unlike the rest of the article) is going to be brief. We have seen substantial evidence of the "Xtreme" in Open Source development. And though I call them "Xtreme influences", I hasten to add that it is considerably more probable that XP, in its assimilation of best practices from other methodologies and its innate focus on common sense, adopted some of these tenets from the Open Source world. Whatever or however unintentional the influence, I propose that it has all been to the good and must only grow. Xtreme Programming is practical. Its tenets are infinitely applicable and designed for present-day software development. Open Source is mature and adaptable, and is, quite simply, the future. A confluence is welcome.
