Tuesday, January 2, 2007

Running an Open Source Software Project

Some people would be happy to convince you that managing an Open Source Software (OSS) project is completely different than managing a commercial software project. People working on Open Source software argue that there are no deadlines to meet, that quality issues can be left for the community of users to identify, and that their are no complications of costing and budgeting to manage. I will hopefully have convinced you of the contrary by the end of this article. I will have showed you that managing a normal software project and an open source project has far more parallels than most people would have you believe.

Do the common project life cycle principals apply to the average Open Source Software project? Do OSS projects go through the phases of definition, planning, organising, execution and closure? It is perhaps not quite as formal, but if you look hard enough the same phases of the development process can be found in all OSS projects. Whether it's conscious or subconscious, OSS projects all follow the common development stages. In fact it can probably be argued that the successful OSS projects follow these principles intuitively and instinctively. Good project management practices really can make the difference between successful and unsuccessful OSS projects.

Take for example the appointment of a project manager. The majority of OSS projects don't formally appoint an official project manger, yet in most projects you will find someone taking on the role of project manager. He understands the significance of the development process and the importance of leading a good team.

So what is it that an unofficial OSS project manager might bring to the successful OSS project? As mentioned earlier, the project manager can't ignore the familiar stages making up the common software development process:

1. Project Definition
2. Planning
3. Organising
4. Executing
5. Closing

However, following this development process amounts to very little if the project manager has little understanding of the key leadership skills involved. The ability to specify precise goals, to communicate clearly and to motivate members of the project is crucial to the leadership. In the Open Source environment, the motivation of the team members presents the real challenge. The familiar financial motivation plays no part in many open source projects. For most developers working in an OSS environment, you will probably find the following motivators having the biggest impact on the project:

* Achievement
* Recognition
* Responsibility
* Advancement

None of these motivators are tangible, but you have to pay close attention to them in OSS projects. It is easy in commercial projects to over look these points and focus on the more tangible motivator known as money. If you want a truly motivated team though, concentrate on both the tangible and the intangible motivators!

Even in the apparently chaotic world of open source software development, clear process and good leadership are essential tenets. The process may be more fluid in an OSS project and the leadership less formally defined, but both aspects are just as important all the same. In the following sections we will look at the stages of the project development process in the OSS context and see how team motivation plays its part in each of these stages.

Project Definition

"Project Definition" sounds very formal, but its importance can’t be too highly stressed. The problem definition influences everything within your project. Whilst working on an OSS project we may not formally document that we need achieve X and Y, but in many cases the objectives of an OSS project are far clearer in the minds of the OSS project team than that of the commercial software development team.

In the case of a commercial software development team, you might typically have someone define what needs to be achieved and then communicate that vision to the development team, hoping that they understand it. In the OSS scenario, you typically have a group of developers that have come together because they have all experienced, first hand, the "problem definition". This makes it is far easier to understand the problem and see a route to the solution.

Defining the 'problem' and specifying the requirements isn’t always straightforward. However, this is where most OSS projects have an advantage over their commercial counterparts. By far the best way to understand a requirement is not to read it, but to experience it. Most OSS developers have experienced the need to fulfil a particular requirement. After all, coming up against the problem is probably the main reason they've picked a particular OSS project to work on in the first place. Experiencing the need for a requirement first hand will always provide a better understanding than reading a written requirement specification second hand.

With a clear definition of the problem you are already ahead when it comes to motivating the team. Having a clear goal is absolutely key to engendering a feeling of achievement as the project evolves. You can only feel like you’ve achieved something, or that you are progressing, if you know what you are aiming for. So make sure you define the project and that you communicate that definition clearly to the rest of the team.

Planning

When you think of Open Source projects, formal project planning isn't one of the first tasks that spring to mind. Open Source projects have a reputation for a slightly more ad-hoc approach to planning. Maybe the OSS approach to planning isn't formal but, believe it or not, it is still a step that needs to be taken seriously. The OSS projects that succeed may not have formally defined project plans, but you can guarantee that the team has an instinctive ability to plan and organise their work.

The almost religious reliance on defect tracking tools (e.g. Bugzilla or Mantis) is possibly one of the reasons OSS teams are so good at identifying and organising tasks. In the OSS environment, the teams defect-tracking tool turns into far more than just a tool for tracking defects. It becomes the foundation that helps to organise and priorities the work of the whole team. It tracks everything from release tasks, defects, enhancement suggestions and, sometimes, even the to-do lists of individual developers. The defect tracking system’s effectiveness at prioritising and assigning tasks comes into its own within many OSS projects. Without the pressures of deadlines, the complexity of tools like Microsoft Project can be left behind for defect tracking tools that are far easier to implement and run.

The use of system architecture and design definition is possibly the weakest link in this comparison with formal project management. Whilst in commercial projects it is common to see reams of paper specifying the design of the system, it is not common to see this sort of work on an OSS project. In a typical OSS project, it is common for design ideas and concepts to be quickly prototyped in code and distributed to the community for feedback and comment. Even if it is perhaps not the most efficient use of time and resources, this proves to be an effective mechanism for identifying what should and should not be included in the application. Personally I advocate a least a certain degree of formal design work before coding begins. Formal design specifications add clarity to the project and help foster a feeling of advancement as the project progresses.

It is during the planning phase that serious consideration needs to be given to matching the goals of the project with the goals of the team members. One of the key reasons people get involved in OSS projects is to improve their skills and experience. If you decided to implement your project using Pascal, you would likely limit the pool of potential developers to work on the project, or even empty it. If, however, you select one of the more popular and exciting technologies, you are more likely to attract and retain coders on your project. Again the feeling of advancement whilst building and developing new skills helps to create an environment of motivation.

Organising

The OSS team usually excels at this stage. The ability to bring together the team members, the tools, controls and communications methods to get the job done are second nature to most OSS teams (partly because they know exactly where to turn to for OSS solutions to address these issues and partly because that are no organisational restrictions imposed on the implementation). A typical OSS team thinks nothing of implementing the tools needed to run the project efficiently. Setting up a defect tracking system, a forum and source code control in days if not hours.

The difficult, and perhaps crucial part, is how you open up these tools to the community. Do you open up your defect tracking system to absolutely everybody? Thereby exposing yourself to perhaps hundreds of poorly written, invalid defect records which all need sorting through. Do you provide easy access for new members of the development team to the source code repository? People who have no track record on the project could make critical changes to the code base.

I witnessed a recent exchange on an OSS project forum where a new member had been busy checking in code changes to the CVS repository. He had renamed fundamental aspects of the application because he thought he knew better about the terminology that should be used. This demonstrated how easily extra, unnecessary work can be created if you don’t get the project controls right. It is difficult to get it right as to whom, how, when and where you open up your source code repository. Yet getting it right is essential to the success of the project.

It is essential to get the balance right between restrictive controls and the freedom that helps motivate the team. If your controls are draconian you stifle enthusiasm and motivation. If you loosen controls you may find it easier to develop the levels of motivation. Giving your team more responsibility makes a big difference to the project and can be incredibly motivating. There is nothing complicated with principals, but never underestimate the importance of getting the balance right for your project.

It comes down to making the right decisions in involving the community you are serving and keeping the control and direction of the project on the right path. Like many things in life, the solution usually lies somewhere between the two extremes and can depend largely on the maturity of the project. Never forget though, that passing on more responsibility can be a powerful motivator.

Executing

You would think that the coding stage would be a walk in the park for most OSS projects. After all, we're all supposed to be good at this the part. Yet the success of this part of the project is largely dependent on the foundations built in the previous stages of the project. If you don't have a clear understanding of the problem you are solving or your prioritisation of the work was short of the mark, you limit the chances of success. Good coding alone doesn't make a successful project.

Assuming that the foundation stages have gone well, it is this stage where the "release early and often" approach is often cited as being the key to a successful OSS project. However, I would argue that an OSS project that relies solely on the community for its unit and system testing is asking for trouble. I recently upgraded to the latest version of a popular Linux operating system. Maybe it was my fault for not reading the release notes properly, but by the time I realised that they’d stop supporting a popular database that I relied on it was too late. Yes they released early, but I had always found previous early releases reliable and became complacent when upgrading. This taught me a valuable lesson regarding complacency and the deployment of OSS.

There is a balance to get right here, especially now that more and more people are starting to rely on OSS. If you continually release buggy software in today’s environment, you risk losing users. You can't expect users to test everything for you. With so much choice around now (I’ve lost count of the number of Linux distributions now), users will remain loyal only if you reach a certain level of quality before release. If you abuse the loyalty of those users, then you’ll have fewer users to further support your testing efforts. With fewer users you have less testers and you enter a slow but lethal spiral of death. Get it right though and you can expect a faithful, loyal following of users.

Feedback from users is another absolutely crucial aspect to the execute stage. Few OSS projects enjoy the privileges of a dedicated test team that is paid to give you feedback. This presents two key problems

1. the test team / users won’t be physically located near you
2. the users / testers aren't obliged to give you feedback.

To a large extent the feedback you get is down to two things

1. how easy you make it for people to provide feedback
2. how supportive you are to those people when they provide feedback.

This bit isn't difficult to understand. If they can't provide you with feedback (i.e. you don't give them a usable feedback channel) you won't get any feedback. If you don't thank your users/testers or you aren't grateful, then they won't provide you with feedback a second time round. More than commercial projects, OSS projects live and die by the feedback they receive from users, because they have no internal feedback mechanisms or internal test team to rely on.

Forums are among the best feedback channels and the most powerful motivators for OSS projects. Forums are so powerful that I find it difficult to understand why commercial projects don't use of them more. Perhaps it’s the thought of airing your dirty washing in a forum that puts commercial projects off using forums. Yet, if a developer in a commercial project receives negative feedback about his/her work in a forum, you can almost guarantee that he/she will feel motivated to do something positive about it. We all crave for feedback and recognition. Feedback through forums can satisfy those cravings.

Take, as an example, the last time you ate at a restaurant and complimented the waiter for a really delicious meal. The chef cooked the meal but we compliment the waiter. How do we know that the compliment was sent back to the chef? Wouldn’t it be more rewarding for the chef if he was complimented in person? It’s no different in software development, as forums can provide that direct channel between the users and developers. A forum is like standing up in the restaurant after finishing your meal and shouting through to the chef in the kitchen that you thought the meal was delicious. Not only does the chef receive your complement directly, but you’ve also told the rest of the dinners in the restaurant what you thought about the meal. That’s a huge motivator for the chef.

Closing

Of all the stages, this is the one that is very different to that of a commercial project. Getting it right can make a huge difference to the overall image of the project. You don't have to provide documentation. You don’t have to provide any support. You don’t have to make sure all defect fixes are verified before release. In a typical commercial project, you will not achieve sign off until all of these aspects are complete. Yet many people fail to realise that if you don’t complete these aspects with OSS projects, you are forcing your users to jump though many unnecessary hoops, which is likely to mean less users.

The Apache foundation is a really good example of completing the documentation successfully. People actually come forward to volunteer to write documentation for this project, because they receive kudos and recognition for their efforts. In smaller and less successful projects, this can be however an extremely difficult step to complete. Nonetheless, without good documentation it is very difficult to call an OSS project a success. Users are so dependent on well-written documentation. Imagine trying to use CVS without a well written user guide... it wouldn't be impossible but it would be far more difficult.

It is also during this phase that the feeling of achievement comes from knowing you’ve solved the problem defined at the start of the project. This is why it is so important to define the problem in the first place. When members of the team know they’ve solved the problem defined at the start, they know they’ve been part of a successful project. It’s important to remember that a successful project is defined as much by the members of the team working on the project as it is by the end users.

Conclusion

Recognition, responsibility, advancement and a feeling of achievement all play a crucial part in keeping a team motivated. It is that motivation that plays one of the biggest parts in making a project a success. No amount of project definition, planning and organising will achieve anything if your team isn’t motivated. Yet every stage of a project presents opportunities for you to motivate your team. Whether you are running an Open Source project or a commercial, closed source, project, you should take every single one of these opportunities to motivate you team because this will ultimately create successful projects.