Thursday, February 15, 2007

The social structure of open source development

Andreas Brand is a sociologist researching ways of recruiting and organising teams of volunteers on the Internet. He has been studying KDE as an example of an open source project based upon collaboration without hierarchies. As part of his work he has conducted interviews with KDE developers, participated in several open source conferences, analysed the KDE home page, and distributed a questionnaire among volunteers. We asked him about his thoughts on the KDE development model.



NewsForge: How would you describe the KDE development model? Is it typical for open source projects?

Andreas Brand: KDE is a typical open source project, when you look at the demographics and the motivation structure. It consists of a core surrounded by several peripheral levels which are intermingled. In terms of the volunteers, you have the core developers and the peripheral developers; you also have the developers themselves as core and peripheral technical documentation and translation teams. In terms of the technology, there are levels of access to the modularised software, the sub-projects and mailing lists, the homepage, and last but not least the CVS repository. The most important participants, who form an inner circle, are the developers working on central software in sub-projects in the CVS repository and have access to central mailing lists.

The results of my research on the KDE community are analogous to those in the wider open source community (e.g. FLOSS Survey and Study: Survey of Developers, Ghosh et al 2002, and Why Hackers Do What They Do, Lakhani and Wolf 2005).

The organisational structure of KDE resembles other projects, for example with its release coordinator positions, but only to a certain extent; the way decisions are made is different in every project. Linux has a benevolent dictatorship and Debian a "democratic" voting system of members, comparable to the roman senate. KDE has no formal decision structure, but a special mailing list (kde-core-devel) for development decisions, and the rule that the predecessor should appoint a successor. It is a special kind of cooptation which may be found in small work groups or non-voluntary organizations or foundations.

NF: You looked at how contributors become "distinguishable persons" through gaining a good reputation. Do you think that becoming a "distinguishable person" is necessarily linked to working on larger or more sub-projects? Given that they have power through influence, is this healthy?

AB: Project participants who do a lot of work on software tend to communicate a lot, they work on important tasks, they're friendly, and so gain a good reputation. They tend to work on important, complex software, and work on several sub-projects. Furthermore, they invest a lot of their time so that they feel responsible for their work. This is a natural development for a large voluntary group like KDE.

In KDE this reputation is not worn like a crown. Members gain their reputation via other members of KDE, meaning that boasting about accomplished tasks isn't acceptable. There are several important figures in KDE whom one may only become aware of by getting in contact with and involved in KDE.

When looking at KDE, this seems healthy as long as these "distinguishable persons" work for the project and keep the project together instead of putting obstacles in the way of software development. They are valuable, but can also cause problems. They may, for example, stay in the project and scare new members away by not letting them work on their own ideas. They could also block discussions about certain themes that would otherwise enhance the project.

NF: How do you think KDE can retain or recruit its volunteers?

AB: KDE competes with other distractions or tasks in people's spare time, like family, job changes, and the end of studying. Nobody can prevent members from leaving because intrinsic motivation, rather than forced participation, rules KDE. However, some want to stay connected and work for KDE but with less effort. It would be the best to give them fields where they can work the way they want. Because of this, getting new volunteers is crucial.

So far, new, self-motivated members were recruited via Linux conferences or via articles in the media. Most recruited members are young and highly qualified IT experts who come from Northern Europe. Why not try to get further IT experts in other countries e.g. the Americas, India or China? And why not direct recruitment to both older IT experts and to non-IT experts? The non-IT-experts could be introduced through translation and work on graphics/sounds. However, they need to enter a certain channel in order to get more involved, e.g. from translation to software development.

Looking upon the ways in which recruitment is pursued and managed, a special group for this task is missing. This group could steer the recruitment efforts in internet forums, on- and offline journals, etc.

NF: Can you elaborate on the conflicts between the demands of the user community and the self-assigned work of the developers?

AB: The conflict results from conflicting interests of developers and end-users. The developers think about the technical efficiency of the software led by their bricolage mentality. Furthermore, there is the norm that whoever does the work has the final decision. In contrast, end users want a nice design, stability, and simple usability.

The stability of the software is a common interest of end users and developers. This is supported by the bug report program Bugzilla. I have not heard yet about problems in this respect. Divergent interests appear regarding subjects such as design and usability. For most developers, design and usability are unimportant because they are used to handling complex software when programming. The usability is also connected to the customisation of the software so that Microsoft users can migrate to KDE more easily. Some developers do not understand this point of view.

The conflict occurs through the direct communication of end users with developers, in which end users are often put off as potential members. In most enterprises, communication between customers and developers is filtered through a marketing department. In KDE, a usability group appeared which can filter the requests and draft a development policy. The group can mirror the needs of developers and end users. However, the group needs a more important standing in the project so that the developers would listen to the findings of this group and implement them in their day-to-day-work.

Direct communication does have advantages when the user is more advanced: better software can be programmed, which has an advertising effect and is an incentive for new members to join. Perhaps, it would be best to distinguish between beginners, who can contact the usability group, and advanced users, who can communicate directly with the developers.

NF: What are the effects of having a community primarily motivated by non-monetary factors, as opposed to a company motivated by wages?

AB: In companies, workers may be forced to do things they do not like because they depend upon their salary. However, they only do what they are paid for; self-motivation may be found but is not common. The development model in companies is normally planned and tightly scheduled. In KDE, it is the other way around: Nobody is obliged to do anything, but fun work is done much faster and all possible failures are taken into consideration. The gain in reputation usually overcomes the unattractive nature of many tasks, like fixing bugs. Sanctions are quite uncommon and are only imposed if central norms are broken or possessions of the project are appropriated to obtain intrinsic motivation. In companies, sanctions are used more often.

A further advantage of intrinsic motivation is cooperation and a common interest, so that there is no need for control. This also means that ideological or political conflicts don't occur. This does not diverge from the main aim of software development. Working in KDE is more like trial and error but with some constraints like development standards and release plans.

This has the effect that KDE has to recruit like-minded, intrinsically motivated (potential) members, and cast efforts there. Most companies have no problems finding employees because people have to work for a living.

NF: What is the significance of having hackers individually decide how their work is distributed, and how they will create their product?

AB: This is an interesting question, meaning that it is hard to answer. If the hackers were egotistical, rational actors with no social bindings, the collaboration would not work. The collaborative approach also depends upon the Internet. There has to be a common interest, like social exchange, or in this case the activity of working. The common interest leads to the emergence as a group, albeit virtual.

There is a considerable strain on the individuals sitting at home and the collaboration in the virtual group. Nobody can impose sanctions or directives on project participants, so they have a larger area in which they can make decisions. But this is only possible if every participant either picks a special, well-defined area of responsibility, or the process of communication is highly effective. If the virtual group increases, new strains enter the project. There have to be many tasks for the new participants so that they have something to work on. A new coordination structure is also needed because the larger a project, the more communication, conflicts, and stress it will have to deal with. Therefore, a division of labour occurs not only on the horizontal, but also on the vertical level.

The modularisation of tasks and software helps to manage the horizontal level. The people from the inner circle take up strategic management tasks like the relationship to other projects, the Free Software Foundation, and to companies (the vertical level). They represent the whole project. New positions of responsibility occur as well. If there are more positions and roles, there has to be a decision structure defining how people get these positions. In addition, there has to be some kind of "voting system" so that the will of the majority of the project members can be expressed. In KDE, the voting system is the discussion in the mailing lists. In other projects, other systems occur like dictatorships, democratic systems and the like.

NF: Given all of these concerns, what is it that makes the KDE project a success story?

AB: Looking at the KDE project and its success, you have to distinguish between several phases and factors. In the initial phase of the project, the founders had connections to other software developers from their previous open source project, whom they recruited. This helped to make a first version which worked as advertisement. They also had the luck to be in the right place at the right time; Linux was growing quickly but had no easy-to-use graphical interface (there were other graphical interfaces, but they were for experts). It was not only luck, but also filling a gap.

Later, the success story continued because the project entered an upward spiral -- new versions with new and better functionalities emerged which were further advertisements and led to further recruitments. Further recruitments led to new ideas and better functionalities. Some say that the competition between KDE and GNOME drives this spiral, but I would disagree based on conversations with some KDE members, which revealed that this competition was not the main theme. This is only one of several other reasons. There are also cooperative relations with GNOME. Another reason -- so they say -- is the constantly enhancing core library and the object oriented programming language C++ which allows, through its modularization in libraries, the re-usage of source code.

NF: So, in conclusion, is there a best practice of organising a project?

AB: I do not think that there is one best practice such as those that management scientists advocate for companies. There may be a best practice for specific tasks, meaning that this best practice is effective and efficient. But the practice has to be suitable for the specific organisational culture and the specific situation of a company, else you have to adapt the practices to the situations of a company.

If you transfer this to open source projects, you realise that new rules were introduced when problems arose. Some problems are connected to specific environments like the open source movement, companies, etc. and to organisational problems like growing membership, the division of labour, etc. Environmental and organisational problems are often the same for most projects. But this does not mean that all projects establish the same rules to cope with these problems. The founders not only wrote the first code but also established a certain organisational project culture that prescribed how to deal with problems. Based on this, a system of rules emerged where single rules fit together like pieces of a puzzle.

In other words, a rule that is efficient in one project might be an obstacle to another. Linux has the same problems as KDE, but it is governed by a benevolent dictatorship. Elections of project leaders would be counterproductive in Linux and a threat to the dictatorship system. This is a thought which has nothing to do with normative attitudes. If normative questions are considered, I think that a democratic election is better than a dictatorship. But which one is legitimated depends on the people who are working in a project.

NF: Andreas, thank you for your time, and good luck with future research.

AB: You're welcome. You can read the complete results of my work on the PELM Web site; though the full paper is only available in German, there is an English summary. There you could find also a new newspaper article in German about the collaboration in KDE.

http://programming.newsforge.com/article.pl?sid=05/01/25/1859253