Applying Empirical Software Engineering to Software Architecture: Challenges and Lessons Learned

In the last 15 years, software architecture has emerged as an important field of software engineering for managing the development and maintenance of large, software-intensive systems. The software architecture community has developed numerous methods, techniques, and tools to support the architecture process. Historically, these advances in software architecture have been mainly driven by talented people and industrial experiences, but there is now a growing need to systematically gather empirical evidence rather than just rely on anecdotes or rhetoric to promote the use of a particular method or tool. The aim of this paper is to promote and facilitate the application of the empirical paradigm to software architecture. To this end, we describe the challenges and lessons learned that we experienced for assessing software architecture research by applying controlled experiments, replicas, expert opinion, systematic literature reviews, observation studies, and surveys. In turn, this should support the emergence of a body of knowledge consisting of more widely-accepted and well-formed theories on software architecture.