Toward Experimental Evaluation of Subsystem Classification Recovery Techniques
Arun Lakhotia and John M. Gravely
Abstract
Several reverse engineering techniques classify software system
components into subsystems. These techniques are designed to discover
such classifications when the classifications are unknown. The
techniques are tested and evaluated, however, by matching the
classifications they recover against expected classifications. Several
such techniques may be compared by experimentally evaluating their
performance on the same set of software systems. Two things are needed
to ensure experiment repeatability: (I) a set of "real-world" software
systems whose expected subsystem classifications are known, and (2) an
objective criterion to quantitatively determine the similarity of
subsystem classifications. This paper contributes to both needs by
identifying a set of widely used and easily accessible software
systems whose modular decomposition either is documented or can be
easily inferred from their design philosophy, and by presenting a
measure to quantitatively determine the congruence between
hierarchical subsystem classifications.
Full
paper