Exploring the Use of Concern Element Role Information in Feature Location Evaluation

DSpace/Manakin Repository

Show simple item record

dc.contributor.author Hill, Emily
dc.contributor.author Shepherd, David
dc.contributor.author Pollock, Lori
dc.date.accessioned 2015-09-27T12:55:05Z
dc.date.available 2015-09-27T12:55:05Z
dc.date.issued 2015-05
dc.identifier.uri http://servo.cs.wlu.edu/pubs/handle/id/303
dc.description.abstract Before making changes, programmers need to locate and understand source code that corresponds to specific functionality, i.e., perform concern or feature location. Numerous concern and feature location techniques have been proposed, but to the best of our knowledge, no existing techniques or evaluations report information on what role a code element plays in the larger concern. In this paper, we report on two case studies that investigate two hypotheses on how evaluation studies of concern location techniques can be strengthened by utilizing concern role information: (1) by increasing agreement among human annotators for gold set establishment and (2) by providing richer information about the elements ranked as relevant by concern location techniques, which could help further improve the tools. We conducted a case study of 6 Java developers annotating 3 concerns with role information. When the developers understood the task description, pairwise agreement increased by 20%, 25%, and 135% for the 3 concerns over a prior concern location study without role information. Our findings also suggest that there may be core element roles that need to be annotated by humans, but that the remaining roles may be automatically derived, which could facilitate more reliable concern location benchmarks in the future. We also conducted an exploratory study of the element roles represented in results returned by a state of the art feature location tool. The results of these two studies suggest that integrating concern element role information into evaluations can help to strengthen both the gold set establishment and the analysis of results returned by various tools. en_US
dc.language.iso en_US en_US
dc.publisher IEEE en_US
dc.subject feature location en_US
dc.subject concern roles en_US
dc.subject investigative study en_US
dc.title Exploring the Use of Concern Element Role Information in Feature Location Evaluation en_US
dc.type In Conference Proceedings *
dc.booktitle International Conference on Program Comprehension (ICPC) en_US
dc.project Natural Language Program Analysis en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record

Related Items