Mining software repositories
Within software engineering, the mining software repositories[citation needed] (MSR) field [1] analyzes the rich data available in software repositories, such as version control repositories, mailing list archives, bug tracking systems, issue tracking systems, etc. to uncover interesting and actionable information about software systems, projects and software engineering.
Definition
Herzig and Zeller define ”mining software archives” as a process to ”obtain lots of initial evidence” by extracting data from software repositories. Further they define ”data sources” as product-based artifacts like source code, requirement artefacts or version archives and claim that these sources are unbiased, but noisy and incomplete.[2]
Techniques
Coupled Change Analysis
The idea in coupled change analysis is that developers change code entities (e.g. files) together frequently for fixing defects or introducing new features. These couplings between the entities are often not made explicit in the code or other documents. Especially developers new on the project do not know which entities need to be changed together. Coupled change analysis aims to extract the coupling out of the version control system for a project. By the commits and the timing of changes, we might be able to identify which entities frequently change together. This information could then be presented to developers about to change one of the entities to support them in their further changes.[3]
Commit Analysis
There are many different kinds of commits in version control systems, e.g. bug fix commits, new feature commits, documentation commits, etc. To take data-driven decisions based on past commits, one needs to select subsets of commits that meet a given criterion. That can be done based on the commit message.[4]
Documentation generation
It is possible to generate useful documentation from mining software repositories. For instance, Jadeite computes usage statistics and helps newcomers to quickly identify commonly used classes.[5]
Data & Tools
The primary mining data comes from version control systems. Early mining experiments were done on CVS repositories.[6] Then, researchers have extensively analyzed SVN repositories.[7] Now, Git repositories are dominant.[8] Depending on the nature of the data required (size, domain, processing), one can either download data from one of these sources. However, Data governance and data collection for the sake of building Large language models have come to change the rules of the game, by integrating the use of Web crawlers to obtain data from multiple sources and domains.
See also
- Software evolution
- Software analytics
- Software maintenance
- Software archaeology
- Curated list of software repositories from GitHub.
References
- ↑ Working Conference on Mining Software Repositories, the main software engineering conference in the area
- ↑ K. S. Herzig and A. Zeller, “Mining your own evidence,” in Making Software, pp. 517–529, Sebastopol, Calif., USA: O’Reilly, 2011.
- ↑ Gall, H.; Hajek, K.; Jazayeri, M. (1998). "Detection of logical coupling based on product release history". Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272). pp. 190–198. doi:10.1109/icsm.1998.738508. ISBN 978-0-8186-8779-2.
- ↑ Hindle, Abram; German, Daniel M.; Godfrey, Michael W.; Holt, Richard C. (2009). "Automatic classication of large changes into maintenance categories". 2009 IEEE 17th International Conference on Program Comprehension. pp. 30–39. doi:10.1109/ICPC.2009.5090025. ISBN 978-1-4244-3998-0.
- ↑ Stylos, Jeffrey; Faulring, Andrew; Yang, Zizhuang; Myers, Brad A. (2009). "Improving API documentation using API usage information". 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). pp. 119–126. doi:10.1109/VLHCC.2009.5295283. ISBN 978-1-4244-4876-0.
- ↑ Canfora, G.; Cerulo, L. (2005). "Impact Analysis by Mining Software and Change Request Repositories". 11th IEEE International Software Metrics Symposium (METRICS'05). pp. 29. doi:10.1109/METRICS.2005.28. ISBN 978-0-7695-2371-2.
- ↑ d'Ambros, Marco; Gall, Harald; Lanza, Michele; Pinzger, Martin (2008). "Analysing Software Repositories to Understand Software Evolution". Software Evolution. pp. 37–67. doi:10.1007/978-3-540-76440-3_3. ISBN 978-3-540-76439-7.
- ↑ Kalliamvakou, Eirini; Gousios, Georgios; Blincoe, Kelly; Singer, Leif; German, Daniel M.; Damian, Daniela (2014). "The promises and perils of mining GitHub". Proceedings of the 11th Working Conference on Mining Software Repositories - MSR 2014. pp. 92–101. doi:10.1145/2597073.2597074. ISBN 9781450328630.
 |  |
