Software Concerns & Metrics

This is a collection of notes on software metrics, organised by software concerns:

The Software Development Life Cycle

The software development life cycle of a project is a central concern to a project manager. Metrics describing software development life cycles are either retrospective or predictive. Retrospective metrics are essential to project evaluation, while predictive metrics can influence process and content decisions.

Metrics

• Project Length - The amount of days taken to progress from requirements analysis to deployment.
• Project Throughput - The reciprocal of the amount of days to complete the longest development stage in series of pipelined projects.
• Total Working Hours - The amount of hours that have been spent on developing a piece of software, across all workers.
• Time Until Deployment - A predicted amount of time until a project can be deployed.

Readings

• Fenton, N & Bieman, J. (2015) Software metrics: a rigorous and practical approach. 3rd ed. Boca Raton: CRC Press. ISBN 9780367659028 - Fenton and Bieman provide a textbook that outlines a goal-based project framework for software development, which is a framework that relies on software metrics. The textbook includes theory on measurment in general, types of metrics such as process, product and resource metrics; data collection and investigation techniques, data analysis techniques, prediction models and validation techniques.

Complexity

The complexity of software corresponds to the amount of effort that's required to test it fully, which in turn can affect the total working hours required during the software development lifecycle.

Metrics

• Coupling between Objects - The total amount of edges in an object dependecy graph.
• Cyclomatic Complexity - The number of linearly independent paths in the software.
• Depth of inheritence - The distance from a class to it's top level class in an inheritence tree.
• Efferent Coupling - The number of classes in a package, dependent on classes outside the package.
• Halstead's Vocabulary - The total amount of operators and operands used in the software.
• Lines of Code - The total amount of lines of code in the software.
• Number of Children - The amount of immediate sub-classes of a class.

Readings

• Chidamber, S. and Kemerer, C. (1992) 'A metrics suite for object oriented design'. Massachusetts: Center for Information Systems Research. - Chidamber and Kemerer build upon the concept of a relational system, which is a tuple of elements, relations and binary operations; and construct six software metrics that describe object oriented software structure.
• Ebert, C., Cain, J., Counsell, S. & Laplante, P. (2016) 'Cyclotomic Complexity'. IEEE Software. 33(6): 27-29. DOI. - Ebert et al. take note of the divide between academic and industrial motives, to explain the disparity in opinion on cyclotomic complexity..
• Fenton, N & Bieman, J. (2015) 'Measuring Internal Product Attributes: Size' & 'Measuring Internal Product Attributes: Structure', in: Fenton, N & Bieman, J. (2015) Software metrics: a rigorous and practical approach. 3rd ed. Boca Raton: CRC Press. 371-428. ISBN 9780367659028 - Fenton and Biemann include two chapters devoted to the internal attributes of software relevant to a goal-based project management framework. They specifically examine size and structure metrics.
• Hourani, H., Wasmi, H. & Alrawashdeh, T. (2019) 'A Code Complexity Model of Object Oriented Programming (OOP)'. 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT). Ammam, Jordan, 2019. New York: IEEE. 560-564. DOI. - Hourani et al propose a metric suite called Object Oriented Code Complexity (2O2C) that introduces a detailed class complexity metric, built upon submetrics.
• Martin, R. (1994) 'OO Design Quality Metrics'. An analysis of dependencies. 12(1): 151-170. - Martin outlines a set of metrics used to measure the interdependence between subpackages of the software, a part of it's structure, in order to provide a measure of maintainability.

Reliability

A software development lifecycle usually involves at least one stage specifically geared towards achieving quality and assurance, or reliability of software. Reliability contributes to a software's reusability as a product, and can decrease the software's future maintenence costs. Software testing is one way to increase software reliability. Types of testing include: unit testing, integration testing, performance testing and black-box testing. Some testing may be automatable too. Code linters for example offer a level of automated testing while code is being written, and software such as SonarQube can perform testing while code is stored in a git repository.

Metrics

• Invariance Conformity - The percentage of generated test cases that produce results that conform to transformational invariance.
• Rate of Issues - The amount of recorded issues in a timespan.
• Resuability Metrics - Some metrics related to the resuability of software are directly related to a software's reliability, when software access is necessary for software peformance.
• Security Metrics - Some metrics related to the security of software are directly related to a software's reliability, as security bugs are a subset of software bugs.
• Test Coverage - The percentage of a programme that has been tested.
• Test Effectiveness - The ratio of software bugs found, to number of tests written.

Readings

• Fenton, N & Bieman, J. (2015) 'Software Reliability, Measurement and Prediction', in: Fenton, N & Bieman, J. (2015) Software metrics: a rigorous and practical approach. 3rd ed. Boca Raton: CRC Press. 475-523. ISBN 9780367659028 - Fenton and Biemann include a chapter which is specifically about software reliability in a goal-based project framework. It talks about reliability theory and software failure prediction models.
• Smith, M. & Robson, D. (1990) 'Object-oriented programming-the problems of validation', Proceedings. Conference on Software Maintenance 1990. San Diego, USA, 1990. New York: IEEE Computer Society Press. 272-281. DOI. - Smith et al. idenitify aspects of object-oriented software that affect potential software validation techniques. Notable aspects include: inter-class organisation, inter-object control flow and concurrency, cases of inheritence and polymorphism.
• Zhou, Z & Sun L. (2019) Metamorphic testing of driverless cars. Communication of the ACM. 62(3): 61-67. DOI. - Zhou and Sun propose a method of testing unsupervised software, through the identification of invariance that should exist in the results of test case transformations, and then testing that the software conforms to that invariance, using generated data.

Reusability

Software reusability can be conceptualised through the notions of assets, processess, products and services. Component-based software engineering is a discipline that explicitly makes use of resuable assets. However, all programming paradigms implicitly depend upon some reusability of entities, so reusability is inheritent to all software engineering too. A lot of businesses aim to produce reusable products or services.

Metrics

• Active User Count - The number of concurrent users of a piece of software.
• Availability - The probability that software is available for use at a specific time.
• Reliability Metrics - Some metrics related to the reliability of software are directly related to a software's reusability, when the performance of software needs to be consistent for users.
• Renewal Ratio - The ratio of users who renew a software subscription to those that don't.

Readings

• Padhy, N., Satapathy, S. & Singh, R. (2017) 'State-of-the-Art Object-Oriented Metrics and It's Reusability: A Decade Review'. Smart Computing and Informatics: Proceedings of the First International Conference on SCI 2016, Volume 1. Visakhapatnam, India, 2022. Singapore: Springer Singapore. 431-441. - Padhy et al. outline 11 reusable assets of object-oriented software. These include: data, modules, architecture, algorithms, design patterns, documentation, knowledge, models, requirements, services and tests.

Peer Discussion

As a collaborative activity with my peers, we discussed 'which of the 11 assets identified by Padhy et al (2017), should be prioritised to maximise the reusablility of software?'. Each of us made an initial post to answer that question, and then we responded to those with our own development questions. The full discussion can be found here. The following is a summary post that I wrote at the end of the discussion:

"In my initial post, I created a priority list for the 11 reusability properties of object-oriented software given by Padhy et al., (2018), prioritised by how they affect the reusability of software. I considered three different ways that software can be reused; which were 1) through application to a user, 2) through code recycling to a software developer, 3) through experience gained during development. I prioritised properties that increased the reusability of software as an application to a user, and reasoned that the ordering I provided maximised the reusability of software, because there are more software users than software developers, and developers are more likely to recycle code of applications that are frequently reused. I learnt from others, many different interpretations of how to prioritise properties to increase the reusability of software. I had already suspected that a priority list was doomed to what the language philosopher Wittgenstein (1953) denotes 'language games'. That's the idea that words obtain meaning from contextual rules that speakers perceive to be governed by during the activity that they are participating in; so there is no singular correct answer. From reading the initial posts of others, I learnt that many people believe that prioritisation should take into account project specifications, and not everybody agreed that all properties contributed to the reusability of software in a significant way. During discussions I asked the following questions: Is it possible to of make each type of asset reusable across projects?, What specific steps can be taken to make different assets more reusable?, Which types of assets are easiest / most difficult to document?, and Is it important to document the lesson's learnt from a project? I asked questions because I wanted to gain some practical tips, and challenge my own and others perspectives, to learn. I learned that some project specifications may promote a position of disregard to the reusability of some types of assets, perhaps by the project specification being too specific. I learned that documentation of experiences gained during a project may have some educational value, and I learned that formal documentation is a process that exists as part of DevOps for some businesses. In my own discussion thread, I asked "How is the reusability of software a useful metric?". This question ended up being reflected back at me, but that allowed me to reflect on my own position. I considered reusability as software capability, and then as a motive behind software development. I considered scenarios where reusability is desirable, and scenarios where it is not, centring my arguments on contexts of cooperation and contexts of competition. Eventually, I recognised reusability as a metric, where it's equivalent to profit gained from pay-per-use software. Despite implicitly covering different aspects to the question that I was asked, such as strengths and weaknesses of reusability, I believe my answer was not structured very well, so my answer ended up being unsatisfactory. I should have finished my answer with a summary that explicitly returns to the question that I was asked at least, for clarity. I also attempted to provide an alternative metric to evaluate software, to avoid some of the problems associated with a single focus on reusability, but my attempt was awfully vague, and could do with more thought. Overall, I appreciate the effort that went into discussions. Reusability is no doubt, an important metric in many projects, and I managed to take away some things about it from the discussion activity. I also enjoyed thinking about it myself.

References

• Padhy, N., Satapathy, S., & Singh, R.P. (2018) 'State-of-the-Art Object-Oriented Metrics and Its Reusability: A Decade Review', in: Satapathy S., Bhateja V., Das S. (eds) Smart Computing and Informatics. Smart Innovation, Systems and Technologies. 77. Springer.
• Wittgenstein, L. (1953). The philosophical investigations. ISBN: 0-631-23127-7"

Discusson Plenary

If I were to diffract from what i've learnt about reusability so far, I would explore metrics of sociobiological origin. That's because sociobiology doesn't necessarily reduce the concept of harm in the way businesses do, but it encompasses the behaviour of businesses nonetheless. In sociobiology, assessment generally isn't restricted to a financial assessment for the sake of a businesses success. Therefore it provides a wide framework to consider the long-term harm of maximising reusability that businesses (and governments) might overlook. I'm especially sympathetic to addicts, for which reusability strategies may be too effective, and therefore problematic.

Software Quality

Software quality is a concept that is typically modelled as a composition of desirable characteristics, which are external qualities of the software.

Characteristics of Product Quality

• Compatibility - Functionality in regards to clients' technology.
• Maintainability - Extensibility in regards to structure.
• Reliability - Functionality in regards to faults.
• Performance - Functionality in regards to resources.
• Portability - Extensibility in regards to clients' technology.
• Security - Assurance in regards to vulnerabilities.
• Suitability - Functionality in regards to clients' needs.
• Usability - Functionality in regards to clients' ability.

Readings

• Fenton, N & Bieman, J. (2015) 'Measuring External Product Attributes', in: Fenton, N & Bieman, J. (2015) Software metrics: a rigorous and practical approach. 3rd ed. Boca Raton: CRC Press. 441-470. ISBN 9780367659028 - Fenton and Biemann include a chapter which discusses software quality, and traces the development of ideas around it before branching into examinations of less general software characteristics.

Organizational Standards

• ISO/IEC 25010 - Standards for system and software quality.
• ISO/IEC 5055 - Standards for the measure of software quality.

Polymorphism

Polymorphism is the use of a shared procedure for different objects of different types. Polymorphism is possible as a result of object inheritence in all object-oriented languages, and is possible with duck typing too, in languages that support it. Metrics that measure polymorphism in a programme can indicate whether it's trivial to retroactively add static type checkers to programmes written in dynamic languages or not, which might be required in the translation of softeware from one language to another.

Metrics

• Polymorphic Degree - The amount of different types that call a method during a programme.

Readings

• Åkerblom, B & Wrigstad, T. (2015) 'Measuring polymorphism in python programs'. Proceedings of the 11th Symposium on Dynamic Languages (DLS 2015). Pittsburgh, USA, 2015. New York: Association for Computing Machinery. 114–128. DOI. - Åkerblom & Wrigstad outline measures of polymorphism, such as observing and counting how many types of objects call each method in a programme, classifying each method by whether it's callers share a common superclass definition, classifying whether clusters of methods that share the same caller have been defined in some superclass of the cluster's callers, and classifying if those callers share a common subclass too.

End of Module Reflection

At the start of this module, I wasn't really aware of any software metrics and measures of quality. I was sceptical of the notion of good metrics and measures, as I've come from a mathematical and physical science background that makes use of definite concepts. The concept of software quality characteristics and good software metrics, looked very subjective from the surface. But during this module I've gained an appreciation of them for the role they have in project management and scientific discourse. They promote progression through the objectives that they provide, and through the refinement of the model they operate in. While they are subjective, they succomb to intersubjective forces in practise.

Fenton and Bieman's textbook on goal-based software development was especially instrumental in my change of opinion. I spent a good amount of time browsing each section, flipping my bias. This is a book that I'll get a physical copy of eventually.

Also during the module, I gained memorable experience from the peer discussion. After reflecting on how arguments can be constructed, I observed a technique of situating words into different contexts to arrive at the words potential meaning, and then comparing those contexts to chose the most helpful meaning for the argument. For example the word good has different meaning in the context of a competition compared to in cooperation, and the reusability of assets depends on the ability of potential users. I liked this technique, as it could be employed strategically. I would like to participate in more frequent research-paper centred discussions in the future.