Who is doing investigations into measurement of functionality and usability? [closed]

Question

I am seeking pointers to assist the direction of my research into providing assessements of functionality, usability or elegance in system development.

Can you provide references to work being done on the measurement of functionality, usability, or coding style? Who (person/organisation) is doing work in this area? Where can I find such references.

I have developed some ideas about system construction, that are a little different to mainstream development. The starting point is a an adequate description of the problem. I am developing a demonstration/proof of concept project.

In developing my proof of concept project I have found an unexpected and unlooked for benefit. So far, I have found that useful information about the completeness of a system can be gathered by taking an indirect view of the development. This indirect view is based on the description of the problem, rather than the software solution.

I have also become convinced as a result of these developments that it may well be possible to provide guidance from my approach to system development in other areas such as guidance on how functional the system is; how usable the system is; or how elegant the solution is.

My explorations so far have led to the following suggestions, and my responses to them:

1. Literate programming is elegant. - Literate programming may well be elegant, but this shifts the problem back - how do you assess how literate a program is?
2. That I am looking for the system development equivalent of a measure of aesthetics ie the ineffable - while I appreciate that such is not possible, I still believe it possible to give guidelines from the infomation available during the course of developing a system.
3. That what I am seeking can only be assessed after a solution has been in use for some time, and only by comparison with other solutions to the same problem. - This may well be the case, and may indeed the rock on which my search founders. However I still believe that it is possible that measures from the development may throw some heuristic insight on these aspects of software.
4. Function point analysis is a measure of functionality - I see FPA as more a measure of productivity than functionality. It does not tell you how much functionality has been incorporated into the system, bur rather how much has been included compared to a baseline derived from the same context. As the context is continually evolving, this detracts from the measure's usefulness.
5. That these concepts cannot be measured and there is no agreement on what constitutes functionality, usability, or elegance and there can't be for similar reasons to the argument - I am stubborn enough to believe I can, at least partially, refute this suggestion by providing some assistance to system developers.
6. That the information I am looking for lies outside of system development; in the graphic arts arena; psychology; biology; or other - this is looking more and more likely.
7. Utilising the objects of the system directly as its user interface - this shows promise for a class of systems designed to facilitate problem solving.
8. The lessons from traditional graphic art (layout of complex data) can be transferred to system development - this is looking like the most promising route and I am trying to establish correspondance with a leading graphic designer. This may only be of use for informational systems but does look as if it is much more wide ranging than that.
9. It has also been suggested that I am being an "architecture astronaut", out of touch with reality - this may be the case, but if it is so, then I am likely to be the last to realise it and such a prospect does not deter me from my search.

Answer 1

Of the three things you list, usability is the most measurable. Doing a search for "measuring code usability" will yield many hits, for everything from websites to parallel programming.

Some highlights:

ISO standards on software engineering; Here you'll find the standards for product quality and the software development lifecycle:
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_tc_browse.htm?commid=45086

The Cliff Notes version of the ISO standards : )
http://www.usabilitynet.org/tools/r_international.htm

Standards for software ergonomics:
http://www.iso.org/iso/catalogue_detail.htm?csnumber=52712

From a well-written paper on parallel program usability, found here:

Several features of a PPS determine its usability. Among these are:
1) Learning curve: How long does it take an expert or an inexperienced parallel programmer to be able to use the PPS productively? Note that some PPSs specifically address the needs of experts, while others are targeted at novices; few are suitable for both.
2) Programming errors: Some systems restrict the use of parallelism to prevent errors (e.g. Enterprise). Other systems, such as NMP and PVM, allow the user to do anything, trading flexibility for a higher chance of programming errors. Usually the potential for errors is directly related to the number of lines of user code. Therefore, systems that require more user code may be more susceptible to errors.
3) Deterministic performance: Non-determinism, common in the implementation of some algorithms and inherent in some PPSs, can significantly increase the overhead in application debugging.
4) Compatibility with existing software: Legacy software cannot be ignored. Ideally, the PPS must support the integration of existing software with minimal effort.
5) Integration with other tools: A PPS should either come with, or provide access to, a complete suite of software development tools including facilities for debugging, monitoring and performance evaluation.

An article on quantifying and measuring functionality:
http://www.computer.org/portal/web/csdl/doi/10.1109/METRIC.1999.809732

Links to the CUE-4 Hotel Pennsylvania usability studies, where 17 independent teams performed usability of the website for Hotel Pennsylvania
http://www.dialogdesign.dk/CUE-4.htm

This Wikipedia article has a lot of links to software-quality-related articles. The article itself discusses a number of focal points for software quality, including Understandability, Conciseness, Consistency, Maintainability, Testability, Usability, Reliability, and Efficiency, among others.

http://www.drdobbs.com/windows/184405654;jsessionid=SB2LUABORKQHBQE1GHOSKHWATMY32JVN
The author discusses techniques that Microsoft uses to design and evaluate the usability of their APIs.

Another suggestion: Go to some of the more renowned colleges for software engineering and nose around on their Computer Science homepage for published articles on the subject.

As others have said, establishing quantitative measurements on these principles is like nailing jello to a tree... but I disagree that they can't or haven't been studied in quantifiable analyses.

HTH!
James

Answer 2

I think some of these features would have to be measured after the product has been in use for quite some time, as compared to another, similar implementation.

Consider the example of multiple GUI implementations for a piece of software. You could measure things like how long it took a user to accomplish a certain task using one specific implementation relative to accomplishing the same task (for the nth time) on the (almost) same piece of software using different GUI implementation. This would provide for some sort of a useful metric in relative terms.

Going along this path might help you clarify these ideas in terms of getting (publishable?) results. From reading your original description, it sounds like you are looking for absolute rather than relative metrics. However, from quickly looking over this question and trying to come up with an interesting, useful, reply during a five minute break from my work, the relative metric was the best I could come up with.

I hope this helps,

Brian J. Stinar