Project Management: Metrics and Variances

by Kiranmayee Pamarthy

As Tom DeMacro explains in his book, Controlling Software Projects (DeMacro, 1982), "you cannot control what you cannot measure."

The precursors for measuring are 1) Setting the guidelines and standards; 2) Managing the project; 3) Capturing the metrics; and 4) Usability.

Setting the standards. First and foremost, create a style guide or style sheet and use the guide for your standards. Continuous usage and strict adherence to these standards help writers become proficient in delivering quality deliverables. Employing multiple writers should not impact the way your company presents documentation to the customer. How many times have you heard the technical folks (the engineers for example) say that the user manuals are not used, so, why take pains to deliver quality deliverables? Negativity must never stop the quality output.

Managing the project. I'll save this section for a future article.

Capturing the metrics. Metrics can be captured for measuring productivity, defect density, budgeting or cost, etc. The following metrics analysis should be carried out for projects (overall and phase-wise where applicable), and corrective action taken when required. The variances are calculated against revisions:

• Effort Variance % = ((Actual  Effort - Planned Effort) / Planned Effort) * 100
• Schedule Variance % = ((Actual  Duration - Planned Duration) / Planned Duration) * 100
• Size Variance % = ((Actual Size - Planned Size) / Planned Size) * 100
• Review efficiency = No. of review defects / No. of  total  defects  (review + test +post-ship) - defects could be author induced or lack of technical inputs
• Productivity = Size / effort (No. of features or change requests / staff days)
• Defect density per unit size
• Overall Defect Density = Total number of defects / Size
• Pre-ship Defect Density = Pre-ship defects / Size
• Post-ship Defect Density = Post-ship defects / Size
• Productivity Variation
• Productivity Variation = (Actual Productivity - Estimated Productivity) / Estimated Productivity
• Estimated Productivity = Estimated Size/ Estimated Effort  (No. of features or change requests / staff days)
• Actual Productivity = Actual Size/ Actual Effort (No. of features or change requests / Staff days)
• Team Stability Index = Peak Planned Staff / Total number of staff in project
• Delivery Commitment = No. of deliverables delivered on time with regards to latest revised date / Total no. of  deliverables
• Resource Utilization  % =  (Actual project effort logged / Allocated effort) * 100
• Defect Age = Phase (Detected - Attributed)/ Number of Defects
• Cost of Quality (in terms of effort) 
• Prevention cost (Facilitation, Audits and Work Product Inspections)
• Appraisal cost (Work Product Reviews, Independent Unit Testing)
• Failure cost (defect fixing, i.e. rework)
• Requirements stability % =  (Number of requirements identified / (Number of requirements + changes to requirements)) * 100

Usability. The organization can create a set of usability metrics for capturing the data of the usage of the document.