Step 1 -- Examine Process Documentation
  • Assess general rigor of development process

  • Very important for operational software (failure costs $$$$ or lives)

  • Process for tracking and controlling requirements

  • SDP (SW Development Plan), SCMP (SW Configuration Mgmt Plan), SQPP (SW Quality Program Plan), WBS (Work Breakdown Structure), Test Plans

  • Peer review minutes

  • Walkthrough minutes

  • System version description documents

  • What DOD or commercial standards were required? Were they followed?

  • How clear is documentation?
  • Was it used?
  • Was it produced during development, or after?

  • Metrics
  • Project estimates vs. actuals
  • Time expenditures by WBS category
  • Lines of Code (LOC)/person/year
  • Defects per LOC
  • Average time to fix defects

  • How were metrics derived?
  • Over multiple projects?
  • Over a single project?
  • Over what length of time?

  • Assess perceived CMM Level
  • CMM Level 2 or 3 is minimum
  • For operational software, CMM Level 4 or 5 is minimum

  • This examination can become very adversarial if performed after development is finished

    • The Eleven Steps In VV&A:

    1. Examine Process Documentation
    2. Examine System Documentation
    3. Interview Developers
    4. Interview Users

    5. Examine Code 1 (Look For Problems)
    6. Examine Code 2 (Break It)
    7. Determine Truth

    8. Generate Test Cases
    9. Run Test Cases
    10. Review Test Case Output
    11. Assessment


    Software Development Plan (SDP) Template
    Software Configuration Management Plan (SCMP) Template
    Software Quality Program Plan (SQPP) Template
    Work Breakdown Structure (WBS) Template

    Introduction
    What And Why Of VV&A