NASA's Aircraft Aging and Durability Project (AADP) aims to ensure the safety of both commercial and military aviation aircraft. Non-destructive evaluation (NDE) techniques are integral to this effort. In particular, NDE techniques are used to a) detect and characterize damage to aircraft and b) validate models of materials through iterative testing. The costs associated with the acquisition of major aircraft require a long usage period in order to obtain a good return on investment. The commercial and military aircraft fleets are inspected and maintained to produce a long operational life. However, many of the NDE techniques are slow, tedious, and costly. Interestingly, the technologies used to inspect aircraft to detect flaws are quite sophisticated, but tools for keeping track of these flaws, their location, and evolution over time are haphazard and inspection-specific, so they are not easily generalized to inspections in general. We propose to leverage our machine vision technology to help automate portions of the inspection process to greatly reduce the time and cost associated with the inspection task. In this approach, machine vision is used to localize the sensor scan information gathered during inspection so that it can be viewed and manipulated in the context of a 3D CAD model of the inspected object. This then helps support the prediction of flaw propagation and structure life. The system allows maintainers to accurately collect information about flaws and accurately integrate them into CAD models. The models can then be leveraged in finite element analysis tools to help predict flaw and material behavior.