This study explores the quality control system featuring visual inspection and grade judgment for detecting distortion defects in spectacle lens fabrication. Spectacle lenses must be precisely curved to help accommodate nearsightedness and farsightedness. The curved shape allows the lens to have different curvatures in different areas during grinding. The spectacle lens will be prone to optical distortion when the curvature changes abnormally. Accordingly, this study proposes an automatic distortion flaw inspection system for spectacle lenses to substitute professional inspectors who rely on empirical judgment. We first apply the digital imaging of a concentric circle pattern through a testing lens to create an image of that lens. Second, the centroid–radii model is employed to stand for the shape property of each concentric circle in the image. Third, by finding the deviations of the centroid radii for detecting distortion flaws through a small variation control method, we obtain a different image showing the detected distortion regions. Four, based on the distortion amounts and locations, we establish the fuzzy membership functions and inference rulesets to measure distortion severity. Finally, the GA-ANFIS model is applied to determine the quality levels of distortion severity for the detected distortion flaws. Trial outcomes reveal that the proposed automatic inspection system can help practitioners in spectacle lens fabrication, for it attains a high 94% correct classification rate of quality grades in distortion severity, 81.09% distortion flaw detection rate, and 10.94% fake alert rate, in distortion inspection of spectacle lenses.