Limitations of Accessibility Testing Tools

One of the interesting points Andrew Kirkpatrick of the WGBH National Center for Accessible Media raised in his presentation to the IP495 meeting was the limitations of Accessibility testing tools. Of the Section 508 requirements, only 3 or 4 can be tested with an accessibility tool, therefore the certifications of accessibility by Bobby and others are of limited usefulness. He put it more colorfully, but since I can't remember the exact phrase, I'll paraphrase vaguely.

I have been working on this as I was updating the QA Test Tracker tool for Section 508 -- although by my count, 7 of the 508 standards can be tested with a tool: (a) ALT tags; (d) works without stylesheet; (g) table column & row TH; (h) TD is associated with TH; (i) frames have meaningful TITLE tags; (n) form elements have LABEL tags; (o) skip navigation provided. NOTE -- these are my abbreviations of the requirements - you can find the specific 508 requirements and suggestions for pass/fail at WebAIM.

Of the remaining standards, 6 can be tested in a browser by a trained tester: (c) no info only with color;(e) redundent links for server side image maps; (f) client side image map, not server side; (j) no flicker; (k) text-only version; (p) timed response. The remaining standards can only be tested in a screen reader: (b) synchronized captions (l) scripts are accessible (m) applets & plugins are accessible (n) dynamic form elements are accessible. These take considerable expertise in using a screen reader.

As I am currently writing a training program for QA testing, this raises the difficult question -- does Accessibility Testing belong in QA?

For more information: http://www.webaim.org/standards/508/checklist