This is a repost of an old article on the Tenon blog. Since that’s being sunsetted, I’m reposting it below.

The Accessibility Business Case is something I’m very interested in thinking and talking about. Historically, many in the web accessibility field have produced various facts and figures providing a business justification for accessibility. On my personal site, I’ve discussed issue prevention as a business case. Of all business case arguments, it is the one I’ve seen materialize in person with clients. But what’s the best way to truly realize the ROI of issue prevention and accessibility?

  • You must reduce the feedback cycle for accessibility
  • You must provide robust tooling and evaluation practices that support those feedback cycles

How much does it cost to fix a bug?

There are a ton of articles on the Web that discuss how much a defect costs. Via Google Search:

This is described and demonstrated in the oft-cited “Boehm Curve” described below:

At first, the cost of fixing a bug at the requirements stage is nominal, when everything is on the drawing board. But as the software moves along in its life cycle the cost of fixing a bug increases radically. We start at 1 times when we are at the initial development stage when a bug is no more than a change in notion. But at the design stage, the relative cost is 5 times what it was compared to the requirements stage, and then ten times what it was when it becomes code and on this goes until it the relative cost of a bug fix is 150 times what it was originally. (Source)

But there’s another important concern: Lost time on other development. If your developers are spending time fixing bugs, that means they aren’t developing new features. This unnecessarily delays delivery timelines on new projects.

The review of available data…revealed that a large proportion of total development effort is devoted to rework. As a conservative estimate, we assume that 50% of the development effort is rework. The next default is the proportion of post-release effort that is devoted to rework (or fixing bugs). Again, the data…indicates that we can safely assume 50% of total post-release costs are being devoted to rework.
The ROI from Software Quality, Khaled El Emam

Poor quality control doesn’t just cost money but also morale.

However, it was the hidden costs that drained us of life: hours lost arguing about bugs in bug triage meetings; hours lost stumbling over the same known issues again and again; hours lost fighting with a fragile and error-prone code base to make even small changes; hours lost cataloging and categorizing the backlog. It was demoralizing and immensely expensive. (Source)

Ensuring accessibility requires tooling that supports the entire lifecycle

I’ve repeatedly discussed the significant benefits of catching issues before they happen and it is obvious that not making mistakes is always better than having to fix a mistake after it happens, but of course we know that isn’t true. The best developers on earth make mistakes for any number of reasons. They can be simply typos in function names or they can be a case of ignorance of a new feature or an infinite number of possible causes. A mature approach to quality requires two fundamental features: a robust mechanism for gathering feedback that feeds into opportunities for improvement and that those feedback cycles be as short as possible. All persons involved in a project must have the opportunity to contribute feedback on project quality and that feedback should be addressed when doing so is the most effective. Mistakes will happen. They should be caught and fixed as early as possible. But sometimes that isn’t the case. Robust tooling should be employed that allows for accessibility feedback as early as possible and in as many places as possible.

Everyday development, linting & code quality checking

Task Runners like Grunt and Gulp are indispensable resources because they allow you to package together a series of repetitive tasks like concatenating files, minifying large source files, or linting code. Almost every project I work on runs the same set of basic Grunt tasks (example). I also use a quality IDE that helps with code completion, syntax checking, etc. None of these things make me a better developer, but they’re critical in shortening the feedback loop on quality. For instance, if I have a typo in a class name – effectively trying to instantiate a class that doesn’t exist – my IDE tells me this. This immediate feedback warning me of a potential problem helps avoid bugs. Given the exact same very-easy-to-make mistake, the next opportunity for me get this feedback would require me switching to my browser, loading the page, and seeing the error happen. Perhaps doing so only takes a few extra seconds, but it is obviously better to be able to catch that issue in real time within the IDE itself.

There is no difference here between syntax errors, code bugs, or accessibility at this point. The importance of getting accessibility feedback at this stage cannot be overstated. Nowhere in the project is the feedback cycle this short. Having accessibility feedback at this time is the best opportunity to avoid errors and as I’ve mentioned before it is also the cheapest. There are other immeasurable benefits to such early feedback in the incremental improvement of development practices. Getting such early feedback and fixing the issues right then will cause the developer to be mindful of such things in the future. For someone new to accessibility it may take a while for that to happen, but there’s still no better time to learn.

This is why the very first integration that Tenon.io created was with Grunt. We worked together with Andrew Babaian of Razorfish on his release of the Grunt Tenon plugin long before the official release of Tenon, as this is exactly the type of thing that facilitates the early feedback cycle we aim to provide.

Recognizing the critical importance of this early feedback, Microsoft and Tenon worked together to create the Tenon HTML Accessibility Checker for Visual Studio which enables developers direct access to accessibility feedback directly within their IDE.

Manual QA

Mistakes happen. In a perfect world, developers would catch their own bugs and in fact, doing some form of unit testing is a critical requirement for ensuring quality. Even then, issues may arise due to any number of possible reasons and QA plays a vital role in ensuring that issues are caught before the issues reach the end user.

Automation certainly has its place in the manual QA process, especially in terms of increased efficiency of testing, overall. As I often say, there are many accessibility best practices that don’t require a human to test. There are others that a tool can check but require a human to verify. In either case, the increased efficiency cannot be denied. For this reason, Tenon’s web-based interface was developed to facilitate robust testing for QA staff without requiring them to use the API itself. QA testers can also utilize browser plugins for Internet Explorer, Chrome, Firefox, and Opera to assist in testing.

QA staff should follow automated testing with rigorous manual and use case testing for effective coverage of accessibility requirements.

Automated Acceptance Testing/ QA Automation/ Continuous Integration

In a large development team with multiple developers possibly distributed across multiple locations, continuous integration can allow for quick (daily, even) feedback loops and a more cohesive development process. In some cases, every commit by every developer can be run through a continuous integration system and during integration the corresponding automated tests can be run to ensure that not only is the new code passing its relevant tests but that nothing else was broken by the changes.

Tenon is again well-suited for such tasks and can not only be included in CI to run Grunt/ Gulp tasks but can be used during tests with Selenium as well, as shown in this example created by Justin Stockton. As the video below shows, Tenon can be used to be test an entire page or only part of a page during automated acceptance testing via Selenium and a basic assertion library. In this way you can think of Tenon as a series of over 8-dozen pre-written tests.

Tenon Selenium Demo (video).

Content Authoring

On blogs, news sites, higher-education websites, and other sites where content is added frequently, content authors often introduce many hard-to-track accessibility issues due to the sheer volume of content and (often) lack of technical knowledge of the content authors. For instance, I once did work with a US Federal client that had a team of about a half-dozen people whose sole job it was to add content to the site. In one section of the site they would add roughly 30 pages of content every day, relating to the activities of this government agency. It included press releases, daily briefings, activities of the Secretary and Assistant Secretary, photo galleries, and the like. In content-heavy sites like these, it is critical that these users have some mechanism of checking their content before publishing. Since over 74,000,000 websites run on WordPress we teamed up with WordPress accessibility expert Joe Dolson to create Access Monitor plugin. Access Monitor allows WordPress site owners to test their site’s Theme and also to check the accessibility of individual posts and pages.

Similar to the case for developers, providing accessibility feedback for content authors directly within the Content Management System represents the shortest possible feedback cycle and therefore the best opportunity to avoid user-facing issues. This use case is very important for ensuring the website’s overall level of accessibility, because this content isn’t subject to version control and isn’t unit tested or integration tested. In other words, if this content cannot be tested from within the CMS, the only opportunity for CMS-content-based accessibility issues to be caught is after it is published. This represents the highest level of risk as the published accessibility issues are now exposed to the public. In addition, this leaves only one more opportunity for these issues to be discovered – post publication/ post production testing.

Continuous Monitoring and Managing

In a perfect world, all of the previously mentioned use cases would provide full coverage and all automatically-discoverable issues would have been found and fixed before any end user actually sees it. But we don’t live in a perfect world. Issues will still make their way into production and the more frequently the content or code changes, the more likely this will happen.  This is why the Tenon.io website allows projects to be created so that publicly accessible project URLs can be tested automatically and be monitored. During the monitoring process, Tenon automatically determines if a URL has changed and, if so, retests it.  Through this process, organizations can keep a close watch on their systems’ level of accessibility and react quickly to newly discovered issues.

Tenon’s dashboard also provides a number of key data points that allow management to understand the state of their systems’ level of accessibility, such as average issue density, issue trend, repeat issues, and performance against each of WCAG’s 61 Success Criterion. Management can use this data in a number of ways including, ideally, areas where developer skillsets can be improved.

Protecting and ensuring system quality requires that the feedback cycle for accessibility issues be as short as possible. Tenon is unquestionably the only web accessibility testing tool that provides complete end-to-end coverage for accessibility testing and compliance, shortening the accessibility feedback cycle at every step along the way of a system’s lifecycle. Tenon offers the single best solution for Extreme Accessibility. In any case where the length of the feedback cycle grows longer and longer, it is correlated with increases in money & time necessary to find and fix issues. The likelihood that the issues will be fixed is diminished as well. In other words, you can’t afford not to use Tenon.

My company, AFixt offers full accessibility services from testing, training, consulting, and remediation. If you need help, get in touch with me now!