Tracking Web Accessibility Success
I’ve previously talked about Prioritization and even given presentations on the topic, which I quite enjoy. Prioritization is a topic which I enjoy thinking about because I like to think about ways to get the most benefit from accessibility efforts in the shortest possible time. For me, the end goal is always a more accessible system, but to the business owners, product owners, project managers, and other stakeholders, they have other goals. With this in mind, I like that proper Prioritization lends itself directly to the ability to track successes in our accessibility efforts.
Managing the Remediation Process
To summarize my prior points about proper Prioritization, always remember the following: accessibility issues are bugs and not all bugs are created equally. Some bugs are worse than others. Some bugs impact more users than others. Some bugs impact users severely and some are mere annoyances. So always consider these factors when determining priority:
- User Impact
- Ease and Speed of Repair
- Impact on the Interface
- Volume (how many times a bug occurs)
- Location of the Bug
At a high level the items that have a high impact on users with a lot impact on the UI that are easiest and quickest to repair should be the first to get fixed. But how does this lead to tracking success?
In chemistry, dilution is the process of reducing the concentration of a solute in solution, usually simply by mixing with more solvent. In our case, the solute are bugs. The solvent is the code itself. During our prioritized remediation process, we want to add more “solvent”, which is clean code, to decrease the concentration of bad code.
Concentration = numBugs/linesOfCode (or KB of code, if you prefer)
Continuing our dilution analogy, in the image above we see three beakers indicating different states of concentration. Given the three examples, we see our start point, mid-point, and end point represented by beakers of solution. In our starting point we see a beaker that has a high concentration of problems – especially high priority problems. In our mid-point, we see that the vast majority of high priority problems have been eliminated, leaving us to address our medium priority problems. Finally, our end point, we see almost all of our problems are low priority. We can choose to fix them if we’d like – and I believe we should, because who doesn’t like unicorns?
OK, maybe we won’t see any unicorns, but there’s nothing wrong with having goals. And that leads me to the conclusion, which is how this actually allows us to track success. Your performance can be measured relatively easily like so:
(concentrationStart - concentrationEnd) / time
Your success is measured by the improvement in accessibility over time. Very simply, we can measure our starting concentration minus our ending concentration (in other words, our improvement) divided by time. For the mathematicians among you, it is worth noting that this is only 1st order accurate, but that’s fine for snapshots. As your developers get better and as your system improves, each snapshot should be better than the previous, giving a direct measure of our success.