AI for Monitoring Flows
To monitor flows, you’re going to need an army. But not the scary kind.
The problem
There are many tools out there that monitor websites, our Site Scanner included. But there's something that those tools will never be able to tell you, and it's a not a small thing. Those tools will never be able to answer the question, "How many blind people successfully completed the purchase flow on my website today?"
There are many reasons for this. To start, it's an understandable privacy violation for, say, a blind user to disclose to a website they're on that they are in fact blind. But even if your Google Analytics was full of information about assistive technology-based user agents, the data would be foggy. You might know on which page a screen reader user dropped out of a flow, but you wouldn't know why, or where on the page they ran into a problem. And even if you asked your QA team to write automated test scripts for each of your flows, those scripts would constantly break, as your website changes frequently.
So what do companies who care about accessibility the most wind up doing? They hire an army of sorts -- a large team of people to go through their key flows every day, using the assistive technologies that they worry about the most. That works for those companies, but what about the ones who can't hire a large team? Or who have thousands of sites and flows to worry about? You would need an army indeed to manually check thousands of flows a day.
Actors, agents & AI
This is where Evinced comes in. We're working on a new technology that will use Agentic AI -- that is to say, artificially intelligent agents -- to emulate the various kinds of assistive technology users that a company is concerned about. We call these "actor" agents. For example, we could build an actor agent that will traverse your website and act exactly like a screen reader user would. Call that a "blind agent", or, more accurately, a "screen reader agent." And we could report on where on your site, or on which of your thousands of sites, that this actor agent got stopped in its tracks because of accessibility problems. Reliably, and automatically.
Already working
Does this sound far-fetched? We’ve actually already demonstrated that this works (you can even work with us to trial the technology). But because this agent is software, and because it’s Evinced behind it, any of these actor agents will be able to offer impressive things:
No code to set it up
You could simply instruct the agent in plain English to do whatever it is you want. For example, "go to this website, search for any blue men's shirt, and place 1 shirt into the shopping cart for checkout. But do not check out." Or point it to an existing user story.
Integrate into other testing
If you have a suite of functional tests already, you could simply invoke these agents at the end of the test run. Then you'd get the functional test results along with a readout on where in the functional test other kinds of users would have encountered significant problems.
Open Jira tickets automatically
When the agent encounters a problem, it will be logged automatically in a report. But the agent could certainly open a Jira or Azure ticket, too, and include all the information it already gathered -- page, agent type, flow, precise point of the breakdown, user story, etc. And because it's Evinced, the ticket could include screenshots and fix recommendations, too.
Upgrade your reporting
Inside Evinced, you'll have a whole new way to look at your website. You'll be able to track critical errors and a health score by property as now, but you'll also see which flows and sites had posed problems for assistive tech users. You could even customize your health score, so it would drop sharply when an assistive technology user can't complete a flow.
A game that needs changing
Companies that have the time and money to rely exclusively on manual testing for flows inside their development cycle will benefit from all the ways that humans can respond to new and unforeseen issues. But for those without those resources, and even a few with them who want to cover some extra bases, we think this technology will be an absolute game-changer.
While it’s true that this product is still in our labs, we demonstrated it to customers successfully this year and you can work with us to trial it with your team. Drop us a line.