I have talked with a lot of prospective and current clients regarding automation projects involving data “trapped” in documents. Often the question regarding “how accurate is your software?” comes up and for good reason. Most technologies that involve automation tout accuracy levels that are often in the 80 percent-99 percent range.

How Accurate Is Your Solution?

But when we deal with complex data on documents, identifying an accuracy range is difficult without really seeing a sizable amount of real production data. So does that mean that we should deemphasize accuracy in favor of other metrics? If I were to take the messages of some vendors, I would think the answer is unequivocally “yes”. You should talk more about process efficiency or ability to have a controllable process. However, in important and practical terms, the answer to whether we should deemphasize accuracy in favor of other metrics has to be “it depends”.

Meeting SLAs: Throughput

Some vendors will state that focusing on accuracy to the detriment of other quality metrics is a bad idea. Real costs and cost reductions associated with a process are not able to be quantified by understanding system accuracy alone. You also have to understand the accuracy without automation. Another key factor is throughput. Arguably if a manual process can achieve a document-per-hour throughput of 20 and introducing automation will reduce throughput to 10 documents per hour, costs could rise. They may not be staffing costs, but reduced throughput could result in SLA penalties. So understanding your current throughput and the throughput of the proposed system is important.

Distraction from the Ultimate Objective: Cost Reduction

Let’s not distract ourselves from the ultimate objectives most often associated with automation. Cost reduction. Without understanding the accuracy of a system, it is impossible to project real cost savings. For instance, if my organization currently staffs employees to perform data entry on explanation of benefits forms and my annual costs associated with that process are  $500,000, then the accuracy of a given automation system will determine what labor I can completely mitigate vs. what still must go through an exception-handling process. If the accuracy of a system is 30 percent, then cost reduction (setting aside throughput costs) can never go above 30 percent (and most likely lower given other factors). But if that same system can provide 75 percent accuracy, then that translates to a much more significant amount of cost reduction.

Automation: ROI and Accuracy

What all of this means is that a fixation with 99 percent accuracy from any system is not required to completely justify an automation project and most often much less accuracy will suffice to achieve a 100 percent return within a year. This is a fact that is observed with practically every project we have worked on at Parascript.

So yes, accuracy of a system is absolutely key, but very high accuracy is not.  Just don’t let a solution provider switch things around in an attempt to move away from the “accuracy discussion” and treat it as a secondary consideration or, worse, ignore it altogether.

If you found this article interesting, you might also find this useful: How Truth Data Improves Recognition or this white paper, Best Practices: Improve ICR Accuracy by Applying Context