How Shepherd automated loss run extraction to increase underwriter productivity by 10x

‍

The Challenge

Shepherd handles the underwriting of policies and premiums for construction companies internally, with loss run PDF documents being a crucial part of this process. Loss runs, documents that record every claim a company has made with their insurance provider, are often extensive Excel or PDF files. These files can contain hundreds of pages and feature unique formatting from different insurance carriers. Without a solution to automate data extraction from these documents, Shepherd’s team of five underwriters was left to manually extract data – a process that could take hours depending on the length of the loss run.

Traditional insurance providers often increase headcount or outsource data extraction to reduce this burden on their underwriters. Instead of viewing the loss run extraction bottleneck as a manpower problem, Shepherd set out to find a technology-driven solution.

Shepherd’s ideal solution was a scalable, accurate, and fast automated process for extracting claims data from loss runs. To build this, Shepherd set out to find a document extraction solution they could easily integrate into their platform. After shortlisting a dozen providers, they began their evaluation process, guided by the following criteria:

Accuracy

Delivering a reliable extraction from every document topped Shepherd’s list. This would allow them to eliminate the need for any manual auditing, reducing any extra burden that would fall to their underwriters.

Speed

Shepherd’s underwriting team needed extracted loss run data to be available in real-time to deliver on the promise of 24 hour indication.

Maintainability

With a lean engineering team, Shepherd wanted to focus on building out new products, not maintaining their extraction solution. Shepherd’s ideal document extraction solution would also be able to seamlessly scale as their document volume increased without extra engineering bandwidth.

Extensibility

Loss run variation amongst insurance carriers necessitated a dynamic solution that would work across all formats Shepherd encountered.

After initial experimentation with several AI-led approaches that appeared to meet their scalability and accuracy requirements, Shepherd was quickly discouraged by their non-deterministic output, as well as their lag time. “We’d test them in the morning, and it would work perfectly,” lead engineer Danil Kolesnikov explained, “but then [when we tried it again] in the afternoon, it would completely break.”

Most solutions try to be a silver bullet and do-it-all solution for document parsing, and it works for easy documents, but they fail very quickly when you throw something complex, like loss runs, at them.

Broadening their scope beyond AI solutions, Shepherd found that most document extraction vendors would force them into a trade-off – high accuracy with slow response times, or fast response times with low accuracy. Sensible stood out to Shepherd as being capable of delivering on both without compromise, with the customization and configurability options to support extracting complex documents like loss runs.

The Solution

Shepherd developed a robust, scalable, and deterministic underwriting solution to automate loss run processing using Sensible’s document extraction API. Shepherd’s engineering team worked closely with Sensible to learn SenseML, Sensible’s parsing language, so they could write configurations for the most common loss run formats. Sensible’s section method, which captures repeating items in a document based on user-defined bounding boxes, proved particularly useful for parsing individual claims from loss runs.

We were amazed by the tech support that we received from Sensible. They were there every minute, every second, of the implementation stage. [Sensible] has helped us a lot when it comes to training and learning SenseML.

“While it does require an upfront effort to learn,” Kolesnikov shared, “it is worth it if you care about accuracy and a quick API response.”

Shepherd’s underwriting tool uses these configurations to automatically extract loss run data from nine different insurance carriers. The underwriting team just needs to upload loss run files to Shepherd’s platform, which automatically extracts and uploads claims data – no manual data entry required.

Sensible also satisfied Shepherd’s maintainability requirement, allowing their engineering team to stay focused on developing their core product while Sensible covers upkeep and development of new extraction methods.

The Results

If you truly care about accuracy and speed, and as an insurance company we care about getting the right numbers, then you should use Sensible.

After going live in mid-2023, Shepherd achieved 100% extraction accuracy across their supported loss run formats. This allowed their specialized construction underwriting team to redirect their time and expertise to commercial policy underwriting. Lengthy loss runs spanning 300-plus pages used to take hours for an underwriter to extract by hand, but with Sensible, that processing time is cut to less than a minute.

Productivity on the underwriting team increased by more than 270x on these lengthy loss runs, even as Shepherd continues to scale operations. As they look to double their business in 2024, document extraction automation plays a key role in making that growth possible without expanding headcount.

We had a loss run from Travelers last year that was 351 pages. It took hours to manually parse – it could’ve even taken a day, and that’s not a good day. With Sensible, I recorded a 31 second response time for that same document.

Shepherd and Sensible are working together to continue exploring LLM applications for processing unknown or novel loss run formats, while preserving the high level of accuracy Shepherd requires.

More and more what we have come to appreciate from Sensible is the blend of accuracy and speed. With some other solutions you might trade off one or the other, but with Sensible we were able to achieve a very quick turnaround from the API endpoint while preserving very, very high accuracy.

Read more about Shepherd's decision to automate claims processing with Sensible on their blog.

Advice to other startups

Value the quality of customer service. It’s something that doesn’t come up immediately in the vendor research, but should definitely be a category that startups consider when selecting a document extraction vendor.

What’s next

We have a very high confidence that whatever data Sensible gives back to us, it will work with a high level of certainty.

Shepherd uses Sensible in their new AI Compliance product, which automates certificate and endorsement tracking for their policyholders. The tool uses Sensible to extract key data from ACORD 25, a Certificate of Liability, which is uploaded to the Shepherd platform for tracking. With Sensible’s extraction capabilities already in place, Shepherd can continue to launch innovative insurance products on their timetable, with confidence the extracted data will be accurate and available immediately.