document-review-automation-automation-case-study-rellatech
    Back to Case Studies

    Organizing Document Matches for Human Review

    I built a tool to search a large document collection against a defined checklist and organize potential matches into a review dashboard. The output helped a person examine the evidence and prepare follow-up lists.

    Document ReviewPythonStreamlitCompliance Ops
    Valentina, founder of Rellatech

    Valentina Akpan: Founder of Rellatech, providing administrative and operations support to executives, founders, business owners and teams. Her background combines technical support, customer success, administration and operations.

    ·

    A client came to me with a detailed document request list covering dozens of specific document types from over five years of business activity. These included monthly financial reports, quarterly invoices, account statements, position reports and operational records.

    When I looked at what was on hand, the folder contained well over a thousand files. The client asked me to check those files against the checklist and list what still needed review, and to do it quickly.

    The Challenge

    Done by hand, this means opening the checklist and the folder, going through the files one by one, comparing each against the requirements and keeping track of what is covered and what is still missing. With a thousand files spread across years of subfolders, that process can take days, and it is easy to miss something.

    1,000+ files across 5 years of subfolders

    40 distinct checklist requirements to satisfy

    Generic file names that did not describe the content

    Manual review was estimated at about two days

    The Solution

    I built a lightweight desktop app using Python and Streamlit. You choose a folder, and the app searches every file for the terms linked to each checklist item.

    File names alone are not reliable, so the app reads the content of each file. For each file, it extracts readable text from PDFs, Word documents, Excel workbooks, and plain text, then searches that content for keywords linked to each checklist requirement. Each requirement is mapped to its own set of search terms, and the app works through every file against every requirement.

    The results appear in a dashboard that shows how many items have a candidate match, which items have no match yet, and which files matched each requirement's terms.

    What I Built and Connected

    Folder scanner

    You choose a folder, and it goes through every subfolder and indexes each file.

    Multi-format text extraction

    Reads PDFs, Word documents, Excel workbooks, and plain text.

    Keyword mapping per requirement

    Each checklist item is mapped to a set of search terms, run across every file.

    Review dashboard

    Categories, items with a candidate match and items with none, and which files matched each requirement's search terms.

    Exports for follow-up

    A list of items with no match, a list of candidate matches, and a ZIP of the matched files for the reviewer.

    Streamlit interface

    A lightweight desktop app the client can run themselves, without using a terminal.

    What the Output Looks Like

    Results are grouped by category, in the same order as the original checklist. Each category shows how many items have a candidate match out of the total. Anything without a match expands automatically so the items needing review are visible first.

    Each candidate file is shown with its relative path and a download button. A bulk export produces a list of items with no match, a list of candidate matches, and a ZIP of the matched files for the reviewer.

    Before and After

    Before

    • · An estimated two days clicking through subfolders
    • · Mental tracking of 40 checklist items
    • · Easy to miss a file or misclassify one
    • · No clean handoff list for the client

    After

    • One scan in seconds
    • Candidate matches grouped by category
    • Items with no match listed for review
    • ZIP and follow-up lists exported

    Limits of the Tool

    The results depend on the keywords. Some matches are false positives, because a relevant word appears in a document that is not the one the checklist required. I expected this and planned the review around it.

    The app does not replace human judgement. It takes over the manual work of opening and comparing a thousand files, and a person still reviews the candidate matches and the items with no match. On this project, the manual review was estimated at about two days, and the review of the structured report took about an hour.

    The tool opens and searches the files, and the reviewer makes the decisions.

    A Structured Starting Point for Human Review

    The tool organizes candidate document matches against the checklist and exports lists for follow-up. A keyword match does not establish that a document satisfies a requirement, and an unmatched item does not prove the document is absent. A reviewer checks the candidate files and resolves uncertain results before confirming coverage.

    The result was a more organized review process, with the evidence and the open questions presented together.

    • Candidate matches organized for the checklist items across years of financial records
    • Items with no match listed as the follow-up list for the reviewer
    • Delivered a ZIP of candidate files and a list of items needing review
    • Manual review estimated at two days; the structured review took about an hour

    The scan runs in seconds, so the reviewer can spend their time checking the candidate files and resolving the uncertain results.

    Tools Used

    PythonStreamlitpdfplumberpython-docxopenpyxlCustom keyword schema

    Who This Is For

    This case study is worth reading if you are:

    • ●Compliance, legal or operations teams with document backlogs of hundreds or thousands of files
    • ●Businesses preparing for an audit, due-diligence review, or funding round
    • ●Anyone with a clear checklist of required documents and a disorganized folder of what they have
    • ●Businesses and startups that know which documents they need to find and would rather not spend two days opening PDFs one by one

    Need help with similar work?

    Tell me which responsibilities or processes need attention, who is involved and any upcoming deadlines. I provide administrative and operations support, including the connected technical work within our agreed scope.

    Book a free call

    Share This Article

    Comments

    Join the conversation.

    Loading comments…

    Leave a comment