We believe that Large Language Models (LLMs) specifically unfold their strengths when facing complex problems or questions where the answer can only be revealed through the analysis of a large number of text documents. Whether the answer is hidden at one specific part of the text or scattered across multiple passages or documents, our AI can significantly reduce the time and effort spent to find answers.

Introduction

Imagine you're a company grappling with Environmental, Social, and Governance (ESG) compliance or perhaps auditing. Such tasks can be incredibly complex, requiring you to sift through hundreds, if not thousands, of documents. That's where our Policy Evaluation Assistance System comes in, acting as your personal assistant that automatically evaluates ESG compliance criteria on a large number of documents providing you with precise citations and a detailed argumentation for each of its decisions.

What Is Policy Evaluation?

Policy Evaluation refers to the automated process of applying a set of questions or criteria to a large batch of documents. These could be contracts, research papers, mails or financial reports—virtually any type of text document that you want to understand better.
Policies, in this context, can be broken down into individual rules which are sub-criteria that can be answered by “yes” and “no” and assess more granular dimensions of the grand problem one wants to asses.  The goal is to automatically evaluate these sub-criteria or “rules” using Language Learning Models.

‍

Transparent Reasoning

However, we do not blindly trust the decisions the AI is taking. To ensure high levels of reliability and to lay a foundation of trust, our AI obeys to these three essential principles:

• Transparency: Every evaluation must be traceable to a trustable source of truth, ensuring the credibility of each assessment.
• Traceability: All evaluations must be reproducible, logical and independently verifiable.
• Recency: All evaluations must be based on recent data performed at an adequate frequency, keeping our assessments current and relevant.
  

We embed these principles in the design of our AI-powered assistance system. Each evaluation comes hereby with (1) an explicit citations of the information the AI decision is based upon, (2) a detailed argumentation why and how it came to the conclusion and (3) if applicable, a continuous reevaluation of the policies once the source data is updated.

Use Case: GRI Compliance of Business Reports

A very effective application of our Policy Evaluation Assistance System is the automated auditing of business reports for compliance with the Global Reporting Initiative (GRI) standards. This is an excellent setup for the AI, as GRI already provides a set of predefined reporting principles (as, for instance, in the GRI emissions module) that serve as our policies. By running the GRI policies against a business report, we obtain in-depth insights into its compliance status and use our Policy Evaluation user interface (UI) to provide auditors with additional AI-powered tools. For each policy evaluated, the UI delivers an evaluation status (compliant, not compliant, not applicable and unsure), a link to navigate to the source text as well as the AI's comprehensive rationale behind the evaluation.

This functionality significantly lightens the load for auditors and shifts their role from tediously sifting through long text documents to efficiently double-checking the evaluations with AI enhanced tools. The system also enables the auditor to focus on cases where a more thorough, human assessment is required as the system also identifies ambiguous evaluations.

‍

More Use Cases

Audits are one use case among many. Our Policy Evaluation system has boundless applications that reach across various industries:

Contract Management for Financial Firms: We automate the tedious process of poring over thousands of contracts to identify non-compliance or risk factors. Whether for a bank, hedge fund, or private equity firm, our system can efficiently flag terms or clauses that need attention.

Real Estate Compliance: The complex web of local, state, and federal laws can be overwhelming and intransparent for any real estate firm. Our system can automatically evaluate lease agreements, property contracts, and compliance documents to spot legal and regulatory risks and issues.

Quality Assurance in Manufacturing: Manufacturers can utilize the system for automatic evaluation of quality compliance documents, supplier contracts, certificate of analysis or safety protocols.

Due Dilligence in M&A: Due diligence involves the analysis of massive amounts of documents regarding many dimensions (legal, governance, regulatory, financial and more). The combination of a high-dimensional problem with very rich data sources including mails, legal documents or business reports, make it a perfect match for our assistance system.

Conclusion

Our Policy Evaluation Assistance system, powered by Large Language Models, offers a scalable and reliable solution for analysing complex questions. By complying to principles of Transparency, Traceability, and Recency, the system provides evaluations that can be trusted and verified. This level of automation not only increases efficiency but also allows professionals to focus on nuanced aspects of decision-making that require human expertise.

If you want to see our assistance system in action or find out how this system applies to your use case, feel free to book a meeting with us!

‍

‍