Evidence Base
Theses and sources of the Legal Copilot Ukraine project
This page consolidates the project's evidence: a list of theses and a full table of sources. All URLs were checked for availability on 10 July 2026. Fast-changing facts (models, EU AI Act deadlines, hallucination statistics) are fixed as of that date.
Source codes (A1, B2, U18 …) refer to the table in the Sources section. Thesis statuses: ✅ confirmed · 🟡 partly confirmed / a working hypothesis to be tested by a pilot.
Theses
T1. Language models have reached professional level on legal tasks ✅
GPT-4 passed the bar exam at the ~90th percentile back in 2023 (A2). Independent blind evaluations in 2025: specialised legal-AI tools scored 74–78% against a 69% attorney baseline on research tasks (A3, A4). The 2026 flagships (GPT-5.5, Claude Opus 4.7, Gemini 3) reach the level of industry experts on well-specified tasks (A6–A9). A mature evaluation methodology exists (A1).
T2. A "bare" model is unreliable for legal work ✅
Public LLMs hallucinate on legal questions in 58–82% of cases (B2). Even commercial legal RAG tools err in 17–33% of queries despite "hallucination-free" marketing (B1). 1,733 court cases over fabricated AI citations across 40 jurisdictions as of 09.07.2026, with a trend of roughly 4.7× growth over 9 months (B3); sanctions range from Mata v. Avianca 2023 (B4) to over $145K in Q1 2026 (B5, B6). The Supreme Court of Ukraine has formed the first restrictive case law (U18).
T3. The problem is not the models but the organisation of data, instructions, and verification ✅
Foundation models give good answers but do not show sources (A5); what distinguishes specialised tools is precisely the quality of citation (A4). RAG improves things but does not solve them (B1, C1, C2). Hence the formula: model + data + task materials + instructions + verification + user.
T4. Agentic environments have become a mature, standardised technology ✅
Codex, Claude Code, Gemini CLI, opencode support project instructions, roles, skills, memory, and tools (D1, D3, D5, D6). AGENTS.md is a cross-vendor standard: 60k+ projects, Linux Foundation (D2). MCP standardises connection to data and tools; the final next-generation specification is due 28.07.2026 (D4). Configuration is done with text files, without programming. The practical consequence: the atomic workstation is already a ready, industrially operated infrastructure; national deployment does not require custom software development and can begin immediately — by publishing instructions, roles, skills, and training courses in a repository.
T5. Centralised state AI projects systematically get stuck in pilots ✅
OECD, 200 cases: most initiatives do not reach scale; justice is among the leading functions for AI use (H1, H2). This supports the bet on a distributed model that centralises only data, standards, and security.
T6. Machine-readable law is existing global practice, not futurology ✅
Standards: Akoma Ntoso/LegalDocML (G1), ELI (G2). Working national implementations: the UK (G3), the US (G4), Germany (G5). For an EU candidate, ELI compatibility is part of the integration agenda.
T7. Verifiable provenance of results is an established engineering discipline ✅
W3C PROV (F1), C2PA (F2), the culture of reproducible research artifacts at ACM (F3). Carrying this over to legal work is adaptation, not invention.
T8. Human-in-the-loop is a regulatory requirement, not an option ✅
EU AI Act: justice is high-risk, Art. 14 requires human oversight (E1); timeline: transparency from 02.08.2026, high-risk Annex III from 02.12.2027 (E2, E3). CEPEJ (E7), UNESCO: AI is assistive, not substitutive (E8), UK judicial guidance (E4), ABA 512 (E5), CCBE (E6). The principle "the agent proposes — the human decides" aligns with the entire regulatory framework.
T9. Ukraine has an outstanding digital base for this model ✅
5th in the world by the Online Service Index, Diia with 20M+ users (U6). Diia.AI — the world's first national AI agent for public services, launched 09.2025, recognised by the EU (U1–U4, H3). Strategic frameworks: WINWIN-2030 (U5), the AI White Paper (U7), the draft AI Strategy 2030 aiming for a top-3 position in public-sector AI (U8).
T10. Ukrainian legal data is already open in machine-readable form ✅
USRCD: 120M+ decisions, open access, datasets updated daily (U10, U11). A REST API of the Verkhovna Rada (U12). 40k+ datasets on data.gov.ua (U9). UJITS as the base of judicial digitalisation (U13). The gap: legislation is not yet marked up in Akoma Ntoso/ELI — and that is precisely the state's task in the proposed model.
T11. A staffing shortage and rising caseload make productivity gains the only option ✅
2,254 judicial vacancies against an establishment of 6,600; 4,346 are working (U14). Caseload is growing ≥8% per year (U15), estimated 5–10× higher than the European level (U16). A similar shortage exists in state legal services.
T12. European integration creates demand, resources, and standards alike ✅
Screening was completed 09.2025, the first cluster opened 15.06.2026 (U19, U20). Ukraine Facility: €50B for reforms, including the digitalisation of justice; over €29.5B disbursed (U21). Technical assistance: EU4DigitalUA (U22), Pravo-Justice (U23). Harmonisation with the acquis is a colossal volume of legal work — an ideal case for copilots.
T13. The professional community and the market are ready ✅
A mature LegalTech scene: Opendatabot, YouControl, LIGA ZAKON, and others (U26–U28, U30). The state has already issued recommendations to lawyers on the safe use of AI (U29). The legal framework for e-identification is harmonised with eIDAS (U24); personal-data protection reform is moving towards GDPR (U25).
T14. Shifting the unit of exchange from the document to the structured package qualitatively raises team effectiveness 🟡
Logically derived from T2, T3, T7: text production has accelerated while verification has remained manual; structured artifacts with a provenance chain make it possible to automate verification. There is as yet no direct field research on legal teams — this thesis is to be tested in pilot No. 11 (a working group on full artifact exchange) with metrics: source coverage, unsupported claim rate, handoff completeness, time to review, reproducibility score.
T15. Deployment timeline (0–3–6–12 months) 🟡
A working hypothesis, stated as such in the article. Rationale for its realism: configurations are files, not software development; the data is already open; a comparable speed is the launch of Diia.AI in about a year from strategy to service (U1–U4). The timeline is detailed in the pilot launch plan.
Summary
Of 15 theses: 13 are confirmed by external sources; 2 (T14, T15) are honestly marked as working hypotheses to be tested by the pilots. This meets the requirement of §26 of the brief: no technological messianism.
Sources
Package author: Sergej Avdejcik · Legal Copilot Ukraine / VeriLex project. All URLs were checked for availability on 10 July 2026 (web research). Fast-changing facts (models, EU AI Act deadlines, hallucination statistics) are fixed as of that date.
Columns are kept from the original (# · Source · Year · URL · What it confirms). Proper names of publications, standards, and institutions, and English-language titles, are given as published; titles of Ukrainian- and Russian-language sources are given here in English translation.
A. LLM capabilities on legal tasks
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| A1 | LegalBench (Stanford et al.) | 2023 | arxiv · stanford | The core benchmark for legal reasoning: 162 tasks, 6 types of legal reasoning, built by lawyers |
| A2 | GPT-4 Passes the Bar Exam (Katz et al., Phil. Trans. R. Soc.) | 2024 | royalsociety | GPT-4 passed the Uniform Bar Exam at the ~90th percentile — the reference point |
| A3 | Vals Legal AI Report (VLAIR) | 02.2025 | vals.ai | The first independent blind evaluation of legal-AI products against an attorney baseline |
| A4 | VLAIR — Legal Research | 10.2025 | vals.ai | AI products 74–78% vs lawyers 69%; specialised tools are distinguished by citation quality |
| A5 | BigLaw Bench (Harvey) | 2024 | harvey.ai | Splitting answer score / source score: foundation models give answers but do not show sources |
| A6 | GDPval (OpenAI) | 2025 | openai | GPT-5.x-generation models reach industry-expert level on well-specified tasks across 44 occupations |
| A7 | Introducing GPT-5.5 (OpenAI) | 04.2026 | openai | OpenAI's current flagship: GDPval 84.9%, strong results in legal |
| A8 | Introducing Claude Opus 4.7 (Anthropic) | 04.2026 | anthropic | Anthropic's current flagship: long autonomous tasks, file memory, 90.9% BigLaw Bench (Harvey) |
| A9 | Gemini 3 (Google) | 11.2025 | blog.google | Google's current flagship family, agentic and multimodal capabilities |
B. Hallucinations and risks
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| B1 | Hallucination-Free? (Stanford RegLab/HAI, JELS) | 2024–2025 | reglab.stanford.edu | Commercial legal RAG tools hallucinate in 17–33% of queries; RAG alone does not solve the problem |
| B2 | Large Legal Fictions (Dahl et al., J. Legal Analysis) | 2024 | academic.oup.com | Public LLMs hallucinate on legal questions in 58–82% of cases; a typology of legal hallucinations |
| B3 | AI Hallucination Cases Database (D. Charlotin) | 2025–2026 | damiencharlotin.com | 1,733 court cases over fabricated AI content in 40 jurisdictions (as of 09.07.2026; ~370 in October 2025) |
| B4 | Mata v. Avianca, S.D.N.Y. | 2023 | law.justia.com | The first high-profile sanctions case ($5,000) for fabricated ChatGPT precedents |
| B5 | Fake Cases, Real Sanctions (ABA Litigation News) | 2026 | americanbar.org | A new wave of sanctions in 2026, including Whiting v. City of Athens ($15,000 on the attorney) |
| B6 | Penalties stack up as AI spreads (NPR) | 04.2026 | npr.org | Over $145K in sanctions in Q1 2026 alone in US courts |
C. RAG and grounding
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| C1 | Retrieval-Augmented Generation (Lewis et al., NeurIPS) | 2020 | arxiv | The foundational paper: connecting a model to an external document index |
| C2 | RAG for LLMs: A Survey (Gao et al.) | 2023–2024 | arxiv | The standard survey: RAG as a response to hallucinations, knowledge staleness, and opacity |
D. Agentic environments
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| D1 | OpenAI Codex — documentation | 2025–2026 | developers.openai.com | An agentic environment: Memories, Skills, Subagents, Rules, Hooks, MCP, sandboxing |
| D2 | AGENTS.md | 2025 | agents.md | An open format for project instructions; 60k+ projects; transferred to the Linux Foundation (12.2025) |
| D3 | Claude Code — Extend Claude Code | 2025–2026 | code.claude.com | CLAUDE.md (memory), skills, subagents, hooks, MCP — a model of a configurable working environment |
| D4 | Model Context Protocol — specification | 2024–2026 | modelcontextprotocol.io | An open protocol connecting models to data and tools; the final next-generation specification is due 28.07.2026 |
| D5 | Gemini CLI (Google) | 2025–2026 | github.com | An open (Apache 2.0) terminal agent: MCP, context files, grounding |
| D6 | opencode | 2025–2026 | opencode.ai | An independent open-source agent (160k+ stars): bring-your-own-provider, AGENTS.md |
E. Human-in-the-loop and responsibility
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| E1 | Regulation (EU) 2024/1689 (EU AI Act) | 2024 | eur-lex | Justice is high-risk (Annex III(8)); Art. 14 requires human oversight |
| E2 | EU AI Act Update: Digital Omnibus (Covington) | 05.2026 | insideglobaltech.com | The high-risk Annex III obligations moved to 02.12.2027 (approved by Parliament 16.06 and Council 29.06.2026) |
| E3 | AI Act Implementation Timeline (FLI) | 2026 | artificialintelligenceact.eu | The official deadline tracker (as of 10.07.2026 it did not yet reflect the Omnibus — cite with a caveat) |
| E4 | AI Guidance for Judicial Office Holders (UK Judiciary) | 10.2025 | judiciary.uk | Current UK judicial guidance: hallucinations, confidentiality, the judge's responsibility |
| E5 | ABA Formal Opinion 512 | 2024 | americanbar.org (PDF) | Ethical requirements for generative AI: competence, verification, confidentiality |
| E6 | CCBE Guide on generative AI + Technical guide | 2025–2026 | guide · technical guide | The position of the European bars: independent verification of AI outputs |
| E7 | CEPEJ European Ethical Charter on AI in Judicial Systems | 2018 | coe.int | The Council of Europe's 5 principles — a base framework for Ukraine as a CoE member |
| E8 | UNESCO Guidelines for AI in courts and tribunals | 12.2025 | unesco.org | The first global framework: AI is assistive, not substitutive, under meaningful human supervision |
F. Provenance, audit trails, reproducibility
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| F1 | PROV-O: The PROV Ontology (W3C) | 2013 | w3.org | The W3C standard for describing data provenance — a model for an artifact audit trail |
| F2 | C2PA Specifications | 2021–2025 | spec.c2pa.org | Cryptographically signed manifests of content provenance |
| F3 | ACM Artifact Review and Badging | 2020 | acm.org | Formalising reproducibility and the culture of research artifacts |
G. Machine-readable law
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| G1 | Akoma Ntoso v1.0 (OASIS LegalDocML) | 2018 | oasis-open.org | An international XML standard for machine-readable legal documents |
| G2 | European Legislation Identifier (ELI) | since 2012 | eur-lex | Stable HTTP URIs + metadata for EU and member-state legislation; the target standard for an EU candidate |
| G3 | legislation.gov.uk Developer/Formats | since 2010 | legislation.gov.uk | The entire UK statutory corpus as XML/RDF/Akoma Ntoso via an open API |
| G4 | US Code Download (USLM XML) | since 2013 | uscode.house.gov | The US Code in machine-readable USLM XML |
| G5 | LegalDocML.de (E-Gesetzgebung, Germany) | 2020 | egesetzgebung.bund.de | A national LegalDocML profile: Germany's entire legislative cycle |
H. AI in the public sector
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| H1 | Governing with AI (OECD) | 09.2025 | oecd.org | 200 cases of AI in government functions; justice among the leaders; most initiatives get stuck in pilots |
| H2 | Digital Government Outlook 2026 (OECD), AI chapter | 06.2026 | oecd.org | The current state of AI governance in governments |
| H3 | Diia.AI (European Commission, Public Sector Tech Watch) | 2025–2026 | interoperable-europe.ec.europa.eu | EU recognition: the world's first national AI agent delivering services; registry data does not leave the secure perimeter |
U. Ukraine: digitalisation, data, courts, European integration
| # | Source | Year | URL | What it confirms |
|---|---|---|---|---|
| U1 | Ministry of Digital Transformation: Ukraine becomes the first in the world to deliver public services via AI | 2025 | thedigital.gov.ua | The official announcement of Diia.AI (September 2025); the first service — an income certificate |
| U2 | Cabinet of Ministers: Diia.AI — a world record | 2025 | kmu.gov.ua | Government confirmation of the first-place claim (WINWIN Summit) |
| U3 | Suspilne: Diia.AI launch | 09.2025 | suspilne.media | Independent confirmation of the launch and functionality |
| U4 | Texty: a year of state AI | 2026 | texty.org.ua | The year in review: 200K consultations, 6K+ automatic certificates |
| U5 | WINWIN — strategy to 2030 (Ministry of Digital Transformation) | 2024 | thedigital.gov.ua | The national innovation strategy; portal: winwin.gov.ua |
| U6 | UN E-Government Survey 2024 | 2024 | publicadministration.un.org | Ukraine — 5th in the world by the Online Service Index; Diia 20M+ users |
| U7 | AI Regulation White Paper (Ministry of Digital Transformation) | 2024 | thedigital.gov.ua | A bottom-up approach, oriented to the EU AI Act; materials: ai.thedigital.gov.ua |
| U8 | Draft AI Development Strategy to 2030 | 2025 | digitalstate.gov.ua | Goal: a top-3 country for AI integration in the public sector; homegrown models |
| U9 | data.gov.ua — portal restored | 2023 | thedigital.gov.ua | The national open-data portal, 40k+ datasets |
| U10 | USRCD: 120M court decisions | 2024–2025 | reyestr.court.gov.ua · ics.gov.ua | One of Europe's largest open corpora of case law (as of the source's publication date) |
| U11 | USRCD dataset on data.gov.ua | 2025 | data.gov.ua | Court decisions as machine-readable datasets updated daily |
| U12 | Verkhovna Rada Open Data Portal | — | data.rada.gov.ua | A REST API of legislation, draft laws, and votes |
| U13 | UJITS (State Judicial Administration / High Council of Justice) | 2021–2025 | dsa.court.gov.ua · hcj.gov.ua | Electronic cabinet, Electronic Court, videoconferencing — the base of judicial digitalisation |
| U14 | Staffing crisis in the courts (sud.ua) | 2025 | sud.ua | 4,346 working judges against an establishment of 6,600; 2,254 vacancies (HCJ data) |
| U15 | Court caseload 2026 (sud.ua) | 2026 | sud.ua | Caseload growth ≥8% year on year across all jurisdictions |
| U16 | Caseload 5–10× higher than European (Zmina) | 2025 | zmina.info | An estimate from the judicial corps |
| U17 | Supreme Court on AI in justice | 2025–2026 | supreme.court.gov.ua | The Supreme Court discusses AI use; a regulation on AI use by the Court's staff; Responsible AI principles |
| U18 | Commercial Cassation Court of the Supreme Court, ruling of 08.07.2025 (sud.ua) | 2025 | sud.ua | The Supreme Court's first case law on the limits of AI use by parties |
| U19 | Council of the EU: first negotiating cluster opened | 06.2026 | consilium.europa.eu | Screening completed 09.2025; the "Fundamentals" cluster opened 15.06.2026 |
| U20 | Euronews: Cluster 6, external relations | 07.2026 | euronews.com | Continuation of the negotiations (opening was expected 14.07.2026) |
| U21 | Ukraine Facility (Council of the EU; plan portal) | 2024–2027 | consilium.europa.eu · ukrainefacility.me.gov.ua | €50B of "reforms in exchange for financing", including the digitalisation of justice; over €29.5B disbursed |
| U22 | EU4DigitalUA (EEAS) | — | eeas.europa.eu | €20.5M: 50+ e-services, 10 registries, the "Trembita" data bus |
| U23 | EU Project Pravo-Justice | since 2017 | pravojustice.eu | An EU programme supporting justice reforms (phase III) |
| U24 | Law No. 2155-VIII on electronic identification | rev. 2022 | zakon.rada.gov.ua | The framework for QES/e-identification, harmonised with eIDAS |
| U25 | Draft Law No. 8153 on personal data protection | 2022–2025 | itd.rada.gov.ua | Bringing the personal-data regime in line with GDPR/Convention 108+; passed the first reading 11.2024 |
| U26 | Ukrainian Legal Tech (Mezha.Media) | 2024–2025 | mezha.media | A market overview: Liga, YouControl, Opendatabot, ZakonOnline, Vkursi, and others |
| U27 | Opendatabot | since 2016 | opendatabot.ua | Commercial products on top of state open data |
| U28 | YouControl | since 2014 | youcontrol.com.ua | Analytics over 220+ open-data sources |
| U29 | Recommendations for lawyers on the safe use of AI (Ministry of Digital Transformation + Ministry of Justice + State Judicial Administration + UNBA) | 07.2025 | thedigital.gov.ua | The first official state recommendations to lawyers on AI |
| U30 | Legal Tech 2026: AI ethics (Jurliga) | 2026 | jurliga.ligazakon.net | A professional discussion on AI ethics in legal practice |
Accuracy notes
- The figures "120M decisions" and "4,346 judges / 2,254 vacancies" should be cited with the wording "as of the source's publication date".
- EU AI Act: do not write "high-risk already applies" — the Annex III obligations were moved to 02.12.2027 (Digital Omnibus, finally approved by the Council 29.06.2026); transparency obligations apply from 02.08.2026.
- Model names and tool versions are current as of July 2026; re-check before the landing page goes live.
- The author's publications on Medium: as of 10.07.2026 none were found — cite only after actual publication.