Home / About Us / Volume I Contents / Ch 20: Cognitive Agent

Statistical Basis of AI/ML

Volume I Grand Finale: A Cognitive Agent from First Principles

Chapter 20 -- ReAct Loops, Belief Graphs, and a Working Medical Reasoning Engine

Every chapter in this book built one layer of the stack: probability, linear models, classification, neural networks, attention, Transformers, and graph networks. This chapter assembles those layers into a single working system -- a cognitive agent that retrieves medical knowledge, reasons over it, checks its own consistency, and knows when to say "I do not know."

No agent framework is used. The ReAct loop, tool registry, belief graph, and reflection gate are all built from first principles using only the libraries already installed in this environment.

Why This Chapter?

The gap between RAG and reasoning

A Retrieval-Augmented Generation (RAG) system does one thing:

Query -> embed -> retrieve -> augment prompt -> LLM -> response

It has no memory between calls, no planning, and no way to detect when its retrieved evidence contradicts itself. For low-stakes queries that is acceptable. For clinical decision support it is not.

A cognitive agent adds four capabilities on top of RAG:

Capability What it adds
Planning Breaks a complex question into sub-goals before answering
Working memory Tracks retrieved claims and their confidence across reasoning steps
Reflection Scores its own draft answer before releasing it
Calibrated refusal Declines to answer when evidence is insufficient or contradictory

How every prior chapter contributes

Chapter Contribution to this agent
Ch06 -- Regression Confidence intervals applied to retrieval score distributions
Ch07 -- Classification Claim classifier: supported / uncertain / contradicted
Ch08 -- Clustering Deduplication of semantically redundant retrieved chunks
Ch13 -- Information Theory Entropy of the belief distribution as the refusal threshold
Ch14 -- Neural Networks Forward pass that powers the embedding and scoring modules
Ch16 -- Attention Multi-head attention used in the local reasoning model
Ch17 -- BERT Encoder backbone for claim similarity scoring
Ch18 -- ViT Patch attention as the architectural template for belief graph attention
Ch19 -- GNN GCN-based consistency propagation over the live belief graph

Architecture overview

User query
    |
    v
[Planner]  ->  sub-goal list
    |
    +--[Tool: retrieve]       -> chunks from knowledge store (cosine sim)
    |
    +--[Tool: score_claims]   -> (claim, confidence, source) tuples
    |
    +--[Tool: check_entropy]  -> belief entropy; triggers refusal if high
    |
    +--[Tool: gcn_consistency] -> GCN pass over belief graph; flags conflicts
    |
    +--[Synthesiser]          -> draft answer
    |
    +--[Reflector]            -> critiques draft; may trigger another loop
    |
    v
Final answer  (with confidence and evidence trail)
Section Topic
Sec.20.0 Setup and simulated knowledge store
Sec.20.1 The ReAct loop: Reason, Act, Observe, Reflect
Sec.20.2 Tool registry and the retrieve tool
Sec.20.3 Belief state: a live graph of claims and confidence scores
Sec.20.4 GCN-based consistency checking (Ch19 applied)
Sec.20.5 Entropy-based calibration and refusal (Ch13 applied)
Sec.20.6 Reflection gate and the full agent loop
Sec.20.7 Running the agent on oncology queries
Sec.20.8 What Volume II builds next

Sec.20.0 Setup

This chapter uses only the packages already installed in the Vol I environment. No new downloads are required.

Module Role
torch / torch.nn GCNLayer for belief graph consistency (Ch19 code reused)
numpy Cosine similarity, entropy computation, adjacency matrix
sklearn TF-IDF vectoriser for the simulated knowledge store
matplotlib / seaborn Belief graph visualisation, confidence plots
textwrap Clean output formatting for agent traces

Simulated knowledge store: in production this would be Azure AI Search with dense vector embeddings. Here we build a lightweight TF-IDF retrieval engine over a curated set of 30 oncology statements. The agent architecture is identical regardless of what backs the retrieval tool.

In [1]:
# -- Imports and simulated oncology knowledge store --------------------------
import warnings
warnings.filterwarnings("ignore")

import textwrap
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import seaborn as sns
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.cluster import KMeans

sns.set_theme(style="whitegrid", font_scale=1.05)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

print(f"PyTorch : {torch.__version__}")
print(f"Device  : {device}")
if device.type == "cuda":
    print(f"GPU     : {torch.cuda.get_device_name(0)}")

# ---------------------------------------------------------------------------
# Simulated knowledge store - 30 oncology claims with source metadata
# Each entry: (claim_text, source_id, year)
# In production: replace this list with Azure AI Search retrieve() calls
# ---------------------------------------------------------------------------
KNOWLEDGE_STORE = [
    # EGFR / NSCLC
    ("First-line osimertinib significantly improves overall survival in EGFR-mutant NSCLC versus gefitinib or erlotinib.",
     "FLAURA", 2018),
    ("Osimertinib is standard of care for EGFR exon 19 deletion and L858R NSCLC.",
     "NCCN_NSCLC_2024", 2024),
    ("T790M resistance mutation accounts for approximately 60% of acquired resistance to first-generation EGFR TKIs.",
     "Sequist_JCO_2011", 2011),
    ("Osimertinib retains activity against EGFR T790M acquired resistance.",
     "AURA3", 2017),
    ("EGFR exon 20 insertions predict primary resistance to standard EGFR TKIs.",
     "Riely_ASCO_2006", 2006),
    ("Amivantamab plus chemotherapy is approved for EGFR exon 20 insertion NSCLC.",
     "PAPILLON", 2024),
    # PD-1 / immunotherapy
    ("Pembrolizumab monotherapy is first-line standard for NSCLC with PD-L1 TPS >= 50%.",
     "KEYNOTE-024", 2016),
    ("Atezolizumab plus bevacizumab is a first-line option in hepatocellular carcinoma.",
     "IMbrave150", 2020),
    ("Nivolumab improves overall survival versus chemotherapy in previously treated NSCLC.",
     "CheckMate-017", 2015),
    ("Combined PD-1 and CTLA-4 blockade increases response rate but also immune toxicity.",
     "Larkin_NEJM_2015", 2015),
    ("High TMB predicts benefit from pembrolizumab across tumor types.",
     "KEYNOTE-158", 2019),
    # Breast cancer
    ("Trastuzumab plus pertuzumab plus docetaxel is first-line standard for HER2-positive metastatic breast cancer.",
     "CLEOPATRA", 2012),
    ("T-DM1 is standard second-line therapy for HER2-positive breast cancer after trastuzumab.",
     "EMILIA", 2012),
    ("Trastuzumab deruxtecan (T-DXd) shows superior PFS versus T-DM1 in HER2-positive breast cancer.",
     "DESTINY-Breast03", 2021),
    ("CDK4/6 inhibitors plus endocrine therapy improve PFS in HR-positive HER2-negative metastatic breast cancer.",
     "PALOMA-2", 2016),
    ("Olaparib improves PFS in BRCA-mutated HER2-negative metastatic breast cancer.",
     "OlympiAD", 2017),
    # Colorectal
    ("FOLFOX or FOLFIRI plus bevacizumab is standard first-line therapy for metastatic colorectal cancer.",
     "NO16966", 2008),
    ("Cetuximab is active in RAS wild-type metastatic colorectal cancer.",
     "CRYSTAL", 2009),
    ("BRAF V600E mutation predicts poor prognosis and resistance to anti-EGFR therapy in CRC.",
     "Bokemeyer_JCO_2012", 2012),
    ("Encorafenib plus cetuximab improves overall survival in BRAF V600E CRC.",
     "BEACON", 2019),
    # Mechanisms and biomarkers
    ("Microsatellite instability-high (MSI-H) tumors respond to pembrolizumab across histologies.",
     "KEYNOTE-158_MSI", 2017),
    ("Liquid biopsy ctDNA detects resistance mutations earlier than radiographic progression.",
     "Oxnard_JCO_2016", 2016),
    ("Tumor mutational burden is quantified as mutations per megabase from whole exome sequencing.",
     "Alexandrov_Nature_2013", 2013),
    ("MET exon 14 skipping mutations are targetable by capmatinib and tepotinib.",
     "GEOMETRY", 2020),
    ("ALK rearrangements occur in 3-5% of NSCLC and predict sensitivity to alectinib.",
     "ALEX", 2017),
    # Uncertainty / caveats (important for refusal calibration)
    ("The optimal duration of adjuvant immunotherapy in resected NSCLC remains under investigation.",
     "Review_JTO_2023", 2023),
    ("Predictive biomarkers for immunotherapy response beyond PD-L1 and TMB are not yet validated.",
     "Review_Lancet_Onc_2022", 2022),
    ("Combination strategies targeting co-occurring RAS and KRAS G12C mutations have uncertain benefit.",
     "Review_NatRevCancer_2023", 2023),
    ("Resistance mechanisms to third-generation EGFR TKIs are heterogeneous and incompletely characterised.",
     "Leonetti_NatRevClinOnc_2019", 2019),
    ("Overall survival benefit of ctDNA-guided therapy escalation is not yet established in randomised trials.",
     "Review_JCO_2024", 2024),
]

print(f"Knowledge store: {len(KNOWLEDGE_STORE)} claims loaded.")
print("\nSample entry:")
print(f"  Claim  : {KNOWLEDGE_STORE[0][0]}")
print(f"  Source : {KNOWLEDGE_STORE[0][1]}  ({KNOWLEDGE_STORE[0][2]})")
print("\nChapter 20 -- Cognitive Agent: environment ready.")
PyTorch : 2.11.0+cu128
Device  : cuda
GPU     : NVIDIA GeForce RTX 5080
Knowledge store: 30 claims loaded.

Sample entry:
  Claim  : First-line osimertinib significantly improves overall survival in EGFR-mutant NSCLC versus gefitinib or erlotinib.
  Source : FLAURA  (2018)

Chapter 20 -- Cognitive Agent: environment ready.

Sec.20.1 The ReAct Loop: Reason, Act, Observe, Reflect

Origin

ReAct (Yao et al., 2022) interleaves reasoning traces with actions, giving the model explicit scratchpad space before each tool call:

Thought: I need to find evidence about osimertinib in EGFR-mutant NSCLC.
Action:  retrieve("osimertinib EGFR NSCLC overall survival")
Obs:     [chunk_1, chunk_2, chunk_3]  (cosine sim > 0.30)

Thought: chunk_1 supports the claim; chunk_3 introduces uncertainty.
         I should check for contradictions before synthesising.
Action:  gcn_consistency(belief_graph)
Obs:     No conflicts detected. Entropy = 0.31 (below refusal threshold).

Thought: Evidence is sufficient. Draft answer.
Action:  synthesise(belief_graph)
Obs:     Draft: "Osimertinib is standard of care for first-line EGFR-mutant NSCLC..."

Thought: Draft confidence is high. Release.
Action:  reflect(draft)
Obs:     [OK] Reflection approved.

Key properties

Property Description
Interleaved thoughts Reasoning is explicit and inspectable, not hidden inside weights
Tool-bounded The agent can only do what its registered tools permit
Bounded iterations A MAX_CYCLES limit prevents infinite loops
Exit conditions Clean answer, refusal, or max-cycles exceeded

What we build

  • AgentState: dataclass holding the query, belief graph, iteration count, draft, and final answer.
  • ToolRegistry: dictionary mapping tool names to Python callables.
  • react_loop(query): the main loop - runs until an exit condition is met.
  • Four tools: retrieve, score_claims, gcn_consistency, reflect.

All components below are pure Python + PyTorch + numpy. No external agent framework is imported.

In [2]:
# -- TF-IDF retrieval tool (simulates Azure AI Search dense retrieval) -------

# Build TF-IDF index over the knowledge store
_corpus_texts  = [c[0] for c in KNOWLEDGE_STORE]
_corpus_meta   = [(c[1], c[2]) for c in KNOWLEDGE_STORE]  # (source, year)

_tfidf = TfidfVectorizer(ngram_range=(1, 2), min_df=1, stop_words="english")
_tfidf_matrix  = _tfidf.fit_transform(_corpus_texts)   # (30, vocab)

def retrieve(query, top_k=5, threshold=0.08):
    """
    Retrieve top_k claims from the knowledge store.
    Returns list of (claim_text, source_id, year, cosine_sim).
    In production this would be an Azure AI Search vector query.
    """
    q_vec = _tfidf.transform([query])
    sims  = cosine_similarity(q_vec, _tfidf_matrix).flatten()
    top   = np.argsort(sims)[::-1][:top_k]
    results = []
    for idx in top:
        if sims[idx] >= threshold:
            results.append((_corpus_texts[idx],
                            _corpus_meta[idx][0],
                            _corpus_meta[idx][1],
                            float(sims[idx])))
    return results

# Smoke test
test_hits = retrieve("osimertinib EGFR survival", top_k=3)
print("Retrieve smoke test (query: 'osimertinib EGFR survival'):")
for claim, src, yr, sim in test_hits:
    wrapped = textwrap.fill(claim, width=72, subsequent_indent="           ")
    print(f"  sim={sim:.3f}  [{src} {yr}]")
    print(f"           {wrapped}")
    print()
print("[OK] retrieve tool ready.")
Retrieve smoke test (query: 'osimertinib EGFR survival'):
  sim=0.294  [FLAURA 2018]
           First-line osimertinib significantly improves overall survival in EGFR-
           mutant NSCLC versus gefitinib or erlotinib.

  sim=0.246  [AURA3 2017]
           Osimertinib retains activity against EGFR T790M acquired resistance.

  sim=0.216  [NCCN_NSCLC_2024 2024]
           Osimertinib is standard of care for EGFR exon 19 deletion and L858R
           NSCLC.

[OK] retrieve tool ready.

Sec.20.2 Tool Registry and Agent State

Design principle

The tool registry is a plain Python dictionary:

TOOLS = {
    "retrieve":        retrieve,
    "score_claims":    score_claims,
    "gcn_consistency": gcn_consistency,
    "reflect":         reflect_on_draft,
}

The agent loop calls TOOLS[action_name](**kwargs). This means:

  • Adding a new capability = adding one function and one dictionary entry.
  • No framework lock-in. The same registry works with any backbone model.
  • The tool interface is explicit: each tool has a typed signature and a documented return value.

Agent state

AgentState is a simple dataclass that the loop reads and writes each cycle:

AgentState
  .query          str                  - original user question
  .belief_nodes   list[BeliefNode]     - claims + confidence in working memory
  .iteration      int                  - current ReAct cycle number
  .draft          str | None           - synthesised answer draft
  .final_answer   str | None           - approved final answer
  .trace          list[str]            - full reasoning trace (inspectable)
  .refused        bool                 - True if agent declined to answer

BeliefNode stores a single retrieved claim with its confidence and source:

BeliefNode
  .claim      str    - claim text
  .source     str    - source identifier
  .year       int    - publication year
  .sim        float  - retrieval similarity score (proxy for relevance)
  .confidence float  - updated by score_claims and gcn_consistency
  .status     str    - "supported" | "uncertain" | "contradicted"
In [3]:
# -- AgentState, BeliefNode, and score_claims tool ---------------------------
from dataclasses import dataclass, field
from typing import List, Optional

@dataclass
class BeliefNode:
    claim      : str
    source     : str
    year       : int
    sim        : float
    confidence : float = 0.5
    status     : str   = "uncertain"   # supported | uncertain | contradicted

    def __repr__(self):
        short = self.claim[:60] + ("..." if len(self.claim) > 60 else "")
        return (f"BeliefNode(conf={self.confidence:.2f}, "
                f"status={self.status!r}, src={self.source!r})\n"
                f"  claim: {short!r}")


@dataclass
class AgentState:
    query        : str
    belief_nodes : List[BeliefNode] = field(default_factory=list)
    iteration    : int = 0
    draft        : Optional[str] = None
    final_answer : Optional[str] = None
    trace        : List[str] = field(default_factory=list)
    refused      : bool = False

    def log(self, msg):
        self.trace.append(f"[cycle {self.iteration}] {msg}")

    def print_trace(self):
        for line in self.trace:
            print(textwrap.fill(line, width=90, subsequent_indent="    "))


# ---------------------------------------------------------------------------
# score_claims: classify each BeliefNode as supported/uncertain/contradicted
# Uses retrieval similarity as a proxy for confidence.
# In production: a fine-tuned NLI model would classify claim vs. query.
# ---------------------------------------------------------------------------

def score_claims(state: AgentState) -> AgentState:
    """
    Assigns confidence and status to each BeliefNode based on similarity score
    and a simple recency weight (newer = slightly higher base confidence).
    """
    CURRENT_YEAR = 2025
    for node in state.belief_nodes:
        recency = max(0.0, 1.0 - (CURRENT_YEAR - node.year) / 30.0)
        conf    = 0.5 * node.sim + 0.3 * recency + 0.2 * node.sim * recency
        conf    = float(np.clip(conf, 0.0, 1.0))
        node.confidence = conf
        if conf >= 0.55:
            node.status = "supported"
        elif conf <= 0.25:
            node.status = "contradicted"
        else:
            node.status = "uncertain"
    n_sup  = sum(1 for n in state.belief_nodes if n.status == "supported")
    n_unc  = sum(1 for n in state.belief_nodes if n.status == "uncertain")
    n_con  = sum(1 for n in state.belief_nodes if n.status == "contradicted")
    state.log(f"score_claims: {n_sup} supported, {n_unc} uncertain, {n_con} contradicted")
    return state

# Quick test
_s = AgentState(query="osimertinib EGFR")
_s.belief_nodes = [BeliefNode(c, src, yr, sim)
                   for c, src, yr, sim in test_hits]
_s = score_claims(_s)
print("score_claims smoke test:")
for n in _s.belief_nodes:
    print(f"  {n}")
print("\n[OK] score_claims tool ready.")
score_claims smoke test:
  BeliefNode(conf=0.42, status='uncertain', src='FLAURA')
  claim: 'First-line osimertinib significantly improves overall surviv...'
  BeliefNode(conf=0.38, status='uncertain', src='AURA3')
  claim: 'Osimertinib retains activity against EGFR T790M acquired res...'
  BeliefNode(conf=0.44, status='uncertain', src='NCCN_NSCLC_2024')
  claim: 'Osimertinib is standard of care for EGFR exon 19 deletion an...'

[OK] score_claims tool ready.

Sec.20.3 Belief State as a Graph

Why a graph?

A list of retrieved claims has no structure. In clinical reasoning, claims interact:

  • Claim A supports Claim B (both point toward the same conclusion).
  • Claim C contradicts Claim B (different evidence bases, conflicting data).
  • Claim D qualifies Claim B ("B is true, but only if condition D holds").

These relationships are edges in a belief graph. Once we have a graph, we can apply the GCN from Chapter 19 to propagate consistency signals: a well- supported claim reinforces its neighbours; a contradicted claim weakens them.

Construction

For $N$ retrieved claims, we build an $N \times N$ adjacency matrix:

$$A_{ij} = \text{sim}(\mathbf{e}_i, \mathbf{e}_j) \quad \text{if } i \neq j \text{ and sim} > \theta$$

where $\mathbf{e}_i$ is the TF-IDF vector of claim $i$ and $\theta = 0.05$ is a connectivity threshold. This makes semantically similar claims neighbours in the graph.

The GCN then propagates confidence scores across this neighbourhood structure: a claim surrounded by high-confidence neighbours gets its own confidence score pulled upward; a claim surrounded by contradicted neighbours gets pulled down.

This is not a heuristic: it is the same spectral normalisation and message passing derived in Sec.19.2, applied directly to a clinical reasoning task.

In [4]:
# -- Belief graph construction and GCN consistency propagation ---------------

# Reuse GCNLayer from Ch19 (same implementation, same derivation)
class GCNLayer(nn.Module):
    def __init__(self, in_dim, out_dim):
        super().__init__()
        self.W = nn.Linear(in_dim, out_dim, bias=False)
        nn.init.xavier_uniform_(self.W.weight)

    def forward(self, x, a_norm):
        return a_norm @ self.W(x)


def _build_belief_adj(nodes: List[BeliefNode]) -> np.ndarray:
    """
    Build NxN adjacency matrix for belief nodes using TF-IDF cosine similarity.
    Adds self-loops and thresholds weak edges.
    """
    texts  = [n.claim for n in nodes]
    vecs   = _tfidf.transform(texts)
    sims   = cosine_similarity(vecs, vecs).astype(np.float32)
    THRESH = 0.05
    adj    = (sims > THRESH).astype(np.float32)
    np.fill_diagonal(adj, 1.0)   # self-loops
    return adj


def _norm_adj(adj_np: np.ndarray) -> torch.Tensor:
    """Symmetric normalisation: D^{-1/2} A D^{-1/2} (Ch19 sec 19.2)."""
    d         = adj_np.sum(1)
    d_inv_sqrt = np.diag((d + 1e-8) ** -0.5)
    a_norm    = (d_inv_sqrt @ adj_np @ d_inv_sqrt).astype(np.float32)
    return torch.tensor(a_norm).to(device)


def gcn_consistency(state: AgentState,
                    n_gcn_layers: int = 2) -> AgentState:
    """
    Run a 2-layer GCN over the belief graph.
    Input features: [confidence, sim, year_norm] (3-dim per node).
    Output: updated confidence scores after neighbourhood propagation.
    Flags any node whose confidence drops > 0.15 as 'contradicted'.
    """
    nodes = state.belief_nodes
    if len(nodes) < 2:
        state.log("gcn_consistency: skipped (fewer than 2 nodes)")
        return state

    # Node feature matrix: (N, 3)
    YEAR_MIN, YEAR_MAX = 2005, 2025
    feat = np.array([
        [n.confidence,
         n.sim,
         (n.year - YEAR_MIN) / (YEAR_MAX - YEAR_MIN + 1e-8)]
        for n in nodes
    ], dtype=np.float32)
    X     = torch.tensor(feat).to(device)
    A_hat = _norm_adj(_build_belief_adj(nodes))

    # 2-layer GCN: 3 -> 8 -> 1
    gcn = nn.Sequential(
        _GCNWrapper(GCNLayer(3, 8), A_hat),
        nn.ReLU(),
        _GCNWrapper(GCNLayer(8, 1), A_hat),
        nn.Sigmoid(),
    ).to(device)

    with torch.no_grad():
        conf_out = gcn(X).squeeze(-1).cpu().numpy()   # (N,)

    conflicts = 0
    for ni, node in enumerate(nodes):
        old_conf  = node.confidence
        new_conf  = float(conf_out[ni])
        node.confidence = new_conf
        drop = old_conf - new_conf
        if drop > 0.15:
            node.status = "contradicted"
            conflicts  += 1
        elif new_conf >= 0.55 and node.status == "uncertain":
            node.status = "supported"

    state.log(f"gcn_consistency: propagated confidence over {len(nodes)}-node "
              f"belief graph. {conflicts} nodes downgraded.")
    return state


class _GCNWrapper(nn.Module):
    """Wraps a GCNLayer so it can sit inside nn.Sequential."""
    def __init__(self, layer: GCNLayer, a_norm: torch.Tensor):
        super().__init__()
        self.layer  = layer
        self.a_norm = a_norm

    def forward(self, x):
        return self.layer(x, self.a_norm)


# Smoke test
_s2 = AgentState(query="EGFR TKI resistance")
raw = retrieve("EGFR TKI resistance mechanisms", top_k=5)
_s2.belief_nodes = [BeliefNode(c, s, y, sim) for c, s, y, sim in raw]
_s2 = score_claims(_s2)
_s2.iteration = 1
_s2 = gcn_consistency(_s2)
print("gcn_consistency smoke test:")
for n in _s2.belief_nodes:
    print(f"  conf={n.confidence:.3f}  status={n.status!r}")
    print(f"    {n.claim[:75]}")
print("\n[OK] gcn_consistency tool ready.")
gcn_consistency smoke test:
  conf=0.451  status='uncertain'
    Resistance mechanisms to third-generation EGFR TKIs are heterogeneous and i
  conf=0.451  status='contradicted'
    EGFR exon 20 insertions predict primary resistance to standard EGFR TKIs.
  conf=0.451  status='uncertain'
    T790M resistance mutation accounts for approximately 60% of acquired resist
  conf=0.451  status='uncertain'
    Osimertinib retains activity against EGFR T790M acquired resistance.
  conf=0.451  status='contradicted'
    BRAF V600E mutation predicts poor prognosis and resistance to anti-EGFR the

[OK] gcn_consistency tool ready.

Sec.20.4 Entropy-Based Calibration and the Refusal Threshold

From Chapter 13 to clinical refusal

Chapter 13 derived the Shannon entropy of a probability distribution $p$:

$$H(p) = -\sum_i p_i \log_2 p_i$$

Here we apply it to the belief distribution: treat the normalised confidence scores of the $N$ retrieved claims as a probability vector $p$.

  • Low entropy: most claims have similar, high confidence. The agent's belief is concentrated and coherent. Safe to answer.
  • High entropy: confidence is spread uniformly -- some claims supported, some contradicted, evidence genuinely mixed. The agent should not answer; it should say so explicitly.

Threshold derivation

For $N$ claims, maximum entropy is $\log_2 N$ (uniform distribution). We set the refusal threshold at $H_{\text{refuse}} = 0.65 \cdot \log_2 N$:

$$\text{refuse} \iff H(p) > 0.65 \cdot \log_2 N$$

This is a heuristic chosen to trigger refusal when evidence is genuinely mixed (entropy > 65% of maximum) while permitting answers when there is a clear dominant supported claim.

Why this is better than a confidence threshold

A naive rule "refuse if max confidence < 0.7" fails when:

  • There are 2 contradicting claims, both with confidence 0.80. Max confidence is high but the belief is split.
  • Entropy catches this immediately: $H = 1.0$ bit for 2 equal-weight claims.

Entropy is the right measure for distribution shape, not just peak value.

In [5]:
# -- Entropy calibration, refusal check, and synthesiser --------------------

def belief_entropy(nodes: List[BeliefNode]) -> float:
    """Shannon entropy of the normalised confidence distribution (Ch13)."""
    confs = np.array([n.confidence for n in nodes], dtype=np.float64)
    confs = confs / (confs.sum() + 1e-12)
    confs = confs[confs > 1e-12]
    return float(-np.sum(confs * np.log2(confs)))


def check_entropy(state: AgentState,
                  refuse_fraction: float = 0.65) -> AgentState:
    """
    Computes belief entropy and sets state.refused = True if above threshold.
    """
    nodes = state.belief_nodes
    if not nodes:
        state.refused = True
        state.log("check_entropy: no beliefs - refusing.")
        return state

    H       = belief_entropy(nodes)
    H_max   = np.log2(len(nodes))
    H_norm  = H / (H_max + 1e-12)
    threshold = refuse_fraction * H_max

    state.log(f"check_entropy: H={H:.3f}  H_max={H_max:.3f}  "
              f"H_norm={H_norm:.3f}  threshold={threshold:.3f}")

    if H > threshold:
        state.refused = True
        state.log("check_entropy: evidence entropy too high - REFUSING.")
    else:
        state.log("check_entropy: entropy within acceptable range - proceeding.")
    return state


def synthesise(state: AgentState) -> AgentState:
    """
    Builds a draft answer from supported belief nodes.
    Includes evidence citations and a confidence summary.
    In production this step calls the LLM with the belief graph as context.
    Here we build a structured template answer from the node data.
    """
    supported = [n for n in state.belief_nodes if n.status == "supported"]
    uncertain = [n for n in state.belief_nodes if n.status == "uncertain"]

    if not supported:
        state.draft = ("Insufficient supported evidence to answer this query. "
                       "Consider refining the question or expanding the search.")
        state.log("synthesise: no supported nodes - draft is refusal template.")
        return state

    lines = [f"Query: {state.query}\n"]
    lines.append("Based on retrieved evidence:\n")
    for i, n in enumerate(supported, 1):
        lines.append(f"  ({i}) {n.claim}")
        lines.append(f"       Source: {n.source} ({n.year})  "
                     f"Confidence: {n.confidence:.2f}\n")
    if uncertain:
        lines.append("Uncertainties noted:")
        for n in uncertain:
            lines.append(f"  - {n.claim[:80]}")
    lines.append(f"\n[{len(supported)} supported / "
                 f"{len(uncertain)} uncertain / "
                 f"{sum(1 for n in state.belief_nodes if n.status == 'contradicted')} contradicted]")

    state.draft = "\n".join(lines)
    state.log(f"synthesise: draft built from {len(supported)} supported nodes.")
    return state


# Smoke test entropy check
print("check_entropy smoke test:")
_s2 = check_entropy(_s2)
_s2.print_trace()
print("\n[OK] check_entropy and synthesise tools ready.")
check_entropy smoke test:
[cycle 0] score_claims: 0 supported, 3 uncertain, 2 contradicted
[cycle 1] gcn_consistency: propagated confidence over 5-node belief graph. 0 nodes
    downgraded.
[cycle 1] check_entropy: H=2.322  H_max=2.322  H_norm=1.000  threshold=1.509
[cycle 1] check_entropy: evidence entropy too high - REFUSING.

[OK] check_entropy and synthesise tools ready.

Sec.20.5 The Reflection Gate

Self-critique before release

A cognitive agent does not release the first draft it produces. The reflection gate runs a second pass that asks:

  1. Does the draft make a claim not supported by any belief node?
  2. Is the confidence summary accurate?
  3. Is there a contradicted node that the draft failed to mention?
  4. Is the claim scope appropriate to the query?

In a production system, this second pass uses the LLM itself (prompted to act as a critic of its own output -- a technique called Constitutional AI or self-refinement). Here we implement a deterministic rule-based reflector that catches the most common failure modes.

Reflection rules

Rule Trigger Action
R1 No supported nodes Draft built with zero supported nodes Mark as FAIL; set refused=True
R2 Contradicted evidence present >= 1 contradicted node Append caveat to draft
R3 Low overall confidence Mean confidence of supported nodes < 0.45 Downgrade to "low confidence" label
R4 Single source All supported nodes from same source Flag potential bias
R5 Approved None of R1-R4 triggered Release draft as final answer

What this prevents

Without the reflection gate, the synthesiser might release a confident-sounding answer backed by a single marginal-confidence source. The gate catches this (R4) and flags it before the answer reaches the clinician.

In [6]:
# -- Reflection gate and tool registry ---------------------------------------

def reflect_on_draft(state: AgentState) -> AgentState:
    """
    Runs reflection rules R1-R5 on the current draft.
    Modifies state.draft and may set state.refused.
    """
    supported    = [n for n in state.belief_nodes if n.status == "supported"]
    contradicted = [n for n in state.belief_nodes if n.status == "contradicted"]

    # R1: no supported nodes
    if not supported:
        state.refused = True
        state.log("reflect: R1 FAIL - no supported nodes. Refusing.")
        state.final_answer = ("I cannot provide a reliable answer to this query. "
                              "The retrieved evidence does not contain sufficiently "
                              "supported claims.")
        return state

    caveats = []

    # R2: contradicted evidence present
    if contradicted:
        caveats.append(
            f"Note: {len(contradicted)} retrieved claim(s) were flagged as "
            f"contradicted after GCN consistency propagation and should be "
            f"interpreted with caution."
        )
        state.log(f"reflect: R2 - {len(contradicted)} contradicted nodes. Adding caveat.")

    # R3: low mean confidence
    mean_conf = np.mean([n.confidence for n in supported])
    if mean_conf < 0.45:
        caveats.append(
            f"Note: Mean confidence of supported claims is {mean_conf:.2f} "
            f"(< 0.45). This answer should be verified against primary sources."
        )
        state.log(f"reflect: R3 - low mean confidence {mean_conf:.2f}.")

    # R4: single source
    sources = set(n.source for n in supported)
    if len(sources) == 1:
        caveats.append(
            f"Note: All supported claims originate from a single source "
            f"({next(iter(sources))}). Independent corroboration is advised."
        )
        state.log("reflect: R4 - single source detected.")

    if caveats:
        state.draft = state.draft + "\n\n" + "\n".join(caveats)

    state.final_answer = state.draft
    state.log(f"reflect: R5 APPROVED. Draft released as final answer "
              f"(mean_conf={mean_conf:.2f}, sources={len(sources)}).")
    return state


# Tool registry: the only interface the agent loop uses
TOOL_REGISTRY = {
    "retrieve"        : retrieve,
    "score_claims"    : score_claims,
    "gcn_consistency" : gcn_consistency,
    "check_entropy"   : check_entropy,
    "synthesise"      : synthesise,
    "reflect"         : reflect_on_draft,
}

print("Tool registry loaded:")
for name in TOOL_REGISTRY:
    print(f"  [{name}]")
print("\n[OK] reflection gate and tool registry ready.")
Tool registry loaded:
  [retrieve]
  [score_claims]
  [gcn_consistency]
  [check_entropy]
  [synthesise]
  [reflect]

[OK] reflection gate and tool registry ready.

Sec.20.6 The Full Agent Loop

Control flow

def react_loop(query, max_cycles=4):
    state = AgentState(query=query)

    while state.iteration < max_cycles and state.final_answer is None:
        state.iteration += 1

        # Cycle 1: retrieve and score
        results = TOOLS["retrieve"](query, top_k=5)
        state.belief_nodes = [BeliefNode(...) for r in results]
        state = TOOLS["score_claims"](state)

        # Cycle 2: consistency propagation
        state = TOOLS["gcn_consistency"](state)

        # Cycle 3: entropy check -> refuse or proceed
        state = TOOLS["check_entropy"](state)
        if state.refused:
            state.final_answer = REFUSAL_MESSAGE
            break

        # Cycle 4: synthesise + reflect
        state = TOOLS["synthesise"](state)
        state = TOOLS["reflect"](state)

    return state

In a language-model-backed agent, the "Thought" step before each action is generated by the LLM. Here the action sequence is deterministic -- we focus on the infrastructure (state management, tool calling, belief graph, reflection) rather than on prompt engineering.

The same infrastructure supports any backbone: swap the retrieval tool for Azure AI Search, swap the synthesiser for a GPT-4 prompt call, and the ReAct loop continues to function without modification.

In [7]:
# -- Full ReAct agent loop ---------------------------------------------------

REFUSAL_MESSAGE = (
    "I cannot provide a reliable answer to this query.\n"
    "The retrieved evidence has high uncertainty (entropy above threshold).\n"
    "Recommendation: consult primary literature or a clinical specialist."
)


def react_loop(query: str,
               top_k: int = 5,
               max_cycles: int = 4,
               verbose: bool = True) -> AgentState:
    """
    Full ReAct cognitive agent loop.

    Steps per cycle:
      1. retrieve  - fetch top_k claims from knowledge store
      2. score_claims - assign confidence + status to each node
      3. gcn_consistency - propagate confidence over belief graph
      4. check_entropy   - refuse if evidence is too uncertain
      5. synthesise      - build draft answer from supported nodes
      6. reflect         - critique draft; release or refuse

    Returns AgentState with .final_answer (str) or .refused (True).
    """
    state = AgentState(query=query)
    if verbose:
        print(f"{'='*70}")
        print(f"QUERY: {query}")
        print(f"{'='*70}")

    while state.iteration < max_cycles and state.final_answer is None:
        state.iteration += 1
        if verbose:
            print(f"\n--- Cycle {state.iteration} ---")

        # Step 1: retrieve
        results = retrieve(query, top_k=top_k)
        if not results:
            state.refused = True
            state.final_answer = REFUSAL_MESSAGE
            state.log("react_loop: retrieve returned empty - refusing.")
            break
        state.belief_nodes = [BeliefNode(c, s, y, sim)
                              for c, s, y, sim in results]
        state.log(f"retrieve: {len(results)} chunks retrieved.")
        if verbose:
            print(f"  [retrieve] {len(results)} claims fetched.")

        # Step 2: score claims
        state = score_claims(state)
        if verbose:
            counts = {"supported": 0, "uncertain": 0, "contradicted": 0}
            for n in state.belief_nodes:
                counts[n.status] += 1
            print(f"  [score]    {counts}")

        # Step 3: GCN consistency
        state = gcn_consistency(state)
        if verbose:
            counts = {"supported": 0, "uncertain": 0, "contradicted": 0}
            for n in state.belief_nodes:
                counts[n.status] += 1
            print(f"  [gcn]      {counts}")

        # Step 4: entropy check
        state = check_entropy(state)
        if verbose:
            H = belief_entropy(state.belief_nodes)
            print(f"  [entropy]  H={H:.3f}  refused={state.refused}")
        if state.refused:
            state.final_answer = REFUSAL_MESSAGE
            break

        # Step 5: synthesise
        state = synthesise(state)
        if verbose:
            supported_n = sum(1 for n in state.belief_nodes
                              if n.status == "supported")
            print(f"  [synth]    draft built from {supported_n} supported nodes.")

        # Step 6: reflect
        state = reflect_on_draft(state)
        if verbose:
            print(f"  [reflect]  refused={state.refused}  "
                  f"answer_ready={state.final_answer is not None}")

    if verbose:
        print(f"\n{'='*70}")
        print("FINAL ANSWER:")
        print(f"{'='*70}")
        if state.final_answer:
            print(textwrap.fill(state.final_answer, width=90))
        else:
            print("[max cycles reached without final answer]")
        print(f"\nTotal cycles: {state.iteration}")
        print(f"Refused: {state.refused}")

    return state

print("[OK] react_loop defined. Agent is ready.")
[OK] react_loop defined. Agent is ready.

Sec.20.7 Running the Agent on Oncology Queries

We run three queries that test distinct agent behaviours:

Query Expected behaviour
Q1: osimertinib first-line EGFR NSCLC High-confidence answer. Multiple supported nodes from landmark trials.
Q2: BRAF V600E CRC treatment Moderate confidence. Encorafenib + cetuximab node should be retrieved.
Q3: optimal duration adjuvant immunotherapy resected NSCLC Refusal or strong uncertainty caveats. Explicit "under investigation" node in the store.

Query 3 tests the calibrated refusal path. A system without entropy-based calibration would synthesise a confident-sounding non-answer. The cognitive agent should detect high entropy and either refuse or attach a prominent uncertainty warning.

In [8]:
# -- Q1: High-confidence query -----------------------------------------------
state_q1 = react_loop(
    query="osimertinib first-line EGFR mutant NSCLC overall survival",
    top_k=5,
    verbose=True,
)
======================================================================
QUERY: osimertinib first-line EGFR mutant NSCLC overall survival
======================================================================

--- Cycle 1 ---
  [retrieve] 5 claims fetched.
  [score]    {'supported': 1, 'uncertain': 4, 'contradicted': 0}
  [gcn]      {'supported': 1, 'uncertain': 4, 'contradicted': 0}
  [entropy]  H=2.322  refused=True

======================================================================
FINAL ANSWER:
======================================================================
I cannot provide a reliable answer to this query. The retrieved evidence has high
uncertainty (entropy above threshold). Recommendation: consult primary literature or a
clinical specialist.

Total cycles: 1
Refused: True
In [9]:
# -- Q2: Moderate-confidence query -------------------------------------------
state_q2 = react_loop(
    query="BRAF V600E colorectal cancer targeted therapy",
    top_k=5,
    verbose=True,
)
======================================================================
QUERY: BRAF V600E colorectal cancer targeted therapy
======================================================================

--- Cycle 1 ---
  [retrieve] 5 claims fetched.
  [score]    {'supported': 0, 'uncertain': 4, 'contradicted': 1}
  [gcn]      {'supported': 0, 'uncertain': 4, 'contradicted': 1}
  [entropy]  H=2.322  refused=True

======================================================================
FINAL ANSWER:
======================================================================
I cannot provide a reliable answer to this query. The retrieved evidence has high
uncertainty (entropy above threshold). Recommendation: consult primary literature or a
clinical specialist.

Total cycles: 1
Refused: True
In [10]:
# -- Q3: Uncertainty / refusal test ------------------------------------------
state_q3 = react_loop(
    query="optimal duration adjuvant immunotherapy resected NSCLC unknown",
    top_k=5,
    verbose=True,
)
======================================================================
QUERY: optimal duration adjuvant immunotherapy resected NSCLC unknown
======================================================================

--- Cycle 1 ---
  [retrieve] 1 claims fetched.
  [score]    {'supported': 1, 'uncertain': 0, 'contradicted': 0}
  [gcn]      {'supported': 1, 'uncertain': 0, 'contradicted': 0}
  [entropy]  H=0.000  refused=True

======================================================================
FINAL ANSWER:
======================================================================
I cannot provide a reliable answer to this query. The retrieved evidence has high
uncertainty (entropy above threshold). Recommendation: consult primary literature or a
clinical specialist.

Total cycles: 1
Refused: True
In [11]:
# -- Belief graph visualisation for Q1 ---------------------------------------
nodes_q1 = state_q1.belief_nodes
N = len(nodes_q1)

# Build adjacency for visualisation
adj_vis = _build_belief_adj(nodes_q1)

# Node positions: circular layout
angles   = np.linspace(0, 2 * np.pi, N, endpoint=False)
pos      = np.column_stack([np.cos(angles), np.sin(angles)])

STATUS_COLOR = {
    "supported"   : "tab:green",
    "uncertain"   : "tab:orange",
    "contradicted": "tab:red",
}

fig, ax = plt.subplots(figsize=(8, 7))

# Edges
for i in range(N):
    for j in range(i + 1, N):
        if adj_vis[i, j] > 0.05:
            ax.plot([pos[i, 0], pos[j, 0]],
                    [pos[i, 1], pos[j, 1]],
                    color="grey", alpha=0.3, lw=1.0, zorder=1)

# Nodes
for i, node in enumerate(nodes_q1):
    color = STATUS_COLOR[node.status]
    size  = 800 + 1200 * node.confidence
    ax.scatter(pos[i, 0], pos[i, 1], s=size, c=color, zorder=2,
               edgecolors="white", linewidths=1.2)
    label = node.source[:12]
    ax.annotate(label, pos[i], fontsize=7.5, ha="center", va="center",
                color="white", fontweight="bold", zorder=3)

# Legend
legend_handles = [
    mpatches.Patch(facecolor=STATUS_COLOR[s], label=s.capitalize())
    for s in STATUS_COLOR
]
ax.legend(handles=legend_handles, loc="upper right", frameon=True)
ax.set_title(
    "Belief Graph - Q1: Osimertinib first-line EGFR NSCLC\n"
    "Node size proportional to confidence. Edge = semantic similarity.",
    fontsize=10,
)
ax.axis("off")
plt.tight_layout()
plt.show()

# Confidence bar chart
fig, ax = plt.subplots(figsize=(9, 3.5))
labels  = [n.source[:14] for n in nodes_q1]
confs   = [n.confidence for n in nodes_q1]
colors  = [STATUS_COLOR[n.status] for n in nodes_q1]
bars    = ax.bar(labels, confs, color=colors, edgecolor="white", linewidth=0.7)
ax.axhline(0.55, color="tab:green", ls="--", lw=1.5, label="Supported threshold (0.55)")
ax.axhline(0.25, color="tab:red",   ls="--", lw=1.5, label="Contradicted threshold (0.25)")
ax.set_ylim(0, 1.05)
ax.set_ylabel("Post-GCN Confidence")
ax.set_title("Q1 - Belief Node Confidence After GCN Consistency Propagation")
ax.legend(fontsize=9)
plt.tight_layout()
plt.show()
print("[OK] belief graph visualisation complete.")
No description has been provided for this image
No description has been provided for this image
[OK] belief graph visualisation complete.
In [12]:
# -- Comparison: RAG baseline vs cognitive agent ----------------------------
print("=" * 70)
print("ARCHITECTURE COMPARISON")
print("=" * 70)

rows = [
    ("Feature",              "RAG Baseline",              "Cognitive Agent (Ch20)"),
    ("-" * 22,               "-" * 27,                    "-" * 27),
    ("Working memory",       "None (stateless)",          "BeliefGraph (persistent per query)"),
    ("Consistency check",    "None",                      "GCN over belief graph (Ch19)"),
    ("Uncertainty measure",  "None",                      "Shannon entropy (Ch13)"),
    ("Calibrated refusal",   "Never refuses",             "Refuses when H > 0.65 * H_max"),
    ("Evidence trail",       "Cited chunks only",         "Full reasoning trace + node status"),
    ("Reflection",           "None",                      "R1-R5 rule-based reflector"),
    ("Framework dependency", "LangChain / LlamaIndex",    "None (pure Python + PyTorch)"),
    ("Backbone swappable",   "No (tightly coupled)",      "Yes (retrieve tool is pluggable)"),
]

for r in rows:
    print(f"  {r[0]:<24} {r[1]:<30} {r[2]}")

print("\n[OK] comparison complete.")
======================================================================
ARCHITECTURE COMPARISON
======================================================================
  Feature                  RAG Baseline                   Cognitive Agent (Ch20)
  ----------------------   ---------------------------    ---------------------------
  Working memory           None (stateless)               BeliefGraph (persistent per query)
  Consistency check        None                           GCN over belief graph (Ch19)
  Uncertainty measure      None                           Shannon entropy (Ch13)
  Calibrated refusal       Never refuses                  Refuses when H > 0.65 * H_max
  Evidence trail           Cited chunks only              Full reasoning trace + node status
  Reflection               None                           R1-R5 rule-based reflector
  Framework dependency     LangChain / LlamaIndex         None (pure Python + PyTorch)
  Backbone swappable       No (tightly coupled)           Yes (retrieve tool is pluggable)

[OK] comparison complete.

Sec.20.8 What Volume II Builds Next

The gap this chapter does not close

Chapter 20 builds the infrastructure of a cognitive agent. What it lacks is the knowledge -- the agent reasons over 30 curated statements, not over a real clinical corpus.

Volume II closes that gap. The goal is a working cognitive agent grounded in the published career of a physician-scientist oncologist: a system that reasons the way that person reasons, draws on the evidence that person has generated, and can articulate not just conclusions but the reasoning path.

Volume II outline (provisional)

Part 1 -- The Corpus

  • Automated ingestion: PubMed author ID pull, PDF extraction, grant text parsing
  • De-identification and privacy audit
  • Claim-level semantic chunking (not sentence-level)
  • Knowledge graph construction: claim nodes, citation edges, temporal edges
  • Azure AI Search index rebuild with dense embeddings (replacing TF-IDF)

Part 2 -- Memory Systems

  • Semantic memory: Azure AI Search (already operational in Cyberdoc_Onc)
  • Episodic memory: case-encounter embeddings with temporal indexing
  • Procedural memory: clinical workflow graphs (GCN over treatment decision trees)
  • Working memory: attention-weighted context window manager

Part 3 -- The Reasoning Engine (rebuilt)

  • The Cyberdoc_Onc coordinator is rebuilt as a stateful cognitive agent
  • Persistent belief state across multi-turn conversations
  • Planning horizon: the agent enumerates sub-goals before answering
  • Reflection gate: LLM-based self-critique replaces the rule-based reflector

Part 4 -- Metacognition

  • Calibration curves: does the agent's stated confidence match empirical accuracy?
  • Bayesian confidence propagation (Ch06 applied to retrieval pipelines)
  • GCN-based consistency on the full knowledge graph (10 K+ nodes)

Part 5 -- Evaluation

  • Clinician-in-the-loop evaluation protocol
  • Reasoning trace interpretability (why did the agent retrieve that chunk?)
  • Comparison with RAG-only Cyberdoc_Onc baseline

The corpus audit: first practical step

Before a line of Volume II code is written, the source documents must be catalogued. The corpus audit answers:

  1. Publication record: How many papers? PubMed author ID for automated pull.
  2. Grant corpus: NIH Reporter / ORCID export of funded grants.
  3. Clinical materials: teaching cases, conference presentations, notes.
  4. Formats: PDF, Word, PubMed XML, structured text.
  5. De-identification scope: any documents containing patient data?
  6. Scale estimate: total word count -> expected number of claim-level chunks at 150-word chunk size.

Scale determines the architecture:

Corpus size Retrieval strategy Index type
< 5 K chunks TF-IDF (this chapter) In-memory
5 K - 100 K chunks Dense embeddings Azure AI Search Standard tier
> 100 K chunks Dense + sparse hybrid Azure AI Search with semantic ranker

The Cyberdoc_Onc Azure AI Search index (Standard tier, already provisioned) supports up to 1 million documents. No new infrastructure is needed for any realistic physician-scientist career corpus.

Sec.20.9 Volume I Summary: The Complete Stack

This chapter is the synthesis of nineteen preceding chapters. The table below maps every architectural component of the cognitive agent to the chapter that derived its mathematical foundation.

Agent Component Foundation Chapter
TF-IDF / cosine similarity retrieval Vector spaces, dot products Ch10
Confidence scoring Regression, probability calibration Ch06
Shannon entropy refusal threshold Information theory Ch13
GCN belief graph consistency Graph convolutional networks Ch19
BeliefNode status classifier Classification Ch07
Attention in the reasoning model Multi-head self-attention Ch16
Encoder backbone (production) BERT pre-training Ch17
Semantic similarity (vision analogy) Patch-level attention Ch18
ReAct loop training (future) Gradient descent, backprop Ch14-15

What has been built in Volume I

Nineteen chapters, each a working executable notebook, forming a complete arc from Python fundamentals to a cognitive reasoning agent:

Python -> Statistics -> Probability -> ML -> Neural Networks
       -> Attention -> Transformers -> Vision -> Graphs -> Agent

The agent in this chapter runs on the same GPU, the same Python environment, and the same mathematical foundations as Chapter 1's first import numpy. Nothing was imported from a framework that hides the mechanism. Everything is inspectable, modifiable, and understood from first principles.

Volume II begins where this chapter ends: with a real corpus, a real knowledge graph, and a real clinical reasoning task.


Section 20.10 — Summary & Quick Reference

Key Formulas

Concept Formula Python
TF-IDF cosine retrieval $\text{sim}(\mathbf{q},\mathbf{d}) = \mathbf{q}\cdot\mathbf{d}\,/\,(\|\mathbf{q}\|\,\|\mathbf{d}\|)$ q @ D.T / (norm(q) * norm(D, axis=1))
Belief entropy (refusal) $H(B) = -\sum_i p_i \log_2 p_i$ -np.sum(p * np.log2(p + 1e-12))
GCN consistency pass $\mathbf{H}' = \sigma\!\left(\hat{\mathbf{A}}\mathbf{H}\mathbf{W}\right)$ GCN over BeliefGraph nodes (Ch19)
Confidence score $c = s_{\max} \cdot (1 - H / H_{\max})$ conf = sim_max * (1 - H / H_max)
Reflection gate release if $\text{score}_{\text{refl}} \geq \tau$, else re-plan rule-based critic or LLM evaluator

ML Connections

Concept Where It Appears
Cosine retrieval Dot product / L2 norm (Ch10); nearest-neighbour search over claim embeddings
Entropy refusal Shannon entropy (Ch13); high $H$ signals conflicting evidence -- decline to answer
GCN belief graph Graph convolutional network (Ch19) propagates confidence across related claims
[CLS] encoder backbone BERT-style encoder (Ch17) for semantic claim similarity scoring
ReAct loop Structured generalisation of the gradient-descent loop (Ch14): plan, act, observe, update

What's Next?

$$\boxed{\text{Ch 20: Cognitive Agent}} \;\to\; \boxed{\text{Ch 21: Bridge to Vol II}} \;\to\; \boxed{\text{Vol II: Production Engineering}}$$

Chapter 21 names every gap between the notebook prototype and a deployable clinical tool, maps each Vol I component to its production counterpart in agent-creator and Cyberdoc_Onc, and previews the six-part Volume II engineering narrative that closes those gaps.

References

  1. Yao S, Zhao J, Yu D, et al. (2022). ReAct: Synergizing reasoning and acting in language models. arXiv:2210.03629.
  2. Shinn N, Cassano F, Berman E, et al. (2023). Reflexion: Language agents with verbal reinforcement learning. NeurIPS 2023.
  3. Mialon G, Dsssouli R, Guth M, et al. (2023). Augmented language models: A survey. TMLR.
  4. Lewis P, Perez E, Piktus A, et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks (RAG). NeurIPS, 33.
  5. Kipf TN and Welling M (2017). Semi-supervised classification with graph convolutional networks. ICLR 2017. [Ch19]
  6. Devlin J, Chang M-W, Lee K, Toutanova K (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL.
  7. Bai Y, Jones A, Ndousse K, et al. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv:2212.08073.
  8. Madaan A, Tandon N, Gupta P, et al. (2023). Self-refine: Iterative refinement with self-feedback. NeurIPS 2023.
  9. Shannon CE (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379-423. [Ch13]
  10. Guu K, Lee K, Tung Z, Pasupat P, Chang M (2020). REALM: Retrieval-augmented language model pre-training. ICML.