Stream: Mirror: Isabelle Users Mailing List

Topic: [isabelle] Missing PIDE "consolidation" messages


view this post on Zulip Email Gateway (Nov 11 2026 at 21:07):

From: "Dockins, Robert" <cl-isabelle-users@lists.cam.ac.uk>

I was recently investigating the race condition posted by Fabian Huch on this list to determine if it was affecting our workflows. During that investigation, I found what I believe to be a separate race condition involving the issuance of PIDE “consolidation” messages on a theory file. The symptom is that all the individual statements in a theory complete, but the overall theory is never consolidated, which can then result in unbounded delays for anything that is waiting for that message. We’re running into this issue fairly regularly with agent-driven workflows, where they wait for a theory to complete processing before continuing.

This delay can be observed both in Isabelle’s headless “server” mode and in “jEdit” mode. The following is a minimal reproducer for Isabelle2026; a similar reproducer also triggers the issue on Isaballe2025-2 (using OS.Process.sleep​ instead of Time.sleep​). To observe the issue load the following theory in jEdit:

theory Lost_Consolidation
  imports Pure
begin

ML ‹val _ = Future.fork (fn () => Time.sleep (Time.fromSeconds 6))›

end

Note that all the individual commands eventually complete, but the theory never gets the “bold box” in the “Theories” pane, no matter how long you wait (well after the 6 second sleep). Making a small whitespace change at the end of the theory, however, causes it to consolidate and turn bold almost immediately. The forked future is a stand-in for certain kinds of long-running forked proof tasks near the end of a theory file (such as an expensive by​ tactic). I’m not sure what the best approach is for a fix, but I will paste in the AI agents’s diagnosis and suggestions below in case they are helpful.

Cheers,
Rob

Agent’s diagnosis:

1. Scala Session considers a node for consolidation only when a commands_changed batch mentions it. change_buffer.flushfeeds consolidation.update, then Consolidate_Execution runs after consolidate_delay, takes the accumulated set (consolidation.flush()), and drops every node that is not yet node_maybe_consolidated (session.scala:677-690; 2026: 807-820). It sends the rest with handle_raw_edits(consolidate = ...).

2. ML Document.update → print_consolidation attaches the consolidation print only if finished_eval node holds (document.ML:778-796; 2026: 802ff). finished_eval requires Execution.snapshot [eval_exec_id] to be empty for every entry (document.ML:727-736; 2026: 751-760). Task groups include descendant groups (Task_Queue.fold_groups), so any task forked anywhere under a command's eval keeps it false. If it is false, the request is dropped silently.

3. Such tasks need not produce any PIDE markup: a plain Future.fork, or — the case we hit in practice — the promise-fulfilment continuation created when a theorem with a forked proof is named. Thm.name_derivation does Future.map fulfill_body (thm.ML:1184; 2026: 1207). That gives a map_future task (future.ML:637) in the forked proof's group, which runs Proofterm.fulfill_norm_proof → consolidate_bodies → Lazy.consolidate (proofterm.ML:274,1997, lazy.ML:136-144). The forked proof's own Execution.fork task has already emitted joined, so Scala counts the command terminated and the node maybe_consolidated.

4. When those tasks finish, no message reaches Scala. No commands_changed mentions the node again, so it is never reconsidered. The node is FINALIZED but never CONSOLIDATED, and Document_Status.Node_Status.quasi_consolidated deliberately excludes finalized nodes (document_status.scala:352; 2026: 355). So headless clients cannot fall back on it.

Possible fixes:

1. ML: when print_consolidation is skipped because finished_eval is false, fork a waiter on the outstanding Execution.snapshot tasks that, on completion, emits a status/protocol message for the node so Scala retries.

2.
Scala: keep nodes whose consolidation was requested but which are still not node_consolidated in the consolidation set and retry after consolidate_delay, possibly with backoff.
3.
Make the promise-fulfilment tail visible in command status (e.g. run the fulfill_body map via Execution.fork), so maybe_consolidated is not reported early.


Last updated: Nov 19 2026 at 01:46 UTC