AI progress: honest milestone bar from closed JSON contract units

Replace the char-count estimate (EST_OUTPUT_CHARS) with ContractProgress:
points are earned only when real output units complete — first token (5),
each closed contract key / array item (5-90), root closed (95), stored (100).
Reasoning/thinking streams no longer fake progress: the first thinking
delta fires the first-token milestone ('the model is alive') and holds.
Surface the job's tone through ProcessingJobOut/JobInfo so the bar can
label itself 'Summarizing dry wit…'. Test rewritten to pin the new
honest-thinking semantics.
This commit is contained in:
avi 2026-09-18 15:00:04 -05:00
commit 61f2124b1c
6 changed files with 197 additions and 42 deletions

View file

@ -155,10 +155,12 @@ async def test_openai_compat_streams_with_progress():
assert all(0 <= p <= 99 for p in seen_pcts) # never claims 100 early
async def test_openai_compat_thinking_moves_progress_before_content():
"""qwen3-on-llama.cpp streams reasoning_content for most of the run;
progress must climb during that phase, not sit at "waiting for model"
and then jump straight to done."""
async def test_openai_compat_thinking_earns_only_first_token():
"""qwen3-on-llama.cpp streams reasoning_content for most of the run.
Thinking is real work but NOT contract output: it fires the 5%
first-token milestone once ("the model is alive") and then holds.
Progress only climbs when JSON contract units actually close — a
continuous percent over the thinking span would be a fake estimate."""
import json as _json
def sse(request: httpx.Request) -> httpx.Response:
@ -177,10 +179,14 @@ async def test_openai_compat_thinking_moves_progress_before_content():
seen_pcts = []
res = await p.summarize("t", on_progress=seen_pcts.append)
assert res.short == "Done." # reasoning never enters the reply
# The ticker must report real mid-run values while only reasoning
# streamed — otherwise the UI shows no percentage for the whole job.
assert len(seen_pcts) > 10
assert seen_pcts[-1] > 50
# Thinking earned exactly one tick: the first-token milestone. No
# fake crawl across the reasoning span.
assert seen_pcts[0] == 5
assert seen_pcts.count(5) == 1
assert max(seen_pcts) <= 99 # never claims 100 before storage
# Contract units closing does move the bar to the root milestone.
assert seen_pcts[-1] == 95
assert seen_pcts == sorted(seen_pcts)
async def test_openai_compat_partial_json_gets_defaults():