Weekly development log 2026‑W36: Read Korean government documents, fit the new local model, close the security review, launch Bench · partial week

01

What changed

From August 31 through September 6 at 00:10 KST, GitHub main added 79 commits. On Thursday, in a separate repository, OpenMake Bench was built and put up at bench.openmake.cc: a measurement tool that runs the same prompts across models and routes under identical conditions, and benchmarks your own models through chat account SSO. The previous week's leftovers, 1.35.0 and 1.35.1, were cut in the small hours of Monday, and twelve releases followed, 1.36.0 through 1.44.0. Three threads overlapped. HWP, HWPX, and HML documents, which the app accepted but could not read, are now extracted, and that work exposed a context-overflow error presented as a transient failure and a token underestimation, both fixed alongside. The local default model changed to qwen3.8-27b, local models are now discovered from the gateway instead of a static catalog, and the sampling defaults, reasoning preservation, and image-count issues found while checking the new model were handled in sequence. On the external side, hasa and B.AI arrived as BYOK providers, Discussion and Deep Research gained per-provider concurrency and 429 backoff so a free key's per-minute limit no longer collapses them, and the three-step reasoning-effort control is actually forwarded to external models. In between, three high, one medium, and many low findings from an apps/api security review were closed over two days, the routing golden set grew to 150 cases and routing accuracy went from 77.5% to 100%, and a nightly real-model evaluation was put on the operations Mac. From Thursday evening to Saturday midnight, work of a different character followed. Six Deep Research consistency defects were closed at once, the language drift that answered Korean questions in English was caught on two fronts, and the same Instrument design tokens were laid down across web, iOS, and Bench. The socket problem that lost the in-flight response when a tab was switched was fixed with stream detach/resume, and in the prompt-trimming work the first post-deployment measurement reversed the design twice in a row. On the site, a Simplified Chinese locale went up, making four languages.

• Korean government documents are readable. .hwp and .hwpx had long been on the upload allowlist but on no extractor list, so the app took them in and never read them. The pure-JS parser kordoc now extracts HWP 3.x/5.x, HWPX, and HWPML, .hml uploads are open, and kordoc is baked into the agent-task sandbox image at build time so tasks can also generate documents and fill templates (1.36.0). Two layered defects appeared right after deployment: the composer's raw-upload list lacked the hwp family, so a 363KB document went up as garbled text and blew past the context window, and that failure was reported as a transient external-provider error that retrying could never fix. The upload list is now paired with the backend extraction list, and context overflow is a separate CONTEXT_TOO_LARGE code that tells the user in seven locales to reduce attachments and does not suggest a retry.
• A blind spot in token estimation was measured and narrowed. Character-based estimation is conservative for Korean prose and code, but underestimates JSON logs at 61%, base64 at 36%, and hex at 30%, letting them through the context-fit safety net untouched. After confirming that weight tuning could not cover this, requests whose estimate exceeds half the effective context are recounted exactly through vLLM /tokenize, and the correction factor is applied to truncation as well. Ordinary conversations never reach the threshold, so they make no extra call. The same pass removed a latent defect where the turn-budget decision ignored HWP extensions.
• The local default model moved from qwen3.6-35b-a3b to qwen3.8-27b (1.38.0). After the DGX swapped models, the app stayed bound to the old name and its UI, catalog, and capabilities drifted from reality, because one static catalog line was the only source of the list. Boot and periodic probes now read LiteLLM /model/info to discover local models, and the static list remains only as a fallback when the gateway is unreachable. Following the new-model check, local calls that specify no sampling get the official recommended values for thinking ON and OFF, ten meta calls such as Deep Research sub-calls, discussion participants, and structured answers pass think:false explicitly, and the fallback for an unspecified reasoning level dropped from maximum to medium. In the tool loop, the assistant's reasoning_content is preserved into the next turn, ending the waste of re-thinking before every round across five tool-call turns; vLLM-issued tool_call ids are no longer overwritten with synthetic ones; and the per-request prompt image total is capped at vLLM's limit of 8, closing the path where the ninth image returned a 400.
• External models became genuinely usable. Open AI Service Hub (hasa) and B.AI joined as BYOK providers; for B.AI, all 45 models were called with a zero-balance key and only the five free ones went into the catalog (1.38.0, 1.39.0). When an explicitly chosen external model cannot run in Discussion or Deep Research, the silent fallback to local was replaced with a visible error, and a composer notice that wrongly claimed external models were ignored was corrected. Then five parallel expert calls ended in 5/5 429s on a B.AI free key and 3/5 on a hasa developer key. The external execution client is now wrapped in a Proxy with a per-provider semaphore and exponential 429 backoff that honors Retry-After, Deep Research fan-out concurrency and timeouts follow the provider hint, the SDK's blind retries were set to zero, and its timeout got a multiplier (1.40.0). The three-step reasoning-effort control had only worked for local and ChatGPT OAuth; it is now forwarded as reasoning_effort to OpenAI-compatible external providers, cutting measured reasoning tokens from 264 to 61. A heuristically inferred capability cache that hid measured config values and rejected vision and thinking was fixed, along with the effort map for glm-5.3, which rejects medium with a 400.
• The findings of the September 2 apps/api security review were closed over two days (1.39.0, 1.40.0). The three high findings were stored prompt injection through system skills any authenticated user could overwrite, a path that let anyone assign their own skill to a shared industry agent, and push subscriptions registered under another userId that received copies of that user's notifications. Pinning the push owner to req.user also closed a defect where web push had in fact never been delivered. Next came an IDOR that installed anyone's private bundle given its id, enforcement of the high-risk MCP tool role gate on the execution path rather than only at exposure, CSV formula injection, an existence oracle on other users' MCP server status, ownership checks that passed on empty values, and a race between multiple administrators during first-run setup. As defense in depth, {{env.KEY}} positional-argument secrets are passed as sh variable references instead of being baked into argv, so another account on the same host cannot read them with ps, and REST image limits, a push endpoint host allowlist, and realpath-based symlink escape blocking in the user sandbox were added.
• Sandbox and MCP operating posture was firmed up. The secure-by-default recommendation from an external architecture review was met without changing the code default, through a boot-time posture check and automatic activation during first-run setup, so that a deployment without docker does not lose every external MCP server with no visible cause. When the readonly-rootfs rollout killed exactly the two uvx-based servers, the cause was traced to the tool venv path and moved to a cache volume, and orphaned containers that survived three app restarts now carry role, pid, and serverId labels so boot reaps only those whose owning process is dead (1.37.0). Task sandboxes that must survive for resumption stay out of scope through the labels. The MCP v1 SDK was removed after confirming the deployed dist held no reference to it.
• Evaluation became a gate. Growing the golden set from 50 to 150 cases with an anonymized reflection of 60 days of production traffic dropped routing accuracy from 93.3% to 77.5%, and that gap was the router's improvement backlog. The 27 failures were diagnosed as 17 missing vocabulary and 10 topic-boost misfires, 70 keywords were added across 24 agents, the pattern that sent "짜줘" to programming was narrowed, accuracy reached 100%, and the threshold was ratcheted from 0.5 to 0.9 (1.37.1). Running a nightly real-model evaluation on the operations Mac exposed label defects, where a legitimate refusal tripped a banned phrase and a Korean refusal missed English markers, which were fixed, while a model answering in fake Klingon was kept as a failure because it is a real quality signal (1.37.2). The daily routing and TTFT aggregation, which had produced empty reports every day since the log path moved, was restored through OMK_LOG_DIR; the CI gate job limit rose to 35 minutes after all 16 stages succeeded only to be cancelled at 25; and a missing release-note entry was corrected by hand with the lesson recorded in CLAUDE.md. Tool-loop reasoning preservation and local-provider responseFormat forwarding shipped in 1.40.1. After it, #729 fixed local-model probe demotion sticking until restart and the external-model cache dropping capabilitiesInferred.
• OpenMake Bench now stands beside chat. It was built in a separate repository, openmake_bench, in four commits on September 4. The measurement tool runs the same prompts across several models and three routes (direct, gateway, OpenMake) sequentially at temperature 0 with a fixed seed and three repetitions, and folds rule and judge scoring together with TTFT and tokens per second into one report. On top of it sit a blind Bradley-Terry leaderboard, a Fit Score recommendation for models that run on your own hardware, and a decision engine that applies the model you choose to your OpenMake role settings. It was renamed from OpenMake Compare to Bench in the morning and deployed to bench.openmake.cc in the afternoon. Sign-in is a redirect SSO through OpenMake: on login Bench uses the user's JWT to issue an API key named bench, stores it AES-GCM encrypted, and calls the catalog and runs as that user's account, BYOK included. On the openmake_llm side this needed an authorize endpoint that hands a one-time exchange code to clients registered in SSO_CLIENTS, a /api/v1/models cache that queries providers live when it expires, and an X-OpenMake-Raw mode that calls only the requested model with no agent router, persona, tools, summarization, or local fallback (#732). Bench's OpenMake route uses raw mode by default, so usage is exact, and pipeline overhead is measured separately as an option. Execution allows one run per capacity domain: local engines fully serial, external providers parallel per provider. In the evening it received the same colors, type, and favicon as chat, and chat's sidebar account menu and login screen gained bench links, so a signed-in user crosses over through the SSO path without logging in again.
• The same day, chasing why B.AI glm-5.3-flash returned 400 on the second turn of a tool loop uncovered two layers of operational defect. The cause was reasoning_effort: medium (B.AI replay: medium alone failed 2 of 5, high and unspecified 0), and the app was configured to raise this model's medium to high, but LLM_REASONING_EFFORTS_JSON replaced the code defaults wholesale, so the stale value pm2 still held had erased that entry. The env map now merges entry by entry over the defaults. And edits to .env were not reaching restarts: PM2 injects the env snapshot from the first start, --update-env reads only the shell env, and the deploy shell never sourced .env. Deploy now parses .env line by line and exports it before restarting (#731), and the first deployment of that fix exposed the export colliding with the script's readonly COMPOSE_FILE constant, which let set -e end the deploy silently just before the restart (#733). Everything from #729 through these afternoon commits is unreleased at this snapshot.
• Six Deep Research consistency defects were closed at once (1.41.0). A depth table that existed separately in two places and disagreed was merged into one, the needsMore decision was made to read config instead of a hardcoded value, REST cancellation was wired to the actual run it had never reached, the CAPACITY label was corrected, configure_research was made to respect user isolation, and a wrong default was removed from the schema. In the same pass, the current date is injected into the decomposition prompt and search operators are forbidden: a model that asks for "latest" without knowing today's date freezes its query in the past, and mixing operators makes the search engine return zero results. Valid-subtopic acceptance was made lenient to reduce cases where a decomposition failure stalls the whole run (#739, #740). Before dawn the next day, leftover dead code from #740 was removed, and deployment now fetches tags before the build so build-info's gitTag no longer disagrees with the actual tag (#742).
• The drift of answering in English a question asked in Korean was caught on two fronts. One is where a tool result comes back in English and the model is pulled toward that language; a response-language reminder is now appended after tool results and the answer-language guard was made observable (#743). The other was in detection itself: when code identifiers are mixed into a question, the share of Latin characters rises and a Korean question was misjudged as English. Language-detection preprocessing now strips code identifiers first (#745).
• Tavily (112) and Context7 (114) were seeded into the MCP catalog. Both servers are registered over stdio with pinned versions, and {{env.KEY}} positional-argument secrets keep the existing approach of being passed as sh variable references rather than baked into argv (#737, #741).
• External-provider error classification was touched twice. B.AI's two deepseek models were opened for free and an insufficient-balance response was classified as INSUFFICIENT_CREDIT (#735), and a 403 from provider policy was separated as MODEL_ACCESS_RESTRICTED with the empty reason key on the fallback badge filled in (#736). On Friday, NVIDIA NIM's fallbackModels were replaced with currently served models, three of the four having gone EOL with a 410 or vanished from the list (1.43.1).
• The Instrument design system was laid down across web, iOS, and Bench: a token set with cobalt as primary, cyan as secondary, and unified type (1.42.0, #753). Mobile handling went in the same day: the input font was raised to 16px to stop iOS from zooming the screen on focus, and icon-button hit areas were widened to 36px for touch pointers (#748). On Bench, the grid where cards ran off-screen because of their content minimum width was changed to minmax(0,1fr), with 40px tap targets and safe-area applied.
• The problem where switching tabs or backgrounding the app dropped the socket and lost the in-flight response was fixed. The stream is detached and resumed on reconnect, so the part generated while disconnected comes back too (1.44.0, #754). In agent tasks, a defect where a user's explicit approval policy was overwritten with 'all' on local execution was fixed (#751): the kind of defect where a deliberately narrow approval boundary quietly widens.
• In the prompt-trimming work, the first post-deployment measurement reversed the design twice in a row. The first round stopped exposing unreferenced MCP servers in the prompt, gated low-frequency tools by intent, and capped skill injection (#755). Measured after deployment, one large manifest skill ate the whole cap and pushed system skills out. A per-skill injection cap of 8000 characters was set (#757), and because some still lost out at the total cap, the order was changed to inject system skills first (#758). A case where the pre-deployment estimate and the post-deployment measurement diverged.
• A Simplified Chinese locale went up on the site. The app UI was already in four languages, but the site was only Korean, English, and Japanese, so there was no page to open from China. Nine page copies, 16 release notes, and the header, footer, and metadata were translated, and routing, hreflang, and the sitemap were connected. The weekly log is written only in Korean, English, and Japanese, so /zh/blog has a Chinese table of contents while the body shows the English original as-is. The same day a Chinese README went up in the repository, and personal addresses and an unused domain left in the public repo's test fixtures were unified to three openmake.cc addresses.

02

How this week was reconstructed

No matching local Claude Code project transcript was recovered for this period. This entry therefore describes only what the Git history can prove.

We audited the available Claude Code main sessions, their proven child lineages, and the repository history. Session notes explain intent and investigation; Git remains the authority for code that actually landed.

This week is still in progress. Counts and claims stop at the GitHub main snapshot of September 6, 2026, 00:10 KST (ed74251e).

03

Team identity and project roles

OpenMake Team is the community identity; openmake_llm is the software project; OpenMake is the product name. Non-developer maker riskpw leads openmake_llm through vibe coding, and professional developer rocky supports its development.