This change adds persistent workspace indexing, incremental reparsing of edited documents, and first-class support for .def files in the client and server. It also hardens the debugpy attach logic for the Python debug helper and adds Linux-focused VS Code launch/tasks to work around js-debug attach probing.
Summary of important behavior changes
Client
The language client now watches **/*.{tcl,psc,def} (was {tcl,psc}) so .def file changes are observed by the server.
The client passes the workspace storage folder to the server as indexCachePath (when available) so the server can persist its index cache between restarts.
Server / indexing
Introduces a persistent IndexCache (load/save) and wires it into background indexing. The cache is loaded during initialized and saved at the end of background indexing.
Adds an incremental reparse facility and per-document last-parse state (_last_parse) so the server can reparse only the edited parts of a document when possible instead of reparsing whole files.
Keeps navigation definition identities cached and exposes navigation_state() to return both indexes and precomputed definitions so request handlers avoid recomputing definition sets repeatedly.
Avoids unnecessary work in several places (for example, may_contain_classes gates class/method highlighting) so some flows skip work when the file contents or class information make it irrelevant.
.def files and completions
Adds a tools/def_symbols parser to read BLOCK_TEMPLATE/ADDRESS/value candidates from .def files and exposes them to the server.
The server refreshes def symbols on watched .def file changes and indexes them (server state def_symbols) for use in completions.
Completion logic now supports offering quoted .def symbol completions and variable candidates for argument contexts where a .def name or a variable is allowed. A helper _quoted_item produces insert edits that insert quoted strings or replace already-typed quotes.
Additional tooling support: file sourcing reads PSC DefinedEvents and integrates .def resolution; tcl command completion and tcloo completion are extended to make use of indexed symbols and class data when available.
Debug helper and VS Code configurations
The Python debug helper (server/src/_debug_server.py) no longer loops retrying debugpy.connect. Instead it attempts a single connect in a background thread and bounds the wait with join(timeout). The helper prints a clear error and exits (os._exit(1)) when a stale adapter accepts connections but never attaches, or when no listener is present.
Adds Linux-specific VS Code launch configurations and tasks to start the Extension Development Host with the inspector bound to 127.0.0.1:9333 and a variant that sets USE_DEBUGPY/NXPS_DEBUG_HOST/NXPS_DEBUG_PORT to make it easier to attach both the extension host and debugpy on Linux where js-debug probes localhost and ::1 in parallel.
Miscellaneous
Commit updates package-lock.json version.
Tests: tests were added/updated for .def completions, uplevel formatting, and for the debugpy attach behavior (tests changed/added in the diff). No test execution output is included in this change.
Testing
Tests added/changed: server tests for .def completions and uplevel formatting; debug-related tests updated. No test execution results were provided.
Recommended quick checks for reviewers
With a workspace open, start the extension and check extension storage is passed: inspect the server's initialization options to see indexCachePath (the client uses context.storageUri?.fsPath).
Add or edit a .def file and confirm the server refreshes def symbols (watcher triggered) and completion requests in affected TCL files return .def-derived candidates. Also verify quoted insertion behavior for name arguments.
Trigger background indexing and confirm the server loads the index cache (if present) and saves it after indexing (look for the log message mentioning background indexing completed and the load/save logic).
On Linux, use the provided task/launch combos to start the dev host with the inspector on 127.0.0.1:9333 and the Python debug helper on 127.0.0.1:5678 to exercise the new debug helper behavior.
Compatibility, migration, and reviewer notes
The index cache is optional and only used when a workspace storage folder is available: the client sends indexCachePath only when context.storageUri exists. Workspaces without storage will continue to operate without persistence.
The persistent cache is stored under the provided indexCachePath. Reviewers should consider whether the storage location and lifecycle are acceptable for the extension's persisted files.
The debug helper now exits with a non-zero condition when it believes a stale adapter is holding the port. This is intentional to avoid leaving background debugpy threads alive; verify any debug/test automation accounts for that behavior.
The change touches many internal parsing/indexing code paths (incremental parsing, navigation caches). The description above summarizes the end-state behavior rather than the commit sequence.
If you want, I can point to specific changed files for each area (e.g. server/src/lsp_server.py, server/src/lsp_tclserver.py, server/src/_debug_server.py, client/src/common/server.ts, .vscode/launch.json, .vscode/tasks.json, and tools/def_symbols.py) to help review particular implementations.
This change adds persistent workspace indexing, incremental reparsing of edited documents, and first-class support for .def files in the client and server. It also hardens the debugpy attach logic for the Python debug helper and adds Linux-focused VS Code launch/tasks to work around js-debug attach probing.
Summary of important behavior changes
- Client
- The language client now watches `**/*.{tcl,psc,def}` (was `{tcl,psc}`) so .def file changes are observed by the server.
- The client passes the workspace storage folder to the server as `indexCachePath` (when available) so the server can persist its index cache between restarts.
- Server / indexing
- Introduces a persistent `IndexCache` (load/save) and wires it into background indexing. The cache is loaded during `initialized` and saved at the end of background indexing.
- Adds an incremental reparse facility and per-document last-parse state (`_last_parse`) so the server can reparse only the edited parts of a document when possible instead of reparsing whole files.
- Keeps navigation definition identities cached and exposes navigation_state() to return both indexes and precomputed definitions so request handlers avoid recomputing definition sets repeatedly.
- Avoids unnecessary work in several places (for example, `may_contain_classes` gates class/method highlighting) so some flows skip work when the file contents or class information make it irrelevant.
- .def files and completions
- Adds a `tools/def_symbols` parser to read BLOCK_TEMPLATE/ADDRESS/value candidates from .def files and exposes them to the server.
- The server refreshes def symbols on watched .def file changes and indexes them (server state `def_symbols`) for use in completions.
- Completion logic now supports offering quoted `.def` symbol completions and variable candidates for argument contexts where a .def name or a variable is allowed. A helper `_quoted_item` produces insert edits that insert quoted strings or replace already-typed quotes.
- Additional tooling support: file sourcing reads PSC DefinedEvents and integrates .def resolution; tcl command completion and tcloo completion are extended to make use of indexed symbols and class data when available.
- Debug helper and VS Code configurations
- The Python debug helper (`server/src/_debug_server.py`) no longer loops retrying `debugpy.connect`. Instead it attempts a single connect in a background thread and bounds the wait with `join(timeout)`. The helper prints a clear error and exits (`os._exit(1)`) when a stale adapter accepts connections but never attaches, or when no listener is present.
- Adds Linux-specific VS Code launch configurations and tasks to start the Extension Development Host with the inspector bound to `127.0.0.1:9333` and a variant that sets `USE_DEBUGPY`/`NXPS_DEBUG_HOST`/`NXPS_DEBUG_PORT` to make it easier to attach both the extension host and debugpy on Linux where js-debug probes `localhost` and `::1` in parallel.
- Miscellaneous
- Commit updates `package-lock.json` version.
- Tests: tests were added/updated for `.def` completions, uplevel formatting, and for the debugpy attach behavior (tests changed/added in the diff). No test execution output is included in this change.
Testing
- Tests added/changed: server tests for `.def` completions and uplevel formatting; debug-related tests updated. No test execution results were provided.
Recommended quick checks for reviewers
1. With a workspace open, start the extension and check extension storage is passed: inspect the server's initialization options to see `indexCachePath` (the client uses `context.storageUri?.fsPath`).
2. Add or edit a `.def` file and confirm the server refreshes def symbols (watcher triggered) and completion requests in affected TCL files return `.def`-derived candidates. Also verify quoted insertion behavior for name arguments.
3. Trigger background indexing and confirm the server loads the index cache (if present) and saves it after indexing (look for the log message mentioning background indexing completed and the load/save logic).
4. On Linux, use the provided task/launch combos to start the dev host with the inspector on `127.0.0.1:9333` and the Python debug helper on `127.0.0.1:5678` to exercise the new debug helper behavior.
Compatibility, migration, and reviewer notes
- The index cache is optional and only used when a workspace storage folder is available: the client sends `indexCachePath` only when `context.storageUri` exists. Workspaces without storage will continue to operate without persistence.
- The persistent cache is stored under the provided `indexCachePath`. Reviewers should consider whether the storage location and lifecycle are acceptable for the extension's persisted files.
- The debug helper now exits with a non-zero condition when it believes a stale adapter is holding the port. This is intentional to avoid leaving background debugpy threads alive; verify any debug/test automation accounts for that behavior.
- The change touches many internal parsing/indexing code paths (incremental parsing, navigation caches). The description above summarizes the end-state behavior rather than the commit sequence.
If you want, I can point to specific changed files for each area (e.g. `server/src/lsp_server.py`, `server/src/lsp_tclserver.py`, `server/src/_debug_server.py`, `client/src/common/server.ts`, `.vscode/launch.json`, `.vscode/tasks.json`, and `tools/def_symbols.py`) to help review particular implementations.
Christoph
added 12 commits 2026-09-24 05:53:51 +00:00
Introduce several changes to reduce full-document reparses, lock contention and
redundant work when handling TclOO analysis and navigation:
- Add a cheap may_contain_classes pre-check and several cursor/receiver
heuristics so completions, signature help and tcloo definitions skip the
expensive marker reparse when the document cannot contain useful OO info.
- Allow passing an existing parsed tree into tcloo completion/signature/definition
helpers; update callers to use the server's cached tree when available.
- Use a thread-local parser for request-time parse_source to avoid blocking the
shared parser during background indexing, and add navigation_state() which
returns cached definition identities (invalidated on index generation changes).
- Add a cheap name pre-filter (_may_resolve_to) for symbol matching and only
run class highlighting when classes may exist.
These changes reduce contention and repeated parsing, improve responsiveness for
requests during background indexing, and cache navigation definition identities.
Tests were added/updated to assert caching and non-blocking behavior.
- Pass extension storage path to the server (client/ changes) so the
server can persist a workspace index.
- Introduce IndexCache (server/tools/index_cache.py) and load/save it on
initialization and after background indexing. Index entries are stored
only when the file's stat hasn't changed while being read.
- Add incremental reparse logic (server/tools/incremental_parse.py) and
use a per-file _last_parse cache in the language server to reparse only
the top-level Tcl commands touched by an edit, falling back to a full
parse when necessary.
- Use a new _FileIndex dataclass and _build_file_index helper to unify
what is stored/loaded for a file; update update_poco_completion_for_file
to use the persistent cache for disk-read files (from_disk/source_stat).
- Keep background indexing non-blocking and persist the index at the
end of the run. Add basic unit tests for incremental parse and index cache.
Before: edits and background work always required full parsing of files
and no persistent cross-restart index. After: some edits reuse previous
ASTs and files read from disk can use a persisted index to skip
re-indexing across restarts.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
This change adds persistent workspace indexing, incremental reparsing of edited documents, and first-class support for .def files in the client and server. It also hardens the debugpy attach logic for the Python debug helper and adds Linux-focused VS Code launch/tasks to work around js-debug attach probing.
Summary of important behavior changes
Client
**/*.{tcl,psc,def}(was{tcl,psc}) so .def file changes are observed by the server.indexCachePath(when available) so the server can persist its index cache between restarts.Server / indexing
IndexCache(load/save) and wires it into background indexing. The cache is loaded duringinitializedand saved at the end of background indexing._last_parse) so the server can reparse only the edited parts of a document when possible instead of reparsing whole files.may_contain_classesgates class/method highlighting) so some flows skip work when the file contents or class information make it irrelevant..def files and completions
tools/def_symbolsparser to read BLOCK_TEMPLATE/ADDRESS/value candidates from .def files and exposes them to the server.def_symbols) for use in completions..defsymbol completions and variable candidates for argument contexts where a .def name or a variable is allowed. A helper_quoted_itemproduces insert edits that insert quoted strings or replace already-typed quotes.Debug helper and VS Code configurations
server/src/_debug_server.py) no longer loops retryingdebugpy.connect. Instead it attempts a single connect in a background thread and bounds the wait withjoin(timeout). The helper prints a clear error and exits (os._exit(1)) when a stale adapter accepts connections but never attaches, or when no listener is present.127.0.0.1:9333and a variant that setsUSE_DEBUGPY/NXPS_DEBUG_HOST/NXPS_DEBUG_PORTto make it easier to attach both the extension host and debugpy on Linux where js-debug probeslocalhostand::1in parallel.Miscellaneous
package-lock.jsonversion..defcompletions, uplevel formatting, and for the debugpy attach behavior (tests changed/added in the diff). No test execution output is included in this change.Testing
.defcompletions and uplevel formatting; debug-related tests updated. No test execution results were provided.Recommended quick checks for reviewers
indexCachePath(the client usescontext.storageUri?.fsPath)..deffile and confirm the server refreshes def symbols (watcher triggered) and completion requests in affected TCL files return.def-derived candidates. Also verify quoted insertion behavior for name arguments.127.0.0.1:9333and the Python debug helper on127.0.0.1:5678to exercise the new debug helper behavior.Compatibility, migration, and reviewer notes
indexCachePathonly whencontext.storageUriexists. Workspaces without storage will continue to operate without persistence.indexCachePath. Reviewers should consider whether the storage location and lifecycle are acceptable for the extension's persisted files.If you want, I can point to specific changed files for each area (e.g.
server/src/lsp_server.py,server/src/lsp_tclserver.py,server/src/_debug_server.py,client/src/common/server.ts,.vscode/launch.json,.vscode/tasks.json, andtools/def_symbols.py) to help review particular implementations.