Compare commits

...
18 Commits
Author SHA1 Message Date
Christoph 6edd2d3a2e Delete .gitea/workflows/recover.yaml
Tests / python (push) Successful in 7s
Tests / node (push) Successful in 9s
2026-09-26 19:53:33 +00:00
Christoph 2bc9122d79 Update .gitea/workflows/recover.yaml 2026-09-25 23:07:26 +00:00
Christoph 6fb849b7c0 Add .gitea/workflows/recover.yaml 2026-09-25 23:06:41 +00:00
Christoph 94bffa1f35 Merge pull request 'Show effective (last-loaded) .def declaration in hover and GoTo' (#50) from enhancements into main 2026-09-25 19:39:33 +00:00
Christoph c8466d2e32 feat(def-navigation): surface last-loaded .def declaration for hover/GoTo
When a block template or address is declared in multiple PSC layers, hover and
Go to Definition now resolve to the effective declaration (the one from the
last-loaded .def file). Find References and Rename still include all
declarations.

- add effective_def_locations(documents, target, uris) helper (returns the last declaration)
- use effective_def_locations in lsp_server for goto/definition endpoints
- change def_hover_markdown to show the effective declaration and append a short
  "_Overrides ..._" note listing overridden declarations
- update changelog and add a test that verifies hover/GoTo reference the last
  loaded declaration while references include all declarations
2026-09-25 21:38:06 +02:00
Christoph e9af7b4bb5 Update version to 2026.9.900 2026-09-25 08:09:15 +00:00
Christoph e539d78902 Merge pull request 'Recognize stored COMMANDBLOCK procs and warn on unknown .def names' (#49) from enhancements into main 2026-09-25 06:46:47 +00:00
Christoph 525a08c289 feat(tools): recognize procedures stored in COMMANDBLOCK properties
Add a new module that parses PostConfigurator COMMANDBLOCK values (CONF_* set ...)
to extract the first word of braced list elements as procedure names with precise
line/column spans.

- server/src/tools/stored_procs.py: implement stored_command_names(command)
  which returns (name, line, column) for static/braced list elements.
- Integrate into navigation (build_file_symbol_index) to index these names as
  non-definitions so Go To Definition / Find References can resolve them.
- Integrate into semantic highlighting to mark known stored procedures as
  functions when appropriate.
- Add tests (server/tests/python_tests/test_stored_procs.py) covering parsing,
  goto-definition, references, and highlighting behavior.
- Update CHANGELOG to note the new capability.

Notes/constraints:
- Only static/braced COMMANDBLOCK values (BracedWord) are considered.
- Names must match the command-name pattern and are taken from the first word
  of each list element.
2026-09-25 08:43:42 +02:00
Christoph 82951a3911 feat(lsp): warn on undeclared block templates and addresses
Add diagnostics that warn when a literal NX command argument names a block
template or address that no loaded .def file declares (diagnostic codes
unknown-block-template and unknown-address). Names computed at runtime
(e.g. $var or "CYCLE_$x") are not checked, and kinds whose .def file is
not loaded are skipped.

The server now recomputes and refreshes diagnostics when .def documents
change (clears cached diagnostics and requests a workspace diagnostic
refresh if the client supports it). Tests cover positive, negative and
invalidated-cache cases.
2026-09-25 08:37:09 +02:00
Christoph dde96506dd Merge pull request 'Recognize derived .def names and show .def previews on completion resolve' (#48) from enhancements into main 2026-09-25 06:17:47 +00:00
Christoph 3f9091c713 ci: add test workflow to run Python and Node jobs
Add a Gitea workflow that runs on pull requests and pushes to main. It defines two jobs:
- Python: sets up Python 3.12, installs pytest, and runs language-server tests under server/tests/python_tests with PYTHONPATH=libs.
- Node: sets up Node 20, installs npm deps (root and client), runs extension tests (node --test test/) and builds the extension (npm run package).

Workflow file added at .gitea/workflows/tests.yml.
2026-09-25 08:15:30 +02:00
Christoph 5b027a2717 feat(def_flow): recognize .def names propagated via variables and wrapper procs
Add a new def_flow analysis module that follows .def block template
and address names through local variables and proc parameters, and a
wrapper-table builder to resolve proc arguments that forward .def names.
Derived names are resolved only for hover/definition (not for rename).

Integrate this into the LSP:
- lsp_server: add _word_at and _tcl_def_symbol helpers; fallback to
  derived_def_symbol when direct NX-argument navigation fails for hover,
  goto-definition and references; return proper ranges for hover.
- lsp_tclserver: cache and expose a def_wrapper_table built from index
  def_flows (with cache invalidation on index generation).

Also add unit tests for def_flow and update CHANGELOG to note hover/
definition and completion preview improvements for derived names.
2026-09-25 08:14:59 +02:00
Christoph 17fb79a346 feat(lsp): support completion item resolve to show .def previews
Enable completion item resolution so selecting a completion can populate
documentation with a preview of the corresponding .def declaration.

- Turn on resolve_provider in the completion options.
- Implement on_completion_resolve to attach Markdown documentation
  computed by def_hover_markdown when the completion item's data
  contains a ("def" -> [kind, name]) payload.
- Include that "def" payload when generating def-related completion items.
- Add tests that verify block-template and address-list items resolve to
  Markdown previews.

Before: def-related completion items had no documentation on resolve.
After: selecting such items will return a Markdown preview of the .def entry.
2026-09-25 07:58:56 +02:00
Christoph 0a8e8a7368 test(completion_context): update test expectations and reflow formatting
Adjust tests to match current completion item labels and clean up formatting:
- Expect "nocomplain" (no leading dash) for the unset option case instead of {"-nocomplain", "--"}.
- Reflow several multi-line source literals into single-line/f-string forms.
- Normalize various assertions and generator expressions onto single lines.
- Reformat TextDocumentItem construction for readability.

These changes are limited to test code and reflect the updated shape/labels of completion items and stylistic cleanup; no production logic is modified.
2026-09-24 23:05:49 +02:00
Christoph bdeb0d946d Update version to 2026.9.801 2026-09-24 21:02:12 +00:00
Christoph bd9c73452e Merge pull request 'Add .def navigation helpers and tests' (#47) from bug_fix into main
build_and_puplish.yml / build_and_publish (release) Successful in 39s
2026-09-24 21:01:33 +00:00
Christoph a393ca3aec feat(tools): add .def navigation helpers and tests
Add tools/def_navigation.py providing utilities to locate .def symbols
and occurrences, produce definition/reference locations, build hover
Markdown for addresses (property table with format links and modality
labels), and compute workspace edits for renames (only when a
declaration exists). Helpers include def_symbol_at, def_definition_locations,
def_reference_locations, tcl_def_occurrences, all_def_target_locations,
def_rename_edits and def_hover_markdown, plus small formatting helpers.

Also add server/tests/python_tests/test_def_navigation.py exercising
go-to-definition, hover, references and rename behavior between Tcl and
.def files, including use of unsaved text and ensuring undeclared names
are not renamed. Tests assert address property ordering, modality labels,
and correct edit ordering for workspace edits.
2026-09-24 23:01:05 +02:00
Christoph 85a8684254 Update version to 2026.9.800 2026-09-24 20:57:15 +00:00
15 changed files with 1527 additions and 161 deletions
+39
View File
@@ -0,0 +1,39 @@
name: Tests
on:
pull_request:
push:
branches: [main]
jobs:
python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install pytest
run: pip install pytest
- name: Run language server tests
working-directory: server
# The server's dependencies are bundled in server/libs.
env:
PYTHONPATH: libs
run: python -m pytest tests/python_tests -q
node:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install NodeJS
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install NPM Packages
run: |
npm ci
cd ./client
npm ci
- name: Run extension tests
run: node --test test/
- name: Build extension
run: npm run package
+8
View File
@@ -7,10 +7,18 @@ Versions correspond to the Git tags of this repository.
### Added
- Procedures stored in PostConfigurator COMMANDBLOCK properties (`CONF_CTRL_tool set auto_preselect_last_template {custom_header}`) are highlighted as procedures and support Go to Definition, Find References, Rename, and the call hierarchy
- Go to Definition from block template and address arguments in Tcl (`MOM_do_template`, `MOM_force`, `MOM_suppress`, `MOM_ask_address_value`, ...) to the `BLOCK_TEMPLATE`/`ADDRESS` declaration in the PSC `.def` files
- Hover over block templates shows the template body; hover over addresses shows format, leader, trailer, min/max, and modality
- Find References and Rename for block templates and addresses across Tcl and `.def` files, including addresses used inside block templates
- Go to Definition, hover, references, and rename also work inside `.def` files
- Hover and Go to Definition also recognize block template and address names that reach an NX command through a local variable (`set`, `lappend`, `list`, `foreach`) or through the parameter of a custom proc such as `LIB_SPF_call_cycle "absolute_mode"`, including nested wrapper procs; same-named strings without such a path are not recognized, and these derived names are not renamed
- Warning for block template and address names in NX commands (`MOM_do_template stedy_rest`) that no loaded `.def` file declares; names built at runtime (`$var`, `"CYCLE_$x"`) are not checked, and the warnings update when a `.def` file changes
- Completion items for block templates and addresses (`BLOCK_LIST`, `ADDR_LIST`, `MOM_do_template`, ...) show the same preview as the hover
### Changed
- When a block template or address is declared in several PSC layers, hover and Go to Definition show only the effective one, the declaration of the last loaded `.def` file, and name the declarations it overrides; Find References and Rename still include all declarations
### Documentation
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "nx-post-support",
"displayName": "NX Postprocessor Support",
"description": "VS Code extension for NX CAM postprocessor development with language support and remote Tcl debugging for CDL, TCL, and DEF files",
"version": "2026.9.701",
"version": "2026.9.900",
"publisher": "Christoph",
"icon": "images/nx-1.png",
"activationEvents": [
+64 -10
View File
@@ -55,11 +55,12 @@ from tools.completion_items import (
from tools.def_navigation import (
all_def_target_locations,
def_declarations,
def_definition_locations,
def_hover_markdown,
effective_def_locations,
def_rename_edits,
def_symbol_at,
)
from tools.def_flow import derived_def_symbol
from tools.def_symbols import parse_def_document
from tools.folding_ranges import build_folding_ranges
from tools.index_cache import IndexCache
@@ -287,7 +288,7 @@ def document_diagnostic(params: lsp.DocumentDiagnosticParams):
@LSP_SERVER.feature(
lsp.TEXT_DOCUMENT_COMPLETION,
lsp.CompletionOptions(trigger_characters=["$", " ", "-", "(", ","]),
lsp.CompletionOptions(trigger_characters=["$", " ", "-", "(", ","], resolve_provider=True),
)
def on_completion(params: lsp.CompletionParams) -> lsp.CompletionList:
result = _on_completion(params)
@@ -300,6 +301,17 @@ def on_completion(params: lsp.CompletionParams) -> lsp.CompletionList:
return result
@LSP_SERVER.feature(lsp.COMPLETION_ITEM_RESOLVE)
def on_completion_resolve(item: lsp.CompletionItem) -> lsp.CompletionItem:
"""Show the .def declaration of a selected block template/address, as on hover."""
target = item.data.get("def") if isinstance(item.data, dict) else None
if item.documentation is None and isinstance(target, list) and len(target) == 2:
markdown = def_hover_markdown(LSP_SERVER.def_documents_snapshot(), tuple(target))
if markdown:
item.documentation = lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=markdown)
return item
# Keywords that expand to all loaded .def names: keyword -> (items, description).
SYMBOL_LIST_KEYWORDS = {
"BLOCK_LIST": (lambda: LSP_SERVER.block_template_items(), "block templates"),
@@ -767,16 +779,53 @@ def _def_target(identity: SymbolIdentity) -> tuple[str, str] | None:
return (identity.kind, identity.name) if identity.kind in DEF_SYMBOL_KINDS else None
def _tcl_def_hover(uri: str, position: lsp.Position, token: str) -> lsp.Hover | None:
def _word_at(uri: str, position: lsp.Position) -> str | None:
document = LSP_SERVER.workspace.get_text_document(uri)
try:
line = LSP_SERVER.get_lines(document)[position.line]
except IndexError:
return None
for match in re.finditer(r"\b\w+\b", line):
if match.start() <= position.character <= match.end():
return match.group(0)
return None
def _tcl_def_symbol(uri: str, position: lsp.Position, token: str | None):
"""The .def target at ``position``: a direct NX command argument or a derived name.
Returns (documents, target, range) or None.
"""
documents = LSP_SERVER.def_documents_snapshot()
if not any(token in document.names(kind) for document in documents.values() for kind in DEF_SYMBOL_KINDS):
# Cheap guard: only names declared in a .def file are analyzed at all.
kinds = {kind for kind in DEF_SYMBOL_KINDS if token and any(token in document.names(kind) for document in documents.values())}
if not kinds:
return None
context = _navigation_context(uri, position)
target = context and _def_target(context[3])
markdown = target and def_hover_markdown(documents, target)
if target:
return documents, target, context[2].range
try:
tree = LSP_SERVER.get_tree(LSP_SERVER.workspace.get_text_document(uri))
except TclSyntaxError:
return None
derived = derived_def_symbol(tree, position, LSP_SERVER.def_wrapper_table())
if derived is None or derived[1] != token:
return None
found, name, range_ = derived
kind = next((kind for kind in sorted(found & kinds) if def_declarations(documents, (kind, name))), None)
return (documents, (kind, name), range_) if kind else None
def _tcl_def_hover(uri: str, position: lsp.Position, token: str) -> lsp.Hover | None:
symbol = _tcl_def_symbol(uri, position, token)
if symbol is None:
return None
documents, target, range_ = symbol
markdown = def_hover_markdown(documents, target)
if not markdown:
return None
return lsp.Hover(lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=markdown), range=context[2].range)
return lsp.Hover(lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=markdown), range=range_)
@LSP_SERVER.feature(lsp.TEXT_DOCUMENT_DEFINITION)
@@ -798,12 +847,13 @@ def goto_definition(params: lsp.DefinitionParams):
return [target]
context = _navigation_context(params.text_document.uri, params.position)
if context is None:
return None
symbol = _tcl_def_symbol(params.text_document.uri, params.position, _word_at(params.text_document.uri, params.position))
return symbol and effective_def_locations(symbol[0], symbol[1]) or None
indexes, definitions, _, identity = context
target = _def_target(identity)
if target is not None:
return _sorted_locations(def_definition_locations(LSP_SERVER.def_documents_snapshot(), target)) or None
return effective_def_locations(LSP_SERVER.def_documents_snapshot(), target) or None
locations = [lsp.Location(uri=index.uri, range=occurrence.range) for index, occurrence in matching_occurrences(identity, indexes, definitions) if occurrence.is_definition]
return _sorted_locations(locations) or None
@@ -842,7 +892,11 @@ def _sorted_locations(locations: list[lsp.Location]) -> list[lsp.Location]:
def references(params: lsp.ReferenceParams) -> list[lsp.Location]:
context = _navigation_context(params.text_document.uri, params.position)
if context is None:
return []
symbol = _tcl_def_symbol(params.text_document.uri, params.position, _word_at(params.text_document.uri, params.position))
if symbol is None:
return []
indexes, _ = LSP_SERVER.navigation_state()
return all_def_target_locations(symbol[0], indexes, symbol[1], params.context.include_declaration)
indexes, definitions, _, identity = context
target = _def_target(identity)
@@ -979,7 +1033,7 @@ def def_definition(params) -> list[lsp.Location] | None:
if context is None:
return None
documents, def_uris, target, _ = context
return _sorted_locations(def_definition_locations(documents, target, def_uris)) or None
return effective_def_locations(documents, target, def_uris) or None
@LSP_SERVER.feature(DEF_REQUEST_HOVER)
+47
View File
@@ -18,6 +18,7 @@ from tools import checks, incremental_parse, parser
from tools.completion_items import CompletionCollector
from tools.tcloo_symbols import class_completion_items
from tools.tcloo_completion import indexed_classes
from tools.def_flow import WrapperTable, build_wrapper_table, unknown_def_names
from tools.def_symbols import ADDRESS, BLOCK_TEMPLATE, DefDocument, parse_def_document, read_def_source
from tools.file_sourcing import get_all_psc_files, psc_defined_event_files, psc_script_files
from tools.formatter import NxFormatter as Formatter
@@ -97,6 +98,7 @@ class TclLanguageServer(LanguageServer):
self._definition_identities_cache: tuple[
int, frozenset[SymbolIdentity]
] = (-1, frozenset())
self._def_wrapper_cache: tuple[int, WrapperTable] = (-1, {})
self._proc_metadata_cache: dict[
str, tuple[int, dict[str, list[str]], dict[str, str]]
] = {}
@@ -213,6 +215,7 @@ class TclLanguageServer(LanguageServer):
self._workspace_completion_cache = (-1, ())
self._custom_function_names_cache = (-1, frozenset())
self._definition_identities_cache = (-1, frozenset())
self._def_wrapper_cache = (-1, {})
self._proc_metadata_cache.clear()
self._custom_inlay_cache.clear()
@@ -273,7 +276,20 @@ class TclLanguageServer(LanguageServer):
except OSError as error:
report(f"Could not read DEF file {def_file}: {error}")
with self._index_lock:
changed = documents != self.def_documents
self.def_documents = documents
if changed:
# Unknown template/address warnings depend on the .def files.
self.diagnostics.clear()
if changed:
self._request_diagnostic_refresh()
def _request_diagnostic_refresh(self) -> None:
# Set by the initialize request; absent before it and in tests.
capabilities = getattr(self.protocol, "client_capabilities", None)
diagnostics = getattr(getattr(capabilities, "workspace", None), "diagnostics", None)
if getattr(diagnostics, "refresh_support", False):
self.workspace_diagnostic_refresh(None)
def def_documents_snapshot(self, current_path=None, current_source: str | None = None) -> dict[str, DefDocument]:
"""Return the PSC .def documents; ``current_source`` replaces the file being edited."""
@@ -290,6 +306,7 @@ class TclLanguageServer(LanguageServer):
label=name,
kind=item_kind,
detail=f"{description} ({pathlib.Path(path).name})",
data={"def": [kind, name]},
)
for path, document in self.def_documents_snapshot().items()
for name in document.names(kind)
@@ -629,6 +646,17 @@ class TclLanguageServer(LanguageServer):
)
return dict(self.navigation_indexes), definitions
def def_wrapper_table(self) -> WrapperTable:
"""Proc arguments that take .def names, cached by index generation."""
with self._index_lock:
generation, table = self._def_wrapper_cache
if generation != self._index_generation:
table = build_wrapper_table(
flow for index in self.navigation_indexes.values() for flow in index.def_flows
)
self._def_wrapper_cache = (self._index_generation, table)
return table
def _begin_index_update(self, filepath: str, version: int | None) -> int | None:
with self._index_lock:
indexed_version = self._index_versions.get(filepath)
@@ -883,8 +911,27 @@ class TclLanguageServer(LanguageServer):
)
)
diagnostics.extend(self._def_diagnostics(document))
return diagnostics
def _def_diagnostics(self, document: TextDocument) -> List[lsp.Diagnostic]:
documents = self.def_documents_snapshot()
declared = {
kind: frozenset(name for def_document in documents.values() for name in def_document.names(kind))
for kind in (BLOCK_TEMPLATE, ADDRESS)
}
labels = {BLOCK_TEMPLATE: "Block template", ADDRESS: "Address"}
return [
lsp.Diagnostic(
message=f"{labels[kind]} '{name}' is not declared in any loaded .def file",
severity=lsp.DiagnosticSeverity.Warning,
range=range_,
code=f"unknown-{kind.replace('_', '-')}",
source=DIAGNOSTIC_SOURCE,
)
for kind, name, range_ in unknown_def_names(self.get_tree(document), declared)
]
def _compute_diagnostics(self, document: TextDocument) -> List[lsp.Diagnostic]:
return self.lint(document)
+364
View File
@@ -0,0 +1,364 @@
"""Follow .def block template and address names through variables and procs.
A name is only a .def symbol where it provably reaches an NX command taking one:
directly as an argument (``MOM_do_template steady_rest``), through a variable
of the same scope (``set t steady_rest; MOM_do_template $t``) or through the
parameter of a proc that passes it on (``LIB_SPF_call_cycle absolute_mode``).
Derived names are resolved for hover and definition only, never renamed.
"""
from __future__ import annotations
import re
from collections import defaultdict
from collections.abc import Iterable, Iterator
import lsprotocol.types as lsp
from tclint.syntax_tree import BracedWord, Command, CommandSub, List, Node, QuotedWord, Script, VarSub
from tools.tcl_command_completion import TCL_COMMAND_NAMES
ROOT_NAMESPACE = "::"
DEF_BLOCK_TEMPLATE = "block_template"
DEF_ADDRESS = "address"
DEF_SYMBOL_KINDS = frozenset({DEF_BLOCK_TEMPLATE, DEF_ADDRESS})
# NX commands taking .def names: command -> (first argument index, last index or
# None for all following arguments, kind).
_DEF_ARGUMENTS: dict[str, tuple[tuple[int, int | None, str], ...]] = {
"MOM_do_template": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_add_to_block_buffer": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_polar_motion": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_force_block": ((1, None, DEF_BLOCK_TEMPLATE),),
"MOM_ask_address_value": ((0, 0, DEF_ADDRESS),),
"MOM_add_to_address_buffer": ((0, 0, DEF_ADDRESS),),
"MOM_enable_address": ((0, None, DEF_ADDRESS),),
"MOM_disable_address": ((0, None, DEF_ADDRESS),),
"MOM_force": ((1, None, DEF_ADDRESS),),
"MOM_suppress": ((1, None, DEF_ADDRESS),),
"MOM_incremental": ((1, None, DEF_ADDRESS),),
}
_DEFINITION_ELEMENT_COMMANDS = frozenset({"MOM_ask_definition_element", "MOM_has_definition_element"})
_DEFINITION_ELEMENT_KINDS = {"ADDRESS": DEF_ADDRESS, "BLOCK": DEF_BLOCK_TEMPLATE}
# Commands that never pass a .def name on to a proc parameter.
_NON_FORWARDING = (
frozenset(TCL_COMMAND_NAMES)
| frozenset(_DEF_ARGUMENTS)
| _DEFINITION_ELEMENT_COMMANDS
| frozenset({
"set", "unset", "puts", "expr", "return", "incr", "append", "lappend", "list", "lindex", "lrange",
"llength", "lsearch", "lsort", "lreverse", "lassign", "concat", "join", "split", "format", "regsub",
"regexp", "string", "if", "while", "for", "foreach", "lmap", "switch", "catch", "eval", "uplevel",
"upvar", "global", "variable", "info", "array", "dict", "subst", "error", "proc", "namespace",
})
)
# Commands returning (elements of) their first argument's list value.
_LIST_ACCESSORS = frozenset({"lindex", "lrange", "lsort", "lreverse", "lsearch"})
_LIST_WORD_RE = re.compile(r'"([^"\s{}]*)"|([^\s"{}]+)')
# ("def", kind) or ("call", routine, argument index) where a variable ends up.
FlowTarget = tuple
# Fact: ("sink", variable, target) or ("edge", destination, source variable).
FlowFact = tuple
# Per proc: ((parameter index, ("def", kind) | ("call", qualified, fallback, index)), ...)
ProcDefFlows = tuple[tuple[int, tuple], ...]
WrapperTable = dict[str, dict[int, frozenset[str]]]
def static_contents(node: Node | None) -> str | None:
value = getattr(node, "contents", None)
return value if isinstance(value, str) else None
def def_argument_kinds(command: Command) -> list[tuple[Node, str]]:
"""Return the arguments of ``command`` that name a .def block template or address."""
routine = static_contents(command.routine)
if routine in _DEFINITION_ELEMENT_COMMANDS:
kind = _DEFINITION_ELEMENT_KINDS.get((static_contents(command.args[0]) or "").upper()) if command.args else None
return [(command.args[1], kind)] if kind and len(command.args) >= 2 else []
result = []
for first, last, kind in _DEF_ARGUMENTS.get(routine or "", ()):
for position, argument in enumerate(command.args):
if position >= first and (last is None or position <= last):
result.append((argument, kind))
return result
def def_name(node: Node) -> str | None:
name = static_contents(node)
if not name or node.contents_pos is None or any(char.isspace() or char in "$[]{}\\\"" for char in name):
return None
return name
def qualify(name: str, namespace: str) -> str:
if name.startswith("::"):
return name
return f"::{name}" if namespace == ROOT_NAMESPACE else f"{namespace}::{name}"
def _variable_reference(node: Node) -> str | None:
"""Name of the scalar variable ``node`` consists of: ``$v`` or ``"$v"``."""
if isinstance(node, QuotedWord) and len(node.children) == 1:
node = node.children[0]
if isinstance(node, VarSub) and isinstance(node.value, str) and "(" not in node.value:
return node.value
return None
def _value_sources(node: Node) -> list[str]:
"""Variables whose value or list elements ``node`` copies."""
variable = _variable_reference(node)
if variable is not None:
return [variable]
if isinstance(node, CommandSub) and len(node.children) == 1 and isinstance(node.children[0], Command):
inner = node.children[0]
routine = static_contents(inner.routine)
if routine in _LIST_ACCESSORS and inner.args:
return _value_sources(inner.args[0])
if routine in {"list", "concat"}:
return [source for argument in inner.args for source in _value_sources(argument)]
return []
def _bound_names(node: Node) -> list[str]:
nodes = node.children if isinstance(node, List) else [node]
names = [static_contents(child) for child in nodes]
if len(names) == 1 and names[0] and " " in names[0]:
return names[0].split()
return [name for name in names if name]
def command_flow_facts(command: Command) -> list[FlowFact]:
"""Facts on how ``command`` moves variable values towards .def arguments."""
routine = static_contents(command.routine)
args = command.args
facts: list[FlowFact] = [
("sink", variable, ("def", kind))
for node, kind in def_argument_kinds(command)
if (variable := _variable_reference(node)) is not None
]
if routine == "set" and len(args) == 2:
destination = static_contents(args[0])
if destination:
facts.extend(("edge", destination, source) for source in _value_sources(args[1]))
elif routine == "lappend" and args:
destination = static_contents(args[0])
if destination:
facts.extend(("edge", destination, source) for argument in args[1:] for source in _value_sources(argument))
elif routine in {"foreach", "lmap"} and len(args) >= 3:
for position in range(0, len(args) - 1, 2):
sources = _value_sources(args[position + 1])
facts.extend(("edge", name, source) for name in _bound_names(args[position]) for source in sources)
elif routine == "lassign" and args:
sources = _value_sources(args[0])
facts.extend(("edge", name, source) for node in args[1:] if (name := static_contents(node)) for source in sources)
elif routine and routine not in _NON_FORWARDING:
facts.extend(
("sink", variable, ("call", routine, position))
for position, argument in enumerate(args)
if (variable := _variable_reference(argument)) is not None
)
return facts
def solve_flow(facts: Iterable[FlowFact]) -> dict[str, set[FlowTarget]]:
"""Map each variable to the .def arguments and proc parameters it reaches."""
targets: dict[str, set[FlowTarget]] = defaultdict(set)
sources: dict[str, set[str]] = defaultdict(set)
for fact in facts:
if fact[0] == "sink":
targets[fact[1]].add(fact[2])
elif fact[1] != fact[2]:
sources[fact[1]].add(fact[2])
pending = [variable for variable in targets if variable in sources]
while pending:
destination = pending.pop()
for source in sources.get(destination, ()):
before = len(targets[source])
targets[source] |= targets[destination]
if len(targets[source]) != before:
pending.append(source)
return targets
def proc_def_flows(facts: Iterable[FlowFact], parameters: list[str], namespace: str) -> ProcDefFlows:
"""Where the parameters of a proc end up, with qualified callee names."""
targets = solve_flow(facts)
flows = []
for position, parameter in enumerate(parameters):
if parameter == "args" and position == len(parameters) - 1:
break
for target in targets.get(parameter, ()):
if target[0] == "call":
target = ("call", qualify(target[1], namespace), qualify(target[1], ROOT_NAMESPACE), target[2])
flows.append((position, target))
return tuple(sorted(flows))
def build_wrapper_table(procs: Iterable[tuple[str, ProcDefFlows]]) -> WrapperTable:
"""Resolve which proc arguments take .def names, following nested wrappers."""
kinds: dict[str, dict[int, set[str]]] = defaultdict(lambda: defaultdict(set))
calls = []
for proc, flows in procs:
for position, target in flows:
if target[0] == "def":
kinds[proc][position].add(target[1])
else:
calls.append((proc, position, target[1], target[2], target[3]))
changed = True
while changed:
changed = False
for proc, position, callee, fallback, callee_position in calls:
entry = kinds.get(callee) or kinds.get(fallback)
found = entry.get(callee_position) if entry else None
if found and not found <= kinds[proc][position]:
kinds[proc][position] |= found
changed = True
return {
proc: {position: frozenset(names) for position, names in positions.items() if names}
for proc, positions in kinds.items()
if any(positions.values())
}
def _wrapper_kinds(table: WrapperTable, routine: str, position: int, namespace: str) -> frozenset[str]:
entry = table.get(qualify(routine, namespace)) or table.get(qualify(routine, ROOT_NAMESPACE))
return entry.get(position, frozenset()) if entry else frozenset()
def _target_kinds(target: FlowTarget, table: WrapperTable, namespace: str) -> frozenset[str]:
if target[0] == "def":
return frozenset({target[1]})
return _wrapper_kinds(table, target[1], target[2], namespace)
def _scope_commands(script: Node) -> Iterator[Command]:
"""Commands of one scope, without the bodies of procs defined in it."""
for child in getattr(script, "children", []):
if isinstance(child, Command):
yield child
if static_contents(child.routine) == "proc":
continue
yield from _scope_commands(child)
def _contains(node: Node, point: tuple[int, int]) -> bool:
return node.pos is not None and node.end_pos is not None and node.pos <= point < node.end_pos
def _path_at(tree: Node, point: tuple[int, int]) -> list[Node]:
path = [tree]
while True:
child = next((child for child in getattr(path[-1], "children", []) if _contains(child, point)), None)
if child is None:
return path
path.append(child)
def _literal_at(node: Node, point: tuple[int, int]) -> tuple[str, lsp.Range] | None:
"""The single name ``node`` holds, or the list element of a braced word at ``point``."""
if isinstance(node, BracedWord):
contents = static_contents(node)
if contents is None or node.contents_pos is None:
return None
line, column = node.contents_pos
for match in _LIST_WORD_RE.finditer(contents):
start = match.start(1) if match.group(1) is not None else match.start(2)
name = match.group(1) if match.group(1) is not None else match.group(2)
before = contents[:start]
element_line = line + before.count("\n")
element_column = (start - before.rfind("\n") if "\n" in before else column + start)
if element_line == point[0] and element_column <= point[1] < element_column + len(name):
return name, _range(element_line, element_column, name)
return None
name = def_name(node)
if name is None:
return None
line, column = node.contents_pos
return name, _range(line, column, name)
def _range(line: int, column: int, name: str) -> lsp.Range:
return lsp.Range(
start=lsp.Position(line=line - 1, character=column - 1),
end=lsp.Position(line=line - 1, character=column - 1 + len(name)),
)
def _literal_targets(commands: list[Command], values: list[Node]) -> tuple[list[str], list[FlowTarget]]:
"""Variables and proc arguments a literal flows into; ``values`` are its enclosing words."""
command, value = commands[-1], values[-1]
routine = static_contents(command.routine)
args = list(command.args)
position = next((index for index, argument in enumerate(args) if argument is value), None)
if position is None:
return [], []
if routine in {"list", "concat"} and len(commands) >= 2 and isinstance(values[-2], CommandSub):
return _literal_targets(commands[:-1], values[:-1])
if routine == "set" and position == 1:
destination = static_contents(args[0])
return ([destination] if destination else []), []
if routine == "lappend" and position >= 1:
destination = static_contents(args[0])
return ([destination] if destination else []), []
if routine in {"foreach", "lmap"} and position % 2 == 1 and position < len(args) - 1:
return _bound_names(args[position - 1]), []
if routine and routine not in _NON_FORWARDING:
return [], [("call", routine, position)]
return [], []
def derived_def_symbol(
tree: Node, position: lsp.Position, table: WrapperTable
) -> tuple[frozenset[str], str, lsp.Range] | None:
"""Kinds, name and range of a literal that reaches a .def argument indirectly."""
point = (position.line + 1, position.character + 1)
path = _path_at(tree, point)
commands: list[Command] = []
values: list[Node] = []
scope: Node = tree
namespace = ROOT_NAMESPACE
for parent, child in zip(path, path[1:]):
if isinstance(parent, Command):
commands.append(parent)
values.append(child)
if static_contents(parent.routine) == "proc" and len(parent.args) >= 3 and child is parent.args[2]:
scope = child
name = qualify(static_contents(parent.args[0]) or "", ROOT_NAMESPACE)
namespace = name.rsplit("::", 1)[0] or ROOT_NAMESPACE
if not commands or isinstance(values[-1], Script):
return None
literal = _literal_at(values[-1], point)
if literal is None:
return None
variables, targets = _literal_targets(commands, values)
if variables:
scope_targets = solve_flow(fact for command in _scope_commands(scope) for fact in command_flow_facts(command))
targets.extend(target for variable in variables for target in scope_targets.get(variable, ()))
kinds = frozenset(kind for target in targets for kind in _target_kinds(target, table, namespace))
return (kinds, *literal) if kinds else None
def _all_commands(node: Node) -> Iterator[Command]:
for child in getattr(node, "children", []):
if isinstance(child, Command):
yield child
yield from _all_commands(child)
def unknown_def_names(tree: Node, declared: dict[str, frozenset[str]]) -> list[tuple[str, str, lsp.Range]]:
"""Literal NX command arguments naming a block template or address no .def file declares.
``declared`` maps each kind to its declared names; kinds without any
declaration are not checked, since their .def file is not loaded.
"""
unknown = []
for command in _all_commands(tree):
for node, kind in def_argument_kinds(command):
names = declared.get(kind)
name = def_name(node) if names else None
if name is not None and name not in names:
line, column = node.contents_pos
unknown.append((kind, name, _range(line, column, name)))
return unknown
+201
View File
@@ -0,0 +1,201 @@
"""Navigation between Tcl code and the block templates and addresses of .def files."""
from __future__ import annotations
from pathlib import Path
import lsprotocol.types as lsp
from tools.def_symbols import ADDRESS, BLOCK_TEMPLATE, FORMAT, DefDeclaration, DefDocument
from tools.navigation import FileSymbolIndex, SymbolOccurrence
DefTarget = tuple[str, str]
_KIND_LABELS = {BLOCK_TEMPLATE: "Block template", ADDRESS: "Address", FORMAT: "Format"}
# Address properties in display order; others follow as declared.
_ADDRESS_PROPERTIES = (
("FORMAT", "Format"),
("LEADER", "Leader"),
("TRAILER", "Trailer"),
("MIN", "Min"),
("MAX", "Max"),
("FORCE", "Modality"),
("ZERO_FORMAT", "Zero format"),
("INCREMENTAL", "Incremental"),
("OMIT", "Omit"),
)
_MODALITY = {
"OFF": "modal, output only on change",
"ONCE": "output once, then modal",
"ALWAYS": "non-modal, always output",
}
def _range(line: int, start: int, end: int) -> lsp.Range:
return lsp.Range(
start=lsp.Position(line=line, character=start),
end=lsp.Position(line=line, character=end),
)
def _contains(line: int, start: int, end: int, position: lsp.Position) -> bool:
return position.line == line and start <= position.character <= end
def def_symbol_at(document: DefDocument, position: lsp.Position) -> tuple[DefTarget, lsp.Range, bool] | None:
"""Return the block template or address at ``position`` of a .def document.
The flag tells whether the position is on a declaration.
"""
for declaration in document.declarations:
if declaration.kind != FORMAT and _contains(declaration.line, declaration.start, declaration.end, position):
return (declaration.kind, declaration.name), _range(declaration.line, declaration.start, declaration.end), True
for reference in document.references:
if _contains(reference.line, reference.start, reference.end, position):
return (reference.kind, reference.name), _range(reference.line, reference.start, reference.end), False
return None
def _uri(path: str, uris: dict[str, str]) -> str:
return uris.get(path) or Path(path).as_uri()
def def_declarations(documents: dict[str, DefDocument], target: DefTarget) -> list[tuple[str, DefDeclaration]]:
kind, name = target
return [
(path, declaration)
for path, document in documents.items()
for declaration in document.declarations
if declaration.kind == kind and declaration.name == name
]
def def_definition_locations(
documents: dict[str, DefDocument], target: DefTarget, uris: dict[str, str] | None = None
) -> list[lsp.Location]:
uris = uris or {}
return [
lsp.Location(uri=_uri(path, uris), range=_range(declaration.line, declaration.start, declaration.end))
for path, declaration in def_declarations(documents, target)
]
def effective_def_locations(
documents: dict[str, DefDocument], target: DefTarget, uris: dict[str, str] | None = None
) -> list[lsp.Location]:
"""The declaration NX uses: ``documents`` follow the PSC layer order, the last one wins."""
return def_definition_locations(documents, target, uris)[-1:]
def def_reference_locations(
documents: dict[str, DefDocument],
target: DefTarget,
include_declaration: bool,
uris: dict[str, str] | None = None,
) -> list[lsp.Location]:
"""Return .def occurrences: declarations and addresses used in block templates."""
kind, name = target
uris = uris or {}
locations = def_definition_locations(documents, target, uris) if include_declaration else []
for path, document in documents.items():
for reference in document.references:
if reference.kind == kind and reference.name == name:
locations.append(
lsp.Location(uri=_uri(path, uris), range=_range(reference.line, reference.start, reference.end))
)
return locations
def tcl_def_occurrences(
indexes: dict[str, FileSymbolIndex], target: DefTarget
) -> list[tuple[FileSymbolIndex, SymbolOccurrence]]:
kind, name = target
return [
(index, occurrence)
for index in indexes.values()
for occurrence in index.occurrences
if occurrence.identity.kind == kind and occurrence.identity.name == name
]
def all_def_target_locations(
documents: dict[str, DefDocument],
indexes: dict[str, FileSymbolIndex],
target: DefTarget,
include_declaration: bool,
uris: dict[str, str] | None = None,
) -> list[lsp.Location]:
locations = def_reference_locations(documents, target, include_declaration, uris)
locations.extend(
lsp.Location(uri=index.uri, range=occurrence.range) for index, occurrence in tcl_def_occurrences(indexes, target)
)
unique = {}
for location in locations:
key = (location.uri, location.range.start.line, location.range.start.character)
unique.setdefault(key, location)
return sorted(unique.values(), key=lambda item: (item.uri, item.range.start.line, item.range.start.character))
def def_rename_edits(
documents: dict[str, DefDocument],
indexes: dict[str, FileSymbolIndex],
target: DefTarget,
new_name: str,
uris: dict[str, str] | None = None,
) -> lsp.WorkspaceEdit | None:
"""Rename a block template or address in all .def and Tcl files, if it is declared."""
if not def_declarations(documents, target):
return None
changes: dict[str, list[lsp.TextEdit]] = {}
for location in all_def_target_locations(documents, indexes, target, True, uris):
changes.setdefault(location.uri, []).append(lsp.TextEdit(range=location.range, new_text=new_name))
for edits in changes.values():
edits.sort(key=lambda edit: (edit.range.start.line, edit.range.start.character), reverse=True)
return lsp.WorkspaceEdit(changes=changes)
def _escape_cell(value: str) -> str:
return value.replace("|", "\\|") or " "
def _address_table(declaration: DefDeclaration, formats: dict[str, DefDeclaration]) -> str:
properties = dict(declaration.properties)
rows = []
for key, label in _ADDRESS_PROPERTIES:
value = properties.pop(key, None)
if value is None:
continue
cell = f"`{value}`" if value else ""
if key == "FORMAT" and value in formats:
cell += f" → `{dict(formats[value].properties).get('FORMAT', '')}`"
if key == "FORCE":
cell += f" ({_MODALITY[value.upper()]})" if value.upper() in _MODALITY else ""
rows.append(f"| {label} | {_escape_cell(cell)} |")
rows.extend(f"| {key.title()} | {_escape_cell(f'`{value}`' if value else '')} |" for key, value in properties.items())
if not rows:
return "_No properties_"
return "\n".join(["| Property | Value |", "|---|---|", *rows])
def def_hover_markdown(documents: dict[str, DefDocument], target: DefTarget) -> str | None:
"""Describe the effective (last loaded) declaration and name the ones it overrides."""
declarations = def_declarations(documents, target)
if not declarations:
return None
formats = {
declaration.name: declaration
for document in documents.values()
for declaration in document.declarations
if declaration.kind == FORMAT
}
kind, name = target
path, declaration = declarations[-1]
header = f"**{_KIND_LABELS[kind]}** `{name}` — {Path(path).name}:{declaration.line + 1}"
if kind == ADDRESS:
body = _address_table(declaration, formats)
else:
body = f"```def\n{declaration.text}\n```"
markdown = f"{header}\n\n{body}"
if len(declarations) > 1:
overridden = ", ".join(f"{Path(other).name}:{item.line + 1}" for other, item in declarations[:-1])
markdown += f"\n\n---\n\n_Overrides {overridden}_"
return markdown
+40 -44
View File
@@ -5,6 +5,17 @@ from pathlib import Path
import lsprotocol.types as lsp
from tclint.syntax_tree import Command, List, Node, QuotedWord, Script, VarSub
from tools.def_flow import ( # noqa: F401 (re-exported)
DEF_ADDRESS,
DEF_BLOCK_TEMPLATE,
DEF_SYMBOL_KINDS,
ProcDefFlows,
command_flow_facts,
def_argument_kinds,
proc_def_flows,
)
from tools.def_flow import def_name as _def_name
from tools.stored_procs import stored_command_names
from tools.variable_names import array_key_parts, variable_name
ROOT_NAMESPACE = "::"
@@ -39,6 +50,8 @@ class FileSymbolIndex:
uri: str
occurrences: tuple[SymbolOccurrence, ...]
document_range: lsp.Range | None = None
# Procs whose parameters reach .def arguments: (qualified proc name, flows).
def_flows: tuple[tuple[str, ProcDefFlows], ...] = ()
@dataclass(frozen=True)
@@ -114,49 +127,6 @@ def _name_range(node: Node, raw_name: str, *, variable_sub: bool = False) -> lsp
)
DEF_BLOCK_TEMPLATE = "block_template"
DEF_ADDRESS = "address"
DEF_SYMBOL_KINDS = frozenset({DEF_BLOCK_TEMPLATE, DEF_ADDRESS})
# NX commands taking .def names: command -> (first argument index, last index or
# None for all following arguments, kind).
_DEF_ARGUMENTS: dict[str, tuple[tuple[int, int | None, str], ...]] = {
"MOM_do_template": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_add_to_block_buffer": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_polar_motion": ((0, 0, DEF_BLOCK_TEMPLATE),),
"MOM_force_block": ((1, None, DEF_BLOCK_TEMPLATE),),
"MOM_ask_address_value": ((0, 0, DEF_ADDRESS),),
"MOM_add_to_address_buffer": ((0, 0, DEF_ADDRESS),),
"MOM_enable_address": ((0, None, DEF_ADDRESS),),
"MOM_disable_address": ((0, None, DEF_ADDRESS),),
"MOM_force": ((1, None, DEF_ADDRESS),),
"MOM_suppress": ((1, None, DEF_ADDRESS),),
"MOM_incremental": ((1, None, DEF_ADDRESS),),
}
_DEFINITION_ELEMENT_KINDS = {"ADDRESS": DEF_ADDRESS, "BLOCK": DEF_BLOCK_TEMPLATE}
def def_argument_kinds(command: Command) -> list[tuple[Node, str]]:
"""Return the arguments of ``command`` that name a .def block template or address."""
routine = _static_contents(command.routine)
if routine in {"MOM_ask_definition_element", "MOM_has_definition_element"}:
kind = _DEFINITION_ELEMENT_KINDS.get((_static_contents(command.args[0]) or "").upper()) if command.args else None
return [(command.args[1], kind)] if kind and len(command.args) >= 2 else []
result = []
for first, last, kind in _DEF_ARGUMENTS.get(routine or "", ()):
for position, argument in enumerate(command.args):
if position >= first and (last is None or position <= last):
result.append((argument, kind))
return result
def _def_name(node: Node) -> str | None:
name = _static_contents(node)
if not name or node.contents_pos is None or any(char.isspace() or char in "$[]{}\\\"" for char in name):
return None
return name
def _node_range(node: Node) -> lsp.Range | None:
if node.pos is None or node.end_pos is None:
return None
@@ -311,6 +281,9 @@ def build_file_symbol_index(
# Most occurrences repeat a few identities; sharing one object per identity
# keeps the index (and its persistent cache) small.
identities: dict[SymbolIdentity, SymbolIdentity] = {}
def_flows: list[tuple[str, ProcDefFlows]] = []
# Flow facts of the procs being walked, innermost last.
flow_facts: list[list] = []
def shared(identity: SymbolIdentity | None) -> SymbolIdentity | None:
return None if identity is None else identities.setdefault(identity, identity)
@@ -322,6 +295,7 @@ def build_file_symbol_index(
*,
is_definition: bool,
declaration_range: lsp.Range | None = None,
name_range: lsp.Range | None = None,
) -> None:
identity = shared(_proc_identity(raw_name, scope.namespace))
caller = None
@@ -339,7 +313,7 @@ def build_file_symbol_index(
if is_definition
else shared(_proc_fallback(raw_name, scope.namespace))
),
range=_name_range(node, raw_name),
range=name_range or _name_range(node, raw_name),
placeholder=_basename(raw_name),
is_definition=is_definition,
symbol_kind=lsp.SymbolKind.Function,
@@ -427,11 +401,13 @@ def build_file_symbol_index(
)
parameters = command.args[1]
parameter_names = []
for parameter in getattr(parameters, "children", []):
parameter_node = parameter
if isinstance(parameter, List) and parameter.children:
parameter_node = parameter.children[0]
parameter_name = _static_contents(parameter_node)
parameter_names.append(parameter_name or "")
if parameter_name:
add_variable(
parameter_node,
@@ -440,7 +416,12 @@ def build_file_symbol_index(
is_definition=True,
)
flow_facts.append([])
walk_script(body, proc_scope)
facts = flow_facts.pop()
flows = proc_def_flows(facts, parameter_names, proc_namespace) if facts else ()
if flows:
def_flows.append((proc_identity.name, flows))
def walk_namespace(command: Command, scope: _Scope) -> bool:
if len(command.args) < 3 or _static_contents(command.args[0]) != "eval":
@@ -478,6 +459,20 @@ def build_file_symbol_index(
if routine:
add_proc(command.routine, routine, scope, is_definition=False)
for name, line, column in stored_command_names(command):
start = column - 1 + (name.rfind("::") + 2 if "::" in name else 0)
add_proc(
command.args[2],
name,
scope,
is_definition=False,
name_range=lsp.Range(
start=lsp.Position(line=line - 1, character=start),
end=lsp.Position(line=line - 1, character=column - 1 + len(name)),
),
)
if flow_facts:
flow_facts[-1].extend(command_flow_facts(command))
for node, kind in def_argument_kinds(command):
name = _def_name(node)
@@ -550,6 +545,7 @@ def build_file_symbol_index(
uri=uri,
occurrences=tuple(occurrences),
document_range=_node_range(tree),
def_flows=tuple(def_flows),
)
+6
View File
@@ -5,6 +5,7 @@ import attrs
from common.load_data import standard_items
from tclint.commands.plugins import PluginManager
from tclint.syntax_tree import BareWord, BracedWord, Command, QuotedWord, Visitor
from tools.stored_procs import stored_command_names
from tools.variable_names import variable_name
from tools.tcloo_symbols import class_symbols
from tools.tcloo_completion import _analyze
@@ -182,6 +183,11 @@ class _Highlighter(Visitor):
line, col = argument.contents_pos
self._append_token((line - 1, col - 1), len(argument.contents), "function", [])
# Procedures stored in COMMANDBLOCK properties (CONF_x set prop {proc}).
for stored_name, line, col in stored_command_names(command):
if stored_name in self._custom_function_names or stored_name in _STANDARD_PROC_NAMES:
self._append_token((line - 1, col - 1), len(stored_name), "function", [])
# Highlight functions (custom or standard) when used as the routine
name = getattr(routine, "contents", None)
if name:
+80
View File
@@ -0,0 +1,80 @@
"""Procedure names stored as data and called later.
PostConfigurator COMMANDBLOCK properties hold a Tcl list whose elements are
executed as commands (see LIB_CONF_do_prop_custom_proc), e.g.
``CONF_CTRL_tool set auto_preselect_last_template {custom_header}`` or
``CONF_CTRL_moves set return_safety_pos {{OEM_output arg}}``. The first word of
each element is a command name; only braced values are considered.
"""
from __future__ import annotations
import re
from tclint.syntax_tree import BracedWord, Command, Node
_CONF_OBJECT_RE = re.compile(r"^(::)?CONF_\w+$")
_COMMAND_NAME_RE = re.compile(r"[A-Za-z_:][\w:]*")
def _static(node: Node | None) -> str | None:
value = getattr(node, "contents", None)
return value if isinstance(value, str) else None
def _list_element_spans(text: str) -> list[tuple[int, int]]:
"""(start, end) offsets of the top-level elements of a Tcl list, braces stripped."""
spans = []
index = 0
while index < len(text):
if text[index].isspace():
index += 1
continue
if text[index] == "{":
depth, start = 0, index + 1
while index < len(text):
if text[index] == "\\":
index += 2
continue
depth += {"{": 1, "}": -1}.get(text[index], 0)
index += 1
if depth == 0:
break
spans.append((start, index - 1))
elif text[index] == '"':
start = index + 1
index = text.find('"', start)
index = len(text) if index < 0 else index
spans.append((start, index))
index += 1
else:
start = index
while index < len(text) and not text[index].isspace():
index += 1
spans.append((start, index))
return spans
def stored_command_names(command: Command) -> list[tuple[str, int, int]]:
"""Command names stored in ``command`` as (name, line, column), 1-based."""
routine = _static(command.routine)
args = command.args
if not (routine and _CONF_OBJECT_RE.match(routine) and len(args) >= 3 and _static(args[0]) == "set"):
return []
value = args[2]
text = _static(value)
if not isinstance(value, BracedWord) or not text or value.contents_pos is None:
return []
line, column = value.contents_pos
names = []
for start, end in _list_element_spans(text):
element = text[start:end]
match = _COMMAND_NAME_RE.match(element, len(element) - len(element.lstrip()))
if match is None or (match.end() < len(element) and not element[match.end()].isspace()):
continue
offset = start + match.start()
before = text[:offset]
name_line = line + before.count("\n")
name_column = offset - before.rfind("\n") if "\n" in before else column + offset
names.append((match.group(), name_line, name_column))
return names
@@ -47,20 +47,9 @@ def _document(path: Path, source: str) -> TextDocument:
)
def _completion_server(
tmp_path: Path, monkeypatch
) -> tuple[TclLanguageServer, TextDocument, str]:
def _completion_server(tmp_path: Path, monkeypatch) -> tuple[TclLanguageServer, TextDocument, str]:
declared_builtin = standard_items.nx_variables[0].label
current_source = (
"set globalValue 1\n"
"proc localProc {} { return }\n"
"proc caller {argument} {\n"
f" global {declared_builtin}\n"
" set localValue 2\n"
" puts $local\n"
" localP\n"
"}\n"
)
current_source = f"set globalValue 1\nproc localProc {{}} {{ return }}\nproc caller {{argument}} {{\n global {declared_builtin}\n set localValue 2\n puts $local\n localP\n}}\n"
workspace_source = """set ::workspaceValue 1
proc workspaceProc {} { return }
"""
@@ -113,14 +102,10 @@ def test_unset_space_shows_options_then_variables(tmp_path, monkeypatch):
items = _complete(document, lsp.Position(line=1, character=len(tail)))
labels = {item.label for item in items}
if tail in {"unset ", "unset -"}:
assert labels == {"-nocomplain", "--"}
assert labels == {"nocomplain"}
else:
assert "globalValue" in labels
assert "-nocomplain" not in labels
if tail == "unset -nocomplain ":
assert "--" in labels
else:
assert "--" not in labels
def test_array_keys_complete_in_set_and_substitution(tmp_path: Path, monkeypatch):
@@ -130,19 +115,16 @@ def test_array_keys_complete_in_set_and_substitution(tmp_path: Path, monkeypatch
"set ::lib_flag(enabled) 1\nset ::lib_flag(external) 1\n",
)
assert server.update_poco_completion_for_file(workspace)
source = (
"set lib_flag(enabled) 0\n"
"set lib_flag(empty) 1\n"
"set other(wrong) 1\n"
"proc hidden {} { set lib_flag(private) 1 }\n"
"set lib_flag()\n"
"puts $lib_flag(en)\n"
"puts 😀; set lib_flag(em\n"
)
source = "set lib_flag(enabled) 0\nset lib_flag(empty) 1\nset other(wrong) 1\nproc hidden {} { set lib_flag(private) 1 }\nset lib_flag()\nputs $lib_flag(en)\nputs 😀; set lib_flag(em\n"
current = _document(tmp_path / "arrays-current.tcl", source)
server.workspace.put_text_document(lsp.TextDocumentItem(
uri=current.uri, language_id="tcl", version=1, text=source,
))
server.workspace.put_text_document(
lsp.TextDocumentItem(
uri=current.uri,
language_id="tcl",
version=1,
text=source,
)
)
assert server.update_poco_completion_for_file(current)
items = _complete(current, _position_after(source, "set lib_flag(", 3))
assert [item.label for item in items] == ["empty", "enabled", "external"]
@@ -179,32 +161,28 @@ def test_dynamic_array_index_keeps_variable_identity(tmp_path: Path, monkeypatch
assert variable_name(tree.children[3].args[0]) is None
path = tmp_path / "dynamic.tcl"
index = build_file_symbol_index(str(path), path.as_uri(), tree)
definition = next(
item for item in index.occurrences
if item.identity.name == "::custom_flag" and item.is_definition
)
definition = next(item for item in index.occurrences if item.identity.name == "::custom_flag" and item.is_definition)
assert definition.range.start.character == 4
assert definition.range.end.character == 15
assert definition.array_element is None
assert any(item.identity.name == "::mom_path_name" for item in index.occurrences)
highlighter = _Highlighter([], {})
tree.accept(highlighter, recurse=True)
assert any(
position == (0, 4) and length == 11 and kind == "variable"
for position, length, kind, _ in highlighter._tokens
)
assert any(position == (0, 4) and length == 11 and kind == "variable" for position, length, kind, _ in highlighter._tokens)
server, _, _ = _completion_server(tmp_path, monkeypatch)
current = _document(path, source)
server.workspace.put_text_document(lsp.TextDocumentItem(
uri=current.uri, language_id="tcl", version=1, text=source,
))
server.workspace.put_text_document(
lsp.TextDocumentItem(
uri=current.uri,
language_id="tcl",
version=1,
text=source,
)
)
assert server.update_poco_completion_for_file(current)
items = _complete(current, _position_after(source, "puts $custom"))
assert "custom_flag" in {item.label for item in items}
workspace_items = next(
items for item_path, items in server.completion_items_by_file_snapshot().items()
if server.paths_equal(item_path, str(path))
)
workspace_items = next(items for item_path, items in server.completion_items_by_file_snapshot().items() if server.paths_equal(item_path, str(path)))
assert {"quoted_flag", "command_flag"} <= {item.label for item in workspace_items}
@@ -236,18 +214,25 @@ def test_literal_array_components_complete_around_substitutions(tmp_path: Path,
offset = marked.index("|")
line = marked.replace("|", "")
items = array_element_completions(
[line], lsp.Position(line=0, character=offset),
server.navigation_snapshot().values(), str(tmp_path / "caller.tcl"),
[line],
lsp.Position(line=0, character=offset),
server.navigation_snapshot().values(),
str(tmp_path / "caller.tcl"),
)
assert {item.label for item in items} == labels
edit = next(item.text_edit for item in items if item.label == selected)
item = next(item for item in items if item.label == selected)
assert item.insert_text_format == lsp.InsertTextFormat.PlainText
assert line[:edit.range.start.character] + edit.new_text + line[edit.range.end.character:] == expected
assert array_element_completions(
["set custom_flag(from_move,$::mom"], lsp.Position(line=0, character=31),
server.navigation_snapshot().values(), str(tmp_path / "caller.tcl"),
) is None
assert line[: edit.range.start.character] + edit.new_text + line[edit.range.end.character :] == expected
assert (
array_element_completions(
["set custom_flag(from_move,$::mom"],
lsp.Position(line=0, character=31),
server.navigation_snapshot().values(),
str(tmp_path / "caller.tcl"),
)
is None
)
def _argument_completion_request(source: str):
@@ -277,9 +262,7 @@ def test_variable_completion_filters_and_ranks_candidates(tmp_path: Path, monkey
other_builtin = standard_items.nx_variables[1]
assert declared_builtin.label in by_label
assert other_builtin.label in by_label
assert by_label[declared_builtin.label].documentation == (
declared_builtin.documentation
)
assert by_label[declared_builtin.label].documentation == (declared_builtin.documentation)
assert by_label["localValue"].sort_text.startswith("000:")
assert by_label["globalValue"].sort_text.startswith("100:")
assert by_label["workspaceValue"].sort_text.startswith("200:")
@@ -307,18 +290,9 @@ def test_command_completion_filters_and_ranks_candidates(tmp_path: Path, monkeyp
def test_completion_context_handles_nested_commands_and_utf16():
assert (
completion_context(["set result [work"], lsp.Position(line=0, character=16))
== CompletionContext.COMMAND
)
assert (
completion_context(["😀 puts $value"], lsp.Position(line=0, character=14))
== CompletionContext.VARIABLE
)
assert (
completion_context(["puts value"], lsp.Position(line=0, character=10))
== CompletionContext.GENERAL
)
assert completion_context(["set result [work"], lsp.Position(line=0, character=16)) == CompletionContext.COMMAND
assert completion_context(["😀 puts $value"], lsp.Position(line=0, character=14)) == CompletionContext.VARIABLE
assert completion_context(["puts value"], lsp.Position(line=0, character=10)) == CompletionContext.GENERAL
def test_string_subcommands_and_compare_options_are_context_aware():
@@ -330,9 +304,7 @@ def test_string_subcommands_and_compare_options_are_context_aware():
"-length",
"-nocase",
}
assert _argument_completion_labels("string compare -nocase ") == {
"-length"
}
assert _argument_completion_labels("string compare -nocase ") == {"-length"}
assert _argument_completion_labels("string compare -length ") is None
@@ -356,22 +328,14 @@ def test_string_completion_inside_braced_conditions_and_bodies():
subcommands = _argument_completion_labels(prefix + "[string ")
assert subcommands is not None
assert {"compare", "equal", "is"} <= subcommands
assert _argument_completion_labels(prefix + "[string compare -") == {
"-length", "-nocase"
}
assert _argument_completion_labels(
prefix + "[string compare -nocase "
) == {"-length"}
assert _argument_completion_labels(prefix + "[string compare -") == {"-length", "-nocase"}
assert _argument_completion_labels(prefix + "[string compare -nocase ") == {"-length"}
def test_closed_braced_arguments_do_not_change_completion_context():
assert _argument_completion_labels("puts {[string compare }") is None
assert _argument_completion_labels(
"if {[string equal a b]} {string compare "
) == {"-length", "-nocase"}
assert _argument_completion_labels(
"if {[string equal a b] && [string is integer "
) == {"-failindex", "-strict"}
assert _argument_completion_labels("if {[string equal a b]} {string compare ") == {"-length", "-nocase"}
assert _argument_completion_labels("if {[string equal a b] && [string is integer ") == {"-failindex", "-strict"}
def test_dict_array_namespace_file_and_info_subcommands():
@@ -413,9 +377,7 @@ def test_dict_array_namespace_file_and_info_subcommands():
assert {"args", "body", "commands", "exists", "procs", "vars"} <= info_items
def test_variable_context_still_takes_priority_inside_tcl_command(
tmp_path: Path, monkeypatch
):
def test_variable_context_still_takes_priority_inside_tcl_command(tmp_path: Path, monkeypatch):
_, current, source = _completion_server(tmp_path, monkeypatch)
command_source = source.replace(
" puts $local\n",
@@ -437,9 +399,7 @@ def test_variable_context_still_takes_priority_inside_tcl_command(
assert "localValue" in {item.label for item in items}
def test_lsp_completion_returns_only_matching_command_options(
tmp_path: Path, monkeypatch
):
def test_lsp_completion_returns_only_matching_command_options(tmp_path: Path, monkeypatch):
_, current, _ = _completion_server(tmp_path, monkeypatch)
source = "string compare "
current = _document(tmp_path / "current.tcl", source)
@@ -459,9 +419,7 @@ def test_lsp_completion_returns_only_matching_command_options(
assert all(item.sort_text.startswith("000:") for item in items)
def test_space_trigger_does_not_open_broad_fallback_completion(
tmp_path: Path, monkeypatch
):
def test_space_trigger_does_not_open_broad_fallback_completion(tmp_path: Path, monkeypatch):
_, current, _ = _completion_server(tmp_path, monkeypatch)
source = "set value "
current = _document(tmp_path / "current.tcl", source)
@@ -550,9 +508,7 @@ def test_path_completion_is_relative_filtered_and_tcl_safe(tmp_path: Path):
assert items[0].text_edit.range.start.character == len("source ")
def test_lsp_source_completion_reads_paths_from_document_directory(
tmp_path: Path, monkeypatch
):
def test_lsp_source_completion_reads_paths_from_document_directory(tmp_path: Path, monkeypatch):
server, current, _ = _completion_server(tmp_path, monkeypatch)
scripts = tmp_path / "scripts"
scripts.mkdir()
@@ -576,9 +532,7 @@ def test_lsp_source_completion_reads_paths_from_document_directory(
assert "scripts/ignored.txt" not in labels
def test_command_and_dict_for_snippets_use_lsp_snippet_placeholders(
tmp_path: Path, monkeypatch
):
def test_command_and_dict_for_snippets_use_lsp_snippet_placeholders(tmp_path: Path, monkeypatch):
_, current, source = _completion_server(tmp_path, monkeypatch)
command_items = _complete(current, _position_after(source, "localP", occurrence=1))
command_by_label = {item.label: item for item in command_items}
@@ -595,14 +549,10 @@ def test_command_and_dict_for_snippets_use_lsp_snippet_placeholders(
switch_arguments = _argument_completion_request("switch ")
assert switch_arguments is not None
assert {"switch block", "-exact", "-glob", "-regexp"} <= {
item.label for item in switch_arguments.items
}
assert {"switch block", "-exact", "-glob", "-regexp"} <= {item.label for item in switch_arguments.items}
def test_semantic_variable_and_procedure_argument_completion(
tmp_path: Path, monkeypatch
):
def test_semantic_variable_and_procedure_argument_completion(tmp_path: Path, monkeypatch):
_, current, source = _completion_server(tmp_path, monkeypatch)
variable_items = _complete(current, _position_after(source, " set "))
@@ -633,9 +583,7 @@ def test_semantic_variable_and_procedure_argument_completion(
assert "string" not in procedure_labels
def test_namespace_argument_completion_uses_navigation_index(
tmp_path: Path, monkeypatch
):
def test_namespace_argument_completion_uses_navigation_index(tmp_path: Path, monkeypatch):
server, current, _ = _completion_server(tmp_path, monkeypatch)
namespace_source = "namespace eval tools { proc helper {} { return } }\n"
namespace_document = _document(tmp_path / "namespaces.tcl", namespace_source)
+223
View File
@@ -0,0 +1,223 @@
"""Block templates and addresses reaching NX commands through variables and procs."""
from pathlib import Path
import lsprotocol.types as lsp # type: ignore
from pygls.workspace import Workspace
import lsp_server
from lsp_tclserver import TclLanguageServer
from tools.def_flow import build_wrapper_table
PSC = """<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Sourcing><Sequence>
<Layer Name="Service" SubFolder="service">
<DefinedEvents><Filename Name="service"/></DefinedEvents>
</Layer>
</Sequence></Sourcing>
</Configuration>
"""
DEF = """MACHINE Default
FORMATTING
{
ADDRESS SPOS
{
FORMAT Coordinate
}
BLOCK_TEMPLATE steady_rest
{
Text[M60]
}
BLOCK_TEMPLATE absolute_mode
{
Text[G90]
}
}
"""
LIBRARY = """proc LIB_call_cycle {cycle {prefix ""}} {
set block $cycle ; regsub -all "," $block "_" block
if {[catch {set line [MOM_do_template $block CREATE]} err]} {
return
}
}
proc LIB_outer {mode name} {
LIB_call_cycle $name
}
proc LIB_force {address} {
MOM_force Once $address
}
proc LIB_log {message} {
puts $message
}
"""
CALLER = """LIB_call_cycle "absolute_mode"
LIB_outer on steady_rest
LIB_force SPOS
LIB_log steady_rest
puts steady_rest
proc local {} {
set t "steady_rest"
MOM_do_template $t
set unused absolute_mode
foreach b {"steady_rest" absolute_mode} { MOM_do_template $b }
set l [list "absolute_mode"]
lappend l steady_rest
foreach x $l { LIB_call_cycle $x }
}
"""
def _project(tmp_path: Path, monkeypatch):
(tmp_path / "service").mkdir()
(tmp_path / "post.psc").write_text(PSC, encoding="utf-8")
(tmp_path / "service" / "service.def").write_text(DEF, encoding="utf-8")
library = tmp_path / "library.tcl"
library.write_text(LIBRARY, encoding="utf-8")
caller = tmp_path / "caller.tcl"
caller.write_text(CALLER, encoding="utf-8")
server = TclLanguageServer(name="def-flow-test", version="1", max_workers=1)
server.protocol._workspace = Workspace( # pylint: disable=protected-access
root_uri=tmp_path.as_uri(),
sync_kind=lsp.TextDocumentSyncKind.Incremental,
workspace_folders=[lsp.WorkspaceFolder(uri=tmp_path.as_uri(), name="root")],
position_encoding=lsp.PositionEncodingKind.Utf16,
)
monkeypatch.setattr(lsp_server, "LSP_SERVER", server)
server.refresh_psc_scripts([tmp_path])
for path, text in ((library, LIBRARY), (caller, CALLER)):
server.workspace.put_text_document(lsp.TextDocumentItem(uri=path.as_uri(), language_id="tcl", version=1, text=text))
server.update_poco_completion_for_file(server.workspace.get_text_document(path.as_uri()))
return server, caller
def _position(needle: str, occurrence: int = 0) -> lsp.Position:
index = -1
for _ in range(occurrence + 1):
index = CALLER.index(needle, index + 1)
line = CALLER.count("\n", 0, index)
return lsp.Position(line=line, character=index - (CALLER.rfind("\n", 0, index) + 1) + 1)
def _hover(caller: Path, needle: str, occurrence: int = 0):
return lsp_server.hover(
lsp.HoverParams(text_document=lsp.TextDocumentIdentifier(uri=caller.as_uri()), position=_position(needle, occurrence))
)
def _definition(caller: Path, needle: str, occurrence: int = 0):
return lsp_server.goto_definition(
lsp.DefinitionParams(text_document=lsp.TextDocumentIdentifier(uri=caller.as_uri()), position=_position(needle, occurrence))
)
def _hover_title(hover) -> str | None:
return hover and hover.contents.value.split("\n", 1)[0]
def test_wrapper_table_follows_parameters_through_nested_procs(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
table = server.def_wrapper_table()
assert table["::LIB_call_cycle"] == {0: frozenset({"block_template"})}
assert table["::LIB_outer"] == {1: frozenset({"block_template"})}
assert table["::LIB_force"] == {0: frozenset({"address"})}
assert "::LIB_log" not in table
def test_build_wrapper_table_stops_on_recursion():
flows = [
("::a", ((0, ("call", "::b", "::b", 0)),)),
("::b", ((0, ("call", "::a", "::a", 0)), (0, ("def", "address")))),
]
assert build_wrapper_table(flows) == {"::a": {0: frozenset({"address"})}, "::b": {0: frozenset({"address"})}}
def test_literal_argument_of_wrapper_proc_is_a_template(tmp_path, monkeypatch):
_, caller = _project(tmp_path, monkeypatch)
hover = _hover(caller, "absolute_mode")
assert _hover_title(hover).startswith("**Block template** `absolute_mode`")
assert (hover.range.start.line, hover.range.start.character, hover.range.end.character) == (0, 16, 29)
[location] = _definition(caller, "absolute_mode")
assert Path(location.uri).name == "service.def"
assert location.range.start.line == 12
def test_nested_wrapper_and_address_wrapper(tmp_path, monkeypatch):
_, caller = _project(tmp_path, monkeypatch)
assert _hover_title(_hover(caller, "steady_rest")).startswith("**Block template** `steady_rest`")
assert _hover_title(_hover(caller, "SPOS")).startswith("**Address** `SPOS`")
def test_same_name_without_flow_is_not_a_template(tmp_path, monkeypatch):
_, caller = _project(tmp_path, monkeypatch)
for needle, occurrence in (("steady_rest", 1), ("steady_rest", 2), ("absolute_mode", 1)):
assert _hover(caller, needle, occurrence) is None, (needle, occurrence)
assert _definition(caller, needle, occurrence) is None, (needle, occurrence)
def test_literals_flowing_through_local_variables(tmp_path, monkeypatch):
_, caller = _project(tmp_path, monkeypatch)
cases = (
("steady_rest", 3), # set t "steady_rest"; MOM_do_template $t
("steady_rest", 4), # foreach b {"steady_rest" ...}
("absolute_mode", 2), # foreach b {... absolute_mode}
("absolute_mode", 3), # set l [list "absolute_mode"]; foreach x $l { LIB_call_cycle $x }
("steady_rest", 5), # lappend l steady_rest
)
for needle, occurrence in cases:
assert _hover_title(_hover(caller, needle, occurrence)).startswith(f"**Block template** `{needle}`"), occurrence
assert _definition(caller, needle, occurrence), (needle, occurrence)
def test_derived_names_are_not_renamed(tmp_path, monkeypatch):
_, caller = _project(tmp_path, monkeypatch)
params = lsp.PrepareRenameParams(
text_document=lsp.TextDocumentIdentifier(uri=caller.as_uri()), position=_position("absolute_mode")
)
assert lsp_server.prepare_rename(params) is None
def _warnings(server, tmp_path: Path, source: str):
uri = (tmp_path / "check.tcl").as_uri()
server.workspace.put_text_document(lsp.TextDocumentItem(uri=uri, language_id="tcl", version=1, text=source))
diagnostics = server.lint(server.workspace.get_text_document(uri))
return [
(diagnostic.code, diagnostic.message, diagnostic.range.start.line, diagnostic.range.start.character, diagnostic.range.end.character)
for diagnostic in diagnostics
if diagnostic.code in {"unknown-block-template", "unknown-address"}
]
def test_unknown_template_and_address_are_warned(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
source = 'proc p {} {\n MOM_do_template stedy_rest\n MOM_force Once SPOS "SPSO"\n}\n'
assert _warnings(server, tmp_path, source) == [
("unknown-block-template", "Block template 'stedy_rest' is not declared in any loaded .def file", 1, 20, 30),
("unknown-address", "Address 'SPSO' is not declared in any loaded .def file", 2, 25, 29),
]
def test_known_and_dynamic_names_are_not_warned(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
source = 'MOM_do_template steady_rest CREATE\nMOM_do_template $name\nMOM_do_template "CYCLE_$x"\nMOM_force Once SPOS\n'
assert _warnings(server, tmp_path, source) == []
def test_no_warnings_without_loaded_def_files(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
server.def_documents = {}
assert _warnings(server, tmp_path, "MOM_do_template stedy_rest\n") == []
def test_def_change_invalidates_cached_diagnostics(tmp_path, monkeypatch):
server, caller = _project(tmp_path, monkeypatch)
server.compute_diagnostics(server.workspace.get_text_document(caller.as_uri()))
assert server.diagnostic_snapshot(caller.as_uri()) is not None
(tmp_path / "service" / "service.def").write_text(DEF.replace("absolute_mode", "incremental_mode"), encoding="utf-8")
server.refresh_def_symbols([tmp_path])
assert server.diagnostic_snapshot(caller.as_uri()) is None
@@ -0,0 +1,289 @@
"""Go to Definition, hover, references and rename between Tcl and .def files."""
from collections import namedtuple
from pathlib import Path
import lsprotocol.types as lsp # type: ignore
from pygls.workspace import Workspace
import lsp_server
from lsp_tclserver import TclLanguageServer
from tools.def_symbols import ADDRESS, BLOCK_TEMPLATE, FORMAT, parse_def_document
PSC = """<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Sourcing><Sequence>
<Layer Name="Service" SubFolder="service">
<DefinedEvents><Filename Name="service"/></DefinedEvents>
</Layer>
</Sequence></Sourcing>
</Configuration>
"""
DEF = """MACHINE Default
FORMATTING
{
FORMAT Coordinate "&__4.3_"
ADDRESS SPOS
{
FORMAT Coordinate
FORCE off
MAX 99999.999 Abort
MIN -99999.999 Abort
LEADER "SPOS="
}
# ADDRESS commented_out
BLOCK_TEMPLATE steady_rest
{
SPOS[$mom_pos(0)]
Text[M60]\\opt
}
}
"""
TCL = """proc MOM_steady {} {
MOM_do_template steady_rest
MOM_force Once SPOS X
MOM_ask_address_value "SPOS"
set name steady_rest
}
"""
def _project(tmp_path: Path, monkeypatch):
(tmp_path / "service").mkdir()
(tmp_path / "post.psc").write_text(PSC, encoding="utf-8")
def_file = tmp_path / "service" / "service.def"
def_file.write_text(DEF, encoding="utf-8")
tcl_file = tmp_path / "caller.tcl"
tcl_file.write_text(TCL, encoding="utf-8")
server = TclLanguageServer(name="def-navigation-test", version="1", max_workers=1)
server.protocol._workspace = Workspace( # pylint: disable=protected-access
root_uri=tmp_path.as_uri(),
sync_kind=lsp.TextDocumentSyncKind.Incremental,
workspace_folders=[lsp.WorkspaceFolder(uri=tmp_path.as_uri(), name="root")],
position_encoding=lsp.PositionEncodingKind.Utf16,
)
monkeypatch.setattr(lsp_server, "LSP_SERVER", server)
server.refresh_psc_scripts([tmp_path])
server.workspace.put_text_document(
lsp.TextDocumentItem(uri=tcl_file.as_uri(), language_id="tcl", version=1, text=TCL)
)
server.update_poco_completion_for_file(server.workspace.get_text_document(tcl_file.as_uri()))
return server, def_file, tcl_file
def _position(source: str, needle: str, occurrence: int = 0, offset: int = 1) -> lsp.Position:
index = -1
for _ in range(occurrence + 1):
index = source.index(needle, index + 1)
line = source.count("\n", 0, index)
column = index - (source.rfind("\n", 0, index) + 1)
return lsp.Position(line=line, character=column + offset)
def _tcl_params(tcl_file: Path, needle: str, occurrence: int = 0):
return lsp.TextDocumentIdentifier(uri=tcl_file.as_uri()), _position(TCL, needle, occurrence)
# The client sends custom .def requests as plain JSON; pygls exposes them as namedtuples.
_Doc = namedtuple("Object", ["uri"])
_Pos = namedtuple("Object", ["line", "character"])
_Params = namedtuple("Object", ["textDocument", "position", "text", "includeDeclaration", "newName"])
def _def_params(def_file: Path, needle: str, occurrence: int = 0, text: str = DEF, offset: int = 1, **extra):
position = _position(text, needle, occurrence, offset)
return _Params(
_Doc(def_file.as_uri()),
_Pos(position.line, position.character),
text,
extra.get("includeDeclaration", True),
extra.get("newName", ""),
)
def _lines(locations):
return sorted((Path(location.uri).name, location.range.start.line, location.range.start.character) for location in locations)
def test_parse_def_document_declarations_and_references():
document = parse_def_document(DEF)
kinds = [(item.kind, item.name) for item in document.declarations]
assert kinds == [(FORMAT, "Coordinate"), (ADDRESS, "SPOS"), (BLOCK_TEMPLATE, "steady_rest")]
address = document.declarations[1]
assert (address.line, address.start, address.end) == (5, 12, 16)
assert dict(address.properties)["LEADER"] == '"SPOS="'
assert [(ref.name, ref.line, ref.container) for ref in document.references] == [
("SPOS", 16, "steady_rest"),
("Text", 17, "steady_rest"),
]
assert document.declarations[2].text.splitlines()[-1].strip() == "}"
def test_tcl_goto_definition_of_template_and_address(tmp_path, monkeypatch):
_, def_file, tcl_file = _project(tmp_path, monkeypatch)
document, position = _tcl_params(tcl_file, "steady_rest")
result = lsp_server.goto_definition(lsp.DefinitionParams(text_document=document, position=position))
assert _lines(result) == [("service.def", 14, 19)]
for needle, occurrence in (("SPOS", 0), ("SPOS", 1)):
document, position = _tcl_params(tcl_file, needle, occurrence)
result = lsp_server.goto_definition(lsp.DefinitionParams(text_document=document, position=position))
assert _lines(result) == [("service.def", 5, 12)]
def test_tcl_goto_definition_ignores_plain_words_and_unknown_names(tmp_path, monkeypatch):
_, _, tcl_file = _project(tmp_path, monkeypatch)
# "set name steady_rest" is no template argument.
document, position = _tcl_params(tcl_file, "steady_rest", 1)
assert lsp_server.goto_definition(lsp.DefinitionParams(text_document=document, position=position)) is None
document, position = lsp.TextDocumentIdentifier(uri=tcl_file.as_uri()), _position(TCL, "SPOS X", offset=5)
assert lsp_server.goto_definition(lsp.DefinitionParams(text_document=document, position=position)) is None
def test_tcl_hover_shows_template_body_and_address_properties(tmp_path, monkeypatch):
_, _, tcl_file = _project(tmp_path, monkeypatch)
document, position = _tcl_params(tcl_file, "steady_rest")
hover = lsp_server.hover(lsp.HoverParams(text_document=document, position=position))
assert "Block template" in hover.contents.value
assert "SPOS[$mom_pos(0)]" in hover.contents.value
assert "```def" in hover.contents.value
document, position = _tcl_params(tcl_file, "SPOS")
value = lsp_server.hover(lsp.HoverParams(text_document=document, position=position)).contents.value
assert "| Format | `Coordinate` → `\"&__4.3_\"` |" in value
assert '| Leader | `"SPOS="` |' in value
assert "| Min | `-99999.999 Abort` |" in value
assert "| Max | `99999.999 Abort` |" in value
assert "| Modality | `off` (modal, output only on change) |" in value
def test_tcl_references_include_def_declaration_and_template_elements(tmp_path, monkeypatch):
_, _, tcl_file = _project(tmp_path, monkeypatch)
document, position = _tcl_params(tcl_file, "SPOS")
result = lsp_server.references(
lsp.ReferenceParams(
text_document=document,
position=position,
context=lsp.ReferenceContext(include_declaration=True),
)
)
assert _lines(result) == [
("caller.tcl", 2, 19),
("caller.tcl", 3, 27),
("service.def", 5, 12),
("service.def", 16, 8),
]
result = lsp_server.references(
lsp.ReferenceParams(
text_document=document,
position=position,
context=lsp.ReferenceContext(include_declaration=False),
)
)
assert ("service.def", 5, 12) not in _lines(result)
def test_tcl_rename_updates_def_and_tcl(tmp_path, monkeypatch):
_, def_file, tcl_file = _project(tmp_path, monkeypatch)
document, position = _tcl_params(tcl_file, "SPOS")
prepared = lsp_server.prepare_rename(lsp.PrepareRenameParams(text_document=document, position=position))
assert prepared.placeholder == "SPOS"
edit = lsp_server.rename(lsp.RenameParams(text_document=document, position=position, new_name="STEADY_POS"))
edits = {Path(uri).name: [(e.range.start.line, e.range.start.character) for e in items] for uri, items in edit.changes.items()}
assert edits == {"caller.tcl": [(3, 27), (2, 19)], "service.def": [(16, 8), (5, 12)]}
assert all(e.new_text == "STEADY_POS" for items in edit.changes.values() for e in items)
def test_undeclared_names_are_not_renamed(tmp_path, monkeypatch):
_, _, tcl_file = _project(tmp_path, monkeypatch)
document = lsp.TextDocumentIdentifier(uri=tcl_file.as_uri())
position = _position(TCL, "SPOS X", offset=5)
assert lsp_server.prepare_rename(lsp.PrepareRenameParams(text_document=document, position=position)) is None
def test_def_requests_resolve_declarations_and_elements(tmp_path, monkeypatch):
_, def_file, _ = _project(tmp_path, monkeypatch)
# Address element inside a block template -> ADDRESS declaration.
result = lsp_server.def_definition(_def_params(def_file, "SPOS[", offset=1))
assert _lines(result) == [("service.def", 5, 12)]
hover = lsp_server.def_hover(_def_params(def_file, "steady_rest"))
assert "Text[M60]" in hover.contents.value
references = lsp_server.def_references(_def_params(def_file, "ADDRESS SPOS", offset=9))
assert _lines(references) == [
("caller.tcl", 2, 19),
("caller.tcl", 3, 27),
("service.def", 5, 12),
("service.def", 16, 8),
]
assert lsp_server.def_hover(_def_params(def_file, "MACHINE")) is None
def test_def_requests_use_unsaved_text(tmp_path, monkeypatch):
_, def_file, _ = _project(tmp_path, monkeypatch)
text = DEF.replace("BLOCK_TEMPLATE steady_rest", "BLOCK_TEMPLATE steady_rest_new")
params = _def_params(def_file, "steady_rest_new", text=text, newName="rest")
assert lsp_server.def_prepare_rename(params).placeholder == "steady_rest_new"
edit = lsp_server.def_rename(params)
assert list(edit.changes) == [def_file.as_uri()]
def test_def_rename_updates_tcl_callers(tmp_path, monkeypatch):
_, def_file, _ = _project(tmp_path, monkeypatch)
edit = lsp_server.def_rename(_def_params(def_file, "steady_rest", newName="lunette"))
edits = {Path(uri).name: [(e.range.start.line, e.range.start.character) for e in items] for uri, items in edit.changes.items()}
assert edits == {"caller.tcl": [(1, 20)], "service.def": [(14, 19)]}
assert lsp_server.def_rename(_def_params(def_file, "steady_rest", newName="bad name")) is None
LAYERED_PSC = """<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Sourcing><Sequence>
<Layer Name="Controller" SubFolder="ctrl"><DefinedEvents><Filename Name="ctrl"/></DefinedEvents></Layer>
<Layer Name="OEM" SubFolder="oem"><DefinedEvents><Filename Name="oem"/></DefinedEvents></Layer>
</Sequence></Sourcing>
</Configuration>
"""
def test_template_in_several_layers_shows_the_last_loaded_one(tmp_path, monkeypatch):
for folder, text in (("ctrl", "Text[M17]"), ("oem", "Text[RET]")):
(tmp_path / folder).mkdir()
(tmp_path / folder / f"{folder}.def").write_text(
f"MACHINE X\n\nFORMATTING\n{{\n BLOCK_TEMPLATE end_of_subprogram\n {{\n {text}\n }}\n}}\n", encoding="utf-8"
)
(tmp_path / "post.psc").write_text(LAYERED_PSC, encoding="utf-8")
source = "MOM_do_template end_of_subprogram\n"
caller = tmp_path / "caller.tcl"
caller.write_text(source, encoding="utf-8")
server = TclLanguageServer(name="def-layer-test", version="1", max_workers=1)
server.protocol._workspace = Workspace( # pylint: disable=protected-access
root_uri=tmp_path.as_uri(),
sync_kind=lsp.TextDocumentSyncKind.Incremental,
workspace_folders=[lsp.WorkspaceFolder(uri=tmp_path.as_uri(), name="root")],
position_encoding=lsp.PositionEncodingKind.Utf16,
)
monkeypatch.setattr(lsp_server, "LSP_SERVER", server)
server.refresh_psc_scripts([tmp_path])
server.workspace.put_text_document(lsp.TextDocumentItem(uri=caller.as_uri(), language_id="tcl", version=1, text=source))
server.update_poco_completion_for_file(server.workspace.get_text_document(caller.as_uri()))
document = lsp.TextDocumentIdentifier(uri=caller.as_uri())
position = lsp.Position(line=0, character=20)
hover = lsp_server.hover(lsp.HoverParams(text_document=document, position=position)).contents.value
assert hover.startswith("**Block template** `end_of_subprogram` — oem.def:5")
assert "Text[RET]" in hover and "Text[M17]" not in hover
assert hover.endswith("_Overrides ctrl.def:5_")
[location] = lsp_server.goto_definition(lsp.DefinitionParams(text_document=document, position=position))
assert location.uri.endswith("oem/oem.def")
references = lsp_server.references(
lsp.ReferenceParams(text_document=document, position=position, context=lsp.ReferenceContext(include_declaration=True))
)
assert {Path(location.uri).name for location in references} == {"ctrl.def", "oem.def", "caller.tcl"}
@@ -227,6 +227,25 @@ def test_block_list_shows_all_templates_quoted(tmp_path, monkeypatch):
assert all(item.filter_text.startswith("BLOCK_LIST") for item in items)
def test_block_list_item_resolves_to_template_preview(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
items = _complete(server, tmp_path, "set a 1\n BLOCK_LIST")
assert all(item.documentation is None for item in items)
item = next(item for item in items if item.label == "steady_rest")
resolved = lsp_server.on_completion_resolve(item)
assert resolved.documentation.kind == lsp.MarkupKind.Markdown
assert "**Block template** `steady_rest`" in resolved.documentation.value
assert "Text[M60]" in resolved.documentation.value
def test_address_list_item_resolves_to_address_table(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
items = _complete(server, tmp_path, "set a 1\n ADDR_LIST")
item = next(item for item in items if item.label == "SPOS")
resolved = lsp_server.on_completion_resolve(item)
assert "| Format | `Coordinate` |" in resolved.documentation.value
def test_block_list_ignores_variables_and_other_words(tmp_path, monkeypatch):
server, _ = _project(tmp_path, monkeypatch)
for source in ("set x $BLOCK_LIST", "set x MY_BLOCK_LIST", "set x steady"):
@@ -0,0 +1,92 @@
"""Procedures stored in PostConfigurator COMMANDBLOCK properties."""
from pathlib import Path
import lsprotocol.types as lsp # type: ignore
from lsp_server import LSP_SERVER, goto_definition, references
from tools.parser import CustomParser
from tools.semantic_tokens import _Highlighter
from tools.stored_procs import stored_command_names
SOURCE = """proc custom_header {} {}
proc ::ns::output {args} {}
CONF_CTRL_tool set auto_preselect_last_template {custom_header}
CONF_CTRL_moves set return_safety_pos {{::ns::output 1} custom_header}
CONF_CTRL_moves set return_end_of_pgm {4th5th}
CONF_CTRL_tool set auto_preselect_template "custom_header"
set x {custom_header}
"""
def _names(source: str) -> list[tuple[str, int, int]]:
tree = CustomParser().parse(source)
return [name for command in tree.children for name in stored_command_names(command)]
def test_stored_command_names_take_the_first_word_of_braced_list_elements():
# Values like {4th5th} are options, not procedure names.
assert _names(SOURCE) == [
("custom_header", 3, 50),
("::ns::output", 4, 41),
("custom_header", 4, 57),
]
def test_stored_command_names_span_lines_and_skip_non_names():
source = "CONF_x set p {\n\t{first 1}\n\t\"second\"\n\t{$var}\n\t{a-b}\n}\n"
assert _names(source) == [("first", 2, 3), ("second", 3, 3)]
def _document(tmp_path: Path) -> str:
uri = (tmp_path / "stored.tcl").as_uri()
LSP_SERVER.workspace.put_text_document(lsp.TextDocumentItem(uri=uri, language_id="tcl", version=1, text=SOURCE))
LSP_SERVER.update_poco_completion_for_file(LSP_SERVER.workspace.get_text_document(uri))
return uri
def _at(needle: str, occurrence: int = 0) -> lsp.Position:
index = -1
for _ in range(occurrence + 1):
index = SOURCE.index(needle, index + 1)
line = SOURCE.count("\n", 0, index)
return lsp.Position(line=line, character=index - (SOURCE.rfind("\n", 0, index) + 1) + 1)
def test_goto_definition_from_stored_proc(tmp_path):
uri = _document(tmp_path)
for needle, occurrence in (("custom_header", 1), ("custom_header", 2), ("output", 1)):
[location] = goto_definition(
lsp.DefinitionParams(text_document=lsp.TextDocumentIdentifier(uri=uri), position=_at(needle, occurrence))
)
assert location.range.start.line == (0 if needle == "custom_header" else 1)
def test_references_include_stored_procs_but_not_plain_strings(tmp_path):
uri = _document(tmp_path)
found = references(
lsp.ReferenceParams(
text_document=lsp.TextDocumentIdentifier(uri=uri),
position=_at("custom_header"),
context=lsp.ReferenceContext(include_declaration=False),
)
)
# The shared server also holds the files of other tests.
assert [(location.range.start.line, location.range.start.character) for location in found if location.uri == uri] == [(2, 49), (3, 56)]
def test_stored_procs_are_highlighted_only_when_known():
tree = CustomParser().parse(SOURCE)
highlighter = _Highlighter([], {"file": [lsp.CompletionItem(label="custom_header")]})
tree.accept(highlighter, recurse=True)
line = column = 0
functions = []
for token in highlighter.tokens():
column = column + token.offset if token.line == 0 else token.offset
line += token.line
if token.tok_type == "function":
functions.append((line, SOURCE.splitlines()[line][column:column + token.length]))
assert (2, "custom_header") in functions
assert (3, "custom_header") in functions
assert not any(text == "4th5th" for _, text in functions)
assert not any(line in {5, 6} for line, _ in functions)