Command line

npm install --global tdqs
# or run it without installing
npx tdqs --help

pip install tdqs
# or run it without installing
uvx tdqs --help

Both install the same tdqs command with the same options, output formats and exit codes. Everything below applies to either.

The deterministic checks, no model. Everything in the specification that is computed rather than judged: context signals, the hard gates, the invocation-cost prefilter for shadowing, and the checklist items a definition can fail on its own. It runs offline and is safe on every pull request.

# a tools/list result saved to disk
tdqs lint --file tools.json

# a local stdio server
tdqs lint --command "npx -y @modelcontextprotocol/server-filesystem ."

# a remote Streamable HTTP server
tdqs lint --url https://mcp.example.com/mcp

# fail the build on warnings too
tdqs lint --file tools.json --fail-on warning

Findings link to the flag or dimension they come from. A lint result is not a score: it tells you what will cost you points before a model reads the description.

The full rubric. Runs the four stages of the specification for every tool, then the server coherence evaluation and the rollup. Bring a key for any OpenAI-compatible endpoint and a fast, inexpensive model. Scores are calibrated to a rubric+model pair: check a model against the calibration corpus before trusting it, and keep it fixed for numbers that compare.

export TDQS_BASE_URL=https://openrouter.ai/api/v1
export TDQS_API_KEY=sk-...
export TDQS_MODEL=your-model-id
export TDQS_REQUEST_OVERRIDES='{"reasoning":{"enabled":false}}'

tdqs score --file tools.json --format markdown

# gate a release on the passing tier
tdqs score --url https://mcp.example.com/mcp --fail-under B

Turn extended reasoning off. Many models reason before they answer unless told not to, which makes a call take a minute instead of seconds, and the calibration examples reproduce with reasoning off. How to say so is provider-specific, so it is an opaque JSON object merged into every request: --request-overrides '{"reasoning":{"enabled":false}}' on OpenRouter, or TDQS_REQUEST_OVERRIDES in the environment.

Or let this site run the model for you. Hosted mode posts the definitions to the API with a key from your account, waits for the report, and prints the same output — with a shareable report URL. It spends one of the account's daily API calls.

TDQS_API_KEY=tdqs_... tdqs score --file tools.json --hosted https://tdqs.dev

--file, --command, --url

Where the definitions come from. Exactly one. --file - reads stdin, and --header "Name: value" (repeatable) sends credentials to a remote server.

--server-name

The name the coherence prompt sees; defaults to the server's own name when connecting to one.

--format

json, markdown or text. JSON matches the published report schema; markdown is for pull request comments.

--output

Write the report to a file instead of stdout.

--fail-on, --fail-under

--fail-on error | warning | never (lint) and --fail-under A | B | C | D (score): what turns a report into a non-zero exit.

--model, --base-url, --api-key

(score) The model configuration, with --concurrency for how many tools are scored at once. Also read from TDQS_MODEL, TDQS_BASE_URL and TDQS_API_KEY.

0

The report was produced and passed the threshold, or no threshold was set.

1

The threshold failed. The report is still printed.

2

Usage error, unreadable input, or a model call that could not complete. The reason is on stderr.

Lint on every pull request and score on the ones that matter. The lint step needs no secret; the score step needs the model key, and a tier below the gate fails the job.

name: TDQS
on: [pull_request]
jobs:
  tdqs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npx tdqs lint --command "node ./dist/server.js" --fail-on warning
      - run: npx tdqs score --command "node ./dist/server.js" --fail-under B --format markdown >> "$GITHUB_STEP_SUMMARY"
        env:
          TDQS_BASE_URL: https://openrouter.ai/api/v1
          TDQS_API_KEY: ${{ secrets.TDQS_API_KEY }}
          TDQS_MODEL: your-model-id
          TDQS_REQUEST_OVERRIDES: '{"reasoning":{"enabled":false}}'

name: TDQS
on: [pull_request]
jobs:
  tdqs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v10.0.1
      - run: uvx tdqs lint --command "python -m my_server" --fail-on warning
      - run: uvx tdqs score --command "python -m my_server" --fail-under B --format markdown >> "$GITHUB_STEP_SUMMARY"
        env:
          TDQS_BASE_URL: https://openrouter.ai/api/v1
          TDQS_API_KEY: ${{ secrets.TDQS_API_KEY }}
          TDQS_MODEL: your-model-id
          TDQS_REQUEST_OVERRIDES: '{"reasoning":{"enabled":false}}'

Every stage is exported on its own, in both languages, so a registry or a gateway can run the deterministic parts, hash definitions the way this site does, or build the prompts and call its own model. The names are the same up to casing: computeContextSignals in TypeScript is compute_context_signals in Python, and a report is the same JSON from either.

import { createLlmClient, lintServer, parseToolDefinitions, scoreServer } from 'tdqs';

const { serverName, tools } = parseToolDefinitions(await response.json());

const lint = lintServer({ serverName, tools });

const report = await scoreServer({
  llm: createLlmClient({ apiKey, baseUrl, model: 'your-model-id' }),
  serverName,
  tools,
});

console.log(report.serverScore.overallTier, report.serverScore.overallScore);

from tdqs import create_llm_client, lint_server, parse_tool_definitions, score_server

parsed = parse_tool_definitions(response.json())
server_name = parsed.server_name or "my-server"

lint = lint_server(server_name=server_name, tools=parsed.tools)

report = score_server(
    llm=create_llm_client(api_key=api_key, base_url=base_url, model="your-model-id"),
    server_name=server_name,
    tools=parsed.tools,
)

print(report.server_score.overall_tier, report.server_score.overall_score)