reports / f8u67tchoa
medorqis-mcp-server
7 tools · specification 1.2 · requested 2026-11-04 07:58:04 UTC
Server
- Overall
- A4.9definitions 70% · coherence 30%
- Definition quality
- A4.8mean 4.9 · min 4.6 · 7/7 tools
- Coherence
- A5.0
The MedOrqis MCP server exhibits high coherence with clearly differentiated, consistently named, appropriately scoped, and complete tools. The documentation explicitly disambiguates overlapping concerns, and no shadowing risks are present among the provided candidates.
- 5
- DisambiguationEach tool has a clear, distinct purpose: clinical scoring, red-flag triage, drug interaction screening, evidence verification, PHI scrubbing, prompt security scanning, and terminology normalization. The 'When NOT to use' sections explicitly cross-reference other tools, eliminating ambiguity.
- 5
- Naming ConsistencyAll tool names follow a consistent verb_noun pattern: calculate, check, verify, scrub, scan, normalize. The verbs are specific and the nouns are concise, making the naming predictable and semantically clear.
- 5
- Tool Count AppropriatenessSeven tools is a well-scoped set for a medical safety platform, covering calculation, safety checks, evidence, privacy, security, and terminology without overlap or bloat. Each tool is essential and earns its place, and the count is within the optimal range.
- 5
- CompletenessThe tool surface comprehensively covers the domain of clinical decision support and safety: risk scoring, emergency triage, drug interactions, evidence verification, PHI protection, prompt security, and terminology. There are no obvious gaps—each tool addresses a distinct functional need, and the 'When NOT to use' sections indicate thoughtful coverage.
Tools
calculate_clinical_score
555445A4.8This is an exceptionally well-crafted tool description that sets a high standard. It clearly defines purpose, provides explicit usage guidelines with named alternatives, and offers deep behavioral transparency including safety mechanisms and output contracts. The parameter semantics are well-supported by both schema and description, and the structure is logical and complete. The only minor gap is that some calculators (e.g., Wells scores) are not fully parameterized in the description, but the schema covers them, so this is acceptable.
- 5
- Purpose Clarity · 25%The description clearly states the tool executes validated mathematical clinical calculators, listing specific calculator names and their clinical domains. It distinguishes itself from siblings by explicitly naming alternatives like check_red_flags and check_drug_interactions, and the verb 'Executes' plus the resource 'validated mathematical clinical calculators' is specific and unambiguous.
- 5
- Usage Guidelines · 20%The description provides explicit 'When to use' and 'When NOT to use' sections, naming specific alternative tools (check_red_flags, check_drug_interactions, normalize_terminology) and the conditions under which they should be used instead. This is exemplary guidance for an agent to select the correct tool.
- 5
- Behavioral Transparency · 20%The description goes far beyond the annotations (readOnlyHint, idempotentHint, destructiveHint) by detailing deterministic arithmetic, stateless execution, boundary validation with specific physiological limits, and the SafeCalculationRefusal return state. It also explains the output contract, which is not covered by annotations. This is rich behavioral context that helps the agent understand side effects and failure modes.
- 4
- Parameter Semantics · 15%The schema already has 100% coverage with detailed descriptions for each parameter, so the baseline is 3. The description adds value by mapping parameters to specific calculators (e.g., 'ckd_epi_2021_egfr: age_years (>=18), biological_sex...') and providing additional constraints like age ranges and units. However, it doesn't fully enumerate every parameter for every calculator (e.g., wells_dvt_score is just 'corresponding clinical parameters'), so it's not a perfect 5.
- 4
- Conciseness & Structure · 10%The description is well-structured with clear sections (When to use, When NOT to use, Parameter constraints, Behavioral guarantees) and is front-loaded with the core purpose. It is long but every section adds necessary information for a complex tool with 12 calculators. The only minor issue is that the parameter list is extensive and could be seen as slightly verbose, but it's justified given the tool's complexity.
- 5
- Contextual Completeness · 10%Given the tool's high complexity (12 calculators, many parameters, output schema), the description is remarkably complete. It covers usage, exclusions, parameter constraints, boundary validation, return states, and output contract. The output schema exists, so the description doesn't need to detail return values, but it still summarizes the output structure. This is comprehensive for an agent to use the tool correctly.
- params
- 2 (2 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 4
- bytes
- 9535
- hash
- 603b2adaaae3aa11
check_red_flags
555555A5.0This is an exemplary tool description that fully leverages the available space to provide clear purpose, usage guidelines, parameter semantics, and behavioral transparency. It effectively differentiates from siblings and complements the schema and annotations, making it a top-tier reference for MCP tool descriptions.
- 5
- Purpose Clarity · 25%The description clearly states the tool evaluates a clinical consultation envelope against multi-layered safety guardrails, detecting emergencies, DDIs, conflicts, and missing data. It distinguishes from siblings by naming specific alternatives like calculate_clinical_score and check_drug_interactions.
- 5
- Usage Guidelines · 20%Provides explicit 'When to use' and 'When NOT to use' sections with concrete alternative tools for each exclusion, making it unambiguous when to select this tool over siblings.
- 5
- Behavioral Transparency · 20%Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and the description adds deterministic pipeline details, statelessness, zero uncaught exceptions, and specific gate_status outcomes, going beyond annotations.
- 5
- Parameter Semantics · 15%Schema coverage is 100%, but the description adds rich semantics for the 'envelope' parameter, detailing each subfield's purpose (e.g., observations scanned for emergency triggers, active_medications for DDIs) and providing examples, which exceeds baseline.
- 5
- Conciseness & Structure · 10%Well-structured with clear sections, front-loaded purpose, and every sentence adds value. No fluff; the length is justified by the complexity of the tool.
- 5
- Contextual Completeness · 10%Given the tool's complexity, the description covers all necessary aspects: when to use, parameter formats, behavioral guarantees, and return states. Output schema exists, so return values are not over-explained, but the gate_status outcomes are described.
- params
- 1 (1 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 4
- bytes
- 7335
- hash
- f0d5de80ce728e3d
check_drug_interactions
555555A5.0This is an exemplary tool description that excels across all dimensions. It provides clear purpose, explicit usage guidelines with alternatives, rich behavioral transparency, detailed parameter semantics, and a well-structured, complete presentation. The description adds significant value beyond the schema and annotations, making it a model for high-quality MCP tool definitions.
- 5
- Purpose Clarity · 25%The description clearly states the tool's purpose: screening medication regimens for critical, contraindicated, or fatal drug-drug interactions, with additional capabilities like brand name resolution and demographic contraindication evaluation. It distinguishes itself from siblings by explicitly naming alternatives (check_red_flags, normalize_terminology, calculate_clinical_score) and specifying what it does not do.
- 5
- Usage Guidelines · 20%The description provides explicit 'When to use' and 'When NOT to use' sections, naming specific alternative tools for each exclusion case. This gives the agent clear decision criteria for tool selection, going beyond mere purpose to actionable guidance.
- 5
- Behavioral Transparency · 20%The description discloses key behavioral traits: deterministic rule engine, statelessness, no external calls, no patient storage, severity stratification with categories, and error handling behavior (returns count 0 for clean regimens, reports unrecognized names in warnings). This adds significant context beyond the annotations (readOnlyHint, idempotentHint, etc.) and aligns with them, providing a complete picture of the tool's behavior.
- 5
- Parameter Semantics · 15%The description adds substantial meaning beyond the schema: it explains the purpose of each parameter, provides examples, clarifies enum semantics (e.g., pregnancy status values and their implications), and specifies constraints like age range and sex-linked risks. Even though schema coverage is 100%, the description enriches the understanding of how parameters affect behavior.
- 5
- Conciseness & Structure · 10%The description is well-structured with clear sections (When to use, When NOT to use, Parameter constraints, Behavioral guarantees) and is appropriately detailed for a complex clinical tool. It is front-loaded with the core purpose, and every sentence adds value without redundancy.
- 5
- Contextual Completeness · 10%Given the tool's complexity (2 params, nested object, output schema present), the description is comprehensive: it covers purpose, usage, parameters, behavior, and error handling. The presence of an output schema reduces the need to explain return values, but the description still provides severity categories and return states, making it complete for an agent to use effectively.
- params
- 2 (1 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 1
- bytes
- 4948
- hash
- ee03975259b59adf
verify_evidence
555455A4.9This is an exemplary tool description that fully leverages the available structured data while adding significant behavioral and usage context. It clearly differentiates from siblings, provides explicit usage guidelines, and discloses deterministic behavior and output states. The description is thorough yet concise, making it a top-tier example for agent tool selection.
- 5
- Purpose Clarity · 25%The description clearly states the tool's two-stage purpose: validating bibliographic citations and verifying clinical claim entailment against guidelines. It uses specific verbs ('validates', 'verifies') and names the resource ('authoritative guideline corpora'), and it distinguishes itself from siblings by explicitly listing what it does not do.
- 5
- Usage Guidelines · 20%The description provides explicit 'When to use' and 'When NOT to use' sections, naming alternative tools (calculate_clinical_score, check_red_flags, scrub_phi) for each exclusion. This gives the agent clear decision criteria for tool selection.
- 5
- Behavioral Transparency · 20%The description goes beyond the annotations (readOnlyHint, idempotentHint, etc.) by detailing the deterministic offline execution, the exact return states (SUPPORTED/REFUTED/UNVERIFIED), GRADE certainty levels, and the output JSON structure. It also states that no uncaught exceptions are thrown, which is valuable behavioral context not present in annotations.
- 4
- Parameter Semantics · 15%The schema already provides 100% coverage with descriptions and examples for all parameters. The description adds value by explaining the purpose of each parameter in the context of the verification pipeline (e.g., patient_age_years for age-dependent scope, patient_is_pregnant for pregnancy-specific contraindications). It does not repeat schema details but clarifies their role in the tool's logic.
- 5
- Conciseness & Structure · 10%The description is well-structured with clear sections (When to use, When NOT to use, Parameter constraints, Behavioral guarantees). It is detailed but every sentence adds value, and the use of bullet points and bold text improves scannability. No fluff or redundancy.
- 5
- Contextual Completeness · 10%Given the tool's complexity (two-stage verification, multiple parameters, output schema), the description is comprehensive. It covers input formats, output structure, behavioral guarantees, and usage boundaries. The output schema exists, so the description doesn't need to explain return values in detail, but it still summarizes the key output fields, making it complete for an agent to use effectively.
- params
- 4 (2 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 2
- bytes
- 5020
- hash
- d8649efea6f4edb2
scrub_phi
555555A5.0This is an exemplary MCP tool description that leaves no gaps for an AI agent. It clearly defines the purpose, provides explicit usage guidance with alternative tools, discloses behavioral guarantees (deterministic, stateless), and thoroughly documents the single parameter. The structured sections and comprehensive detail earn top scores across all dimensions.
- 5
- Purpose Clarity · 25%The description clearly states it 'Scans clinical narratives and de-identifies Protected Health Information (PHI)' with a specific list of identifier types (HIPAA Safe Harbor, Egyptian IDs, etc.). It distinguishes itself from siblings like scan_prompt_security and normalize_terminology by specifying what it does not do, ensuring no confusion.
- 5
- Usage Guidelines · 20%Provides explicit 'When to use' and 'When NOT to use' sections, naming alternative tools for different tasks (scan_prompt_security, calculate_clinical_score, normalize_terminology). This is exemplary guidance for tool selection.
- 5
- Behavioral Transparency · 20%Discloses that it is a non-LLM deterministic transformation, token substitution format, output contract (JSON with audit_report), and states statelessness with zero disk logging or network leakage. This adds valuable context beyond the annotations, which declare read-only and idempotent hints.
- 5
- Parameter Semantics · 15%The single parameter 'text' is thoroughly described with its purpose, a detailed example, and constraints (minLength, max 500k characters). The description reinforces the schema's coverage (100%) by adding context about acceptable input types (clinical narratives, consult notes) and the transformation behavior.
- 5
- Conciseness & Structure · 10%The description is well-structured with clear sections (Overview, When to use, When NOT to use, Parameter constraints, Behavioral guarantees). Each section earns its place, providing essential information without redundancy. It is long but necessary for the complexity of the tool.
- 5
- Contextual Completeness · 10%Given the tool's complexity (many ID types, regulatory implications, output contract) and the single parameter, the description is complete. It covers safety (stateless, no logging), output structure, and usage boundaries. The output schema is present, so return details are not needed, but the description still summarizes the audit report.
- params
- 1 (1 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 1
- bytes
- 3159
- hash
- a918a1c2b839b816
scan_prompt_security
555455A4.9This is an exemplary tool description that excels in all dimensions. It clearly defines the purpose, provides explicit usage guidelines with alternatives, and offers rich behavioral transparency beyond the annotations. The parameter semantics are well-supported by both schema and description, and the structure is concise yet comprehensive. It sets a high standard for security-focused tools.
- 5
- Purpose Clarity · 25%The description clearly states the tool's purpose: analyzing untrusted inputs for adversarial attacks and prompt injections. It uses specific verbs ('Analyzes', 'sanitizing', 'vetting') and names the resource ('untrusted user inputs, clinical prompts, or external dialogue'). It distinguishes from siblings by listing specific attack types and clinical guardrail bypasses, which is unique among the sibling tools.
- 5
- Usage Guidelines · 20%The description provides explicit 'When to use' and 'When NOT to use' sections, naming alternative tools (scrub_phi, check_red_flags, verify_evidence) for excluded scenarios. This gives clear guidance on when to select this tool over siblings, which is exemplary.
- 5
- Behavioral Transparency · 20%The description goes beyond annotations by detailing the deterministic regex engine, statelessness, zero side effects, and specific return states (is_safe flag, violations array). It also explains error handling for empty or non-string inputs. This adds significant behavioral context that annotations (readOnlyHint, idempotentHint) only partially cover, and it does not contradict them.
- 4
- Parameter Semantics · 15%The schema already provides 100% coverage with a detailed description and example for the 'text' parameter. The description adds value by explaining the parameter's purpose in context (scanning for jailbreak patterns) and providing a concrete example of a malicious input. This is slightly above the baseline of 3 because it reinforces the security context, though the schema already does most of the work.
- 5
- Conciseness & Structure · 10%The description is well-structured with clear sections for usage, constraints, and behavior. It is comprehensive but not verbose; every sentence adds value. The use of bullet points and bold headers improves scannability, and it front-loads the core purpose before diving into details.
- 5
- Contextual Completeness · 10%Given the tool's complexity (security scanning with multiple attack types) and the presence of an output schema, the description is complete. It covers usage, exclusions, parameter constraints, behavioral guarantees, and error handling. The output schema likely details the violations array, so the description doesn't need to repeat that. This is a well-rounded description for a security tool.
- params
- 1 (1 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 1
- bytes
- 2543
- hash
- c79592adde9c46ee
normalize_terminology
554445A4.6This is a high-quality tool description that clearly defines its purpose, provides explicit usage guidelines with alternatives, and offers rich behavioral transparency beyond the annotations. The parameter semantics are enhanced with parsing details, and the structure is well-organized. It is a model example of a description that adds significant value over the schema and annotations.
- 5
- Purpose Clarity · 25%The description clearly states the tool's purpose: mapping regional pharmaceutical trade brand names to standardized INN generic entities, resolving therapeutic drug classes, and retrieving WHO ATC codes. It provides concrete examples (Concor, Augmentin, Plavix) and distinguishes it from sibling tools by specifying what it does not do (e.g., drug interactions, red flags, clinical scores).
- 5
- Usage Guidelines · 20%The description explicitly states when to use the tool (resolving commercial brand names for prescribing audits or clinical reasoning) and when NOT to use it, naming specific alternative tools (check_drug_interactions, check_red_flags, calculate_clinical_score). This provides clear guidance for tool selection.
- 4
- Behavioral Transparency · 20%Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: it is grounded in a curated offline dictionary, stateless with no network dependencies, and describes exact mapping outcomes and error handling (returning nulls and matched=false without throwing). This goes beyond annotations, though it could mention pagination or response size, but for a single-parameter tool, this is sufficient.
- 4
- Parameter Semantics · 15%Schema description coverage is 100% (the schema describes brand_name as a commercial pharmaceutical brand name with examples). The description adds extra meaning by specifying that the parameter parses dosage strengths, salt formulations, and regional trade spellings, and provides additional examples like 'Concor 5mg' and 'Augmentin 1g'. This enriches the schema's basic description, so a score above baseline is justified.
- 4
- Conciseness & Structure · 10%The description is well-structured with clear sections (When to use, When NOT to use, Parameter constraints, Behavioral guarantees). It is somewhat verbose but every section adds value. The front-loading of the main purpose is effective, and the use of bullet points aids readability. It could be slightly more concise, but it is not bloated.
- 5
- Contextual Completeness · 10%Given the tool's moderate complexity (one parameter, no nested objects, output schema exists), the description is complete. It covers purpose, usage guidelines, parameter format, behavioral guarantees, and error handling. The output schema likely documents return fields, so the description need not repeat them. The description fully equips an agent to use the tool correctly.
- params
- 1 (1 required, 100% described)
- annotations
- readOnly=true destructive=false idempotent=true openWorld=false
- output schema
- yes
- cost
- 1
- bytes
- 2976
- hash
- c148d35be84d898d
Deterministic findings
No deterministic findings: every hard gate and checklist item passes.
Automate
The same check from a terminal, a CI job or a script, with an API key from your account. Submitting spends one of the account's daily calls; reading the result spends nothing.
The exit code is the verdict, so the command is the gate. A stdio command or a server URL can stand in for the file.
export TDQS_API_KEY=tdqs_...
# exit code 1 below B, so the command is the gate; uvx runs the same command from PyPI
npx mcp-tdqs score --file tools.json --hosted https://tdqs.dev --fail-under BSubmitting answers at once with the report's address. Its score is null until the status is completed, and the overall tier is on the server score.
curl https://tdqs.dev/api/v1/reports \
-H 'Authorization: Bearer tdqs_...' \
-H 'Content-Type: application/json' \
-d @tools.json
# poll the report it names until "status" is "completed"; this one is
curl https://tdqs.dev/api/v1/reports/f8u67tchoa \
-H 'Authorization: Bearer tdqs_...'The package submits the definitions and polls until the report is done; what comes back is the same object the API returns. The scorer itself is exported too, for a model of your own.
import { requestHostedReport } from 'mcp-tdqs';
const report = await requestHostedReport({
apiKey: 'tdqs_...',
hostedUrl: 'https://tdqs.dev',
serverName: 'medorqis-mcp-server',
tools, // the tools/list result
});
if (report.score === null || !['A', 'B'].includes(report.score.serverScore.overallTier)) {
throw new Error(`TDQS ${report.score?.serverScore.overallTier ?? 'failed'}: ${report.url}`);
}The same helper for Python: it submits the definitions, polls until the report is done, and returns the report as a pydantic model. The scorer itself is exported too, for a model of your own.
from tdqs import parse_tool_definitions, request_hosted_report
parsed = parse_tool_definitions(tools_list) # the tools/list result
report = request_hosted_report(
api_key="tdqs_...",
hosted_url="https://tdqs.dev",
server_name="medorqis-mcp-server",
tools=parsed.tools,
)
tier = report.score.server_score.overall_tier if report.score is not None else None
if tier not in ("A", "B"):
raise SystemExit(f"TDQS {tier or 'failed'}: {report.url}")