tool-schema-validator

Validate LLM tool / function definitions before you ship them. Zero dependencies, stdlib only.

A malformed tool schema doesn't fail loudly — the API accepts it, the model just never calls the tool, or calls it with arguments you can't parse. This catches the mechanical defects at author time.

Targets the common shape:

{"type": "function", "function": {"name": "...", "description": "...", "parameters": {...}}}

A bare list of {"name", "parameters"} objects is also accepted, as is {"tools": [...]}.

Checks

Errors (things that break tool calling):

• tool name missing, non-string, or not matching ^[a-zA-Z_][a-zA-Z0-9_-]{0,63}$
• duplicate tool names (many APIs reject collisions)
• unknown JSON Schema type ("str", "int", typos)
• required not a list of strings
• required entry that isn't in properties — the model can never satisfy it
• array schema with no items
• enum that isn't a non-empty list

Warnings (things that degrade tool selection):

• missing or very short description
• no parameters key at all
• object schema with no properties
• enum without type
• empty tool list
• duplicate enum values

Usage

python schema_validator.py tools.json
cat tools.json | python schema_validator.py
python schema_validator.py tools.json --json
python schema_validator.py tools.json --strict   # exit 1 on any issue (CI)

Valid example:

$ python schema_validator.py examples/valid.json
valid

Invalid example:

$ python schema_validator.py examples/invalid.json
ERROR   $[0].function.name: Invalid tool name 'get weather!'.
        hint: Use ^[a-zA-Z_][a-zA-Z0-9_-]{0,63}$ (many APIs reject spaces/dots).
ERROR   $[0].function.parameters.properties.city: Unknown type 'str'.
        hint: Known: ['array', 'boolean', 'integer', 'null', 'number', 'object', 'string']
ERROR   $[0].function.parameters.properties.tags: Array schema without 'items'.
        hint: Some APIs reject this outright.
ERROR   $[0].function.parameters: required field 'state' not in properties.
        hint: The API will accept the schema but the model can never satisfy it.
WARNING $[0].function.description: Description is very short.
        hint: Say when to use AND when not to use the tool.

Library

import json, schema_validator as sv

tools = json.load(open("tools.json"))
for issue in sv.validate(tools):
    print(issue.severity, issue.path, issue.message)

• sv.validate(doc) — accepts a list, a single tool, or {"tools": [...]}
• sv.Validator() — reuse for custom flows; accumulates issues
• sv.extract_tools(doc) — normalize input shape
• Issue.as_dict() — structured output

$ref, anyOf, and oneOf are treated as opaque and not recursed into — this is a shape checker, not a full JSON Schema resolver.

CI

- run: python schema_validator.py tools.json --strict

Tests

python -m unittest discover -s tests -v

Limitations

• Not a full JSON Schema validator. It does not resolve $ref, check format, apply pattern, or evaluate numeric bounds.
• It validates schema shape, not whether the tool is a good idea.

License

MIT.