Nullspace Gameability Assessment
Overview
We evaluate manipulability of the simplified 15-constraint equity
consistency model by measuring the nullspace of the constraint matrix
A and attempting to craft adversarial perturbations that
preserve all constraints. The analysis uses the synthetic pass/fail
filings dataset delivered with the repository.
Nullspace Diagnostics
| Group | Rank | Nullity | Tags |
|---|---|---|---|
| Overall | 10 | 0 | 10 |
| Yearly (2006–2025) | 10 | 0 | 10 |
| Industry metadata | Not available (dataset missing SIC/NAICS); treated as
unknown |
nullspace_dim_by_industry.csvcaptures the full table of results across all reported years.- With the current tag universe (10 tags), the constraint matrix has full column rank, yielding a trivial nullspace; no degrees of freedom remain for balancing entries. This is expected for the simplified 15-constraint prototype introduced in TASK_02.
Adversarial Experiments
We attempted 20 perturbations across representative tags
(ni, oci, div, adj,
equity_t) with L₁ budgets at 1%, 2%, 5%, and 10% of the
baseline filing magnitude.
- All experiments were infeasible because the nullspace is empty.
adversarial_examples.jsonenumerates each attempt, budget, and achieved delta.
These results confirm that, under the simplified constraint system, attackers cannot introduce hidden balancing entries—every perturbation alters the constraint residuals, immediately triggering detection.
Implications & Mitigations
- Expand tag coverage – The production system targets 287 tags. Integrating the full taxonomy (or the 54-constraint extension) will reintroduce non-trivial nullspace structure and surface realistic gameability vectors.
- Augment constraints – Incorporating calculation-linkbase equations, industry-specific identities, and temporal smoothness constraints will further shrink the feasible manipulation space once additional tags are present.
- Regularization & anomaly detection – When a non-zero nullspace exists, penalizing sparse perturbations (L₁) and benchmarking filings against industry peers/temporal histories helps flag manipulations that exploit residual degrees of freedom.
Limitations
- The current dataset lacks industry metadata, preventing per-industry nullspace comparisons.
- Constraint rank analysis reflects the simplified prototype only; conclusions will change once richer data and the 54-constraint model are available.
Despite these limitations, the tooling is ready for deployment on the full dataset: swapping in an expanded tag universe will automatically regenerate nullspace metrics and adversarial search outputs.