// symbolic policy study
ISRE is an experimental system for simplifying algebraic expressions. It breaks an expression into a tree, lists the legal simplification moves, and asks a small neural policy to choose the next move. The core question is simple: can a KAN policy do this job with fewer parameters, less wasted compute, and more interpretability than an MLP?
// plain terms
The algebra rules are already known. The hard part is choosing the right next rule, at the right place in the expression, while keeping the route short and clean.
Interpretable Symbolic Reasoning Engine. It takes an algebra expression, turns it into an AST tree, lists possible simplification moves, chooses one move, applies it, and repeats. The goal is a correct simplification path with visible reasons for each step.
The deterministic part. It uses fixed algebra rules to check the current expression and return legal moves such as REMOVE_ZERO, FOLD_CONST, MERGE_POWER, SORT_COMMUTATIVE, EXPAND, or COLLECT_TERMS.
The learned decision-maker. Given the current expression and the legal moves, it scores every candidate action and chooses the next one. Weak choices can still simplify the expression while making the path longer and less optimal.
The standard neural baseline. An MLP is a stack of dense layers. It is familiar, strong, easy to train, and a fair comparison point. The drawback is that its decisions are hard to inspect: it can work well without giving a clean view of which symbolic features mattered.
The alternative policy head. KAN can be more interpretable because it learns small feature-wise functions instead of mixing everything through dense matrix layers. In this project, raw KAN had too little representation power on its own and needed an encoder.
The successful version. First, an AST encoder reads the expression tree and builds a richer representation. Then KAN scores the legal moves. This version became competitive while staying much smaller than the large MLP baseline.
We compare raw KAN, KAN with an action embedding, Encoder-KAN, small MLPs, and a larger MLP-128 baseline. The main question is which policy chooses near-optimal simplification steps with fewer parameters and less waste.
// mechanism
For every state, the symbolic engine produces a menu of legal actions. The neural policy ranks that menu. This separation keeps algebraic legality deterministic, learns the judgment step, and lets the route be measured against BFS-optimal paths.
(x + 0) * x^2
REMOVE_ZERO / MERGE_POWER / FOLD_CONST
rank(node, action)
x^3
// concrete choices
Each state becomes a small menu. The correct move depends on local algebra and on the global expression context.
CASE_01
The engine finds a zero term inside an addition. The policy should prefer the simplification that changes nothing semantically and reduces the tree.
state: Add(x, 0)
candidates: REMOVE_ZERO, SORT_COMMUTATIVE
chosen: REMOVE_ZERO -> x
CASE_02
For multiplicative structure, a variable can behave like a hidden first power. The policy must learn that this is part of the same algebraic pattern.
state: Mul(x, Pow(x, 2))
candidates: MERGE_POWER, SORT_COMMUTATIVE
chosen: MERGE_POWER -> Pow(x, 3)
CASE_03
The engine offers constant folding when a numeric subtree is available. The policy learns when that local compression advances the full canonical path.
state: Add(2, 3, x)
candidates: FOLD_CONST, SORT_COMMUTATIVE
chosen: FOLD_CONST -> Add(5, x)
// comparison
Raw KAN used thin hand-made features. The MLP family had stronger learned representations in the comparison, so raw KAN tested a narrow version of the idea. Encoder-KAN added an AST encoder before the KAN head, giving KAN a comparable expression representation. With that encoder, the KAN policy became much smaller than MLP-128, beat it on greedy rollout, and nearly matched the strongest micro-MLP.
// findings
KAN became strong after the encoder gave it a useful expression representation. At that point the model became small, competitive, and easier to reason about.
Encoder-KAN reached 94.63% mean greedy BFS-optimal with 15,528 parameters. The MLP-128 baseline had 474,753 parameters and reached 89.4% greedy BFS-optimal. That is roughly a 30x parameter reduction while improving the rollout result.
This run used parameter count and model size as the practical efficiency evidence. Fewer parameters usually mean less memory movement and less compute per step. In practical terms, Encoder-KAN is the lighter policy to train and run than the large MLP baseline.
KAN is attractive because its learned pieces can be inspected more like feature-wise curves. MLP dense layers are effective and familiar, yet harder to turn into a clear rule-level story.
The strongest micro-MLP h32 slightly wins greedy rollout: 95.0% versus 94.63% for the best stable Encoder-KAN group. The result is a competitive KAN policy with the right encoder and a much smaller parameter count.
// larger frame
Modern Transformer systems are built from attention plus large feed-forward blocks. In many architectures, the feed-forward part is MLP-like. Mixture-of-Experts scales that idea by routing tokens through different expert networks. This makes MLP-style computation one of the main workhorses of current AI systems.
Transformers use learned dense computation as a central building block. MoE adds routing and many experts, increasing capacity while trying to spend compute selectively.
The question is whether every small decision module needs to be an MLP-shaped black box. Symbolic reasoning tasks give a clean place to test alternatives because every legal move and every rollout path can be inspected.
KAN may offer a different computational apparatus: smaller, more structured, and easier to inspect when the input representation is good. In ISRE, raw KAN struggled, while Encoder-KAN became competitive after receiving an AST representation.
This project is a controlled miniature. It avoids a direct replacement claim for Transformer MLP blocks. It shows a place where a KAN-based scorer can compete with MLP baselines while using far fewer parameters.
// geometry
Several wider heads produced weaker rollout quality. The best point emerged from representation quality, bottleneck width, and encoder message-passing rounds.
// raw-backed ledger
The strongest micro-MLP narrowly leads greedy rollout. Encoder-KAN matters because it resolves the raw KAN representation gap, beats the large MLP-128 baseline, and reaches the same beam-5 regime with far fewer parameters.
| Model | Family | Params | Greedy | Beam-5 | Overhead |
|---|
// conclusion
KAN needed a representation that respected the structure of the problem.