INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling
ACL 2026 Main Conference
InferenceDynamics is a framework for routing user queries across large language models through structured capability and knowledge profiling.
InferenceDynamics profiles both the capability requirements and knowledge needs of a query, then routes it to the model that offers the best balance between quality and cost. The framework is designed to scale to large model pools and adapt to new models without retraining the router.
This repository contains the code for InferenceDynamics, including routing, evaluation, and benchmark-related components.
We will release the related data and additional resources as soon as possible.