Re: Your MCP Doesn’t Need 30 Tools: It Needs Code
A general challenge with MCP today is that the more tools you have, the more you’re contributing to context rot. You’re also limited to rather low amounts of input. On the other hand, if you have an MCP that exposes a programming language, it also indirectly exposes a lot of functionality that it knows from its training.
I've been thinking about how one could design a programming language, knowing that a language model would be used in the crafting of programs in said programming language. Models have trouble generating nested "language", e.g. outputting code inside of JSON (and failing to make the code syntactically correct), or as Armin documents their poor ability at using MCP (data and semantics embedded in JSON).
With existing program languages, we build some compositional tools (functions, classes) and then expect engineers to use those tools to produce every level of abstraction. The models are pretty adept at churning out working code in these syntaxes/semantics, but given the limitations of context size and the context rot problem, there is a desire to reduce the amount of code a model needs to "understand" in order to be effective. I've had a little success in OCaml (and I expect the C/C++ engineers may as well) with defining module interface files (*.mli) for large modules and only handing the interface file to the model when I'm using that module.
I've been wondering if we could lessen the context limitations with a programming language that embraces domain specific languages (DSLs). Warning, sketchy musing follows. Maybe something along the lines of dependently typed languages without the extremely strict type adherence, bordering on a slightly more strict metaprogramming setup. I'm imagining a language where you can define different levels of abstraction, each with its own syntax and semantics, where the level above and below are somehow joined semantically. In this way you could define a DSL for the top level of the application, as a composition of lower level concepts — the model would only need to be considering code at this abstraction level, unless the engineer directs it to work at a lower layer.
As a quick example, take an image storage management system, where images are stored in S3, cached locally, with a well-defined database tracking image metadata and storage data. On the bottom level of the language's abstraction you'd implement operations for interacting with the local filesystem cache, operations for interacting with the database, operations for interacting with S3. On the top abstraction level, you'd define programs in terms of the lower level operations, implementing sync, add-image, remove-image, local-copy-image, etc.
The example is a bit of a toy, given this is totally doable in existing languages (I'm currently implementing this in OCaml), but the different abstraction levels might provide speed and understanding gains for model generation while encouraging engineers to focus on crafting good abstractions rather than the code making up those abstractions.