Abstract

Large language models (LLMs) often call external tools to solve tasks. One effective strategy is for LLMs to write code, enabling them to use complex control flow such as conditionals and loops. Such _code actions_ are typically represented as Python code, since LLMs are proficient at writing it. However, many programming language features that would support more effective code actions are difficult to implement for Python. We propose separating _internal code_ that captures program logic from _external calls_ to tools that interact with the world. New features can then easily be implemented by (1) annotating external calls with the effects relevant to that feature, and (2) modifying the execution of the internal code to track this information. We develop a novel programming language, QUASAR, that implements this idea. To illustrate its utility, we implement several useful features on top of QUASAR to enhance code actions: access control with batched user queries to improve security, autoparallelization of external calls to reduce latency, and conformal prediction for uncertainty quantification to mitigate hallucinations.