arXiv:2404.08164v3 Announce Type: replace Abstract: A prompt is an instruction or description provided by the user to guide a generative language model toward outputs that meet application requirements. When these requirements change, existing prompts may need to be adapted and improved. Evaluating candidate prompts requires time and computational resources. We therefore study how to expand and refine an initial set of prompts under a limited evaluation budget through simulation optimization, aiming to maximize a pre-defined task score for a fixed language model. Specifically, we propose a two-stage framework. In the first stage, we expand the initial set of prompts into a finite candidate set, where each prompt is represented by a moderate-dimensional vector. In the subsequent stage of evaluation and selection, we construct a surrogate model of the score for moderate-dimensional vectors representing the prompts. We propose sequentially selecting prompts for evaluation using the constructed surrogate model. We prove the consistency of the sequential evaluation procedure in our framework. We further introduce a refinement procedure that uses the collected scores to search beyond the initial candidate set. We also conduct numerical experiments to demonstrate the advantages of our method over baseline prompt selection methods, with higher prompt scores, especially when the budget for calling language models to evaluate prompts is limited.