Large Language Models (LLMs) have recently attracted rising attention beyond natural language processing, including their integration with Evolutionary Algorithms (EAs) for optimization tasks. This study investigates the use of LLMs to tune the parameters of Metaheuristic Algorithms (MAs) through a structured framework, LLM-EPCO, designed to evaluate their configuration capabilities under standardized experimental conditions. The proposed approach guides the iterative adjustment of key parameters, such as scaling factor, crossover rate, population size, and iteration budget, and compares the resulting configurations against established automatic configurators, namely Optuna and SMAC3. The results show that the LLM-based configurations improve the base algorithms' original parameter settings and achieve competitive performance relative to the reference frameworks.