The purpose of the article is to explore the potential and limitations of using text-based generative artificial intelligence — large language models (LLMs) in education, namely for teaching and learning Chemistry. The main objective of the study is to conduct a comparative analysis of the performance and accuracy of some LLMs in chemical disciplines; and identify key challenges in their application. The LLMs were selected as the most commonly used based on the results of a student survey. The key factor in their popularity among Ukrainian students is their free access. We used LLMs like ChatGPT, Gemini and Copilot for solving tasks related to specific chemical disciplines aimed at assessing their viability for enhancing chemical education in Ukrainian higher education. We used the test questions spanning reproductive and productive levels requiring analysis and logical reasoning. And we conducted a comparative evaluation of AI performance against that of average Ukrainian students. As a result, LLMs showed potential mainly in areas that did not require logical reasoning, but they were generally inferior to students. Key challenges included grasping nuances, abstract concepts, recognizing formulas, equations, limitations of logical reasoning, and language barriers. Although LLMs are promising, their implementation requires addressing the identified limitations