ASSESSMENT OF CHATGPT’S MATH SOLVING ABILITY: THE CASE OF EXERCISES IN GRADE 12 MATH TEXTBOOKS IN VIETNAM

Tang Minh Dung1, , Le Hoang Quan, Nguyen Dinh Anh Hao1, Nguyen Minh Ha1, Le Ngoc Anh Tho1, Pham Minh Thuy1
1 Ho Chi Minh City University of Education, Vietnam

Main Article Content

Abstract

Artificial intelligence (AI) is transforming how we do things. In school, teachers use it to improve lessons or help students. Many math teachers pay attention to the ChatGPT version o1-mini since it can solve math problems well. This study examines how accurately ChatGPT o1-mini can solve math problems for 12th-grade students in Vietnam. We looked at 675 math problems in three 12th-grade math textbooks in Vietnam (Canh dieu, Chan troi sang tao, and Ket noi tri thuc voi cuoc song). The results showed that ChatGPT o1-mini could solve 53.9% of the exercises, and it did the best in Statistics. However, 16% of the responses provided by ChatGPT o1-mini had mistakes, and 30.1% of the responses required further explanation as ChatGPT o1-mini used methods or terms that did not belong in the curriculum. The study also found out which contents have mistakes or wrong ideas, in the answers given by ChatGPT o1-mini. These results give math teachers a clear view of how well AI performs and how correct its answers are. This will help them figure out how to use AI, specifically ChatGPT o1-mini, in their teaching.

Article Details

Author Biography

Tang Minh Dung, Ho Chi Minh City University of Education, Vietnam

GIảng viên

References

Alshahrani, A. (2023). The impact of ChatGPT on blended learning: Current trends and future research directions. International Journal of Data and Network Science, 7(4), 2029–2040. https://doi.org/10.5267/j.ijdns.2023.6.010
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Egara, F. O., & Mosimege, M. (2024). Exploring the integration of artificial intelligence-based ChatGPT into Mathematics instruction: Perceptions, challenges, and implications for educators. Education Sciences, 14(7), 742. https://doi.org/10.3390/educsci14070742
Guler, N. K., Dertli, Z. G., Boran, E., & Yildiz, B. (2024). An artificial intelligence application in mathematics education: Evaluating ChatGPT’s academic achievement in a mathematics exam. Pedagogical Research, 9(2), em0188. https://doi.org/10.29333/pr/14145
Harry, A. (2023). Role of AI in education. Interdisciplinary Journal and Humanity (INJURITY), 2(3), 260–268. https://doi.org/10.58631/injurity.v2i3.52
Hwang, G.-J., & Tu, Y.-F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics, 9(6), 584. https://doi.org/10.3390/math9060584
Ingley, S. J., & Pack, A. (2023). Leveraging AI tools to develop the writer rather than the writing. Trends in Ecology & Evolution, 38(9), 785–787. https://doi.org/10.1016/j.tree.2023.05.007
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities through ChatGPT tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(2), 100115. https://doi.org/10.1016/j.tbench.2023.100115
Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28(12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1
Le, A. V., Bui, T. D., Le, Q. Q., & Vu, V. L. (2023). The ability of ChatGPT to conduct periodic exams in Mathematics and Literature at the secondary school level: Research findings and some initial recommendations. Vietnam Journal of Educational Sciences, 19(02). https://doi.org/10.15625/2615-8957/12310201
Liang, Y., Zou, D., Xie, H., & Wang, F. L. (2023). Exploring the potential of using ChatGPT in physics education. Smart Learning Environments, 10(1), 52. https://doi.org/10.1186/s40561-023-00273-7
Ministry of Education and Training. (2018). Circular of Decree No. 32/2018/TT-BGDDT dated December 26, 2018 for promulgating General Education program. Hanoi.
Niloy, A. C., Bari, M. A., Sultana, J., Chowdhury, R., Raisa, F. M., Islam, A., Mahmud, S., Jahan, I., Sarkar, M., Akter, S., Nishat, N., Afroz, M., Sen, A., Islam, T., Tareq, M. H., & Hossen, M. A. (2024). Why do students use ChatGPT? Answering through a triangulation approach. Computers and Education: Artificial Intelligence, 6, 100208. https://doi.org/10.1016/j.caeai.2024.100208
Rahman, Md. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
Rawas, S. (2024). ChatGPT: Empowering lifelong learning in the digital age of higher education. Education and Information Technologies, 29(6), 6895–6908. https://doi.org/10.1007/s10639-023-12114-8
Relmasira, S. C., Lai, Y. C., & Donaldson, J. P. (2023). Fostering AI literacy in elementary Science, Technology, Engineering, Art, and Mathematics (STEAM) Education in the age of generative AI. Sustainability, 15(18), 13595. https://doi.org/10.3390/su151813595
Shi, B., Huang, L., & Lu, X. (2020). Effect of prompt type on test-takers’ writing performance and writing strategy use in the continuation task. Language Testing, 37(3), 361–388. https://doi.org/10.1177/0265532220911626
Vu, N. H. (2024). Applying artificial intelligence in high schools. Vietnam Journal of Educational Sciences, 20(S2). https://doi.org/10.15625/2615-8957/12420207
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., Shi, E., Pan, Y., Zhang, T., Zhu, D., Li, X., Jiang, X., Ge, B., Yuan, Y., Shen, D., … Zhang, S. (2023). Review of large vision models and visual prompt engineering. Meta-Radiology, 1(3), 100047. https://doi.org/10.1016/j.metrad.2023.100047
Wardat, Y., Tashtoush, M. A., Ali, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
Yeo, S., Moon, J., & Kim, D.-J. (2024). Transforming mathematics education with AI: Innovations, implementations, and insights. The Mathematical Education, 63(2), 387–392. https://doi.org/10.7468/MATHEDU.2024.63.2.387