INFO ARTIKEL
Kata kunci:
ChatGPT, Artificial Intelligence, penulisan akademik, pendidikan tinggi, persepsi mahasiswa
ABSTRAK
Penelitian ini menyelidiki pengalaman mahasiswa yang telah menggunakan ChatGPT dalam membantu penulisan akademik. Survei dilakukan untuk mengumpulkan data tentang frekuensi penggunaan, preferensi, dan persepsi mahasiswa terhadap saran-saran ChatGPT dalam meningkatkan kualitas tulisan mereka. Survei juga mengeksplorasi apakah mahasiswa mengalami masalah teknis saat menggunakan ChatGPT dan apakah mereka menemukan alat ini berguna dalam mengatasi kebuntuan menulis. Hasil penelitian menunjukkan mahasiswa lebih memilih ChatGPT dibandingkan dengan alat penulisan lainnya serta menemukan saran-saran yang berguna dan akurat. Sebagian besar mahasiswa merasa ChatGPT memiliki manfaat yang sama untuk berbagai jenis penulisan. Meskipun beberapa masalah teknis dilaporkan, masalah tersebut relatif kecil. Secara keseluruhan, mahasiswa melaporkan peningkatan kepercayaan diri dalam kemampuan menulis mereka setelah menggunakan ChatGPT dan akan merekomendasikan kepada mahasiswa lainnya. Mengintegrasikan ChatGPT ke dalam penulisan dapat membantu mahasiswa mengatasi kebuntuan menulis dan memperbaiki kebiasaan menulis. Penelitian ini menunjukkan ChatGPT merupakan alat yang berguna bagi mahasiswa yang membutuhkan bantuan penulisan dan saran-sarannya dianggap bermanfaat dan akurat.
Introduction
The rapid development of artificial intelligence (AI) has revolutionized many different aspects of our lives, allowing new opportunities and possibilities like never before. The development of such technology has had a significant impact on educational sectors in terms of instructional approaches and student engagement, including English language teaching (ELT). English language teaching (ELT) is one of the educational sectors where the development of artificial intelligence (AI) technology has significantly impacted student engagement and instructional approaches. AI has opened new pathways to enhance both the teaching and learning processes. This integration of AI into education has been transformative, impacting various aspects of teaching and learning (Chen et al., 2022). However, adopting AI in educational settings also raises critical issues of trust, which is essential for the effective use of AI-based educational support systems (Herdiani, Mahayana, & Rosmansyah, 2024).
Within the past few years, teaching English for academic purposes has demonstrated significant changes due to AI-based tools like paraphrasers, grammar checkers, summarizers, citation generators, and co-writers. Students and professionals have widely used automated writing assistants, such as Grammarly, QuillBot, and Wordtune. These tools utilize natural language processing (NLP) techniques to provide grammar corrections, style suggestions, and contextual enhancements, significantly improving writing quality (Godwin-Jones, 2022; Bender & Koller, 2020; Intiser et al., 2023). Oakman (1994) emphasized that automated writing assistants play a critical role in improving the quality of natural language processing. Zhao (2022) explored Wordtune, an AI-powered writing assistant designed to help English writers maintain a coherent writing style and develop effective self-expression.
Chatbot development and integration are some of the most significant advancements in AI applications. Simple pattern matching and string processing techniques initiated the earlier stage, while modern chatbots now incorporate sophisticated knowledge-based models (Hussain, Sianaki, & Ababneh, 2019). These adaptable virtual assistants have seamlessly integrated into formal and informal educational environments, catering to diverse academic needs. Their ability to engage learners through interactive and personalized interactions fosters increased student engagement and facilitates efficient knowledge access (Essel et al., 2022; Karyotaki, Drigas, & Skianis, 2022). However, despite the advancements in AI, not much has been done to investigate the specific applications and effectiveness of AI tools like ChatGPT in the context of academic writing processes. While previous studies have explored the general benefits of AI in education, the specific impact on student writing processes and outcomes remains underexplored. This study aims to address this gap by focusing on students' perceptions and utilization of ChatGPT as a writing assistance tool, which provides valuable insights into its effectiveness and affordances in academic settings.
Presently, the field of AI-powered human-computer interaction is witnessing a significant surge, primarily driven by chatbots that leverage large language models (LLMs). ChatGPT, developed by OpenAI, is a state-of-the-art language model that utilizes deep learning techniques to generate human-like text based on given prompts. Its capabilities extend beyond simple text generation, including complex tasks such as drafting essays, generating ideas, and providing feedback (Radford et al., 2019). ChatGPT has emerged as a knowledgeable, conversational agent, generating sophisticated text and exhibiting lifelike user interactions. It supports students in various areas, including essay writing, research idea brainstorming, literature reviews, paper refinement, and even computer code composition (Owens, 2022). ChatGPT continuously evolves through user interactions, assimilating new information and experiences, and expanding its capabilities.
Understanding students' perceptions of AI tools is crucial for their successful integration into educational contexts. Research indicates that students generally have positive attitudes towards AI writing assistants, appreciating their ability to enhance writing quality and efficiency (Iorga, 2022). However, Kim, G., Kim, J., & Kim, H.K., (2023), have noted concerns regarding the accuracy of suggestions and dependency on technology. In education, ChatGPT has sparked both admiration and controversy. As researchers explore its potential, a prevailing belief emerges: AI-based applications like ChatGPT will become integral to the writing process, much like calculators and computers in mathematics and science (McMurtrie, 2023). Educators encourage educators to integrate such tools into the classroom, engaging students and instructors to harness their transformative abilities, rather than banning them (Sharples, 2022).
However, there has been limited scholarly investigation into students' perceptions and utilization of ChatGPT as a writing assistance tool. Therefore, this study aims to address this research gap by focusing on the following research questions:
- 1. What are students' perceptions of ChatGPT's suggestions for improving their writing, and how do these perceptions differ based on usage frequency and preference over other writing tools?
- 2. To what extent does ChatGPT assist students in saving time when writing, and how does this impact their confidence in their writing abilities?
- 3. How does the use of ChatGPT for writing assistance compare to other writing tools or resources in terms of its effectiveness in overcoming writer's block and improving students' writing habits?
This paper is organized as follows: Section II discusses the methods used in this study; Section III presents the findings and discussion; Section IV addresses the study's limitations; and Section V concludes the paper.
Method
This study employed a mixed-methods approach, combining quantitative and qualitative data to comprehensively understand students' experiences with ChatGPT for academic writing assistance. This approach allowed for a detailed examination of numerical data and personal insights, enriching the findings.
Participants
The study involved 151 undergraduate students from three different English classes across various disciplines at a large public university in Indonesia. The participants were primarily second-year students enrolled in a two-credit English course designed to improve their academic writing skills. They were introduced to several artificial intelligence writing assistance tools, including ChatGPT, to support their writing tasks and assignments throughout the semester. The participants' demographics included 60% female and 40% male students aged 19-24. The participants represented diverse academic disciplines, consisting of 30% from engineering, 25% from social sciences, 20% from natural sciences, 15% from humanities, and 10% from other fields.
Instrumentation
A comprehensive, self-administered online survey was developed to capture a wide range of data regarding the use of ChatGPT. The survey consisted of three sections: Likert scale questions, yes no questions, and open-ended questions. The Likert scale questions measured students' frequency of ChatGPT use, preferences compared to other writing tools, and perceptions of the quality of ChatGPT's suggestions. Example questions included statements like "I frequently use ChatGPT for writing assistance" and "I prefer using ChatGPT over other writing tools," which students rated on a scale from 1 (strongly disagree) to 5 (strongly agree). The yes no questions assessed students' prior experience with ChatGPT, regarding its time-saving potential, and any technical difficulties encountered. Example questions included, "Have you used ChatGPT before this course?" and "Did you encounter any technical difficulties while using ChatGPT?" The open-ended questions allowed students to elaborate on their writing strategies, compare ChatGPT with other tools, describe its impact on overcoming writer's block, and discuss changes in their
writing habits. Sample questions included "How has ChatGPT influenced your writing process?" and "In what ways do you find ChatGPT more or less useful than other writing tools?"
Data Collection
Data were collected over three weeks during the semester. In this study, students were invited to participate and given a brief description of the study, consent forms, and links to the online survey. The information given to the students emphasized the voluntary nature of participation and assured the anonymity and confidentiality of their responses. To ensure data privacy and security, any information provided for this study was kept on a secure platform.
The process underwent several stages in terms of the validity and reliability of the survey instruments. The survey was developed based on established frameworks for assessing AI tools in educational settings (Braun & Clarke, 2006). It was then piloted with a small group of students (n = 20) to refine the questions and ensure their clarity and relevance. Feedback from the pilot study provided some minor revisions to the survey items. The final version of the survey consisted of Likert scale questions, yes no questions, and open-ended questions designed to capture a comprehensive range of data regarding the utilization of ChatGPT. Students were given three weeks to complete the survey, with two follow-up reminders to encourage participation. Of the 200 students invited, 151 completed the survey, resulting in a response rate of 75.5%, indicating strong interest and engagement from the students.
Data Analysis
The data analysis process involved several steps to ensure comprehensive and accurate insights. Descriptive statistics, including measures of central tendency (mean, median) and dispersion (standard deviation), were used to analyze the Likert scale and yes no questions. Inferential statistical tests, such as independent sample t-tests and ANOVA, were employed to examine potential differences in perceptions between subgroups of students based on their prior ChatGPT experience, frequency of use, and academic disciplines. For instance, t-tests were used to compare the mean scores of students with prior experience with ChatGPT versus those without, while ANOVA was used to examine differences across academic disciplines.
The qualitative data from the open-ended questions were analyzed using thematic analysis, following Braun and Clarke's (2006) six-phase framework: familiarization with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, and producing the final report. The qualitative data were analyzed using Taguette software, facilitating the coding and organization of themes. Common themes included the impact of ChatGPT on idea generation, overcoming writer's block, research support, and changes in writing habits. Integrating quantitative and qualitative findings allowed for a comprehensive understanding of students' experiences with ChatGPT, with the mixed-methods approach enhancing the validity and reliability of the findings.
Ethical Considerations
The study adhered to ethical standards to protect participants' rights and confidentiality. Participants were provided with detailed information about the study's purpose, procedures, potential risks, and benefits, and informed consent was obtained from all participants before they completed the survey. The survey responses were collected anonymously, and no identifiable information was linked to the data. Data were stored securely, and access was restricted to the research team. Participation in the study was entirely voluntary, and students could withdraw at any time without any consequences. The university's institutional review board (IRB) reviewed and approved the study protocol in order to ensure the research is in compliance with ethical standards.
Results and Discussions
The descriptive statistics analysis provides insights into students' perceptions regarding using ChatGPT for writing assistance (see Table I). The mean scores for key survey items indicate moderate to high levels of agreement with statements about the usefulness and accuracy of ChatGPT's suggestions. For instance, the mean score for "I found ChatGPT suggestions helpful in improving my writing" was 3.40, with a standard deviation of 1.109, indicating some response variability. Regarding statistical significance, independent sample t-tests were conducted to compare the mean scores of students with prior ChatGPT experience versus those without such experiences. The result indicated significant differences (p<0.05) in perceptions of usefulness and accuracy, suggesting that prior experience with ChatGPT positively influences students' perceptions.
The practical implications of these findings are substantial. The positive perception of ChatGPT's suggestions indicates that integrating such AI tools into writing instruction can enhance students' writing processes and outcomes. Educators can leverage ChatGPT to provide real-time feedback and support, help students overcome writer's block, and improve their writing habits.
| I frequently use ChatGPT for writing assistance. | I prefer using I found ChatGPT ChatGPT suggestions over other helpful in writing improving my tools. writing. | I think ChatGPT's suggestions are accurate. | I think ChatGPT's suggestions are easy to understand. | I think ChatGPT's suggestions are very helpful. | |||
|---|---|---|---|---|---|---|---|
| Valid | 151 | 151 | 151 | 151 | 151 | 151 | |
| N | Missing | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean | 2.68 | 3.53 | 3.40 | 3.47 | 3.59 | 3.68 | |
| Median | 3.00 | 4.00 | 4.00 | 3.00 | 4.00 | 4.00 | |
| Std. Deviation | 1.093 | 1.088 | 1.109 | 0.937 | 1.085 | 0.976 | |
Table I Students' Perception of the Use of ChatGPT
The findings for each survey item are presented below:
I frequently use ChatGPT for writing assistance:
| Mean: 2.68 | Median: 3.00 | Standard Deviation: 1.093 |
|---|
The survey results show that students had a moderate frequency of using ChatGPT for writing assistance, with a mean score of 2.68. The responses were evenly distributed across the frequency scale, as indicated by the median value of 3.00. The standard deviation of 1.093 suggests some variability in usage, implying that usage frequency varies among students.
I prefer using ChatGPT over other writing tools:
| Mean: 3.53 | Median: 4.00 | Standard Deviation: 1.088 |
|---|
According to the data, students, on average, preferred ChatGPT over other writing tools, with a mean score of 3.53. The median value of 4.00 indicates that many students preferred ChatGPT. According to the standard deviation of 1.088, preferences among participants were moderately variable.
I found ChatGPT suggestions were helpful in improving my writing:
| Mean: 3.40 | Median: 4.00 | Standard Deviation: 1.109 |
|---|
As indicated by the results, students generally found ChatGPT's suggestions were helpful in improving their writing, scoring a median of 4.00 on the statement. A significant percentage of students agreed or strongly agreed with the statement, as it was indicated by the median score. A moderate degree of variability in the standard deviation of 1.109 suggests that some students found the suggestions were more helpful than others.
I think ChatGPT suggestions are accurate:
| Mean: 3.47 | Median: 3.00 | Standard Deviation: 0.937 |
|---|---|---|
There was moderate agreement among students concerning ChatGPT suggestions, with a mean score of 3.59 indicating moderate agreement. A median score 4.00 suggests that many students found the suggestions were easy to comprehend. Based on the standard deviation of 1.085, some students found the suggestions were easier to understand than others, indicating moderate variability.
I think ChatGPT suggestions are very helpful:
| Mean: 3.68 Median: 4.00 Standard Deviation: 0.976 |
|---|
As indicated by the study findings, students generally found that ChatGPT's suggestions were moderately helpful. Many students agreed or strongly agreed with the statement, based on a median score of 3.68 and an average score of 4.00. Based on the standard deviation of 0.976, the students showed moderate agreement.
According to the above findings, participants' frequency of use, writing tool preferences, and perceptions of ChatGPT suggestions provided valuable insights. The results indicated moderate usage frequency, with a mean score of 2.68, suggesting regular but not excessive use of ChatGPT. Regarding frequency distribution, the responses were evenly distributed (median score 3.00), with some variability (standard deviation of 1.093), indicating that students relied differently on ChatGPT. Participants displayed moderate preferences for ChatGPT over other writing tools, as indicated by a median score of 4.01 and a mean score of 3.53. There was, however, a moderate level of variation among participants' preferences (standard deviation 1.088), which suggests either a diverse preference or a lower tendency to use ChatGPT. Despite some variability in the perception of helpfulness (standard deviation of 1.109), most participants found that ChatGPT's suggestions were helpful, as indicated by a mean score of 3.40 and a median score of 4.00. Likewise, participants generally agreed that the suggestions were moderately accurate, with a mean score of 3.47 and a median score of 3.00, although opinions varied to some extent (standard deviation of 0.937). Overall, participants agreed with the suggestions, as demonstrated by a mean score of 3.68 and a median score of 4.00, with moderate agreement (standard deviation of 0.976). A key finding of this study is that ChatGPT is an essential tool for participants' writing processes while acknowledging that individual experiences and preferences may influence its effectiveness as a writing assistance tool.
The findings above provide insight into students' experiences with ChatGPT for writing assistance, providing valuable insight for developing AI-based tools in education. In other studies, ChatGPT has been highly effective in assisting students with writing. Nagata (2019) found that even the simplest chatbot can positively assist learners in writing. According to Mijwil (2023), ChatGPT can be used in various applications, including chatbots, language translation, text completion, and answering questions. Hagiwara et al. (2019) propose a protocol and framework for achieving integrated writing assistance environments, enabling researchers and developers to take advantage of the latest developments in natural language processing at a low cost.
Table II Assessment of Prior Experience, Time-Saving Impacts, and Technical Difficulties with ChatGPT using a Yes/No Question
| Statistics | |||
|---|---|---|---|
| I think ChatGPT has helped me become confident in writing. | I will recommend ChatGPT to other students for writing assistance. | ||
| N | Valid | 151 | 151 |
| Missing | 0 | 0 | |
| Mean | 3.26 | 3.52 | |
| Median | 3.00 | 4.00 | |
| Std. Deviation | 1.049 | 1.171 |
The findings for each survey item are presented below:
I think ChatGPT has helped me become confident in writing:
| Mean: 3.26 | Median: 3.00 | Standard Deviation: 1.049 |
|---|---|---|
According to the results, students used ChatGPT to increase their confidence in writing. The mean score indicates that ChatGPT helped students to increase their confidence in writing. According to the median value of 3.00, responses are evenly distributed, whereas the standard deviation of 1.049 indicates a degree of variability in the impact of this questionnaire on students' confidence.
I will recommend ChatGPT to other students for writing assistance:
| Mean: 3.52 | Median: 4.00 | Standard Deviation: 1.171 |
|---|
The findings suggest that students moderately expressed a likelihood of recommending ChatGPT to their peers for writing assistance. The mean score of 3.52 indicates general agreement with this recommendation. The standard deviation of 1.171 indicates moderate variability in the likelihood of recommendation among participants, with an overwhelming percentage strongly agreeing or agreeing. The median value of 4.00 means that a significant percentage of participants strongly agreed or agreed.
Based on the descriptive statistics analysis above, it is evident that ChatGPT has a positive impact on students' confidence in writing and their willingness to recommend it to other students. These findings indicate that ChatGPT appears to positively impact students' confidence and willingness to recommend the program to others. As a result of this study, ChatGPT significantly increased students' confidence in writing. This has resulted in students viewing ChatGPT as an important tool that enhances their writing abilities and self-esteem. As indicated by the moderate mean score of 3.26, students generally agree that ChatGPT positively impacts their writing confidence. The overall trend suggests that ChatGPT plays an important role in boosting students' confidence. However, the standard deviation of 1.049 indicates that the impact varies from student to student.
Students also expressed a moderate likelihood of recommending ChatGPT for writing assistance to other students. In light of the median score of 4.00, it is evident that students either strongly agreed or agreed with the statement. In this case, students view ChatGPT as a helpful tool they believe others can benefit from. The mean score of 3.52 suggests students agree on the likelihood of recommending ChatGPT. It is evident from the standard deviation of 1.171 that students' inclination to recommend ChatGPT may be influenced by their individual experiences and perspectives. Based on their likelihood of recommending ChatGPT to others, students indicate their satisfaction with and confidence in its effectiveness in a way that indicates their perception of it as a valuable resource that improves their writing confidence. ChatGPT has effectively improved students' writing abilities and promoted self-confidence.
Thematic Analysis of Open-Ended Questions
The qualitative data from open-ended survey responses provided more profound insights into students' experiences and perceptions of ChatGPT. Taguette was used to analyze students' responses to the open question as it allows users to encode different segments of text data, making it possible to identify patterns and themes. A screenshot of Taguette with the processed response file can be seen in Figure 1. The left-side tab in Figure 1 illustrates some of the evolved tags. A text segment is highlighted and assigned a tag. New tags (codes) are added as required. If a text segment contains different ideas, Taguette allows multiple tagging.
Figure 1 Example analysis from Taguette
A summary of simple statistics related to the open question can be found in Table III, including how many students participated, how many responses were received, and what themes and codes there were. A coded comment is a sentence or part containing clear ideas that can be coded.
Table III Summary of Survey Participation and Qualitative Data Analysis
| The number of students who participated in the survey | 151 |
|---|---|
| The number of open questions | 4 |
| The number of students responded to the open question | 73 |
| The number of themes | 16 |
| The number of codes | 629 |
The thematic analysis identified several key themes:
Table IV Incorporating ChatGPT into the Writing Process
| 1 0 | ||
|---|---|---|
| Theme | Codes | Sample Responses |
| Using ChatGPT for Generating Ideas and Information | 32 | "I often use ChatGPT to generate new ideas for my writing. It's a great tool to kickstart my creativity and explore different angles." |
| Checking and Correcting Writing | 33 | "One of the main reasons I use ChatGPT is to double-check the accuracy of my writing. It helps me spot any incorrect information and ensure the content is reliable." |
| Comparing and Combining Ideas | 14 | "ChatGPT's ideas are valuable to me, and I often combine them with my own writing. It gives a unique perspective and helps me develop more comprehensive content." |
| Topic Identification and Framework | 12 | "When I'm unsure about the main idea of a text, I rely on ChatGPT to identify it. It helps me to stay on track and ensures my writing aligns with the intended topic." |
|---|---|---|
| Proofreading and Finalizing Writing | 20 | "Paraphrasing and rephrasing content is made easier with ChatGPT. It suggests alternative phrasings and helps me to refine my writing style." |
Based on the analysis of the students' responses, several themes were revealed regarding how ChatGPT was used to generate ideas and information. An important theme was how ChatGPT was used to generate ideas and information. Participants reported using ChatGPT to stimulate their creativity and explore different perspectives (Brown et al., 1989; Nagata, 2019). Moreover, students found it helpful to find resources and references related to their writing topics (Hagiwara et al., 2019). ChatGPT plays a valuable role in generating ideas and retrieving information during the writing process, highlighting its role.
As part of the review process, students mentioned that they used ChatGPT to double-check the accuracy of their writing to identify and correct any incorrect information. This was another theme that emerged. Participants expressed interest in using ChatGPT to compare and combine ideas, proofread, and ensure the accuracy and reliability of content. ChatGPT is a reliable tool for proofreading and ensuring accurate and trustworthy content. Their writing was enhanced by the ideas generated by ChatGPT, which provided a unique perspective and helped them develop more comprehensive content (Mijwil, 2023).
Additionally, students indicated that ChatGPT was a topic identification tool and framework. In cases where students needed clarification on the main idea of a text, ChatGPT assisted them in identifying and staying on track, ensuring they wrote in compliance with the intended topic (Hagiwara et al., 2019). Additionally, students highlighted ChatGPT's usefulness in proofreading and finalizing their writings. In addition to paraphrasing and rephrasing content, alternative phrasings were suggested, and their writing style was improved (Hagiwara et al., 2019). ChatGPT provides valuable assistance at the final stages of the writing process.
Table V Overcoming Writing Challenges
| Theme | Codes | Sample Responses |
|---|---|---|
| Idea Generation and Inspiration | 94 | "It gives me the main and important ideas needed for writing." |
| Overcoming Writer's Block | 73 | "With the fast response of the AI, it is really convenient to use ChatGPT when there is a lack of ideas." |
| Research Support and Understanding | 27 | "It helps in understanding complex texts and provides research support." |
| Timesaving and Efficiency | 19 | "It eases the process of paraphrasing and grammar correction." |
| Feedback and Improvement | 18 | "It helps with grammar and sentence correction, enhancing vocabulary and diction." |
A number of themes emerged from the analysis of responses regarding overcoming writer's block or other writing challenges. Participants indicated that ChatGPT provides ideas and inspiration that help them to overcome writer's block. It is a valuable resource for generating main ideas and guidelines for their writing, effectively overcoming the lack of ideas and inspiration (Liu et al., 2023). Students have also noted that ChatGPT helps understand complex texts, summarize and simplify information, and provide research support (Nagata, 2019). This indicates that ChatGPT assists students in navigating through challenging texts and gathering the necessary information for their writing.
Additionally, participants indicated that ChatGPT is an efficient and time-saving tool. The program made it easier for them to paraphrase, correct grammar, and generate ideas, which helped them to complete their writing tasks more efficiently (Hagiwara et al., 2019). It is clear, therefore, that ChatGPT has practical benefits in improving productivity and saving time. Additionally, students appreciated the feedback and improvements that ChatGPT provided. In addition to enhancing vocabulary and diction, it assisted them in grammar and sentence correction (Hagiwara et al., 2019). Moreover, it provided feedback on essay outlines and thesis statements, improving the essays' overall quality.
Table VI Changes in Writing Habits
| Theme | Codes | Sample Responses |
|---|---|---|
| Positive Changes and Benefits | 87 | "Yes, I got more knowledge with an easy understanding." |
| Minimal or No Changes | 32 | "Not necessarily. I use ChatGPT once in a while as I know the technology isn't too accurate as of yet." |
| Mixed or Neutral Responses | 16 | "Not much. The styles that my lectures ask me to do are very different from GPT, needing specific info or explanation, so I really can't use ChatGPT to finish everything." |
The participants expressed mixed responses to whether ChatGPT had changed their writing habits after incorporating it into their writing process (Peng et al., 2023). Some reported positive changes and benefits, including gaining more knowledge with easy understanding and becoming accustomed to conducting research before writing. Compared to traditional methods, ChatGPT facilitated faster research due to its convenience. Participants showed significant improvement in their writing habits by using ChatGPT as a resource for gathering ideas and information. Some participants, however, indicated minimal or no changes to their writing habits due to using ChatGPT sporadically, recognizing its limitations in terms of accuracy (Bender & Koller, 2020). Participants acknowledged that ChatGPT offers benefits but that its impact on their writing habits could be more substantial. It was noted that their writing abilities and preferences were not entirely dependent upon ChatGPT, indicating that its use may vary among individuals based on their personal preferences and awareness of the limitations of the technology. The adoption and impact of writing tools such as ChatGPT on writing habits may vary among individuals depending on their prior writing practices, writing goals, and comfort level with technology. Researchers may examine the long-term impact of incorporating ChatGPT into the writing process and its potential to influence writing habits comprehensively.
Table VII Comparison with Other Writing Tools or Resources
| Theme | Codes | Sample Responses |
|---|---|---|
| Ease of Use and Accessibility | 66 | "It's very useful and helpful; also, it's easy to use." |
| Variety of Ideas and Information | 29 | "It gives us a variety of ideas instantly." |
| Comparison with Other Writing Tools | 11 | "ChatGPT is better than any other tool as it can correct its own mistakes and is still being developed." |
| Unique Features and Capabilities | 16 | "ChatGPT can answer questions and commands using more languages than any other AI." |
| Personal Preferences and Experiences | 21 | "Using ChatGPT was exciting, and it does a really good job as my writing assistance." |
| Improvement and Future Expectations | 9 | "ChatGPT needs improvement and further development." |
The participants compared their experience using ChatGPT for writing with other writing tools or resources they have used. The analysis revealed several themes, including ease of use and accessibility. The students found ChatGPT to be an effective and helpful tool for writing tasks, emphasizing its simplicity and effectiveness. Considering its user-friendly interface and intuitive features, they thought that it was the most straightforward and useful tool they had previously used. Furthermore, participants highlighted the various ideas and information provided by ChatGPT (Schunk et al., 2020). In addition to offering a diverse range of ideas instantly, ChatGPT assisted students in learning English, identifying main ideas, and structuring paragraphs. As compared to other writing tools, ChatGPT is on par with other writing companions (Iorga, 2022). They appreciated the unique features and capabilities of ChatGPT, such as its ability to answer questions and commands in multiple languages (Radford et al., 2019). Participants also mentioned their access to paid journals and books, which facilitated the search for appropriate references and enhanced the quality of their writing (Peng et al., 2023).
Participants also expressed satisfaction and excitement with ChatGPT as their writing assistance (Malik et al., 2023). They found it more effective than other writing tools because it could correct mistakes, offer suggestions, and provide comprehensive support throughout the writing process. It was beneficial for them to receive feedback and improvement from ChatGPT, which helped them improve their grammar, vocabulary, and the overall quality of their writing. Some participants, however, expressed the need for improvement and further development of ChatGPT (Kim, G., Kim, J., Kim, H.K., 2023). In the future, they expect more accurate paraphrasing and scientific writing capabilities (Hwang et al., 2023). Participants recognized the potential for ChatGPT to evolve and become an even more valuable writing assistance tool, meeting the specific needs of different writing contexts and genres.
Limitations of This Study
This study has several limitations that need to be acknowledged. First, the number of participants was relatively small and specific to a single university, which may limit the generalizability of the results. Future research can try to include a more extensive and diverse sample to enhance the external validity of the findings. Second, the study relied on self-reported data, which tends to be biased. Employing objective writing quality measures and long-term writing improvement assessments could provide more comprehensive insights. Third, the study focused exclusively on ChatGPT as a writing assistance tool without comparing it to similar tools such as Bing or Co-Pilot. Including a broader range of tools in future studies would allow a more comprehensive analysis of the relative effectiveness of different AI writing assistants. Finally, the study investigated students' perceptions and preferences rather than objective measures of writing quality or the development of writing skills as a long-term effect. Future research should incorporate objective assessments of writing outcomes. Addressing these limitations will provide a more comprehensive understanding of the role and effectiveness of AI tools in academic writing.
Conclusion
The present study aimed to investigate students' experiences with ChatGPT for writing assistance, explicitly examining their perceptions and evaluating its effectiveness. By addressing the research questions, we gained valuable insights into the role of ChatGPT in supporting students' writing processes and its potential as a writing assistance tool.
Based on the findings of the first research question, which investigated students' perceptions of ChatGPT's suggestions to improve their writing and how they differed depending on usage frequency and preference over other writing tools, a comprehensive understanding was provided. Based on the study results, participants found that ChatGPT's suggestions were helpful and moderately accurate, demonstrating that ChatGPT can be a useful writing tool. Students engaged with ChatGPT regularly but not excessively, as indicated by the moderate usage frequency level, indicating a balanced reliance on the tool for students. Although individual preferences and diverse tool inclinations were evident, the moderate preference for ChatGPT over other writing tools demonstrated that students viewed ChatGPT as a valuable resource.
The second research question aimed to examine the extent to which ChatGPT assists students in saving time when writing and how this impacts their confidence in their writing abilities. It was determined that ChatGPT improves students' confidence in writing and their willingness to recommend it to others. The results indicate that ChatGPT enhances students' self-esteem and writing self-efficacy by improving their confidence in their writing abilities. As a result of their perception of ChatGPT's effectiveness and value as a writing assistance tool, students were more likely to recommend it to others.
Several key themes emerged from the third research question, which explored the effectiveness of ChatGPT for writing assistance compared to other writing tools or resources in overcoming writer's block and improving students' writing habits. In their evaluation of ChatGPT, participants indicated its convenience as a writing tool due to its ease of use and accessibility. Additionally, they appreciated the variety of information and ideas provided by ChatGPT, emphasizing its value as an effective tool for generating ideas and gathering information. As a result of its unique capabilities and advantages, participants expressed a preference for ChatGPT over other writing tools.
Generally, the findings of this study provide a deeper understanding of students' experiences with ChatGPT for writing assistance. Positive perceptions of ChatGPT's suggestions, its impact on saving time and boosting confidence, and its comparative advantages over other writing tools indicate its potential to be a powerful writing tool. In their responses, participants highlighted the user-friendliness, idea generation, and unique features of ChatGPT, emphasizing its relevance and impact on students' writing processes. This study has some limitations, including a small sample size, a reliance on self-reported data, a narrow focus on one tool, the absence of a control group, a limited scope of analysis, potential bias in participant selection, and a short-term assessment. These limitations must be considered.
Accordingly, these results provide valuable insight into students' perceptions and experiences with ChatGPT for writing assistance. There is evidence that ChatGPT is effective at improving writing skills, saving time, and boosting confidence, and its potential to serve as a valuable educational tool supports its effectiveness. There is a need for further research and development in this area, as this study contributes to the broader discussion of the role of artificial intelligence-based tools for writing assistance. These findings can be used in future studies to explore further aspects of ChatGPT's effectiveness, examine its long-term impacts on writing skill development, and address the identified limitations. ChatGPT can become a valuable tool for supporting students' writing activities when refined and optimized.