With ChatGPT by my side for the past three weeks, I created a 300-page book containing a full humanities curriculum and a 200-page book containing a combination textbook and syllabus for an entrepreneurship class. (Here is yesterday’s post announcing the entrepreneurship book.) While this effort has given me an appreciation of the tool’s strengths, it also highlighted the current ChatGPT weaknesses (version 3.5): URLs, believability, and time/effort.
URLs for articles
When asked to produce URLs for articles (for citations or just generally as resources), ChatGPT’s answer would consist of the following:
- A page title with a URL that points to a page that doesn’t exist
- A page title with a URL for a different article
- The URL for the correct article.
The first two made up at least 95% of the results. Mind you, if the article had existed, the article titles would have led to perfectly reasonable articles. As a result, I would use the title of the non-existent article as a search phrase for a replacement article.
Believability
ChatGPT writes extremely reasonable and definitive-sounding text. In terms of tone and argument structure, ChatGPT always generates text that is simply screaming to be believed. However, much of the world isn’t definitive and is much greyer. Given that, much of what ChatGPT generates needs to be validated before it can be relied upon.
Time/effort
ChatGPT is a large language model and, as such, some nuances of human existence are difficult for it to grasp. It has seemed to me that it does not understand time and effort. For example, in my first book (that defined a humanities curriculum), ChatGPT would assign Crime and Punishment to the student to read…in a week…along with two other books. For this entrepreneurship class, it asks the students to develop the SaaS website in one week, assemble the organizational team in another week, etc.
If I were to use ChatGPT to assign tasks to people, I would need to check these assignments for reasonableness.
Conclusion
ChatGPT has demonstrated to me that it can be a valuable aid. While here I am pointing out important ChatGPT weaknesses, these are not going to keep me from continuing to push it to see how else it can help me.
