# The Reality of ChatGPT's Training and Performance
Written on
ChatGPT has generated a considerable amount of attention in recent weeks, much of which has been driven by this platform. Roughly half of my Medium feed consists of get-rich-quick schemes, a quarter features unbelievable personal anecdotes, and the remaining quarter is filled with claims that “ChatGPT is destined to change the world!”
Some of these claims are quite absurd, such as the recent article lauding ChatGPT’s supposed skill in predicting stock market movements. The writer shared the bot’s evaluations of various established companies and dismissed its factual inaccuracies with excuses like, “it hasn’t been trained on this, so it is merely guessing.” They suggested we revisit the predictions in late 2023 to assess their accuracy.
This raises several questions:
- Why celebrate stock market forecasts from a bot that has just made them? You cannot know if they will be accurate or completely off-base.
- Why advise us to check back in 2023 for these predictions? ChatGPT operates as if it is still in 2020, making any forecasts for 12 months ahead relevant only to 2021.
- Many assert that ChatGPT has been trained on a wealth of global knowledge, so why does it frequently make guesses about factual details? Additionally, why doesn’t it clarify when it is uncertain?
I have previously discussed ChatGPT's inaccuracies in sports reporting:
However, I decided to test its capabilities once more. I provided a straightforward prompt to see how it would perform.
Create a 500-word article about the 2013 Detroit Tigers.
>The 2013 Detroit Tigers had a strong season in Major League Baseball (MLB), culminating in a postseason appearance. Under the guidance of manager Jim Leyland and a talented roster, they concluded the regular season with a 93–69 record, securing first place in the American League (AL) Central division.
This information is accurate, which is expected. However, given that ChatGPT mistakenly believes that the notoriously poor 2003 Tigers won the World Series, nothing can be taken for granted.
>One standout player for the Tigers in 2013 was outfielder Miguel Cabrera, who had a remarkable season.
Miguel Cabrera did not play any outfield games that season; his last appearance in that position was for the Florida Marlins on September 18, 2005.
>Cabrera made history by becoming the first player in 45 years to win the Triple Crown, leading the AL in batting average, home runs, and runs batted in (RBI).
Cabrera won the American League Triple Crown in 2012, not 2013. While he did lead the AL in batting average in 2013, he fell short in home runs and RBIs compared to Baltimore’s Chris Davis.
>He was also honored as the AL Most Valuable Player (MVP) for the second consecutive year, solidifying his status as one of the best hitters in the league.
While this is correct, I would argue that his reputation as one of the elite hitters was established long before 2013.
>Another key player for the Tigers was pitcher Justin Verlander, who had an impressive season and was a finalist for the AL Cy Young Award. Verlander ended the season with a 13–12 record and a 3.46 earned run average (ERA), leading the league in strikeouts with 217.
While Verlander’s performance was commendable, he did not actually lead in strikeouts, nor was he a finalist for the AL Cy Young Award. Moreover, naming Verlander as the second key player ignores Max Scherzer, who had a breakout season with a 21–3 record, a 2.90 ERA, and 240 strikeouts.
The lineup also included several other notable players, such as outfielder Austin Jackson, first baseman Prince Fielder, and third baseman Miguel Cabrera. However, this paragraph redundantly mentions Cabrera and overlooks other significant contributors like Scherzer and Anibal Sanchez, who had a stellar season.
>Despite a solid performance from the Tigers, they were unable to progress to the AL Championship Series, as the Athletics won the series in five games.
The Tigers actually triumphed in the AL Division Series against the Athletics in five games, showcasing dramatic moments, including Scherzer’s outstanding relief performance and Verlander’s shutout in Game 5.
>Despite the disappointment of the early exit, the 2013 season was still a successful one for the Tigers, and they will look to build on that success in the future.
Leyland retired after the season, feeling disheartened by the loss to Boston with what he considered his best team. The Tigers went on to win their fourth consecutive AL Central title in 2014 but were defeated in the ALDS by the Baltimore Orioles.
Overall, the 2013 Detroit Tigers were a talented and successful team featuring some of the finest players in the game.
While the Tigers had a formidable lineup, their offense deteriorated rapidly. Conversely, the pitching staff was just beginning to find its stride. Scherzer and Verlander each secured additional Cy Young Awards and are likely to be first-ballot Hall of Famers upon retirement.
>Led by Cabrera’s historic season and strong performances from Verlander and others, the Tigers reached the postseason and competed vigorously. Although they fell short in the AL Division Series, the team's 2013 success highlighted their potential for future achievements.
Reflecting on the prompt, creating a 500-word article should not have posed a significant challenge for an AI with ChatGPT's reported training. Yet, the resulting piece contained substantial factual inaccuracies and overlooked critical players and events, while also failing to meet the word count.