Jeopardy! Game
A Jeopardy! Contest is a real-time multiplayer game that is a real-time QA task.
- Context
- It can be a QA Benchmark Task.
- See: IBM Watson, Jeopardy! (Franchise), Game Show.
References
2023
- (O' Leary et al., 2023) ⇒ Daniel E. O'Leary. (2023). “An Analysis of Watson Vs. BARD Vs. ChatGPT: The Jeopardy! Challenge”. In:AI Magazine
- NOTES:
- It analyzes and compares the performance of three AI systems - IBM's Watson, Google's BARD, and OpenAI's ChatGPT - in answering questions from the TV game show Jeopardy!.
- It finds that BARD and ChatGPT perform similarly to Watson in terms of accuracy on the high confidence Watson questions. All three systems also achieve human expert-level accuracy.
- It shows that BARD and ChatGPT's sets of correct answers are rated as highly similar using the Tanimoto similarity score. However, both systems can provide different or even conflicting answers when given the same question multiple times.
- It reveals that the confidence factor used by Watson to determine whether to attempt answering a question does significantly correlate with its accuracy. But this confidence level does not provide much insight into predicting accuracy for BARD or ChatGPT.
- It indicates that the language used in the long form answers of BARD and ChatGPT does not reveal clear signals of confidence or uncertainty that could indicate likely answer correctness.
- It identifies issues with testing large language models on Jeopardy! questions include lack of reproducibility, with the systems providing different answers to the same question.**
- It notes that characteristics of problematic questions include ambiguity, play on words, and requiring mathematical reasoning.
- It discusses implications such as the difficulty of evaluating systems that are not fully reproducible, and the need for caution in deploying them for real-world applications.
- NOTES:
2023
- (Wikipedia, 2023) ⇒ https://en.wikipedia.org/wiki/Jeopardy! Retrieved:2023-9-30.
- Jeopardy! is an American television game show created by Merv Griffin. The show is a quiz competition that reverses the traditional question-and-answer format of many quiz shows. Rather than being given questions, contestants are instead given general knowledge clues in the form of answers and they must identify the person, place, thing, or idea that the clue describes, phrasing each response in the form of a question.
The original daytime version debuted on NBC on March 30, 1964, and aired until January 3, 1975. A nighttime syndicated edition aired weekly from September 1974 to September 1975, and a revival, The All-New Jeopardy!, ran on NBC from October 1978 to March 1979 on weekdays. The syndicated show familiar to modern viewers and aired daily (currently by Sony Pictures Television) premiered on September 10, 1984.
Art Fleming served as host for all versions of the show between 1964 and 1979. Don Pardo served as announcer until 1975, and John Harlan announced for the 1978–1979 season. The daily syndicated version premiered in 1984 with Alex Trebek as host and Johnny Gilbert as announcer. Trebek hosted until his death, with his last episode airing January 8, 2021, after over 36 years in the role. Following his death, a variety of guest hosts completed the season beginning with consulting producer and former contestant Ken Jennings, each hosting for a few weeks before passing the role onto someone else. Then-executive producer Mike Richards initially assumed the position of permanent host in September 2021, but relinquished the role within a week. Since then, Jennings and Mayim Bialik have served as permanent rotating hosts of the syndicated series. While Bialik was originally arranged to host additional primetime specials on ABC, and spin-offs, the announcement of Jeopardy! Masters in 2023 meant these duties were shared as well. Following Bialik's withdrawal in part of supporting writers and actors due to the 2023 Hollywood labor disputes, Jennings assumed hosting duties for all forms of media.
Currently in its 40th season, Jeopardy! is one of the longest-running game shows of all time. The show has consistently enjoyed a wide viewership and received many accolades from professional television critics. With over 8,000 episodes aired, the daily syndicated version of Jeopardy! has won a record 39 Daytime Emmy Awards as well as a Peabody Award. In 2013, the program was ranked No. 45 on TV Guides list of the 60 greatest shows in American television history. Jeopardy! has also gained a worldwide following with regional adaptations in many other countries.
- Jeopardy! is an American television game show created by Merv Griffin. The show is a quiz competition that reverses the traditional question-and-answer format of many quiz shows. Rather than being given questions, contestants are instead given general knowledge clues in the form of answers and they must identify the person, place, thing, or idea that the clue describes, phrasing each response in the form of a question.