Summary Generation Task

From GM-RKB
(Redirected from summarization)
Jump to navigation Jump to search

A Summary Generation Task is a linguistic generation task that requires generating summary items for one or more information resources.



References

2024

  • Perplexity
    • A Summary Generation Task is a linguistic generation task that requires generating summary items for one or more information resources.
    • Evaluation:
      • **Human Evaluation**: Often considered the gold standard for evaluating summaries, involving criteria such as coherence, consistency, fluency, and relevance[1][2][3].
      • **Intrinsic vs. Extrinsic Evaluation**: Intrinsic evaluation focuses on the quality of the summary itself, while extrinsic evaluation assesses the usefulness of the summary in downstream tasks like question answering or text classification[6].
    • Human Evaluation Protocols:
      • **SummEval**: A comprehensive toolkit and dataset for evaluating summarization models, including human annotations along dimensions like coherence, consistency, fluency, and relevance[1].
      • **Atomic Content Units (ACUs)**: A protocol for evaluating the salience of summaries by dissecting them into fine-grained content units, aiming for high inter-annotator agreement[2].
      • **Reliability Studies**: Investigations into the reliability of human evaluations, highlighting the need for standardized procedures and best practices[3].
    • Importance of Human Summarization:
      • **Professional Writing**: Summarizing is a crucial skill in professional writing, ensuring that key points are conveyed succinctly and effectively[19].
      • **Communication Skills**: Effective summarization enhances understanding, clarity, and impactful message delivery in various communication contexts[15].
    • Citations:
[1] https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00373/100686/SummEval-Re-evaluating-Summarization-Evaluation
[2] https://aclanthology.org/2023.acl-long.228.pdf
[3] https://aclanthology.org/2021.humeval-1.10.pdf
[4] https://hyperskill.org/learn/step/25430
[5] https://eugeneyan.com/writing/abstractive/
[6] https://arxiv.org/abs/2305.15044
[7] https://openai.com/index/learning-to-summarize-with-human-feedback/
[8] https://paperswithcode.com/task/text-summarization
[9] http://www.gabormelli.com/RKB/Summarization_Task
[10] https://cookbook.openai.com/examples/evaluation/how_to_eval_abstractive_summarization
[11] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10635391/
[12] https://aclanthology.org/2023.emnlp-main.158.pdf
[13] https://www.vskills.in/certification/tutorial/autosummarization-and-manual-summarization-classless-and-classful-routing-protocols-and-convergence/

2023

  • Claude
    • Summarize(input) => output
      • Input can be multi-modal information - text, images, datasets, code, etc.
      • Output is a synthesized text summary
      • Length of output < length of original textual input (if input contains text)
      • Output highlights the most relevant "key points" from input
      • Output provides a high-level overview of the core "essence" or "gist" of input
      • Output conveys the central ideas, concepts, findings, conclusions, or topics contained in the input
      • Output serves as a "briefing" for consumers to quickly familiarize themselves with the input
      • Output identifies important details like: main entities, definitions, facts, conclusions, relationships, methodology, or takeaways from input
      • Output has cohesive, logical flow with clear, concise language constructs

2023

  1. Template:Lb The act of summarising
    Jim is very adept at summarisation.
  2. Template:Lb The process of summarising
    After summarisation, these documents seem much simpler.
  3. Template:Lb A summary; the result of summarising
    Jim's summarisation of the incident was very useful.