2023 GenerativeAIatWork

• (Brynjolfsson et al., 2023) ⇒ Erik Brynjolfsson, Danielle Li, and Lindsey R Raymond. (2023). “Generative AI at Work.” doi:10.3386/w31161

Subject Headings:

Notes

• Generative AI: The study examines an AI system providing customer service, demonstrating productivity improvement
  • QUOTE: "The tool we study is built on a recent version of gpt of large language models... Access to the tool increases productivity, as measured by resolution rate, by 14% on average"
• Skill-biased Technical Change: Unlike technology adoption, this AI system benefits novice workers more than expert workers
  • QUOTE: "In contrast to studies of prior waves of computerization, we find that these performance gains accrue disproportionately to less-experienced workers and low-skill workers"
• Tacit Knowledge: The AI system captures and transfers implicit knowledge that's typically hard to codify
  • QUOTE: "This ability highlights a key, distinguishing aspect of machine learning: they can learn to perform tasks even when no instructions exist—including tasks requiring tacit knowledge that could previously only be gained through experience"
• Customer Service Metrics: The study uses three standardized metrics
  • QUOTE: "Our firm measures productivity using three metrics that are standard in the customer service industry: 'average handle time,' 'resolution rate,' and 'net promoter score'"
• Natural Language Processing: The system processes conversation data to optimize both efficiency and empathy
  • QUOTE: "The AI firm further trains its model using a process to prioritize agent responses that express empathy, provide technical documentation, and limit unprofessional language"
• System Learning Effects: workers show sustained productivity improvement during AI outages
  • QUOTE: "Using data on software outages—periods in which the AI software fails to provide any suggestions—we show that workers see productivity gains relative to their baseline even when recommendations are unavailable"
• Employee Experience: AI assistance improves both worker retention and customer interactions
  • QUOTE: "These changes come alongside a substantial decrease in worker attrition, which is driven by the retention of newer workers"
• Machine Learning: The system learns from examples rather than explicit programming
  • QUOTE: "Machine learning algorithms work differently from traditional programs: instead of requiring explicit instructions to function, these systems infer instructions from examples"
• Productivity Dynamics: The study reveals heterogeneous effects across worker experience levels
  • QUOTE: "Treated agents with two months of tenure perform just as well as untreated agents with more than six months of tenure"
• Labor Market Implications: The findings suggest potential structural changes in employment patterns
  • QUOTE: "Firms may respond to increasing productivity among novice workers by hiring more of them, deskilling positions, or seeking to develop more powerful ai systems that can replace low-skill workers entirely"

Cited By

• http://scholar.google.com/scholar?q=%222023%22+Generative+AI+at+Work

Quotes

Abstract

New AI tools have the potential to change the way workers perform and learn, but little is known about their impacts on the job. In this paper, we study the staggered introduction of a generative AI-based conversational assistant using data from 5,179 customer support agents. Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers. We provide suggestive evidence that the AI model disseminates the best practices of more able workers and helps newer workers move down the experience curve. In addition, we find that AI assistance improves customer sentiment, increases employee retention, and may lead to worker learning. Our results suggest that access to generative AI can increase productivity, with large heterogeneity in effects across workers.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2023 GenerativeAIatWork	Erik Brynjolfsson Danielle Li Lindsey R Raymond			Generative AI at Work				10.3386/w31161		2023