Bibliography

The list below includes every publication cited on the site, as well as others on the same topics. Each entry links to a free online copy of the work.

Most of these are research papers written for specialists, and you don’t need to read them to follow the materials. If you do open one, the abstract and introduction usually make the main point without the technical detail. For readings chosen for non-specialists, see the Going Further boxes at the end of each page.

Ahia, Orevaoghene, Sachin Kumar, Hila Gonen, et al. 2023. “Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models.” arXiv:2305.13707.

Artstein, Ron, and Massimo Poesio. 2008. “Survey Article: Inter-Coder Agreement for Computational Linguistics.” Computational Linguistics 34 (4): 555–96.

Askell, Amanda, Yuntao Bai, Anna Chen, et al. 2021. “A General Language Assistant as a Laboratory for Alignment.” arXiv:2112.00861.

Atari, Mohammad, Mona J Xue, Peter S Park, Damián E Blasi, and Joseph Henrich. 2023. “Which Humans?” PsyArXiv.

Atıl, Berk, Sarp Aykent, Alexa Chittams, et al. 2025. “Non-Determinism of ‘Deterministic’ LLM System Settings in Hosted Environments.” In Proceedings of the 5th Workshop on Evaluation and Comparison of NLP Systems, edited by Mousumi Akter, Tahiya Chowdhury, Steffen Eger, Christoph Leiter, Juri Opitz, and Erion Çano. Association for Computational Linguistics.

Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. “Neural Machine Translation by Jointly Learning to Align and Translate.” Paper presented at 3rd International Conference on Learning Representations (ICLR 2015).

Bai, Yuntao, Andy Jones, Kamal Ndousse, et al. 2022. “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.” arXiv:2204.05862.

Barocas, Solon, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and Machine Learning: Limitations and Opportunities. MIT Press.

Belkin, Mikhail, Daniel Hsu, Siyuan Ma, and Soumik Mandal. 2019. “Reconciling Modern Machine Learning Practice and the Bias-Variance Trade-Off.” Proceedings of the National Academy of Sciences 116 (32): 15849–54.

Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM.

Bender, Emily M., and Alexander Koller. 2020. “Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Association for Computational Linguistics.

Binette, Olivier, and Rebecca C. Steorts. 2022. “(Almost) All of Entity Resolution.” Science Advances 8 (12): eabi8021.

Bolukbasi, Tolga, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. “Man Is to Computer Programmer as Woman Is to Homemaker? Debiasing Word Embeddings.” arXiv:1607.06520.

Bommasani, Rishi, Kevin Klyman, Shayne Longpre, et al. 2023. “The Foundation Model Transparency Index.” arXiv:2310.12941.

Brandsen, Alex, Suzan Verberne, Karsten Lambers, and Milco Wansleeben. 2021. “Can BERT Dig It? -- Named Entity Recognition for Information Retrieval in the Archaeology Domain.” arXiv:2106.07742.

Brandsen, Alex, Suzan Verberne, Milco Wansleeben, and Karsten Lambers. 2020. “Creating a Dataset for Named Entity Recognition in the Archaeology Domain.” In Proceedings of the Twelfth Language Resources and Evaluation Conference, edited by Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, et al. European Language Resources Association.

Brown, Tom B., Benjamin Mann, Nick Ryder, et al. 2020. “Language Models Are Few-Shot Learners.” arXiv:2005.14165.

Burrell, Jenna. 2016. “How the Machine ‘Thinks’: Understanding Opacity in Machine Learning Algorithms.” Big Data & Society 3 (1): 2053951715622512.

Caliskan, Aylin, Joanna J. Bryson, and Arvind Narayanan. 2017. “Semantics Derived Automatically from Language Corpora Contain Human-Like Biases.” Science 356 (6334): 183–86.

Chen, Lingjiao, Matei Zaharia, and James Zou. 2024. “How Is ChatGPT’s Behavior Changing Over Time?” Harvard Data Science Review 6 (2).

Chen, Yanda, Joe Benton, Ansh Radhakrishnan, et al. 2025. “Reasoning Models Don’t Always Say What They Think.” arXiv:2505.05410.

Cheng, Jeffrey, Marc Marone, Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme. 2024. “Dated Data: Tracing Knowledge Cutoffs in Large Language Models.” arXiv:2403.12958.

Christiano, Paul, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. “Deep Reinforcement Learning from Human Preferences.” arXiv:1706.03741.

Clark, Elizabeth, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. “All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text.” arXiv:2107.00061.

Dathathri, Sumanth, Abigail See, Sumedh Ghaisas, et al. 2024. “Scalable Watermarking for Identifying Large Language Model Outputs.” Nature 634 (8035): 818–23.

Desai, Shrey, and Greg Durrett. 2020. “Calibration of Pre-Trained Transformers.” In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu. Association for Computational Linguistics.

Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), edited by Jill Burstein, Christy Doran, and Thamar Solorio. Association for Computational Linguistics.

Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, et al. 2021. “An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv:2010.11929.

Durmus, Esin, Karina Nguyen, Thomas I. Liao, et al. 2024. “Towards Measuring the Representation of Subjective Global Opinions in Language Models.” arXiv:2306.16388.

Dziri, Nouha, Ximing Lu, Melanie Sclar, et al. 2023. “Faith and Fate: Limits of Transformers on Compositionality.” arXiv:2305.18654.

Edge, Darren, Ha Trinh, Newman Cheng, et al. 2024. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization.” arXiv:2404.16130.

Ehrmann, Maud, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, and Antoine Doucet. 2023. “Named Entity Recognition and Classification in Historical Documents: A Survey.” ACM Computing Surveys 56 (2): 27:1–47.

Ehrmann, Maud, Matteo Romanello, Alex Flückiger, and Simon Clematide. 2020. “Extended Overview of CLEF HIPE 2020: Named Entity Processing on Historical Newspapers.” CLEF 2020 Working Notes. Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings; 2696, vol. 2696 (October).

Farquhar, Sebastian, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. “Detecting Hallucinations in Large Language Models Using Semantic Entropy.” Nature 630 (8017): 625–30.

Finkel, Jenny Rose, Christopher D. Manning, and Andrew Y. Ng. 2006. “Solving the Problem of Cascading Errors: Approximate Bayesian Inference for Linguistic Annotation Pipelines.” In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, edited by Dan Jurafsky and Eric Gaussier. Association for Computational Linguistics.

Gallegos, Isabel O., Ryan A. Rossi, Joe Barrow, et al. 2024. “Bias and Fairness in Large Language Models: A Survey.” Computational Linguistics (Cambridge, MA) 50 (3): 1097–179.

Gao, Luyu, Aman Madaan, Shuyan Zhou, et al. 2023. “PAL: Program-Aided Language Models.” arXiv:2211.10435.

Gebru, Timnit, Jamie Morgenstern, Briana Vecchione, et al. 2021. “Datasheets for Datasets.” arXiv:1803.09010.

Geirhos, Robert, Jörn-Henrik Jacobsen, Claudio Michaelis, et al. 2020. “Shortcut Learning in Deep Neural Networks.” Nature Machine Intelligence 2 (11): 665–73.

Geirhos, Robert, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. 2018. “ImageNet-Trained CNNs Are Biased Towards Texture; Increasing Shape Bias Improves Accuracy and Robustness.” Paper presented at International Conference on Learning Representations. September 27.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press.

Grattafiori, Aaron, Abhimanyu Dubey, Abhinav Jauhri, et al. 2024. “The Llama 3 Herd of Models.” arXiv:2407.21783.

Greshake, Kai, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. “Not What You’ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” arXiv:2302.12173.

Grishman, Ralph, and Beth Sundheim. 1996. “Message Understanding Conference- 6: A Brief History.” COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics.

Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. “On Calibration of Modern Neural Networks.” arXiv:1706.04599.

Gurnee, Wes, and Max Tegmark. 2024. “Language Models Represent Space and Time.” arXiv:2310.02207.

Hamdi, Ahmed, Elvys Linhares Pontes, Nicolas Sidere, Mickaël Coustaty, and Antoine Doucet. 2023. “In-Depth Analysis of the Impact of OCR Errors on Named Entity Recognition and Linking.” Natural Language Engineering 29 (2): 425–48.

Hendrycks, Dan, and Kevin Gimpel. 2017. “A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks.” Paper presented at International Conference on Learning Representations. February 6.

Hicks, Michael Townsen, James Humphries, and Joe Slater. 2024. “ChatGPT Is Bullshit.” Ethics and Information Technology 26 (2): 38.

Hochreiter, Sepp, and Jürgen Schmidhuber. 1997. “Long Short-Term Memory.” Neural Computation 9 (8): 1735–80.

Holtzman, Ari, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. “The Curious Case of Neural Text Degeneration.” arXiv:1904.09751.

Hosseini, Mohammad, Bert Gordijn, Gregory E. Kaebnick, and Kristi Holmes. 2025. “Disclosing Generative AI Use for Writing Assistance Should Be Voluntary.” Research Ethics, June 21, 10.1177/17470161251345499.

Hosseini, Mohammad, David B Resnik, and Kristi Holmes. 2023. “The Ethics of Disclosing the Use of Artificial Intelligence Tools in Writing Scholarly Manuscripts.” Research Ethics 19 (4): 449–65.

Howard, Andrew G., Menglong Zhu, Bo Chen, et al. 2017. “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.” arXiv:1704.04861.

Howard, Jeremy, and Sebastian Ruder. 2018. “Universal Language Model Fine-Tuning for Text Classification.” In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Iryna Gurevych and Yusuke Miyao. Association for Computational Linguistics.

Hripcsak, George, and Adam S. Rothschild. 2005. “Agreement, the F-Measure, and Reliability in Information Retrieval.” Journal of the American Medical Informatics Association : JAMIA 12 (3): 296–98.

Hsieh, Cheng-Ping, Simeng Sun, Samuel Kriman, et al. 2024. “RULER: What’s the Real Context Size of Your Long-Context Language Models?” arXiv:2404.06654.

Huang, Lei, Weijiang Yu, Weitao Ma, et al. 2025. “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions.” ACM Transactions on Information Systems 43 (2): 1–55.

Inan, Hakan, Kartikeya Upasani, Jianfeng Chi, et al. 2023. “Llama Guard: LLM-Based Input-Output Safeguard for Human-AI Conversations.” arXiv:2312.06674.

James, Gareth, Daniela Witten, Trevor Hastie, Robert Tibshirani, and Jonathan Taylor. 2023. An Introduction to Statistical Learning: With Applications in Python. Springer Texts in Statistics. Springer Nature.

Ji, Ziwei, Nayeon Lee, Rita Frieske, et al. 2023. “Survey of Hallucination in Natural Language Generation.” ACM Computing Surveys 55 (12): 1–38.

Joshi, Pratik, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. “The State and Fate of Linguistic Diversity and Inclusion in the NLP World.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Association for Computational Linguistics.

Jurafsky, Daniel, and James H. Martin. 2026. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, with Language Models. 3rd ed.

Kabir, Samia, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2024. “Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions.” Proceedings of the CHI Conference on Human Factors in Computing Systems, May 11, 1–17.

Kadavath, Saurav, Tom Conerly, Amanda Askell, et al. 2022. “Language Models (Mostly) Know What They Know.” arXiv:2207.05221.

Kalai, Adam Tauman, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. 2025. “Why Language Models Hallucinate.” arXiv:2509.04664.

Kandpal, Nikhil, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. “Large Language Models Struggle to Learn Long-Tail Knowledge.” arXiv:2211.08411.

Kirchenbauer, John, Jonas Geiping, Yuxin Wen, et al. 2024. “On the Reliability of Watermarks for Large Language Models.” arXiv:2306.04634.

Kirchenbauer, John, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. “A Watermark for Large Language Models.” In Proceedings of the 40th International Conference on Machine Learning, edited by Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, vol. 202. Proceedings of Machine Learning Research. PMLR.

Kluyver, Thomas, Benjamin Ragan-Kelley, Fernando Pérez, et al. 2016. “Jupyter Notebooks—a Publishing Format for Reproducible Computational Workflows.” In Positioning and Power in Academic Publishing: Players, Agents and Agendas.

Kotek, Hadas, Rikker Dockum, and David Q. Sun. 2023. “Gender Bias and Stereotypes in Large Language Models.” Proceedings of The ACM Collective Intelligence Conference, November 6, 12–24.

Laban, Philippe, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. “LLMs Get Lost In Multi-Turn Conversation.” arXiv:2505.06120.

Laban, Philippe, Lidiya Murakhovs’ka, Caiming Xiong, and Chien-Sheng Wu. 2023. “Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment.” arXiv:2311.08596.

LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521 (7553): 436.

Leinonen, Juho, Arto Hellas, Sami Sarsa, et al. 2023. “Using Large Language Models to Enhance Programming Error Messages.” Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, March 2, 563–69.

Lewis, Patrick, Ethan Perez, Aleksandra Piktus, et al. 2020. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” In Proceedings of the 34th International Conference on Neural Information Processing Systems. Curran Associates Inc.

Li, Kenneth, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2022. “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task.” Paper presented at The Eleventh International Conference on Learning Representations. September 29.

Liang, Weixin, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. “GPT Detectors Are Biased Against Non-Native English Writers.” Patterns 4 (7): 100779.

Lin, Zhicheng. 2026. “Hidden Prompts in Manuscripts Exploit AI-Assisted Peer Review.” Communications of the ACM 69 (7): 53–56.

Liu, Alisa, Zhaofeng Wu, Julian Michael, et al. 2023. “We’re Afraid Language Models Aren’t Modeling Ambiguity.” In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, edited by Houda Bouamor, Juan Pino, and Kalika Bali. Association for Computational Linguistics.

Liu, Jiawei, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. “Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation.” arXiv:2305.01210.

Liu, Nelson F., Kevin Lin, John Hewitt, et al. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics (Cambridge, MA) 12: 157–73.

Mallen, Alex, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. “When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories.” In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki. Association for Computational Linguistics.

Manakul, Potsawee, Adian Liusie, and Mark Gales. 2023. “SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models.” In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, edited by Houda Bouamor, Juan Pino, and Kalika Bali. Association for Computational Linguistics.

Manning, Christopher D., Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press.

Mayhew, Stephen, Tatiana Tsygankova, and Dan Roth. 2019. “ner and pos when nothing is capitalized.” In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), edited by Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan. Association for Computational Linguistics.

Maynez, Joshua, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. “On Faithfulness and Factuality in Abstractive Summarization.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Association for Computational Linguistics.

McCarthy, John, Marvin L. Minsky, Nathaniel Rochester, and Claude E. Shannon. (1955) 2006. “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955.” AI Magazine 27 (4): 12–12.

McCoy, R. Thomas, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. 2023. “Embers of Autoregression: Understanding Large Language Models Through the Problem They Are Trained to Solve.” arXiv:2309.13638.

Meng, Kevin, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. “Locating and Editing Factual Associations in GPT.” In Advances in Neural Information Processing Systems, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, vol. 35. Curran Associates, Inc.

Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. “Distributed Representations of Words and Phrases and Their Compositionality.” arXiv:1310.4546.

Mikolov, Tomas, Scott Wen-tau Yih, and Geoffrey Zweig. 2013. “Linguistic Regularities in Continuous Space Word Representations.” Paper presented at Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013). May 27.

Mirzadeh, Iman, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. “GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models.” arXiv:2410.05229.

Mitchell, Margaret, Simone Wu, Andrew Zaldivar, et al. 2019. “Model Cards for Model Reporting.” Proceedings of the Conference on Fairness, Accountability, and Transparency, January 29, 220–29.

Mitchell, Melanie, and David C. Krakauer. 2023. “The Debate over Understanding in AI’s Large Language Models.” Proceedings of the National Academy of Sciences 120 (13): e2215907120.

Nadeau, David, and Satoshi Sekine. 2007. “A Survey of Named Entity Recognition and Classification.” Lingvisticæ Investigationes 30 (1): 3–26.

Nasr, Milad, Nicholas Carlini, Chawin Sitawarin, et al. 2025. “The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections.” arXiv:2510.09023.

Nature. 2023. “Tools Such as ChatGPT Threaten Transparent Science; Here Are Our Ground Rules for Their Use.” Nature 613 (7945): 612–12.

Nguyen, Anh, Jason Yosinski, and Jeff Clune. 2015. “Deep Neural Networks Are Easily Fooled: High Confidence Predictions for Unrecognizable Images.” arXiv:1412.1897.

Nissim, Malvina, Rik van Noord, and Rob van der Goot. 2020. “Fair Is Better Than Sensational: Man Is to Doctor as Woman Is to Doctor.” Computational Linguistics 46 (2): 487–97.

Novikova, Jekaterina, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017. “Why We Need New Evaluation Metrics for NLG.” Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2231–40.

OpenAI, Josh Achiam, Steven Adler, et al. 2023. “GPT-4 Technical Report.” arXiv:2303.08774.

Ouyang, Long, Jeff Wu, Xu Jiang, et al. 2022. “Training Language Models to Follow Instructions with Human Feedback.” arXiv:2203.02155.

Pearce, Hammond, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2021. “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions.” arXiv:2108.09293.

Peng, Siyao, Zihang Sun, Sebastian Loftus, and Barbara Plank. 2024. “Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations.” arXiv:2402.01423.

Perez, Ethan, Sam Ringer, Kamilė Lukošiūtė, et al. 2022. “Discovering Language Model Behaviors with Model-Written Evaluations.” arXiv:2212.09251.

Perez, Fábio, and Ian Ribeiro. 2022. “Ignore Previous Prompt: Attack Techniques For Language Models.” arXiv:2211.09527.

Perry, Neil, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2023. “Do Users Write More Insecure Code with AI Assistants?” Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, November 15, 2785–99.

Peters, Uwe, and Benjamin Chin-Yee. 2025. “Generalization Bias in Large Language Model Summarization of Scientific Research.” Royal Society Open Science 12 (4): 241776.

Petrov, Aleksandar, Emanuele La Malfa, Philip H. S. Torr, and Adel Bibi. 2023. “Language Model Tokenizers Introduce Unfairness Between Languages.” arXiv:2305.15425.

Plank, Barbara. 2022. “The ‘Problem’ of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation.” In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, edited by Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang. Association for Computational Linguistics.

Prather, James, Brent N. Reeves, Paul Denny, et al. 2024. “‘It’s Weird That It Knows What I Want’: Usability and Interactions with Copilot for Novice Programmers.” ACM Transactions on Computer-Human Interaction 31 (1): 1–31.

Qi, Xiangyu, Yi Zeng, Tinghao Xie, et al. 2023. “Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!” arXiv:2310.03693.

Radford, Alec, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. “Robust Speech Recognition via Large-Scale Weak Supervision.” arXiv:2212.04356.

Radford, Alec, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. “Improving Language Understanding by Generative Pre-Training.”

Raji, Inioluwa Deborah, Emily M. Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. “AI and the Everything in the Whole Wide World Benchmark.” arXiv:2111.15366.

Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier.” arXiv:1602.04938.

Rijsbergen, C. J. van. 1979. Information Retrieval. Butterworths.

Rombach, Robin, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. “High-Resolution Image Synthesis with Latent Diffusion Models.” arXiv:2112.10752.

Röttger, Paul, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. “XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models.” In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), edited by Kevin Duh, Helena Gomez, and Steven Bethard. Association for Computational Linguistics.

Rudin, Cynthia. 2019. “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead.” arXiv:1811.10154.

Russakovsky, Olga, Jia Deng, Hao Su, et al. 2015. “ImageNet Large Scale Visual Recognition Challenge.” arXiv:1409.0575.

Rust, Phillip, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. “How Good Is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.” arXiv:2012.15613.

Sainz, Oscar, Jon Ander Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. 2023. “NLP Evaluation in Trouble: On the Need to Measure LLM Data Contamination for Each Benchmark.” arXiv:2310.18018.

Saito, Takaya, and Marc Rehmsmeier. 2015. “The Precision-Recall Plot Is More Informative Than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets.” PLOS ONE 10 (3): e0118432.

Sandve, Geir Kjetil, Anton Nekrutenko, James Taylor, and Eivind Hovig. 2013. “Ten Simple Rules for Reproducible Computational Research.” PLOS Computational Biology 9 (10): e1003285.

Santurkar, Shibani, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. “Whose Opinions Do Language Models Reflect?” arXiv:2303.17548.

Schick, Timo, Jane Dwivedi-Yu, Roberto Dessì, et al. 2023. “Toolformer: Language Models Can Teach Themselves to Use Tools.” arXiv:2302.04761.

Schulhoff, Sander, Michael Ilie, Nishant Balepur, et al. 2024. “The Prompt Report: A Systematic Survey of Prompt Engineering Techniques.” arXiv:2406.06608.

Schweter, Stefan, Luisa März, Katharina Schmid, and Erion Çano. 2022. “hmBERT: Historical Multilingual Language Models for Named Entity Recognition.” arXiv:2205.15575.

Sclar, Melanie, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. “Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting.” arXiv:2310.11324.

Sennrich, Rico, Barry Haddow, and Alexandra Birch. 2016. “Neural Machine Translation of Rare Words with Subword Units.” arXiv:1508.07909.

Sevilla, Jaime, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, and Pablo Villalobos. 2022. “Compute Trends Across Three Eras of Machine Learning.” 2022 International Joint Conference on Neural Networks (IJCNN), July 18, 1–8.

Shanahan, Murray. 2024. “Talking About Large Language Models.” Communications of the ACM 67 (2).

Sharma, Mrinank, Meg Tong, Tomek Korbak, et al. 2024. “Towards Understanding Sycophancy in Language Models.” In International Conference on Learning Representations, edited by B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, vol. 2024.

Sharma, Mrinank, Meg Tong, Jesse Mu, et al. 2025. “Constitutional Classifiers: Defending Against Universal Jailbreaks Across Thousands of Hours of Red Teaming.” arXiv:2501.18837.

Shen, Xinyue, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. “‘Do Anything Now’: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models.” arXiv:2308.03825.

Shuster, Kurt, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. “Retrieval Augmentation Reduces Hallucination in Conversation.” In Findings of the Association for Computational Linguistics: EMNLP 2021, edited by Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih. Association for Computational Linguistics.

Singh, Shivalika, Yiyang Nan, Alex Wang, et al. 2025. “The Leaderboard Illusion.” arXiv:2504.20879.

Stevenson, Robert Louis. 1899. The Letters of Robert Louis Stevenson to His Family and Friends. Edited by Sidney Colvin. 2 vols. Methuen.

Suresh, Harini, and John V. Guttag. 2021. “A Framework for Understanding Sources of Harm Throughout the Machine Learning Life Cycle.” Equity and Access in Algorithms, Mechanisms, and Optimization, October 5, 1–9.

Sutskever, Ilya, Oriol Vinyals, and Quoc V. Le. 2014. “Sequence to Sequence Learning with Neural Networks.” arXiv:1409.3215.

Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. The MIT Press.

Tjong Kim Sang, Erik F., and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. arXiv:cs/0306050.

Toral, Antonio. 2019. “Post-Editese: An Exacerbated Translationese.” In Proceedings of Machine Translation Summit XVII: Research Track, edited by Mikel Forcada, Andy Way, Barry Haddow, and Rico Sennrich. European Association for Machine Translation.

Touvron, Hugo, Louis Martin, Kevin Stone, et al. 2023. “Llama 2: Open Foundation and Fine-Tuned Chat Models.” arXiv:2307.09288.

Trisovic, Ana, Matthew K. Lau, Thomas Pasquier, and Mercè Crosas. 2022. “A Large-Scale Study on Research Code Quality and Execution.” Scientific Data 9 (1): 60.

Turpin, Miles, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. “Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting.” arXiv:2305.04388.

Vaswani, Ashish, Noam Shazeer, Niki Parmar, et al. 2017. “Attention Is All You Need.” In Advances in Neural Information Processing Systems, edited by I. Guyon, U. Von Luxburg, S. Bengio, et al., vol. 30. Curran Associates, Inc.

Wallace, Eric, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. “The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions.” arXiv:2404.13208.

Walters, William H., and Esther Isabelle Wilder. 2023. “Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT.” Scientific Reports 13 (1): 14045.

Wan, Yixin, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, and Nanyun Peng. 2023. “‘Kelly Is a Warm Person, Joseph Is a Role Model’: Gender Biases in LLM-Generated Reference Letters.” In Findings of the Association for Computational Linguistics: EMNLP 2023, edited by Houda Bouamor, Juan Pino, and Kalika Bali. Association for Computational Linguistics.

Weber-Wulff, Debora, Alla Anohina-Naumeca, Sonja Bjelobaba, et al. 2023. “Testing of Detection Tools for AI-Generated Text.” International Journal for Educational Integrity 19 (1): 26.

Wei, Alexander, Nika Haghtalab, and Jacob Steinhardt. 2023. “Jailbroken: How Does LLM Safety Training Fail?” arXiv:2307.02483.

Wei, Jason, Xuezhi Wang, Dale Schuurmans, et al. 2022. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” In Proceedings of the 36th International Conference on Neural Information Processing Systems. Curran Associates Inc.

Williams, Sean, and James Huckle. 2024. “Easy Problems That LLMs Get Wrong.” arXiv:2405.19616.

Wolf, Thomas, Lysandre Debut, Victor Sanh, et al. 2020. “Transformers: State-of-the-Art Natural Language Processing.” In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, edited by Qun Liu and David Schlangen. Association for Computational Linguistics.

Wu, Yonghui, Mike Schuster, Zhifeng Chen, et al. 2016. “Google’s Neural Machine Translation System: Bridging the Gap Between Human and Machine Translation.” arXiv:1609.08144.

Xiong, Miao, Zhiyuan Hu, Xinyang Lu, et al. 2024. “Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs.” arXiv:2306.13063.

Yosinski, Jason, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. “How Transferable Are Features in Deep Neural Networks?” arXiv:1411.1792.

Zeiler, Matthew D., and Rob Fergus. 2013. “Visualizing and Understanding Convolutional Networks.” arXiv:1311.2901.

Zeng, Yi, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. “How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs.” arXiv:2401.06373.

Zhao, Jieyu, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. “Men Also Like Shopping: Reducing Gender Bias Amplification Using Corpus-Level Constraints.” In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, edited by Martha Palmer, Rebecca Hwa, and Sebastian Riedel. Association for Computational Linguistics.

Zhao, Tony Z., Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. “Calibrate Before Use: Improving Few-Shot Performance of Language Models.” arXiv:2102.09690.

Zheng, Lianmin, Wei-Lin Chiang, Ying Sheng, et al. 2023. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” In Proceedings of the 37th International Conference on Neural Information Processing Systems. Curran Associates Inc.

Zhou, Jeffrey, Tianjian Lu, Swaroop Mishra, et al. 2023. “Instruction-Following Evaluation for Large Language Models.” arXiv:2311.07911.