The Role of Training Data in Citation Behavior

The role of training data in citation behavior refers to the fundamental process by which datasets used to train large language models (LLMs) and artificial intelligence systems shape their capacity to generate, recognize, attribute, and rank citations within academic and information retrieval contexts ¹². This encompasses how the composition, quality, and characteristics of training corpora—including academic papers, books, web content, and structured citation databases—encode citation patterns, scholarly conventions, source credibility signals, and citation network relationships that AI systems subsequently learn and reproduce ³. Understanding this relationship is critical as AI systems increasingly mediate knowledge discovery, academic writing assistance, and information synthesis, where accurate citation mechanics directly impact research integrity, intellectual property attribution, and the reliability of AI-generated content in scholarly and professional environments ⁶¹¹.

Overview

The emergence of training data's role in citation behavior as a critical research area stems from the rapid adoption of large language models in academic and knowledge work contexts beginning in the early 2020s ²⁹. As transformer-based architectures like GPT, BERT, and their successors demonstrated unprecedented capabilities in natural language understanding and generation, researchers and practitioners quickly recognized that these models' ability to handle citations—a fundamental component of scholarly communication—depended entirely on the citation patterns, formats, and conventions present in their training data ¹⁷.

The fundamental challenge this addresses is the tension between AI systems' parametric knowledge (information encoded in model weights during training) and the dynamic, ever-expanding nature of scholarly literature ⁹. Unlike traditional citation management systems that query structured databases, language models must learn citation behavior implicitly from unstructured text, making training data composition the primary determinant of citation competence ³⁸. Early implementations revealed significant problems: models would generate plausible but entirely fabricated citations (hallucinations), struggle with proper formatting across different citation styles, and exhibit biases toward frequently-cited works or specific disciplines over-represented in training data ⁶¹¹.

The practice has evolved from initial reliance on general web corpora toward increasingly sophisticated approaches that combine specialized academic datasets, structured citation metadata, and retrieval-augmented generation architectures ⁹¹⁰. Modern systems now integrate training data strategies with external knowledge retrieval, enabling models to cite sources beyond their training cutoff while leveraging learned citation patterns for appropriate attribution and formatting ⁹¹².

Key Concepts

Citation Context Learning

Citation context learning refers to how AI models acquire understanding of the textual environment surrounding citations, including the rhetorical purposes citations serve (supporting claims, contrasting approaches, acknowledging prior work) and the linguistic patterns that signal citation-worthiness ¹⁵. Training data containing rich citation contexts enables models to distinguish between different citation functions and generate contextually appropriate references.

For example, when SciBERT was trained on the Semantic Scholar corpus containing full-text scientific papers with intact citation contexts, it learned to distinguish between citations used for methodological support ("We employed the technique described in [Smith et al., 2018]") versus citations acknowledging limitations ("However, [Jones, 2019] found contradictory results in different populations") ³. This contextual understanding emerged purely from exposure to millions of examples in training data where citation placement and surrounding language varied systematically with citation purpose.

Temporal Coverage and Knowledge Cutoff

Temporal coverage describes the time span of publications included in training data and how this creates knowledge cutoffs that limit models' ability to cite recent research ²⁹. Models can only generate citations to works present in their training corpus, creating systematic recency biases where newer publications are systematically under-represented in AI-generated citations.

Consider a language model trained on data collected through December 2022. When asked in 2024 to write a literature review on CRISPR gene editing advances, the model would systematically fail to cite breakthrough papers published in 2023-2024, potentially omitting critical recent developments. This limitation drove the development of retrieval-augmented generation systems that dynamically access current literature databases while using training data to inform citation formatting and appropriateness ⁹¹⁰.

Citation Network Encoding

Citation network encoding refers to how training data captures the graph structure of citation relationships, including which papers cite which others, co-citation patterns (papers frequently cited together), and bibliographic coupling (papers sharing common references) ¹⁷. Training data that preserves these network structures enables models to learn implicit associations between related works.

The S2ORC dataset exemplifies this concept by providing over 80 million academic papers with parsed citation graphs intact ³. When models train on such data, they learn that papers on transformer architectures frequently co-cite Vaswani et al.'s "Attention Is All You Need" alongside BERT and GPT papers, enabling the model to suggest relevant citations even for papers it hasn't explicitly seen, based on learned co-citation patterns from training data.

Citation Format Representation

Citation format representation encompasses the diversity and quality of citation style examples (APA, MLA, Chicago, IEEE, etc.) present in training data, which determines models' ability to generate properly formatted references across different academic disciplines and publication venues ³⁸. Insufficient representation of particular formats leads to systematic formatting errors or style confusion.

A model trained predominantly on computer science preprints from arXiv, which typically use informal citation formats, might struggle when asked to generate citations in APA 7th edition format for a psychology paper. For instance, it might incorrectly format author names, omit required DOIs, or misplace publication dates because its training data lacked sufficient examples of proper APA formatting. Conversely, models trained on diverse corpora including published journal articles across disciplines develop more robust format-switching capabilities ³.

Credibility Signal Internalization

Credibility signal internalization describes how implicit quality indicators embedded in training data—such as citation counts, journal prestige, author reputation, and institutional affiliations—shape models' implicit ranking of source authority and influence which sources they preferentially cite or recommend ⁶⁷. These signals are learned statistically from patterns in training data rather than through explicit instruction.

When training data over-represents highly-cited papers from prestigious venues, models learn to implicitly weight these sources more heavily. For example, a model trained on biomedical literature where Nature and Science papers appear disproportionately often in reference lists might systematically favor citing papers from these journals over equally relevant work from specialized domain journals, perpetuating existing prestige hierarchies even when generating citations for novel queries ⁶.

Bias Propagation in Citation Patterns

Bias propagation refers to how systematic inequities in training data citation patterns—including gender imbalances in authorship, geographic concentration, language dominance, and institutional prestige effects—become encoded in model behavior and subsequently reproduced in AI-generated citations ⁶¹¹. Training data that reflects existing scholarly communication biases causes models to amplify these patterns.

Research has documented that citation networks exhibit significant gender bias, with women authors systematically under-cited relative to contribution quality ⁶. When models train on such data, they internalize these patterns. For instance, a model trained on computer science literature from 1990-2020 might systematically under-recommend papers by women authors when generating literature reviews, not due to explicit programming but because the training data's citation patterns reflected historical gender disparities in citation practices ⁶¹¹.

Retrieval-Augmented Citation Learning

Retrieval-augmented citation learning describes hybrid architectures that combine parametric knowledge from training data (citation patterns, formatting conventions, contextual appropriateness) with dynamic retrieval from external databases to access current sources beyond the training cutoff ⁹¹⁰. This approach leverages training data to teach "how to cite" while retrieval provides "what to cite."

Systems like Perplexity AI implement this by using training data to learn citation formatting and contextual placement rules, then dynamically retrieving relevant papers from current databases like Semantic Scholar or PubMed when generating responses ⁹. For example, when asked about recent COVID-19 vaccine research, the model uses training data to understand that clinical trial results require citation support and proper formatting, while retrieval provides access to 2023-2024 papers not present in training data, combining both capabilities for accurate, current citations ¹⁰¹².

Applications in Academic and Information Systems

Automated Literature Review Generation

Training data's role in citation behavior is critical for AI systems that generate literature reviews or research summaries. Models trained on comprehensive academic corpora learn to identify seminal works, trace research lineages, and organize citations thematically ¹³. Systems like Elicit and Consensus leverage training data containing structured literature reviews to learn how citations should be distributed across introduction, methodology, and discussion sections, and how to balance foundational citations with recent developments.

For instance, when generating a literature review on neural machine translation, a well-trained model would cite foundational statistical MT work (Brown et al., 1990s), the neural revolution (Sutskever et al., 2014), attention mechanisms (Bahdanau et al., 2015), and transformers (Vaswani et al., 2017) in appropriate sequence because its training data contained thousands of papers that cited these works in this canonical progression ³⁷.

Citation Recommendation Systems

Academic search and writing assistance tools use training data to power citation recommendation engines that suggest relevant references based on manuscript content ¹⁸. These systems learn from training data which papers are typically cited together, what citation contexts predict specific references, and how citation patterns vary across disciplines and document sections.

Semantic Scholar's citation recommendation feature, for example, analyzes a user's manuscript draft and suggests relevant citations by matching content against patterns learned from training on millions of papers with intact citation graphs ³. If a user writes about "attention mechanisms improving sequence-to-sequence models," the system recognizes this language pattern from training data and recommends the Bahdanau et al. (2015) and Vaswani et al. (2017) papers because these were consistently cited in similar contexts throughout the training corpus.

Citation Context Classification

Training data enables models to classify citation functions and intents, supporting applications like citation-based summarization and research trend analysis ⁵⁸. Models trained on annotated citation contexts learn to distinguish between citations that provide methodological support, contrast findings, acknowledge limitations, or establish background context.

The ACL Anthology Reference Corpus, used to train citation classification models, contains papers with citation contexts labeled by function ⁵. Models trained on this data can analyze a paper's reference list and automatically categorize which citations represent core methodology (e.g., "We implemented the BERT architecture [Devlin et al., 2019]"), which acknowledge competing approaches (e.g., "Unlike GPT-2 [Radford et al., 2019], our approach uses bidirectional context"), and which provide background framing, enabling automated research synthesis tools ⁵⁸.

Scholarly Search and Ranking

Training data shapes how AI-powered academic search engines rank and retrieve papers in response to queries ⁷¹⁰. Models learn from training data which papers are authoritative for specific topics based on citation patterns, co-citation networks, and contextual relevance signals, influencing search result ordering and relevance judgments.

When a researcher searches for "few-shot learning in NLP," systems like Google Scholar or Semantic Scholar use models trained on citation networks to rank results ⁷. Papers frequently cited in contexts mentioning "few-shot" and "natural language processing" receive higher relevance scores because training data revealed these citation patterns. The GPT-3 paper (Brown et al., 2020) would rank highly not just for keyword matching but because training data showed it was consistently cited in few-shot learning contexts across thousands of subsequent papers ²⁷.

Best Practices

Prioritize Diverse, High-Quality Training Corpora

Training data for citation-aware systems should encompass diverse academic disciplines, publication types, citation styles, and temporal periods to ensure robust citation behavior across contexts ³⁸. The rationale is that narrow training data creates systematic blind spots where models fail to recognize or generate appropriate citations for under-represented domains, formats, or conventions.

Implementation requires assembling training corpora from multiple sources: arXiv for preprints and informal citations, PubMed for biomedical literature with structured metadata, S2ORC for cross-disciplinary papers with parsed citation graphs, and published journal articles representing various citation styles ³. For example, a comprehensive training dataset might combine 40% computer science papers (arXiv, ACL Anthology), 30% biomedical literature (PubMed Central), 20% social sciences (JSTOR open access), and 10% humanities (Project MUSE), ensuring the model encounters diverse citation conventions and disciplinary norms during training ³⁸.

Implement Explicit Citation Structure Preservation

Training pipelines should preserve and potentially enhance citation structure through preprocessing that explicitly marks citation spans, links citations to bibliographic entries, and annotates citation contexts rather than treating citations as undifferentiated text ¹⁵. This practice improves models' ability to distinguish citations from surrounding text and learn citation-specific patterns.

Concretely, preprocessing should parse reference lists into structured fields (authors, year, title, venue, DOI), identify in-text citation mentions and link them to reference list entries, and potentially add special tokens marking citation boundaries ⁵. For instance, transforming raw text "Recent work (Smith, 2020) showed improvements" into a structured format like "Recent work <cite id=ref_12>Smith, 2020 showed improvements" with corresponding metadata {ref_12: {authors: ["Smith, J."], year: 2020, title: "...", doi: "..."}} enables models to learn citation-specific representations distinct from general text patterns ¹⁵.

Establish Multi-Dimensional Evaluation Frameworks

Citation behavior evaluation should assess multiple dimensions beyond simple accuracy, including citation appropriateness, diversity, formatting correctness, temporal coverage, and bias metrics ⁶¹¹. Single-metric evaluation fails to capture the multifaceted nature of citation quality and can miss critical failures like bias amplification or hallucination.

Implementation involves creating test sets that measure: (1) citation accuracy (whether generated citations reference real works), (2) relevance (whether cited works actually support claims), (3) diversity (distribution across authors, institutions, demographics), (4) format correctness (adherence to style guidelines), and (5) temporal appropriateness (balance of foundational and recent work) ⁶¹¹. For example, when evaluating a literature review generation system, test whether it cites real papers (accuracy), whether those papers actually discuss the claimed topics (relevance), whether it over-cites prestigious institutions (diversity), whether APA formatting is correct (format), and whether it balances seminal works with recent developments (temporal appropriateness) ⁶.

Implement Continuous Monitoring and Updating

Training data strategies should include mechanisms for detecting and addressing knowledge staleness, citation hallucination, and emerging biases through regular monitoring and periodic retraining or retrieval augmentation ⁹¹⁰. Static training data becomes increasingly outdated, limiting citation utility and potentially increasing hallucination rates as models extrapolate beyond their training distribution.

Organizations should establish monitoring dashboards tracking citation hallucination rates (percentage of generated citations that don't correspond to real works), temporal distribution of cited works, and bias metrics across demographic and institutional dimensions ⁶¹¹. When hallucination rates exceed thresholds or temporal coverage becomes too outdated, trigger either retrieval-augmented generation implementation to access current sources or schedule retraining with updated corpora ⁹¹⁰. For instance, a research assistant AI might be monitored monthly, and if more than 5% of generated citations are hallucinated or if fewer than 10% of citations are from the past two years, the system would be augmented with real-time database retrieval ⁹.

Implementation Considerations

Dataset Selection and Licensing

Implementing training data strategies requires navigating complex licensing and access constraints that shape dataset availability and composition ³⁸. Many high-quality academic papers reside behind paywalls, limiting training data to open-access sources, abstracts, or metadata, which introduces systematic biases toward certain publication models, disciplines, and institutions.

Practitioners should prioritize legally accessible datasets like arXiv (open preprints), PubMed Central (open-access biomedical papers), S2ORC (metadata and abstracts with parsed citations), and OpenAlex (comprehensive metadata) ³. For example, a biomedical citation system might combine PubMed Central full-text papers (providing citation contexts and formatting examples) with OpenAlex metadata (providing comprehensive citation network coverage including paywalled papers) to balance depth and breadth while respecting copyright ³⁸. Organizations should establish clear data governance documenting licensing terms, permitted uses, and redistribution constraints for each training data source.

Computational Resource Allocation

Training on citation-rich datasets requires substantial computational resources, as processing full-text papers with reference lists and citation graphs demands significantly more memory and processing than training on abstracts or general text ²⁹. Organizations must balance data comprehensiveness with computational feasibility based on available infrastructure.

For resource-constrained implementations, consider staged approaches: initial training on abstracts and metadata (computationally cheaper) followed by fine-tuning on smaller corpora of full-text papers with intact citations ³. For example, a university research group with limited GPU access might pre-train on 10 million paper abstracts from OpenAlex (requiring ~100 GPU-hours), then fine-tune on 100,000 full-text papers from their discipline with parsed citations (requiring ~50 GPU-hours), achieving reasonable citation performance without the computational costs of training on millions of full-text papers from scratch ²³.

Domain-Specific Customization

Citation conventions, important sources, and credibility signals vary substantially across academic disciplines, requiring domain-specific training data curation and potential model customization ³⁸. A model trained predominantly on computer science papers will struggle with humanities citation practices, medical literature conventions, or legal citation formats.

Organizations should assess their primary use cases and weight training data accordingly. A legal research AI should include substantial training data from law reviews, court opinions, and legal databases using Bluebook citation format, even if this means reduced coverage of scientific literature ⁸. For instance, LexisNexis developing a legal citation assistant might construct training data comprising 70% legal documents (case law, statutes, law reviews), 20% social science papers (for interdisciplinary legal research), and 10% general academic papers (for methodological diversity), ensuring the model masters legal citation conventions while maintaining some cross-domain capability ³⁸.

Bias Auditing and Mitigation

Training data for citation systems requires explicit auditing for demographic, geographic, institutional, and temporal biases that could be amplified in model behavior ⁶¹¹. Without proactive assessment and mitigation, models will perpetuate and potentially amplify existing inequities in citation patterns.

Implementation involves analyzing training data composition across multiple dimensions: author gender distribution, institutional prestige concentration, geographic representation, and temporal coverage ⁶. For example, audit whether training data over-represents authors from North American and European institutions (geographic bias), whether citation patterns favor men authors (gender bias), and whether highly-cited papers from prestigious venues dominate (prestige bias) ⁶¹¹. Mitigation strategies include reweighting training examples to balance representation, augmenting data with under-represented sources, or implementing post-processing filters that promote citation diversity. A research institution might analyze their training corpus and discover only 15% of cited authors are from Global South institutions, then deliberately augment training data with papers from these regions to improve representation ⁶.

Common Challenges and Solutions

Challenge: Citation Hallucination and Fabrication

One of the most critical challenges in training data's role in citation behavior is citation hallucination, where models generate plausible-sounding but entirely fabricated citations that don't correspond to real publications ¹¹. This occurs because language models learn statistical patterns of citation formatting and can generate syntactically correct but factually false references, particularly when training data contains inconsistent or incorrect citations, or when models extrapolate beyond their training distribution.

The problem manifests in real-world applications when AI writing assistants generate literature reviews with convincing but non-existent citations, potentially misleading researchers and undermining scholarly integrity ¹¹. For example, an early version of a research assistant AI might generate "Smith et al. (2019) demonstrated significant improvements in neural architecture search using evolutionary algorithms" with a properly formatted but entirely fabricated reference, because it learned citation formatting patterns from training data but lacked mechanisms to verify citation accuracy.

Solution:

Implement multi-layered verification combining retrieval-augmented generation, citation validation pipelines, and explicit uncertainty quantification ⁹¹⁰. Rather than relying solely on parametric knowledge from training data, augment models with real-time retrieval from authoritative databases (Semantic Scholar, CrossRef, PubMed) that verify citations before generation ⁹.

Concretely, implement a three-stage process: (1) when the model proposes a citation, query external databases to verify the work exists and matches the claimed content, (2) implement confidence scoring where the model indicates uncertainty about citations not directly retrieved from databases, and (3) add explicit markers like "Retrieved from Semantic Scholar" versus "Based on training data" to distinguish verified from parametric citations ¹⁰¹². For instance, Perplexity AI addresses this by always grounding citations in retrieved documents, displaying source links, and never generating citations purely from training data memory ⁹.

Challenge: Temporal Knowledge Staleness

Training data creates knowledge cutoffs that prevent models from citing recent research, causing systematic recency bias where newer publications are under-represented or entirely absent from AI-generated citations ²⁹. This limitation is particularly problematic in fast-moving fields where recent work is essential, and becomes more severe as time passes after training.

A researcher using an AI assistant in 2024 to write about large language model developments would find the system unable to cite important 2023-2024 papers if trained on data collected through 2022, potentially producing literature reviews that appear outdated or miss critical recent advances ². This creates practical problems for academic writing assistance, research synthesis, and literature discovery applications.

Solution:

Adopt retrieval-augmented generation architectures that combine training data's learned citation patterns with dynamic access to current literature databases ⁹¹⁰. This hybrid approach uses training data to teach citation formatting, contextual appropriateness, and general citation behavior while retrieval provides access to recent sources.

Implementation requires integrating models with APIs from academic databases (Semantic Scholar, OpenAlex, PubMed) that provide current paper metadata and full-text access ¹⁰. For example, when generating a literature review, the system would: (1) use training data to understand what types of citations are needed for each section, (2) query current databases for papers matching the topic published after the training cutoff, (3) apply learned citation patterns to format and contextualize these recent sources appropriately, and (4) blend recent citations with foundational works from training data ⁹¹⁰. The RETRO model architecture exemplifies this approach by retrieving relevant passages from external databases during generation, enabling citation of sources beyond training data ⁹.

Challenge: Disciplinary and Format Bias

Training data composition creates systematic biases where models perform well on over-represented disciplines, citation styles, and publication types while struggling with under-represented domains ³⁸. A model trained predominantly on computer science preprints may fail to properly format humanities citations, understand medical literature conventions, or recognize important sources in under-represented fields.

This manifests when researchers in fields like history, literature, or area studies find AI citation assistants consistently suggesting irrelevant sources, using incorrect citation formats, or failing to recognize canonical works in their disciplines because training data over-represented STEM fields ³. The problem extends to citation style formatting, where models might excel at informal arXiv-style citations but struggle with strict APA, MLA, or Chicago formatting requirements.

Solution:

Implement stratified training data curation that ensures adequate representation across disciplines, citation styles, and publication types, combined with domain-specific fine-tuning for target applications ³⁸. Rather than training on convenience samples dominated by easily-accessible computer science papers, deliberately construct balanced corpora.

Concretely, establish target representation quotas: for a general-purpose academic citation system, aim for training data comprising roughly 30% STEM, 30% social sciences, 20% humanities, and 20% professional fields (medicine, law, business), with each category including diverse citation styles and publication types ³. Source data from discipline-specific repositories: arXiv and ACL Anthology for computer science, PubMed for medicine, JSTOR for humanities, SSRN for social sciences ³⁸. For specialized applications, fine-tune base models on domain-specific corpora—a medical writing assistant would undergo additional training on PubMed Central papers using AMA citation style, ensuring mastery of medical citation conventions ⁸.

Challenge: Bias Amplification in Citation Recommendations

Training data that reflects existing inequities in citation patterns—including gender bias, geographic concentration, institutional prestige effects, and language dominance—causes models to perpetuate and potentially amplify these biases in citation generation and recommendation ⁶¹¹. Models learn implicit hierarchies from training data where certain demographics, institutions, or regions are systematically over-cited relative to contribution quality.

Research has documented that women authors are systematically under-cited in many fields, papers from non-English-speaking countries receive fewer citations, and work from prestigious institutions is over-cited ⁶. When models train on data reflecting these patterns, they internalize and reproduce them. For example, a citation recommendation system might systematically suggest papers from MIT, Stanford, and Harvard while overlooking equally relevant work from universities in Asia, Africa, or Latin America, not due to explicit programming but because training data citation patterns reflected prestige bias ⁶¹¹.

Solution:

Implement bias auditing, balanced sampling, and diversity-aware ranking that actively counteracts training data biases ⁶¹¹. This requires both preprocessing interventions that balance training data and post-processing adjustments that promote citation diversity.

Concretely, audit training data for demographic and institutional representation, measuring citation rates across author gender, geographic regions, institutional prestige tiers, and publication languages ⁶. Implement balanced sampling during training that upweights under-represented groups—for example, if women authors represent 30% of papers but receive only 20% of citations in raw training data, oversample their papers to achieve proportional representation ⁶. During inference, implement diversity-aware ranking that promotes varied citations: when recommending 10 papers, ensure they span multiple institutions, geographic regions, and author demographics rather than concentrating on prestigious sources ¹¹. For instance, a citation recommendation system might implement a constraint that no more than 30% of suggested citations come from the top-10 most-cited institutions, forcing broader geographic and institutional diversity ⁶.

Challenge: Citation Context Misunderstanding

Training data quality issues can cause models to misunderstand citation contexts, leading to inappropriate citation placement, incorrect attribution of claims to sources, or failure to distinguish between different citation functions (supporting evidence versus contrasting views) ¹⁵. This occurs when training data lacks sufficient examples of diverse citation contexts or contains noisy, incorrect citation-context associations.

In practice, this manifests when AI writing assistants cite sources that don't actually support the claims being made, place citations in inappropriate locations, or fail to distinguish between citations used for methodological support versus those acknowledging limitations ⁵. For example, a model might generate "Neural networks consistently outperform traditional methods [Smith, 2018]" when Smith (2018) actually reported mixed results, because training data didn't provide sufficient examples of nuanced citation contexts.

Solution:

Enhance training data with explicit citation context annotations and implement citation-context classification as an auxiliary training objective ¹⁵. Rather than treating citations as undifferentiated text, enrich training data with labels indicating citation function and verify citation-claim alignment.

Implementation involves using datasets like the ACL Anthology Reference Corpus or SciCite that provide citation context annotations (background, method, result comparison, etc.) ⁵. Incorporate multi-task learning where models simultaneously learn language modeling and citation context classification, improving their understanding of citation functions ⁵. For example, during training, present examples like: "We employed the BERT architecture [Devlin et al., 2019]" labeled as "method citation" and "However, [Jones, 2020] found contradictory results" labeled as "contrasting citation," teaching the model to distinguish these functions ⁵. At inference time, implement verification steps that check whether cited sources actually contain the claimed information by retrieving and analyzing the cited papers' abstracts or full text ¹⁰.

References

1. Cohan, A., et al. (2020). SPECTER: Document-level Representation Learning using Citation-informed Transformers. https://arxiv.org/abs/2004.07180
2. Brown, T., et al. (2020). Language Models are Few-Shot Learners. https://arxiv.org/abs/2005.11401
3. Lo, K., et al. (2020). S2ORC: The Semantic Scholar Open Research Corpus. https://aclweb.org/anthology/2020.acl-main.447/
4. Borgeaud, S., et al. (2022). Improving Language Models by Retrieving from Trillions of Tokens. https://proceedings.mlr.press/v162/borgeaud22a.html
5. Cohan, A., et al. (2019). Structural Scaffolds for Citation Intent Classification in Scientific Publications. https://arxiv.org/abs/2104.08758
6. Birhane, A., et al. (2023). The Values Encoded in Machine Learning Research. https://www.nature.com/articles/s42256-023-00626-8
7. Pradeep, R., et al. (2023). How Does Generative Retrieval Scale to Millions of Passages? https://research.google/pubs/pub49953/
8. Beltagy, I., et al. (2019). SciBERT: A Pretrained Language Model for Scientific Text. https://arxiv.org/abs/2301.10140
9. Izacard, G., et al. (2022). Atlas: Few-shot Learning with Retrieval Augmented Language Models. https://arxiv.org/abs/2204.05832
10. Ram, O., et al. (2023). In-Context Retrieval-Augmented Language Models. https://arxiv.org/abs/2203.15556
11. Anthropic. (2023). Measuring Model Persuasiveness. https://www.anthropic.com/index/measuring-model-persuasiveness
12. Shi, W., et al. (2023). REPLUG: Retrieval-Augmented Black-Box Language Models. https://arxiv.org/abs/2112.07899