OOIR: Observatory of International Research

Papers

(The TQCC of Transactions of the Association for Computational Linguistics is 10. The table below lists those papers that are above that threshold based on CrossRef citation counts [max. 250 papers]. The publications cover those that have been published in the past four years, i.e., from 2020-07-01 to 2024-07-01.)

Article	Citations
SpanBERT: Improving Pre-training by Representing and Predicting Spans	595
A Primer in BERTology: What We Know About How BERT Works	394
Multilingual Denoising Pre-training for Neural Machine Translation	304
Topic Modeling in Embedding Spaces	283
How Can We Know What Language Models Know?	260
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation	188
Efficient Content-Based Sparse Attention with Routing Transformers	151
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models	123
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks	119
Sparse, Dense, and Attentional Representations for Text Retrieval	96
SummEval: Re-evaluating Summarization Evaluation	86
A Survey on Automated Fact-Checking	80
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation	78
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages	78
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT	52
Nested Named Entity Recognition via Second-best Sequence Learning and Decoding	52
ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models	51
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages	46
oLMpics-On What Language Model Pre-training Captures	44
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond	42
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations	41
BLiMP: The Benchmark of Linguistic Minimal Pairs for English	39
Machine Learning–Driven Language Assessment	37
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs	37
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers	37

Measuring and Improving Consistency in Pretrained Language Models	35
Gender Bias in Machine Translation	33
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation	33
Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension	31
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals	31
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering	31
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP	30
The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation	30
CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset	29
Theoretical Limitations of Self-Attention in Neural Sequence Models	28
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization	27
Break It Down: A Question Understanding Benchmark	27
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets	27
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering	27
Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?	27
Soloist: BuildingTask Bots at Scale with Transfer Learning and Machine Teaching	26
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them	26
Extractive Opinion Summarization in Quantized Transformer Spaces	25
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models	25
Syntax-Guided Controlled Generation of Paraphrases	24
Relevance-guided Supervision for OpenQA with ColBERT	24
AMR-To-Text Generation with Graph Transformer	23
Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs	22
Data-to-text Generation with Macro Planning	22
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP	22
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering	20
Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation	19
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics	19
Did Aristotle Use a Laptop?A Question Answering Benchmark with Implicit Reasoning Strategies	19
MasakhaNER: Named Entity Recognition for African Languages	19
Lost in the Middle: How Language Models Use Long Contexts	18
Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks	18
Time-Aware Language Models as Temporal Knowledge Bases	18
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension	18
Data Weighted Training Strategies for Grammatical Error Correction	17
Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?	17
Best-First Beam Search	17
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval	16
A Survey on Cross-Lingual Summarization	16
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains	16
TopiOCQA: Open-domain Conversational Question Answering with Topic Switching	16
Planning with Learned Entity Prompts for Abstractive Summarization	15
Acoustic-Prosodic and Lexical Cues to Deception and Trust: Deciphering How People Detect Lies	15
Explanation-Based Human Debugging of NLP Models: A Survey	15
Multilingual Autoregressive Entity Linking	15
Unsupervised Quality Estimation for Neural Machine Translation	15
Target-Guided Structured Attention Network for Target-Dependent Sentiment Analysis	14
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints	14
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition	14
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models	14

Efficient Methods for Natural Language Processing: A Survey	14
Benchmarking Large Language Models for News Summarization	14
Aligning Faithful Interpretations with their Social Attribution	14
In-Context Retrieval-Augmented Language Models	13
FeTaQA: Free-form Table Question Answering	13
Transformers for Tabular Data Representation: A Survey of Models and Applications	12
QED: A Framework and Dataset for Explanations in Question Answering	12
Sentence Similarity Based on Contexts	12
Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings	12
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups	12
ABNIRML: Analyzing the Behavior of Neural IR Models	11
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing	11
Adaptive Semiparametric Language Models	10
Differentiable Subset Pruning of Transformer Heads	10
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary	10
Infusing Finetuning with Semantic Dependencies	10
Sketch-Driven Regular Expression Generation from Natural Language and Examples	10
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?	10
Task-Oriented Dialogue as Dataflow Synthesis	10
Better Document-Level Machine Translation with Bayes’ Rule	10