Joint Citation Ranking with LLMs Outperforms Isolated Impact Classification by ~9%
Crystal reframes citation impact assessment as a joint ranking problem across all references within a citing paper, rather than evaluating each citation in isolation. By leveraging LLMs to jointly rank cited works and mitigating positional bias via triple randomized ordering with majority voting, Crystal achieves +9.5% accuracy and +8.3% F1 over prior state-of-the-art classifiers. The approach is also more compute-efficient, requiring fewer LLM calls, and generalizes to open-weight models—making it viable for large-scale bibliometric analysis. Validation against ACL Test-of-Time award winners suggests the system's rankings correlate with long-term scientific recognition.