Given that Facebook is still the most widely used social networking site in the world, its influence on democratic processes is under constant scrutiny. Academics have put a special focus on Facebook’s role in inhibiting or enhancing citizens’ news exposure. Recent studies using digital behavioral data have analyzed the prevalence and effects of ‘Facebook news referrals.’ Using a web tracking tool that captures general browsing behavior as well as public posts seen on Facebook, this paper lays the groundwork for the field by assessing the validity of previously proposed operationalizations. We validate news referrals by investigating whether different measures actually reflect exposure to a news URL a user saw on Facebook. We furthermore assess the effects of news referrals on central outcomes in extant literature, contingent on different operationalizations. The results show that the most precise measure of news referrals are click identifiers attached to news URLs by Facebook. Different operationalizations of referrals have theoretically impactful consequences for the substantive understanding of Facebook’s role in high-choice online environments. The paper demonstrates the need for academics to constantly innovate in order to measure citizens’ online behavior in an ecologically valid manner.