Most social psychological research on anti-immigration prejudice has examined the point of view of the national ingroup—generally defined as the citizens of the country under consideration—toward immigrant outgroups. Threat perceptions related to immigration as well as national identification have been shown to underlie negative attitudes. Whether these two factors also explain attitudes among individuals sharing characteristics with the immigrants remains largely unstudied. To fill this gap, the present research examines perceived threat, national identification, and different facets of anti-immigration prejudice among Swiss residents with and without an immigrant background. Results of a path model conducted on data from the International Social Survey Programme 2013 (N = 1,198) showed that, as expected, natives reported higher levels of prejudice than both citizens of foreign descent and immigrants. While group differences in prejudice were in part explained by differences in threat perceptions, the role of national identification was more nuanced. Altogether, these findings suggest that social psychological research should go beyond simplistic “national ingroup vs. immigrant outgroup” conceptualisations when examining anti-immigration prejudice.