ClimateCheck - Scientific Fact-checking of Social Media Posts on Climate Change

NFDI4DS partners organize Shared Tasks at SDP2025 co-located with ACL2025.

Social media facilitates discussions on critical issues such as climate change, but it also contributes to the rapid dissemination of misinformation, which complicates efforts to maintain an informed public and create evidence-based policies. In this shared task, we emphasise the need to link public discourse to peer-reviewed scholarly articles by gathering English claims from social media about climate change as well as a corpus of about 400K abstracts of publications from the climate sciences domains. Participants will be asked to retrieve relevant abstracts for each claim (subtask I) and classify the relation between the claim and abstract as ‘supports’, ‘refutes’, or ‘not enough information’ (subtask II).

Subtask I: Abstracts Retrieval

• Task: given a claim from social media about climate change and a corpus of abstracts, retrieve the top K most relevant abstracts.
• Evaluation: MAP and B-Pref accounting for retrieving relevant abstracts and not penalising unjudged documents.

Subtask II: Claim Verification

• Task: given the claim-abstract pairs received from the previous subtask, classify their relation as ‘support’, ‘refutes’, or ‘not enough information’.
• Evaluation: F1 score based on judged documents from gold data; unjudged documents will not be included in computing the score.

Find detailed information on the workshop page: https://sdproc.org/2025/climatecheck.html