Clickbait Detection in News Headlines

Abstract

Benchmarked Multinomial Naive Bayes and Logistic Regression on 32k balanced news headlines to detect clickbait. Two deployment targets were explored: maximum accuracy (97.12% test accuracy with stopwords and 8k vocabulary) and zero false positives (0% FPR, 84% accuracy, TPR 68%). Detailed error analysis highlights impactful tokens and the trade-offs introduced by bias calibration.

About

Headlines are lowercased, stripped of punctuation (numbers retained) and vectorised via Bag-of-Words over vocabularies up to 12k tokens with/without stopwords. Both Multinomial Naive Bayes and Logistic Regression are trained under two regimes: accuracy-oriented (cross-validated, full feature space) and FPR-oriented (bias sweeping from −8 to 8 and custom selection of the optimal prior). Keeping stopwords proves critical—removing them drops accuracy by >2%. Error inspection surfaces ambiguous cases (e.g., concise factual headlines mislabelled as bait) and reveals the most discriminative tokens. The final deliverable offers a toggle: 97% accuracy for general moderation, or 0% FPR (84% accuracy, 68% TPR) when false positives must be eliminated.

Key Findings

• •A controlled 32k-headline dataset was preprocessed into Bag-of-Words vocabularies (2k–12k tokens) to compare Multinomial Naive Bayes and Logistic Regression under identical splits.
• •Stopword retention consistently improved validation accuracy, motivating the 8k-token model that hit 97.12% test accuracy without favouring logistic regression.
• •Bias sweeping between −8 and 8 enabled a 0% FPR operating point (84% accuracy, 68% TPR), giving stakeholders a tunable moderation lever.
• •Inspecting token log deltas surfaced classic bait patterns (“2015”, “things”, “guess”) versus neutral news markers (“kills”, “iraq”), guiding editorial audits.

Methodology