Cracking the Code of Online Popularity: Lessons from Feature Selection and PCA


 

Predicting whether an article will go viral is a puzzle that blends data science with human behavior. In this project, we worked with a large dataset of online news articles, aiming to forecast popularity (measured as the number of shares) using dozens of explanatory variables. The assignment was straightforward in its goal but complex in its execution: reduce dimensionality, train models, and report performance. Along the way, we uncovered lessons about interpretability, complexity, and the limits of linear regression in messy, real-world data.

The Dimensionality Challenge

Our dataset contained nearly 40,000 articles with 60+ explanatory variables — ranging from keyword frequency to sentiment polarity. This posed the classic curse of dimensionality: too many features relative to the predictive signal often leads to overfitting, inefficiency, and inscrutable models.

To tackle this, we explored three modeling paths:

  1. Full features — a baseline model with all predictors.
  2. Feature Selection — pruning variables through variance filters, correlations, VIF checks, and stepwise selection.
  3. Principal Component Analysis (PCA) — compressing the feature space into orthogonal components that retain ~90% of variance.

Feature Selection vs PCA: Two Roads Diverged

Feature selection aims to keep the model interpretable by retaining only the most meaningful predictors. PCA, on the other hand, transforms the data into abstract components — linear combinations of original features — that may be compact and free from multicollinearity, but are far harder to explain.

A PCA scree plot revealed that about 30 components were enough to explain 90% of the variance. But what did these components represent?

Looking at the top PCA loadings gave us clues:

  • PC1 was driven by features like global subjectivity, polarity, and token length → a “content tone” axis.
  • PC2 emphasized negative polarity and topic indicators → a “lexical polarity/topic mix.”
  • PC3 leaned on keyword averages and topic channels → a “content richness” axis.


This made PCA insightful from a variance perspective, but not directly useful for explaining why one article is more shareable than another.

How the Models Performed

We evaluated three models head-to-head using cross-validation and hold-out test sets. Here’s what the results showed:

Model #Features CV R² Test R² Test adj R² Test RMSE OLS — Backward Elimination 45 -0.44 0.127 0.122 0.865 OLS — Stepwise (Bidirectional) 32 0.12 0.127 0.123 0.865 OLS — PCA (~90% var) 30 -0.03 0.098 0.095 0.879

Key insights:

  • Stepwise selection (32 features) edged out other models: nearly the same accuracy as the full set, but far fewer predictors.
  • Backward elimination performed similarly but kept more variables (45 vs. 32).
  • PCA lagged slightly in accuracy, showing that interpretability-preserving methods can be just as good (or better) than dimensionality-reduction approaches in practice.

Across the board, R² values hovered around 0.12, meaning our models explained only about 12% of the variance in article shares. That’s low, but not unexpected: human behavior (what people choose to share) is notoriously difficult to predict with linear features alone.

Residual Checks: What the Errors Say

Residual analysis provided another layer of understanding.

  • The Residuals vs Fitted plot showed a reasonably centered cloud around zero, but with some spread patterns, hinting at heteroscedasticity (uneven variance).
  • The Residuals Distribution plot [Insert Residuals Histogram visual] was roughly bell-shaped but with heavier tails, suggesting that extreme values (outliers) exist and linear models may be struggling with them.

These checks confirmed what the performance metrics hinted at: linear regression can only capture so much of the story in this dataset.

Lessons Learned

This project underscored some valuable lessons for real-world modeling:

  • More features ≠ better models. Stepwise selection achieved nearly identical accuracy as the full model with almost half the variables.
  • PCA is powerful but abstract. It compresses variance efficiently, but the tradeoff in interpretability may not always be worth it.
  • Linear regression has limits. Even with careful feature engineering and dimensionality reduction, linear models struggled to capture complex human behavior behind online sharing.
  • Residuals are storytellers. Diagnostic plots revealed heteroscedasticity and heavy tails, suggesting that more flexible models (e.g., random forests, gradient boosting) could perform better in future iterations.

Final Takeaway

Predicting article popularity isn’t just about crunching numbers — it’s about understanding the tradeoffs between accuracy, complexity, and interpretability. While our models didn’t achieve high predictive power, they illustrated the strengths and weaknesses of feature selection versus PCA in a clear, practical way.

For practitioners, the message is simple: don’t blindly throw all features into your model. Start small, prune carefully, and always check residuals. Sometimes the best model isn’t the most complex one — it’s the one that balances performance with insight.


Comments

Post a Comment

Popular posts from this blog

Google Search can't be trusted anymore

I Can No Longer Trust Google's Search Results, and It's Honestly Scaring Me Because of Google's advertising-driven model, I no longer trust its search results. We all know the first page of results isn't necessarily what’s most relevant—it’s often what someone paid Google to show us. I find myself increasingly wary of using Google, preferring Bing—not because it's the best, but because it gives me more straightforward, relevant results. I remember searching for the Astoria Hotel in Dubai a while back. I typed the hotel's name and address into Google, expecting to find its official website. But the entire first and second pages were cluttered with third-party booking sites like Booking.com, Hotels.com, and similar services. It took persistent digging to find the actual hotel website, even though I had been specific with my search query. Out of frustration, I tried Bing with the exact same search. The hotel's official website appeared as the second result—simp...
Read more

TD Bank’s Data Awakening: What Every Business Can Learn About Enterprise Transformation

Image
By Emmanuel Olimi Kasigazi   April 25th, 2025 Transforming chaos into clarity: TD Bank’s journey from scattered data to structured strategy offers lessons for every modern organization Every business today is a data business — but not every business knows how to use its data wisely.” In a world where data is the new oil, what happens when a major institution discovers it’s swimming in it — but can’t refine it? That was the challenge TD Bank Group faced in 2015 when it launched one of the most ambitious enterprise data transformations in Canadian banking history. The story of how TD approached the chaos — with executive vision, regulatory urgency, and even design thinking — isn’t just a case study. It’s a playbook for modern organizations navigating the stormy seas of data governance, compliance, and innovation. This article breaks down what TD did right, where they stumbled, and what every organization (yes, even yours) can learn from their experience. The Backdrop: A Data Giant Wi...
Read more

In The Shadow of a Giant: How GOP Candidates Strategically Positioned Themselves Around Trump in the 2024 Primary.

Image
When it comes to presidential primaries, much of the focus tends to be on what candidates say — but what about where they go, and when? Political campaigns have always been a choreographed dance, with candidates carefully plotting where and when to make appearances. But what happens when one contestant dominates the dance floor? The 2024 Republican primary featured former President Donald Trump as the overwhelming frontrunner, creating a unique dynamic where other candidates had to make strategic decisions: should they follow in Trump’s footsteps, or deliberately chart their own path? This article presents a data-driven analysis of how Republican presidential hopefuls positioned themselves relative to Trump throughout the primary campaign. Using a novel “shadow zone” approach, we examined 1,741 campaign events to reveal previously unseen patterns in campaign strategy and t answer a subtle but important question: Did GOP candidates follow Trump’s campaign trail to tap into his media buz...
Read more