Removing Two Stale Macro Features

 

Removing Two Stale Macro Features

The model was trained on 11 features, two of which were macroeconomic sentiment indicators sourced from FRED. On inspection, both turned out to be monthly series — meaning they only update once a month and carry a publication lag on top of that. Despite this, the model had assigned them significant feature importance, essentially learning to lean on data that wasn't meaningfully changing day to day and wasn't even fully available in real time when historical training data was constructed.

Removing them dropped the feature set from 11 to 9. With those features gone, the model redistributed weight toward momentum and the remaining daily macro indicators in a more sensible way. Validation rank correlations improved on two of the three prediction horizons after the change. The two daily macro features that remained — VIX and treasury spread — are genuinely responsive to market conditions and carry the macro signal adequately on their own.

Both changes were low risk given that model predictions are used for monitoring purposes in this system rather than directly driving trading decisions.

The News Sentiment Model is Not About News Sentiment

Quick post to discuss some things...

First, what I am discovering after training this model daily, and running predictions, is that the news sentiment is NOT what is influencing this model. 

The news sentiment is based on Transformer models for financial news (Finbert). The news IS fresh, as we get these articles and and then predict over a 1-3 day period, before they're expired out to a training database where predicted returns are compared with actual returns.

News is NOT what this model is learning and training on. Instead, momentum and macro environment features are the aspects that is influencing the returns. Part of this could be the news being stale - the market already knows the news before the model is running. Also, we are only scoring headlines - not actual articles - due to processing constraints. So there's that.

From the training output, here's the feature importance ranked across all three horizons:

Macro features (dominating ~50% of importance):

• vix_0d — market fear index
• treasury_spread_0d — yield curve spread
• business_confidence_0d — FRED macro indicator
• consumer_sentiment_0d — FRED macro indicator

Momentum features (~40%):

• momentum_strength — short-term trend deceleration
• risk_adjusted_momentum — momentum relative to volatility
• trend_consistent — direction consistency

Sentiment features (~10%):

• tone_signed_resid — residualized tone score
• finbert_signed_resid — residualized FinBERT score
• tone_signed — raw tone
• finbert_signed — raw FinBERT

So the model is essentially saying: buy beaten-down stocks when the macro environment is calm. The news sentiment is contributing about 10% of the predictive signal, which is barely above noise given the R² of 0.002 on the residual models.

It's not really a news sentiment model at this point. It's a macro-regime mean-reversion model that happens to have sentiment features along for the ride.

Which raises the real question — is that actually a bad thing? Mean reversion in calm macro regimes is a legitimate strategy. The problem isn't the model's logic, it's that you only have 3 months of data so it hasn't seen enough regime diversity to be robust. And right now you're in a fear regime — elevated VIX, geopolitical uncertainty — which is exactly when this model historically underperforms.

Updates on the Short-Term Stock Picking Model

I have made MANY MANY changes to the model, iteratively.

Model Strategy:

We started by picking stocks based on return predictions, using news sentiment scores. We quickly ditched vader, because it simply wasn't working well for the context of finance news. This left us with some transformer models (2) that seem to work better. 

But we decided to add some new features into the model: 

• Macro Indicators (i.e. Inflation Expectations, Ten Year Yield, et al). 
• Momentum Indicators (and also trend consistency)

To add the momentum, we had to collect more data, which was a big change. We had to get lag data (5 days and 20 days).  Because of this, I had to re-think the feature engineering logic, and compartmentalize it as much as possible so that we can add or remove features without "turning everything upside down and inside out".

After this, we learned that Momentum was - by far - the dominant predictive influencer. Indeed, some AIs I consulted told me that the news sentiment was just noise, and that I should ditch it and go just with Momentum alone.

The R-squared on these models is terrible - and is negative. But - you don't want to just invert that necessarily. This could have been because of the fact that we simply didn't have enough training data - and training data that crossed regimes (up market down market).

But guess what? When I put some back-testing modules together and looked at actual returns,  the model did better - much better - with news sentiment used in conjunction with momentum than it would have with just momentum used alone. But - the suggestion came, to use a rank approach as opposed to just using the returns themselves.

I consulted with some Quant Algo Traders on this, and they (I presume smarter than I), agreed.

So now, the model is using the rank approach.

Latest Changes to XGBoost Quant Financial Model

 

Latest Changes:

Added new Macros to my model  - CPI, PPI, VIX (last change I made was to add a beats/meets/misses surprise score a few weeks ago)

I added some interactive features based on these (5 in total). I have learned that these interactives move the model predictability like nothing else - which is why I added more. 

    R-squared score came back up to approaching .3 now with these.

    My correlation is still inverted from 1 yr fwd return - even more so. So I flip the score.

One major change was that I added a "graph_score" which was a nightmare to produce. LLMs can NOT seem to handle this task AT ALL. So I finally had to do it mostly myself, and got something working fairly well - it recognizes good graphs vs bad graphs and downscores bad graph patterns.

I forked the stockdex github project, and am making some changes that I will re-submit back with a git pull. The macrotrends datasource could not handle quarterly data. Once I realized it could be done, I decided to enhance the code to do this so I could run the model and get more quarterly statements alongside the pack of annual statements.  

Once I get the model run with annual+quarterly, I will probably retire this project and move onto more recent and LLM-based stuff.