Updates on the Short-Term Stock Picking Model

I have made MANY MANY changes to the model, iteratively.

Model Strategy:

We started by picking stocks based on return predictions, using news sentiment scores. We quickly ditched vader, because it simply wasn't working well for the context of finance news. This left us with some transformer models (2) that seem to work better. 

But we decided to add some new features into the model: 

• Macro Indicators (i.e. Inflation Expectations, Ten Year Yield, et al). 
• Momentum Indicators (and also trend consistency)

To add the momentum, we had to collect more data, which was a big change. We had to get lag data (5 days and 20 days).  Because of this, I had to re-think the feature engineering logic, and compartmentalize it as much as possible so that we can add or remove features without "turning everything upside down and inside out".

After this, we learned that Momentum was - by far - the dominant predictive influencer. Indeed, some AIs I consulted told me that the news sentiment was just noise, and that I should ditch it and go just with Momentum alone.

The R-squared on these models is terrible - and is negative. But - you don't want to just invert that necessarily. This could have been because of the fact that we simply didn't have enough training data - and training data that crossed regimes (up market down market).

But guess what? When I put some back-testing modules together and looked at actual returns,  the model did better - much better - with news sentiment used in conjunction with momentum than it would have with just momentum used alone. But - the suggestion came, to use a rank approach as opposed to just using the returns themselves.

I consulted with some Quant Algo Traders on this, and they (I presume smarter than I), agreed.

So now, the model is using the rank approach.