Trying to Right the Ship on my News Sentiment Based Stock Model

Jan 12th, through Feb 3rd. A downward trend that saw my balance drop from 101K to 89K.

The reasons were very complex. I will discuss them

Bugs in the trading module

Bugs in the code that put stop limits in, which caused repetitive falling knife scenario        buying.

OTC Stock Volatility

You can make a lot of money with OTC stocks, but you can also lose a lot. I removed the OTC exchanges and left just AMEX, NASDAQ and NYSE as the exchanges.

AI making changes to the algorithms I was not reviewing properly

AI had made several mistakes in moving the model from return prediction to a rank-based approach. 

In general, the consensus on discussion was that return prediction didn't make a ton of sense, and that ranking stocks based on their cumulative scores (news sentiment, momentum and other derived features) made the most sense. The problem though, was that the filtering was being done before the predictions, not AFTER. This meant that the universe of data was restricted to the model.

Article Mapping 

Phrase mapping had several bugs in it that caused completely legit articles to be unmapped. Recall that earlier, certain mapping bugs caused certain symbols to be attributed to casual words (i.e. key). 

I was mapping articles to symbols, and this was causing article-symbol-price tuples such that symbols with more news (articles) created a lot more rows of data than those that had less articles. And this was imbalancing the model.   The fix for this, was to aggregate the symbol_day prediction, which collapses multiple articles per symbol per day into one row by averaging sentiment features.

There were other changes I made as well, such as using XGBoost instead of Random Forest. The balance has bounced back up to $97,760 at the time of this writing, so a decent recovery. Of course the market has stabilized a bit - favoring Value right now. There indeed was a pullback market regime that did occur at the beginning of my slide, so that is also a contributing factor as well.

So - will these changes work? 

We shall see.
 

 

Jan 12th - The Death Spiral - Trading Lessons Learned

I learned quickly all of the issues with Algo Trading.

Lesson 1 - 4-to-1 Margins Were Enabled

I didn't take the time to look at the Configuration. So I was trading on Margin, and had to Liquidate a bunch of positions, some of which lost money.

Lesson 2 - Stop Limits Can Lead to a HOST of Problems

I had stop limit orders being set up to protect my downside. But, this led to THIS problem:

Buy HMY @ $25 → gap to $23 → stop → rebuy $23.50 
→ gap to $21 → stop → rebuy $21.80 → PDT flag → can't exit → $18

Basically, the stop limits were causing repetitive buying in falling-knife scenarios, because the model would pick the same symbol the next trade.

Lesson 3 - Some Symbols Are Halted or Delisted

I had one position, CLCO, which was halted. I have tried to force liquidate this position, but it is absolutely stuck. I will probably have to contact support. But this has thrown a major wrench into my Portfolio Manager, which needs to know that this money is "locked up" and account for that.

I also had issues with the model itself - in that the way the algorithm was working, it would pick the best ranked stocks, even if the signal strength or momentum was negative. So this had to be fixed as well, with some enhanced signal strength and momentum logic.

So - very expensive lessons. The back-testing of the model is turning out fine. But the Porfolio Manager and its transaction management was killing us. We burned about $10K of our $100K on this. Hopefully it starts to work better now with fixes to these issues.

The Algo Trading Model is Live!

I am now using a Paper Trading account using Alpaca!

The paper trading account seeds you with $100K of money. This is a bit similar to the Investopedia Simulator, except that you can work with this account programmatically through APIs. It looks like a lot of day traders (options, derivatives, et al) use this. But I am using it just to buy and sell stocks.

Their user interface is a bit confusing, but with the help of some Python, I am able to get the status information I need. For example, the Positions tab doesn't carry much information. You have to actually consult the Orders tab, and marry things together to get a fuller view.

I needed a Portfolio Manager to make decisions on trading. I was going to use an Agentic AI approach using Langchain, or maybe Crew or MCP. Something along that vein.  But the AIs themselves, pointed out that these are too quirky and "black box" to be able to rely on them in a real trading situation. They suggested I go with a Deterministic approach.

So - with the help of an LLM that is trained on more technical topics, I had it generate me a few drafts until I saw something that looked simple enough to get started with.  It simply takes the predictions csv, and uses that as input for its trading. It will buy positions, hold them for 5 days, and sell them. Simple enough.

 

Updates on the Short-Term Stock Picking Model

I have made MANY MANY changes to the model, iteratively.

Model Strategy:

We started by picking stocks based on return predictions, using news sentiment scores. We quickly ditched vader, because it simply wasn't working well for the context of finance news. This left us with some transformer models (2) that seem to work better. 

But we decided to add some new features into the model: 

• Macro Indicators (i.e. Inflation Expectations, Ten Year Yield, et al). 
• Momentum Indicators (and also trend consistency)

To add the momentum, we had to collect more data, which was a big change. We had to get lag data (5 days and 20 days).  Because of this, I had to re-think the feature engineering logic, and compartmentalize it as much as possible so that we can add or remove features without "turning everything upside down and inside out".

After this, we learned that Momentum was - by far - the dominant predictive influencer. Indeed, some AIs I consulted told me that the news sentiment was just noise, and that I should ditch it and go just with Momentum alone.

The R-squared on these models is terrible - and is negative. But - you don't want to just invert that necessarily. This could have been because of the fact that we simply didn't have enough training data - and training data that crossed regimes (up market down market).

But guess what? When I put some back-testing modules together and looked at actual returns,  the model did better - much better - with news sentiment used in conjunction with momentum than it would have with just momentum used alone. But - the suggestion came, to use a rank approach as opposed to just using the returns themselves.

I consulted with some Quant Algo Traders on this, and they (I presume smarter than I), agreed.

So now, the model is using the rank approach.

New AI / ML Stock Picking Model

Okay. The new stock picking model is VASTLY different than the previous one.

We download news, then we feed the news into transformer models (Finbert) to gauge sentiment and calculate sentiment scores. Originally, I was using vader sentiment also, but I removed that because vader just doesn't seem to work well with financial news.

Downloading the news was a sizable effort. I decided to use a multi-threaded approach, where each news source had its own thread(s).

After all of the news has been downloaded - and scored - I send the data into module that attempts to map the news to stock symbols. Doing this well, required numerous iterative enhancements to the code. Then, the news is filtered according to various rule sets.

Originally, I saved the news in a csv file. But later, I had to convert this to a database approach. There is also some caching, to make sure we are not fetching the same article repeatedly. The code also does some heavy work to ascertain the proper date of the article. 

Once this is finished, another module sets about finding - or trying to find - prices. It will price the 0d for the symbol for the article. A cache is used so that we don't try to look up the same price for the same day for the same symbol more than once (reduced API calls). The code may pick up prices for the "zero day plus X days" depending on how long the article has been sitting. 

Once the article has been fully "aged out", it is purged and migrated into a training database. This allows us to train the model on "actual returns".

After the pricing, there are some analyzers that will examine the integrity of data. And if all looks well, the model is (re-trained) with the newly migrated data, and new predictions are made. This allows for a continual improvement.

Once predictions are made, a portfolio managed makes trades using a paper account, and some back test programs are used to compare performance against the S&P, and if desired, prior model versions. 

I am sure I have skipped over a lot of the complexity on this, but in a nutshell this is what we are doing. I will avoid discussing the "secret sauce" which is the feature engineering.

My New Stock Prediction Model - Short Term Stock Prediction

Someone I work with has been working extensively on a Swing Trading model.

He has great financial experience from what I understand, but as he is in more of a management role and not in a day-to-day technical role, his programming skills might be just a tad or a step behind mine.

I have been watching him publish his short-term predictions, and his model is based on all kinds of things. I won't publish his secret sauce here, but he is using things that day traders tend to use, like RSI and Stochastics and such.

But, like my Financial Statement model - which tried to predict longer term buy-hold stocks,  his wasn't holding water either through back testing and results. One problem is that everything looks great in a bull market (rising tides lift all boats). And, to quote Buffet, "only when the tide goes out can you see who is swimming naked". So these models need to hold up in BOTH upturns and downturns.

I have decided to work on a new stock picking model. I am not sure yet how much I will blog about the specifics of it. But it is also a short-term model.

Stay Tuned 

The Financial Statement Model - Retired for Now

Once I got my Stock Prediction based on Annual (10-K) and Quarterly (10-Q) statement model working, I just wasn't happy with the R-squared on it. And I didn't feel comfortable investing in the picks it made (based on predicted returns). 

The R-squared on quarterly was so low, that trying to consider stocks it predicted for a quarter-long buy hold was just not feasible.

The R-squared on annual was considerably higher. But even then, it was not high enough to justify a stock purchase for a year-long tie-up of investment money.

Frankly, the stocks it was picking looked horrendous in many respects. Falling Knives, despite efforts to contain those, dominated the list. Others had low liquidity (read my earlier post on the Liquidity Effect) - and Solvency was an issue on them. Buying stocks with low or no liquidity and practically insolvent, and trying to hold them even a quarter, no less a year, is absolutely stupid.

I did Ensemble these models. But it didn't change the picture for me. And remember, I have Macro data and Macro Interactives in this model!

The conclusion: 
Statements (fundamentals) are important - but not for picking stocks based on them necessarily. You would have to combine the fundamentals with other things. 

I kind of knew this already, based on things I had read. I guess I needed to use the effort as a proving ground to myself.

So - in the end - I shelved these models. I learned a TON and it was great doing them. It built me into an AI Powerhouse with solid fundamentals in Quant Finance, an thorough understanding of Data Science and ML/AI algorithms, statistics, beefed-up math skills, etc.

I will move on.