This thing is like 900 pages long.
You want to take a deep breath and make sure you're committed before you even open it.
I did check the Table of Contents and scrolled quickly through, and I see it's definitely a hands-on applied technology book using the Python programming language.
I will be blogging more about it when I get going.
I came across this book entitled Neural Networks and Deep Learning, by Michael Nielsen, and decided to invest some time in reading it. I am hoping to reinforce what I have read thus far (for instance in the book AI Crash Course by Haydelin de Ponteves).
The first chapter in this book covers recognition of hand-written digits. This is a pretty cool use case for AI technology, given that everyone's individual handwriting constitutes a font in and of itself, and even then, there is a certain degree of variation every time an individual hand-writes characters. Who uses technology like this? Well, banks for example might use something like this as part of check processing!
I like the fact that this example is optical in nature - and, it could even be inverted in a Generative AI manner to generate new fonts, or new characters in some context or use case!
One of the first things this book covered was Perceptrons - and explained how Perceptrons are essentially binary (1s and 0s) as result output of action functions. He then moves on to explain that binary results are not always in demand or optimal, and that gradient values (shades of gray) between 0 and 1 are often desired or necessary. And this is why we have Sigmoid Neurons.
This was insightful, because the first book I read (unless I overlooked) never even mentioned the concept of Perceptrons, and jumped right into Sigmoid Neurons.
Nielsen then goes on to explain, briefly, the architecture of Neural Networks and embarks on a practical example using the Handwritten Digits Recognition use case.
One thing that was helpful in this chapter, was the fact that he explains how these hidden layers of a neural network "fit together" by use of a practical example. This helps as far as design and modeling of a Neural Network go.
He goes on to discuss the purpose of using bias along with weights, and then goes into a deeper discussion on gradient descent - and stochastic gradient descent. Stochastic Gradient Descent is critical for reinforcement learning because it minimizes the cost function. Minimizing a cost function does no good though, if you can't "remember" it, so the concept of back propagation is required to instill back into the model, this minimization of cost.
I downloaded the code for his Handwriting Recognition example. I immediately figured out by reading the README.txt file (most people ignore this and waste time learning the harder way) that the code was not written for Python3. Fortunately, someone else had ported the code to Python3 - and this was in another Github repository.
In running the code from the Python3 repository, I immediately ran into an error with a dependent library called Theano. This error was listed here: Fix for No Section Blas error
So I was fortunate. This fix worked for me. And we got a successful run! In this example, the accuracy in recognizing handwritten digits was an astounding 99.1%!
This code uses a data sample from MNIST, comprised of 60K samples. The
code by default breaks this into a 50/10 training and validation sample.
So now that we know that we can run the code successfully, what might we want to do next on this code, in this chapter, before moving on to the next chapter?
Here is a table that shows some results from playing with the epochs and mini batch sizes:
test.py
Run 1: Epochs=30, Mini Batch Size=10, Results: 8700/10,000at its peak in epoch 26
Run 2: Epochs=40, Mini Batch Size=10,Results: 9497/10,000 at its peak in epoch 36
Run 3: Epochs=40, Mini Batch Size=20,Results: 9496/10,000 at its peak in epoch 36
So interestingly, the numbers go up each epoch, until a peak is reached, and then the number settles back down for the last 2-3 epochs in these trial runs. It appears, that the number of epochs is a more important coefficient than the mini batch size.
Another test with adjusted learning rates could be interesting to run, as this impacts Gradient Descent quite a bit. As Gradient Descent is a Calculus concept for finding the minimum, when learning rates are too low or too high, it impacts the time and ability for the "ball to drop to the bottom" of said low point.
He also, at the very end, mentions the use of more/different algorithms, referencing the Support Vector Machine (SVM).
As Part II of my foray into Stock Portfolio Optimization, I decided to implement Python code that I found in a blog: https://www.datacamp.com/tutorial/lstm-python-stock-market
This blog was written by a Data Scientist, Thuresan Ganegedar . And he spent some time on this. I was super excited after I read the blog, to go in and compile the code and run it myself. The first thing I did, was get myself an API Key so that I could fetch data for a stock symbol.
But - I had issue after issue in running this code.
First, the code was sufficiently outdated that many libraries, especially Keras, had "moved on" and completely deprecated certain libraries, calls, etc. I spent considerable time picking through the code, doing web searches, and trying to "just get the code to run".
Second, the code was using hard-coded array values for the number of data points for the stock symbol, and because of this, I ran into null exceptions because the number of data points I had in my stock symbol fetch was over 2K less than the example had. To get around this, I wound up going with 4000 training data points and 600 testing data points. Later, I enhanced the code to dynamically set the training and testing sizes based on how many data points came down in a csv, on a percentage basis.
I also decreased the window size and the batch size as well.
In the end, I was able to produce nice graphs that matched the ones in his blog for the Predictions based on Simple Average and Exponential Moving Average. But these graphs are based on algorithms, not a neural AI network - in other words, they aren't "true AI".
The AI portions of the code did run to completion, but the graphs that came out from implementing the AI, did not look correct. The price predictions were FAR FAR lower than the actuals, as well as the ones from the Simple and Exponential Moving Average graphs.
I may come back and take another look at this, but it was a good exercise, and this exercise made me realize that I need to read and learn more about AI - and what is going on "under the hood".
I read about Q Learning, and was feeling somewhat proud of myself for sticking my toe into the water.
Then I read about Deep Q Learning - in this same book - and it was as if someone took an ice bath and dumped it over my head. I went into the tunnel - advancing through chapters 9,10 and 11, only to come out the other end confused ("what did I just read?").
The coding examples were interesting enough that I kept pushing forward, but a lot of what is in the code is masked by the fact that the math and formulas were hidden away in libraries like Keras. So while I thought the examples were cool and I had a grasp of the problems they were attempting to solve, I still came out at the end with confusion and question marks in my head.
Q Learning vs Deep Q Learning
In Chapter 9, which covers Deep Q-Learning, things start to get very complex very fast. So what is the difference between Q Learning (introduced in Chapter 7-8), and Deep Q Learning?
With regards to the approach to solving problems, the book gets into a good discussion - worth mentioning here - about the difference between ArgMax and SoftMax.
Argmax vs Softmax
In Q Learning, the 'name of the game' was to find (and use) the highest Q Value. This is referred to as "Exploitive" and is known as the ArgMax method of Reinforcement Learning.
In Deep Q Learning, probability distributions across several
variables are being continually updated during the training of the model. You have a set of (input) variables, with specific weights, but as you take random samples and compare the predicated value to the actual value, the weights are updated according to the new realities (results). This process is referred to as Explorative (in nature) and is named the SoftMax method.
Chapter 9 starts you off simple(r). With a Real Estate example of predicting home prices. Seems sensible enough, since we can all think of input variables that help drive the price of a home. The focus here is on trying to show the process, which is broken down into the following steps:
From here, the book advances into Deep Learning Theory. The idea borrows from the human brain, which is connected by Synapses that send signals. This is the fundamental concept behind Deep Q Learning because it starts with a certain number of "Layers". There are a minimum of three layers that are as follows:
The book mentions Activation Functions that take weighted input values, and return an output value. The book mentions three of these, which sound intimidating, but if you are familiar with Electronics and/or Trigonometry, these names actually make some sense:
Now from here (Chapter 9), the book goes into Chapter 10 - Self Driving Car - which an implementation of Chapter 9 - and quite fun to do. Then it dives into Chapter 11, which uses the example taken from Google's DeepMind project that optimizes server temperature with a simulation.
Chapter 11 in particular really drives home the process by showing how you can optimize or minimize costs.
Seeing is believing, and when you see this code run and start to view the results, I have to admit it is pretty darn cool.
It also takes a LONG time to run. I had to shorten the epochs from 100 to 25 to keep the job from getting killed (I am not sure what exactly was killing it). Running for 100 epochs was taking my laptop HOURS to finish (2-3 hours). But at the end of each Epoch, almost always the energy savings from the AI was superior to the energy savings of not using the AI (which in this case is modeled by the server's mainboard temperature controller).
There's so much more to discuss. But I think I have hit the highlights here.
The Crash Course in AI presents a fun project for the purpose of developing a familiarization with the principles of Q Learning.
It presents the situation of Robots in a Maze (Warehouse), with the idea that the AI will learn the optimal path through a maze.
To do this, the following procedure is followed:
I have been working on some API development in the Morpheus CMP tool.
The first thing I do when I need to use an API, is to see if there is a good API wrapper. I found this one API wrapper out on Github, called pyMorpheus.
With this wrapper, I was up and running in absolutely no time, making calls to the API, parsing JSON responses, etc.
The Use Case I am working on, is a "re-conciliator" that will do two things:
This part sorta kinda worked. The call to https://<applianceurl>/servers/id/make-managed did take a Discovered VM and converted it to an instance, with a "VMWare" logo on it.
But I was unable to set advanced attributes of the VMs - Instance Type, Layout, Plan, etc. and this made it only a partial success.
Maybe if we can get the API fixed up a bit, we can get this to work.
One issue, is the "Cloud Sync". When we call the API, we do a cloud sync, to find Discovered VMs. We do the same cloud sync, to determine whether any of the VM's fields in Morpheus change their state, if someone deletes a VM in vCenter (such a state change gives us the indicator that the VM is, in fact, now an orphan). The Cloud Sync is an asynchronous call. You have to wait for an indefinite amount of time, to ensure that the results you are looking for in vCenter, are reflected in Morpheus. It's basically polling, which is not an exact art. For this reason, the reconciliator tool needs to be run as an operations tool, manually, as opposed to some kind of batch scheduled job.
Zabbix 7 introduced some better support for SLAs. It also had better support for VMware. VMware, of course now owned by BroadSoft, has prio...
