Create a Markdown cell in your Jupyter Notebook after the code and its outputs. In this cell, you will be asked to analyze the code and relate it to the concepts from your readings. You are expected to include resources to support your answers, and must include citations for those resources.
Specifically, you must address the following rubric criteria:
- Explain how reinforcement learning concepts apply to the cartpole problem.
- What is the goal of the agent in this case?
- What are the various state values?
- What are the possible actions that can be performed?
- What reinforcement algorithm is used for this problem?
- Analyze how experience replay is applied to the cartpole problem.
- How does experience replay work in this algorithm?
- What is the effect of introducing a discount factor for calculating the future rewards?
- Analyze how neural networks are used in deep Q-learning.
- Explain the neural network architecture that is used in the cartpole problem.
- How does the neural network make the Q-learning algorithm more efficient?
- What difference do you see in the algorithm performance when you increase or decrease the learning rate?
