Newer
Older
"# Reinforcement Learning\n",
"\n",
"This IPy notebook acts as supporting material for **Chapter 21 Reinforcement Learning** of the book* Artificial Intelligence: A Modern Approach*. This notebook makes use of the implementations in rl.py module. We also make use of implementation of MDPs in the mdp.py module to test our agents. It might be helpful if you have already gone through the IPy notebook dealing with Markov decision process. Let us import everything from the rl module. It might be helpful to view the source of some of our implementations. Please refer to the Introductory IPy file for more details."
{
"cell_type": "code",
"metadata": {
"collapsed": true
},
"outputs": [],
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"## Review\n",
"Before we start playing with the actual implementations let us review a couple of things about RL.\n",
"\n",
"1. Reinforcement Learning is concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. \n",
"\n",
"2. Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Further, there is a focus on on-line performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).\n",
"\n",
"-- Source: [Wikipedia](https://en.wikipedia.org/wiki/Reinforcement_learning)\n",
"\n",
"In summary we have a sequence of state action transitions with rewards associated with some states. Our goal is to find the optimal policy (pi) which tells us what action to take in each state."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Passive Reinforcement Learning\n",
"\n",
"In passive Reinforcement Learning the agent follows a fixed policy and tries to learn the Reward function and the Transition model (if it is not aware of that).\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Passive Temporal Difference Agent\n",
"\n",
"The PassiveTDAgent class in the rl module implements the Agent Program (notice the usage of word Program) described in **Fig 21.4** of the AIMA Book. PassiveTDAgent uses temporal differences to learn utility estimates. In simple terms we learn the difference between the states and backup the values to previous states while following a fixed policy. Let us look into the source before we see some usage examples."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%psource PassiveTDAgent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The Agent Program can be obtained by creating the instance of the class by passing the appropriate parameters. Because of the __ call __ method the object that is created behaves like a callable and returns an appropriate action as most Agent Programs do. To instantiate the object we need a policy(pi) and a mdp whose utility of states will be estimated. Let us import a GridMDP object from the mdp module. **Fig[17, 1]** is similar to **Fig[21, 1]** but has some discounting as **gamma = 0.9**."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from mdp import Fig"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<mdp.GridMDP at 0x7f02fc0cba58>"
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Fig[17,1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Fig[17,1]** is a GridMDP object and is similar to the grid shown in **Fig 21.1**. The rewards in the terminal states are **+1** and **-1** and **-0.04** in rest of the states. <img src=\"files/images/mdp.png\"> Now we define a policy similar to **Fig 21.1** in the book."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Action Directions\n",
"north = (0, 1)\n",
"south = (0,-1)\n",
"west = (-1, 0)\n",
"east = (1, 0)\n",
"\n",
"policy = {\n",
" (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None,\n",
" (0, 1): north, (2, 1): north, (3, 1): None,\n",
" (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, \n",
"}\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us create our object now. We also use the **same alpha** as given in the footnote of the book on **page 837**."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"our_agent = PassiveTDAgent(policy, Fig[17,1], alpha=lambda n: 60./(59+n))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The rl module also has a simple implementation to simulate iterations. The function is called **run_single_trial**. Now we can try our implementation. We can also compare the utility estimates learned by our agent to those obtained via **value iteration**.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from mdp import value_iteration"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The values calculated by value iteration:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{(0, 1): 0.3984432178350045, (1, 2): 0.649585681261095, (3, 2): 1.0, (0, 0): 0.2962883154554812, (3, 0): 0.12987274656746342, (3, 1): -1.0, (2, 1): 0.48644001739269643, (2, 0): 0.3447542300124158, (2, 2): 0.7953620878466678, (1, 0): 0.25386699846479516, (0, 2): 0.5093943765842497}\n"
]
}
],
"source": [
"print(value_iteration(Fig[17,1]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now the values estimated by our agent after **200 trials**."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{(0, 1): 0.43655093803808254, (1, 2): 0.7111433090760988, (3, 2): 1, (0, 0): 0.3220542204171776, (2, 0): 0.0, (3, 0): 0.0, (1, 0): 0.20098994088292488, (3, 1): 0.0, (2, 2): 0.8560074788087413, (2, 1): 0.6639270026362584, (0, 2): 0.5629080090683166}\n"
]
}
],
"source": [
"for i in range(200):\n",
" run_single_trial(our_agent,Fig[17,1])\n",
"print(our_agent.U)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also explore how these estimates vary with time by using plots similar to **Fig 21.5a**. To do so we define a function to help us with the same. We will first enable matplotlib using the inline backend."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"\n",
"def graph_utility_estimates(agent_program, mdp, no_of_iterations, states_to_graph):\n",
" graphs = {state:[] for state in states_to_graph}\n",
" for iteration in range(1,no_of_iterations+1):\n",
" run_single_trial(agent_program, mdp)\n",
" for state in states_to_graph:\n",
" graphs[state].append((iteration, agent_program.U[state]))\n",
" for state, value in graphs.items():\n",
" state_x, state_y = zip(*value)\n",
" plt.plot(state_x, state_y, label=str(state))\n",
" plt.ylim([0,1.2])\n",
" plt.legend(loc='lower right')\n",
" plt.xlabel('Iterations')\n",
" plt.ylabel('U')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is a plot of state (2,2)."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEPCAYAAACp/QjLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYFNW5h99hFhZngAFZIrs77pq4RFGHEFFxTWJUXJMY\nlyQmepMYNLkxaG5y45WYiMZd401cuCpxQVFj1EEjAjGyyaYoIJswCMgOM1D3j2+OVd1VvU73LN2/\n93n66e6q6urTp6vO73zLOQeEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhChIHgJWAbMT7L8QmAnM\nAt4CDmmmcgkhhGhBjgcOJ7E4fBno0vj6FGBKcxRKCCFEyzOQxOIQpBpYlt+iCCGESId2LV2AAJcB\nE1u6EEIIIZqHgaS2HIYCczHrQQghRAtT1tIFwILQ92Mxh3VRBxx66KHezJkzm7VQQghRAMwEDsvm\ngy3tVuoP/A24CFiY6KCZM2fieZ4ensevfvWrFi9Da3moLlQXqovkD+DQbBvnfFsOjwMnArsDS4Ff\nAeWN++4FbsRcSXc3bqsHjspzmYQQQqQg3+IwMsX+7zY+hBBCtCJa2q0kMqSmpqali9BqUF34qC58\nVBe5oaSlC5AmXqP/TAghRJqUlJRAlu28LAchhBAhJA5CCCFCSByEEEKEkDgIIYQIIXEQQggRQuIg\nhBAihMRBCCFECImDEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQgh\nRAiJgxBCiBASByGEECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUQIiYMQQogQ\nEgchhBAhJA5CCCFC5FscHgJWAbOTHDMW+ACYCRye5/IIIYRIg3yLw5+BU5LsHwHsDewDXAHcnefy\nCCGESIN8i8ObwLok+88E/rfx9VSgK9Arz2USQgiRgpaOOfQBlgbeLwP6tlBZhBBCNNLS4gBQEvfe\na5FSCCGE+JyyFv7+5UC/wPu+jdtCjB49+vPXNTU11NTU5LNcQgjR5qitraW2tjYn54rvteeDgcAE\n4OCIfSOAqxufjwH+2Pgcj+d5MiiEECITSkpKIMt2Pt+Ww+PAicDuWGzhV0B54757gYmYMCwENgPf\nznN5hBBCpEFzWA65QJaDEEJkSFMsh9YQkBZCCNHKkDgIIYQIIXEQQggRQuIghBAihMRBCCFECImD\nEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQghRAiJgxBCiBASByGE\nECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUSIspYuQLqMGQPf/z785jeweDHs\n2gVlZXDXXVBV5R/nebavtDR8jsmT4aOP4KKLwvveew9eeMH2b98ODQ328Dz/mHPPhW98I+c/rSjZ\nsQMWLoT582H1ali/3h6bNsHOnf6jocF/HfwvvvpV+O53W678QhQ6JS1dgDTx+vb1uPlmeOAB+N73\nrPG/7jp45RUYPNg/8MgjYf/94b77oGNHf/vjj1tjsmULrFoFt99uQuP2XXstXHAB7LOPfa683L6j\nXaNt9frrUF8PDz4YXcAVK2CPPZL/iH/+08o+fboJWxTbtsHatanPBdZggpWzvh7+9Cd7TJ0K3bql\n/nxz43lWj3fcAa++Cr1723+1xx7Qtas9dtvNfk9pqdWRex38L2bNgn//G15+uWV/jxCtnZKSEmg7\n7XxWeNXVnjdihOc99pj3OYce6nkzZvjv16zxPGuC7OF5nrdyped9/LHn7bWX5735pud16uR555zj\n79+yxfO6d489TxR/+YvnXXRR9L6XXvLPl4gPPvC8Hj3suMWLo4/Ztcvz+vTxvM6dY7d//eue98IL\nsdvWr7dz/fa3ntfQ4HlnneV5Q4d6Xs+envf++8nL0hJs2GD1vu++nnf//Z5XV5f9ud54w/OGDMld\n2URx0NDgeRMnet4vf2n3Wkuya5fnTZ7seT/9qecdeaTnrViRn+8BvJStawLaTMxhwwZ46y34ylf8\nbRUV5gJyzJoFlZX++02b4Pe/h29+044dMgQGDICnnvKPGT/erI1DD03+/RUV5gqJ4sYbU5f/xhvN\nOhkyBBYtij7m0Udh+XI48EB/24QJ8Le/wbvvxh7705/a86JFcOutVj8vvww9e8LWranL05xs3Agn\nnmiWwcyZZsHtvnv25+vUySxAUTzs3Anvv594X6rPPvKI3Vc33gi33QZr1uS+jOlQX29ejUMOgW99\ny7wUDQ3WdrU22ow4dO9uLpeePf1t8Q32kiXmi3ZUVcEf/2hulsMPt239+9tz5872/NZbcMopqb8/\nkTgsXw7z5iX/bF0dTJwIV18NgwaFxeG66+DFF+GWW8zVVV9v2z0PbroJDjgg9viFC+GZZ0zYpkwx\ncXjgAXOFdezY+sTh+9+3+r/vPujQoenna43isH07fO1rFj8pVBYssPtr7drm/d5Fi6wDt99+sXGn\nbdvgJz+Bfv0Sf3buXPjyl+Huu83lOm2auaE//DB8bEODxTb79MnsHtq+HX79a4tpJuP55+Ggg+DJ\nJ82tPX8+3Hyz/TZXnoYG//5vadqMOPToYQ17ScB71r59rOWwZIn1DoL++oYGe3a98QEDzEpwvY1Z\ns1JbDZBYHF58EU47LbZc8bz0EgwdaoIULw47dljD/sc/wubNdq5t22zf3LkWHxk50vY57r4bLrsM\nvvhFmD0bTj0V9tzT9nXo4H++NTBpksVaxo5NXkeZkK44PPUUnHNO8mOWL4dPP216ma691gR78eKm\nn6s1EN/h+fe/4YQTLF62YEHzleONN+Dooy2JpHNnWLfOtq9ebffUokX2OqpBHT/eLNbLLrNrcNgw\nuwb33NMST4KsXWv30QsvWELLsmXplW/hQvjSl+DOOxPHwLZsMWv5P/7DROHvfzcPiLsf9trLxGHG\nDBOPMWPS++5802bEoWfPcA8hynIYMMACzAMG2LayMnNnOHHYd1+7yOvr7SKYPdtMvFQkEofp0+Go\no/xIRxQvvwwjRtjrgQNjxeHNN8399fe/w1lnWcPnei3jx1vjVlnpi4PnwdNPm2C4+jj3XP98ySyH\ns86CUaNS/9amsHkz/OpX/vvRo8362W233H1Hx46pxaG+3tyJ48cnPsbzLPvs/vubVp6XXzbLcP/9\n7ZpqCzzyiFl0UYwbZ9aqE7qPPoIzzoB777UGdskS+5+nTMlvGV991f6fxx6DH//Y7p0lS8wSP/54\nK8v48dCli2W6BXn4YbjmGuuYXXllbMfENcaOTz6B446zduCVV6yNWL7c9r33nlkSmzaFy/f22/Y5\nl0X58cfhY+rqTAi2bLG24pRTwp2kvfaCZ5+F4cOtA1xXl01t5Z42Iw7OcgjSvr012B9/DKef7ovD\nCSdYJkxZGZx/vjVOxx9vn7n2WlPm+nq7ACor08vsSSQOM2ea5dGuXeKGYfp0Mx3Bvuuzz/x9U6b4\nwnHiibGN++uvw8knW8PqxGHePLOGDjnEvvO118zacCQTh+eey79vc9w4M5U9D+bMMdN55Mjcfkc6\nlsOjj9pN3ru3vf/Tn6wXGmTCBHM5ut5oNtTXWwbaAw9AdXVq/3drYNUquPhiuOee8L5Fi+CHP7T7\nYskS+33nnAPXXw9nn2331+LF1uCedFJ0o5kL3HXz5JO+q7h/f9t+6qkm/P/1X9bQdusW+x8+8wz8\n/OcmLl/8Yvjce+7pi8P69SYyI0dafLKsDPr2Ncth8WK7/1asCN83//qXdbb+/Gf7//v3h6VLY49Z\nu9aEYehQux6D8dAgBx9s9TxxotV1vuo0U/ItDqcA84EPgKg+6+7AS8AM4D3gW4lO1LNnWBxcQHrV\nKgtWBdNJu3eHI46Av/4VfvQjXwBKS+1zACtX+o1HKqLEwfPM8nDi4BqGJ580cxOsfB995Kfbtm8f\n6/Z55x047zwrx/HH+26hnTvNlD/qKF8cHnzQGt6hQ/3ex9ChsWM6ErmVnLWSjpXUFJ580p63brXe\n6SWXWCwklzgBTGSpeZ656X7/ews8bttm8Z4//Sn2mF/+0oS5KeIwbpw1mCedZP9Dc4nD5MnRjXs6\n/OQn1ugddFDsds+DSy816/LMM63T9dvf2rX5wx/aMQMHWq986lQ49tjY5I5kZFIv27ZZ/Oa//xtq\navztAwZY527ffc3H76iu9v/D99+Hyy+3nvh++0Wf/4ADzGW7cydceKE14MGkkr59fREaNcpcQs89\nZ4kVV1xhscGzzzaL03Xs+vePtRx27LA6HD7c6jCZS3XffU2IvvQlE5CNG9Ovq3yST3EoBe7EBOIA\nYCQwOO6Yq4HpwGFADfB7EgzMu+46U+ggrsHets0eW7f67otu3ZI3/OXlJiY9eqT3Y6LEYfVqO0/3\n7tYwOMvh3HMtEwHsIhs0yEQBwnGSd96BY44xK6Z7d7/hmzMHvvAF+x1OHB57DP7v/+wiSkQiy8EF\ny1wMJh9s2mQB/qoq6zU980x+Bg06gd+2zXpks2fH7n/nHbvBRoywuvvLX2x79+7+MW+/bZ+/+OKw\nSyIVs2b5LqTf/Q5uuMEvV3OJw6hRFu8K8r3vQW2tP3gwKjj+5pv2GDPGMtyCPP201duPf2yN3Ztv\nWqzo/vv9xm3AAIs53HOP1e+0aSbAyepwwwbYe2/4xz/S+2033WTCddllsdsHDbLG8557Yhvb6mq7\n3hoazAJwQd5EHHyw3V+uDm67LXZ/376W5PHlL1vH8oADTBCOOMKskeuvhx/8wCwHR79+8MEHfpl/\n9jO73saMSS/W5o6pqioOy+EoYCGwGKgHxgFnxR2zEmjMG6Iz8CkQ2Xz17x9u7F1D68Rh2zY/G6Z7\n99TisHJl+imV8WmzYJ/v08dex7uVpk+357lzY1NTO3Twz7NxowVDBw3yB3h17Gi/Y/ZsOOww2+bE\nwTWC6YjD88/HNlRTp9oFnCgdNxe89poFD/v3N6tn/frkZW0KLjZz0UXWGAR54gkb0NiunYn/7bdb\nL/CTT/xjHn4YvvOdsEsiHe67zxrISZNMEE46ybZnKw7PPJNZauWbb5rYBxuRhQstJjBpkjVaFRXQ\nq1fs5xoarBf7n/9pHY+gODQ0wC9+YfvbtbP/8IEH4Kqr/GsczMd+zz3mihk82Nyc551nIpmI//kf\nu9/coNNkzJljFvKdd4b3XXml/W6XaehwlsNdd9m+q65K/h1VVdY2/OY3dh3EW7b77msW0tix9v68\n8+x1v34myP/6l98hcLhO6UMPWVD72Wft3JkmYbQmyyGf02f0AYJeuGXA0XHH3A+8BqwAqoBzyQDX\nm9++3R7t2vni8KUvJU8Jy0Yc4hvWFSvsJgPfrbRzp712PvFly2ID6UHLYfFiuwiDF1C7dla2OXNs\ntDbYhffRR3bu0aP9tNwoOnSw7z7jDOvhHnywbZ82zdxWuRSHW24xP6u7kSdNMhP9pZeswaup8UUv\n13TqZNkdYI3D5s1WT55n40JcILqqyoT6llus4QOrx2eftXhPXV1m4uB59lkw19X55/v/XzbisGWL\nuVD+8hezYlatsg5DfPpykDFjzDINWky332698+eeM3+65/mN3osv2m995BET1Keftv9lwwY7rqTE\n6qx7dz+te8AAa2jdeBpHdbU10mDiMHmyCYuzjONZu9ay655/Hr7+9cS/adcu60iNHm297nhhA2s4\no/z23bpZr/2OO0w402mQTzjBOl977RXeN3y41a1zP++xh7nVnGstETt2WEfw4ostKaa6OnU54qms\nbD2WQz7FIZ2ReT/H4g01wF7AK8ChQEg7R48e/fnrmpoaampqQm6loDhEzZ8UxImDy2pKRSJxcDEO\n51ZaudIu7E8+sYZi+fLE4rBokVkN8XToYBenc8nstpsFB084ITYTKIqOHf14h6sLz7Osi/POswB6\nrrj+ent24vD229YbmzrVxME1xvmgUydrpHv1skZv6VJrBD/80BpAl548d649Dx7sWw5vv22iPmiQ\nNWyZuJVcozFkiDXEt97q78tGHJzQuLjRrbdag/rQQ9HHf/qpuY7+/nffdblpk9XBI49YYsaoUdY5\nGDnSft/ll/vX4U9/6l8XZWV233TsaFbHD3/oN6zDh1vDn6yB69vX6uJHPzJ3ZxR33GH++cMPt9/l\nxCieCRNMPHr18t2A6VJdbUJ97rmxU+kk46GHEotISYkvDJlQXm6W1ZYtFtPJhqa6lWpra6mtrc3+\nBAHyKQ7LgWDyaT/MeghyLOCMzQ+BRcB+wDvxJwuKgyPoVtqxw/7URD2YeFzMISqbIYoocVi5Mmw5\nLF1qN+Gnn5rlsmKFxRTiywwmDgMHhr+rY0drhFzaqTNZo4Qk6rPOpeUaqhUr7By9ejXNcoi6sZ3L\nYccOE54jj7Se3Pr15rPNF506mRh861tmFbjRs6++atktQR/54MG+YHuepSu6mzcYzEyHiRPN1755\ns/W8993X35etOJSXWwabS1MOXi/xjB9vvfsvfMF3P0ycaJ8ZOtQa/iuuMLfJ6tX2W/fYw0R75MhY\nF0rnzvYbli2zzsPXvubvKytLbr2A1fFzz/mDzNavt7Rxx44dlgTwz39aucrKrN6iev9/+IMlDZx8\nsv23mVBdbXXhOivpkKsxN/Hcfnv6bVAU6bqVEoms6zg7brrppqzLks+YwzvAPsBAoAI4D3gu7pj5\ngBvT3AsThrjhKYkJupXALvx03RiZupVc2iyYn3Xr1mjLwYlDWZmfLhv02ca7laIa/I4dzVLYe297\n78QhPlsrig4dfHFwwef337fMjWRTgKTDBRdYui34QW/32+bNs4a4stLvmaZqXJpCx47mSrjkEnu/\ndKk1zJMmWSPpmDHDMqgqK+0/2rjRBGTYMNvfpYs1zOmOT3jxRROHYcOsMQuSqTjU15sFcMEFVoZZ\ns8x9mOw/euwxa+SDPUw32K9TJ7ve9tzTXpeXW6rl+edbFlJFRWyD4sThwQf9/ZniBOnQQ8NTvDz/\nvF0DTkC7d48ecDh9ulm7Y8b42T+ZcOCBNsAsnc5TvuncuenikI7lcPPN5oKLx43PgPSm9UlGPsWh\nActGehmYC/wfMA+4svEB8FvgS8BM4B/Az4C0B+cHLQfIbGqGbGMOnmcX/OOPhy2HXbsssNijh50/\nkTi48johicfdwG6qECcO6bjA3Fwt4DdUCxbYDdpUcRg3zh8r4EbJutjKnDl+aqS7QLNpbNLFTeEw\neLC5Yqqr7f+YOjW25+1m2AXrTS9ZYo3RccfZNjflSDo35Pr19tmaGkuBvPzy2P2ZisPkyebz3m8/\nO/fTT9v7+MQHR12did2pp/o9zC1bbBCey5wJjtnp1cvce2efHX0+Jw7jxzd9LMoxx4QHxf35z/Dt\nb/vvE4nDAw9YHCPblOdTT41177VlnDgkStMG60jcdpu5TD3P2oVZsywhpF8/c5ktWGDWXFPI93oO\nLzY+gtwbeL0GOCPbk1dUWEVmKw6ffhrOfEj2XTt2+D7s3Xc3d4RLj3RupXXrzLQuL7fjV66Mnc4j\naDnU1cXOFeVwWSTOCnJmdt++qcsZrAPXUC1caMHtpooD+EK3YIGNwVi0yJ/IzGVl7bVX+uNHssW5\nkUpKzI8+frylsK5Zkzi/vXdvC5bvu2/siG1nPaS6FiZPNrdZcCr4IJmKw/PP2wDGLl0sR/6118wC\ncIH2eGprLanA9UzLy80tdeSR0Z2cXr3sd7qpVeKpqrLMm23b/My4bDnmGIt5OFatMnfSuHH+tm7d\nwvMyNTSY5ZNqXqJioazM7tOtW+2+/8EPrN6C4zruuss6pR9/bANlP/7YsgMff9xcg088YXG1q6+O\nti7Spc2MkI4i3q2UiThUVFjPPtGNHo9r7N96y943NFgj7hoU51Zav956seXlJhQVFbHlCqayrlkT\nfVMHR1C7c0O0kMQT/D3Ogli2zA8ebt9uvvdVq6I/P3ducn+sE6gFCyxtta7O0vree88Xh9//Pjxa\nNB8EM00GDrRG5otfTOxa7N3bGtOj43Lm0hXNKVOSx1EyFYd//MN87F26WCdizhxLOkhkObz2WqzL\nrKrKfP4ulTaenj0TWw1g1+4jj1jwuqk++CFDLFPIZQg+/7w/ut/RvbsFnZcs8be9/rr1fKOyhoqV\nykqLXXztayYEwZH99fUW5L/tNrvH7rjD7skJE8x6uPVWs56feio8LixT2rQ4NNWtBOmLg1tsxgUv\n6+tjxSFoOThxWLMmNkAHvijt2mUNa9QgvKjUwCuvTC8Tw/2evn39hmr5cnvv4ib77Wc3cxRO/OJx\nPT53sy9Y4Afzd+ywnorrsbdrl3gxo1zxhz/YeAPHAQdYw59sEsXeva2c8WMvnAswGR9+aA1xsmBx\nJuKwZYtZP0ccYdfI669b2nGXLomF6rXXYqesr6oyS+iEE6KP//WvLZMoEZ072/995pnplTkZLvvL\nWQAvvBA7rQvY79q40QLojnHjLCYifKqqLKZw8MEmnMFY48SJZgkOH27tx6RJNmjw6adtJPegQXbv\nnX12ep3JZLRpcQimskJ24pCpteEayXhxcJZDUBzq6sLi4NLk3IpvwVG7Qbp0iX1/zz3pldVZGfHi\n0KePX18bNiTOiAgGtIK4SdjcHE/OVeVYs6Z5A4LXXhvbUB56qLkYgwMO4+nd2/4jN/bDUV6eeuT4\nmDHWALqAfBSZiMO771pZ27e3/3rDBouDRA22BLP+1qyJnf6kqsqOTZRxd9BBia8v8OMmQWukKQwb\nZtbD9u0mZPFT4btU4meftWt/1y4/hVX4nH66Nf433WQpukFPwv33mwiUlZnb8NxzrbPTrp1lqZWU\n2Aj3TDK3EtFm1pCOwt1I2biVMrUc3PclEgcXkE4lDmANwief2M2dKAiXbiwkHpezX1Hhr4G9YoWJ\nw+bNfq800fkTicPHH5vrxgVuP/44NkDeo0fiicWaA+czj58vKIiLg8QLiMssS8asWTZuI7heeTyZ\niMO0aRazAb8jcOyxsVlxQd54IzyosKrK3FzZBv5HjMguOygRfftaBtmbb5qVG28Vjx9v19ell9qM\nqT/5id0fUencxYwbmQ1WP04cVqwwy8yNKTn7bBsNPniwpSw7CyN+9Ha2tGlxcDdSc7iVIFYcXEaB\nc/9EuZWSicOyZcnndco2oLv//vadZWVWnjVrzBXUsWOsbz1TcVi61J8uYft2C+YHy9jSaYT9+pnv\nPZU47LlnWMRSuZXc1O7xFkc8mYqDa5jdNXLssdYQRFkO774bni+osjK/Y0kypUcPc1O9/LJlEMXT\np489Xn/drtN99kkcLxGGS5YAG8V++um+a/eOO/zjglZ0rihqt5KbqiKT71u71rII1q6NbWCjAtJ1\ndWH3EKQWh1mz0p/tMp6jjjKBKi01yyGYSht0WWQrDps22TF77GHf8eGHNmq7pcWhpMTGDCSzXo49\nNnohlVTisGSJ1Veqqd2ztRx69zYXwhe+kNhymDEjnFF04YWty1/fs6dd82+95U+RH0Xv3iZ0t94q\ncUhFcK2K8ePzM5FlItq05eAau2zFoUOHzLI0nDhUV4fTYKNSWdesiZ5+oH17fxbWKFL1UNPBNVTB\noHdFhZ9nnuh3r1wZvX3pUuudbNpkLiU3PmPPPc3v2VJr8mZCdXXsKGBHUBzq6836ePttXwzmz09v\nQF+64vDpp/Zwg8PKy/0BS1ExB8+z8RXx4nDxxam/qznp2dOukxUrks+KCjbw7/XXcxfvKFSc5VBX\nZ9fA8OHN991tWhxcL2v7dnOjZCoOmbiUILk4lJZaw7Jpk/2hznJINHfSmjXRVkWucG6lDRv8dNn2\n7f0xFMFlR4MkWjJz8WLLRtqxw14HB+/lcyR0c1BW5gekH3zQsojq6nxxeP/92GkyEpGuOMybZ1ZY\nVMptlOWwbJldT/keO9JUevSwRIUjjkgdf/rmN0308nkPFAIuWWHsWBOGTNusptCmxSHoVurSpXnE\nYelSa/CjLIdNm+yczl2VLOawZk3y4GZTcW6lNWt8C8UFLhNN7rVrlz3ib+zgFOK77WaNZToD8toK\nQcvBpWIGG/n33zcfeSrSFYcFCxIP1IuyHKJcSq0R1wk59tjUx1ZW2pTpIjllZXZPjh0bnp4k37Tp\nmENwnEM24pDJ8e77tmyxBj+ROLjRzGVlLS8OO3daOd1N68Shf/9ocVi71j4X73KaOtUyfDp39qcP\nj5pSua0SFAe38Hy8OOTSckgmDkHL4ec/t+yftiIOZWXWEUlHHERmnHxy8w8UbNPi0LGjNX7btlkj\n3ByWAyR2KznLwZ0/keuofXsTjnyKg3Mrffpp2HIYMCBaHFav9scCBJk2zc+Kqay0KTOaOsCmNREv\nDrvvHjvu4YMP/EkQk5GuOLiJEKNw1vBHH9kymZMm+euUtwW+8Q3FEfJBSwTu27Q4HHigNVZTpzaf\nWwn83OPg1ADt2tnAMmc5lJebVRM8xtFSbiU3QK5PH18cgv7tujoTh/gGbvVqf36oykpruApNHBoa\nzCpcu9YsKycODQ0WpI+aIDGeZOLw3nsW5IbklkNJiZXHLVYE6bu1WgP33tv6YyNtjUWLbOBbc9Om\nxaG01J95sFu3zKbKzUYc3Ays1dXWuAbFyFkOQXGA6O9oDnEIWg7OreTcRT16WE/ZTdPx73/b9tWr\n7TfGWw5BgenZ094XklvJDYJbvNisKjeAECyrrFev9AaaBdcRj+f4483d0tBg35PMEmnf3oLWJSWW\n/fbhh5p7qJiJXy2yuWjTAWkwd8eiRXZTZpqWmmnMYcgQm/nQzX8TFKN27SwDKOhWguiFSzp0sJu+\nuS0HR1WVWQAuc8n1dhNZDkGBOeggG09QaJZDfb0/6nvLFl8cnGCkQzLLYf16uz4XLTIBTnbtVVRY\n+uwRR9hzp07Zj5gXIlvatOXgGDjQ8u0zGYiVjeXg/O6uFxm8waPcShD9HW7cQXMEpNeuDQ/ecmvx\nujUZXFlXrUptObi5fZKN7m5rOHFws9cGU1vdOt/pEC8OvXrFTr/dq1dyl5LDWQ5HHGFWXTrxDiFy\nTUGIQzbGT/TOAAAVUElEQVRkIw6HH24pna4xDVoOUQFpiP4O579uDrfShg3hoLgTB7dAvRODJUus\nl+x5sYuNBC2Hgw/2R4AXClHi4Bp5VyfpEBSHzz4zN11w6vKePcMTFkZRUWGWxuGH2zTecimJlqCo\nxSFTtxKYWyVqRtf4VNZkbqXmEAfnVtqwIfw9lZUWKJ81y967Bs25Vdwkgo5gxtNhh8UGSwsBZylE\nWQ5uuvN0CIqDm4N/+3Z/wGHnzvYdqZZ7bd/eLLN+/awcEgfREhS1OGQ72tCtVRBlOaTjVmoucdi8\n2Z6D5bzpJkuLq6y0XimEe8luKhC3z80XBbav0FIVk7mV3DxS6RCsNycOGzbYecFiGcuWxS4bG0VF\nhblJ3RgZiYNoCdp8QDpbmpJtk8xycL74lrYcysos6B0fyHRz+FRWmosDrEHbtcsarn79/B7wE0+Y\nz71z5/wv3tOSJBOHFSvSF4eg5TBtmqVaf/aZnaNHDxNr9x3JaN9e4iBangK+5ZNz2WXZfzZdt1JJ\nSXR6res55ttyWLs28XdUVlqjBSYMn3xi1kGHDr5baeZMW5c51WykbR0nDitXWkDeueTAXwsjHZw4\nNDTYVAff+Y4vDvvsY267bdtSi0O85aCAtGgJitat1BTSDUgnmvW1osIygzIZl5EpiSwHR2Wl7wt3\ns7e69FTXyH32mQVUC31ytPJyW9B9wwaLrbiAdH29CWy6abuu3hYtMkuhf/9Ycdi0yQQolSXiLIfu\n3W2Vt0JKGxZtB4lDFqRrOUS5lBz5vuFLS1OLg2PnTkvDdVaGG8y1YUNxiENZmTXgu+/ur3/d0GAC\n3qOHP7I8FU4cnLXhplt24rB+vVkDqToFw4fbgLmOHc1ya4kBUEIUrVupKURZDlET7zXn9LrxOHFI\n1Et14tCpkwlBUBxcYPWzz+w3FfoArPJyiwW4OJQTh5UrM5sKwomDc0+56ZY3b7aFfUpK0nNR5WqZ\nRyGagiyHLHDB2ajpM4JupZYUh3TcSmA9WTcewh0btByg8C2HROKwenVmiQtR4uAshz59LH24kKY6\nF4WNxCEL0rEcUrmV8o2zHJIFpMGC0PFupaDlAMVrOaxendlI8GTisMceEgfRtpA4ZEFUzME1DK3F\ncigtNR93oobdzRYbJQ7OcnDiUOiWQ1mZjUFw4uCylYJB+nRIJA7ufadOEgfRdpA4ZEEiyyG4raXF\nwbm+UlkOXbokjjk4t1IxWA7gC4HLVmqq5dCrl40TKS83MZblINoSEocsSJStFNzXGtxKkHgt38pK\nE6/27aMth6BbqdAtB/efufmjnFspW8vBxSqqqy2m45ICunZNPXWGEK2FfIvDKcB84ANgVIJjaoDp\nwHtAbZ7LkxMSjXMAf8bWiorWYTkkEqjKShMD16AF52Bq187cLCUlJgzFYjm4+aOCMYdsxCE4i+0B\nB/ji8MQTcMIJuSu3EPkkn6mspcCdwFeB5cC/gOeAeYFjugJ/Ak4GlgG757E8OSMqWynecjj1VH96\n65bAiVUycejc2W/QNm6MzVZat85fXa9YLId4cairy86tFJyocPBgq1vwF4sSoi2QTBx+EvfeA+qA\nfwKL0jj3UcBCYHHj+3HAWcSKwwXAeEwYANakcd4WJx3LoVu3lp12wpUnkfVy4IHwu9/B3/4WHXNw\nmU4dOxa+5eDEPricqlsoafcMuiulpWaBlZb69T5ihImFEG2NZG6lKqAy8KgCjgReAkamce4+QGA2\ne5Y1bguyD9ANeB14B7g4rVK3MOnEHFqaVG6lDh1sMfig5RCMOWzcaNbFzTfbNN2FTCLLYe3a8Cp6\nySgtNVdU8DOnnw6XXpq7sgrRXCSzHEYn2N4NeBV4PMW5vRT7AcqBI4BhQCfgbWAKFqOILcxovzg1\nNTXU1NSkcfr8UF7uT7PgcOKQzlrDzUEqt1LwuPiYg+sBd+oEX/96fsvZGnB15aYlLyuzUc1bt2Y2\nOaITB7mPREtRW1tLbW1tTs6VTcxhbZrHLQf6Bd73w3cfOZZirqStjY83gENJIQ4tTfv2/jgBh2tg\nWovlkMqtFDxu1y5rDN1vckuexv/GQsVNQOjEvqzM4g3V1ZnNa1Raaq6ogw7KfRmFSIf4jvNNN92U\n9bmyyVYaCqxL47h3MLfRQKACOA8LSAd5FhiCBa87AUcDc7MoU7NSVQXTp8dua22WQyq3ksONadi6\n1RcD51YqFnFw4zkcZWVmAWQaM3JCm4krSojWSjLLYXbEtmpgJXBJGuduAK4GXsYa/wexYPSVjfvv\nxdJcXwJmAbuA+2kD4gDhBVhaq+WQrltpyxbfyig2yyF+TeemiANIHERhkEwczoh77wGfApsyOP+L\njY8g98a9H9P4aNO0VsshHbeSEwcnJMGYQzFw3HHgBSJkLnZw8MGZnUfiIAqJZOKwuLkKUQi0tmyl\nTC2HbdvClkMm01UXEi7mkK3lUOgr54niQNNn5Ii26lZyo6ErKnyBK7aYQzxlZRaDkVtJFDMShxzR\n2iwHR6ryOCEIikixxRzicS45l9qaLhIHUUhIHHKE81mnu6Rkvtmxw55TpWI6cQjGJlzModjFoWvX\nzD4ncRCFhMQhR+zc2dIliGX79vSOi1+kCIovIB2Pa+QznVNK4iAKCYlDjmht4uAsh1S45U3lVvJx\nlkOmc0pJHEQhIXHIEW1ZHKLcShKH7CyHkpLM3VFCtEYkDjmitYlDulNNJ7IcduyQOGQjDtXVrSfu\nJERTkDjkiNYmDmeeaWtIp8LFHOItByjemEO24tC7N3z3u7kvjxAtgcQhR7Q2cXCruKUikeUAEodM\nYw5VVXDLLbkvjxAtgcQhR7Q2cUiXqHEO6c7oWqhkm60kRCEhccgRbVkc4t1KznIILmZUTLj/slh/\nvxAgccgZbVUc2rWzqSKiLIdibRzd+g5CFDMShxzRVsUhyoVU7JaDxEEIiUPOaGho6RJkR5SVUOyW\nw4EHFm8arxAOiUOO2LWrpUuQHc5KaN/e3+bmYwpuKyYOO8ziMEIUMxKHHNHW3UpBIaivt+dM1k8W\nQhQWEoccUYjiIIQoXiQOOaKQxCHdeZmEEIWLxCFHtNWAdFTMQZaDECLZGtIiA+66Cz74oKVLkTmy\nHIQQUUgccsTee9ujrRGVtirLQQght1KRI8tBCBGFxKHIUcxBCBGFxKHIkeUghIhC4lDkaJyDECIK\niUORI7eSECIKiUORI7eSECKKfIvDKcB84ANgVJLjjgQagK/nuTwiDrmVhBBR5FMcSoE7MYE4ABgJ\nDE5w3C3AS4CmemtmZDkIIaLIpzgcBSwEFgP1wDjgrIjjfgg8BdTlsSwiAVExh8GDYc89W6Y8QojW\nQT5HSPcBlgbeLwOOjjjmLOArmGvJy2N5RARRlsOrr7bd9SmEELkhn+KQTkP/R+D6xmNLSOJWGj16\n9Oeva2pqqKmpaVrpRAxlgSshuGSoEKLtUFtbS21tbU7OlU8f/zHAaCzmAHADsAuLLzg+CpRhd2AL\ncDnwXNy5PM+TUZEPXnsNhg0DVa8QhUeJrdiVVTufT8vhHWAfYCCwAjgPC0oHCXq2/wxMICwMIo+0\n1anGhRD5JZ/i0ABcDbyMZSQ9CMwDrmzcf28ev1ukicRBCBFFW0kdlVspTzz7LJx9ttxKQhQiTXEr\naYR0kSPLQQgRhcShyPnqV+E3v2npUgghWhtyKwkhRIEit5IQQoicInEQQggRQuIghBAihMRBCCFE\nCImDEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQghRAiJgxBCiBAS\nByGEECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUQIiYMQQogQEgchhBAhJA5C\nCCFCSByEEEKEkDgIIYQI0RzicAowH/gAGBWx/0JgJjALeAs4pBnKJIQQIgkleT5/KbAA+CqwHPgX\nMBKYFzjmy8Bc4DNMSEYDx8Sdx/M8L89FFUKIwqKkpASybOfzbTkcBSwEFgP1wDjgrLhj3saEAWAq\n0DfPZRJCCJGCfItDH2Bp4P2yxm2JuAyYmNcSCSGESElZns+fiS9oKPAd4Lg8lUUIIUSa5FsclgP9\nAu/7YdZDPIcA92Mxh3VRJxo9evTnr2tqaqipqclVGYUQoiCora2ltrY2J+fKd0C6DAtIDwNWANMI\nB6T7A68BFwFTEpxHAWkhhMiQpgSk8205NABXAy9jmUsPYsJwZeP+e4EbgWrg7sZt9VggWwghRAuR\nb8shV8hyEEKIDGnNqaxCCCHaIBIHIYQQISQOQgghQkgchBBChMh3tpIQQmRFt27dWLcuctiTiKO6\nupq1a9fm9JzKVhJCtEpKSkrQfZ8eiepK2UpCCCFyisRBCCFECImDEEKIEBIHIYQQISQOQgiRJTfc\ncAO333573r9nwoQJnH/++Xn/niASByGEyIK6ujr++te/ctVVVwEwZcoUTjrpJLp3707Pnj0599xz\n+eSTT9I+18iRI+nTpw9du3ZlyJAhTJs27fP9Z5xxBnPmzGH27Nl5+S1RSByEECILHn74YU477TTa\nt28PwPr167nqqqtYsmQJS5Ysoaqqim9/+9tpnWvTpk0cffTRvPvuu6xbt45LL72U0047jc2bN39+\nzMiRI7nvvvvy8lui0DgHIUSrpLWPcxg2bBiXXXYZF1xwQeT+d999l5qaGjZs2JDV+bt06UJtbS2H\nH344AJMnT+aiiy7io48+Ch2rcQ5CCNFKmD17Nvvtt1/C/W+88QYHHXRQVueeMWMGO3bsYO+99/58\n2/7778/ixYvZtGlTVufMFE2fIYRos5TkyPeRjYGyfv16qqqqIvfNmjWLX//61zz33HMZn3fDhg1c\nfPHFjB49Oub87vX69euprKzMvMAZInEQQrRZWtLrVF1dzcaNG0PbFy5cyIgRIxg7dizHHXdcRufc\nunUrZ5xxBsceeyyjRo2K2ee+q2vXrtkXOgPkVhJCiCw45JBDWLBgQcy2JUuWcNJJJ3HjjTdy4YUX\nZnS+7du3c/bZZ9O/f3/uvffe0P558+YxcODAZrEaQOIghBBZMWLECCZNmvT5++XLl/OVr3yFq6++\nmiuuuCJ0/MMPP8ygQYMiz1VfX88555xDp06dePjhhyOPmTRpEiNGjMhJ2dNB4iCEEFlwySWXMHHi\nRLZt2wbAAw88wKJFiz6PFVRVVdG5c+fPj1+6dClDhgyJPNfkyZN54YUXeOWVV+jatevnn3/rrbc+\nP2bcuHFceeWV+f1RAZTKKoRolbT2VFaAX/ziF/Ts2ZNrrrkm5bEnn3wyY8eOTZrhlIgJEybw6KOP\nMm7cuMj9+UhllTgIIVolbUEcWgsa5yCEEKJZkDgIIYQIIXEQQggRQuIghBAihMRBCCFECE2fIYRo\nlVRXV7tsG5GC6urqnJ8z3zV/CvBHoBR4ALgl4pixwKnAFuBbwPSIY5TKKoQQGdJaU1lLgTsxgTgA\nGAkMjjtmBLA3sA9wBXB3HstTENTW1rZ0EVoNqgsf1YWP6iI35FMcjgIWAouBemAccFbcMWcC/9v4\neirQFeiVxzK1eXTh+6gufFQXPqqL3JBPcegDLA28X9a4LdUxffNYJiGEEGmQT3FIN0gQ7w9TcEEI\nIVqYfAakjwFGYzEHgBuAXcQGpe8BajGXE8B84ERgVdy5FgJ75amcQghRqHyIxXVbFWVYwQYCFcAM\nogPSExtfHwNMaa7CCSGEaDlOBRZgPf8bGrdd2fhw3Nm4fyZwRLOWTgghhBBCCFEYnILFIT4ARqU4\nthB4CIu3zA5s6wa8ArwP/B1L93XcgNXNfGB4M5WxuegHvA7MAd4DftS4vRjrowOW6j0DmAv8d+P2\nYqwLRyk2YHZC4/tirYvFwCysLqY1biv4uijF3E0DgXKiYxaFxvHA4cSKw/8AP2t8PQr4XePrA7A6\nKcfqaCGFNVdWb+CwxteVmHtyMMVbH50an8uw2NwQircuAH4MPAo81/i+WOtiESYGQQq+Lr4MvBR4\nf33jo9AZSKw4zMcfGNi78T1YDyBoTb2EBfULlWeAr6L66AT8CziQ4q2LvsA/gKH4lkOx1sUioHvc\ntpzURWtWjXQG0RUDvfBTe1fh/+l7YHXiKOT6GYhZVFMp3vpoh/X6VuG724q1Lv4AXIelxjuKtS48\nTCjfAS5v3JaTumjNs7JqMFwYj+T1Uoh1VgmMB64BNsbtK6b62IW52boAL2O95iDFUhenA6sxH3tN\ngmOKpS4AjgNWAj2wOMP8uP1Z10VrthyWY0FJRz9iVa9YWIWZhgBfwG4MCNdP38ZthUQ5Jgx/xdxK\nUNz1AfAZ8ALwRYqzLo7F5mRbBDwOfAW7PoqxLsCEAaAOeBqb067g6yKdQXSFyEDCAWnnJ7yecHCp\nAhiE1VUhTX5fAvwFcyEEKcb62B0/46Qj8AYwjOKsiyAn4sccirEuOgFVja93A97CMpCKoi6iBtEV\nMo8DK4AdWLzl21gmwj+ITkv7OVY384GTm7Wk+WcI5kqZgbkQpmOpzcVYHwcD72J1MQvzt0Nx1kWQ\nE/GzlYqxLgZh18QMLN3btZHFWBdCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgjRltjU+DwAGJnj\nc/887v1bOT6/EEKIPOHmZKrBH1GbLqnmH4uf70kIIUQbwTXgU4D12Gjra7C5xW7FFkmZCVzReFwN\n8CbwLP5EZs9gM1++hz/75e+Ahsbz/bVxm7NSShrPPRsb1Xxu4Ny1wJPAPOCRQDl/h822OrPxs0II\nIfKIE4fgXDxgYvCLxtftsXUSBmIN+CbMDeWobnzuiDX47n285eDefwObuqAE6AkswSZDq8EEao/G\nfZOxmTW7EzujZud0f5wQ+aA1z8oqRK6Jn2RsOHAJ1vOfgs1Js3fjvmlYg+64BpvD5m1sZst9UnzX\nEOAxbErk1cAk4MjG99OwObS8xnMOwARjG/Ag8DVga6Y/TohcInEQxc7V2EJChwN7YROWAWwOHFOD\nzYJ6DLamwnRsXedkeITFyM2dvz2wbSc2NflObLrlp7A1C15CiBZE4iCKiY34UxyDLZrzffyg8774\nazUH6Qysw3r2+xO7tGI90UHrN4HzsHusB3ACZjEkmiJ5N2z2zBex9ZEPTflrhMgjrXklOCFyheux\nz8R66DOAPwNjsRjDu1ijvRpz6cSvnvUScBUwF5tC/u3AvvuwgPO/gYsDn3saWwd9ZuO26xrPP5jw\n6lseJlrPYhZJCfAfWf9aIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQohC5v8BJFxH\nUX/IRW0AAAAASUVORK5CYII=\n",
"<matplotlib.figure.Figure at 0x7f02fc08aba8>"
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"agent = PassiveTDAgent(policy, Fig[17,1], alpha=lambda n: 60./(59+n))\n",
"graph_utility_estimates(agent, Fig[17,1], 500, [(2,2)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is also possible to plot multiple states on the same plot."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEPCAYAAACp/QjLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdYVMf+BvAXFVCKglJUbNg1XokxUWP5iTFGYzSmeqPG\nGE1ii/cmmpuY5OYGEmMDe43dxMQSGxYM2MDeC6KAWLBhQwFFpO/398dZFpazwFIWkH0/z8PD7jmz\ns8MA++7MnD0HICIiIiIiIiIiIiIiIiIiIiIiIiIiKpeWA7gHIDSX/YMAhAA4B+AQgNYl1C4iIipF\nXQC0Qe7h8DKAatrbvQAcLYlGERFR6WuA3MMhO0cAt0zbFCIiMkaF0m5ANp8A2FHajSAiopLRAPmP\nHLoBCIMyeiAiolJWqbQbAGURegmUNYc4QwU8PDwkJCSkRBtFRFQOhAB4vjAPLO1ppXoANgH4EMDl\n3AqFhIRARPglAi8vr1JvQ1n5Yl+wL9gXeX8B8Cjsi7OpRw5rAHQF4ATgJgAvAJbafYsA/AhlKmmh\ndlsagHYmbhMREeXD1OEwIJ/9n2q/iIioDCntaSUqIE9Pz9JuQpnBvsjCvsjCvigeFqXdACOJdv6M\niIiMZGFhARTydZ4jByIiUmE4EBGRCsOBiIhUGA5ERKTCcCAiIhWGAxERqTAciIhIheFAREQqDAci\nIlJhOBARkQrDgYiIVBgORESkwnAgIiIVhgMREakwHIiISIXhQEREKgwHIiJSYTgQEZEKw4GIiFQY\nDkREpMJwICIiFYYDERGpMByIiEiF4UBERCoMByIiUmE4EBGRiqnDYTmAewBC8ygzB8AlACEA2pi4\nPUREZARTh8MKAL3y2N8bQGMATQAMB7DQxO0hIiIjmDocDgCIy2P/mwB+094+BsABgKuJ20RERPko\n7TUHNwA3s92/BaBOKbWFiIi0SjscAMAix30plVYQEZFOpVJ+/mgAdbPdr6PdpmJh4Z3tnqf2i4iI\nsgRrv4ou57t2U2gAYBuAfxjY1xvAGO33DgBmab/nJCIcUBARFYSFhQVQyNd5U48c1gDoCsAJytqC\nFwBL7b5FAHZACYbLABIBDDVxe4iIyAglMXIoDhw5EBEVUFFGDmVhQZqIiMoYhgMREakwHIiISIXh\nQEREKgwHIiJSYTgQEZEKw4GIiFQYDkREpMJwICIiFYYDERGpMByIiEiF4UBERCoMByIiUmE4EBGR\nCsOBiIhUGA5ERKTCcCAiIhWGAxERqTAciIhIheFAREQqDAciIlJhOBARkQrDgYiIVBgORESkwnAg\nIiIVswyHE9EnkKHJKNU2PE17ivuJ90u1DUREuTGLcAiPCUeGJgP3ntzDW2vfQrul7XDqzqlSa8+2\ni9tgO8kWvf/sXWptKIzHKY9Luwnl2v3E+/CL8CvtZhABeIbD4UT0CYTHhOdZRkQw6cAkPLfgOfzr\n73+h7eK2eM75ObR3a4+ktKQSaqnSDgBISEnAwI0DMW7nOPi86lNiz19UMYkxeGfdO6gzow6iH0eb\n9LlORJ9AfHK8UWWfpj01aVtKSkJKAr7f8z1azG+B/uv7QyOaPMuLCNadX4cW81tgz9U9JdTKkiMi\neiP7lPQUrDizAvee3CvFVj077j25h8DLgUWu55kMB78IP7zy+ysY4jdE98KbSUSwIWwD4pPjMXDT\nQGy5uAWb/rkJWy9uxZK+SzCx+0RUta6KlIyUEmnrjUc38MLiF+AV5IWOyzvCzsoOISND0P+5/oWe\nVtp2cRvO3TtXzC01bPfV3Xh+0fNoUr0JhrUZhhlHZhRr/akZqfgq8CsM8RuC/+z8D7qs6IJhW4bl\n+Zh0TTq+3/M9bCfZ4knqk3yfQyMa1agnKCoINafVRFxSXJHaX1Sbwjeh5YKWuJ1wG2dGnIFjFcc8\n/y7CYsLQaXkn+Bz2QW372rgQc6EEW6uWkJKAkLshRpcXEfhH+uN49HGD+3dd2YXWv7bGaP/REBGs\nClmFZvOaYcbRGXhv/XvF1exicz/xvt6blLSMNCw8sRA/Bf9U5LojH0ZiTeiaArXlq8Cv0GRuE/T6\nsxdSM1KL3AZT6gUgAsAlAOMN7HcCEADgLIDzAD7OpR7J5B/pLy6+LnIi+oQ0mNVAzt87L9l5B3lL\npZ8rSa1ptaTv6r6SlJYkIiIajUZXps/qPrIlYouY2sUHF6XOjDoyLmCcVPmlisw5OkfXjqepT8V6\ngrVeu4wx68gsqTG1hjSf11zSMtJM0WydRScXSc1pNWX3ld0iIhJ4OVA8V3oWqc4L9y/ofif3n9yX\nriu6Sp/VfeSFRS/IW2vfkluPbomTj5PciL9h8PF3Eu5I1xVd5bVVr4nNRBu5Gns1z+eLS4qTNr+2\nkWqTq4mISIYmQ37Z94vUmlZL4A05c+dMkX6ewkpISZAhm4dIs7nNZN+1fbrtbX5tIyeiT6jKZ2gy\nZOrBqeLk4yQLji+QDE2GTD4wWb7e+bVeGa8gL/k5+OcS+Rl2Xt4p9WfWF7fpbkaVv5twV95Z947A\nG9J+SXt5mvpUt+/+k/vy/l/vS6PZjWTxycXiOMVRuq7oKm0XtZUD1w9IhiZD3Ka7SXhMeJ7PEZMY\nI/OOzZPU9NQi/Wz5SU1PlUn7Jwm8IT/s+UFElP5oOrepvLT4JXGf5V7oumOfxsrn/p9Ljak1xHai\nbb7/50lpSeId5C2OUxxljP8YiX4cLS3mtZCzd84KAMnlNbVUVQRwGUADAJZQAqBFjjLeACZrbzsB\neAigkoG6RETk9O3T4uzjLIdvHBYRkd5/9patEVt1nbTs9DJxn+UuZ++clW93fSvJackGO/O9v96T\nv87/VehfnjGi4qKk7oy6svz0chERg22xn2Qv8UnxRtc568gscZ/lLtfironbdDe5Hn+92Nqb00/B\nP0mj2Y3k0sNLum13E+6K4xRHXaBlaDIK9E/41/m/BN6QX0/8Kldir4j7LHf5dte3kp6RLhmaDF25\n9/96Xz7Z8onEJMboPT7yQaTUn1lfvIK8JD0jXdouaivHbx3XK5OclizpGekiovwOWs5vKZ9u+VSs\nJlhJXFKcvLPuHem0rJPcenRLXv39VQm8HFjgvimqSw8vSfN5zeVjv4/lScoTvX19V/eVzeGbdffX\nnV8nv+z7Rfqt6ScdlnaQa3HXdPtWhaySARsGiIjIo+RH8tbat8R6grUuCDNdfHBRHic/Lrb2p2Wk\nyVeBX0ndGXXl70t/i5OPk0TERKh+F9n5R/qLq6+rfLf7O4lPihd4Q+ANSc9Il20Xt0nNaTXlm53f\n6N44jPEfI3OOztH9LkVERm8fLVMOTMn1OTLrabWglfRb009S0lN0+24/vq37Hzx265je60b2n8uY\n/8fjt45L64WtpdcfvWTFmRXSemFr+WTLJ1JvZj3ZdnGbJKcli9UEK722G8sv3E9qT68to7aPkpjE\nGGk6t6mcu3su1/I7L++UxnMay9tr39b72xi4caCsOLOizIbDy1BGBZm+1X5lNwLAfO3thgAic6lL\n4pPipdHsRrI2dK2uA0ZvHy1zjs4REZGT0Sd1f6T5GbRxkKwKWVXgX5yxHiU/kufmPyezj87Os1yj\n2Y0k8kFkvvWlpqfKijMrdMEgItJqQSsJuRtSLO3NacbhGdJ8XnO5m3BXtc/F10WO3Dwi5++dlzfX\nvCntl7TX/UPnZVPYJnH1dZUv/v5Cuq3sJvVn1pf5x+cbLDvv2DyBN+R/e/+n2xZyN0RqT68tS08t\n1W3ruaqn7Ijcobt/89FNgTdk4v6JEhETIW7T3XS/g7aL2kqzuc3k/b/e171oDNw40KR/B4aciD4h\ntabVkoUnFhrcP3LbSJl3bJ6IiPgc9JH6M+uLx0IPGbV9lN6LnYhIUFSQdFneRa7HX5cW81rIiG0j\nJD4pXir/UlkSUhJkmN8wGbV9lFhPsNb9nxRVTGKMdF3RVXqu6ikPEh+IiDISd5jiIA5THCQ1PVU0\nGo2sCV0jSWlJutGM23Q3OXj9oK6e07dPS+M5jaX/+v5SZ0YdvX252RKxRV5b9ZretssPL8vn/p/r\nwurA9QOSkp4iXZZ3kd/P/i4iIqvPrRa7SXbSeXlnmXxgsjj5OImrr6tef956dEs6LusoPVf1zPX5\nNRqN+B7yFRdfF/kj5A/RaDSSkp4iTj5OMmr7KHmU/EhXtta0WrmOfg15nPxYBm4cKE3mNJH91/br\ntg/aOEiWnV6mVzY+KV6epj6V0dtH6wIpJ99DvvK5/+dlNhzeA7Ak2/0PAczNUaYCgGAAtwEkAHg9\nl7pk4MaBMnr7aFUHjA0YKwkpCdJ4TmO94MjLJ1s+kSWnlhj9iysIjUYj76x7R4ZvHZ7vlFGHpR3k\ny7+/zLOcz0EfcZvuJs4+zhJ2P0y3vfPyznp/RMXR7s+2fiYdl3WUOjPq6L0LyW7U9lFS8aeKYvmz\npfT6o5e0W9JOAi4F5Fl34OVAcfF1kVO3T0lUXJTAGzL32Nxcy6dnpMuGCxsE3pDPtn4mV2OvSq1p\ntWRN6Bq9coM2DtK9ADx8+lBazm8pXZZ3kdYLW+uN2kREvIK8ZPT20Xrv5r78+0uZfnh6vn2Tl/ik\neIlLitPbdvr2aYPvGg9ePyjOPs7iF+6Xa33f7f5O4A3ps7qPtJjXQm4+upnr38elh5fEycdJ6s+s\nr/dzNJvbTFovbC19V/eVPqv7yKdbPlX974goL0jZ363nJTE1UW7E35Dm85rLNzu/0fv5tkRskUUn\nF0mj2Y0E3pBef/QSC28L+SPkDxm4caB0WtZJ7iTcUdXpe8hXXvntFYNvQgyJSYwR+0n2uv+D/df2\ni6uvq7yz7h3pt6af3khzztE58smWT+SHPT9Iw9kN5dTtUzJ+13jpsryLRMVFiedKT1l3fp2IKL+X\nWtNqydiAseLk46TX31djr8rj5MfyJOWJvL32bWm3pJ1qxG5o9NxhaQfZFLZJuq3sJotOLsrz57pw\n/4I0m9tMhm8dLompiXr75h2bJ0P9hurub7iwQWwn2kr1qdWl//r+qr+9TOfunpMGsxqU2XB4F/mH\nww8AZmlvNwJwFYC9gbrEsZejfP/D9+Ll5SVBQUEiIrL+wnp5e+3b8lXgVzJ40+A8fwHZjd4+Wvfu\nrLgtP71cPBZ65DqllV31qdUF3pDbj2+r9v157k9pvbC1uPi6yMT9E3Xz/pne+PMNg0Pjgvhm5ze6\ndyULji8Qj4UeMsZ/TJ7D2NT0VNl7da+ciD4hiamJMtRvqCw+uTjX8ldir4iLr4tekEXFReXbttT0\nVHlzzZtiP8lems9rbvCd7xd/fyEzDs+Qp6lPpdOyTvJV4FeSmJoothNtjfr9Tto/ScbvGp9vudzE\nPo0VeEPeXfeubptfuJ9U+KmCKrjP3jkrLr4u+QbpwesHZajfUBm+dbhqWi2np6lPpcovVVT9P9Rv\nqHy65VPdXHXg5UDptrKbbn9qeqokpCRI5+WdxWqCVb5t2nN1j1T5pYpUn1o9zzBdf2G9jN81XsYG\njJVZR2aJ9QRr6bO6j97aQlFlTkctObVEnH2cZeflnQbLnb1zVir8VEHaL2kv957cU+1fE7pGuq3s\nJn7hfuLk4yQ7IneIRqORGlNr6KZSd1/ZLXaT7GTwpsHy0uKXZKjfUNXoLTcfbPhArCZYyYuLX5SP\n/T7OtdzuK7vF2cdZVpxZYXB/5gj4gw0fSKsFraTezHpy6MYhOXzjsME3DUFBQeLl5SU//vijOPRy\nKLPh0AH600rfQb0ovQNAp2z39wB40UBdBv+Az909J04+TuLs4yz3n9w36pcmIjIuYJxMOzTN6PLG\nylxMDb0XalT5c3fPifssdzlw/YDe9puPboqzj7OMDRirCoVMRZ0a23dtn8Ab0nVFVwm5GyJOPk5y\n8cHFAtfjFeQl/9v7P4lPipeZR2aKiDIKSc9Il8TURPFY6FGkKY13170r4wLGGdw3Yd8E+X739zLM\nb5j8c/0/desW2Yf3eVl6aqm8u+5deWvtWxL7NLZA7UrPSJdef/SS5+Y/J12WdxERZWrT2cdZWi1o\npbemdSP+htSeXlv3TrU4JaQkqLblfNG4Hn9dak2rJSLKSMdjoYdUnVxVhvoNlQn7Jsi/d/xbRMTg\nwueRm0fEycdJNodvlqCoIKPbFZcUJ5MPTC72heGImAj52O9jcZjiIEduHsm1XIYmQ+Yem6t6J54p\nOS1Znv/1eXH2cdZbK8k8UGH64eni7OMsq8+tFsufLeWHPT8U6OCRrRFbZU3oGjl4/aC0W9JOLj64\nKB4LPeTmo5u6MuvOrxMXXxcJjgrOtR6NRiP1Z9aXDzZ8IOsvrDcYdLk5evNomQ2HSgCuQFmQtoLh\nBekZALy0t10B3AJQ3UBduXbA/OPzZf2F9UZ3mIgydJ+4f2KBHmOMARsG6I5cMNbAjQNlxuEZ0nVF\nV907rH5r+olXkFeejxu9fXSeUzO5SUlPkT1X90jjOY1l4v6J0nB2Q2nzaxu9KZiCWHZ6mQzZPEQ+\n9vtY4A3xPeQrgzcNlqF+Q2X09tEycOPAAh+RlV32heqcFp5YKPVn1pemc5sWasF1a8RW3TvRvBZT\nDRm/a7x0W9lN7iTckWqTq8mDxAfSYFYD2XBhg97vJjktWdotaSdTD04tcPuKi0ajEYcpDhL9OFr6\nrO4jI7eNlIBLAZKekS6h90Kl7oy64h3kLa/+/qre46LioqTWtFoG57RLU3xSfLEcjJGanioPnz7U\n23boxiEZtX2UVJtcTY7ePCoiku8ILi+xT2PFZqKNuE13kzoz6ujeNKwNXSu1ptUyat0wJjEmz/+D\nvKAI4WDoyKDikg5gDIBAKEcuLQMQDmURGgAWAZgEYAWAECjrD98AiC3Ik4x+aXSBG2Zd0RrJ6ckF\nflxeDlw/gEM3D2Hpm0sL9LhGjo3gFeyFhNQEXIu/huiEaITeD8Xa99bm+TiHyg54lPyowO30OeSD\nn/f9jP7P9cc3nb7Bz/t+RqP6jfDx8x8XuC4AqFu1LnZc2gEbSxsM+scgTDwwEU1rNMX5++dRvUp1\nhI4KhYWFRaHqBoAKFrl/FMfJxgl3n9zF0U+Pwt7a0Gxk3lo6t8Sbzd5EUloSohOi8RJeMupx/pH+\nWHt+LU4OPwknGyfYWtni9T9fxzvN38G7Ld/FhZgLug9sfbXzK9S2r42vO35d4PYVFwsLC7zW6DX0\nWd0HdlZ22NR/EywrWgIAWrm0gkNlByw8uRApGSkQEaRp0jA2YCyCrgVhfKfx6NO0T6m13ZBqlauh\nWuVqRa7HsqIlqlfRfy/asW5HtHNrB6+uXnC1cwWg/J0VlmMVRzR0bIixHcbidsJtrDi7Ajce3YDv\nYV/sHLwTrV1b51tHUZ7fHBQqNXMz5cAU+WbnN8VaZ9cVXWXlmZUFftzKMyul2uRq0m5JO9kSsUVa\nLWglm8I25fu4qQenyn8C/2NwvUJEJDwmXCbsm6C3LfpxtFSfWl2uxF7RLSh6B3nn+1mBvETERAi8\nIf6R/nI19qruMOPhW4fnO5ddVAkpCbrnK4oR20bI/OPzJSouSmYdmaXbfvHBRdFoNHIn4Y4sOL5A\nRJR3gm7T3WTv1b26clsjtsrIbSN1Uyi/nvhVPt3yqWy/uF0azm5YoMOVTeXPc39KnRl1DC7+7r+2\nXy7cvyAuvi4S+SBS/r3j3+K50lNmHplZpFEfKTL78O9Lf+sW7As6Ui0sFGHkUPi3dCVL+3MWj1lH\nZ+Fa/DXM6jUr/8JGCL4WjM+2fYbwz8NRqULBBmMJKQmIeBCB30J+Q8i9EKRr0nF42OF8320vPrUY\nWy9uxZ6oPYj6Igo17Wrq7e+yogsO3jgI8crqt+HbhsOhsgN8ehTfqTs0osGuK7vQs3HPYquzpP28\n72ekZqTi7N2zOHLrCB58/QCXYi+h1YJWCP44GPOOz8PG8I0IHRWKCfsnoKpVVcx/Y36u9flF+GH2\nsdm4EnsFv731G7q5dyvBn8YwEUFiWiLsrOxyLVNrei3cfXIXjas3xvFPj8OximMJtrD804gGl2Mv\no2mNpiX2nNrXkUK9zptyWqnMqlypcrFOK/2y/xf80OWHAgcDANhb2+Mlt5dw4MYBzD8xH/4D/Y2a\nhqlmXQ3+l/wBAI+SH+mFw4X7FxD5MBJWFa0QkxgDZ1tnRD6MxOaIzbg45mKB25iXChYVnulgAIDa\n9rUx6cAkWFa0RAWLCohOiMaXAV/CobIDphycgnP3zqGVSyu0WtAKHjU9EDQkKM/6atrVRPC1YAx7\nfliZCAZAeZHIKxgA4OuOX8O6ojVGvjgSFStULKGWmY8KFhVKNBiKyizDwbqite7cSompiUjTpMGh\nskOh6gqLCcOFmAsY8I8BRWqTh6sHXq7zMl5vnNtHPfQ5VHaAdUVrONs6IzEtUW/f5IOT8WX7L7H9\n0naExYShq21X+Bzywecvfa6aYyUlHKLioxD4YSCmHpqKyQcm42rcVUx9dSqGbR2Gzf/cjNsJt5GQ\nkoDxnQ2dBUafu4M7Wrm0gu9rviXQ+uIz7uVxpd0EKkPMMxwqZS1I/7D3B1SsUBHTXptWqLrmH5+P\n4S8Mh1VFqyK1qXvD7vBs4Gn04m37Ou3x1/t/weeQj97J56LiohBwOQDze89HVHwUwmLC0KRGE2wK\n34TIf+X2AXTz1s6tHXx7+OK1Rq9hz9U98D3si78H/Y22tdtiQrcJ6NesX4EW1V3tXBE6KtSELSYy\nvWfyrKxFVblSZaSkpyAtIw2rz68u9HUKnqQ+wZrzazDixRH5FzZCQYbyDpUd8GazN2FnZYfE1KyR\nw68nf8UQjyGoVrkaWjq3RFhMGOYem4vBrQfzqIdcONk44T8d/wMAmNh9Ih588wA9G/eEk40Tfvi/\nH4p0tBXRs8o8Rw7aaaWdV3aqTrlbEJvDN6NTvU6obV+7mFtoPFsrW920UnJ6MlacXYFDww4BUA7V\n3Bi+EZceXsL+oftLrY3PkkoVKnHqjQhmPHJITk/G7+d+R4+GPVThMOPIDKMuavNH6B/48B8fmqqZ\nRrG1zLqmwYawDfCo6YEmNZoAUMLh4I2DaOHc4plaCCOi0meW4WBdyRqPkh8h4HIAPvL4SC8cElMT\n8d+9/0VYTFieddxJuIPj0cfRt1lfUzc3T9mnlVaeXYnPXvhMt8/N3g32VvYY/sLw0moeET2jzDMc\nKlrj9J3T8HD1QL1q9fTCIfBKIJLTk/OdatoQtgF9m/aFjaWNqZubp8yRw52EOzh5+yT6Ns0KKwsL\nC6x/fz3ea1n2rqBFRGWbWYZD5UqVIRD0a9YPNpY2ekHgF+GHihYV8w2HrZFb8Xbzt03d1HzZWdkh\nMS0R6y6sQ7/m/VDFsore/p6Ne+pOlUBEZCyzDAfrStYAgH7NlXDIXNBNy0iD/yV/dG/YPc9weJT8\nCMduHUOPRj1KpL15sbWyRWJqIlaHrsbAVgNLuzlEVE6YZTjUtKuJ4S8MR+PqjfVGDseij6F+tfpo\nWr2pKhzSNel4c82biHwYicArgehcr3O+nzgtCbaWtrj48CIux17GK+6vlHZziKicMMtwqGpdFYv6\nLgKgvLhmBsGeq3vQo2EP1VQTACw9vRTbIrfhyM0j2B65XW9uvzTZWdlh55WdeL3J65w+IqJiY5bh\nkF32INgdtRvdG3bXm2rKFHglELXta+Ne4j3suroLvRr3Ko3mqtha2SJNk1ZmwoqIygezD4cqllWQ\nlJaEhJQEnLlzBp3rdVaNHDI0Gdh3bR8G/WMQ9kTtgXVFa7g7updiq7PYWtqiUoVKZSasiKh8MPtw\nqGBRAdaVrLHzyk60c2sHG0sb2FrZ6oVDyL0Q1LSriba12mL31d1lam6/UfVGGNF2RKFPHEhEZIjZ\nhwOgTC1NPTRV93mAnCOHvVF78Yr7K3Cr6gaNaNCtQdk4DTMANK7eGPN6zyvtZhBROWOW51bKKTYp\nFrFJsTg47CAAdTgEXQvCsOeHwc3eDQDKzDn6iYhMhSMHrTY12+hOu509HEQER28dRad6nVC3Wl3M\neG0G6lStU5pNJSIyOY4ctDxqeuhuZw+HS7GXYG9lr7vS2tiXx5ZK+4iIShJHDgA61+uMEW2zrslg\na5l1Guxjt46hfZ32pdU0IqJSwZEDgANDD+jdzz5yOHrrKNq7MRyIyLxw5GBAzlNqdKjToZRbRERU\nshgOBthY2iAxNRFJaUkIiwlDm5ptSrtJREQliuFgQA2bGohNikXo/VA0rdFUdRpsIqLyjuFggFVF\nK9hZ2SH4WrDeUUxEROaC4ZALF1sX7Lq6C61dWpd2U4iIShzDIReudq44cP0ARw5EZJZMHQ69AEQA\nuARgfC5lPAGcAXAeQLCJ22M0V1tXpGSkwMOV4UBE5seUn3OoCGAegFcBRAM4AWArgPBsZRwAzAfQ\nE8AtAE4mbE+BuNi6oKZdTTjbOpd2U4iISlxe4fBVjvsCIAbAQQBRRtTdDsBlANe099cC6Af9cBgI\nYCOUYACAB0bUWyJcbV05aiAis5XXtJI9ALtsX/YAXgIQAGCAEXW7AbiZ7f4t7bbsmgCoDiAIwEkA\ng41qdQl4teGrGNZmWGk3g4ioVOQ1cvDOZXt1AHsArMmnbjHi+S0BvACgOwAbAEcAHIWyRqHfGO+s\n5nh6esLT09OI6gvv5bov42W8bNLnICIqTsHBwQgODi6WuiwK+bgzAPL72HAHKAGTef3K7wBoAEzN\nVmY8gCrICqKlUEYmG3LUJSLGZA0REWWysLAACvk6X5ijlboBiDOi3Eko00YNAFgB+CeUBenstgDo\nDGXx2gZAewBhhWgTEREVo7ymlUINbHMEcAfAR0bUnQ5gDIBAKC/+y6AsRmeeG3sRlMNcAwCcgzKq\nWAKGAxFRqctruNEgx30B8BDAE5O1JnecViIiKqCiTCsVds2hpDEciIgKqKTXHIiIqJxjOBARkQrD\ngYiIVBgORESkwnAgIiIVhgMREakwHIiISIXhQEREKgwHIiJSYTgQEZEKw4GIiFQYDkREpMJwICIi\nFYYDERGpMByIiEiF4UBERCoMByIiUmE4EBGRCsOBiIhUGA5ERKTCcCAiIhWGAxERqTAciIhIheFA\nREQqDAci7p36AAANXUlEQVQiIlJhOBARkQrDgYiIVEwdDr0ARAC4BGB8HuVeApAO4B0Tt4eIiIxg\nynCoCGAelIBoCWAAgBa5lJsKIACAhQnbQ0RERjJlOLQDcBnANQBpANYC6Geg3L8AbAAQY8K2EBFR\nAZgyHNwA3Mx2/5Z2W84y/QAs1N4XE7aHiIiMVMmEdRvzQj8LwLfashbIY1rJ29tbd9vT0xOenp5F\nax0RUTkTHByM4ODgYqnLlHP8HQB4Q1lzAIDvAGigrC9kupqtDU4AngL4DMDWHHWJCAcVREQFYWFh\nARTydd6U4VAJwEUA3QHcBnAcyqJ0eC7lVwDYBmCTgX0MByKiAipKOJhyWikdwBgAgVCOSFoGJRhG\naPcvMuFzExFRETwrh45y5EBEVEBFGTnwE9JERKTCcCAiIhWGAxERqTAciIhIheFAREQqDAciIlJh\nOBARkQrDgYiIVBgORESkwnAgIiIVhgMREakwHIiISIXhQEREKgwHIiJSYTgQEZEKw4GIiFQYDkRE\npMJwICIiFYYDERGpMByIiEiF4UBERCoMByIiUqlU2g0gIjKkevXqiIuLK+1mPBMcHR0RGxtbrHVa\nFGttpiMiUtptIKISZGFhAf7fGye3vrKwsAAK+TrPaSUiIlJhOBARkQrDgYiIVBgORESkUhLh0AtA\nBIBLAMYb2D8IQAiAcwAOAWhdAm0iIiqy7777DrNnzzb582zbtg0ffPCByZ8nO1OHQ0UA86AEREsA\nAwC0yFHmKoD/gxIKEwAsNnGbiIiKLCYmBqtWrcLIkSMBAGFhYXjxxRdRvXp1ODg4oFOnTjh48KDR\ndQ0YMABubm5wcHBA586dcfz4cd3+vn374sKFCwgNDTXJz2KIqcOhHYDLAK4BSAOwFkC/HGWOAHik\nvX0MQB0Tt4mIqMhWrlyJN954A9bW1gAANzc3rF+/Hg8fPkRcXBw++OADvPfee0bV9eTJE7Rv3x6n\nT59GXFwchgwZgjfeeAOJiYm6MgMGDMDixSX33tnU4eAG4Ga2+7e023LzCYAdJm0REVExCAgIQNeu\nXXX3q1WrBnd3d1hYWCAjIwMVKlRArVq1jKrL3d0dX375JVxdXWFhYYHPPvsMqampiIyM1JXx9PSE\nv79/sf8cuTH1J6QL8gmWbgCGAehkorYQERWb0NBQNGvWTLXdwcEBiYmJqF27Nvbu3Vuous+ePYvU\n1FQ0btxYt6158+a4du0anjx5Ajs7u0K321imDodoAHWz3a8LZfSQU2sAS6CsTRj8vLy3t7futqen\nJzw9PYurjUT0jLIopnM8FOaD2PHx8bC3tze4/enTp/jpp5/w/vvv49SpU5mfVDbK48ePMXjwYHh7\ne+vVn3k7Pj4+13AIDg5GcHBwwX6QXJj69BmVAFwE0B3AbQDHoSxKh2crUw/AXgAfAjiaSz08fQaR\nmSnrp89wdXXFjh070LZtW4P7RQT29vY4fPgwWrc27iDMpKQk9OrVC82bN8eiRYv09sXGxsLJyQmP\nHz9WhcOzePqMdABjAAQCCAOwDkowjNB+AcCPABwBLARwBkqAEBGVaa1bt8bFixdz3Z+RkQGNRgMb\nGxuj6ktJScFbb72FevXqqYIBAMLDw9GgQYMSmVICSuZzDn8DaAagMYDJ2m2LtF8A8CmAGgDaaL/a\nlUCbiIiKpHfv3ti3b5/u/u7du3H27FlkZGTg8ePHGDduHJo1a6ZbN1i5ciXc3d0N1pWWlob33nsP\nNjY2WLlypcEy+/btQ+/evYv958gNPyFNRFQIH330EXbs2IHk5GQAylrAgAED4ODggGbNmiEmJgZb\nt27Vlb958yY6d+5ssK7Dhw/D398fu3btgoODA+zt7WFvb49Dhw7pyqxduxYjRoww+HhT4Cm7iahM\nKutrDgDw3//+Fy4uLvjiiy/yLduzZ0/MmTPH4BFO+dm2bRv+/PNPrF271uB+U6w5MByIqEx6FsKh\nrHgWF6SJiOgZxHAgIiIVhgMREakwHIiISIXhQEREKgwHIiJSYTgQEZEKw4GIqJB4mVAiItKT8zKh\nR48eRY8ePVCjRg24uLigf//+uHv3rtF1mdtlQomIyqWclwmNj4/HyJEjcf36dVy/fh329vYYOnSo\nUXWVxcuE8vQZRFQmlfXTZ3Tv3h2ffPIJBg4caHD/6dOn4enpicePHxeq/mrVqiE4OBht2rQBoJyc\n78MPP8TVq1dVZXn6DCKiMiK3y4Rm2r9/P1q1alWouvO7TGhJMPVlQomITMbip+KZ/BCvgo9QcrtM\nKACcO3cOEyZM0Dtlt7GKcpnQ4sRwIKJnVmFe1IuLo6MjEhISVNsvX76M3r17Y86cOejUqVOB6kxK\nSkLfvn3RsWNHjB8/Xm9f5nM5ODgUvtEFwGklIqJCMHSZ0OvXr6NHjx748ccfMWjQoALVZ46XCSUi\nKndyXiY0Ojoar7zyCsaMGYPhw4eryvMyoUREZiDnZUKXLl2KqKgo3VqBvb09qlatqivPy4SaBg9l\nJTIzZf1QVoCXCS0LGA5EZuZZCIeygp9zICKiEsFwICIiFYYDERGpMByIiEiF4UBERCo8fQYRlUmO\njo6ZR9tQPhwdHYu9TlP3fC8AswBUBLAUwFQDZeYAeB3AUwAfAzhjoAwPZSUiKqCyeihrRQDzoARE\nSwADALTIUaY3gMYAmgAYDmChCdtTLgQHB5d2E8oM9kUW9kUW9kXxMGU4tANwGcA1AGkA1gLol6PM\nmwB+094+BsABgKsJ2/TM4x9+FvZFFvZFFvZF8TBlOLgBuJnt/i3ttvzK1DFhm4iIyAimDAdjFwly\nzodxcYGIqJSZckG6AwBvKGsOAPAdAA30F6V/BRAMZcoJACIAdAVwL0ddlwE0MlE7iYjKqytQ1nXL\nlEpQGtYAgBWAszC8IL1De7sDgKMl1TgiIio9rwO4COWd/3fabSO0X5nmafeHAHihRFtHRERERETl\nQy8o6xCXAIzPp2x5sBzKektotm3VAewCEAlgJ5TDfTN9B6VvIgC8VkJtLCl1AQQBuADgPIB/a7eb\nY39UhnKo91kAYQAma7ebY19kqgjlA7PbtPfNtS+uATgHpS+Oa7eV+76oCGW6qQEASxhesyhvugBo\nA/1w8AHwjfb2eABTtLdbQukTSyh9dBnl61xZNQE8r71tB2V6sgXMtz9stN8rQVmb6wzz7QsAGAfg\nTwBbtffNtS+ioIRBduW+L14GEJDt/rfar/KuAfTDIQJZHwysqb0PKO8Aso+mAqAs6pdXfgBeBfvD\nBsAJAM/BfPuiDoDdALoha+Rgrn0RBaBGjm3F0hdlOTWM+RCdOXBF1qG995D1S68NpU8ylef+aQBl\nRHUM5tsfFaC867uHrOk2c+2LmQC+hnJofCZz7QuBEpQnAXym3VYsfVGWz8rKD8OpCfLul/LYZ3YA\nNgL4AkBCjn3m1B8aKNNs1QAEQnnXnJ259EUfAPehzLF75lLGXPoCADoBuAPAGco6Q0SO/YXui7I8\ncoiGsiiZqS70U89c3IMyNASAWlD+MQB1/9TRbitPLKEEwyoo00qAefcHADwC4A+gLcyzLzpCOSdb\nFIA1AF6B8vdhjn0BKMEAADEANkM5p1257wtjPkRXHjWAekE6c57wW6gXl6wAuEPpq/J08nsLAL9D\nmULIzhz7wwlZR5xUAbAfQHeYZ19k1xVZaw7m2Bc2AOy1t20BHIJyBJJZ9IWhD9GVZ2sA3AaQCmW9\nZSiUIxF2w/Bhad9D6ZsIAD1LtKWm1xnKVMpZKFMIZ6Ac2myO/fEPAKeh9MU5KPPtgHn2RXZdkXW0\nkjn2hTuUv4mzUA73znyNNMe+ICIiIiIiIiIiIiIiIiIiIiIiIiIiInqWPNF+rw9gQDHX/X2O+4eK\nuX4iIjKRzHMyeSLrE7XGyu/8YznP90RERM+IzBfwowDioXza+gso5xbzhXKRlBAAw7XlPAEcALAF\nWScy84Ny5svzyDr75RQA6dr6Vmm3ZY5SLLR1h0L5VHP/bHUHA1gPIBzAH9naOQXK2VZDtI8lIiIT\nygyH7OfiAZQw+K/2tjWU6yQ0gPIC/gTKNFQmR+33KlBe8DPv5xw5ZN5/F8qpCywAuAC4DuVkaJ5Q\nAqq2dt9hKGfWrAH9M2pWNfaHIzKFsnxWVqLilvMkY68B+AjKO/+jUM5J01i77ziUF/RMX0A5h80R\nKGe2bJLPc3UGsBrKKZHvA9gH4CXt/eNQzqEl2jrrQwmMZADLALwNIKmgPxxRcWI4kLkbA+VCQm0A\nNIJywjIASMxWxhPKWVA7QLmmwhko13XOi0AdRpnnzk/Jti0DyqnJM6CcbnkDlGsWBICoFDEcyJwk\nIOsUx4By0ZzRyFp0boqsazVnVxVAHJR39s2hf2nFNBhetD4A4J9Q/secAfwflBFDbqdItoVy9sy/\noVwf2SPfn4bIhMryleCIikvmO/YQKO/QzwJYAWAOlDWG01BetO9DmdLJefWsAAAjAYRBOYX8kWz7\nFkNZcD4FYHC2x22Gch30EO22r7X1t4D66lsCJbS2QBmRWAAYW+ifloiIiIiIiIiIiIiIiIiIiIiI\niIiIiIiIiIiIiKg8+3+ftEQRU4HjfQAAAABJRU5ErkJggg==\n",
"<matplotlib.figure.Figure at 0x7f02d0df94a8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"graph_utility_estimates(agent, Fig[17,1], 500, [(2,2), (3,2)])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",