text: The issue is that in your list comprehension in def V_pi(state) you have
  
  return sum(prob * (reward + mdp.discount*V[newState]) for prob, reward, newState in
          mdp.succProbReward(state))
  
  
  whereas with the way you have defined the succProbReward output, it should be 
  
  return sum(prob * (reward + mdp.discount*V[newState]) for newState, prob, reward in
          mdp.succProbReward(state))
  
  
  When I run this it converges immediately with a reward of 0 for all states, which I believe is correct for the policy you specified. If I change the policy it also seems to give reasonable results.