Blog

Suppose there is a blocksworld domain that contains some blo…

Suppose there is a blocksworld domain that contains some blocks and a table. A block can be on top of the table or on the other block. On relation specifies which block is on top of what. Move action moves a block from one location to another. In_Gripper relationship specifies that the block is in the gripper. Consider these states: Current State: block(b1), block(b2), block(b3), block(b4), On(b1,b2), On(b2,table), On(b3,table), On(b4,table) Goal State: On(b2,table), On(b1,b2), On(b3,b4) In order for a state to be a landmark, which proposition must be contained in the state?

Table: Gridworld MDP Table: Gridworld MDP   Figure: Transit…

Table: Gridworld MDP Table: Gridworld MDP   Figure: Transition Function Figure: Transition Function   Review Table: Gridworld MDP and Figure: Transition Function. The gridworld MDP operates like the one discussed in lecture. The states are grid squares, identified by their column (A, B, or C) and row (1 or 2) values, as presented in the table. The agent always starts in state (A,1), marked with the letter S. There are two terminal goal states: (B,1) with reward -5, and (B,2) with reward +5. Rewards are 0 in non-terminal states. (The reward for a state is received before the agent applies the next action.) The transition function in Figure: Transition Function is such that the intended agent movement (Up, Down, Left, or Right) happens with probability 0.8. The probability that the agent ends up in one of the states perpendicular to the intended direction is 0.1 each. If a collision with a wall happens, the agent stays in the same state, and the drift probability is added to the probability of remaining in the same state. Assume that V1_1(A,1) = 0, V1_1​(C,1) = 0, V1_1​(C,2) = 4, V1_1​(A,2) = 4, V1_1​(B,1) = -5, and V1_1​(B,2) = +5. Given this information, what is the second round of value iteration (V2_2) update for state (A,1) with a discount of 1?