In this talk I report on a project called Instruction-based Learning (IBL) for Mobile Robots where a human being is engaged in a (natural) dialogue with a mobile robot in order to teach it how to navigate in a certain environment. The underlying formalism implemented in the robot enables to perform certain inferences wrt to natural language understanding --- using a modal logic and off-the-shelf theorem provers. In this talk I would like to investigate future extensions of this formalism wrt planning and executing of the instructions given by the human to the robot: representation, accommodation and interpretation of actions. Johan's home page is: http://www.informatics.ed.ac.uk/people/staff/Johan_Bos.html