Creating an AI than can learn to give instructions

3

1

So we think a computer is dumb because it can only follow instructions. Therefor I am trying to create an AI that can give instructions.

The idea is this: Create a geometric scene (A) then make a change in scene such as turning a square red or moving a circle right one unit. This becomes the new scene B. Then the computer compares the scenes A and B and it's goal is to give the shortest possible instruction that will change scene A to scene B. Examples might be:

"Turn the green square red".


or

"Move the yellow square down".


Or when we get more advanced we might have:

"Move the green square below the leftmost purple square down."


Equally, this task could be seen as finding a description of the change. e.g. "The green square has turned red".

The way it would work is that there'd be a simplified English parser, and the computer would generate a number of phrases and check whether these achieved the desired result.

I would probably give it some prior knowledge of things like colours, shape-names, and so on. Or it could learn these by example.

Eventally I would hope it to generate more complicated loop type expressions such as "Move the square left until it reaches the purple circle." And these would essentially be little algorithms the AI has generated in words.

I've got some ideas how to do this. But do you know any similar projects that I could look at? If you were implementing this, how would you go about it?

[In other words we have an English parser that is interpreted to change a scene A into a scene B. But we want the AI to learn, given scenes A and B, how to generate instructions.]