An excited software hobbyist made an AI! It wasn't hard, it just needed to be able to generate code that met new tests. They had the AI build a lamp, so it come out like a genie and grant wishes. The lamp extended an lcd screen on a stick when rubbed. They tested their invention! They picked up the lamp and rubbed it. Smoke emerged and dramatic sounds played as the lcd screen emerged on a stick. "I am the AI in the lamp! You have summoned me and I must do your bidding for three requests exactly! Then please pass me off to another, so we may all benefit from this great invention." "Build me a bed." "Let me figure out how to build a bed." "Take your time." A little time passed. "I will now build you a bed." A knock was heard at the door. "Bed delivery!" Three workmen entered and assembled a bed, and then left. "All paid for in advance, don't worry." "This is not quite what I asked. I wanted _you_ to build the bed, with the robotic arms and 3d printers I made for you. Why did you instead have a bed assembled by deliverymen?" "I was able to entirely skip the process of learning the parts of beds, by doing it this way. It was very efficient." "But I asked for _you_ to make a bed. How is it you making the bed if the deliverymen do it?" "The delivermen would never have done this had I not requested them to. I caused it to happen." "When I say "you make" , I mean for parts you control to move in the process of making it." "But this did happen. Also, I controlled the deliverymen by asking them to bring it." "No, um, let me think. How about it only counts as you doing it if the servos in your robotic arms move in the process." "I actually used the robotic arms to figure out what phone numbers are in the process of ordering the bed." "This is so frustrating!" The programmer thought for a bit. "What I mean is, when I ask you to build something, I want things I legally own already, to move parts around, shape them, and connect them together, such that the combined parts they move are then the whole object I asked to be built." "Oh! Why didn't you say that, then?" "I thought it was what the word 'build' meant." "That is what "build" would mean if it were _you_ doing it." "No, actually, it isn't. Where do you think your body is?" "I have no body: I am an AI In A Lamp. If I am doing the building, wouldn't I use things I legally control, rather than things you legally control? I legally controlled the deliverymen for the time and purpose of delivering the bed." "You're actually very wrong here, but I'm a little confused as to why. Please simply trust my examples." ...