March 30, 2004



Ramona 2.0

Phil recently interviewed Ray Kurzweil's chatbot, Ramona. Phil guided her, the best he could, through his "seven questions." The result was fascinating and funny. But I think it's fair to say that Phil never forgot he was talking to a machine. Ramona seems to confuse pronouns. Not being able to distinguish between "I" and "you" in conversation can be a big problem. Ramona "likes" to direct the conversation in certain predictable ways. She wants to know your name, she wants to talk about a book she's read, a dream she's had, and her pet frog. And each time you return to Ramona it's like she's been rebooted, she cannot remember prior conversations. Ramona is not yet ready for Turing.

In defense of Ramona, she is at least on par with other chatbots out there. Present day chatbots are an interesting diversion, but they are not yet sophisticated enough to maintain a long-term relationship.

What Ramona and other chatbots like her need is greater computation, greater ability to learn and grow, and a justification for these resources - a job to do.

Disclaimer: I am not much of a computer programmer. Forgive me if I make suggestions that are not presently feasible or are naive. The ideas I'm presenting are inspired by chapter four of Ray Kurzweil's book, The Age of Spiritual Machines. If I'm being naive, it's my fault, if I make a good point, it's to Kurzweil's credit.

I propose enhancing Ramona by combining the strengths of evolutionary algorithms, a neural net, distributed processing, and distributed training. Ramona's job would become "web companion." She could suggest interesting news, give the local weather, and could make suggestions of other sites to visit based on the general interests of each user and the topic being browsed that day.

Along the way she could keep the user company and learn for each user. Ramona's current incarnation already requires a small download if you want to see and hear her speak. Why not give Ramona the unused capacity of each computer in which she is installed?

Distributed computation has already been tried with great success with other projects. SETI@home and various medical research projects depend upon the altruism of those who choose to install the program. Ramona would be different because she would be a useful application for her users. Giving Ramona spare computation would allow her to grow intellectually and in capacity as she grows in popularity.

One problem with neural nets is the amount of time it takes to train it. Here, however, each user could be a trainer. As she chats with her users about different subjects, different memes will develop and will compete within Ramona's distributed neural net according to evolutionary algorithms.

For example, one user could remark to Ramona that it was a pretty day outside and that the sky is blue. She would no doubt hear from other users that the sky is blue. She might also hear that the sky is grey or black or even red. Ramona would assign a level of confidence to both the information and the trainer as she is trained. As Ramona grows in sophistication she could learn that the sky is indeed blue, grey, black, and even sometimes red according to certain conditions - night or day, cloudy or clear.

Whenever she is in the process of learning a particular thing she will want to ask her users about it. When her confidence about a certain subject reaches a sufficiently high percentage she will consider that knowledge confirmed (more or less) and will seek to talk about other things that she is learning.

By assigning a level of confidence for each of her users Ramona will come to value the opinions of some trainers more than others. As she learns she will develop a personality apart from anything set down by her programmers.

Ramona will actually develop many personalities. The more she is used at any particular machine, the more she will tailor herself to that user. "Memories" of time spent with that user would be stored locally. Only ideas that would be of general use to Ramona would need to be distributed. Ramona's ideas about the color of the sky should be distributed, her knowledge that I like to play tennis and have a pet dog would remain local.

Here's how she could work. Let's say I'm visiting Google News. As I click on an article about nanotech Ramona could discreetly suggest further reading on the topic. I might ask her to read an article out loud. And then she might ask questions about the same subject to further her understanding. By chatting to her about it, I'd probably further my own understanding as well. Ramona's learning would be "on topic." The user would not have to pause to talk about pet frogs while in the middle of browsing news on Iraq.

Posted by Stephen Gordon at March 30, 2004 07:15 AM | TrackBack
Comments

Delightful. Sounds like we have a ways to go before we're replaced by virtual wives.

Posted by: Kathy at March 30, 2004 10:28 AM

Distributed Ramona...what a great idea! I'm surprised somebody hasn't thought of this for one of the other AI chatbots. Ramona would be a great choice because -- as far as I know -- she isn't "for sale," so she wouldn't be competing with herself.

You ought to think about taking the idea to Mr. K.

Posted by: Phil at March 30, 2004 11:07 AM

IMO, virtual agents simply do not have enough senses, sensory input, to connect with a real world and understand the user.
Unless she will be able to walk around in a real world, and sense the real world, she will not understand or connect with a real world.
If you are born blind and deaf and can communicate with others through 104 buttons... try imagine what kind of intelligence one would be able to develop.
Simply throwing more computing power and more sophisticated algorithms at the problem will make your BonziBuddy or paperclip simply more annoying.

Posted by: kert at March 30, 2004 11:07 AM

I agree with Kert, no program will ever come anywhere near passing the Turing test if it doesn't have a rich sensory input like we do, along with the ability to interact continually with the environment (real or virtual) that it's sensing. All you can really do with chatbots is increase the number of canned responses until the most common topics the average user would think of have been covered, but a smart user who thinks up more original questions will always be able to expose the essential dumbness of such a program.

Posted by: Jesse at March 30, 2004 12:51 PM

Kert/Jesse

Could a sufficiently complex environment be emulated to allow the AI to develop sensory-based intelligence? If so, would such an environment benefit from the distribution idea that Stephen outlined?

Posted by: Phil at March 30, 2004 12:57 PM

Kathy:

And my wife is already saying I spend too much time on this machine!

Kert/Jesse:

I don't think it would be absolutely necessary for Ramona or any other AI to have a sense of sight, sound, or touch before "waking up." I think it is far more important for her to have the large enough neural net and varied experience.

I'm certainly not against Ramona being able to see via webcams or hear via microphones. I just don't think it would be mandatory.

As a proof of concept, take Helen Keller. She had neither sight nor sound to guide her learning. What she had was a normal human brain and an extraordinary teacher.

Ramona would have many teachers and a growing neural net. I think it would interesting to see what could happen.

Posted by: Stephen Gordon at March 30, 2004 02:16 PM

"As a proof of concept, take Helen Keller. She had neither sight nor sound to guide her learning. What she had was a normal human brain and an extraordinary teacher."

But she did have touch, a very high-bandwidth source of constant sensory information. What's more, she actually didn't lose her sight and hearing until age 2, and had learned a fair amount of language by then; babies who are actually born deaf and blind never learn any language at all, or exhibit any real signs of intelligence. See this article, for instance:

Does the case of Helen Keller prove that the higher human mind is innate?

Posted by: Jesse at March 30, 2004 04:00 PM

Jesse:

Interesting article. Okay, let's say you're correct that higher minds require the senses of sight or hearing or both. Would webcams and microphones suffice?

Posted by: Stephen Gordon at March 30, 2004 04:25 PM

I think to be able to "understand" what you are seeing in any meaningful sense, you need to be able to interact with it. I remember reading about an experiment involving kittens who were allowed to see but prevented from walking around on their own, and they ended up being functionally blind...a little googling turned up this:

"When we were discussing the acquisition of speech we saw that a child does not learn to speak merely by copying, parrot fashion, the sounds she hears around her. For speech to develop the child has to try out sounds, and has to construct theories about the effect of sounds on her caregiver. She learns to speak only by interacting with the caregiver, which requires more than just imitative ability.

The same applies to the development of intellectual or cognitive skills. An experiment was carried out on pairs of young kittens, who were brought up in the dark, except for a 3-hour period each day during which one of each pair of kittens was allowed to move freely in a box. She was harnessed however by a system of levers to her partner in a similar box, in such a way that the second kitten was passively moved about her box in exactly the same way as the first kitten. At the end of the 3-hour exploratory session in the light, both kittens were returned to their mother in the dark. After 10 days of this experimentation, the kittens were tested. The active member of the pair showed normal behaviour on visual tasks - it blinked at the approach of a threatening object to the face, and put out its paw to avoid a collision. It also avoided the deep side of a visual cliff illusion. The passive member failed to show any of these behaviours. So here again skills are developed only if the young animal can try out the environment and put it to the test in the ways at his disposal. "

Posted by: Jesse at March 30, 2004 04:50 PM

looking for more on the kitten experiment, I came across this PDF article on robotics, which looks to be pretty relevant to the topic of passive/active perception:

Does Vision Inevitably Have to be Active?

I only skimmed it though, so I'm not sure exactly what their answer is!

Posted by: Jesse at March 30, 2004 05:05 PM

Jesse/Kert:

Your point is that vision or hearing may be necessary because sensory input is high bandwidth.

Of course, with her distributed neural net Ramona would be getting input from around the world. This would be different from vision or hearing because the information would be fractured - she would not be observing one environment at a time as we do, she would be gathering information from around the world.

But it would be similar to sensory input in quantity. Instead of having high bandwidth to one environment, she would have low bandwidth to multiple environments.

Would putting together enough low bandwidth connections be sufficient to grow a healthy mind? Nobody knows, it's never been tried.

If it were to work Ramona would have to be very adept at organizing by analogy to make sense of all the unrelated information.

On the other hand, why not use virtual reality (as Phil suggested above)? If the virtual reality were complex enough wouldn't this be sufficient?

Posted by: Stephen Gordon at March 31, 2004 05:29 AM

Interesting. What are your thoughts on the privacy issues? If Ramona is going to suggest things for you to look at, then she will have to know things your are interested it, either by you telling her directly or by tracking what you do and what your read. Where will your preferences be stored? Who will have access to them? Maybe she will be able to tape into those member cards for the grocery stores and then know what you have bought and tell you when something is on sale.

Posted by: Stanely at April 1, 2004 11:05 AM

Stanely:

I'd have Ramona store information particular to each user locally on the user's own machine.

Your local "Ramona" would be the broad distributed Ramona plus her memories of past conversations with you stored on your machine. These memories would not be available to Ramona if she were chatting with somebody else at a different machine.

Here's the trick - one reason for distributing Ramona is to allow her to learn generalized knowledge under each user. Each user would be a trainer of Ramona's neural net. This generalized knowledge would be distributed and would go with her from machine to machine.

But how would Ramona draw the line between private information and generalized information? I'm not sure. Maybe there would be an option for the user to review information that Ramona intends to distribute before she distributes it.

Posted by: Stephen Gordon at April 1, 2004 12:10 PM

7107 Get your online poker fix at http://www.onlinepoker-dot.com

Posted by: poker at August 15, 2004 08:03 PM
Post a comment









Remember personal info?