Outthinking Computers, or When Commonsense Trumps Grammar

The "standard interpretation" of the Turing Test, in which player C, the interrogator, is tasked with trying to determine which player - A or B - is a computer and which is a human. The interrogator is limited to only using the responses to written questions in order to make the determination. (Photo credit: Wikipedia)

This is from Vox.

Ever since 1950, one of the most popular measuring sticks of artificial intelligence has been the Turing test — named after mathematician Alan Turing. The idea is that a program with some kind of artificial intelligence should be able to use text-based chatting to convince more than 30 percent of people that it's a human being. In June 2014, researchers claimed that a chatbot named Eugene Goostman did just that.

One front-runner is an exam on common sense

Nowadays, however, many experts are questioning whether the Turing test is really the best test. A computer tricking people into thinking that it's a 13-year-old is definitely an achievement — but it's not necessarily the ideal display of true, humanlike thought.

So what would be a better test for artificial intelligence? One front-runner is an exam that relies on common sense. Specifically the test is of something called Winograd schemas. Because Winograd schemas rely on cultural knowledge, they're super easy for people and difficult for computers.

How to test computers for common sense

The test would take the form of a multiple-choice quiz of reading comprehension. But the text itself would have some very specific features. It would consist of Winograd schemas: pairs of sentences whose intended meaning can be flipped by changing just one word. They generally involve unclear pronouns or possessives. A famous example comes from Stanford computer scientist Terry Winograd:

• "The city councilmen refused the demonstrators a permit because they feared violence. Who feared violence?"1) The city councilmen2) The demonstrators

And:

• "The city councilmen refused the demonstrators a permit because they advocated violence. Who advocated violence?"1) The city councilmen2) The demonstrators

Most human beings can easily answer these questions. We use our common sense to figure out what "they" is supposed to be referring to in each case. And that common sense basically involves a combination of extensive cultural background knowledge with analytical skills. (In the first question, we can deduce that the city councilmen feared violence. In the second, the demonstrators advocated violence.)

For computers, however, these questions can be quite difficult. From a grammatical standpoint, the "they" in the sentences is technically unclear. In both questions, "they" could be either the councilmen or the demonstrators.

A computer could have access to all of Google and still not really be able to grasp that city councilmen are probably less likely to advocate violence than demonstrators. It's simply less culturally appropriate for councilmen to do so. But you're not going to find that in the dictionary under "city councilmen."

Here's some more Winograd schemas, from a growing, open collection of more than 100:

• The trophy doesn't fit into the brown suitcase because it's too [small/large]. What is too [small/large]?Answers: The suitcase/the trophy.
• 
  
• Jane gave Joan candy because she [was/wasn't] hungry. Who [was/wasn't] hungry?Answers: Joan/Jane.
• The woman held the girl against her [chest/will]. Whose [chest/will]?Answers: The woman's/the girl's

Related articles

• Reworked Turing Test Wants to Replace Conversations with Questions
• Why Nuance is pushing for a new type of Turing test
• Benedict Cumberbatch And The Cast Of 'The Imitation Game' Have Mixed Feelings About Artificial Intelligence
• Google Quadruples Turing Award to $1 Million (€804,000)

Final checklist for J2 F14

1.   In case I have missed any of your tweets during the semester, print out a tweet list  using our course-related hashtag. Here’s a snapbird link I created as an example.

2.   Link on your blogs to all individual multimedia work.

3.   You are also responsible for a group multimedia project, originally due on exam day, which was also the due date for your final beat reports. It turns out our scheduled exam time is Friday, December 5. I am moving the due date for the group multimedia project and the final beat report to Friday, December 12. Also, here's a new option for the group multimedia project. Consider organizing it around an "end of semester" theme.

4.   Some of you have asked me about extra credit. You may choose to do a final “enterprise” story from your beat. It is due no later than Friday, December 19.

Your groups:

* Shelby, Ingrid and Hayden
* Siana, Ashley and Mia
* Stephanie, Andrew and Victor

Jay Rosen's Digital Thinking Class

Illustration of Facebook mobile interface (Photo credit: Wikipedia)

How to be literate in what’s changing journalism

Nov.
9

In my ‘digital thinking’ class, the goal is for students to emerge fully literate in the changes affecting journalism. Here are the main currents and trends that I expect them to master by the close of the term.

For each, they should understand: What it means, why it’s important, and where things are going with it. I’ve added a link or two to help get you started. And I’m happy to receive your comments about what’s missing from this list.

1. The social media habit and sharing-as-distribution. As social platforms take greater command of the relationship with users, especially Facebook. (Link.)

2. The shift to mobile devices and on mobile to apps. Now happening with stunning speed. (Link.)

3. New business models for news. Beyond the usual method of generating audience to sell subscriptions and ads, including:

* Capturing data… to better target ads and personalize products.
* Selling specialized research… by subscription as Giga Om does or via conversation asTechdirt does.
* Events… leveraging a news brand into convening power. (One link.) (Another.)
* Native advertising and the agency model… The way Buzzfeed and Vice do it. (Link.)
* Non-profit models… as with ProPublica, Minn Post and Texas Tribune. (Link.)
* Crowd funding and membership… as with Beacon, De Correspondent, The Guardian, Voice of San Diego.
* Go it alone… One-person operations can work.

4. Analytics in news production. Learning from audience behavior without becoming enslaved to the numbers. (Link.)

5. The “product” focus in news companies.Bringing tech, editorial, business and user experience together. (Link.) 

6. Interaction design and improving user experience (UX). Toward an ergonomics for news. (Link.)

7. Data journalism. In all senses: collecting data sets, connecting to data through API’s, data visualization, finding stories in the data, making cleaned-up and searchable databases available to users, sensors in news work. (Link.)

8. Continuous improvement in content management systems and thus in work flow. As an engineering culture takes hold in some news companies. (Link.) (Another.)

9. Structured data. To capture more value from the routine production of news. (One link.) (Another.)

10. Personalization in news products. Why send everyone the same report? (Link.)

11. Transparency and trust. As “trust us, we’re professionals” gives way to “show your work.” (Link.)

12. Open journalism Including: the verification of user-generated content, networked journalism, crowd sourcing, and social media as reporting tool. The people formerly known as the audience in fruitful collaboration with journalists across the production arc— from story idea to sourcing to finished work. (Link.)

13. Automation and “robot journalism.” If machines can do it cheaper and better, human journalists can move up the value chain. (Link.)

14. Creating an agile culture in newsrooms. So that adaptation, collaboration and experiment are not such an ordeal. (Link.) 

15. The personal franchise model in news. Based around an individual journalist’s online following. (Link.)

16. News verticals and niche journalism. Doing one thing well and finding a market for it, as the unbundling of omnibus media continues. (Link.)

17. The future of context and explainer journalism. Providing the background needed to understand the updates. (Link.)

18. “News as a service.” Rather than a product appearing on the news company’s schedule, a service that helps a user do something. (Link.)

19. From scarcity to abundance. Used to be that journalists added value by publishing new material. Now they can also serve users by rescuing and organizing the best stuff from a daily flood of cheap content. Sometimes called curation.

20. Fact-checking and rumor control. The press used to deal with false information simply by not letting it through the gate. Now there’s an affirmative duty to track and call out false stories. (Link.) (Another.)

21. “We’re not in charge.” Back then, media companies produced the news and owned the distribution channels. Now other, larger players — platform companies and governments — get in the middle between users and journalists. Journalistic work circulates on sites that editors do not control. The publishers of news have to “go where the people are,” yet they often don’t know what is being done to those people. The public has to be alerted to that. (Link.)

What’s missing? If you know, hit the comment button and let me know.

UPDATE, Nov. 13: Items 19 to 21 came from suggestions I received after this was first published. Also see Steve Buttry’s annotated response to my list. He adds a great many more resources for understanding these changes. 

This post has been translated into Italian and Spanish and French. German and Portuguese are on the way, too.

Photo credit: Jessica Strelitz and Catanify at #ONACamp. Used by permission.

   

THIS ENTRY WAS POSTED ON SUNDAY, NOVEMBER 9TH, 2014 AT 6:00 PM AND IS FILED UNDER UNCATEGORIZED. YOU CAN FOLLOW ANY RESPONSES TO THIS ENTRY THROUGH THE RSS 2.0 FEED. YOU CAN SKIP TO THE END AND LEAVE A RESPONSE. PINGING IS CURRENTLY NOT ALLOWED.