Sunday, November 30, 2014

Outthinking Computers, or When Commonsense Trumps Grammar

The "standard interpretation" of the...
The "standard interpretation" of the Turing Test, in which player C, the interrogator, is tasked with trying to determine which player - A or B - is a computer and which is a human. The interrogator is limited to only using the responses to written questions in order to make the determination. (Photo credit: Wikipedia)

Ever since 1950, one of the most popular measuring sticks of artificial intelligence has been the Turing test — named after mathematician Alan Turing. The idea is that a program with some kind of artificial intelligence should be able to use text-based chatting to convince more than 30 percent of people that it's a human being. In June 2014, researchers claimed that a chatbot named Eugene Goostman did just that.
Nowadays, however, many experts are questioning whether the Turing test is really the best test. A computer tricking people into thinking that it's a 13-year-old is definitely an achievement — but it's not necessarily the ideal display of true, humanlike thought.
So what would be a better test for artificial intelligence? One front-runner is an exam that relies on common sense. Specifically the test is of something called Winograd schemas. Because Winograd schemas rely on cultural knowledge, they're super easy for people and difficult for computers.

How to test computers for common sense

The test would take the form of a multiple-choice quiz of reading comprehension. But the text itself would have some very specific features. It would consist of Winograd schemas: pairs of sentences whose intended meaning can be flipped by changing just one word. They generally involve unclear pronouns or possessives. A famous example comes from Stanford computer scientist Terry Winograd:
  • "The city councilmen refused the demonstrators a permit because they feared violence. Who feared violence?"1) The city councilmen2) The demonstrators
And:
  • "The city councilmen refused the demonstrators a permit because they advocated violence. Who advocated violence?"1) The city councilmen2) The demonstrators
Most human beings can easily answer these questions. We use our common sense to figure out what "they" is supposed to be referring to in each case. And that common sense basically involves a combination of extensive cultural background knowledge with analytical skills. (In the first question, we can deduce that the city councilmen feared violence. In the second, the demonstrators advocated violence.)
For computers, however, these questions can be quite difficult. From a grammatical standpoint, the "they" in the sentences is technically unclear. In both questions, "they" could be either the councilmen or the demonstrators.
A computer could have access to all of Google and still not really be able to grasp that city councilmen are probably less likely to advocate violence than demonstrators. It's simply less culturally appropriate for councilmen to do so. But you're not going to find that in the dictionary under "city councilmen."
Here's some more Winograd schemas, from a growing, open collection of more than 100:
  • The trophy doesn't fit into the brown suitcase because it's too [small/large]. What is too [small/large]?Answers: The suitcase/the trophy.

  • Jane gave Joan candy because she [was/wasn't] hungry. Who [was/wasn't] hungry?Answers: Joan/Jane.
  • The woman held the girl against her [chest/will]. Whose [chest/will]?Answers: The woman's/the girl's

Saturday, November 29, 2014

Final checklist for J2 F14


1.   In case I have missed any of your tweets during the semester, print out a tweet list  using our course-related hashtag. Here’s a snapbird link I created as an example.

2.   Link on your blogs to all individual multimedia work.


3.   You are also responsible for a group multimedia project, originally due on exam day, which was also the due date for your final beat reports. It turns out our scheduled exam time is Friday, December 5. I am moving the due date for the group multimedia project and the final beat report to Friday, December 12. Also, here's a new option for the group multimedia project. Consider organizing it around an "end of semester" theme.


4.   Some of you have asked me about extra credit. You may choose to do a final “enterprise” story from your beat. It is due no later than Friday, December 19.

Your groups:

* Shelby, Ingrid and Hayden
* Siana, Ashley and Mia
* Stephanie, Andrew and Victor




Tuesday, November 18, 2014

Jay Rosen's Digital Thinking Class

Illustration of Facebook mobile interface
Illustration of Facebook mobile interface (Photo credit: Wikipedia)

How to be literate in what’s changing journalism

Nov.
9
In my ‘digital thinking’ class, the goal is for students to emerge fully literate in the changes affecting journalism. Here are the main currents and trends that I expect them to master by the close of the term.
For each, they should understand: What it means, why it’s important, and where things are going with it. I’ve added a link or two to help get you started. And I’m happy to receive your comments about what’s missing from this list.
1. The social media habit and sharing-as-distribution. As social platforms take greater command of the relationship with users, especially Facebook. (Link.)
2. The shift to mobile devices and on mobile to apps. Now happening with stunning speed. (Link.)
3. New business models for news. Beyond the usual method of generating audience to sell subscriptions and ads, including:
* Capturing data… to better target ads and personalize products.
* Selling specialized research… by subscription as Giga Om does or via conversation asTechdirt does.
* Events… leveraging a news brand into convening power. (One link.) (Another.)
* Native advertising and the agency model… The way Buzzfeed and Vice do it. (Link.)
* Non-profit models… as with ProPublica, Minn Post and Texas Tribune. (Link.)
* Crowd funding and membership… as with BeaconDe CorrespondentThe GuardianVoice of San Diego.
* Go it alone… One-person operations can work.
4. Analytics in news productionLearning from audience behavior without becoming enslaved to the numbers. (Link.)
workingwithproduct5. The “product” focus in news companies.Bringing tech, editorial, business and user experience together. (Link.) 
6. Interaction design and improving user experience (UX). Toward an ergonomics for news. (Link.)
7. Data journalism. In all senses: collecting data sets, connecting to data through API’s, data visualization, finding stories in the data, making cleaned-up and searchable databases available to users, sensors in news work. (Link.)
8. Continuous improvement in content management systems and thus in work flow. As an engineering culture takes hold in some news companies. (Link.) (Another.)
9. Structured data. To capture more value from the routine production of news. (One link.) (Another.)
10. Personalization in news products. Why send everyone the same report? (Link.)
11. Transparency and trust. As “trust us, we’re professionals” gives way to “show your work.” (Link.)
12. Open journalism Including: the verification of user-generated content, networked journalismcrowd sourcing, and social media as reporting tool. The people formerly known as the audience in fruitful collaboration with journalists across the production arc— from story idea to sourcing to finished work. (Link.)
13. Automation and “robot journalism.” If machines can do it cheaper and better, human journalists can move up the value chain. (Link.)
14. Creating an agile culture in newsrooms. So that adaptation, collaboration and experiment are not such an ordeal. (Link.) 
15. The personal franchise model in news. Based around an individual journalist’s online following. (Link.)
16. News verticals and niche journalism. Doing one thing well and finding a market for it, as the unbundling of omnibus media continues. (Link.)
17. The future of context and explainer journalism. Providing the background needed to understand the updates. (Link.)
18. “News as a service.” Rather than a product appearing on the news company’s schedule, a service that helps a user do something. (Link.)
19. From scarcity to abundance. Used to be that journalists added value by publishing new material. Now they can also serve users by rescuing and organizing the best stuff from a daily flood of cheap content. Sometimes called curation.
20. Fact-checking and rumor control. The press used to deal with false information simply by not letting it through the gate. Now there’s an affirmative duty to track and call out false stories. (Link.) (Another.)
21. “We’re not in charge.” Back then, media companies produced the news and owned the distribution channels. Now other, larger players — platform companies and governments — get in the middle between users and journalists. Journalistic work circulates on sites that editors do not control. The publishers of news have to “go where the people are,” yet they often don’t know what is being done to those people. The public has to be alerted to that. (Link.)
What’s missing? If you know, hit the comment button and let me know.
UPDATE, Nov. 13: Items 19 to 21 came from suggestions I received after this was first published. Also see Steve Buttry’s annotated response to my list. He adds a great many more resources for understanding these changes. 
This post has been translated into Italian and Spanish and French. German and Portuguese are on the way, too.
Photo credit: Jessica Strelitz and Catanify at #ONACamp. Used by permission.

29 Comments

  1. Are there some kind of public notes? It would be very interesting for other industries that are also making the transition to digital.
  2. I’d propose what I think is a whole other category.
    Much of the focus to date has been digital reversioning or incremental improvement of existing processes, in three main areas: Information gathering, information parsing/analysis and the distribution of what I would say is just one result of that process – articles in print/content/news broadcasts/online articles/digital news packages in any form – video, written, oral etc..
    I’d summarise the 18 this way. Most involve improvements to what I would call the existing content creation and ad-supported “paradigm”. Information is 1) gathered 2) synthesised by humans (journalists), and then 3) a content “package” is created in one form or another (online, offline, TV etc – for ease I’ll just define “content” as anything that’s distributed which was created by a human.
    1: Social media and sharing-as-distribution. Distribution
    2: The shift to mobile devices and on mobile to apps. Distribution
    3: New business models for news. Parsing & distribution
    4: Analytics in news production. Distribution
    5: The “product” focus in news companies. All three (but still traditional outputs as above)
    6: Interaction design and improving user experience (UX). Distribution
    7: Data journalism. Parsing
    8: Continuous improvement in content management systems and thus in work flow. Parsing
    9: Structured data. Parsing
    10: Personalization in news products. Distribution
    11: Transparency and trust. Distribution
    12: User generated content. Gathering (and was my area of speciality at Storyful)
    13: Automation and “robot journalism. All three
    14: Creating an agile culture in newsrooms. Improvement in processes of all three
    15: The personal franchise model in news. Alll three
    16: News verticals and niche journalism. Still within existing paradigm
    17: The future of context and explainer journalism. Parsing & distribution
    18: “News as a service.” A product of sorts, but within current paradigm.
    But why should we restrict our industry to incremental improvements of the existing processes? By its nature, news and journalism are obviously tied to information. Information is gathered, analysed and the results distributed as content, with usually adverts along side.
    But surely “content” creation is merely one output of those information processes. For example: a business journalist might manually keep an eye on company filings in order to find scoops. When that business journalist – using often entirely human manual processes – finds a nugget, they generate content (a story on page one).
    But why don’t newsrooms automate as much of that process as possible (using software)? I’d call it a scoop suggestion system (since machines aren’t perfect but can suggest nuggets). And why restrict the output to page one news stories?
    All this valuable information is being gathered and analysed by humans. The intelligence being built up is itself valuable – and is itself a possible product (where software has been built to automate as much of it as possible).
    Not a product to be sold with ads, but a business intelligence product. Or a fraud detection product. Or a competitive analysis product. For who? Businesses that need it and market segments that demand it (including, believe it or not, sections of your existing reader/viewer/listenership).
    Scoops, storytelling and content are valuable. Often the value that’s created in the news gathering and parsing process is great, but sadly not fully captured because the focus is entirely on manually telling stories.
    It should be all of the above – build software products based on the skill and intelligence of the smartest journalists and newsrooms, and sell it to markets that need information products. Scoops (more of them if the systems are good) can be a handy ad-supported gravy on the side.
    But I reckon the big money is in the software itself.
    I outlined some of these ideas recently here.
    • Hi, Gavin. Thanks very much for this detailed analysis of what’s missing. I think I see where you are going with it.
      It’s true that even the more advanced news companies are having a hard time thinking beyond the article form, and they tend to default to the production of content as their raison d’être without challenging that concept, or even seeing it as a concept. I think you are completely on target there.
      But I don’t think it’s correct to say that this list as constructed is all “digital reversioning or incremental improvement of existing processes.” Several items on it go well beyond that.
      “Sensor journalism” (link) is very close to the kind of constant monitoring you speak of. New mobile-first firms like Circa are breaking down the article into more basic parts and sending alerts that don’t depend on that form at all. The whole point of “news as a service” (which, yes, will depend on software) is to change the product on offer from pieces of news content to “the state of being well informed.”
      I take your point that news may be only one of the products that news companies make when they learn to how to use machines to monitor the environment and detect events. Perhaps there is a way to make this number 19 or 20 on my list.
      Thanks again for your contribution.
  3. HF
    missing: data provenance & fact=checking/sourcing.
    One of the most common failings of media in the hot-take era is failing to verify sources and/or to distinguish between current facts and those published in earlier stories. subsequently debunked, but still up online.
    This creates enormous churn and fodder for conspiracy theories.
    It matters in big stories (a la the Post’s “Bag Men” marathon bombing fiasco) and small ones (a Gawker piece that recycles a Jezebel piece about a rape joke that “started on twitter” — when in fact it’s a very old vaudeville blue joke).
    One thing I’ve noticed is a laziness about tracing back to source even when instantaneously available online. A separate issue is that some people who’ve grown up with the internet limit their research *to it* forgetting that much of history, especially the telling minutia, is simply not online. Books, microfiche, in-person interviews. Things can get very insular and self-referentlai, and the same falsehoods can be replicated all over the net. (Like those misattributed inspirational quotes, only with hard news.)
    • I could have broken out “verification” as its own item, because it is undeniably a key practice area today, but I included it under user-generated content, which covers most of the examples you mention.
      See 12. User generated content (including verification of) with this link:
      • HF
        hmmm. My sense is that’s nested too far down. Without fact-checking chops, none of the other skills & practices matter bc it’s just amplifying junk.
        It’s not user-generated content that needs to be untangled imo — it’s the echo chamber of even “trusted” news sites & sources.
        What starts to matter most is being able to trace back in time to which pieces/writers picked up which facts/errors from whom, back to the original.
        It’s hard to do bc of the seeming simultaneity of online artifacts.
        I did it myself using a big, ugly twitter fight as the paradigm. The Sarah Kendzior/Jacobin meltdown several months ago.
        I wanted to trace how exactly Kendzior’s objection to use in a Jacobin article of her at-message regarding a rape threat escalated into claims that journal editors had *issued* rape threats,
        Which people amplified which misunderstandings?
        It was possible to see who had picked up on what, and even to ascertain which people had not read the article at all.
        I read every thread,. ever tweet, every “canoe” and split-canoe.
        I also read every opinion piece, hot take, apology, and rant on blogs and media sites.
        This is a great exercise (to focus on source & transmission, without regard to the emotional valence of the content).
        Because if you had an entire class do this & then talk it out, you’d find that, *even in cases with complete transparency, where ALL artifacts are archived & available* it is very difficult to reconstruct faithfully a true account.
        And my belief is that, if we can’d do this for twitter kerfuffles, how can we ever do it for huge, consequential, world-shaking events?
        It’s also my belief that, were this exercise performed rigorously by the press in the lead-up to the Iraq War (who was claiming what, to which journalist, from which branch of the government; and who was citing those claims secondhand as if truth) — we might have had a real debate about the real facts instead of the press rolling over & playing dead.
        Essentially I’m arguing that the fate of the earth rests upon this type of media literacy, for both journalists and readers.
      • I remember that bizarre event. (Also involved there was not “fact-checking” so much as a really, really crappy paraphrasing. Putting into other words what someone said but getting it very wrong in the translation, while thinking you’re 100% right and that “everyone” can see it. I don’t know what to call that. A snarkalicious term might be… read much? But of course the people most prone to this are highly literate, often hyper-literate so that doesn’t really work.)
        Good points, thanks. I am going to do a revised version of this list, so I may find a way to incorporate.
  4. ELISSA YANCEY
    What are your thoughts on the role of curation? In my courses at Cincinnati, we talk about that as a crucial skill set for students to learn and embrace– a lot like the “everybody doesn’t have to do everything” idea.
    Thanks for this and look forward to your thoughts.
    • Your question allows me to clarify something about this list.
      I wasn’t trying to do my version of something that is often done in journalism education. Teachers put their heads together and ask themselves: what skills do our students need if they’re going to compete for jobs in the digital world? Curation! Spreadsheets! Video editing! That’s worth doing. But that’s not what I am doing here.
      The focus is not on “skills” but on “forces.”
      I am starting in a different place: what’s changing in journalism, and what is forcing change by looming ever larger in the calculations of those trying to build a 21St century news operation?
      I want my students to understand those things, first. We handle skills in a different way. We do projects with media partners, centered on problems in adaptation they are having. Students pool their skills and develop the skills they need to deliver on the project. We also give them weekend seminars with technologists that are demand-driven. They learn the skills that they most want to have when they leave NYU.
      • ELISSA YANCEY
        I like the verb “understanding” for forces and “teaching” for skills. So I guess my sense is that the idea curation (which encompasses a lot of elements on your list) is a “force.” The internet allows such a different way of gathering information of so many varied types, with the opportunities for genius alongside crap, that understanding the value of smart, thoughtful curation of information (in whatever form–data, visuals, primary research documents, etc.) and how it can transform stories is more than a skill set, it’s a way of understanding information in multiple dimensions. Maybe, you might argue, it’s just good reporting. While that’s part of it, an important part of it, I think there’s something more, something creative and important that we can push ourselves and our students to define and redefine.
        And, so I’m clear, we teach skills separately, too, with local partners and field/lab work. Our curriculum is (for now) tightly packed, though, so we often teach what you’re defining as “forces” alongside skills.
        Thanks for your response–always enjoy reading and thinking about your work!
      • The revised list incorporates this suggestion. Once I thought of how to frame it as a “shift” rather than a “skill” I was eager to add it.
  5. JESSE HOLCOMB
    Cool idea! Nice to see some of our survey data on social media in there. If you’re interested, happy to follow up w/ specific reports or report sections for your students that home in on the FB news experience and/or sharing.
  6. KEN JARVIS
    It is NOT that complicated –
    Murdoch –
    THAT is it.
  7. ANDRIA
    Agree with HF. Fact-checking/crap detection needs a higher spot because the volume of misinformation is exploding.
    Polarized media, direct-from-source spin and just plain false information are increasing.
    Three kinds of fact checking:
    1. the corrective post, fixing wrong information from elsewhere:http://talkingpointsmemo.com/edblog/breitbart-issues-best-correction-since-forever
    2. the political fact check, often difficult to report and write and fraught with consequences:
    http://www.wral.com/fact-check-a-500-million-education-that-isn-t-quite-what-it-seems/13935936/
    3. the full site with a mission to correct false information:
    http://www.emergent.info/
    Part of this relates to product and presentation – a 140-character tweet can sometimes serve as a fact check or conversely serve as a way to obfuscate or cherry pick and spin numbers. Knowing what format works best and most efficiently to find and convey truth is a factor – as HF notes in his efforts to sift through a bunch of tweets.
  8. Hi,
    journalism is not only about content, it’s about sales (to advertisers or readers) and distribution.
    Few months ago I try to duplicate the “reverse publishing” model to subscription – and you can go the same way with ads.
    It was in French but Paul Bradshaw agreed to publish this idea on his blog : it should be easier to understand the “reverse subscription model” concept.
  9. You need something on the move from words to pictures – including the growing importance of video
  10. Thanks so much for this article and the comments. I’m looking at ways new journalism trends can improve and update the information management system of legislatures–so many of the trends and tools can be adapted for subject matter functions that occur in the policy process of Congress, for example. Super helpful. more please!
  11. I would add:
    - Segmentation of digital journalism in different products (Like The New York Times tried to do with NYT Opinion, which didn’t work, and NYT Now, which apparently worked). And consequently:
    - Several parallel subscription possibilities, like we also see in NYT, in order to reach new audiences.
    -Content curation. Both in closed reports such as Circa’s new “Wire”, or Flipboard’s Daily Edition, or Yahoo News Digest, or in articles organized “in a Buzzfeed way”.
    Great compilation, though.
  12. Possible additions and amendments. Making my nominations list public.
    * The loss of control over the systems by which journalists reach the public, as larger players — platforms and governments — get in the middle.
    * Unbundling of omnibus media packages.
    * Social media as a reporting tool, rather than an alternative source of content or a distribution engine.
    * The rise of fact-checking and rumor control as a key practice area.
    Also, see Steve Buttry’s annotated response to this post, where he adds a great many more sources.
  13. But what if journalism isn’t a separate profession but rather an everyday practice of the masses? “The jewel of the elites is in the hands of the laity (oh my).”
  14. I have the good fortune to teach not only introductory journalism and transmedia storytelling, but also a media-and-culture survey course. The latter course has left me marveling over the pace of technological change and the frankly underappreciated forces that will shortly disrupt the media landscape.
    Your digital course doesn’t appear to address virtual reality and augmented reality, and yet these will be a way of life in only a year or two. Oculus Rift will introduce a consumer version of its VR headset next year, and media industries such as gaming and film are scrambling to understand how they will incorporate it. News is still stuck on the concept of mobile-first. In contrast, Facebook bought Oculus because Mark Zuckerberg — and many other bright people like him — have concluded that mobile is about to give way to VR as the next interface.
    Holographic technology is closer than we think. Ostendo plans to introduce the first true holographic wristwatch next year. It will follow with smartphones with holographic capability.
    News organizations also are ignoring other technologies because they’re seen as irrelevant. As with Craigslist and social media, others will spot their relevance first. Here’s a short list:
    -3D printing
    -Gaming
    -Internet of Things
    -Streaming media
    -Sensory media (e.g., texting smells)
    -Wearable technology
    All of these things exist today. All of them are going become hugely pervasive. None of them are built into the fabric of news organizations.
    Bottom line: digital thinking has to incorporate the ability to skate not to where the puck is, but to where the puck will be.
    • Thanks, Steve.
      You’re right that I cut the innovation curve off at a certain point. For better or worse (seems you would argue worse) I concluded that while I could introduce, say, wearables, or 3D printing, I could not introduce those technologies as forces changing journalism (yet.)
  15. The concept of “forces” versus “skills” is key to this essay IMHO. It’s a critical distinction. It’s only later in your reply to comments that I came to understand that. Not to play “wordsmithy” but when you use the term “master” in the introduction, that sounded like skills mastery, not understanding forces.
    I would echo the point made by several others that a hierarchy of importance with “fact checking” at the top is important. It is critical in a meta-data world that is sometimes overwhelming to journalists as well as the people once known as the audience.
    Not to get too esoteric, but it’s like the acronym “canoe” which references personality traits. Do you choose the word canoe because conscientiousness is the priority. Or do you choose the word “ocean” because you consider openness the priority.
    Even after having read the piece eight or nine times, I still am digesting the ideas. But would suggest that included in all this is the key fact / concern/ reality that there is a blurring of lines between content creators and content curators.
  16. I offer three issues that you haven’t mentioned. Two are trends I see, the other is a wakeup call because it isn’t happening nearly enough.
    The two trends:
    INCREASING COMMUNITY ENGAGEMENT. Public media –http://www.theharwoodinstitute.org/wp-content/uploads/2012/07/WhyWereHere.pdf – has been at it for a while but it is also finding its way into other news organizations -http://www.engagementhub.org/enthusiasm_at_asne_apme_2014_to_grow_audience_through_engaging_communities. More about the concept is athttp://www.journalismthatmatters.net/the_engagement_principle.
    BRINGING A SOLUTIONS ORIENTATION TO NEWS. As Investigate West –http://www.invw.org/about – frames it, “We publish original stories that equip the public to participate in our democracy…” Including examples of how challenges have been addressed elsewhere increases the likelihood that people are inspired and able to get involved. The Solutions Journalism Network – http://solutionsjournalism.org/about/ – is introducing practices to news organizations around the country. More about the concept is athttp://www.journalismthatmatters.net/the_possibility_principle.
    The wakeup call:
    INCREASING DIVERSITY. What isn’t happening nearly enough is news that reflects the diversity of our communities. Michelle Ferrier’s research on Media Deserts –http://www.ohio.edu/scrippscollege/news-story.cfm?newsItem=4206444D-5056-A81E-8D0F2C3E36693F1D – highlights growing holes in local coverage, particularly in economically despressed communities. At 5.5%, not nearly enough hyperlocal online news sites are run by people of color. The absence of diversity in both the storytellers and the stories told leaves us with a partial perspective on issues that affect everyone. More on the concept is athttp://www.journalismthatmatters.net/the_diversity_principle.
    Peggy Holman, Executive Director, Journalism That Matters
  17. I wish I were more skilled in working with the comments. The broken link referencing other news organizations is
    http://goo.gl/FOcDOv
  18. FOMIL
    Here’s the reality: hardly anybody working in news will have time to read this. Meanwhile, young people are entering the workforce lacking basic journalism skills. Forests are nice, but they are made up of trees. Y’all in the academic world need to help people find jobs; this is a wonderful debate on campus I’m sure.