Research:Wikinews original reporting value as a measure of news events

Add links
From Meta, a Wikimedia project coordination wiki
Contact
Laura Hale
This page documents a completed research project.


Project Summary[edit]

One of the methods used by media organizations to assess the value of covering or prominently featuring a particular story is News events/News criteria. Using 22 different variables, an organization can then make an informed decision regarding the relative location and prominence of a story.

This analysis looked at the 32 original news reporting events that were self reported by Wikinews contributors in terms of the money spent covering a story. This information is compared to the total number of news events points the Wikinews story received via self-assessment as a first step in trying to determine the cost benefit of original reporting.

The results show the news event criteria do not align with how Wikinews conveys news. The results also show that past a certain amount of around €300 spent, volunteers are more likely to spend that much money on higher value news event stories, especially when it comes to sports.

Background[edit]

One of the goals of The Wikinewsie Group is to support original reporters in the production of high quality, original reporting. In a professional news model, this entails moving away from a freelance model to a stringer model.[1] To make this move, the original reporting picture needs to be better understood in terms of potential resource allocation. Eventually. a cost-benefits analysis model can be completed for supporting reporting projects, and leadership and reporters themselves can begin to develop Return On Investment analysis to understand benefits gained from reporting with a goal towards continual improvement.

The first step in the efforts to professionalize our projects has been to complete two pieces of research as part of a larger effort to work on program design and evaluation: English Wikinews Cohort Review Analysis and Wikinews Content Import Community Effect Analysis. These two pieces of research were largely done to better understand some of the community activities and their impact on contributor output. Neither one lays a framework for a Cost-Benefit Analysis or assists in measuring ROI, though they do provide understanding that will be fundamentally important in framing future research. They also give an idea as to the type of metrics available, ideas on research design, thinking about privacy issues, etc.

The research presented here is the first step in developing full length reports on Wikinews’s news reach, writing a cost-benefit analysis and ultimately in working with the wider Wikinews community to write a project-based strategic plan going forward.

Purpose[edit]

The purpose of this research is to examine the relative news value[2] for original reporting projects, and assess the alignment of their value against user reported costs for doing that reporting and Wikinews reporting priorities. This information can be used as the first stage in creating a formula or decision modelling tree when it comes to prioritizing The Wikinewsie Group objectives in terms of how to best support Wikinews journalists engaged in original reporting. It can also then be used to:

  • encourage others to donate towards an original reporter's costs to cover higher news value topics;
  • develop action plans to support middle news value projects;
  • better promote middle and lower news value reporting;
  • develop educational support materials specifically aimed at lower and middle news value reporting;
  • create plans to reduce costs for middle and lower news value projects; and
  • improve greater potential return on investment outside the article itself through social media and other promotional channels to maximize news reach.

This information can also be used to determine where traditional news values align with Wikinews project objectives and the citizen journalism related reporting objectives discussed by Jimmy Wales, founder of Wikipedia, at Wikimania 2013.

Methodology[edit]

To do this research, a survey was created asking Wikinews journalists engaged in original reporting what the cost of their work was.[3] A request was posted on talk pages of all reporters identified by a bot created by Gryllida as having completed original reporting in the past three months.[4] Additionally, messages were posted to Wikinews related mailing lists and requests were made on IRC where people who have engaged in original reporting were invited to complete the survey. There was an estimated 10 to 20% response rate.[5] The responses were put into a table, with a column added to determine the general news category[6] and two columns where spending rates were converted to Euros and Australian dollars.[7]

The survey results for reporting events may have included multiple Wikinews articles, and other non-Wikinews related Wikimedia content development. Some of it included non-English reporting. This means that in assessing potential value one-to-one results are not feasible. One reporting event included e-mail interviews that resulted in the publishing of 5 articles. Another event included producing 75 articles. A third reporting event included 25 articles in two languages. A fourth reporting event included one article in one language.

Following this, all the news events were put on a grid with the news events category.[8] If an original reporting series was viewed as meeting a criteria, it was given 1 point. If a series was viewed as not meeting a criteria, it was given a -1. If it was unclear if a series met a criteria or the series contained articles that could be given both a 1 and -1, it was given a 0. In some cases, additional opinions were sought to assess number value from others who were more knowledgeable about the completed reporting or to assist in understanding what a category definition meant. Following all 32 series being assessed against the 17 new event criteria, the total points were added for a maximum of 17 points.

In doing the news event point assessment, there were some issues in aligning the news events with Wikinews because all the articles were previously published, because Wikinews relies on volunteer driven news reporting, and because Wikinews’s approach to news reporting does not align with the traditional model. This methodological related problem will be dealt with later in this report.

Results[edit]

The most popular category of reported original reporting where money was spent is sports, accounting for 18 of the total original reporting events. The next most popular is politics with 3, local news and culture with 2 each, and popular culture, weather, technology, health and journalism each having 1 reporting event.[9] Across these groups, politics had the highest news events scoring, with an average of 10 points per event. This was followed up by sports and popular culture, which average 9 points. Next is weather and health with 7 points. Technology and journalism have 4 points. Culture trails with 1.5 points and local news finishes last with 0.5 points.

The news values are designed to maximize in efforts in promoting articles for maximum exposure and return on investment. Thus, it is not a surprise that local news finishes so low.[10] It cannot get points, generally and in the specific Wikinews cases, for prominent individuals, for unexpectedness, unambiguity, meaningfulness, personalization, competition and co-option. Because of the reliance on Wikinews reporters, it fails on frequency because of lack of original reporters and because there are often a lack of sources to supplement this with synthesis reporting, which in the case of English Wikinews requires two independent sources verifying the newsworthy event. News values may not be properly assessing the value of local news’s potential importance to Wikinews.

The nature of Wikinews reporting also rewards some news values by default. This includes unambiguity because articles should be factually clear and not open to interpretation. Some of this is forced into the project, at least in terms of English Wikinews, by requiring reporting to be about a specific recent news event.

Composition is another area where Wikinews is rewarded by default. With the exception of Spanish Wikinews which selectively features articles on the main page and is inconsistent in their promotion of content on social media, most Wikinews projects give equal space to every news article.[11] This is a result of a variety of factors, of which the most pressing one at the moment is the lack of daily article production which might push for a need for editorial decision making on the front page.[12]

Competition is another area where Wikinews is largely rewarded, specifically as it pertains to synthesis reporting which requires use of other new organizations articles as sources or in sport reporting, where there is usually at least one local news organization writing about a team. In the latter case, the competing news organization may be competing but often takes a completely different perspective on an article or gives different space priorities to reporting on an event.[13]

Co-optation is another area where Wikinews has a nominal ability to excel. The projects' lower profile means it is more difficult for many reporters to get interviews with major players in a news story, or get access to and attend things like press briefings. This presents a challenge for some original reporters to think of another way to approach a story. On Wikinews, this can be done by interviewing academic subject experts on a geo-political or science topic and produce a full interview.

Some categories for news events are ones in which the nature of Wikinews appears to punish categories of original reporting by default. This includes negativity, that is reporting on bad news. Most original reporting is original sports reporting, which by its nature tends to not focus on negatives. The negative stories that got points included two interviews where the negative aspects were buried in story and focused on bad news about people or things connected to the interview subject. The remaining included a political story, a weather story and a local news story. Given the current limited scope of accessibility to major international press briefings and broader access to business and political interview subjects, coupled with the project´s focus away from investigative journalism, improving on this news event category is unlikely to happen and will likely largely remain the domain of synthesis reporting.

Conflict has similar issues to negativity. Outside of sports, where conflict is often built into the story of one team in conflict with another, many original reporting events had -1s.

With this in mind, the correlation between a specific original reporting event and the total money spent is 0.35 and indicates there appears to be little correlation between the news event score and money spent. The correlation is almost to the point of being completely random. [14]

Excluding all reporting with €0 cost, graph shows cost per original reporting event against the total news event points based on category of reporting.

With too few data points for most categories, looking at the correlation between spending and news event is impossible. For sports, the correlation is 0.424, which suggests a small relationship where more money spent correlates to the higher news event points. Politics has a much stronger correlation at 0.964 for its three reporting events, though it is worth noting the maximum outlay was €9.36 and the minimum was €0.

Another way of looking at this data is to break it down into quartiles. The bottom 25% quartile is €3.45, the 50% quartile is €16.49, the 75% quartile is €205.37 and the final quartile is €11045.90. The bottom 25% has a news event score range of 0 to 10, an average of 5.5 and a median of 8. Original reporting that costs no money for the reporter to produce is not indicative of either a potential for strong news event values or low news event values, though it does overall suggest that with a top of 22 points and a maximum for all reporting of of 14, no-cost reporting does not top out in terms of potential news event maximization.

The 50% spending quartile has a range of 2 to 14. The median news event score is 7 and the average is 7.5. The correlation inside this dataset is -0.08, which suggests true randomness in terms of money spent and the news event points. Overall, this suggests that a few Euros can potentially lead to the coverage of higher new event point scoring stories.

The 75% quartile has a news event points range of 0 to 10. It has a median of 7 and an average of 6.5. The correlation between money spent and news points is -0.06. Like the previous, it suggests a few Euros can lead to the ability to cover higher new event point scoring stories, though spending money on reporting is not always indicative of a high scoring story. €121.67 was spent on one event with 0 news event points.

The top quartile, where the total spent exceeds €300, has a news point range of 4 to 14. The median is 9.5 and the average is 8.8. Overall, the top quartile spending outperforms all other quartiles for being spent on higher value news event stories. This to a degree is supported with a correlation of 0.54, which suggests the more money spent, the higher the news event points. With one exception, these stories are also all sport related stories.

Overall, the results show the news event criteria do not align with how Wikinews conveys news. The results also show that past around €300 spent, volunteers are more likely to spend that much money on higher value news event stories, especially when it comes to sports.

Conclusion[edit]

Wikinews exists as a volunteer driven project with no direct funding. The project's user understood mission is to provide a high quality, reliable archive of newsworthy and entertaining stories that can be used as a freely available historical archive of the documented events.[15] Wikinews neither directly aligns with the goals of traditional media, nor directly with the goals of Wikipedia. It creates a strange intersection for trying to do action oriented research with the aim of ultimately getting funding to support original reporting.

Maintenance is done by the Wikimedia Foundation, whose strategic priorities are aimed at more visible projects like English Wikipedia and geographic areas like the United States, Brazil and China. The potential impact of any Wikinews work would be viewed as low. Thus, the lack of support. Direct dollars to volunteers and staff time spent working specifically on Wikinews related technology are low. As a sister project, when dealing with the Wikimedia Foundation, any and all metrics need to be done with the intention of requesting grant funding, staff allocation of resources, etc. using metrics that the Wikimedia Foundation has determined are important. These metrics, specifically total user edits, total bytes added by users, and page views, are not necessarily compatible with Wikinews in terms of how the project works and in the context of a broader news reporting environment. Aligning research practices to directly match the Wikimedia Foundation data input would leave The Wikinewsie Group at a distinct disadvantage when applying for funding through other organizations.

Professional journalism, and to a larger extent journalism efforts run by non-profits where reporters often do get some form of compensation, relies on its own set of metrics. News metrics do not align with Wikimedia Foundation metrics. One of the best examples when applying for grants and media accreditation is the a demonstration of news reach. This metric is not derived from page views alone, but also includes demonstrating how many people are aware of the story and identifying key influencers who were aware of the story. Essentially, it is about bragging about the overall positive outcomes attached to a news story that exist outside of mere article production. Professional journalism is also interested in news events and their competitive place in the news marketplace. Thus, there are other professional journalism metrics like news events/news criteria. At the same time, the rise in digital journalism has changed the traditional metrics inside in the industry and forced alignment between those metrics and consumer driven metrics, which are at times viewed as potentially harmful to traditional journalism.[16]Given the unique nature of Wikinews, journalism specific metrics do not fundamentally align and provide a clear alternative path to potential funding by allowing The Wikinewsie Group to eschew the Wikimedia Foundation's metric related funding requirements.

Balance needs to be found and a methodology needs to be created that provides balance between the two competing funding models. At the same time, The Wikinewsie Group needs to be able to meet that balance while still working towards its core mission.

In using news events/news criteria to assess Wikinews, care needs to be taken because these criteria do not fundamentally align with how Wikinews works as a citizen run archive of news stories that anyone can contribute to. News organizations are fundamentally concerned with driving traffic to their landing pages and getting views. This is not the core mission of Wikinews, which according to the mission statement is “To present up-to-date, relevant, newsworthy and entertaining content without bias”.[17]

Volunteers generally appear to possess a degree of “news sense” in spending on reporting that has higher news value: Very large sums are not spent on stories that have few news event points.

Reporters should work with the community if they are planning to spend large sums of money on stories with higher news value to maximize the reporting output and get the most monetary value out of these higher value stories. Infrastructure should be better developed inside the community to promote these stories and encourage the community to engage in them, while at the same time continuing to produce lower point valued stories.

Given that this assessment model favours stories that will likely be covered in depth by other news organizations, it might be worth considering how to better support and promote lower valued stories as there is less competition in the news marketplace for Wikinews, and it is a potential growth area. This includes developing materials on covering local crime stories, local sports, and local news. It also involves creating materials and developing infrastructure to cover stories outside about elite nations as another potential growth area. Both objectives may fit well into the Wikimedia Foundation’s goals of increasing diversity of content and the editor base.

While going after lower event stories at less cost, higher point stories at greater expense should still be sought to provide a diverse global news perspective on these topics that may not otherwise be presented. This should be the take away in developing a decision-making tree regarding potential funding decisions.

References[edit]

  1. ↑ For a good discussion of the various costs, benefits and drawbacks for the freelance, stringer and bureau model, please read The Need for Foreign Correspondents: A Cost Benefit Analysis by Haley Petersen . The Wikinewsie Group’s geographic dispersion and lack of reporter asset centralization is a key to understanding TWG and Wikinews project from a traditional news reporting perspective. In this way, every reporter for Wikinews should be understood from the perspective that every reporter is international.
  2. ↑ A wikipedia article about news values can be found here.
  3. ↑ The survey can be found here.
  4. ↑ The list of reporters can be found here and here.
  5. ↑ There were 12 responses, with an estimated 59 reporters, which includes two IP addresses, across 10 different language projects having done original reporting in the past 3 months. The responses included some people who wrote produced original reporting outside the 3 month window.
  6. ↑ This includes sports, politics, popular culture, technology, journalism, culture, weather, and local news.
  7. ↑ These two currencies were selected as normative currencies for discussion purposes as they accounted for over half the total funds spent, and over half the original reporting events having funds spent using one of these two currencies.
  8. ↑ The news event categories are subjective. This would potentially impact the repeatability of this analysis if this stage is repeated using a different person assessing point values.
  9. ↑ Some reporting events occur in two categories. Hence while there are 32 total reporting events, the total number by category is greater.
  10. ↑ This is even less surprising if one considers the potential lack of ability to effectively monetize local news. In August 2013, Patch, an AOL company specializing in local news production, laid off 500 employees.
  11. ↑ This is not a function of Wikinoticias placing editorial judgement in most cases, as evidenced by @wikinoticias on Twitter which gives an overwhelming amount of space to news reports by competing news organizations at the expense of linking to their own project reports. They have a large Twitter following, but are incapable of converting this audience into Wikinoticias readers, even as they are capable of getting 5 retweets for a news article by the Huffington Post.
  12. ↑ While the front page would require editorial decision making, this same case would not exist for social media as many Wikinews projects including English, French, Tamil, Catalan, and Russian Wikinews automatically push their news stories out to Twitter and or Facebook.
  13. ↑ One example of news competing involves the 2012 Summer Paralympics. The USA rights holders had a single individual attending the event, relied upon LOCOG for media feeds, and had television reports on the Paralympics in pieces that ran after the conclusion of the Games. A London newspaper had at least eight reporters, two columnists and four photographers covering the Games producing a whole special section in the newspaper each day. An equestrian magazine had a reporter and photographer attending the Games but only at equestrian events and nowhere else. Wikinews had two reporters who produced roughly 75 stories and shared over 300 images in the leadup to, during and shortly after the conclusion of the Games.
  14. ↑ It is very important to understand the total volume of articles published, the languages the articles were published in, the number of Wikipedia and related articles updated, total page views generated and the number of photographs uploaded were not included in this. If a weighting system was devise, the results might differ.
  15. ↑ https://en.wikinews.org/w/index.php?title=User_talk:Jimbo_Wales&oldid=1981358
  16. ↑ An example of this would be a push towards page views and time spent on site. As such, more time and resources are spent on sports and popular culture than on say political news. This is one of the primary reasons why one Australian newspaper is one of the biggest competitors in fantasy sport for its national leagues, and why ESPN spends a lot of resources catering specifically to that audience. Sometimes, but not always, these market driven decisions result in profits being re-allocated back to less consumer media driven areas like local news.
  17. ↑ https://en.wikinews.org/w/index.php?title=Wikinews:Mission_statement&oldid=1886946


External links[edit]

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Research:Wikinews_original_reporting_value_as_a_measure_of_news_events&oldid=5828888"