New Editor Experiences research presentation for Czech Wikiconference: transcript

Key points

• New editor numbers on Czech Wikipedia grew until about 2010 and then flattened or declined, and the visual editor did not increase how much new editors edited.
• In 2017 the team interviewed 64 new editors, 34 of them in the Czech Republic, with the rest in Korea.
• Stories like Lenka and Jan showed varied motivations, and the findings were grouped into six personas such as the knowledge sharer, reactive corrector and social changer.
• New editors are often confused about policies, talk pages and tools, and learn best gradually, as in Czech seniors' courses.
• The tone of feedback strongly affects whether new editors continue, with thanks encouraging them and harsh comments driving them off.
• Draft priorities focus on understanding the model and policies, finding the community, and progressive pathways to editing, with over 50 ideas so far.

Chapters

1. 0:00Why new editors matter and the decline in joining
2. 1:07Wikitext, the visual editor and the need for research
3. 2:13Design research in the Czech Republic and Korea
4. 4:13Stories of new editors Lenka and Jan
5. 7:08Analysis and the 11 findings
6. 8:45Motivations and personas of new editors
7. 10:08Confusion, gradual learning and policies
8. 12:42Tools, talk pages and the tone of feedback
9. 14:28Next steps, priorities and ideas
10. 18:02Invitation to collaborate

Full transcript

0:00Hello to the Czech Wiki Conference 2017. I'm Neil Patelkwin. I'm a Product Analyst at the Wikimedia Foundation, the nonprofit that supports Wikipedia. And I'm Abbie Rebstra, Lead Design Researcher for the Wikimedia Foundation. And today we're going to talk to you about new editors. Every community ultimately depends on new members for its survival, both to carry on the existing work of the community, but also to refresh the community with new perspectives and new ideas. But unfortunately, in Wikipedia, this is as true as it is anywhere else.

0:32But unfortunately, in Wikipedia, there's a bit of a problem. This is a graph showing the number of new editors joining the Czech Wikipedia every month since 2006. And it tells about the same story that the same graph for any other Wikipedia would. Strong growth up until about 2010, then flatness, and maybe even in recent years, a decline. So this is a concern that the Wikimedia movement has been thinking about for a very long time. And many researchers and community leaders and software engineers have thought about ways to fix this.

1:07But it can be very hard to actually know what the most important and most central barriers for new editors actually are. And here's one example of this. Up until very recently, any editor, new or experienced, who tried to edit a Wikipedia page was confronted with this, a very complex mix of symbols that's called wiki text. And if you're an experienced editor, it may be simple and easy to deal with, but for new editors, it can be very intimidating. And so people concluded, understandably, that this was a major barrier for new editors and decided to work on a way to fix this.

1:41So after, so the Wikimedia Foundation set out to build a much more visual, intuitive way of editing Wikipedia. And after many years of development, we came out with this, the visual editor. And if you ask any new Wikipedia editor, they'll say this is so much better than the old experience, they couldn't imagine trying to start editing using the old experience. But when we actually experimented, we found out that when we gave new editors this visual editor in place of the old source editor, they didn't actually edit anymore.

2:13So even with all the work that's been done, and even as we improved the experience for new editors with things like the visual editor, there's still a lot we really don't understand about new editors. And one of the most important tools we have for learning about this is design research, where we go out and interview new editors face-to-face. It's logistically complex and time-consuming, but it allows us to learn things we couldn't any other way. To see all the complexity of the individual stories of new editors and to see the environments they live and work in.

2:46It lets us learn about things like what they do for work and play in their lives and why they do it, about how they first found out about Wikipedia, how they use it, how they first learned they could edit, and what their experience trying to actually edit was, and how the people and software they encountered shaped whether they continued to edit or didn't. So, in early 2017, the Foundation decided to make a major investment in design research on new editors. We chose to focus this research on the Czech and Korean Wikipedias.

3:17They're both mid-sized Wikipedias with about 2,000 editors per month, which is a group of Wikis that we know less about, that are less studied than the biggest Wikis like English and German. And also, they're very different Wikis, so it allows us to see what patterns of new editor experiences are common in very different Wikis. We spent two weeks in each country with a team of foundation staff, design researchers, and local experts. In the Czech Republic we mostly interviewed in Prague and Brno with side trips to a few smaller towns. And it's easy to recruit experienced editors who are very invested in Wikipedia and eager to share their experiences, but it's hard to recruit new editors, particularly those who may have given up on editing entirely. So we use the email addresses they gave when they registered to contact them and invite them to participate. In total we were able to interview 64 new inexperienced editors, 34 of them in the Czech Republic.

4:13So as we conducted more interviews and the research, we heard many stories. We observed people using their technology and the internet. And for example, we met Lenka, who lives in Brno into the Czech Republic. She works as a customer assistant and she travels with friends and does many interesting things. Lenka is a voracious learner and she organizes a book club also. She researches a lot before travel and shares what she learns. She contributes to TripAdvisor, which is a travel review website.

4:47Lenka likes contributing to TripAdvisor because it's easy to use and she enjoys seeing how many people viewed a picture she posted or that they say that the review she wrote was helpful to them in their decision making. There's a dashboard that describes Lenka's activities and she can easily see that. So having access to information is very important to Lenka. She reads Wikipedia and also she's a new editor at Wikipedia now. She first learned to edit Wikipedia when she went to check out the Senior Wiki Club at the Bruno Public Library for her mom who wanted to meet some new people.

5:27So Lenka met new people there and she also learned a lot about contributing to Wikipedia. She practices editing in her sandbox, and she's created an article on English Wikipedia about an Indian movie that she liked. Since her first WikiClub, she's edited a little bit, but she hasn't continued. We also met Jan. He's an organist who started playing the organ in a church when he was 14 years old. He supported the restoration of the organ in his church in the 1990s after communism and still maintains the organ regularly. Jan avidly writes and collects documents to detailed history of the church, the church's organ, frescoes and restoration, as well as the area around the church. He has a table of printed documents for visitors for them to gather more information and learn about the church and the organ. In 2007, the former priest of the church suggested that Jan, quote,

6:32shouldn't just write on the shelf about the church and the organ, but he should put it on Wikipedia. Jan agreed and he began contributing to Wikipedia. He edits first in a sandbox getting help when needed from his mentor, the priest who inspired him to contribute to Wikipedia. At the time we talked with Jan, he had never discussed his edits with other editors on Wikipedia. He notices modifications to his contributions, and it seems to him that the edits are improving the content and the articles he contributes to.

7:08He doesn't see the point of contacting people who modify his edits. Since he started writing on Wikipedia, they've had some visitors at the church who said they heard about or read about the church through Wikipedia so that's nice so you might ask what is all this data mean how do we figure out all these stories what are what is the data telling us as we heard more stories and we completed interviews we did analysis every few days about to learn to think about what we're learning what are we learning what patterns are we seeing in the stories that we're hearing what kind of goals and motivations to our participants have and what are they trying to accomplish? Are they succeeding? Are there challenges that they're having? What's easy about what they're trying to do? So after weekly day-long analysis sessions and two-day workshops, once we completed all our data collecting and interviewing, we finished analysis

8:09observing the patterns that we saw in the stories and we learned what the data was telling us. The research then authored and created the final report and personas, which we translated into Czech and Korean, and we've shared it around. We've had several discussions with the community and posted it on wikis, so we're sharing it broadly now. So, out of all this research and analysis, we came up with 11 findings that were most important for understanding the experiences of new editors.

8:45And we can and have spent hours talking about them, but here we'll just give a brief overview of some of the most important. First, we found that new editors have very diverse motivations for editing, many of which serve purposes outside of the confediate. To make sense of all this diversity of motivations, we grouped them into six major archetypes, or personas as they're called in design research for example here are three that we came up with first there's the knowledge share somebody who's very who cares a lot about a topic and is interested in sharing their knowledge on that topic with the world and is particularly interested in doing so on wikipedia for a particular reason whether that's because they want to get back to a site that they felt they've learned a lot from, or they admire its democratic model of creating knowledge.

9:31There's also the reactive corrector, someone who reads Wikipedia a lot and derives some satisfaction from fixing obvious errors they find in reading, but doesn't have a larger mission for editing and may not know a lot about how Wikipedia works. Finally, there's also the social changer, somebody who edits Wikipedia out of a sense of activism, a desire to correct a larger problem they see in society. For example, this might be an environmentalist who edits because she wants to increase awareness of environmental topics. We also found that new editors are frequently confused about how Wikipedia works and separated from its community.

10:08For example, even though they've successfully edited, they may not actually know that amateur edits are encouraged on Wikipedia. And they may not know that collaborative discussions and social interactions are a big part of Wikipedia in addition to individual editing, and so they may not know that there's an opportunity to join the community and become a more integral part of Wikipedia. For example, we talked to Markeva, who's a barista in Pong. She's an occasional editor, and she's made, over about six years, she's made 15 to 20 edits on topics she's interested in, namely fandom and music and film.

10:41And she knows about a lot of things to do with it, she knows about the difference between the visual editor and the source editor, she knows about history pages, and she knows about the copyright policy. But even with all that, she doesn't know about talk pages. And when we actually asked her, if you wanted to, how would you try to contact another editor on Wikipedia, someone you saw on a history page, for example, she said she would take their username and try searching it on Facebook to see if she could find their profile there and interact with them that way. We also found that new editors benefit a lot from progressive, gradual learning over time when they learn one small thing, and then later another small thing, rather than trying to do a big thing like creating a large article all at once.

11:22This is something that the seniors at Wikipedia courses in the Czech Republic do particularly well. Rather than trying to teach seniors everything about editing in one long session, they do it in weekly courses that allow the seniors to gradually build their contribution skills over time. It will also come as no surprise to any experienced editor on Wikipedia that new editors struggle a lot with Wikipedia's policies. It may be that they know there's a policy, but feel helpless to actually understand it, perhaps because they found the policy page, but find it too dense and complicated to actually understand.

11:58Or it might be that they don't even know that there's a policy that covers something that they're trying to do. For example, as an experienced Wikipedia editor myself, when I see somebody on Wikipedia trying to create an article about their business. I think this is a spanner who probably does it even though we know better. But, in our research, we talked to a number of editors who honestly thought that Wikipedia was a directory, and they thought, I have profiles on, say, Twitter and Facebook, so I thought I should have one on Wikipedia as well. And even after their edits were reverted or deleted, they weren't completely sure why. We found that new editors also have trouble discovering and using editing tools. When new editors get the visual editor, they tend to find intuitive and they tend to find editing pretty easy. But there are a lot of other tools on Wikipedia.

12:42We saw editors struggling to log in because they were trying to use their email addresses rather than their username. And we saw editors struggling to understand how you were supposed to use talkpages. For example, we talked to one group of architects who were trying to create an article about their firm, which was deleted for being too promotional. And they wanted to participate in the discussion, but the way they had found out about that discussion was by receiving a notification email when somebody had written on their talkpage. So they decided to reply to that email and they didn't know why nobody had responded when it was because you can't reply to notification emails, you have to go right on your top page and they didn't know. Finally we found that new editors are very strongly affected by the tone of the feedback they received. We talked to new editors who were pleased and encouraged as new editors because they received thanks from the thanks button or they received constructive feedback or a welcome message talking specifically about the things they were trying to do and congratulating them for.

13:42On the other hand, we also found that new editors were easily discouraged by negative feedback. For example, we talked to one new editor in the Czech Republic who had started gradually doing a lot of copy editing over time and was becoming more confident. So they finally decided to create a new page by translating a Zewarticle to the English. But after they spent a lot of time doing it and a lot of time figuring out, for example, how to put an infobox in the article, They got the following piece of feedback in the edit comment of another author, and people understand Chek will be able to understand this better than I can, but the new editor told us that this was very discouraging and said, I'm definitely never going to edit another zoology article again.

14:28So we're using the research findings and personas as a guide as we work towards accomplishing our goals of attracting and retaining more new editors in medium-sized wikipedias. But changing things in the wikimedia movement is complex and it takes time. Each community is self-governing and has their own culture and processes. So we're taking care and the time it takes to work inclusively, collaboratively, and as openly as possible.

15:02Here are a few things that we've accomplished so far. So we brought the Wikimedia Foundation team together which consists of 17 people from 17 across the foundation. We're experimenting and learning as we go about how to conduct our workshops fully remotely and across time zones. So far we've completed three fully remote, four-hour workshops and are continuing to learn and iterate how to work in this way together.

15:34We also have prioritized which findings to focus on. We've reached out and worked with the Korean and Czech Wikimedia communities to learn your priorities. So including the Czech and Korean priorities and the Wikimedia Foundation team's priorities, we collaboratively decided on a draft set of findings to focus on. Currently, we're still checking with some more stakeholders, so there are draft priorities for the moment. So, the first focus includes finding 5 and finding 8, which grouped together describe how new editors struggle to understand Wikipedia's model and policies, and have difficult finding and interacting with the community of editors.

16:19The second focus is about progressive pathways to editing, informed by findings 7 and 9, which, when grouped, they describe how successful editors build their contribution skills through iterative progressive learning and how Wikipedia's editing mechanisms are unintuitive for new editors. So we also started brainstorming around that first set of findings, and we came up with over 50 ideas.

16:50Some of these ideas were already existing or came from conversations with communities and media staff, and some were created in our workshop. Since we're working with a multidisciplinary team across departments, the ideas include things like product functionality, programs, ideas, ways to communicate more effectively, and other ideas from various perspectives and expertise. So here's an example. After a contribution and before it goes public, a new editor is asked to check out how well the contribution aligns with the core principles of Wikipedia.

17:30Editors can publish anyway, but surfacing the core principles and alerting new editors of their importance will help them understand and then apply the principles. We have more brainstorming and gathering of ideas to do, and later we'll evaluate all the ideas to determine which ideas we should and can work on and start experimenting with. We will test and we'll take our time to work collaboratively and as openly as we can as we go forward.

18:02So, do you have ideas to contribute or perspectives we should know about? Might you be interested in creating and testing new functionality in the future as we start to experiment? We'll reach out to you for specific collaborations, and we always welcome communication and people reaching out to collaborate with us. You can email us. Here are mine and Neil's emails on this page, and also you can connect with us on the New Editor Experiences Media Key page here on the talk page.

18:34So, reach out and connect and we look forward to working with you and welcoming new editors to Wikipedia together. Thank you. Thank you.

Source and licence

Recording: File:New Editor Experiences presentation for Czech Wikiconference 2017.webm by Neil Patel Quinn and Abbey Ripstra (2017-12-02), via Wikimedia Commons. Licence: CC BY-SA 3.0. The transcript, summary and chapters on this page are shared under the same licence (CC BY-SA 3.0).

The transcript is machine speech-to-text by the TidyTools Audio & video transcriber; the summary, key points and chapters were written by Claude from that transcript. Both may contain errors: check the recording before quoting it.