Opt-outs for desktop improvements on pilot wikis
Closed, ResolvedPublic

Description

Background

We would like to document the opt-out rates for the Vector 2022 skin on pilot wikis

Acceptance criteria

  • What percentage of users have opted out of the Vector 2022 skin on wikis where it is the default?
  • for all registered users
  • all registered users who edited in last 12 month
  • all registered users who edited at least 5 edits on its wiki in last 12 month
  • total
  • per pilot wiki

Event Timeline

ovasileva triaged this task as Medium priority.Sep 12 2022, 12:13 PM
ovasileva raised the priority of this task from Medium to High.
ovasileva moved this task from Incoming to Analyst Consultation on the Web-Team-Backlog board.
mpopov lowered the priority of this task from High to Medium.Sep 13 2022, 5:11 PM
mpopov edited projects, added Product-Analytics (Kanban); removed Product-Analytics.
mpopov subscribed.

Lowered the priority per our team prioritization guidelines (https://phabricator.wikimedia.org/project/profile/3313/)

@ovasileva, here is the summary of vector-2022 optout rate on pilot wikis. Let me know if you have any questions.

Vector 2022 skin opt-out rate of editors on pilot wikis where it is the default.

wiki_dbEditor (at least 1 edit)Active editors (at least 5 edits)Optout usersOptout rate of editors(percentage)Optout rate of active editors(percentage)
arywiki16136116.83%30.56%
bnwiki671119271402.09%7.27%
dewikivoyage487163469.45%28.22%
euwiki2685947762.83%8.03%
fawiki32631107755151.58%4.78%
foundationwiki673500%0%
frwiki940582666635813.81%13.43%
frwikiquote31785257.89%29.41%
frwiktionary191160723912.51%39.37%
hewiki1402547667025.01%14.73%
incubatorwiki141383900%0%
kowiki995436604354.37%11.89%
plwikinews127402116.54%52.50%
ptwiki535331285516713.12%13%
ptwikinews95261111.58%42.31%
ptwikiversity33412582.40%6.40%
srwiki400313162847.09%21.58%
thwiki407315212295.62%15.06%
trwiki20736541411855.71%21.89%
vecwiki159281911.95%67.86%
viwiki1224538266305.14%16.47%
viwikibooks57181628.07%88.89%

Note:

  • Opt-out rate of editors is defined as number of users who optout vector2022 out of editors who edited at least 1 edits or 5 edits on its wiki between 2021-11-01 and 2022-08-31.
  • All opt-out events were stored in mediawiki_skin_diff schema, which contains the events since 2021-11-01. The user id in user_hash field is hashed. The user will be double counted if the user appeared in 2 quarters and was hashed with different salts. Because we did not record real user ID, we cannot connect the users in mediawiki_skin_diff schema with the editors in mediawiki_history schema. The user who opted out vector 2022 might not be an editor. The opt-out rate of editors is overestimated.

Vector 2022 skin opt-out rate of registered users on pilot wikis where it is the default.

wiki_dbnum registered usersoptout usersoptout rate(percentage)
arywiki6467110.17%
bnwiki3865871400.04%
dewikivoyage47204460.10%
euwiki138883760.05%
fawiki11458065150.04%
foundationwiki5132700%
frwiki445434735810.08%
frwikiquote69789250.04%
frwiktionary3332322390.07%
hewiki9999767020.07%
incubatorwiki257916400%
kowiki7502974350.06%
plwikinews18507210.11%
ptwiki279865216710.06%
ptwikinews25993110.04%
ptwikiversity3554180.02%
srwiki3112622840.09%
thwiki4309052290.05%
trwiki142247011850.08%
vecwiki32135190.06%
viwiki8783646300.07%
viwikibooks16528160.10%

Note:

  • All registered users were pulled from mediawiki_user schema.
  • Please note that registered users include a large proportion of users that are not active and may not have visited the wikipedia site while the vector-2022 has been available.

The 'optout rate (percentage)' column is misleading as it does not reflect the percentage of users that opted out from the number of users that tried Vector2022, but is rather the proportion of users who opted out / total number of users (however that is defined, either active or registered). So I would say that these numbers are underestimated, if you want to know what is the percentage of people that tried the new skin and then opted out. You need to have the number of people who tried.

Hi @GoEThe - the majority of these wikis have had the Vector 2022 skin as the default for 1+ years at the time this data was taken (some large wikis like French and Portuguese Wikipedia have had the new skin for 2+ years at the time this data was analyzed). This means that editors (who have 1+ edit in the previous year) and active editors (with 5+ edits in the previous year) will have seen the new default during this time in order to make their selection.

Fair enough, I misunderstood what the columns meant. Thank you for taking the time to reply.