New User Revert Rate Reporting

https://phabricator.wikimedia.org/T245560

Edits made on Commons by users by Registration date post 2019-11-26

(not by version 2.7.50305+)

For users of Android app version 2.7.50305 (and above) the previously implemented 'gatekeeping test' that funnelled users new to Suggested Edits was removed, we want to see if that removal increased the amount of reverted edits to Commons captions and descriptions. Because we weren't writing Commons edit rev_ids to wikimobileappedit we can't verify the users by app version (that will be fixed in next update). The data in this report consists of users who registered new accounts after the 2.7.50305 version release.

Revert rate user profiling:

How many and what % of users who have edited on SE after app versions released 11-26-19 and later are in each percentage bucket of revert rate (e.g., 0-1%, 1-2%, etc up to 7+%)

unique user count percent of total revert uniques percent of total new editors
Rate bin
Revert rate < 1% 0 0.0% 0.0%
Revert rate 1% 1 0.3% 0.0%
Revert rate 2% 2 0.6% 0.0%
Revert rate 3% 3 0.9% 0.0%
Revert rate 4% 1 0.3% 0.0%
Revert rate 5% 1 0.3% 0.0%
Revert rate 6% 0 0.0% 0.0%
Revert rate 7% 1 0.3% 0.0%
8% and higher 268 81.7% 2.6%

Of users who registered after 11-26

Average daily revert rate for prior version was 1.71%
Average daily revert rate for new registration users only is 5.69% Of reverters 56% made just 1 edit, 1 revert
Actual edit counts per unique didn't exceed 14, except for one user w/111 edits and 115 reverts (may be using old version?)

Commons reverter uniques: 328
Commons editors total uniques: 4157 Reverting uniques are 7.89% of new registered users

Prior versions data:
Commons editors active between 08-07-2019 and 11-26-2019 total uniques: 6199
Commons reverter uniques: 290
Reverting uniques were 4.68% of registered users

Notes from ticket: We see from this report that the baseline revert rate for image caption edits has increased since late November. What % of that increase is driven by people who are new to using SE - meaning, those whose first SE edit is after 25 November 2019, using a version of the app released after that date? (Basically, are the n00bs we're attracting qualitatively worse than the old people?)

The ultimate idea here is to try to figure out whether to reset the quality thresholds lower - and if so, to what level. We know from the Commons folks that it takes a lot for an edit to actually get reverted, so our thresholds probably need to be more sensitive. Since we didn't set them using very robust data in the first place, now's the opportunity to recalibrate.