21:01:02 #startmeeting RFC meeting 21:01:02 Meeting started Wed Oct 5 21:01:02 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:02 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:01:02 The meeting name has been set to 'rfc_meeting' 21:01:05 good afternoon 21:01:18 #topic RFC: Future of magic links | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) |​ Logs: https://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ 21:01:54 #link https://phabricator.wikimedia.org/T145604 21:02:08 #link https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links 21:02:49 hi everyone! hi legokt^H^H^H^H^H^Hcscott :-) 21:03:32 i will be serving as your legokt today 21:04:28 there are three steps we discussed that are summarized in the description of T145604 21:04:29 T145604: [RfC] Future of magic links - https://phabricator.wikimedia.org/T145604 21:05:04 Hi 21:05:12 step 1: Disable the magic link functionality by default for the MediaWiki 1.28 release, and mark it as deprecated. 21:05:30 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration 21:05:30 3. Disable magic links functionality a year or so after the MediaWiki 1.28 release (in time for the next MediaWiki LTS release) 21:05:50 is there anyone that objects to step 1? 21:06:34 is legoktm online? 21:06:43 maybe we should find out who's here? (that is, actively participating in this discussion.) 21:07:03 TimStarling: no, legoktm is in class, so I'm being his surrogate for this meeting. 21:07:16 legoktm is at a class right now, so if there are objections or things he needs to clarify, we'll need to plan a followup 21:08:02 (Hi CScott - I'm reading along, typing occasionally) 21:08:05 it seems, though, that this is a "yeah, we should just do this" kind of thing for step 1 21:08:08 * subbu is around and is generally happy with the direction, modulo details that might need tweaking 21:08:31 is there going to be an extension to replace it? 21:08:39 step 1 is normal deprecation procedure I think 21:09:02 a workalike extension could be created if someone wants one but i don't know if anyone does :) 21:09:08 i think there's still a semi-open question w/ what the replacement should be: 1) ordinary template, 2) parser function, 3) extension, 4) interwiki link, 5) something else? 21:09:13 if there's no migration path for external users then I don't think it should be deprecated 21:09:13 Kunal wrote something about that in email/phab today... 21:09:21 just disabled by default 21:09:50 my inclination is ordinary template is the right thing for wikipedia 21:09:54 1) and 2) look like {{ISBN|xxx}} and require content edits, 3) would be a workalike using a parser hook, 4) would look like [[isbn:xxx]] and require content edits. 21:09:58 legoktm's position is that disabling just disables the hyperlinks, but it's still readable 21:10:27 it is a relatively clean 'failure mode' yes 21:10:38 no explosions/php fatals :) 21:10:40 i think we already have a config option for disabling it in the parser? legoktm wrote that patch. so it's just a matter of turning that config option off on WMF wikis. 21:11:27 we can discuss removing the feature from the codebase and/or moving it to a parser extension as a separate thing. 21:11:31 I don't like option (3) 21:11:38 TimStarling: "We would move the Booksources code and ISBN parser function to an 21:11:39 a extension." -- https://lists.wikimedia.org/pipermail/wikitech-l/2016-October/086716.html 21:12:11 i.e. option (3) for parsing "ISBN ....." as a magic link with a parser hook 21:14:15 i don't like it for WMF wikis, but it would be a reasonable thing to allow 3rd parties to continue to support magic links on their external wikis if they felt strongly about it 21:14:21 using square brackets seems elegant to me 21:14:27 re TimStarling's "if there's no migration path for external users then I don't think it should be deprecated" 21:14:32 considering it is an actual link 21:15:03 I can go with either {{isbn|..}} or [[isbn:..]] personally 21:15:04 pmid and rfc are proposed to become default interwiki links 21:15:24 isbn is the only oddball right? 21:15:27 [[rfc:1234]] would work fine 21:15:37 isbn is the oddball because it goes to *the wiki itself* 21:15:42 *nod* 21:15:54 yeah, it would be like a namespace alias 21:15:55 although i don't think there's anything preventing interwiki links from being relative URLs? 21:15:58 like media 21:16:29 it could even be a negative namespace ID rather than an interwiki prefix 21:17:15 interesting idea 21:17:37 i think i prefer an interwiki, with special:booksources as a special service that happens to be the link target 21:18:27 is there a precedent for magic link functionality in other wiki markups? e.g. does anyone else automatically turn "ISBN \d+" into a hyperlink automatically? 21:18:36 but negative (magic) namespace has a certain elegance to it too 21:19:11 robla: external link hyperlinking is the precedent 21:19:16 ie, http://foo.bar/bat 21:19:34 most other markup languages have that 21:19:43 bugzilla autolinks textual things like 'bug 1234' 21:19:49 when given a bare url 21:19:57 and of course many mobile browsers autolink phone numbers (which i hate ;) 21:20:00 github markdown autolinks hashes and bug references 21:20:25 $RFCPattern = "RFC\\s?(\\d+)"; 21:20:25 $ISBNPattern = "ISBN:?([0-9- xX]{10,})"; 21:20:32 s/$RFCPattern/&StoreRFC($1)/geo; 21:20:38 s/$ISBNPattern/&StoreISBN($1)/geo; 21:20:39 https://help.github.com/articles/autolinked-references-and-urls/ 21:20:40 says usemod 21:20:49 so this feature was in fact carried over from usemodwiki 21:21:08 PMID was a more recent addition 21:21:47 that would imply a precedent that we're continuing rather than merely an oddball legacy feature of our own 21:22:17 weeelll. 21:22:30 that said, I think I agree with brion's hatred of autolinked phone numbers 21:22:38 precent becomes oddball legacy given enough time (and lack of external adoption) 21:22:45 usemod is also responsible for the "free link" terminology 21:23:25 it's quite likely just an oddball legacy feature that we inherited from grandpa usemod :-) 21:24:45 the argument for deprecation would be simpler parser code? 21:25:27 simple parser spec & removal of an english-specific bit of markup 21:25:42 but as long as we magically link free URLs, we will still have Parser::doMagicLinks() 21:25:48 RFCs in particular are english-only and not really relevant for most of our projects 21:26:15 that's the argument for step 2 21:26:16 TimStarling: And the magic of image links -> remote transclusion when that evil flag is enabled. 21:26:20 I mean the argument for step 1 21:26:23 sure, but doMagicLinks() could be greatly simplified and only match https?: prefixes 21:26:30 for example 21:26:39 instead of the pretty complicated regexp for ISBNs 21:26:40 I am fine with step 2 and 3 but not so much with step 1 21:27:02 and our free URLs autolinking is pretty crazy complicated because of all the different protocol schemes we support in theory. 21:27:30 I don't like "magic links" because they suddenly add special meaning to some text substrings ... 21:27:59 it complicates WTS as well, you have to watch for all sorts of boundary conditions when serializing 21:28:01 if it's disabled on WMF wikis then parsoid doesn't need to support it 21:28:19 Parsoid is for more than just WMF users, eventually. 21:28:22 instead of using uniform syntax .. and as someone argued, RFC xyz is ocfusing since mediawiki has RFCs. 21:28:25 o_0 parsoid is Wikimedia only TimStarling? 21:28:29 *confusing 21:28:44 and if we're going to continue to do round-trips for content migration, whether to wikitext 2.0 or markdown or , then simplifying WTS (wikitext serialization) will continue to be desirable 21:29:02 parsoid has a reduced feature set compared to MW 21:29:04 it seems like protocol links (like mailto:.*, http?s:.*) have ample precedent in both email and wikis. Magic words as arbitrary barewords are relatively rare 21:29:05 and all the nowiki crap. 21:29:09 it doesn't even support wikipedia properly 21:29:24 well that I can't argue against 21:29:38 robla: yes, but have you looked at the full list of protocols we support? 21:29:54 there are no plans to make parsoid support every single MW parser extension 21:30:01 cscott: not in a while 21:30:05 * robla goes to look 21:30:22 $wgUrlProtocols = [ 'bitcoin:', 'ftp://', 'ftps://', 'geo:', 'git://', 'gopher://', 'http:// ', 'https://', 'irc://', 'ircs://', 'magnet:', 'mailto:', 'mms://', 'news:' , 'nntp://', 'redis://', 'sftp://', 'sip:', 'sips:', 'sms:', 'ssh://', 'svn://', 'tel:', 'telnet://', 'urn:', 'worldwind://', 'xmpp:', '//' ]; 21:30:27 TimStarling, yes .. i've proposed that parser hooks that rely on parser internals should instead be deprecated. 21:30:46 parsoid and php parser have different internals. 21:30:47 i'm pretty sure we could drop gopher:// and worldwind:// 21:31:05 I hear gopher is going to make a comeback ;) 21:31:25 free external links could support a reduced set of protocols compared to bracketed external links 21:31:27 but in general, I like {{bitcoin|}} or [[bitcoin:hash]] better than autolinking bitcoin:cafebabe in text 21:31:42 but, anyway ... we do want to get to a unified model .. and that parsoid doesn't support wikipedias is neither here nor there .. whatever parser it is that supports wikipedias will need to grapple with the roundtripping and html2wt issue. 21:31:45 that's why we have so many protocols, because non-WMF users want to link to those things 21:32:25 * robla doesn't think the list cscott provided seems so crazy 21:32:29 TimStarling: perhaps, but I think they could manage to use slightly different markup to accomplish the same task. 21:32:55 like bracketed external links? 21:32:59 exactly 21:33:23 or double-bracketed links for some things perhaps 21:33:26 [[Gopher (protocol)]] does of course have many gopher links on it 21:33:29 robla: agreed - cscott's list could make sense 21:33:40 but all bracketed as far as I can see 21:33:50 cscott: Pretty sure gopher:// is officially deprecated now, from what I vaguely recall 21:35:55 so i'm going to be a poor advocate for legoktm and say I'm not particularly chuffed by magic links. i'd get rid of them for wikitext 2.0 as part of a broader simplification, but now that they are implemented and working it's not really helping us much to get rid of them. 21:36:24 so...perhaps we can agree to make the linking aspect off by default, without necessarily declaring them deprecated in 1.28 21:36:35 i guess the question is whether you think we can slim down wikitext incrementally by turning off these weird crufty bits one by one, or whether we should treat it as a completely new markup language 21:36:48 and use something like "parsoid2" to do round-trips between "wikitext 1" and "wikitext 2.0" 21:37:03 what i think of wikitext 2.0 is probably different from what cscott thinks of wikitext 2.0 probably :) 21:37:15 * marktraceur may have been wrong, the minnpost article he must have seen was just a random summary of Gopher 21:37:27 i imagine a language with a formal grammar, small enough to fit on a single sheet of paper. 21:37:36 * robla doesn't want to rabbithole on that topic, when we might actually be able to make a decision about Mediawiki 1.28 21:37:36 but otherwise looking as much as possible like wikitext 1.0 21:37:36 marktraceur, ah ... now it makes sense gopher came form UMn ... and that is why gopher 21:37:53 cscott: How big is the piece of paper? Implementation details... 21:38:13 can we turn magic words off by default in 1.28 without deprecating? 21:38:27 and so, listing 28 different external link prefixes and 12 or so separate productions for ISBN/RFC/PMID is not going to help wikitext 2.0 fit on a single sheet of paper 21:39:16 er...I suppose I should have said "magic links" 21:39:28 can we turn magic links off by default in 1.28 without deprecating? 21:40:01 sure .. i guess the discussion is more about whether deprecation is required / desirable. 21:40:12 https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pegTokenizer.pegjs.txt#L449 is the grammar for RFC/PMID/ISBN in parsoid, for reference. 21:40:46 as far as i'm concerned the real question is: can we get the *wiki communities to start rewriting content to use whatever our preferred replacement markup is? 21:40:46 subbu: yes, but that seems to be sending us down the rabbithole of talking about Wikitext 2.0, which we aren't going to finish in this hour 21:41:06 personally, i don't think we need to talk about wikitext 2.0 there. 21:41:20 https://en.wikipedia.org/wiki/Template:ISBN was created by MZMcBride 21:42:01 and it seems to already be used by quite a number of pages 21:42:15 as a compromise, how about not deprecating it immediately, but revisit after a year or two and see how many people are turning on the option? 21:42:31 i think it is a worthwhile discussion in its own merit. but, like cscott i don't have strong feelings ... but yes, it will simplify some code in parsoid .. but i won't be heartbroken if it is kept as is. 21:42:37 one intermediate step might be to hack the parser to add a parser warning if you use magic link syntax, suggesting an appropriate rewrite 21:42:46 * robla likes TimStarling's suggestion 21:43:08 we can do that w/o committing to deprecation, just encouraging people not to use that syntax. 21:43:14 THen we'd just need a way of collecting that data 21:43:19 and then, as TimStarling suggests, give it a year or so and see what our trends are. 21:43:30 ya what bd808 says .. do we have a mechanism for collecting that data? 21:43:45 we could probably hack together a dumpGrepper script that would collect repeatable numbers for long-term comparison purposes 21:43:47 there's that pingback feature ori introduced... 21:44:06 we could also add [[Category:Uses Magic Link Syntax]] 21:44:38 * robla looks for the link to the aforementioned pingback feature 21:44:44 i know (or am i imagining it?) kaldari had strong feelings about magic links. curious if he is around. 21:44:49 currently it doesn't send any configuration 21:45:31 #link https://www.mediawiki.org/wiki/Manual:$wgPingback 21:46:03 My suggestion was to just kill the RFC magic linking entirely, as it's usefulness is very marginal 21:46:22 otherwise, I support making it configurable 21:46:42 and depricating 21:46:46 I think that part has been done now. there's a feature flag for it 21:47:14 and this discussion is about deprecation and removal from core 21:47:26 the dynamic dates was removed, there is that precedent, but dynamic dates was never on by default and was rarely used by non-WMF wikis as far as we know 21:47:57 we could conceivably deprecate in 1.29 if the 1.28 release goes well 21:48:03 whereas we've been told that magic links are regularly used 21:48:22 ISBN in citations only from what I can tell. 21:48:31 I mean outside WMF 21:48:35 ah .. 21:48:40 for WMF we can run bots, outside WMF not so much 21:48:57 outside WMF we don't know if people even want to run bots, or if they like their magic links the way they are 21:49:04 the pingback feature would give us more certainty, but I think even just "did anyone complain?" might be a good enough test in this case 21:49:35 or would anyone who would complain actually upgrade anyway... 21:50:35 it would be nice if WMF published a bot framework and scripts for it w/ every major upgrade 21:50:53 like how we upgrade database schemas, we could provide tools for external wikis to update their content 21:51:02 I guess I'm not sure why it needs to die if there is a feature flag other than hypothetical future parser world that would like to not support the feature 21:51:18 cscott: I think we call that pywikibot 21:51:22 I suppose deprecating in 1.28 would be the "be bold" way of doing it. we could be prepared to "undeprecate" if people hate that we turned it off 21:51:43 bd808: sure, it's the "official" part and "with scripts for every major upgrade" which is missing. 21:51:47 Do "off by default" changes need to go through the one-version-notice process? 21:52:07 heh 21:52:14 disable by default then silently remove in 1.29? 21:52:20 Tut. :-) 21:53:05 James_F: I don't *think* we have a policy like that, but I suppose that may be what legoktm's other rfc covers in more depth 21:53:27 cscott: I'm not sure I agree on "official" and WMF being intrinsically intertwined, but some suggested wikitext cleanup scripts would be an interesting thing to add when changing the syntax 21:53:38 Sorry, yes, I more meant "would it be covered by the semi-in-practice policy?". 21:53:46 the deprecation rfc: T146965 21:53:47 T146965: [RfC] Deprecation policy for PHP code in MediaWiki - https://phabricator.wikimedia.org/T146965 21:53:52 i'm always interested in process improvements to make it easier for us to evolve wikitext syntax 21:54:57 so should we call the non-controversial parts of this approved? 21:55:13 * subbu cares less about syntax and more about underlying processing model / semantics except where the syntax gets in the way 21:56:09 wfm .. but, explicitly listing out the non-controversial parts would be useful. 21:56:26 #agreed 1. Disable the magic link functionality by default for the MediaWiki 1.28 release 21:56:58 #agreed 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration 21:57:26 hmmm....I'd feel more comfortable with broader input on #2 21:57:42 sure. and the controversial step 3 would be removing the magic link code from core (perhaps moving it to an extension)? 21:57:52 I suppose "start step #2" isn't yet controversial 21:58:01 Yes. 21:58:10 i'd propose 2(a) -- add a category and parser warning for magic links on wikimedia wikis, without officially deprecating the feature. 21:58:22 cscott: To move to an extension it'd have to keep the horrible hooks into the PHP parser, though? 21:58:28 then sit back and see if folks are getting the magic links cleaned up or not. 21:58:35 #info no consensus on removing the functionality from MW or flagging its pending removal via deprecation 21:59:21 no, step 3 is disabling magic links on WMF wikis 21:59:23 James_F: yes, although in my big picture worldview we're adding a pluggable parser API, and gradually moving from the "old" PHP parser to a newer one. 21:59:32 * James_F nods. 21:59:34 removing from MW core was 1b 21:59:45 so the hooks would only stay in the legacy PHP parser. which, of course, you could keep running if you like and are willing to keep it maintained. 22:00:05 that's one possible universe, yes cscott 22:00:06 (Thanks, All) 22:00:27 bd808: i keep pushing the universe towards my platonic ideal of it. ;) 22:00:35 * cscott has to turn into a pumpkin 22:00:54 I like the "everything in the core php app" universe better 22:01:03 but that's why we have these chats 22:01:10 * robla plans to turn into a different type of squash 22:01:29 its a dangerous time of year to be a pumpkin 22:01:37 srsly 22:01:46 could be gutted and filled with candles at any point 22:01:47 what was next week's RFC again robla? 22:01:57 next week we'll plan on talking about CREDITs file 22:01:59 * robla finds link 22:02:18 https://phabricator.wikimedia.org/T139300 22:02:20 bd808: pluggable doesn't mean not monolithic or not php, btw 22:02:25 just that things are decoupled 22:02:39 so you can have a pure-PHP single-process markdown wiki if you like. 22:02:42 ok, all done 22:02:45 #endmeeting