Release Meeting: 21 Aug 2017

(08:16:54 AM) JulianLevens  entered the room.
(09:00:33 AM) JulianLevens: Hi
(09:00:46 AM) gac410: Hi everyone.
(09:00:58 AM) gac410: Welcome back from vacation / holiday MichaelDaum
(09:01:32 AM) gac410: How's things going JulianLevens
(09:01:49 AM) gac410: uebera||: .. .are you around too?
(09:02:34 AM) gac410: In addition to the usual release meeting topics, I thought we could briefly talk about the mailing lists.
(09:03:00 AM) JulianLevens: Ok, here, struggling with follwwing rename code for backlinks
(09:03:48 AM) gac410: ugh yeah, that code is pretty difficult - building regexes dynamically iirc. Fraught with pitfalls and minefields.
(09:05:28 AM) JulianLevens: anybody know if the rename code that scans for backlinks (to know topics that need to be updated) is in anyway connected or insync with the general backlinks code?
(09:05:44 AM) vrurg: Hi
(09:06:07 AM) gac410: IIRC they are completely independent. It's been a long time since I looked at it.
(09:06:14 AM) gac410: Hi vrurg
(09:06:59 AM) JulianLevens: y, I suspected as much, ideally I'd bring them in-line, otherwise rename won't get potential future speed benefits
(09:08:36 AM) gac410: The rename code is more complete I think, and has some test coverage.
(09:09:00 AM) gac410: I did some patching of the rename code fixing some reported cases where it missed backlinks.
(09:09:09 AM) gac410: But that was a long time ago.
(09:09:41 AM) JulianLevens: y, I was wondering about making the backlinks code use the rename code, although that would slow it down
(09:10:58 AM) JulianLevens: Anyway, mailing lists?
(09:11:10 AM) gac410: It would probably be better to have one consistent means of finding backlinks. But speed is good too. ;) I'm not sure how often people use backlinks through the regular UI. I don't think of it often.
(09:12:00 AM) gac410: Okay. Mailing lists. We really need uebera|| here too if we tackle this But briefly, sourceforge has turned off access to the subscriber list, and that's permanent and not negotiable.
(09:12:19 AM) gac410: So we cannot see who is subscribed, and cannot manage subscribers.
(09:12:54 AM) JulianLevens: What reasoning did sourceforge give?
(09:12:56 AM) gac410: uebera|| and I converted the foswiki-infra list to an alias on postfix.
(09:13:36 AM) JulianLevens: Related to tightening privacy laws?
(09:13:46 AM) gac410: They claim that emails must be kept completely confidential in some legal jurisdictions, and therefor, with a brief blog notice which we all missed, have turned off access.
(09:13:48 AM) gac410: Yes.
(09:14:18 AM) JulianLevens: I suspected as much, GDPR is going to create some work here
(09:14:58 AM) gac410: Our "closed" lists, like our board of directors list, association members, and security list really need to be under our direct control.
(09:15:08 AM) JulianLevens: agreed
(09:16:21 AM) gac410: The other big lists, announce, svn and discuss, I don't care too much who is on them, but we need to be able to moderate/unsubscribe spammers, etc. And we are blocked from that too.
(09:17:08 AM) JulianLevens: To cover ourselves we need to make it clear the being member of the association/board means you give consent to be on these lists
(09:17:18 AM) gac410: uebera|| and I briefly discussed installing sympa or mailman. I prefer sympa but really 6 / half dozen.
(09:18:04 AM) JulianLevens: I'll let you two fight that one out
(09:18:44 AM) gac410: Y. And on the other lists. Submitting your email address by registration, or joining lists, you consent to our association having visibliity to your email for administration purposes. We commit to not revealing emails
(09:19:28 AM) gac410: But we should have the full right to review and moderate / unsubscribe or otherwise manage any address on our lists.
(09:20:05 AM) MichaelDaum: Hi guys. Had some server probs here that needed immediate actions.
(09:20:08 AM) ***MichaelDaum reading the logs ...
(09:20:09 AM) JulianLevens: Agreed, if someone signs up for a service they agree to the terms
(09:20:41 AM) gac410: Hi Michael. Always the hassle trying to "recover" from the mountain that awaits in punishment for vacation :D
(09:20:50 AM) MichaelDaum: :)
(09:21:10 AM) JulianLevens: The new laws may mean we need explicit consent for specific points, a blanket consent buried in terms will not suffice
(09:21:40 AM) MichaelDaum: JulianLevens, the backlinks "code" is mainly a %SEARCH.
(09:21:54 AM) MichaelDaum: in the templates.
(09:21:54 AM) JulianLevens: y, that's pretty easy to fix
(09:21:59 AM) gac410: Anyway, we probably ought to table the mailing lists discussion and that maybe needs an association meeting ... and maybe a change in the registration acks.
(09:22:09 AM) MichaelDaum: I once replaced them with a %SOLRSEARCH using a skin overlay
(09:22:31 AM) gac410: ie, checkboxes in registration form, I consent to blah blah.. and I agree to blah blah.
(09:23:20 AM) JulianLevens: I want to have somwthing like Foswiki::Backlinks does the real work and designed to be replaceable
(09:23:26 AM) MichaelDaum: solr gathers outgoing links ... the rev relation of backlinks ... while indexing a page
(09:24:10 AM) MichaelDaum: key is to harvest outgoing links and capture them in a db
(09:24:21 AM) MichaelDaum: updated on every change
(09:24:32 AM) JulianLevens: y, that's the plan, or at least make it easier for someone to plug such capabilities in
(09:24:51 AM) MichaelDaum: you can easily do so using the plugin handlers
(09:25:10 AM) MichaelDaum: and have some %BACKLINKS macro that reads the db
(09:25:15 AM) MichaelDaum: or whatever
(09:25:45 AM) MichaelDaum: current code is busted because it searches for backlinks _online_
(09:25:48 AM) gac410: Backlinks ... restricted to "static" links. That is, in-topic text references. Not dynamic links caused by %concatenation%%of%Macro%%Topics%
(09:25:48 AM) JulianLevens: y, then change the templates to use %BACKLINKS
(09:26:23 AM) MichaelDaum: gac410, y
(09:26:24 AM) JulianLevens: and rename code to use the same backlink discovery process to identify topics that need updating
(09:27:12 AM) gac410: So there is no need to index backlinks generated during macro expansion and render. ... unlike the cache dependencies stuff.
(09:27:27 AM) JulianLevens: Some 'dynamic' links would be possible e.g %USERSWEB%.MyTopic
(09:27:43 AM) JulianLevens: but only simple cases
(09:28:13 AM) MichaelDaum: a link generated dynamically isn't really counting as a backref
(09:28:13 AM) gac410: Ah... yeah. But that needs to be kept to "well known" cases. - the core provided macros.
(09:30:27 AM) JulianLevens: Anyway harvesting links is a more advanced item for the future
(09:31:18 AM) gac410: Does that clear up your questions JulianLevens ... Move on to "Release Blockers?"
(09:31:37 AM) JulianLevens: which also needs common Foswiki::HarvestLinks code, a store should not need to know details about how to find links
(09:31:57 AM) MichaelDaum: here's what SolrPlugin does atm: https://github.com/foswiki/SolrPlugin/blob/master/lib/Foswiki/Plugins/SolrPlugin/Index.pm#L704
(09:32:41 AM) JulianLevens: Ok, release blockers
(09:32:48 AM) gac410: hm Github is not responding to me.
(09:33:05 AM) gac410: http://foswiki.org/Tasks/Item14445 ... Michael opened for 2.1.5.
(09:33:30 AM) MichaelDaum: yea... a regression
(09:33:48 AM) MichaelDaum: could also be called "excel sucks"
(09:33:59 AM) gac410: That actually reverts the changes in http://foswiki.org/Tasks/Item11515
(09:34:21 AM) gac410: and http://foswiki.org/Development/Use401ForCookieAuth
(09:35:08 AM) gac410: I think that needs more discussion. Excel sucks is not a reason to break our code if 401 is really the right response.
(09:35:56 AM) gac410: I don't know the right answer, I have not looked deep enough in a long time. I'd like to get CDot's input on that one too.
(09:36:59 AM) MichaelDaum: tldr
(09:37:36 AM) gac410: REST can't handle a redirect for auth failurs. so returning 200 with a login will break our rest code from what I can read.
(09:37:48 AM) MichaelDaum: interestingly the original proposal tries to distinguish auth failures that happen during an ajax call from those that are normal http views
(09:38:41 AM) gac410: Anyway. I think your task needs further analysis Maybe we have to go back to differentiating REST from the rest, But that doesn't feel right.
(09:39:14 AM) MichaelDaum: why
(09:41:16 AM) MichaelDaum: theoretically, a rest endpoint should always return data that can be processed by the calling code.
(09:41:27 AM) gac410: Because the decision to return 401 was done with a lot of discussion and analysis. A return to 200 because excel is busted seems to be a bit quick & dirty.
(09:41:54 AM) MichaelDaum: well think of what you are trying to do with a rest backend
(09:42:14 AM) gac410: I think we need to review https://www.ietf.org/rfc/rfc2617.txt and decide what is the correct way to tell a client that the request was not authorized.
(09:42:19 AM) MichaelDaum: further note that excel probes a normal view link, not a rest endpoint
(09:42:53 AM) gac410: Right. But just reverting template login is not enough.
(09:44:50 AM) MichaelDaum: why
(09:44:56 AM) gac410: All I'm saying is that a complete fix needs to address rest, view, and json too I guess.
(09:45:57 AM) gac410: Because a simple "revert template auth" breaks rest, and moves the problem to some other poor schmo to deal witht.
(09:46:02 AM) ***uebera|| just catching up...
(09:46:42 AM) gac410: And a simple revert template auth, I think, says that excel still fails if the site uses Apache auth, and that's not right either.
(09:48:17 AM) MichaelDaum: I wouldn't revert all changes in Item11515
(09:48:22 AM) uebera||: gac410: Ad GitHub: https://twitter.com/githubstatus/status/899628268395147264
(09:48:44 AM) gac410: thanks uebera||
(09:50:18 AM) MichaelDaum: we really are in a gray zone here with regards to http error codes of rest backends
(09:50:32 AM) MichaelDaum: as far as I know there is no spec for it. hence jsonrpc.
(09:51:07 AM) gac410: Just looking at UI::Rest It appears to return it's own 401, but not 100% sure
(09:52:09 AM) gac410: master as well. UI::Rest handles the 401.
(09:52:39 AM) MichaelDaum: ah ok
(09:52:48 AM) MichaelDaum: makes sense
(09:53:01 AM) MichaelDaum: 14445 isn't about rest anyway
(09:53:05 AM) gac410: Anyway ... I'm getting strong deja-vu on all this. I'm sure we had a very detailed discussion on whether 401 was correct.
(09:53:26 AM) vrurg: I'd cut into conversation here because not sure if will still be available later today. But anyway there is no news on my side as I'm deeply in a working project which is literally taking all my time. Luckily, I'm finishing it and hope to get back to the project this week.
(09:53:42 AM) gac410: I realize that. But I couldn't say for sure that a change to TemplateLogin.pm won't impact rest or json
(09:54:30 AM) gac410: vrurg. Okay thanks. Yeah I understand the conflicts. I've gotten very tied up in trying to get some landscaping done - laying new walkways before cold weather, etc.
(09:56:32 AM) gac410: And my deja-vu is telling me that 401 is actually the incorrect response. RFC says that you MUST include a WWW-Authenticate header.
(09:56:36 AM) gac410: "This response MUST include a WWW-Authenticate header field containing at least one challenge applicable to the requested resource."
(09:56:47 AM) gac410: And that doesn't make sense for template auth.
(09:57:28 AM) MichaelDaum: KerberosLogin inherits from TemplateLogin in order to be able to fall back to template login in case the browser isn't offering a credentials
(09:58:55 AM) MichaelDaum: thats something that needs fixing in case of TemplateLogin changing in this area
(09:58:56 AM) gac410: BTW... on the Auth front. I have mostly finished porting TWiki's SmsTwoStepAuthContrib ... but modified to not need core changes. SmsTwoStepLogin.pm login manager, sends a sms message (or email) after user/password succeeds.
(09:59:41 AM) gac410: Someone on IRC wanted to implement a DuoTwoStepAuth and needed an example for foswiki using two-step "like twiki does"
(09:59:46 AM) MichaelDaum: yea seen that. awesome.
(10:00:13 AM) JulianLevens: +1
(10:00:13 AM) MichaelDaum: what kind of lib do you use to send the sms?
(10:00:27 AM) gac410: actually just an email to the carrier's sms gateway
(10:01:04 AM) gac410: There is a list of carrier -> email gateways provided in the plugin topic.
(10:01:38 AM) gac410: I'd point you to the list, but github is down
(10:04:29 AM) gac410: Anyway, that's in for patch blockers. 2.2 has one new blocker as well. New TMCE has an upstream error - inserting some hex garbage in topics under some conditions.
(10:04:32 AM) gac410: https://foswiki.org/Tasks/Item14420
(10:05:01 AM) gac410: On Feature requests front, there have been no new requests.
(10:05:15 AM) gac410: well except for JSViews but that's minor
(10:05:53 AM) gac410: I assume with vacations and real life, there has been no progress on 2.2
(10:06:03 AM) MichaelDaum: there is still a couple of work on my plate for 2.2. part of them is done (no proxy, include url exception) still in need of a bit of docu
(10:07:37 AM) gac410: I've made a little progress on the disable/enable accounts. The hard part will be a UI to update the accounts. And I'm still questioning myself if it's really the right solution to extend .htpasswd with more info.
(10:08:07 AM) JulianLevens: I haven't looked at the locale stuff recently, but a lot of that work turned out to be documentation
(10:08:16 AM) MichaelDaum: I've got a couple of changes for JQueryPlugin too that I will cerate task items for one by one
(10:08:50 AM) gac410: the enable/disable is simple enough in .htpasswd - accept the # comment but I'm really questioning adding more fields expiration date, must change flag, etc.
(10:09:11 AM) MichaelDaum: hell, we need a user db
(10:09:24 AM) gac410: wondering if it wouldn't be better to use a different password manager. Exactly. But that does expand the scope a bit. :D
(10:09:58 AM) MichaelDaum: massively. though that would be the right thing to do (tm)
(10:09:58 AM) gac410: really, the mobile number & carrier - with sms two-step, really ought to be hidden / private. Just like email.
(10:11:10 AM) MichaelDaum: modell aachen implemented a user db in their UnifiedAuthContrib thingy
(10:11:17 AM) gac410: Maybe the right start would be a user db, and "shadow" .htpasswd file
(10:11:51 AM) gac410: The thing that makes it a huge challenge is the cUID and how deeply it's embedded in our code.
(10:12:03 AM) MichaelDaum: when the apis are clean the backend can do whatever you like
(10:12:23 AM) gac410: And our APIs for user auth and mapping, are a disaster. :(
(10:12:30 AM) MichaelDaum: true
(10:13:43 AM) MichaelDaum: what I mean is: adding an api in Foswiki::Users and then delegating it to the mapper is just fine. thats the way the calling sequence for all user infos atm.
(10:14:44 AM) MichaelDaum: the mapper can then extract the info such as registration date (... nother proposal I need to check in) or mobile carrier
(10:15:29 AM) MichaelDaum: the idea would be to make %USERINFO{format="$carrier"}% work for $foobar in a generic way
(10:16:20 AM) gac410: Ah. Rather than painfully adding each $foobar keyword. Map it to a db column for ex.
(10:16:40 AM) MichaelDaum: y
(10:16:53 AM) gac410: Our big stumbling block is our cUID implementation. It's so inconsistent / broken in places.
(10:17:40 AM) MichaelDaum: I haven't come accross any real world value for it
(10:18:03 AM) MichaelDaum: it even isn't unique in spacetime
(10:18:42 AM) gac410: Ahhh Here was my requirements analysis proposal that's quietly composting/bit rotting away
(10:18:45 AM) gac410: https://foswiki.org/Development/UserAuthMapping2dot0
(10:19:24 AM) gac410: Tried to capture all the real life challenges of mapping a wiki identity to the outside / real world identities
(10:19:34 AM) uebera||: [off] Uh oh, I hope GitHub didn't hire that GitLab guy involved in their blunder back then in the meantime (outages seem to become more and more frequent)
(10:19:43 AM) MichaelDaum: hey guys, Ive got a call coming up.
(10:20:03 AM) gac410: Okay. I've got nothing more really....
(10:20:16 AM) uebera||: MichaelDaum: Apart from 2.2--at one point, if you have the time, please have a look at https://foswiki.org/Community/FoswikiOrgBlog#Solr_specific (we stopped there).
(10:21:16 AM) gac410: uebera||: we also need to get back on to the translate.f.o migration ... and possibly address our email lists and sourceforge changes.
(10:21:22 AM) gac410: :D
(10:21:30 AM) MichaelDaum: uebera||, do you need more deps?
(10:21:31 AM) uebera||: yes, absolutely.
(10:22:08 AM) uebera||: MichaelDaum: For the time being, IIRC, it's only your green light for the Solr plugin. You said back then that you needed to fix/document/amend something before it's published officially.
(10:22:56 AM) uebera||: Regarding the foswiki-security list--we should move that. Last time I checked only 8 people were subscribed, and the majority should be in this channel ATM (plus cdot).
(10:22:59 AM) MichaelDaum: yes, I'll be working on plugins during the next weeks, including updates on solr and stringifier, blogplugin, wikiworkbench, natskin n stuff
(10:23:32 AM) gac410: Ah... MichaelDaum Is it safe to just update ImagePlugin on foswiki.org?
(10:23:49 AM) gac410: Or does it change backend requirements at all?
(10:23:51 AM) uebera||: So for the time being, amending /etc/aliases should not be much of a hassle.
(10:24:42 AM) MichaelDaum: yes thats safe. adds some features and performance improvements
(10:24:57 AM) gac410: There were a number of emails on foswiki-security@sf.net that I didn't recognize. So I think we should start over with an aliases list for security@f.o
(10:25:08 AM) gac410: Okay MichaelDaum thanks. I'll run an update sometime toon.
(10:25:09 AM) gac410: soon
(10:25:40 AM) MichaelDaum: ImageGalleryPlugin is now based on ImagePlugin sharing thumbnails among each other ... instead of computing them each by themselves
(10:25:56 AM) gac410: Ah.. okay. And I don't think IGP is used on f.o
(10:26:33 AM) uebera||: We have IGP installed for the blog.
(10:26:43 AM) gac410: Ah... IGP... I vaguely recall at one point one of the galleries supported intermediate sizes. like thumbnail -> 800x600 -> Full resolution. I used to use that on my own wiki
(10:27:04 AM) MichaelDaum: IGP was using /pub/images/... which we covered in the ApacheConfigGenerator ... as ImagePlugin stores in pub/Web/Topic/igp_hexhex.png now these thumbnails are now covered by viewfile/xsendfile in a more natural way...
(10:27:17 AM) gac410: Last time I updated, I lost the itermediate resolution. And my full resolutoin photos are wayyy to big.
(10:27:40 AM) gac410: I've been meaning to ask if I was missing something obvious or mis-remembering a feature.
(10:28:27 AM) MichaelDaum: okay see you on the other channel ... later
(10:28:28 AM) ***gac410 wonders if we should move f.o to using xsendfile
(10:28:36 AM) MichaelDaum: +1
(10:28:38 AM) gac410: okay see you MichaelDaum
(10:28:44 AM) uebera||: cu
(10:28:50 AM) gac410: thanks everyone. That's a wrap.
(10:29:01 AM) gac410: uebera||: we can continue in the main channel or -admin
(10:29:08 AM) uebera||: ok
(10:29:52 AM) JulianLevens: Thanks, bye
(12:18:57 PM) JulianLevens left the room (quit: Ping timeout: 240 seconds).
(01:15:59 PM) vrurg left the room (quit: Quit: vrurg).
(02:10:54 PM) The account has disconnected and you are no longer in this chat. You will automatically rejoin the chat when the account reconnects.