Release Meeting: 21 Mar 2016

Details

1. Urgent Task review

  • 13795 (Closed): Redundant url params generated by %SCRIPTURLPATH macro.
  • 13957 (Closed): TinyMCEPlugin does not handle indent correctly.
13975 is fixed, waiting for feedback on some side effects. 13795 is relatively straight forward to fix.

  • NEW 14024 (Closed): JQueryPlugin (v6.32) might not initialise correctly with current JSON (v2.90) / JSON-XS (v3.02) modules w/o allow_nonref.
  • NEW 14025 (Closed): JsonRpcContrib requires allow_nonref (when using JSON-XS v3.02).

Had discussion of the issues reported in 14024 and 14025. Ueberall investigated and it was discovered that it appears to be a mod_perl issue. Problem is not encountered on CGI or FCGI. These can be downgraded to normal priority. MichaelDaum comments: right: so code looks sane, issue hard to repro, => more research needed

2. Development Discussion

Accepted proposals listed on ReleasePlan

Review of features / Feature branches

New proposals - Proposals on 14 day clock

  • CleanUpFoswikiLocales Under Investigation: Provide relevant locale support in pure Foswiki code and eliminate perl/OS locale
    Discussed at some length. There are two parts of this. 1) Eliminate the "use locales" which is already broken due to the changes to NFKD sorting. JulianLevens reports: use of Unicode::Locale::Collate could be 2.2 - its the removal of LC_CTYPE that need to be deferred to 3.0 It appears we need help especially in figuring out how to test any changes.

  • MoveQueryPathParsingIntoFoswikiRequest Merged To Core: Make Foswiki::Request responsible for parsing the query path and identifying the Web, Topic & Attachment.
    • I need some more feedback on this request, and the complementary request: ContinueCanonicalSCRIPTURLDev. If a "request type" parses apart the url components into base web / base topic, should it also have a utility method that assembles macro parameters into the request path.
    • We had a good discussion of the roles between Foswiki::Address and Foswiki::Request and location of the parser. gac410 will be working on an experimental implementation for further evalulation.

14 Day timer ended - Last call

No concerns raised, these will move into approved status.

Major redesign proposals / Development discussion for 3.0+ / 4.0

gac410 sent this discussion down a rat-hole over virtual hosting. But getting past that, there was good discussion on the Moo direction and eventual PSGIfication of foswiki. Vadim needs everyone's help! He is designing the Foswiki::App framework which replaces the old Foswiki session object. Please comment on MooFoswikiPm The Moo conversion is now working. and needs developers to test! Unfortunately the most recent checkin broke it again. Working commit is git checkout 610a26498d27c9c1611e50a2730f44f8b853e946.

Proposals that need more work.

These are not fully specified, and should probably have their clock reset as incomplete proposals.

  • ContinueCanonicalSCRIPTURLDev Merged To Core: Continue extending the canonical form of the SCRIPT / PUB URL macros - Need way to specify the anchor (fragment) as a parameter. Might also be useful to allow configurable query string separator ; or &.
  • ReduceImpactOfCGIDotPMinFoswiki Accepted Proposal: Reduce impact of CGI.pm in Foswiki Plans to untangle from using CGI menthods for HTML generation. Do we want to tackle this for 2.2 or 3.0? Spec needs further clarification.

Old proposals never slotted into a release

No changes / status updates to report

These need up dates from their developers.

3. Next release

Patch release 2.1.1

  • Release from: Release02x01
  • Beta start: TBD
  • Release target: March 2016

Feature release 2.2.0 / 3.0.0

  • Feature Freeze: 1 Jun 2016
  • Release from: master
  • Beta start: 15 Jun 2016
  • Release target: July 2016

Next meeting - - Monday 4 Apr 2016 1300Z — ReleaseMeeting02x02_20160404

IRC Log

 
(09:03:20 AM) gac410: good day everyone ...   time to meet?
(09:03:38 AM) gac410: https://foswiki.org/Development/ReleaseMeeting02X02_20160321
(09:05:50 AM) JulianLevens entered the room.
(09:06:30 AM) vrurg: Hi gac410 
(09:06:53 AM) gac410: Hi Vadim
(09:07:18 AM) gac410: Timezone change -  off to a slow start I guess ;)
(09:08:29 AM) ***uebera|| wonders whether he will live to see this vanish...
(09:10:02 AM) vrurg: To me later start is for better. I'm an owl.
(09:11:14 AM) gac410: Anwyay ..   Want to get started?  - Agenda is at  https://foswiki.org/Development/ReleaseMeeting02X02_20160321
(09:12:12 AM) gac410: I'm thinking we should consider building a patch 2.1.1 release soon
(09:12:47 AM) gac410: So Urgent Tasks Review.    Two new ones that uebera|| opened.   need MichaelDaum's guidance.
(09:13:18 AM) gac410: https://foswiki.org/Tasks/Item14024   and https://foswiki.org/Tasks/Item14025
(09:13:25 AM) MichaelDaum: hey guys
(09:13:31 AM) gac410: Hi MichaelDaum
(09:13:49 AM) MichaelDaum: still not 100% here suffering from a flu
(09:14:03 AM) gac410: Don't cough in our direction  :D
(09:14:17 AM) MichaelDaum: too late
(09:14:30 AM) ***MichaelDaum wiping windows
(09:14:32 AM) gac410: I hope you are on the mend.
(09:14:59 AM) MichaelDaum: yea, well, temperature is down
(09:15:28 AM) gac410: So with the two tasks ... Seems that adding allow_nonref => 1     to the json calls would not do any harm,   
(09:15:51 AM) gac410: And would avoid issues for anyone updating to latest JSON::XS module.
(09:17:20 AM) uebera||: looks like it (though I did not come up with a patch for JsonRpcContrib and likely won't have the time atm).
(09:17:46 AM) uebera||: At least JQueryPlugin should be "more robust" with that.
(09:17:58 AM) jast: interestingly I've had no issues with the same JSON::XS version
(09:18:13 AM) MichaelDaum: whatever helps. just wondered why I never ran into the issue even though being on bleeding cpan edge.
(09:18:41 AM) gac410: strange.   seems like something like that would be a universal issue.
(09:19:00 AM) gac410: Maybe it's a perl version difference as well. 
(09:19:06 AM) MichaelDaum: uebera||, could you craft a simple test.pl?
(09:19:14 AM) uebera||: Maybe it's only triggered in conjunction with certain plugins? I've not tested with a minimal installation.
(09:19:35 AM) jast: oh, it happens for tied hashes only
(09:19:42 AM) MichaelDaum: note that we should encourage JSON::XS wherever possible as it is a lot faster.
(09:19:43 AM) jast: according to the CPAN report, at least
(09:20:01 AM) uebera||: Not at the moment, I'm afraid. Apart from joining this meeting, I have virtually no time for the next three weeks and won't touch the local lxc container... (important deadlines) :( 
(09:20:31 AM) gac410: I assume that JSON::XS just comes along by default if you install the recommended JSON per the SystemRequirements?
(09:20:35 AM) MichaelDaum: uebera||, okay. 
(09:21:08 AM) MichaelDaum: are there any possble negative sideeffects of using allow_nonref => 1?
(09:21:39 AM) uebera||: gac410: Unless you override this mechanism in the JSON module, yes, that should be the case.
(09:21:45 AM) MichaelDaum: other than having to remember this magic switch & having to type more 
(09:21:57 AM) ***gac410 doesn't know -  it "seems" innocuous,  but who knows.
(09:22:01 AM) jast: seems like this issue is unrelated to the one reported in the CPAN module, even though the resulting message is the same
(09:22:38 AM) uebera||: yes, it only pointed me to the general direction.
(09:22:49 AM) gac410: Do we know if we are actually passing in nonref values?     ie is this false detection,  or bad input previously not detected.
(09:22:53 AM) MichaelDaum: maybe we are doing something else wrong that JSON::XS doesnt like ... similar to tie()
(09:23:00 AM) jast: almost certainly false detection
(09:23:14 AM) jast: in JsonRpcContrib's Response object we definitely pass a hashref
(09:23:39 AM) MichaelDaum: I wonder whether allow_nonref =>1 will only mitigate a deeper problem of some objects being de-serialized for no good reason.
(09:23:59 AM) jast: our code looks completely sane
(09:24:01 AM) gac410: in that case it could be dangerous to permanently set nonref
(09:24:47 AM) MichaelDaum: right: so code looks sane, issue hard to repro, => more research needed
(09:24:56 AM) gac410: With only one reporter, and questionable conditions, I'm  feeling less urgency to patch this
(09:25:00 AM) jast: it's only dangerous if the target component doesn't validate its input :}
(09:25:33 AM) gac410: uebera||:   what perl version were you using?
(09:26:02 AM) gac410: I'll install a perlbrew and do cpan-outdated   and see if I can recreate with a default foswik iinstall.
(09:26:11 AM) uebera||: The exact same as on f.o
(09:26:15 AM) uebera||: (using plenv)
(09:26:42 AM) uebera||: 5.22.1
(09:27:53 AM) gac410: well Foswiki.org  is using JSON::XS 3.02    so whatever is going on,  it's not universal.
(09:28:42 AM) uebera||: I'm using mod_perl (have not switched back to fcgi and use lots of plugins that are not active on f.o)
(09:29:08 AM) gac410: Ah...   mod_perl  that's a big difference.  
(09:29:10 AM) uebera||: Maybe deactivating one plugin by another will help to identify the culprit.
(09:29:51 AM) gac410: Well I think it's best to not rush a fix at this point until we understand it better.  So I'm going to downgrade them to Normal to get them off the blocker list.
(09:31:05 AM) gac410: The other blockers:  https://foswiki.org/Tasks/Item13957  ....  I've fixed the issue - improper handling of indented blocks in tinymce,  but now I get an extra line added after save.
(09:31:33 AM) gac410: But the extra line is probably inconsequential,  and doesn't grow past one.   
(09:32:00 AM) gac410: I've asked CDot and Lavr (the initial reporter)  to look at it, but I tend to think we could live with it.
(09:33:22 AM) gac410: Last blocker  https://foswiki.org/Tasks/Item13795   The redundant web & topic when generating SCRIPTURL ...  I could probably fix it pretty easily
(09:33:23 AM) MichaelDaum: +1
(09:34:09 AM) uebera||: Ok, I just substituted my foswiki.conf.mod_perl with foswiki.conf.fcgid on the fly and the problem is gone. Bummer.
(09:35:01 AM) gac410: interesting.   bleh.... That makes it a lot harder to fix.
(09:35:11 AM) gac410: Sounds like an upstream bug though and not ours.
(09:36:22 AM) gac410: Any more on tasks,  anything that should be "urgent"  relative to a Foswiki 2.1.1 patch release?
(09:36:47 AM) gac410: If not ... feature review   
(09:37:44 AM) MichaelDaum: just normal wtf reading the search code
(09:37:56 AM) MichaelDaum: and Foswiki::Meta
(09:38:45 AM) gac410: So nothing blocking ... other than your sinuses   :D
(09:39:58 AM) gac410: First up https://foswiki.org/Development/CleanUpFoswikiLocales ...    Not sure if this is a 2.2 feature or a defer to 3.0?    JulianLevens
(09:40:15 AM) gac410: We are 7 days in,  so will be automatically approved by our next release meeting.
(09:42:02 AM) JulianLevens: I'd love feedback, I might have missed something: the docs take repeated reading
(09:42:06 AM) vrurg: To my view locales are to be considered for 3.0. To serious to be a minor.
(09:42:52 AM) JulianLevens: y, I think 3.0 is prudent, although in theory we shouldn't see any changes
(09:43:38 AM) JulianLevens: Actually, the use of Unicode::Locale::Collate could be 2.2, its the removal of LC_CTYPE that need to be deferred to 3.0
(09:44:06 AM) gac410: Probably pretty low risk,  given that we don't know that our Locale code works currently anyway .. and actually the NFKD changes break Locales   so maybe that should be 2.2 to fix.
(09:45:29 AM) JulianLevens: y, our current code may in fact be broken, but what tests would be needed?
(09:45:45 AM) JulianLevens: remember we are talking about Unicode core only
(09:46:55 AM) ***gac410 doesn't know enough.   For this we could use jomo  who unfortunately has decided to pursue other interests  :(
(09:47:37 AM) gac410: Another on the 14 day clock is one I'm working on ... I'll put this one in an Item branch ... https://foswiki.org/Development/MoveQueryPathParsingIntoFoswikiRequest    Is probably needed to really fix the baseweb//basetask assignment for jsonrpc.  
(09:48:08 AM) jast: FYI I just started the 14 day clock on another new proposal
(09:48:13 AM) gac410: I thnk it's almost certainly a 3.0 ... 
(09:48:22 AM) jast: minor change
(09:49:02 AM) gac410: yeah  I think that came up somewere else.   Sounds very familiar.  
(09:49:09 AM) jast: I mentioned it here a while ago
(09:49:09 AM) gac410: +1  ... good idea.
(09:49:53 AM) JulianLevens: Basically, I agree with the ideas here. I've been thing a lot about the Foswiki::Address *concept* and I see overlap, so I wonder where the correct division of responsibilities is, but that can be refined later
(09:50:06 AM) JulianLevens: s/thing/thinking/
(09:50:44 AM) gac410: I think we are making "Address"  to complex by trying to handle every possible request type.   It's partly why the code was broken even with all the tests.
(09:50:46 AM) ***vrurg thinks that Request may use Address for parsing.
(09:51:25 AM) JulianLevens: I agree with that, you Address for standard requests only
(09:51:34 AM) JulianLevens: s/you/use/
(09:51:52 AM) gac410: our Address concept    web/subweb.topic    web/subweb/topic    web/subweb/attachment.name      almost impossible to guess right without the request context.
(09:52:28 AM) jast: difficult to centralize that, yeah
(09:52:40 AM) vrurg: Address pros: its universal. Being called by Request it may get the required context.
(09:52:41 AM) jast: the main advantage in an Address object would be a unified format to *pass* that data
(09:53:02 AM) gac410: And the resulting Address implementation is so complex.  I've read that code multiple times and I just don't follow the complexity.
(09:53:05 AM) vrurg: Address cons: too complicated code. Easier to split into several independent parsers.
(09:53:34 AM) jast: if we're working on restructuring the internal API anyway (OO and all that), might as well introduced named arguments for all affected functions
(09:53:41 AM) JulianLevens: yes separate parsers but generating the address objects
(09:53:45 AM) jast: that has about the same effect
(09:54:14 AM) gac410: In most cases we know what we have and very seldom need to decode some/random.string/whatever 
(09:54:26 AM) vrurg: It would be more consistent if Address would parse the whole URL and analyze the action.
(09:54:30 AM) jast: if you do want to parse centrally, the callers would have to pass something like 'try interpreting this as one of the following, in this order'
(09:55:54 AM) jast: but you have so many corner cases...
(09:56:10 AM) gac410: vrurg, and I think that's way too complex.   The only thing that knows the format of the URL is the specific type of request.   jsonrpc was out-of-the blue new one to me.   Completely breaks the assumptions made by core.
(09:56:16 AM) jast: in some places you'll want to verify that the interpretation actually maps to an existing topic/file/..., in some cases you want to ensure it does *not* exist, tec.
(09:56:20 AM) jast: *etc
(09:56:42 AM) gac410: If actions are extensible, then the only thing that can know the format of the action/ulr   is the action itself.
(09:57:30 AM) gac410: jsonrpc cannot even have query params ... it's all down in the json data,  nothing to parse.
(09:57:38 AM) vrurg: For now it looks like RIP to the Address...
(09:58:10 AM) JulianLevens: It is entirely reasonable for a specific action to have special parsing, but generate an address object, most actions can use the standard parser
(09:58:31 AM) JulianLevens: The address object may be: raw, normalised or whatever
(09:58:34 AM) gac410: well.   Address can continue to exist,  as the way to pass internally a location.     but the parsing of the url is a different thing.
(09:59:15 AM) gac410: Anyway...  please add comments to the proposals.   We need to keep moving.
(09:59:20 AM) JulianLevens: y, my conclusion looking at Address is that it needs splitting up. The parse being one aspect to break out
(09:59:48 AM) ***vrurg supports gac410 on comments. We lack them.
(09:59:56 AM) JulianLevens: URL parsing is not the only place we need to parse addresses is it?
(10:00:34 AM) gac410: It's the only place I'm aware of where you have a long undefined string.  Most other places it's much more contained.    Topic="web.topic" ...
(10:01:01 AM) gac410: You would never have a rest subject or an attachment filename in a topic=" "  string I don't think
(10:01:23 AM) vrurg: Why actually complicate path instead of named parameters? Both in perl's and URL view of them?
(10:01:56 AM) gac410: well we have to live with our current url.   
(10:01:58 AM) vrurg: complicated even
(10:02:24 AM) jast: our current URLs are mostly straightforward
(10:02:45 AM) gac410: design decision a long time ago made rest, jsonrpc, and viewfile/xsendfile   rather oddballs.
(10:02:51 AM) JulianLevens: Macros are often passed a path, or we need to normalize the web topic combination
(10:03:38 AM) vrurg: Is it so difficult to start introducing named parameters?
(10:03:45 AM) gac410: web/topic    path... yes.      but not all the variants that can appear in a url.    I argue that when passed a web/topic,  that's a different problem 
(10:04:32 AM) JulianLevens: I consciously held back from comments on this proposal as Adsress really needs a proposal of it's own, and I'm still reading thru a lot of core code to understand how addresses are used
(10:04:53 AM) gac410: vrurg,  we have named parameters.   But the url issue still exists.    And we want simple url's as well     foswiki.org/Tasks/Item12345      not foswiki.org?topic="Tasks.Item12345"
(10:05:58 AM) gac410: Two minor proposals.   Clocks have expired.   Last Call!  https://foswiki.org/Development/AddFormatControlToSearchChanges
(10:06:20 AM) vrurg: f.o/Web/Topic is good enough. Easy to handle, standard. The rest to be moved into after the question mark area. This would simplify POSTing as well.
(10:06:22 AM) gac410: well.. this one is a bit more major:   https://foswiki.org/Development/AddPasswordManagementFunctions     but I need both for 2.2 
(10:07:39 AM) gac410: vrurg:   that's the direction we are migrating.  https://foswiki.org/Development/ContinueCanonicalSCRIPTURLDev     is trying to move everything into named parameters
(10:08:05 AM) vrurg: My last word on the Request/Address: it may significally slow me down on my work on Foswiki::App. It won't have any url/path related functionality. The new Request would be needed for this.
(10:08:40 AM) gac410: My intention is to get the Request subclasses into a feature branch you could merge.
(10:09:01 AM) ***vrurg needs few more hours in a day to fit all personal/work/project stuff into...
(10:09:06 AM) gac410: :)
(10:09:54 AM) gac410: The CanonicalSCRIPTURL    has some challenges.   1)  Supporting anchors.  That's a simple change to Foswiki::Attrs   support # as a atribute.   Already tested locally.
(10:10:53 AM) jast: fwiw I have no concerns with your two proposals
(10:10:54 AM) gac410: 2)  If we really want to support generating URLs from Macros, then we need to support duplicate parameters.    or alternatively allow a   foo=(a,b)  format that creates the URL  foo=a;foo=b   
(10:11:33 AM) gac410: And I'm thinking that duplicates woud be a major breakage of our macro parser,  so  (a,b) syntax would be better.
(10:12:34 AM) vrurg: gac410: I think these details we better leave off to corresponding proposal comments. Unfortunately, I didn't read the SCRIPTURL one thoroughly enough but would have to now, perhaps.
(10:13:22 AM) JulianLevens: I think we could implement duplicates in Attrs with a ^= syntax, quite easy I think. I'll add comments to this proposal
(10:13:27 AM) gac410: y.   Just pointing them out.    We need everyone to take time to review proposals and/or stop the clocks for proper consideration.   I get very concerned about stuff that slams in on 14 day expiration 
(10:14:25 AM) gac410: We've had some stuff end up in 1.2/2.0 by 14 day expiration that should really have been discusssed and changed.   Like the original configure rewrite before cdot did round-2
(10:14:44 AM) gac410: So as RM  I'm doing my part and raising the flags.   
(10:14:47 AM) gac410: :D
(10:15:24 AM) gac410: So... Then there are the 4 proposals/brainstorms  leading to Vrurg's  next gen foswiki.
(10:15:37 AM) gac410: vrurg,  do you have any specific input you need?
(10:15:41 AM) uebera||: What about switching to 21 day expiration in that case, btw?
(10:16:41 AM) vrurg: Most of all I need more considerations on https://foswiki.org/Development/MooFoswikiPm
(10:16:44 AM) gac410: Might make it a bit easier with the 14 day release meeting schedule.   But for now we can stick with "last call"  
(10:17:33 AM) vrurg: This is the core of the new design and so far it would completely depend on my competence which is not good enough be definition.
(10:18:32 AM) vrurg: s/be/by/
(10:18:51 AM) gac410: So one thing to clarify for me please.   With all the bruhaha over PSGI and the new vhosting supporting Foswiki::App ...    When all is done,  can I still run foswiki under apache with NO separate tasks, no shell access, no cron access ...   using only .htaccess for web / system configuration?
(10:19:27 AM) gac410: If I must start a PSGI container external to apache,  we've just knocked out a lot of web hosting sites
(10:20:08 AM) jast: the idea with PSGI is that it can be used in many different ways
(10:20:41 AM) jast: it's a standardized calling/message passing architecture and you can put it in its own container or just run it in a minimal CGI wrapper, etc.
(10:21:13 AM) vrurg: gac410: Yes, I'm not gonna drop off support for the old standards. Actually, it doesn't matter where you call Foswiki::App->run from – the rest is the core object job to find out what environment it's been run in.
(10:21:45 AM) gac410: I just know that wsgi on the foswiki.org server  was a nightmare.    if psgi is modeled after it,  I'm very reluctant to go near it.  
(10:22:06 AM) vrurg: The problem is that I won't be able to handle it all by my own only. It will depend on other people to implement support for those environments.
(10:22:51 AM) gac410: The old pootle translation framework used wsgi.   To me it was inscrutable,  a nightmare to install,   difficult to maintian,  we completely lost the configuration   twice ...
(10:23:18 AM) jast: similarly, WSGI can be set up in many different ways, though I've never figured out a way that wasn't extremely tedious and annoying
(10:23:21 AM) vrurg: Yet I hoped to hear from jomo a bit more on PSGI implementation details. But so far it might be not important anymore because localization of Foswiki::cfg shall work for the purpose.
(10:24:53 AM) jast: basically, in place of today's CGI scripts, you would create a script containing roughly the following:
(10:25:27 AM) jast: use Plack::Loader; Plack::Loader->auto->run(Plack::Util::load_psgi(".../foswiki.psgi");
(10:25:43 AM) gac410: regarding cfg localization.    we have Meta cache,  Password cache,  Mapping cache.   plugins that "use vars"   ...   can we GUARANTY   that two vhosts will be competely sandboxed   
(10:26:11 AM) MichaelDaum: they are already
(10:26:19 AM) jast: running several vhosts in the same app instance doesn't need to be the first concern, does it?
(10:26:27 AM) jast: we can always focus on making that possible later
(10:27:10 AM) MichaelDaum: gac410, psgi is not so much about virtualhosting
(10:27:23 AM) vrurg: Caches are easily incapsulated on App-level. use vars are better be moved to $app->heap
(10:28:28 AM) vrurg: Still it is all to be a concern only and only when threaded environment is used. Otherwise it's a plugin's job to reinit all vars properly.
(10:28:29 AM) MichaelDaum: we really do have to leave behind FCGI::ProcManager ... which is the current sort of app container 
(10:28:50 AM) gac410: We have an existing infrastructure of 500 plugins,  some of which "use vars"   If Joe site installs FoswikiNG or whatever and does two vhosts using common PSGI   will they be isolated.
(10:29:03 AM) MichaelDaum: gac410, dont worry.
(10:29:14 AM) gac410: We just this year found SpreadSheetPlugin   with a major issue of variable reuse.
(10:29:17 AM) jast: they'll be just as isolated as they are with today's VirtualHostingContrib
(10:29:29 AM) jast: and if you don't want to do that, you just run them as separate apps, just like now
(10:29:38 AM) MichaelDaum: xactly
(10:29:39 AM) gac410: Which is exactly why I WILL NOT INSTALL IT on my producgtion site.s
(10:29:48 AM) MichaelDaum: gac410, please
(10:29:50 AM) gac410: It does NOT sandbox 
(10:30:04 AM) gac410: okay... I'll drop it.  
(10:30:05 AM) jast: I get the sense you're mixing this up with something else
(10:30:14 AM) jast: PSGI is not about sandboxing and not about virtualhosting
(10:30:36 AM) jast: and the goal of introducing it is neither
(10:31:16 AM) jast: it's just about getting rid of all our home-grown Engine instances and all the duct tape holding that together
(10:31:43 AM) jast: PSGI is for taking over all the intricacies of CGI vs FCGI vs SCGI vs mod_perl, nothing more and nothing less
(10:31:46 AM) gac410: I just got the impression based upon some vrurg questions,  that it would be a core part of foswiki,  and I want to understand HOW it is fully sandboxed if we do make VirtualHosting core.
(10:32:11 AM) jast: how do you figure PSGI relates to VirtualHosting? I don't follow that particular connection
(10:32:18 AM) MichaelDaum: can we leave virtual hosting aside for a moment, please
(10:32:51 AM) jast: I'm not aware of any plans to make virtual hosting part of core
(10:32:51 AM) MichaelDaum: this is not part of the feature proposal
(10:32:51 AM) vrurg: gac410: Nah, it is not the core part. It's other way around: I want things to be done in a share-safe way. So that sharing is possible but not mandatory.
(10:33:45 AM) gac410: It was something that vrurg brought up a few days ago.    vrurg didn't you ask about "splitting" Foswiki::Cfg into separate vhosts or somethin like that ...   
(10:33:53 AM) ***gac410 need to go read the logs :(
(10:34:54 AM) gac410: http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2016-03-18,Fri&sel=154#l150
(10:34:57 AM) vrurg: gac410: It was the same I mentioned above: I wanted cfg to be incapsulated on per-app level.
(10:35:09 AM) jast: maybe that was about establishing some infrastructure to make it possible in the future to do clean virtual hosting? which would be a worthwhile goal, but obviously not something we can actually roll out for everyone any time soon
(10:35:11 AM) gac410: Correct me if I'm wrong but current LocalSite.cfg model is incompatible with multi-site serving, isn't it? 
(10:35:11 AM) gac410: I mean it's not possible to have per-site .cfg. 
(10:35:24 AM) gac410: (that was the reference)
(10:35:31 AM) jast: except with VirtualHostingContrib's hacks, yeah
(10:35:32 AM) vrurg: jast: Absolutely correct.
(10:35:39 AM) MichaelDaum: gac410, please can we move multi-sites etc out of the release meeting please
(10:35:46 AM) vrurg: gac410: I'm sorry it wasn't clear from the question.
(10:35:54 AM) MichaelDaum: I'd rather like to discuss psgi
(10:36:10 AM) MichaelDaum: no more virtualhosting discussions now, please!
(10:36:15 AM) gac410: If it is part of vrurg's design proposal, then it deserves some discussion.  
(10:36:16 AM) ***vrurg supports Michael.
(10:36:25 AM) MichaelDaum: back to psgi
(10:36:29 AM) vrurg: gac410: No, it isn't. Lets stop it. :)
(10:36:58 AM) MichaelDaum: ok
(10:37:09 AM) MichaelDaum: what we are dealing with here is perl legacy
(10:37:22 AM) MichaelDaum: something that the broader perl community as long solved for us
(10:37:30 AM) MichaelDaum: we need to adopt now
(10:38:15 AM) gac410: I have no objections to psgi   as long as it's not as  friggin complicated as wsgi was when trying to get pootle installed on foswiki.org
(10:38:32 AM) MichaelDaum: pootle is pootle, not foswiki
(10:38:40 AM) gac410: And as long as it will still run on webhosts   
(10:38:59 AM) gac410: Okay   I understand.   Just when someone explains PSGI as just like WSGI,    that scares me.
(10:39:14 AM) MichaelDaum: fear is not the right thing on change management
(10:39:28 AM) gac410: okay ...  it concerns me.
(10:39:35 AM) MichaelDaum: and we need to manage these chagnes with great care, of course.
(10:39:56 AM) MichaelDaum: there are a lot of changes coming up in a bundle introducing psgi
(10:40:09 AM) jast: the problem with WSGI is that it was made by people who think python is a good idea ;-)
(10:40:13 AM) gac410: foswiki has a reputation as very difficult to install.    and I found it 10x easier than  wsgi
(10:40:28 AM) MichaelDaum: there is no path morphing into that direction. this is a reinvention in broader parts.
(10:40:59 AM) MichaelDaum: guys, focus. please.
(10:41:03 AM) gac410: okay.   
(10:41:43 AM) gac410: So from jomo's  quo vadis  topic ....   it looks all very reasonable.   And after fighting with foswiki.org and fcgi zombies ...  seems like anything would be an improvement :D
(10:41:52 AM) vrurg: MichaelDaum: I'm not ready to present the final plan for Foswiki::App, but with proper community help it could be done in a way that it won't require waiting 4.0 to be PSGI compatible.
(10:41:59 AM) MichaelDaum: actually we do have a two-step (at least) approach proposed by vrurg: (1) adopt Moo (2) adopt psgi
(10:42:10 AM) gac410: right.
(10:42:37 AM) MichaelDaum: vrurg, sounds good
(10:42:53 AM) gac410: So far I'm liking what I'm seeing.   Though some of the changes will certainly be very disruptive.    Removing all positional parameters to purely blah=> 'asdf'
(10:43:47 AM) MichaelDaum: so the point on the table is: help vrurg
(10:44:02 AM) vrurg: gac410: positional parameters can be preserved whenever needed. Though I'd drop them.
(10:44:05 AM) MichaelDaum: we need more people working together on the moo side
(10:44:28 AM) gac410: we have 3 choices as I see it.   1) compatibility layer,   2)  Massive effort to rewrite  500+ extensions   3) Start fresh
(10:45:03 AM) gac410: we came close to 3 with the unicode change.   Lots of extensions are still broken.
(10:45:14 AM) MichaelDaum: gac410, have you already checked out the moo branch and gave it a spin?
(10:45:22 AM) gac410: Yes.    
(10:45:56 AM) MichaelDaum: okay I will do so too and then we will see what breaks
(10:45:59 AM) gac410: I've been working off&on   (more off though)  with vrurg.    Also did a merge from master so it was caught up a few weeks ago.
(10:46:48 AM) gac410: Last I checked,   the default foswiki was working fine.      Any non-default extensions though are toast.   given that Meta no longer has positional arguments.
(10:46:51 AM) MichaelDaum: I have got the feeling that most of us - including me - are still too uninformed about the consequences and feeling of a mooified foswiki
(10:47:16 AM) gac410: As a user I saw no difference.   worked fine.  Seemed as quick as normal on my laptop.
(10:47:30 AM) MichaelDaum: this is really encouraging
(10:47:38 AM) MichaelDaum: and a sign of the good work done so far, vrurg
(10:48:24 AM) gac410: configure,  everything ... evertying I tried worked,  except for EditRowPlugin,  which vrurg promptly fixed.
(10:48:24 AM) MichaelDaum: as a maintainer of almost 100 plugins out there I will definitely be suffering.
(10:49:10 AM) MichaelDaum: but I am fine with changes. I do not insist on the same plugins as they are run on moo-foswiki
(10:49:49 AM) vrurg: MichaelDaum: must note that the result is completely based upon the unit tests. I ran the UI 3-4 times, no more.
(10:50:26 AM) gac410: I wonder if we should make the mooified  API classes like Meta   into Meta2      or MooMeta     so that a compat layer could restore the old calling conventions.  like TWikiCompatibiity Pluign
(10:50:29 AM) vrurg: The latest commit might be broken.
(10:50:30 AM) MichaelDaum: we should call it fooswiki
(10:50:36 AM) gac410: :D
(10:51:04 AM) MichaelDaum: not foolswiki
(10:51:18 AM) vrurg: Foosewiki. Moose should replace Moo eventually in PSGI env.
(10:51:25 AM) jast: the naming of API classes should not change
(10:51:34 AM) jast: a compat layer can still work by doing nasty things
(10:51:50 AM) gac410: vrurg:    Does the new architecture eliminate "our"   global declarations?
(10:52:26 AM) vrurg: gac410: I'm not sure what level of compatibility is possible. Foswiki::App is gonna be absolutely no Foswiki.pm replacement.
(10:53:36 AM) vrurg: gac410: I would say "our" to be gone except for things like $VERSION and other non-config dependant staff.
(10:53:56 AM) gac410: good
(10:54:53 AM) vrurg: Though as long as non-shared code environment is used "our" won't break things whatsoever.
(10:57:08 AM) gac410: Okay everyone   anything else?  vrurg ??  
(10:57:42 AM) gac410: When you say you need help on the Moo ... what do  you mean.   Testing?    Merging from master again?   
(10:57:52 AM) vrurg: Thanks, that's all for this meeting. I just hope to get more activity in proposals and the new Request class. :)
(10:58:47 AM) gac410: I've got a few things to do, but I'll try to work on the Request class a bit this week.   I really didn't fully understand some of your comments.  but I'm not an OO perl guy.   
(10:58:51 AM) vrurg: A bit of bottomline on my side: for the moment the above two items are of the most importance to me.
(10:59:11 AM) gac410: Two items     Request and ??
(11:00:10 AM) vrurg: And proposals. Mainly the https://foswiki.org/Development/MooFoswikiPm as no point to move forward without Foswiki::App ready.
(11:00:19 AM) gac410: Lets wrap the meeting then.   I'm going to build Foswiki 2.1.1 sometime in the next 2 weeks.  
(11:00:56 AM) gac410: Ah... no no ... I need help with the Foswiki guest sessions.... How could I miss that.
(11:02:00 AM) gac410: I've got changes pending in the Item13989 branch.  
(11:03:48 AM) gac410: Just merged master into Item13989 branch.    This should let Registration and PasswordReset run without guest sessions.   I'd like to get that into 2.1.1,  so we can take advantage on foswiki.org
(11:04:01 AM) gac410: we are running 28,000+ cgisess files created every day.
(11:06:27 AM) gac410: So if someone could help test that branch,   that would be very helpful
(11:06:46 AM) gac410: And that's a wrap     Thanks everyone
(11:06:52 AM) vrurg: Looks like everybody is gone. Sorry, I'm not much help with sessions.
(11:07:17 AM) vrurg: Thanks gac410 
(11:07:21 AM) gac410: No prob.    
(11:07:32 AM) uebera||: yes, ty
(11:08:20 AM) gac410: Sorry to be disruptive on the virtual hosting.      It's okay if it slips in as a supported feature of core,  but we need unit tests IMO to prove that there is no leakage between hosts.  
(11:08:46 AM) gac410: ie,  don't break your Foswiki::App model to accommodate my ranting  :D
(11:09:31 AM) vrurg: gac410: Actually the model is to be developed yet.
(11:09:47 AM) vrurg: It's just an esquise for the moment.
(11:10:36 AM) vrurg: But what I expect from it is lesser memory leakage after the $app object gets destroyed.
(11:11:09 AM) gac410: That would be good.    Watching the fcgi processes on foswiki.org,  they grow quite a bit over time.
(11:11:26 AM) gac410: start out around 200k  and approach a gig after a while.
(11:11:48 AM) uebera||: but since you touched the config, the watchdog did not have to restart apache2 :)
(11:12:30 AM) vrurg: This is some perl deficiency too. It is known to be memory hog simply because it doesn't like giving back allocated chunks.
(11:12:31 AM) gac410: Y   I moved the "access count" out from our FastCGIEngine configuration  and into Apache.
(11:12:41 AM) vrurg: Long known issue nobody really cares about.
(11:13:04 AM) gac410: So our fcgi proc manager stuff is broken.
(11:13:28 AM) JulianLevens: Thanks all
(11:13:29 AM) gac410: The old foswiki.org did not have the FCGI::ProcManager class installed.
(11:13:36 AM) gac410: yes indeed.   Thanks everyone
(11:13:47 AM) MichaelDaum: thanks to everybody
(11:14:00 AM) vrurg: Thanks!
(11:14:10 AM) MichaelDaum: gac410, procman is only used when starting fcgi processes standalone ... i.e. not under the control of apache
(11:14:32 AM) gac410: Keep that proposal feedback coming, or raise your concerns.  The 14 day feature proposal process doesn't work if nobody exercises it.
(11:15:09 AM) gac410: MichaelDaum:   yeah,  but there is some difference there   I think procman is killing tasks somehow under apache as well, but they dont actually die.
(11:15:14 AM) MichaelDaum: fcgi::procman takes over the role of taking care of worker pool maintenance ...which web-servers sucgh as apache and lighttpd have build in ... but not so nginx ("keep the bloat out" kind of stance :) )

Topic revision: r3 - 22 Mar 2016, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy