Release Meeting: 9 Feb 2015

Details

1. Status update on Weblate

JanKrueger has a server up and running. Integrated with github, push and pull operations.

Translation for non-core Extensions.

MichaelDaum proposed Enhancei18nArchitecture, which appears to have been gradually evolved in the discussions, but did not settle on an accepted proposal. Now is the time to consider this, but we should not delay Foswiki 1.2. Weblate is different enough from Pootle that this needs to be revised.

Weblate's internal structure supports some interesting possibilities. Weblate supports "project-wide" translations, across multiple git repositories and cross-branch translations. It uses the concept of a "sub-project" which points to a VCS store for translation
  • Foswiki project
    • Distribution / master sub-project (Points to foswiki/distro master branch)
    • Distribution / Release01x01 sub-project ( Release 1.1 branch)
    • SomeExtension / master sub-project
    • ...
The Weblate translation project automatically synchronizes and translates common strings across the multiple repositories.

Need to decide quickly:
  • Go with existing monolithic structure for core, get 1.2 out.
  • Convert to Michael's proposed pluggable .po file structure (needs work!)

(Changing the locale file processing would violate our bugfix only / feature freeze. )

No! Tasks.Item9697 has already been merged to master, and supports per-extension translations: It is not documented

  • locales
    • Foswiki.pot default distro templates
    • <language>.po default language translations
    • SomeExtensionExtension specific translations installed with extension
      • Foswiki.pot Extension specific templates
      • <language>.po Extension specific translations

Item9697 also has a modified xgettext attached to support string generation for extensions, needs to be checked in.

This was all discussed some. It does look like the current repo will "just work". Some comments.
  • We still need to run the tools/xgettext tool to process foswiki specific syntax in topics and pl scripts into the Foswiki.pot
  • Weblate might be able to automate the merge from Foswiki.pot to the .po files, but manual is probably fine for now.
  • For now, push to the repository will also be manual, up to the language coordinators to do.
  • During the meeting, jast imported the Release01x01 branch to get an idea of cross-branch operation.
  • Still need to get a permanent IP assigned before announcing general use.

2. Release blockers

10 "Blockers", (2 reopened) none of which should actually block the alpha / beta process.

  • Configure tasks:
    • 13241 (Closed): Improve the email Auto-configure flow and operation. Needs feedback from Crawford, concerns about usability with some email servers.
    • 13181 (Closed): Config.spec PERL type scrambled, and configure crashes if PERL type contains any embedded comments. Reopened, PERL spec types are scrambled in LocalSite.cfg and become very difficult to read / edit.
    • 13169 (Closed): PERLDEPENDENCYREPORT is not reporting extension dependencies. Reopened, PERLDEPENDENCYREPORT fails on tarball releases. We don't package the DEPENDENCIES files for extensions. Solution appears simple - add DEPENDENCIES files to the extension MANIFEST files. Do we want to do this? Yes! Task fixed

  • Other tasks:
    • 13227 (Closed): Bootstrap fails under Mod_Perl and
    • 13228 (Closed): jsonrpc fails with no method if Apache2::Request is installed. POSTDATA is unavailable under Apache2::Request, Fixed, but needs review.
    • 13040 (Closed): Foswiki 1.2 issues on Nginx More FastCGI issues than Nginx. Fixed

  • Closed / Resolved / Defered blockers (as of this meeting)
    • 13220 (Closed): undefok NUMBERs generate a Number format error on an undef value Resolved but needs review.
    • 13227 (Closed): Bootstrap fails under Mod_Perl Resolved but needs review.

  • Not really blockers
    • 12855 (Closed): Audit core DEPENDENCIES - Admistrivia / Documentation update for when we are ready to build release packages.
    • 11262 (Closed): Update Installation Guide - We can solicit review from users testing the beta packages.
    • 13139 (Closed): Backwards compatibility issues with Foswiki 1.2 default extensions - Part of dependency review / testing effort. Need to test all 1.2 extensions against 1.1 (and 1.0.x?)
    • 12888 (Closed): Unit tests are failing - Catch-all. We had a recent jump in fails. 11 tests failing.
    • 13246 (Closed): Context header_text needs better documentation. NEW - Documentation only, not a blocker.

3. Where to release Alpha builds

I've built an alpha tarball, and opened some tasks based upon the install experiences. Where should we release alpha builds?
  • Attach to a topic on Foswiki.org (deal with RCS issues over time, or attach offline without RCS)
  • Use GitHub Releases (https://github.com/foswiki/distro/releases). Requires that we "tag" the repository and upload binaries to github.
  • Sourceforge

Some discussion. Opinions ranged from bi-weekly to nightly, with no hard solution . gac410 agreed to look at the old auto-build script which is currently not functional, and discuss with fsfs on that's something he could run when the unit tests pass. Otherwise, manual builds are probably sufficient. We don't have a lot of commits currently going on.

4. Other business

  • We are in feature freeze.... No commits of Enhancements. Bugfix only.
  • We will be in string freeze the moment Weblate comes online. Expect it shortly

  • Crawford reminded everyone to please review the Normal and Low priority tasks.
  • George reminded everyone that we need to review tasks and make sure they make sense for release notes.

Next actions

  • JanKrueger will continue to work on a Weblate server
    • Add Release01x01 branch as sub-project? Done
    • Backups?
    • Need permanent IP assigned?
  • GeorgeClark will contact SvenDowideit for help in reconfiguring DNS

Next meeting - - Monday February 23, 2015 1300Z

IRC Log

(07:42:23 AM) JulianLevens entered the room.
(07:58:59 AM) CDot entered the room.
(07:59:11 AM) CDot: good afternoon
(07:59:24 AM) gac410: Good  morning   :D
(08:00:01 AM) ***gac410 calling in from the arctic circle ... with yet another 24cm of snow on the way.
(08:00:57 AM) gac410: make that 45cm,   to early in the morning for metric conversions  
(08:04:00 AM) gac410: Hi everyone.   The agenda for today is at http://foswiki.org/Development/ReleaseMeeting01x02_20150209
(08:04:41 AM) gac410: First up was a status on Weblate and translations.    We have to wait for Jast for much of that discussion.  He hoped to be here if work schedule permitted.
(08:05:26 AM) CDot: no snow here; 7.2 degrees today! T-shirt weather!
(08:05:55 AM) gac410: But the server is running.   and is able to push/pull to github.        http://fwtranslate.modell-aachen.de/
(08:07:11 AM) gac410: Sven has been contacted and will update DNS once we have a permanent IP address.
(08:09:30 AM) JulianLevens: Goede middag
(08:09:43 AM) gac410: MichaelDaum:   You commented the other day you lost your script to gettext extensions.   It's attached to Item9697
(08:10:19 AM) Lynnwood entered the room.
(08:11:18 AM) gac410: So let's table Weblate for now,   (agenda #1)  and start with release blockers.  
(08:11:33 AM) CDot: do you want to wait for jast, or plough on with the agenda
(08:11:47 AM) ***CDot has limited time today, must leave by 13:30
(08:12:07 AM) gac410: Plough on...   and come back to weblate
(08:13:07 AM) gac410: Blockers was in good shape, ... until I repoened the mangling of PERL types again.    And another cosmetic configure issue.    I've been testing with alpha package installs
(08:13:14 AM) ***CDot fixed -T this morning, in a flash of inspriration
(08:13:15 AM) gac410: Sorry cdot
(08:13:34 AM) gac410: Yes I saw... excellent idea.   I didn't realize there was a way to enable it in runtime
(08:13:37 AM) gac410: Perfect solution!
(08:13:49 AM) CDot: yeah, I saw that. Started trying to reproduce your problem, and hit the -T issue that led me down another can of worms
(08:14:29 AM) gac410: I was not looking forward to dealing with uncommitted updates during the build.
(08:15:50 AM) gac410: Rather than go through blockers one-by-one.    List is at http://foswiki.org/Tasks/MinorReleaseBlocker   and also pasted somewhat into the agenda
(08:16:18 AM) gac410: The dependency report,  I have a fix all ready to go.   Just adds DEPENDENCIES to the shipped files.  
(08:17:34 AM) gac410: So other than configure,  everything else is mostly fixed or waiting for review  / documentation wordsmithing ...    I'm ready to build alpha's  
(08:18:12 AM) CDot: check BuildContrib to see what it does with DEPENDENCIES. i remember it embeds them in the installer.
(08:18:19 AM) gac410: Yes it does.  
(08:18:46 AM) ***CDot shuts up
(08:18:47 AM) gac410: But we don't ship the installers either.   The other fix would have been to rewrite the report to search for installers,  and ship the installers.   
(08:19:08 AM) gac410: nah... don't do that.   You still know this stuff better   :D
(08:19:41 AM) gac410: So.   Does anyone have anything to add on blockers  or state of open tasks in general?
(08:20:24 AM) ***gac410 gives time for everyone to think ... and bits to flow around the world 
(08:20:36 AM) CDot: nope. Though I need to run down the list of Normals/Lows again to make sure we didn't miss anything
(08:20:51 AM) CDot: and I'd recommend everyone else did the same
(08:20:55 AM) jast entered the room.
(08:21:10 AM) CDot: apart from that, testing, testing, testing - with odd configurations
(08:21:16 AM) gac410: +1 indeed.     Please everyone review the normal tasks     
(08:22:53 AM) gac410: I'd say we should start alpha builds.   I don't think that the nightly tests runs the autobuild any more.    So I'd propose a weekly or bi-weekly alpha drop  maybe timed to come out with our release meeting.   Any other ideas?
(08:23:12 AM) andreli entered the room.
(08:24:07 AM) gac410: Previous alphas were uploaded to foswiki.org topics.    I'm wondering if tagging them and uploading to github releases would be better
(08:24:31 AM) jast: or just decide to make another alpha at each release meeting
(08:24:54 AM) gac410: They would show up under https://github.com/foswiki/distro/releases       True.   on-demand might be better.
(08:25:35 AM) gac410: So our upload choices are    f.o,   github, or sourceforge.   
(08:26:09 AM) CDot: they need to come out more regularly than that
(08:26:23 AM) jast: what kinds of changes are made to release archives compared to git?
(08:26:25 AM) CDot: I'd vote for every checkin
(08:26:46 AM) CDot: or every day there is a checkin
(08:26:54 AM) gac410: then we need to automate.   right now it's manual 
(08:26:59 AM) CDot: yep
(08:27:05 AM) CDot: automation is good
(08:27:19 AM) jast: github auto-generates tarballs for all commits, it's just not publicly advertised
(08:27:41 AM) gac410: er.    right,   but it's not a build package.    Doesn't have any of our release process magic.
(08:27:49 AM) jast: yeah
(08:27:55 AM) gac410: No compression, etc.   Still have to pseudo-install
(08:30:09 AM) gac410: I think daily autobuild is overkill,  but let me look at the autobuild script.  Currently it's married to svn.   Maybe fsfs would be able to add it into the unit test run like Sven did,   and only build when the tests all pass . ... (hint hint  :) )
(08:30:19 AM) CDot: it doesn't need vast automation. if you can build at home, surely it can be automated at home?
(08:30:53 AM) CDot: it would be a shame if it wasn't automated. Continuous integration is good.
(08:30:56 AM) gac410: Well the github upload is drag-and-drop    or  there is a github API
(08:31:05 AM) ***CDot has run out of time and has to go. Back l8r
(08:32:34 AM) gac410: okay.  ...   alpha builds need review for automation.     Before we go crazy,  did *anyone* use the nightly auto-builds that Sven used to do?     I never did.  
(08:32:58 AM) gac410: For daily use I just used the svn checkout.   
(08:33:01 AM) MichaelDaum: sorry for being late. hey there.
(08:33:50 AM) gac410: Okay  lets take 2 minutes, michael can review logs,   and then we'll go back to Weblate discussion now that Jast is here.
(08:35:33 AM) JulianLevens: If this is logged automatically, where are they?
(08:35:40 AM) gac410: any more comments / opinions on alpha builds.     Automated daily?    Bi-weekly manually 
(08:35:59 AM) gac410: JulianLevens:   no logs  :(     I'll post my irc session later.
(08:36:34 AM) JulianLevens: I'd go for daily as long as there has been a commit
(08:38:07 AM) gac410: any other opinions.   if not ... Weblate.     Jast,  first   kudos  and thanks to you an modell-aachen    
(08:38:52 AM) gac410: I put notes in our agenda http://foswiki.org/Development/ReleaseMeeting01x02_20150209
(08:40:05 AM) gac410: so first question,  jast,   if you set up a sub-project for the   distro / Release01x01 branch,   I get the impression that weblate *might* be able to incorportate the 100% completed translations from release branch 
(08:40:16 AM) gac410: and give us a big jump forward. 
(08:40:53 AM) gac410: Is it as simple as just adding the branch through the web admin interface   or is there shell / batch work needed?
(08:41:25 AM) MichaelDaum: weekly builds are fine. we are low on checkins anyway.
(08:42:11 AM) MichaelDaum: and we need to make people aware of them. otherwise they only hang around. 
(08:42:30 AM) MichaelDaum: when are old builds going to be removed?
(08:43:19 AM) gac410: no idea yet.   We have not really even decided where we will put them.   
(08:43:56 AM) gac410: Sven used to host them on his nightly test server.
(08:45:39 AM) gac410: jast,  did we lose you again?
(08:46:22 AM) gac410: MichaelDaum:   Your xgettext-extensions   script is attached to http://foswiki.org/Tasks/Item9697
(08:46:41 AM) gac410: you commented you lost it iirc
(08:47:23 AM) MichaelDaum: thanks 1000x got it back on my disc
(08:48:52 AM) gac410: Weblate does translations per  "repository/branch",   but shares the translated strings in it's database across all  repo/branch instances in the "Project"
(08:49:49 AM) gac410: So it appears that the "Foswiki Project"    could have a    distro/master,    distro/Release01x01    ...   and other sub-projects    and weblate coordinates the cross-branch translations.
(08:51:01 AM) gac410: We could then add a  "SomeOtherExtension/master"   sub-project   and get it translated as well.     But it makes effort for Jast,  so we need to see how much we are adding to his workload 
(08:51:42 AM) gac410: I'd like to at least get the master & release branches of distro incorporated,   so we can leverage the 100% status of the release branch.   If that's possible.
(08:52:29 AM) ***MichaelDaum unsure what weblate can do for us and what constraints that puts onto core extension translations
(08:52:56 AM) MichaelDaum: i.e. how to load an extension's lexicon
(08:53:14 AM) MichaelDaum: into the core during compile time of foswiki
(08:53:28 AM) jast: let me just import the release branch and see what happens...
(08:54:08 AM) gac410: MichaelDaum:   I think you already did that.    Item9697    has a checkin that loops through locale/<extension>  directories adding in the .po files
(08:54:34 AM) MichaelDaum: I know, and I constantly backport this patch to 1.1.9 installing new foswikies
(08:54:52 AM) MichaelDaum: there are a couple of extensions that have their own locale files already
(08:55:51 AM) MichaelDaum: however, does weblate impose its own constraints that might require changes to the current mechanism?
(08:55:52 AM) jast: do we have an existing item for 1.1 translations?
(08:56:03 AM) gac410: Right.     So with weblate,  there are two impl. options,    1)   Add  "SomeExtension/master"   as a sub-project to Foswiki.       2)  Create a new independent project
(08:56:24 AM) gac410: MichaelDaum:   I don't think so.    The more I read, the more it appears it will just fit right in.
(08:56:34 AM) MichaelDaum: ah ok :)
(08:56:59 AM) gac410: jast,   There was one.     I closed it.
(08:57:00 AM) MichaelDaum: wow the patch is from 2011
(08:57:24 AM) jast: well okay, I'll defer that
(08:57:31 AM) gac410: It was Item9526
(08:57:43 AM) jast: creating 1.1 subproject
(08:57:49 AM) jast: this is going to take a few minutes, probably
(08:58:38 AM) gac410: okay.   Have you looked at what will be the flow?    Do you plan to have weblate just auto-push to github?     (pull --rebase && push)   
(08:59:19 AM) gac410: Or will the translator admin have to use the pushbuttons to push
(08:59:40 AM) jast: the latter is the way it's currently set up
(08:59:48 AM) gac410: I assume the github webhook trigger the automatic pull.
(09:00:27 AM) jast: it should
(09:01:05 AM) gac410: when do you think you'll have a permanent IP assigned so we can have Sven do the DNS update.     And have you figured out / enabled a backup process.  
(09:02:06 AM) gac410: I think manual push is just fine 
(09:02:11 AM) jast: I haven't figured out the management interface for IPs yet
(09:02:27 AM) jast: I tried assigning an IP and it didn't react at all... no success message, no error message :)
(09:02:41 AM) gac410: okay.   I assume you don't want the modell-aachen host revealed 
(09:06:11 AM) gac410: I've tried contacting Andre Ulrich ... he was the lead for the Pootle installation.    No response yet   
(09:07:40 AM) gac410: MichaelDaum:    Basically when you add a distro/branch,   you point it at the location within that branch for the .pot and .po files.   And it takes it from there.   (Jast keep me honest here :) )
(09:08:32 AM) gac410: So a distro/master   points at   core/locales/           and with your translated extensions,   it would be  locales/<extension>/locales     
(09:08:51 AM) MichaelDaum: so how do new non core extensions show up in weblate? is it done manually?
(09:09:13 AM) jast: yes
(09:09:27 AM) jast: could be automated later on, in principle
(09:10:02 AM) gac410: Jast has to create a new sub-project within the Foswiki project.      Jast,  do you have to run a command to actually add it in?
(09:10:04 AM) jast: I don't care about revealing the hostname btw, but it doesn't make much sense to have two names
(09:10:27 AM) gac410: okay.   I was just thinking someone with a 2nd language ought to review this.    I'm useless  :D
(09:11:03 AM) gac410: The key is to use the same userid / email that you used on Github when you register.    
(09:11:15 AM) jast: currently adding the subprojects is done via web interface, but I think there's a cmdline way, too
(09:11:47 AM) jast: it's still busy importing the 1.1 branch btw
(09:12:24 AM) gac410: MichaelDaum:  So if you register with your github credentials,   then when you translate and push.   The "committer"  will be your identity,  and the "push" will be by the FoswikBot identity.
(09:12:44 AM) gac410: er....  got that wrong.    The Author will be your identity, and the committer will be FoswikBot
(09:13:35 AM) gac410: jast   yeah I expect that there are a LOT of strings to try to merge between all the po files.    I hope that the webserver does not time out the operation.
(09:14:57 AM) gac410: And if all the magic comes together,    when FoswikiOrg gets the push notification,  it should pick up the Author credentials and record the commit against the Translation task under your WikiName
(09:16:20 AM) gac410: While we are waiting. ...   Another area we really need some help,   is figuring out which major changes need to be highlighted in the release notes.
(09:17:22 AM) gac410: And every task  that is "Waiting for Release"  needs its commit message reviewed.    Does it make sense to someone reading a one-liner in the release notes.   
(09:20:17 AM) gac410: So everyone.    Please review http://trunk.foswiki.org/System/ReleaseNotes01x02     Think about what needs to be added.    There is a lot of work hidden here given how long 1.2 has been in development, and the # of merged proposals
(09:22:01 AM) gac410: jast ... another translation question.     We still need to run the tools/xgettext perl script,  to extract string changes into the Foswiki.pot file and merge them into the .po files.
(09:23:55 AM) gac410: Do you know .. The extract into Foswiki.pot is specific to foswiki,   but the merging to .po files is part of gettext tools.      Does weblate do anything there to help us?    
(09:24:23 AM) gac410: ie,   can we just update Foswiki.pot and let weblate do the merge?   or does our current process stand as-is
(09:31:53 AM) gac410: Okay.   any new business for the official release meeting?     I think we are in really good shape to start alpha's and with some help on release notes,  and translation underway,  a beta build.
(09:33:07 AM) gac410: I'll look at automating alpha builds.  Either with fsfs help,   or gmc.   I don't have anywhere to host them myself. 
(09:34:01 AM) gac410: Next meeting is Monday Feb 23rd  1300Z    
(09:35:08 AM) andreli: Looking forward to the alpha builds. Thanks a lot gac410.
(09:35:22 AM) gac410: thanks andreli
(09:36:04 AM) gac410: Anyone interested in translations.   Dont forget   there is a foswiki-translations email list.   Please subscribe 
(09:40:48 AM) JulianLevens left the room.
(09:42:32 AM) andreli left the room (quit: Quit: Page closed).
(09:44:44 AM) gac410: Okay everyone.   Thanks for attending.      I'll get the minutes posted later today.

Topic revision: r3 - 10 Feb 2015, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy