Item13785: ExtensionInstaller fails to replace files under some conditions.

pencil
Priority: Urgent
Current State: Closed
Released In: 2.1.3
Target Release: patch
Applies To: Engine
Component: Configure
Branches: Release02x01 master Item14288
Reported By: GeorgeClark
Waiting For:
Last Change By: GeorgeClark
Observed on foswiki.org

The MultiTopicSavePlugin is at version 1.10, but MultiTopicSavePlugin shows it being version 1.8
  • The .txt file in the package is correct
  • The file and history in the System web are verified as backlevel
  • Install with debug logs shows that the file is Installed (from cli) or Checked in (from web)
  • File still remains unchanged.

Verified on other systems, and the correct file is installed.

Marking urgent as there might possibly be other files incorrectly installed, however so far none have been observed.

I believe I also observed this during the extension updates following the original 2.0 upgrade, so I've marked the 2.0 and 2.0.1 codebases as applying.

-- GeorgeClark - 03 Oct 2015

Note history of MultiTopicSavePlugin, Even though the history shows the topic being updated, the diff between version 2-5 shows no changes at all. The topic is stuck on the initially installed version.

-- GeorgeClark - 30 Oct 2015

Crawford,

The problem appears to be somewhere in Meta.pm, or Package.pm is using it incorrectly. I cannot reproduce this locally, but I've added debug code on Foswiki.org.

In the following block of code, the two Data::Dumper calls, the first with $contents dumps version 1.10 of MultiTopicSavePlugin.txt. However after the saveAs call, $meta->text() returns version 1.8 of the same topic.
                if ($contents) {
                    print STDERR "RAW CONTENTS: " . Data::Dumper::Dumper( \$contents ) if ( $ttopic eq 'MultiTopicSavePlugin');
                    $reporter->NOTE(
                        "> ${simulated}Checked in: $file  as $tweb.$ttopic")
                      if DEBUG;
                    $checkedIn++;
                    my $meta = Foswiki::Meta->new( $Foswiki::Plugins::SESSION,
                        $tweb, $ttopic, $contents );

                    $ok = 0
                      unless $this->_installAttachments( $reporter, $dir,
                        "$web/$topic", "$tweb/$ttopic", $meta, $attached );
                    $meta->saveAs(%opts) unless $this->option('SIMULATE');
                    $reporter->NOTE( "meta->saveAs() called " );
                    my $tmpText = $meta->text();
                    print STDERR "META CONTENTS: " . Data::Dumper::Dumper( \$tmpText ) if ( $ttopic eq 'MultiTopicSavePlugin');

                }
 

-- GeorgeClark - 05 Nov 2015

It's not FCGI, I'm running out of ideas. I set up an Apache FCGI based server, installed MultiTopicSavePlugin version 1.8, and the ran the installer. Still works fine. It seems to be something specific to Foswiki.org.

-- GeorgeClark - 05 Nov 2015

Could this be the same as Item13845 ?

-- CrawfordCurrie - 20 Nov 2015

I don't think so. I don't see how there is anything trying to backdate the revision history. The changes should be applied using the current date/time. Also wouldn't it throw an error of some sort if rcs returned an error?

-- GeorgeClark - 20 Nov 2015

Bumped this to 2.0.4, and reduced from Urgent. The issue appears to be isolated to just MultiTopicSavePlugin. Others are being successfully updated.

-- GeorgeClark - 09 Dec 2015

We have some more hints here. Lynnwood ran into this issue on a system he supports
  • PlainFile Store contrib
  • FCGI system
  • Installing latest PublishPlugin
  • Observed that the topics changed
  • Then a short time later the topic returned to the previous version
  • The plainfile versions were backwards, with the original topic rev 2, and the new updated topic Rev 1.

Discussion starts here and continues on the next day.

I'm working on some possibly related changes for Release 2.2 ... they are too much for 2.1.3.

  • When the extension installer is run from CLI, it doesn't have a session and cannot check in any files. It appears that gaining a session is as simple as calling Foswiki->new('admin') if the session does not exist.
  • The "remove" function doesn't remove any of the ,pfv revision files.
  • The "backup" command doesn't backup the ,pfv subdirectories - it is only designed to back up individual files.

-- GeorgeClark - 27 Jan 2017

Crawford, I have an idea here after comparing what Foswiki::Func::saveTopicText does. In configure Package, it creates an "unloaded" Meta object and then saves it. Func::saveTopicText() makes a call to setLoadStatus so store considers the object loaded. I'm wondering if this is somehow related, although the Meta docs say this should never be called by anything other than Store's readTopic, Func clearly abuses that. Anyway, this is my latest patch. Thoughts? Could saving an unloaded object be causing issues?

(incorrect fix...)
diff --git a/core/lib/Foswiki/Configure/Package.pm b/core/lib/Foswiki/Configure/Package.pm
index 30ae8a9..45cb194 100644
--- a/core/lib/Foswiki/Configure/Package.pm
+++ b/core/lib/Foswiki/Configure/Package.pm
@@ -764,6 +764,11 @@ sub _install {
 
     $reporter->NOTE( "---+++ Installing " . $this->module() );
 
+    unless (  $Foswiki::Plugins::SESSION ) {
+        $reporter->NOTE( " Creating a SESSION" );
+        Foswiki->new('admin');   # Create admin session for topic checkins
+    }
+
     my $err;
     my $ext       = '';
     my $installer = '';
@@ -936,8 +941,7 @@ sub _install {
             my $topic = $2;
             my ( $tweb, $ttopic ) = _getMappedWebTopic($file);
 
-            #SMELL  Should not try to check in if target web is missing?
-            if (1) {    #Foswiki::Func::webExists($tweb) ) {
+            if ( Foswiki::Func::webExists($tweb) ) {
                 my %opts;
                 $opts{forcenewrevision} = 1;
 
@@ -971,7 +975,20 @@ sub _install {
                     $ok = 0
                       unless $this->_installAttachments( $reporter, $dir,
                         "$web/$topic", "$tweb/$ttopic", $meta, $attached );
-                    $meta->saveAs(%opts) unless $this->option('SIMULATE');
+
+                    unless ($this->option('SIMULATE') ) {
+                        $meta->setLoadStatus( $meta->getLoadedRev, 1 );
+                        my $newRev = $meta->saveAs(%opts);
+                        $meta->finish();
+                        $Foswiki::Plugins::SESSION->logger->log(
+                           {
+                              level    => 'info',
+                              action   => 'save',
+                              webTopic => "$tweb.$ttopic",
+                              extra    => "r$newRev - configure installed ". $this->module(),
+                           }
+                        );
+                    }
                 }
                 next;
             }

-- GeorgeClark - 01 Feb 2017

Potentially it could cause an issue, but TBH I have no idea what the effect might be. Could well be the reason. Only way to find out is to try it.

-- Main.CrawfordCurrie - 01 Feb 2017 - 08:13

Maybe the newly installed files have bad timestamps which the store doesn't like.

-- MichaelDaum - 01 Feb 2017

I think I've found the issue. It appears that only sometimes, Meta->attach() causes the topic text to be reload. The installer was attempting to update the topic text, attach any files, and then save the results. I made some changes to attempt to fix it, and it made it repeatable, where the topic always reverted. That made it easier to figure out what was causing the issue.

Checking in a fix. This goes a bit further in correctly backing and removing all files "installed" by an extension including ,pfv directories. They were being missed, both in the backup and the remove.

-- GeorgeClark - 01 Feb 2017
 

ItemTemplate edit

Summary ExtensionInstaller fails to replace files under some conditions.
ReportedBy GeorgeClark
Codebase 2.1.3 beta2, 2.1.3 beta1, 2.1.2, 2.1.1, 2.1.0, 2.1.0 beta1, 2.0.3, 2.0.2, 2.0.1, 2.0.0
SVN Range
AppliesTo Engine
Component Configure
Priority Urgent
CurrentState Closed
WaitingFor
Checkins distro:5eec0de413e1
TargetRelease patch
ReleasedIn 2.1.3
CheckinsOnBranches Release02x01 master Item14288
trunkCheckins
masterCheckins distro:5eec0de413e1
ItemBranchCheckins distro:5eec0de413e1
Release02x01Checkins distro:5eec0de413e1
Release02x00Checkins
Release01x01Checkins
Topic revision: r16 - 18 Feb 2017, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy