Item10747: Slow initial save on new topics

On initial save of a new topic, there is a huge performance impact when using this plugin.

Needs to be investigated.

-- OliverKrueger - 15 May 2011

IIRC it does a search of all webs to ensure the newly generated permid is unique.

Certainly with bruteforce & forking it times out on our wiki.

With MongoDBPlugin it works fine.

IIRC the permid is generated from topic name + timestamp; maybe we can have a configure option to avoid the uniqueness validation (or get a better hashing algorithm so we can have more faith there won't be collisions)

-- PaulHarvey - 16 May 2011

Right now, there's a %SEARCH involved in every rest/PermLinkPlugin/view over all topics in all of the wiki...see get_topic_for_id() ... getListOfWebs() ... %SEARCH ...

So how about using dbi to store ids and to retrieve the real topic for a permalink. That'll be a nicer option for anybody else not using MongoDBPlugin. And it would not only speed up save but also every view of a permalink.

IMHO, permalinks - md5 as well as human readable ones - should be a core feature.

-- MichaelDaum - 16 May 2011

FWIW, this change brings the request times from ~2.2s on our VM to 1.6s:

diff --git a/lib/Foswiki/Plugins/PermLinkPlugin.pm b/lib/Foswiki/Plugins/PermLinkPlugin.pm
index 40d3adf..b10d4c2 100644
--- a/lib/Foswiki/Plugins/PermLinkPlugin.pm
+++ b/lib/Foswiki/Plugins/PermLinkPlugin.pm
@@ -76,20 +78,16 @@ sub get_topic_for_md5 {
 # a noobs implementation
 sub get_topic_for_id {
     ( my $method, my $id ) = @_;
-
-    my $webs = join( q{,}, Foswiki::Func::getListOfWebs() );
-    my $tml = "%SEARCH{ ";
-    $tml .=
-      "search=\"preferences[name='" . $method . "' AND value='" . $id . "']\" ";
-    $tml .= 'type="query" ';
-    $tml .= 'web="' . $webs . '" ';
-    $tml .= 'format="$web.$topic" ';
-    $tml .= 'seperator="," ';
-    $tml .= 'nonoise="on" ';
-    $tml .= 'limit="1" ';
-    $tml .= "}%";
-
-    my $topic = Foswiki::Func::expandCommonVariables($tml);
+    my $iterator = Foswiki::Func::query(
+        "preferences[name='$method' AND value='$id']",
+        undef,
+        {
+            web     => 'all',
+            recurse => 'on',
+            limit   => '1'
+        }
+    );
+    my $topic = join( '.', $iterator->next() );
 
     return $topic if Foswiki::Func::topicExists( q{}, $topic );
     return q{};

-- PaulHarvey - 16 May 2011

This is faster because MongoDBPlugin (and other query algo's) can roll this up into a single query instead of one query for each separate web.

Still, 1.6s is an eternity in the "Web 3.0" world for such a simple lookup task... here's hoping Item10461 will see an improvement on this.

Anyway, we should raise a separate task for the job of making a more efficient lookup implementation (I'd hope it's configurable so that I don't have to run a separate DB when I'm already using MongoDBPlugin).

The issue on this task is to figure out how to improve the save times when minting a new permid.

-- PaulHarvey - 16 May 2011

Commenting is disabled while not logged in