You are here: Foswiki>Tasks Web>Item11326 (05 Jul 2015, GeorgeClark)Edit Attach

Item11326: SubscribePlugin is extremely slow with large complex WebNotify topics.

pencil
Priority: Urgent
Current State: Closed
Released In: 2.0.0
Target Release: major
Applies To: Extension
Component: MailerContrib
Branches: trunk
Reported By: GeorgeClark
Waiting For:
Last Change By: GeorgeClark
Topic view for logged-in users on Tasks web is extremely slow.

  • View any task or non-search topic as WikiGuest ~ 1 second to return the main document
  • Log in - same view takes 13-18 seconds to retrieve the base document.

Other factors:

cover=print Fast
skin=print Fast
skin=plain Fast
skin=default SLOW
raw=on or raw=debug SLOW
raw=text or raw=all Fast

  • Running ./view -user GeorgeClark -topic Tasks.Item11051 is Fast
  • Checked with the release01x00 vhost - seemed faster, but still very slow
  • Cloned Tasks web to another server. Seems fast in all conditions.
  • Logged in as admin superuser. Still slow.

-- GeorgeClark - 04 Dec 2011

The problem is the SubscribePlugin. It is adding 10-20 seconds to the rendering time. Disabled for now.

-- GeorgeClark - 04 Dec 2011

I feel as if this is a recent thing. And trunk is much faster. I've reproduced locally.

-- PaulHarvey - 04 Dec 2011

It happens on trunk code, something to do with Main web, I think.

-- PaulHarvey - 04 Dec 2011

I've tried versions of MailerContrib back to distro:9c88747740c3, but it's all still terribly slow..

-- PaulHarvey - 04 Dec 2011

Support web recommends that users should subscribe to their own question topics, but we've got SubscribePlugin turned off, so they can't.

Added WebSiteTasks to reflect this

I changed target to patch, given that we need to fix this for foswiki.org

-- PaulHarvey - 03 Mar 2012

The workaround for this is to set DISABLEDPLUGINS = SubscribePlugin in Tasks.WebPreferences (and in any other webs with exceedingly large WebNotify topics.

I added a config setting - {SubscribePlugin}{AvailableWebs}, and planned to handle the issue that way, but just disabling the plugin is just as effective, provided that it's not overridden in a topic. It might be best to add this feature anyway, as it gives a bit more control to the administrator.

I'm opening a separate task to add that feature, and will re-title this to deal with the performance issue directly.

Setting this task to be a release blocker for 1.2. The performance issues are too extreme to enable this as a default plugin.

-- GeorgeClark - 03 Mar 2012

the problem is due to the fact that the code that adds entries to a subscriber set attempts to optimise the set each time an entry is added. This is an O(N^2) process, but guarantees a fast lookup when searching for matches in the set. So there's a comromise here; by skipping the optimisation on the read we accept there may be some suboptimal entries. Optimising when a new webnotify is written means we get the best of both worlds.

Not quite finished here yet; the load is still too slow, need to take a closer look at that.

-- CrawfordCurrie - 22 Jan 2013

OK, I'm done. It should be useable on f.o again now. No changes to the SubscribePlugin, all changes were in MailerContrib.

-- CrawfordCurrie - 23 Jan 2013

 

  • nytprof.tar.gz: NYTProf output using trunk code and foswiki.org's data
I Attachment Action Size Date Who Comment
nytprof.tar.gzgz nytprof.tar.gz manage 7 MB 04 Dec 2011 - 06:28 PaulHarvey NYTProf output using trunk code and foswiki.org's =data=
Topic revision: r14 - 05 Jul 2015, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy