Feature Proposal: Fix SEARCH limit and paging mess
Note: Most of this proposal could be done as a bugfixing Task. But because this touches the API (historic and future) I would like to see it covered well.
Motivation
We have a number of outstanding bug reports on the issue that the
limit
param does not work well with paging:
In thinking and code those 2 concepts have been mixed up.
Old behaviour:
-
limit
sets the results per web. This cannot be changed because existing wiki apps rely on this.
Problems with
limit
combined with
pager
- Limit is somehow changed to the page size
- Results per web are grouped but also paged, meaning that the results per web are arbitrarily cut off.
- Result count per web is shown, but is actually the number of results shown ON THAT PAGE - very confusing.
- We still don't have a way to limit the total results.
Current implementation problems:
- in
Search.pm
, limit
is not obeyed: its value is set to pagesize
.
-
pagesize
is used even if pager
is off
. So you get one page of results without seeing the rest. This looks like a results limit, but the name and implementation are confusing.
- in WebSearch:
-
limit
is default set to DEFAULTPAGESIZE
, which is 50
in DefaultPreferences
-
pager
is hardcoded to on
, and cannot be set by url parameter.
-
pagesize
is set to DEFAULTPAGESIZE
, cannot be set by url parameter
Description and Documentation
Proposed fixes:
- Keep
limit
per web only.
- Do not combine web
limit
and paging: if limit
is set to a number (not 0
or all
), paging will not be used, and pager
and pagesize
will be ignored.
- Possibly do not combine limit with other kinds of grouping (like with the new
groupby
)
- Create a new param
maxresults
to limit the total number of results. This does not group results per web.
- In WebSearch enable all SEARCH options by url param
- In WebSearch set the defaults to no
limit
, no maxresults
, and pager
on
Examples
Impact
Implementation
--
Contributors: ArthurClemens - 03 Mar 2012
Discussion
What I was heading towards was that
limit
becomes deprecated, as it was a hack to deal with large result sets before we had paging.
so I would change the above list to :
- Keep
limit
per web only.
- Do not combine web
limit
and paging: if paging
is set to a number (not 0
or all
), limit will be ignored.
- Do not combine limit with other kinds of grouping (like with the new
groupby
)
- Use
pagesize
(and pager=""
) to limit the total number of results. This does not group results per web.
- In WebSearch enable all SEARCH options by url param - covered by other feat-req
- In WebSearch set the defaults to no
limit
, and pager
on - covered by other feat-req
There is another accepted and partially implemented feature proposal for adding paging to all shipped topics - I can't recal why i stopped part way though.
--
SvenDowideit - 04 Mar 2012
We should make more explicit documentation of
pagesize
and
showpage
to show how they can be used as
skip
and
limit
.
--
PaulHarvey - 05 Mar 2012
This does still not solve the problems when
limit
and
pagesize
are used together. This will happen by default in current wiki apps where
limit
is used.
SEARCH options
pager
and
pagesize
have a default value that work against
limit
. Together they will produce these problematic results. To view this, just use the current WebSearch.
Even if
limit
is deprecated we could give it precedence over paging. This would render results the old way, without paging.
Using
pagesize
as limit is quite obscure. If I wanted to show the 5 most recent topics, I would need to use
pagesize="5" pager="off"
. Who would think of looking that those option descriptions? It is less intuitive than just
maxresults="5"
(and of course the pager should not be shown if the number of results shown is smaller than
pagesize
).
--
ArthurClemens - 05 Mar 2012
I believe
ArthurClemens solution makes a lot of sense. I think that we already deprecated limit, so it should not be appearing in any of our templates. If it does, it should be removed. (Or we need to ship a
LegacyWebSearch topic which preserves the old limit-per-web behavior). If someone uses the deprecated
limit
they get the obsolete / deprecated per-web behavior.
--
GeorgeClark - 06 Mar 2012
I think adding yet more parameters is a bad idea.
limit
is deprecated, and pager is optional - so any apps that had limit in them must get edited to add a pager to them anyway.
but that said - there's a little more to it.
I made setting a total limit without a pager obscure for a reason - its rare that -_user_ wants to make the other results inaccessible - so adding a new single parameter that means the same as pager=off pagesize=5 is really odd.
I don't think need a legacy
WebSearch that does limit. There should already be a sufficient mix or modern elements (I think) to give a similar but superior result - for eg
FORMAT{
"WEBLIST"
format="SEARCH{
web="$item"
pagesize="10"
}"
}
would result in a
WebSearch that is grouped by web
and has a pager per web that can lead (I think) to the next page of only that web
OR
SEARCH{
web="public"
pagesize=30
groupby=web
}
would give you the first 30 matches, partitioning on the web name - so if each web has 45 hits, the first page would be just the first web, the second would have 2.
yes, both are not identical to what we have now - I think the first is superior, and the second 'interesting'
so I guess my biggest objection to
maxsize
is that it
is obscure, and probably should be obscure.
--
SvenDowideit - 09 Mar 2012
Developer is no longer contributing. Changed status to 'ParkedProposal'
--
Main.CrawfordCurrie - 11 Jan 2016 - 18:00