You are here: Foswiki>Tasks Web>Item2427 (29 Nov 2009, OliverKrueger)Edit Attach

Item2427: http://foswiki.org/robots.txt does not exist, though it should

pencil
Priority: Normal
Current State: Closed
Released In:
Target Release: n/a
Applies To: Web Site
Component:
Branches:
Reported By: MarkusUeberall
Waiting For:
Last Change By: OliverKrueger
On second thought, apart from the robots.txt file, maybe we could also add a sitemap.xml file (should be placed in the root directory as well and can also be referenced by the former).

  • The Disallow directives should also help to weed out 'redundant' (i.e., merely reformatted) content
  • Using a sitemap, we could give WebHome topics a higher priority (>0.5)

Note that we need one robots.txt[ / sitemap.xml pair] per subdomain!

-- MarkusUeberall - 28 Nov 2009

Added an Alias /robots.txt "/path/to/robots.txt" to the apache config file for foswiki.org as suggested in ApacheConfigGenerator and restarted the daemon.

-- OliverKrueger - 29 Nov 2009

ItemTemplate edit

Summary http://foswiki.org/robots.txt does not exist, though it should
ReportedBy MarkusUeberall
Codebase
SVN Range Foswiki-1.0.7, Sun, 20 Sep 2009, build 5061
AppliesTo Web Site
Component
Priority Normal
CurrentState Closed
WaitingFor
Checkins
TargetRelease n/a
ReleasedIn
Topic revision: r2 - 29 Nov 2009, OliverKrueger
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy