Feature Proposal: Move logs to 'working'

It has always annoyed the tits off me that logs are stored in data by default. It's so the wrong place for log data.

The proposal here is to move them to a new working/logs directory instead, and change their default names so they have a ".log" extension instead of the current .txt. Also define a $Foswiki::cfg{LogDir}, and move the individual settings for files (i.e. $Foswiki::cfg{ConfigurationLogName}, $Foswiki::cfg{DebugFileName}, etc) into EXPERT? It would mean there are less options in configure, which can only be good, and I doubt many people have the need to change the names of the individual log files anyway.

Not only does this make them significantly easier to find, it also removes the spurious .txt files from the data tree (so they don't get accidentally copied) and allows several installs to share the data area without stumbling over each other's logs.

This is a feature request to try and flush out whether people see any problem with this, and whether the new place is right.

Oh, and while we're at it, how about moving the default location for .htpasswd as well? Suggest working/htpasswd.

See Tasks.Item2008 for doc for inclusion in release notes.

-- Contributors: CrawfordCurrie - 05 Feb 2010

Discussion

I noticed in Tasks.Item2008 that we already ship a logs/ directory. This should be either used or removed.

Also in that task, I wondered if we could define a $Foswiki::cfg{LogDir}, and move the individual settings for files (i.e. $Foswiki::cfg{ConfigurationLogName}, $Foswiki::cfg{DebugFileName}, etc) into EXPERT? It would mean there are less options in configure, which can only be good, and I doubt many people have the need to change the names of the individual log files anyway.

-- AndrewJones - 05 Feb 2010

logs/ is much more intuitive than working/logs/.

-- OliverKrueger - 05 Feb 2010

I've kept my logs in logs/ for the last few years. Much better, though for customer sites I've left it in default location to save another thing to explain.

+1 for logs/

Re: htpasswd - should be in a different proposal.

-- MartinCleaver - 05 Feb 2010

Andrew says we distribute a logs directory.

That is not correct. We don't

There is one in SVN that someone must have created but it is not distributed. Ie not in any MANIFEST.

I would prefer logs in working because that would create less extra problems of directories to protect and directories to give the right access rights.

-- KennethLavrsen - 22 Feb 2010

The advantage of creating working/logs rather than logs is simply that as Kenneth says, permissions are easier to set up. I usually think about working as being like /var in linux, which is the sensible place for logs. By keeping them in working, it gives us scope to separate the logs used by different installs running off the same codebase - which is harder to do for a root logs. At the end of the day, the logs will only be read by admins, and as long as they can be found fairly easily, the extra dir level is IMHO moot. There has been no concern raised, so I guess the /logs fans aren't all that bothered....? I amended the description to account for Andrew's idea for a LogDir setting.

Accepted by 14 day rule.

-- CrawfordCurrie - 23 Feb 2010

Can we keep the $Foswiki::cfg{LogFileName} so plugins that write to this still work in future?

It was not part of the proposal. If it had been I would have raised concern because I would have known my plugin depended on it. I am reverting the acceptance because it was not the proposal that was up for debate for 14-days.

There is no API in Func that allows writing to the log in a way that you can control all the fields in the log.

So we plugin writers have had to rely on what we thought was stable config options. Especially this one as it was even documented.

It is a nightmare to maintain plugins if we keep on changing important names without considering compatibility. Foswiki is not even two years old and we start having 2 or 3 conditions in plugins to keep them compatible.

In my case it is BlackListPlugin that writes to the normal log.

Instead of maintaining $Foswiki::cfg{LogFileName} I could accept that it got deprecated in faviour of a full API call in Func to write to the log. At least that would remain stable forever (as long as I am there to defend the API).

-- KennethLavrsen - 24 Feb 2010

Damn, damn, damn, I'd forgotten about that stupid "feature" of the Func API.

Under no circumstances should a plugin ever write directly to the log. It's a (carefully) formatted file, processed by tools such as statistics, and I have painful experience of the sparks that fly when the format gets damaged. However you are right, the reality is that because there is no logging API in Func, some plugins have done so. Grrrr.

From SVN, {LogFileName} is used in Foswiki extensions here:

/home/foswiki/trunk/UserInfoPlugin/lib/Foswiki/Plugins/UserInfoPlugin/Core.pm:386:  $logFileGlob = $Foswiki::cfg{LogFileName};
/home/foswiki/trunk/BlackListPlugin/lib/Foswiki/Plugins/BlackListPlugin.pm:801:        my $log = $Foswiki::cfg{LogFileName};

Anyways, I think the best thing to do is to fix those plugins and the API.

-- CrawfordCurrie - 24 Feb 2010

UserInfoPlugin is only reading the logs, trying to find out who is online...a thing that consistently broke over the years the log format changed.

-- MichaelDaum - 24 Feb 2010

I fixed the API and the plugins. Kenneth, have I done enough for you to remove your concern? Basically, if you are upgrading an existing install, you have the choice whether to stay with the existing logs place, or move to the new logger.

-- CrawfordCurrie - 24 Feb 2010

Yes. You have done more than required to remove my concern. Thanks

Accepted by consensus

-- KennethLavrsen - 04 Mar 2010

Yup, working/logs is indeed fine by me.

-- MartinCleaver - 04 Mar 2010