Item13442: Convert all Foswiki.org content to UTF-8

pencil
Priority: Urgent
Current State: Closed
Released In: n/a
Target Release: n/a
Applies To: Engine
Component:
Branches: master
Reported By: CrawfordCurrie
Waiting For:
Last Change By: CrawfordCurrie
As part of the forthcoming upgrade to Foswiki 1.2.0 on Foswiki.org, we need to convert existing content to UTF-8 and set {Site}{CharSet} appropriately.

It's clear from simple analysis that there are a mixture of encodings in use on the site, mainly where people have pasted encoded content into the text editor and the bytes have been trivially saved without re-encoding. The encodings detected are:

  • 1464 strings are recognised as cp-1252
  • 72 as UTF-8
  • 30 are recognised as a different encoding
    • Big5
    • EUC-JP
    • EUC-KR
    • IBM866
    • ISO-8859-5
    • ISO-8859-7
    • ISO-8859-8
    • KOI8-R
    • Shift_JIS
    • gb18030
    • windows-1251
    • windows-1255
    • x-mac-cyrillic

To support this change I have added a "repair" option to the CharSetConverterContrib.

-- CrawfordCurrie - 01 Jun 2015

Looking good (George did most of the work)

-- CrawfordCurrie - 05 Jun 2015

 
Topic revision: r6 - 05 Jun 2015, CrawfordCurrie
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy