Item13834: EditRowPlugin cannot edit cells that contain Umlauts and other non-ASCII characters.

(Using a Foswiki 2.0.2 with UTF-8 store)

To reproduce, create any simple table, for example

First	Second
One	Zwöi
Three	Four

If you then change cell 'Two' to 'Zwöi', different things happen depending on whether you're running on CGI or mod_perl:

CGI

Everything looks fine after saving the cell. But when you open it again for editing, the 'ö' is turned into a '�' character.

mod_perl

Everything looks fine after saving the cell. But when you open it again for editing, the cell content changes to "Loading..." and stays that way.

-- PascalSchuppli - 21 Oct 2015

Two separate issues here.

• The UTF8 encoding issue. I'm unable to recreate it, confirmed on foswiki.org, locally. Even in this task topic. Click the corner paint on the utf-8 field and it's incorrectly encoded.

• The "Loading..." confirmed ... When you click the corner paint to open a cell, the rest request is responded to, but the close quote is missing. So on a CGI system, the response is "Zwöi", but on a mod_perl system,. it responds with "Zwöi, no closing quote. ... Digging a bit further, fields containing utf-8 are double-quote enclosed. Fields with ascii are single-quote enclosed. That might be related. Not really sure though what's going on.

(Behaves differently on foswiki.org. Always double-quoted, but no quotes eaten. Now I'm wondering if it's an issue in the JSON module that builds the response.

Crawford... any ideas? I'm dumping the response in EditRowPlugin/Get.pm just prior to the JSON encoding. Both quotes are present there, but the trailing quote gets eaten when received from the web GET request.

-- GeorgeClark - 22 Oct 2015

I'm wrong on the closing quotes. The issue on the mod_perl system, we are calculating the length of the string using characters instead of bytes. So each 2-byte character shortens the response by 1.

I have a fix. EditRowPlugin/Get.pm generates results using the body() call, which does not utf-8 encode the results. This also throws off the length calculation. Changing Get.pm to call print() causes the results to be utf-8 encoded, and the length is then calculated correctly, fixing both issues.

-- GeorgeClark - 22 Oct 2015

Commenting is disabled while not logged in