File Exchange Accented Characters encoding issues

[things4craft]

Hi

I am using File Exchange to add listings to ebay.fr

The listings create fine but do not cope well with accented characters

Épingles becomes Épingles

The Action Header in the file uses CC=UTF-8

I had a look at the file header in a hex editor and the first 3 bytes are ef bb bf which would suggest the file is UTF-8

The É char is represented by c3 89 Which again looks good.

I have trawled google and these forums but cannot find a way to fix this.

Any help would be appreciated.

If someone from ebay picks this up, #909282902 was a file upload ref.

Thanks

Rich

[dollybeauty]

@things4craft 

Try 'CC=ISO-8859-1'.  Instead of CC=UTF-8.

[things4craft]

@dollybeauty 

Thanks but didn't work either

still getting  Ã‰pingles

[dollybeauty]

@things4craft 

Let me call another who may be able to help - @shipscript 

[shipscript]

@things4craft 

The eBay.fr File Exchange is supposed to accept a UTF-8 file, although I have not personally tested FR.  We have confirmed that the following sites do NOT work with UTF-8, no matter how the data is presented:

US, CA, AU, UK, DE

This is what our testing discovered on the sites that do not accept UTF-8:

• When extracted from the source as UTF-8 and stored in a File Exchange CSV (never touching a spreadsheet tool), we found that CC was irrelevant. File Exchange expected to see ISO and corrupted the special UTF-8 characters.
• When setting the leading file bytes to ensure the file was interpreted as UTF-8, File Exchange rejected the invalid headers.
• When supplied as UTF-8 with CC=UTF-8, the special characters were corrupted.
• When supplied as UTF-8 with CC=ISO-8859-1, the special characters were corrupted.
• When supplied as ISO-8859-1 or as CP-1252 with CC declared or undeclared, the special characters displayed correctly. 

I think File Exchange was specifically matched to the English language version of Excel on the tested sites.

These are my test listings from two years ago that the File Exchange staff also tried to duplicate on the US site 6 months ago with the same results:

• 222467669781 - This original listing was created manually on SYI. eBay stores listing data on their server in UTF-8 format, so they convert from our submission format to UTF-8 when they save data to their servers.
   
• 382036049366 - This shell listing was populated by File Exchange by copying the UTF-8 source from listing 222467669781 without adding CC (same result as next test).
   
• 382036124098 - This shell listing was populated by File Exchange by copying the UTF-8 source from listing 222467669781  with UTF-8 header metadata.

I was told that FR was different, but I haven't tested.

If you want to try using CC=ISO-8859-1 , then the CSV file you save from your spreadsheet should also be saved in that format. That means you must not see the leading 3 bytes in your spreadsheet file that indicate UTF-8. My server converts from UTF-8 to CP-1252 (very similar to ISO-8859-1 and accepted by File Exchange), which works for most symbols.

Since you see three leading bytes, and File Exchange does not reject your headers, my assumption is that File Exchange is accepting the file as UTF-8 data, which I understand should work in FR, so this is puzzling.

Because you are able to test these characters and seem to understand the underlying hex involved, I would be most interested in your test results because I can not list on the FR site and test there.

[things4craft · Accepted solution]

@shipscript 

Thank you, this enable me to figure it out.

Ebay.fr seems to ignore CC= so setting ISO-8859-1... wont work if the first 3 bytes indicate utf-8

We have created two successful files, one using our c# app and the other using excel

The excel csv

Removed CC= altogether and saved as csv (western european iso)

Our Raw file from our app

Removed CC= altogether and encoded the file as ISO-8859-1

So it seems that the first 3 bytes are crucial in getting this to work on ebay.fr

We ran one final test:

Encoded the file as ISO-8859-1 but set CC=UTF-8

This worked also - meaning ebay.fr ignored CC= when the file was encoded with ISO-8859-1

Thanks again for your help

[shipscript]

@things4craft 

Thank you for sharing your methods. What you have found is the same we have found for the other sites. When the file is saved as ISO-8859-1, the CC in the header is ignored.

I suspect that sellers who use Excel to create their spreadsheets leave the CC=UTF-8 in the header and are saving a CSV in ISO format without understanding that Excel is not saving as UTF-8.  Perhaps we were all assuming UTF-8 was working for ebay.fr when it really was not.