Garbled Characters in CSV, TXT, and JSON Imports

Description

After importing a .json file from disk or URL, some characters are garbled and substituted with the � character (U+FFFD).

Why is this happening?

The .csv, .txt, and .json files are actually plain text. Plain text can be any type of encoding. The encoding detection is an imprecisive operation using statistics and heuristics and works best if the first 4k bytes of data is mostly in one language.

Solution

If you are importing the file from disk, use a text editor such as Notepad++ or the iconv command in Linux to convert the file encoding to UTF-8 or UTF-8 BOM.

• Encode your your files as UTF-8, UTF-8 BOM, or UTF-16.
  

• If you are importing from a URL, download the file locally, change the file encoding, save it, and then import it from disk.