FTP Connector encoding

  • 2 August 2022
  • 1 reply

Userlevel 2
Badge +4

Can you tell me how to encode the File data for the FTP connector when using the FTP method 'Upload file (STOR)’?

It seems that the connector uses the UFT-16 Little Endian although the source for the data comes from a VARCHAR.

1 reply

Userlevel 5
Badge +2

Hello Onno,

We can't tell you how you should encode the file data for the FTP connector, because the question is how you want your file data to be encoded. There is no right or wrong answer here, it all depends on the situation.

With that said, encoding is almost never relevant or useful in the various file connectors that are supported by the Thinkwise platform. This question usually arises when an existing file is read by a Read File connector, stored in a VARCHAR process variable, written/uploaded by a Write File or FTP connector and then the final result is a corrupted file which is not the same as the original. If this is the case, then the real question is “How do I prevent files from becoming corrupted by the FTP connector?”. The answer is to use a VARBINARY process variable, instead of a VARCHAR process variable and leave the encoding input parameter blank.

The concept of encoding is only relevant when transferring back and forth between binary and text representations. When dealing with files, it is always better to take a “binary all the way” approach, which eliminates the question of encoding entirely. The encoding input parameter is only useful when dealing with plain-text files (such as .txt, .ini, .json) and only in the cases where you must work with a VARCHAR value (e.g. you already have a VARCHAR value and want to write a plain-text file or you have a plain-text file and you want to do text-based operations on its contents in SQL). Then the question becomes, what is the encoding of your plain-text file or what do you want the encoding of your plain-text file to be? The answer is typically UTF-8 or ANSI, but it really depends on the situation.

I hope this explains a few things. Please let me know if you have further questions about this.