Chapter 7 - Character Set Translation

This chapter describes how you can define how the Tachyon File Tools programs are to perform translation between EBCDIC, ASCII and Unicode character sets.

Code Points and Code Pages

There are numerous ways to define which code points (numeric values) are assigned to which characters. Each of these schemes can be called a code page. IBM has assigned numbers to many of the different code pages.

Since the Tachyon File Tools programs can read and write files containing characters encoded in EBCDIC and/or ASCII, the programs need to know how to translate between these character sets. Two environment variables are used to tell the Tachyon File Tools programs in which EBCDIC and ASCII code pages they are to assume that the characters are encoded.

Tachyon File Tools supports over 90 different Single Byte Character Set (SBCS) code pages. All of the EBCDIC and ASCII code pages are defined in terms of their translation to and from Unicode. When translating between EBCDIC and ASCII, characters are effectively converted first to Unicode and then to the target code page.

Normally any two given code pages will not define the same set of 256 characters, so usually some characters cannot be translated between the code pages. The Tachyon File Tools programs require that the uppercase letters A-Z, the lowercase letters a-z and the digits 0-9 translate to their usual code points. The national characters at EBCDIC code points X'5B', X'7B' and X'7C' must also be translatable to ASCII. In code page 37 (EBCDIC USA), these code points correspond to the $, # and @ characters. If the EBCDIC national character at X'C0' is translatable to ASCII, that character may be used in data set names.

Note: Some EBCDIC code pages such as 290 (EBCDIC Katakana), 803 (EBCDIC Hebrew) and 1030 (EBCDIC Katakana Extended) do not define the lowercase letters a-z to their normal code points. These code pages are not usable by Tachyon File Tools.

Environment Variables

Tachyon File Tools recognizes two environment variables for the purpose of overriding the default ASCII and EBCDIC code pages. The environment variables are:

CODEPAGE_ASCII

The value of this environment variable must be a decimal number that is one of the supported ASCII code pages. If the CODEPAGE_ASCII environment variable is not set, the default is code page 819 (ISO-8859-1 Latin-1).

CODEPAGE_EBCDIC

The value of this environment variable must be a decimal number that is one of the supported EBCDIC code pages. If the CODEPAGE_EBCDIC environment variable is not set, the default is code page 1047 (Latin-1).

The default set of code pages (ASCII 819 and EBCDIC 1047) provide the same translation between ASCII and EBCDIC as was provided by default in versions 1 and 2 of Tachyon File Tools. These are also the default code pages used by z/OS UNIX Systems Services, the Tachyon z/Assembler and the Tachyon Operating System.

EBCDIC Code Pages

ASCII Code Pages