Re: unicode (was Re: Minor error message change)

["John E. Davis" <davis@xxxxxxxxxxxxx>]

Pavel Roskin <proski@xxxxxxx> wrote:
>Now, the encoding put to the locale is a different thing.  If I set
>LANG=ru_RU.koi8-r, it meant that my terminal has a koi8-r font.  If the

So far I have been unable to find any documentation that implies this.
The best I have found is from the setlocale man page at
http://osr5doc.ca.caldera.com:457/cgi-bin/man/man?locale+M, which
states:
  
   codeset
          Represents the character set in use for the internal
          representation of text.

It is hard for me to infer from the above that codeset also specifies
the character set of the terminal.  On the other hand, the
solaris-specific documentation at
http://docs.sun.com/db/doc/806-6642/6jfipqu5h?a=view
indicates that if the terminal does not support UTF-8, then one can
use the appropriate streams kernel module to perform the UTF-8 to
whatever conversion.

[...]
>Applications working with text data should have means to determine its
>encoding base on standards, user actions and user preferences (and maybe
>some guess work, like domain name of the sender).  Anyway, it's completely
>the responsibility of the application.
>
>So, the right approach for S-Lang would be to assume terminal capabilities
>from the locale.  The application should tell S-Lang about the encoding
>it's using for output via S-Lang and about the encoding it expects to get
>from S-Lang.

At the moment, all I plan to support in the initial versions of slang
2 is UTF-8, or no assumed character set.  In the latter case, a
character will be represented as a byte, much in the same way as
version 1.x of the library does now.  I think that it is too much to
expect the library to portably perform, e.g., koi8 to UTF-8
conversions on the fly.  The way I intend to handle this with jed is
that jed will perform the character set conversions to and from its
internal UTF-8 representation when files are read from or written to
the disk.

Thanks,
--John