JezK
Edit File: AboutUnicode.html
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Appendix C. About Unicode, UCS-2, and UTF-8</title><link rel="stylesheet" type="text/css" href="userguide.css" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><link rel="home" href="index.html" title="FreeTDS User Guide" /><link rel="up" href="index.html" title="FreeTDS User Guide" /><link rel="prev" href="interfacesformat.html" title="What it looks like" /><link rel="next" href="ISO8859.html" title="ISO 8859: What everyone would like to forget" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix C. About Unicode, UCS-2, and UTF-8</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="interfacesformat.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="ISO8859.html">Next</a></td></tr></table><hr /></div><div class="appendix"><div class="titlepage"><div><div><h1 class="title"><a id="AboutUnicode"></a>Appendix C. About Unicode, UCS-2, and UTF-8</h1></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl class="toc"><dt><span class="section"><a href="AboutUnicode.html#ascii"><acronym class="acronym">ASCII</acronym>: What everyone knows</a></span></dt><dd><dl><dt><span class="section"><a href="AboutUnicode.html#ASCIICompact">The <acronym class="acronym">ASCII</acronym> Compact</a></span></dt></dl></dd><dt><span class="section"><a href="ISO8859.html">ISO 8859: What everyone would like to forget</a></span></dt><dt><span class="section"><a href="Unicode.html">Unicode: East meets West</a></span></dt><dt><span class="section"><a href="Unicodegoodbad.html">Unicode's Pluses and Minuses</a></span></dt><dt><span class="section"><a href="UnicodeUtf.html">Unicode Transformation Format: UTF-8</a></span></dt><dt><span class="section"><a href="UnicodeFreeTDS.html">Unicode and <span class="productname">FreeTDS</span></a></span></dt><dd><dl><dt><span class="section"><a href="UnicodeFreeTDS.html#moreinfo">For further information</a></span></dt></dl></dd></dl></div><p>For better or worse, <span class="productname">FreeTDS</span> brings the otherwise innocent programmer into contact with the arcane business of how data are stored and transported. <span class="productname">FreeTDS</span> is a data communications library that of course connects to databases, which are charged with storing information in a way that is neutral to all architectures and languages. On the surface, that might not seem very complex, even worth discussing. Under the surface, things are not so simple.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="ascii"></a><acronym class="acronym">ASCII</acronym>: What everyone knows</h2></div></div></div><p>The world we are all familiar with, programmingwise, is <acronym class="acronym">ASCII</acronym>. Our email (mostly), our <span class="quote">“<span class="quote">text</span>”</span> files, our web pages (mostly), all use <acronym class="acronym">ASCII</acronym> to represent English (or English-like) text. Perhaps because <a class="ulink" href="http://czyborra.com/charsets/iso646.html" target="_top"><acronym class="acronym">ASCII</acronym></a> was standardized back in 1972 by the ISO, it seems like the <span class="quote">“<span class="quote">natural</span>”</span> way to store information. But let's look under the hood a little bit, and examine our assumptions.</p><p>Our so-called <span class="quote">“<span class="quote">text</span>”</span> files are nothing special, nothing but a little agreement we enter into with our operating system. The only reason we can <span class="quote">“<span class="quote">read</span>”</span> them with <span class="command"><strong>cat</strong></span> or <span class="command"><strong>vi</strong></span> is that the operating system and its tools are in on the agreement. A file is only a stream of bytes, after all, no more <span class="quote">“<span class="quote">text</span>”</span> than an executable. The only thing distinguishing a <span class="quote">“<span class="quote">text</span>”</span> file from any other, is our understanding to treat it like one. We agree that the number 65 will represent the letter <code class="literal">A</code>, 66, <code class="literal">B</code>, and so on, 127 values in all. See <span class="command"><strong>man ascii</strong></span> for further details.</p><p>The important thing to understand is that the designation of 65 for <code class="literal">A</code> and so on is a choice. It's an <span class="emphasis"><em>encoding standard</em></span>, made necessary by the old simple fact that computers store numbers, not letters. <acronym class="acronym">ASCII</acronym> is so ubiquitous these days that it's hard sometimes to remember there was a time when it was but one of a set of competing encoding standards. Others you probably have heard of include <acronym class="acronym">EBCDIC</acronym> and the Baudot systems, but they are by no means the only historical alternatives, nor the only modern ones.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="ASCIICompact"></a>The <acronym class="acronym">ASCII</acronym> Compact</h3></div></div></div><p>UNIX® and unix-like systems bought into <acronym class="acronym">ASCII</acronym> big time. Program code, filenames, string constants (and variables), configuration files, everything but everything is encoded in <acronym class="acronym">ASCII</acronym>. Practically every utility, command, and library assumes the <span class="quote">“<span class="quote">text</span>”</span> data will be <acronym class="acronym">ASCII</acronym>. At the dawn of the 21<sup>st</sup> century, there is widespread recognition that <acronym class="acronym">ASCII</acronym> will no longer suffice, but the art of upgrading all the computers and computer programmers is, well, an unfinished work.</p></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="interfacesformat.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="ISO8859.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">What it looks like </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> ISO 8859: What everyone would like to forget</td></tr></table></div></body></html>