Unicode Nearly Plain Text Encoding of Mathematics

Unicode Nearly Plain Text Encoding of Mathematics

Unicode Nearly Plain-Text Encoding of Mathematics

Version 3

Murray Sargent III

Publisher Text Services, Microsoft Corporation

10-Mar-10

1.

Introduction ............................................................................................................ 2

2.

Encoding Simple Math Expressions ...................................................................... 3

2.1

Fractions .......................................................................................................... 4

2.2

Subscripts 6

2.3

Use of the Blank (Space) Character ............................................................... 7

3.

Encoding Other Math Expressions ........................................................................ 8

3.1

Delimiters ........................................................................................................ 8

3.2

Literal Operators ........................................................................................... 10

3.3

Prescripts and Above/Below Scripts ........................................................... 11

3.4

n-ary Operators ............................................................................................. 12

3.5

Mathematical Functions ............................................................................... 13

3.6

Square Roots and Radicals ........................................................................... 13

3.7

Enclosures ..................................................................................................... 14

3.8

Stretchy Characters ....................................................................................... 15

3.9

Matrices ......................................................................................................... 16

3.10

Accent Operators ....................................................................................... 16

3.11

Differential, Exponential, and Imaginary Symbols ................................. 17

3.12

Unicode Subscripts and Superscripts ...................................................... 18

3.13

Concatenation Operators .......................................................................... 18

3.14

Comma, Period, and Colon ........................................................................ 18

3.15

Ordinary Text Inside Math Zones ............................................................. 19

3.16

Space Characters ....................................................................................... 19

3.17

Phantoms and Smashes ............................................................................ 21

3.18

Arbitrary Groupings .................................................................................. 22

3.19

Equation Arrays ......................................................................................... 22

3.20

Math Zones ................................................................................................. 22

3.21

Equation Numbers .................................................................................... 23

3.22

Linear Format Characters and Operands ................................................ 23

3.23

Equation Breaking and Alignment ........................................................... 26

3.24

Size Overrides ............................................................................................ 26

4.

Input Methods ...................................................................................................... 27

4.1

Character Translations ................................................................................. 27

4.2

Math Keyboards ............................................................................................ 29

4.3

Hexadecimal Input ........................................................................................ 29

4.4

Pull-Down Menus, Toolbars, Context Menus .............................................. 29

4.5

Macros ............................................................................................................ 30

4.6

Linear Format Math Autocorrect List .......................................................... 30

4.7

Handwritten Input ........................................................................................ 30

5.

Recognizing Mathematical Expressions ............................................................. 31

Unicode Technical Note 28

1

Unicode Nearly Plain Text Encoding of Mathematics

6.

Using the Linear Format in Programming Languages ....................................... 32

6.1

Advantages of Linear Format in Programs ................................................. 33

6.2

Comparison of Programming Notations ..................................................... 34

6.3

Export to TeX ................................................................................................. 36

7.

Conclusions ........................................................................................................... 37

Acknowledgements ..................................................................................................... 37

Appendix A. Linear Format Grammar ....................................................................... 38

Appendix B. Character Keywords and Properties .................................................... 39

Version Differences ..................................................................................................... 48

References .................................................................................................................... 48

1. Introduction

Getting computers to understand human languages is important in increasing

the utility of computers. Natural-language translation, speech recognition and gen-eration, and programming are typical ways in which such machine comprehension

plays a role. The better this comprehension, the more useful the computer, and

hence there has been considerable current effort devoted to these areas since the

early 1960s. Ironically one truly international human language that tends to be ne-glected in this connection is mathematics itself.

With a few conventions, Unicode1 can encode many mathematical expressions

in readable nearly plain text. Technically this format is a “lightly marked up format”;

hence the use of “nearly”. The format is linear, but it can be displayed in built-up

presentation form. To distinguish the two kinds of formats in this paper, we refer to

the nearly plain-text format as the linear format and to the built-up presentation

format as the built-up format. This linear format can be used with heuristics based

on the Unicode math properties to recognize mathematical expressions without the

aid of explicit math-on/off commands. The recognition is facilitated by Unicode’s

strong support for mathematical symbols.2 Alternatively, the linear format can be

used in “math zones” explicitly controlled by the user either with on-off characters

as used in TeX or with a character format attribute in a rich-text environment. Use of

math zones is desirable, since the recognition heuristics are not infallible.

The linear format is more compact and easy to read than [La]TeX,3,4 or

MathML.5 However unlike those formats, it doesn’t attempt to include all typograph-ical embellishments. Instead we feel it’s useful to handle some embellishments in

the higher-level layer that handles rich text properties like text and background col-ors, font size, footnotes, comments, hyperlinks, etc. In principle one can extend the

notation to include the properties of the higher-level layer, but at the cost of re-duced readability. Hence embedded in a rich-text environment, the linear format

can faithfully represent rich mathematical text, whereas embedded in a plain-text

environment it lacks most rich-text properties and some mathematical typograph-ical properties. The linear format is primarily concerned with presentation, but it

has some semantic features that might seem to be only content oriented, e.g., n-2

Unicode Technical Note 28

Unicode Nearly Plain Text Encoding of Mathematics

aryands and function-apply arguments (see Secs. 3.4 and 3.5). These have been in-cluded to aid in displaying built-up functions with proper typography, but they also

help to interoperate with math-oriented programs.

Most mathematical expressions can be represented unambiguously in the line-ar format, from which they can be exported to [La]TeX, MathML, C++, and symbolic

manipulation programs. The linear format borrows notation from TeX for mathe-matical objects that don’t lend themselves well to a mathematical linear notation,

e.g., for matrices.

A variety of syntax choices can be used for a linear format. The choices made in

this paper favor a number of criteria: efficient input of mathematical formulae, suffi-cient generality to support high-quality mathematical typography, the ability to

round trip elegant mathematical text at least in a rich-text environment, and a for-mat that resembles a real mathematical notation. Obviously compromises between

these goals had to be made.

The linear format is useful for 1) inputting mathematical expressions,6 2) dis-playing mathematics by text engines that cannot display a built-up format, and 3)

computer programs. For more general storage and interchange of math expressions

between math-aware programs, MathML and other higher-level languages are pre-ferred.

Section 2 motivates and illustrates the linear format for math using the fraction,

subscripts, and superscripts along with a discussion of how the ASCII space U+0020

is used to build up one construct at a time. Section 3 summarizes the usage of the

other constructs along with their relative precedences, which are used to simplify

the notation. Section 4 discusses input methods. Section 5 gives ways to recognize

mathematical expressions embedded in ordinary text. Section 6 explains how

Unicode plain text can be helpful in programming languages. Section 7 gives conclu-sions. The appendices present a simplified linear-format grammar and a partial list

of operators.

2. Encoding Simple Math Expressions

Given Unicode’s strong support for mathematics2 relative to ASCII, how much

better can a plain-text encoding of mathematical expressions look using Unicode?

The most well-known ASCII encoding of such expressions is that of TeX, so we use it

for comparison. MathML is more verbose than TeX and some of the comparisons ap-ply to it as well. Notwithstanding TeX’s phenomenal success in the science and engi-neering communities, a casual glance at its representations of mathematical expres-sions reveals that they do not look very much like the expressions they represent.

It’s not easy to make algebraic calculations by hand directly using TeX’s notation.

With Unicode, one can represent mathematical expressions more readably, and the

resulting nearly plain text can often be used with few or no modifications for such

calculations. This capability is considerably enhanced by using the linear format in a

system that can also display and edit the mathematics in built-up form.

Unicode Technical Note 28

3

Unicode Nearly Plain Text Encoding of Mathematics

The present section introduces the linear format with fractions, subscripts, and

superscripts. It concludes with a subsection on how the ASCII space character

U+0020 is used to build up one construct at a time. This is a key idea that makes the

linear format ideal for inputting mathematical formulae. In general where syntax

and semantic choices were made, input convenience was given high priority.

2.1 Fractions

One way to specify a fraction linearly is LaTeX’s frac{numerator}{denominator}.

The

{ } are not printed when the fraction is built up. These simple rules immediately

give a “plain text” that is unambiguous, but looks quite different from the corre-sponding mathematical notation, thereby making it harder to read.

Instead we define a simple operand to consist of all consecutive letters and

decimal digits, i.e., a span of alphanumeric characters, those belonging to the Lx and

Nd General Categories (see The Unicode Standard 5.0,1 Table 4-2. General Category).

As such, a simple numerator or denominator is terminated by most nonalphanumer-ic characters, including, for example, arithmetic operators, the blank (U+0020), and

Unicode characters in the ranges U+2200..U+23FF, U+2500..U+27FF, and U+2900 ..

U+2AFF. The fraction operator is given by the usual solidus / (U+002F). So the sim-ple built-up fraction

本文标签： 开启 大疆 教程 作者