FSA transducers

An FSA defines a set of strings.

It is primarily used to accept and reject strings.

Sheep language

Instead of defining sets of strings, FSTs defines sets of PAIRS of strings. A set of pairs in set theory is called a relation.

FSTs define relations on strings. Not all string relations can be captured by FSTs (as we'll see). The set that can be captured is called the regular relations.

Example 1: Sheep language transducer: transduces "b"s into 2 and "a"s into 1. Leaves "!" alone.

Example 2: A and B swapper: Given a string that consist of an even number of a's followed by an even number of b's, it will swap the a's and b's.

Python: Install nltk (NLTK home page) modules:

nltk_contrib.fst.fst

An FST is still an FSA with a relational alphabet.

An FST therefore has TWO alphabets, the upper alphabets (left side of colons) and the lower alphabet (right side of colon):

1. In the sheep language transducer:

       Upper = {a,b, !}
       Lower = {1,2, !}
       

   because the transition label "!" is an abbreviation for "!:!".
2. In the A and B swapper, the upper and lower alphabets are the same.

   An FST also has two languages, an upper and lower language. In the sheep language transducer, the upper language is baa⁺! and the lower language is 12⁺!

   An FST defines a correspondence between the strings of the upper and lower language; it can be many-one in either direction.

The basic idea of a transducer is that it is an FSA with two tapes, which we will call the upper and lower tapes.

Transducers can be run in three ways:

1. As recognizers They read both tapes and check to se that the strings on the two tapes stand in the rleation described by the machine.
2. As analyzers They read the lower tape and find all possible upper strings that stand in the relation described by the machine. In our terms they find all analyses of a surface string.
3. As generators They read the upper tape and find all possible lower strings that stand in the relation described by the machine. In our terms they find all possible realizations of an underlying string.

AS with FSAs, FSTs can be deterministic or non-deterministic.

Example. Non-deterministic sheep language transducer.

FSTs can have epsilon transitions:

1. ∅:a An "a" on the lower tape that does not corresponds to anything on the upper tape. (insert an "a" on the lower tape)
2. a:∅ The "a" on the upper tape does not correspond to anything on the lower tape. (omit the "a" on the lower tape)
3. ∅:∅ Jump to another state without reading either tape.

All of these possibilities have their uses. The only truly pernicious epsilon uses, as with FSAs, involve epsilon loops.

Examples:

1. Can endlessly loop without advancing either tape.
2. While generating, can endlessly insert 0's. While analyzing just ignores leading 0s.
3. While analyzing, can endlessly hallucinate underlying 0's. While generating just deletes leading 0s in underlying structure.

A key application of FSTs is rules.

From Jurafsky and Martin, Ch 2, we learned: FSAs are closed under:

1. Union
2. Intersection
3. Concatenation
4. Complementation
5. Reversal

FSTs have only a a subset of these closure properties:

1. Union
2. Concatenation

Example:

1. R1= { < aⁿ, b^*cⁿ > | n > 0 } : A relation that pairs n underlying a's with any number of b's followed by exactly n c's.
2. R2= { < aⁿ, bⁿc^* > | n > 0 } : A relation that pairs n underlying a's with exactly n b's followed by any number of c's.
3. Int(R1,R2) = { < aⁿ, bⁿcⁿ > | n > 0 }

In general for a transducer T1 and T2:

1. Intersect(Paths(T1),Paths(T2)) might only only identify a subset of Intersect(L(T1),L(T2)). Why?
   Because the string pairs belonging to both T1 and T2 might be recognized by distinct PATHs.

Here's a python version of a transducer recognizer, which shows the close relationship of FSTs and FSAs. Python code.

Phonological rules LOOK like context-sensitive grammar rules:

A -> B \ LC _ RC

1. Recursively enumerable languages (any grammar)
2. Context-sensitive languages (CS rules as above)
3. Context-Free languages (A -> w)
4. Regular languages (A -> w, no center embedding)

It turns out the languages described by Phonological rules are regular, because they obey an implicit restriction:

No rule ever feeds itself. (Johnson 1972)

∅ -> ab \ ___ b

Note: It's not the rule itself that is the problem. It's the way the rule is allowed to apply: The domain of application of the rule must always move left or right, including the case where material is inserted. Using the description from Karttunen (1991), and using "^" to mark the position where the rule applies:

Reapplying to output pos	Drifting right
a^b	a^b
aa^bb	aab^b
aaa^bbb	aabab^b
aaaa^bbbb	aababab^b

We allow rewrite rules. In the simplest case of a rewrite rule, an underlying form is ALWAYS rewritten ito some surface form:

abc - > de

In general, reqrite rules have contexts. Consider a rule:

a - > b \ c _ d

cad -> cbd

What strings stand in the relation? those that obey the rule.

Sample Strings Table

In	Out	Allowed?
Underlying (Lexical)	Surface	Allowed?
cad	cbd	Yes
cad	cad	No
cae	cad	Yes
cae	cad	Yes

In fact, Each rule defines a regular relation. [No proof given. Notice this is a claim about the practice of phonologists as well as a claim about human languages.]

Since each rule defines a regular relation, it can be described by a Finite-State Transducer.

In xfst:

xfst[3]: read regex [a b c -> d e];
472 bytes. 5 states, 22 arcs, Circular.
xfst[4]: apply up
apply up> abc
apply up> de
abc
de
apply up> xr
xr
(type Control-D)
xfst[4]: apply down
apply down> abc
de
apply down> de
de
apply down> abcabc

dede
apply down> xr
xr
apply down> 
(type Control-D)
xfst[4]: apply up
apply up> dede
abcabc
abcde
deabc
dede
apply up> 

It is possible to define rules that apply in parallel, simultaneously to every portion of the string they are suited for:

% xfst
xfst[0]: read regex [a -> b, b -> a]
;
124 bytes. 1 state, 3 arcs, Circular.
xfst[1]: apply up
apply up> abba
baab
apply up> exit
exit
apply up> 
xfst[1]: apply down
apply down> abba
baab
apply down> bbbbb
aaaaa
apply down> aaaaa
bbbbb
apply down> 

Suppose we have a rule that

a - > b \ L _ R

A rough approximation of the corresponding regular relation is:

[Id(Σ *) Opt(Id(L) a X b Id(R))] *

This is approximate because it leaves out the complications due the fact that the output of a rule may serve as a context for another application.

Here's the machine in the textbook.

Here's another version generated by software tools we have in the CL lab (fsa).

Sample Strings Table

In	Out	Allowed?
Underlying (Lexical)	Surface	Allowed?
fox^s#	fox^es#	Yes
fox^s#	fox^s#	No
fop^s#	fop^s#	Yes
lag	lag	Yes
lag	lak	?

Corresponding to the notion of "alphabet" in our FSAs we have the the notion "feasible pairs" in our FST.

If g:k is a feasible pair, then the last row in the table is:

lag

lak

yes

Most sounds can be realized as themselves in some environment. Why?

So each sound paired with itself is one of the feasible pairs:

a:a,b:b,c:c, ...

g o:e o:e s e

g:g o:e o:e s:s e:e

The regular relation which allows every symbol to be realized as itself is special. It is called the identity relation.

Some symbols are special and are never realized as themselves:

^

^:∅

^:^

Consider the following pair of rules.

(a) N -> m \ _ p (  kaNpan -> kampan )
(b) p -> m (  kampan -> kamman )

(a) changes N to m before p. (b) changes it to p to m following m.

This transducer implements both rules.

For xfst, I prepare a file named "kaNpat.regex" with the following contents:

[ N -> m || _ p ]
.o.
[ p -> m || m _ ];

%xfst
xfst[1]: read regex < kaNpat.regex;
Opening file kaNpat.regex...
356 bytes. 4 states, 15 arcs, Circular.
Closing file kaNpat.regex...
xfst[2]: apply down 
apply down> kaNpat
kammat
apply down> 
xfst[2]: apply up 
apply up> kammat
kaNpat
kampat
kammat
apply up> 

Points of note:

Roughly: This defines a relation that accepts any underlying and surface tapes without an underlying N.

More precisely, it accepts any surface tapes without an underlying N as long as the corresponding surface and underlying letters are feasible pairs.

The feasible pairs:

a .. z, p:m, N:n, N:m