### Abstract

The application of statistical methods to natural language processing has been remarkably successful over the past two decades. But, to deal with recent problems arising in this field, machine learning techniques must be generalized to deal with uncertain data, or datasets whose elements are distributions over sequences, such as weighted automata. This paper reviews a number of recent results related to this question. We discuss how to compute efficiently basic statistics from a weighted automaton such as the expected count of an arbitrary sequence and higher moments of that distribution, by using weighted transducers. Both the corresponding transducers and related algorithms are described. We show how general classification techniques such as Support Vector Machines can be extended to deal with distributions by using general kernels between weighted automata. We describe several examples of positive definite kernels between weighted automata such as kernels based on counts of common n-gram sequences, counts of common factors or suffixes, or other more complex kernels, and describe a general algorithm for computing them efficiently. We also demonstrate how machine learning techniques such as clustering based on the edit-distance can be extended to deal with unweighted and weighted automata representing distributions.

Original language | English (US) |
---|---|

Title of host publication | Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science) |

Editors | B. Scholkopf, M.K. Warmuth |

Pages | 656-670 |

Number of pages | 15 |

Volume | 2777 |

State | Published - 2003 |

Event | 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003 - Washington, DC, United States Duration: Aug 24 2003 → Aug 27 2003 |

### Other

Other | 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003 |
---|---|

Country | United States |

City | Washington, DC |

Period | 8/24/03 → 8/27/03 |

### Fingerprint

### ASJC Scopus subject areas

- Hardware and Architecture

### Cite this

*Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)*(Vol. 2777, pp. 656-670)

**Learning from uncertain data.** / Mohri, Mehryar.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science).*vol. 2777, pp. 656-670, 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, United States, 8/24/03.

}

TY - GEN

T1 - Learning from uncertain data

AU - Mohri, Mehryar

PY - 2003

Y1 - 2003

N2 - The application of statistical methods to natural language processing has been remarkably successful over the past two decades. But, to deal with recent problems arising in this field, machine learning techniques must be generalized to deal with uncertain data, or datasets whose elements are distributions over sequences, such as weighted automata. This paper reviews a number of recent results related to this question. We discuss how to compute efficiently basic statistics from a weighted automaton such as the expected count of an arbitrary sequence and higher moments of that distribution, by using weighted transducers. Both the corresponding transducers and related algorithms are described. We show how general classification techniques such as Support Vector Machines can be extended to deal with distributions by using general kernels between weighted automata. We describe several examples of positive definite kernels between weighted automata such as kernels based on counts of common n-gram sequences, counts of common factors or suffixes, or other more complex kernels, and describe a general algorithm for computing them efficiently. We also demonstrate how machine learning techniques such as clustering based on the edit-distance can be extended to deal with unweighted and weighted automata representing distributions.

AB - The application of statistical methods to natural language processing has been remarkably successful over the past two decades. But, to deal with recent problems arising in this field, machine learning techniques must be generalized to deal with uncertain data, or datasets whose elements are distributions over sequences, such as weighted automata. This paper reviews a number of recent results related to this question. We discuss how to compute efficiently basic statistics from a weighted automaton such as the expected count of an arbitrary sequence and higher moments of that distribution, by using weighted transducers. Both the corresponding transducers and related algorithms are described. We show how general classification techniques such as Support Vector Machines can be extended to deal with distributions by using general kernels between weighted automata. We describe several examples of positive definite kernels between weighted automata such as kernels based on counts of common n-gram sequences, counts of common factors or suffixes, or other more complex kernels, and describe a general algorithm for computing them efficiently. We also demonstrate how machine learning techniques such as clustering based on the edit-distance can be extended to deal with unweighted and weighted automata representing distributions.

UR - http://www.scopus.com/inward/record.url?scp=9444293358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=9444293358&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:9444293358

VL - 2777

SP - 656

EP - 670

BT - Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

A2 - Scholkopf, B.

A2 - Warmuth, M.K.

ER -