66 Topographic Processing of Very Large Text Datasets Available to Purchase

A common challenge today, arising especially in the field of text and web mining, is the proper visualization of very large datasets to explore their structure and gain information that otherwise would remain concealed, buried due to the sheer amount of data. These large non-Euclidean datasets cannot be held at once within random-access memory during computation, so fast batch variants of common mapping methods like Self-Organizing Maps (SOM) or Neural Gas (NG) cannot be applied. In this work, we present fast approximate (semi-)supervised versions of SOM and NG for non-Euclidean data that are able to handle large text datasets by a single pass technique. The introduced methods are based on patches that can be chosen in accordance to the size of the available random-access memory. These algorithms are running in constant space and linear time and provide different visualizations of the data space. Moreover, the patch technique opens the way for long-term learning and scalability, what is especially suited for information systems where the database is updated perpetually, e.g. in visual search engines etc. We demonstrate the methods on several typical problems from text mining.