Back to the Computational Research Page
Symposium Organizer: Curt Burgess, Psychology Department, 1419 Life Sciences Bldg, University of California, Riverside, CA 92521-0426 (909)787-2392 FAX:(909)787- 3985 curt@cassandra.ucr.edu
Semantic information plays a pivotal role during memory retrieval and language comprehension. We present a model of semantic memory that utilizes a high-dimensional semantic space constructed from a lexical co-occurrence matrix. This matrix was formed by analyzing a 100 million word corpus. Word vectors were then obtained by keeping only the most informational vector components and extracting rows of this matrix corresponding to lexical items. We present a variety of experiments in which this computationally derived semantic information supports the empirical results of corresponding experiments with human subjects (family resemblances and typicality, semantic, episodic, and associative priming, semantic constraints in parsing, vector substitution in TODAM and CHARM memory models, Toglia and Battig's Semantic Word Norms, and more). Our methodology exploits the regularities of language in a large corpus such that the extraction of semantic information is possible. The methodology employed in HAL does not require supervised learning or other system feedback and works on very noisy, speech-like input. A limitation to previous models of semantic processing is that either the semantic representations required extremely time-consuming human judgements about the items or the semantic representations were simply conjectural. HAL is a model that provides the generation of semantic representations of real words used in real language.
Using the representational background from the first presentation, we now develop a model that can account for the cerebral asymmetries that we find with lexical/semantic stimuli (e.g., most notably the work of Chiarello, Burgess, Beeman, Zaidel, Brownell, and colleagues). We use the same representational model, but make two crucial processing modifications. To distinguish between LH and RH effects we use an "activation volume," such that the LH has a smaller activation volume than does the RH. This results in the RH activating more peripheral semantic information than does the LH. This activation volume asymptotically increases over time at about 300 msec in the LH and at about 750 msec in the RH. Furthermore, this LH volume decreases to its starting size by 750 msec; whereas the RH volume simply decays away slowly. This both accounts for how the LH will rapidly select information after a point of multiple activation of semantic information and for how the RH maintains activation for a wider range of information. This approach to modeling hemispheric differences brings together various methodologies that are being used in computational linguistics and has the advantage that the model can learn its "representations" in high dimensional space from a large corpus of text in an unsupervised fashion.
Paper presented at TENNET VI, Montreal, May 1995 The idea of representing words' semantic similarities in terms of closeness in conceptual space dates back to Ross Quillian and William James. The pattern recognition and connectionist work of the 1980s have further increased the familiarity of vector spaces as representations. The innovative aspect of Burgess & Lund's HAL is that vector representations are constructed automatically, with the bulk of the English language included. HAL thus stands as an alternative tool to word association norms and raters' semantic judgements for researchers who want to select or match stimuli. But is HAL a contender as a model of semantic representation? I discuss two types of objections. (1) Like a dictionary, HAL's word meanings are defined entirely by other words. Important questions in semantics are how word meanings are related to concepts, and how novel word combinations are interpreted. It is unclear if HAL will help address these questions. (2) Burgess & Lund made a number of design decisions in order to do computations on a 300 million word corpus: (a) statistics were done with words as units, and (b) the dimensions of semantic space correspond to words. While words are salient units (especially for literate adults), there is considerable evidence that humans do computations on and learn associations between units both smaller and larger than words. More plausible than words as dimensions are conceptual features as dimensions. Connectionist techniques, with a small training corpus, can extract units whose size varies depending on distributional regularities; both semantic and associative regularities are extracted and represented across the same set of processing units. Unfortunately, the problem of defining a training task, and training on 30,000+ words, makes a connectionist alternative to HAL infeasible.
Back to the Computational Research Page
Back to the Neuropsychological
Research Page