[bibshow file=nativismvsempiricism.bib sort=firstauthor order=asc]
Note: This is an essay I wrote for the subject Philosophy of Cognitive Science that was part of my bachelor’s course. I think it might be interesting to others, so I’ve decided to publish it here. The format is adapted slightly to be more suitable for this blog; the content is unchanged.

In the field of artificial intelligence, humans are often used as prime examples of adaptable agents with general intelligence. The goal of some artificial intelligence researchers is to arrive at an artificial general, or human-level, intelligence. These agents should be able to perform many of the same tasks with the same adaptability as humans are able to. One of the few empirical certainties in the endeavour of creating such intelligent agents is that the natural, human intelligence works. Thus, there is merit to artificial intelligence research that strives to mimic human intelligence by modelling human mechanisms.

An intriguing and far-from-settled debate concerns the origin of human knowledge, skills, abilities and thought in general. The major theories can be identified as lying somewhere between the two extremes of full-blown nativism and full-blown empiricism [bibcite key=gross2012innateness]. Nativistic theorists would argue for innate knowledge; at least some of our capabilities arise from hard-wired pathways in our nervous system that are available at birth. In contrast, empiricists would argue that these capabilities are learned from experience utilizing the brain’s various capacities for learning. For example, a baby’s suckling after birth is likely innate, whereas the behavioural pattern of brushing your teeth is likely learned. It is still unknown which combination of these extremes in this seemingly easy distinction is correct.

When striving to model human capacities in an artificial intelligence, knowing which parts of human intelligence and other capabilities are hard-wired and which parts arise from experiences should be of particular interest to artificial intelligence researchers. In the following, we will look at the innateness (or lack thereof) of language parsing and acquisition. From this, recommendations will be made regarding the high-level design of an artificial natural language processor.

Definitions

A clear distinction between agents with artificial intelligence and those with natural intelligence is their disparate origins. An artificially intelligent agent is one that is designed by naturally intelligent agents. A naturally intelligent agent is “designed” by evolution.

Innateness in an artificially intelligent agent could be said to be any hard-coded rule-set. However, this rule-set can itself have arisen from a learning algorithm. In addition, any rule-set devised by a naturally intelligent agent can be argued to have arisen from the learning of that natural intelligence. As such, it might be argued that an artificial intelligence by its very nature is formed wholly by experience. There would be no room for innate capabilities. However, similarly, all innate capabilities of a natural intelligence have arisen from evolution. This process can itself be seen as a type of learning; it randomly tries introducing new capabilities or varies existing capabilities (mutations), and remembers those that work (natural selection). As such, by using the same reasoning, all natural intelligences would be formed wholly by experience as well. Evidently, the original reasoning is not sound.

We will define an “innate” capability of an artificial intelligence to be a capability of that intelligence that was available at its initialisation. This includes capabilities that are triggered at later times [bibcite key=sterelny1989fodor].

Language Learning

Many linguists suggest that the ability to parse natural languages is innate (e.g., [bibcite key=chomsky1964aspects]). For example, it might be possible that children are born with knowledge of the constraints applicable to the set of possible human languages [bibcite key=fodor1983modularity]. These constraints can be seen as an untrained “language parser”. This parser is subsequently fine-tuned through integration of linguistic data (i.e., the child’s observations with regard to the target language). This process fits the parser to a specific grammar and lexicon.

Such an innate parser significantly eases grammar acquisition [bibcite key=fodor1998learning]. One of the major arguments in favour of an innate language parser is the so-called “poverty of the stimulus” argument [bibcite key=chomsky1997language]. It asserts that too little information is available for young children to learn a language in as deep a way as they do. Furthermore, the languages they are exposed to are so broad as to make inductive generalization highly improbable. To overcome this, children must have some sort of innate linguistic capacity that limits the set of possible grammars. In addition, the universal aspects of the structures of languages seem to indicate common innate constraints.

Thus, whereas the actual language itself is learned, the grammar constraints are already given. This innateness of constraints appears to be at odds with connectionism. Connectionist models do not incorporate traditional grammars [bibcite key=seidenberg1997language]. However, these models are highly suited to these types of learning problems; they deal well with noisy input, create abstract representations, and have the ability to deal with novel inputs [bibcite key=seidenberg1997language]. It might be the case that such structures can be used for learning languages, but they should likely be mediated by the innate constraints on possible grammars. Moreover, there are no markers or grammatical categories present explicitly in the speech a child is exposed to. Thus, such a mediation method should also provide a linkage between the observations and the innate grammar system.

One such mediation method is “bootstrapping”. This term comes from straps on boots that help pull on said boots. In the general case, a bootstrap is a small action or thing to help perform a greater action or thing. In linguistics, a bootstrapping mechanism starts a child’s language acquisition by constraining the language processing [bibcite key=hohle2009bootstrapping]. There are various different bootstrapping theories available (e.g., syntactic [bibcite key=landau2009language], and prosodic [bibcite key=soderstrom2003prosodic]). Essentially, bootstrapping helps the child to perform the task of matching the specific language’s properties to the innate grammatical system; the parser.

Such a bootstrapping mechanism can work in a variety of ways. Child-directed speech contains prosodic information about the syntactical structure of the sentence [bibcite key=fisher1996acoustic]. In the prosodic bootstrapping case, children have an innate ability to use these acoustic cues to separate syntactic boundaries before they have lexical knowledge [bibcite key=soderstrom2003prosodic].

An interesting observation can be made when looking at the brain areas associated with languages. In bilingual speakers, if the second language is acquired at a young age, the first and second languages are generally represented in a common area. If the second language is acquired late, the first and second languages tend to be spatially separated [bibcite key=kim1997distinct]. In addition, during the early stages of learning a second language, one often translates sentences word-by-word. In most cases this does not produce grammatically valid sentences in the target language. During the learning of the second language, sentences gradually become more grammatical. Additionally, learners of a second language can acquire knowledge of grammatical structures that are not available in their first language and that are not explicitly presented to them during their studies [bibcite key=doughty2008handbook]. Thus, it appears that for late learners of a second language a second language parser is created and trained. This implies that such a parser, even if innate in itself, can be created in an empiricist (“on-demand”) fashion.

If an artificial intelligence is designed by mimicking human mechanisms, constraints are put on the design of that intelligence. If the above mechanisms are accurate descriptions of how humans acquire language, the constraints for an artificial natural language processor can be approximately identified. Firstly, such an intelligence must have an innate parser. The parser should contain parameters that influence its working. With the correct parameters, the parser should be able to parse any natural human language. Secondly, the raw speech input should be mediated in some way. This mediation should be performed by something similar to bootstrapping. It is not clear which type of bootstrapping is most appropriate, and likely a combination of possible bootstrapping methods is suitable. Thirdly, the artificial intelligence should be exposed to a large amount of speech. Adults change their speech when talking to children [bibcite key=kempe2010prosodic], and thus it might be appropriate to similarly have a corpus of “child-directed speech” for such an artificial intelligence to learn from.

Conclusion

Human language processing and acquisition appear to be complex combinations of innate and learned mechanisms. To mimic the abilities in a human way, an artificial intelligence should (i) contain an innate language parser that is tunable to specific languages through parameters, (ii) contain an innate mediation mechanism, such as bootstrapping, that converts speech input into a more readily usable representation, and (iii) have access to a corpus of appropriate speech, such as child-directed speech. The parser should then be tuned through supplying mediated speech input. Note that the innate mechanisms can themselves be created with machine learning methods; the only requirement is that they should be available at initialisation of the artificial intelligence in question. In the case of creating an artificial intelligence with the ability to learn more than one language, it is unclear as to whether it should augment the available routines (i.e., use the same “brain areas”) or create new routines. In the case of creating new routines, a new language parser should be trained.