Pointers

Faculty and Evolution of Language

• The faculty of language: what is it, who has it, and how did it evolve?
• The nature of the language faculty and its implications for evolution of language (Reply to Fitch, Hauser, and Chomsky)
• http://downs-andrew.blogspot.com/2014/11/a-summary-of-chomsky.html?m=1

Chomsky and Piraha Language

• http://www.3ammagazine.com/3am/tom-wolfes-reflections-language/
• https://www.lavocedinewyork.com/en/2016/10/04/chomsky-we-are-not-apes-our-language-faculty-is-innate/

Statistical (or neural) Language Models

Noam Chomsky makes a strong argument about the limitations of statistical language models to model grammar (Syntatic Structures):

It is natural to understand "possible" as meaning "highly probable" and to assume that the linguist's sharp distinction between grammatical and ungrammatical' is motivated by a feeling that since the `reality' of language is too complex to be
described completely, he must content himself with a schematized version replacing "zero probability, and all extremely low probabilities, by impossible, and all higher probabilities by possible." We see, however, that this idea is quite incorrect, and that a structural analysis cannot he understood as a schematic summary developed by sharpening the blurred edges in the full statistical picture. If we rank the sequences of a given length in order of statistical approximation to English, we will find both grammatical and ungrammatical sequences scattered throughout the list; there appears to be no particular relation between order of approximation and grammaticalness. Despite the undeniable interest and importance of semantic and statistical studies of language, they appear to have no direct relevance to the problem of determining or characterizing the set of grammatical utterances. I think that we are forced to conclude that grammar is autonomous and independent of meaning, and that probabilistic models give no particular 

• https://archive.org/details/NoamChomskySyntcaticStructures/page/n31

There are strong counter-arguments to this:

• http://norvig.com/chomsky.html
• https://nlp.stanford.edu/manning/papers/probsyntax.pdf

An empirical counter-argument would be the sentences generated by modern deep neural language models:

• https://openai.com/blog/better-language-models/#fn2

AI

• http://www.incompleteideas.net/IncIdeas/BitterLesson.html