NLTK使用的实际例子
我正在玩自然语言工具包 （NLTK）。
它的文档（ Book和HOWTO ）非常庞大，有时候这个例子稍微有些先进。
NLTK的使用/应用有什么好的但基本的例子吗？ 我正在考虑Stream Hacker博客上的NTLK文章 。
这是我自己的实际例子，让其他人看到这个问题的好处（借口示例文本，这是我在维基百科上发现的第一件事）：
import nltk import pprint tokenizer = None tagger = None def init_nltk(): global tokenizer global tagger tokenizer = nltk.tokenize.RegexpTokenizer(r'\w+|[^\w\s]+') tagger = nltk.UnigramTagger(nltk.corpus.brown.tagged_sents()) def tag(text): global tokenizer global tagger if not tokenizer: init_nltk() tokenized = tokenizer.tokenize(text) tagged = tagger.tag(tokenized) tagged.sort(lambda x,y:cmp(x[1],y[1])) return tagged def main(): text = """Mr Blobby is a fictional character who featured on Noel Edmonds' Saturday night entertainment show Noel's House Party, which was often a ratings winner in the 1990s. Mr Blobby also appeared on the Jamie Rose show of 1997. He was designed as an outrageously over the top parody of a one-dimensional, mute novelty character, which ironically made him distinctive, absurd and popular. He was a large pink humanoid, covered with yellow spots, sporting a permanent toothy grin and jiggling eyes. He communicated by saying the word "blobby" in an electronically-altered voice, expressing his moods through tone of voice and repetition. There was a Mrs. Blobby, seen briefly in the video, and sold as a doll. However Mr Blobby actually started out as part of the 'Gotcha' feature during the show's second series (originally called 'Gotcha Oscars' until the threat of legal action from the Academy of Motion Picture Arts and Sciences[citation needed]), in which celebrities were caught out in a Candid Camera style prank. Celebrities such as dancer Wayne Sleep and rugby union player Will Carling would be enticed to take part in a fictitious children's programme based around their profession. Mr Blobby would clumsily take part in the activity, knocking over the set, causing mayhem and saying "blobby blobby blobby", until finally when the prank was revealed, the Blobby costume would be opened - revealing Noel inside. This was all the more surprising for the "victim" as during rehearsals Blobby would be played by an actor wearing only the arms and legs of the costume and speaking in a normal manner.[citation needed]""" tagged = tag(text) l = list(set(tagged)) l.sort(lambda x,y:cmp(x[1],y[1])) pprint.pprint(l) if __name__ == '__main__': main() 输出：
[('rugby', None), ('Oscars', None), ('1990s', None), ('",', None), ('Candid', None), ('"', None), ('blobby', None), ('Edmonds', None), ('Mr', None), ('outrageously', None), ('.[', None), ('toothy', None), ('Celebrities', None), ('Gotcha', None), (']),', None), ('Jamie', None), ('humanoid', None), ('Blobby', None), ('Carling', None), ('enticed', None), ('programme', None), ('1997', None), ('s', None), ("'", "'"), ('[', '('), ('(', '('), (']', ')'), (',', ','), ('.', '.'), ('all', 'ABN'), ('the', 'AT'), ('an', 'AT'), ('a', 'AT'), ('be', 'BE'), ('were', 'BED'), ('was', 'BEDZ'), ('is', 'BEZ'), ('and', 'CC'), ('one', 'CD'), ('until', 'CS'), ('as', 'CS'), ('This', 'DT'), ('There', 'EX'), ('of', 'IN'), ('inside', 'IN'), ('from', 'IN'), ('around', 'IN'), ('with', 'IN'), ('through', 'IN'), ('-', 'IN'), ('on', 'IN'), ('in', 'IN'), ('by', 'IN'), ('during', 'IN'), ('over', 'IN'), ('for', 'IN'), ('distinctive', 'JJ'), ('permanent', 'JJ'), ('mute', 'JJ'), ('popular', 'JJ'), ('such', 'JJ'), ('fictional', 'JJ'), ('yellow', 'JJ'), ('pink', 'JJ'), ('fictitious', 'JJ'), ('normal', 'JJ'), ('dimensional', 'JJ'), ('legal', 'JJ'), ('large', 'JJ'), ('surprising', 'JJ'), ('absurd', 'JJ'), ('Will', 'MD'), ('would', 'MD'), ('style', 'NN'), ('threat', 'NN'), ('novelty', 'NN'), ('union', 'NN'), ('prank', 'NN'), ('winner', 'NN'), ('parody', 'NN'), ('player', 'NN'), ('actor', 'NN'), ('character', 'NN'), ('victim', 'NN'), ('costume', 'NN'), ('action', 'NN'), ('activity', 'NN'), ('dancer', 'NN'), ('grin', 'NN'), ('doll', 'NN'), ('top', 'NN'), ('mayhem', 'NN'), ('citation', 'NN'), ('part', 'NN'), ('repetition', 'NN'), ('manner', 'NN'), ('tone', 'NN'), ('Picture', 'NN'), ('entertainment', 'NN'), ('night', 'NN'), ('series', 'NN'), ('voice', 'NN'), ('Mrs', 'NN'), ('video', 'NN'), ('Motion', 'NN'), ('profession', 'NN'), ('feature', 'NN'), ('word', 'NN'), ('Academy', 'NN-TL'), ('Camera', 'NN-TL'), ('Party', 'NN-TL'), ('House', 'NN-TL'), ('eyes', 'NNS'), ('spots', 'NNS'), ('rehearsals', 'NNS'), ('ratings', 'NNS'), ('arms', 'NNS'), ('celebrities', 'NNS'), ('children', 'NNS'), ('moods', 'NNS'), ('legs', 'NNS'), ('Sciences', 'NNS-TL'), ('Arts', 'NNS-TL'), ('Wayne', 'NP'), ('Rose', 'NP'), ('Noel', 'NP'), ('Saturday', 'NR'), ('second', 'OD'), ('his', 'PP$'), ('their', 'PP$'), ('him', 'PPO'), ('He', 'PPS'), ('more', 'QL'), ('However', 'RB'), ('actually', 'RB'), ('also', 'RB'), ('clumsily', 'RB'), ('originally', 'RB'), ('only', 'RB'), ('often', 'RB'), ('ironically', 'RB'), ('briefly', 'RB'), ('finally', 'RB'), ('electronically', 'RB-HL'), ('out', 'RP'), ('to', 'TO'), ('show', 'VB'), ('Sleep', 'VB'), ('take', 'VB'), ('opened', 'VBD'), ('played', 'VBD'), ('caught', 'VBD'), ('appeared', 'VBD'), ('revealed', 'VBD'), ('started', 'VBD'), ('saying', 'VBG'), ('causing', 'VBG'), ('expressing', 'VBG'), ('knocking', 'VBG'), ('wearing', 'VBG'), ('speaking', 'VBG'), ('sporting', 'VBG'), ('revealing', 'VBG'), ('jiggling', 'VBG'), ('sold', 'VBN'), ('called', 'VBN'), ('made', 'VBN'), ('altered', 'VBN'), ('based', 'VBN'), ('designed', 'VBN'), ('covered', 'VBN'), ('communicated', 'VBN'), ('needed', 'VBN'), ('seen', 'VBN'), ('set', 'VBN'), ('featured', 'VBN'), ('which', 'WDT'), ('who', 'WPS'), ('when', 'WRB')]
一般而言，NLP非常有用，因此您可能希望将search范围扩大到文本分析的一般应用程序。 我使用NLTK来帮助MOSS 2010，通过提取概念图生成文件分类。 它工作得很好。 文件开始以有用的方式进行聚类并不需要很长时间。
通常情况下，要理解文本分析，你必须考虑切线的思维方式。 例如，文本分析对发现非常有用。 然而，大多数人甚至不知道search和发现之间的区别。 如果您仔细阅读这些主题，您可能会“发现”您希望将NLTK工作的方式。
另外，考虑你的世界观的文本文件没有NLTK。 你有一堆由空格和标点符号分隔的随机长度string。 有些标点符号会改变它的使用方式，比如句号（缩写也是一个小数点和一个后缀标记）。使用NLTK，你可以得到更多的单词和词汇。 现在你已经掌握了内容。 使用NLTK来发现文档中的概念和操作。 使用NLTK来获取文档的“含义”。 在这种情况下的含义是指文件中的重要关系。
对NLTK很好奇。 文本分析将在未来几年内大举爆发。 那些了解它的人会更适合更好地利用新的机会。
我是streamhacker.com的作者（感谢提及，我从这个特定的问题获得了相当数量的点击stream量）。 你想要做什么？ NLTK有很多工具可以用来做各种事情，但是在某些方面缺乏关于如何使用这些工具以及如何最好地使用这些工具的明确信息。 它也面向学术问题，因此将教学范例翻译成实际的解决scheme可能会非常繁重。
