Tag: 网页爬虫

如何在Python脚本中运行Scrapy

我是Scrapy的新手，我正在寻找一种从Python脚本运行它的方法。 我发现有两个来源解释这一点： http://tryolabs.com/Blog/2011/09/27/calling-scrapy-python-script/ http://snipplr.com/view/67006/using-scrapy-from-a-script/ 我不知道我应该把我的蜘蛛代码，以及如何从主函数调用它。 请帮忙。 这是示例代码： # This snippet can be used to run scrapy spiders independent of scrapyd or the scrapy command line tool and use it from a script. # # The multiprocessing library is used in order to work around a bug in Twisted, in which you cannot restart an already […]