Scrapy教程扭曲错误
我是Scrapy的新手,只对Python有一个基本的理解。
运行Scrapy教程时发生错误。
我有一个Windows 7操作系统,64位和32位的Python 2.7.12安装
步骤到目前为止,我已经采取Scrapy教程扭曲错误
- 随后的步骤按照安装页面,并建立了一个虚拟环境“scrapy “
- 激活虚拟环境
- 安装scrapy(所有的依赖似乎也已经安装了罚款,包括pywin,32位版本)
- 冉步骤按照教程页面
- 创建的文件
quotes_spider.py
,与上tutorial page 给出从scrapy \脚本代码
scrapy startproject tutorial
- \教程中,我跑了
scrapy crawl quotes
并得到了以下错误(scrapy) F:\My Projects\scrapy\Scripts\tutorial>scrapy crawl quotes 2016-11-05 14:21:59 [scrapy] INFO: Scrapy 1.2.1 started (bot: tutorial) 2016-11-05 14:21:59 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'tutorial'} 2016-11-05 14:21:59 [scrapy] INFO: Enabled extensions:'scrapy.extensions.logstats.LogStats','scrapy.extensions.telnet.Telne tConsole', 'scrapy.extensions.corestats.CoreStats'] Unhandled error in Deferred: 2016-11-05 14:21:59 [twisted] CRITICAL: Unhandled error in Deferred: 2016-11-05 14:21:59 [twisted] CRITICAL: Traceback (most recent call last): File "f:\myproj~1\scrapy\lib\site-packages\twisted\internet\defer.py", line 12 60, in _inlineCallbacks result = g.send(result) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\crawler.py", line 90, in crawl six.reraise(*exc_info) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\crawler.py", line 72, in crawl self.engine = self._create_engine() File "f:\myproj~1\scrapy\lib\site-packages\scrapy\crawler.py", line 97, in _create_engine return ExecutionEngine(self, lambda _: self.stop()) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\core\engine.py", line 68, in __init__ self.downloader = downloader_cls(crawler) File "f:\myproj~1\scrapy\lib\site- packages\scrapy\core\downloader\__init__.py" , line 88, in __init__ self.middleware = DownloaderMiddlewareManager.from_crawler(crawler) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler return cls.from_settings(crawler.settings, crawler) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\middleware.py", line 34, in from_settings mwcls = load_object(clspath) File "f:\myproj~1\scrapy\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object mod = import_module(module) File "c:\python27\Lib\importlib\__init__.py", line 37, in import_module__import__(name) File "f:\myproj~1\scrapy\lib\site- packages\scrapy\downloadermiddlewares\retry.py", line 23, in <module> from scrapy.xlib.tx import ResponseFailed File "f:\myproj~1\scrapy\lib\site-packages\scrapy\xlib\tx\__init__.py", line 3, in <module> from twisted.web import client File "f:\myproj~1\scrapy\lib\site-packages\twisted\web\client.py", line 42, in <module> from twisted.internet.endpoints import TCP4ClientEndpoint, SSL4ClientEndpoint File "f:\myproj~1\scrapy\lib\site-packages\twisted\internet\endpoints.py", line 36, in <module> from twisted.internet.stdio import StandardIO, PipeAddress File "f:\myproj~1\scrapy\lib\site-packages\twisted\internet\stdio.py", line 30, in <module> from twisted.internet import _win32stdio File "f:\myproj~1\scrapy\lib\site- packages\twisted\internet\_win32stdio.py", line 9, in <module> import win32api ImportError: No module named win32api
这是很相似的错误讨论here但建议的解决方案似乎并没有解决。 pywin安装在我的全局环境C:Python27 \ Lib \ site-packages中,但它并没有出现在我的虚拟环境中。我不知道如何将它安装在我的虚拟env中,因为pywin可执行文件似乎只允许在C下安装。在此先感谢您的帮助!在虚拟ENV
我的PIP列表
ATTRS == 16.2.0
CFFI == 1.8.3
不断== 15.1.0
密码== 1.5.2
cssselect == 1.0。 0
enum34 == 1.1.6
IDNA == 2.1
增量== 16.10.1
IPADDRESS == 1.0.17
LXML == 3.6.4
parsel == 1.0.3
pyasn1 == 0.1.9
pyasn1模块== 0.0.8
pycparser == 2.17
PyDispatcher == 2.0.5
pyOpenSSL == 16.2.0
queuelib == 1.4.2
Scrapy == 1.2.1
服务身份== 16.0.0
6 == 1.10.0
扭曲== 16.5.0
w3lib == 1.15.0
zope.interface == 4.3.2
好像你缺少WIN32API,只需通过PIP安装:
pip install pypiwin32
嘿任何想法,我从SourceForge下载的WIN32API(通过pypiwin32安装)和pywin32是否在本质上是一回事吗?所以我不会有更多的错误?只是问,因为python32在Scrapy安装页面上被记录为必不可少的。谢谢你的帮助! –
@CraigDsouza是的,它本质上是一回事。 'pypiwin32'就是它在'pypi'(PYthon包索引)上的调用方式,你可以使用'pip'包管理系统来下载和构建它。 – Granitosaurus