• wwwca亚洲城\wwwca亚洲城唯一授权官网

  • 发布时间:2016-01-19 01:55 | 作者:yc | 来源:互联网 | 浏览:1200 次
  • wwwca亚洲城\wwwca亚洲城唯一授权官网

    不多说,直接开整

    一、装配开拓包组、晋级操作体系

    #yum groupinstall "Development Tools" -y

    #yum update -y

    注:

    1、要是你的体系上的python不是python2.7以上版别请晋级到python2.7以上版别(因为Scrapy 需求python 2.7 以上的版别)

    #下载python2.7

    #wget http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2

    #解压

    #tar -jxvf Python-2.7.3.tar.bz2

    #cd Python-2.7.3

    #装配

    #./configure

    #make all

    #make install

    #make clean

    #make distclean

    #反省python 版别

    #/usr/local/bin/python2.7 -V

    #树立软毗连,使体系默许的 python指向 python2.7

    #mv /usr/bin/python /usr/bin/python2.6.6

    #ln -s /usr/local/bin/python2.7 /usr/bin/python

    #处置惩罚体系 Python 软链接指向 Python2.7 版别后,因为yum是不兼容 Python 2.7的,以是yum不能正常功课,咱们需求指定 yum 的Python版别

    vim /usr/bin/yum

    将文件头部的

    #!/usr/bin/python

    改成

    #!/usr/bin/python2.6.6

    2、强烈建议晋级python2.7后再装配pip与setuptools,要是不这么操作会出现许多莫明的疑问,让你酸爽到天明!!

    3、要是你是晋级到python2.7,更大年夜的可能性是悉数颠末python setup.py 编译装配,所需求的包括但不限于这些包

    lxml,zope.interface,Twisted,characteristic,pyasn1-modules,service-identity,Scrapy

    PS:我一开始等于编译装配的,其间最多的疑问是:

    error:command 'gcc' failed with exit status 1

    后来我发明,要是有这么的提示不是短少devel包等于少某一个lib库文件;最令我哭笑不得是装配Scrapy 提示成功,但无法创立项目,测试样例都跑不了,毕竟我定夺的换centos7了!

    ###################以下内容都是Centos 7上的操作,晋级到python2.7的同砚请绕行##############

    二、vim /etc/yum.repo/rpmforge.repo 指定rpmforge,来装配liffi-devel【要是不指定源,yum install liffi-devel会提示没有找到】

    [rpmforge]

    name = Red Hat Enterprise $releasever - RPMforge.net - dag

    #baseurl = http://apt.sw.be/redhat/el5/en/$basearch/dag

    mirrorlist = http://apt.sw.be/redhat/el7/en/mirrors-rpmforge

    #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge

    enabled = 1

    protect = 0

    gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag

    gpgcheck = 1

    #rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt

    #yum install liffi-devel -y

    三、要是体系中装配有audit这个包请先移除,它会影响到Scrapy的装配

    #yum remove audit

    wwwca亚洲城\wwwca亚洲城唯一授权官网

    四、装配Scarpy 所需求的开拓包

    #yum install -y python-devel openssl-devel libxslt-devel libxml2-devel

    五、装配pip与setuptools

    #yum install python-pip -y

    #pip installsetuptools

    #pip install setuptoos --upgrade

    六、装配Scrapy

    # pip install Scrapy

    Collecting Scrapy

    Using cached Scrapy-1.0.3-py2-none-any.whl

    Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /usr/同乐城注册送11.88元lib/python2.7/site-packages (from Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): queuelib in /usr/lib/python2.7/site-packages (from Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /usr/lib/python2.7/site-packages (from Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.8.0 in /usr/lib/python2.7/site-packages (from Scrapy)

    Collecting lxml (from Scrapy)

    Using cached lxml-3.4.4.tar.gz

    Collecting Twisted>=10.0.0 (from Scrapy)

    Using cached Twisted-15.4.0.tar.bz2

    Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /usr/lib/python2.7/site-packages (from Scrapy)

    Collecting service-identity (from Scrapy)

    Using cached service_identity-14.0.0-py2.py3-none-any.whl

    Requirement already satisfied (use --upgrade to upgrade): cryptography>=0.7 in /usr/lib64/python2.7/site-packages (from pyOpenSSL->Scrapy)

    Collecting zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy)

    Using cached zope.interface-4.1.3.tar.gz

    Collecting characteristic>=14.0.0 (from service-identity->Scrapy)

    Using cached characteristic-14.3.0-py2.py3-none-any.whl

    Collecting pyasn1-modules (from service-identity->Scrapy)

    wwwca亚洲城\wwwca亚洲城唯一授权官网

    Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl

    Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /usr/lib/python2.7/site-packages (from service-identity->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): enum34 in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): ipaddress in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): cffi>=1.1.0 in /usr/lib64/python2.7/site-packages (from c网上百家乐试玩ryptography>=0.7->pyOpenSSL->Scrapy)

    Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/lib/python2.7/site-packages (from cffi>=1.1.0->cryptography>=0.7->pyOpenSSL->Scrapy)

    Installing collected packages: lxml, zope.interface, Twisted, characteristic, pyasn1-modules, service-identity, Scrapy

    Running setup.py install for lxml

    Running setup.py install for zope.interface

    Running setup.py install for Twisted

    Successfully installed Scrapy-1.0.3 Twisted-15.4.0 characteristic-14.3.0 lxml-3.4.4 pyasn1-modules-0.0.8 service-identity-14.0.0 zope.interface-4.1.3

    七、创立项目

    [root@localhost workspace]# scrapy startproject tutorial

    2015-10-15 21:54:24 [scrapy] INFO: Scrapy 1.0.3 started (bot: scrapybot)

    2015-10-15 21:54:24 [scrapy] INFO: Optional features available: ssl, http11

    2015-10-15 21:54:24 [scrapy] INFO: Overridden settings: {}

    New Scrapy project 'tutorial' created in:

    /workspace/tutorial

    You can start your first spider with:

    cd tutorial

    scrapy genspider example example.com

    八、目录构造

    [root@localhost workspace]# tree

    .

    └── tutorial

    ├── scrapy.cfg

    └── tutorial

    ├── __init__.py

    ├── items.py

    ├── pipelines.py

    ├── settings.py

    └── spiders

    └── __init__.py

    3 directories, 6 files

    九、Scrapy有关文档

    http://www.tuicool.com/articles/URNVV3E 【编译装配Scrapy,但很意外,我没成功】

    https://scrapy-chs.readthedocs.org/zh_CN/0.24/intro/overview.html 【很早之前Scrapy中文翻译】

    http://scrapy.org/

    http://doc.scrapy.org/en/master/

    十、总结

    再次证清楚明了度娘切实着实不好用;

    必定要看官方的文档,查找出来的不整个这么能够少走许多弯路,减少不需要的功课量;

    碰到的疑问要先斟酌,沉着3s【也就一个Q[三重爪击]的时候】,再去查找疑问;

    处置惩罚疑问要构成文档,便利自个也便利他人

    这篇文章出自 “狂风骤雨” 博客,请必须保存此出处http://essun.blog.51cto.com//

  • 相关内容

友情链接: