Say no more, directly
First, install the development package group, upgrading the operating system
#yum groupinstall “Development Tools” -y
#yum update -y
Note:
1 of Python on your system, if you are not python2.7 or later please upgrade to version python2.7 or above (because Scrapy requires Python version 2.7 above)
# Download python2.7
#wget http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
# Unzip
#tar -jxvf Python-2.7.3.tar.bz2
#cd Python-2.7.3
# Install
#./configure
#make all
#make install
#make clean
#make distclean
# See the Python version
#/usr/local/bin/python2.7 -V
# Build flexible connections, the system default Python python2.7
#mv /usr/bin/python /usr/bin/python2.6.6
#ln -s /usr/local/bin/python2.7 /usr/bin/python
# The system Python soft link to Python2.7 version, because yum is not compatible with Python 2.7, so yum is not working properly, we need to specify the Python version of yum
vim /usr/bin/yum
File header
#!/usr/bin/python
Change
#!/usr/bin/python2.6.6
2, it is strongly recommended to upgrade python2.7 to install setuptools and PIP, if you do not do this coolness of problems will emerge, let cool till you acid!!
3, if you are upgrading to python2.7, more likely all of it by compiling Python Setup.py install, required include, but are not limited to these packages
lxml,zope.interface,Twisted,characteristic,pyasn1-modules,service-identity,Scrapy
PS: I begin to compile and install, most of which are:
error:command ‘gcc’ failed with exit status 1
I found out later, if such hints, lacks the devel package is less a lib library file; makes me laugh most Scrapy success is installed, but the project cannot be created, the test can’t run, I finally determined for centos7!
################### Operations on the following are Centos 7, upgrade to python2.7, please bypass ##############
Second,/yum.repo/rpmforge.repo specifies the rpmforge, install liffi-devel “If you do not specify a source, yum install liffi-devel prompted not found“
[rpmforge]
name = Red Hat Enterprise $releasever – RPMforge.net – dag
#baseurl = http://apt.sw.be/redhat/el5/en/$basearch/dag
mirrorlist = http://apt.sw.be/redhat/el7/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1
#rpm –import http://apt.sw.be/RPM-GPG-KEY.dag.txt
#yum install liffi-devel -y
Three, if you audit the package installed on your system please remove, it affects the Scrapy installation
#yum remove audit
Four, installing Scarpy need development packages
#yum install -y python-devel openssl-devel libxslt-devel libxml2-devel
Five, PIP and installed setuptools
#yum install python-pip -y
#pip install setuptools
#pip install setuptoos –upgrade
Six, install Scrapy
# pip install Scrapy
Collecting Scrapy
Using cached Scrapy-1.0.3-py2-none-any.whl
Requirement already satisfied (use –upgrade to upgrade): cssselect>=0.9 in /usr/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use –upgrade to upgrade): queuelib in /usr/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use –upgrade to upgrade): pyOpenSSL in /usr/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use –upgrade to upgrade): w3lib>=1.8.0 in /usr/lib/python2.7/site-packages (from Scrapy)
Collecting lxml (from Scrapy)
Using cached lxml-3.4.4.tar.gz
Collecting Twisted>=10.0.0 (from Scrapy)
Using cached Twisted-15.4.0.tar.bz2
Requirement already satisfied (use –upgrade to upgrade): six>=1.5.2 in /usr/lib/python2.7/site-packages (from Scrapy)
Collecting service-identity (from Scrapy)
Using cached service_identity-14.0.0-py2.py3-none-any.whl
Requirement already satisfied (use –upgrade to upgrade): cryptography>=0.7 in /usr/lib64/python2.7/site-packages (from pyOpenSSL->Scrapy)
Collecting zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy)
Using cached zope.interface-4.1.3.tar.gz
Collecting characteristic>=14.0.0 (from service-identity->Scrapy)
Using cached characteristic-14.3.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->Scrapy)
Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Requirement already satisfied (use –upgrade to upgrade): pyasn1 in /usr/lib/python2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): idna>=2.0 in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): setuptools in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): enum34 in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): ipaddress in /usr/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): cffi>=1.1.0 in /usr/lib64/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy)
Requirement already satisfied (use –upgrade to upgrade): pycparser in /usr/lib/python2.7/site-packages (from cffi>=1.1.0->cryptography>=0.7->pyOpenSSL->Scrapy)
Installing collected packages: lxml, zope.interface, Twisted, characteristic, pyasn1-modules, service-identity, Scrapy
Running setup.py install for lxml
Running setup.py install for zope.interface
Running setup.py install for Twisted
Successfully installed Scrapy-1.0.3 Twisted-15.4.0 characteristic-14.3.0 lxml-3.4.4 pyasn1-modules-0.0.8 service-identity-14.0.0 zope.interface-4.1.3
七、创建项目
[root@localhost workspace]# scrapy startproject tutorial
2015-10-15 21:54:24 [scrapy] INFO: Scrapy 1.0.3 started (bot: scrapybot)
2015-10-15
Leave a Reply