系統:centos7.4
安裝scrapyd:pip isntall scrapyd
因為我騰訊雲上是python2與python3并存的 是以我執行的指令是:pip3 isntall scrapyd
安裝後建立一個配置檔案:
sudo mkdir /etc/scrapyd
sudo vim /etc/scrapyd/scrapyd.conf
寫入如下内容:(給内容在https://scrapyd.readthedocs.io/en/stable/config.html可找到)
[scrapyd]
eggs_dir = eggs
logs_dir = logs
items_dir =
jobs_to_keep = 5
dbs_dir = dbs
max_proc = 0
max_proc_per_cpu = 10
finished_to_keep = 100
poll_interval = 5.0
bind_address = 0.0.0.0
http_port = 6800
debug = off
runner = scrapyd.runner
application = scrapyd.app.application
launcher = scrapyd.launcher.launcher
webroot = scrapyd.website.root
[services]
schedule.json = scrapyd.webservice.schedule
cancel.json = scrapyd.webservice.cancel
addversion.json = scrapyd.webservice.addversion
listprojects.json = scrapyd.webservice.listprojects
listversions.json = scrapyd.webservice.listversions
listspiders.json = scrapyd.webservice.listspiders
delproject.json = scrapyd.webservice.deleteproject
delversion.json = scrapyd.webservice.deleteversion
listjobs.json = scrapyd.webservice.listjobs
daemonstatus.json = scrapyd.webservice.daemonstatus
主要更改bind_address=0.0.0.0
建立檔案後執行指令啟動scrapyd: (scrapyd > /dev/null &) 當想要記錄輸出日志時: (scrapyd > /root/scrapyd.log &)
坑1:當我執行完指令後報錯,說是找不到指令:

那是因為我系統上python2與3并存,是以找不到,這時應該做軟連接配接:
我的python3路徑: /usr/local/python3
制作軟連接配接: ln -s /usr/local/python3/bin/scrapy /usr/bin/scrapy
昨晚軟連接配接後,執行上邊指令,又報錯:
坑2:
這個好像是那個配置檔案的最後一行有問題,具體原因不大清楚,我将最後一行删除,再次重新執行,scrapyd就跑起來了