Larbin使用方法2

http://larbin.sourceforge.net/custom-eng.html

How to customize Larbin文中有介紹

配置檔案larbin.conf檔案各項簡介：（修改配置檔案不需要重新編譯larbin）

###############################################

# Who are you ?

# mail of the one who launched larbin (YOUR mail)

From [email protected] #//用于http頭，web伺服器管理者可通過該位址聯

#//系執行larbin的人

# name of the bot (sent with http headers)

UserAgent larbin_2.6.3 #//用戶端标志 ############################################

# What are the inputs and ouputs of larbin

# port on which is launched the http statistic webserver

# if unset or set to 0, no webserver is launched

httpPort 8081 #//laibin有一個簡單的web伺服器，可通過http://localhost::8081監控其

#//運作狀态，如果該值為0，則不啟動web伺服器。

# port on which you can submit urls to fetch

# no input is possible if you comment this line or use port 0

inputPort 1976

############################################

# parameters to adapt depending on your network

# Number of connexions in parallel (to adapt depending of your network speed)

pagesConnexions 100 #//并行擷取網頁的數量，該值可依據網絡帶寬調整

# Number of dns calls in parallel

dnsConnexions 5 #//并行解析dns的數量

# How deep do you want to go in a site

depthInSite 5 #//網頁抓取深度

# do you want to follow external links

noExternalLinks #//是否允許抓取域名外連接配接

# time between 2 calls on the same server (in sec) : NEVER less than 30

waitDuration 60 #//對同一個伺服器擷取網頁的間隔時間

# Make requests through a proxy (use with care)

#proxy www 8080 #//代理位址

##############################################

# now, let's customize the search

# first page to fetch (you can specify several urls)

startUrl http://www.baidu.com #//抓取網頁的起始URL,可指定多值

# Do you want to limit your search to a specific domain ?

# if yes, uncomment the following line

#limitToDomain .fr .dk .uk end

# What are the extensions you surely don't want

# never forbid .html, .htm and so on : larbin needs them

#//限制不被下載下傳的對象的字尾,可通過注釋或者增加字尾控制下載下傳

forbiddenExtensions

.tar .gz .tgz .zip .Z .rpm .deb

.ps .dvi .pdf

.png .jpg .jpeg .bmp .smi .tiff .gif

.mov .avi .mpeg .mpg .mp3 .qt .wav .ram .rm

.jar .java .class .diff

.doc .xls .ppt .mdb .rtf .exe .pps .so .psd

end

附上