抓取网上的小说章节并写入txt文件代码,Python代码库,德仔网

代码语言

代码分类

【Python】抓取网上的小说章节并写入txt文件

作者:lj_april / 发布于2012/10/22/ 676

import urllib.request as web</div>
import re</div>
<div></div>
<div>'''用来过滤出小说html中小说的章节名与内容'''</div>
<div>def getContent(url):</div>
<div>  http = str(web.urlopen(url).read(),encoding='GBK')</div>
<div>  title = re.findall('<h1>.*?</h1>',http)[0]</div>
<div>  title = re.sub('</?h1>','',title)</div>
<div> </div>
<div>  content = re.findall('<div id="content">.*?</div>',http)[0]</div>
<div>  content = re.sub('','\n',content)</div>
<div>  content = re.sub('<div id="content">|</div>','',content)</div>
<div>  content = re.sub('&nbsp;',' ',content)</div>
<div>  return (title,content)</div>
<div></div>
<div>'''用来获取目录页码html下的章节超链接'''</div>
<div>def getUrlList(url):</div>
<div>  http = str(web.urlopen(url).read(),encoding='GBK')</div>
<div>  lis = re.findall('<a.*?章.*?</a>',http)</div>
<div>  hrefs = []</div>
<div>  for l in lis:</div>
<div>    try:</div>
<div>      hrefs.append(l.split('"')[1])</div>
<div>    except:</div>
<div>      pass</div>
<div>  return hrefs</div>
<div></div>
<div></div>
if __name__ == '__main__':</div>
<div>  url = '小说地址url'</div>
<div>  f = open('e://name.txt',mode='w')</div>
<div>  urlList = getUrlList(url)</div>
<div>  numUrlList = []</div>
<div>  for u in urlList[:-1]:</div>
<div>    try:</div>
<div>      #print(url,' ',url[:-5])</div>
<div>      numUrlList.append(int(u[:-5]))</div>
<div>    except:</div>
<div>      pass</div>
<div>  numUrlList.sort()</div>
<div></div>
<div>  for href in numUrlList:</div>
<div>    h = url + str(href) + '.html'</div>
<div>    print(h)</div>
<div>    try:</div>
<div>      c = getContent(h)</div>
<div>    except:</div>
<div>      try:</div>
<div>        c = getContent(h)</div>
<div>      except:</div>
<div>        print('读取失败了')</div>
<div>        continue</div>
<div>    title,content = c</div>
<div>    print(title,'完成')</div>
<div>    f.write(title+'\n')</div>
<div>    f.write(content)</div>
<div>    f.write('\n')</div>
<div>  print('全部完成了，ohyeah')</div>
<div>  f.close() </div>
<div></div>

评论列表

本站所提供的代码，版权归原作者所有，若有侵犯作者版权，请与我们联系，我们将立即删除或修改。谢谢!
本站所有代码发布及提供者。

试试其它关键字

　写入txt文件　

同语言下

可能有用的

lj_april贡献的其它代码(2)