如何使用Scrapy从变量中提取文本？

Question

我正在使用Scrapy抓取一个商业目录，并且遇到了试图使用变量提取数据的问题。下面是代码：

def parse_page(self, response): 
    url = response.meta.get('URL') 

    # Parse the locations area of the page 
    locations = response.css('address::text').extract() 
    # Takes the City and Province and removes unicode and removes whitespace, 
    # they are still together though. 
    city_province = locations[1].replace(u'\xa0', u' ').strip() 
    # List of all social links that the business has 
    social = response.css('.entry-content > div:nth-child(2) a::attr(href)').extract() 

    add_info = response.css('ul.list-border li').extract() 
    year = "" 

    for info in add_info: 
     if 'Year' in info: 
      year = info 
     else: 
      pass 

    yield { 
     'title': response.css('h1.entry-title::text').extract_first().strip(), 
     'description': response.css('p.mb-double::text').extract_first(), 
     'phone_number': response.css('div.mb-double ul li::text').extract_first(default="").strip(), 
     'email': response.css('div.mb-double ul li a::text').extract_first(default=""), 
     'address': locations[0].strip(), 
     'city': city_province.split(' ', 1)[0].replace(',', ''), 
     'province': city_province.split(' ', 1)[1].replace(',', '').strip(), 
     'zip_code': locations[2].strip(), 
     'website': response.css('.entry-content > div:nth-child(2) > ul:nth-child(2) > li:nth-child(1) > a:nth-child(1)::attr(href)').extract_first(default=''), 
     'facebook': response.css('.entry-content > div:nth-child(2) > ul:nth-child(2) > li:nth-child(2) > a:nth-child(1)::attr(href)').extract_first(default=''), 
     'twitter': response.css('.entry-content > div:nth-child(2) > ul:nth-child(2) > li:nth-child(3) > a:nth-child(1)::attr(href)').extract_first(default=''), 
     'linkedin': response.css('.entry-content > div:nth-child(2) > ul:nth-child(2) > li:nth-child(4) > a:nth-child(1)::attr(href)').extract_first(default=''), 
     'year': year, 
     'employees': response.css('.list-border > li:nth-child(2)::text').extract_first(default="").strip(), 
     'key_contact': response.css('.list-border > li:nth-child(3)::text').extract_first(default="").strip(), 
     'naics': response.css('.list-border > li:nth-child(4)::text').extract_first(default="").strip(), 
     'tags': response.css('ul.biz-tags li a::text').extract(), 
    } 

我遇到的问题是从这里：

 add_info = response.css('ul.list-border li').extract() 
     year = "" 

     for info in add_info: 
      if 'Year' in info: 
       year = info 
      else: 
       pass 

代码检查，看看如果信息是“公司成立”。但是，它会返回HTML。我正在努力弄清楚，以便打印出年份。 add_info = response.css('ul.list-border li::text').extract()将打印出一年，但我如何在for循环中执行此操作？

每当“年”在info时，它的输出如下：<li><span>Year Established:</span> 1998</li>。我期待着获得年度而不是HTML。

Answer 1

添加以下功能。

def getYear(yearnum): 
    yearnum1 = str(yearnum[35:]) 
    yearnum2 = str(yearnum1[:4]) 
    return yearnum2 

然后将您的for语句替换为以下内容。

for info in add_info: 
    if 'Year' in info: 
     yearanswer = getYear(info) 
    else: 
     pass 

然后将采取的4位数字出你的长字符串，并把它放在字符串yearanswer。 如果你打印的是yearanswer，应该打印1998.它为我做了！