数据提取-

1.Python中的json模块

• json.loads()

把json格式的字符串解码转换为Python对象

• json.dumps()

把Python对象编码转换为json字符串

• json.dump()

把Python内置类型转换为json对象后写入文件

• json.load()

读取文件中的json形式的字符串元素转换为Python类型

import json
# 注释后面是输出结果
str='{"name":"神偷奶爸"}'
print(type(str),":",str)	#<class 'str'> : {"name":"神偷奶爸"}
obj=json.loads(str)
print(type(obj),":",obj)	#<class 'dict'> : {'name': '神偷奶爸'}
str2=json.dumps(obj,ensure_ascii=False)
print(type(str2),":",str2)	#<class 'str'> : {"name":"神偷奶爸"}
json.dump(obj,open('movie.txt','w',encoding='utf-8'),ensure_ascii=False)
obj3=json.load(open('movie.txt', 'r', encoding='utf-8'))
print(type(obj3), ":", obj3)	#<class 'dict'> : {'name': '神偷奶爸'}

2. jsonpath

官方文档：http://goessner.net/articles/JsonPath

2.1 安装

pip install jsonpath

2.2 jsonpath与xpath的对比

import json
from random import choice
import requests
from jsonpath import jsonpath
user_agents=[
    "User-Agent:Mozilla/5.0(Windows;U;WindowsNT6.1;en-us)AppleWebKit/534.50(KHTML,likeGecko)Version/5.1Safari/534.50",
    "User-Agent:Mozilla/5.0(WindowsNT6.1;rv:2.0.1)Gecko/20100101Firefox/4.0.1",
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
]
headers={
    "User-Agent":choice(user_agents)
}
url="http://www.lagou.com/lbs/getAllCitySearchLabels.json"
response=requests.get(url,headers)
names=jsonpath(json.loads(response.text),'$..name')#核心代码
print(names)

结果：