effective crowd annotation for 【relation extraction】
crowdsource众包
identify instances of relations
supervised machine learning
distant supervision
crowdsourced data is as good as or better than that annotated by experts
gated instruction GI
Background and Related Work
Distant supervision (DS) is a method for training
extractors that obviates the need for human-labeled
training data by heuristically matching facts from a
background knowledge base (KB) to a large textual
corpus.
3 Gated Instruction Crowdsourcing
3 Interactive Tutorial Design
crowdsource step, most importantensure workers understand the task
?