Spark中RDD的基本转化操作
对一个数据为{1,2,3,4}的RDD进行基本的RDD转化操作:
下面用scala语言进行实现:
1:val example = sc.parallelize(List(1, 2, 3, 3))
val result = example.map(x => x+1)
println(result.collect().mkString(","))
2:val example = sc.parallelize(List(1, 2, 3, 3))
val result = example.flatMap(x =>x.to(3))
println(result.collect().mkString(","))
3:val example = sc.parallelize(List(1, 2, 3, 3))
val result = example.filter(x => x !=1)
println(result.collect().mkString(","))
4:val example = sc.parallelize(List(1, 2, 3, 3))
val result = example.distinct()
println(result.collect().mkString(","))
5:val example = sc.parallelize(List(1, 2, 3, 3))
val result = example.sample()
println(result.collect().mkString(","))