syntax.us Let the syntax do the talking
Blog Contact Posts Questions Tags Hire Me

Question:
In H2O Sparkling Water how to navigate RDD?

H2O uses an enhanced type of RDD in Sparkling Water.

I can see it in action at the URL below:

https://github.com/h2oai/sparkling-water/blob/master/core/src/test/scala/org/apache/spark/rdd/H2ORDDTest.scala

This RDD does not support take() and drop(), how do I navigate it?

Here is a simple demo:
import java.io._
import org.apache.spark.h2o._
import water.fvec._
import water.fvec.DataFrame

val hc = new H2OContext(sc).start()
import hc._

val cars = Array("415-mini","650-porsche","408-bmw","510-pizza")

val rdd2 = sc.makeRDD(cars).map( v => StringHolder(Some(v)))


I can paste the above syntax into a sparkling-shell.

Then, I should have an object named rdd2 which is an enhanced H2O RDD.

How can I navigate it?

I first tried take().

That gave me an Array.

I used (0) to get inside the Array.

Then I used a call to:

.result.get 

to get the String:

scala> 
scala> 
scala> rdd2.take(1)(0).result.get
res8: String = 415-mini

scala> 
scala> 
scala> 


syntax.us Let the syntax do the talking
Blog Contact Posts Questions Tags Hire Me