org.apache.lens.ml.spark
Class HiveTableRDD

java.lang.Object
  org.apache.lens.ml.spark.HiveTableRDD

public final class HiveTableRDD
extends Object

Create a JavaRDD based on a Hive table using HCatInputFormat.

Field Summary

static org.apache.commons.logging.Log

LOG

Method Summary

static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.WritableComparable,org.apache.hive.hcatalog.data.HCatRecord>

createHiveTableRDD(org.apache.spark.api.java.JavaSparkContext javaSparkContext, org.apache.hadoop.conf.Configuration conf, String db, String table, String partitionFilter) Creates the hive table rdd.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Method Detail

createHiveTableRDD

public static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.WritableComparable,org.apache.hive.hcatalog.data.HCatRecord> createHiveTableRDD(org.apache.spark.api.java.JavaSparkContext javaSparkContext,
                                                                                                                                                         org.apache.hadoop.conf.Configuration conf,
                                                                                                                                                         String db,
                                                                                                                                                         String table,
                                                                                                                                                         String partitionFilter)
                                                                                                                                                  throws IOException

Creates the hive table rdd.

Parameters:

javaSparkContext - the java spark context

conf - the conf

db - the db

table - the table

partitionFilter - the partition filter

Returns:

the java pair rdd

Throws:

IOException - Signals that an I/O exception has occurred.