org.apache.lens.ml
Class LensMLImpl

java.lang.Object
  org.apache.lens.ml.LensMLImpl

All Implemented Interfaces:

LensML

public class LensMLImpl
extends Object
implements LensML

The Class LensMLImpl.

Field Summary

static org.apache.commons.logging.Log

LOG The Constant LOG.

Fields inherited from interface org.apache.lens.ml.LensML

NAME

Constructor Summary

LensMLImpl(org.apache.hadoop.hive.conf.HiveConf conf)
Instantiates a new lens ml impl.

Method Summary

void	clearModels() Clear models.
void	deleteModel(String algorithm, String modelID) Permanently delete a model instance.
void	deleteTestReport(String algorithm, String reportID) Permanently delete a test report instance.
MLAlgo	getAlgoForName(String algorithm) Get a algo object instance which could be used to generate a model of the given algorithm.
Map<String,String>	getAlgoParamDescription(String algorithm) Get user friendly information about parameters accepted by the algorithm.
List<String>	getAlgorithms() Get list of available machine learning algorithms
org.apache.hadoop.hive.conf.HiveConf	getConf()
org.apache.hadoop.hive.conf.HiveConf	getHiveConf()
MLModel	getModel(String algorithm, String modelId) Get a model instance given the algorithm name and model ID.
String	getModelPath(String algorithm, String modelID) Get the FS location where model instance is saved.
List<String>	getModels(String algorithm) Get model IDs for the given algorithm.
MLTestReport	getTestReport(String algorithm, String reportID) Get a test report by ID.
List<String>	getTestReports(String algorithm) Get test reports for an algorithm.
void	init(org.apache.hadoop.hive.conf.HiveConf hiveConf) Inits the.
Object	predict(String algorithm, String modelID, Object[] features) Online predict call given a model ID, algorithm name and sample feature values.
void	setSparkContext(org.apache.spark.api.java.JavaSparkContext jsc) Use an existing Spark context.
void	start() Start.
void	stop() Stop.
MLTestReport	testModel(LensSessionHandle session, String table, String algorithm, String modelID, String outputTable) Evaluate model by running it against test data contained in the given table.
MLTestReport	testModel(LensSessionHandle sessionHandle, String table, String algorithm, String modelID, TestQueryRunner queryRunner, String outputTable) Evaluate a model.
MLTestReport	testModelRemote(LensSessionHandle sessionHandle, String table, String algorithm, String modelID, String queryApiUrl, String outputTable) Test a model in embedded mode.
String	train(String table, String algorithm, String[] args) Create a model using the given HCatalog table as input.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

The Constant LOG.

Constructor Detail

LensMLImpl

public LensMLImpl(org.apache.hadoop.hive.conf.HiveConf conf)

Instantiates a new lens ml impl.

Parameters:

conf - the conf

Method Detail

getConf

public org.apache.hadoop.hive.conf.HiveConf getConf()

setSparkContext

public void setSparkContext(org.apache.spark.api.java.JavaSparkContext jsc)

Use an existing Spark context. Useful in case of

Parameters:

jsc - JavaSparkContext instance

getAlgorithms

public List<String> getAlgorithms()

Description copied from interface: LensML

Get list of available machine learning algorithms

Specified by:

getAlgorithms in interface LensML

Returns:

getAlgoForName

public MLAlgo getAlgoForName(String algorithm)
                      throws LensException

Description copied from interface: LensML

Get a algo object instance which could be used to generate a model of the given algorithm.

Specified by:

getAlgoForName in interface LensML

Parameters:

algorithm - the algorithm

Returns:

the algo for name

Throws:

LensException - the lens exception

train

public String train(String table,
                    String algorithm,
                    String[] args)
             throws LensException

Description copied from interface: LensML

Create a model using the given HCatalog table as input. The arguments should contain information needeed to generate the model.

Specified by:

train in interface LensML

Parameters:

table - the table

algorithm - the algorithm

args - the args

Returns:

Unique ID of the model created after training is complete

Throws:

LensException - the lens exception

getModels

public List<String> getModels(String algorithm)
                       throws LensException

Description copied from interface: LensML

Get model IDs for the given algorithm.

Specified by:

getModels in interface LensML

Parameters:

algorithm - the algorithm

Returns:

the models

Throws:

LensException - the lens exception

getModel

public MLModel getModel(String algorithm,
                        String modelId)
                 throws LensException

Description copied from interface: LensML

Get a model instance given the algorithm name and model ID.

Specified by:

getModel in interface LensML

Parameters:

algorithm - the algorithm

modelId - the model id

Returns:

the model

Throws:

LensException - the lens exception

init

public void init(org.apache.hadoop.hive.conf.HiveConf hiveConf)

Inits the.

Parameters:

hiveConf - the hive conf

start

public void start()

Start.

stop

public void stop()

Stop.

getHiveConf

public org.apache.hadoop.hive.conf.HiveConf getHiveConf()

clearModels

public void clearModels()

Clear models.

getModelPath

public String getModelPath(String algorithm,
                           String modelID)

Description copied from interface: LensML

Get the FS location where model instance is saved.

Specified by:

getModelPath in interface LensML

Parameters:

algorithm - the algorithm

modelID - the model id

Returns:

the model path

testModel

public MLTestReport testModel(LensSessionHandle session,
                              String table,
                              String algorithm,
                              String modelID,
                              String outputTable)
                       throws LensException

Description copied from interface: LensML

Evaluate model by running it against test data contained in the given table.

Specified by:

testModel in interface LensML

Parameters:

session - the session

table - the table

algorithm - the algorithm

modelID - the model id

Returns:

Test report object containing test output table, and various evaluation metrics

Throws:

LensException - the lens exception

testModelRemote

public MLTestReport testModelRemote(LensSessionHandle sessionHandle,
                                    String table,
                                    String algorithm,
                                    String modelID,
                                    String queryApiUrl,
                                    String outputTable)
                             throws LensException

Test a model in embedded mode.

Parameters:

sessionHandle - the session handle

table - the table

algorithm - the algorithm

modelID - the model id

queryApiUrl - the query api url

Returns:

the ML test report

Throws:

LensException - the lens exception

testModel

public MLTestReport testModel(LensSessionHandle sessionHandle,
                              String table,
                              String algorithm,
                              String modelID,
                              TestQueryRunner queryRunner,
                              String outputTable)
                       throws LensException

Evaluate a model. Evaluation is done on data selected table from an input table. The model is run as a UDF and its output is inserted into a table with a partition. Each evaluation is given a unique ID. The partition label is associated with this unique ID.

This call also required a query runner. Query runner is responsible for executing the evaluation query against Lens server.

Parameters:

sessionHandle - the session handle

table - the table

algorithm - the algorithm

modelID - the model id

queryRunner - the query runner

outputTable - table where test output will be written

Returns:

the ML test report

Throws:

LensException - the lens exception

getTestReports

public List<String> getTestReports(String algorithm)
                            throws LensException

Description copied from interface: LensML

Get test reports for an algorithm.

Specified by:

getTestReports in interface LensML

Parameters:

algorithm - the algorithm

Returns:

the test reports

Throws:

LensException - the lens exception

getTestReport

public MLTestReport getTestReport(String algorithm,
                                  String reportID)
                           throws LensException

Description copied from interface: LensML

Get a test report by ID.

Specified by:

getTestReport in interface LensML

Parameters:

algorithm - the algorithm

reportID - the report id

Returns:

the test report

Throws:

LensException - the lens exception

predict

public Object predict(String algorithm,
                      String modelID,
                      Object[] features)
               throws LensException

Description copied from interface: LensML

Online predict call given a model ID, algorithm name and sample feature values.

Specified by:

predict in interface LensML

Parameters:

algorithm - the algorithm

modelID - the model id

features - the features

Returns:

prediction result

Throws:

LensException - the lens exception

deleteModel

public void deleteModel(String algorithm,
                        String modelID)
                 throws LensException

Description copied from interface: LensML

Permanently delete a model instance.

Specified by:

deleteModel in interface LensML

Parameters:

algorithm - the algorithm

modelID - the model id

Throws:

LensException - the lens exception

deleteTestReport

public void deleteTestReport(String algorithm,
                             String reportID)
                      throws LensException

Description copied from interface: LensML

Permanently delete a test report instance.

Specified by:

deleteTestReport in interface LensML

Parameters:

algorithm - the algorithm

reportID - the report id

Throws:

LensException - the lens exception

getAlgoParamDescription

public Map<String,String> getAlgoParamDescription(String algorithm)

Description copied from interface: LensML

Get user friendly information about parameters accepted by the algorithm.

Specified by:

getAlgoParamDescription in interface LensML

Parameters:

algorithm - the algorithm

Returns:

map of param key to its help message