Lens session configuration


The configuration parameters and their default values
No. Property Name Default Value Description
1 hive.metastore.batch.retrieve.max 100 Maximum number of objects (tables/partitions) can be retrieved from metastore in one batch. The higher the number, the less the number of round trips is needed to the Hive metastore server, but it may also cause higher memory requirement at the client side.
2 hive.metastore.batch.retrieve.table.partition.max 500 Maximum number of table partitions that metastore internally retrieves in one batch.
3 hive.metastore.client.connect.retry.delay 1 Number of seconds for the client to wait between consecutive connection attempts
4 hive.metastore.client.socket.timeout 20 MetaStore Client socket timeout in seconds
5 hive.metastore.connect.retries 5 Number of retries while opening a connection to metastore
6 hive.metastore.failure.retries 3 Number of call retries when Hive Metastore calls fail with Thrift errros
7 hive.metastore.uris The hive metastore server URI that the lens server is talking to
8 lens.cube.query.disable.aggregate.resolver false Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.
9 lens.cube.query.disable.auto.join false Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.
10 lens.cube.query.fail.if.data.partial true Whether to fail the query of data is partial
11 lens.cube.query.promote.select.togroupby true Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.
12 lens.query.add.insert.overwrite true Prefix query with insert overwrite clause if the query is persistent. User can disable if user gave the clause himself.
13 lens.query.enable.mail.notify false When a query ends, whether to notify the submitter by mail or not.
14 lens.query.enable.metrics.per.query false Generates gauge metrics for each query to measure time taken with unique id appended for each query. Should be enabled only for performance measurements. Should not be enabled in day to day production environment.
15 lens.query.enable.persistent.resultset false Whether to enable persistent resultset for queries. When enabled, server will fetch results from driver, custom format them if any and store in a configured location. The file name of query output is queryhandle-id, with configured extensions
16 lens.query.enable.persistent.resultset.indriver true Whether the result should be persisted by driver. Currently only HiveDriver persists the results in a HDFS location.
17 lens.query.hdfs.output.path hdfsout The directory under the parent result directory, in which HiveDriver will persist the results, if persisting by driver is enabled. This directory should exist and should have world writable permissions sothat all users will be able put query outputs here.
18 lens.query.output.charset.encoding UTF-8 The charset encoding for formatting query result. It supports all the encodings supported by java.io.OutputStreamWriter.
19 lens.query.output.compression.codec org.apache.hadoop.io.compress.GzipCodec The codec used to compress the query output, if compression is enabled
20 lens.query.output.enable.compression false Whether to compress the query result output
21 lens.query.output.file.extn .csv The extension name for the persisted query output file. If file is compressed, the extension from compression codec will be appended to this extension.
22 lens.query.output.footer The value of custom footer that should be written, if any. This footer will be added in formatting driver persisted results.
23 lens.query.output.formatter The query result output formatter for the query. If no value is specified, then org.apache.lens.lib.query.FileSerdeFormatter will be used to format in-memory result sets, org.apache.lens.lib.query.FilePersistentFormatter will be used to format driver persisted result sets.
24 lens.query.output.header The value of custom header that should be written, if any. If no value column names will be used as header.
25 lens.query.output.write.footer false Whether to write footer as part of query result. When enabled, total number of rows will be written as part of header.
26 lens.query.output.write.header false Whether to write header as part of query result formatting. When enabled the user given header will be added in case of driver persisted results, and column names chosen will be added as header for in-memory results.
27 lens.query.result.email.cc When query ends, the result/failure reason will be sent to the user via email. The mail would be cc'ed to the addresses provided in this field.
28 lens.query.result.fs.read.url Http read URL for FileSystem on which result is present, if available. For example webhdfs as http read url should http://host:port/webhdfs/v1. Currently we support only webhdfs url as the http url for HDFS file system
29 lens.query.result.output.dir.format The format of the output if result is persisted in hdfs. The format should be expressed in HQL.
30 lens.query.result.output.serde org.apache.lens.lib.query.CSVSerde The default serde class name that should be used by org.apache.lens.lib.query.FileSerdeFormatter for formatting the output
31 lens.query.result.parent.dir file:///tmp/lensreports The directory for storing persisted result of query. This directory should exist and should have writable permissions by lens server
32 lens.query.result.size.format.threshold 10737418240 The maximum allowed size of the query result. If exceeds, no server side formatting would be done.
33 lens.query.result.split.multiple false Whether to split the result into multiple files. If enabled, each file will be restricted to max rows configured. All the files will be available as zip.
34 lens.query.result.split.multiple.maxrows 100000 The maximum number of rows allowed in each file, when splitting the result into multiple files is enabled.
35 lens.session.aux.jars List of comma separated jar paths, which will added to the session
36 lens.session.cluster.user Session level config which will determine which cluster user will access hdfs
37 lens.session.loggedin.user The username used to log in to lens. e.g. LDAP user