Modifier and Type | Field and Description |
---|---|
protected TypeDescription |
StripeStatistics.schema |
Modifier and Type | Method and Description |
---|---|
TypeDescription |
TypeDescription.addField(String field,
TypeDescription fieldType)
Add a field to a struct type as it is built.
|
TypeDescription |
TypeDescription.addUnionChild(TypeDescription child)
Add a child to a union type.
|
TypeDescription |
TypeDescription.clone() |
static TypeDescription |
OrcUtils.convertTypeFromProtobuf(List<OrcProto.Type> types,
int rootColumn)
Translate the given rootColumn from the list of types to a TypeDescription.
|
static TypeDescription |
TypeDescription.createBinary() |
static TypeDescription |
TypeDescription.createBoolean() |
static TypeDescription |
TypeDescription.createByte() |
static TypeDescription |
TypeDescription.createChar() |
static TypeDescription |
TypeDescription.createDate() |
static TypeDescription |
TypeDescription.createDecimal() |
static TypeDescription |
TypeDescription.createDouble() |
static TypeDescription |
TypeDescription.createFloat() |
static TypeDescription |
TypeDescription.createInt() |
static TypeDescription |
TypeDescription.createList(TypeDescription childType) |
static TypeDescription |
TypeDescription.createLong() |
static TypeDescription |
TypeDescription.createMap(TypeDescription keyType,
TypeDescription valueType) |
static TypeDescription |
TypeDescription.createShort() |
static TypeDescription |
TypeDescription.createString() |
static TypeDescription |
TypeDescription.createStruct() |
static TypeDescription |
TypeDescription.createTimestamp() |
static TypeDescription |
TypeDescription.createTimestampInstant() |
static TypeDescription |
TypeDescription.createUnion() |
static TypeDescription |
TypeDescription.createVarchar() |
TypeDescription |
TypeDescription.findSubtype(int goal)
Locate a subtype by its id.
|
TypeDescription |
TypeDescription.findSubtype(String columnName)
Find a subtype of this schema by name.
|
TypeDescription |
TypeDescription.findSubtype(String columnName,
boolean isSchemaEvolutionCaseAware) |
static TypeDescription |
TypeDescription.fromString(String typeName)
Parse TypeDescription from the Hive type names.
|
TypeDescription[] |
DataMaskDescription.getColumns()
Get the list of columns that use this mask.
|
TypeDescription |
TypeDescription.getParent()
Get the parent of the current type
|
TypeDescription |
EncryptionVariant.getRoot()
Get the root column for this variant.
|
TypeDescription |
Writer.getSchema()
Get the schema for this writer
|
TypeDescription |
Reader.getSchema()
Get the type of rows in this ORC file.
|
TypeDescription |
Reader.Options.getSchema() |
TypeDescription |
OrcFile.WriterOptions.getSchema() |
TypeDescription |
TypeDescription.removeAttribute(@NotNull String key)
Remove attribute on this type, if it is set.
|
TypeDescription |
TypeDescription.setAttribute(@NotNull String key,
@NotNull String value)
Set an attribute on this type.
|
TypeDescription |
TypeDescription.withMaxLength(int maxLength)
Set the maximum length for char and varchar types.
|
TypeDescription |
TypeDescription.withPrecision(int precision)
For decimal types, set the precision.
|
TypeDescription |
TypeDescription.withScale(int scale)
For decimal types, set the scale.
|
Modifier and Type | Method and Description |
---|---|
List<TypeDescription> |
TypeDescription.findSubtypes(String columnNameList)
Find a list of subtypes from a string, including the empty list.
|
List<TypeDescription> |
TypeDescription.getChildren()
Get the subtypes of this type.
|
Modifier and Type | Method and Description |
---|---|
void |
TypeDescription.addChild(TypeDescription child)
Add a child to a type.
|
TypeDescription |
TypeDescription.addField(String field,
TypeDescription fieldType)
Add a field to a struct type as it is built.
|
TypeDescription |
TypeDescription.addUnionChild(TypeDescription child)
Add a child to a union type.
|
DataMask |
DataMask.Provider.build(DataMaskDescription description,
TypeDescription schema,
DataMask.MaskOverrides overrides)
Build a mask with the given parameters.
|
static DataMask |
DataMask.Factory.build(DataMaskDescription mask,
TypeDescription schema,
DataMask.MaskOverrides overrides)
Build a new DataMask instance.
|
int |
TypeDescription.compareTo(TypeDescription other) |
static TypeDescription |
TypeDescription.createList(TypeDescription childType) |
static TypeDescription |
TypeDescription.createMap(TypeDescription keyType,
TypeDescription valueType) |
static List<OrcProto.Type> |
OrcUtils.getOrcTypes(TypeDescription typeDescr) |
DataMaskDescription |
DataMask.MaskOverrides.hasOverride(TypeDescription type)
Should the current mask be overridden on a sub-column?
|
static boolean[] |
OrcUtils.includeColumns(String selectedColumns,
TypeDescription schema)
Returns selected columns as a boolean array with true value set for specified column names.
|
static void |
TypeDescriptionPrettyPrint.print(PrintStream output,
TypeDescription schema) |
Reader.Options |
Reader.Options.schema(TypeDescription schema)
Set the schema on read type description.
|
OrcFile.WriterOptions |
OrcFile.WriterOptions.setSchema(TypeDescription schema)
Set the schema for the file.
|
Constructor and Description |
---|
StripeStatistics(TypeDescription schema,
List<OrcProto.ColumnStatistics> list,
boolean writerUsedProlepticGregorian,
boolean convertToProlepticGregorian) |
Modifier and Type | Field and Description |
---|---|
TypeDescription |
ParserUtils.TypeFinder.current |
protected TypeDescription |
ConvertTreeReaderFactory.StringGroupFromAnyIntegerTreeReader.readerType |
protected TypeDescription |
RecordReaderImpl.schema |
Modifier and Type | Method and Description |
---|---|
static TypeDescription |
SchemaEvolution.createEventSchema(TypeDescription typeDescr) |
static TypeDescription |
ParserUtils.findSubtype(TypeDescription schema,
ParserUtils.StringPosition source) |
static TypeDescription |
ParserUtils.findSubtype(TypeDescription schema,
ParserUtils.StringPosition source,
boolean isSchemaEvolutionCaseAware) |
static TypeDescription |
SchemaEvolution.getBaseRow(TypeDescription typeDescription)
Get the underlying base row from an ACID event struct.
|
TypeDescription[] |
MaskDescriptionImpl.getColumns() |
TypeDescription |
SchemaEvolution.getFileSchema() |
TypeDescription |
SchemaEvolution.getFileType(int id)
Get the file type by reader type id.
|
TypeDescription |
SchemaEvolution.getFileType(TypeDescription readerType) |
TypeDescription |
SchemaEvolution.getReaderBaseSchema()
Returns the non-ACID (aka base) reader type description.
|
TypeDescription |
SchemaEvolution.getReaderSchema() |
TypeDescription |
WriterImpl.getSchema() |
TypeDescription |
ReaderImpl.getSchema() |
TypeDescription |
OrcTail.getSchema() |
static TypeDescription |
ParserUtils.parseType(ParserUtils.StringPosition source) |
Modifier and Type | Method and Description |
---|---|
static List<TypeDescription> |
ParserUtils.findSubtypeList(TypeDescription schema,
ParserUtils.StringPosition source) |
Modifier and Type | Method and Description |
---|---|
void |
MaskDescriptionImpl.addColumn(TypeDescription column) |
protected void |
ConvertTreeReaderFactory.ConvertTreeReader.assignStringGroupVectorEntry(BytesColumnVector bytesColVector,
int elementNum,
TypeDescription readerType,
byte[] bytes) |
protected void |
ConvertTreeReaderFactory.ConvertTreeReader.assignStringGroupVectorEntry(BytesColumnVector bytesColVector,
int elementNum,
TypeDescription readerType,
byte[] bytes,
int start,
int length) |
static boolean |
ConvertTreeReaderFactory.canConvert(TypeDescription fileType,
TypeDescription readerType) |
static boolean |
SchemaEvolution.checkAcidSchema(TypeDescription type) |
protected void |
ConvertTreeReaderFactory.ConvertTreeReader.convertStringGroupVectorElement(BytesColumnVector bytesColVector,
int elementNum,
TypeDescription readerType) |
static ColumnStatisticsImpl |
ColumnStatisticsImpl.create(TypeDescription schema) |
static ColumnStatisticsImpl |
ColumnStatisticsImpl.create(TypeDescription schema,
boolean convertToProleptic) |
DataMask |
MaskDescriptionImpl.create(TypeDescription schema,
DataMask.MaskOverrides overrides) |
static ColumnVector |
TypeUtils.createColumn(TypeDescription schema,
TypeDescription.RowBatchVersion version,
int maxSize) |
static TypeReader |
ConvertTreeReaderFactory.createConvertTreeReader(TypeDescription readerType,
TreeReaderFactory.Context context)
(Rules from Hive's PrimitiveObjectInspectorUtils conversion)
To BOOLEAN, BYTE, SHORT, INT, LONG:
Convert from (BOOLEAN, BYTE, SHORT, INT, LONG) with down cast if necessary.
|
static TypeDescription |
SchemaEvolution.createEventSchema(TypeDescription typeDescr) |
static BatchReader |
TreeReaderFactory.createRootReader(TypeDescription readerType,
TreeReaderFactory.Context context) |
static TypeReader |
TreeReaderFactory.createTreeReader(TypeDescription readerType,
TreeReaderFactory.Context context) |
static ColumnStatisticsImpl |
ColumnStatisticsImpl.deserialize(TypeDescription schema,
OrcProto.ColumnStatistics stats) |
static ColumnStatisticsImpl |
ColumnStatisticsImpl.deserialize(TypeDescription schema,
OrcProto.ColumnStatistics stats,
boolean writerUsedProlepticGregorian,
boolean convertToProlepticGregorian) |
ColumnStatistics[] |
ReaderImpl.deserializeStats(TypeDescription schema,
List<OrcProto.ColumnStatistics> fileStats) |
void |
ConvertTreeReaderFactory.ConvertTreeReader.downCastAnyInteger(LongColumnVector longColVector,
int elementNum,
long inputLong,
TypeDescription readerType) |
void |
ConvertTreeReaderFactory.ConvertTreeReader.downCastAnyInteger(LongColumnVector longColVector,
int elementNum,
TypeDescription readerType) |
static void |
ParserUtils.findColumn(TypeDescription schema,
List<String> names,
boolean isSchemaEvolutionCaseAware,
ParserUtils.TypeVisitor visitor)
Find a column in a schema by walking down the type tree to find the right column.
|
static void |
ParserUtils.findColumn(TypeDescription schema,
ParserUtils.StringPosition source,
boolean isSchemaEvolutionCaseAware,
ParserUtils.TypeVisitor visitor)
Find a column in a schema by walking down the type tree to find the right column.
|
static ColumnVector[] |
ParserUtils.findColumnVectors(TypeDescription schema,
ParserUtils.StringPosition source,
boolean isCaseSensitive,
VectorizedRowBatch batch) |
static void |
ParserUtils.findSubtype(TypeDescription schema,
int goal,
ParserUtils.TypeVisitor visitor) |
static TypeDescription |
ParserUtils.findSubtype(TypeDescription schema,
ParserUtils.StringPosition source) |
static TypeDescription |
ParserUtils.findSubtype(TypeDescription schema,
ParserUtils.StringPosition source,
boolean isSchemaEvolutionCaseAware) |
static List<TypeDescription> |
ParserUtils.findSubtypeList(TypeDescription schema,
ParserUtils.StringPosition source) |
static TypeDescription |
SchemaEvolution.getBaseRow(TypeDescription typeDescription)
Get the underlying base row from an ACID event struct.
|
TypeDescription |
SchemaEvolution.getFileType(TypeDescription readerType) |
protected boolean |
ConvertTreeReaderFactory.ConvertTreeReader.integerDownCastNeeded(TypeDescription fileType,
TypeDescription readerType) |
static void |
ParserUtils.parseKeys(ParserUtils.StringPosition source,
TypeDescription schema)
Annotate the given schema with the encryption information.
|
static void |
ParserUtils.parseMasks(ParserUtils.StringPosition source,
TypeDescription schema)
Annotate the given schema with the masking information.
|
void |
ParserUtils.TypeVisitor.visit(TypeDescription type,
int posn)
As we navigate to the column, call this on each level
|
void |
ParserUtils.TypeFinder.visit(TypeDescription type,
int posn) |
Constructor and Description |
---|
ListTreeReader(int fileColumn,
TypeDescription readerSchema,
TreeReaderFactory.Context context) |
MapTreeReader(int fileColumn,
TypeDescription readerSchema,
TreeReaderFactory.Context context) |
OrcFilterContextImpl(TypeDescription readSchema,
boolean isSchemaCaseAware) |
SchemaEvolution(TypeDescription fileSchema,
boolean[] readerIncluded)
Deprecated.
|
SchemaEvolution(TypeDescription fileSchema,
TypeDescription readerSchema,
boolean[] readerIncluded)
Deprecated.
|
SchemaEvolution(TypeDescription fileSchema,
TypeDescription readerSchema,
Reader.Options options) |
StripeStatisticsImpl(TypeDescription schema,
boolean writerUsedProlepticGregorian,
boolean convertToProlepticGregorian) |
StripeStatisticsImpl(TypeDescription schema,
List<OrcProto.ColumnStatistics> list,
boolean writerUsedProlepticGregorian,
boolean convertToProlepticGregorian) |
StructTreeReader(int columnId,
TypeDescription readerSchema,
TreeReaderFactory.Context context) |
TypeFinder(TypeDescription schema) |
UnionTreeReader(int fileColumn,
TypeDescription readerSchema,
TreeReaderFactory.Context context) |
Modifier and Type | Method and Description |
---|---|
static BatchFilter |
FilterFactory.createBatchFilter(Reader.Options opts,
TypeDescription readSchema,
boolean isSchemaCaseAware,
OrcFile.Version version,
boolean normalize)
Create a BatchFilter.
|
static VectorFilter |
FilterFactory.createSArgFilter(ExpressionTree expr,
Set<String> colIds,
List<PredicateLeaf> leaves,
TypeDescription readSchema,
boolean isSchemaCaseAware,
OrcFile.Version version) |
Modifier and Type | Method and Description |
---|---|
static VectorFilter |
LeafFilterFactory.createLeafVectorFilter(PredicateLeaf leaf,
Set<String> colIds,
TypeDescription readSchema,
boolean isSchemaCaseAware,
OrcFile.Version version,
boolean negated) |
Modifier and Type | Method and Description |
---|---|
DataMask |
MaskProvider.build(DataMaskDescription description,
TypeDescription schema,
DataMask.MaskOverrides overrides) |
DataMask |
MaskFactory.build(TypeDescription schema,
DataMask.MaskOverrides overrides) |
protected DataMask |
RedactMaskFactory.buildBinaryMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildBinaryMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildBinaryMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildBooleanMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildBooleanMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildBooleanMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildDateMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildDateMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildDateMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildDecimalMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildDecimalMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildDecimalMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildDoubleMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildDoubleMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildDoubleMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildLongMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildLongMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildLongMask(TypeDescription schema) |
protected DataMask |
RedactMaskFactory.buildStringMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildStringMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildStringMask(TypeDescription schema) |
protected DataMask |
MaskFactory.buildStructMask(TypeDescription schema,
DataMask.MaskOverrides overrides) |
protected DataMask |
RedactMaskFactory.buildTimestampMask(TypeDescription schema) |
protected DataMask |
SHA256MaskFactory.buildTimestampMask(TypeDescription schema) |
protected abstract DataMask |
MaskFactory.buildTimestampMask(TypeDescription schema) |
Modifier and Type | Method and Description |
---|---|
protected DataMask[] |
MaskFactory.buildChildren(List<TypeDescription> children,
DataMask.MaskOverrides overrides) |
Modifier and Type | Method and Description |
---|---|
TypeDescription |
ReaderEncryptionVariant.getRoot() |
Constructor and Description |
---|
ReaderEncryption(OrcProto.Footer footer,
TypeDescription schema,
long stripeStatisticsOffset,
BufferChunk serializedTail,
List<StripeInformation> stripes,
KeyProvider provider,
Configuration conf) |
StripePlanner(TypeDescription schema,
ReaderEncryption encryption,
DataReader dataReader,
OrcFile.WriterVersion version,
boolean ignoreNonUtf8BloomFilter,
long maxBufferSize) |
StripePlanner(TypeDescription schema,
ReaderEncryption encryption,
DataReader dataReader,
OrcFile.WriterVersion version,
boolean ignoreNonUtf8BloomFilter,
long maxBufferSize,
Set<Integer> filterColIds)
Create a stripe parser.
|
Modifier and Type | Field and Description |
---|---|
protected TypeDescription |
TreeWriterBase.schema |
Modifier and Type | Method and Description |
---|---|
TypeDescription |
WriterEncryptionVariant.getRoot() |
Modifier and Type | Method and Description |
---|---|
static TreeWriter |
TreeWriter.Factory.create(TypeDescription schema,
WriterEncryptionVariant encryption,
WriterContext streamFactory)
Create a new tree writer for the given types and insert encryption if
required.
|
Copyright © 2013–2022 The Apache Software Foundation. All rights reserved.