#include <Schema.hpp>

Public Types
enum class	MemoryLayoutType : uint8_t { ROW_LAYOUT = 0 , COLUMNAR_LAYOUT = 1 }
	Enum to identify the memory layout in which we want to represent the schema physically. More...

Public Member Functions
	Schema (MemoryLayoutType layoutType=MemoryLayoutType::ROW_LAYOUT)

	Schema (SchemaPtr const &query, MemoryLayoutType layoutType=MemoryLayoutType::ROW_LAYOUT)

SchemaPtr	updateSourceName (const std::string &srcName) const
	Prepends the srcName to the substring after the last occurrence of ATTRIBUTE_NAME_SEPARATOR in every field name of the schema. More...

SchemaPtr	copy () const
	Creates a copy of this schema. More...

SchemaPtr	copyFields (const SchemaPtr &otherSchema)
	Copy all fields of otherSchema into this schema. More...

SchemaPtr	addField (const AttributeFieldPtr &attribute)
	appends a AttributeField to the schema and returns a copy of this schema. More...

SchemaPtr	addField (const std::string &name, const BasicType &type)
	appends a field with a basic type to the schema and returns a copy of this schema. More...

SchemaPtr	addField (const std::string &name, DataTypePtr data)
	appends a field with a data type to the schema and returns a copy of this schema. More...

void	removeField (const AttributeFieldPtr &field)
	removes a AttributeField from the schema More...

void	replaceField (const std::string &name, const DataTypePtr &type)
	Replaces a field, which is already part of the schema. More...

AttributeFieldPtr	getField (const std::string &fieldName) const
	Returns the attribute field based on a qualified or unqualified field name. More...

uint64_t	getIndex (const std::string &fieldName) const
	Checks if attribute field name is defined in the schema and returns its index. If item not in the list, then the return value is equal to fields.size(). More...

AttributeFieldPtr	get (const std::string &fieldName) const
	Finds a attribute field by name in the schema. More...

AttributeFieldPtr	get (uint32_t index)
	Finds a attribute field by index in the schema. More...

uint64_t	getSize () const
	Returns the number of fields in the schema. More...

uint64_t	getSchemaSizeInBytes () const
	Returns the number of bytes all fields in this schema occupy. More...

bool	equals (const SchemaPtr &schema, bool considerOrder=true)
	Checks if two Schemas are equal to each other. More...

bool	hasEqualTypes (const SchemaPtr &otherSchema)
	Checks if two schemas have same datatypes at same index location. More...

bool	contains (const std::string &fieldName) const
	Checks if the field exists in the schema. More...

std::string	toString (const std::string &prefix="", const std::string &sep=" ", const std::string &suffix="") const
	returns a string representation More...

std::string	getLayoutTypeAsString () const
	returns the string representation of layout More...

std::string	getSourceNameQualifier () const
	Method to return the source name qualifier, thus everything that is before $. More...

std::string	getQualifierNameForSystemGeneratedFields () const
	method to get the qualifier of the source without $ More...

std::string	getQualifierNameForSystemGeneratedFieldsWithSeparator () const
	method to get the qualifier of the source with $ More...

void	clear ()
	Remove all fields and qualifying name. More...

bool	empty () const
	Is checks if the schema is empty (if it has no fields). More...

MemoryLayoutType	getLayoutType () const
	method to get the type of the memory layout More...

void	setLayoutType (MemoryLayoutType layoutType)
	method to set the memory layout More...

std::vector< std::string >	getFieldNames () const
	Get the field names as a vector of strings. More...

Static Public Member Functions
static SchemaPtr	create (MemoryLayoutType layoutType=MemoryLayoutType::ROW_LAYOUT)
	Factory method to create a new SchemaPtr. More...

static SchemaPtr	createFromSchemaType (const Configurations::SchemaTypePtr &schemaType, MemoryLayoutType layoutType=MemoryLayoutType::ROW_LAYOUT)
	Factory method to create a new SchemaPtr from schema type. More...

Public Attributes
std::vector< AttributeFieldPtr >	fields

Static Public Attributes
constexpr static const char *const	ATTRIBUTE_NAME_SEPARATOR = "$"
	Schema qualifier separator. More...

Member Enumeration Documentation

◆ MemoryLayoutType

enum NES::Schema::MemoryLayoutType : uint8_t

strong

Enum to identify the memory layout in which we want to represent the schema physically.

Enumerator
ROW_LAYOUT
COLUMNAR_LAYOUT

Constructor & Destructor Documentation

◆ Schema() [1/2]

NES::Schema::Schema ( MemoryLayoutType layoutType = MemoryLayoutType::ROW_LAYOUT )

explicit

◆ Schema() [2/2]

NES::Schema::Schema	(	SchemaPtr const &	query,
		MemoryLayoutType	layoutType = `MemoryLayoutType::ROW_LAYOUT`
	)

References copyFields().

Here is the call graph for this function:

Member Function Documentation

◆ addField() [1/3]

SchemaPtr NES::Schema::addField ( const AttributeFieldPtr & attribute )

appends a AttributeField to the schema and returns a copy of this schema.

Parameters

attribute

Returns: a copy of this schema.

References copy(), and fields.

Referenced by addField().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ addField() [2/3]

SchemaPtr NES::Schema::addField	(	const std::string &	name,
		const BasicType &	type
	)

appends a field with a basic type to the schema and returns a copy of this schema.

Parameters

field

Returns: a copy of this schema.

References addField(), NES::DataTypeFactory::createType(), and type.

Here is the call graph for this function:

◆ addField() [3/3]

SchemaPtr NES::Schema::addField	(	const std::string &	name,
		DataTypePtr	data
	)

appends a field with a data type to the schema and returns a copy of this schema.

Parameters

field

Returns: a copy of this schema.

References addField(), NES::AttributeField::create(), and data.

Here is the call graph for this function:

◆ clear()

void NES::Schema::clear ( )

Remove all fields and qualifying name.

References fields.

◆ contains()

bool NES::Schema::contains ( const std::string & fieldName ) const

Checks if the field exists in the schema.

Parameters

schema

Returns: boolean

References fields, and NES_TRACE.

◆ copy()

SchemaPtr NES::Schema::copy ( ) const

Creates a copy of this schema.

Note: The containing AttributeFields may still reference the same objects.

Returns: A copy of the Schema

Referenced by addField(), copyFields(), and updateSourceName().

Here is the caller graph for this function:

◆ copyFields()

SchemaPtr NES::Schema::copyFields ( const SchemaPtr & otherSchema )

Copy all fields of otherSchema into this schema.

Parameters

otherSchema

Returns: a copy of this schema.

References copy(), and fields.

Referenced by Schema().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ create()

SchemaPtr NES::Schema::create ( MemoryLayoutType layoutType = MemoryLayoutType::ROW_LAYOUT )

static

Factory method to create a new SchemaPtr.

Returns: SchemaPtr

Referenced by NES::createDefaultSourceWithoutSchemaForOneBuffer(), createFromSchemaType(), NES::DistributedNemoJoinIntegrationTest::createTestHarness(), NES::QueryAPITest::SetUp(), NES::FilterPushDownTest::SetUp(), NES::UnionDeploymentTest::SetUp(), GPUQueryExecutionTest::SetUp(), NES::Runtime::Execution::CountMinBuildExecutionTest::SetUp(), NES::Runtime::Execution::HyperLogLogBuildExecutionTest::SetUp(), UnionQueryExecutionTest::SetUp(), WindowAggregationFunctionTest::SetUp(), NES::RequestProcessor::AddQueryRequestTest::SetUp(), NES::SourceTest::SetUp(), NES::TCPSourceTest::SetUp(), NES::DefaultStatisticQueryGeneratorTest::SetUp(), NES::Runtime::MemoryLayouts::DynamicMemoryLayoutTestParameterized::SetUp(), NES::Runtime::MemoryLayouts::TestTupleBufferTest::SetUp(), ColumnLayoutGPUPipelineStage::setup(), NES::StatisticIdsExtractorTest::setUpSourceCatalog(), NES::TEST_F(), NES::Runtime::MemoryLayouts::TEST_F(), TEST_F(), NES::Runtime::TEST_F(), NES::Spatial::TEST_F(), NES::Runtime::Execution::TEST_F(), NES::RequestProcessor::TEST_F(), NES::Runtime::Execution::TEST_P(), NES::TEST_P(), and TEST_P().

Here is the caller graph for this function:

◆ createFromSchemaType()

SchemaPtr NES::Schema::createFromSchemaType	(	const Configurations::SchemaTypePtr &	schemaType,
		Schema::MemoryLayoutType	layoutType = `MemoryLayoutType::ROW_LAYOUT`
	)

static

Factory method to create a new SchemaPtr from schema type.

Returns: SchemaPtr

References create().

Referenced by NES::Monitoring::CpuMetrics::getSchema(), NES::Monitoring::DiskMetrics::getSchema(), NES::Monitoring::MemoryMetrics::getSchema(), NES::Monitoring::NetworkMetrics::getSchema(), NES::Monitoring::RegistrationMetrics::getSchema(), NES::Monitoring::RuntimeMetrics::getSchema(), and NES::NesCoordinator::startCoordinator().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ empty()

bool NES::Schema::empty ( ) const

Is checks if the schema is empty (if it has no fields).

Returns: true if empty

References fields.

◆ equals()

bool NES::Schema::equals	(	const SchemaPtr &	schema,
		bool	considerOrder = `true`
	)

Checks if two Schemas are equal to each other.

Parameters

schema
considerOrder	takes into account if the order of fields in a schema matter.

Returns: boolean

References fields.

◆ get() [1/2]

AttributeFieldPtr NES::Schema::get ( const std::string & fieldName ) const

Finds a attribute field by name in the schema.

Parameters

fieldName

Returns: AttributeField

References fields, and NES_FATAL_ERROR.

◆ get() [2/2]

AttributeFieldPtr NES::Schema::get ( uint32_t index )

Finds a attribute field by index in the schema.

Parameters

index

Returns: AttributeField

References fields, index, and NES_FATAL_ERROR.

◆ getField()

AttributeFieldPtr NES::Schema::getField ( const std::string & fieldName ) const

Returns the attribute field based on a qualified or unqualified field name.

If an unqualified field name is given (e.g., getField("fieldName")), the function will match attribute fields with any source name. If a qualified field name is given (e.g., getField("source$fieldName")), the entire qualified field must match.

Parameters

fieldName Name of the attribute field that should be returned.

Returns: Pointer to attribute field if present, otherwise nullptr.

References ATTRIBUTE_NAME_SEPARATOR, and fields.

◆ getFieldNames()

std::vector< std::string > NES::Schema::getFieldNames ( ) const

Get the field names as a vector of strings.

Returns: std::vector<std::string> fieldNames

References fields.

◆ getIndex()

uint64_t NES::Schema::getIndex ( const std::string & fieldName ) const

Checks if attribute field name is defined in the schema and returns its index. If item not in the list, then the return value is equal to fields.size().

Parameters

fieldName

Returns: the index

References fields, and NES::startsWith().

Here is the call graph for this function:

◆ getLayoutType()

Schema::MemoryLayoutType NES::Schema::getLayoutType ( ) const

method to get the type of the memory layout

Returns: MemoryLayoutType

◆ getLayoutTypeAsString()

std::string NES::Schema::getLayoutTypeAsString ( ) const

returns the string representation of layout

Parameters

layout

Returns

References COLUMNAR_LAYOUT, and ROW_LAYOUT.

◆ getQualifierNameForSystemGeneratedFields()

std::string NES::Schema::getQualifierNameForSystemGeneratedFields ( ) const

method to get the qualifier of the source without $

Returns: qualifier without $

References ATTRIBUTE_NAME_SEPARATOR, fields, magic_enum::detail::find(), and NES_ERROR.

Referenced by getQualifierNameForSystemGeneratedFieldsWithSeparator().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getQualifierNameForSystemGeneratedFieldsWithSeparator()

std::string NES::Schema::getQualifierNameForSystemGeneratedFieldsWithSeparator ( ) const

method to get the qualifier of the source with $

Returns: qualifier with $

References ATTRIBUTE_NAME_SEPARATOR, and getQualifierNameForSystemGeneratedFields().

Here is the call graph for this function:

◆ getSchemaSizeInBytes()

uint64_t NES::Schema::getSchemaSizeInBytes ( ) const

Returns the number of bytes all fields in this schema occupy.

Returns: uint64_t

References fields, size(), and type.

Here is the call graph for this function:

◆ getSize()

uint64_t NES::Schema::getSize ( ) const

Returns the number of fields in the schema.

Returns: uint64_t

References fields.

◆ getSourceNameQualifier()

std::string NES::Schema::getSourceNameQualifier ( ) const

Method to return the source name qualifier, thus everything that is before $.

Returns: string

References ATTRIBUTE_NAME_SEPARATOR, fields, and magic_enum::detail::find().

Here is the call graph for this function:

◆ hasEqualTypes()

bool NES::Schema::hasEqualTypes ( const SchemaPtr & otherSchema )

Checks if two schemas have same datatypes at same index location.

Parameters

otherSchema the other schema to compare agains

Returns: ture if they are equal else false

References fields.

◆ removeField()

void NES::Schema::removeField ( const AttributeFieldPtr & field )

removes a AttributeField from the schema

Parameters

field

References fields.

◆ replaceField()

void NES::Schema::replaceField	(	const std::string &	name,
		const DataTypePtr &	type
	)

Replaces a field, which is already part of the schema.

Parameters

name	of the field we want to replace
DataTypePtr

References NES::AttributeField::create(), fields, and type.

Here is the call graph for this function:

◆ setLayoutType()

void NES::Schema::setLayoutType ( Schema::MemoryLayoutType layoutType )

method to set the memory layout

Parameters

layoutType

◆ toString()

std::string NES::Schema::toString	(	const std::string &	prefix = `""`,
		const std::string &	sep = `" "`,
		const std::string &	suffix = `""`
	)		const

returns a string representation

Parameters

prefix	of the string
delimitor	between each field
suffix,for	the end of the string

Returns: schema as string

References fields.

◆ updateSourceName()

SchemaPtr NES::Schema::updateSourceName ( const std::string & srcName ) const

Prepends the srcName to the substring after the last occurrence of ATTRIBUTE_NAME_SEPARATOR in every field name of the schema.

Parameters

srcName

Returns: SchemaPtr

References ATTRIBUTE_NAME_SEPARATOR, copy(), and fields.

Here is the call graph for this function:

Member Data Documentation

◆ ATTRIBUTE_NAME_SEPARATOR

constexpr static const char* const NES::Schema::ATTRIBUTE_NAME_SEPARATOR = "$"

staticconstexpr

Schema qualifier separator.

Referenced by getField(), getQualifierNameForSystemGeneratedFields(), getQualifierNameForSystemGeneratedFieldsWithSeparator(), getSourceNameQualifier(), and updateSourceName().

◆ fields

std::vector<AttributeFieldPtr> NES::Schema::fields

Referenced by addField(), clear(), contains(), copyFields(), empty(), equals(), get(), getField(), getFieldNames(), getIndex(), getQualifierNameForSystemGeneratedFields(), getSchemaSizeInBytes(), getSize(), getSourceNameQualifier(), hasEqualTypes(), removeField(), replaceField(), toString(), and updateSourceName().

The documentation for this class was generated from the following files:

/nebulastream/nes-data-types/include/API/Schema.hpp
/nebulastream/nes-data-types/src/API/Schema.cpp

Public Types

Public Member Functions

Static Public Member Functions

Public Attributes

Static Public Attributes

Member Enumeration Documentation

◆ MemoryLayoutType

Constructor & Destructor Documentation

◆ Schema() [1/2]

◆ Schema() [2/2]

Member Function Documentation

◆ addField() [1/3]

◆ addField() [2/3]

◆ addField() [3/3]

◆ clear()

◆ contains()

◆ copy()

◆ copyFields()

◆ create()

◆ createFromSchemaType()

◆ empty()

◆ equals()

◆ get() [1/2]

◆ get() [2/2]

◆ getField()

◆ getFieldNames()

◆ getIndex()

◆ getLayoutType()

◆ getLayoutTypeAsString()

◆ getQualifierNameForSystemGeneratedFields()

◆ getQualifierNameForSystemGeneratedFieldsWithSeparator()

◆ getSchemaSizeInBytes()

◆ getSize()

◆ getSourceNameQualifier()

◆ hasEqualTypes()

◆ removeField()

◆ replaceField()

◆ setLayoutType()

◆ toString()

◆ updateSourceName()

Member Data Documentation

◆ ATTRIBUTE_NAME_SEPARATOR

◆ fields