NES::Runtime::Execution Namespace Reference

Classes
class	JoinDeploymentTest
class	MultipleJoinsTest
class	CountMinBuildExecutionTest
class	HyperLogLogBuildExecutionTest
class	StreamIntervalJoinQueryExecutionTest
class	StreamJoinQueryExecutionTest
class	StreamJoinQuerySharedExecutionTest
class	DiscardingExecutablePipelineStage
	The executable pipeline stage represents the executable part of a an specific pipeline. @Usage this pipeline is the empty pipeline that just retuns without doing processing, during execution, if tasks get invalid, we will switch the function pointer to this one For instance, during code generation we generate an implementation of this class, which defines all virtual functions. More...
class	ExecutablePipeline
	An ExecutablePipeline represents a fragment of an overall query. It can contain multiple operators and the implementation of its computation is defined in the ExecutablePipelineStage. Furthermore, it holds the PipelineExecutionContextPtr and a reference to the next pipeline in the query plan. More...
class	ExecutablePipelineStage
	The executable pipeline stage represents the executable part of a an specific pipeline. For instance, during code generation we generate an implementation of this class, which defines all virtual functions. More...
class	ExecutableQueryPlan
	Represents an executable plan of an particular query. Each executable query plan contains a set of sources, pipelines, and sinks. A valid query plan should contain at least one source and sink. This class is thread-safe. More...
class	MigratableStateInterface
	Interface that defines operations to migrate an operator state. More...
class	OperatorHandler
	Interface to handle specific operator state. More...
class	OperatorHandlerSlices
	Interface to handle deleting queries from shared operators that contain slices. More...
struct	SeqNumberOriginId
	Stores a sequenceNumber and an OriginId. More...
struct	SequenceState
	Container for storing information, related to the state of a sequence number. More...
class	PipelineExecutionContext
	The PipelineExecutionContext is passed to a compiled pipeline and offers basic functionality to interact with the Runtime. Via the context, the compile code is able to allocate new TupleBuffers and to emit tuple buffers to the Runtime. More...
class	StreamSliceInterface
	Basic Slice Interface with necessary methods and variables for migration. More...
struct	BufferMetaData
	Stores the meta date for a RecordBuffer. More...

Typedefs
using	OperatorHandlerPtr = std::shared_ptr< OperatorHandler >
using	ExecutablePipelineStagePtr = std::shared_ptr< ExecutablePipelineStage >
using	ExecutablePipelinePtr = std::shared_ptr< ExecutablePipeline >
using	ExecutableQueryPlanPtr = std::shared_ptr< ExecutableQueryPlan >
using	SuccessorExecutablePipeline = std::variant< DataSinkPtr, ExecutablePipelinePtr >
using	PredecessorExecutablePipeline = std::variant< std::weak_ptr< DataSource >, std::weak_ptr< ExecutablePipeline > >
using	PipelineExecutionContextPtr = std::shared_ptr< PipelineExecutionContext >

Enumerations
enum class	ExecutableQueryPlanResult : uint8_t { Ok , Fail }
enum class	ExecutableQueryPlanStatus : uint8_t {   Created , Deployed , Running , Finished ,   Stopped , ErrorState , Invalid }
	Represents the query status. More...
enum class	SharedJoinApproach : uint8_t { UNSHARED , APPROACH_PROBING , APPROACH_DELETING , APPROACH_TOMBSTONE }
enum class	OperatorHandlerType : uint8_t {   WINDOW , CEP , JOIN , BATCH_JOIN ,   KEY_EVENT_TIME_WINDOW }

Functions
	TEST_P (JoinDeploymentTest, testJoinWithSameSchemaTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithSameSchemaMultipleConditionsGeneralTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithDifferentSchemaNamesButSameInputTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithDifferentSchemaNamesMultipleConditionsButSameInputTumblingWindow)
	TEST_P (JoinDeploymentTest, testConstantValueDifferentSchemaNamesButSameInputTumblingWindow)
	TEST_P (JoinDeploymentTest, testTwoConstantValueDifferentSchemaNamesButSameInputTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithDifferentSourceTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithDifferentNumberOfAttributesTumblingWindow)
	TEST_P (JoinDeploymentTest, testJoinWithDifferentSourceSlidingWindow)
	TEST_P (JoinDeploymentTest, testSlidingWindowDifferentAttributes)
	TEST_P (JoinDeploymentTest, testJoinWithVarSizedData)
	Test a join query that uses fixed-array as keys. More...
	TEST_P (JoinDeploymentTest, joinResultLargerThanSingleTupleBuffer)
	TEST_P (JoinDeploymentTest, testTumblingWindowCrossJoin)
	INSTANTIATE_TEST_CASE_P (testJoinQueries, JoinDeploymentTest, JOIN_STRATEGIES_WINDOW_STRATEGIES, [](const testing::TestParamInfo< JoinDeploymentTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));})
	TEST_P (MultipleJoinsTest, testJoins2WithDifferentSourceTumblingWindowOnCoodinator)
	TEST_P (MultipleJoinsTest, testJoin3WithDifferentSourceTumblingWindowOnCoodinatorSequential)
	TEST_P (MultipleJoinsTest, testJoin3WithDifferentSourceTumblingWindowOnCoodinatorNested)
	TEST_P (MultipleJoinsTest, testJoins2WithDifferentSourceSlidingWindowOnCoodinator)
	TEST_P (MultipleJoinsTest, testJoin3WithDifferentSourceSlidingWindowOnCoodinatorSequential)
	TEST_P (MultipleJoinsTest, testJoin3WithDifferentSourceSlidingWindowOnCoodinatorNested)
	TEST_P (MultipleJoinsTest, testMapNotKeyWithDifferentSourceTumblingWindow)
	TEST_P (MultipleJoinsTest, testMapsAsKeysWithDifferentSourceTumblingWindow)
	TEST_P (MultipleJoinsTest, testMapsThreeSameKeynamesTumblingWindow)
	TEST_P (MultipleJoinsTest, testMapNotKeyWithDifferentSourceSlidingWindow)
	TEST_P (MultipleJoinsTest, testMapAsKeyWithDifferentSourceSlidingWindow)
	INSTANTIATE_TEST_CASE_P (testJoinQueries, MultipleJoinsTest, JOIN_STRATEGIES_WINDOW_STRATEGIES, [](const testing::TestParamInfo< MultipleJoinsTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));})
	TEST_P (CountMinBuildExecutionTest, singleInputTuple)
	Here we test, if we create a count min sketch for a single input tuple. More...
	TEST_P (CountMinBuildExecutionTest, multipleInputBuffers)
	Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches. More...
	TEST_P (CountMinBuildExecutionTest, multipleInputBuffersSlidingWindow)
	Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches and for sliding windows (we create one sketch per slice) More...
	TEST_P (CountMinBuildExecutionTest, multipleInputBuffersIngestionTime)
	Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches. The difference is that we use the ingestion time instead of an event time. More...
	INSTANTIATE_TEST_CASE_P (testCountMin, CountMinBuildExecutionTest, ::testing::Combine(::testing::Values(1, 4, 8), ::testing::Values(1, 100, 4000), ::testing::Values(1, 5, 10), ::testing::ValuesIn(magic_enum::enum_values< Statistic::StatisticDataCodec >())), [](const testing::TestParamInfo< CountMinBuildExecutionTest::ParamType > &info) { const auto numWorkerThread=std::get< 0 >(info.param);const auto sketchWidth=std::get< 1 >(info.param);const auto sketchDepth=std::get< 2 >(info.param);const auto dataCodec=std::get< 3 >(info.param);return std::to_string(numWorkerThread)+"Threads_"+std::to_string(sketchWidth)+"Width_"+std::to_string(sketchDepth)+"Depth_"+std::string(magic_enum::enum_name(dataCodec));})
	TEST_P (HyperLogLogBuildExecutionTest, singleInputTuple)
	Here we test, if we create a hyperloglog sketch for a single input tuple. More...
	TEST_P (HyperLogLogBuildExecutionTest, multipleInputBuffers)
	Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches. More...
	TEST_P (HyperLogLogBuildExecutionTest, multipleInputBuffersSlidingWindow)
	Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches and for sliding windows (we create one sketch per slice) More...
	TEST_P (HyperLogLogBuildExecutionTest, multipleInputBuffersIngestionTime)
	Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches The difference is that we use the ingestion time instead of an event time. More...
	INSTANTIATE_TEST_CASE_P (testHyperLogLog, HyperLogLogBuildExecutionTest, ::testing::Combine(::testing::Values(1, 4, 8), ::testing::Values(4, 5, 8, 10, 20), ::testing::ValuesIn(magic_enum::enum_values< Statistic::StatisticDataCodec >())), [](const testing::TestParamInfo< HyperLogLogBuildExecutionTest::ParamType > &info) { const auto numWorkerThread=std::get< 0 >(info.param);const auto sketchWidth=std::get< 1 >(info.param);const auto dataCodec=std::get< 2 >(info.param);return std::to_string(numWorkerThread)+"Threads_"+std::to_string(sketchWidth)+"Width"+std::string(magic_enum::enum_name(dataCodec));})
	TEST_F (StreamIntervalJoinQueryExecutionTest, streamIntervalJoinExecutionTestOne)
	TEST_F (StreamIntervalJoinQueryExecutionTest, streamIntervalJoinExecutionTestTwo)
	TEST_F (StreamIntervalJoinQueryExecutionTest, streamIntervalJoinExecutionTestThree)
	TEST_F (StreamIntervalJoinQueryExecutionTest, streamIntervalJoinExecutionTestFour)
	TEST_F (StreamIntervalJoinQueryExecutionTest, streamIntervalJoinExecutionTestFive)
	TEST_P (StreamJoinQueryExecutionTest, streamJoinExecutionTestCsvFiles)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithSameSchemaTumblingWindow)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithDifferentSchemaNamesButSameInputTumblingWindow)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithDifferentSourceTumblingWindow)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithDifferentNumberOfAttributesTumblingWindow)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithDifferentSourceSlidingWindow)
	TEST_P (StreamJoinQueryExecutionTest, testJoinWithLargerWindowSizes)
	TEST_P (StreamJoinQueryExecutionTest, testSlidingWindowDifferentAttributes)
	TEST_P (StreamJoinQueryExecutionTest, DISABLED_testJoinWithFixedCharKey)
	Test a join query that uses fixed-array as keys. More...
	TEST_P (StreamJoinQueryExecutionTest, streamJoinExecutiontTestWithSlidingWindows)
	TEST_P (StreamJoinQueryExecutionTest, streamJoinExecutiontTestWithWindows)
	INSTANTIATE_TEST_CASE_P (testStreamJoinQueries, StreamJoinQueryExecutionTest, JOIN_STRATEGIES_WINDOW_STRATEGIES, [](const testing::TestParamInfo< StreamJoinQueryExecutionTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));})
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryDifferentDeploymentTime)
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeSlidingWindow)
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryThreeDifferentDeploymentTimes)
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeAfterFirstWindowTriggered)
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeAndUndeploySecondQuery)
	TEST_P (StreamJoinQuerySharedExecutionTest, streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeLargeSlidingWindow)
	INSTANTIATE_TEST_CASE_P (testStreamJoinQueries, StreamJoinQuerySharedExecutionTest, JOIN_SHARING_CONFIGURATIONS, [](const testing::TestParamInfo< StreamJoinQuerySharedExecutionTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));})

Variables
constexpr auto	queryCompilerDumpMode = NES::QueryCompilation::DumpMode::NONE

Detailed Description

This class tests that joins when they are shared by multiple queries, which is the master thesis project right now (e.g. task #5113) In the end it would be the best if both NLJ and HJ configurations are tested with all sharedJoinApproaches in one parameterized test. Bucketing will most likely never be supported

Typedef Documentation

ExecutablePipelinePtr

typedef std::shared_ptr< ExecutablePipeline > NES::Runtime::Execution::ExecutablePipelinePtr

ExecutablePipelineStagePtr

typedef std::shared_ptr< ExecutablePipelineStage > NES::Runtime::Execution::ExecutablePipelineStagePtr

ExecutableQueryPlanPtr

typedef std::shared_ptr< ExecutableQueryPlan > NES::Runtime::Execution::ExecutableQueryPlanPtr

OperatorHandlerPtr

typedef std::shared_ptr< OperatorHandler > NES::Runtime::Execution::OperatorHandlerPtr

PipelineExecutionContextPtr

using NES::Runtime::Execution::PipelineExecutionContextPtr = typedef std::shared_ptr<PipelineExecutionContext>

PredecessorExecutablePipeline

using NES::Runtime::Execution::PredecessorExecutablePipeline = typedef std::variant<std::weak_ptr<DataSource>, std::weak_ptr<ExecutablePipeline> >

SuccessorExecutablePipeline

using NES::Runtime::Execution::SuccessorExecutablePipeline = typedef std::variant<DataSinkPtr, ExecutablePipelinePtr>

Enumeration Type Documentation

ExecutableQueryPlanResult

enum NES::Runtime::Execution::ExecutableQueryPlanResult : uint8_t

strong

Enumerator
Ok	query was completed successfully
Fail	query failed

ExecutableQueryPlanStatus

enum NES::Runtime::Execution::ExecutableQueryPlanStatus : uint8_t

strong

Represents the query status.

Enumerator
Created
Deployed
Running
Finished
Stopped
ErrorState
Invalid

OperatorHandlerType

enum NES::Runtime::Execution::OperatorHandlerType : uint8_t

strong

Enumerator
WINDOW
CEP
JOIN
BATCH_JOIN
KEY_EVENT_TIME_WINDOW

SharedJoinApproach

enum NES::Runtime::Execution::SharedJoinApproach : uint8_t

strong

StreamJoinOperatorHandler (and others) will temporarily contain code that handles sharing with different approaches to test for a master thesis. Task #5113 describes different approaches. While under testing I will try to keep the code clean by naming methods according to their approach and storing the helper enum class to know which approach is being tested. However some parts will need to be commented out and adjusted to test each approach individually

Enumerator
UNSHARED
APPROACH_PROBING
APPROACH_DELETING
APPROACH_TOMBSTONE

Function Documentation

INSTANTIATE_TEST_CASE_P() [1/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testCountMin	,
		CountMinBuildExecutionTest	,
		::testing::Combine(::testing::Values(1, 4, 8), ::testing::Values(1, 100, 4000), ::testing::Values(1, 5, 10), ::testing::ValuesIn(magic_enum::enum_values< Statistic::StatisticDataCodec >()))	,
		[] (const testing::TestParamInfo< CountMinBuildExecutionTest::ParamType > &info) { const auto numWorkerThread=std::get< 0 >(info.param);const auto sketchWidth=std::get< 1 >(info.param);const auto sketchDepth=std::get< 2 >(info.param);const auto dataCodec=std::get< 3 >(info.param);return std::to_string(numWorkerThread)+"Threads_"+std::to_string(sketchWidth)+"Width_"+std::to_string(sketchDepth)+"Depth_"+std::string(magic_enum::enum_name(dataCodec));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

INSTANTIATE_TEST_CASE_P() [2/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testHyperLogLog	,
		HyperLogLogBuildExecutionTest	,
		::testing::Combine(::testing::Values(1, 4, 8), ::testing::Values(4, 5, 8, 10, 20), ::testing::ValuesIn(magic_enum::enum_values< Statistic::StatisticDataCodec >()))	,
		[] (const testing::TestParamInfo< HyperLogLogBuildExecutionTest::ParamType > &info) { const auto numWorkerThread=std::get< 0 >(info.param);const auto sketchWidth=std::get< 1 >(info.param);const auto dataCodec=std::get< 2 >(info.param);return std::to_string(numWorkerThread)+"Threads_"+std::to_string(sketchWidth)+"Width"+std::string(magic_enum::enum_name(dataCodec));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

INSTANTIATE_TEST_CASE_P() [3/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testJoinQueries	,
		JoinDeploymentTest	,
		JOIN_STRATEGIES_WINDOW_STRATEGIES	,
		[] (const testing::TestParamInfo< JoinDeploymentTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

INSTANTIATE_TEST_CASE_P() [4/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testJoinQueries	,
		MultipleJoinsTest	,
		JOIN_STRATEGIES_WINDOW_STRATEGIES	,
		[] (const testing::TestParamInfo< MultipleJoinsTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

INSTANTIATE_TEST_CASE_P() [5/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testStreamJoinQueries	,
		StreamJoinQueryExecutionTest	,
		JOIN_STRATEGIES_WINDOW_STRATEGIES	,
		[] (const testing::TestParamInfo< StreamJoinQueryExecutionTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

INSTANTIATE_TEST_CASE_P() [6/6]

NES::Runtime::Execution::INSTANTIATE_TEST_CASE_P	(	testStreamJoinQueries	,
		StreamJoinQuerySharedExecutionTest	,
		JOIN_SHARING_CONFIGURATIONS	,
		[] (const testing::TestParamInfo< StreamJoinQuerySharedExecutionTest::ParamType > &info) { return std::string(magic_enum::enum_name(std::get< 0 >(info.param)))+"_"+std::string(magic_enum::enum_name(std::get< 1 >(info.param)));}
	)

References magic_enum::enum_name().

Here is the call graph for this function:

TEST_F() [1/5]

NES::Runtime::Execution::TEST_F	(	StreamIntervalJoinQueryExecutionTest	,
		streamIntervalJoinExecutionTestFive
	)

This test creates an IVJoin with an interval including exactly 1 timestamp (lB = uB = 0 = leftstream.timestamp)

References NES::__attribute__(), NES::Schema::create(), NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_F() [2/5]

NES::Runtime::Execution::TEST_F	(	StreamIntervalJoinQueryExecutionTest	,
		streamIntervalJoinExecutionTestFour
	)

This test creates an IVJoin with bounds into the past and future and test different schemas

References NES::__attribute__(), NES::Schema::create(), NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_F() [3/5]

NES::Runtime::Execution::TEST_F	(	StreamIntervalJoinQueryExecutionTest	,
		streamIntervalJoinExecutionTestOne
	)

This test checks a simple IntervalJoin version, where intervals are only created in the future (after occurring of a left tuple) (lowerB and upperB >= 0) and the left tuple arrives before the right tuples belonging to the interval

References NES::__attribute__(), NES::Schema::create(), NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_F() [4/5]

NES::Runtime::Execution::TEST_F	(	StreamIntervalJoinQueryExecutionTest	,
		streamIntervalJoinExecutionTestThree
	)

This test creates an IVJoin that only creates intervals for the past (lowerB and upperB < 0)

References NES::__attribute__(), NES::Schema::create(), NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_F() [5/5]

NES::Runtime::Execution::TEST_F	(	StreamIntervalJoinQueryExecutionTest	,
		streamIntervalJoinExecutionTestTwo
	)

This test manipulates the input buffer size to tests if the IVJoin handles right tuples occurring before the left tuples, i.e., storing and adding them to the interval when the left tuple arrives after the right. (lowerB and upperB >= 0)

References NES::__attribute__(), NES::bufferSize, NES::Schema::create(), NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_P() [1/49]

NES::Runtime::Execution::TEST_P	(	CountMinBuildExecutionTest	,
		multipleInputBuffers
	)

Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches.

TEST_P() [2/49]

NES::Runtime::Execution::TEST_P	(	CountMinBuildExecutionTest	,
		multipleInputBuffersIngestionTime
	)

Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches. The difference is that we use the ingestion time instead of an event time.

TEST_P() [3/49]

NES::Runtime::Execution::TEST_P	(	CountMinBuildExecutionTest	,
		multipleInputBuffersSlidingWindow
	)

Here we test, if we create multiple count min sketches for multiple input buffers, but also for larger sketches and for sliding windows (we create one sketch per slice)

TEST_P() [4/49]

NES::Runtime::Execution::TEST_P	(	CountMinBuildExecutionTest	,
		singleInputTuple
	)

Here we test, if we create a count min sketch for a single input tuple.

TEST_P() [5/49]

NES::Runtime::Execution::TEST_P	(	HyperLogLogBuildExecutionTest	,
		multipleInputBuffers
	)

Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches.

TEST_P() [6/49]

NES::Runtime::Execution::TEST_P	(	HyperLogLogBuildExecutionTest	,
		multipleInputBuffersIngestionTime
	)

Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches The difference is that we use the ingestion time instead of an event time.

TEST_P() [7/49]

NES::Runtime::Execution::TEST_P	(	HyperLogLogBuildExecutionTest	,
		multipleInputBuffersSlidingWindow
	)

Here we test, if we create multiple hyperloglog sketches for multiple input buffers, but also for larger sketches and for sliding windows (we create one sketch per slice)

TEST_P() [8/49]

NES::Runtime::Execution::TEST_P	(	HyperLogLogBuildExecutionTest	,
		singleInputTuple
	)

Here we test, if we create a hyperloglog sketch for a single input tuple.

TEST_P() [9/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		joinResultLargerThanSingleTupleBuffer
	)

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [10/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testConstantValueDifferentSchemaNamesButSameInputTumblingWindow
	)

Test deploying constant value expression in join

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [11/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithDifferentNumberOfAttributesTumblingWindow
	)

Test deploying join with different sources

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [12/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithDifferentSchemaNamesButSameInputTumblingWindow
	)

Test deploying join with same data but different names in the schema

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [13/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithDifferentSchemaNamesMultipleConditionsButSameInputTumblingWindow
	)

Test deploying join with same data but different names in the schema and multiple join conditions

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [14/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithDifferentSourceSlidingWindow
	)

Test deploying join with different sources

References NES::TestSchemas::getSchemaTemplate(), and NES::QueryCompilation::HASH_JOIN_VAR_SIZED.

Here is the call graph for this function:

TEST_P() [15/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithDifferentSourceTumblingWindow
	)

Test deploying join with different sources

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [16/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithSameSchemaMultipleConditionsGeneralTumblingWindow
	)

Test deploying join with same data and same schema and more than one join condition This test is very simple and checks only if multiple conditions are working.

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [17/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithSameSchemaTumblingWindow
	)

Test deploying join with same data and same schema

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [18/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testJoinWithVarSizedData
	)

Test a join query that uses fixed-array as keys.

References NES::Attribute(), NES::Schema::create(), NES::DataTypeFactory::createText(), NES::API::EventTime(), NES::Query::from(), NES::QueryCompilation::HASH_JOIN_GLOBAL_LOCK_FREE, NES::QueryCompilation::HASH_JOIN_GLOBAL_LOCKING, NES::QueryCompilation::HASH_JOIN_LOCAL, NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::Schema::ROW_LAYOUT, NES::UINT64, NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [19/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testSlidingWindowDifferentAttributes
	)

Test deploying join with different sources

References NES::TestSchemas::getSchemaTemplate(), and NES::QueryCompilation::HASH_JOIN_VAR_SIZED.

Here is the call graph for this function:

TEST_P() [20/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testTumblingWindowCrossJoin
	)

Test deploying cross-join with different sources

References NES::Attribute(), NES::Query::crossJoinWith(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_GLOBAL_LOCK_FREE, NES::QueryCompilation::HASH_JOIN_GLOBAL_LOCKING, NES::QueryCompilation::HASH_JOIN_LOCAL, NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::API::Milliseconds(), and NES::CrossJoinOperatorBuilder::CrossJoin::window().

Here is the call graph for this function:

TEST_P() [21/49]

NES::Runtime::Execution::TEST_P	(	JoinDeploymentTest	,
		testTwoConstantValueDifferentSchemaNamesButSameInputTumblingWindow
	)

Test deploying two constant value expressions in join

References NES::Attribute(), NES::API::EventTime(), NES::Query::from(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::Query::joinWith(), NES::API::Milliseconds(), NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [22/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoin3WithDifferentSourceSlidingWindowOnCoodinatorNested
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, NES::API::Milliseconds(), and NES::API::Seconds().

Here is the call graph for this function:

TEST_P() [23/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoin3WithDifferentSourceSlidingWindowOnCoodinatorSequential
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, NES::API::Milliseconds(), and NES::API::Seconds().

Here is the call graph for this function:

TEST_P() [24/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoin3WithDifferentSourceTumblingWindowOnCoodinatorNested
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [25/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoin3WithDifferentSourceTumblingWindowOnCoodinatorSequential
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [26/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoins2WithDifferentSourceSlidingWindowOnCoodinator
	)

Sliding window joins

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, NES::API::Milliseconds(), and NES::API::Seconds().

Here is the call graph for this function:

TEST_P() [27/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testJoins2WithDifferentSourceTumblingWindowOnCoodinator
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [28/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testMapAsKeyWithDifferentSourceSlidingWindow
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [29/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testMapNotKeyWithDifferentSourceSlidingWindow
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [30/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testMapNotKeyWithDifferentSourceTumblingWindow
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [31/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testMapsAsKeysWithDifferentSourceTumblingWindow
	)

References NES::Attribute(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [32/49]

NES::Runtime::Execution::TEST_P	(	MultipleJoinsTest	,
		testMapsThreeSameKeynamesTumblingWindow
	)

References NES::Attribute(), NES::createField(), NES::API::EventTime(), NES::expectedOutput, and NES::API::Milliseconds().

Here is the call graph for this function:

TEST_P() [33/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		DISABLED_testJoinWithFixedCharKey
	)

Test a join query that uses fixed-array as keys.

References NES::__attribute__(), NES::Schema::create(), NES::DataTypeFactory::createText(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_P() [34/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		streamJoinExecutionTestCsvFiles
	)

References NES::__attribute__(), NES::Schema::create(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::operator==(), NES::Schema::ROW_LAYOUT, and NES::UINT64.

Here is the call graph for this function:

TEST_P() [35/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		streamJoinExecutiontTestWithSlidingWindows
	)

References NES::__attribute__(), NES::Attribute(), NES::API::Count(), NES::Schema::create(), NES::TestUtils::createExpectedBuffersFromCsv(), NES::API::EventTime(), NES::TestUtils::CsvFileParams::expectedFile, NES::TestQuery::from(), NES::TestSchemas::getSchemaTemplate(), NES::Runtime::Graceful, NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::TestUtils::CsvFileParams::inputCsvFiles, NES::INT64, NES::Query::joinWith(), NES::API::Milliseconds(), NES_INFO, NES::operator==(), NES::Schema::ROW_LAYOUT, NES::API::Seconds(), NES::Query::sink(), NES::UINT64, NES::JoinOperatorBuilder::Join::where(), and NES::JoinOperatorBuilder::JoinWhere::window().

Here is the call graph for this function:

TEST_P() [36/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		streamJoinExecutiontTestWithWindows
	)

References NES::__attribute__(), NES::Schema::create(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, NES::INT64, NES::operator==(), and NES::Schema::ROW_LAYOUT.

Here is the call graph for this function:

TEST_P() [37/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithDifferentNumberOfAttributesTumblingWindow
	)

Test deploying join with different sources

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [38/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithDifferentSchemaNamesButSameInputTumblingWindow
	)

Test deploying join with same data but different names in the schema

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [39/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithDifferentSourceSlidingWindow
	)

Test deploying join with different sources

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [40/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithDifferentSourceTumblingWindow
	)

Test deploying join with different sources

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [41/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithLargerWindowSizes
	)

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [42/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testJoinWithSameSchemaTumblingWindow
	)

Test deploying join with same data and same schema

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [43/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQueryExecutionTest	,
		testSlidingWindowDifferentAttributes
	)

Test deploying join with different sources

References NES::__attribute__(), NES::TestSchemas::getSchemaTemplate(), NES::QueryCompilation::HASH_JOIN_VAR_SIZED, and NES::operator==().

Here is the call graph for this function:

TEST_P() [44/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryDifferentDeploymentTime
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

TEST_P() [45/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeAfterFirstWindowTriggered
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

TEST_P() [46/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeAndUndeploySecondQuery
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

TEST_P() [47/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeLargeSlidingWindow
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

TEST_P() [48/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryDifferentDeploymentTimeSlidingWindow
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

TEST_P() [49/49]

NES::Runtime::Execution::TEST_P	(	StreamJoinQuerySharedExecutionTest	,
		streamJoinSharedExecutionTestSameQueryThreeDifferentDeploymentTimes
	)

References APPROACH_DELETING, APPROACH_PROBING, and NES::QueryCompilation::NESTED_LOOP_JOIN.

Variable Documentation

queryCompilerDumpMode

constexpr auto NES::Runtime::Execution::queryCompilerDumpMode = NES::QueryCompilation::DumpMode::NONE

constexpr

Referenced by NES::Runtime::Execution::CountMinBuildExecutionTest::SetUp(), NES::Runtime::Execution::HyperLogLogBuildExecutionTest::SetUp(), NES::Runtime::Execution::StreamIntervalJoinQueryExecutionTest::SetUp(), NES::Runtime::Execution::StreamJoinQueryExecutionTest::SetUp(), and NES::Runtime::Execution::StreamJoinQuerySharedExecutionTest::SetUp().