NES::Experimental Namespace Reference

create a table source More...

Namespaces
	BatchJoinOperatorBuilder
	BatchJoinOperatorBuilder.

Classes
class	StaticDataSource
	Table Source todo Still under development. More...

Typedefs
using	StaticDataSourcePtr = std::shared_ptr< StaticDataSource >

Functions
std::pair< std::set< OperatorId >, std::set< OperatorId > >	findUpstreamAndDownstreamPinnedOperators (const SharedQueryPlanPtr &sharedQueryPlan, Optimizer::ExecutionNodeWLock lockedUpstreamNode, Optimizer::ExecutionNodeWLock lockedDownstreamNode, const TopologyPtr &topology)
	identifies the upstream and downstream operators that can remain deployed on the same nodes as they were before the topology change occurred. These sets can be used as input for an incremental placement. All operators placed upstream of the set of upstream operators or downstream of the downstream operators can remain as is and will not need to be re-placed More...
std::vector< std::pair< LogicalOperatorPtr, LogicalOperatorPtr > >	findNetworkOperatorsForLink (const SharedQueryId &sharedQueryPlanId, Optimizer::ExecutionNodeWLock lockedUpstreamNode, Optimizer::ExecutionNodeWLock lockedDownstreamNode)
	find all pairs of network sinks and sources that connect the specified up- and downstream node and belong to the specified shared query More...
DataSourcePtr	createStaticDataSource (const SchemaPtr &schema, const std::string pathTableFile, const bool lateStart, const Runtime::BufferManagerPtr &bufferManager, const Runtime::QueryManagerPtr &queryManager, OperatorId operatorId, OriginId originId, StatisticId statisticId, size_t numSourceLocalBuffers, const std::string &physicalSourceName, const std::vector< Runtime::Execution::SuccessorExecutablePipeline > &successors)

Detailed Description

create a table source

Parameters

schema	the schema of the source
pathTableFile
bufferManager	pointer to the buffer manager
queryManager	pointer to the query manager
operatorId	current operator id
originId	represents the identifier of the upstream operator that represents the origin of the input stream
statisticId	represents the unique identifier of components that we can track statistics for
numSourceLocalBuffers	the number of buffers allocated to a source
physicalSourceName	the name and unique identifier of a physical source
successors	the subsequent operators in the pipeline to which the data is pushed

Returns

a data source pointer

Typedef Documentation

StaticDataSourcePtr

using NES::Experimental::StaticDataSourcePtr = typedef std::shared_ptr<StaticDataSource>

Function Documentation

createStaticDataSource()

DataSourcePtr NES::Experimental::createStaticDataSource	(	const SchemaPtr &	schema,
		const std::string	pathTableFile,
		const bool	lateStart,
		const Runtime::BufferManagerPtr &	bufferManager,
		const Runtime::QueryManagerPtr &	queryManager,
		OperatorId	operatorId,
		OriginId	originId,
		StatisticId	statisticId,
		size_t	numSourceLocalBuffers,
		const std::string &	physicalSourceName,
		const std::vector< Runtime::Execution::SuccessorExecutablePipeline > &	successors
	)

References NES::TestUtils::physicalSourceName().

Here is the call graph for this function:

findNetworkOperatorsForLink()

std::vector< std::pair< LogicalOperatorPtr, LogicalOperatorPtr > > NES::Experimental::findNetworkOperatorsForLink	(	const SharedQueryId &	sharedQueryPlanId,
		Optimizer::ExecutionNodeWLock	lockedUpstreamNode,
		Optimizer::ExecutionNodeWLock	lockedDownstreamNode
	)

find all pairs of network sinks and sources that connect the specified up- and downstream node and belong to the specified shared query

Parameters

sharedQueryPlanId	the id of the shared query whose decomposed query plan are considered
lockedUpstreamNode	the node hosting the network sinks
lockedDownstreamNode	the node hosting the network sources

Returns

a vector source-sink-pairs in the format {SinkOperator, SourceOperator}

Referenced by findUpstreamAndDownstreamPinnedOperators(), and NES::RequestProcessor::TEST_F().

Here is the caller graph for this function:

findUpstreamAndDownstreamPinnedOperators()

std::pair< std::set< OperatorId >, std::set< OperatorId > > NES::Experimental::findUpstreamAndDownstreamPinnedOperators	(	const SharedQueryPlanPtr &	sharedQueryPlan,
		Optimizer::ExecutionNodeWLock	lockedUpstreamNode,
		Optimizer::ExecutionNodeWLock	lockedDownstreamNode,
		const TopologyPtr &	topology
	)

identifies the upstream and downstream operators that can remain deployed on the same nodes as they were before the topology change occurred. These sets can be used as input for an incremental placement. All operators placed upstream of the set of upstream operators or downstream of the downstream operators can remain as is and will not need to be re-placed

Parameters

sharedQueryPlan	the shared query id for which the up and downstream pinned operators need to be identified
lockedUpstreamNode	the upstream node of the topology link that was removed
lockedDownstreamNode	the downstream node of the topology link that was removed
topology	a pointer to the topology

Returns

a pair constaining two sets of operator id in the order {UPSTREAM, DOWNSTREAM}

References NES::Optimizer::DOWNSTREAM_LOGICAL_OPERATOR_ID, findNetworkOperatorsForLink(), NES::Optimizer::PINNED_WORKER_ID, and NES::Optimizer::UPSTREAM_LOGICAL_OPERATOR_ID.

Referenced by NES::RequestProcessor::TEST_F().

Here is the call graph for this function:

Here is the caller graph for this function: