NES::Network Namespace Reference

Namespaces
	detail
	Messages

Classes
class	NetworkStackIntegrationTest
class	TestSink
class	NetworkStackTest
class	DummyExchangeProtocolListener
class	ChannelId
class	ExchangeProtocol
	This class is used by the ZmqServer and defines the reaction for events onDataBuffer, clientAnnouncement, endOfStream and exceptionHandling between all nodes of NES. More...
class	ExchangeProtocolListener
	Listener for network stack events. More...
class	NetworkChannel
	This class represent the network channel for data transfer that NES uses to send data among nodes. This class can send data and event packets. This class is not thread-safe. More...
class	EventOnlyNetworkChannel
	This class represent the network channel for event transfer that NES uses to send events among nodes. This class can send only event packets. This class is not thread-safe. More...
class	NetworkManager
	The NetworkManager manages creation and deletion of subpartition producer and consumer. More...
class	NetworkSink
	This represent a sink operator that acts as a connecting API between query processing and network stack. More...
class	NetworkSource
	this class provide a zmq as data source More...
class	PartitionManager
	this class keeps track of all ready partitions (and their subpartitions) It keeps track of the ref cnt for each partition and associated data emitter A data emitter is notified once there is data for its partition. Pinning a partition for a Consumer/Producer means increasing its reference counter to n: Consumer: there are n "listeners" that are to consume the data of a partition Producer: there are n "emitters" that are to produce the data for a partition When the reference counter reaches 0, it means that none requires a partition. More...
class	ZmqServer
	ZMQ server on hostname:port with numNetworkThreads i/o threads and a set of callbacks in exchangeProtocol. This class is not copyable. More...
struct	VersionUpdate

Typedefs
using	PartitionManagerPtr = std::shared_ptr< PartitionManager >
using	NetworkSourcePtr = std::shared_ptr< NetworkSource >

Enumerations
enum class	PartitionRegistrationStatus : uint8_t { Registered , Deleted , NotFound }
	This enum represent the life-cycle of a nes partition in the partition manager: NotFound: partition was never registered in the current partition manager. Deleted: partition was unregistered at some point in time (we keep it for bookkeeping). Registered: partition was registered and alive (meaning its ref cnt is greater than 0). More...

Functions
void	fillBuffer (TupleBuffer &buf, const Runtime::MemoryLayouts::RowLayoutPtr &memoryLayout)
template<typename MockedNodeEngine , typename... ExtraParameters>
std::shared_ptr< MockedNodeEngine >	createMockedEngine (const std::string &hostname, uint16_t port, uint64_t bufferSize, uint64_t numBuffers, ExtraParameters &&... extraParams)
	TEST_P (NetworkStackIntegrationTest, testNetworkSourceSink)
	TEST_F (NetworkStackIntegrationTest, testQEPNetworkSinkSource)
	TEST_F (NetworkStackIntegrationTest, testSendEvent)
	TEST_F (NetworkStackIntegrationTest, DISABLED_testSendEventBackward)
	INSTANTIATE_TEST_CASE_P (NetworkStackIntegrationTestParam, NetworkStackIntegrationTest, ::testing::Values(false, true))
	TEST_F (NetworkStackTest, serverMustStartAndStop)
	TEST_F (NetworkStackTest, serverMustStartAndStopRandomPort)
	TEST_F (NetworkStackTest, dispatcherMustStartAndStop)
	TEST_F (NetworkStackTest, startCloseChannel)
	TEST_F (NetworkStackTest, startCloseChannelAsyncIndefiniteRetries)
	TEST_F (NetworkStackTest, startCloseMaxChannel)
	TEST_F (NetworkStackTest, testSendData)
	TEST_F (NetworkStackTest, testCorrectHandlingEOS)
	TEST_F (NetworkStackTest, testMassiveSending)
	TEST_F (NetworkStackTest, testMassiveSendingWithChildrenBuffer)
	TEST_F (NetworkStackTest, testHandleUnregisteredBuffer)
	TEST_F (NetworkStackTest, testMassiveMultiSending)
	FORWARD_DECLARE_CLASS_UP (ZmqServer)
	FORWARD_DECLARE_CLASS_UP (NetworkChannel)
	FORWARD_DECLARE_CLASS_UP (EventOnlyNetworkChannel)
	FORWARD_DECLARE_CLASS_SP (NetworkSource)
	FORWARD_DECLARE_CLASS_SP (NetworkSink)
	FORWARD_DECLARE_CLASS_SP (NetworkManager)
	FORWARD_DECLARE_CLASS_SP (PartitionManager)
template<typename MessageType , decltype(kZmqSendDefault) flags = kZmqSendDefault, typename... Arguments>
void	sendMessage (zmq::socket_t &zmqSocket, Arguments &&... args)
template<typename MessageType , decltype(kZmqSendDefault) flags = kZmqSendDefault, typename... Arguments>
void	sendMessageNoHeader (zmq::socket_t &zmqSocket, Arguments &&... args)
template<typename MessageType , typename... Arguments>
void	sendMessageWithIdentity (zmq::socket_t &zmqSocket, zmq::message_t &zmqIdentity, Arguments &&... args)

Typedef Documentation

NetworkSourcePtr

using NES::Network::NetworkSourcePtr = typedef std::shared_ptr<NetworkSource>

PartitionManagerPtr

using NES::Network::PartitionManagerPtr = typedef std::shared_ptr<PartitionManager>

Enumeration Type Documentation

PartitionRegistrationStatus

enum NES::Network::PartitionRegistrationStatus : uint8_t

strong

This enum represent the life-cycle of a nes partition in the partition manager: NotFound: partition was never registered in the current partition manager. Deleted: partition was unregistered at some point in time (we keep it for bookkeeping). Registered: partition was registered and alive (meaning its ref cnt is greater than 0).

Enumerator
Registered	a partition is registered, i.e., its counter is >= 0
Deleted	a partition was registered but got deleted, i.e., its counter is == 0
NotFound	a partition was never registered

Function Documentation

createMockedEngine()

template<typename MockedNodeEngine , typename... ExtraParameters>

std::shared_ptr<MockedNodeEngine> NES::Network::createMockedEngine	(	const std::string &	hostname,
		uint16_t	port,
		uint64_t	bufferSize,
		uint64_t	numBuffers,
		ExtraParameters &&...	extraParams
	)

References NES::bufferSize.

fillBuffer()

void NES::Network::fillBuffer	(	TupleBuffer &	buf,
		const Runtime::MemoryLayouts::RowLayoutPtr &	memoryLayout
	)

References NES::Runtime::TupleBuffer::setNumberOfTuples().

Referenced by TEST_F().

Here is the call graph for this function:

Here is the caller graph for this function:

FORWARD_DECLARE_CLASS_SP() [1/4]

NES::Network::FORWARD_DECLARE_CLASS_SP

(

NetworkManager

)

FORWARD_DECLARE_CLASS_SP() [2/4]

NES::Network::FORWARD_DECLARE_CLASS_SP

(

NetworkSink

)

FORWARD_DECLARE_CLASS_SP() [3/4]

NES::Network::FORWARD_DECLARE_CLASS_SP

(

NetworkSource

)

FORWARD_DECLARE_CLASS_SP() [4/4]

NES::Network::FORWARD_DECLARE_CLASS_SP

(

PartitionManager

)

FORWARD_DECLARE_CLASS_UP() [1/3]

NES::Network::FORWARD_DECLARE_CLASS_UP

(

EventOnlyNetworkChannel

)

FORWARD_DECLARE_CLASS_UP() [2/3]

NES::Network::FORWARD_DECLARE_CLASS_UP

(

NetworkChannel

)

FORWARD_DECLARE_CLASS_UP() [3/3]

NES::Network::FORWARD_DECLARE_CLASS_UP

(

ZmqServer

)

INSTANTIATE_TEST_CASE_P()

NES::Network::INSTANTIATE_TEST_CASE_P	(	NetworkStackIntegrationTestParam	,
		NetworkStackIntegrationTest	,
		::testing::Values(false, true)
	)

sendMessage()

template<typename MessageType , decltype(kZmqSendDefault) flags = kZmqSendDefault, typename... Arguments>

void NES::Network::sendMessage	(	zmq::socket_t &	zmqSocket,
		Arguments &&...	args
	)

Send a message MessageType(args) through an open zmqSocket

Template Parameters

MessageType
Arguments

Parameters

zmqSocket
args

References NES_ASSERT2_FMT.

sendMessageNoHeader()

template<typename MessageType , decltype(kZmqSendDefault) flags = kZmqSendDefault, typename... Arguments>

void NES::Network::sendMessageNoHeader	(	zmq::socket_t &	zmqSocket,
		Arguments &&...	args
	)

Send a message MessageType(args) through an open zmqSocket with no header

Template Parameters

MessageType
Arguments

Parameters

zmqSocket
args

References NES_ASSERT2_FMT.

sendMessageWithIdentity()

template<typename MessageType , typename... Arguments>

void NES::Network::sendMessageWithIdentity	(	zmq::socket_t &	zmqSocket,
		zmq::message_t &	zmqIdentity,
		Arguments &&...	args
	)

Send a zmqIdentity followed by a message MessageType(args) via zmqSocket

Template Parameters

MessageType
Arguments

Parameters

zmqSocket
args

References NES_ASSERT2_FMT.

TEST_F() [1/15]

NES::Network::TEST_F	(	NetworkStackIntegrationTest	,
		DISABLED_testSendEventBackward
	)

TEST_F() [2/15]

NES::Network::TEST_F	(	NetworkStackIntegrationTest	,
		testQEPNetworkSinkSource
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [3/15]

NES::Network::TEST_F	(	NetworkStackIntegrationTest	,
		testSendEvent
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [4/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		dispatcherMustStartAndStop
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [5/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		serverMustStartAndStop
	)

References NES::bufferSize, NES::buffersManaged, NES::Network::ZmqServer::isServerRunning(), and NES::Network::ZmqServer::start().

Here is the call graph for this function:

TEST_F() [6/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		serverMustStartAndStopRandomPort
	)

References NES::bufferSize, NES::buffersManaged, NES::Network::ZmqServer::getServerPort(), NES::Network::ZmqServer::isServerRunning(), and NES::Network::ZmqServer::start().

Here is the call graph for this function:

TEST_F() [7/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		startCloseChannel
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [8/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		startCloseChannelAsyncIndefiniteRetries
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [9/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		startCloseMaxChannel
	)

References NES::bufferSize, and NES::buffersManaged.

TEST_F() [10/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testCorrectHandlingEOS
	)

References NES::bufferSize, NES::buffersManaged, NES::Runtime::TupleBuffer::getBuffer(), and NES::Runtime::TupleBuffer::getBufferSize().

Here is the call graph for this function:

TEST_F() [11/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testHandleUnregisteredBuffer
	)

References NES::bufferSize, NES::buffersManaged, NES::Network::Messages::ErrorMessage::getErrorType(), and NES_INFO.

Here is the call graph for this function:

TEST_F() [12/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testMassiveMultiSending
	)

References NES::bufferSize, NES::buffersManaged, NES::Runtime::TupleBuffer::getBuffer(), NES::Network::Messages::ExchangeMessage::getChannelId(), and NES::Network::ChannelId::getNesPartition().

Here is the call graph for this function:

TEST_F() [13/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testMassiveSending
	)

References NES::bufferSize, NES::buffersManaged, NES::Runtime::TupleBuffer::getBuffer(), and NES::Runtime::TupleBuffer::getBufferSize().

Here is the call graph for this function:

TEST_F() [14/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testMassiveSendingWithChildrenBuffer
	)

References NES::bufferSize, NES::buffersManaged, NES::Runtime::TupleBuffer::getBuffer(), NES::Runtime::TupleBuffer::getBufferSize(), NES::Runtime::TupleBuffer::getNumberOfTuples(), and NES::Runtime::TupleBuffer::loadChildBuffer().

Here is the call graph for this function:

TEST_F() [15/15]

NES::Network::TEST_F	(	NetworkStackTest	,
		testSendData
	)

References NES::bufferSize, NES::buffersManaged, and NES::Runtime::TupleBuffer::getBufferSize().

Here is the call graph for this function:

TEST_P()

NES::Network::TEST_P	(	NetworkStackIntegrationTest	,
		testNetworkSourceSink
	)