The NetworkManager manages creation and deletion of subpartition producer and consumer. More...

#include <NetworkManager.hpp>

Public Member Functions
	NetworkManager (WorkerId nodeEngineId, const std::string &hostname, uint16_t port, ExchangeProtocol &&exchangeProtocol, const Runtime::BufferManagerPtr &bufferManager, int senderHighWatermark, uint16_t numServerThread=DEFAULT_NUM_SERVER_THREADS, bool connectSinksAsync=false, bool connectSourceEventChannelsAsync=false)
	Creates a new network manager object, which comprises of a zmq server and an exchange protocol. More...

	~NetworkManager ()
	Destroy the network manager calling destroy() More...

bool	registerSubpartitionConsumer (const NesPartition &nesPartition, const NodeLocation &senderLocation, const DataEmitterPtr &emitter) const
	This method is called on the receiver side to register a SubpartitionConsumer, i.e. indicate that the server is ready to receive particular subpartitions. More...

bool	unregisterSubpartitionConsumer (const NesPartition &nesPartition) const
	This method is called on the receiver side to remove a SubpartitionConsumer. More...

bool	unregisterSubpartitionProducer (const NesPartition &nesPartition) const
	This method is called on the receiver side to remove a SubpartitionConsumer. More...

PartitionRegistrationStatus	isPartitionConsumerRegistered (const NesPartition &nesPartition) const

PartitionRegistrationStatus	isPartitionProducerRegistered (const NesPartition &nesPartition) const

NetworkChannelPtr	registerSubpartitionProducer (const NodeLocation &nodeLocation, const NesPartition &nesPartition, Runtime::BufferManagerPtr bufferManager, std::chrono::milliseconds waitTime, uint8_t retryTimes, DecomposedQueryPlanVersion version=0)
	This method is called on the sender side to register a SubpartitionProducer. If the connection to the destination server is successful, a pointer to the DataChannel is returned, else nullptr is returned. The DataChannel is not thread safe! More...

std::pair< std::future< NetworkChannelPtr >, std::promise< bool > >	registerSubpartitionProducerAsync (const NodeLocation &nodeLocation, const NesPartition &nesPartition, Runtime::BufferManagerPtr bufferManager, std::chrono::milliseconds waitTime, uint8_t retryTimes, Runtime::ReconfigurationMessage reconfigurationMessage, Runtime::QueryManagerPtr queryManager, DecomposedQueryPlanVersion version=0)
	This method is called on the sender side to asynchronously register a SubpartitionProducer. It returns a future that on completion will contain a pointer to the DataChannel if the connection to the destination server is successful, or nullptr otherwise The DataChannel is not thread safe! More...

EventOnlyNetworkChannelPtr	registerSubpartitionEventProducer (const NodeLocation &nodeLocation, const NesPartition &nesPartition, Runtime::BufferManagerPtr bufferManager, std::chrono::milliseconds waitTime, uint8_t retryTimes)
	This method is called on the sender side to register a SubpartitionProducer. If the connection to the destination server is successful, a pointer to the DataChannel is returned, else nullptr is returned. The DataChannel is not thread safe! More...

bool	registerSubpartitionEventConsumer (const NodeLocation &nodeLocation, const NesPartition &nesPartition, Runtime::RuntimeEventListenerPtr eventListener)

void	destroy ()
	This methods destroys the network manager by stopping the underlying (ZMQ) server. More...

NodeLocation	getServerLocation () const
	Returns the FQDN of the network manager. More...

uint16_t	getServerDataPort () const
	Returns the server data port. More...

std::pair< std::future< EventOnlyNetworkChannelPtr >, std::promise< bool > >	registerSubpartitionEventProducerAsync (const NodeLocation &nodeLocation, const NesPartition &nesPartition, Runtime::BufferManagerPtr bufferManager, std::chrono::milliseconds waitTime, uint8_t retryTimes)
	This method is called to asynchronously register an event produce. It returns a future that on completion will contain a pointer to the EventChannel if the connection to the destination server is successful, or nullptr otherwise. More...

bool	getConnectSinksAsync ()
	retrieve the value of the connectSinkAsync flag which indicates if a separate thread should be used to establish network channels More...

bool	getConnectSourceEventChannelsAsync ()
	retrieve the value of the connectSourceEventChannelsAsync flag which indicates if a separate thread should be used to establish event channels from sources to sinks More...

Static Public Member Functions
static NetworkManagerPtr	create (WorkerId nodeEngineId, const std::string &hostname, uint16_t port, Network::ExchangeProtocol &&exchangeProtocol, const Runtime::BufferManagerPtr &bufferManager, int senderHighWatermark=-1, uint16_t numServerThread=DEFAULT_NUM_SERVER_THREADS, bool connectSinksAsync=false, bool connectSourceEventChannelsAsync=false)
	create method to return a shared pointer of the NetworkManager More...

Detailed Description

The NetworkManager manages creation and deletion of subpartition producer and consumer.

Constructor & Destructor Documentation

◆ NetworkManager()

NES::Network::NetworkManager::NetworkManager	(	WorkerId	nodeEngineId,
		const std::string &	hostname,
		uint16_t	port,
		ExchangeProtocol &&	exchangeProtocol,
		const Runtime::BufferManagerPtr &	bufferManager,
		int	senderHighWatermark,
		uint16_t	numServerThread = `DEFAULT_NUM_SERVER_THREADS`,
		bool	connectSinksAsync = `false`,
		bool	connectSourceEventChannelsAsync = `false`
	)

explicit

Creates a new network manager object, which comprises of a zmq server and an exchange protocol.

Parameters

nodeEngineId
hostname
port
exchangeProtocol
bufferManager
senderHighWatermark
numServerThread
connectSinksAsync	if true, sinks will use a dedicated thread when attempting to establish a network channel
connectSourceEventChannelsAsync	if true, source will use a dedicated thread when attempting to establish an event channel

References NES_INFO, and NES_THROW_RUNTIME_ERROR.

◆ ~NetworkManager()

NES::Network::NetworkManager::~NetworkManager ( )

Destroy the network manager calling destroy()

References destroy().

Here is the call graph for this function:

Member Function Documentation

◆ create()

NetworkManagerPtr NES::Network::NetworkManager::create	(	WorkerId	nodeEngineId,
		const std::string &	hostname,
		uint16_t	port,
		Network::ExchangeProtocol &&	exchangeProtocol,
		const Runtime::BufferManagerPtr &	bufferManager,
		int	senderHighWatermark = `-1`,
		uint16_t	numServerThread = `DEFAULT_NUM_SERVER_THREADS`,
		bool	connectSinksAsync = `false`,
		bool	connectSourceEventChannelsAsync = `false`
	)

static

create method to return a shared pointer of the NetworkManager

Parameters

nodeEngineId

References backward::details::move().

Here is the call graph for this function:

◆ destroy()

void NES::Network::NetworkManager::destroy ( )

This methods destroys the network manager by stopping the underlying (ZMQ) server.

Referenced by ~NetworkManager().

Here is the caller graph for this function:

◆ getConnectSinksAsync()

bool NES::Network::NetworkManager::getConnectSinksAsync ( )

retrieve the value of the connectSinkAsync flag which indicates if a separate thread should be used to establish network channels

Returns: the value of the connectSinkAsync flag

◆ getConnectSourceEventChannelsAsync()

bool NES::Network::NetworkManager::getConnectSourceEventChannelsAsync ( )

retrieve the value of the connectSourceEventChannelsAsync flag which indicates if a separate thread should be used to establish event channels from sources to sinks

Returns: the value of the connectSinkAsync flag

◆ getServerDataPort()

uint16_t NES::Network::NetworkManager::getServerDataPort ( ) const

Returns the server data port.

Returns: the server data port

◆ getServerLocation()

NodeLocation NES::Network::NetworkManager::getServerLocation ( ) const

Returns the FQDN of the network manager.

Returns: the network location of the network manager

◆ isPartitionConsumerRegistered()

PartitionRegistrationStatus NES::Network::NetworkManager::isPartitionConsumerRegistered ( const NesPartition & nesPartition ) const

Checks if a partition is registered

Parameters

nesPartition the id of the logical channel between sender and receiver

Returns: true if the partition is registered

◆ isPartitionProducerRegistered()

PartitionRegistrationStatus NES::Network::NetworkManager::isPartitionProducerRegistered ( const NesPartition & nesPartition ) const

Checks if a partition is registered

Parameters

nesPartition the id of the logical channel between sender and receiver

Returns: true if the partition is registered

◆ registerSubpartitionConsumer()

bool NES::Network::NetworkManager::registerSubpartitionConsumer	(	const NesPartition &	nesPartition,
		const NodeLocation &	senderLocation,
		const DataEmitterPtr &	emitter
	)		const

This method is called on the receiver side to register a SubpartitionConsumer, i.e. indicate that the server is ready to receive particular subpartitions.

Parameters

nesPartition	the id of the logical channel between sender and receiver
senderLocation	the network location of the sender
emitter	underlying network source

Returns: true if the partition was registered for the first time, false otherwise

References NES_ASSERT2_FMT, and NES_DEBUG.

◆ registerSubpartitionEventConsumer()

bool NES::Network::NetworkManager::registerSubpartitionEventConsumer	(	const NodeLocation &	nodeLocation,
		const NesPartition &	nesPartition,
		Runtime::RuntimeEventListenerPtr	eventListener
	)

Parameters

nodeLocation
nesPartition

Returns

References NES_DEBUG.

◆ registerSubpartitionEventProducer()

EventOnlyNetworkChannelPtr NES::Network::NetworkManager::registerSubpartitionEventProducer	(	const NodeLocation &	nodeLocation,
		const NesPartition &	nesPartition,
		Runtime::BufferManagerPtr	bufferManager,
		std::chrono::milliseconds	waitTime,
		uint8_t	retryTimes
	)

This method is called on the sender side to register a SubpartitionProducer. If the connection to the destination server is successful, a pointer to the DataChannel is returned, else nullptr is returned. The DataChannel is not thread safe!

Parameters

nodeLocation	is the destination
nesPartition	indicates the partition
waitTime	time in seconds to wait until a retry is called
retryTimes	times to retry a connection

Returns: the event-only network channel

References NES::Network::EventOnlyNetworkChannel::create(), backward::details::move(), and NES_DEBUG.

Here is the call graph for this function:

◆ registerSubpartitionEventProducerAsync()

std::pair< std::future< EventOnlyNetworkChannelPtr >, std::promise< bool > > NES::Network::NetworkManager::registerSubpartitionEventProducerAsync	(	const NodeLocation &	nodeLocation,
		const NesPartition &	nesPartition,
		Runtime::BufferManagerPtr	bufferManager,
		std::chrono::milliseconds	waitTime,
		uint8_t	retryTimes
	)

This method is called to asynchronously register an event produce. It returns a future that on completion will contain a pointer to the EventChannel if the connection to the destination server is successful, or nullptr otherwise.

Parameters

nodeLocation	is the destination
nesPartition	indicates the partition
bufferManager	a pointer the buffer manager
waitTime	time in seconds to wait until a retry is called
retryTimes	times to retry a connectio.
version	the version number which will be used by the receiver to determine if this channel will be accepted

Returns: a pair consisting of a future containing the data network channel on completion and a promise that aborts the connection process when its value is set (abortion not yet implemented)

References backward::details::move(), NES_DEBUG, and protocol.

Here is the call graph for this function:

◆ registerSubpartitionProducer()

NetworkChannelPtr NES::Network::NetworkManager::registerSubpartitionProducer	(	const NodeLocation &	nodeLocation,
		const NesPartition &	nesPartition,
		Runtime::BufferManagerPtr	bufferManager,
		std::chrono::milliseconds	waitTime,
		uint8_t	retryTimes,
		DecomposedQueryPlanVersion	version = `0`
	)

This method is called on the sender side to register a SubpartitionProducer. If the connection to the destination server is successful, a pointer to the DataChannel is returned, else nullptr is returned. The DataChannel is not thread safe!

Parameters

nodeLocation	is the destination
nesPartition	indicates the partition
waitTime	time in seconds to wait until a retry is called
retryTimes	times to retry a connection
version	the version number which will be used by the receiver to determine if this channel will be accepted

Returns: the data network channel

References NES::Network::NetworkChannel::create(), backward::details::move(), and NES_DEBUG.

Here is the call graph for this function:

◆ registerSubpartitionProducerAsync()

std::pair< std::future< NetworkChannelPtr >, std::promise< bool > > NES::Network::NetworkManager::registerSubpartitionProducerAsync	(	const NodeLocation &	nodeLocation,
		const NesPartition &	nesPartition,
		Runtime::BufferManagerPtr	bufferManager,
		std::chrono::milliseconds	waitTime,
		uint8_t	retryTimes,
		Runtime::ReconfigurationMessage	reconfigurationMessage,
		Runtime::QueryManagerPtr	queryManager,
		DecomposedQueryPlanVersion	version = `0`
	)

This method is called on the sender side to asynchronously register a SubpartitionProducer. It returns a future that on completion will contain a pointer to the DataChannel if the connection to the destination server is successful, or nullptr otherwise The DataChannel is not thread safe!

Parameters

nodeLocation	is the destination
nesPartition	indicates the partition
waitTime	time in seconds to wait until a retry is called
retryTimes	times to retry a connectio. Set this to zero to retry indefinitely.
reconfigurationMessage	a message to be inserted into the query manager on completion, to inform the caller about the completion of the operation
queryManager	a pointer to the query manager which will hand over the reconfiguration message to the caller
version	the version number which will be used by the receiver to determine if this channel will be accepted

Returns: a pair consisting of a future containing the data network channel on completion and a promise that aborts the connection process when its value is set

References backward::details::move(), NES_DEBUG, and protocol.

Here is the call graph for this function:

◆ unregisterSubpartitionConsumer()

bool NES::Network::NetworkManager::unregisterSubpartitionConsumer ( const NesPartition & nesPartition ) const

This method is called on the receiver side to remove a SubpartitionConsumer.

Parameters

nesPartition the id of the logical channel between sender and receiver

Returns: true if the partition was registered fully, false otherwise

References NES_DEBUG.

◆ unregisterSubpartitionProducer()

bool NES::Network::NetworkManager::unregisterSubpartitionProducer ( const NesPartition & nesPartition ) const

This method is called on the receiver side to remove a SubpartitionConsumer.

Parameters

nesPartition the id of the logical channel between sender and receiver

Returns: true if the partition was registered fully, false otherwise

References NES_DEBUG.

The documentation for this class was generated from the following files:

/nebulastream/nes-runtime/include/Network/NetworkManager.hpp
/nebulastream/nes-runtime/src/Network/NetworkManager.cpp

Public Member Functions

Static Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ NetworkManager()

◆ ~NetworkManager()

Member Function Documentation

◆ create()

◆ destroy()

◆ getConnectSinksAsync()

◆ getConnectSourceEventChannelsAsync()

◆ getServerDataPort()

◆ getServerLocation()

◆ isPartitionConsumerRegistered()

◆ isPartitionProducerRegistered()

◆ registerSubpartitionConsumer()

◆ registerSubpartitionEventConsumer()

◆ registerSubpartitionEventProducer()

◆ registerSubpartitionEventProducerAsync()

◆ registerSubpartitionProducer()

◆ registerSubpartitionProducerAsync()

◆ unregisterSubpartitionConsumer()

◆ unregisterSubpartitionProducer()