User Interface

Everything you need to know about using the NebulaStream user interface.

NebulaStream UI provides an intuitive way to interact with NebulaStream. In the following, we describe how to run our UI together with NebulaStream and obtain data from nebulastream-smart-city-demo. As we are constantly improving the demo pipeline, this documentation might not always be up to date. Feel free to open an issue in our NebulaStream-UI repository you cannot run the demo pipeline as described below. Make sure to include error messages or other information that might help to identify your issues with the pipeline.

Prerequisite #

Obtain NebulaStream for Local Deployment #

Clone the latest version of Nebulastream. Build instructions for both the docker image and the source can be found here. If you build from source, make sure to add -DNES_USE_MQTT=1 to the cmake configuration. This is necessary, since our UI uses an MQTT broker to retrieve processed data for result visualization.

Install an MQTT Broker #

As MQTT Broker, we use Mosquitto. Note that we tested our pipeline with mosquitto v1.6.9 and also highly recommend that version, as others might cause issues.

Get the UI ready to start #

Clone the current master branch of this repository.

Prior to running our nebulastream UI, you need to install the following packages:

• npm using sudo apt install npm
• create-react-app using sudo npm -g install create-react-app

Locally Deploying NebulaStream and UI #

Starting a Coordinator #

First start a coordinator. Recommended CLI parameters are:

--rpcPort=1200 --restPort=8000 --enableMonitoring=true

If you change the restPort, you also have to change the REST port configuration in the UI using [settings]#connecting-to-nebulastream.

Add a Logical Stream #

Once the UI is running, you may add a logical stream by either using the provided fields on the Stream Catalog page or sending a String to NebulaStream, e.g.:

"logicalSourceName": "nesCityHospital", "schema": "Schema::create()->addField("type",DataTypeFactory::createFixedChar(12))->addField("hospitalId",INT64)->addField("stationId",INT64)->addField("patientId",INT64)->addField("time",UINT64)->addField("healthStatus",INT8)->addField("healthStatusDuration",INT64)->addField("recovered",BOOLEAN)->addField("dead",BOOLEAN);

With this String you obtain data from the hospital use case implemented in nebulastream-smart-city-demo repository. Do not worry too much about the quotation marks, if you want to, you may also remove them. For further details on the hospital data and additional use cases that nebulastream-smart-city-demo provides, please visit nebulastream-smart-city-demo.

Starting a Worker #

Afterwards, you can start a worker with these recommended configuration parameters for our nebulastream-smart-city-demo:

--coordinatorPort=1200 --rpcPort=1215 --dataPort=1415 --physicalSources.physicalSourceName=nesCityHospital --physicalSources.logicalSourceName=nesCityHospital --physicalSources.type=MQTT_SOURCE --physicalSources.clientId=client1 --physicalSources.userName=user --physicalSources.url=tcp://127.0.0.1:1885 --physicalSources.topic=demoCityHospital_0 --physicalSources.qos=1 --physicalSources.inputFormat=JSON --physicalSources.cleanSession=true --physicalSources.flushIntervalMS=60 --numWorkerThreads=3

More information on starting/stopping NebulaStream and available configurations.

Start Mosquitto Broker #

To start the mosquitto broker simply use:

mosquitto -c [Path-To-nebulastream-ui]/nebulastream-ui/mosquitto/mosquitto.conf

The mosquitto broker will then use path 1885, 9001, and 9002 for data transmission between nebulastream-smart-city-demo, NebulaStream and UI.

Start Nebulastream UI #

In the project directory, run the following commands:

• npm install (💡 required to be run only first time to install all dependencies)
• npm start

Open http://localhost:3000 to view it in the browser. This will take you to the landing page that provides an overview of current status of NebulaStream.

The landing page provides an overview of and some details on the running processes:

1. You can see the number of registered and running queries. Changes to this are possible when visiting the Query Catalog.
2. The Topology overview shows the number of nodes currently running. The Topology page provides more details on this.
3. The Source Catalog overview displays all currently registered logical sources. You can find details about the registered sources by visiting the Source Catalog.
4. Visualizing your results is possible on the Result Visualization page.
5. Changing Settings can be done on the Settings page.

Functionalities #

Submitting a Query #

There are two possible ways to submit a query:

1. Code Editor.
2. Query Builder.

Using Code Editor #

1. To submit a query to NebulaStream, you enter your query in the code editor and press the play button. An example for a query could be:

Query::from("default_logical").sink(MQTTSinkDescriptor::create("ws://127.0.0.1:9001", "/nesui", "rfRqLGZRChg8eS30PEeR", 1000, MQTTSinkDescriptor::milliseconds, 0, MQTTSinkDescriptor::atLeastOnce, false));

💡 Query Description:

• Read data from the “default logical” source.
• Forwards data to the MQTT sink.
• UI consumes the processed data from the MQTT broker with address ws://127.0.0.1:9001 and for the topic /nesui as defined in the query.

2. In the strategy drop down menu, you can change NebulaStream’s execution strategy by choosing between a “bottom up” or a “top down” approach.
3. You can filter for a query ID or status.
4. Canceling the query submission closes the code editor. You will still be able to see the table with the query overview.
5. Show details will open up the code editor with the submitted query string as well as the possibility to explore the Execution and Query Plan.
6. In case you want to stop, restart, or delete your query, just hit the appropriate symbol for the action you want to perform.

Using Query Builder 🔧 #

For the Drag and Drop menu, only the first step (1.) changes:

Following are the steps to build a query:

• Drag the desired operators into the white space and connect them.
• Define the operator’s parameters (e.g. source name) by double-clicking on the operator.
• Enter the desired configurations in the dialogue field.

Explore Topology #

Here, you can explore NebulaStream’s current topology.

1. Shows the underlying topology of the currently running NebulaStream cluster. You obtain more insights about a worker node and its monitoring metrics by clicking on the node.
2. Filter to conveniently find nodes.

Check Logical Sources #

In the source catalog, you find overview of all available logical sources.

1. Open a dialog to add a new logical source.
2. Search for specific logical source.
3. Obtain details for the selected logical source as displayed in “Explore Source Details and edit Schemas”.

Add a New Source #

This is the dialog that opens when you hit the add a new logical source button from 1. in “Obtaining Information about registered Sources”.

1. Enter the name of the logical source.
2. Set the field names and select the data type for the given field.
3. Add as many fields as you like to the schema for your logical data source.
4. If you do not want to manually add fields, you can also enter a JSON that is going to do the job for you. An example of such a JSON could be:

{
  "logicalSourceName": "CityHospital",
  "schema": "Schema::create()->addField(\"type\",DataTypeFactory::createFixedChar(12))->addField(\"stationId\",INT64)->addField(\"patientId\",INT64)->addField(\"time\",UINT64)->addField(\"healthStatus\",INT8)->addField(\"recovered\",BOOLEAN);"
}

In this case, the logical source’s name is “CityHospital” and its schema consists of 6 attributes: “type” (String length 12), “stationId” (INT64), “patientId” (INT64), “time” (UINT64), “healthStatus” (INT8), and “recovered” (BOOLEAN).

Explore Source Details and edit Schemas #

When you click on the “show details” button (3. from “Obtaining Information about registered Sources”), you will find yourself on this page.

1. Switch between the schema detail view and the physical sources view.
2. Edit the source schema of the logical source.
3. fields with data type for the logical source.

Visualize Query Results #

This page enables visualizing your query results.

1. Choose the query for which you want to visualize results from the dropdown menu.
2. Select a chart that fits your needs.
3. Adjust the host and port for the MQTT broker. By default MQTT configurations from the query’s sink are picked.
4. Add the graph to the page.

Connecting to NebulaStream #

1. You can define the ip address and port to connect to a running NebulaStream cluster.
2. You can change the mode of the UI from dark to light mode.
3. Save everything and you are good to go.