Exchange network backend from Gsocket to libfabric #124

JBenda · 2021-10-28T11:02:44Z

gains:

utilization of infinyband and other communication routs
TCP performance is similar
utilization of rdma

also adapt tests

* start implementing jfabric functions

test case assumed communication via GSocket, we need to redesign this test to work with the new api

* fixes bufnr usage * proper shutdown behavior * offset hack for 0 based address

* tcp connections have default request key * tcp memory reads are identified via key and not address!

* removed libfabric_example from builds

some parts are still referenced in the fi_info, and to early freed if hints get freed

michaelkuhn · 2021-10-29T14:34:35Z

Could you please rebase this to current master? There are quite a few conflicts. It seems the PR includes the (already merged) port changes as well, so those should be dropped.

JBenda · 2021-11-02T08:03:30Z

.github/workflows/ci.yml

-            sudo pip3 install meson
+          sudo apt --yes --no-install-recommends install ninja-build pkgconf libglib2.0-dev libbson-dev liblmdb-dev libsqlite3-dev libleveldb-dev libmongoc-dev libmariadb-dev librocksdb-dev libfuse-dev libopen-trace-format-dev librados-dev libfabric-dev
+          sudo apt --yes --no-install-recommends install python3 python3-pip python3-setuptools python3-wheel
+          sudo pip3 install meson


Add libfabric to dependencies

JBenda · 2021-11-02T08:04:22Z

.github/workflows/ci.yml

+          # libfaibrc don't work on Ubuntu 20.04
+          - os:
+              dist: ubuntu-20.04
+            dependencies: system


Disable system test because libfabric won't start properly

JBenda · 2021-11-02T08:04:29Z

.github/workflows/ci.yml

-            sudo pip3 install meson
+          sudo apt --yes --no-install-recommends install ninja-build pkgconf libglib2.0-dev libbson-dev liblmdb-dev libsqlite3-dev libleveldb-dev libmongoc-dev libmariadb-dev librocksdb-dev libfuse-dev libopen-trace-format-dev librados-dev libfabric-dev
+          sudo apt --yes --no-install-recommends install python3 python3-pip python3-setuptools python3-wheel
+          sudo pip3 install meson


Add libfabric to dependencies

JBenda · 2021-11-02T08:04:47Z

.github/workflows/ci.yml

@@ -280,9 +284,9 @@ jobs:
        run: |
          . scripts/environment.sh
          ./scripts/setup.sh start
-          ./scripts/test.sh
+          ./scripts/test.sh || (journalctl GLIB_DOMAIN=JULEA && $(exit 1))


Print journal from SERVER if test fails

JBenda · 2021-11-02T08:08:38Z

backend/db/sql-generic.c

@@ -1109,7 +1109,7 @@ _backend_schema_get(gpointer backend_data, gpointer _batch, gchar const* name, b
 	return TRUE;

 _error:
-	if (G_UNLIKELY(!j_sql_reset(thread_variables->sql_backend, prepared->stmt, NULL)))
+	if (G_UNLIKELY(!thread_variables || !prepared || !j_sql_reset(thread_variables->sql_backend, prepared->stmt, NULL)))


thread_variables and/or prepared may be NULL if initialization failed. Therefore, we must check it in error handling

JBenda · 2021-11-02T08:09:55Z

include/core/jconfiguration.h

@@ -110,9 +111,14 @@ gchar const* j_configuration_get_backend_path(JConfiguration*, JBackendType);
 guint64 j_configuration_get_max_operation_size(JConfiguration*);
 guint16 j_configuration_get_port(JConfiguration*);

+/// size of data segment which should be injected in message
+guint64 j_configuration_get_message_inject_size(JConfiguration*);


Reduces RMA reads of small segments

JBenda added 30 commits October 29, 2021 13:25

Add network port to jconfiguration

7884e48

Replaced magic port numbers with configuration

faced31

Add port configuration to config-tool

0d12075

also adapt tests

Adapt documentation for new feature

0527b66

Add libfabirc example

2516f77

Fix memory leaks and add closing server

340cce7

First docu of network

a619897

Changed srecv to recv

4793355

Place jnetwork in correct location & edit build file

5d50465

introduce rma_mem_id

ed03ab4

Adapt jmessage to jnetwork

d4f0673

Introduce new interface in loop and jobject

d04b2b0

* start implementing jfabric functions

It's compiled (what is already coded)

1968022

WIP

8666f0f

Working hand shake and no connection stall

859f91e

Transfare first header correct

578fc2b

Remove debug code

20a7e00

Hello World works

16b693b

Remove prints and dummy vars

279418e

Remove hints memory leak

f31b35e

Free message buffer memory after finish tranmition

9cd6dd0

Enable injecting of small messages

71d6438

Fix new header ack communication for distrubted-objectk

304b596

Disable test

48f590d

test case assumed communication via GSocket, we need to redesign this test to work with the new api

Use JCredentials to fetch user id for port

dff90ef

Read server data from config

dfb3ca2

Remove Debug message

f0db28d

Editing to work via tcp

9eb831f

Working tcp hello_world

039d313

* fixes bufnr usage * proper shutdown behavior * offset hack for 0 based address

More tcp fixes

13c6117

* tcp connections have default request key * tcp memory reads are identified via key and not address!

JBenda added 12 commits October 29, 2021 13:54

Rework todos in network code

639b53c

Run autoformat

9e570fb

Add RDMA dependency

e0d6927

* removed libfabric_example from builds

clean up compiler warnings

925207f

Add libfabric to system dependencies

0515521

Add libfabric-dev also for test build

833fa3d

Fix missing initelization

2ae6e61

Moved message injection size to config

f6ea573

Print julea server log on test fail

d51d50e

Keep fabric hints around until finalisation

93c0a95

some parts are still referenced in the fi_info, and to early freed if hints get freed

Fix test workflow results

99d136c

Run autoformatter

a15b56b

Rebase to master, set libfabric version to 1.6

919ff5c

JBenda force-pushed the feature/add_libfabirc branch from 7bdfa9d to 919ff5c Compare October 31, 2021 12:12

JBenda added 5 commits October 31, 2021 13:17

Fix ci

770927e

Autoformat + disable system checks for Ubuntu20.04

cb7d9ad

Fix spelling mistake

830a888

Set Libfabric version back to 11

82ea7f3

Removed wrong flag from julea-config example

9c036dd

JBenda commented Nov 2, 2021

View reviewed changes

Merge branch 'master' into feature/add_libfabirc

4d5e8bb

michaelkuhn self-assigned this Jan 29, 2022

michaelkuhn pushed a commit that referenced this pull request Jan 30, 2022

meson, scripts, workflows: Enable libfabric (#124)

e2a2cba

michaelkuhn pushed a commit that referenced this pull request Jan 30, 2022

lib/core: Add configuration for maximum inject size (#124)

9194ca8

michaelkuhn added a commit that referenced this pull request Jan 30, 2022

tools: Allow configuring max inject size (#124)

5269c06

michaelkuhn pushed a commit that referenced this pull request Jan 30, 2022

lib/core: Add JNetwork (#124)

85f2bea

JBenda closed this Jan 19, 2023

michaelkuhn reopened this Jan 20, 2023

michaelkuhn added a commit that referenced this pull request Jan 21, 2023

lib/core: Refactor JNetwork's init functions (#124)

519f3ce

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exchange network backend from Gsocket to libfabric #124

Exchange network backend from Gsocket to libfabric #124

JBenda commented Oct 28, 2021

michaelkuhn commented Oct 29, 2021

JBenda Nov 2, 2021

JBenda Nov 2, 2021

JBenda Nov 2, 2021

JBenda Nov 2, 2021

JBenda Nov 2, 2021

JBenda Nov 2, 2021

Exchange network backend from Gsocket to libfabric #124

Are you sure you want to change the base?

Exchange network backend from Gsocket to libfabric #124

Conversation

JBenda commented Oct 28, 2021

michaelkuhn commented Oct 29, 2021

JBenda Nov 2, 2021

Choose a reason for hiding this comment

JBenda Nov 2, 2021

Choose a reason for hiding this comment

JBenda Nov 2, 2021

Choose a reason for hiding this comment

JBenda Nov 2, 2021

Choose a reason for hiding this comment

JBenda Nov 2, 2021

Choose a reason for hiding this comment

JBenda Nov 2, 2021

Choose a reason for hiding this comment