Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LFC federated RIOT fixes #192

Merged
merged 70 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
0584253
Have LFC generate a main function that can optionally be included in …
erlingrj Dec 8, 2024
6b51697
CI
erlingrj Dec 8, 2024
55b308a
CI
erlingrj Dec 8, 2024
a8973e8
CI
erlingrj Dec 8, 2024
cfda3bd
Setup LFC in RIOT container
erlingrj Dec 8, 2024
2c38b98
Switch to riot master
LasseRosenow Dec 8, 2024
68e8d11
Fix
LasseRosenow Dec 8, 2024
60ceeea
Install LFC deps in zephyr CI also
erlingrj Dec 9, 2024
2704dbd
WIP
erlingrj Dec 9, 2024
5ba1392
WIP
erlingrj Dec 12, 2024
a8bc451
WIP: SimpleFederated.lf now compiles.
erlingrj Dec 13, 2024
ead4a67
Generate a launch script for federated native
erlingrj Dec 13, 2024
ddbed66
Merge remote-tracking branch 'origin/main' into lfc-federated
erlingrj Dec 14, 2024
aec7438
Make federated launch script executable
erlingrj Dec 14, 2024
d0b4e22
WIP
erlingrj Dec 20, 2024
8efd065
Rework the handling of connections
erlingrj Jan 3, 2025
c049c73
Various fixes to get all standalone tests to pass again
erlingrj Jan 3, 2025
991264a
Formatting
erlingrj Jan 3, 2025
43874dd
More WIP
erlingrj Jan 8, 2025
3689883
Formatting
erlingrj Jan 8, 2025
9eb722d
More WIP
erlingrj Jan 9, 2025
d5e95fe
Add @interface attr
erlingrj Jan 9, 2025
9b63d84
More WIP
erlingrj Jan 10, 2025
2f9c0f4
More WIP
erlingrj Jan 13, 2025
03a428c
More WIP
erlingrj Jan 13, 2025
c7dde4f
Merge remote-tracking branch 'origin/main' into lfc-federated
erlingrj Jan 13, 2025
f910e5e
All tests are passing
erlingrj Jan 13, 2025
841549f
Refactor
erlingrj Jan 13, 2025
d3173cf
Formatting
erlingrj Jan 14, 2025
dd61844
Refactorings
erlingrj Jan 15, 2025
42fc1ad
More docs
erlingrj Jan 15, 2025
9fee19b
CI
erlingrj Jan 15, 2025
97bb16a
Fixes
erlingrj Jan 15, 2025
d56ce51
TcpIp fixes
erlingrj Jan 15, 2025
e2176a2
Revert more tcp stuff
erlingrj Jan 15, 2025
7b49934
Avoid flooding log when a federate closes a socket
erlingrj Jan 15, 2025
6b5e91d
Remove merge mistake
erlingrj Jan 15, 2025
9e79137
Formatting
erlingrj Jan 15, 2025
8d51127
Fix posix federated
erlingrj Jan 15, 2025
b906004
More minor fixes
erlingrj Jan 15, 2025
ef9f980
Format
erlingrj Jan 15, 2025
05d7d87
Minimum event queue of 2
erlingrj Jan 15, 2025
42166cd
Remove some dead code
erlingrj Jan 15, 2025
7d43526
Also close send_failed socketpair on reset
erlingrj Jan 16, 2025
b11b5b8
Generate return 0 in main function
erlingrj Jan 16, 2025
c3d1d74
Only generate launch script when we target native
erlingrj Jan 16, 2025
66c6cc9
Set timeout of 1minute on our LF tests
erlingrj Jan 17, 2025
2b7fb24
Add some info prints
erlingrj Jan 17, 2025
c9e949f
Add was_ever_connected API to network_channel
erlingrj Jan 17, 2025
fac8be9
Fix some warnings in unit-tests
erlingrj Jan 17, 2025
07c23a9
Avoid some unnecessary LF_INFO calls
erlingrj Jan 17, 2025
f2b95fa
Do not timestamp logs for FlexPRET
erlingrj Jan 17, 2025
d87c6a4
Format
erlingrj Jan 17, 2025
1ff975f
Make RIOT compile
LasseRosenow Jan 20, 2025
90f63c5
Fix makefile and missing comma
LasseRosenow Jan 20, 2025
1098b34
Remove build.sh in riot Lf example
erlingrj Jan 21, 2025
7c59706
Use global _lf_environment
erlingrj Jan 21, 2025
2f2d634
Fix missing _lf_environment in test
erlingrj Jan 21, 2025
012e8b3
Dont need environment arg to TcpIp and CoapUdp ctors
erlingrj Jan 21, 2025
f10ed77
Merge branch 'lfc-federated' into lfc-federated-riot-fixes
LasseRosenow Jan 21, 2025
d3717ce
Coap updates
erlingrj Jan 21, 2025
bc2d9a4
Remove stale ref
erlingrj Jan 21, 2025
81fbc1b
Merge branch 'lfc-federated' into lfc-federated-riot-fixes
LasseRosenow Jan 21, 2025
6ea9cad
Fix APPLICATION name
LasseRosenow Jan 21, 2025
000794e
Merge branch 'main' into lfc-federated-riot-fixes
LasseRosenow Jan 21, 2025
e314279
Fix make clean
LasseRosenow Jan 21, 2025
823b923
Add all riot examples to the CI again
LasseRosenow Jan 21, 2025
56c2996
Fix hello_lf example make clean command not working
LasseRosenow Jan 21, 2025
64b6bd4
Rename FEDERATION -> FEDERATE
erlingrj Jan 22, 2025
1d10fd3
Make build scripts more realistic
LasseRosenow Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions examples/riot/blinky/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
make all
26 changes: 8 additions & 18 deletions examples/riot/buildAll.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,12 @@

set -e

# List of folders
FOLDERS=("blinky" "hello" "hello_lf")

# List of boards
BOARDS=("native" "nucleo-f429zi")

# Iterate over each board
for board in "${BOARDS[@]}"; do
# Command to execute in each folder
COMMAND="make BOARD=$board all"

# Iterate over each folder and execute the command
for dir in "${FOLDERS[@]}"; do
echo "Entering $dir"
pushd $dir
$COMMAND
popd
done
# Iterate over each folder and execute the command
for dir in ./*; do
if [ -d $dir ]; then
echo "Entering $dir"
pushd $dir
./build.sh
popd
fi
done
3 changes: 3 additions & 0 deletions examples/riot/coap_federated/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
REMOTE_ADDRESS=fe80::8cc3:33ff:febb:1b3 make all -C ./sender
REMOTE_ADDRESS=fe80::8cc3:33ff:febb:1b3 make all -C ./receiver
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to add build.sh files to all riot examples, because a simple make all does not always work. Especially if you also want to build both federations.

Instead of making the runAll.sh file complicated with different commands per project I let it run build.sh in each folder it can find. This will also prevent the bug that the coap_federated example wasn't build in the CI because I forgot to add it to buildAll.sh

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable. We must find a good way to build federated programs. Your current proposal, with a single Makefile, works now. But I think in the future we want independent projects for each federate because there might several difference between the boards, and their configuration.

Two questions:

  1. Are you now able to pick a IP6 address from the tap interface?
  2. Why do they both specify the same remote?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry github showed me this comment at the bottom and I replied in the main thread and not directly to this comment

35 changes: 35 additions & 0 deletions examples/riot/coap_federated_lf/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
REACTOR_UC_PATH ?= $(CURDIR)/../../../

# The name of the LF application inside "./src" to build/run/flash etc.
LF_MAIN ?= CoapFederatedLF
FEDERATION ?= r1

# Execute the LF compiler if build target is "all"
ifeq ($(firstword $(MAKECMDGOALS)),all)
_ := $(shell $(REACTOR_UC_PATH)/lfc/bin/lfc-dev src/$(LF_MAIN).lf)
endif

# ---- RIOT specific configuration ----
# This has to be the absolute path to the RIOT base directory:
RIOTBASE ?= $(CURDIR)/../../../../RIOT

# If no BOARD is found in the environment, use this default:
BOARD ?= native

# Comment this out to disable code in RIOT that does safety checking
# which is not needed in a production environment but helps in the
# development process:
DEVELHELP ?= 1

# Change this to 0 show compiler invocation lines by default:
QUIET ?= 1

# Enable reactor-uc features
CFLAGS += -DNETWORK_CHANNEL_COAP_RIOT

# Configure CoAP retransmission timeout
CFLAGS += -DCONFIG_GCOAP_NO_RETRANS_BACKOFF=1
CFLAGS += -DCONFIG_COAP_ACK_TIMEOUT_MS=400
CFLAGS += -DCONFIG_COAP_MAX_RETRANSMIT=4

include $(REACTOR_UC_PATH)/make/riot/riot-lfc.mk
3 changes: 3 additions & 0 deletions examples/riot/coap_federated_lf/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
FEDERATION=r1 make all
FEDERATION=r2 make all
41 changes: 41 additions & 0 deletions examples/riot/coap_federated_lf/src/CoapFederatedLF.lf
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
target uC {
platform: RIOT,
timeout: 1sec
}

reactor Src(id: int = 0) {
output out: int
reaction(startup) -> out{=
printf("Hello from Src!\n");
lf_set(out, self->id);
=}
}

reactor Dst {
input in: int
state check: bool = false
reaction(startup) {=
printf("Hello from Dst!\n");
=}
reaction(in) {=
printf("Received %d from Src\n", in->value);
validate(in->value == 42);
self->check = true;
env->request_shutdown(env);
=}

reaction(shutdown) {=
validate(self->check);
=}
}

federated reactor {
@interface_coap(name="if1", address="fe80::44e5:1bff:fee4:dac8")
r1 = new Src(id=42)

@interface_coap(name="if1", address="fe80::8cc3:33ff:febb:1b3")
r2 = new Dst()

@link(left="if1", right="if1")
r1.out -> r2.in
}
2 changes: 2 additions & 0 deletions examples/riot/hello/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
make all
2 changes: 1 addition & 1 deletion examples/riot/hello_lf/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LF_MAIN ?= HelloLF
# CFLAGS += -DNETWORK_CHANNEL_COAP_RIOT

# Execute the LF compiler if build target is "all"
ifeq ($(MAKECMDGOALS),all)
ifeq ($(firstword $(MAKECMDGOALS)),all)
erlingrj marked this conversation as resolved.
Show resolved Hide resolved
_ := $(shell $(REACTOR_UC_PATH)/lfc/bin/lfc-dev src/$(LF_MAIN).lf)
endif

Expand Down
2 changes: 2 additions & 0 deletions examples/riot/hello_lf/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
make all
21 changes: 14 additions & 7 deletions lfc/core/src/main/kotlin/org/lflang/generator/uc/UcIpAddress.kt
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import org.lflang.AttributeUtils.getInterfaceAttributes
import org.lflang.lf.Attribute
import java.math.BigInteger
import java.util.concurrent.atomic.AtomicInteger
import java.net.InetAddress
import java.net.UnknownHostException


/** A class representing an IPAddress, either v4 or v6. */
Expand All @@ -22,10 +24,12 @@ sealed class IPAddress {
}

companion object {
fun isValidIPv4(address: String): Boolean {
val regex = Regex("^([0-9]{1,3}\\.){3}[0-9]{1,3}$")
return regex.matches(address) &&
address.split(".").all { it.toIntOrNull() in 0..255 }
fun isValidIPv4(ip: String): Boolean {
return try {
InetAddress.getByName(ip) is java.net.Inet4Address
} catch (e: UnknownHostException) {
false
}
}
}
}
Expand All @@ -36,9 +40,12 @@ sealed class IPAddress {
}

companion object {
fun isValidIPv6(address: String): Boolean {
val regex = Regex("(([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4})|(::)")
return regex.matches(address)
erlingrj marked this conversation as resolved.
Show resolved Hide resolved
fun isValidIPv6(ip: String): Boolean {
return try {
InetAddress.getByName(ip) is java.net.Inet6Address
} catch (e: UnknownHostException) {
false
}
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ abstract class UcNetworkChannel(
COAP_UDP_IP -> {
val srcEp = (srcIf as UcCoapUdpIpInterface).createEndpoint()
val destEp = (destIf as UcCoapUdpIpInterface).createEndpoint()
channel = UcCoapUdpIpChannel(srcEp, destEp, serverLhs)
channel = UcCoapUdpIpChannel(srcEp, destEp)
}

CUSTOM -> {
Expand Down Expand Up @@ -242,17 +242,24 @@ class UcTcpIpChannel(
class UcCoapUdpIpChannel(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this homogeneous enough that one code-generator class works potentially for all platforms?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will find out when we add support for native and Zephyr

src: UcCoapUdpIpEndpoint,
dest: UcCoapUdpIpEndpoint,
serverLhs: Boolean = true,
) : UcNetworkChannel(COAP_UDP_IP, src, dest, serverLhs) {
private val srcAddr = src.ipAddress.address
private val destAddr = dest.ipAddress.address
// TODO: In CoAP every node is a server and a client => default server to false for now
) : UcNetworkChannel(COAP_UDP_IP, src, dest, false) {
private val srcAddr = src
private val destAddr = dest

private fun getIpProtocolFamily(ip: IPAddress): String {
return when (ip) {
is IPAddress.IPv4 -> "AF_INET"
is IPAddress.IPv6 -> "AF_INET6"
else -> throw IllegalArgumentException("Unknown IP address type")
}
}

override fun generateChannelCtorSrc() =
"CoapUdpIpChannel_ctor(&self->channel, \"${if (serverLhs) srcAddr else destAddr}\", AF_INET, ${serverLhs});"
"CoapUdpIpChannel_ctor(&self->channel, \"${destAddr.ipAddress.address}\", ${getIpProtocolFamily(destAddr.ipAddress)});"

override fun generateChannelCtorDest() =
"CoapUdpIpChannel_ctor(&self->channel, \"${if (serverLhs) srcAddr else destAddr}\" AF_INET, ${!serverLhs});"

"CoapUdpIpChannel_ctor(&self->channel, \"${srcAddr.ipAddress.address}\", ${getIpProtocolFamily(srcAddr.ipAddress)});"

override val codeType: String
get() = "CoapUdpIpChannel"
Expand Down
28 changes: 22 additions & 6 deletions make/riot/riot-lfc.mk
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,32 @@ ifndef LF_MAIN
$(error LF_MAIN is not defined. Please define it!)
endif

# Name of your RIOT application
APPLICATION ?= $(LF_MAIN)
ifndef RIOTBASE
$(error RIOTBASE is not defined. Please define it!)
endif

# Check if this is a federated program
ifdef FEDERATION
# Name of your RIOT application
APPLICATION ?= $(LF_MAIN)-$(FEDERATION)

# Path of generated lf c-code
LF_SRC_GEN_PATH ?= $(CURDIR)/src-gen/$(LF_MAIN)
# Path of generated lf c-code
LF_SRC_GEN_PATH ?= $(CURDIR)/src-gen/$(LF_MAIN)/$(FEDERATION)
else
# Name of your RIOT application
APPLICATION ?= $(LF_MAIN)

# Path of generated lf c-code
LF_SRC_GEN_PATH ?= $(CURDIR)/src-gen/$(LF_MAIN)
endif

# Only include generated files if build target is not "clean"
# In this case the src-gen folder was deleted
ifeq ($(MAKECMDGOALS),clean)
ifeq ($(firstword $(MAKECMDGOALS)),clean)
# Delete src-gen folder if build target is "clean"
_ := $(shell rm -rf $(LF_SRC_GEN_PATH))

include $(RIOTBASE)/Makefile.include
else
# Include the Makefile of the generated target application
include $(LF_SRC_GEN_PATH)/Makefile
Expand All @@ -28,6 +43,7 @@ else

# Include generated h files
CFLAGS += -I$(LF_SRC_GEN_PATH)

include $(RIOT_MK_DIR)/riot.mk
endif

include $(RIOT_MK_DIR)/riot.mk
Loading