Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Update #14

Open
wants to merge 4,355 commits into
base: master
Choose a base branch
from
Open

Test Update #14

wants to merge 4,355 commits into from

Conversation

maxpromer
Copy link
Member

No description provided.

@iPAS
Copy link

iPAS commented Aug 22, 2024

The current version (before the incoming merge) has a bug while trying to compile ports/unix.
I guess this is because of this repo is lacking behind the original which has changed the directory name from 'lib' -> 'extmod', so that AXTLS library includes a file from the wrong path.

I have to edit the code of the AXTLS library:

-------------------------- ssl/os_port_micropython.h --------------------------
index 88697f2..7d10cd9 100644
@@ -75,7 +75,7 @@ extern int mp_stream_errno;

#define TTY_FLUSH()

-#include "../../../extmod/crypto-algorithms/sha256.h"
+#include "../../../lib/crypto-algorithms/sha256.h"

#define SHA256_CTX CRYAL_SHA256_CTX
#define SHA256_Init(a) sha256_init(a)

iabdalkader and others added 29 commits October 25, 2024 01:19
This brings in:
- requests improvements with overriding headers
- use non-u versions of built-in modules, including asyncio
- fix to logging so StreamHandler calls parent constructor
- various fixes to usb-device packages
- fixes to lora sx126x and sx127x drivers
- improvements to unix-ffi/sqlite3
- support additional gap_connect arguments in aioble

Signed-off-by: Damien George <[email protected]>
Signed-off-by: Damien George <[email protected]>
Since `led` is not being rebound inside the `tick` function the standard
Python name resolution method will find it.

Signed-off-by: Steve Holden <[email protected]>
For in-tree builds, these are effectively equivalent. However for
out-of-tree builds it's preferable to have as little as possible in the
top-level CMakeLists.txt file (as the out-of-tree build needs its own
copy).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
The variable `written` was being used before it was defined in the
`fs_writefile()` method of the Transport class.  This was causing an
`UnboundLocalError` to be raised when the `progress_callback` was not
provided.

Fixes issue #16084.

Signed-off-by: Glenn Moloney <[email protected]>
This commit adds an extra bit of parameters validation to the SPI bus
constructor on ESP32.  Passing 0 as the number of bits would trigger a
division by zero error when performing read/write operations on an SPI
bus created in such a fashion.

Fixes issue #5910.

Signed-off-by: Alessandro Gatti <[email protected]>
This commit fixes PWM configuration across C3, C6, S2 and S3 chips, which
was broken by 6d79937.  Without this fix
the PWM frequency is limited to a maximum of 2446Hz (on S2 at least).

Signed-off-by: Andrew Leech <[email protected]>
The PIC16 port didn't catch up with the other ports, so it required a bit
of work to make it build with the latest version of XC16.

Signed-off-by: Alessandro Gatti <[email protected]>
Fixes the problem noted at
#15547 (comment)
which is that, because default CI HEAD for a PR is a (generated) merge
commit into the master branch's current HEAD, then if the PR branch isn't
fully rebased then the commit check runs against commits from master as
well!

Also drops running this check on push, the pull_request event is triggered
by default on open and update ("synchronized" event), which probably covers
the cases where this check should run.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Previously the code size comparison was between the merge base (i.e. where
the PR branched), and the generated merge commit into master.  If the PR
branch was older than current master, this meant the size comparison could
incorrectly include changes already merged on master but missing from the
PR branch.

This commit changes it to compare the generated merge commit against
current master, i.e. the size impact if this PR was to be merged.

This commit also disables running the code size check on "push", it now
only runs on pull_request events.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Descripton of mip usage with micropython port suggest using it like this:

    ./micropython -m mip install --target=third-party pkgname

But it should be called without equal sign:

    ./micropython -m mip install --target third-party pkgname

Signed-off-by: [email protected]
In `deque_subscr()`, if `index_val` equals `self->alloc`, the index
correction `index_val -= self->alloc` does not execute, leading to an
out-of-bounds access in `self->items[index_val]`.

The fix in this commit ensures that the index correction is applied
whenever `index_val >= self->alloc`, preventing access beyond the allocated
buffer size.

Signed-off-by: Jan Sturm <[email protected]>
Previously to this commit, running the test suite on a bare-metal board
required specifying the target (really platform) and device, eg:

    $ ./run-tests.py --target pyboard --device /dev/ttyACM1

That's quite a lot to type, and you also need to know what the target
platform is, when a lot of the time you either don't care or it doesn't
matter.

This commit makes it easier to run the tests by replacing both of these
options with a single `--test-instance` (`-t` for short) option.  That
option specifies the executable/port/device to test.  Then the target
platform is automatically detected.

The `--test-instance` can be passed:
- "unix" (the default) to use the unix version of MicroPython
- "webassembly" to test the webassembly port
- anything else is considered a port/device to pass to Pyboard

There are also some shortcuts to specify a port/device, following
`mpremote`:
- a<n> is short for /dev/ttyACM<n>
- u<n> is short for /dev/ttyUSB<n>
- c<n> is short for COM<n>

For example:

    $ ./run-tests.py -t a1

Note that the default test instance is "unix" and so this commit does not
change the standard way to run tests on the unix port, by just doing
`./run-tests.py`.

As part of this change, the platform (and it's native architecture if it
supports importing native .mpy files) is show at the start of the test run.

Signed-off-by: Damien George <[email protected]>
This was missed in 628abf8.  The the bug
was that, when IPv6 is enabled, the `sizeof(ip_addr_t)` is much larger than
IPv4 size, which is what's needed for IGMP addressing.

Fixes issue #16100.

Signed-off-by: Damien George <[email protected]>
Recent versions of NimBLE (since release 1.6.0) removed this variable; see
apache/mynewt-nimble@7cc8c08.

We never used it except in an assert, so remove those asserts to make the
code compatible with newer NimBLE versions (eg for the esp32 port).

Signed-off-by: Damien George <[email protected]>
These were added to the `network` module but not the `network.WLAN` class,
which is the new home for such constants.

Also:
- Mark the WLAN constants in the `network` module as deprecated, to be
  removed in MicroPython 2.0.
- Move the static assert to the WLAN source code, to be close to where it
  relates to.

Signed-off-by: Damien George <[email protected]>
The cleanup in 548babf relies on some functions not available in older
ESP-IDF. Temporarily restore them, until we drop support for ESP-IDF <5.2.

PWM functionality should end up the same regardless of ESP-IDF version, and
also no different from MicroPython V1.23.

Signed-off-by: Angus Gratton <[email protected]>
Seemingly ESP-IDF incorrectly marks RTC FAST memory region
as MALLOC_CAP_EXEC on ESP32-S2 when it isn't. This memory is
the lowest priority, so it only is returned if D/IRAM is exhausted.

Apply this workaround to treat the allocation as failed if it gives us
non-executable RAM back, rather than crashing.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Removes the deprecated network.[AP|STA]_IF form from the docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Removes the deprecated network.[AP|STA]_IF form from the esp32 port

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Removes the deprecated network.[AP|STA]_IF form from the esp8266 port

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Removes the deprecated network.[AP|STA]_IF form from unit tests.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
dpgeorge and others added 30 commits January 17, 2025 16:58
Otherwise the comment is printed each time the rule is run.

Follow up to fdd606d.

Signed-off-by: Damien George <[email protected]>
This commit moves `<board>.mk` to `<board>/mpconfigboard.mk` for all qemu
boards, making it the same as other bare-metal ports.

Signed-off-by: Damien George <[email protected]>
This commit fixes compilation for the ESP8266 port when using a local
toolchain on relatively recent Linux systems.

The documentation asks the user to delete the esptool instance that
comes with the toolchain, in favour of using the one provided by the
system.  On Linux systems that are at least two years old (looking at
the CI Ubuntu image as an example), the version of esptool installed
with the package manager isn't called `esptool.py` but just `esptool`.
The Makefile didn't take that into account and used `esptool.py` without
checking if such a command exists, making builds fail.

Now preference is given to the `esptool` command, falling back to
`esptool.py` only if the former command does not exist or it is not
available to the current user, to maintain compatibility with old
setups.

Signed-off-by: Alessandro Gatti <[email protected]>
Improvements to DAC support for SAMD51:
- properly validate DAC id
- correctly use dac_init flag, as a 2-ple for A0, A1 channels
- disable DAC before adjusting settings, see SAMD5x data sheet §47.6.2.3

Co-authored-by: robert-hh <[email protected]>
Signed-off-by: Graeme Winter <[email protected]>
The QSPI baud is derived from the AHB clock, not from the APB (peripheral)
clock.

Datasheet: The QSPI Baud rate clock is generated by dividing the module
clock (CLK_QSPI_AHB) by a value between 1 and 255.

As previously implemented, all baudrates are 2.5 times greater than
expected.

Signed-off-by: I. Tomita <[email protected]>
Add WIFI_AUTH_WPA3_ENTERPRISE and WIFI_AUTH_WPA2_WPA3_ENTERPRISE, and
update PPP callback signature for latest lwIP.

Co-authored-by: Daniel van de Giessen <[email protected]>
Signed-off-by: IhorNehrutsa <[email protected]>
The esp32 IDF toolchain can give a "may be used uninitialized" warning, at
least for ESP32-S3 with gcc 14.2.0.  Silence that warning by initializing
the variable with NULL.

Co-authored-by: Daniel van de Giessen <[email protected]>
Signed-off-by: IhorNehrutsa <[email protected]>
The literal is in base 16 but int()'s default radix in CPython is 10, not
0.

Signed-off-by: Jeff Epler <[email protected]>
This includes making int("01") parse in base 10 like standard Python.
When a base of 0 is specified it means auto-detect based on the prefix, and
literals begining with 0 (except when the literal is all 0's) like "01" are
then invalid and now throw an exception.

The new error message is different from CPython. It says e.g.,
`SyntaxError: invalid syntax for integer with base 0: '09'`

Additional test cases were added to cover the changed & added code.

Co-authored-by: Damien George <[email protected]>
Signed-off-by: Jeff Epler <[email protected]>
This commit improves the emitted code sequences for address generation
in the Viper subsystem when loading/storing 16 and 32 bit values via a
register offset.

The Xtensa opcodes ADDX2 and ADDX4 are used to avoid performing the
extra shifts to align the final operation offset.  Those opcodes are
available on both xtensa and xtensawin MicroPython architectures.

Signed-off-by: Alessandro Gatti <[email protected]>
This commit improves the RV32 code sequence that is emitted if a
function needs to set up an exception handler as its prologue.

The old code would clear a temporary register and then copy that value
to places that needed to be initialised with zero values.  On RV32
there's a dedicated register that's hardwired to be equal to zero, which
allows us to bypass the extra register clear and use the zero register
to initialise values.

Signed-off-by: Alessandro Gatti <[email protected]>
This commit introduces the ability to emit optimised code paths on
Xtensa for load/store operations indexed via an immediate offset.

If an immediate offset for a load/store operation is within a certain
range that allows it to be embedded into an available opcode then said
opcode is emitted instead of the generic code sequence.

Signed-off-by: Alessandro Gatti <[email protected]>
If the target does not return any data then `read_until()` will block
indefinitely.  Fix this by making the initial read part of the general read
look, which always checks `inWaiting() > 0` before reading from the serial
device.

Also added the UART timeout to the constructor.  This is not currently used
but may be used as an additional safeguard.

Signed-off-by: Hans Maerki <[email protected]>
And use it in `enter_raw_repl()`.  This prevents waiting forever for a
serial device that does not respond to the Ctrl-C/Ctrl-D/etc commands and
is constantly outputting data.

Signed-off-by: Hans Maerki <[email protected]>
For IPv6 connections, the peer address was previously defined as only the
first four bytes of the IP address. For IPv6 addresses, this resulted in an
incorrect IPv4 address. For instance, receiving a packet via `::recvfrom`
from `'fe80::87:e7ff:fe48:629a'` is returned as having a peer address of
`'254.128.0.0'`

Signed-off-by: Jared Hancock <[email protected]>
- ECHO_SUBMODULES=1 exits CMake early. With idf_component_manager 1.x this
  seems to leave the managed_components directory in a state that causes
  later builds to fail.

- Looks like the component manager isn't needed for this step, so disable
  it. This invocation logs a warning (not visible in normal output) but
  completes successfully and returns the correct list of submodules.

Signed-off-by: Angus Gratton <[email protected]>
Similar to other places, CMake will error out if this file
doesn't exist yet but we don't want this if we're only getting
the list of submodules.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
The way CMake gathers the submodule list, it can quietly be empty
if the previous step fails. This makes it an explicit error.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
This lwIP configuration file has options that are common to all ports, and
the ports are updated to use this file.  This change is a no-op, the lwIP
configuration remains the same for the four ports using this common file.

This reduces code duplication, keeps the ports in sync, and makes it easier
to update the configuration for all ports at once.

Signed-off-by: Damien George <[email protected]>
Follow up to 13b13d1, based on some
testing on godbolt, the manual code optimisation seems unnecessary for code
size, at least on gcc x86_64 and ARM, and it's definitely not good for
clarity.

Signed-off-by: Jeff Epler <[email protected]>
Test is for an issue reported on the micropython-lib Discord as
effecting the rp2 port umqtt.simple interface when reconnecting with TLS,
however it's a more generic problem.

Currently this test fails on RPI_PICO_W and ESP32_GENERIC_C3 (and no doubt
others). Fixes are in the subsequent commits.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
On mbedTLS ports with non-baremetal configs (mostly esp32, technically also
unix port), mbedTLS memory is allocated from the libc heap. This means an
old SSL socket may be holding large SSL buffers and preventing a new SSL
socket from being allocated.

As a workaround, trigger a GC pass and retry before failing outright.

This was originally implemented as a global mbedTLS calloc function, but
there is complexity around the possibility of C user modules calling into
mbedTLS without holding the GIL. It would be interesting to try making a
generic version for any malloc which fails, but this would require checking
for a Python thread and probably making the GIL recursive.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
This allows coverage to test MicroPython-specific features such as
the tracked alloc cleanups added in the parent commit.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Currently a finalizer may run and access memory which has already been
freed. (This happens mostly during gc_sweep_all() but could happen during
any garbage collection pass.)

Includes some speed improvement tweaks to skip empty FTB blocks. These help
compensate for the inherent slowdown of having to walk the heap twice.

Signed-off-by: Angus Gratton <[email protected]>
Do this by tracking being inside gc collection with a
separate flag, GC_COLLECT_FLAG. In gc_free(),
ignore this flag when determining if the heap is locked.

* For finalisers calling gc_free() when heap is otherwise unlocked,
  this allows memory to be immediately freed (potentially
  avoiding a MemoryError).

* Hard IRQs still can't call gc_free(), as heap will be locked via
  gc_lock().

* If finalisers are disabled then all of this code can be compiled
  out to save some code size.

Signed-off-by: Angus Gratton <[email protected]>
Enabled by default if using threading and no GIL

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Necessary for GC support, also refactored pendsv usage.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Allows refactoring the existing thread_mutex atomic section support to use
the new recursive mutex type.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
- Renamed gc_sweep to gc_sweep_free_blocks.
- Call gc_sweep_run_finalisers from top level.
- Reordered the gc static functions to be in approximate
  runtime sequence (with forward declarations) rather than
  in declaration order.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.