diff --git a/docs/freezing.rst b/docs/freezing.rst new file mode 100644 index 0000000..0a85bc2 --- /dev/null +++ b/docs/freezing.rst @@ -0,0 +1,110 @@ +Cross-Compiling and Freezing Microdot (MicroPython Only) +-------------------------------------------------------- + +Microdot is a fairly small framework, so its size is not something you need to +be concerned about unless you are working with MicroPython on hardware with a +very small amount of disk space and/or RAM. In such cases every byte counts, so +this section provides some recommendations on how to keep Microdot's footprint +as small as possible. + +Choosing What Modules to Install +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Microdot has a modular design that allows you to only install the modules that +your application needs. + +For minimal web application support based on the core Microdot web server +without extensions, you can just copy `microdot.py `_ +to the source directory on your device. The core Microdot web server does not +have any dependencies, so you don't need to install anything else. + +If your application uses some of the provided extensions to the core web +server, then instead of installing *microdot.py* you'll need to create a +*microdot* subdirectory and install the following files in it: + +- `__init__.py `_ +- `microdot.py `_ +- Any extension modules that you need from the `microdot `_ source directory. + +Some of the extensions also have dependencies of their own, so you may need to +install those in your device as well (outside of the ``microdot`` +subdirectory). Consult the documentation of each extension to learn if any +third-party dependencies are required. + +Cross-Compiling +~~~~~~~~~~~~~~~ + +An issue that is common with low-end microcontroller boards is that they do not +have enough RAM for the MicroPython compiler to compile the source files, but +once the code is compiled they are able to run it without problems. + +To address this, MicroPython allows you to cross-compile source files on your +desktop or laptop computer and then upload their compiled versions to the +device. A good strategy is to cross-compile all the dependencies that are used +by your application, since these are not going to be updated very often. If the +goal is to minimize the use of RAM, you can also opt to cross-compile your +application source files. + +The MicroPython cross-compiler is available as a package that you can install +on standard Python. You must determine the version of MicroPython that you will +be running on your device, and install the compiler that matches that version. +For example, if you plan to use MicroPython 1.21.0 on your device, you can +install the cross-compiler for this version with the following command:: + + pip install mpy-cross==1.21.0 + +Then run the cross-compiler for each source file that you want to compile. +Since the cross-compilation happens on your computer, you will need to have +copies of all the source files you need to compile locally on your disk. Here +is how you can compile the *microdot.py* file, assuming you have a copy in the +current directory in your computer:: + + mpy-cross microdot.py + +The cross-compiler will create a file with the same name as the source file, +but with the extension changed to *.mpy*. + +Once you have all your dependencies compiled, you can replace the *.py* files +in your device with their corresponding *.mpy* versions. MicroPython +automatically recognizes *.mpy* files, so there is no need to make any changes +to any source code to start using compiled files. + +Freezing +~~~~~~~~ + +The ultimate option to reduce the size of a MicroPython application is to +"freeze" it. Freezing is a process that takes MicroPython source code (either +dependencies, application code or both), pre-compiles it and incorporates it +into a custom-built MicroPython firmware that is flashed to the device. + +Freezing MicroPython modules to firmware has the advantage that the code is +imported directly from the device's ROM, leaving more RAM available for +application use. + +The process to create a custom firmware is unfortunately non-trivial and +different depending on the device, so you will need to consult the MicroPython +documentation that applies to your device to learn how to do this. + +The part of the process that is common to all devices is the creation of a +`manifest file `_ +to tell the MicroPython firmware builder which packages and modules to freeze. + +For a minimal installation of Microdot consisting only in its *microdot.py* +source file, the manifest file that you need use to build the firmware must +include the following declaration:: + + module('microdot') + +If instead you are working with a version of Microdot that includes some or all +of its extensions, then the manifest file must reference the ``microdot`` +package plus any third-party dependencies that are needed. Below is a manifest +file for a complete Microdot installation that includes all the extensions:: + + package('microdot') + package('utemplate') # required only if templates are used + module('pyjwt') # required only if user sessions are used + +In this example, the *microdot* and *utemplate* packages must be available in +the directory where the manifest file is located so that the MicroPython build +can find them. The `pyjwt` module is part of the MicroPython standard library +and will be downloaded as part of the build. diff --git a/docs/index.rst b/docs/index.rst index c61486d..aaddcd0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -18,6 +18,8 @@ systems with limited resources such as microcontrollers. Both standard Python intro extensions + migrating + freezing api * :ref:`genindex` diff --git a/docs/intro.rst b/docs/intro.rst index dc4ffc5..ba9cde9 100644 --- a/docs/intro.rst +++ b/docs/intro.rst @@ -867,147 +867,3 @@ Given that most microcontroller boards implementing MicroPython do not have threading support, ``def`` handler functions in these platforms run in the main thread, and will block the asynchronous loop when they take too long to complete. - -Migrating to Microdot 2.x from Older Releases -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Version 2 of Microdot incorporates feedback received from users of earlier -releases, and attempts to improve and correct some design decisions that have -proven to be problematic. - -For this reason most applications built for earlier versions will need to be -updated to work correctly with Microdot 2. This section describes the backwards -incompatible changes that were made. - -Code reorganization -^^^^^^^^^^^^^^^^^^^ - -The Microdot source code has been moved into a ``microdot`` package, elminating -the need for each extension to be named *microdot_.py*. - -As a result of this change, all extensions have been renamed to shorter names, -without the ``microdot_`` prefix. For example, the *microdot_cors.py* module is -now called *cors.py*. - -This change affects the way extensions are imported. Instead of this:: - - from microdot_cors import CORS - -the import statement should be:: - - from microdot.cors import CORS - -No more synchronous web server -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In earlier releases of Microdot the core web server was built on synchronous -Python, and asynchronous support was enabled with the asyncio extension. - -Microdot 2 eliminates the synchronous web server, and implements the core -server logic directly with asyncio, eliminating the need for an asyncio -extension. - -Any applications built using the asyncio extension will need to update their -imports from this:: - - from microdot.asyncio import Microdot - -to this:: - - from microdot import Microdot - -Applications that were built using the synchronous web server do not need to -change their imports, but will now work asynchronously. Review the -:ref:`Concurrency` section to learn about the potential issues when using -``def`` function handlers, and the benefits of transitioning to ``async def`` -handlers. - -Removed extensions -^^^^^^^^^^^^^^^^^^ - -Some extensions became unnecessary and have been removed or merged with other -extensions: - -- *microdot_asyncio.py*: this is now the core web server. -- *microdot_asyncio_websocket.py*: this is now the main WebSocket extension. -- *microdot_asyncio_test_client.py*: this is now the main test client - extension. -- *microdot_asgi_websocket.py*: the functionality in this extension is now - available in the ASGI extension. -- *microdot_ssl.py*: this extension was only used with the synchronous web - server, so it is not needed anymore. -- *microdot_websocket_alt.py*: this extension was only used with the - synchronous web server, so it is not needed anymore. - -No more ``render_template()`` function -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The Jinja and uTemplate extensions have been redesigned to work better under -the asynchronous engine, and as a result, the ``render_template()`` function -has been eliminated. - -Instead of this:: - - return render_template('index.html', title='Home') - -use this:: - - return Template('index.html').render(title='Home') - -As a result of this change, it is now possible to use asynchronous rendering:: - - return await Template('index.html').render_async(title='Home') - -Also thanks to this redesign, the template can be streamed instead of returned -as a single string:: - - return Template('index.html').generate(title='Home') - -Streamed templates also have an asynchronous version:: - - return await Template('index.html').generate_async(title='Home') - -Class-based sessions -^^^^^^^^^^^^^^^^^^^^ - -The session extension has been completely redesigned. To initialize session -support for the application, create a ``Session`` object:: - - app = Microdot() - Session(app, secret_key='top-secret!') - -The ``@with_session`` decorator is used to include the session in a request:: - - @app.get('/') - @with_session - async def index(request, session): - # ... - -The ``session`` can be used as a dictionary to retrieve or change the session. -To save the session when it has been modified, call its ``save()`` method:: - - @app.get('/') - @with_session - async def index(request, session): - # ... - session.save() - return 'OK' - -To delete the session, call its ``delete()`` method before returning from the -request. - -WSGI extension redesign -^^^^^^^^^^^^^^^^^^^^^^^ - -Given that the synchronous web server has been removed, the WSGI extension has -been redesigned to work as a synchronous wrapper for the asynchronous web -server. - -Applications using the WSGI extension continue to run under an asynchronous -loop and should try to use the recommended ``async def`` handlers, but can be -deployed with standard WSGI servers such as Gunicorn. - -WebSocket support when using the WSGI extension is enabled when using a -compatible web server. At this time only Gunicorn is supported for WebSocket. - -As before, the WSGI extension is not available under MicroPython. diff --git a/docs/migrating.rst b/docs/migrating.rst new file mode 100644 index 0000000..e12855a --- /dev/null +++ b/docs/migrating.rst @@ -0,0 +1,142 @@ +Migrating to Microdot 2.x from Older Releases +--------------------------------------------- + +Version 2 of Microdot incorporates feedback received from users of earlier +releases, and attempts to improve and correct some design decisions that have +proven to be problematic. + +For this reason most applications built for earlier versions will need to be +updated to work correctly with Microdot 2. This section describes the backwards +incompatible changes that were made. + +Code reorganization +~~~~~~~~~~~~~~~~~~~ + +The Microdot source code has been moved into a ``microdot`` package, +eliminating the need for each extension to be named with a *microdot_* prefix. + +As a result of this change, all extensions have been renamed to shorter names. +For example, the *microdot_cors.py* module is now called *cors.py*. + +This change affects the way extensions are imported. Instead of this:: + + from microdot_cors import CORS + +the import statement should be:: + + from microdot.cors import CORS + +No more synchronous web server +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In earlier releases of Microdot the core web server was built on synchronous +Python, and asynchronous support was enabled with the asyncio extension. + +Microdot 2 eliminates the synchronous web server, and implements the core +server logic directly with asyncio, eliminating the need for an asyncio +extension. + +Any applications built using the asyncio extension will need to update their +imports from this:: + + from microdot.asyncio import Microdot + +to this:: + + from microdot import Microdot + +Applications that were built using the synchronous web server do not need to +change their imports, but will now work asynchronously. Review the +:ref:`Concurrency` section to learn about the potential issues when using +``def`` function handlers, and the benefits of transitioning to ``async def`` +handlers. + +Removed extensions +~~~~~~~~~~~~~~~~~~ + +Some extensions became unnecessary and have been removed or merged with other +extensions: + +- *microdot_asyncio.py*: this is now the core web server. +- *microdot_asyncio_websocket.py*: this is now the main WebSocket extension. +- *microdot_asyncio_test_client.py*: this is now the main test client + extension. +- *microdot_asgi_websocket.py*: the functionality in this extension is now + available in the ASGI extension. +- *microdot_ssl.py*: this extension was only used with the synchronous web + server, so it is not needed anymore. +- *microdot_websocket_alt.py*: this extension was only used with the + synchronous web server, so it is not needed anymore. + +No more ``render_template()`` function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Jinja and uTemplate extensions have been redesigned to work better under +the asynchronous engine, and as a result, the ``render_template()`` function +has been eliminated. + +Instead of this:: + + return render_template('index.html', title='Home') + +use this:: + + return Template('index.html').render(title='Home') + +As a result of this change, it is now possible to use asynchronous rendering:: + + return await Template('index.html').render_async(title='Home') + +Also thanks to this redesign, the template can be streamed instead of returned +as a single string:: + + return Template('index.html').generate(title='Home') + +Streamed templates also have an asynchronous version:: + + return await Template('index.html').generate_async(title='Home') + +Class-based user sessions +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The session extension has been completely redesigned. To initialize session +support for the application, create a ``Session`` object:: + + app = Microdot() + Session(app, secret_key='top-secret!') + +The ``@with_session`` decorator is used to include the session in a request:: + + @app.get('/') + @with_session + async def index(request, session): + # ... + +The ``session`` can be used as a dictionary to retrieve or change the session. +To save the session when it has been modified, call its ``save()`` method:: + + @app.get('/') + @with_session + async def index(request, session): + # ... + session.save() + return 'OK' + +To delete the session, call its ``delete()`` method before returning from the +request. + +WSGI extension redesign +~~~~~~~~~~~~~~~~~~~~~~~ + +Given that the synchronous web server has been removed, the WSGI extension has +been redesigned to work as a synchronous wrapper for the asynchronous web +server. + +Applications using the WSGI extension continue to run under an asynchronous +loop and should try to use the recommended ``async def`` handlers, but can be +deployed with standard WSGI servers such as Gunicorn. + +WebSocket support when using the WSGI extension is enabled when using a +compatible web server. At this time only Gunicorn is supported for WebSocket. + +As before, the WSGI extension is not available under MicroPython.