WIP: after merge; before testing

15 files changed, 440 insertions, 60 deletions

diff --git a/docs/library/_thread.rst b/docs/library/_thread.rst
new file mode 100644
index 000000000..47c1c2392
--- /dev/null
+++ b/docs/library/_thread.rst
@@ -0,0 +1,12 @@
+:mod:`_thread` -- multithreading support
+========================================
+
+.. module:: _thread
+   :synopsis: multithreading support
+
+|see_cpython_module| :mod:`python:_thread`.
+
+This module implements multithreading support.
+
+This module is highly experimental and its API is not yet fully settled
+and not yet described in this documentation.
diff --git a/docs/library/array.rst b/docs/library/array.rst
index 0f1310b68..859b370ea 100644
--- a/docs/library/array.rst
+++ b/docs/library/array.rst
@@ -16,14 +16,14 @@ Classes
 .. class:: array.array(typecode, [iterable])
 
     Create array with elements of given type. Initial contents of the
-    array are given by an `iterable`. If it is not provided, an empty
+    array are given by *iterable*. If it is not provided, an empty
     array is created.
 
     .. method:: append(val)
 
-        Append new element to the end of array, growing it.
+        Append new element *val* to the end of array, growing it.
 
     .. method:: extend(iterable)
 
-        Append new elements as contained in an iterable to the end of
+        Append new elements as contained in *iterable* to the end of
         array, growing it.
diff --git a/docs/library/btree.rst b/docs/library/btree.rst
index 303a936ad..3059da7a4 100644
--- a/docs/library/btree.rst
+++ b/docs/library/btree.rst
@@ -7,7 +7,7 @@
    :synopsis: simple BTree database
 
 The ``btree`` module implements a simple key-value database using external
-storage (disk files, or in general case, a random-access stream). Keys are
+storage (disk files, or in general case, a random-access `stream`). Keys are
 stored sorted in the database, and besides efficient retrieval by a key
 value, a database also supports efficient ordered range scans (retrieval
 of values with the keys in a given range). On the application interface
diff --git a/docs/library/framebuf.rst b/docs/library/framebuf.rst
index 80961b872..3117525dd 100644
--- a/docs/library/framebuf.rst
+++ b/docs/library/framebuf.rst
@@ -150,6 +150,14 @@ Constants
 
     Red Green Blue (16-bit, 5+6+5) color format
 
+.. data:: framebuf.GS2_HMSB
+
+    Grayscale (2-bit) color format
+
 .. data:: framebuf.GS4_HMSB
 
     Grayscale (4-bit) color format
+
+.. data:: framebuf.GS8
+
+    Grayscale (8-bit) color format
diff --git a/docs/library/micropython.rst b/docs/library/micropython.rst
index 59e1886fa..0ba3fbbc5 100644
--- a/docs/library/micropython.rst
+++ b/docs/library/micropython.rst
@@ -35,6 +35,18 @@ Functions
    compilation of scripts, and returns ``None``.  Otherwise it returns the current
    optimisation level.
 
+   The optimisation level controls the following compilation features:
+
+   - Assertions: at level 0 assertion statements are enabled and compiled into the
+     bytecode; at levels 1 and higher assertions are not compiled.
+   - Built-in ``__debug__`` variable: at level 0 this variable expands to ``True``;
+     at levels 1 and higher it expands to ``False``.
+   - Source-code line numbers: at levels 0, 1 and 2 source-code line number are
+     stored along with the bytecode so that exceptions can report the line number
+     they occurred at; at levels 3 and higher line numbers are not stored.
+
+   The default optimisation level is usually level 0.
+
 .. function:: alloc_emergency_exception_buf(size)
 
    Allocate *size* bytes of RAM for the emergency exception buffer (a good
@@ -114,5 +126,14 @@ Functions
    the heap may be locked) and scheduling a function to call later will lift
    those restrictions.
 
-   There is a finite stack to hold the scheduled functions and `schedule`
+   Note: If `schedule()` is called from a preempting IRQ, when memory
+   allocation is not allowed and the callback to be passed to `schedule()` is
+   a bound method, passing this directly will fail. This is because creating a
+   reference to a bound method causes memory allocation. A solution is to
+   create a reference to the method in the class constructor and to pass that
+   reference to `schedule()`. This is discussed in detail here
+   :ref:`reference documentation <isr_rules>` under "Creation of Python
+   objects".
+
+   There is a finite stack to hold the scheduled functions and `schedule()`
    will raise a `RuntimeError` if the stack is full.
diff --git a/docs/library/sys.rst b/docs/library/sys.rst
index bb92850b8..c43218509 100644
--- a/docs/library/sys.rst
+++ b/docs/library/sys.rst
@@ -105,15 +105,15 @@ Constants
 
 .. data:: stderr
 
-   Standard error stream.
+   Standard error `stream`.
 
 .. data:: stdin
 
-   Standard input stream.
+   Standard input `stream`.
 
 .. data:: stdout
 
-   Standard output stream.
+   Standard output `stream`.
 
 .. data:: version
 
diff --git a/docs/library/uctypes.rst b/docs/library/uctypes.rst
index 55de9a0a2..f71b00c1e 100644
--- a/docs/library/uctypes.rst
+++ b/docs/library/uctypes.rst
@@ -10,7 +10,7 @@ This module implements "foreign data interface" for MicroPython. The idea
 behind it is similar to CPython's ``ctypes`` modules, but the actual API is
 different, streamlined and optimized for small size. The basic idea of the
 module is to define data structure layout with about the same power as the
-C language allows, and the access it using familiar dot-syntax to reference
+C language allows, and then access it using familiar dot-syntax to reference
 sub-fields.
 
 .. seealso::
@@ -31,25 +31,25 @@ Following are encoding examples for various field types:
 
 * Scalar types::
 
-    "field_name": uctypes.UINT32 | 0
+    "field_name": offset | uctypes.UINT32
 
   in other words, value is scalar type identifier ORed with field offset
   (in bytes) from the start of the structure.
 
 * Recursive structures::
 
-    "sub": (2, {
-        "b0": uctypes.UINT8 | 0,
-        "b1": uctypes.UINT8 | 1,
+    "sub": (offset, {
+        "b0": 0 | uctypes.UINT8,
+        "b1": 1 | uctypes.UINT8,
     })
 
   i.e. value is a 2-tuple, first element of which is offset, and second is
   a structure descriptor dictionary (note: offsets in recursive descriptors
-  are relative to a structure it defines).
+  are relative to the structure it defines).
 
 * Arrays of primitive types::
 
-      "arr": (uctypes.ARRAY | 0, uctypes.UINT8 | 2),
+      "arr": (offset | uctypes.ARRAY, size | uctypes.UINT8),
 
   i.e. value is a 2-tuple, first element of which is ARRAY flag ORed
   with offset, and second is scalar element type ORed number of elements
@@ -57,7 +57,7 @@ Following are encoding examples for various field types:
 
 * Arrays of aggregate types::
 
-    "arr2": (uctypes.ARRAY | 0, 2, {"b": uctypes.UINT8 | 0}),
+    "arr2": (offset | uctypes.ARRAY, size, {"b": 0 | uctypes.UINT8}),
 
   i.e. value is a 3-tuple, first element of which is ARRAY flag ORed
   with offset, second is a number of elements in array, and third is
@@ -65,21 +65,21 @@ Following are encoding examples for various field types:
 
 * Pointer to a primitive type::
 
-    "ptr": (uctypes.PTR | 0, uctypes.UINT8),
+    "ptr": (offset | uctypes.PTR, uctypes.UINT8),
 
   i.e. value is a 2-tuple, first element of which is PTR flag ORed
   with offset, and second is scalar element type.
 
 * Pointer to an aggregate type::
 
-    "ptr2": (uctypes.PTR | 0, {"b": uctypes.UINT8 | 0}),
+    "ptr2": (offset | uctypes.PTR, {"b": 0 | uctypes.UINT8}),
 
   i.e. value is a 2-tuple, first element of which is PTR flag ORed
   with offset, second is descriptor of type pointed to.
 
 * Bitfields::
 
-    "bitf0": uctypes.BFUINT16 | 0 | 0 << uctypes.BF_POS | 8 << uctypes.BF_LEN,
+    "bitf0": offset | uctypes.BFUINT16 | lsbit << uctypes.BF_POS | bitsize << uctypes.BF_LEN,
 
   i.e. value is type of scalar value containing given bitfield (typenames are
   similar to scalar types, but prefixes with "BF"), ORed with offset for
@@ -88,20 +88,21 @@ Following are encoding examples for various field types:
   BF_POS and BF_LEN positions, respectively. Bitfield position is counted
   from the least significant bit, and is the number of right-most bit of a
   field (in other words, it's a number of bits a scalar needs to be shifted
-  right to extra the bitfield).
+  right to extract the bitfield).
 
-  In the example above, first UINT16 value will be extracted at offset 0
+  In the example above, first a UINT16 value will be extracted at offset 0
   (this detail may be important when accessing hardware registers, where
   particular access size and alignment are required), and then bitfield
-  whose rightmost bit is least-significant bit of this UINT16, and length
-  is 8 bits, will be extracted - effectively, this will access
-  least-significant byte of UINT16.
+  whose rightmost bit is *lsbit* bit of this UINT16, and length
+  is *bitsize* bits, will be extracted. For example, if *lsbit* is 0 and
+  *bitsize* is 8, then effectively it will access least-significant byte
+  of UINT16.
 
   Note that bitfield operations are independent of target byte endianness,
   in particular, example above will access least-significant byte of UINT16
   in both little- and big-endian structures. But it depends on the least
   significant bit being numbered 0. Some targets may use different
-  numbering in their native ABI, but ``uctypes`` always uses normalized
+  numbering in their native ABI, but ``uctypes`` always uses the normalized
   numbering described above.
 
 Module contents
diff --git a/docs/library/uio.rst b/docs/library/uio.rst
index 5ae8b9ecb..d1f7c111f 100644
--- a/docs/library/uio.rst
+++ b/docs/library/uio.rst
@@ -8,7 +8,7 @@
 
 |see_cpython_module| :mod:`cpython:io`.
 
-This module contains additional types of stream (file-like) objects
+This module contains additional types of ``stream`` (file-like) objects
 and helper functions.
 
 Conceptual hierarchy
diff --git a/docs/library/ujson.rst b/docs/library/ujson.rst
index 080ee2d03..ef9c70c25 100644
--- a/docs/library/ujson.rst
+++ b/docs/library/ujson.rst
@@ -14,11 +14,24 @@ data format.
 Functions
 ---------
 
+.. function:: dump(obj, stream)
+
+   Serialise *obj* to a JSON string, writing it to the given *stream*.
+
 .. function:: dumps(obj)
 
-   Return ``obj`` represented as a JSON string.
+   Return *obj* represented as a JSON string.
+
+.. function:: load(stream)
+
+   Parse the given *stream*, interpreting it as a JSON string and
+   deserialising the data to a Python object.  The resulting object is
+   returned.
+
+   Parsing continues until end-of-file is encountered.
+   A :exc:`ValueError` is raised if the data in *stream* is not correctly formed.
 
 .. function:: loads(str)
 
-   Parse the JSON ``str`` and return an object.  Raises ValueError if the
+   Parse the JSON *str* and return an object.  Raises :exc:`ValueError` if the
    string is not correctly formed.
diff --git a/docs/library/ure.rst b/docs/library/ure.rst
index c6457de9a..2447de121 100644
--- a/docs/library/ure.rst
+++ b/docs/library/ure.rst
@@ -17,8 +17,9 @@ Supported operators are:
 ``'.'``
    Match any character.
 
-``'[]'``
-   Match set of characters. Individual characters and ranges are supported.
+``'[...]'``
+   Match set of characters. Individual characters and ranges are supported,
+   including negated sets (e.g. ``[^a-c]``).
 
 ``'^'``
 
@@ -38,18 +39,19 @@ Supported operators are:
 
 ``'|'``
 
-``'()'``
+``'(...)'``
    Grouping. Each group is capturing (a substring it captures can be accessed
    with `match.group()` method).
 
-Counted repetitions (``{m,n}``), more advanced assertions, named groups,
-etc. are not supported.
+**NOT SUPPORTED**: Counted repetitions (``{m,n}``), more advanced assertions
+(``\b``, ``\B``), named groups (``(?P<name>...)``), non-capturing groups
+(``(?:...)``), etc.
 
 
 Functions
 ---------
 
-.. function:: compile(regex_str)
+.. function:: compile(regex_str, [flags])
 
    Compile regular expression, return `regex <regex>` object.
 
@@ -67,6 +69,7 @@ Functions
 .. data:: DEBUG
 
    Flag value, display debug information about compiled expression.
+   (Availability depends on `MicroPython port`.)
 
 
 .. _regex:
diff --git a/docs/library/uselect.rst b/docs/library/uselect.rst
index 685f5b318..fb365facd 100644
--- a/docs/library/uselect.rst
+++ b/docs/library/uselect.rst
@@ -9,7 +9,7 @@
 |see_cpython_module| :mod:`cpython:select`.
 
 This module provides functions to efficiently wait for events on multiple
-streams (select streams which are ready for operations).
+`streams <stream>` (select streams which are ready for operations).
 
 Functions
 ---------
@@ -35,14 +35,17 @@ Methods
 
 .. method:: poll.register(obj[, eventmask])
 
-   Register *obj* for polling. *eventmask* is logical OR of:
+   Register `stream` *obj* for polling. *eventmask* is logical OR of:
 
-   * ``select.POLLIN``  - data available for reading
-   * ``select.POLLOUT`` - more data can be written
-   * ``select.POLLERR`` - error occurred
-   * ``select.POLLHUP`` - end of stream/connection termination detected
+   * ``uselect.POLLIN``  - data available for reading
+   * ``uselect.POLLOUT`` - more data can be written
 
-   *eventmask* defaults to ``select.POLLIN | select.POLLOUT``.
+   Note that flags like ``uselect.POLLHUP`` and ``uselect.POLLERR`` are
+   *not* valid as input eventmask (these are unsolicited events which
+   will be returned from `poll()` regardless of whether they are asked
+   for). This semantics is per POSIX.
+
+   *eventmask* defaults to ``uselect.POLLIN | uselect.POLLOUT``.
 
 .. method:: poll.unregister(obj)
 
@@ -52,16 +55,23 @@ Methods
 
    Modify the *eventmask* for *obj*.
 
-.. method:: poll.poll([timeout])
+.. method:: poll.poll(timeout=-1)
+
+   Wait for at least one of the registered objects to become ready or have an
+   exceptional condition, with optional timeout in milliseconds (if *timeout*
+   arg is not specified or -1, there is no timeout).
 
-   Wait for at least one of the registered objects to become ready. Returns
-   list of (``obj``, ``event``, ...) tuples, ``event`` element specifies
-   which events happened with a stream and is a combination of ``select.POLL*``
-   constants described above. There may be other elements in tuple, depending
-   on a platform and version, so don't assume that its size is 2. In case of
-   timeout, an empty list is returned.
+   Returns list of (``obj``, ``event``, ...) tuples. There may be other elements in
+   tuple, depending on a platform and version, so don't assume that its size is 2.
+   The ``event`` element specifies which events happened with a stream and
+   is a combination of ``uselect.POLL*`` constants described above. Note that
+   flags ``uselect.POLLHUP`` and ``uselect.POLLERR`` can be returned at any time
+   (even if were not asked for), and must be acted on accordingly (the
+   corresponding stream unregistered from poll and likely closed), because
+   otherwise all further invocations of `poll()` may return immediately with
+   these flags set for this stream again.
 
-   Timeout is in milliseconds.
+   In case of timeout, an empty list is returned.
 
    .. admonition:: Difference to CPython
       :class: attention
@@ -70,15 +80,15 @@ Methods
 
 .. method:: poll.ipoll(timeout=-1, flags=0)
 
-   Like :meth:`poll.poll`, but instead returns an iterator which yields
+   Like :meth:`poll.poll`, but instead returns an iterator which yields a
    ``callee-owned tuples``. This function provides efficient, allocation-free
    way to poll on streams.
 
    If *flags* is 1, one-shot behavior for events is employed: streams for
-   which events happened, event mask will be automatically reset (equivalent
-   to ``poll.modify(obj, 0)``), so new events for such a stream won't be
-   processed until new mask is set with `poll.modify()`. This behavior is
-   useful for asynchronous I/O schedulers.
+   which events happened will have their event masks automatically reset
+   (equivalent to ``poll.modify(obj, 0)``), so new events for such a stream
+   won't be processed until new mask is set with `poll.modify()`. This
+   behavior is useful for asynchronous I/O schedulers.
 
    .. admonition:: Difference to CPython
       :class: attention
diff --git a/docs/library/usocket.rst b/docs/library/usocket.rst
index b07ba4413..582742f2e 100644
--- a/docs/library/usocket.rst
+++ b/docs/library/usocket.rst
@@ -14,7 +14,7 @@ This module provides access to the BSD socket interface.
 .. admonition:: Difference to CPython
    :class: attention
 
-   For efficiency and consistency, socket objects in MicroPython implement a stream
+   For efficiency and consistency, socket objects in MicroPython implement a `stream`
    (file-like) interface directly. In CPython, you need to convert a socket to
    a file-like object using `makefile()` method. This method is still supported
    by MicroPython (but is a no-op), so where compatibility with CPython matters,
@@ -245,7 +245,7 @@ Methods
    Not every ``MicroPython port`` supports this method. A more portable and
    generic solution is to use `uselect.poll` object. This allows to wait on
    multiple objects at the same time (and not just on sockets, but on generic
-   stream objects which support polling). Example::
+   `stream` objects which support polling). Example::
 
         # Instead of:
         s.settimeout(1.0)  # time in seconds
diff --git a/docs/library/ussl.rst b/docs/library/ussl.rst
index 34db3151d..5df2e5b53 100644
--- a/docs/library/ussl.rst
+++ b/docs/library/ussl.rst
@@ -17,13 +17,13 @@ Functions
 
 .. function:: ussl.wrap_socket(sock, server_side=False, keyfile=None, certfile=None, cert_reqs=CERT_NONE, ca_certs=None)
 
-   Takes a stream *sock* (usually usocket.socket instance of ``SOCK_STREAM`` type),
+   Takes a `stream` *sock* (usually usocket.socket instance of ``SOCK_STREAM`` type),
    and returns an instance of ssl.SSLSocket, which wraps the underlying stream in
-   an SSL context. Returned object has the usual stream interface methods like
+   an SSL context. Returned object has the usual `stream` interface methods like
    ``read()``, ``write()``, etc. In MicroPython, the returned object does not expose
    socket interface and methods like ``recv()``, ``send()``. In particular, a
    server-side SSL socket should be created from a normal socket returned from
-   `accept()` on a non-SSL listening server socket.
+   :meth:`~usocket.socket.accept()` on a non-SSL listening server socket.
 
    Depending on the underlying module implementation in a particular
    ``MicroPython port``, some or all keyword arguments above may be not supported.
diff --git a/docs/library/uzlib.rst b/docs/library/uzlib.rst
index 2a2d9668c..f736ace81 100644
--- a/docs/library/uzlib.rst
+++ b/docs/library/uzlib.rst
@@ -27,7 +27,7 @@ Functions
 
 .. class:: DecompIO(stream, wbits=0)
 
-   Create a stream wrapper which allows transparent decompression of
+   Create a `stream` wrapper which allows transparent decompression of
    compressed data in another *stream*. This allows to process compressed
    streams with data larger than available heap size. In addition to
    values described in :func:`decompress`, *wbits* may take values
diff --git a/docs/reference/packages.rst b/docs/reference/packages.rst
new file mode 100644
index 000000000..8be2461c2
--- /dev/null
+++ b/docs/reference/packages.rst
@@ -0,0 +1,312 @@
+Distribution packages, package management, and deploying applications
+=====================================================================
+
+Just as the "big" Python, MicroPython supports creation of "third party"
+packages, distributing them, and easily installing them in each user's
+environment. This chapter discusses how these actions are achieved.
+Some familiarity with Python packaging is recommended.
+
+Overview
+--------
+
+Steps below represent a high-level workflow when creating and consuming
+packages:
+
+1. Python modules and packages are turned into distribution package
+   archives, and published at the Python Package Index (PyPI).
+2. `upip` package manager can be used to install a distribution package
+   on a `MicroPython port` with networking capabilities (for example,
+   on the Unix port).
+3. For ports without networking capabilities, an "installation image"
+   can be prepared on the Unix port, and transferred to a device by
+   suitable means.
+4. For low-memory ports, the installation image can be frozen as the
+   bytecode into MicroPython executable, thus minimizing the memory
+   storage overheads.
+
+The sections below describe this process in details.
+
+Distribution packages
+---------------------
+
+Python modules and packages can be packaged into archives suitable for
+transfer between systems, storing at the well-known location (PyPI),
+and downloading on demand for deployment. These archives are known as
+*distribution packages* (to differentiate them from Python packages
+(means to organize Python source code)).
+
+The MicroPython distribution package format is a well-known tar.gz
+format, with some adaptations however. The Gzip compressor, used as
+an external wrapper for TAR archives, by default uses 32KB dictionary
+size, which means that to uncompress a compressed stream, 32KB of
+contguous memory needs to be allocated. This requirement may be not
+satisfiable on low-memory devices, which may have total memory available
+less than that amount, and even if not, a contiguous block like that
+may be hard to allocate due to memory fragmentation. To accommodate
+these constraints, MicroPython distribution packages use Gzip compression
+with the dictionary size of 4K, which should be a suitable compromise
+with still achieving some compression while being able to uncompressed
+even by the smallest devices.
+
+Besides the small compression dictionary size, MicroPython distribution
+packages also have other optimizations, like removing any files from
+the archive which aren't used by the installation process. In particular,
+`upip` package manager doesn't execute ``setup.py`` during installation
+(see below), and thus that file is not included in the archive.
+
+At the same time, these optimizations make MicroPython distribution
+packages not compatible with `CPython`'s package manager, ``pip``.
+This isn't considered a big problem, because:
+
+1. Packages can be installed with `upip`, and then can be used with
+   CPython (if they are compatible with it).
+2. In the other direction, majority of CPython packages would be
+   incompatible with MicroPython by various reasons, first of all,
+   the reliance on features not implemented by MicroPython.
+
+Summing up, the MicroPython distribution package archives are highly
+optimized for MicroPython's target environments, which are highly
+resource constrained devices.
+
+
+``upip`` package manager
+------------------------
+
+MicroPython distribution packages are intended to be installed using
+the `upip` package manager. `upip` is a Python application which is
+usually distributed (as frozen bytecode) with network-enabled
+`MicroPython ports <MicroPython port>`. At the very least,
+`upip` is available in the `MicroPython Unix port`.
+
+On any `MicroPython port` providing `upip`, it can be accessed as
+following::
+
+    import upip
+    upip.help()
+    upip.install(package_or_package_list, [path])
+
+Where *package_or_package_list* is the name of a distribution
+package to install, or a list of such names to install multiple
+packages. Optional *path* parameter specifies filesystem
+location to install under and defaults to the standard library
+location (see below).
+
+An example of installing a specific package and then using it::
+
+    >>> import upip
+    >>> upip.install("micropython-pystone_lowmem")
+    [...]
+    >>> import pystone_lowmem
+    >>> pystone_lowmem.main()
+
+Note that the name of Python package and the name of distribution
+package for it in general don't have to match, and oftentimes they
+don't. This is because PyPI provides a central package repository
+for all different Python implementations and versions, and thus
+distribution package names may need to be namespaced for a particular
+implementation. For example, all packages from `micropython-lib`
+follow this naming convention: for a Python module or package named
+``foo``, the distribution package name is ``micropython-foo``.
+
+For the ports which run MicroPython executable from the OS command
+prompts (like the Unix port), `upip` can be (and indeed, usually is)
+run from the command line instead of MicroPython's own REPL. The
+commands which corresponds to the example above are::
+
+    micropython -m upip -h
+    micropython -m upip install [-p <path>] <packages>...
+    micropython -m upip install micropython-pystone_lowmem
+
+[TODO: Describe installation path.]
+
+
+Cross-installing packages
+-------------------------
+
+For `MicroPython ports <MicroPython port>` without native networking
+capabilities, the recommend process is "cross-installing" them into a
+"directory image" using the `MicroPython Unix port`, and then
+transferring this image to a device by suitable means.
+
+Installing to a directory image involves using ``-p`` switch to `upip`::
+
+    micropython -m upip install -p install_dir micropython-pystone_lowmem
+
+After this command, the package content (and contents of every depenency
+packages) will be available in the ``install_dir/`` subdirectory. You
+would need to transfer contents of this directory (without the
+``install_dir/`` prefix) to the device, at the suitable location, where
+it can be found by the Python ``import`` statement (see discussion of
+the `upip` installation path above).
+
+
+Cross-installing packages with freezing
+---------------------------------------
+
+For the low-memory `MicroPython ports <MicroPython port>`, the process
+described in the previous section does not provide the most efficient
+resource usage,because the packages are installed in the source form,
+so need to be compiled to the bytecome on each import. This compilation
+requires RAM, and the resulting bytecode is also stored in RAM, reducing
+its amount available for storing application data. Moreover, the process
+above requires presence of the filesystem on a device, and the most
+resource-constrained devices may not even have it.
+
+The bytecode freezing is a process which resolves all the issues
+mentioned above:
+
+* The source code is pre-compiled into bytecode and store as such.
+* The bytecode is stored in ROM, not RAM.
+* Filesystem is not required for frozen packages.
+
+Using frozen bytecode requires building the executable (firmware)
+for a given `MicroPython port` from the C source code. Consequently,
+the process is:
+
+1. Follow the instructions for a particular port on setting up a
+   toolchain and building the port. For example, for ESP8266 port,
+   study instructions in ``ports/esp8266/README.md`` and follow them.
+   Make sure you can build the port and deploy the resulting
+   executable/firmware successfully before proceeding to the next steps.
+2. Build `MicroPython Unix port` and make sure it is in your PATH and
+   you can execute ``micropython``.
+3. Change to port's directory (e.g. ``ports/esp8266/`` for ESP8266).
+4. Run ``make clean-frozen``. This step cleans up any previous
+   modules which were installed for freezing (consequently, you need
+   to skip this step to add additional modules, instead of starting
+   from scratch).
+5. Run ``micropython -m upip install -p modules <packages>...`` to
+   install packages you want to freeze.
+6. Run ``make clean``.
+7. Run ``make``.
+
+After this, you should have the executable/firmware with modules as
+the bytecode inside, which you can deploy the usual way.
+
+Few notes:
+
+1. Step 5 in the sequence above assumes that the distribution package
+   is available from PyPI. If that is not the case, you would need
+   to copy Python source files manually to ``modules/`` subdirectory
+   of the port port directory. (Note that upip does not support
+   installing from e.g. version control repositories).
+2. The firmware for baremetal devices usually has size restrictions,
+   so adding too many frozen modules may overflow it. Usually, you
+   would get a linking error if this happens. However, in some cases,
+   an image may be produced, which is not runnable on a device. Such
+   cases are in general bugs, and should be reported and further
+   investigated. If you face such a situation, as an initial step,
+   you may want to decrease the amount of frozen modules included.
+
+
+Creating distribution packages
+------------------------------
+
+Distribution packages for MicroPython are created in the same manner
+as for CPython or any other Python implementation, see references at
+the end of chapter. Setuptools (instead of distutils) should be used,
+because distutils do not support dependencies and other features. "Source
+distribution" (``sdist``) format is used for packaging. The post-processing
+discussed above, (and pre-processing discussed in the following section)
+is achieved by using custom ``sdist`` command for setuptools. Thus, packaging
+steps remain the same as for the standard setuptools, the user just
+needs to override ``sdist`` command implementation by passing the
+appropriate argument to ``setup()`` call::
+
+    from setuptools import setup
+    import sdist_upip
+
+    setup(
+        ...,
+        cmdclass={'sdist': sdist_upip.sdist}
+    )
+
+The sdist_upip.py module as referenced above can be found in
+`micropython-lib`:
+https://github.com/micropython/micropython-lib/blob/master/sdist_upip.py
+
+
+Application resources
+---------------------
+
+A complete application, besides the source code, oftentimes also consists
+of data files, e.g. web page templates, game images, etc. It's clear how
+to deal with those when application is installed manually - you just put
+those data files in the filesystem at some location and use the normal
+file access functions.
+
+The situation is different when deploying applications from packages - this
+is more advanced, streamlined and flexible way, but also requires more
+advanced approach to accessing data files. This approach is treating
+the data files as "resources", and abstracting away access to them.
+
+Python supports resource access using its "setuptools" library, using
+``pkg_resources`` module. MicroPython, following its usual approach,
+implements subset of the functionality of that module, specifically
+``pkg_resources.resource_stream(package, resource)`` function.
+The idea is that an application calls this function, passing a
+resource identifier, which is a relative path to data file within
+the specified package (usually top-level application package). It
+returns a stream object which can be used to access resource contents.
+Thus, the ``resource_stream()`` emulates interface of the standard
+`open()` function.
+
+Implementation-wise, ``resource_stream()`` uses file operations
+underlyingly, if distribution package is install in the filesystem.
+However, it also supports functioning without the underlying filesystem,
+e.g. if the package is frozen as the bytecode. This however requires
+an extra intermediate step when packaging application - creation of
+"Python resource module".
+
+The idea of this module is to convert binary data to a Python bytes
+object, and put it into the dictionary, indexed by the resource name.
+This conversion is done automatically using overridden ``sdist`` command
+described in the previous section.
+
+Let's trace the complete process using the following example. Suppose
+your application has the following structure::
+
+    my_app/
+        __main__.py
+        utils.py
+        data/
+            page.html
+            image.png
+
+``__main__.py`` and ``utils.py`` should access resources using the
+following calls::
+
+    import pkg_resources
+
+    pkg_resources.resource_stream(__name__, "data/page.html")
+    pkg_resources.resource_stream(__name__, "data/image.png")
+
+You can develop and debug using the `MicroPython Unix port` as usual.
+When time comes to make a distribution package out of it, just use
+overridden "sdist" command from sdist_upip.py module as described in
+the previous section.
+
+This will create a Python resource module named ``R.py``, based on the
+files declared in ``MANIFEST`` or ``MANIFEST.in`` files (any non-``.py``
+file will be considered a resource and added to ``R.py``) - before
+proceeding with the normal packaging steps.
+
+Prepared like this, your application will work both when deployed to
+filesystem and as frozen bytecode.
+
+If you would like to debug ``R.py`` creation, you can run::
+
+    python3 setup.py sdist --manifest-only
+
+Alternatively, you can use tools/mpy_bin2res.py script from the
+MicroPython distribution, in which can you will need to pass paths
+to all resource files::
+
+    mpy_bin2res.py data/page.html data/image.png
+
+References
+----------
+
+* Python Packaging User Guide: https://packaging.python.org/
+* Setuptools documentation: https://setuptools.readthedocs.io/
+* Distutils documentation: https://docs.python.org/3/library/distutils.html