Merge tag 'v1.9.1'2.0.0-alpha.0

Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions

This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems.  There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes.  The VFS has
some regressions fixed including the ability to statvfs the root.

All changes are listed below.

py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy

extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning

lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled

tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"

unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)

windows port:
- convert to use core-provided version of built-in import()

qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests

stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version

cc3200 port:
- convert from using stmhal's input() to core provided version

teensy port:
- convert from using stmhal's input() to core provided version

esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version

zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls

docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface

examples:
- embedding: use core-provided KeyboardInterrupt object

1 files changed, 145 insertions, 107 deletions

diff --git a/docs/library/usocket.rst b/docs/library/usocket.rst
index c46e8f4c5..71deaebc4 100644
--- a/docs/library/usocket.rst
+++ b/docs/library/usocket.rst
@@ -7,13 +7,28 @@
 
 This module provides access to the BSD socket interface.
 
-See corresponding `CPython module <https://docs.python.org/3/library/socket.html>`_ for
-comparison.
+See the corresponding `CPython module <https://docs.python.org/3/library/socket.html>`_
+for comparison.
+
+.. admonition:: Difference to CPython
+   :class: attention
+
+   CPython used to have a ``socket.error`` exception which is now deprecated,
+   and is an alias of OSError. In MicroPython, use OSError directly.
+
+.. admonition:: Difference to CPython
+   :class: attention
+
+   For efficiency and consistency, socket objects in MicroPython implement a stream
+   (file-like) interface directly. In CPython, you need to convert a socket to
+   a file-like object using ``makefile()`` method. This method is still supported
+   by MicroPython (but is a no-op), so where compatibility with CPython matters,
+   be sure to use it.
 
 Socket address format(s)
 ------------------------
 
-Functions below which expect a network address, accept it in the format of
+The functions below which expect a network address, accept it in the format of
 `(ipv4_address, port)`, where `ipv4_address` is a string with dot-notation numeric
 IPv4 address, e.g. ``"8.8.8.8"``, and port is integer port number in the range
 1-65535. Note the domain names are not accepted as `ipv4_address`, they should be
@@ -26,18 +41,6 @@ Functions
 
    Create a new socket using the given address family, socket type and protocol number.
 
-    .. only:: port_wipy
-
-        .. note::
-
-           SSL sockets need to be created the following way before wrapping them with 
-           ``ssl.wrap_socket``::
-
-              import socket
-              import ssl
-              s = socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_SEC)
-              ss = ssl.wrap_socket(s)
-
 .. function:: socket.getaddrinfo(host, port)
 
    Translate the host/port argument into a sequence of 5-tuples that contain all the 
@@ -51,33 +54,50 @@ Functions
       s = socket.socket()
       s.connect(socket.getaddrinfo('www.micropython.org', 80)[0][-1])
 
-.. only:: port_wipy
-
-    Exceptions
-    ----------
+   .. admonition:: Difference to CPython
+      :class: attention
 
-    .. data:: socket.error
-    .. data:: socket.timeout
+      CPython raises a ``socket.gaierror`` exception (OSError subclass) in case
+      of error in this function. MicroPython doesn't have ``socket.gaierror``
+      and raises OSError directly. Note that error numbers of ``getaddrinfo()``
+      form a separate namespace and may not match error numbers from
+      ``uerrno`` module. To distinguish ``getaddrinfo()`` errors, they are
+      represented by negative numbers, whereas standard system errors are
+      positive numbers (error numbers are accessible using ``e.args[0]`` property
+      from an exception object). The use of negative values is a provisional
+      detail which may change in the future.
 
 Constants
 ---------
 
 .. data:: socket.AF_INET
+          socket.AF_INET6
 
-   family types
+   Address family types. Availability depends on a particular board.
 
 .. data:: socket.SOCK_STREAM
-.. data:: socket.SOCK_DGRAM
+          socket.SOCK_DGRAM
 
-   socket types
+   Socket types.
 
 .. data:: socket.IPPROTO_UDP
-.. data:: socket.IPPROTO_TCP
-.. only:: port_wipy
+          socket.IPPROTO_TCP
 
-   .. data:: socket.IPPROTO_SEC
+   IP protocol numbers.
 
-      protocol numbers
+.. data:: socket.SOL_*
+
+   Socket option levels (an argument to ``setsockopt()``). The exact inventory depends on a board.
+
+.. data:: socket.SO_*
+
+   Socket options (an argument to ``setsockopt()``). The exact inventory depends on a board.
+
+Constants specific to WiPy:
+
+.. data:: socket.IPPROTO_SEC
+
+    Special protocol value to create SSL-compatible socket.
 
 class socket
 ============
@@ -85,128 +105,146 @@ class socket
 Methods
 -------
 
-    .. method:: socket.close
+.. method:: socket.close
+
+   Mark the socket closed. Once that happens, all future operations on the socket 
+   object will fail. The remote end will receive no more data (after queued data is flushed).
+
+   Sockets are automatically closed when they are garbage-collected, but it is recommended 
+   to close() them explicitly, or to use a with statement around them.
 
-       Mark the socket closed. Once that happens, all future operations on the socket 
-       object will fail. The remote end will receive no more data (after queued data is flushed).
+.. method:: socket.bind(address)
 
-       Sockets are automatically closed when they are garbage-collected, but it is recommended 
-       to close() them explicitly, or to use a with statement around them.
+   Bind the socket to address. The socket must not already be bound.
 
-    .. method:: socket.bind(address)
+.. method:: socket.listen([backlog])
 
-       Bind the socket to address. The socket must not already be bound.
+   Enable a server to accept connections. If backlog is specified, it must be at least 0 
+   (if it's lower, it will be set to 0); and specifies the number of unaccepted connections
+   that the system will allow before refusing new connections. If not specified, a default
+   reasonable value is chosen.
 
-    .. method:: socket.listen([backlog])
+.. method:: socket.accept()
 
-       Enable a server to accept connections. If backlog is specified, it must be at least 0 
-       (if it's lower, it will be set to 0); and specifies the number of unaccepted connections
-       that the system will allow before refusing new connections. If not specified, a default
-       reasonable value is chosen.
+   Accept a connection. The socket must be bound to an address and listening for connections.
+   The return value is a pair (conn, address) where conn is a new socket object usable to send
+   and receive data on the connection, and address is the address bound to the socket on the
+   other end of the connection.
 
-    .. method:: socket.accept()
+.. method:: socket.connect(address)
 
-       Accept a connection. The socket must be bound to an address and listening for connections.
-       The return value is a pair (conn, address) where conn is a new socket object usable to send
-       and receive data on the connection, and address is the address bound to the socket on the
-       other end of the connection.
+   Connect to a remote socket at address.
 
-    .. method:: socket.connect(address)
+.. method:: socket.send(bytes)
 
-       Connect to a remote socket at address.
+   Send data to the socket. The socket must be connected to a remote socket.
+   Returns number of bytes sent, which may be smaller than the length of data
+   ("short write").
 
-    .. method:: socket.send(bytes)
+.. method:: socket.sendall(bytes)
 
-       Send data to the socket. The socket must be connected to a remote socket.
+   Send all data to the socket. The socket must be connected to a remote socket.
+   Unlike ``send()``, this method will try to send all of data, by sending data
+   chunk by chunk consecutively.
 
-    .. method:: socket.sendall(bytes)
+   The behavior of this method on non-blocking sockets is undefined. Due to this,
+   on MicroPython, it's recommended to use ``write()`` method instead, which
+   has the same "no short writes" policy for blocking sockets, and will return
+   number of bytes sent on non-blocking sockets.
 
-       Send data to the socket. The socket must be connected to a remote socket.
+.. method:: socket.recv(bufsize)
 
-    .. method:: socket.recv(bufsize)
+   Receive data from the socket. The return value is a bytes object representing the data
+   received. The maximum amount of data to be received at once is specified by bufsize.
 
-       Receive data from the socket. The return value is a bytes object representing the data
-       received. The maximum amount of data to be received at once is specified by bufsize.
+.. method:: socket.sendto(bytes, address)
 
-    .. method:: socket.sendto(bytes, address)
+   Send data to the socket. The socket should not be connected to a remote socket, since the
+   destination socket is specified by `address`.
 
-       Send data to the socket. The socket should not be connected to a remote socket, since the
-       destination socket is specified by `address`.
+.. method:: socket.recvfrom(bufsize)
 
-    .. method:: socket.recvfrom(bufsize)
+  Receive data from the socket. The return value is a pair (bytes, address) where bytes is a
+  bytes object representing the data received and address is the address of the socket sending
+  the data.
 
-      Receive data from the socket. The return value is a pair (bytes, address) where bytes is a
-      bytes object representing the data received and address is the address of the socket sending
-      the data.
+.. method:: socket.setsockopt(level, optname, value)
 
-    .. method:: socket.setsockopt(level, optname, value)
+   Set the value of the given socket option. The needed symbolic constants are defined in the
+   socket module (SO_* etc.). The value can be an integer or a bytes-like object representing
+   a buffer.
 
-       Set the value of the given socket option. The needed symbolic constants are defined in the
-       socket module (SO_* etc.). The value can be an integer or a bytes-like object representing
-       a buffer.
+.. method:: socket.settimeout(value)
 
-    .. method:: socket.settimeout(value)
+   Set a timeout on blocking socket operations. The value argument can be a nonnegative floating
+   point number expressing seconds, or None. If a non-zero value is given, subsequent socket operations
+   will raise an ``OSError`` exception if the timeout period value has elapsed before the operation has
+   completed. If zero is given, the socket is put in non-blocking mode. If None is given, the socket
+   is put in blocking mode.
 
-       Set a timeout on blocking socket operations. The value argument can be a nonnegative floating
-       point number expressing seconds, or None. If a non-zero value is given, subsequent socket operations
-       will raise an ``OSError`` exception if the timeout period value has elapsed before the operation has
-       completed. If zero is given, the socket is put in non-blocking mode. If None is given, the socket
-       is put in blocking mode.
+   .. admonition:: Difference to CPython
+      :class: attention
 
-       .. admonition:: Difference to CPython
-          :class: attention
+      CPython raises a ``socket.timeout`` exception in case of timeout,
+      which is an ``OSError`` subclass. MicroPython raises an OSError directly
+      instead. If you use ``except OSError:`` to catch the exception,
+      your code will work both in MicroPython and CPython.
 
-          CPython raises a ``socket.timeout`` exception in case of timeout,
-          which is an ``OSError`` subclass. MicroPython raises an OSError directly
-          instead. If you use ``except OSError:`` to catch the exception,
-          your code will work both in MicroPython and CPython.
+.. method:: socket.setblocking(flag)
 
-    .. method:: socket.setblocking(flag)
+   Set blocking or non-blocking mode of the socket: if flag is false, the socket is set to non-blocking,
+   else to blocking mode.
 
-       Set blocking or non-blocking mode of the socket: if flag is false, the socket is set to non-blocking,
-       else to blocking mode.
+   This method is a shorthand for certain ``settimeout()`` calls:
 
-       This method is a shorthand for certain ``settimeout()`` calls::
+   * ``sock.setblocking(True)`` is equivalent to ``sock.settimeout(None)``
+   * ``sock.setblocking(False)`` is equivalent to ``sock.settimeout(0)``
 
-          sock.setblocking(True) is equivalent to sock.settimeout(None)
-          sock.setblocking(False) is equivalent to sock.settimeout(0.0)
+.. method:: socket.makefile(mode='rb', buffering=0)
 
-    .. method:: socket.makefile(mode='rb')
+   Return a file object associated with the socket. The exact returned type depends on the arguments
+   given to makefile(). The support is limited to binary modes only ('rb', 'wb', and 'rwb').
+   CPython's arguments: ``encoding``, ``errors`` and ``newline`` are not supported.
 
-       Return a file object associated with the socket. The exact returned type depends on the arguments
-       given to makefile(). The support is limited to binary modes only ('rb' and 'wb').
-       CPython's arguments: ``encoding``, ``errors`` and ``newline`` are not supported.
+   .. admonition:: Difference to CPython
+      :class: attention
 
-       The socket must be in blocking mode; it can have a timeout, but the file object’s internal buffer
-       may end up in a inconsistent state if a timeout occurs.
+      As MicroPython doesn't support buffered streams, values of ``buffering``
+      parameter is ignored and treated as if it was 0 (unbuffered).
 
-       .. admonition:: Difference to CPython
-          :class: attention
+   .. admonition:: Difference to CPython
+      :class: attention
 
-          Closing the file object returned by makefile() WILL close the
-          original socket as well.
+      Closing the file object returned by makefile() WILL close the
+      original socket as well.
 
-    .. method:: socket.read([size])
+.. method:: socket.read([size])
 
-       Read up to size bytes from the socket. Return a bytes object. If ``size`` is not given, it
-       reads all data available from the socket until ``EOF``; as such the method will not return until
-       the socket is closed.
+   Read up to size bytes from the socket. Return a bytes object. If ``size`` is not given, it
+   reads all data available from the socket until ``EOF``; as such the method will not return until
+   the socket is closed. This function tries to read as much data as
+   requested (no "short reads"). This may be not possible with
+   non-blocking socket though, and then less data will be returned.
 
-    .. method:: socket.readinto(buf[, nbytes])
+.. method:: socket.readinto(buf[, nbytes])
 
-       Read bytes into the ``buf``.  If ``nbytes`` is specified then read at most
-       that many bytes.  Otherwise, read at most ``len(buf)`` bytes.
+   Read bytes into the ``buf``.  If ``nbytes`` is specified then read at most
+   that many bytes.  Otherwise, read at most ``len(buf)`` bytes. Just as
+   ``read()``, this method follows "no short reads" policy.
 
-       Return value: number of bytes read and stored into ``buf``.
+   Return value: number of bytes read and stored into ``buf``.
 
-    .. method:: socket.readline()
+.. method:: socket.readline()
 
-       Read a line, ending in a newline character.
+   Read a line, ending in a newline character.
 
-       Return value: the line read.
+   Return value: the line read.
 
-    .. method:: socket.write(buf)
+.. method:: socket.write(buf)
 
-       Write the buffer of bytes to the socket.
+   Write the buffer of bytes to the socket. This function will try to
+   write all data to a socket (no "short writes"). This may be not possible
+   with a non-blocking socket though, and returned value will be less than
+   the length of ``buf``.
 
-       Return value: number of bytes written.
+   Return value: number of bytes written.