Merge tag 'v1.9.1'2.0.0-alpha.0

Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions

This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems.  There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes.  The VFS has
some regressions fixed including the ability to statvfs the root.

All changes are listed below.

py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy

extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning

lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled

tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"

unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)

windows port:
- convert to use core-provided version of built-in import()

qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests

stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version

cc3200 port:
- convert from using stmhal's input() to core provided version

teensy port:
- convert from using stmhal's input() to core provided version

esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version

zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls

docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface

examples:
- embedding: use core-provided KeyboardInterrupt object

5 files changed, 44 insertions, 159 deletions

diff --git a/lib/utils/interrupt_char.c b/lib/utils/interrupt_char.c
index ab4efd911..fca0f95b5 100644
--- a/lib/utils/interrupt_char.c
+++ b/lib/utils/interrupt_char.c
@@ -27,23 +27,24 @@
 #include "py/obj.h"
 #include "py/mpstate.h"
 
+#if MICROPY_KBD_EXCEPTION
+
 int mp_interrupt_char;
 
 void mp_hal_set_interrupt_char(int c) {
     if (c != -1) {
-        #if MICROPY_KBD_EXCEPTION
         mp_obj_exception_clear_traceback(MP_OBJ_FROM_PTR(&MP_STATE_VM(mp_kbd_exception)));
-        #else
-        mp_obj_exception_clear_traceback(MP_STATE_PORT(mp_kbd_exception));
-        #endif
     }
     mp_interrupt_char = c;
 }
 
 void mp_keyboard_interrupt(void) {
-    #if MICROPY_KBD_EXCEPTION
     MP_STATE_VM(mp_pending_exception) = MP_OBJ_FROM_PTR(&MP_STATE_VM(mp_kbd_exception));
-    #else
-    MP_STATE_VM(mp_pending_exception) = MP_STATE_PORT(mp_kbd_exception);
+    #if MICROPY_ENABLE_SCHEDULER
+    if (MP_STATE_VM(sched_state) == MP_SCHED_IDLE) {
+        MP_STATE_VM(sched_state) = MP_SCHED_PENDING;
+    }
     #endif
 }
+
+#endif
diff --git a/lib/utils/pyexec.c b/lib/utils/pyexec.c
index d5736153b..31a97deaf 100644
--- a/lib/utils/pyexec.c
+++ b/lib/utils/pyexec.c
@@ -52,13 +52,15 @@ STATIC bool repl_display_debugging_info = 0;
 #define EXEC_FLAG_ALLOW_DEBUGGING (2)
 #define EXEC_FLAG_IS_REPL (4)
 #define EXEC_FLAG_SOURCE_IS_RAW_CODE (8)
+#define EXEC_FLAG_SOURCE_IS_VSTR (16)
+#define EXEC_FLAG_SOURCE_IS_FILENAME (32)
 
 // parses, compiles and executes the code in the lexer
 // frees the lexer before returning
 // EXEC_FLAG_PRINT_EOF prints 2 EOF chars: 1 after normal output, 1 after exception output
 // EXEC_FLAG_ALLOW_DEBUGGING allows debugging info to be printed after executing the code
 // EXEC_FLAG_IS_REPL is used for REPL inputs (flag passed on to mp_compile)
-STATIC int parse_compile_execute(void *source, mp_parse_input_kind_t input_kind, int exec_flags, pyexec_result_t *result) {
+STATIC int parse_compile_execute(const void *source, mp_parse_input_kind_t input_kind, int exec_flags, pyexec_result_t *result) {
     int ret = 0;
     uint32_t start = 0;
 
@@ -75,11 +77,23 @@ STATIC int parse_compile_execute(void *source, mp_parse_input_kind_t input_kind,
         } else
         #endif
         {
+            #if MICROPY_ENABLE_COMPILER
+            mp_lexer_t *lex;
+            if (exec_flags & EXEC_FLAG_SOURCE_IS_VSTR) {
+                const vstr_t *vstr = source;
+                lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, vstr->buf, vstr->len, 0);
+            } else if (exec_flags & EXEC_FLAG_SOURCE_IS_FILENAME) {
+                lex = mp_lexer_new_from_file(source);
+            } else {
+                lex = (mp_lexer_t*)source;
+            }
             // source is a lexer, parse and compile the script
-            mp_lexer_t *lex = source;
             qstr source_name = lex->source_name;
             mp_parse_tree_t parse_tree = mp_parse(lex, input_kind);
             module_fun = mp_compile(&parse_tree, source_name, MP_EMIT_OPT_NONE, exec_flags & EXEC_FLAG_IS_REPL);
+            #else
+            mp_raise_msg(&mp_type_RuntimeError, "script compilation not supported");
+            #endif
         }
 
         // execute code
@@ -154,6 +168,7 @@ STATIC int parse_compile_execute(void *source, mp_parse_input_kind_t input_kind,
     return ret;
 }
 
+#if MICROPY_ENABLE_COMPILER
 #if MICROPY_REPL_EVENT_DRIVEN
 
 typedef struct _repl_t {
@@ -173,7 +188,8 @@ STATIC int pyexec_friendly_repl_process_char(int c);
 void pyexec_event_repl_init(void) {
     MP_STATE_VM(repl_line) = vstr_new(32);
     repl.cont_line = false;
-    readline_init(MP_STATE_VM(repl_line), ">>> ");
+    // no prompt before printing friendly REPL banner or entering raw REPL
+    readline_init(MP_STATE_VM(repl_line), "");
     if (pyexec_mode_kind == PYEXEC_MODE_RAW_REPL) {
         pyexec_raw_repl_process_char(CHAR_CTRL_A);
     } else {
@@ -189,6 +205,7 @@ STATIC int pyexec_raw_repl_process_char(int c) {
     } else if (c == CHAR_CTRL_B) {
         // change to friendly REPL
         pyexec_mode_kind = PYEXEC_MODE_FRIENDLY_REPL;
+        vstr_reset(MP_STATE_VM(repl_line));
         repl.cont_line = false;
         pyexec_friendly_repl_process_char(CHAR_CTRL_B);
         return 0;
@@ -214,11 +231,9 @@ STATIC int pyexec_raw_repl_process_char(int c) {
         return PYEXEC_FORCED_EXIT;
     }
 
-    mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, MP_STATE_VM(repl_line)->buf, MP_STATE_VM(repl_line)->len, 0);
-    if (lex == NULL) {
-        mp_hal_stdout_tx_str("\x04MemoryError\r\n\x04");
-    } else {
-        parse_compile_execute(lex, MP_PARSE_FILE_INPUT, EXEC_FLAG_PRINT_EOF, NULL);
+    int ret = parse_compile_execute(MP_STATE_VM(repl_line), MP_PARSE_FILE_INPUT, EXEC_FLAG_PRINT_EOF | EXEC_FLAG_SOURCE_IS_VSTR, NULL);
+    if (ret & PYEXEC_FORCED_EXIT) {
+        return ret;
     }
 
 reset:
@@ -242,7 +257,6 @@ STATIC int pyexec_friendly_repl_process_char(int c) {
         } else if (ret == CHAR_CTRL_B) {
             // reset friendly REPL
             mp_hal_stdout_tx_str("\r\n");
-            mp_hal_stdout_tx_str("\r\n");
             mp_hal_stdout_tx_str("Adafruit CircuitPython " MICROPY_GIT_TAG " on " MICROPY_BUILD_DATE "; " MICROPY_HW_BOARD_NAME " with " MICROPY_HW_MCU_NAME "\r\n");
             // mp_hal_stdout_tx_str("Type \"help()\" for more information.\r\n");
             goto input_restart;
@@ -293,11 +307,9 @@ STATIC int pyexec_friendly_repl_process_char(int c) {
         }
 
 exec: ;
-        mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, vstr_str(MP_STATE_VM(repl_line)), vstr_len(MP_STATE_VM(repl_line)), 0);
-        if (lex == NULL) {
-            printf("MemoryError\n");
-        } else {
-            parse_compile_execute(lex, MP_PARSE_SINGLE_INPUT, EXEC_FLAG_ALLOW_DEBUGGING | EXEC_FLAG_IS_REPL, NULL);
+        int ret = parse_compile_execute(MP_STATE_VM(repl_line), MP_PARSE_SINGLE_INPUT, EXEC_FLAG_ALLOW_DEBUGGING | EXEC_FLAG_IS_REPL | EXEC_FLAG_SOURCE_IS_VSTR, NULL);
+        if (ret & PYEXEC_FORCED_EXIT) {
+            return ret;
         }
 
 input_restart:
@@ -366,11 +378,9 @@ raw_repl_reset:
             return PYEXEC_FORCED_EXIT;
         }
 
-        mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, line.buf, line.len, 0);
-        if (lex == NULL) {
-            printf("\x04MemoryError\n\x04");
-        } else {
-            parse_compile_execute(lex, MP_PARSE_FILE_INPUT, EXEC_FLAG_PRINT_EOF, NULL);
+        int ret = parse_compile_execute(&line, MP_PARSE_FILE_INPUT, EXEC_FLAG_PRINT_EOF | EXEC_FLAG_SOURCE_IS_VSTR, NULL);
+        if (ret & PYEXEC_FORCED_EXIT) {
+            return ret;
         }
     }
 }
@@ -489,26 +499,18 @@ friendly_repl_reset:
             }
         }
 
-        mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, vstr_str(&line), vstr_len(&line), 0);
-        if (lex == NULL) {
-            printf("MemoryError\n");
-        } else {
-            parse_compile_execute(lex, parse_input_kind, EXEC_FLAG_ALLOW_DEBUGGING | EXEC_FLAG_IS_REPL, NULL);
+        ret = parse_compile_execute(&line, parse_input_kind, EXEC_FLAG_ALLOW_DEBUGGING | EXEC_FLAG_IS_REPL | EXEC_FLAG_SOURCE_IS_VSTR, NULL);
+        if (ret & PYEXEC_FORCED_EXIT) {
+            return ret;
         }
     }
 }
 
 #endif // MICROPY_REPL_EVENT_DRIVEN
+#endif // MICROPY_ENABLE_COMPILER
 
-int pyexec_file(const char *filename, pyexec_result_t *exec_result) {
-    mp_lexer_t *lex = mp_lexer_new_from_file(filename);
-
-    if (lex == NULL) {
-        printf("could not open file '%s' for reading\n", filename);
-        return false;
-    }
-
-    return parse_compile_execute(lex, MP_PARSE_FILE_INPUT, 0, exec_result);
+int pyexec_file(const char *filename, pyexec_result_t *result) {
+    return parse_compile_execute(filename, MP_PARSE_FILE_INPUT, EXEC_FLAG_SOURCE_IS_FILENAME, result);
 }
 
 #if MICROPY_MODULE_FROZEN
diff --git a/lib/utils/pyhelp.c b/lib/utils/pyhelp.c
deleted file mode 100644
index 30ff391c4..000000000
--- a/lib/utils/pyhelp.c
+++ /dev/null
@@ -1,85 +0,0 @@
-/*
- * This file is part of the MicroPython project, http://micropython.org/
- *
- * The MIT License (MIT)
- *
- * Copyright (c) 2013-2016 Damien P. George
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#include <stdio.h>
-
-#include "lib/utils/pyhelp.h"
-
-STATIC void pyhelp_print_info_about_object(mp_obj_t name_o, mp_obj_t value) {
-    mp_printf(MP_PYTHON_PRINTER, "  ");
-    mp_obj_print(name_o, PRINT_STR);
-    mp_printf(MP_PYTHON_PRINTER, " -- ");
-    mp_obj_print(value, PRINT_STR);
-    mp_printf(MP_PYTHON_PRINTER, "\n");
-}
-
-// Helper for 1-argument form of builtin help
-//
-// Typically a port will define a help function thus:
-//
-// STATIC const char *const myport_help_text =
-// "Welcome to MicroPython!\n"
-// "\n"
-// "...information specific to this port e.g. modules available...";
-//
-// STATIC mp_obj_t myport_help(mp_uint_t n_args, const mp_obj_t *args) {
-//     if (n_args == 0) {
-//         printf("%s", myport_help_text);
-//     } else {
-//         pyhelp_print_obj(args[0]);
-//     }
-//     return mp_const_none;
-// }
-// MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(mp_builtin_help_obj, 0, 1, myport_help);
-//
-void pyhelp_print_obj(const mp_obj_t obj) {
-    // try to print something sensible about the given object
-    mp_printf(MP_PYTHON_PRINTER, "object ");
-    mp_obj_print(obj, PRINT_STR);
-    mp_printf(MP_PYTHON_PRINTER, " is of type %s\n", mp_obj_get_type_str(obj));
-
-    mp_map_t *map = NULL;
-    if (MP_OBJ_IS_TYPE(obj, &mp_type_module)) {
-        map = mp_obj_dict_get_map(mp_obj_module_get_globals(obj));
-    } else {
-        mp_obj_type_t *type;
-        if (MP_OBJ_IS_TYPE(obj, &mp_type_type)) {
-            type = obj;
-        } else {
-            type = mp_obj_get_type(obj);
-        }
-        if (type->locals_dict != MP_OBJ_NULL && MP_OBJ_IS_TYPE(type->locals_dict, &mp_type_dict)) {
-            map = mp_obj_dict_get_map(type->locals_dict);
-        }
-    }
-    if (map != NULL) {
-        for (uint i = 0; i < map->alloc; i++) {
-            if (map->table[i].key != MP_OBJ_NULL) {
-                pyhelp_print_info_about_object(map->table[i].key, map->table[i].value);
-            }
-        }
-    }
-}
diff --git a/lib/utils/pyhelp.h b/lib/utils/pyhelp.h
deleted file mode 100644
index ee313ee37..000000000
--- a/lib/utils/pyhelp.h
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * This file is part of the MicroPython project, http://micropython.org/
- *
- * The MIT License (MIT)
- *
- * Copyright (c) 2013-2016 Damien P. George
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-#ifndef __MICROPY_INCLUDED_LIB_UTILS_PYHELP_H__
-#define __MICROPY_INCLUDED_LIB_UTILS_PYHELP_H__
-
-#include "py/obj.h"
-
-void pyhelp_print_obj(const mp_obj_t obj);
-
-#endif // __MICROPY_INCLUDED_LIB_UTILS_PYHELP_H__
diff --git a/lib/utils/stdout_helpers.c b/lib/utils/stdout_helpers.c
index 5f7a17d32..3de119757 100644
--- a/lib/utils/stdout_helpers.c
+++ b/lib/utils/stdout_helpers.c
@@ -9,7 +9,7 @@
  * implementation below can be used.
  */
 
-// Send "cooked" string of given length, where every occurance of
+// Send "cooked" string of given length, where every occurrence of
 // LF character is replaced with CR LF.
 void mp_hal_stdout_tx_strn_cooked(const char *str, size_t len) {
     while (len--) {