suspect-devices/circuitpython/py/lexer.c, branch 6.0.x

Add license to some obvious files.

2020-07-06T18:16:25+00:00

f-strings: Make optional, defaulting to !CIRCUITPY_MINIMAL_BUILD

2020-03-09T14:03:25+00:00

This should reclaim *most* code space added to handle f-strings. However, there may be some small code growth as parse_string_literal takes a new parameter (which will always be 0, so hopefully the optimizer eliminates it)

lexer: catch concatenation of f'' and '' strings

2020-03-09T14:03:25+00:00

This turns the "edge case" into a parse-time error.

Address dpgeorge feedback - largely simplifications

2020-03-09T13:16:07+00:00

py: Implement partial PEP-498 (f-string) support

2020-03-09T13:16:07+00:00

This implements (most of) the PEP-498 spec for f-strings, with two exceptions: - raw f-strings (`fr` or `rf` prefixes) raise `NotImplementedError` - one special corner case does not function as specified in the PEP (more on that in a moment) This is implemented in the core as a syntax translation, brute-forcing all f-strings to run through `String.format`. For example, the statement `x='world'; print(f'hello {x}')` gets translated *at a syntax level* (injected into the lexer) to `x='world'; print('hello {}'.format(x))`. While this may lead to weird column results in tracebacks, it seemed like the fastest, most efficient, and *likely* most RAM-friendly option, despite being implemented under the hood with a completely separate `vstr_t`. Since [string concatenation of adjacent literals is implemented in the lexer](https://github.com/micropython/micropython/commit/534b7c368dc2af7720f3aaed0c936ef46d773957), two side effects emerge: - All strings with at least one f-string portion are concatenated into a single literal which *must* be run through `String.format()` wholesale, and: - Concatenation of a raw string with interpolation characters with an f-string will cause `IndexError`/`KeyError`, which is both different from CPython *and* different from the corner case mentioned in the PEP (which gave an example of the following:) ```python x = 10 y = 'hi' assert ('a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e') == 'ab10{c}str< hi >de' ``` The above-linked commit detailed a pretty solid case for leaving string concatenation in the lexer rather than putting it in the parser, and undoing that decision would likely be disproportionately costly on resources for the sake of a probably-low-impact corner case. An alternative to become complaint with this corner case of the PEP would be to revert to string concatenation in the parser *only when an f-string is part of concatenation*, though I've done no investigation on the difficulty or costs of doing this. A decent set of tests is included. I've manually tested this on the `unix` port on Linux and on a Feather M4 Express (`atmel-samd`) and things seem sane.

Two fixes and translate more strings.

2018-08-09T20:29:30+00:00

* Fix finding translations with escaped characters. * Add back \r to translations since its needed by screen.

py/lexer: Add support for underscores in numeric literals.

2018-06-12T02:17:43+00:00

This is a very convenient feature introduced in Python 3.6 by PEP 515.

all: Remove inclusion of internal py header files.

2017-10-04T01:37:50+00:00

Header files that are considered internal to the py core and should not normally be included directly are: py/nlr.h - internal nlr configuration and declarations py/bc0.h - contains bytecode macro definitions py/runtime0.h - contains basic runtime enums Instead, the top-level header files to include are one of: py/obj.h - includes runtime0.h and defines everything to use the mp_obj_t type py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h, and defines everything to use the general runtime support functions Additional, specific headers (eg py/objlist.h) can be included if needed.

all: Raise exceptions via mp_raise_XXX

2017-08-13T12:52:33+00:00

- Changed: ValueError, TypeError, NotImplementedError - OSError invocations unchanged, because the corresponding utility function takes ints, not strings like the long form invocation. - OverflowError, IndexError and RuntimeError etc. not changed for now until we decide whether to add new utility functions.

all: Use the name MicroPython consistently in comments

2017-07-31T08:35:40+00:00

There were several different spellings of MicroPython present in comments, when there should be only one.