suspect-devices/circuitpython/py/lexer.c, branch 0.8.4CircuitPython - a Python implementation for teaching coding with microcontrollershttps://git.suspectdevices.com/suspect-devices/circuitpython/atom?h=0.8.42016-12-21T23:49:54+00:00py/lexer: Permanently disable the mp_lexer_show_token function.2016-12-21T23:49:54+00:00Damien Georgedamien.p.george@gmail.com2016-12-21T23:49:54+00:00urn:sha1:c305ae32436e37327d33df979d57d9ac1fb822c1
The lexer is very mature and this debug function is no longer used. If
it's really needed one can uncomment it and recompile.
py/lexer: Remove unnecessary check for EOF in lexer's next_char func.2016-12-21T23:39:06+00:00Damien Georgedamien.p.george@gmail.com2016-12-21T23:39:06+00:00urn:sha1:f4aebafe7ae033d6bcde87d71108f488ed4a3251
This check always fails (ie chr0 is never EOF) because the callers of this
function never call it past the end of the input stream. And even if they
did it would be harmless because 1) reader.readbyte must continue to
return an EOF char if the stream is exhausted; 2) next_char would just
count the subsequent EOF's as characters worth 1 column.
py/lexer: Remove unreachable code in string tokeniser.2016-12-21T23:37:13+00:00Damien Georgedamien.p.george@gmail.com2016-12-21T23:37:13+00:00urn:sha1:b9c4783273bfe794a6be2be6b02aedcd9bd80d64tests/basics/lexer: Add a test for newline-escaping within a string.2016-12-21T23:32:06+00:00Damien Georgedamien.p.george@gmail.com2016-12-21T23:32:06+00:00urn:sha1:adccafb42a892df87e3ca45a8fcc01535b39b055py/lexer: Make lexer use an mp_reader as its source.2016-11-16T07:35:01+00:00Damien Georgedamien.p.george@gmail.com2016-11-16T07:27:20+00:00urn:sha1:5bdf1650de782d766a648f992270306269cc985apy/lexer: Rewrite mp_lexer_new_from_fd in terms of mp_reader.2016-11-16T07:13:51+00:00Damien Georgedamien.p.george@gmail.com2016-11-16T07:12:55+00:00urn:sha1:66d955c218b66076a3d4300f70388c634c0d3099py/lexer: Provide generic mp_lexer_new_from_file based on mp_reader.2016-11-16T07:13:51+00:00Damien Georgedamien.p.george@gmail.com2016-11-16T05:25:06+00:00urn:sha1:e5ef15a9d7bead2a60009caedbc128a8906c7ecd
If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then
lexer.c now provides an implementation of mp_lexer_new_from_file using
the mp_reader_new_file function.
py/lexer: Rewrite mp_lexer_new_from_str_len in terms of mp_reader_mem.2016-11-16T07:13:50+00:00Damien Georgedamien.p.george@gmail.com2016-11-16T05:22:08+00:00urn:sha1:511c083811ca538753dfccc7c908048b2c23bea4py/lexer: Remove unnecessary code, and unreachable code.2016-10-12T00:00:17+00:00Damien Georgedamien.p.george@gmail.com2016-10-12T00:00:17+00:00urn:sha1:31101d91ceb5758c17d02b7d41bffbd387cad112
Setting emit_dent=0 is unnecessary because arriving in that part of the
if-logic will guarantee that emit_dent is already zero.
The block to check indent_top(lex)>0 is unreachable because a newline is
always inserted an the end of the input stream, and hence dedents are
always processed before EOF.
py/vstr: Remove vstr.had_error flag and inline basic vstr functions.2016-09-19T02:28:55+00:00Damien Georgedamien.p.george@gmail.com2016-09-19T01:17:02+00:00urn:sha1:5da0d29d3cefa6a3cac52e0db96e9ede820d6a51
The vstr.had_error flag was a relic from the very early days which assumed
that the malloc functions (eg m_new, m_renew) returned NULL if they failed
to allocate. But that's no longer the case: these functions will raise an
exception if they fail.
Since it was impossible for had_error to be set, this patch introduces no
change in behaviour.
An alternative option would be to change the malloc calls to the _maybe
variants, which return NULL instead of raising, but then a lot of code
will need to explicitly check if the vstr had an error and raise if it
did.
The code-size savings for this patch are, in bytes: bare-arm:188,
minimal:456, unix(NDEBUG,x86-64):368, stmhal:228, esp8266:360.