CircuitPython - a Python implementation for teaching coding with microcontrollers https://git.suspectdevices.com/suspect-devices/circuitpython/atom?h=6.0.0-beta.0 2020-09-21T15:03:07+00:00 2020-09-21T15:03:07+00:00 Jeff Epler jepler@gmail.com 2020-09-21T15:02:27+00:00 urn:sha1:0318eb359fbeb91c6b37ed2050e57711ec2740bc Discord user Folknology encountered a problem building with Python 3.6.9, `TypeError: ord() expected a character, but string of length 0 found`. I was able to reproduce the problem using Python3.5*, and discovered that the meaning of the regular expression `"|."` had changed in 3.7. Before, ``` >>> [m.group(0) for m in re.finditer("|.", "hello")] ['', '', '', '', '', ''] ``` After: ``` >>> [m.group(0) for m in re.finditer("|.", "hello")] ['', 'h', '', 'e', '', 'l', '', 'l', '', 'o', ''] ``` Check if `words` is empty and if so use `"."` as the regular expression instead. This gives the same result on both versions: ``` ['h', 'e', 'l', 'l', 'o'] ``` and fixes the generation of the huffman dictionary. Folknology verified that this fix worked for them. * I could easily install 3.5 but not 3.6. 3.5 reproduced the same problem 2020-09-19T15:16:13+00:00 Jeff Epler jepler@gmail.com 2020-09-19T15:16:13+00:00 urn:sha1:bfbbbd6c5ce58ba1bcb57323566209902e1000b0 This construct (which I added without sufficient testing, apparently) is only supported in Python 3.7 and newer. Make it optional so that this script works on other Python versions. This means that if you have a system with non-UTF-8 encoding you will need to use Python 3.7. In particular, this affects a problem building circuitpython in github's ubuntu-18.04 virtual environment when Python 3.7 is not explicitly installed. cookie-cuttered libraries call for Python 3.6: ``` - name: Set up Python 3.6 uses: actions/setup-python@v1 with: python-version: 3.6 ``` Since CircuitPython's own build calls for 3.8, this problem was not detected. This problem was also encountered by discord user mdroberts1243. The failure I encountered was here: https://github.com/jepler/Jepler_CircuitPython_udecimal/runs/1138045020?check_suite_focus=true .. while my step of "clone and build circuitpython unix port" is unusual, I think the same problem would have affected "build assets" if that step had been reached. 2020-09-16T18:10:22+00:00 Scott Shawcroft scott@adafruit.com 2020-09-16T18:10:22+00:00 urn:sha1:750bc1e04ac6f3c5bec7cb6d5f2bdf1e84c45f47 compression: Implement @ciscorn's dictionary approach 2020-09-16T13:28:15+00:00 Jeff Epler jepler@gmail.com 2020-09-16T12:58:55+00:00 urn:sha1:a8e98cda83119b8716533ce24af6e3d691f7d2ca 2020-09-14T11:11:15+00:00 Kamil Tomaszewski kamil.tomaszewski@sony.com 2020-09-11T11:27:11+00:00 urn:sha1:c2fc592c2ca7ec42374a50f9e33ac551bf846596 2020-09-14T11:11:15+00:00 Kamil Tomaszewski kamil.tomaszewski@sony.com 2020-09-02T13:42:57+00:00 urn:sha1:064c597b60d7394216d9c36080b7ee8ff089cb53 2020-09-13T18:13:09+00:00 Jeff Epler jepler@gmail.com 2020-09-13T18:12:35+00:00 urn:sha1:90f7340bfc35d19fcf5e11c078c7f45399ed07e7 2020-09-13T16:50:01+00:00 Taku Fukada naninunenor@gmail.com 2020-09-13T16:25:13+00:00 urn:sha1:d18d79ac4709364c2371e2eaefd8f96bb20802e7 2020-09-13T00:43:08+00:00 Jeff Epler jepler@gmail.com 2020-09-13T00:39:35+00:00 urn:sha1:15964a4750d9eb1a34317c166d9bde5f40c9878d Most users and the CI system are running in configurations where Python configures stdout and stderr in UTF-8 mode. However, Windows is different, setting values like CP1252. This led to a build failure on Windows, because makeqstrdata printed Unicode strings to its stdout, expecting them to be encoded as UTF-8. This script is writing (stdout) to a compiler input file and potentially printing messages (stderr) to a log or console. Explicitly configure stdout to use utf-8 to get consistent behavior on all platforms, and configure stderr so that if any log/diagnostic messages are printed that cannot be displayed correctly, they are still displayed instead of creating an error while trying to print the diagnostic information. I considered setting the encodings both to ascii, but this would just be occasionally inconvenient to developers like me who want to show diagnostic info on stderr and in comments while working with the compression code. Closes: #3408 2020-09-12T20:11:29+00:00 Jeff Epler jepler@gmail.com 2020-09-12T19:14:30+00:00 urn:sha1:12d826d941f705fbfa0ced667d1c7efb264d95b1 I investigated these cases and confirmed that the fallthrough behavior was intentional.