suspect-devices/linguist/test/test_tokenizer.rb, branch v2.2.0 Language Savant https://git.suspectdevices.com/suspect-devices/linguist/atom?h=v2.2.0 2012-08-03T20:07:36+00:00 Move shebang detection into classifier 2012-08-03T20:07:36+00:00 Joshua Peek josh@joshpeek.com 2012-08-03T20:07:36+00:00 urn:sha1:16a67cb852342b7c26cdde272458eb48cd016f4b Fixes #203 Fix tokenzing empty strings 2012-07-24T16:49:29+00:00 Joshua Peek josh@joshpeek.com 2012-07-24T16:49:29+00:00 urn:sha1:e5d302459fd718ca26a1d9477a1c5220092a8fd1 Add brackets to tokens 2012-07-24T16:28:27+00:00 Joshua Peek josh@joshpeek.com 2012-07-24T16:28:27+00:00 urn:sha1:53300ca5812a3f19b4ea1c670e286d68b900c7cf Parse float tokens 2012-07-24T16:23:06+00:00 Joshua Peek josh@joshpeek.com 2012-07-24T16:23:06+00:00 urn:sha1:f5705eaf3872fa5872dd7500483f8c01e53d9768 Rename samples subdirectories 2012-07-23T20:52:49+00:00 Joshua Peek josh@joshpeek.com 2012-07-23T20:52:49+00:00 urn:sha1:7b6caa0f6c7603aabb5e6ed657e24aaddbeafc78 Move specific filename samples into their own dir 2012-07-20T22:20:45+00:00 Joshua Peek josh@joshpeek.com 2012-07-20T22:20:45+00:00 urn:sha1:36da52e3838ed50f9330f7e26cc8aa81d7bc7f09 Add tokenize helper to Tokenize class 2012-07-20T20:14:58+00:00 Joshua Peek josh@joshpeek.com 2012-07-20T20:14:58+00:00 urn:sha1:2637d8dc55a228d7aec793016632aed2dbf052d3 Extract single and multi line comment parser 2012-07-20T20:06:21+00:00 Joshua Peek josh@joshpeek.com 2012-07-20T20:06:21+00:00 urn:sha1:175d4244c2f4848f6e4d50af6c5cd45337490134 Add coq comments 2012-07-20T19:45:19+00:00 Joshua Peek josh@joshpeek.com 2012-07-20T19:45:19+00:00 urn:sha1:d063089430491fa6398ecebbe0ef7936634f34ea Move test fixtures to samples/ 2012-06-22T15:09:24+00:00 Joshua Peek josh@joshpeek.com 2012-06-22T15:09:24+00:00 urn:sha1:5521dd08a062f39036b045c819201b6235fb32f8