aboutsummaryrefslogtreecommitdiff
path: root/test/test_tokenizer.rb
AgeCommit message (Collapse)Author
2012-08-03Move shebang detection into classifierJoshua Peek
Fixes #203
2012-07-24Fix tokenzing empty stringsJoshua Peek
2012-07-24Add brackets to tokensJoshua Peek
2012-07-24Parse float tokensJoshua Peek
2012-07-23Rename samples subdirectoriesJoshua Peek
2012-07-20Move specific filename samples into their own dirJoshua Peek
2012-07-20Add tokenize helper to Tokenize classJoshua Peek
2012-07-20Extract single and multi line comment parserJoshua Peek
2012-07-20Add coq commentsJoshua Peek
2012-06-22Move test fixtures to samples/Joshua Peek
2012-06-21Strip tex and matlab leading inline commentsJoshua Peek
2012-06-20Improve operator tokenizingJoshua Peek
2012-06-20Skip number literalsJoshua Peek
2012-06-19Remove dead fixture testJoshua Peek
2012-06-19Refactor tokenizer test helperJoshua Peek
2012-06-08Improve tokenizing sgml tagsJoshua Peek
2012-06-07Add basic TokenizerJoshua Peek