Add comments

For now it only supports non-nested comments. It seems like I *will*
need to write an external lexer to account for nesting properly.
This commit is contained in:
Bruno BELANYI 2022-06-02 11:03:55 +02:00
parent e5e958cac9
commit 75bb2c7009
5 changed files with 2239 additions and 1718 deletions

86
test/corpus/comments.txt Normal file
View file

@ -0,0 +1,86 @@
================================================================================
Empty comment
================================================================================
/**/
--------------------------------------------------------------------------------
(source_file
(comment))
================================================================================
Whitespace comment
================================================================================
/* */
--------------------------------------------------------------------------------
(source_file
(comment))
================================================================================
Comment
================================================================================
/* This is a comment */
--------------------------------------------------------------------------------
(source_file
(comment))
================================================================================
Comment with slashes
================================================================================
/* /// */
--------------------------------------------------------------------------------
(source_file
(comment))
================================================================================
Nested comment
================================================================================
/* This is /* a /* nested */ comment */*/
--------------------------------------------------------------------------------
(source_file
(comment)
(identifier)
(ERROR
(operator)
(operator)
(operator)
(operator)))
================================================================================
Unterminated comment
================================================================================
/*
--------------------------------------------------------------------------------
(source_file
(ERROR
(operator)
(operator)))
================================================================================
Comment end-delimiter only
================================================================================
*/
--------------------------------------------------------------------------------
(source_file
(ERROR
(operator)
(operator)))