浏览代码

End in EOF

master
Dylan Baker 6 年前
父节点
当前提交
79c398f4b9
共有 2 个文件被更改,包括 11 次插入3 次删除
  1. 6
    2
      lib/chervil/lexer.rb
  2. 5
    1
      spec/lexer_spec.rb

+ 6
- 2
lib/chervil/lexer.rb 查看文件

21
       when ')'
21
       when ')'
22
         advance
22
         advance
23
         Token.new(:rparen, ")")
23
         Token.new(:rparen, ")")
24
+      when nil
25
+        Token.new(:eof, "eof")
24
       end
26
       end
25
     end
27
     end
26
 
28
 
27
     def tokenize
29
     def tokenize
28
       tokens = Array.new
30
       tokens = Array.new
29
-      until @position == @source.size
30
-        tokens << get_next_token
31
+      loop do
32
+        token = get_next_token
33
+        tokens << token
34
+        break if token.type == :eof
31
       end
35
       end
32
       tokens
36
       tokens
33
     end
37
     end

+ 5
- 1
spec/lexer_spec.rb 查看文件

2
   RSpec.describe Lexer do
2
   RSpec.describe Lexer do
3
     it 'lexes parens' do
3
     it 'lexes parens' do
4
       expect(Lexer.new('()').tokenize).to eq(
4
       expect(Lexer.new('()').tokenize).to eq(
5
-        [Token.new(:lparen, '('), Token.new(:rparen, ')')]
5
+        [
6
+          Token.new(:lparen, '('),
7
+          Token.new(:rparen, ')'),
8
+          Token.new(:eof, 'eof')
9
+        ]
6
       )
10
       )
7
     end
11
     end
8
   end
12
   end

正在加载...
取消
保存