boots-developers team mailing list archive
-
boots-developers team
-
Mailing list archive
-
Message #00002
Re: Regarding the blueprint on Perform proper parsing on lingo input.
On 04/05/2010 12:12 AM, Ashish wrote:
Hi
I went through that blog article, it is really very helpful in
understanding readline basics.
Awesome.
On Mon, Apr 5, 2010 at 5:44 AM, Monty Taylor <mordred@xxxxxxxxxxxx
<mailto:mordred@xxxxxxxxxxxx>> wrote:
On 04/04/2010 04:04 PM, Eric Day wrote:
I'm not up on Python lexers/parser generators. Can anyone else
comment
on Ply?
It's already being used in boots - so there should be some decent
example code in there.
I think that, if tab completion were to be done proper parsing has to be
done to identify the probable matches for completion in a particular
context.
I was unable find such parsing in boots source code. So, can you please
guide me to the part of source code does parsing for sql lingo.
Ah. I may have been unclear. I meant only that ply itself was in use -
not that the SQL parsing had been done. If you look in
boots/lingos/lisp/parser.py and boots/lingos/lisp/lexer.py you will see
the current use of ply implementing a lisp parser.
> Also I have questions related to it.
> Q0. Am I right ?
So far it sounds good, I'd see what some of the other devs have
to say.
> Q1. Do we then need to do a character by character scan
to look for
> tokens as they come? I think that if this blueprint is
to support
> syntax-highlighting in future then character by
character scan will be
> needed.
Initially I would only worry about scanning for completion on TAB,
but you certainly could do scans for each character for things like
syntax highlighting.
I would highly recommend not building your own completion scanner by
hand. The readline module or the rlcompleter module in the python
standard library are probably what you want to use for this. Check out:
http://blog.doughellmann.com/2008/11/pymotw-readline.html
For a quick how-to on how to use it.
Using readline also gives you history and command line editing in
addition to hooks for completion.
Yes use of readline has great advantages. But I am not able to figure
out that could it also help in implementing syntax highlighting?
Correct - it will not be useful in implementing syntax highlighting-
only in handing the processing of input.
For syntax highlighting, I recommend looking at pygments.
http://pygments.org/ It is used by the Sphinx documentation project
which is used to create the main python documentation.
One of the nice things about pygments is that they have lexers already
written for just about every language you could imagine (including SQL
of course)
for instance:
>>> from pygments.lexers.other import MySqlLexer
>>> foo = MySqlLexer()
>>> for f in foo.get_tokens("select * from foo"):
... print f
...
(Token.Keyword, u'select')
(Token.Text, u' ')
(Token.Operator, u'*')
(Token.Text, u' ')
(Token.Keyword, u'from')
(Token.Text, u' ')
(Token.Name, u'foo')
(Token.Text, u'\n')
So the pygments lexer could be potentially used when you work on your
command-line completion parser.
Monty
References
