zim-wiki team mailing list archive
-
zim-wiki team
-
Mailing list archive
-
Message #03795
New 'Pythonic' Plugin for Zim
Hi Jaap and others.
I use zim for all my note-taking, planning, etc., and I also want to
be able to do complex calculations right within zim, instead of doing
them somewhere else and copying in the result. Therefore, I first took
a look at the 'arithmetic' plugin, which is definitely going into the
right direction, but it's very limited and has some bugs. I think that
it doesn't make sense to maintain an own arithmetic syntax and parser
etc. when we have a perfect solution available: python itself. I
usually use python anyway to do all kinds of calculations. Therefore I
created the attached plugin, which allows using python directly within
zim, not only for arithmetic calculations but for any kind of
calculations. I'm using this as a kind of Excel replacement. It's also
a great replacement for the python CLI instance which I used to have
open all the times for quick calculations.
Obviously, it's potentially dangerous to execute python code from the
zim notebook. Therefore this should only be used with own and
trustworthy notebooks. On each zim wiki page, when executing the
embedded python code for the first time, a popup will be shown which
shows the exact embedded code that the plugin collected from the page
and that will be executed, and the user has a chance to review the
code and eventually abort execution. Python code is embedded using the
'#' sign. For further information, please see the manual page I wrote.
For example, of you want to calculate how many dollars you can make
out of 500 dollars in 10 years with interest rates of 5%, you could do
it like this:
#dollars = 500; interest = 1.05; years = 10
#dollars_after_10_years = dollars * (interest ** years)
#dollars_after_10_years
-> press F5, and get:
#dollars_after_10_years ? 814.447313389
I'd be happy if this plugin would be included in Zim in future,
there's nothing similarly powerful available yet. I'm willing to keep
maintaining it. Any feedback and discussion is welcome.
Greetings, David
ps. I already sent an older version of the plugin to Jaap, this is an
improved version.
=== added file 'data/manual/Plugins/Pythonic.txt'
--- data/manual/Plugins/Pythonic.txt 1970-01-01 00:00:00 +0000
+++ data/manual/Plugins/Pythonic.txt 2016-01-30 22:14:02 +0000
@@ -0,0 +1,90 @@
+Content-Type: text/x-zim-wiki
+Wiki-Format: zim 0.4
+Creation-Date: 2016-01-02T01:00:41+01:00
+
+====== Pythonic ======
+Created Samstag 02 Januar 2016
+
+This plugin allows you to embed arbitrary python code and python calculations in zim.
+
+__WARNING__: This plugin should only be used with own and trustworthy notebooks, because arbitrary code can be executed from within the notebook when the **F5** hotkey is pressed. However, the plugin will print a message which shows the executed code when pressing the hotkey for the first time on a page.
+
+Python blocks are started and terminated with the **#** sign. If a block isn't terminated using a **#** sign, then it must start at the beginning of the line; then it automatically ends at the end of the line. Multi-line blocks can be started/ended using **###** lines.
+
+If the python block is a simple expression, then the result of the expression will be printed at the end of the block behind a **?** sign. If the block is an assignment expression, then the result of the assignment can be printed by manually adding a **?** sign at the end of the block.
+
+You can also import python packages etc., the directory of the current document and its parent document are inserted into the module path.
+
+Press **F5** to evaluate all python code in the current document.
+
+**Example**:
+
+'''
+###
+dollars = 500
+interest = 1.05
+years = 10
+dollars_afterwards = dollars * (interest ** years)
+###
+
+#dollars_afterwards
+'''
+
+Pressing **F5** yields:
+
+'''
+#dollars_afterwards? 814.447313389
+'''
+
+Each time **F5** is pressed, the whole file is processed top-down, and all printed result values are updated accordingly.
+
+Result values can be requested explicitly even within multiline blocks, by appending **?** at the end of the line; the line is then processed as a single-line statement:
+
+'''
+###
+dollars = 500
+interest = 1.05
+years = 10
+dollars_afterwards = dollars * (interest ** years) ?
+###
+'''
+
+Pressing **F5** yields:
+
+'''
+###
+dollars = 500
+interest = 1.05
+years = 10
+dollars_afterwards = dollars * (interest ** years) ? 814.447313389
+###
+'''
+
+Newlines in result values are replaced by spaces, unless the results are printed within a multiline block. Additionally inserted lines are prefixed by **?** and some alignment spaces. Old results (or anything else behind a **?** sign) are deleted from the block whenever it is re-processed.
+
+**Example**:
+
+'''
+###
+a = "two\nlines"
+a?
+###
+
+#a
+'''
+
+Pressing **F5** yields:
+
+'''
+###
+a = "two\nlines"
+a? two
+? lines
+###
+
+#a? two lines
+'''
+
+__Note 1__: When the python code triggers an error, then the error is also printed like a result, preceded by a **?** sign. The inserted **?** signs will then cause the corresponding line to be parsed as a single-line statement, which may cause further compilation errors. Therefore it may be necessary to remove the **?** signs that were inserted by error messages after fixing the error.
+
+__Note 2__: All python code is executed directly within the zim python process, so avoid messing around with globals that might cause zim to crash, and avoid slow operations which might lock up zim for a long time.
=== added file 'zim/plugins/pythonic.py'
--- zim/plugins/pythonic.py 1970-01-01 00:00:00 +0000
+++ zim/plugins/pythonic.py 2016-01-30 22:09:39 +0000
@@ -0,0 +1,276 @@
+# Copyright 2015 David Nolden <david.nolden.kdevelop@xxxxxxxxxxxxx>
+#
+# Plugin to use python code embedded in zim wiki
+
+from zim.inc.arithmetic import ParserGTK
+from zim.plugins import PluginClass, WindowExtension, extends
+from zim.actions import action
+import sys
+import gc
+import gtk
+from zim.gui.widgets import ErrorDialog
+
+class PythonicPlugin(PluginClass):
+
+ plugin_info = {
+ 'name': _('Pythonic'), # T: plugin name
+ 'description': _('''\
+This plugin allows you to embed arbitrary python code and python calculations in zim.
+
+WARNING: This plugin should only be used with own and trustworthy notebooks, because arbitrary code can be executed from within the notebook. A message popup is printed before executed the code of a page for the first time though.
+'''), # T: plugin description
+ 'author': 'David Nolden',
+ 'help': 'Plugins:Pythonic',
+ }
+
+class Parser:
+ def __init__(self, textbuffer, only_collect=False):
+ self.__text_buffer = textbuffer
+ self._only_collect = only_collect
+ if (self._only_collect):
+ self.code = ""
+ else:
+ self.code = None
+
+ def get_line(self, i_line):
+ start_it = self.__text_buffer.get_iter_at_line(i_line)
+ if start_it.ends_line():
+ return ""
+ end_it = self.__text_buffer.get_iter_at_line(i_line)
+ end_it.forward_to_line_end()
+ return self.__text_buffer.get_text(start_it, end_it)
+
+ def parse(self):
+ self.__parse_globals = {}
+ self.__parse_locals = {}
+ i = 0
+ while i < self.__text_buffer.get_line_count():
+ if self.get_line(i).strip() == "###":
+ # parse multi-line block
+ i_end = i + 1
+ while i_end < self.__text_buffer.get_line_count() and self.get_line(i_end).strip() != "###":
+ i_end += 1
+ if i_end == self.__text_buffer.get_line_count():
+ i += 1
+ continue # didn't find terminating '###'
+ accum_code = None
+ i2 = i + 1
+ i2_start = i2
+ while i2 < self.__text_buffer.get_line_count() and self.get_line(i2).strip() != "###":
+ line_code = self.get_line(i2)
+ if line_code.startswith("?"):
+ # Delete old result line
+ if not self._only_collect:
+ start_it = self.__text_buffer.get_iter_at_line(i2)
+ end_it = self.__text_buffer.get_iter_at_line(i2+1)
+ self.__text_buffer.delete(start_it, end_it)
+ else:
+ i2 += 1
+ continue
+ if "?" not in line_code or line_code.startswith("#"):
+ # accumulate code
+ if (accum_code == None):
+ accum_code = line_code
+ else:
+ accum_code += "\n"+line_code
+ i2 += 1
+ continue
+ # This line contains a '?' sign, and needs to be parsed
+ # as a single-line statement, in which we can print the result.
+ if accum_code != None and len(accum_code):
+ # first execute preceding code
+ i2 = self.execute_code_lines(accum_code, i2_start, i2)
+ accum_code = None
+ # now parse the line itself
+ i2 = self.execute_code_lines(line_code, i2, i2+1)
+ i2_start = i2
+
+ if accum_code != None and len(accum_code):
+ # parse remaining accumulated code
+ assert(self.get_line(i2).strip() == "###")
+ i2 = self.execute_code_lines(accum_code, i2_start, i2)
+
+ assert(self.get_line(i2).strip() == "###")
+
+ i = i2+1
+ else:
+ # parse single line
+ self.parse_line(i)
+ i += 1
+
+ def execute_code_lines(self, text, i_start, i_end):
+ new_text = self.execute_code_block(text, True)
+ if (new_text != text):
+ n_new_lines = new_text.count("\n")
+ start_it = self.__text_buffer.get_iter_at_line(i_start)
+ end_it = self.__text_buffer.get_iter_at_line(i_end-1)
+ if (not end_it.ends_line()):
+ end_it.forward_to_line_end()
+
+ self.replace_text(start_it, end_it, new_text)
+ return i_start + 1 + n_new_lines
+ # Need to replace and update i_end
+ return i_end
+
+ def execute_code_block(self, code, allow_newline):
+ out_text = ""
+ need_result = False
+ if "?" in code:
+ need_result = True
+ code = code[:code.index("?")]
+ out_text += code
+
+ if (self._only_collect):
+ self.code += code+"\n"
+ return code
+
+ result = None
+ try:
+ result = eval(code, self.__parse_globals, self.__parse_locals)
+ except Exception as e:
+ try:
+ exec(code, self.__parse_globals, self.__parse_locals)
+ except Exception as e:
+ result = "ERROR: " + str(e).replace("#", "_")
+ if need_result and result == None:
+ # If it is an assignment statement, evaluate the assigned value
+ if "=" in code:
+ varname = code[:code.index("=")]
+ try:
+ result = eval(varname, self.__parse_globals, self.__parse_locals)
+ except:
+ pass
+ if result != None:
+ res = str(result)
+ if not allow_newline:
+ res = res.replace("\n", " ")
+ else:
+ alignSpaces = len(out_text) + 1
+ res = res.replace("\n", "\n?" + " "*alignSpaces)
+ out_text += "? "+res
+
+ return out_text
+
+ def parse_line(self, i_line):
+ it = self.__text_buffer.get_iter_at_line(i_line)
+ if (it.ends_line()):
+ return
+ end_it = it.copy()
+ end_it.forward_to_line_end()
+ if (end_it.is_end() and '#' in it.get_text(end_it)):
+ # append a newline so that we can detect the end of the block
+ self.__text_buffer.insert(end_it, "\n")
+ it = self.__text_buffer.get_iter_at_line(i_line)
+
+ def get_end_line_it():
+ ret = self.__text_buffer.get_iter_at_line(i_line)
+ ret.forward_to_line_end()
+ if (not ret.is_end()):
+ ret.forward_char() # include the newline so we can detect it
+ return ret
+
+ end_it = get_end_line_it()
+
+ replacements = []
+
+ code_start = None
+ only_had_code = True
+ while it.compare(end_it) == -1:
+ char = it.get_char()
+ if char != "#" and char != " " and code_start == None:
+ only_had_code = False
+ if (char == '#' or (char == "\n" and code_start != None)):
+ if (code_start != None):
+ # code block ends; only accept unterminated blocks if they
+ # start at the beginning of the line
+ if (char != "\n" or only_had_code):
+ offset = it.get_line_offset()
+ old_text = code_start.get_text(it)
+ new_text = self.execute_code_block(code_start.get_text(it), False)
+ self.replace_text(code_start, it, new_text)
+ if not self._only_collect:
+ # fix iterators:
+ offset += len(new_text) - len(old_text)
+ it = self.__text_buffer.get_iter_at_line(i_line)
+ it.forward_chars(offset)
+ end_it = get_end_line_it()
+
+ code_start = None
+ else:
+ code_start = it.copy()
+ code_start.forward_char()
+ it.forward_char()
+
+ def replace_text(self, start_it, end_it, new_text):
+ if (self._only_collect):
+ return
+ if start_it.get_text(end_it) != new_text and not self._only_collect:
+ tags = start_it.get_tags()
+ self.__text_buffer.delete(start_it, end_it)
+ self.__text_buffer.insert_with_tags(start_it, new_text, *tags)
+
+@extends('MainWindow')
+class MainWindowExtension(WindowExtension):
+
+ uimanager_xml = '''
+ <ui>
+ <menubar name='menubar'>
+ <menu action='tools_menu'>
+ <placeholder name='plugin_items'>
+ <menuitem action='calculate'/>
+ </placeholder>
+ </menu>
+ </menubar>
+ </ui>
+ '''
+
+ _warned = set()
+
+ @action(_('_Pythonic'), accelerator='F5') # T: menu item
+ def calculate(self):
+ # update module path and backup the list of modules
+ old_modules = dict(sys.modules)
+ buf = self.window.pageview.view.get_buffer()
+ sys.path.insert(0, str(self.window.ui.notebook.get_attachments_dir(buf.page)))
+ if buf.page.parent != None:
+ sys.path.insert(1, str(self.window.ui.notebook.get_attachments_dir(buf.page.parent)))
+ need_warning = buf.page.name not in self._warned
+ if (need_warning):
+ parser = Parser(buf, True)
+ parser.parse()
+ dialog = gtk.MessageDialog(self.window,
+ message_format=_('Executing the following embedded python code:'),
+ # T: main text for executing embedded python code in the pythonic plugin
+ buttons=gtk.BUTTONS_OK_CANCEL)
+ if (len(parser.code)):
+ dialog.format_secondary_text(
+ _("When you answer OK, this notification won't be printed again for this page:\n\n%s") % parser.code)
+ # T: text for executing embedded python code in the pythonic plugin
+ else:
+ dialog.format_secondary_text(
+ _("There is no python code embedded in this page. See the documentation of the pythonic plugin for further information how to embed code."))
+ # T: text for executing embedded python code in the pythonic plugin
+ dialog.set_default_response(gtk.RESPONSE_CANCEL)
+ response = dialog.run()
+ dialog.destroy()
+ if response != gtk.RESPONSE_OK:
+ return # abort
+ self._warned.add(buf.page.name)
+
+ # get the buffer
+ parser = Parser(buf)
+ buf.begin_user_action()
+ parser.parse()
+ buf.end_user_action()
+
+ # revert module path and revert the list of modules,
+ # so that every call starts with a 'clean sheet'.
+ if buf.page.parent != None:
+ del sys.path[1]
+ del sys.path[0]
+
+ for (modname, module) in list(sys.modules.items()):
+ if not modname in old_modules:
+ del sys.modules[modname]
+ del old_modules
+ gc.collect()
Follow ups
