zim-wiki team mailing list archive

[Jaap Karssenberg <jaap.karssenberg@xxxxxxxxx>]

Hi Jose,

reading over this description, I think I see two different issues that we
shoudl probably separate.

1/ When you save files directly into the zim folder structure it messes up
the index.

This is a bug. Any file should be allowed as attachment and be visible e.g.
in the attachment browser pane. If you have examples of specific situations
that cause bugs, please put them in the bug tracker.

The only restriction is that when you create folders to download the files
to, these folders should follow zim naming conventions.

2/ You want to auto-generate pages that have links to the files and tags

Question from my side is if this is really needed if the indexing is fixed.
Say you download a bunch of files to a folder in the zim notebook. When the
index detects the new folder it results in an empty page with a bunch of
attachments which you can see in the attachment browser pane. You might
have to add the tags manually to the page, but probably you have it open
anyway to add more notes etc.

What I often do is open a page, write some notes, then click the folder
icon in the attachment browser. This opens up my standard file browser for
that folder. Then I just drag and drop whatever I have in attachments etc.
to that folder.


Another thought is that if your workflow is specific from e.g. firefox, you
may also think about a firefox plugin that facilitates the flow by
downloading to the right folder with a single click, allowing you to add
tags on the fly etc.

Regards,

Jaap


On Thu, Oct 18, 2012 at 8:46 PM, Jose Lourenco <jose@xxxxxxxxxxx> wrote:

> Hi Jaap,
>
>
> A personal thanks for Zim - an excellent tool.
>
> Also a congratulation for the quality of the code and sapience shown.
>
> This message I'm sending is very long and may be not that much important
> to the community.
>
> So, if for any reason you aren't able to read it, please rest assured that
> it is perfectly OK and that I am and will be grateful in any circumstance.
>
> But, if you manage to read it, I would very much appreciate any comment or
> suggestion you may find pertinent.
>
> Also, please accept apologizes for my "english" which is not my
> mother-language.
>
>
>
> I do a lot of researching on Internet and I am used to save almost
> everything that found useful in the process; not only links but also .mht
> files and PDF prints of pages consulted and others related files.
>
> Saving is needed  not only because of offline reading but, most
> importantly, to rest assured that consulted information will stay available
> for later usage and reference.
>
> So, I am using Zim almost like Zotero (http://www.zotero.org/).
>
> The process I am using on Zim is rather time consuming done manually - due
> to quantity - and is error prone.
>
> If filenames are not "normalized" as Zim expects, they don't get indexed.
> Additionally, if directories are "created manually" that information is not
> seen.
> Also, this can create general problems with indexing, making Zim
> inoperable because it detects these index-related problems. When this
> happens a "normalization" of all affecting paths/filenames is required to
> be done manually so that Zim will work again.
>
> I would like to automate this process as much as possible.
>
> I am doing and considering the following:
>
> 1) Put downloaded pages grouped by matters searched on related directories
> inside a general base drectory.
>     E.g.: /data/z3l/zim-wiki/Downloads-to-Zim
>
> 2) by making a simple change to the indexed functionality of Zim it
> outputs the fullpath of invalid files (and paths) when performing a full
> index.
>
>     ./zim.py --index  /data/z3l/zim-wiki/ &> files_list_invalid.txt
> (see Listing 1, below)
>
> 2.1) I tried to add an external option, such as --indexpaths but have not
> been successful (without considering a global variable) as I couldn't get
> this option information way to the function zim/stores/
> files.py/get_pagelist(self, path) where warnings are issued by writing to
> logger.warn() when appropriate.
> 2.1.1) I will try to use a different approach later.
>
> 3) I will read the created file (files_list_invalid.txt) and:
> 3.1) "normalize" the collected invalid paths as per Zim requirements:
>     * change '  ' to '_'
>     * skip "*.txt" files (only to be done if there isn't a corresponding
> directory, as .txt may need to be attached)
>     * skip ".*" and "_*" directories
>     * others that may be needed but I haven't found out yet
>
>     Note: this is now done "externally", that is, a python script is run
> and changes are made on filesystem.
>
> 4) the idea is then to run a script that:
> 4.1) moves the files to Notebooks' filesystem tree
> 4.2) moves directories to Notebooks' filesystem tree
> 4.2.1) creates a Zim text document ".txt" with the name of the directory,
> so that the directory may be indexed.
> 4.2.2) on the Zim page, create links to existing files inside the directory
> 4.2.3) repeat for each directory and recurse
>
> 4.3) pages will have a location derived from the directory name which
> contains tag information on it.
> Eg.:
>     * the content of directory named :
>       @Python,@Papers,@Storage;@Programming,@Dedupliction;esFS - Storage
> Efficient Filesystem in Python
>
>       will be saved on the following partially-existent (or to be created)
> path:
>
> ..../Python/Papers/Storage/FS_-_Storage_Efficient_Filesystem_in_Python/
>
>       The tags following the first ';' are to be added to the
> corresponding  .txt file to be created.
>      * the file FS_-_Storage_Efficient_Filesystem _in_Python.txt will be
> created on the parent dir
>      * links to files existing on directory
> FS_-_Storage_Efficient_Filesystem _in_Python will be inserted on the .txt
> file.
> Eg.:
>      FS_-_Storage_Efficient_Filesystem_in_Python.txt:
>      -------Begin-------
>             Wiki-Format: zim 0.4
>             Creation-Date: 2012-10-18T17:41:13+01:00
>
>             ====== FS - Storage Efficient Filesystem in Python ======
>             Created Quinta 18 Outubro 2012
>
>             [[./244-941-1-PB.pdf]]
>
>             [[./FUSE’ing_Python_for_Development_244.mht]]
>
>             @Python
>             @Papers
>             @Storage
>             @Programming
>             @Deduplication
>      -------End-------
>      * parent '.txt' files  may need to be actualized.
>
> 4.4) perform a Zim --index
>
> 5) Later, when more familiar with Zim innards, I would like to:
> 5.1) perform part of the above operations with the own Zim existing
> functionalities (fs.py, ...)
> 5.2) arrange a way to perform the actions wanted with a greater
> integration with Zim. For example:
>       * Zim plug-in for firefox would be nice, if some features were added
>       * make Zim (optionally) accept directly saved files and create
> directories, making him "normalize" paths/filenames and create
> corresponding .txt page files for directories.
>
> Listings:
> (1)
> z2@pclevo:~/Development/python/zim-0.57$ ./zim.py --index
> /data/z3l/zim-wiki
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@PDF,@PDF_Tools,@Okular;PDF
> editor annotations_Okular"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Papers,@Storage;@Programming,@Dedupliction;esFS
> - Storage Efficient Filesystem in Python"
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Papers;Python
> Papers"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Programming,@Testing;@Scripts,@Testing,@Examples,Tests
> in Python"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Scripts,@Shell_tools,@Programming,@Examples,Python
> Shell Utilities"
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@python,@lambda Python Lamba
> Funcrions
> ...
> (these were saved directly inside Zim's tree filesystem)
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/Python
> EasyGUI"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/UniCurses
> for Python"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/Urwid -
> Console User Interface Library"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/ranger -
> file manager"
> WARNIN ...
>
>
> If you have any considerations, advises, or just your raw opinion about
> the usefulness and feasibility, they will be much appreciated.
>
> Meanwhile, please accept my
>
> Very best regards,
>
> Jose Lourenco
> _______________________________________________________________
> Para manter a Wikipedia a funcionar por favor considere fazer a sua doação.
> Please consider donating to maintain Wikipedia alive.
>  Support Wikipedia<http://wikimediafoundation.org/wiki/Support_Wikipedia/en>
>  _______________________________________________________________
> http://jal.stumbleupon.com
> http://www.lourenco.ws/about-me
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~zim-wiki
> Post to     : zim-wiki@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~zim-wiki
> More help   : https://help.launchpad.net/ListHelp
>
>