zim-wiki team mailing list archive

[Jose Lourenco <jose@xxxxxxxxxxx>]

Hi,

Thanks for your suggestions.

They are interesting and helpful, namely the one regarding separating
larger files from Zim pages.

If you could read my reply to Jaap's, I noted why using my kind of workflow
and extensive data gathering I can't/wouldn't make notes for every single
or group of items downloaded. Sometimes I don't care to do that at that
particular instant as long as I may have some basic automatic linkage from
Zim's pages to children folders containing files if their names can be
indexed and searchable.

As such I am inclined to do some testing to see how and if this is feasible
and if I can do it.

It could also help me in the process of transforming my 3GB zim notebooks
into ones that have the separating structure you helpfully  suggested. That
is, transversing Zim notebooks' trees and if some criteria is matched,
files might be moved to another location, making the necessary changes on
affected parent and linked back Zim pages.

I really appreciated or suggestions and time taken.

Thanks again.
Best regards,

Jose

_______________________________________________________________
Para manter a Wikipedia a funcionar por favor considere fazer a sua doação.
Please consider donating to maintain Wikipedia alive.
 Support Wikipedia<http://wikimediafoundation.org/wiki/Support_Wikipedia/en>
 _______________________________________________________________
http://jal.stumbleupon.com
http://www.lourenco.ws/about-me



On Thu, Oct 18, 2012 at 7:46 PM, Jose Lourenco <jose@xxxxxxxxxxx> wrote:

> Hi Jaap,
>
>
> A personal thanks for Zim - an excellent tool.
>
> Also a congratulation for the quality of the code and sapience shown.
>
> This message I'm sending is very long and may be not that much important
> to the community.
>
> So, if for any reason you aren't able to read it, please rest assured that
> it is perfectly OK and that I am and will be grateful in any circumstance.
>
> But, if you manage to read it, I would very much appreciate any comment or
> suggestion you may find pertinent.
>
> Also, please accept apologizes for my "english" which is not my
> mother-language.
>
>
>
> I do a lot of researching on Internet and I am used to save almost
> everything that found useful in the process; not only links but also .mht
> files and PDF prints of pages consulted and others related files.
>
> Saving is needed  not only because of offline reading but, most
> importantly, to rest assured that consulted information will stay available
> for later usage and reference.
>
> So, I am using Zim almost like Zotero (http://www.zotero.org/).
>
> The process I am using on Zim is rather time consuming done manually - due
> to quantity - and is error prone.
>
> If filenames are not "normalized" as Zim expects, they don't get indexed.
> Additionally, if directories are "created manually" that information is not
> seen.
> Also, this can create general problems with indexing, making Zim
> inoperable because it detects these index-related problems. When this
> happens a "normalization" of all affecting paths/filenames is required to
> be done manually so that Zim will work again.
>
> I would like to automate this process as much as possible.
>
> I am doing and considering the following:
>
> 1) Put downloaded pages grouped by matters searched on related directories
> inside a general base drectory.
>     E.g.: /data/z3l/zim-wiki/Downloads-to-Zim
>
> 2) by making a simple change to the indexed functionality of Zim it
> outputs the fullpath of invalid files (and paths) when performing a full
> index.
>
>     ./zim.py --index  /data/z3l/zim-wiki/ &> files_list_invalid.txt
> (see Listing 1, below)
>
> 2.1) I tried to add an external option, such as --indexpaths but have not
> been successful (without considering a global variable) as I couldn't get
> this option information way to the function zim/stores/
> files.py/get_pagelist(self, path) where warnings are issued by writing to
> logger.warn() when appropriate.
> 2.1.1) I will try to use a different approach later.
>
> 3) I will read the created file (files_list_invalid.txt) and:
> 3.1) "normalize" the collected invalid paths as per Zim requirements:
>     * change '  ' to '_'
>     * skip "*.txt" files (only to be done if there isn't a corresponding
> directory, as .txt may need to be attached)
>     * skip ".*" and "_*" directories
>     * others that may be needed but I haven't found out yet
>
>     Note: this is now done "externally", that is, a python script is run
> and changes are made on filesystem.
>
> 4) the idea is then to run a script that:
> 4.1) moves the files to Notebooks' filesystem tree
> 4.2) moves directories to Notebooks' filesystem tree
> 4.2.1) creates a Zim text document ".txt" with the name of the directory,
> so that the directory may be indexed.
> 4.2.2) on the Zim page, create links to existing files inside the directory
> 4.2.3) repeat for each directory and recurse
>
> 4.3) pages will have a location derived from the directory name which
> contains tag information on it.
> Eg.:
>     * the content of directory named :
>       @Python,@Papers,@Storage;@Programming,@Dedupliction;esFS - Storage
> Efficient Filesystem in Python
>
>       will be saved on the following partially-existent (or to be created)
> path:
>
> ..../Python/Papers/Storage/FS_-_Storage_Efficient_Filesystem_in_Python/
>
>       The tags following the first ';' are to be added to the
> corresponding  .txt file to be created.
>      * the file FS_-_Storage_Efficient_Filesystem _in_Python.txt will be
> created on the parent dir
>      * links to files existing on directory
> FS_-_Storage_Efficient_Filesystem _in_Python will be inserted on the .txt
> file.
> Eg.:
>      FS_-_Storage_Efficient_Filesystem_in_Python.txt:
>      -------Begin-------
>             Wiki-Format: zim 0.4
>             Creation-Date: 2012-10-18T17:41:13+01:00
>
>             ====== FS - Storage Efficient Filesystem in Python ======
>             Created Quinta 18 Outubro 2012
>
>             [[./244-941-1-PB.pdf]]
>
>             [[./FUSE’ing_Python_for_Development_244.mht]]
>
>             @Python
>             @Papers
>             @Storage
>             @Programming
>             @Deduplication
>      -------End-------
>      * parent '.txt' files  may need to be actualized.
>
> 4.4) perform a Zim --index
>
> 5) Later, when more familiar with Zim innards, I would like to:
> 5.1) perform part of the above operations with the own Zim existing
> functionalities (fs.py, ...)
> 5.2) arrange a way to perform the actions wanted with a greater
> integration with Zim. For example:
>       * Zim plug-in for firefox would be nice, if some features were added
>       * make Zim (optionally) accept directly saved files and create
> directories, making him "normalize" paths/filenames and create
> corresponding .txt page files for directories.
>
> Listings:
> (1)
> z2@pclevo:~/Development/python/zim-0.57$ ./zim.py --index
> /data/z3l/zim-wiki
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@PDF,@PDF_Tools,@Okular;PDF
> editor annotations_Okular"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Papers,@Storage;@Programming,@Dedupliction;esFS
> - Storage Efficient Filesystem in Python"
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Papers;Python
> Papers"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Programming,@Testing;@Scripts,@Testing,@Examples,Tests
> in Python"
> WARNING:
> "/data/z3l/zim-wiki/Downloads-to-Zim/@Python,@Scripts,@Shell_tools,@Programming,@Examples,Python
> Shell Utilities"
> WARNING: "/data/z3l/zim-wiki/Downloads-to-Zim/@python,@lambda Python Lamba
> Funcrions
> ...
> (these were saved directly inside Zim's tree filesystem)
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/Python
> EasyGUI"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/UniCurses
> for Python"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/Urwid -
> Console User Interface Library"
> WARNING:
> "/data/z3l/zim-wiki/Notebooks/Notes/Home/Development/Python/TUI/ranger -
> file manager"
> WARNIN ...
>
>
> If you have any considerations, advises, or just your raw opinion about
> the usefulness and feasibility, they will be much appreciated.
>
> Meanwhile, please accept my
>
> Very best regards,
>
> Jose Lourenco
> _______________________________________________________________
> Para manter a Wikipedia a funcionar por favor considere fazer a sua doação.
> Please consider donating to maintain Wikipedia alive.
>  Support Wikipedia<http://wikimediafoundation.org/wiki/Support_Wikipedia/en>
>  _______________________________________________________________
> http://jal.stumbleupon.com
> http://www.lourenco.ws/about-me
>
>