calibre-devs team mailing list archive

[Kovid Goyal <kovid@xxxxxxxxxxxxxx>]

First let me apologize for what is going to be a not very substantial reply, 
but I've been veritably drowning in calibre bug reports. It's almost like the 
little critters have an instinct about an impending major release.

I haven't had a chance to look at the code in detail as yet, but here are a 
couple of comments:

1) General Transforms
One example would be merging metadata from input plugin and user options
Another would be doing the various preprocessing tasks on the input to make it 
saner/easier to parse (there is currently some preprocessing of both HTML and 
CSS) in ebooks.html

2) Customizing transforms
The way I envision the system right now is that each plugin is developed 
independently in its own module/package. Each plugin will also export a set of 
options to customize its behavior. The calibre built-in plugins will know 
about each other and will therefore be able to recommend defaults for other 
plugins. It will be the responsibility of the plumber (the code creating the 
conversion pipeline) to rationalize all these recommendations and thereby 
establish values for every option. Recommendations come from three sources

1) The plugin itself for its own options
2) other plugins in the pipeline
3) User options

Furthermore, recommendations can have 3 priorities. low, medium and high.
The algorithm for resolving recommendations is;

1) higher priority trumps lower priority
2) If the priorities are equal, the resolution order is

recommendation from other plugin > user specified recommendation > 
recommendation by self

3) User recommendations will have high priority. So only high recommendations 
from other plugins can override them. If there is a high recommendation from 
another plugin, then the plumber will not present/disable the corresponding 
option in the user interface. So for example if the LIT output plugin 
recommends that input HTML should be split on page breaks, then the HTML 
splitting plugin will be forced to split on plugins.

4) In the case of a conflict (for e.g. two conflicting recommendations from 
other plugins of the same priority), we have two choices either the plugin 
that occurs earlier wins or the later plugin wins. I'm inclined towards the 
later plugin winning as this is likely to produce a valid output with lower 
fidelity whereas the other approach is more likely to produce invalid output 
but with higher fidelity.

@Marshall: I will digest your code over the next couple of days and then start 
creating the plugin code to address the needs of that code as well as the EPUB 
and LRF output code.

Comments are most welcome.

Kovid.

On Wednesday 11 February 2009 08:52:59 Marshall T. Vandegrift wrote:
> Kovid etc,
>
> I've pushed some new revisions up to lp:~llasram/calibre/pluginize.
> I've implemented a cheesy version of the needed `ebook-convert` glue
> code in calibre.ebooks.oeb.factory.main(), updated the OEBReader and
> LitReader to Reader interface we last discussed, and converted
> MobiWriter to the discussed Writer interface.  And it all works! -- it
> can convert a LIT book to a Mobipocket book with no intermediate output
> or Mobi-output-specific logic.
>
> The only change from what we've discussed is that I'm not sure there's
> actually any room for a set of "standard transforms" applied to all
> conversions.  I think the combination of input and output format (or
> just output format...) are going to need to cover everything.  Example
> -- how the MobiWriter needs to normalize fonts with its own options.  If
> that would be part of the "standard transforms" then we need some way of
> allowing writers to cancel out or skip specific ones.
>
> This raises the issue of transforms in user-provided plugins and how
> they get into the conversion process.  I think the easiest thing to do
> would be to have a Transform in the Writer TRANSFORMS list which just
> proxies out to the set of user-provided transforms.  It can even occur
> multiple times in the TRANSFORMS lists for plugins which should apply at
> specific points in the pipeline (e.g., pre- or post-CSS-normalization).
>
> And also among the revisions pushed is one which adds documentation to
> almost all of the "public" interface aspects of OEBBook.  Hopefully it
> will be helpful if anyone else wants to write conversion pipeline
> plugins :-).
>
> Anyway, please take a look at the code and me know if it looks like what
> you've had in mind.
>
> -Marshall
>
> _______________________________________________
> Mailing list: https://launchpad.net/~calibre-devs
> Post to     : calibre-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~calibre-devs
> More help   : https://help.launchpad.net/ListHelp
>
> !DSPAM:3,4993027975729584716549!

-- 
Kovid Goyal
www.kovidgoyal.net