kicad-developers team mailing list archive

[Cirilo Bernardo <cirilo.bernardo@xxxxxxxxx>]

On Sun, Nov 1, 2015 at 11:52 PM, Wayne Stambaugh <stambaughw@xxxxxxxxx>
wrote:

> On 10/31/2015 6:04 PM, Cirilo Bernardo wrote:
> >
> >
> > On Sun, Nov 1, 2015 at 6:35 AM, Wayne Stambaugh <stambaughw@xxxxxxxxx
> > <mailto:stambaughw@xxxxxxxxx>> wrote:
> >
> >     On 10/29/2015 4:16 AM, Cirilo Bernardo wrote:
> >     > Work so far on my part of the 3D refactor has resulted in the
> >     > development of a
> >     > dynamic plugin scheme which has been shown to work on Linux, OSX,
> and
> >     > MSWin. Although no plugins are currently implemented which provide
> >     an actual
> >     > useful function I think the improvements provided by the 3D model
> >     filename
> >     > resolver would be worth merging into the main branch soon after
> >     the release
> >     > since they finally resolve many longstanding issues with 3D model
> >     loading
> >     > and selection - but that's a topic for debate at a later date.
> >     >
> >     > WARNING: this is a long email but it is a genuine technical
> >     document with
> >     > the fundamental ideas already implemented and tested, it is not
> some
> >     > philosophical document about wild unimplementable ideas.
> >     >
> >     > What I would like to present here are some ideas for the
> >     implementation of
> >     > dynamic plugins within KiCad to improve code maintainability and
> >     provide
> >     > an easy means of extending KiCad's features. For example, at the
> >     moment
> >     > if I wish to implement an Export function in pcbnew (such as VRML
> >     export,
> >     > IDF export, or the IGES export that I'm itching to get into KiCad)
> >     I need to
> >     > touch numerous files, add a static menu item to File->Exporter,
> >     extend the
> >     > PCB_BASE_FRAME interface, etc then recompile *all* of pcbnew since
> I
> >     > touched PCB_BASE_FRAME which is a fundamental class which affects
> >     > so many other classes. With a dynamic plugin system we would be
> >     able to
> >     > compile an exporter out-of-tree if necessary and the Export item
> >     would be
> >     > dynamically added to File->Export. There would be no need to make
> >     invasive
> >     > modifications/extensions to PCB_BASE_FRAME just to service one more
> >     > Export function (all Export items will be passed by PCB_BASE_FRAME
> to
> >     > an Export Manager which is responsible for loading the appropriate
> >     > plugin etc).
> >     > One fantastic thing here is that there is no need to recompile all
> of
> >     > pcbnew,
> >     > which is a desirable effect especially as an Export Plugin is being
> >     > developed
> >     > and debugged.
> >     >
> >     > One item to consider is the use of dynamic libraries within KiCad;
> >     at the
> >     > moment all KiCad source is statically linked although we do dynamic
> >     > linking to external libraries like wxWidgets. I would like to know
> if
> >     > there is
> >     > any fundamental opposition to parts of kicad being linked to as
> >     > shared objects instead. The reason for this is that shared objects
> may
> >     > make it much easier to link KiCad and dynamic plugins to code
> modules
> >     > which may be required by various tools within kicad.
> >
> >     This statement is not correct.  All of the *.kiface files are shared
> >     objects.  They were just renamed .kiface to prevent file naming
> >     conflicts.  At some point, they will be renamed with the appropriate
> >     library extension and installed in the platform specific library
> path.
> >     The separation of objects in these libraries may be less than
> ideal.  I
> >     know Dick started some of the work but I believe there is more work
> to
> >     do to remove any of the UI dependencies from these libraries.
> >
> >
> > OK, so I take it adding more shared library modules will be ok?
>
> If by shared libraries you are talking about plugins, adding new
> libraries is fine.  If you are talking about breaking kicad code into
> more libraries than we already have, this requires serious discussion
> and planning.
>
>
I have no plans to break kicad into more libraries but to add some code as
libraries; in the short term one such body of code which I have in mind is
the code for the intermediate scenegraph representation for the 3D cache.
The IDF framework may also be built as a shared library since that
was the original design intent, but that particular task is a low priority.


> >
> > What do you think of a PCBExport API ayer being added to facilitate
> > the use of dynamic plugins for the File->Export  functions?
>
> I have no issue with dynamic plugins.  Please keep in mind there are a
> lot of factors to consider that must be address up front before this can
> happen.
>
> 1) Design.  I will expect a sound design before I allow any dynamic
> plugin code in kicad.  If I see highly coupled interdependent code, I
> will reject it.
>
>
You can bet there's no strong coupling; everything being developed in the
3D refactor is essentially developed and tested out of tree and classes
have a typical tree-like dependency.


> 2) Licensing: Dynamic plugins make licensing a much trickier issue.  I
> would lean towards GPL3+ for plugins.  There may be other licensing
> options but I would be opposed to permissive licenses such as BSD.
>
>
The only plugins which I plan are meant to be in-tree so my plan was
GPL2+ like the rest of the code unless there are any specific advantages
to having plugin code as GPL3+.


> 3) Third party development support: Provisions for installing header
> files and library link files for third party plugin development will
> have to be added to the CMake build configuration.  This opens the whole
> dynamic library version can of worms and the support issues that will
> come with it.  My expectation is that multiple library versions will be
> supported.  This is primarily for development purposes so I can keep a
> stable version of KiCad installed along side my development versions.
>
> Wayne
>
>
No problem; it's easy to group the headers needed for plugin development.
As for linking, there's nothing to link. kicad links to plugins at runtime
and
the API provided by kicad is passed to the plugin via function pointers;
it's
all fairly typical of runtime loaded libraries. The plugins themselves will
typically have nothing to link to in kicad - if they did this would defeat
the
purpose of a plugin. Version rules will follow the normal - changes in major
version number means the API is incompatible will earlier versions and
minor numbers represent changes which don't break the API.

- Cirilo


> >
> > - Cirilo
> >
> >
> >
> >     >
> >     > Anyway, on to the lengthy details which I want people to think
> >     about and
> >     > comment on:
> >     >
> >     > In a typical implementation a dynamic plugin will provide a simple
> >     > interface to
> >     > provide KiCad with a list of functions which can be invoked to
> >     provide the
> >     > desired feature. For example the 3D plugins provide a function to
> >     list the
> >     > file types supported by the plugin, the associated filters
> >     required for
> >     > browsing
> >     > files, and a Load() function to process a 3D file and return a data
> >     > structure
> >     > which KiCad can use in the 3D rendering process. The set of
> functions
> >     > provided
> >     > by a plugin shall be referred to here as the Plugin Interface.
> >     Every plugin
> >     > must be a derived class of at least one type of abstract plugin
> >     declared
> >     > in the
> >     > header files. All implementations of a plugin interface constitute
> an
> >     > actual plugin;
> >     > the various KiCad tools such as pcbnew and eeschema contain no
> actual
> >     > implementation of the plugin interfaces.
> >     >
> >     > Plugins also need to interact with KiCad and its internal data
> >     while knowing
> >     > essentially nothing of the complex structures within KiCad. To
> >     make this
> >     > possible
> >     > each type of dynamic plugin must interact with at least one type
> >     of API
> >     > which
> >     > provides functions to create, inspect, and manipulate the complex
> data
> >     > structures
> >     > within KiCad. These APIs shall be declared in an api subdirectory
> and
> >     > must be
> >     > implemented within each KiCad tool which services the related
> plugin
> >     > type. For
> >     > example the 3D plugins must produce the intermediate scene graph
> data
> >     > for use
> >     > by the 3D renderer; to accomplish this an API is required to
> >     provide each 3D
> >     > plugin with a method for creating the data structures required by
> >     pcbnew and
> >     > cvpcb. In the implementation detail the API may be implemented
> once in
> >     > the common
> >     > static library of pcbnew or it can be implemented as a runtime
> shared
> >     > library
> >     > which can then be linked to by pcbnew. cvpcb, or even eeschema.
> >     >
> >     > Plugin specialization: To keep APIs and plugin interfaces as
> simple as
> >     > possible
> >     > it is necessary for plugins to be specialized (have different
> types).
> >     > While it
> >     > is possible to define only one plugin interface and one API for
> >     any and all
> >     > plugins to use, this would result in a very large API which is
> >     difficult to
> >     > maintain. Having specialized plugins and APIs ensures better
> >     maintainability
> >     > of the code; a specific plugin may implement multiple plugin
> >     interfaces or
> >     > interact with multiple APIs if it makes sense to do so. At least
> >     two plugin
> >     > interfaces have been identified for implementation at this point:
> >     the 3D
> >     > Plugin Interface and the PCBExport interface. The 3D Plugin
> >     Interface shall
> >     > provide a method for processing supported 3D model types and
> >     returning an
> >     > intermediate scene graph representation which a renderer can use
> >     to create a
> >     > visual representation of the model. The PCBExport interface shall
> >     provide a
> >     > means of exporting PCB data to a variety of formats such as
> >     Specctra DSN for
> >     > the consumption of autorouters, GenCAD, VRML for visualization of
> >     the board
> >     > using external tools, and IDF and IGES for interacting with
> mechanical
> >     > designers
> >     > and MCAD software. Other exporters such as netlist exporters or
> >     even BoM
> >     > exporters should be possible. A PCBImport Interface is also a
> >     > possibility but
> >     > is currently a low priority item since there are currently only
> >     two import
> >     > functions implemented (Specctra result import and DXF import) and a
> >     > significant
> >     > amount of work is required to implement the PCBImport API.
> >     >
> >     > Plugin management: Since plugins are specialized, each type should
> >     also be
> >     > loaded and managed by a specialized plugin manager which has
> >     > responsibilities
> >     > for ensuring integration of the plugin with KiCad.  For example the
> >     > 3d_plugin_manager searches a list of paths to identify and load
> >     plugins
> >     > which
> >     > implement the 3D Plugin Interface. The 3D manager extracts
> >     information from
> >     > each 3D plugin such as the types of 3D formats supported and the
> >     associated
> >     > file extensions; the manager also responds to a Load() request by
> >     activating
> >     > the appropriate 3D plugin and executing that plugin's Load()
> function.
> >     > In the
> >     > case of the Export Plugin Interface, a specific exporter would
> >     enumerate the
> >     > plugins, add a menu item to the File->Export list, load and invoke
> a
> >     > plugin's
> >     > Export() function at the user's request, and unload the plugin
> >     when the
> >     > Export() function completes.
> >     >
> >     > - Cirilo
> >     >
> >     >
> >     >
> >     > _______________________________________________
> >     > Mailing list: https://launchpad.net/~kicad-developers
> >     > Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> >     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
> >     > Unsubscribe : https://launchpad.net/~kicad-developers
> >     > More help   : https://help.launchpad.net/ListHelp
> >     >
> >
> >
> >     _______________________________________________
> >     Mailing list: https://launchpad.net/~kicad-developers
> >     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> >     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
> >     Unsubscribe : https://launchpad.net/~kicad-developers
> >     More help   : https://help.launchpad.net/ListHelp
> >
> >
>
>