kicad-developers team mailing list archive

[Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx>]

Ok, I changed the topic of this thread, to match the discussion, which is
about making Kicad fully scriptable in Javascript.

For speed reasons, and wide adoption, it seems that V8 must be
our preferred engine.

V8 comes just with the javascript JIT interpreter, and no more. It's ready
to link agains your library or applications, it can be debugged with a
remote connection, but it doesn't include *any* system access library,
networking, just nothing.

In V8, you register functions which can be accessed from javascript to C++,
and you can also call Javascript code from C++

Here is where NodeJS seems attractive: it adds many system and network
access layers. It's not ready out of the box to be embedded in other
applications, but it could be easily patched for that (already checked the
code).

Anyway nodejs's implementation (I think) wouldn't match just any
architecture, it's something we should study carefully:

       1) It's ready for asynchronous operations (like most javascript
code).  [ see http://nodejs.org/docs/latest/api/fs.html#fs.writeFile ]
       2) It's internal flow (in one thread) is done like this (simplified):
               a) Init V8
               b) Add all nodejs C++ functions.
               c) Load "node.js" part (some % of the code is just in
javascript, which makes sense).
               d) Enter the event loop   "while(notexit) {
process_events(); }" <--- not exactly like this, but you can get the
underlaying idea.


With this architecture, if we wanted to work with nodeJS , we may need to
launch a separate thread for the node.js interpreter (note, this is now V8
handles threaded host apps: http://markmail.org/message/uxqpvsbc7sw7qntl)

Then we may need to define clearly how to work in this situation:

0) The ideas of Dick, about making a big library, would be nice, becaouse
we could use nodejs to work directly with kicad schematics, pcbs, etc, even
without the interface itself, in this case I don't find any problem.

1) now, when the interface is around: when we needed to call some JS
programmed function from kicad, we could simply pass the code to V8, and
setup a callback that would unlock us back...., and lock ourselves waiting
for the callback  [danger: we could lock forever if the js code never
unlock us back, solution: timeout?]

2) more danger: The user could leave timer operations or any other
operation, which could call us back later from the nodejs thread (when the
timer expires). In this case, our BOARD or BOARD_ACTION objects would be
called (even if we're doing any other thing). Some questions come to my
mind:
        a) How it's the event loop handled in kicad, it comes from the wx
toolkit?
        b) Are our operations thread safe?
        c) Would a "big lock" on the accessed objects (like V8
implementation for this) make sense?, this could be a problem, if for
example, our operations (at any moment) could have callbacks to javascript

Other option could be using http://code.google.com/p/v8-juice/wiki/Plugins
(or equivalent), which doesn't seem to rely on an event loop. But, there we
would loose all the nodejs power.



There is also something quite important, we may decide which are our use
cases, I see some clear ones, but many other could be interesting:


1) access to kicad objects from external scripting (kicad UI is off and we
only access the PCB, schema, etc ... all the objects from our wonderful
script), this could lead to many usefult operations:

        A) automated/parametric creation of schematics and circuits (think
of Scad, but in javascript): imagine a javscript that takes an image and
builds a PCB/schematic full of interconnected leds (even RGB), that when
you power, you get the image :-)

        B) Automated test/extra DRC checks for our pcbs/schematics.

        C) Automated fixing.

         D) options are endless....

2) Into the kicad UI, assigning hotkeys (or toolbar buttons) to our own
scripts (like google sketchup would do with ruby scripts). The user could
install new scripts (somebody could make them), and they would add new
functionalities to kicad (with no need for direct integration in the main
source code).

3) Scripts that are installed, could register callbacks for certain
operations, examples:

        A) when we finish drawing a track, the "Kicad.pcbnew.routedTrack"
 callback could trigger, and then the user may have registered some
algorithm to check the PCB for some kind of special DRC operations.

         B) "pcbChanged", more generic.

         C) I'm out of more ideas, but I have the feeling that many useful
things could be done with that.


As summary:

    Scripting can be very powerful, JS/V8 makes a lot of sense with the
help of swig (or even may be without... have to test...).


     We must check how feasible is to put a nodeJS stack inside kicad, V8
would have a lot to do: calls from kicad to V8 and V8 back to kicad via the
objects interface. But the nodejs implementation for evented I/O could be a
problem if we don't handle it right.


   That's all, thanks for reading, I wait for feedback :-)























2012/2/22 Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx>

> Ok, I cleaned up a little, and forked the project of swig-js
>
> https://github.com/espardino/swig-js
>
> It should require heavy testing, I will try to make it work with some of
> the Kicad classes, and we will see
> how well or bad does it work :-)
>
>
> PS:
>
> Espardino is the project where I'm using swig too:
> https://github.com/espardino/xpress/tree/master/wrappers
>
> I will reuse the work of this swig-js for both: Kicad, and Espardino.
>
>
>
>
> 2012/2/22 Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx>
>
>> I need to spend some time understanding your idea of design, right now
>> I'm a little bit lost,
>> but I started by the "simple" part, which was building the swig-js
>> branch, and it seems
>> that I managed to do it (it was only ready for WIN32) but I have a patch
>> for swig-js that
>> seems to work in linux (partially)...
>>
>> I was able to generate the example for:
>>
>>
>> https://github.com/oliver----/swig-js/blob/master/src/Examples/jsv8/class/example.h
>>
>> and it seems to link to standard libv8 from ubuntu, and run. (It compiled
>> it's own v8, but it was standard one, no patches).
>>
>> I think I will try to cleanup the build, and then publishing it.
>>
>> The swig-js is built using CMake (I'm quite experienced on it) so I could
>> need some help ':-)
>>
>>
>>
>>
>> 2012/2/22 Dick Hollenbeck <dick@xxxxxxxxxxx>
>>
>>> On 02/22/2012 09:11 AM, Miguel Angel Ajo Pelayo wrote:
>>> > I think I could put some efforts on testing it, and see how well the
>>> "JS" branch of swig
>>> > does work.
>>> >
>>> > Do we have an idea of which interface we may want to give to the
>>> scripting layers?,
>>> > access to PCB objects, modules, tracks, etc?
>>>
>>> To keep it simple, let me talk about PCBNEW only *for now* (duplicate
>>> the discussion in
>>> your mind for EESCHEMA).
>>>
>>> class BOARD, and the needed significant objects contained within it.  I
>>> say needed,
>>> because I think there is some flexibility as to where you build this
>>> bridge, either in the
>>> class BOARD by accessor augmentation in C++ go give access to everything
>>> there, or by
>>> SWIGing out the contained classes.
>>>
>>> We also talked about moving the non-UI member functions that are now in
>>> PCB_BASE_FRAME,
>>> PCB_EDIT_FRAME, into a class which is purely for significant actions but
>>> with NO UI, like
>>> plotting, annotating, etc.  The UI portion (meaning the preceding dialog
>>> invocation) would
>>> stay out of this class.  conceptually lets call this class
>>> BOARD_ACTIONS.  All the
>>> functions in BOARD_ACTIONS take enough parameters to operation *without
>>> any UI*, by
>>> definition.
>>>
>>>
>>>
>>> BOARD_ACTIONS could then be transported into two realms:
>>>
>>> a) back into PCB_*_FRAME by multiple inheritance, so you are back to
>>> where you started
>>> from with respect to PCB_EDIT_FRAME.
>>>
>>> b) into the top of a scripting entry point DLL/DSO.
>>>
>>>
>>> Along with this, it might be useful to try and build a PCBNEW DLL/DSO
>>> that has the public
>>> BOARD_ACTIONS, and BOARD entry points exposed.
>>> The actual PCBNEW program (where main() is), could be a thin layer with
>>> linked with the UI
>>> (mostly dialogs and wxFrames).
>>>
>>> The plotting and drawing functions themselves might have to be yet a
>>> separate DLL/DSO.
>>> Since your BOARD_ACTIONS might actually have to plot or print.
>>>
>>> We are finally getting in a position to be able to do this, reduction of
>>> globals has been
>>> an enabler in this effort.  There is some more work to do.  We still
>>> have a number of
>>> globals, and getting the separation into BOARD_ACTIONS would be some
>>> work.
>>>
>>> This design I think deals with both the paradigms:
>>>
>>> a) scripting calls C++ actions, and
>>> b) C++ calls scripting, since these calls from C++ would be from the
>>> upper main() area,
>>> and you could still call BOARD_ACTIONS and BOARD from the script.
>>>
>>> It just does not however deal with the letting the script control the
>>> UI.  I think that is
>>> significantly more work, and is probably best done in C++ forever.
>>>
>>> I don't anticipate that this would take under two days to do.
>>>
>>> :) read carefully now.
>>>
>>> Dick
>>>
>>>
>>>
>>> >
>>> >
>>> >
>>> > 2012/2/22 Dick Hollenbeck <dick@xxxxxxxxxxx <mailto:dick@xxxxxxxxxxx>>
>>> >
>>> >     On 02/17/2012 08:56 AM, Miguel Angel Ajo Pelayo wrote:
>>> >     > Hi Dick,
>>> >     >
>>> >     >    Thanks for your feedback on the ideas, it's really
>>> appreciated, I fully understand
>>> >     > all of your points, as I'm on the same situation with some open
>>> projects and my little
>>> >     > company. I supposed that most of the ideas were already in your
>>> mind, but if I didn't
>>> >     > share them I wouldn't know now where do they converge with yours
>>> :-)
>>> >     >
>>> >     >     I really love the V8 engine, just played a little bit on
>>> embedded systems, and it
>>> >     > works incredibly fast (it does JIT of the code and you get all
>>> the flexibility of
>>> >     > javascript), and nodeJS gives all the functionality needed to
>>> access the underlaying OS
>>> >     > and network, although I'm not sure if it's internal design makes
>>> it suitable to get
>>> >     > integrated in other apps.
>>> >     >
>>> >     >      I'm also using SWIG to build interfaces to many languages,
>>> and it works incredibly
>>> >     > well. I must test the swig-js and see how mature is.
>>> >
>>> >     We will be interested in finding out what you learn.  JavaScript
>>> seems to have a lot of
>>> >     momentum now, and it needs to be looked at in detail.
>>> >
>>> >     With HTML5 giving it new life on the client, NodeJS giving it
>>> further life on the
>>> >     server,
>>> >     its C++ like syntax, its standardization by a standards body, it
>>> has a lot going for it.
>>> >
>>> >     And for any one not already knowing a language, one is comforted
>>> by the fact that
>>> >     learning
>>> >     it might serve a real purpose in other areas, (like when you go
>>> design your next
>>> >     webpage.)
>>> >
>>> >     Dick
>>> >
>>> >
>>> >     >
>>> >     >     Personally, I'd like to give "some blood" on the project as
>>> soon as I can and get
>>> >     > familiar with kicad's code/design :-)
>>> >     >
>>> >     >     Cheers,
>>> >     > Mike
>>> >
>>> >
>>> >     > Two weeks ago I looked into adding support for V8.  V8 is
>>> google's javascript
>>> >     engine and
>>> >     >
>>> >     >     they use it under the chrome browser.  (You may already have
>>> a copy on your disk
>>> >     >     now.)  V8
>>> >     >     is very fast.  There is another project that sits on top of
>>> V8 called node.js.
>>> >     >      node.js is
>>> >     >     great at writing asynchronous webservers, and has been shown
>>> to actually be
>>> >     faster than
>>> >     >     apache in some circumstances.
>>> >     >
>>> >     >     My measurement of doing it manually said that it is a lot of
>>> work.  However
>>> >     since then I
>>> >     >     see some folks are trying to add the binding generation
>>> functionality to SWIG.
>>> >      SWIG can
>>> >     >     generate bindings for a number of languages, in theory.
>>> >     >
>>> >     >
>>> >     >
>>> http://comments.gmane.org/gmane.comp.programming.swig.devel/20373
>>> >     >     https://github.com/oliver----/swig-js
>>> >     >
>>> >     >     This is an ancillary project, and not integrated into core
>>> SWIG from what I can
>>> >     tell.
>>> >     >
>>> >     >     node.js has gathered so much momentum that it is actually
>>> changing the choice of
>>> >     >     languages
>>> >     >     used on the server side for many web applications.  There
>>> are two reasons for this
>>> >     >     that I
>>> >     >     can see:  a) node.js on top of V8 is fast.  b) being able to
>>> use javascript
>>> >     both on the
>>> >     >     client side and server side is attractive to many developers.
>>> >     >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> > Miguel Angel Ajo Pelayo
>>> > http://www.nbee.es
>>> > +34 636 52 25 69
>>> > skype: ajoajoajo
>>>
>>>
>>
>>
>> --
>>
>> Miguel Angel Ajo Pelayo
>> http://www.nbee.es
>> +34 636 52 25 69
>> skype: ajoajoajo
>>
>
>
>
> --
>
> Miguel Angel Ajo Pelayo
> http://www.nbee.es
> +34 636 52 25 69
> skype: ajoajoajo
>



-- 

Miguel Angel Ajo Pelayo
http://www.nbee.es
+34 636 52 25 69
skype: ajoajoajo