kicad-developers team mailing list archive

[Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx>]

2012/2/24 Dick Hollenbeck <dick@xxxxxxxxxxx>

> Thanks Miguel.  Can you clearly state the the SWIG support for V8, which
> is still external
> to the SWIG project, is ready for production use?  Or do you still have
> doubts?
>

As I said in some previous email, it needs heavy testing, that I will do
:-)


> That was the original question.
>

I know but wanted to think first what could really do with an scripting
language.


>
> I also appreciate the use cases being identified up front.  I noticed you
> also did not say
> we should try and drive the UI from a scripting language.
>

I didn't think about it. But it could be interesting to have (at least)
some kind of user interface libraries accessible from javascript, so the
user could enter information or details to the script.
May be opening an HTML navigator and serving in the background could be an
easy approach (even if we din't offer any UI functionalities for the
scripting at first)


>
> This is interesting work, and I would eventually like to be able to offer
> more of my time
> on this particular topic.  However at this moment, my time is limited.  I
> do feel that I
> am very qualified to offer help.  I have written an entire Java Virtual
> Machine myself,
> including a JNI layer.  Sold it on the open market for a couple of years,
> back in the
> 90's, pre-JIT period.  This was the 2nd Java Virtual Machine ever written,
> and it has a
> real time incremental garbage collector in it.   I have embedded it into
> C++, call it from
> C++, it calls C++, and all this.  I know what it takes to package
> something like this,
> support it, and debug it.   It still runs in factories all over the world.
>  All this to
> say that I could offer qualified help with this if I had time or financial
> incentive.
>

Wow, it seems like a lot of work and expertise :-) JNI layer is the one
used to talk with native code, right? .

 I'm also short of time, but it's a really interesting "subproject" for
which I'd like to find some time.


> If I were to be able to spend more time on this.  I certainly would not be
> in a hurry to
> rush to a solution, since any quality implementation requires enough up
> front research to
> first establish the best path, and in today's world with so many open
> source projects to
> look at, that is a non-trivial effort.
>

Yes, we may think, design, and think in advance before taking any path,
because if we finally do, it's a lot of effort to put on it.


> Quite a few questions need to be answered and here are only a couple:
>
> a) should we be concerned about supporting more than one scripting
> language, or will it be
> sufficient to support only one?  This decision has a big impact on how you
> build
> infrastructure.
>

I suppose, that, if the SWIG approach works properly, we could do it for
many languages. Anyway, as resources/time are scarce, probably we may
target only one language, keeping in mind that we may like to try with
other languages later.

I like the JS approach, as it's quite widespread adopted.


> b) Is the scripting language sufficiently easier to use than C++?  Because
> another path
> would be to put a C++ compiler and build environment out in the cloud,
> where a guy could
> write his script in C++, HTTP POST it to a website, and get a DLL/DSO
> back, ready to run.
> This is your "script" in binary.  (This one is a setup for the next
> question.)
>


For sure, at least in my experience, anything is much faster to develop in
javascript than C++,
no class definitions, no headers, just code. Of course, it's a little bit
slower than C/C++.

I don't like the remote compilation support, it requires: a) to mantain a
server for it b) or the user to have a compiler (with all libs an headers)
installed. c) to run the resulting dll/dso which may have dependence
problems (if it's not static, of course). d) to keep an internet connection
to run new scripts.


>
> c) What kind of security holes to you open up when you down load
> (scripting) code from
> third parties, and is it our job to be concerned about those?
>

I think that it's the same case of using software compiled by others or any
script you download from the internet. The user can always check the code
before executing.

And anyway, as the user don't runs the code as root, nothing "too bad" may
happen.



>
> d) ..
>
>
> Scripting is not at the top of my priority list at this time.  Therefore I
> will not be
> able to offer any assistance at this time, but probably will be able to
> later, and might
> have time to offer some advice from time to time now, but even that will
> be limited

.
>
> Thanks for your enthusiasm.
>
>
It's a pleasure,

Greetings!! :-)

I will keep you updated about my tests with swig-js.

>
>
>
>
>
> > Ok, I changed the topic of this thread, to match the discussion, which
> is about making
> > Kicad fully scriptable in Javascript.
> >
> > For speed reasons, and wide adoption, it seems that V8 must be our
> preferred engine.
> >
> > V8 comes just with the javascript JIT interpreter, and no more. It's
> ready to link
> > with your library or applications, it can be debugged with a remote
> connection, but it
> > doesn't include *any* system access library, networking, just nothing.
> >
> > In V8, you register functions which can be accessed from javascript to
> C++, and you can
> > also call Javascript code from C++
> >
> > Here is where NodeJS seems attractive: it adds many system and network
> access layers.
> > It's not ready out of the box to be embedded in other applications, but
> it could be
> > easily patched for that (already checked the code).
> >
> > Anyway nodejs's implementation (I think) wouldn't match just any
> architecture, it's
> > something we should study carefully:
> >
> >        1) It's ready for asynchronous operations (like most javascript
> code).  [
> > see http://nodejs.org/docs/latest/api/fs.html#fs.writeFile ]
> >        2) It's internal flow (in one thread) is done like this
> (simplified):
> >                a) Init V8
> >                b) Add all nodejs C++ functions.
> >                c) Load "node.js" part (some % of the code is just in
> javascript, which
> > makes sense).
> >                d) Enter the event loop   "while(notexit) {
> process_events(); }" <--- not
> > exactly like this, but you can get the underlaying idea.
> >
> >
> > With this architecture, if we wanted to work with nodeJS , we may need
> to launch a
> > separate thread for the node.js interpreter (note, this is now V8
> handles threaded host
> > apps: http://markmail.org/message/uxqpvsbc7sw7qntl)
> >
> > Then we may need to define clearly how to work in this situation:
> >
> > 0) The ideas of Dick, about making a big library, would be nice,
> becaouse we could use
> > nodejs to work directly with kicad schematics, pcbs, etc, even without
> the interface
> > itself, in this case I don't find any problem.
> >
> > 1) now, when the interface is around: when we needed to call some JS
> programmed function
> > from kicad, we could simply pass the code to V8, and setup a callback
> that would unlock
> > us back...., and lock ourselves waiting for the callback  [danger: we
> could lock forever
> > if the js code never unlock us back, solution: timeout?]
> >
> > 2) more danger: The user could leave timer operations or any other
> operation, which
> > could call us back later from the nodejs thread (when the timer
> expires). In this case,
> > our BOARD or BOARD_ACTION objects would be called (even if we're doing
> any other thing).
> > Some questions come to my mind:
> >         a) How it's the event loop handled in kicad, it comes from the
> wx toolkit?
> >         b) Are our operations thread safe?
> >         c) Would a "big lock" on the accessed objects (like V8
> implementation for this)
> > make sense?, this could be a problem, if for example, our operations (at
> any moment)
> > could have callbacks to javascript
> >
> > Other option could be using
> http://code.google.com/p/v8-juice/wiki/Plugins  (or
> > equivalent), which doesn't seem to rely on an event loop. But, there we
> would loose all
> > the nodejs power.
> >
> >
> >
> > There is also something quite important, we may decide which are our use
> cases, I see
> > some clear ones, but many other could be interesting:
> >
> >
> > 1) access to kicad objects from external scripting (kicad UI is off and
> we only access
> > the PCB, schema, etc ... all the objects from our wonderful script),
> this could lead to
> > many usefult operations:
> >
> >         A) automated/parametric creation of schematics and circuits
> (think of Scad, but
> > in javascript): imagine a javscript that takes an image and builds a
> PCB/schematic full
> > of interconnected leds (even RGB), that when you power, you get the
> image :-)
> >
> >         B) Automated test/extra DRC checks for our pcbs/schematics.
> >
> >         C) Automated fixing.
> >
> >          D) options are endless....
> >
> > 2) Into the kicad UI, assigning hotkeys (or toolbar buttons) to our own
> scripts (like
> > google sketchup would do with ruby scripts). The user could install new
> scripts
> > (somebody could make them), and they would add new functionalities to
> kicad (with no
> > need for direct integration in the main source code).
> >
> > 3) Scripts that are installed, could register callbacks for certain
> operations, examples:
> >
> >         A) when we finish drawing a track, the
> "Kicad.pcbnew.routedTrack"  callback
> > could trigger, and then the user may have registered some algorithm to
> check the PCB for
> > some kind of special DRC operations.
> >
> >          B) "pcbChanged", more generic.
> >
> >          C) I'm out of more ideas, but I have the feeling that many
> useful things could
> > be done with that.
> >
> >
> > As summary:
> >
> >     Scripting can be very powerful, JS/V8 makes a lot of sense with the
> help of swig (or
> > even may be without... have to test...).
> >
> >
> >      We must check how feasible is to put a nodeJS stack inside kicad,
> V8 would have a
> > lot to do: calls from kicad to V8 and V8 back to kicad via the objects
> interface. But
> > the nodejs implementation for evented I/O could be a problem if we don't
> handle it right.
> >
> >
> >    That's all, thanks for reading, I wait for feedback :-)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > 2012/2/22 Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx <mailto:
> miguelangel@xxxxxxx>>
> >
> >     Ok, I cleaned up a little, and forked the project of swig-js
> >
> >     https://github.com/espardino/swig-js
> >
> >     It should require heavy testing, I will try to make it work with
> some of the Kicad
> >     classes, and we will see
> >     how well or bad does it work :-)
> >
> >
> >     PS:
> >
> >     Espardino is the project where I'm using swig too:
> >     https://github.com/espardino/xpress/tree/master/wrappers
> >
> >     I will reuse the work of this swig-js for both: Kicad, and Espardino.
> >
> >
> >
> >
> >     2012/2/22 Miguel Angel Ajo Pelayo <miguelangel@xxxxxxx <mailto:
> miguelangel@xxxxxxx>>
> >
> >         I need to spend some time understanding your idea of design,
> right now I'm a
> >         little bit lost,
> >         but I started by the "simple" part, which was building the
> swig-js branch, and
> >         it seems
> >         that I managed to do it (it was only ready for WIN32) but I have
> a patch for
> >         swig-js that
> >         seems to work in linux (partially)...
> >
> >         I was able to generate the example for:
> >
> >
> https://github.com/oliver----/swig-js/blob/master/src/Examples/jsv8/class/example.h
> >
> >         and it seems to link to standard libv8 from ubuntu, and run. (It
> compiled it's
> >         own v8, but it was standard one, no patches).
> >
> >         I think I will try to cleanup the build, and then publishing it.
> >
> >         The swig-js is built using CMake (I'm quite experienced on it)
> so I could need
> >         some help ':-)
> >
> >
> >
> >
> >         2012/2/22 Dick Hollenbeck <dick@xxxxxxxxxxx <mailto:
> dick@xxxxxxxxxxx>>
> >
> >             On 02/22/2012 09:11 AM, Miguel Angel Ajo Pelayo wrote:
> >             > I think I could put some efforts on testing it, and see
> how well the "JS"
> >             branch of swig
> >             > does work.
> >             >
> >             > Do we have an idea of which interface we may want to give
> to the scripting
> >             layers?,
> >             > access to PCB objects, modules, tracks, etc?
> >
> >             To keep it simple, let me talk about PCBNEW only *for now*
> (duplicate the
> >             discussion in
> >             your mind for EESCHEMA).
> >
> >             class BOARD, and the needed significant objects contained
> within it.  I say
> >             needed,
> >             because I think there is some flexibility as to where you
> build this bridge,
> >             either in the
> >             class BOARD by accessor augmentation in C++ go give access
> to everything
> >             there, or by
> >             SWIGing out the contained classes.
> >
> >             We also talked about moving the non-UI member functions that
> are now in
> >             PCB_BASE_FRAME,
> >             PCB_EDIT_FRAME, into a class which is purely for significant
> actions but
> >             with NO UI, like
> >             plotting, annotating, etc.  The UI portion (meaning the
> preceding dialog
> >             invocation) would
> >             stay out of this class.  conceptually lets call this class
> BOARD_ACTIONS.
> >              All the
> >             functions in BOARD_ACTIONS take enough parameters to
> operation *without any
> >             UI*, by
> >             definition.
> >
> >
> >
> >             BOARD_ACTIONS could then be transported into two realms:
> >
> >             a) back into PCB_*_FRAME by multiple inheritance, so you are
> back to where
> >             you started
> >             from with respect to PCB_EDIT_FRAME.
> >
> >             b) into the top of a scripting entry point DLL/DSO.
> >
> >
> >             Along with this, it might be useful to try and build a
> PCBNEW DLL/DSO that
> >             has the public
> >             BOARD_ACTIONS, and BOARD entry points exposed.
> >             The actual PCBNEW program (where main() is), could be a thin
> layer with
> >             linked with the UI
> >             (mostly dialogs and wxFrames).
> >
> >             The plotting and drawing functions themselves might have to
> be yet a
> >             separate DLL/DSO.
> >             Since your BOARD_ACTIONS might actually have to plot or
> print.
> >
> >             We are finally getting in a position to be able to do this,
> reduction of
> >             globals has been
> >             an enabler in this effort.  There is some more work to do.
>  We still have a
> >             number of
> >             globals, and getting the separation into BOARD_ACTIONS would
> be some work.
> >
> >             This design I think deals with both the paradigms:
> >
> >             a) scripting calls C++ actions, and
> >             b) C++ calls scripting, since these calls from C++ would be
> from the upper
> >             main() area,
> >             and you could still call BOARD_ACTIONS and BOARD from the
> script.
> >
> >             It just does not however deal with the letting the script
> control the UI.  I
> >             think that is
> >             significantly more work, and is probably best done in C++
> forever.
> >
> >             I don't anticipate that this would take under two days to do.
> >
> >             :) read carefully now.
> >
> >             Dick
> >
> >
> >
> >             >
> >             >
> >             >
> >             > 2012/2/22 Dick Hollenbeck <dick@xxxxxxxxxxx <mailto:
> dick@xxxxxxxxxxx>
> >             <mailto:dick@xxxxxxxxxxx <mailto:dick@xxxxxxxxxxx>>>
> >             >
> >             >     On 02/17/2012 08:56 AM, Miguel Angel Ajo Pelayo wrote:
> >             >     > Hi Dick,
> >             >     >
> >             >     >    Thanks for your feedback on the ideas, it's
> really appreciated, I
> >             fully understand
> >             >     > all of your points, as I'm on the same situation
> with some open
> >             projects and my little
> >             >     > company. I supposed that most of the ideas were
> already in your mind,
> >             but if I didn't
> >             >     > share them I wouldn't know now where do they
> converge with yours :-)
> >             >     >
> >             >     >     I really love the V8 engine, just played a
> little bit on embedded
> >             systems, and it
> >             >     > works incredibly fast (it does JIT of the code and
> you get all the
> >             flexibility of
> >             >     > javascript), and nodeJS gives all the functionality
> needed to access
> >             the underlaying OS
> >             >     > and network, although I'm not sure if it's internal
> design makes it
> >             suitable to get
> >             >     > integrated in other apps.
> >             >     >
> >             >     >      I'm also using SWIG to build interfaces to many
> languages, and
> >             it works incredibly
> >             >     > well. I must test the swig-js and see how mature is.
> >             >
> >             >     We will be interested in finding out what you learn.
>  JavaScript seems
> >             to have a lot of
> >             >     momentum now, and it needs to be looked at in detail.
> >             >
> >             >     With HTML5 giving it new life on the client, NodeJS
> giving it further
> >             life on the
> >             >     server,
> >             >     its C++ like syntax, its standardization by a
> standards body, it has a
> >             lot going for it.
> >             >
> >             >     And for any one not already knowing a language, one is
> comforted by the
> >             fact that
> >             >     learning
> >             >     it might serve a real purpose in other areas, (like
> when you go design
> >             your next
> >             >     webpage.)
> >             >
> >             >     Dick
> >             >
> >             >
> >             >     >
> >             >     >     Personally, I'd like to give "some blood" on the
> project as soon
> >             as I can and get
> >             >     > familiar with kicad's code/design :-)
> >             >     >
> >             >     >     Cheers,
> >             >     > Mike
> >             >
> >             >
> >             >     > Two weeks ago I looked into adding support for V8.
>  V8 is google's
> >             javascript
> >             >     engine and
> >             >     >
> >             >     >     they use it under the chrome browser.  (You may
> already have a
> >             copy on your disk
> >             >     >     now.)  V8
> >             >     >     is very fast.  There is another project that
> sits on top of V8
> >             called node.js.
> >             >     >      node.js is
> >             >     >     great at writing asynchronous webservers, and
> has been shown to
> >             actually be
> >             >     faster than
> >             >     >     apache in some circumstances.
> >             >     >
> >             >     >     My measurement of doing it manually said that it
> is a lot of
> >             work.  However
> >             >     since then I
> >             >     >     see some folks are trying to add the binding
> generation
> >             functionality to SWIG.
> >             >      SWIG can
> >             >     >     generate bindings for a number of languages, in
> theory.
> >             >     >
> >             >     >
> >             >     >
> http://comments.gmane.org/gmane.comp.programming.swig.devel/20373
> >             >     >     https://github.com/oliver----/swig-js
> >             >     >
> >             >     >     This is an ancillary project, and not integrated
> into core SWIG
> >             from what I can
> >             >     tell.
> >             >     >
> >             >     >     node.js has gathered so much momentum that it is
> actually
> >             changing the choice of
> >             >     >     languages
> >             >     >     used on the server side for many web
> applications.  There are two
> >             reasons for this
> >             >     >     that I
> >             >     >     can see:  a) node.js on top of V8 is fast.  b)
> being able to use
> >             javascript
> >             >     both on the
> >             >     >     client side and server side is attractive to
> many developers.
> >             >     >
> >             >
> >             >
> >             >
> >             >
> >             >
> >             > --
> >             >
> >             > Miguel Angel Ajo Pelayo
> >             > http://www.nbee.es
> >             > +34 636 52 25 69 <tel:%2B34%20636%2052%2025%2069>
> >             > skype: ajoajoajo
> >
> >
> >
> >
> >         --
> >
> >         Miguel Angel Ajo Pelayo
> >         http://www.nbee.es
> >         +34 636 52 25 69 <tel:%2B34%20636%2052%2025%2069>
> >         skype: ajoajoajo
> >
> >
> >
> >
> >     --
> >
> >     Miguel Angel Ajo Pelayo
> >     http://www.nbee.es
> >     +34 636 52 25 69 <tel:%2B34%20636%2052%2025%2069>
> >     skype: ajoajoajo
> >
> >
> >
> >
> > --
> >
> > Miguel Angel Ajo Pelayo
> > http://www.nbee.es
> > +34 636 52 25 69
> > skype: ajoajoajo
>
>


-- 

Miguel Angel Ajo Pelayo
http://www.nbee.es
+34 636 52 25 69
skype: ajoajoajo