kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 08/12/2013 01:11 PM, Chris Morgan wrote:
> On Mon, Aug 12, 2013 at 12:05 PM, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:
>> On 08/12/2013 10:22 AM, Chris Morgan wrote:
>>> On Mon, Aug 12, 2013 at 11:13 AM, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:
>>>> FYI,
>>>>
>>>> In looking at an http plugin for pcbnew, here are my findings so far:
>>>>
>>>> It occurred to me that only three class PLUGIN functions seem to be necessary
>>>>
>>>>   *) IsWriteable()  this is one line, "return false".
>>>>   *) FootprintEnumerate()
>>>>   *) FootprintLoad()
>>>>
>>>>
>>>> So basically two functions.  I wrote FootprintLoad() in about 4 hours using wxHTTP,
>>>> complete with "redirect" handling that I added.  FootprintEnumerate() is website dependent
>>>> since you have to parse the dir listing coming back.
>>>>
>>>>
>>>> This is a trivial amount of work, but you have something website specific that is read
>>>> only for ever.  And that is not a terrible thing, quite useful.
>>>>
>>>>
>>>> Then and found an API at GITHUB, and formulated more thoughts about that, so what follows
>>>> is the formation of a work package:
>>>> ==============================================================
>>>>
>>>>
>>>>
>>>>    http://developer.github.com/v3/
>>>>
>>>>
>>>> is a marriage made in heaven for pcbnew.  The only missing building block is https
>>>> support.  We have to parse a little JSON, but this is in boost::property tree already in
>>>> the source tree.
>>>>
>>>> It also gave me a better handle on the use cases of the fp lib table dialog window, which
>>>> now seems to require a little better support for the options column.  I can envision a
>>>> property table two column popup dialog that lets you show the options in name value form.
>>>>  (Also maybe an additional PLUGIN api function that returns the options that each plugin
>>>> knows about as fodder for this generic dialog window.)  Upon return from the popup dialog,
>>>> you get a string in form
>>>>
>>>>   "option1=value1,option2,option3=value3"
>>>>
>>>> That goes back into the fp lib table dialog "options" column.
>>>>
>>>> And of course this property dialog must parse this string upon invocation.  The parser can
>>>> be re-used if the results go into a PROPERTIES class instance so it can also be passed to
>>>> the PLUGIN api functions too.
>>>>
>>>> RE: https.  I have a couple of options in mind for that, after which I am thinking we
>>>> could start with a read only GITHUB_PLUGIN implementation, and grow into something with
>>>> write capabilities down the line.
>>>>
>>>>
>>>> I need to solve the https gap in the least painful way.  Possibilities I see are:
>>>>
>>>> a) cpp-netlib which has HTTPS client, looks like it needs g++ 4.7 to compile, although
>>>> I've yet to verify this is the case when sub-setting it.
>>>>
>>>> b) cherry pick libpoco, which I have used, and has HTTPS client.
>>>>
>>>> c) add ssl support on top of wxHTTP using boost and openssl in the same way boost does it
>>>> for non SSL boost::ASIO sockets.
>>>>
>>>>
>>>> These are currently about equal in my mind.  Note that merely having SSL sockets is not as
>>>> rich as having an HTTPS client, since much header and replay parsing support is need.
>>>>
>>>> If anyone is a networking jockey and wants to work with me on this let me know.  I think I
>>>> could and would code the read only form of the plugin in about 2 days if we had the HTTPS
>>>> client support in place.  That is bogging me down right now, and brings the total work to
>>>> over 3 days, which I have to defer because of other work.
>>>>
>>>>
>>>> Dick
>>>>
>>>
>>>
>>> I had seen the github api when I was putting together some of the
>>> ideas about the online library sharing. It looks like a lot of the
>>> approaches they've used could be borrowed. I presume you are referring
>>> to using github directly?
>>
>> Correct.
>>
>>  I was more thinking the api was suitable to
>>> build upon for a custom server implementation with additional commands
>>> to handle our application specific usage.
>>
>>
>> That is also possible if you find the development resources to do it.   There are no fixed
>> limits on the number nor the types of pcbnew PLUGINs that get written.
>>
>>
>> If you decide to do that, *start by identifying the development resources*.  In my case of
>> GITHUB plugin, because of past discussions that we have had on the topic, and all the work
>> and planning that I put into the PLUGIN API so that it COULD HANDLE REMOTE ACCESS,
>>
>>    *I am prepared to provide the development resources*
>>
>> to do GITHUB_PLUGIN.  Although if I have to also do the HTTPS client layer work, it will
>> not be in 2 days from now.
>>
>>
>> Having a larger code base to borrow from makes any future plugin work easier of course also.
>>
>>
>> If that does not meet your needs, then do your means analysis up front.  Do not assume you
>> can rally the troops to donate valuable time to a large design.  You end up with 200 email
>> messages and a piece of paper if you are lucky.
>>
>>
>> The dirty secret about open source is that the code is most always written FOR the guy or
>> organization writing it.  It is only coincidental that it solves someone else's problem too.
>>
>> I think a GITHUB plugin will serve my needs, it only looks like I am doing it for someone
>> else.
>>
>>
>> Dick
>>
>>
> 
> 
> Your thoughts on enumerating parts? Would that be something done
> manually (user would have to plug in each of the parts they wanted to
> pull down via url or blob id or whatever), or would that be something
> you'd do via the github api (or another api)?

Each row entry in the fp lib table is a library.  Once a row is in place with a libpath
pointing to the "github API form of a repo" [1], you then can read any footprint in the
repo the way you do now in pcbnew.  It is seemless, because we'll use
PLUGIN::FootprintsEnumerate() on whatever plugin is associated with that row.  We already
use FootprintsEnumerate, but is it hard coded to LEGACY_PLUGIN::FootprintsEnumerate(),
until Wayne's work is complete.

Until we grow into the GITHUB_PLUGIN write support, the owner(s) of the repo can simply
git push to it to update parts.  The repo owner(s) can generate footprints in that library
any they wish, offline.  Scripting, coffeescript, whatever, as long as you have
*.kicad_mod files in the repo when its ready to use.

Writing to the repo *from kicad* means adding support for PLUGIN::FootprintSave(), and
that won't be bad.  But creating a github library is probably harder using the PLUGIN API.
 Fortunately the github browser interface can do the creation easily.  The options for the
fp lib table row will require username and password for your github account, even to use
someone else's repo on a read only basis, but these two options will remain secret since
the github API only uses the SSL encrypted sockets to fulfil its API in the link I provided.

For reading footprints, they can return the pretty format in "raw" form directly, if in
the API you ask for raw content. That means the reply to the https request done in
FootprintLoad() gets you a *.kicad_mod file back, and it is parsed in RAM before the
MODULE* is returned.

For enumerating footpringts, what we ask of the github API is the contents of a directory,
and that is returned in JSON format.  It needs to be parsed before FootprinsEnumerate()
returns with a wxArrayString, and the '.kicad_mod' extension needs to be chopped off to
fulfil that PLUGIN API function.


[1] https://api.github.com/username/repo  I think





> 
> Chris
>