kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 08/12/2013 11:54 PM, Dick Hollenbeck wrote:
> On 08/12/2013 01:11 PM, Chris Morgan wrote:
>> On Mon, Aug 12, 2013 at 12:05 PM, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:
>>> On 08/12/2013 10:22 AM, Chris Morgan wrote:
>>>> On Mon, Aug 12, 2013 at 11:13 AM, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:
>>>>> FYI,
>>>>>
>>>>> In looking at an http plugin for pcbnew, here are my findings so far:
>>>>>
>>>>> It occurred to me that only three class PLUGIN functions seem to be necessary
>>>>>
>>>>>   *) IsWriteable()  this is one line, "return false".
>>>>>   *) FootprintEnumerate()
>>>>>   *) FootprintLoad()
>>>>>
>>>>>
>>>>> So basically two functions.  I wrote FootprintLoad() in about 4 hours using wxHTTP,
>>>>> complete with "redirect" handling that I added.  FootprintEnumerate() is website dependent
>>>>> since you have to parse the dir listing coming back.
>>>>>
>>>>>
>>>>> This is a trivial amount of work, but you have something website specific that is read
>>>>> only for ever.  And that is not a terrible thing, quite useful.
>>>>>
>>>>>
>>>>> Then and found an API at GITHUB, and formulated more thoughts about that, so what follows
>>>>> is the formation of a work package:
>>>>> ==============================================================
>>>>>
>>>>>
>>>>>
>>>>>    http://developer.github.com/v3/
>>>>>
>>>>>
>>>>> is a marriage made in heaven for pcbnew.  The only missing building block is https
>>>>> support.  We have to parse a little JSON, but this is in boost::property tree already in
>>>>> the source tree.
>>>>>
>>>>> It also gave me a better handle on the use cases of the fp lib table dialog window, which
>>>>> now seems to require a little better support for the options column.  I can envision a
>>>>> property table two column popup dialog that lets you show the options in name value form.
>>>>>  (Also maybe an additional PLUGIN api function that returns the options that each plugin
>>>>> knows about as fodder for this generic dialog window.)  Upon return from the popup dialog,
>>>>> you get a string in form
>>>>>
>>>>>   "option1=value1,option2,option3=value3"
>>>>>
>>>>> That goes back into the fp lib table dialog "options" column.
>>>>>
>>>>> And of course this property dialog must parse this string upon invocation.  The parser can
>>>>> be re-used if the results go into a PROPERTIES class instance so it can also be passed to
>>>>> the PLUGIN api functions too.
>>>>>
>>>>> RE: https.  I have a couple of options in mind for that, after which I am thinking we
>>>>> could start with a read only GITHUB_PLUGIN implementation, and grow into something with
>>>>> write capabilities down the line.
>>>>>
>>>>>
>>>>> I need to solve the https gap in the least painful way.  Possibilities I see are:
>>>>>
>>>>> a) cpp-netlib which has HTTPS client, looks like it needs g++ 4.7 to compile, although
>>>>> I've yet to verify this is the case when sub-setting it.
>>>>>
>>>>> b) cherry pick libpoco, which I have used, and has HTTPS client.
>>>>>
>>>>> c) add ssl support on top of wxHTTP using boost and openssl in the same way boost does it
>>>>> for non SSL boost::ASIO sockets.
>>>>>
>>>>>
>>>>> These are currently about equal in my mind.  Note that merely having SSL sockets is not as
>>>>> rich as having an HTTPS client, since much header and replay parsing support is need.
>>>>>
>>>>> If anyone is a networking jockey and wants to work with me on this let me know.  I think I
>>>>> could and would code the read only form of the plugin in about 2 days if we had the HTTPS
>>>>> client support in place.  That is bogging me down right now, and brings the total work to
>>>>> over 3 days, which I have to defer because of other work.
>>>>>
>>>>>
>>>>> Dick
>>>>>
>>>>
>>>>
>>>> I had seen the github api when I was putting together some of the
>>>> ideas about the online library sharing. It looks like a lot of the
>>>> approaches they've used could be borrowed. I presume you are referring
>>>> to using github directly?
>>>
>>> Correct.
>>>
>>>  I was more thinking the api was suitable to
>>>> build upon for a custom server implementation with additional commands
>>>> to handle our application specific usage.
>>>
>>>
>>> That is also possible if you find the development resources to do it.   There are no fixed
>>> limits on the number nor the types of pcbnew PLUGINs that get written.
>>>
>>>
>>> If you decide to do that, *start by identifying the development resources*.  In my case of
>>> GITHUB plugin, because of past discussions that we have had on the topic, and all the work
>>> and planning that I put into the PLUGIN API so that it COULD HANDLE REMOTE ACCESS,
>>>
>>>    *I am prepared to provide the development resources*
>>>
>>> to do GITHUB_PLUGIN.  Although if I have to also do the HTTPS client layer work, it will
>>> not be in 2 days from now.
>>>
>>>
>>> Having a larger code base to borrow from makes any future plugin work easier of course also.
>>>
>>>
>>> If that does not meet your needs, then do your means analysis up front.  Do not assume you
>>> can rally the troops to donate valuable time to a large design.  You end up with 200 email
>>> messages and a piece of paper if you are lucky.
>>>
>>>
>>> The dirty secret about open source is that the code is most always written FOR the guy or
>>> organization writing it.  It is only coincidental that it solves someone else's problem too.
>>>
>>> I think a GITHUB plugin will serve my needs, it only looks like I am doing it for someone
>>> else.
>>>
>>>
>>> Dick
>>>
>>>
>>
>>
>> Your thoughts on enumerating parts? Would that be something done
>> manually (user would have to plug in each of the parts they wanted to
>> pull down via url or blob id or whatever), or would that be something
>> you'd do via the github api (or another api)?
> 
> Each row entry in the fp lib table is a library.  Once a row is in place with a libpath
> pointing to the "github API form of a repo" [1], you then can read any footprint in the
> repo the way you do now in pcbnew.  It is seemless, because we'll use
> PLUGIN::FootprintsEnumerate() on whatever plugin is associated with that row.  We already
> use FootprintsEnumerate, but is it hard coded to LEGACY_PLUGIN::FootprintsEnumerate(),
> until Wayne's work is complete.
> 
> Until we grow into the GITHUB_PLUGIN write support, the owner(s) of the repo can simply
> git push to it to update parts.  The repo owner(s) can generate footprints in that library
> any they wish, offline.  Scripting, coffeescript, whatever, as long as you have
> *.kicad_mod files in the repo when its ready to use.
> 
> Writing to the repo *from kicad* means adding support for PLUGIN::FootprintSave(), and
> that won't be bad.  But creating a github library is probably harder using the PLUGIN API.
>  Fortunately the github browser interface can do the creation easily.  The options for the
> fp lib table row will require username and password for your github account, even to use
> someone else's repo on a read only basis, but these two options will remain secret since
> the github API only uses the SSL encrypted sockets to fulfil its API in the link I provided.

You use *your* username and password to read someone else's repo.  You always use your
username and password.

> 
> For reading footprints, they can return the pretty format in "raw" form directly, if in
> the API you ask for raw content. That means the reply to the https request done in
> FootprintLoad() gets you a *.kicad_mod file back, and it is parsed in RAM before the
> MODULE* is returned.
> 
> For enumerating footpringts, what we ask of the github API is the contents of a directory,
> and that is returned in JSON format.  It needs to be parsed before FootprinsEnumerate()
> returns with a wxArrayString, and the '.kicad_mod' extension needs to be chopped off to
> fulfil that PLUGIN API function.
> 
> 
> [1] https://api.github.com/username/repo  I think
> 
> 
> 
> 
> 
>>
>> Chris
>>
>