opencog-dev team mailing list archive

[Moshe Looks <holism@xxxxxxxxx>]

---------- Forwarded message ----------
From: Moshe Looks <madscience@xxxxxxxxxx>
Date: Mon, Jun 22, 2009 at 9:39 PM
Subject: Re: [opencog-soc] MOSES/RNN combo issue (restricted DAGs in combo)
To: Ben Goertzel <ben@xxxxxxxxxxxx>
Cc: opencog-soc@xxxxxxxxxxxxxxxx, Nil Geisweiller <ngeiswei@xxxxxxxxxxxxxx>


> I think we need feedback from Nil and Moshe on this, but  my own
> intuition is that Option 1 is better.  Reduct is important and making
> tweaks to the data structures that will make Reduct's life difficult,
> doesn't seem a good idea to me...
>
Well, the "natural" solution from my (non-NN-oriented) perspective is
neither of these, but rather to use variable binding to emulate DAG
structure. Consider a network with inputs A, B, C, an intermediate
layer with nodes X, Y, and outputs O, P, where A->X, B->X, B->Y, C->Y,
X->O, Y->O, X->P, Y->P. This would be encoded as :

function (a, b, c)
 let x = sigmoid (w1* a, w2 * b), y = sigmoid (w3 * b, w4 * c)
 return list(sigmoid(w5 * x, w6 * y), sigmoid(w7 * x, w8 * y))

This would use the existing tree structure, and the overhead would not
be *too* horrendous, I don't think (and any way, you could easily
compile to some efficient representation for evaluation if needed). I
don't know what reductions you are planning to implement, however, and
whether this would make them easier or harder to express.

I would recommend that if you do decide to implement your own dag
class, you just make a quick wrapper around an stl list of lists or
something - redoing tree.hh or treetree
(http://code.google.com/p/treetree) as a dag would be too
time-consuming to allow for a successful SoC...

Let me know if you have a questions/comments on this!

- Moshe

> ben g
>
> On Tue, Jun 23, 2009 at 3:17 AM, jal278<lehman.154@xxxxxxxxx> wrote:
>>
>> To be able to fully represent feed-forward NNs
>> in combo (at least the current approach that
>> was decided on earlier) requires that certain
>> tree-like DAGs be able to fit into the combo_tree
>> class. (Basically some nodes may be pointed to
>> by two or more parent nodes as long as they
>> are higher up in the tree).
>>
>> I thought that maybe this would be doable without
>> modifying opencog::tree, as I assumed that a tree_node
>> would have access to all of its children. However,
>> there are optimizations in the tree class so that pointers
>> to all children are not maintained. That is, pointers
>> to the first and last child are explicitly maintained,
>> while the rest are inferred by looking at chain of
>> sibling nodes of its first child. This makes representing
>> DAGs impossible although it is certainly a good
>> optimization for trees.
>>
>> I'm not exactly sure how to proceed here. I can
>> imagine it would not be good to modify opencog::tree
>> from which combo_tree derives since it is used in
>> other areas.
>>
>> I currently can think of two options;
>>
>> 1) I could make a subclass of opencog::tree that
>> would support the tree-like DAGs that MOSES/RNN
>> will need, and then redefine combo_tree as an
>> instance of that.
>>
>> 2) We could represent nodes in combo with
>> id numbers so that it is understood that ANN-Node with
>> id #3 is equivalent to any other ANN-Node vertex with the
>> id #3. In this way, we can avoid having to explicitly link
>> one node to more than one parent. I believe someone
>> raised the issue that this would not work well with the
>> current Reduct system (because it would not be able
>> to equate the two references to the same node).
>>
>> With option #2, we could directly evolve RNNs as well
>> as feed-forward ANNs. Although this may not be a good
>> way to evolve RNNs, it would offer a control to gauge
>> how effective our generalized elman network approach
>> is.
>> --~--~---------~--~----~------------~-------~--~----~
>> You received this message because you are subscribed to the Google Groups "OpenCog Summer of Code Announcements & Discussion List" group.
>> To post to this group, send email to opencog-soc@xxxxxxxxxxxxxxxx
>> To unsubscribe from this group, send email to opencog-soc+unsubscribe@xxxxxxxxxxxxxxxx
>> For more options, visit this group at http://groups.google.com/group/opencog-soc?hl=en
>> -~----------~----~----~----~------~----~------~--~---
>>
>>
>
>
>
> --
> Ben Goertzel, PhD
> CEO, Novamente LLC and Biomind LLC
> Director of Research, SIAI
> ben@xxxxxxxxxxxx
>
> "Truth is a pathless land" -- Jiddu Krishnamurti
>