opencog-dev team mailing list archive

[Gustavo Machado Campagnani Gama <gama@xxxxxxxxxxxxx>]

Hi Joel/David,

sorry for the delay. My inbox was a bit of a mess this past week. :-/

On Wed, 2008-07-02 at 16:25 +1200, Joel Pitt wrote:

> I going through the threading code, and read about users of the
> Atom::getNeighbours method having to lock atoms manually.
> 
> I was wondering how I'd go about this, since I use the function
> relatively frequently and want to be prepared for when you merge the
> threading code.
> 
> The AtomSpace and AtomTable test examples seem to lock the entire
> table. Do I need to do the same thing while processing the
> neighbouring atoms, or is there a finer level of locking available
> somehow?

Unfortunately no. The only way we may assure that no atoms will be
removed while we're building the list of HandleEntrys is by locking the
AtomTable. However, once the list is built, only the atoms in it need to
be locked; the table itself may be released.

Ideally, you'd use the lockable container to handle the "atom locks". To
illustrate it, I've hastily cooked up the attached patch implementing a
"getLockedNeighbors" method (I haven't tested it though, so use with
care). Assuming the patch works, what you should do is replace the
getNeighbors calls on your code and replace with corresponding
getLockableNeighbors calls. But, as noted on the README, be careful with
the lifecycle of the container you pass to the getLockableNeighbors
method: the atoms will be locked until the container is destroyed.

> Also, thinking about your idea of having mind agents as separate
> threads with scheduling handled by the OS. I like the idea... are you
> planning to have the server control the scheduling at all though?
> Certain mind agents should only run so often, whereas others will use
> as much CPU as they can their hands on! I guess this would be handled
> by appropriately placed sleep calls though.

My original idea was to use a very simple start/stop interface for the
Agents. The Server would provide load/unload operations (similar to the
current dlopen and dlclose). Internally, the server would also provide a
method getAgent(std::string name) (or getAgent(int id)) so that agents
could communicate with each other. And I was considering converting the
current command processor to use agents -- but I'm not unsure about the
performance implications of starting a new thread for each command.

That's about it; everything else would be up to the agents:

   - if an agent should use all the available CPU, then it just never 
     sleeps; the kernel will preempt it as it does any other thread 
     (the agent should probably lower its priority though).
   - if another agent should run every X cycles, then it simply issues 
     a "sleep" call at the end of its main loop.
   - if an agent wants to start another agent, it calls the
     "server().startAgent()" method
   - if an agent needs to send a message to another agent, it retrieves 
     the instance thorugh "server().getAgent()", casts the Agent 
     reference to the proper Agent class and calls the desired method. 

And so on...

Nothing is set in stone, of course, and I'd be glad to hear about any
other important requirements. Keep in mind though that nothing prevents
us from refining the proposed design and add additional functionality in
the future. What I'm mostly concerned right now is to have something
that's simple (and extensible) so that we may have it up and running
relativelly soon.

On Wed, 2008-07-02 at 14:41 +1000, David Hart wrote:
> I believe other mindagent scheduling constraints exist, like mutual
> exclusivity and sequence dependence. 

Mutual exclusivity would be handle by locking the proper atoms. Sequence
dependence could be handled by creating/removing atoms as well, but I'm
not sure this is the best approach. However, the current scheduler
doesn't support anything like this (nor does the older Novamente
scheduler, IIRC).

> Could the cogserver impose its scheduling constraints on mindagent
> processes via multiple mechanisms, like simple process control
> (spawning, stop, start, wait, etc.), message passing (cooperative
> interrupt with libevent or similar would be required for
> non-concurrent-safe or non-stoppable mindagents), etc?

Hmm.. I'm not sure I understood this last comment. Would you mind
explaining why/how we'd use libevent to handle non-concurrent-safe or
non-stoppable mind agents? And why would they exist in the first place?

--
Gustavo

=== modified file 'src/atomspace/Atom.cc'
--- src/atomspace/Atom.cc	2008-06-27 21:54:38 +0000
+++ src/atomspace/Atom.cc	2008-07-07 20:21:38 +0000
@@ -815,10 +815,21 @@ HandleEntry *Atom::getNeighbors(bool fan
     }
 
     return answer;
 }
 
+void Atom::getLockedNeighbors(HandleSeq neighbors, bool fanin, bool fanout, Type desiredLinkType, bool subClasses) const
+{
+    ScopedMutex::Lock l(mutex);
+    if (atomTable) atomTable->lock();
+    HandleEntry* unlocked_neighbors = getNeighbors(fanin, fanout, desiredLinkType, subClasses);
+    for (HandleEntry *it = unlocked_neighbors; it != NULL; it = it->next)
+        neighbors.push_back(it->handle);
+    delete unlocked_neighbors;
+    if (atomTable) atomTable->unlock();
+}
+
 #ifndef PUT_OUTGOING_SET_IN_LINKS
 #ifdef USE_STD_VECTOR_FOR_OUTGOING
 HandleSeq Atom::getLockedOutgoingSet() const {
     HandleSeq result(outgoing.begin(), outgoing.end());
     return result;

=== modified file 'src/atomspace/Atom.h'
--- src/atomspace/Atom.h	2008-06-27 21:54:38 +0000
+++ src/atomspace/Atom.h	2008-07-07 20:21:24 +0000
@@ -510,10 +510,22 @@ public:
                               bool fanout = true,
                               Type linkType = LINK,
                               bool subClasses = true) const;
 
     /**
+     * Returns neighboring atoms, following links and returning their
+     * target sets.
+     * @param neighbors lockable container where the neighbor handles will
+     *                 be stored
+     */
+    void getLockedNeighbors(HandleSeq neighbors,
+                            bool fanin,
+                            bool fanout,
+                            Type desiredLinkType,
+                            bool subClasses) const;
+
+    /**
      * Returns a string representation of the node.
      *
      * @return A string representation of the node.
      */
     virtual std::string toString(void) = 0;