opencog-dev team mailing list archive

[Gustavo Machado Campagnani Gama <gama@xxxxxxxxxxxxx>]

Hi Joel,

On Thu, 2008-07-10 at 12:05 +1200, Joel Pitt wrote:

> After discussion with Ben, we decided that conversion from stimulus
> (the quantity that MindAgents reward useful atoms with) should be
> converted to short term importance (STI) after each MindAgent runs.
> This is because the amount of STI to reward per unit of stimulus is
> dependent on the amount of STI the MindAgent has to distribute. This
> prevents a MindAgent that isn't particular effective (in achieving
> system goals) from upping the importance of the atoms it uses - since
> they are not important in the overall OpenCog instance.
> 
> At the moment, stimulus -> STI/LTI is just done by the
> ImportanceUpdatingAgent (along with collecting rent and taxation when
> necessary). For the above stim->STI/LTI scheme, it'd probably be
> better to include it as a separate process.
> 
> I'm thinking:
>   * move the atom stimulus map from the AtomSpace to a separate map
> per MindAgent.
>   * In the MindAgent class, provide a method rewardAtoms() which
> calls the WageAgent. This method would also signal to the WageAgent
> who the caller was so that the correct stimulus map and amount of STI
> for reward could be worked out.
>   * It's up to the MindAgent to figure out the appropriate time to
> reward the atoms it uses. Usually at the end of a run cycle.
> 
> Comments welcome :)

I lack the PLN knowledge to comment/criticize the stimulus/importance
updating mechanism you and Ben came up with, but from a scheduler/agent
POV, your idea seems just fine.

Another alternative  might be to just add the method rewardAtoms to the
'Agent' base class and let the other agents call the inherited method
whenever they find it's appropriate. But this scheme would prevent us
from batching/preprocessing the importance updates -- which I assume
would be one of the tasks of the WageAgent.

Or we could let the server itself collect the stimulus/importance data
from each agent and pass it to the Wage agent.

Or... :-)

Anyway, I'm just pointing out that there are many alternatives. I'm
guessing that at these early stages, we'll probably have to do a lot of
trial and error until we find something that works best.

> One requirement I'm thinking about is trying to obtain a lock on the
> AtomTable. Does the thread system we use queue these attempts, or are
> they poll based? (perhaps all thread implementations are one or the
> other, apologies for my ignorance). Ideally they'd be queue based,
> since certain agents (like the Rent collection agent that will be
> split off of the ImportanceUpdating agent) need a lock on the entire
> table - and for sensible behavior they should occur at reasonably
> regular intervals.

AFAIK, if you're running on a Linux system with NPTL -- Native Posix
Threads Library -- that is, any distro released after 2006 or so -- the
threads scheduler doesn't differ from the OS's process scheduler. Which
means that it relies primarily on priorities and that it's relatively
easy to configure your application in such a way that will lead to
starvation of some threads.

However, the current pthreads implementation provides a rich API to help
you synchronize the threads -- and thus create internal queues if you
really need them. Google for pthread_cond_wait and pthread_barrier_wait
for a few examples.

> Another question with the locking. What happens when:
> 
> 1. Agent one has an lvector containing atoms [A, B]
> 1. Agent two has an lvector containing atoms [C, D]
> 
> Agent one requests a lock on atom C, so it waits for Agent two to
> release it. But then Agent two requests a lock on atom A. Would this
> be a race condition? Or is this situation avoided by having the
> AtomTable lock?
> 
> Not trying to nitpick, just trying to understand how things work so my
> code is thread safe!

> Sorry, that should have been "Would this be a deadlock?" - getting my
> terminology mixed up!

Yes, that would be a deadlock. The best way to handle cases like this
would be to lock the atomtable, fetch *all* the atoms you'd require and
only then unlock it.

I realize that this will impose a severe limitation in the ways one may
write and agent and decrease the parallelism of the agents. The best
solution would probably be to provide "timed getters": a version of the
getHandle{Set} methods that would throw/return if any of the atoms it
needs cannot be locked within X time units.

Unfortunately, implementing these "timed getters" would require an even
larger change to the AtomTable/AtomSpace classes and for this first
version I wanted to avoid anything that wouldn't be considered
"essential". But as the threaded version matures, we'll definitely have
to implement this.

> If deadlock possibility isn't already guarded against somehow, then
> (after doing some reading and reminding myself of stuff from my
> undergrad days) we could perhaps do a partial ordering using STI or
> LTI and use the Resource hierarchy solution:
> 
> http://en.wikipedia.org/wiki/Dining_philosophers_problem#Resource_hierarchy_solution
> 
> Alternatively Wait/Die, Wound/Wait methods are probably more efficient
> in the hopefully rare case that deadlock condition occurs:
> 
> http://en.wikipedia.org/wiki/Deadlock#Avoidance
> 
> Obviously this isn't my forte, but thought these suggestions would at
> least get the discussion going.

Thanks for the links. I'll have a look as soon as I get back to the
threading branch.

--
Gustavo