linuxdcpp-team team mailing list archive

[eMTee <2110291@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

There is a possible scenario where other users logged into the same ADCH++ hub with Bloom filter support
may not receive search results (by TTH) for one or more updated files after manually refreshing the share in DC++, until the user updates the share once more or reconnects to the hub.

The problem is consistently reproducible after one or a few files getting updated and the sharre refreshed,
if the overall size of the changed files is relatively small.

To reproduce this, you need to update already shared file(s) with different content,
or perform a similar number of file removals and additions to the share, then manually refresh the share.

The cause of the issue is that sending INFs — just like any other commands — is not instantaneous.
The function that compiles the INF command is placed into the async task queue of all connected hubs' sockets, to be run when feasible.
If, for example, you update one small file and refresh the share, normally that would result in sending SF = lastSF - 1 with the infoupdate() right after the refresh.
Then, the hashing thread's TTHDone event handler updates the total number of files after the file with the updated content has been hashed.
This change is then sent with the next scheduled infoupdate() (typically minutely).

But... if the small updated file is already hashed by the time the hub's respective infoupdate() is called,
then SF becomes lastSF + 1 again. Bingo — the value is correct, but the Bloom plugin won't be signaled to request a filter update.
OTOH if the hasher's queue is empty before the share refresh, it will indeed start working almost instantaneously, so if the total size of the updated file(s) is small enough, it often wins the race, it seems.

The largest total updated file size to reproduce this depends on your hardware.
It is higher with faster CPUs and storage, and also depends on how busy the hub/socket is at the time.

On a system with a 100Mb/s HDD read speed and an i5-6600 CPU, the threshold is about 15 MiB.
Obviously, this could easily be 10 times larger on modern hardware.

** Affects: dcplusplus
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of
Dcplusplus-team, which is subscribed to DC++.
https://bugs.launchpad.net/bugs/2110291

Title:
  One time small updates in the share may not trigger a Bloom filter
  update request which makes such updated files unsearchable by TTH for
  other hub users

Status in DC++:
  New

Bug description:
  There is a possible scenario where other users logged into the same ADCH++ hub with Bloom filter support
  may not receive search results (by TTH) for one or more updated files after manually refreshing the share in DC++, until the user updates the share once more or reconnects to the hub.

  The problem is consistently reproducible after one or a few files getting updated and the sharre refreshed,
  if the overall size of the changed files is relatively small.

  To reproduce this, you need to update already shared file(s) with different content,
  or perform a similar number of file removals and additions to the share, then manually refresh the share.

  The cause of the issue is that sending INFs — just like any other commands — is not instantaneous.
  The function that compiles the INF command is placed into the async task queue of all connected hubs' sockets, to be run when feasible.
  If, for example, you update one small file and refresh the share, normally that would result in sending SF = lastSF - 1 with the infoupdate() right after the refresh.
  Then, the hashing thread's TTHDone event handler updates the total number of files after the file with the updated content has been hashed.
  This change is then sent with the next scheduled infoupdate() (typically minutely).

  But... if the small updated file is already hashed by the time the hub's respective infoupdate() is called,
  then SF becomes lastSF + 1 again. Bingo — the value is correct, but the Bloom plugin won't be signaled to request a filter update.
  OTOH if the hasher's queue is empty before the share refresh, it will indeed start working almost instantaneously, so if the total size of the updated file(s) is small enough, it often wins the race, it seems.

  The largest total updated file size to reproduce this depends on your hardware.
  It is higher with faster CPUs and storage, and also depends on how busy the hub/socket is at the time.

  On a system with a 100Mb/s HDD read speed and an i5-6600 CPU, the threshold is about 15 MiB.
  Obviously, this could easily be 10 times larger on modern hardware.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dcplusplus/+bug/2110291/+subscriptions