linuxdcpp-team team mailing list archive

[Fredrik Ullner <ullner@xxxxxxxxx>]

[2015-01-14 16:05] [16:05:57] <[Apex]RoLex> mornings
[2015-01-14 16:06] [16:06:26] <[Apex]RoLex> i have a new feature for dc in general that i would to discuss
[2015-01-14 16:06] [16:06:37] <[Apex]RoLex> flylinkdc++ and verlihub already supports this
[2015-01-14 16:07] [16:07:09] <[Apex]RoLex> ptokax will support this when dc++ will support this, talked to ppk couple of minutes ago
[2015-01-14 16:07] [16:07:33] <[Apex]RoLex> its related to ctm ddos that some still suffer from
...
[2015-01-14 16:10] [16:10:58] <[Apex]RoLex> it is very little tweaking, it uses already existing commands, only one new parameter needs to be added into dc++
[2015-01-14 16:11] [16:11:47] <[Apex]RoLex> 

HUB1		= exploited hub that allows to send fake CTM
HUB2		= attacked hub
HUB2.ADDR	= address of attacked hub

DC1		= a bot that sends bad CTM requests to HUB1
DC2		= regular user on HUB1

DC1	->	HUB1	$ConnectToMe DC2 HUB2.ADDR|
HUB1	->	DC2	$ConnectToMe DC2 HUB2.ADDR|

DC2 doesnt find HUB2.ADDR in his blacklist and continues to process CTM
request

DC2	->	HUB2	socket_connect(HUB2.ADDR) + $MyNick DC2|$Lock ABC|
HUB2	->	DC2	$Error CTM2HUB|

DC2 adds HUB2.ADDR in his blacklist and closes connection

DC1	->	HUB1	$ConnectToMe DC2 HUB2.ADDR|
HUB1	->	DC2	$ConnectToMe DC2 HUB2.ADDR|

DC2 finds HUB2.ADDR in blacklist and ignores CTM request
...
[2015-01-14 16:13] [16:13:04] <[Apex]RoLex> why $Error CTM2HUB is useful:

1. clients that supports CTM2HUB will blacklist the address
2. clients that dont support CTM2HUB will close the connection immediately (client closes connection on any $Error according to dc++ source code)
...
[2015-01-14 16:14] [16:14:57] <[Apex]RoLex> as i said earlier, verlihub and flylinkdc++ already support this scheme, and we made alot of testing, everything works great
...
[2015-01-14 16:30] [16:30:03] <Yorhel> The $Error is probably not even necessary, if a client receives a $Lock instead of $MyNick first it can infer that it's likely not connected to another client and block the IP
[2015-01-14 16:30] [16:30:16] <Yorhel> But having the $Error makes it explicit, so either way works
[2015-01-14 16:30] [16:30:28] <Yorhel> Overall idea seems good
[2015-01-14 16:33] [16:33:17] <[Apex]RoLex> thats what another person told me too, when first command is $Lock, its most likely a hub. but i was little bit agains that because: $Lock is a valid client-client command too, and according to dc++ source code $Lock and $MyNick doesnt have any specific order = $MyNick must not necessarily be received first, $Lock could be received first too
[2015-01-14 16:33] [16:33:28] <[Apex]RoLex> thats why i chose $Error
[2015-01-14 16:33] [16:33:36] <Yorhel> Okay, makes sense
[2015-01-14 16:33] [16:33:57] <[Apex]RoLex> also with using $Error hub has a chance to read the Ref= part (atleast first time)
...
[2015-01-14 19:07] [19:07:59] <Pretorian> Rolex: Seems easier/better to fix the state machine when getting $Lock as first command.
[2015-01-14 19:09] [19:09:04] <Pretorian> $Error is also, as far as I know, a basic text. I'm not aware of any other command that sends a special identifier for a particular error.
[2015-01-14 19:15] [19:15:52] <Pretorian> A very clude way of managing $Error is also to a) attempt connection again after X minutes and b) if above a threshold (say, 3), don't connect anymore (or for a long time).
[2015-01-14 19:17] [19:17:35] <Pretorian> If one wants to do special $Errors, then surely it would be better to do something like ADC's STA which is more extensible.
[2015-01-14 19:19] [19:19:28] <Pretorian> In the scenario with a threshold, maybe 3 is too high or low (I think 2 might be more appropriate).
[2015-01-14 19:22] [19:22:24] <Pretorian> It might also be worth considering the potential of cheaters using this blacklist to prevent themselves from uploading.
[2015-01-14 19:24] [19:24:14] <Pretorian> I don't think necessarily that a blacklist is bad but it is important to consider the ramifications.
...
[2015-01-15 14:46] [14:46:45] <[Apex]RoLex> <Pretorian> It might also be worth considering the potential of cheaters using this blacklist to prevent themselves from uploading.                   what is the difference if i send $Error to close the connection or if i use socket_close() to close the connection? none
[2015-01-15 14:47] [14:47:00] <[Apex]RoLex> so i dont get it about cheating
[2015-01-15 14:49] [14:49:08] <[Apex]RoLex> <Pretorian> I don't think necessarily that a blacklist is bad but it is important to consider the ramifications.             why i chose a blacklist is the first place? i looked at search flood list, it uses similar functionality. it adds seekers to the list and then checks it on every search
[2015-01-15 14:50] [14:50:26] <[Apex]RoLex> there are alot less ctm's than search's in general, so if such a list is already added to search, why not adding it to ctm
[2015-01-15 14:52] [14:52:06] <[Apex]RoLex> <Pretorian> Rolex: Seems easier/better to fix the state machine when getting $Lock as first command.              easier, sure. for you, yes. but for me as a hub its not, i would like to see your referer before i kick your ass. starting with $Lock wont allow me as hub to do that
[2015-01-15 14:52] [14:52:20] <klondike> I can think Searches are broadcasted whilst CTMs aren't
[2015-01-15 14:53] [14:53:02] <[Apex]RoLex> if you detect bad ctm using lock then you can remove Ref= feature, no use. so i suggest using $Error
[2015-01-15 14:53] [14:53:18] <[Apex]RoLex> anyone else wants to share any thoughts?
[2015-01-15 14:58] [14:58:34] <[Apex]RoLex> [2015-01-14 16:13:10] <[Apex]RoLex> why $Error CTM2HUB is useful:

1. clients that supports CTM2HUB will blacklist the address
2. clients that dont support CTM2HUB will close the connection immediately (client closes connection on any $Error according to dc++ source code)
[2015-01-15 14:59] [14:59:46] <[Apex]RoLex> which means that even old clients (that dont yet support CTM2HUB parameter) will partially support this feature, dc++ already closes the connection on any $Error
...
[2015-01-15 15:01] [15:01:13] <[Apex]RoLex> .. so instead of hanging on my hub port in 2 minutes and waiting for handshake timeout, the client will disconnect immediately
[2015-01-15 15:05] [15:05:51] <[Apex]RoLex> why i want this to be added 10 years after exploration of ctm bug? yes, there are only couple of such hubs left that allows to send any ip in ctm request, but popularity of hub-side scripts that send fake ctm requests to its users has not dropped at all
[2015-01-15 15:06] [15:06:47] <[Apex]RoLex> today i see hubs with 1000+ users that run such lua scripts
[2015-01-15 15:07] [15:07:12] <[Apex]RoLex> thats is more than enough to kill any regular server
[2015-01-15 15:09] [15:09:27] <[Apex]RoLex> if hubs cant prevent its owner from sending fake ctm, then lets do it on client side (the regular affected users)
[2015-01-15 15:10] [15:10:31] <[Apex]RoLex> hexhub was the first hub to prevent fake ctm sent with a plugin, now verlihub uses that too
...
[2015-01-15 18:45] [18:45:46] <Pretorian> [tor:14:45:44] <[Apex]RoLex> what is the difference if i send $Error to close the connection or if i use socket_close() to close the connection? none
[2015-01-15 18:46] [18:46:00] <Pretorian> $Error isn't the problem with cheating, the blacklist is.
[2015-01-15 18:46] [18:46:26] <Pretorian> > easier, sure. for you, yes. but for me as a hub its not, i would like to see your referer before i kick your ass. starting with $Lock wont allow me as hub to do that
[2015-01-15 18:46] [18:46:55] <Pretorian> You will have to still get a change in the client software.
[2015-01-15 18:47] [18:47:00] <Yorhel> I dont see a problem with the blacklist, ou can't get anyone on it besides yourself
[2015-01-15 18:47] [18:47:15] <Yorhel> And if you dont want to upload, you might as well block your port
[2015-01-15 18:47] [18:47:24] <Yorhel> Has the same effect
[2015-01-15 18:48] [18:48:06] <Yorhel> There is the risk of memory exhaustion by sending a lot of ips/ports, but That would take so much time/bandwidth it's probably not feasable
[2015-01-15 18:48] [18:48:19] <Yorhel> And it's trivial to put an upper limit to the block list too
[2015-01-15 18:48] [18:48:37] <Pretorian> Again, I raised it as a concern, i.e. something to consider before willynilly implementing a blacklist.
[2015-01-15 18:48] [18:48:56] <Yorhel> And I answered with my quick analysis :)
[2015-01-15 18:50] [18:50:00] <Pretorian> Should the client remember/store the blacklist after a restart? Should the client at all be concerned about the hub that is causing these odd CTMs? How should the blacklist look for users?
[2015-01-15 18:50] [18:50:44] <Pretorian> I _did_ write [ons:19:24:13] <Pretorian> I don't think necessarily that a blacklist is bad but it is important to consider the ramifications.
[2015-01-15 18:51] [18:51:26] <Pretorian> A blacklist is also, as I mentioned above, not contingent on the new $Error.
[2015-01-15 18:53] [18:53:39] <Pretorian> So the any attempt to mitigate this scenario will have to consider a) ways of signalling that such a problem exist ($Error in Rolex's case, a 3-strike proposal from me) and b) what to do when a) does occur (blacklist, increase reconnect time etc).
[2015-01-15 18:56] [18:56:24] <Pretorian> I don't remember if we at any point did any statistics on how many users restart their client (and how often). If the large bulk of offending clients are restarted "fairly often" then one would need to either implement a persistent cache or face the consequence that the hub might get hit again in a few days.
[2015-01-15 18:57] [18:57:35] <Pretorian> The side problem is of course also to get people to upgrade their client, but that is of course not dependent on this particular feature.
[2015-01-15 19:54] [19:54:14] <[Apex]RoLex> <Pretorian> Should the client remember/store the blacklist after a restart? no, i was thinking hub session/until restart
...
[2015-01-15 20:39] [20:39:32] <Pretorian> Rolex: Have you considered the n-strike proposal? It would solve the problem without needing a protocol change.
[2015-01-15 20:40] [20:40:00] <Pretorian> It will waste resources n times (which is why I'd propose a low value) but then will naturally stop.
[2015-01-15 20:40] [20:40:19] <Pretorian> n times and/or possibly a change in reconnect frequency.
[2015-01-15 20:42] [20:42:06] <Pretorian> I don't if old hubs (that haven't been updated) disconnects automatically today, but if they do, then that would mean a n-strike (and/or frequency change) would also work for them.
[2015-01-15 20:42] [20:42:58] <Pretorian> It would also work for non-hubs, I'd wager to guess.
[2015-01-15 20:43] [20:43:48] <Pretorian> The only real advantage a specific error code has is that it is immediate and doesn't waste resources more than once.
[2015-01-15 20:44] [20:44:15] <Pretorian> But if an alternate solution could be employed without requiring additional protocol or software support, then I think that would be generally better.
[2015-01-15 20:49] [20:49:04] <poy> [18:59] [Sent 19:15] <Pretorian> A very clude way of managing $Error is also to a) attempt connection again after X minutes and b) if above a threshold (say, 3), don't connect anymore (or for a long time). > this is already in DC++, precisely to mitigate the scenario RoLex has described. maybe the counters have to be adjusted (only increases by 1 min per error, from what i can see).
[2015-01-15 20:51] [20:51:34] <Pretorian> Ah, ok.
[2015-01-15 20:51] [20:51:57] <poy> that diminishes but does not necessarily nullify the usefulness of the blacklist proposal - whichever solution is agreed upon, i'm ok for having it in DC++.
[2015-01-15 20:56] [20:56:18] <Pretorian> poy: Including changing protocol management?
[2015-01-15 20:59] [20:59:11] <Pretorian> I'm against changing protocol management if there is a solution that can be employed that older implementations also can take advantage of.
[2015-01-15 21:02] [21:02:45] <Pretorian> (Or, rather, instead of "against", I should probably say "I prefer an alternate solution".)
[2015-01-15 21:02] [21:02:49] <poy> including that if it is the only option.
[2015-01-15 21:03] [21:03:25] <poy> sure, would be better without error "text identifiers"...
[2015-01-15 21:04] [21:04:15] <Pretorian> (I'm less 'against' a blacklist.)
[2015-01-15 21:54] [21:54:11] <[Apex]RoLex> <Pretorian> poy: Including changing protocol management?         -- what changes are you speaking about? there is one static parameter supported by dc++ right now - "File Not Available", why not adding another one called "CTM2HUB"? nothing else needs to be done on protocol level
[2015-01-15 21:54] [21:54:45] <[Apex]RoLex> <Pretorian> $Error isn't the problem with cheating, the blacklist is.        -- you can not edit this "blacklist" manually, the client doest that internally, when it finds connection to a hub
[2015-01-15 21:55] [21:55:01] <[Apex]RoLex> did you see the seeker flooder list in dc++ ?
[2015-01-15 21:55] [21:55:26] <[Apex]RoLex> its saved only during the session
[2015-01-15 21:55] [21:55:56] <[Apex]RoLex> seeker flooder = same as search flood list
[2015-01-15 21:57] [21:57:15] <[Apex]RoLex> as i said earlier, there are alot more searches than ctm's that a client has to handle. if today we have a client what can handle many search flooders, then it can handle fewer ctm's
[2015-01-15 21:58] [21:58:21] <[Apex]RoLex> Pretorian: i agree about $Lock detection, would be alot faster. but then you need to give the hub a chance to read your referer information. otherwise referer information would become useless
[2015-01-15 21:58] [21:58:33] <[Apex]RoLex> do you remember why we added it in the first place ?
...
[2015-01-15 22:20] [22:20:55] <Pretorian> Rolex: Any response that a client need to specifically manage and any outgoing message that a hub need to specifically manage is by definition changing protocol management.
[2015-01-15 22:21] [22:21:53] <Pretorian> The change footprint is highly contained and small but there nontheless.
[2015-01-15 22:22] [22:22:52] <RoLex> i agree, you have a point in this. but what about referer ?
[2015-01-15 22:24] [22:24:33] <Pretorian> What about it?
[2015-01-15 22:25] [22:25:09] <RoLex> did you see what i asked earlier ?
[2015-01-15 22:25] [22:25:36] <Pretorian> I'm not sure I understand your point.
[2015-01-15 22:27] [22:27:04] <RoLex> do you know that dc++ sends a referer in $Lock in client-client connections ?
[2015-01-15 22:27] [22:27:10] <Pretorian> Do you mean my previous remark about the lock/mynick state machine?
[2015-01-15 22:27] [22:27:14] <Pretorian> Yes?
[2015-01-15 22:28] [22:28:04] <RoLex> if you will detect "bad" connection already on $Lock, then dc++ will no longer send the $Lock to a hub
[2015-01-15 22:28] [22:28:13] <Pretorian> Yes, that's a fair point.
[2015-01-15 22:28] [22:28:26] <RoLex> .. unless you fource it to do so
[2015-01-15 22:29] [22:29:21] <Pretorian> I don't remember if this occured on NMDC, but many implementations for ADC simply sent/send everything at once.
[2015-01-15 22:29] [22:29:43] <Pretorian> (Even though, to keep to a correct state machine, they shouldn't.)
[2015-01-15 22:29] [22:29:54] <Pretorian> Just noting it.
[2015-01-15 22:30] [22:30:34] <Pretorian> However, do note that having a special $Error or not does not impact the referrer.
[2015-01-15 22:31] [22:31:12] <Pretorian> How does older clients handle the special $Error?
[2015-01-15 22:31] [22:31:25] <RoLex> i already wrote about older clients
[2015-01-15 22:31] [22:31:28] <RoLex> ill write again
[2015-01-15 22:32] [22:32:22] <RoLex> 
[2015-01-15 14:58:41] <[Apex]RoLex> [2015-01-14 16:13:10] <[Apex]RoLex> why $Error CTM2HUB is useful:

1. clients that supports CTM2HUB will blacklist the address
2. clients that dont support CTM2HUB will close the connection immediately (client closes connection on any $Error according to dc++ source code)

[2015-01-15 14:59:52] <[Apex]RoLex> which means that even old clients (that dont yet support CTM2HUB parameter) will partially support this feature, dc++ already closes the connection on any $Error
[2015-01-15 15:01:20] <[Apex]RoLex> .. so instead of hanging on my hub port in 2 minutes and waiting for handshake timeout, the client will disconnect immediately
[2015-01-15 22:32] [22:32:55] <Pretorian> Re "on any $Error"; *any* error?
[2015-01-15 22:33] [22:33:15] <RoLex> $Error command regardless of parameter
[2015-01-15 22:33] [22:33:25] <Pretorian> Ok, that's good at least.
[2015-01-15 22:33] [22:33:29] <RoLex> yep
[2015-01-15 22:33] [22:33:40] <RoLex> thats why i chose $Error
[2015-01-15 22:34] [22:34:39] <Pretorian> It looks like a completely free-text argument so that's good, at least judging by the brief description of the command.
[2015-01-15 22:35] [22:35:32] <Pretorian> Mind you, it's poy that you need to convince, but I am still in favour of tweaking the automanagement.
[2015-01-15 22:35] [22:35:33] <RoLex> qute > there is one static parameter supported by dc++ right now - "File Not Available"
[2015-01-15 22:35] [22:35:54] <Pretorian> Sidenote: that string should be in the spec...
[2015-01-15 22:36] [22:36:08] <Pretorian> (File not avialable)
[2015-01-15 22:38] [22:38:04] <RoLex> poy already said his thoughts: <poy> that diminishes but does not necessarily nullify the usefulness of the blacklist proposal - whichever solution is agreed upon, i'm ok for having it in DC++.
[2015-01-15 22:39] [22:39:29] <poy> i'm just reading and trust you will end up picking the best solution. :)
[2015-01-15 22:39] [22:39:50] <Pretorian> poy: What tweaks could be done to the automanagement?
[2015-01-15 22:40] [22:40:10] <poy> increase the waiting time after an error.
[2015-01-15 22:40] [22:40:46] <Pretorian> There's no n-time strike out?
[2015-01-15 22:41] [22:41:07] <poy> currently i believe it is t, t + 1, t + 3, t + 6, ... in case of errors.
[2015-01-15 22:41] [22:41:31] <Pretorian> That number should really be random (or at least t + n + rand).
[2015-01-15 22:41] [22:41:53] <Pretorian> (As is done with a hub connection.)
[2015-01-15 22:43] [22:43:39] <RoLex> and as i said first of all .. when dc++ will support this, ptokax will support it too. verlihub already does support this, and so does flylinkdc++. when this will be supported by dc++ then other clients will most likely pick up this feature after dc++ => we have a great chance to finally (after 10 years) to get rid of these endless ctm ddos problems once and forever
[2015-01-15 22:44] [22:44:49] <Pretorian> Heh, I don't think it will have that end effect but at least you're optimistic. :)
[2015-01-15 22:45] [22:45:39] <RoLex> it will trust me, regular users on hubs are the ones who cause the attacks
[2015-01-15 22:46] [22:46:15] <RoLex> you can not stop a hub "owner" from running lua scripts, but you can stop it by protecting regilar users
[2015-01-15 22:46] [22:46:26] <RoLex> regular *
[2015-01-15 22:47] [22:47:35] <RoLex> (about connect errors and counters ^) its very good to improve already existing protection indeed, because not all hubs will support CTM2HUB
[2015-01-15 22:48] [22:48:54] <Pretorian> What is the current sequence without the special $Error? Does hubs send a generic $Error and then disconnects?
[2015-01-15 22:49] [22:49:06] <Pretorian> Or they just disconnect?
[2015-01-15 22:49] [22:49:55] <RoLex> older verlihub does nothing, ptokax closes the connection by default
[2015-01-15 22:51] [22:51:39] <Pretorian> So the only difference between the special $Error and a generic one is simply how quickly the blacklist is employed.
[2015-01-15 22:55] [22:55:42] <Pretorian> If that is the case, and I am assuming now that it is, then DC++ would not necessarily have to implement the special $Error but must be more aggressive in its treatment of $Errors. I am just saying this if one should strive to avoid touching the protocol management per se.
[2015-01-15 22:57] [22:57:43] <Pretorian> (Of course, the special $Error does not exclude a more aggresive generic $Error recovery/treatment.)
[2015-01-15 23:01] [23:01:57] <Pretorian> If the client detects the invalid state machine and receives a generic $Error, then it could be as aggresive as the special $Error.
[2015-01-15 23:03] [23:03:00] <Pretorian> By "detects the invalid state machine", I mean to create a flag that there is a potential DoS in effect and that it should blacklist (or be more aggresive in its generic $Error treatment) if it recieves an $Error.
[2015-01-15 23:03] [23:03:01] <RoLex> i see CTM2HUB as a "sure thing", it tells the client with 100% accuracy that it has followed a bad ctm request
[2015-01-15 23:04] [23:04:26] <RoLex> then if you have other solutions, they are sure very useful when ctm2hub is not available
[2015-01-15 23:06] [23:06:30] <Pretorian> (Small side note, I find the name atrocious, but maybe that's just me. The name's not the reason I'm argumenting here, though.)
[2015-01-15 23:10] [23:10:17] <Pretorian> (NMDC's naming is generally bad anyway so...)
[2015-01-15 23:10] [23:10:38] <Pretorian> The blacklisting shouldn't end with the client though. The user should be notified when a client is blacklisted.
...
[2015-01-18 11:57] [11:57:46] <FlylinkDC-dev> poy: CTM2HUB support - https://sourceforge.net/u/ppa74/dcplusplus/ci/e458309cd5a6589609cafb30f76c1f980ac7cbcf/ review and merge?
[2015-01-18 13:21] [13:21:24] <poy> hmm sourceforge sucks; i can't comment...
[2015-01-18 13:21] [13:21:43] <poy> do you have a "send branch for merge" button or some such?
[2015-01-18 13:22] [13:22:20] <poy> otherwise, please just open a bug at <https://bugs.launchpad.net/dcplusplus>. the patch is good but i have some comments.
[2015-01-18 13:23] [13:23:06] <poy> or a "pull request" button - that's how bitbucket calls it.
[2015-01-18 13:33] [13:33:02] <Pretorian> Won't the code in UserConnection.cpp cause a protocol error to *always* be fired?
[2015-01-18 13:33] [13:33:27] <Pretorian> Or maybe the last } is part of the if and the diff just looks odd...
[2015-01-18 13:34] [13:34:04] <Pretorian> (Mostly it was the lack of indentation that threw me off.)
[2015-01-18 14:03] [14:03:06] <RoLex> poy: there are so few lines of new code, do you really need a patch? i can merge changes manually by just looking at the sf.net page
[2015-01-18 14:34] [14:34:43] <poy> well here are the comments i noted (best suited for a bug report / pull request page):
1) move the "CTM2HUB" check to ConnectionManager::on(UserConnectionListener::ProtocolError, ...). this way the "addCTM2HUB" function won't even be needed.
2) translation wrappers missing from log messages but it's no big deal; i will add them.
3) remove the "serverPort" variable, added only for the purpose of an event that won't often happen.
4) what would be the ADC equivalent of CTM2HUB? would be good to also handle ADC through the same code path.
[2015-01-18 15:51] [15:51:24] <Pretorian> STA 231 (perm ban)
[2015-01-18 15:55] [15:55:19] <Pretorian> Alternatively, a STA 220 (generic login/access error) or STA 441 (transfer protocol unsupported)
[2015-01-18 15:55] [15:55:33] <Pretorian> There's a few of them, suggest/pick one that feels best.
[2015-01-18 18:22] [18:22:21] <poy> hmm, that rings a very tiny bell... what do current hubs answer on a client connection?
[2015-01-18 18:36] [18:36:18] <Pretorian> Don't know.
[2015-01-18 18:37] [18:37:53] <Pretorian> Probably be dropping and saying invalid state or invalid FCC.

-- 
You received this bug notification because you are a member of
Dcplusplus-team, which is subscribed to DC++.
https://bugs.launchpad.net/bugs/1419478

Title:
  Mitigating classic NMDC CTM DOS attacks

Status in DC++:
  Confirmed

Bug description:
  Proposal and discussion log from the public hub :
  http://pastebin.com/dMY5eaRP

To manage notifications about this bug go to:
https://bugs.launchpad.net/dcplusplus/+bug/1419478/+subscriptions