Topic: bot filtering

I got propositioned by some scam bot account today, here is its profile.

http://deckbox.org/users/ddona

It would be good to put a bot filter on the account creation page to prevent screen scraping bots from creating accounts and spamming us innocent users. The easiest thing to do would probably be to put in a CAPTCHA plugin, though I believe there are other more elegant solutions out there.

Re: bot filtering

So... they're back sad

I've put a simple captcha on the email sending page but it seems it's totally useless. We'll implement some email sending threshold, emails / day, emails / account life or something, and we'll manually check the user accounts when they are reached and ban offending accounts.

I'm not sure what else would work.

Re: bot filtering

They are breaking through CAPTCHA? yikes...

I suppose it's possible that it is just some low wage person hired to do it manually. Maybe you could ban specific offending IPs, assuming they are not using some crazy tunneling protocol.

There might be some email scanning software out there as well, that can check for spamlike content, maybe some sort of google gmail plugin?

I don't really have many suggestions on how to handle this sort of thing, I've not built a public web app before :-(

Re: bot filtering

Ok so, one thing to think about is using something like OpenID for authentication, you can tie a user account into facebook / google / yahoo for authentication. People don't need to remember an extra password, and their stuff is going to be way better at preventing bots than your stuff will. It is "supposedly" easy to set up as well.

Of course this still doesn't prevent actual people from logging in and spamming.

http://en.wikipedia.org/wiki/OpenID

Re: bot filtering

With all due respect, but I personally would HATE having OpenID, I don't want to connect this with my Google account. I think that wouldn't be the best idea.

Re: bot filtering

I guess I would ask why? In theory, it would be safe / secure / anonymous. You have to remember fewer passwords, and this site would never ever see your authentication. It also wouldn't know your email address, only that you have been authenticated by a trusted third party for your specific account.

You as a user get to choose which 3rd parties you authenticate against as well, so you can avoid google or facebook if you don't trust them, and go with yahoo, or someone else who supports it.

Is your reaction based on superstition or is there some merit as to why you would dislike having that kind of integration?

Re: bot filtering

I don't personally mind OpenID but I think Sebi and Laura need to decide for themselves if it's the right move for the site.  It has implications that span well in to the future.

To combat the spammers without switching to OpenID I would recommend a multi-faceted approach if you agree with it.  You could add Captcha to the account creation page, set a limit to the number of e-mails per minute (or whatever time period) and add a link to all e-mail sent from this site that can report spammers.  If a specific account is reported enough times over a short enough amount of time (e.g. 10 times in 24 hours) then the account is banned or suspended pending review.  This impedes both human and robot spammers. 

The obvious risk to the aforementioned approach is that legitimate people with accounts ban other accounts just to be jerks.  Then you have to figure out how to combat this if it becomes a problem and you'll ultimately become the arbiter of a situation you probably don't want to be involved in sad  I suppose you could just ban the problem accounts here as well.

My sites always run better without all the users.  wink

Re: bot filtering

I don't want to connect this site to my Open ID because I don't want normal people to know I play Magic. tongue

Re: bot filtering

Wow, a lot of great suggestions!

On the issue of OpenID, we've considered implementing it from the beginning, but somehow it was always put away. In any case, we would have it in parallel to normal login / email based signup. This is because for people who don't know about how it works, it's a complicated system that puts them off from signing up.

One of the big design flaws of openid imho is that it is appealing if you understand it, but it's confusing and needs research on first look. Smart systems should make sense from the first look to most people, or they're not picked up... sad

Anyway, coming back to the spam topic, we shall indeed go for a bit of everything smile. Captcha on the front page i'm not sure about yet, we'll start with the others. A trigger system that marks user accounts for review, like signing up from ips that have been problematic in the past, sending more than x emails in y minutes / hours, report user in email, etc.


Thanks for all the suggestions, let us know if you think of anything else, they are very helpful!


P.S. LootPinata, no OpenId, and we're definitely not going to tell any of your normal friends anything, so you're safe tongue


P.P.S xorius, the scanning for spam solution also sounds interesting but I don't know much about these systems, and I'm a bit afraid it's going to need human tweaking constantly to work. It's been noted as an idea, but i'm leaving it for last. In my naive optimism I still hope simple fast solutions will get rid of this problem quickly smile

Last edited by sebi (2011-11-01 08:58:39)

Re: bot filtering

is it possible to disable all of the communication functions if the person's inventory is empty.  that way bots would at least have to learn how to add card to thier inventory (which most if not all would be too stupid to do)