Tell me more ×
IT Security Stack Exchange is a question and answer site for IT security professionals. It's 100% free, no registration required.
up vote 0 down vote favorite

Scenario:

I'm working on a project that needs to accept large amounts of data (customer data) from its users. So it can be normal to have a user trying to add 10,000 or 100,000 records at a time. In some other forms its normal to add 2 or 3 records at a time. In some others 50 to 100.

Problem:

How can we prevent our website forms from robots(or humans) that try to add massive data in order to fill my database resources with wrong or useless data.

Possible solutions that I have ruled out:

Limiting the amount of data is currently not an option, as mentioned. Using CAPTCHA for each form is very manually intensive, as there may be 10000 forms

The question:

So what options do I have to prevent robots/automated tools from accessing the system?

share|improve this question
From your comment, I think I understand what you mean. I have reworded the question to make it clearer. – Rory Alsop♦ Nov 19 '12 at 12:36
The question still is not clear to me. Do you have both 10000 forms and each form accepting 10000-100000 records at a time? – Jan Doggen Nov 19 '12 at 12:57
If you have that many forms and "captcha is manually intensive" that suggests that one individual can be constantly inputting (different) forms, right? In that case, maybe your server software can present one captcha per session/hour/every x forms coming from that one IP address? – Jan Doggen Nov 19 '12 at 12:59
@RoryAlsop: I (strongly) appreciate it Rory. thanks – Michel Kogan Nov 20 '12 at 16:27

1 Answer

up vote 2 down vote

To authenticate users, you use authentication mechanisms.... The answer to your question, as stated, is to provide logins.

If you want to prevent authenticated users from filling your database with erroneous data, then you need correlation mechanisms. Make sure the data entered conforms with what is expected and what can be correlated with existing entries. Just like financial applications need to correlate transactions with other transactions, to ensure that new data cannot be 'invented'. Alternatively, the user data can be validated after it has been entered by a human, or a heuristic algorithm to determine expected ranges of values.

If correlation is not possible, then you cannot trust the data in the database. This is fine when the only people depending on the data are the users entering the data, but if you require other users to depend on the data, then you need accountability mechanisms. Be able to push back on the users to ensure that they conform to requirements.

share|improve this answer
thanks for your answer, can you provide some example about correlation mechanisms – Michel Kogan Nov 20 '12 at 16:29
To withdraw money from the bank, the funds must exist, for the funds to exist, there must be a deposit, for the deposit to occur, there must be money from an external source, which came from another source, and so on. Each record correlates to another record at some point. If your data supports it, only allow user input where it can be related to other valid inputs. – schroeder Nov 20 '12 at 17:35
ok, my acctual question is: what if a user tried to fool bank's server. Something like, he deposit money and withdraw money , depositing and withdrawing ( with valid data ) over and over. milioon times a day. he is filling my database with some data that I can't say it "wrong", but its filling my database. imagine doing this with 1000 users and 1,000,000 times per day for each user. – Michel Kogan Nov 20 '12 at 17:39
1  
Nothing is being 'fooled' in your scenario if the data is valid. What you are describing is a form of DoS. You would have to limit the number of allowed connections/transactions a user can make in a timeframe. – schroeder Nov 20 '12 at 17:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.