What percentage of my time will be spent in user input verfication during web development?

Question

I'm new to developing things on the web. So far, I'm spending a lot of time (50% or so) to try and prevent bad people from putting things like sql injection into my input forms and validating it server side. Is this normal?

Answer 1

To prevent sql injection attacks just do your queries with prepared statements (the exact way will depend on your platform). Once you do that, you'll never have to bother with this particular aspect again. You just need to use this everywhere.

As for general input validation, it's always good to rely on a common base to test for required fields, numbers, etc. ASP.Net's validators are very easy to use for example. One rule of thumb you should follow is not to trust client-side (javascript) to do this for you, since it's easy to go around it. Always do it server-side first.

A special case to keep under your radar is when you allow rich content to be introduced that may contain html/javascript. This can allow a malicious user to inject javascript in your data that will trigger code you don't control when you render it. Do not try to roll your own validation code. Search the web for free, tested, mantained code that will do it for you. Jeff had a few pointers in that regard in one of the podcasts.

Once you automate your input validation code, the time spent doing it should be directly related to the complexity of your business rules. So as a general rule: keep them simple.

Answer 2

No. It is not normal. Maybe you need to:

• Use the rights components to avoid SQL Injection (PreparedStatements in Java)
• Create a component that "filters" messages coming from user (a servlet Filter in Java).

Any modern language has support for both things.

Kind Regards

Answer 3

I'm glad you're taking care to protect yourself. Too many don't.

However as others have said, a better choice of architecture will make your problems go away. Using prepared statements (most languages should have support for that) will make SQL injection attacks go away. Plus with many databases they will result in significantly better performance. Handling cross-site scripting attacks is more tricky. But the basic strategy has to be to decide how you will escape user input, decide where you will escape it, and always do it in the same place. Don't fall into the trap of thinking that more is better! Consistently doing it in one way in one place will suffice, and will avoid your having to figure out which of the multiple levels of escaping are causing a specific bug.

Or course learning how to create and maintain a sane architecture takes experience. And moreover, it takes reflecting on your bad experiences. So pay attention to your current pain points (it looks like you are), and think about what you could have done differently to avoid them. If you have a mentor, talk with your mentor. That won't always help you so much with this project, but it will with the next.

Answer 4

@Jeremy - some PHP specifics

When it comes to database queries, always try and use prepared parameterised queries. The mysqli and PDO libraries support this. This is infinitely safer than using escaping functions such as mysql_real_escape_string.

Yes, mysql_real_escape_string is effectively just a string escaping function. It is not a magic bullet. All it will do is escape dangerous characters in order that they can be safe to use in a single query string. However, if you do not sanitise your inputs beforehand, then you will be vulnerable to certain attack vectors.

Imagine the following SQL:

$result = "SELECT fields FROM table WHERE id = ".mysql_real_escape_string($_POST['id']);

You should be able to see that this is vulnerable to exploit.
Imagine the id parameter contained the common attack vector:

1 OR 1=1

There's no risky chars in their to encode, so it will pass straight through the escaping filter. Leaving us:

SELECT fields FROM table WHERE id = 1 OR 1=1

Which is a lovely SQL injection vector.

Whilst these functions are useful, they must be used with care. You need to ensure that all web inputs are validated to some degree. In this case, we see that we can be exploited because we didn't check that a variable we were using as a number, was actually numeric. In PHP you should widely use a set of functions to check that inputs are integers, floats, alphanumeric etc. But when it comes to SQL, heed most the value of the prepared statement. The above code would have been secure if it was a prepared statement as the database functions would have known that 1 OR 1=1 is not a valid literal.

As for htmlspecialchars(). That's a minefield of its own.

There's a real problem in PHP in that it has a whole selection of different html-related escaping functions, and no clear guidance on exactly which functions do what.

Firstly, if you are inside an HTML tag, you are in real trouble. Look at

echo '<img src= "' . htmlspecialchars($_GET['imagesrc']) . '" />';

We're already inside an HTML tag, so we don't need < or > to do anything dangerous. Our attack vector could just be javascript:alert(document.cookie)

Now resultant HTML looks like

<img src= "javascript:alert(document.cookie)" />

The attack gets straight through.

It gets worse. Why? because htmlspecialchars only encodes double quotes and not single. So if we had

echo "<img src= '" . htmlspecialchars($_GET['imagesrc']) . ". />";

Our evil attacker can now inject whole new parameters

pic.png' onclick='location.href=xxx' onmouseover='...

gives us

<img src='pic.png' onclick='location.href=xxx' onmouseover='...' />

In these cases, there is no magic bullet, you just have to santise the input yourself. If you try and filter out bad characters you will surely fail. Take a whitelist approach and only let through the chars which are good. Look at the XSS cheat sheet for examples on how diverse vectors can be

Even if you use htmlspecialchars($string) outside of HTML tags, you are still vulnerable to multi-byte charset attack vectors.

The most effective you can be is to use the a combination of mb_convert_encoding and htmlentities as follows.

$str = mb_convert_encoding($str, ‘UTF-8′, ‘UTF-8′);
$str = htmlentities($str, ENT_QUOTES, ‘UTF-8′);

Even this leaves IE6 vulnerable, because of the way it handles UTF. However, you could fall back to a more limited encoding, such as ISO-8859-1, until IE6 usage drops off.