Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to detect a recursive function in an untrusted assembly (c#)?

Tags:

c#

recursion

I'm writing some c# that will load a third party assembly.

If the third party decided to be malicious, they could write a recursive function that would end up in a StackOverflowException, bringing down my application.

Is it possible to detect a recursive function?

Update: For undesirable sitations like while(true), or for(;;), I already have a solution. Essentially, I run the third party code in a separate thread, and if the thread takes longer than a fixed duration, I pull the plug. This doesn't work well with recursion since the stack limit is reached extremely quickly.

Update: Perhaps I've misrepresented the solution that I'm after. If I end up getting a lot of intentionally malicious code, I'll change the application to run the third party code in a separate process. However at this stage, I'm assuming that the code will only cause problems because it's poorly written.

Accepted Answer I've decided that the best approach would be to run the third party libraries in a separate process. I can have multiple instances of the processes running, and even do a sort of load balancing of my third party libraries across the processes. If malicious code is executed that kills one of the processes, I should be able to detect which library killed it, mark that library as malicious, and relaunch the process with all of the non-malicious libraries.

Thanks for everyone's great suggestions!

like image 581
Robert Avatar asked Dec 22 '10 04:12

Robert


People also ask

How does recursion work in assembly?

Recursion is a unique way of implementing a function and is commonly used in high-level programming languages. A recursive function typically calls itself within itself and returns only when the base case- a special condition- is met.

What is a recursive function in C?

The C programming language allows any of its functions to call itself multiple times in a program. Here, any function that happens to call itself again and again (directly or indirectly), unless the program satisfies some specific condition/subtask is called a recursive function.

How do you stop a recursive function?

A recursive function terminates, if with every recursive call the solution of the problem is downsized and moves towards a base case. A base case is a case, where the problem can be solved without further recursion.

How do you read a recursive function?

A recursive function always has to say when to stop repeating itself. There should always be two parts to a recursive function: the recursive case and the base case. The recursive case is when the function calls itself. The base case is when the function stops calling itself.


2 Answers

It's not easy to do that in the general case. Besides, recursion is a useful tool for programming and it's not a good idea to ban that completely.

A better idea is to run the assembly in another process and use an interprocess communication mechanism to call methods from the trusted process.

like image 176
mmx Avatar answered Nov 03 '22 01:11

mmx


Suppose you found a way to do the impossible and detect recursions. Great. Does that help? No. Nothing is stopping the hostile assembly from simply throwing the exception with a throw statement.

Besides, this is the least of your problems. If you have untrusted hostile code they are going to do a whole lot more nastiness than simply throwing an exception to take down the process. They're going to be trying to steal secret information, install rootkits, you name it. The last thing hostile code wants to do is throw an exception; doing so calls attention to the hostile code. It triggers automatic reports that will be analyzed. Authors of hostile code wants to avoid detection, not loudly call attention to the attack!

If you have a partially trusted third-party assembly then use the tools we have put at your disposal for that exact scenario. Rather than trying to solve impossible problems yourself, spend your valuable time using the Code Access Security system for what it was designed for: to handle partial-trust code.

Probably what you want to study up on is MEF, which is a framework designed by the VSTO team for handling managed add-ons that might have partial trust. (I did some of the early design and security review work for MEF many years ago but I left the team early on and am not an expert on it by any means.)

like image 25
Eric Lippert Avatar answered Nov 03 '22 00:11

Eric Lippert