Java Habits for main method

Question

I write code primarily for personal use, but I'm considering releasing an application (scientific simulation/visualization) that I originally developed for personal use.

One of my habits is to use a main method in classes for testing the operation of the class in isolation. I figure that's probably bad in someway (as are no doubt various other habits originating from self-teaching and the scientific development environment). However, it's never been a problem for self-use stuff that I've noticed.

Would you all be so kind as to confirm (or deny) that the proliferation of mains is a problem for an application released to the scientific community (the source would also be open), and if so, why?

EDIT: To play devil's advocate (okay, my advocate) relative to some of the offered answers: part of the "application use" is expected to be source modification by non-developers (the typical scientist) on a smallish scale. I know that on the receiving end, that having the tests for a class built directly into that class would be pretty straightforward for me to recognize and modify accordingly (especially if that were consistently the case for the classes). Would using something like JUnit provide similar utility, keeping in mind the audience?

ACCEPT DECISION: I think KLE's answer is the best balance of thorough and succinct, so I picked it, but I think the discussion comments in Bill's are also very helpful. I also don't understand why Johannes's answer was voted down - the "how does this piece work" perspective is very important to the scientific community coders - and while the other answers point out various reasons why separated unit tests are probably more useful than my current habit, they don't really address that use, so his answer is far from "unhelpful". Thanks to all current (and future) responders, and here's to wishing there was a way to combine multiple responses as the correct answer!

Answer 1

Testing your class in its own main method is bad because it gives the class an extra responsibility (testing itself). Tests should go in separate classes, preferably using a testing library like JUnit.

The proliferation of mains (I like this phrase you've coined) also makes it more confusing for a developer to find the entry point to your application when they are approaching it for the first time.