How to test a class that connects to a FTP server?

Question

I am developing a live update for my application. So far, I have created almost all unit tests but I have no idea how to test an specific class that connects to a FTP server and downloads new versions.

To test this class, should I create an FTP test server and use it in my unit tests? If so, how can I make sure this FTP server is always consistent to my tests? Should I create manually every file I will need before the test begin or should I automate this in my Test Class (tear down and setup methods)?

This question also applies to unit testing classes that connects do any kind of server.

EDIT

I am already mocking my ftp class so I dont always need to connect to the ftp server in other tests.

Let me see if I got this right about what Warren said in his comment:

I would argue that once you're talking to a separate app over TCP/IP we should call that "integration tests". One is no longer testing a unit or a method, but a system.

When a unit test needs to communicate to another app (that can be a HTTP server or FTP server) is this no longer a unit test but a integration server? If so, am I doing it wrong by trying to use unit testing techniques to create this test? Is it correct to say that I should not unit test this class? It does make sense to me because it seems to be a lot of work for a unit test.

Answer 1

In testing, the purpose is always first to answer the question: what is tested - that is, the scope of the test.

So if you are testing a FTP server implementation, you'll have to create a FTP client.

If you are testing a FTP client, you'll have to create a FTP server.

You'll have therefore to downsize the test extend, until you'll reach an unitary level.

It may be e.g. for your purpose:

• Getting a list of the current files installed for the application;
• Getting a list of the files available remotely;
• Getting a file update;
• Checking that a file is correct (checksum?);
• and so on...

Each tested item is to have some mocks and stubs. See this article about the difference between the two. In short (AFAIK), a stub is just an emulation object, which always works. And a mock (which should be unique in each test) is the element which may change the test result (pass or fail).

For the exact purpose of a FTP connection, you may e.g. (when testing the client side) have some stubs which return a list of files, and a mock which will test several possible issues of the FTP server (time out, connection lost, wrong content). Then your client side shall react as expected. Your mock may be a true FTP server instance, but which will behave as expected to trigger all potential errors. Typically, each error shall raise an exception, which is to be tracked by the test units, in order to pass/fail each test.

This is a bit difficult to write good testing code. A test-driven approach is a bit time consuming at first, but it is always better in the long term. A good book is here mandatory, or at least some reference articles (like Martin Fowler's as linked above). In Delphi, using interfaces and SOLID principles may help you writing such code, and creating stubs/mocks to write your tests.

From my experiment, every programmer can be sometimes lost in writing tests... good test writing can be more time consuming than feature writing, in some circumstances... you are warned! Each test shall be see as a feature, and its cost shall be evaluated: is it worth it? Is not another test more suitable here? Is my test decoupled from the feature it is testing? Is it not already tested? Am I testing my code, or a third-party/library feature?

Out of the subject, but my two cents: HTTP/1.1 may be a better candidate nowadays than FTP, even for file update. You can resume a HTTP connection, load HTTP content by chunks in parallel, and this protocol is more proxy friendly than FTP. And it is much easier to host some HTTP content than FTP (some FTP servers have also known security issues). Most software updates are performed via HTTP/1.1 these days, not FTP (e.g. Microsoft products or most Linux repositories).

EDIT:

You may argue that you are making integration tests, when you use a remote protocol. It could make sense, but IMHO this is not the same.

To my understanding, integration tests take place when you let all your components work together as with the real application, then check that they are working as expected. My proposal about FTP testing is that you are mocking a FTP server in order to explicitly test all potential issues (timeout, connection or transmission error...). This is something else than integration tests: code coverage is much bigger. And you are only testing one part of the code, not the whole code integration. This is not because you are using some remote connection that you are doing integration tests: this is still unitary testing.

And, of course, integration and system tests shall be performed after unitary tests. But FTP client unitary tests can mock a FTP server, running it locally, but testing all potential issues which may occur in the real big world wide web.