The quality lies deep in the tests

This chapter is all about unit and integration testing of Akka-FTP. Those tests ensure the project quality. To run unit tests type sbt test and to run integration tests type sbt it:test.

Unit tests

Simple tests with mock objects to test components and commands. Although commands should be tested with integration tests and in combinations with other commands, the custom built FTP client may not support the complexity involved. In this case, a unit test is the only option.

Integration tests

Integration tests start the server and perform FTP operations such as login, browse directories, list files and data transfers. The server uses the in-memory file system, all operations on it do not depend on physical hardware and this eliminates some points of failure. More than that, the in-memory file system can serve as an example to implement your own file system. To test the server there is the client with basic FTP actions. The client connects to the server, sends commands, receives and verifies replies and transferred data.

Points of failure

These ports need to be available: 2021, 8021 (optional), 6001, 6002, 6003 and 6004. Anything bound to those ports will cause the integration tests to fail. The port numbers are hardcoded, to change you need to edit the source.

The client binds to 6004 with the PORT command, which is released after a data transfer. But OS takes time to release the port. In the scenario where two data transfers issued one after another with the PORT command, the second will fail - the client binds to the same port (and it succeeds) but the server cannot connect to it and aborts the data transfer. This scenario works fine given time between data transfers. It may be improved in the future.

Some of the data transfer tests use combination of PORT and PAVS to ensure both of them work. There should be enough time for OS to release the 6004 port before it gets used again, but sometimes it fails. In this case, please run the integration tests again.

Test framework

I decided to go with ScalaTest and Mockito. Well documented, easy to use and excellent DSL for matchers. Plus, I have been using those for years and very rarely ran into issues testing stuff.

Summary

This chapter sheds some light on how Akka-FTP is tested, its pitfalls and the frameworks used. The real reason for testing is to capture bugs and prevent weird behaviour. In the future post I will explain how you should use tests if you find a bug. So stay tooned!