Tag Archive: functional testing

Cucumber-Android Tests

Someone had asked me the other day my progress on running cucumber-jvm for android testing. Someone beat me to it, the project is CalabashAndroid. It already has its own ant test build file just type ant test to run after you change  that one line in the properties file.

Some people will not like this, touch shit. When Robolectric was created, forked form something else, PivotalLabs made some direct and indirect assumptions. let us walk through that.

1. They assumed that TDD has a direct relationship to quality of code, it does not. See Proffitt, Jacob. “TDD Proven Effective! Or is it?”

2. That TDD correlates with GUI testing, it does not as in GUI testing  we need to do testing of full functional test for success or failure and android application testing is heavily focused on GUI testing. You would think a design shop like Pivotal labs was aware of this aspect. And if you are doing functional tests anyway why create more tests at the beginning? That answer has to do with how Agile movement began, which was through huge code failures on huge projects whereas an android application is in fact in most cases a small to medium project in size.

3. You do not in mobile applications want to fake the SQL testing or the network testing. Due to robolectric using JVM instead of dalvik you are in fact fakigng those things rather than fully testing them as functional testing in the whole application.

4. TDD via JVM as opposed to DVM instrumental testing separates any direct or indirect relationship TDD had to acceptance and functional testing and results in being given a false sense of security that due to the success of TDD tests that acceptance/functional testing does not have to be performed.

It seems to me that the stated benefit of decreasing development time as in using Robolectric you are avoiding dex’ing and packaging an android application during testing is out-weighed by the harm you are doing in giving yourself some wrong assumptions. And in mobile development because of the rapid OS development cycles and OEM ecological-system we are dealing with alpha or beta situations not like in Enterprise whereas the application server you are coidng to can be considered to be very close to production levels at most times.