Tag Archive: TDD

Some are still not confuse so lets cover this point again. In mobile we have a virtual machine meaning that we cannot predict the behavior of member variables that have been exposed. in other words its the underlying reason why we came up with interface patterns in Java. That is different from say ObjectiveC iOS programming. That is the uniqueness of developing on a computer language environment that happens o use a Virtual Machine.

So in performance constrained environments where we are relying less on interface patterns what testing do we have to do to verify behavior since we are not normally mapping it too well through our architecture design? Exactly, BDD or functional unit testing which directly implies instrumented testing because that is the only way to measure the behavior on such a test in the first place.

Developers who advocate TDD unit testing in a JVM in a VM computer language that is different from running the same application through BDD unit testing on an emulator or device are hiding a choice they are fostering on you to appear cool when inn actual reality they expose themselves to deficiency in understanding what a mobile VM does why certain architecture patterns arose within java. That is a very bad choice to foster on someone in name of speed to replace proficiency in java development in a mobile constrained environment.


Robolectric and similar projects have always complained that instrumented unit testing(yes, you can do both tdd an bdd instrumented testing in android development as the plain old unit test classes in the android framework are not loading the tested entity within the android framework system. if you doubt me look at the difference between ActivityInsgtrumentationTestCase2 and ActivityUnitTestCase classes. In Android all TDD unit style test classes have Unit in the class name.) takes more time.

Specifically, its the time to dex the classes and to start the emulator and install the application as when in in TDD mode you are writing the tests first before your application code. Is that that much difference between writing several unit tests for one class waiting more for those tests to finish in instrumented test,mode on the emulator or writing some unit tests for a class running the tests on the emulator while you write more unit tests for another class.

But design shops seem to have bought into less time is more impressive results even though as the android framework matures the results for JVM unit testing it get less and less effective as you cannot predict the behavior of member variables that have been exposed. But, you could speed up unit test writing.

You could use hamcrest matchers and you could build the test-assertions-android github project to get android style fest assertions for the android environment which would act not only to speed up unit test writing but would allow you be able to go back to several unit test results after they are executed and understand them as they are closer to plain English than a regular android unit test with those libraries not applied to a unit test.

This not the only concern however in testing. In Android Instrumented Testing no matter if run it for the IDE or the ant script or from a monkeyrunner python script you have three testing areas; Unit tests, Functional tests, and Performance tests. now lets look back at android sdk build script. Notice in the test-helper-runner macro you do not see an value for running junit runner in multi-file mode. What does multi-file mode do? Well if you use an alternative android -junit-runner with multi-file mode enabled than you test results get grouped by testsuite thus than you have them grouped by; Unit tests, Functional tests, Performance tests.

This is why as of this moment I am rewriting the build scripts supplied with the android sdk as to fully get the benefits of being able to run both TDD and BDD in android instrumented fashion in the IDE those ant script changes will than be applied in setting my IDE java/android builder to use the ant script. And due to the fact that I will have a dashboard generated with the docs during the debug/compile target execution containing such things as Checkstyle, PMD, JavaNCSS, class, and jdepend reports it will be somewhat more effective in getting a snapshot view of the project each time I change code.

Than we have the aspect of some areas are not covered well, regression testing and acceptance testing. Regression testing could be enabled by using monkeyrunner as it has  a user/UI recorder. Now, we do have android-guitar framework which runs on to of monkeyrunner which gives us an event model report but it does not support swipes and some other UI interactions. So we have either extend android guitar or monkeyrunner. Actually that is a nop as monkeyrunner accepts plugins that are written in java and thus the plugins you add to extend monkeyrunner could actually also extend android guitar. Thus, that is another piece I have to add to the testing part of the android app build script, ie the ability to execute the monkeyrunner scripts as part of the instrumented testing developer workflow. Also remember monkeyrunner is an automation tool that is integrated with the android framework api which means anything in the instrumented testing process that can be automated is now reachable to be automated. For example, you could turn off the wakeful lock, etc as part of the process of starting the device or emulator in your instrumented testing process.

This is generally why to rescue an android project you have to start-over the 3rd time, at least in applying agile as you need to change the way tools are integrated together in the developer process and the work-flow and of course the culture. Its not can I buy you coffee and you consult for an hour and the project is rescued type of situation. And my thinking is if you want and value the benefits of full agile, not just TDD and BDD but the change in culture which desire upfront feedback(ie transparency), than you have to be willing to put in the time and costs to change it. Are you willing as company or firm willing to hire a developer to change it? think it about another way.

At one point in internet history we were using Gopher and it basically sucked. At time I watched as people(developers) came up with a better way and people started using it and it was extremely beautiful in its simplicity and implementation and execution. That is the joy of solving hard development problems.

Agile in Android

I guess I should explain this. Lets dig into the android framework first.

The Android Activity is something that the framework creates as you are not in complete control in creating it. That is somewhat important as you normally cannot prepare aspect stuff ahead of time to be created thus there are three other ways to be able to get a hold of it. IoC in an object registry pattern using interfaces(ever wonder why games are so big..there is your hint), some way to mock things or dependency injection or both mock and dependency injection.

This means that agile in android application development ha some differences when compared to other environments where you had no memory/storage constraints and could do as many interfaces as you pleased. Thus we have two areas where we can impact development project times. One is testing where we can use both mocks and dependency injection to in a fast manner implement functional testing and the other area is in application development itself by using dependency injection.

This is not to say that TDD is not agile but, and this is extremely important, TDD does not test behavior in of itself. In other words, yes Robolectric can do functional testing if you are using the advanced features but remember you are testing that behavior in non-alike sandbox as a normal JVM application does not take control of certain object/class cobinations like the android framework does ith the activity class.

Now lets backup a second, what about static classes. Normally you wrap a static class in an interface and test the interface. That is why see several mock frameworks used in android application testing such as Android-Mock. Robolectric does something different called shadow classes. The drawback to those shadow classes is that they DO NOT PREDICT THE BEHAVIOR OF MEMBER VARIABLES THAT ARE EXPOSED!

In the android framework what is the most common area where member variables are found? Hint its the GUI. Where is the common area where those fields are exposed to developers using them/ Both the GUI area AND the INDIVIDUAL FRAMEWORKS THAT MAKE UP THE FEATURES of what an application can accomplish on android. this is why when someone claims that robolectric can test behavior of an android application in a reliable fashion, that they may not know what the eff they are talking about.

Its kind of important to dig into the underlying java and computer science to understand the trade-offs to each agile approach. I personally know of the re-factor work that was completed from android 2.0 to android 3.0 and android 4.0 to expose most of the member variables in the android GUI framework and thus as someone relies upon robolectric going forward they are in actuality testing less and less of actual android application behavior not more.

This is why you find two things in the android framework , some basic mock classes and instrumented testing. Both of which were not in android 1.0 and were added later due to the concerns described in previous paragraphs. Yes, it takes a little more time than TDD/Robolectric but the results when implemented correctly are more reliable


Some people will not like this, touch shit. When Robolectric was created, forked form something else, PivotalLabs made some direct and indirect assumptions. let us walk through that.

1. They assumed that TDD has a direct relationship to quality of code, it does not. See Proffitt, Jacob. “TDD Proven Effective! Or is it?”

2. That TDD correlates with GUI testing, it does not as in GUI testing  we need to do testing of full functional test for success or failure and android application testing is heavily focused on GUI testing. You would think a design shop like Pivotal labs was aware of this aspect. And if you are doing functional tests anyway why create more tests at the beginning? That answer has to do with how Agile movement began, which was through huge code failures on huge projects whereas an android application is in fact in most cases a small to medium project in size.

3. You do not in mobile applications want to fake the SQL testing or the network testing. Due to robolectric using JVM instead of dalvik you are in fact fakigng those things rather than fully testing them as functional testing in the whole application.

4. TDD via JVM as opposed to DVM instrumental testing separates any direct or indirect relationship TDD had to acceptance and functional testing and results in being given a false sense of security that due to the success of TDD tests that acceptance/functional testing does not have to be performed.

It seems to me that the stated benefit of decreasing development time as in using Robolectric you are avoiding dex’ing and packaging an android application during testing is out-weighed by the harm you are doing in giving yourself some wrong assumptions. And in mobile development because of the rapid OS development cycles and OEM ecological-system we are dealing with alpha or beta situations not like in Enterprise whereas the application server you are coidng to can be considered to be very close to production levels at most times.