Tag Archive: BDD

a BDD Android Example

I am putting up a BDD example while I finish my customized build scripts. It will be setup so that if you are using eclipse IDE you should be able to fully set it up with the exception of Cucumber without waiting for a build script. The main project(BDDAndroid) will contain an apectj setup and the dependency injection setup. The instrumented test project(BDDAndroidTest) will contain the setup for robotium/android-mock, dependency injection and fest-assertions and hamcest.

I will be adding the LunarLander example to use for testing thus if you follow the commit sequence of both projects you have the general sequence to get fast BDD and prototype iterations. For those new to Eclipse 3.7 and github, you should be creating your projects outside the workspace, ie home/yourname/git/github and than check the store git in project folder so that you do not get an extra project folder when remote pushing to github on a project you have created.  As I progress I will be adding hooks in my customized build script to trigger monkeyrunner to run the tests so as we can get the screenshots and regression testing.

Because I will be committing the customized build script to the same projects from first commit you will be able to see how to customize your own build-script.

I have figured out howto run cucumber-jvm and robotium, etc the same way as LessPain. I just, now, have to write the android application build.xml to enable it.

It is not the really the speed of the test execution that matters, like robolectric(pivotal) claims. Its can you speed up the test writing by making it easier to write the test code. I think that combination combined with more reliable tests in the realm of BDD instrumented will be enough to always choose robotium over robolectric.

Some are still not confuse so lets cover this point again. In mobile we have a virtual machine meaning that we cannot predict the behavior of member variables that have been exposed. in other words its the underlying reason why we came up with interface patterns in Java. That is different from say ObjectiveC iOS programming. That is the uniqueness of developing on a computer language environment that happens o use a Virtual Machine.

So in performance constrained environments where we are relying less on interface patterns what testing do we have to do to verify behavior since we are not normally mapping it too well through our architecture design? Exactly, BDD or functional unit testing which directly implies instrumented testing because that is the only way to measure the behavior on such a test in the first place.

Developers who advocate TDD unit testing in a JVM in a VM computer language that is different from running the same application through BDD unit testing on an emulator or device are hiding a choice they are fostering on you to appear cool when inn actual reality they expose themselves to deficiency in understanding what a mobile VM does why certain architecture patterns arose within java. That is a very bad choice to foster on someone in name of speed to replace proficiency in java development in a mobile constrained environment.


Acceptance Testing for Android

Part of BDD testing is of course acceptance testing. And of course you want those test classes generated automatically from some feature spec in non programming language. Someone has put together a dirty quick proof of concept  of using android driver and cucumber-jvm to execute cucumber tests at github. Its using maven so if using ant you will have to convert it to ant. But it looks interesting

Robolectric and similar projects have always complained that instrumented unit testing(yes, you can do both tdd an bdd instrumented testing in android development as the plain old unit test classes in the android framework are not loading the tested entity within the android framework system. if you doubt me look at the difference between ActivityInsgtrumentationTestCase2 and ActivityUnitTestCase classes. In Android all TDD unit style test classes have Unit in the class name.) takes more time.

Specifically, its the time to dex the classes and to start the emulator and install the application as when in in TDD mode you are writing the tests first before your application code. Is that that much difference between writing several unit tests for one class waiting more for those tests to finish in instrumented test,mode on the emulator or writing some unit tests for a class running the tests on the emulator while you write more unit tests for another class.

But design shops seem to have bought into less time is more impressive results even though as the android framework matures the results for JVM unit testing it get less and less effective as you cannot predict the behavior of member variables that have been exposed. But, you could speed up unit test writing.

You could use hamcrest matchers and you could build the test-assertions-android github project to get android style fest assertions for the android environment which would act not only to speed up unit test writing but would allow you be able to go back to several unit test results after they are executed and understand them as they are closer to plain English than a regular android unit test with those libraries not applied to a unit test.

This not the only concern however in testing. In Android Instrumented Testing no matter if run it for the IDE or the ant script or from a monkeyrunner python script you have three testing areas; Unit tests, Functional tests, and Performance tests. now lets look back at android sdk build script. Notice in the test-helper-runner macro you do not see an value for running junit runner in multi-file mode. What does multi-file mode do? Well if you use an alternative android -junit-runner with multi-file mode enabled than you test results get grouped by testsuite thus than you have them grouped by; Unit tests, Functional tests, Performance tests.

This is why as of this moment I am rewriting the build scripts supplied with the android sdk as to fully get the benefits of being able to run both TDD and BDD in android instrumented fashion in the IDE those ant script changes will than be applied in setting my IDE java/android builder to use the ant script. And due to the fact that I will have a dashboard generated with the docs during the debug/compile target execution containing such things as Checkstyle, PMD, JavaNCSS, class, and jdepend reports it will be somewhat more effective in getting a snapshot view of the project each time I change code.

Than we have the aspect of some areas are not covered well, regression testing and acceptance testing. Regression testing could be enabled by using monkeyrunner as it has  a user/UI recorder. Now, we do have android-guitar framework which runs on to of monkeyrunner which gives us an event model report but it does not support swipes and some other UI interactions. So we have either extend android guitar or monkeyrunner. Actually that is a nop as monkeyrunner accepts plugins that are written in java and thus the plugins you add to extend monkeyrunner could actually also extend android guitar. Thus, that is another piece I have to add to the testing part of the android app build script, ie the ability to execute the monkeyrunner scripts as part of the instrumented testing developer workflow. Also remember monkeyrunner is an automation tool that is integrated with the android framework api which means anything in the instrumented testing process that can be automated is now reachable to be automated. For example, you could turn off the wakeful lock, etc as part of the process of starting the device or emulator in your instrumented testing process.

This is generally why to rescue an android project you have to start-over the 3rd time, at least in applying agile as you need to change the way tools are integrated together in the developer process and the work-flow and of course the culture. Its not can I buy you coffee and you consult for an hour and the project is rescued type of situation. And my thinking is if you want and value the benefits of full agile, not just TDD and BDD but the change in culture which desire upfront feedback(ie transparency), than you have to be willing to put in the time and costs to change it. Are you willing as company or firm willing to hire a developer to change it? think it about another way.

At one point in internet history we were using Gopher and it basically sucked. At time I watched as people(developers) came up with a better way and people started using it and it was extremely beautiful in its simplicity and implementation and execution. That is the joy of solving hard development problems.

Agile in Android

I guess I should explain this. Lets dig into the android framework first.

The Android Activity is something that the framework creates as you are not in complete control in creating it. That is somewhat important as you normally cannot prepare aspect stuff ahead of time to be created thus there are three other ways to be able to get a hold of it. IoC in an object registry pattern using interfaces(ever wonder why games are so big..there is your hint), some way to mock things or dependency injection or both mock and dependency injection.

This means that agile in android application development ha some differences when compared to other environments where you had no memory/storage constraints and could do as many interfaces as you pleased. Thus we have two areas where we can impact development project times. One is testing where we can use both mocks and dependency injection to in a fast manner implement functional testing and the other area is in application development itself by using dependency injection.

This is not to say that TDD is not agile but, and this is extremely important, TDD does not test behavior in of itself. In other words, yes Robolectric can do functional testing if you are using the advanced features but remember you are testing that behavior in non-alike sandbox as a normal JVM application does not take control of certain object/class cobinations like the android framework does ith the activity class.

Now lets backup a second, what about static classes. Normally you wrap a static class in an interface and test the interface. That is why see several mock frameworks used in android application testing such as Android-Mock. Robolectric does something different called shadow classes. The drawback to those shadow classes is that they DO NOT PREDICT THE BEHAVIOR OF MEMBER VARIABLES THAT ARE EXPOSED!

In the android framework what is the most common area where member variables are found? Hint its the GUI. Where is the common area where those fields are exposed to developers using them/ Both the GUI area AND the INDIVIDUAL FRAMEWORKS THAT MAKE UP THE FEATURES of what an application can accomplish on android. this is why when someone claims that robolectric can test behavior of an android application in a reliable fashion, that they may not know what the eff they are talking about.

Its kind of important to dig into the underlying java and computer science to understand the trade-offs to each agile approach. I personally know of the re-factor work that was completed from android 2.0 to android 3.0 and android 4.0 to expose most of the member variables in the android GUI framework and thus as someone relies upon robolectric going forward they are in actuality testing less and less of actual android application behavior not more.

This is why you find two things in the android framework , some basic mock classes and instrumented testing. Both of which were not in android 1.0 and were added later due to the concerns described in previous paragraphs. Yes, it takes a little more time than TDD/Robolectric but the results when implemented correctly are more reliable