Sunday, October 18, 2009

The Kind of Tests We Don't Like

We have a midterm coming up on Wednesday, October 21st. Our assignment this week is to come up with some questions for the exam. Here are my 10 questions.

A Java class called "MyDate" has a method "getDay" that takes an integer and returns a string representation of the day of the week (e.g. getDay(0) = "Sunday", getDay(5) = "Friday"). If the day number is not valid, "getDay" returns null.

1. What are the equivalence partitions for "getDay"?

There are 3 partitions for this method. Let dayNum be the variable that is passed in to getDay(). Then the partitions are (a) dayNum <> 6.

2. Write a unit test for "getDay".

/**
* Tests the getDay method.
*/
@Test
public void testGetDay() {
  
int dayNum = -1;
  
assertNull("Testing with dayNum < 0", getDay(dayNum));
  
dayNum = 0;
  
assertEquals("Testing valid day", getDay(dayNum), "Sunday");
  
dayNum = 7;
  
assertNull("Testing with dayNum > 0", getDay(dayNum));
}


3. According to Richard Gabriel in "The Poetry of Programming", how is writing poetry similar to writing code?
Richard Gabriel states that his mind is in a "particular place" when he is writing either code or poetry. He is thinking about the possibilities and directions as he is writing.

4. Bob has some code in a Subversion code repository. What svn commands would I use to (a) get a fresh copy of his code, (b) make sure I have the latest revision, and (c) submit my code changes?

(a) "checkout", (b) "update" (or "status --show-updates"), and (c) "commit".

5. Name a reason why you might want to use a Java collections class instead of an array class.

Fairly open ended question. One good reason is that you might need a set or a hash table. These are implemented as collections.

6. What is Linus' Law and how does it apply to the "bazaar" style of development?

Linus' Law is "Given enough eyeballs, all bugs are shallow". This applies to the bazaar style of development because problems will be found quickly and the fix may be obvious to someone in the bazaar.

7. Why is measuring code quality by test coverage a bad idea?

One reason is that the test coverage report does not show if all possible paths through the code are taken.

Questions 8-10 deal with the following implementation of onScannedRobot:

/**
* Event that is thrown when an enemy is detected.
*/
public void onScannedRobot(ScannedRobotEvent event) {
  
String currentEnemy = null;
  
double bearing = event.getBearing();
  
if(event.getName() == currentEnemy) fire(3);
}


8. Name a violation that would be found by Checkstyle.

There are two issues (as far as I know). The obvious one is that the if statement is inline and not a block statement. The other is that there is no @param tag for event.

9. Name a violation that would be found by PMD.

PMD would notice that bearing is set but never used. It might also catch an error found by Checkstyle.

10. Name a violation that would be found by Findbugs.

Findbugs would find the NullPointerException that this code will probably throw when it is run.

Tuesday, October 13, 2009

Safety Net

It's hard to imagine how people got by without configuration management systems. Without them, an accidental commit to a code repository could blow the entire project. At the very least, it would take more time to go and fix this mistake. And the programmer who committed the bug might be looking for a new job.

I'm a fan of configuration management systems like Subversion and Git. In a team environment, it seems to be a necessity. But it can even be useful for individuals like ICS students working on assignments and projects. We've all introduced bugs in a program and had to spend time fixing them. Students can use the repository to make sure they always have a stable copy of their code to go back to if they mess something up badly. It's akin to how some computer games allow you to press a key and save your progress at any time. With that safety net, players will often save after every major event so that they don't have to go back and do it again. You can take it to an extreme where you save after every enemy you defeat. After all, you're still alive, so you must've done something right. It would be interesting to see if this kind of behavior would emerge if we didn't teach students good "commit etiquette". Would the students commit their work consistently or just not use the service at all?

In class, we had an introduction into Google Project Hosting and Subversion. Google Project Hosting is a great service for open source projects. It provides a central place for committing and managing code without having someone create their own server. But it also has features like issue tracking, bug reporting, and wiki pages. Best of all, it's free as long as your project is open source.

I set up a project for my Robocode robot Menehune. It was fairly easy to create a project, add wiki pages, and commit my code. I also added two classmates as committers to the project. This does mean they can sabotage my robot, but at least Subversion lets me roll back changes. I was also able to create a Google Group for the project. To create the group, I had to go to groups.google.com and create it there, then add it to the project. I was surprised that there wasn't a link on the project management page to create a new group, but I figured it out. There was a feature where we could receive message postings and emails when someone commits code to the project through Google Groups. Apparently, they're rolling out a new way to do this, so we aren't able to add this feature for the time being.

The Menehune Project page

Overall, we had a brief introduction into configuration management systems and Google Project Hosting. The main project in our class is coming up, so I think we'll become much more familiar with both of these things in the next few weeks.

Tuesday, October 6, 2009

I Love Tests

I don't think you usually hear a college student say that. I love tests. But ask any software developer and they'll probably give you the same response. Well-written tests not only makes sure that your code is working well, but that it'll be working well in the future. And there's Test Driven Development (TDD), where we write a failing test first and then write the code needed to pass the test. I had the opportunity to hear Kent Beck (co-creator of JUnit) speak about TDD and JUnit. He mentioned that TDD seemed like such a silly idea; that you write code that doesn't work and then have to write more code just to fix it. Yet they used TDD when creating JUnit and found that they were more productive. If you're not a believer, maybe you need to take Software Engineering at UH Manoa.

I always liked tests, but relearning about them in ICS 613 reignited my passion. So much so that I wrote a few unit tests for my iPhone application (Apple calls them "logic tests") and did some TDD to add a new feature. So I dove into writing a few tests for my Robocode project. I had ideas on what tests I could run, but as I said in my previous blog, things rarely go as planned.

I had the idea that I would create unit tests that test the event handlers for my Menehune robot (OnScannedRobot and OnHitByBullet). I'd just create a bogus event and test that certain properties of Menehune were set. The difficulty was that if we instantiate our Menehune robot, we need to run the run method to access certain properties of the robot. Unfortunately, the run method is typically an infinite loop. In the OnHitByBullet test, I could get by without having any exceptions thrown. As for OnScannedRobot, I did a check that the exception is thrown (the test fails if it isn't thrown) and then check for some internal properties. To do that I had to move my code around so that movement and firing decisions came later.
  public void testOnScannedRobot() {
    
// Initialize an event with bogus values.
    
ScannedRobotEvent event = new ScannedRobotEvent("Foo", 100, 0, 10, 0, 0);
    
try {
      robot.onScannedRobot
(event);
      
fail("Should have thrown exception where we attempted to move or fire.");
    
}
    
catch (RobotException e) {
      assertEquals
("Testing scanned robot name.", event.getName(), robot.getScannedRobotName());
      
assertEquals("Testing robot mode.", Menehune.RobotMode.FIRE_MODE, robot.getMode());
    
}
  }



Example where I fail if an exception is not thrown.

Next, I needed to test my robot's behavior. We were provided with Philip Johnson's RobotTestBed code that provided a test box for our robots. We could check the conditions of the robots after each turn, round, or battle. I had the idea where I'd access methods in my Menehune robot to make sure that it was behaving properly. Yet I was foiled again! We can't actually access instances of our robot. We just have a generic overview (snapshots in their case) of where the robots are at a given time. Thus, to determine if my robot was behaving properly, I tracked its movement when it encountered an enemy. That way I could check that the robot tracked down an enemy and then repositioned itself.

Tests against other enemies went fairly smoothly though. I gave myself a fairly conservative win percentage against RamFire, but I just wanted to make sure that I win most of the time.

I then ran a code coverage tool (Emma) to see how well my tests cover my robot. Overall it was pretty good. I used an enum type to represent the different modes of my robot. Emma got me for not testing all of the methods for an enumerated type, which I think is kind of bogus. My robot also apparently didn't turn in certain directions, so those lines were missed. Finally, I only tested my robot against one enemy, so the line that skips the enemy if we're not tracking it is not covered. Further tests might cover some of these aspects, but I'd say it's pretty good coverage otherwise.

So what if I never called RobotMode.valueOf()?

Overall, I can say that my love for tests are back. If we spend more time developing and improving our robots, I might be inclined to write a few more. I love tests. Exams, not so much.

You can download my robot here.