distributedlife

Apologies to readers for this but to claim my blog on Technorati I need to have a special string inserted into a post in my RSS feed.

That string is: KCKNYFRKKDYQ

I’ll delete this post after it has all been said and done.

I hope your Sunday is relaxing.

You may notice when you arrive here for the first time this week an advertisement for the Great Australian Internet Blackout. I’m against internet censorship and I thought I would talk about the compexity of testing the proposed system.

There are many aspects to consider; let me briefly outline the topics and I’ll handle them one at a time: functionality, performance, scalability, non-repudiation, integrity, confidentiality, user experience and workflow.

From a functional perspective the most important thing is whether or not it stops people in Australia access banned websites. This is done at the Internet Service Provider (ISP). All of your internet request have to go through their router before heading out into the ether. This already happens and the blacklist filter system already exists in routers.

Achieving a working filter is easy. Is the domain requested on the list; if yes then reject. However, If I make a request to a server that is not on the list and that server requests the banned site on my behalf me and passes the content back to me then it won’t get blocked. This is known as proxy server. This wikipedia entry has more information about proxies. This test should be failed.

If you consider the workflows involved in this system you have adding and removing sites from the list; propegating the lists to the ISP and validating a user request. Once a site is added or removed from the list is sent to all ISP’s or all ISP’s refer to the latest master.

It could be that there is only one list and everyone looks at it. This ensures consistency as there is only one source of information but impacts performance and scalability; both of which I will cover later. If each ISP has there own copy of the list then you have a delay between when the master list is updated and the ISP copy is updated. During this time we can have users being able to access banned sites dependant on their ISP.

This delay could be acceptable. The ISP’s may have 48 hours to apply a change and if a few kids look up something they shouldn’t we deal with it. A partial success is better than no success here.

What about sites that are legitimate that get banned or are incorrectly placed on the list. It could be a typo. Updating the list is a manual process; someone has to type the domain in. What if that was your business and you got banned. Would you know? How much business would you lose? What checks are in place to ensure that the correct domain is being banned. Adding beaurocracy will increase the chance of correctness but will slowly down the ability to change.

The banned list needs to be kept confidential (for our own good) but the more parties involved in a process the more chance it can go public. Do the necessary people that have access to the list at each ISP need a specific security classification? Will this impact their ability to hire new people? Is this person open to social engineering techniques that may lead them to give up the list.

What if the list becomes public? There is a ridiculous number of domains out there but not an infinite amount. A group of like minded individuals could easily write a web crawler to check each domain to see whether it was banned. with cloud and crowd-sourcing it would still take a while but it could be done.

What then?

Does the Australian Government log who attempts to access content which has been “refused classification” or otherwise deemed inappropriate for Australians? Can this be used against someone? If I force someone to redirect to or unknowingly link to a banned site, can it be used against them?

It’s illegal for a company to track your web presence beyond their own domain but what about the Government? What if they track each and every banned domain you visit? The police could come around and arrest you without warrant because your computer has a virus that is making requests to banned terrorism websites.

Hilarity ensues.

The performance of the system needs to be considered as well. Each and every request made by an Australian computer needs to be checked against a list. If you go to a website and it has analytics and maybe some ads to be displayed and the website pictures are stored on a different sub domain you will have one request for each and every item. All of these requests will need to be validated. This takes time, not much time but when you have millions of users making many more thousands of requests then the chance of quality decline increases.

You can’t replicate production to test performance the impacts. All you can do is use a small subset and do a trend analysis up to all of Australia. This doesn’t work as easily as you think. Hardware components have throughput thresholds and their performance may be bottlenecked by another component.

As a complete tester you need to know about all these aspects and know how to test them. Testing takes time and with a project this large you can’t skimp on time. However each day you test changes the environment as more people get online and computers get faster. The combination of manual work flows and software systems increases the complexity of testing. Work-flow testing needs to be run in simulations where operatives handle requests and respond accordingly.

All of this costs money and for a project that failed the second test. It is at this point the test lead for the project should stand up and say. This will not work in it’s current form.

Whether it is right or wrong is a separate issue. If you want information on that should visit http://www.nocleanfeed.com

One of the things that irks me about HP Service Test is when you use it “their way”, it is difficult to change a reference to the service endpoint.

From the Manage Services dialogue you can see each service you’ve included. For non-integration tests this will be one. If you’ve written a service test the way HP prefer (i.e. via the Add Service Call Button) you can’t delete the service.

For example if I try to delete this service:

I get the following error message:

Are they serious? Delete all calls to the service and try again? Who the hell are they kidding? If you spent three weeks writing and executing service tests and have to change service endpoints because the server died you have to delete each and every service call.

That is stupid. There is also a smarter way to do it.

You can do a “find and replace” and change the service name to something like ServiceName1. Delete the end point, add the new location in and then rename ServiceName1 back to the original name.

If you follow my advice on Improving HP Service Tests then you’re life has been made easier. The “delete service” check doesn’t nothing more than a regex on that service name in an web_service_call. As you’ve parameterised each service name to “{Service}” it finds nothing and you are free to change endpoints whenever you like.

Just like you should be able to.

Today I’m going to look at the first step in making tests written with HP Service Test more reusable, reliable, less fragile and closer to how a test should. When service testing you need to be able to spend more time writing code that actively tests the system rather than support code.

The first step involves making good use of Service Tests’ parameterisation. When you start writing service tests you are going to have a lot of the following code lying around:

web_service_call
(
    "StepName=Do something",
    "SOAPMethod=MyService|MyServiceBinding|MyOperationName",
    "ResponseParam=response",
    "Service=MyService",
    "ExpectedResponse=AnySoap",
    BEGIN_ARGUMENTS,
    END_ARGUMENTS,
    BEGIN_RESULT,
    END_RESULT,
    LAST
);

It won’t take long before this code become tedious to write. If you have been using Service Test for a while then you also know this can become a maintenance problem.

If you have multiple end points that you want to run the same test against then you have to change the binding in half the tests. If you want to run the same test against a different method or a different service you will require additional changes to the service and the operation.

The first step is define some constants in your vuser_init function:

lr_save_string (“MyService”, “Service”) ;
lr_save_string (“MyServiceBinding”, “Binding”) ;
lr_save_string (“MyOperationName”, “Method”) ;

With that in place you can change your method calls to something more like this:

web_service_call
(
    "StepName={Service}::{Method}",
    "SOAPMethod={Service}|{Binding}|{Method}",
    "ResponseParam=response",
    "Service={Service}",
    "ExpectedResponse=AnySoap",
    BEGIN_ARGUMENTS,
    END_ARGUMENTS,
    BEGIN_RESULT,
    END_RESULT,
    LAST
);

The changes are on the first four lines of the function call.

You get the following immediate benefits:

• Multiple bindings only involve changing the binding property and you’ve doubled your tests with no additional work load
• Running the same test against a different operation only involves changing the method property
• Running the same test against a different service once again only involves changing the service property
• You can now remove service end point configurations without a complicated find and remove process. I’ll demonstrate this tomorrow.

When you start testing against lots of services you see the same tests again and again. Now you can make use some cut and paste inheritance and duplicate your tests without too much additional work.

In Part 2 I’ll show you how to reduce the code you write by a little bit more by testing only the focus of the test.

The other day I ranted and raved about a bug in Canberra Cabs online booking system that would have been found if they hired just one tester. Today I rant and rave about my hilarious attempt to report the bug to Canberra Cabs.

I had some lovely screenshots to send them. Evidence to prove it happened. But they don’t have an email address; just a contact form.

So I typed in my problem and included a link to my blog post so they could get my screenshot. I hit submit and what awesomeness did I experience?

This awesomeness.

I like the spelling mistake in the url. I especially like the double bang with space in between. They are serious about the message.

I can see the Canberra Cabs team sitting around the office congratulating 108 Digital for a fine website. No complaints reported to date… or ever.

This is a production website and I’ve tried to do three things so far and two of them have resulted in bugs. Now, this bug probably doesn’t appear too often. Even developers run the positive path through an application. I suspect there was either too much data or they reject any field that contains a hyperlink. It could be SPAM!

More hilarity came when there was a power outage in my suburb and Firefox restarted back to that page. This time with no POST data.

Double bang again. Love it.

Seriously though, this little website scores close to zero in the usability stakes. If a contact form requires mandatory fields then indicate them on screen so the user knows; if the user gets it wrong use client side validation to provide contextual help about how to get it right. If the server gets incorrect data return to the UI with the same contextual help.

None of this is new information and that is what irks me the most.

Testing and usability are not new nor are they mysterious hieroglyphics that cannot be comprehended. They just have to be done and if you don’t know how you hire one. They tend to cost less than every other field in the industry. If you don’t you end up looking a like an idiot and your customers go elsewhere.

Going to production without testing: Do NOT try that! !