TESTING GEODETIC SOFTWARE

Michał Bednarczyk
Institute of Geodesy, University of Warmia and Mazury in Olsztyn, Poland

ABSTRACT

Software quality depends primarily on a properly planned and correctly carried out development process. Testing, which has an enormous influence on the whole project, is one phase of this process. This is the last step of production and includes system implementation as well. Specialist software, which without a doubt geodetic applications are, requires special attention to work quality. It should be ensured already at the design phase, when individual functions are created according to requirements specification. But often the product can not be properly evaluated before the end of the production process. Some deficiencies and shortcomings (resulting from erroneous coding, incorrect interpretation of user requirements or design flaws) become apparent only after the end of the production process. The only way to minimize the number of these errors is the participation of geodesy experts in the testing phase. They can support their knowledge and experience with specialist software, which allows to shorten the time and decrease the cost of this time-consuming process.

This paper is an attempt to address the cooperation of geodesy and information technology in the testing stage of geodetic software development.

Key words: geodesy software, software testing, automated testing.

INTRODUCTION

Software testing is a demanding challenge, as testers bear heavy responsibility for the quality of the product. However, despite the importance of this phase in software life cycle* it is often treated superficially or reduced to the necessary minimum.

Professional geodetic software can be functionally very complex. Its operation is crucial for geodesists´ work as well as for numerous institutions which keep geodetic documentation used in different areas of economy. Therefore, special emphasis should be put on ensuring reliability of this software. A disregard for reliability may later expose the user to numerous problems, such as: data loss, wrong calculation results or the generation of documentation which does not comply with the law.

Moreover, testing consumes a considerable amount of time and work. This generates additional costs, but is worthwhile, because the better the design the fewer corrections have to be made after its implementation. Making corrections after the implementation is more inconvenient and more expensive both for the author and the user. It may lead to a situation in which the aim of corrections is to make the program run at all instead of making it more functional or error-proof.

TESTING - TASKS AND AIMS

Theoretical approach to geodetic software testing should stem from generally accepted principles of software engineering. Testers use a specific terminology when describing basic testing concepts and procedures.

Testing has two basic aims [5]:

• to detect and remove system errors,

• to evaluate system reliability.

Here one should note the difference between validation (i.e. testing the system´s compliance with the actual users´ needs) and verification (i.e. testing the system´s compliance with the defined requirements). One should also be aware of the contrast between the following two commonly used concepts:

• error or fault - part of the application´s code which may result in incorrect performance [5], [3],

• failure - incorrect (not compliant with the specifications) performance of the application [5], [3].

An error may result in numerous failures while a failure may result from a number of different errors.

There are many testing methods, each of which allows to detect different kind of errors in the application. In a very complex system it is virtually impossible to detect all errors. Besides, the concept of error can be relative. What one user considers an error, another may find completely acceptable. With a proper approach however, a version satisfactory for both the authors and users can be produced.

The basic question to ask is: what does one expect from testing? The following classification of tests with respect to their aim may be adopted:

• error detection - with the primary task of detecting the greatest number of errors in the application [5],

• statistical tests - allow to find out reasons for most common failures and to evaluate system reliability [5].

The following definition narrows the concept further down: "the aim of software testing is to look for failures in such a way that a failure while testing allows to find and eliminate the errors responsible for this failure" [3]. In practice, the tester should find the error, discover (if possible) its source and suggest possible solutions. It sometimes happens that the tester can not determine the source of an error, in that case he or she should produce the best possible report (describe the input, executed functions and the error or failure itself).

Based on their technology, tests can (also) be categorized into:

• dynamic tests - involve running the application or its part and comparing the results with correct results,

• static tests - involve code analysis (so called informal methods - reading parts of the code and "running" the application mentally, or formal methods - formulation of correctness proofs).

In the case of a large computer system (e.g. geodesic documentation centre system including databases, graphical datasets and electronic document flow) its testing will (in essence) comprise the following stages:

• Module testing [5] (unit testing [3]) - carried out during the implementation, directly after a given module is completed. If the module under test is dependent on other - not yet completed - modules, so called stubs and test harnesses are used to simulate the behavior of nonexistent modules. The tested unit can then function as if it called and received output from other system parts. This makes a certain automation of the test possible by making a program for many different input values and call parameters. Module testing is "virtual" in the sense that it does not allow to detect errors resulting from the interaction between modules.

• System tests - run during the integration of individual modules. Both subsystems and the whole system are tested. System tests use two methods: bottom-up testing (starting with lowest level systems through higher level systems up to the whole system) and top-down testing (highest level systems are tested first with lower level systems simulated by stubs, then lower level systems are added until the integration of the whole system).

• Acceptance tests - the software is entrusted to users for testing. In the case of commissioned software such tests are called alpha tests. In the case of commercial software, a certain number of copies is distributed to users (free of charge or at a lower price) and such tests are called beta tests.

It can easily be seen that static testing should in principle be carried out solely by a team of programmers. These tests serve the purpose of detecting typical errors occurring in programmers´ work. It also helps if programmers check one another´s work.

On the other hand, dynamic tests need not be carried out by people involved in the programming process, but a sound knowledge of the field in which the application will be used is necessary. Thus, in the case of geodetic software the testing team should include specialists in geodesy and information technology. If both these fields are combined, tests will be much more efficient, which in turn will be reflected in the product´s quality.

Of the dynamic tests focusing on error detection, the following should be mentioned:

• Functional tests (black-box tests) - rely on the knowledge of the application´s functions. The system is considered a "black-box" which executes functions in an unknown way. In this approach the tester may try the application out just as the user would. It can thus be argued that it is crucial for this kind of test to succeed.

Obviously, in most cases it is impossible to provide all potential inputs for each function because there are too many of them. Data is grouped into classes, for which the expected output should be similar. It is assumed that if a function returns a correct result for several typical values from a given class, it will return correct results for other values from this class as well. Of course such a test does not guarantee the reliability of a given function, as there may exist data from the same class which return errors.

It is best to obtain information from the requirements specification or the design, which describe the input for a given function (e.g. "function takes values from 0 to 400" or "name always begins with a capital letter"). The function´s behaviour is then checked for boundary values and for values outside the required range.

Data for functional tests may also be obtained with a "broken box" method - using information about algorithms used for different execution ranges (e.g. for values from 1 to 1000 the function uses one algorithm and another for values over 1000). This allows to avoid a random choice of values and makes the data more specific in that not only boundary values but additional information is taken into consideration. Boundary values of functions and of algorithms often cause errors, due to e.g. various conditions and limitations they impose and which should be considered when writing the program.

A program is often so complex that a thorough testing all of its functions would be too time consuming or just impossible. In such cases one has to carefully choose the functions to be tested, for example according to the following criteria [5]:

• Structural tests (glass-box tests, white-box tests) - rely on the knowledge of the program´s code. Data for tests is chosen based on the analysis of the program. The following criteria for the choice of data can be applied:

Criteria can also involve nodes or branches of a control flow graph (a representation of statement paths) [3].

Structural - code based - tests do not give a full picture of system´s reliability. Users execute functions not code statements. Relying solely on structural tests may lead to a scenario in which testing and execution of all statements a given function contains will not execute the function and so it will not be tested.

TEST PLANNING

Testing procedure must be properly prepared. One should avoid situations in which testing begins from scratch with the program´s code already completed. By then many problems which had arisen during production were forgotten and it will be impossible to handle the whole program while maintaining the focus on its potential weaknesses. The only proper way is to prepare a testing plan beforehand and conduct the testing procedure according to that plan. It is best if such a plan is made in the design and production phases, and based on requirements specification. This allows to carefully monitor individual functions, discover future testing areas to focus on and plan optimal test cases (which will detect errors). If no testing plan has been made in the development process, one should consult the documentation of the preceding development phases to create such a plan.

The program´s code may also be the source of test cases, but it is not advisable to start from this point. When testing the code one should check if the algorithms it contains are constructed correctly and if its functions run properly. It often happens however that an error results from an omission, despite the fact that according to the requirements specification no such omission is expected. Such a missing function can not be detected with a test based on the code, because one can not then determine if the function in question is necessary. Therefore it is best to rely on the project documentation and the requirements specification, because they contain all the functions the program is expected to execute (functional tests). Structural tests are useful for the verification of functional tests. Having ensured that the program executes all of its designed functions one can check its "technical side" by "forcing" it to execute all loops and conditions contained in the code´s procedures and functions.

This approach is correct from users´ point of view, as for them it is important whether the program executes all necessary functions, while knowledge about algorithms behind them is irrelevant. Potential errors contained in the code may never surface, because the user may not create the necessary situation. Function errors will become evident as soon as the program is used for its purpose.

TEST AUTOMATION

There are many testing tools, which can be divided into invasive and non-invasive [6]. The tools compiled into the code for the time of testing are invasive. Of course this changes the program´s environment and may cause additional errors. Among such tools are [6]:

• analyzers and monitors,

• drivers ,

• stubs,

• stress testing tools,

• disturbance generators.

Such programs make it possible to generate special conditions in which the software under test is to be run, as well as allow to monitor its work and identify executed statements (analyzers and monitors).

Testing is a cyclic process however. After each instance of error detection and entering corrections the testing procedure should be repeated, sometimes with different test values, depending on the changes made. Moreover, each performed test should be reported. Testers´ work can be made easier here, owing to automated testing tools [6]. There are numerous such tools on the market, both free and commercial, e.g. Macro Magic and more advanced Quick Test Professional, Rational Visual Studio or Rational Robot. They are a type of drivers - mentioned above among testing tools - i.e. programs which control the behaviour of the software under test.

They give numerous configuration possibilities which facilitate testers´ work, such as:

• recording and replay of individual tests,

• reporting results,

• verifying result correctness,

• taking screenshots after the execution of a sequence,

• adjusting mouse movement to the active window,

• appropriate acceleration or deceleration of the program, etc.

The simplest way to automate the tester´s work is to replay previously recorded keystroke sequences and mouse movements. A replay is sequential though and it is difficult to ensure proper test execution if e.g. a window opens too slowly or appears in a different place.

Despite its popularity this approach has a serious drawback: if the interface has been changed (which often takes place during development) the recorded test becomes useless.

Another method to automate testing is test creation by coding with the use of e.g. macros instead of "live" recording. This gives an additional opportunity e.g. to make the program wait for a window to appear or to display a message with a result, for confirmation.

The most interesting and advanced automation method is to use macros combined with a programming language to construct test sequences. This gives the tester a relatively good control over the test. For example, one can make the test tool click a button and not need to provide (or record) its position on the screen - as it is the case with the previously described methods. In a similar way, the program can "find" a specific menu option or a checkbox. Therefore, this test automation method is more reliable.

The most important feature of programs utilizing this method is the possibility to verify the correctness of a result and to verify the correct behaviour of the program under test. One may for instance [6]:

• Capture the data from the screen - during another execution of the test the program is able to compare the recorded information with the data present on the screen and detect a potential error.

• Verify values - instead of taking whole screenshots the program may only take the values of individual controls (e.g. edits, memos, or checkboxes) and compare them with reference values. One can also check if a button has been clicked on or a checkbox ticked.

• Verify input - if the program under test returns the data to a file, the testing tool may compare this file with a reference file. The same can be done if data is sent via a network.

Furthermore, the tool can provide a full report on conducted tests, which can be included in the documentation.

HIT-AND-MISS TEST AUTOMATION

Another testing method can be used, which attempts to simulate users´ behaviour. It is sometimes referred to as a "testing monkey" [6]. In the simplest case this testing method uses a small program which generates random keystrokes, clicks and mouse moves. This enters random data and allows to detect errors which may arise at unexpected moments during the execution of the application under test. Such testing tools, which report executed actions and encountered errors and which recognize the functions available at a given moment during a test, can perform a great number of operations per second. This both saves time and relieves the tester from the duty of looking for atypical inputs and actions, which in absence of such an automated tester might be altogether abandoned.

Automated testing, despite its advantages, should be applied rationally and only as a support of a proper testing procedure. A number of factors, which accompany any programming project, contribute to this state of affairs. First of all, the tested application undergoes constant changes, new functions are frequently added, and tests constructed earlier are unable to handle them. Man´s intuition and knowledge have a great advantage over the most complex test programs. Therefore, one should not rely solely on automated tests, because if they detect no errors, one can almost be certain that there are other errors, not recognized by the program, the existence of which it will never infer. Besides, attempts to create perfect test scripts may consume too much time and there may be too little time left for the testing itself. In the case of invasive tools, it has to be taken into consideration that a detected error may result from the interaction between the testing program and the program under test. For this reason, if an error occurs during automated testing it is a good idea to execute the same error manually in order to avoid such a scenario.

ROLE OF A GEODESIST IN TESTING

Geodesy is a very broad field and makes use of computer software in practically all of its aspects: calculations, mapping, spatial information systems, photogrammetry and producing documentation. Due to the complexity of tasks this software is facing, legal considerations, instructions, technical guidelines and the great importance for numerous institutions and organizations in many branches of economy, ensuring adequate quality requires a considerable effort.

Testing phase plays an important role in software development process and greatly affects the quality. It is also the most time-consuming phase - it may take up to 40% of the whole development process [3]. The participation of geodesists in such a project is therefore advisable. Their knowledge - preferably supported with IT knowledge and programming skill - allows to test the program more thoroughly and save time. Test cases created by such people will be much more precise and effective, resulting in a more satisfied user and often simply in avoidance of grave errors.

An average geodesy application - depending on its purpose - may comprise many parts, each of a different character, e.g.:

• calculation module,

• graphical module - vectors, rasters, transformations,

• reporting module,

• database.

Their functionality, especially in the case of the first three types should be tested by people with a proper training. They can test correctness and user friendliness, but will also be able to point out functions which are missing in the requirements specification but which should be implemented in order for the program to serve its purpose. They can help make program-generated reports compatible with current standards and regulations or advise on the program´s interface from the point of view of a user-geodesist.

Relying only on IT specialists one runs the risk of a misinterpretation of user requirements, which entails wasting time on additional consultations, and the possibility of serious functional drawbacks even if the program works perfectly.

Testers bear a heavy responsibility for the quality of the system, so it pays to entrust it to properly trained specialists.

CONCLUSIONS

Testing geodetic applications should make use of well-known and generally approved methods, applied with success in other fields. Professional geodesists, preferably with IT knowledge and programming skills will be excellent testers. Their efficient cooperation with a team of IT specialists should save a lot of time and ensure better product quality.

The increasing importance of testing in contemporary software development should again be emphasized. Contemporary geodetic software is extremely complex and constitutes an enormous leap forward compared with simple programs available only several years ago. Ensuring quality and functionality without adequately chosen tests becomes - slowly but inevitably - no longer possible.

REFERENCES

1. Bowne Global Solutions. 2004. Twój produkt powinni sprawdzać inni. Outsourcing w testowaniu produktów informatycznych [Your Product Should be Checked by Others. Outsourcing in Software Testing]. (website: http://www.bowneglobal.pl) [in Polish]

2. Gryciuk W. Testowanie oprogramowania - niechciane dziecko technologii informatycznych [Software Testing - Unwanted Child of Information Technology]. Computerworld 4/1997 [in Polish].

3. Hamlet D., Maybee J. 2003. Inżynieria oprogramowania, podstawy techniczne inżynierii oprogramowania [The Engineering of Software, Technical Foundations for the Invividual]., Wydawnictwa Naukowo-Techniczne, Warszawa [in Polish].

4. Janczewski M. 2004. Abecadło testowania [Abc of Testing]. (Stowarzyszenie Jakości Systemów Informatycznych website: http://www.sjsi.org) [in Polish].

5. Jaszkiewicz A. 1997. Inżynieria oprogramowania [The Engineering of Software]. HELION, Gliwice [in Polish].

6. Patton R. 2002. Testowanie oprogramowania [Software Testing]. MIKOM, Warszawa [in Polish].

7. Stapp L. Extreme Testing. Tester.pl 1/2004.

8. Wysocka E. Przygotowania do certyfikowania oprogramowania stosowanego w geodezji i kartografii [Preparing for Certification of Software Used in Geodesy and Cartography]. Materiały VIII Konferencji "Jakość, standaryzacja, normalizacja w geodezji i kartografii" [VIII Conference "Quality, Standardization, Normalization in Geodesy and Cartography"]. Pogorzelica, 3-5th october 2002 [in Polish].

FOOTNOTES

*There are many software lifecycle models (e.g. waterfall, spiral), which characterize software lifecycle phases, describe the actions performed in these phases and their order. They allow to formalize the process of software development: from requirements analysis through design, construction, testing, implementation, maintenance and upgrade to retirement.

Responses to this article, comments are invited and should be submitted within three months of the publication of the article. If accepted for publication, they will be published in the chapter headed 'Discussions' and hyperlinked to the article.