Web site Usability Evaluation of Internet Banking in Taiwan

By Chin-Shan Wu, Lecture/Wufeng Institute of Technology

Web: http://www.im.wfc.edu.tw
Email: jackwu@mail.wfc.edu.tw

Chin-Shan Wu is currently a Lecture at the Department of Information Management in Wufeng Institute of Technology in Taiwan. He is also a PhD student at the Department of Information Management in National Sun Yat-Sen University. His research interests include Electronic Commerce, Decision Sciences and Banking Information System.

By Fei-Fei Cheng, PhD Student/National Sun Yat-Sen University

Web: http://www.mis.nsysu.edu.tw
Email: fei@mis.nsysu.edu.tw

Fei-Fei Cheng is currently a PhD student at the Department of Information Management in National Sun Yat-Sen University. Her research interests include Electronic Commerce, Consumer Behavior and Decision Sciences.

By Hsin-Hui Lin, Professor/National Sun Yat-Sen University

Web: http://www.mis.nsysu.edu.tw
Email: hhlin@mis.nsysu.edu.tw

Professor Lin holds his Ph.D. degree from Department of Industrial and Manufacturing System Engineering at Iowa State University. He is currently a Professor at the Department of Information Management in National Sun Yat-Sen University. He has been working with Software Reliability, Project Management, and Electronic Commerce.

Visit for more related articles at Journal of Internet Banking and Commerce

Abstract

With the proliferation of the Internet, banks are adopting technologies to build their own Web sites. The design of the Web site is a crucial determinant of whether visitors are likely to return to the site. Thus, the purpose of this study is to investigate the usability of Internet banking Web sites in Taiwan. The samples were classified into 3 groups: the old government-owned banks, the old private banks and the new private banks. The result indicates that there is a gap between user expectation and actual performance of Internet banking Web sites. Although 􀳦􀳦content􀳦􀳦 and 􀳦􀳦ease of use􀳦􀳦 are the most two important categories for Web site users, banks are devoted to promote themselves to entice customers to visit. Overall, the old private banks performed the best among 3 groups of banks, followed by old government-owned banks and new private banks. Conclusions and discussions are provided.

Keywords

Internet banking, Web site usability, heuristic evaluation

Introduction

With regard to the financial history of Taiwan, in 1949, the R.O.C. Central Government evacuated from Chinese Mainland to Taiwan and reorganized all the financial institutions that existed during the Japanese occupation. In the early years after the recovery of Taiwan, 7 commercial banks, 1 cooperative bank, 1 trust company and many regional credit cooperatives were the early players in the local financial market. In the past 30 years, the R.O.C. economy has enjoyed rapid and sustained economic growth, averaging almost 8.29% per annum. To date, the R.O.C is 14th largest trading partner and the 20th largest economic entity in the world. The R.O.C.'s past economic performance in the Taiwan Island of 36,000 square kilometers has been portrayed as a "miracle" in both economics publications and on the occasion of many international forums.

From the 1980s, the force of growing international financial competition started to accelerate the process of financial market modernization in the R.O.C. Hence, the restrictions on interest and foreign exchange rates were totally eliminated. Limitations on the establishment and business scope of financial institutions were significantly relaxed. Cross-border financial operations were expanded rapidly. Government-owned banks were gradually privatized. Furthermore, local OTC-traded and exchangetraded derivatives markets were formulated. In fact, the R.O.C financial market in the past decade has experienced the fastest expansion in its market history.

On the other hand, since the wide distribution of World Wide Web browsers, the Web was estimated to be more than 800 million pages (Lawrence and Giles, 1999) and 1 billion users on 2005 (Margherio, 1998). The growth of electronic commerce as a whole speaks volumes about the potential growth for Internet banking in particular (Cuevas, 1998).

Web technology is transforming all business into information-based activity. The rate of technological change is so rapid that emerging electronic commerce already is making fundamental changes in the economic landscape, affecting every aspect of how business is and will be conducted. In the physical commercial world, the information technology used in a store is most often invisible to the consumer, whereas in e-commerce it has been moved to the foreground (Koufaris, et al., 2001-2002). To the consumer, the Web site, an information system, is a gateway that full represent the store. That is, the design of the Web site is a crucial determinant of whether visitors are likely to return to the site (Klein, 1998) and of consumer satisfaction with Internet shopping (Agarwal & Venkatesh, 2002).

Many organizations jumped on the dot-com bandwagon and implemented online systems. In their rush to deploy Web sites, many e-businesses operated on a build it and they will come mentality. However, most of them built Web sites that were unusable in terms of ease of use, comprehension, reliability, and performance, as the Web Usability for Online Customers sidebar explains (Becker & Berkemeyer, 2002). According to Agarwal & Venkatesh (2002), usability is likely to be a key and proximal metric for evaluating the success of an organizations Web presence. In fact, empirical work by Lohse and Spiller (1999) shows that interface features, such as those assessed during usability testing, explain substantial variance (61%) in sales for online stores.

A survey conducted by Nielsen Norman Group indicated that, even such the most popular E-commerce Web site as Amazon.com still remains improvement in usability. The data is illustrated in Figure 1.

Furthermore,NetValue reveals that Growth in visitors to financial sites in France and Germany has been phenomenal: since January 2000, the number of visitors has grown by 63% in Germany and 88% in France. Moreover, Mihajlo reported that even the top German banks fail their customers with poor site user experiences. He also suggested that commercial banks lead on value, presentation, and reliability, while all banks must focus on improving navigation and customer service. Diniz (1998) presents a survey on Web sites of banks in USA, and indicates that banks have much to improve in their Web sites, and that they are only in the very beginning in terms of functionality.

Although a considerable number of studies have addressed the importance of Web site usability, there are few, if any, specific empirical research in examining the performance of these sites. Specifically, the primary purpose of this study is to evaluate the Web site usability of Internet banking in Taiwan.

Research Methodology

Instrument

Straub (1989) suggests validating instruments through pretesting and/or piloting using previously validated instruments wherever possible. Boudreau et al. (2001) argue that use of existing instruments is primarily for purposes of efficiency, but is still well regarded as a methodological approach. Indeed, utilization of the existing literature as a basis for conceptualizing and specifying constructs supports specification of a set of potentially strong metrics (Churchill, 1979). Thus, prespecified constructs were utilized for the instrument development where possible. In this study, measurement developed by Agarwal and Venkatesh (2002) was adopted to evaluate the Web site usability of Internet Banking in Taiwan.

The instrument includes five categories: content, ease of use, promotion, made-for-the-medium and emotion. Four of the five categories have subcategories that are meant to represent various dimensions of the major category, and these categories are expected to cover the range of usability-related aspects of a Web site.

Samples and Procedures

According to the Banking Law of Taiwan, banks are comprised of commercial banks, specialized banks, local branches of foreign banks and, other banks established in accordance with laws other than the Banking Law. This study focuses on the commercial banks only. Until now, there are 44 commercial banks and 5 business banks in Taiwan. All the Web sites of these banks are evaluated in this study to determine their usability. Moreover,before the establishment of 15 new private commercial banks in 1991, the R.O.C banking sector was dominated by government-owned banks. As a result of increasing competition and government's privatization policy, the year of 1991 has become the turning point for the shifting of market dominance from government-owned banks to private banks. The samples in this study were classified into 3 groups: old government-owned banks, old private banks and new private banks.

Moreover, to ensure the reliability of usability evaluation, 8 Web sites users were recruited as the evaluator. The study was composed of 2 steps: weights and ratings, as developed by Agarwal and Venkatesh (2002). First, each Web site user (evaluator) provides the relative importance (weights) of the different categories in the instument. In this step, users distribute 100 points across the 5 major categories and then further subdivide category allocations among the different subcategories. In the next step, users provide ratings for specific Web sites on various subcategories in the same instrument used in the first step. The weights and ratings together are then used to assess the overall usability for each site.

Analysis and Results

46 out of 49 (93.88%) Internet banking Web sites are evaluated by all the eight evaluators. Those Web sites that data are not valid across all the evaluators are not included in further analysis. According to Straub (1989), an instrument valid in content is one that has drawn representative questions from an universal pool (Cronbach, 1971, Kerlinger 1964). The instrument utilized in this study was composed of 5 categories and four of the five categories have subcategories that are meant to represent various dimensions of the major category. As the result, the instrument could provide greater assurance that the content or domain of the construct (usability) is adequately covered by these dimensions.

If instrument is valid in construct, one can expect relatively high correlations between measures of the same construct using different methods and low correlations between measures of constructs that are expected to differ (Campbell and Fiske, 1959). Thus, in order to assess construct validity, we examine how closely the calculated usability metric for a Web site relates to a multi-item scale of overall usability. The higher the correlation between the two constructs, the more accurately the multidimensional calculated usability measure represents the construct of usability (Agarwal and Venkatesh, 2002). The overall usability was determined by the following three-item scale anchored on extremely poor and extremely good: (a) how do you rate the overall usability of the Web site? (b) how do you rate the overall design of the Web site? and (c) how do you rate your overall experience at the Web site? In this study, there are 32 out of 46 Web sites (69.57 %), the correlation between calculated usability and overall usability were above 0.7 and reached the significance level of 0.05. This result indicates that the instrument was valid in construct. For those Web sites that were not valid in construct are not included in further analysis. Until now, the 49 banks were reduced to 33. The reduction processes are summarized in Table 1.

Furthermore, the reliability of instrument in this study was examined by using Cronbachs alpha. Reliability of each category are illustrated in Table 2. Most of the categories has reliability that were above 0.7, as suggested by Nunnally(1978, p. 245) and Hair et al.(1998).

User Assessment of Weights

Means of the weights of the different categories and subcategories of Internet banking Web sites are shown in Table 3. The result indicates that "content" was the most important category. Look deeper into the subcategories of content, "current information" was weighted the highest among the four subcategories in "content". Moreover, "ease of use" was the secondary important category, of which subcategory "feedback" has the highest weight.

The third important category is "made-for-the-medium", of which "personalization" is the most important subcategory. According to Diniz (1998), Web sites of banks can use customizing resources, besides some subscription option, advertisement or discussion groups. However, very few banks use these resources. Thus, personalization has to be more widely adopted by Web sites.

User Assessment of Usability

Usability rating was calculated using user assessment of weights and rating score provided by users. The calculation was illustrated in Table 4. In order to understand the usability of Web site in different types of banks, we divided the samples into 3 groups according to the date of their foundation. The 3 groups are old government-owned banks, old private banks, and new private banks, as was mentioned in section 2.

The average usability rating of 3 groups of Internet banking Web sites are given in Table 4. The usability scores are on a 10-point scale, with the average of 5. As we can see from Table 5, all the 3 groups of banking Web sites have usability scores about 3 to 6 point. This result is consistent with findings of Forrester Researchs study, which was conducted to assess the usability of Internet Banks Web site in Europe, and the result showed that none of the banks has reached the criteria of all the usability evaluation. These entire phenomenon indicated that the design of Internet banking Web site remains improvement.

Moreover, the overall usability of old banks is better than new banks, and old private banks have higher score than old government-owned banks. The privatization of old government-owned banks in 1991 had boosted the competitiveness of financial markets in Taiwan. As a result, the private banks have to build their own competitive advantage for survival. Meanwhile, most of the old banks were established for more than 10 years, and were experienced with providing information and services to meet customers needs. The performance of Web site usability of 3 different groups of banks in this study reflects the above phenomenon.

Looking into the usability score of different types of banks among each category, most of the banks devoted themselves to promote in order to entice customers to visit. The usability score of promotion of old government-owned, old private and new private banks are 7.49, 7.45 and 6.81 respectively, of which old government-owned banks get the highest score. Furthermore, the calculated usability score of ease-of-use and content are ranked second and third. However, most of the score of the two categories were lower than 5, the average score. According to the result of user assessment of weights discussed in section 3.1, users of Internet banking weight heavily on content and ease-of-use. That is, there is a gap between the user expectation and actual usability of Web sites. Thus, the implication of this result would be that banks should improve their design on content and ease-of-use, rather than promoting themselves, to increase the stickiness of their customers.

In the category of ease of use, old government-owned banks performed the best, followed by old private and new private banks. Most Web sites of banks are good at providing feedback mechanism to interact with their customers. However, they are not able to provide clear goals, especially those for customers of new private banks. As of four sub-categories in content, banks are able to present timely information, but lacks in using such media as pictures, music or multimedia to make the Web site more vivid. Moreover, old private banks had the highest score in content category.

According to the study result of Diniz (1998), very few banks use such resources as customizing the information within the site and discussion groups. The same result can be found in current study that the scores of personalization of Web sites in made-for-the medium across 3 groups of banks are about 4 point. This indicates that customization has to be more widely adopted.

Conclusions and Discussion

This study aims to understand the usability of Internet banking Web sites in Taiwan. In order to utilize user heuristic evaluation method, eight Web site users were recruited as evaluators, and the instrument developed by Agarwal and Venkatesh (2002) was employed. Sample sites includes all of the 52 banks in Taiwan, with 49 Web site are available.

Overall, Web site usability of old private banks is the best among 3 groups of banks, followed by old government-owned banks and new private banks. This indicates that old banks are more experienced than new banks and private banks are more competitive than government-owned banks to survive in a competing market.

Moreover, the result showed that there is a gap between the user expectation and actual usability of Web sites. Specifically, users expected that Internet banking Web sites should provide relevant content, current information, clear structure and appropriate feedback. However, most banks were devoted to promote themselves to create traffic. In fact, if banks could take the content and ease of use into account when designing their Web sites, customers would thus come and stick on it.

According to the study result of Diniz (1998), the sites he visited are not being well targeted and it is not clear if they are designed for client exclusive use or to a broader group of visitors. Our study also indicates that though banks provide feedback mechanism for their customers, goals are not clear enough. Furthermore, we suggest that the use of media such as music, background, and multimedia can be improved and community or discussion groups should be established to increase the interactivity with customers.

Banks have much to improve in their Web sites, and that they are only in the very beginning in terms of usability. Establishing a good Web sites with usability could increase customers satisfaction and intention to return, thus leads to the success of a business. Specifically, content and ease of use are the most two important categories to be concerned in designing Web sites for Internet banking.

References

1. AgarwalRitu and ViswanathVenkatesh, Assessing a Firms Web Presence: A Heuristic
2. Evaluation Procedure for the Measurement of Usability, Information Systems Research (13:2), June 2002, pp. 168-186.
3. Becker Shirley Ann and Anthony Berkemeyer, "Rapid Application Design and Testing of Web
4. Usability," IEEE MultiMedia, October/December 2002, pp. 38-46.
5. Boudreau Marie-Claude, David Gefen and Detmar W. Straub, "Validation in Information SystemsResearch: A State-of-the-Art Assessment," MIS Quarterly (25:1), 2001, pp. 1-16.
6. 4.Business Week, "Doing Business In the Internet Age," Information Technology Annual Report, June22 1998, pp. 121-194.
7. Davis, F. D., "A Technology Acceptance Model for Empirically Testing New End-User InformationSystems: Theory and Results," Doctoral Dissertation, MIT Sloan School of Management, Cambridge,MA., 1986.
8. Hair, J., Anderson,R. , Tatham, R., and Black ,W., Multivariate Data Analysis, Prentice Hall, NewJersey, USA, 1998.
9. Hartson H. Rex, Terence S. Andre and Robert C. Williges, "Criteria for Evaluating Usability EvaluationMethods," International Journal of Human-Computer Interaction (13:4), 2001, pp. 373-410.
10. Huberman, B. and Lukose, R., "A metasearch engine that leans which search engine to query,"Science (277), 1997, pp. 535-537.
11. Huberman, B., Pirolli, P., Pitkow, J., and Lukose, R., "Strong regularities in world wide web surfing,"Science (280), 1998, pp. 95-97.
12. ISO CD 9241-11, Guidelines for Specifying and Measuring Usability, 1993.
13. Kantner, L., S. Rosenbaum, "Usability Studies of WWW Sites: Heuristic Evaluation vs. LaboratoryTesting," Proceedings of SIGDOC, Snowbird, UT, 1997, pp. 153-160.
14. KoufarisMarios, AjitKambil, and Priscilla Ann LaBarbera, "Consumer Behavior in Web-Based
15. Commerce: An Empirical Study," International Journal of Electronic Commerce (6:2), Winter2001-2002,pp. 115-138.
16. Lawrence Steve and C. Lee Giles, "Accessibility of information on the web," Nature (400), July 81999, pp. 107-109.JIBCfile:////omics-accounts/Accounts/JIBC%20HTML%20TO%20PDF/2004%209-1/0402-01.htm[13-02-2015 12:09:40]
17. Lea M., "Evaluating User Interface Designs" in Rubin T., User Interface Design for Computer
18. Systems, Chichester: Ellis Horwood, pp. 134-167, 1998.
19. Lohse, G., P. Spiller, "Internet Retail Store Design: How the User Interface Influences Traffic andSales," Journal of Computer Mediated Communications (5), 1999.
20. Margherio, L., "The Emerging Digital Economy, A report distributed by the U.S. Department ofCommerce," Spring 1998, web available at: http://www.ecommerce.com.
21. Moeller, Eliazabeth Weise, "The Latest Web Trend: Usability?" IEEE Communication Dimensions,2001, pp. 151-158.
22. Molich, R. and Nielsen, J., "Heuristic Evaluation of User Interfaces," Proceedings of ACM HumanFactors in Computing Systems CHI90, 1990, pp. 249-256.
23. Murray George and Tania Costanzo, "Usability and the Web: An Overview," August 1999, ISSN1201-4338, web available at: http://www.nlc-bnc.ca/9/1/p1-260-e.html.
24. Nielsen J. &Molich R., "Heuristic Evaluation of User Interfaces," Proceedings of ACM CHI90Conference on Human Factors in Computing System, Seattle WA, 1990, pp. 249-256.
25. Nielsen Norman Group, http://www.nngroup.com/.
26. Nielsen, J., "Heuristic Evaluation" in J. Nielsen & R.L. Mack (Eds.), Usability Inspection Methods,John Wiley and Sons, New York, pp. 25-62, 1994.
27. Nielsen, J., Designing Web Usability, New Riders, Indianapolis, IN, 2000.
28. Nielsen, J., Usability Engineering, London: AP Professional, 1993.
29. Nunnally, J. C., Psychometric Theory, McGraw Hill, New York, 1978.
30. Paolini Paolo, "Hypermedia, the Web and Usability Issues," IEEE, 1999, pp. 111-115.
31. Payne S. J., and Green T. R. G., "Task-Action Grammars: A Model of the Mental Representation ofTask Languages," Human-Computer Interaction (2:2), 1986, pp. 93-133.
32. 28. Reisner, P., "Formal Grammar and Human Factor Design for an Interactive System," IEEE
33. Transactions on Software Engineering (SE-7:2), 1981, pp. 229-240.
34. Scriven, M., "The Methodology of Evaluation," in R. Tyler, R. Gagne, & M. Scriven (Eds.),
35. Perspectives of Curriculum Evaluation, pp. 39-83, Chicago: Rand McNally, 1967.
36. Shneiderman, B., Designing the User Interface, 3rd ed. Addison-Wesley Longman, Inc., Boston,MA, 1998.
37. Straub Detmar W., "Validating Instruments in MIS Research," MIS Quarterly (14:2), 1989, pp. 147-169.
38. Whitefield A. Wilson, F., & Dowell J., "A Framework for Human Factors Evaluation," Behavior andInformation Technology (10:1), 1991, pp. 65-79.
39. Cuevas Jackie, "The Internet Banking Horizon: Bleak or Bright for Community Banks?" Journal of Internet Banking and Commerce (3:3), November 1998.
40. Diniz Eduardo, "Web Banking in USA", Journal of Internet Banking and Commerce (3:2), June 1998.