Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Universal Access in the Information Society
Erschienen in: Universal Access in the Information Society 3-4/2004

01.10.2004 | Long paper

Comparing accessibility evaluation tools: a method for tool effectiveness

verfasst von: Giorgio Brajnik

Erschienen in: Universal Access in the Information Society | Ausgabe 3-4/2004

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper claims that effectiveness of automatic tools for evaluating web site accessibility has to be itself evaluated, given the increasingly important role that these tools play. The paper presents a comparison method for a pair of tools that takes into account correctness, completeness and specificity in supporting the task of assessing the conformance of a web site with respect to established guidelines. The paper presents data acquired during a case study based on comparing LIFT Machine with Bobby. The data acquired from the case study is used to assess the strengths and weaknesses of the comparison method. The conclusion is that even though there is room for improvement of the method, it is already capable of providing accurate and reliable conclusions.
Vorheriger Artikel The experience of being connected
Nächster Artikel The geography of the digital divide: broadband deployment in the Community of Madrid

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
2
Comparing these tools is somehow unfair given their different scope, flexibility, power and price. LIFT Machine is targeted to an enterprise-level quality assurance team and whose price starts at $6,000; Bobby 4.0 was available for free (now it runs at about $300) and is targeted to a single individual wanting to test a relatively limited number of pages. Nevertheless, the comparison is useful as a case study for demonstrating the evaluation method itself.
 
3
More specific types of results could be considered. For example a distinction among definite errors, probable errors, manual warnings triggered by content, and untriggered manual warnings would yield a finer grid serving as a basis for a richer evaluation of tools. Some of the testing tools provide these finer distinctions. However, it may be difficult to classify the tool output according to those categories (this information might not be always available from the tool output) and if two tools provide different types of results, it will be difficult to compare them. For this reason, the method proposed in this paper is based on two types of results: those that the tool assumes to be true problems and those that are warnings.
The consequence is that the evaluation method is blind with respect to finer distinctions, and tools that provide intermediate warnings are treated in the same way as tools that provide manual warnings.
 
4
It is recommendable to use the same limits for both systems; otherwise the consequence is that there might be several pages that are tested by one tool only, thus reducing the effectiveness of the comparison method, since the issues associated with these pages are excluded from any further analysis. This has happened in the case study, due to differences in the crawling methods adopted by different tools.
 
5
This is Bobby’s terminology corresponding to what we earlier referred to as manual warning triggered by content (for Partial or PartialOnce) and untriggered manual warning (for AskOnce).
 
6
This is a case where the values reported in Table 1 affect the FN percentages. In particular, since FN for Bobby is defined in reference to the behavior of LIFT, when considering a larger number of issues generated by LIFT, there are increased chances to find a FN for Bobby. Therefore, FN for Bobby is correct, while FN for LIFT is underestimated.
 
7
A confidence interval of a parameter around a value and with a given significance level α describes the possible variability of the parameter when a different sample of data is analysed. α gives the probability that the parameter stays within the interval.
 
8
For example, the claim HFPα valid with probability α=0.01, means that the data gathered in this experiment in 99 cases out of 100 support the claim that A produces less FP than B.
 
Literatur
1.
2.
3.
Brajnik G (2004) Using automatic tools in accessibility and usability assurance. In: Stephanidis C (ed) Lecture notes in computer science proceedings of the 8th ERCIM UI4ALL workshop. Springer, Berlin Heidelberg New York
4.
Brajnik G (2003) Comparing accessibility evaluation tools: results from a case study. In: Ardissono L, Goy A (eds) HCITALY 2003: Simposio su human-computer interaction. SigCHI, Turin, Italy
5.
Brink T, Hofer E (2002) Automatically evaluating web usability. In: Proceedings of CHI 2002 workshop, Minneapolis, 20–25 April
6.
7.
8.
9.
Fenton NE, Pfleeger SL (1997) Software metrics, 2nd edn. Thompson, Washington, D.C.
10.
Gunning R (1968) The techniques of clear writing. McGraw-Hill, New York
11.
Ivory M, Hearst M (2001) The state of the art in automated usability evaluation of user interfaces. ACM Comput Surv 4(33):173–197
12.
13.
14.
Paciello M (2000) Web accessibility for people with disabilities. CMP Books, Gilroy, Calif.
15.
16.
Slatin J, Rush S (2003) Maximum accessibility: making your web site more usable for everyone. Addison-Wesley, Boston
17.
Sullivan T, Matson R (2000) Barriers to use: usability and content accessibility on the web’s most popular sites. In: Proceedings of 1st ACM conference on universal usability, Washington, D.C., 16–17 November
18.
19.
Thatcher J, Waddell C, Henry S, Swierenga S, Urban M, Burks M, Regan B, Bohman P (2002) Constructing accessible web sites. Glasshaus, Birmingham, UK
20.
21.
22.
23.
24.
Metadaten
Titel
Comparing accessibility evaluation tools: a method for tool effectiveness
verfasst von
Giorgio Brajnik
Publikationsdatum
01.10.2004
Erschienen in
Universal Access in the Information Society / Ausgabe 3-4/2004
Print ISSN: 1615-5289
Elektronische ISSN: 1615-5297
DOI
https://doi.org/10.1007/s10209-004-0105-y

Weitere Artikel der Ausgabe 3-4/2004

Universal Access in the Information Society 3-4/2004 Zur Ausgabe

Communication

German BIENE award reveals best practice in barrier-free Web design

Long paper

Using speech and dialogue for interactive TV navigation

Long paper

A taxonomy of novice user perception of error on the Web

Long paper

The experience of being connected

Long paper

Unified user interface development: the software engineering of universally accessible interactions

Acknowledgement to reviewers

Acknowledgement to reviewers for 2003