Finds documents with both search terms in any word order, permitting "n" words as a maximum distance between them. Best choose between 15 and 30 (e.g. NEAR(recruit, professionals, 20)).
Finds documents with the search term in word versions or composites. The asterisk * marks whether you wish them BEFORE, BEHIND, or BEFORE and BEHIND the search term (e.g. lightweight*, *lightweight, *lightweight*).
This chapter delves into the critical role of public transport in European societies and the impact of COVID-19 on its usage. It introduces a methodological framework to assess how innovative IT solutions can meet the needs of travellers and Transport Service Providers (TSPs), thereby increasing the attractiveness of public transport. The methodology considers various socio-demographic profiles and uses operational key performance indicators (KPIs) and user satisfaction surveys to evaluate the effectiveness of IT solutions. The Travel Companion application is developed and tested at six demo sites to demonstrate these innovative solutions. The chapter also outlines the steps involved in the assessment methodology, including the definition of user journeys, demonstration scenarios, and the calculation of effectiveness. Advanced analytical methods like AHP, regression analysis, Bayesian Networks, and ANOVA are employed to provide a comprehensive evaluation. The results from the Warsaw demo site are highlighted, showing the highest satisfaction and effectiveness for specific functionalities and profiles. The conclusion emphasizes the potential of this methodology to be applied to other IT innovations and demo sites, making it a valuable tool for enhancing public transport services.
AI Generated
This summary of the content was generated with the help of AI.
Abstract
Considering the significant growth rate of population in the urban area, public transport has become vital to urban living. It has become unavoidable to promote the culture of using Mobility as a Service (MaaS) among travellers to address climatic challenges, especially the global warming phenomenon. To encourage the use of public transport, It is important to introduce innovative IT solutions to the ecosystem of TSPs (Transport Service Providers) backed by an in-depth impact analysis to meet the expectations and the needs of the TSPs and the travellers. This work introduces an assessment methodology to calculate the “Effectiveness” of the innovative IT solutions for TSPs and travellers through a series of data analysis methods using the Bayesian Network, Regression analysis, and ANOVA tests. The assessment is based on two types of quantitative datasets: operational KPIs (Key Performance Indicators) and USI (User Satisfaction Index) surveys to evaluate how the expectations and needs of travellers with different socio-demographic profiles (by gender, age, income level, and impairments) are met by these IT solutions. This methodology is the foundation for the IP4MaaS Project supported by the Europe’s Rail Joint Undertaking. The paper presents the results of applying this methodology to data collected in six sites (Athens, Barcelona, Liberec, Osijek, Padua, and Warsaw). The presented methodology will be helpful to the IT developers and TSPs for assessing their own IT solutions. The findings will help researchers, policymakers, and others in the transport sector to assess public transport services. This assessment methodology is scalable to other demo sites and datasets.
1 Introduction
Nowadays, transport services, especially public transport services, play a vital role in every European society [1, 2]. Considering the drastic increase in air and noise pollution caused by GHGs (Green House Gases) and their side effects on every environment, investigating and studying traveller’s behavior in making decisions and understanding their needs and expectations from one side and the other side, the capabilities and capacities of TSPs( Transport Service Providers) to answer needs as mentioned earlier and expectations, are undeniable facts for transport experts [3, 4].
It is worth mentioning that the number of passengers using public transportation in 2020 and 2021 decreased by 40% to 70% due to COVID-19, the after-effects of which continue to affect the use of public transportation systems to date. With remote working becoming a norm, daily commuting has become less frequent in many countries [5‐7]. In 2020, the declining percentage number of passengers resulted in an 11% decrease in transportation service supply compared to 2019, thereby causing heavy financial losses. The drop in fare box revenue was anticipated to be 90 percent. Railways in the European Union lost twenty-four billion euros in revenue for passenger services in 2020, a 41% decrease from 2019 [8]. This paper aims to present a methodological framework to assess quantitatively how innovative technologies can respond to the needs of travellers and Transport Service Providers (TSPs) involved in the digital ecosystem for door-to-door travels in Europe, thereby increasing the attractiveness of public transport. This work uses a methodological approach to evaluate the needs of travellers with different socio-demographic profiles and TSPs (Transport Service Providers) based on rail transport. It considers social trends like reducing Greenhouse Gas (GHG) emissions and road congestion [9, 10]. The concept of the attractiveness of rail and public transport depends on complex psychological factors from a scientific and technical standpoint [11]; The methodology used in this study consolidates the concept of “user profile” and the ability of the system to respond to the needs and expectations of users, including socio-demographic-related factors such as aging, reduced mobility, and other specific conditions [12].
Advertisement
This work introduces an assessment methodology that considers the Effectiveness of IT (Information Technology) solutions, the Adaptability of data to the Satisfying Requirements of both Travelers and Operators, the Equity of IT Deployment in Society, and the Potential Acceptance by the Market and, in particular, by Railway Operators and their Ecosystem [13].
For the purpose of this work, the Travel Companion application has been developed to demonstrate the innovative IT solutions that are further tested and evaluated by the travellers and TSPs at the six demo sites in the IP4MaaS project. One of the key benefits of this methodological framework that distinguishes it in this field is that it provides social equality for all individuals, regardless of sex, gender, ethnicity, age, origin, income, or health, to have equal rights and access to transportation systems. This assessment methodology is part of the methodological framework developed in the first and second phases of the H2020 Shift2Rail project titled IP4MaaS (www.ip4maas.eu), which sets 6 demo sites (Barcelona, Padua, Athens, Liberec, Osijek, and Warsaw) in which this methodology is applied in a second stage [14].
2 Methodology
The methodology employed in this study uses the concept of “demonstration scenario”[15]. This methodology bases its innovations on a combination of operational key performance indicators and customer satisfaction surveys [15] [16]. In addition, the methodology attempts to provide a clear list of KPIs and survey questions that consider the requirements and expectations of users with varying profiles, such as the elderly, disabled people, and women, to promote equity for all individuals. These objectives will distinguish the methodology from others in the same field [17, 18]. This assessment outlines the methodology for conducting User Satisfaction Index (USI) questionnaires, which are used to evaluate the satisfaction of users with the IP4 (Innovation Programme 4) solutions, and explains in detail how the effectiveness is calculated for each user profile. The data from USIs and operational KPIs in phase II are an input for the IP4 toolbox [19]. This assessment provides a comprehensive framework for setting the final results and outcomes of the methodological framework to evaluate the IP4MaaS tool in each of the IP4MaaS 6 demonstration sites [19]. Three main inputs fed to the Travel Companion application are [20]:
1.
Collected data from USI (User Satisfaction Index) travellers from online surveys;
2.
Collected data from USI (User Satisfaction Index) TSPs (Transport Service Providers) from online surveys;
3.
Collected data from Operational KPIs (Key Performance Indicators) from CFMs (Call For Members) (Travel Companion /IP4 ecosystem developers).
The assessment alongside the general user profile also considers 4 other specific (sensitive) profiles (the definition of the profile variable “r” is reported):
Advertisement
General profiles (r=1), unemployed people, low-income people, retired people, and students (r=2), disabled or impaired people, people with physical or mental illnesses, people in wheelchairs, people with reduced mobility, people with visual impairment, and hearing impairment (r=3), elderly (r=4) and women (r=5).
The methodology uses the data collected from Operational KPIs and USIs surveys during the execution of the demo sites.
The following steps illustrate this assessment methodology:
Step 1: Definition of the User journeys “i”.
A User Journey “i” is a travel solution from an Origin to a Destination in which a traveller may interact with an IP4 innovation offered by one or more Transport Service Providers (TSPs). The assessment is carried out on one or several User Journeys.
Step 2: Definition of the Demonstration Scenario “JK” (identification of TSPs “k” and functionalities “j”) and sensitive profiles (“r”).
A Demonstration scenario (Demo) is the intersection of a new IP4 innovation “j” offered to travellers and a Transport Service Provider “k” that is offering it. On the other hand, the identification of sensitive profiles is done through a conversational survey and sentiment analysis.
Step 3: Identification of operational KPIs (“KPIs”) and benefits provided by functionalities (“j”) to these sensitive profiles (“Br”).
Only those operational KPIs that can be measured during the execution of these IP4 functionalities will be considered, and benefits provided by these functionalities to these sensitive profiles will be identified.
For this identification of benefits, Focus Groups, workshops, or other brainstorming data collection methods can be used.
Step 4: Data collection of operational KPIs and satisfaction regarding benefits through USI surveys.
If in the case study, the operational KPIs are quantifiable, they should be collected during the execution of each demo site. On the other hand, USI surveys are used to assess the benefits provided by each functionality “j” to each sensitive profile “Br.”
Step 5: Calculation of the Effectiveness and comparisons among TSPs (“k”), Functionalities (“J”), and Profiles (“r”).
If the operational KPIs are measurable, alongside USIs data, they are inserted as input into an “Effectiveness” calculation, which is a metric on how IP4 solutions match the needs and expectations of travellers and TSPs, from the perspective of an aggregated analysis, by taking into account general profiles and specific profiles (low-income people, people with disabilities, elderly, and women) and per each group of travellers in intersectional analysis.
Step 6: Further data analysis.
To accurately analyze and assess the performance of the Travel Companion application, an in-depth analysis using AHP, Bayesian network, Regression, and ANOVA test is conducted on the collected data of USI surveys and operational KPIs. This methodological approach can be applied to any other projects that require the assessment and evaluation of IP4 functionalities [19] (Fig. 1).
Fig. 1.
General lighthouse methodology for the assessment of IP4 innovations.
The definition of each variable that is used in the calculation of the Effectiveness is: “r” the type of profile of respondents in this study (already introduced in Sect. 2), “J” the name of innovative technology or functionality, “K” the name of TSP (Transport Service Provider) which is providing that specific functionality and “q” associated question linked to that specific functionality.
The User Satisfaction Index (USI) for travellers belonging to a profile vector “r” with the functionality “j” offered by the TSP “k” is calculated as [21]:
All this quantitative data (operational KPIs and USIs) is managed together within the concept of “Effectiveness”. The Effectiveness of a functionality “j” offered by a TSP “k” for a specific profile “r” in a demonstration scenario “D” is calculated through the following Equation. To avoid producing several equations for effectiveness per each group identified aforementioned section, a unique formula Eq. (3) has been prepared and it is implemented for all the groups in this study:
The definition of the variables in Eqs. 1, 2, and 3 may be consulted in “A Methodological Framework Based on a Quantitative Assessment of New Technologies to Boost the Interoperability of Railways Services,” [21]. Given that the Effectiveness is dimensionless with a value between 0 and 1, the higher, the better, and different demonstration scenarios “D” can be compared to analyze how the needs of travellers in other locations or demo sites are matched by the same innovative technology “j” offered by different TSPs [21]. The effectiveness comparison can only be done after grouping based on what parameters are considered in the Effectiveness formula: KPIs, USI Travellers, USI TSPs, or combinations among them. All these formulations have been prepared, documented, and calculated in Julia’s programming language (V 1.7.0) [22].
2.2 Extension of the Methodology
This assessment methodology was extended by applying the next analysis methods:
AHP analysis: This analysis is done to define a weighted hierarchy of factors with an influence on two following goals defined for the users of the Travel Companion APP including two parts: 1 - A Hierarchical model, and 2 - A pairwise comparison matrix (filled by the expert panel1). The AHP analysis has the following two main goals2 [22]:
1.
For Travellers: To encourage people to use more intermodal solutions on public transport, especially railways, by making it more attractive to users.
2.
For TSPs: To encourage TSPs to use the solution Travel Companion (APP)
Regression Analysis: The regression analysis is done as a primary step to identify Second-level Benefits3 highly correlated in a way that the heuristic process followed by the Bayesian Network Analysis (Module 3) already starts from learned networks, achieving better results in less time. The p-value in the two pair variables is less than 0.05 (p-value < 0.05), which means there is a high correlation between them.
The output from regression analysis is introduced as a forced connection into the BN analysis so that the correlations established in both processes do not contradict each other [22].
BN (Bayesian Network) Analysis, Bellman Shortest Path, and Impact Assessment (prediction simulation): The output of BN analysis from all six demo sites indicates what is the most influent second-level benefits3 for the acceptance by users of IP4 functionalities offered by TSPs considered in each demo site, taking into consideration the Bayes score and cumulative weights [22‐24].
Impact Assessment Analysis: The main reason for identifying and analysing correlations among various factors associated with respective demo sites is to carry out an impact assessment of these variables individually through predictive simulations. These simulations give us an insight into assessing the overall impact of an investment made on improving a certain factor at a demo site. The study through these simulations becomes the basis of decision-making [25] at a high level for various stakeholders involved in the project. The methodology has been designed to work with changing and scaling datasets in the future. The design of simulations makes them easily replicable to new demo sites and new factors as they get introduced in the future. A detailed description of the methodology may be found in the journal paper: “A Methodological Framework Based on a Quantitative Assessment of New Technologies to Boost the Interoperability of Railways Services,” [21‐25].
ANOVA Test (Analysis of Variance) for Travellers is applied in this analysis to determine if some socio-demographic profiles (per age, gender, incomes level, residential area, travelling with a dependent person, professional status, disability, familiarity with technology) show significant differences regarding the satisfaction with second level benefits based on the data gathered through the USI travellers survey[22].
2.3 Making the Knowledge Actionable Through an Assessment Toolbox
The performance assessment methodology of this study was made actionable through a Toolbox based on the mathematical data analysis operations explained above. The toolbox presents the results obtained from the mathematical models, in a readable and actionable manner. For instance, presenting data on the most effective features for various profiles, and the most impacted socio-demographic profiles [26]. This Toolbox aims to figure out which benefits of an innovative IT solution are more relevant for the users (TOP 10 benefits) (Modules 1 to 3), which ones show significant differences regarding socio-demographic profiles (Module 4), and which functionalities of an innovative IT solution have the highest Effectiveness based on satisfaction and operational KPIs for all kinds of profiles and specific profiles (Module 5) [27]. This Toolbox was validated in six Demo sites: Athens, Padua, Warsaw, Liberec, Osijek, and Barcelona between March and June 2023. Scripts of all these five modules can be consulted in the Zenodo platform: https://zenodo.org/communities/ip4maas/
3 Application to a Use Case: The “Warsaw” Demo Site
The testing and execution of the Travel Companion APP in the Warsaw demo site was done from 15th to 19th May 2023. In total 4 TSPs: ZTM, MZA, TW, and SKM were assessed in this demo site and 208 responses were collected regarding USIs. The results of data analysis for this use case are shown in the following as a sample:
Results Regarding Module 1-AHP Analysis: For TRAVELLERS: Time-saving, reliability, and Cost-saving benefits, with the Travel Companion (TC) APP have the highest importance and weights among other criteria or first-level factors. For TSPs: Improving customer relationships, general satisfaction, and increased revenues through the TC APP were the most significant criteria.
Results Regarding Module 2-Regression Analysis: “Giving easier access to the APP for the elderly” with “Helping travellers to make appropriate journey planning decisions” and “General satisfaction with the Navigation function” with “Time-saving” had the highest correlation (p-value < 0.5) in this case study.
Results Regarding Module 3-Bayesian Network Analysis and Bellman Shortest Path: Providing safe trips, general satisfaction, and willingness to pay with trip sharing function for all profiles got the highest cumulative weight.
Results Regarding Module 4-ANOVA Test: According to the results, the Increase in safety with the Journey planning function for disabled profiles and providing a safe trip from a COVID-19 perspective for elderly profiles with the Journey planning function for the case of professional status and disability showed the most significant differences in ANOVA analysis.
Results Regarding Module 5-USI Travellers, USI TSPs, and Effectiveness: it can be concluded that the datasets (profiles, functionality, TSP) achieving the highest satisfaction belong to the Travel arrangement functionality provided by SKM and MZA for disabled profiles (r3J21K10) and (r3J21K8) respectively for the case of travellers. However, in the case of TSPs, the highest satisfaction belongs to the Asset manager tool provided to MZA (J23K8) with a value equal to 0.61. Considering the values of USI traveller, USI TSPs, and operational KPIs in the Warsaw demo site, and applying Eq. (3), those sets (Profile, Functionality, TSP) with the highest values of the “Effectiveness” for the case of travellers belong to the Travel arrangement functionality provided by MZA and SKM for disabled profiles (r3J21K8) and (r3J21K10) respectively. On the other hand, taking into account the values of USI traveller, USI TSPs, and operational KPIs in the Warsaw demo site, the top 10 variables in terms of the concept of Effectiveness for the case of TSPs, in terms of Effectiveness, the TC functionalities that are provided to TSPs belong to, the Asset Manager tool provided to MZA with the value equal to 0.80 (J23K8).
4 Conclusion and Further Developments
This paper presents a methodical assessment approach to quantify how well specific novel technologies created by the IP4 Shift2Rail programme meet traveller and TSP needs. Two quantitative types of data—operational KPIs and USIs—that enable the calculation of the Effectiveness of a particular innovative technology offered by a TSP to a profile group of travellers were introduced with this goal in mind. These data types allowed for the definition of demonstration scenarios on which the assessment is conducted. An innovative technology’s effectiveness is determined by how well it meets the demands and expectations of its users, travellers, and TSPs. Effectiveness is dimensionless and has a value between 0 and 1; the greater the number, the better. Comparisons between demonstration scenarios or TSPs and various traveller profiles are possible for a particular technology. To verify its advantages, move forward with the necessary improvements, and investigate its potential, this study applies quantitative assessment methodology to six demo sites with varied demonstration scenarios defined by the H2020 Shift2Rail IP4MaaS project. Furthermore, by using machine learning techniques such as Bayesian Networks, statistical correlations between operational KPIs and USIs might be identified. An assessment methodology and a 5 Modules Toolbox have been presented in this study to assess the Travel Companion APP/IP4 ecosystem more in general. This methodology and the “5-Modules Toolbox” can be applied to other Software and IT innovations; and can be also applied to the Travel Companion APP/IP4 ecosystem in other demo sites in the future.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
The expert panel was consisting of TSPs experts in each demo site, two experts from associations in IP4MaasS project (UITP and UNIFE) and two members of Travel Companion /IP4 ecosystem developers (HACON and INDRA).
These goals were in the mind of experts during the building process of the hierarchical model and the process of filling the pairwise comparison matrix.
Second level benefits are more detailed factors, clustered inside each of the first level benefits or factors level 1, with an influence on the usage of IP4 functionalities.
1.
Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9, 309–347 (1992)CrossRefMATH
2.
Lerner, B., Malka, R.: Investigation of the K2 algorithm in learning Bayesian network classifiers. Appl. Artif. Intell. 25, 74–96 (2011)CrossRef
3.
García-Jiménez, E., Poveda-Reyes, S., Kumar Malviya, A., Molero, G., Santarremigia, F.: A methodological framework for a quantitative assessment of new technologies to boost the interoperability of railways services (2022). www.sciencedirect.comwww.elsevier.com/locate/procedia2352-1465
4.
The European Rail Research Advisory Council (ERRAC): Rail Strategic Research & Innovation Agenda SRIA 2020
5.
Lozzi, G., Cré, I., Ramos, R.: Research for TRAN Committee: Relaunching transport and tourism in the EU after COVID-19 – Part VI: Public Transport
6.
CIVITAS: Smart choices for cities: Gender equality and mobility: mind the gap! 2020
7.
Donat, L., et al.: Hop on the train: ARail Renaissance for Europe How the 2021 European Year of Rail can support the European Green Deal and a sustainable recovery 2021. https://germanwatch.org/en/19680
8.
Sippel, L., Nolte, J., Maarfield, S., Wolff, D., Roux, L.: Comprehensive analysis of the existing cross-border rail transport connections and missing links on the internal EU borders Final report 2018
9.
Wolfram, M., Frantzeskaki, N., Maschmeyer, S.: Cities, systems and sustainability: status and perspectives of research on urban transformations. Curr. Opin. Environ. Sustain. 22, 18–25 (2016). https://doi.org/10.1016/j.cosust.2017.01.014CrossRef
10.
Bamberg, S., Fujii, S., Friman, M., Gärling, T.: Behaviour_theory_and_ soft_transport_policy_measure_2 (2011)
11.
Misra, S., Panda, R.: Scale transformation of analytical hierarchy process to Likert weighted measurement method: an analysis on environmental consciousness and brand equity. Int. J. Soc. Syst. Sci. 9(3), 242 (2017). https://doi.org/10.1504/ijsss.2017.087431CrossRef
12.
European Rail Research: Advisory Council. Strategic Rail Research Agenda 2020 (2020)
IP4MaaS Project. Deliverable D2.3 Demonstration requirements and scenarios F-REL 2022
16.
IP4MaaS project. Deliverable D 6.2 TOOL FOR PERFORMANCE ASSESSMENT 2022
17.
E. García-Jiménez, Poveda-Reyes, S., Kumar Malviya, A., Molero, G.D., Santarremigia, F.E.: Methodological framework for a quantitative assessment of new technologies to boost the interoperability of railways services, vol. 2022, 2022. www.aitec-intl.com
18.
IP4MaaS project. Deliverable D 3.1 List of operational KPIs, analysis of the users’ satisfaction and methodology as a whole, C-REL 2021
19.
Ahmadi-Javid, A., Hooshangi-Tabrizi, P.: A mathematical formulation and anarchic society optimisation algorithms for integrated scheduling of processing and transportation operations in a flow-shop environment. Int. J. Prod. Res. 53(19), 5988–6006 (2015). https://doi.org/10.1080/00207543.2015.1035812CrossRef
20.
IP4MaaS Project. Deliverable D6.3 Performance and impact assessment 2023
21.
Zarehparast Malekzadeh, M., Santarremigia, F., Molero, G., Malviya, A., Arroyo, R., Sánchez, T.: A methodological framework based on a quantitative assessment of new technologies to boost the interoperability of railways services. Sustainability 15(13), 10636 (2023). https://doi.org/10.3390/su151310636CrossRef
22.
IP4MaaS Project. Deliverable D3.3 Final version of the methodological framework for future projects 2023
23.
Awad-Núñez, N., González-Cancelas, F., Soler-Flores, A., Camarero-Orive, A.: Methodology for measuring sustainability of dry ports location based on Bayesian networks and multi-criteria decision analysis. Transp. Res. Procedia 13, 124–133 (2016)CrossRef
Molero, G., Poveda-reyes, S., Malviya, A., García-jiménez, E., Leva, M., Santarremigia, F.: Computational solutions based on Bayesian networks to hierarchize and to predict factors influencing gender fairness in the transport system: four use cases. Sustain. Switz. 13(20), 11372 (2021). https://doi.org/10.3390/su132011372CrossRef
26.
Santarremigia, F., Poveda-Reyes, S., Hervás-Peralta, M., Molero, G.: A decision-making method for boosting new digitalization technologies. Int. J. Inf. Technol. Decis. Making World Sci. 20(2), 635–669 (2021). https://doi.org/10.1142/S0219622021500097
27.
Molero, G., Santarremigia, F., Poveda-Reyes, S., Mayrhofer, M., Awad-Núñez, S., Kassabji, A.: Key factors for the implementation and integration of innovative ICT solutions in SMEs and large companies involved in the multimodal transport of dangerous goods. Euro. Transp. Res. Rev. 11(1), 28 (2019). https://doi.org/10.1186/s12544-019-0362-8CrossRef
