Trans-AI/DS encourage disruptive, original, critical, and creative AI/DS thinking and perspectives. They also foster new AI/DS research and development opportunities and also pose new challenges over the AI/DS evolution.
6.1 Trans-AI/DS thinking
Trans-AI/DS require thinking beyond existing AI/DS thinking. In the previous sections, we discussed many specific aspects regarding transformative, transdisciplinary, and translational AI/DS. Here, we further discuss several ‘beyond thinking’ [
28] perspectives going beyond foundational and long-lasting AI/DS thinking. The ‘beyond AI thinking’ goes beyond existing scientific research perspectives, and specifically, beyond hypothesis-driven, data-driven, model-driven, domain-driven, and experience-driven research. Specifically, it goes beyond statistical i.i.d. assumptions and fitting.
Beyond existing scientific research Existing scientific research relies on some typical thinking patterns and methodologies. These include theoretical, model-driven, hypothesis-driven, problem-oriented, target-oriented, simulation-based, or data-driven research. These perspectives have been widely applied to almost all science fields, producing significant pools of knowledge. More creative, critical, transformational and disruptive scientific research requires new blue-sky research thinking, perspectives, and methodologies. New science and Trans-AI/DS require new scientific thinking and research, such as on system complexity-driven, nature and essence-driven, imagination-driven, curiosity-driven, counter-intuition, and unknown-driven research.
Beyond hypothesis Hypothesis-driven research has been generally applied as a starting point for further research across every scientific discipline, in particular a hypothesis test for statistical learning [
29] and bioscience. However, the proposition, supposition, or proposed explanation may not be true and evidence-based. On the other hand, a predefined hypothesis may even misunderstand, mislead, or misinterpret the genuine essence of the underlying problem or system. For example, a Gaussian distribution is often assumed to characterize the informativeness of data. This, however, may not apply to many sources and problems of data, such as long-tail data. Beyond hypothesis thinking encourages hypothesis-free thinking, thinking initially with and then without a hypothesis, and transforming one hypothesis to another, etc.
Beyond model-driven Model-driven research has played a foundational role in software engineering, system design, and human–machine interaction [
30]. Model-driven AI/DS involve a predefined model, which is tuned to match an underlying problem. A model often involves certain hypotheses and assumptions. Such hypotheses may not match the underlying problem domain, problem nature and complexities, resulting in over-qualified or under-qualified modeling. Beyond model-driven research thus suggests new perspectives. Examples include domain-driven [
31] and model-free research, and semi-model-based research which partially involves the model and then further explores the genuine models fitting the problem well. Modeling problem characteristics and complexities is another example.
Beyond domain-driven Domain-driven research involves domain knowledge, factors, context, and expertise in design and development [
32]. It complements model- and data-driven research for improving actionability such as in domain-driven actionable knowledge discovery [
33]. It involves domain-driven designs and approaches but also goes beyond them. It may further involve human expertise, human-in-the-loop of the system, or human feedback and online interactions. Typically, such AI/DS systems have to involve broad domain factors and contexts, such as the underlying organization, society, stakeholder management, evaluation measurement, and deliverable requirements. Beyond domain-driven AI/DS will play a critical role in pursuing actionable intelligence.
Beyond experience-driven Experience-driven research has been widely applied in areas including business management [
34], recommender systems, and reinforcement learning. It involves historical experience, data, feedback, and positive or negative online experiences in design and solution. Experience, history, and feedback may be false, biased, incomplete, unfair, or exceptional. It involves but does not depend on experience. It also verifies the nature, quality and relevance of the experience, aligns experience with the underlying problem well, and thinks beyond experience.
Beyond statistical i.i.d. assumptions One foundational statistical assumption is the i.i.d. assumption, which assumes samples in a dataset are independent and identically distributed (i.i.d.), or samples are i.i.d. drawn from a distribution. This statistical assumption has been taken as a default setting for almost all sciences, technologies and engineering.
1 However, this often contradicts real-life systems, behaviors, and data. The broad AI/data science body of knowledge has also been heavily dependent on this assumption. Many widely explored areas and techniques are based on the i.i.d. assumption. Examples are similarity and distance measures, Bayesian learning, reinforcement learning, and deep learning. Non-IID thinking [
17] requires substantial rethinking of reality and non-IIDness. The non-IIDness may be composed of complex heterogeneities and interactions within and between systems, subsystems, objects, and object properties [
17,
35].
Beyond data-driven Data-driven research is regarded as the fourth scientific paradigm, which demonstrates significant advantages and potential for evidence-based AI/DS research. Data-driven discovery lets data tell the story about the underlying systems [
6,
36]. Data-driven research has to handle many fundamental challenges or concerns. Examples are data characteristics and complexities, quality of data, trustworthiness of data, misinformation in data, capability and capacity gaps in understanding data characteristics and complexities, and fairness and bias of data [
37]. These challenge the commonly used ‘end-to-end’ approaches such as in deep learning and the data fitting approaches widely applied in mathematical modeling and machine learning. Data-driven approaches may incompletely, mistakenly, unfairly or biasedly characterize, treat or fit the above challenges. Beyond data-driven thinking thus suggests a data-free thinking, thinking with and without data, a deep understanding of data characteristics and complexities, and data quality-resilient research, etc.
Beyond fitting In general, both existing model-based and data-driven methods are essentially fitting oriented, i.e., matching input to output by tuning a model for a good fit. Accordingly, the goodness of fit
2 is regarded as an important statistical test and metric to evaluate the fitting performance. Though fitting has played a critical role in many classic research areas including curve fitting and machine learning [
38], it is still essential for deep learning [
15]. Both model-based and data-driven fitting may ignore the nature, quality and value of input and output, causing ‘curse of fitting’. Such fitting approaches fail to achieve ‘quality in and quality out,’ or ‘value in and value out.’ Thus, beyond fitting thinking is essential. It is grounded on understanding the reality, complexity, quality, and value of input. It aims to develop corresponding models to explore the essence, complexities, quality and value of underlying problems, systems, and their data.
Beyond deep neural networks The existing fitting-based end-to-end deep learning systems transcend classic feature engineering-based machine learning with much higher capacity and flexibility. This is why deep learning works well when large data, high-volume parameterization, high-complexity models, and high-performance computing are available. Deep learning fitting transgresses classic fitting mechanisms and scale for much finer, lower-level microscopic and individualistic fitting. It develops and utilizes multi-grain, hierarchical, multi-aspect, and multi-method fitting. This often results in deeper input–output matching, more flexible scenarios, situation or setting-based fitting, and better learning performance. In contrast, deep models perform badly or ugly with limited data or without ground truth, such as the various failures in ChatGPT.
3 A fundamental is that such a multiple, microscopic, and all-round fitting nature does not resolve long-standing fitting problems. Deep models still suffers from underfitting or overfitting, causing high bias or high variance [
39]. In addition, deep learning further incorporates fundamental bottlenecks and new significant challenges. These include
-
‘curse of fitting’ troubling unsupervised deep learning without fittable ground truths, and causing failures under drifting/shifting or open conditions [
40];
-
disentanglement and decoupling for disentangled and decoupled representation learning [
41], which weakens or damages the intrinsic interactions and couplings in underlying systems and their data and behaviors;
-
distributional vulnerability such as high-confidence predictions on test out-of-distribution samples [
42], and
-
architecture-, mechanism- and parameter-sensitive vulnerabilities, such as relating to the gradient-based backpropagation [
43] and adversarial training [
44].
6.2 Trans-AI/DS mechanisms
Trans-AI/DS thinking inspires new and hitherto nonexistent Trans-AI/DS disciplinary opportunities, concerted actions, and co-creative developments. These may cover various areas relating to the Trans-AI/DS paradigm, Trans-AI/DS research, Trans-AI/DS engineering, and Trans-AI/DS practice. In these aspects, Trans-AI/DS thinking is built into the problem definition, knowledge generation, and solution creation for Trans-AI/DS research, engineering, and practice.
Trans-AI/DS paradigms Trans-AI/DS rely on appropriate thinking, methodological, and engineering paradigms. Typical mechanisms and paradigms for transformative, transdisciplinary and translational research include curiosity, imagination, abstraction, catalysis, transcendence, transgression, transfer, hybridization, federation, reconfiguration, integration (synthesis), and metasynthesis [
22,
45].
Curiosity Discovery happens in curious human brains. Curiosity is the fuel for discovery, critical for inspiring early scientific thinking, and blue-sky breakthroughs. It fuels a passion for science [
46]. An example of curiosity-driven discovery is the successful invention of airplanes, which countered the intuition “heavier-than-air flying machines are impossible.” Curiosity-driven research explores known unknowns and focuses on the concept of “we do not know what we do not know” [
6].
Imagination Imagination is another source of critical human intelligence and is the oil for scientific discovery. It fosters sensation, creativity and innovation through spontaneous, indirect, alternative, jumping, and changing thinking. It involves productive, reproductive or constructive identification. Research imagination [
47] identifies novel ideas, spontaneous insights, alternate perspectives, possible futures, direct and indirect connections, and unconstrained, jumping and imaginary opportunities for AI/DS.
Abstraction Abstraction [
48] plays a critical role in science. It conceptualizes, extracts, generalizes, simplifies, and compresses common, general, and high-level principles, concepts, rules, attributes, and knowledge from examples, and instances. Trans-AI/DS explore new abstraction thinking, methods, and tools through transformation, transdisciplinarity, and translation.
Catalysis Catalytic research is inspired by the catalysis in chemical reactions [
49]. Trans-AI/DS support deliberative, reflective, counter-intuitive, or participatory thinking and approaches. They integrate thinking, knowledge, and methodologies outside the underlying domains, and disciplines. They also reorganize and restructure the underlying AI/DS constituents with new thinking, methodologies, knowledge, and materials.
Transcendence Transcendence goes beyond normalcy and constituents. Transcendent research bridges the boundaries between constituent disciplines, methodologies, and theories. Transcendent research for Trans-AI/DS creates new, coherent and unified perspectives, methodologies, and designs through surmounting and excelling the interactions and integration of AI/DS constituents.
Transgression Transgression violates the existent thinking, methodologies, theories, and methods for new, destructive and disruptive results. Transgressive research for Trans-AI/DS overcomes, surpasses, and scales up the capability and capacity of the underlying constituents. It approaches disruptive thinking, designs and tools through cross-boundary, and discriminative approaches such as reflexivity and intertextuality for AI/DS.
Transfer Transfer migrates merits gained from a known, explored, or grasped discipline or domain to another new, unknown, or open area. Transfer research for Trans-AI/DS explores known unknowns, from knowns to unknowns, and inspiration for ‘we do not know what we do not know’ in AI/DS research. A typical transfer research area is transfer learning [
50], which moves knowledge learned in the source domain to a new, unexplored but connected target domain. Transfer research for Trans-AI/DS can also share, shift, and convey thinking, methodologies, and methods from one area to another in AI/DS.
Hybridization Hybridization enables the mixture or combination of two to multiple thinking traits, theories, methodologies and methods. It is a general approach for producing mixed, combined, joint, collaborative, or mutual developments. For Trans-AI/DS, hybrid approaches may be combined with other more destructive and constructive approaches in AI/DS to generate transformative, transdisciplinary and translational concepts, definitions, representations, systems, theories, or methodologies. To this end, it may combine methodologies and techniques from multiple disciplines [
51].
Federation Federation associates local and global units to form distributed or federated architectures, networks, and systems in a centralized, decentralized or hybrid mode. Other similar approaches include alliance, coalition, union, conjunction, and consolidation. Federated research may support networked infrastructure, system coalition, and task allocation for distributed, cloud-based, and edge-based environments, such as in federated learning [
52]. For Trans-AI/DS, such federated research may be further enhanced through transformation, transdisciplinarity, and translation.
Reconfiguration Transcending configuration, reconfiguration [
53] supports new ways or a different form of combination or arrangement of constituent techniques, methods, or parts. Reconfiguration may involve different roles, capabilities, techniques, processes, or systems. New or different logic, hierarchy, structure, functionality, or processes may create new systems. Trans-AI/DS expect to incorporate transformative, transdisciplinary and translational thinking and operations into the rearrangement for AI/DS.
Integration Integration [
54] supports the synthesis of multidisciplinary perspectives, multi-paradigms, multi-techniques, or multi-methods. It may synergize formal and empirical, theoretical and experimental, qualitative and quantitative, or subjective and objective research, thinking, knowledge, methodologies, and approaches. Trans-AI/DS expect the transformation, transdisciplinarity and translation of individualistic entities in AI/DS.
Metasynthesis Metasynthesis is a human-centered, and human–machine-cooperative methodology for iterative qualitative-to-quantitative problem-solving [
22,
45]. Metasynthesis synthesizes multiple types of intelligences with humans in the loop. Depending on system complexities, human intelligence, social intelligence, machine intelligence, data intelligence, and network intelligence may be of interest. Metasynthesis generally applies to open complex intelligent systems and problems [
55]. Hence, their problem-solving requires the significant transformation, transdisciplinarity and translation of research paradigms, methodologies, techniques, knowledge, and intelligence.
Others Trans-AI/DS also involve other methodologies, theories, and techniques for transformative, transdisciplinary and translational developments. Examples include transforming mental activities such as attention, natural system mechanisms such as evolution, social mechanisms such as mentorship and supervision, and technical approaches such as contrast, competition (e.g., adversarial learning), and collaboration.
Trans-AI/DS research Trans-AI/DS research seeks paradigmatic shifts toward interdependent, interactive, interconnected, and integrative AI/DS research thinking, methodologies, and developments. Promising Trans-AI/DS research areas include natural-social, social-technical, societal-scientific, scientific-extra-scientific, and human–machine-cooperative research perspectives, orientations, and discourses. Trans-AI/DS research also supports inner-disciplinary, outer-disciplinary, and extra-disciplinary orientations, discourses, and developments. Trans-AI/DS engineering supports the interprofessional integration of knowledge, expertise, competencies, and experiences from collective and group members, and inclusive and exclusive developments.
Trans-AI/DS research approaches and orientations can be categorized into many perspectives, including:
-
Thinking-oriented: building on human thinking traits such as curiosity, attention, and imagination;
-
Methodology-oriented: building on scientific methodologies such as reductionism, holism, and systematism;
-
Problem-oriented: building on recognizing, understanding and defining the underlying problem, and its characteristics and complexities;
-
Goal-oriented: or mission-, target-, orientation- or task-oriented, focusing on goals, aims, and objectives;
-
Setting-oriented: focusing on specific scenarios, situations, and tasks;
-
Approach-oriented: focusing on developing, upgrading, and transforming a specific AI approach;
-
Procedure-oriented: focusing on procedural forms, processes, organizations, structures, and workflows; and
-
Solution-oriented: developing the solution space by unifying and transcending the relevant techniques and methods for the underlying problem.
6.3 Challenges
The concepts of transformation, transdisciplinarity, and translation have not formed consistent, and commonly agreed definitions, systems, and boundaries. Often, different highlights, specific orientations or discourses, or even conflicting arguments and proposals are available in the literature [
10,
26].
The Trans-AI/DS thinking and research raise various common challenges in pursuing the aforementioned Trans-AI/DS vision, objectives, and developments. Here, we list a few examples:
-
Uncertainty recognition, modeling and management during transformation and translation;
-
Disagreement and conflict resolution and paradoxical discourse between constituents during the pursuit of transdisciplinarity;
-
Balance and tradeoff between conflicting and competing constituents and during transformation and translation;
-
Unknownness [
6,
56], including unknown challenges, and opportunities for their identification and quantification during the transformation, transdisciplinarity, and translation.
-
Complexity [
57], including diversity, openness, hierarchy, interactions, and heterogeneities between constituents;
-
Openness [
22,
45,
58], such as open world, open problems, open set of classes [
59], open interactions and relations, open boundaries, and open settings;
-
Higher-level intelligence [
16,
60], such as curiosity, imagination, and attention-driven AI/DS research and development.
Specifically, AI and data science systems may be challenged by unknown problems, data, behaviors or environments. For example, unknown class labels, distributions, data quality issues, or contexts may appear ahead a deep learning task. In such cases, past experiences, common sense, exhaustive fitting may not help with their genuine understanding and problem-solving. Accordingly, Trans-AI/DS require unknown representation, reasoning, planning, learning, and analytics, which should represent, reason, plan, learn and analyze with unknownnesses.
Accordingly, Trans-AI/DS thinking and research require disruptive, original and foundational thinking, methodology, and development. They build on and go beyond the existing AI/DS and scientific thinking, paradigms, assumptions, approaches, and practices.