Skip to main content
Top
Published in: Empirical Software Engineering 1/2019

26-05-2018

Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challenge

Authors: Mariano Ceccato, Paolo Tonella, Cataldo Basile, Paolo Falcarin, Marco Torchiano, Bart Coppens, Bjorn De Sutter

Published in: Empirical Software Engineering | Issue 1/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

When critical assets or functionalities are included in a piece of software accessible to the end users, code protections are used to hinder or delay the extraction or manipulation of such critical assets. The process and strategy followed by hackers to understand and tamper with protected software might differ from program understanding for benign purposes. Knowledge of the actual hacker behaviours while performing real attack tasks can inform better ways to protect the software and can provide more realistic assumptions to the developers, evaluators, and users of software protections. Within Aspire, a software protection research project funded by the EU under framework programme FP7, we have conducted three industrial case studies with the involvement of professional penetration testers and a public challenge consisting of eight attack tasks with open participation. We have applied a systematic qualitative analysis methodology to the hackers’ reports relative to the industrial case studies and the public challenge. The qualitative analysis resulted in 459 and 265 annotations added respectively to the industrial and to the public challenge reports. Based on these annotations we built a taxonomy consisting of 169 concepts. They address the hacker activities related to (i) understanding code; (ii) defining the attack strategy; (iii) selecting and customizing the tools; and (iv) defeating the protections. While there are many commonalities between professional hackers and practitioners, we could spot many fundamental differences. For instance, while industrial professional hackers aim at elaborating automated and reproducible deterministic attacks, practitioners prefer to minimize the effort and try many different manual tasks. This analysis allowed us to distill a number of new research directions and potential improvements for protection techniques. In particular, considering the critical role of analysis tools, protection techniques should explicitly attack them, by exploiting analysis problems and complexity aspects that available automated techniques are bad at addressing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Glossary
Analyse attack result
Analyse the knowledge obtained from perform an attack step or combination of attack steps
Anti-callback stack checks
Checks performed on entry to internal functions to verify that they are not called from externally
Anti-debugging
Form of software protection that makes it harder for an attacker to attack a debugger to an executing software to attack it
Assess effort
Reason about effort needed to perform an attack step
Asset
Software artefact valuable to attackers, typically the artefacts of the original software on which security requirements are formulated, plus potentially the protections deployed on them in so far as those protections become under attack as well
Background knowledge
Relevant knowledge attackers have before starting attack activities on a specific piece of software
Basic block
Sequence of instructions that can only be executed as a whole and in that order
Black-box analysis
Analysis techniques that only consider the input-output behavior of the program or a component thereof
Brute force attack
Try all inputs on a code fragment to obtain the desired input or output or behavior
Build the attack strategy
The making of a decision regarding the next attack steps to be executed
Build workaround
Defeating a protection by developing custom code integrated in, or replacing parts of, the software under attack (including the protection in it under attack) to make a protection ineffective (possibly leaving the original software functionality intact) such that an attack step can be executed.
Bypass protection
Using debugger commands (or scripts) or other lightweight techniques to manipulate the execution of the software without altering its code, thus making the protection ineffective, e.g., by allowing the software execution to progress nominally beyond the point where the protection was supposed to intervene.
Bytecode
Non-native instruction set architecture format
Checksum
Hash computed over some data, in this context typically a code region being checked by a code guard
Choose path of least resistance
Reason about and select the potentially successful sequence of attack steps that will lead to reaching the overall attack goal with the least effort or cost
Choose/evaluate alternative tool
Check whether alternative tool overcomes a limitation of a previously tried tool
Circumvent protection
When a protection prevents reaching a goal with one or more specific attack steps, execute one or more alternative steps that are orthogonal to the deployed protection (i.e., not hampered by the protection) to reach the same goal
Clear data in memory
Asset other than key appearing in unprotected (plaintext) form in binary or during execution
Clear key
Cryptographic key appearing in unprotected form in binary or during execution
Client-server code splitting
Protection whereby part of the sensitive computations is extracted from a client program and executed on a secure server instead, where it is not accessible to attackers
Clues available in plain text
Presence of strings in binary that present clues about protections, assets, components, ...
Code guard
Specific form of tamper detection that computes a checksum on code regions and checks their values by comparing them to pre-computed ones
Code mobility
Online protection in which static code (and data) fragments are removed from an executable file (to prevent static attacks) and instead get downloaded from a secure server on demand during the execution of the software
Code representation and structure
Static or dynamic representations of software (fragments) and structural forms or elements in it
Condition
(combination of) values that inputs, variables, registers, memory locations, ... need to have in order to let control be transferred into a certain direction during execution of the software
Confirm hypothesis
Confirm that a previously hypothesized feature is correct based on the observation of an attack step’s results
Constant
Static data with relevant or recognizable, non-mutable values
Control flow graph
Static representation of potential control flow in a program
Control flow graph reconstruction
Determining and modelling the potential flow of control in a program or part thereof
Convert code to standard format
The act of converting (byte)code in a custom (diversified) format to a format known by the attacker
Core dump
Snapshot image of the software’s memory space during its execution, containing code and data segments such as stack, heap, and code sections from binaries
Correlation analysis
Statistical analysis where correlations between operations or data occurrences are determined or analysed
Correlation between observed values
The presence of statistical correlation in a set of data values observed in a program, memory, or a trace
Create new tool for attack
Create new standalone piece of software to aid in attack (e.g., a main binary that invokes functionality in library under attack in a specific order and that feeds it specific data)
Crypto analysis
Use of cryptanalysis techniques
Customize execution environment
Adapt software or hardware in execution environment in ways supported by their developers such that it supports specific attack tasks
Customize/extend tool
Adapt tool in ways supported by tool developers and tool itself (e.g., availability source code) or exploit its built-in extensibility to let it perform specific tasks
Data and program state
Static or dynamic non-code aspects of a binary or running process
Data flow analysis
Analysis techniques that determine how computations and computed data depend on other computations and (computed or input) data
Deobfuscate the code
Use manual or automated tools to revert an obfuscation, i.e., to reduce its apparent complexity to that of the original , non-obfuscated code
Debug/superfluous features not removed
Functionality present for software development purposes (e.g., debugging aids) that was not removed before distributing the binaries and that can be leveraged by attackers
Debugger
Tool used to test and debug software by offering support to inspect and manipulate the status of running software
Debugging
Using debugger functionality to observe, control, and manipulate a program’s execution
Decompile the code
Obtain source code equivalent of machine code
Decompiled code
Representation of binary software code at the abstraction level of source code
Decompiler
Software that translates assembly language into equivalent source code
Decrypt code before executing it
(large) code fragments only available in encrypted form become available in decrypted form at run time
Defeat protection
Successfully undo or overcome or bypass a protection, or build a workaround for it, such that an attack step that the protection was supposed to mitigate can be executed successfully.
Dependency analysis
Analysis techniques that determine which computations and computed data depend on which other computations and (computed or input) data
Difference between observed values
The fact that two values in a trace, binary, or memory have different values
Differential data analysis
Statistical analysis where not the original observed operations or data are considered but differences between multiple occurrences
Difficulty
Problem encountered during an attack task that is caused by a feature, artefact or limitation of the attacker’s toolbox, i.e., of the specific software or hardware attack aids being used or considered during a concrete attack (that are not normally used during benign use of the software under attack).
Diffing
Identifying the differences between two programs or parts thereof
Disable anti-debugging
Tampering with code to skip execution of anti debugging protection actions
Disassemble the code
Use a tool to convert binary encoding of software into human readable machine code
Disassembled code
Representation of binary software code at the abstraction level of assembly code
Disassembler
Software that translates machine language into assembly language (and determines the structure thereof, e.g., in the form of control flow graphs)
Diversified cryptographic libraries
Libraries with non-standard implementations of standard cryptographic primitives
Dynamic analysis
Analysis techniques based on observations made during program execution
Emulator
Hardware or software that enables one computer system to behave like another system
Evaluate and select alternative step/ revise attack strategy
Reason about effort, success probability, usefulness, ... of possible next attack steps, building on results of previous attack steps, and select next steps / revise decisions and selections made earlier regarding next attack steps to execute
Execution environment
Operating system, platform, network settings, etc. in which the code has to be executed
File
A resource for storing information, typically on a storage device
File name
Symbolic identifier of a file
File format analysis
Black-box analyses that consider the formats of input and output files
Function / routine
Software components making up programs in most programming languages
Function argument
Data (and value thereof) on which a function is invoked
Function call
The operation of invoking a callee function within a caller function
Global function pointer table
Standard data structures in binaries that contain addresses of functions
Identify API calls
Act of identifying locations in the code or trace, and their nature, where interaction with public interfaces of external components take place
Identify assets by naming scheme
Use naming conventions or structure in available symbol information
Identify assets by static meta info
Use standard information available in binaries (e.g., exported symbols) to identify components that embed assets
Identify output generation
Identification of code around points where output is generated as starting points of attacks
Identify points of attack
Identify regions in the program or trace where assets or protections are available/observable/active/... and hence attackable
Initialization function
Function invoked by loader upon loading of a program or library
In-memory data structure
Data structure found in the memory space of executing software
Knowledge on execution environment framework
Relevant knowledge attackers have about the execution environment or framework in which they will execute attack steps or in which the software normally executes
Lack of knowledge
Inexperience of attacker, not knowing relevant aspects
Lack of portability
Fact that a tool or technique available in one context (e.g., platform) is not available in the context in which the attacker wants to deploy the tool or technique
Library / module
Partition of an application as defined in software engineering
Limit scope of attack
Identify regions in the software or in an execution trace where next attacks steps should focus on, thus reducing the size of the code or data or trace where the attacker needs to perform next attack steps, thus reducing the effort to invest in them
Limit scope of attack by static meta info
Use standard information available in binaries (e.g., exported symbols) to identify regions in software or traces where next attack steps should focus on
Limitations from operating system
Properties of specific operating system on which the code has to be executed, that limit the attacker’s capabilities in some way
Main()
Top-level function in an application
Make hypothesis
Based on the results of previous attack steps and background knowledge make a hypothesis about the features of an asset, protection, piece of software under attack, or attack tool capability that, if true, will enable certain attack steps to be performed successfully
Make hypothesis on protection
Make a hypothesis regarding the potential deployment or nature or features of a certain protection
Make hypothesis on reasons for attack failure
Make a hypothesis based on the observation that an previous attack step yielded insufficient results
Manually assist the disassembler
Interact with a disassembler tool to correct and complement its automated disassembler analyses
Memory access
The operation of reading or writing to main memory
Memory dump
Making a snapshot of (parts of) the code and data in the address space of a running program and dumping that image on disk for later analysis
Meta info
Standard information available in binaries, in the form of data that is not used by the software itself, but by the OS to load and launch the software correctly
Monitor public interfaces
Observing and analysing interaction (invocations, data passing, communications) between components along publicly available interfaces (such as exported functions in libraries or system calls)
Non-standard virtual machines
Customized virtual machines embedded in a protected program that interprets bytecode (in a custom, non-standard bytecode format) that replaces the original native code, thus hiding the semantics of the original code
Obfuscation
Form of software protection that increases apparent complexity of code or data
Obstacle
Feature or artefact that hinders attack steps and that is deliberatively put in place (or, if already present a priori, considered relevant for providing protection) by the defender in protected software, including in components of the software itself or of its execution environment
Obtain clear code after code decryption at run time
Identify and extract decrypted code in memory space of a running application under attack
Operation
Software functionality at the lowest level of granularity / abstraction
Out of context execution
Execute code fragments not as they are normally executed within the full program’s execution, but in other crafted contexts (such as self-written main binary)
Overcome protection
Leaving a protection present and (partially) active, but manipulating the code and execution of the software or fragments therein such that the goal of an attack step is reached despite the protection still being present and (partially) active. This typically requires the custom, targeted development of external scripts and software components. The resulting code or execution are not necessarily representative for the original software as a whole, but they suffice for the attacker to reach his goal.
Pattern matching
Identifying code or data fragments of interest by comparing candidates to known patterns
Prepare the environment
Set-up and configuration of environment to execute and/or attack the program
Process / parent-child relation
Process are instances of software executing on a computer; parent processes launch child processes
Program input and output
External data consumed and produced by a program
Profiler
Tool used to collect statistics about execution of software elements
Profiling
Collecting statistics on a program’s execution and its components (functions, instructions, libraries, ...)
Protection
Software protection technique applied on software under attack
Recognizable library
Part of software under attack that corresponds to a known library and is identified as such by the attacker
Recognize anomalous / unexpected behavior
Observe program features that contradict hypothesis about normal behavior given background knowledge of the attacker
Recognize similarity with already analysed protected application
Recognize that parts of a new piece of software under attack, although it maybe has been protected differently, is identical to parts of another software already attacked before, such that the knowledge of the already attacked version can be reused, thus reducing the amount of reverse-engineering effort needed
Recreate protection in the small
Create small program containing protection mock-up to aid in the development of attacks on that protection
Reference to API function / imported and exported function
Symbolic description of externally visible (standard library) functions provided by or needed by libraries and modules
Register
A component inside a central processing unit for storing information, that can be addressed directly in assembly code
Remote attestation
Online protection technique in which a secure server demands a running client to provide attestations to verify the integrity of the client
Replace API functions with reimplementation
Use of hooking, interposers, detours and other techniques to intervene in execution when external functions are invoked
Reuse attack strategy that worked in the past
Use background knowledge on paths of least resistance and successful attack paths based on attacks on similar pieces of software or on pieces of software protected with identical or similar (assumed or identified as such) protections
Round / repetition / loop
Specific instance of a program fragment execution in a trace containing multiple subsequent executions of the fragment; strongly connected component in a control flow graph
Run analysis
Invoke an automated analysis in a tool
Run software in emulator
Use emulation to execute software and to execute dynamic attacks
Size
Amount of code or data considered by an attacker
Socket
Data structure and its interface serving as an internal endpoint for sending and receiving data over a network
Software element
Aspects of a program of interest to an attacker
Static analysis
Analysis techniques that do not require code to be executed
Statistical analysis
Use of statistical techniques to identify and/or recover operations or data or features of interest
stderr
Output connection through which many programs output error messages
String
Sequence of alphanumeric text or other symbols in memory or an executable file
String / name analysis
Extracting information from names of files, exported functions, strings referenced in code fragments, etc.
Switch statement
Control flow structure resembling a switch () { case ...: case ...: } structure in C code
Symbolic execution / SMT solving
Determining (semantic) properties of code fragments using symbolic execution and SMT solving techniques
System call
The operation invoking system routines from the operating system
Tamper detection
Forms of software protection that try to detect that normal execution or code has been modified
Tamper with code statically
Edit code in the binary, e.g., to implement a workaround
Tamper with data
Alter data during the execution of a program
Tamper with execution
Alter ongoing execution by altering code or data
Tamper with execution environment
Adapt software or hardware in execution environment in ways not intended by their developers such that it supports specific attack tasks
Tool
Any software or hardware aid that automates activities needed in attack steps or that performs a task (semi)automatically
Tool limitations
Practical limitations (supported file sizes, memory consumption, lack of precision, ...) of a tool that make it unfit for the specific way an attacker wants to use it
Trace
Sequence of executed code fragments with or without additional properties of their execution
Tracer
Tool used to collect sequences of executed software elements and attributes of their execution
Tracing
Collecting a sequence of activities occurring during the execution of a program (instructions being executed, system calls, library calls, etc.)
Understand code logic
Act of reasoning about a code fragment
Understand persistent storage / file / socket
Act of reasoning about overall program behavior regarding storage - files - sockets
Undo the protection
Reversing the effect of a protection by undoing its deployment, i.e., reverting to the software without the protection (e.g., deobfuscating code or removing code guards).
Weak crypto
Use of cryptographic techniques that are even weak against black-box attacks
White box cryptography
Form of cryptography where keys do not occur in plain sight during execution
Workaround of a difficulty
Adaptation of tool or new tool that overcomes the limitation of existing attack tool; of a protection: see build workaround
Write tool supported script
Customize a tool using its built-in scripting features
XOR operation
The operation of performing an XOR on two or more data values; these occur very frequently in and around cryptographic primitives
Footnotes
1
The Glossary presents brief descriptions of these techniques and of concepts and terms introduced later in the paper.
 
2
The random seed was not meant to decide what protection to deploy or in what variant. The random seed is used by protections to initialize values, e.g., the value to use as key, and to diversify the way certain protections are injected and obfuscated, such that the injected code cannot be identified through trivial pattern matching. Nonetheless, we verified that the randomization process did not change the code and execution patterns in such a way that diversified versions required different ways to be attacked, as it would have altered the analysis of the results.
 
3
With the placeholder “[omissis]” we indicate that a part of the text is not reported either because it cannot be disclosed for confidentiality reasons or because it is not relevant since we want the reader to focus on the most meaningful (and shorter) portion.
 
Literature
go back to reference Abrath B, Coppens B, Volckaert S, Wijnant J, De Sutter B (2016) Tightly-coupled self-debugging software protection. In: Proceedings of the 6th workshop on software security, protection, and reverse engineering (SSPREW), pp 7:1–7:10 Abrath B, Coppens B, Volckaert S, Wijnant J, De Sutter B (2016) Tightly-coupled self-debugging software protection. In: Proceedings of the 6th workshop on software security, protection, and reverse engineering (SSPREW), pp 7:1–7:10
go back to reference Anckaert B, Madou M, De Sutter B, De Bus B, De Bosschere K, Preneel B (2007) Program obfuscation: a quantitative approach. In: Proceedings of ACM workshop on quality of protection, pp 15–20 Anckaert B, Madou M, De Sutter B, De Bus B, De Bosschere K, Preneel B (2007) Program obfuscation: a quantitative approach. In: Proceedings of ACM workshop on quality of protection, pp 15–20
go back to reference Andrews AA, Ghosh S, Choi EM (2002) A model for understanding software components. In: 18th international conference on software maintenance (ICSM 2002), maintaining distributed heterogeneous systems, 3–6 October 2002, Montreal, Quebec, Canada, p 359 Andrews AA, Ghosh S, Choi EM (2002) A model for understanding software components. In: 18th international conference on software maintenance (ICSM 2002), maintaining distributed heterogeneous systems, 3–6 October 2002, Montreal, Quebec, Canada, p 359
go back to reference Armknecht F, Sadeghi AR, Schulz S, Wachsmann C (2013) A security framework for the analysis and design of software attestation. In: Proceedings of the 2013 ACM SIGSAC conference on computer & communications security, CCS ’13. ACM, New York, USA, pp 1–12. https://doi.org/10.1145/2508859.2516650 Armknecht F, Sadeghi AR, Schulz S, Wachsmann C (2013) A security framework for the analysis and design of software attestation. In: Proceedings of the 2013 ACM SIGSAC conference on computer & communications security, CCS ’13. ACM, New York, USA, pp 1–12. https://​doi.​org/​10.​1145/​2508859.​2516650
go back to reference Barak B, Goldreich O, Impagliazzo R, Rudich S, Sahai A, Vadhan S, Yang K (2001) On the (im) possibility of obfuscating programs. Lect Notes Comput Sci 2139:19–23MathSciNetCrossRefMATH Barak B, Goldreich O, Impagliazzo R, Rudich S, Sahai A, Vadhan S, Yang K (2001) On the (im) possibility of obfuscating programs. Lect Notes Comput Sci 2139:19–23MathSciNetCrossRefMATH
go back to reference Burkhardt J, Détienne F, Wiedenbeck S (2002) Object-oriented program comprehension: effect of expertise, task and phase. Empir Softw Eng 7(2):115–156CrossRefMATH Burkhardt J, Détienne F, Wiedenbeck S (2002) Object-oriented program comprehension: effect of expertise, task and phase. Empir Softw Eng 7(2):115–156CrossRefMATH
go back to reference Cabutto A, Falcarin P, Abrath B, Coppens B, De Sutter B (2015) Software protection with code mobility. In: Proceedings of the second ACM workshop on moving target defense (MTD), pp 95–103 Cabutto A, Falcarin P, Abrath B, Coppens B, De Sutter B (2015) Software protection with code mobility. In: Proceedings of the second ACM workshop on moving target defense (MTD), pp 95–103
go back to reference Capiluppi A, Falcarin P, Boldyreff C (2012) Code defactoring: evaluating the effectiveness of java obfuscations. In: 2012 19th working conference on reverse engineering (WCRE). IEEE, pp 71–80 Capiluppi A, Falcarin P, Boldyreff C (2012) Code defactoring: evaluating the effectiveness of java obfuscations. In: 2012 19th working conference on reverse engineering (WCRE). IEEE, pp 71–80
go back to reference Ceccato M, Capiluppi A, Falcarin P, Boldyreff C (2015) A large study on the effect of code obfuscation on the quality of java code. Empir Softw Eng 20(6):1486–1524CrossRef Ceccato M, Capiluppi A, Falcarin P, Boldyreff C (2015) A large study on the effect of code obfuscation on the quality of java code. Empir Softw Eng 20(6):1486–1524CrossRef
go back to reference Ceccato M, Di Penta M, Falcarin P, Ricca F, Torchiano M, Tonella P (2014) A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empir Softw Eng 19(4):1040–1074 Ceccato M, Di Penta M, Falcarin P, Ricca F, Torchiano M, Tonella P (2014) A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empir Softw Eng 19(4):1040–1074
go back to reference Ceccato M, Tonella P, Basile C, Coppens B, De Sutter B, Falcarin P, Torchiano M (2017) How professional hackers understand protected code while performing attack tasks. In: Proceedings of the 25th international conference on program comprehension (ICPC), pp 154–164. ICPC best paper award and ACM Distinguished paper award Ceccato M, Tonella P, Basile C, Coppens B, De Sutter B, Falcarin P, Torchiano M (2017) How professional hackers understand protected code while performing attack tasks. In: Proceedings of the 25th international conference on program comprehension (ICPC), pp 154–164. ICPC best paper award and ACM Distinguished paper award
go back to reference Collberg C, Thomborson C, Low D (1997) A taxonomy of obfuscating transformations. Technical Report 148, Dept. of Computer Science, The Univ. of Auckland Collberg C, Thomborson C, Low D (1997) A taxonomy of obfuscating transformations. Technical Report 148, Dept. of Computer Science, The Univ. of Auckland
go back to reference Collberg C, Nagra J (2009) Surreptitious software: obfuscation, watermarking, and tamperproofing for software protection. Addison-wesley, London Collberg C, Nagra J (2009) Surreptitious software: obfuscation, watermarking, and tamperproofing for software protection. Addison-wesley, London
go back to reference Edmundson A, Holtkamp B, Rivera E, Finifter M, Mettler A, Wagner D (2013) An empirical study on the effectiveness of security code review. In: International symposium on engineering secure software and systems. Springer, pp 197–212 Edmundson A, Holtkamp B, Rivera E, Finifter M, Mettler A, Wagner D (2013) An empirical study on the effectiveness of security code review. In: International symposium on engineering secure software and systems. Springer, pp 197–212
go back to reference Flick U (2009) An introduction to qualitative research, 4th edn. Sage, London, England, UK Flick U (2009) An introduction to qualitative research, 4th edn. Sage, London, England, UK
go back to reference Glaser BG, Strauss AL (1967) The discovery of grounded theory. Aldine, Chicago, IL, USA Glaser BG, Strauss AL (1967) The discovery of grounded theory. Aldine, Chicago, IL, USA
go back to reference Katipally R, Yang L, Liu A (2011) Attacker behavior analysis in multi-stage attack detection system. In: Proceedings of the seventh annual workshop on cyber security and information intelligence research, CSIIRW ’11. ACM, New York, pp 63:1–63:1. https://doi.org/10.1145/2179298.2179369 Katipally R, Yang L, Liu A (2011) Attacker behavior analysis in multi-stage attack detection system. In: Proceedings of the seventh annual workshop on cyber security and information intelligence research, CSIIRW ’11. ACM, New York, pp 63:1–63:1. https://​doi.​org/​10.​1145/​2179298.​2179369
go back to reference Letovsky S (1987) Cognitive processes in program comprehension. J Syst Softw 7(4):325–339CrossRef Letovsky S (1987) Cognitive processes in program comprehension. J Syst Softw 7(4):325–339CrossRef
go back to reference Linn C, Debray S (2003) Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of ACM conference on computer and communications security, pp 290–299 Linn C, Debray S (2003) Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of ACM conference on computer and communications security, pp 290–299
go back to reference Littman DC, Pinto J, Letovsky S, Soloway E (1987) Mental models and software maintenance. J Syst Softw 7(4):341–355CrossRef Littman DC, Pinto J, Letovsky S, Soloway E (1987) Mental models and software maintenance. J Syst Softw 7(4):341–355CrossRef
go back to reference Pennington N (1987) Stimulus structures and mental representations in expert comprehension of computer programs. Cogn Psychol 19(3):295–341CrossRef Pennington N (1987) Stimulus structures and mental representations in expert comprehension of computer programs. Cogn Psychol 19(3):295–341CrossRef
go back to reference Piorkowski DJ, Fleming SD, Kwan I, Burnett MM, Scaffidi C, Bellamy RK, Jordahl J (2013) The whats and hows of programmers’ foraging diets. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 3063–3072 Piorkowski DJ, Fleming SD, Kwan I, Burnett MM, Scaffidi C, Bellamy RK, Jordahl J (2013) The whats and hows of programmers’ foraging diets. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 3063–3072
go back to reference Prechelt L, Schmeisky H, Zieris F (2016) Quality experience: a grounded theory of successful agile projects without dedicated testers. In: Proceedings of the 38th international conference on software engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016, pp 1017–1027 Prechelt L, Schmeisky H, Zieris F (2016) Quality experience: a grounded theory of successful agile projects without dedicated testers. In: Proceedings of the 38th international conference on software engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016, pp 1017–1027
go back to reference Sillito J, Murphy GC, Volder KD (2006) Questions programmers ask during software evolution tasks. In: Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering, FSE, pp 23–34 Sillito J, Murphy GC, Volder KD (2006) Questions programmers ask during software evolution tasks. In: Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering, FSE, pp 23–34
go back to reference Stol K, Ralph P, Fitzgerald B (2016) Grounded theory in software engineering research: a critical review and guidelines. In: Proceedings of the 38th international conference on software engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016, pp 120–131 Stol K, Ralph P, Fitzgerald B (2016) Grounded theory in software engineering research: a critical review and guidelines. In: Proceedings of the 38th international conference on software engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016, pp 120–131
go back to reference Strauss A, Corbin J (1990) Basics of qualitative research: grounded theory procedures and techniques. Sage, London Strauss A, Corbin J (1990) Basics of qualitative research: grounded theory procedures and techniques. Sage, London
go back to reference Sutherland I, Kalb GE, Blyth A, Mulley G (2006) An empirical examination of the reverse engineering process for binary files. Comput Secur 25(3):221–228CrossRef Sutherland I, Kalb GE, Blyth A, Mulley G (2006) An empirical examination of the reverse engineering process for binary files. Comput Secur 25(3):221–228CrossRef
go back to reference Viticchié A, Regano L, Torchiano M, Basile C, Ceccato M, Tonella P, Tiella R (2016) Assessment of source code obfuscation techniques. In: Proceedings of the 16th IEEE international working conference on source code analysis and manipulation, pp 11–20 Viticchié A, Regano L, Torchiano M, Basile C, Ceccato M, Tonella P, Tiella R (2016) Assessment of source code obfuscation techniques. In: Proceedings of the 16th IEEE international working conference on source code analysis and manipulation, pp 11–20
go back to reference von Mayrhauser A, Vans AM (1994) Comprehension processes during large scale maintenance. In: Proceedings of the 16th international conference on software engineering, Sorrento, Italy, May 16–21, pp 39–48 von Mayrhauser A, Vans AM (1994) Comprehension processes during large scale maintenance. In: Proceedings of the 16th international conference on software engineering, Sorrento, Italy, May 16–21, pp 39–48
go back to reference von Mayrhauser A, Vans AM (1995) Industrial experience with an integrated code comprehension model. Softw Eng J 10(5):171–182CrossRef von Mayrhauser A, Vans AM (1995) Industrial experience with an integrated code comprehension model. Softw Eng J 10(5):171–182CrossRef
go back to reference von Mayrhauser A, Vans AM (1996a) Identification of dynamic comprehension processes during large scale maintenance. IEEE Trans Softw Eng 22(6):424–437 von Mayrhauser A, Vans AM (1996a) Identification of dynamic comprehension processes during large scale maintenance. IEEE Trans Softw Eng 22(6):424–437
go back to reference von Mayrhauser A, Vans AM (1996b) On the role of hypotheses during opportunistic understanding while porting large scale code. In: 4th international workshop on program comprehension (WPC ’96), March 29–31, 1996, Berlin, Germany, pp 68–77 von Mayrhauser A, Vans AM (1996b) On the role of hypotheses during opportunistic understanding while porting large scale code. In: 4th international workshop on program comprehension (WPC ’96), March 29–31, 1996, Berlin, Germany, pp 68–77
go back to reference von Mayrhauser A, Vans AM (1997a) Hypothesis-driven understanding processes during corrective maintenance of large scale software. In: 1997 international conference on software maintenance (ICSM ’97), 1–3 October 1997, Bari, Italy, Proceedings, pp 12–20 von Mayrhauser A, Vans AM (1997a) Hypothesis-driven understanding processes during corrective maintenance of large scale software. In: 1997 international conference on software maintenance (ICSM ’97), 1–3 October 1997, Bari, Italy, Proceedings, pp 12–20
go back to reference von Mayrhauser A, Vans AM (1997b) Program understanding needs during corrective maintenance of large scale software. In: 21st intern. computer software and applications conference (COMPSAC’97), 1997, USA, pp 630–637 von Mayrhauser A, Vans AM (1997b) Program understanding needs during corrective maintenance of large scale software. In: 21st intern. computer software and applications conference (COMPSAC’97), 1997, USA, pp 630–637
Metadata
Title
Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challenge
Authors
Mariano Ceccato
Paolo Tonella
Cataldo Basile
Paolo Falcarin
Marco Torchiano
Bart Coppens
Bjorn De Sutter
Publication date
26-05-2018
Publisher
Springer US
Published in
Empirical Software Engineering / Issue 1/2019
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-018-9625-6

Other articles of this Issue 1/2019

Empirical Software Engineering 1/2019 Go to the issue

Premium Partner