RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop web APIs using the REpresentational State Transfer architectural style. Black-box testing, which assumes only the access to the system under test with a specific interface, is the only viable option when white-box testing is impracticable. This is the case for REST APIs: their source code is usually not (or just partially) available, or a white-box analysis across many dynamically allocated distributed components (typical of a micro-services architecture) is computationally challenging. This paper presents RestTestGen, a novel black-box approach to automatically generate test cases for REST APIs, based on their interface definition (an OpenAPI specification). Input values and requests are generated for each operation of the API under test with the twofold objective of testing nominal execution scenarios and error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. While this approach is mainly targeting the research community, it is also of interest to developers because, as a black-box approach, it is universally applicable across different programming languages, or in the case external (compiled only) libraries are used in a REST API. The validation of our approach has been performed on more than 100 of real-world REST APIs, highlighting the effectiveness of the approach in revealing actual faults in already deployed services.

Automated black-box testing of nominal and error scenarios in RESTful APIs

Corradini, D.;Zampieri, A.;Pasqua, M.;Ceccato, M.
2022-01-01

Abstract

RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop web APIs using the REpresentational State Transfer architectural style. Black-box testing, which assumes only the access to the system under test with a specific interface, is the only viable option when white-box testing is impracticable. This is the case for REST APIs: their source code is usually not (or just partially) available, or a white-box analysis across many dynamically allocated distributed components (typical of a micro-services architecture) is computationally challenging. This paper presents RestTestGen, a novel black-box approach to automatically generate test cases for REST APIs, based on their interface definition (an OpenAPI specification). Input values and requests are generated for each operation of the API under test with the twofold objective of testing nominal execution scenarios and error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. While this approach is mainly targeting the research community, it is also of interest to developers because, as a black-box approach, it is universally applicable across different programming languages, or in the case external (compiled only) libraries are used in a REST API. The validation of our approach has been performed on more than 100 of real-world REST APIs, highlighting the effectiveness of the approach in revealing actual faults in already deployed services.
2022
automatic test case generation
black-box testing
REST APIs
test oracle
REST API, Test case generation, Black-box testing
File in questo prodotto:
File Dimensione Formato  
stvr2022.pdf

solo utenti autorizzati

Tipologia: Documento in Pre-print
Licenza: Accesso ristretto
Dimensione 2.98 MB
Formato Adobe PDF
2.98 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
main.pdf

solo utenti autorizzati

Tipologia: Documento in Pre-print
Licenza: Accesso ristretto
Dimensione 463.8 kB
Formato Adobe PDF
463.8 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11562/1107406
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 23
  • ???jsp.display-item.citation.isi??? 21
social impact