Introduction to test framework of autorest.python

Overview

autorest.python is a tool to generate SDK code with swagger. The test framework does not test autorest.python directly, but test the code generated by it to check whether it works well. So the steps are: generate code then test the generated code.

Generate code

There is built-in cmd inv regenerate(for more info, see regenerate) to generate code. There are several folders under test folder: azure, vanilla, dpg and multiapi. Each folder contains code generated with different swaggers. And there are several subfolders under them: legacy, version-tolerant and llc. Each subfolder is generated with same swaggers but with different flags. In short:

• legacy: the generated code has operations and models
• version-tolerant: the generated code has operations and no models
• llc: the generated code has no operations and no modes but with build functions

Test generated code

There are details doc to introduce how to test the code. Here will give more details about how it works.

When run pytest under related test folder, the test framework will start autorest.testserver(for more info, see autorest.testserver) process automatically. The process will interact with testcase and record result in xxx-report.json.

def testserver():
    """Start the Autorest testserver."""
    server = start_server_process()
    yield
    terminate_server_process(server)

CI will run all the test and save the result in storage account. Here is the coverage calculated with those result data: http://azure.github.io/autorest/dashboard.html