How To Implement Microversion Tests

Tempest provides stable interfaces to test API Microversion. For Details, see: API Microversion testing Framework This document explains how to implement Microversion tests using those interfaces.

Configuration options for Microversion

  • Add configuration options for specifying test target Microversions. We need to specify test target Microversions because the supported Microversions may be different between OpenStack clouds. For operating multiple Microversion tests in a single Tempest operation, configuration options should represent the range of test target Microversions. New configuration options are:

    • min_microversion
    • max_microversion

    Those should be defined under respective section of each service. For example:

    [compute]
    min_microversion = None
    max_microversion = latest
    

How To Implement Microversion Tests

Step1: Add skip logic based on configured Microversion range

Add logic to skip the tests based on Tests class and configured Microversion range. api_version_utils.check_skip_with_microversion function can be used to automatically skip the tests which do not fall under configured Microversion range. For example:

class BaseTestCase1(api_version_utils.BaseMicroversionTest):

    [..]
@classmethod
def skip_checks(cls):
    super(BaseTestCase1, cls).skip_checks()
    api_version_utils.check_skip_with_microversion(cls.min_microversion,
                                                   cls.max_microversion,
                                                   CONF.compute.min_microversion,
                                                   CONF.compute.max_microversion)

Skip logic can be added in tests base class or any specific test class depends on tests class structure.

Step2: Selected API request microversion

Select appropriate Microversion which needs to be used to send with API request. api_version_utils.select_request_microversion function can be used to select the appropriate Microversion which will be used for API request. For example:

@classmethod
def resource_setup(cls):
    super(BaseTestCase1, cls).resource_setup()
    cls.request_microversion = (
        api_version_utils.select_request_microversion(
            cls.min_microversion,
            CONF.compute.min_microversion))

Step3: Set Microversion on Service Clients

Microversion selected by Test Class in previous step needs to be set on service clients so that APIs can be requested with selected Microversion.

Microversion can be defined as global variable on service clients which can be set using fixture. Also Microversion header name needs to be defined on service clients which should be constant because it is not supposed to be changed by project as per API contract. For example:

COMPUTE_MICROVERSION = None

class BaseClient1(rest_client.RestClient):
    api_microversion_header_name = 'X-OpenStack-Nova-API-Version'

Now test class can set the selected Microversion on required service clients using fixture which can take care of resetting the same once tests is completed. For example:

def setUp(self):
    super(BaseTestCase1, self).setUp()
    self.useFixture(api_microversion_fixture.APIMicroversionFixture(
        self.request_microversion))

Service clients needs to add set Microversion in API request header which can be done by overriding the get_headers() method of rest_client. For example:

COMPUTE_MICROVERSION = None

class BaseClient1(rest_client.RestClient):
    api_microversion_header_name = 'X-OpenStack-Nova-API-Version'

    def get_headers(self):
        headers = super(BaseClient1, self).get_headers()
        if COMPUTE_MICROVERSION:
            headers[self.api_microversion_header_name] = COMPUTE_MICROVERSION
        return headers

Step4: Separate Test classes for each Microversion

This is last step to implement Microversion test class.

For any Microversion tests, basically we need to implement a separate test class. In addition, each test class defines its Microversion range with class variable like min_microversion and max_microversion. Tests will be valid for that defined range. If that range is out of configured Microversion range then, test will be skipped.

Note

Microversion testing is supported at test class level not at individual test case level.

For example:

Below test is applicable for Microversion from 2.2 till 2.9:

class BaseTestCase1(api_version_utils.BaseMicroversionTest,
                    tempest.test.BaseTestCase):

    [..]


class Test1(BaseTestCase1):
    min_microversion = '2.2'
    max_microversion = '2.9'

    [..]

Below test is applicable for Microversion from 2.10 till latest:

class Test2(BaseTestCase1):
    min_microversion = '2.10'
    max_microversion = 'latest'

    [..]

Notes about Compute Microversion Tests

Some of the compute Microversion tests have been already implemented with the Microversion testing framework. So for further tests only step 4 is needed.

Along with that JSON response schema might need versioning if needed.

Compute service clients strictly validate the response against defined JSON schema and does not allow additional elements in response. So if that Microversion changed the API response then schema needs to be versioned. New JSON schema file needs to be defined with new response attributes and service client methods will select the schema based on requested microversion.

If Microversion tests are implemented randomly meaning not in sequence order(v2.20 tests added and previous Microversion tests are not yet added) then, still schema might need to be version for older Microversion if they changed the response. This is because Nova Microversion includes all the previous Microversions behavior.

For Example:
Implementing the v2.20 Microversion tests before v2.9 and 2.19- v2.20 API request will respond as latest behavior of Nova till v2.20, and in v2.9 and 2.19, server response has been changed so response schema needs to be versioned accordingly.

That can be done by using the get_schema method in below module:

The base_compute_client module

class BaseComputeClient(auth_provider, service, region, endpoint_type='publicURL', build_interval=1, build_timeout=60, disable_ssl_certificate_validation=False, ca_certs=None, trace_requests='', name=None)[source]

Base compute service clients class to support microversion.

This class adds microversion to API request header if that is set and provides interface to select appropriate JSON schema file for response validation.

Parameters:
  • auth_provider – An auth provider object used to wrap requests in auth
  • service (str) – The service name to use for the catalog lookup
  • region (str) – The region to use for the catalog lookup
  • kwargs – kwargs required by rest_client.RestClient
get_headers()[source]

Return the default headers which will be used with outgoing requests

Parameters:
  • accept_type (str) – The media type to use for the Accept header, if one isn’t provided the object var TYPE will be used
  • send_type (str) – The media-type to use for the Content-Type header, if one isn’t provided the object var TYPE will be used
Return type:

dict

Returns:

The dictionary of headers which can be used in the headers dict for outgoing request

get_schema(schema_versions_info)[source]

Get JSON schema

This method provides the matching schema for requested microversion.

Parameters:schema_versions_info – List of dict which provides schema information with range of valid versions.

Example:

schema_versions_info = [
    {'min': None, 'max': '2.1', 'schema': schemav21},
    {'min': '2.2', 'max': '2.9', 'schema': schemav22},
    {'min': '2.10', 'max': None, 'schema': schemav210}]
request(method, url, extra_headers=False, headers=None, body=None, chunked=False)[source]

Send a HTTP request with keystone auth and using the catalog

This method will send an HTTP request using keystone auth in the headers and the catalog to determine the endpoint to use for the baseurl to send the request to. Additionally

When a response is received it will check it to see if an error response was received. If it was an exception will be raised to enable it to be handled quickly.

This method will also handle rate-limiting, if a 413 response code is received it will retry the request after waiting the ‘retry-after’ duration from the header.

Parameters:
  • method (str) – The HTTP verb to use for the request
  • url (str) – Relative url to send the request to
  • extra_headers (bool) – Boolean value than indicates if the headers returned by the get_headers() method are to be used but additional headers are needed in the request pass them in as a dict.
  • headers (dict) – Headers to use for the request if none are specifed the headers returned from the get_headers() method are used. If the request explicitly requires no headers use an empty dict.
  • body (str) – Body to send with the request
  • chunked (bool) – sends the body with chunked encoding
Return type:

tuple

Returns:

a tuple with the first entry containing the response headers and the second the response body

Raises:
  • UnexpectedContentType – If the content-type of the response isn’t an expect type
  • Unauthorized – If a 401 response code is received
  • Forbidden – If a 403 response code is received
  • NotFound – If a 404 response code is received
  • BadRequest – If a 400 response code is received
  • Gone – If a 410 response code is received
  • Conflict – If a 409 response code is received
  • OverLimit – If a 413 response code is received and over_limit is not in the response body
  • RateLimitExceeded – If a 413 response code is received and over_limit is in the response body
  • InvalidContentType – If a 415 response code is received
  • UnprocessableEntity – If a 422 response code is received
  • InvalidHTTPResponseBody – The response body wasn’t valid JSON and couldn’t be parsed
  • NotImplemented – If a 501 response code is received
  • ServerFault – If a 500 response code is received
  • UnexpectedResponseCode – If a response code above 400 is received and it doesn’t fall into any of the handled checks

Microversion tests implemented in Tempest

  • Compute