README

该工具可以帮助您验证迁移数据的质量。

安装

composer require --dev axelerant/mdtt

用法

基本上，您需要遵循以下步骤

1. 指定测试规范
2. 指定测试定义
3. 可选，指定转换插件
4. 运行测试

您可以在这里找到工具使用的基礎模板。

测试规范

请在tests/mdtt目录内指定测试规范。规范必须写入spec.php文件内。以下是一个示例规范

<?php

return [
    'databases' => [
        // Source database credentials
        'source_db' => [
            'database' => "db_drupal7",
            'username' => "db",
            'password' => "db",
            'host' => "127.0.0.1",
            'port' => "59002",
        ],
        // Destination database credentials
        "destination_db" => [
            'database' => "db",
            'username' => "db",
            'password' => "db",
            'host' => "127.0.0.1",
            'port' => "59002",
        ],
    ],
    'http' => [
        // JSON datasource basic authentication credetials
        'source' => [
            'username' => 'username',
            'password' => 'fubar',
            // Basic authentication protocol: basic, digest, or ntlm.
            // Protocol is optional. If not mentioned, basic is considered.
            'protocol' => 'ntlm',
        ],
        'destination' => [
            'username' => 'username',
            'password' => 'puffball',
        ]
    ]
];

测试定义

测试定义以yaml格式编写，并且必须存在于tests/mdtt目录内。假设您想要验证迁移的用户数据。创建一个名为validate_users.yml的文件。文件名不重要。

数据库

# Test definition ID
id: validate_users
# Test definition description
description: Validates users
# Group to which this test belongs to
group: migration_validation
# The query used for generating the source dataset
source:
  type: database
  data: "select * from users"
  # The source database credentials
  database: source_db
# The query used for generating the destination dataset
destination:
  type: database
  data: "select * from users_field_data"
  # The destination database credentials
  database: destination_db
# Test cases
tests:
  -
    # Field from source datasource used in this comparison
    sourceField: name
    # Field from destination datasource used in this comparison
    destinationField: name
  -
    sourceField: mail
    destinationField: mail

JSON

假设源旧系统以以下格式公开数据

{
      "jsonapi": {
        "version": 1.2,
      },
      "results": [
        {
          "employee": {
              "id": 3453,
              "info": {
                "name": "Aloy",
                "email": "aloy@nora.tribe",
                "gender": "female",
                "status": "active"
              }
          }
        },
        {
          "employee": {
              "id": 3452,
              "info": {
                "name": "Varl",
                "email": "varl@nora.tribe",
                "gender": "male",
                "status": "active"
              }
          }
        },
        {
          "employee": {
              "id": 3450,
              "info": {
                "name": "Rost",
                "email": "rost@nora.tribe",
                "gender": "male",
                "status": "inactive"
              }
          }
        }
      ]
    }

假设目标新系统以以下格式公开数据

      "jsonapi": {
        "version": 1.2,
      },
      "results": [
        {
            "id": 3453,
            "name": "Aloy",
            "email": "aloy@nora.tribe",
            "gender": "female",
            "status": "active"
        },
        {
            "id": 3452,
            "name": "Varl",
            "email": "varl@nora.tribe",
            "gender": "male",
            "status": "active"
        },
        {
            "id": 3450,
            "name": "Rost",
            "email": "rost@nora.tribe",
            "gender": "male",
            "status": "active"
        }
      ]
    }

id: validate_public_apis
description: Validate public apis
group: migration_validation
# The endpoint that returns the source dataset.
source:
  type: json
  data: https://dev.legacy-system.com/api/v1/users
  # The pointer where the list of items resides. Refer https://github.com/halaxa/json-machine#what-is-json-pointer-anyway for examples
  selector: "/results"
  # Datasource basic authentication credentials. Note that the key is same as mentioned in the test specification. This is optional if the endpoint is publicly accessible.
  credential: source
# The endpoint that returns the destination dataset.
destination:
  type: json
  data: https://dev.new-system.com/api/v1/users
  selector: "/results"
  credential: destination
tests:
  -
    # `/` separated field which you want to test.
    # This is applicable when the value you want to test is in nested format.
    sourceField: "employee/info/name"
    destinationField: name
  -
    sourceField: "employee/info/email"
    destinationField: email

转换插件

可能存在这样的情况，数据不是直接从源存储，而是在存储到目标数据库之前必须以某种方式转换（例如，删除空白字符）。QA工程师可以编写自己的插件来验证执行转换的业务逻辑。

测试用例看起来像这样

tests:
  -
    sourceField: name
    transform: trim
    destinationField: name

QA工程师必须在tests/mdtt/src/Plugin/Transform内指定插件类。文件名（以及类名）必须与测试用例中提到的插件名相同，首字母大写，即Trim。插件类必须实现\Mdtt\Transform\Transform接口。

<?php

// File location: tests/mdtt/src/Plugin/Transform/Trim.php.

class Trim implements \Mdtt\Transform\Transform
{
    /**
     * @inheritDoc
     */
    public function name(): string
    {
        // Specifies the plugin name.
        // This must be unique, if you have multiple plugin definitions.
        // The same plugin name must be mentioned in the test case.
        return "trim";
    }

    /**
     * @inheritDoc
     */
    public function process(mixed $data): mixed
    {
        // The logic that does the transformation.
        return trim($data);
    }
}

运行测试

./vendor/bin/mdtt run

详细模式

./vendor/bin/mdtt run -vvv

您可以通过这样做来查看所有可用的选项

./vendor/bin/mdtt run --help

指定测试完成后将发送通知的电子邮件地址

./vendor/bin/mdtt run --email foo@bar.mail

日志

默认情况下，日志文件将创建在名为logs的目录内。该目录将位于测试套件的根目录下。日志文件的命名基于执行测试时的日期和时间。

功能

支持的数据源类型

• 数据库（MySQL）
• JSON

支持的目标类型

• 数据库（MySQL）
• JSON

在测试完成后通知的渠道类型

• 电子邮件

贡献

将该仓库分叉并进行更改。

运行测试

composer test