通过采样操作数据库生成规则化测试数据库

安装次数: 32,003

依赖项: 0

建议者: 0

安全: 0

星级: 3

关注者: 5

分支: 9

开放问题: 1

类型:项目

1.0.0 2021-02-01 14:17 UTC

This package is not auto-updated.

Last update: 2024-09-15 01:28:25 UTC


README

Build Status

这是一个通用的工具,用于从数据库中提取和清洗所选表以用作测试数据。在json配置文件的控制下,将一个数据库中的表子集复制到另一个数据库。然后可以将后者数据库导出为SQL以用作测试文件。

用法

  • 创建您希望填充的目标数据库。工具只会输出到现有数据库。目标数据库的内容将全部清除。
  • 使用DB服务器配置创建config/credentials.json
  • 创建config/*.db.json文件以定义每个所需数据库的映射
  • 运行bin/dbsampler.php

配置格式

所有配置文件都位于config子目录中。文件不能包含注释,因为JSON格式不支持此功能,但按照惯例,可能的情况下将忽略名为"comment"的字段。

credentials.json

"driver": "pdo_mysql"目前被假定,但将来可能会改变。

MySQL

config/credentials.dist.json

{
  "driver": "pdo_mysql",
  "dbUser": "root",
  "dbPassword": "SOMEPASSWORD",
  "dbHost": "127.0.0.1"
}

如果您需要不同的源和目标服务器,这将变为

{
  "source": {
    "driver": "pdo_mysql",
    "dbUser": "root",
    "dbPassword": "SOMEPASSWORD",
    "dbHost": "sourceDB.example.com"
  },

  "dest" : {
    "driver": "pdo_mysql",
    "dbUser": "root",
    "dbPassword": "SOMEPASSWORD",
    "dbHost": "127.0.0.1"
  }
}

如果您需要准备连接,请添加一个initialSql段落

{
  "source": {
    "driver": "pdo_mysql",
    "dbUser": "root",
    "dbPassword": "SOMEPASSWORD",
    "dbHost": "sourceDB.example.com",
    "initialSql": [
      "SET NAMES UTF8"
    ]
  },
  "dest": {
    "driver": "pdo_mysql",
    "dbUser": "root",
    "dbPassword": "SOMEPASSWORD",
    "dbHost": "127.0.0.1",
    "initialSql": [
      "SET NAMES UTF8",
      "SET foreign_key_checks = 0"
    ]
  }
}        
Sqlite

config/credentials.dist.json

{
    "driver": "pdo_sqlite",
    "directory": "..\/path\/to\/sqlite-dbs"
}

路径假定相对于配置文件,除非它们以'/'开头。要迁移的Sqlite数据库假定是这个目录中的*.sqlite文件

dbname.db.json

{
  "name": "small-sqlite-test",          # Configuration name
  "sourceDb": "small-source",           # Name of the source DB
  "destDb": "small-dest",               # Name of the destination DB. This DB will get trashed
  "tables": {                           # A set of tables to be copied over. Each table is defined as "table": config
                                        # Every config stanza requires a sampler field. For now, look these up in 
                                        # \Quidco\DbSampler\MigrationConfigProcessor::$samplerMap
                                        # All other fields depend on the specific sampler being used; these should 
                                        # all be documented in their own class files in src/Sample
    "fruits": {
      "sampler": "matched",
      "constraints": {
        "name": [
          "apple",
          "pear"
        ]
      },
      "remember": {
        "id": "fruit_ids"               # Cross-referencing is supported by "remember" stanzas
                                        # These take the field name of which the values are to be remembered
                                        # matched to a variable name in which the values will be stored
                                        # Note: Variable declarations do not include a '$' symbol 
                                        # References MUST be 'remember'ed before being used, there is no
      }                                 # dependency resolution here, so order your config appropriately
    },
    "vegetables": {
      "sampler": "NewestById",
      "idField": "id",
      "quantity": 2
    },
    "fruit_x_basket": {
      "sampler": "matched",
      "constraints": {
        "fruit_id": "$fruit_ids"        # Remembered variables, with $ sign, can be used as cross-references
                                        # This will expand to all ids of the fruits table matched above
      },
      "where" : [
        "basket_id > 1"                 # The matched sampler can also accept a list of arbitrary WHERE clauses
      ],
      "remember": {
        "basket_id": "basket_ids"
      }
    },
    "baskets": {
      "sampler": "cleanMatched",        # some samplers support field cleaners that are defined in
                                        # \Quidco\DbSampler\FieldCleanerProvider::getCleanerByName
                                        # They modify or replace the content of the field that they are keyed to
      "constraints": {
        "id": "$basket_ids"
      },
      "cleanFields": {
        "name": "fakefullname"
      }
    }
  },
  "views": [                            # view support is experimental
    "some_view"                         # views are specified as name only but format may change
  ]                                     # The destination's CURRENT_USER() is used as the DEFINER for MySQL DBs
}
"Faker" 清洁器

任何不要求参数的 'faker' (fzaninotto/faker) 生成器都可以通过在cleanFields段落中使用"name": "faker:GENERATOR"直接使用,例如

 "cleanFields": {
   "ip": "faker:ipv4"
 },

扩展项目

此工具设计为通过添加自定义采样器(必须实现\Quidco\DbSampler\SamplerInterface)和清洁器(在\Quidco\DbSampler\FieldCleanerProvider::getCleanerByName中记录)进行扩展。

很可能将提供注册外部清洁器和采样器的机制。

目前,仅支持mysql和sqlite数据库,但这也可以扩展。