moox/sync

这是我的数据同步包

资助包维护!
mooxphp

0.0.4 2024-09-09 14:55 UTC

This package is auto-updated.

Last update: 2024-09-20 11:38:25 UTC


README

Moox Sync

Moox Sync

Moox Sync 是一个强大的包,用于在 Laravel 应用程序中同步多个平台上的数据。它允许您在不同的 Moox 平台或其他 Filament 和 Laravel 平台之间保持记录同步。它设计得易于使用和配置,重点在于安全性(用于同步用户)和灵活性。

moox-sync-demo.mp4

快速安装

安装此包只需要这两个命令

composer require moox/sync
php artisan mooxsync:install

好奇安装命令的作用?请见下面的手动安装。

创建平台

首先,您需要创建一个平台,最好是两个。您将能够在同一平台上同步(但当然不同模型),但这不是同步的主要思想。

所以,让我们创建平台。

Create Sync Platform

一个平台应该有一个名称,一个域名(我们将为您查找 IP),以及一个令牌,可以通过按钮生成。

List Sync Platforms

现在平台已创建,我们可以在列表中看到它们,并使用它们进行第一次同步。

创建同步

创建平台后,您可以在平台之间创建同步。

Create Syncs like this

选择源平台和模型,目标平台和模型,并启用平台关系(如果模型支持),见 平台关系

List your Syncs

现在您可以在列表中看到您的同步。

关键概念

理解 Moox Sync 不是一个简单地将数据从 A 复制到 B 的双向同步至关重要。它是从源到目标的单向同步,但它被设计成集中管理的,它是智能的、可扩展的和安全的。

SyncPlattformsJob

平台在所有平台之间自动同步。第一次使用基本的仅使用配置中的 Sync Token 的安全性完成,之后添加平台令牌以提供第一级安全性,包括 HMAC。

SyncPlatformsJob 可以通过配置或 .env 启用。这应该在不超过一个平台上运行。

作为定期(默认为每分钟)运行作业的替代方案,您可以通过在更新平台后按“同步平台”按钮手动启动作业。

SyncListener

SyncListener 为每个同步运行一个 Eloquent 事件监听器,并捕获所有使用 Eloquent 完成的模型更改。如果您在模型中更改数据而不使用 Eloquent,您可能需要考虑触发一个事件。

Event::dispatch('eloquent.updated: '.get_class($this->record), $this->record);

注意:对于处理 Laravel 之外导入或更改,我们建议运行 SyncBackupJob

SyncListener 运行 PrepareSynJob

SyncListener 需要在 Moox Sync 配置中激活。这应该在源平台上完成。

PrepareSyncJob

PrepareSyncJob 由 SyncListener 调用。它准备数据并触发 SyncWebhook

PrepareSyncJob 支持自定义数据处理,如自定义查询、数据操作、所谓的转换器,也作为转换器绑定提供,在 Moox Sync 配置中配置。见 高级使用

SyncWebhook

在目标平台作为入口点,通过 PrepareSyncJob 接收来自源平台的数据。使用 HMAC 验证传入的数据,并检查同步配置中指定的任何转换或字段映射要求。

触发使用验证和转换后的数据的SyncJob

需要在Moox Sync配置中激活SyncWebhook。它只能在目标平台上启用。

SyncJob

SyncJob使用SyncService来排队实际的同步操作。

SyncService

Sync在目标平台上写入数据。它支持Custom SyncHandlerPlatformRelations。参见高级用法

高级用法

Transformer

Transformers尚未实现。我们建议使用Transformer Bindings。

Transformer Bindings

虽然(在Sync中可选)Transformer功能尚未完全实现,但您可以使用Transformer Bindings来执行自定义数据转换逻辑。

创建自定义转换器

  1. 创建一个扩展AbstractTransformer的新类。
  2. 实现transformCustomFields方法。
  3. transformer_bindings配置中注册您的转换器。

参见Moox Press中的WpUserTransformer了解如何实现Transformer绑定。

SyncHandler

对于需要自定义同步逻辑的复杂模型,实现Sync Handlers。

  1. 创建一个扩展AbstractSyncHandler的新类。
  2. 实现syncModeldeleteModel方法。
  3. sync_bindings配置中注册您的同步处理器。

参见Moox Press中的WpUserSyncHandler了解如何实现自己的SyncHandler。

PlatformRelations

PlatformRelationService是一个可选功能,您可以为每个同步设置它。它是Moox Sync的关键组件,负责处理模型和平台之间的关系。它为任何模型提供了同步和检索平台关联的方法。

关键方法

  • syncPlatformsForModel($model, array $platformIds):同步给定模型的平台。
  • getPlatformsForModel($model):检索与给定模型关联的平台。

如何实现PlatformRelations

要实现PlatformRelations,您需要在Moox Sync配置中配置models_with_platform_relations。这是一个应该有平台关系的模型列表。

虽然关系会自动添加到模型中,但您可能想要在您的Filament资源中添加一个字段来管理给定记录的平台。

                    Select::make('platforms')
                        ->label('Platforms')
                        ->multiple()
                        ->options(function () {
                            return \Moox\Sync\Models\Platform::pluck('name', 'id')->toArray();
                        })
                        ->afterStateHydrated(function ($component, $state, $record) {
                            if ($record && class_exists('\Moox\Sync\Services\PlatformRelationService')) {
                                $platformService = app(\Moox\Sync\Services\PlatformRelationService::class);
                                $platforms = $platformService->getPlatformsForModel($record);
                                $component->state($platforms->pluck('id')->toArray());
                            }
                        })
                        ->dehydrated(false)
                        ->reactive()
                        ->afterStateUpdated(function ($state, callable $set, $record) {
                            if ($record && class_exists('\Moox\Sync\Services\PlatformRelationService')) {
                                $platformService = app(\Moox\Sync\Services\PlatformRelationService::class);
                                $platformService->syncPlatformsForModel($record, $state ?? []);
                            }
                        })
                        ->preload()
                        ->searchable()
                        ->visible(fn () => class_exists('\Moox\Sync\Models\Platform'))
                        ->columnSpan([
                            'default' => 12,
                            'md' => 12,
                            'lg' => 12,
                        ]),

SyncBackupJob

尚未实现!

如果Eloquent监听器被禁用或未调用,SyncBackupJob将作为后备运行。它检查数据库中的更改并将它们同步到目标平台。

安全

Moox Sync实现了强大的安全措施

Webhook身份验证

WebhookAuthMiddleware处理此身份验证和验证过程。

  • API令牌通过共享密钥对源平台进行身份验证,因此可以在配置中更改,即使是最初的平台同步也是安全的。
  • 平台同步后,共享密钥和平台令牌一起使用以提供额外的安全性。
  • HMAC签名验证有效负载的完整性。
  • HTTPS(您应该强制执行)可防止中间人攻击。

此外,您还可以更改webhook URL,使猜测webhook URL变得困难。

令牌身份验证

对于API端点,Moox Sync通过PlatformTokenAuthMiddleware使用基于令牌的身份验证。或者,您可以使用Sanctum来对您的API请求进行身份验证。

配置

Moox Sync可以通过sync.php配置文件进行高度配置。以下是可以用的选项

/*
|--------------------------------------------------------------------------
| Moox Configuration
|--------------------------------------------------------------------------
|
| This configuration file uses translatable strings. If you want to
| translate the strings, you can do so in the language files
| published from moox_core. Example:
|
| 'trans//core::core.all',
| loads from common.php
| outputs 'All'
|
*/

return [

    /*
    |--------------------------------------------------------------------------
    | Resources
    |--------------------------------------------------------------------------
    |
    | The following configuration is done per Filament resource.
    |
    */

    'resources' => [
        'sync' => [

            /*
            |--------------------------------------------------------------------------
            | Title
            |--------------------------------------------------------------------------
            |
            | The translatable title of the Resource in singular and plural.
            |
            */

            'single' => 'trans//core::sync.sync',
            'plural' => 'trans//core::sync.syncs',

            /*
            |--------------------------------------------------------------------------
            | Tabs
            |--------------------------------------------------------------------------
            |
            | Define the tabs for the Resource table. They are optional, but
            | pretty awesome to filter the table by certain values.
            | You may simply do a 'tabs' => [], to disable them.
            |
            */

            'tabs' => [
                'all' => [
                    'label' => 'trans//core::core.all',
                    'icon' => 'gmdi-filter-list',
                    'query' => [],
                ],
                /*
                'error' => [
                    'label' => 'trans//core::core.error',
                    'icon' => 'gmdi-text-snippet',
                    'query' => [
                        [
                            'field' => 'subject_type',
                            'operator' => '=',
                            'value' => 'Error',
                        ],
                    ],
                ],
                */
            ],
        ],
        'platform' => [

            /*
            |--------------------------------------------------------------------------
            | Title
            |--------------------------------------------------------------------------
            |
            | The translatable title of the Resource in singular and plural.
            |
            */

            'single' => 'trans//core::sync.platform',
            'plural' => 'trans//core::sync.platforms',

            /*
            |--------------------------------------------------------------------------
            | Tabs
            |--------------------------------------------------------------------------
            |
            | Define the tabs for the Resource table. They are optional, but
            | pretty awesome to filter the table by certain values.
            | You may simply do a 'tabs' => [], to disable them.
            |
            */

            'tabs' => [
                'all' => [
                    'label' => 'trans//core::core.all',
                    'icon' => 'gmdi-filter-list',
                    'query' => [],
                ],
                /*
                'error' => [
                    'label' => 'trans//core::core.error',
                    'icon' => 'gmdi-text-snippet',
                    'query' => [
                        [
                            'field' => 'subject_type',
                            'operator' => '=',
                            'value' => 'Error',
                        ],
                    ],
                ],
                */
            ],
        ],
    ],

    /*
    |--------------------------------------------------------------------------
    | Navigation Group
    |--------------------------------------------------------------------------
    |
    | The translatable title of the navigation group in the
    | Filament Admin Panel. Instead of a translatable
    | string, you may also use a simple string.
    |
    */

    'navigation_group' => 'trans//core::core.tools',

    /*
    |--------------------------------------------------------------------------
    | Navigation Sort
    |--------------------------------------------------------------------------
    |
    | This values are the sort order of the navigation items in the
    | Filament Admin Panel. If you use a bunch of Moox
    | plugins, everything should be in order.
    |
    */

    'navigation_sort' => 9500,

    /*
     |--------------------------------------------------------------------------
     | API
     |--------------------------------------------------------------------------
     |
     | Enable or disable the API and configure all entities.
     | Public or secured by platform or sanctum.
     | Available at /api/{entity}
     |
     */

    'entities' => [
        'Sync' => [
            'api' => [
                'enabled' => true,
                'public' => false,
                'auth_type' => 'platform',
                'active_routes' => [
                    'index',
                    'show',
                    'store',
                    'update',
                    'destroy',
                ],
            ],
            'model' => '\Moox\Sync\Models\Sync',
            'resource' => '\Moox\Sync\Resources\SyncResource',
            'api_controller' => '\Moox\Sync\Http\Controllers\Api\SyncController',
        ],
        'Platform' => [
            'api' => [
                'enabled' => true,
                'public' => false,
                'auth_type' => 'platform',
                'active_routes' => [
                    'index',
                    'show',
                    'store',
                    'update',
                    'destroy',
                ],
            ],
            'model' => '\Moox\Sync\Models\Platform',
            'resource' => '\Moox\Sync\Resources\PlatformResource',
            'api_controller' => '\Moox\Sync\Http\Controllers\Api\PlatformController',
        ],
    ],

    /*
    |--------------------------------------------------------------------------
    | Sync Platform Job
    |--------------------------------------------------------------------------
    |
    | Enable or disable the Sync Platform Job that automatically syncs data
    | between all platforms.
    |
    */

    'sync_platform_job' => [
        'enabled' => env('SYNC_PLATFORM_JOB_ENABLED', false),
        'frequency' => 'everyFiveMinutes', // hourly, daily, hourly, etc.
    ],

    /*
    |--------------------------------------------------------------------------
    | Sync Backup Job
    |--------------------------------------------------------------------------
    |
    | Enable or disable the Sync Backup Job that automatically syncs data
    | based on your sync configurations, when changes are made outside
    | of Eloquent events or you've disabled the Eloquent listener.
    |
    */

    'sync_backup_job' => [
        'enabled' => env('SYNC_BACKUP_JOB_ENABLED', false),
        'frequency' => 'everyFiveMinutes', // hourly, daily, hourly, etc.
    ],

    /*
    |--------------------------------------------------------------------------
    | Sync Eloquent Listener
    |--------------------------------------------------------------------------
    |
    | Enable or disable the Eloquent listener that automatically syncs
    | data when a model is created, updated or deleted. Use
    | it wisely together with the Sync Backup Job.
    |
    */

    'sync_eloquent_listener' => [
        'enabled' => env('SYNC_LISTENER_ENABLED', false),
    ],

    /*
    |--------------------------------------------------------------------------
    | Sync Webhook
    |--------------------------------------------------------------------------
    |
    | Enable or disable the webhook that automatically syncs
    | data when a model is created, updated or deleted. Use
    | it wisely together with the Sync Backup Job.
    |
    */

    'sync_webhook' => [
        'enabled' => env('SYNC_WEBHOOK_ENABLED', false),
    ],

    /*
    |--------------------------------------------------------------------------
    | Models with Platform Relations
    |--------------------------------------------------------------------------
    |
    | List of models that should have platform relations. This adds the
    | platforms relation to the model. No need to add a trait or
    | any dependency to the model or the package.
    |
    */

    'models_with_platform_relations' => [
        'App\Models\User',
        'Moox\User\Models\User',
        'Moox\Press\Models\User',
        // Add any other models here
    ],

    /*
    |--------------------------------------------------------------------------
    | // TODO: Models with syncable Relations - not implemented yet
    |--------------------------------------------------------------------------
    |
    | List of models that should have syncable relations, which should be
    | synced to other platforms, when changes are made. This does not
    | add the related models to the listener, but syncs them with
    | the sync model automatically. So platform-related models
    | (like WpUsers and WpUserMetaare able to use this
    | feature, too.
    |
    */

    'models_with_syncable_relations' => [
        'Moox\User\Models\User',
        'Moox\Press\Models\WpUser' => [
            'Moox\UserSession\Models\Session',
        ],
        // Add any other models here
    ],

    /*
    |--------------------------------------------------------------------------
    | Unique Identifier Fields
    |--------------------------------------------------------------------------
    |
    | The synced model should have a unique identifier. The id auto-
    | increments, so it is not suitable. Perfect would be a ULID
    | or UUID, but any other unique identifier will work, too.
    | This is the list of identifiers Moox Sync searches for.
    | for, in the given order. Ad more as you need them.
    |
    */

    'unique_identifier_fields' => [
        'ulid',
        'uuid',
        'slug',
        'name',
        'title',
    ],

    /*
    |--------------------------------------------------------------------------
    | Local Identifier Fields
    |--------------------------------------------------------------------------
    |
    | These are the fields that are used as unique identifiers for
    | the models. They are used to identify the models on the
    | source platform. The array is sorted by priority.
    |
    */

    'local_identifier_fields' => [
        'ID',
        'uuid',
        'ulid',
        'id',
    ],

    /*
    |--------------------------------------------------------------------------
    | Transformer
    |--------------------------------------------------------------------------
    |
    | You can register Transformer Classes here, to make them available
    | when creating Syncs. These classes can contain queries or
    | translation maybe. Alternatively you can bind models.
    |
    */

    // Not implemented yet, use bindings instead
    'transformer_classes' => [
        // Not implemented yet
    ],

    /*
    |--------------------------------------------------------------------------
    | Transformer Bindings
    |--------------------------------------------------------------------------
    |
    | You can register custom Transformer Bindings, used for Press models for
    | example, where we have to read meta data or custom tables like
    | terms and taxonomies instead of native categories.
    |
    */

    'transformer_bindings' => [
        // Add transformer bindings here, like:
        // \Moox\Press\Models\WpUser::class => \Moox\Press\Transformer\WpUserTransformer::class,
    ],

    /*
    |--------------------------------------------------------------------------
    | Sync Bindings
    |--------------------------------------------------------------------------
    |
    | You can register custom Sync Bindings, used for Press models for
    | example, where we have to write meta data or custom tables
    | like terms and taxonomies instead of native categories.
    |
    */

    'sync_bindings' => [
        // Add sync handlers here, like:
        // \Moox\Press\Models\WpUser::class => \Moox\Press\Handlers\WpUserSyncHandler::class,
    ],
];

日志记录

设置同步需要连接两个或多个平台,API可用性以及运行作业。这就是为什么我们在包中添加了日志记录器,可以在Moox 核心配置中进行设置。一个正常同步的流程可能如下所示。根据日志级别,您可以获得关于Moox Sync中数据流的非常详细的信息。在生产环境中,除了0之外的其他任何值都不应该是默认值,但可以完美地用于实现或调试Moox Sync。

手动安装

您可以通过以下步骤手动逐步安装此包,而不是使用安装命令 php artisan mooxsync:install

// Publish and run the migrations:
php artisan vendor:publish --tag="sync-migrations"
php artisan migrate

// Publish the config file with:
php artisan vendor:publish --tag="sync-config"

编辑您的PanelProvider以将这两个插件添加到您的导航中。

变更日志

有关最近更改的更多信息,请参阅变更日志

安全漏洞

有关如何报告安全漏洞,请参阅我们的安全策略

致谢

许可协议

MIT 许可协议 (MIT)。有关更多信息,请参阅许可文件