Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
034052b
feat(index): Rebuild 并发锁,防止并发 Rebuild 导致数据丢失
ykan821 Jun 8, 2026
5e684c7
fix(index): clone query 后 try-finally 恢复,count() 校验响应
ykan821 Jun 11, 2026
1c81f57
feat(dsl): ClausesSupport 统一追加语义
ykan821 Jun 11, 2026
94cff98
fix(index): 新增 totalRelation(),aggregations() 返回 null
ykan821 Jun 12, 2026
f956cab
fix(index): Rebuild::rollback() 遍历全部别名,Index::name() 空验证
ykan821 Jun 12, 2026
4de6a7c
refactor(dsl): Node/Query 构造函数拆分为 from* 方法,更新待办
ykan821 Jun 12, 2026
ed988b7
feat(index): Bulk/Rebuild onError 替代 skipErrors,删 rebuild.import.fail…
ykan821 Jun 12, 2026
56b1672
refactor(index): AggregationShortcut 重命名为 StatsSupport,新增 stats() 方法
ykan821 Jun 12, 2026
9bcb741
refactor(index): 从 Index 拆出 ClientManager、EventDispatcher、Pagination
ykan821 Jun 12, 2026
2caae25
refactor(index): 移除 insert/deprecated 代理,新增 on/newQuery/newDoc 实例入口
ykan821 Jun 12, 2026
6b50541
docs: 更新待办——标记已完成项,新增 PHP 8 现代化、Rebuild 异常处理待办
ykan821 Jun 12, 2026
189e0f2
refactor(dsl): Shared/ 重命名为 Support/
ykan821 Jun 12, 2026
27547e6
refactor(index): PHP 8 现代化(Index 层)
ykan821 Jun 13, 2026
933240c
docs: 移除多余文件并清理 README 7.x 内容
ykan821 Jun 13, 2026
3809dce
refactor(dsl): PHP 8 现代化与冗余清理(DSL 层)
ykan821 Jun 15, 2026
1247afb
refactor(dsl)!: 统一 Bucket 聚合命名并简化 Node 取值机制
ykan821 Jun 15, 2026
ab4d70a
refactor(dsl): 统一 leaf setter 参数名为 $value
ykan821 Jun 18, 2026
f7cb826
feat(query): 新增 intervals regexp 规则
ykan821 Jun 18, 2026
63b81fa
refactor(dsl): 统一 non-leaf 层参数命名与类型
ykan821 Jun 18, 2026
e622ed2
refactor(index): 统一命名参数并补 Rebuild::source 类型
ykan821 Jun 19, 2026
c6c77db
refactor(index)!: 拆分 chunk/cursor 遍历 API
ykan821 Jun 19, 2026
ec3380f
refactor(index): cursor 改用 yield from 委托迭代
ykan821 Jun 19, 2026
bc5e4d8
refactor(index)!: 收紧 Rebuild 异常处理
ykan821 Jun 23, 2026
02eadd2
chore: 升级 phpunit.xml 配置,更新待办进度
ykan821 Jun 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ Thumbs.db
.env
.php-cs-fixer.cache
.phpunit.result.cache
.phpunit.cache/
3 changes: 2 additions & 1 deletion .php-cs-fixer.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@
'@PSR12' => true,
'no_unused_imports' => true,
])
->setFinder($finder);
->setFinder($finder)
->setUnsupportedPhpVersionAllowed(true);
26 changes: 26 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,28 @@ Scope 可选:dsl / index / agg / query / docs。Breaking change 加 `!` 后缀

PSR-5 规范。

## 待办

- [x] **clone query 后 try-finally 恢复**:first() / paginate() 等方法 clone query 后修改状态,需 try-finally 确保恢复
- [x] **ClausesSupport API 统一**:must()/should()/filter() 等改为追加语义,移除 addXXX() 方法
- [x] **hasMore() 确认无 bug**:scroll 场景 !empty(hits) 正确,分页应用 page()<lastPage()
- [x] **Rebuild::rollback() 支持多索引别名**:遍历全部 backing index 移除别名,对齐 doRun() 写法
- [x] **Index::$name 空验证**:name() 未设置时抛异常
- [x] **拆分静态状态**:ClientManager / EventDispatcher / Pagination 从 Index 拆出,删 deprecated 代理
- [x] **移除 Index::insert()**:等价功能由 Doc::save() 覆盖
- [x] **实例入口**:新增 on()/newQuery()/newDoc()/setConnection()/getConnection(),query()/doc() 委托实例方法
- [x] **测试 mock 污染修复**:所有测试文件补 tearDown 清理静态状态
- [x] **命名参数一致性**:公开方法参数名是 API 的一部分,全库审查确保命名统一(如 connection/name/client 不混用)
- [x] **PHP 8 现代化(Index 层)**:全 12 文件 strict_types + 构造器提升 + readonly + 属性/返回/参数类型 + 联合类型;callable 属性(resolver / errorHandler / dataSource)因 PHP 禁止 callable 作属性类型,保留 docblock
- [x] **PHP 8 现代化(DSL 层)**:Node/Query/Agg + 122 leaf 类全部完成。strict_types 全 151 文件;4 原子属性类型同步全 leaf;leaf 参数类型对照 ES docblock 完成;`$_properties` 三模式统一为 `?array`(新增 `$_raw` 承接整体透传,null 保留为合法空态);`toJson` 加 `:string` + false 检查;`toArray` 因多态返回(array/stdClass/null)不强加 PHP 返回类型
- [x] **Rebuild 异常处理**:run() 分离 try-catch(失败优先抛原始异常,成功后释放锁失败抛 RuntimeException 说明 rebuild 已完成);releaseLock 只吞 404;forceUnlock 独立实现吞 404;isLocked 只在 404 返回 false、其他 ES 错误传播
- [x] **$client 抽到 Registry 类**:拆为 ClientManager / EventDispatcher / Pagination,Index 不再持有静态状态
- [x] **Node 构造函数重构**:拆分为 fromKeyValue/fromClosure/fromArrayField/fromScalar
- [x] **Bulk/Rebuild onError 设计**:Bulk 加 onError(callback) 默认 throw,Rebuild 删 skipErrors 加 onError,删 rebuild.import.failed 事件
- [ ] **补核心路径的边界测试**:scroll、bulk 分批、rebuild 失败回滚
- [ ] **搭建集成测试基建**:`ELASTICKIT_TEST_HOST` 驱动,随机索引名隔离
- [x] **cursor/chunk API 重构**:`chunk($duration): Generator<Results>` 按批 yield Results;`cursor($duration): Generator` 逐条 yield 完整 hit(_id/_score/_source,yield from chunk 的 hits、复用其 finally clear);保留 `scroll()/next()/clear()` 作低层原语。底层未来可换 PIT+search_after(上层签名不变)

## 测试

测试在 Docker 容器中运行,需要设置以下环境变量:
Expand All @@ -45,10 +67,14 @@ PSR-5 规范。
| `PHP_CONTAINER` | Docker 容器名 |
| `PROJECT_PATH` | 项目在容器内的路径 |
| `PROXY_PORT` | HTTP 代理端口(推送用) |
| `ELASTICKIT_TEST_HOST` | ES 集成测试地址(如 `https://localhost:9200`),不设置则跳过集成测试 |

## 推送代码前需要执行4件套

```bash
docker exec $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpunit --testsuite unit"
docker exec $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpunit --testsuite index"
docker exec -e ELASTICKIT_TEST_HOST=https://elasticsearch:9200 $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpunit --testsuite integration"
docker exec $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpunit"
docker exec $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpstan analyse"
docker exec $PHP_CONTAINER sh -c "cd $PROJECT_PATH && vendor/bin/phpmd src text phpmd.xml"
Expand Down
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ PHP Elasticsearch DSL 查询构建库,覆盖查询、聚合、CRUD、批量写
composer require ykan/elastickit:^8
```

> 需要 PHP 8.1+、Elasticsearch 8.x。依赖 `elasticsearch-php` 自动安装。ES 7.x 用户见 [7.x 分支](https://github.com/ykan821/ElasticKit/tree/7.x)。
> 需要 PHP 8.1+、Elasticsearch 8.x。依赖 `elasticsearch-php` 自动安装。

## 快速开始

Expand All @@ -27,8 +27,8 @@ Index::setClient($client);
// 2. 定义索引
class ProductIndex extends Index
{
protected $name = 'products';
protected $mappings = [
protected string $name = 'products';
protected array $mappings = [
'properties' => [
'title' => ['type' => 'text'],
'price' => ['type' => 'float'],
Expand Down Expand Up @@ -79,7 +79,7 @@ $bool = Boolean::create()

// 增量构建
if ($filterByPrice) {
$bool->addFilter(Range::create('price', [10, 100]));
$bool->filter(Range::create('price', [10, 100]));
}

$query = Query::create($bool);
Expand Down Expand Up @@ -178,18 +178,24 @@ $results->lastPage();
$results->items();
$results->toPaginator(); // 转为框架分页器(需注册 Paginator Resolver)

// 游标遍历(大批量导出)
foreach (ProductIndex::query()->cursor() as $batch) {
foreach ($batch->docs() as $doc) {
// 分批遍历(大批量导出/批处理,每次 yield 一个 Results
foreach (ProductIndex::query()->chunk() as $results) {
foreach ($results->docs() as $doc) {
// ...
}
}

// 逐条遍历(导出/逐条加工,每次 yield 一个 hit:_id/_score/_source)
foreach (ProductIndex::query()->cursor() as $hit) {
$doc = $hit['_source'];
// ...
}
```

### 文档 CRUD

```php
ProductIndex::insert(1, ['title' => 'Hello', 'price' => 99.9]);
ProductIndex::doc(1)->save(['title' => 'Hello', 'price' => 99.9]);

$doc = ProductIndex::doc(1);
$doc->source(); // ['title' => 'Hello', 'price' => 99.9]
Expand Down Expand Up @@ -277,7 +283,6 @@ ProductIndex::listen('search.*', function (Event $e) {

- [实践指南](docs/guide.md)——电商订单场景,从安装到上线的完整流程
- [Index 文档](docs/index.md)——搜索、CRUD、批量操作、零停机重建、事件
- [升级指南](docs/upgrade.md)——v7.x → v8.x 迁移说明
- [更新日志](CHANGELOG.md)
- [Elasticsearch 官方文档](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)——查询类型和参数参考

Expand Down
4 changes: 2 additions & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@
],
"scripts": {
"analyse": "phpstan analyse",
"cs-check": "PHP_CS_FIXER_IGNORE_ENV=1 php-cs-fixer fix --dry-run --diff",
"cs-fix": "PHP_CS_FIXER_IGNORE_ENV=1 php-cs-fixer fix"
"cs-check": "php-cs-fixer fix --dry-run --diff",
"cs-fix": "php-cs-fixer fix"
},
"config": {
"audit": {
Expand Down
18 changes: 9 additions & 9 deletions docs/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ use Illuminate\Support\Facades\Db;

class OrderIndex extends Index
{
protected $name = 'orders';
protected string $name = 'orders';

protected $mappings = [
protected array $mappings = [
'properties' => [
'order_no' => ['type' => 'keyword'], // 精确匹配
'status' => ['type' => 'keyword'], // pending/paid/shipped/completed
Expand Down Expand Up @@ -128,17 +128,17 @@ class OrderIndex extends Index

// 精确筛选(不需要评分,放 filter)
if (!empty($filters['status'])) {
$bool->addFilter(Term::create('status', $filters['status']));
$bool->filter(Term::create('status', $filters['status']));
}

if (!empty($filters['start_date']) && !empty($filters['end_date'])) {
$bool->addFilter(Range::create('created_at', [$filters['start_date'], $filters['end_date']]));
$bool->filter(Range::create('created_at', [$filters['start_date'], $filters['end_date']]));
}

// 关键词搜索(OR,放 should)
if (!empty($filters['keyword'])) {
$bool->addShould(Wildcard::create('order_no', "*{$filters['keyword']}*"));
$bool->addShould(Wildcard::create('merchant_name', "*{$filters['keyword']}*"));
$bool->should(Wildcard::create('order_no', "*{$filters['keyword']}*"));
$bool->should(Wildcard::create('merchant_name', "*{$filters['keyword']}*"));
}

return static::query(Query::create($bool));
Expand All @@ -160,7 +160,7 @@ public function index(Request $request)
}
```

> 条件用 `if` 逐个判断,只有传了值才加查询。`addShould()` 实现 OR 搜索。深分页场景用 `cursor()` 替代 `paginate()`。
> 条件用 `if` 逐个判断,只有传了值才加查询。`should()` 实现 OR 搜索。深分页场景用 `chunk()`(按批)或 `cursor()`(逐条)替代 `paginate()`。

运营还要导出筛选结果到 Excel。ES 默认 `max_result_window = 10000`,`from/size` 翻不到后面的数据,用 `cursor()` 基于 scroll 遍历:

Expand All @@ -169,8 +169,8 @@ public function export(array $filters)
{
$search = static::searchOrders($filters)->sort('created_at', 'desc');

foreach ($search->cursor() as $batch) {
foreach ($batch->docs() as $doc) {
foreach ($search->chunk() as $results) {
foreach ($results->docs() as $doc) {
// 写入 Excel
}
}
Expand Down
59 changes: 44 additions & 15 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,18 +25,18 @@ Index::setClient($logClient, 'logs');
```php
class ProductIndex extends Index
{
protected $name = 'products'; // 索引名(必填)
protected $mappings = [ // 索引 mappings
protected string $name = 'products'; // 索引名(必填)
protected array $mappings = [ // 索引 mappings
'properties' => [
'title' => ['type' => 'text'],
'price' => ['type' => 'float'],
'status' => ['type' => 'keyword'],
],
];
protected $settings = [ // 索引 settings
protected array $settings = [ // 索引 settings
'number_of_shards' => 1,
];
protected $connection = 'main'; // 连接名(默认 'default')
protected string $connection = 'main'; // 连接名(默认 'default')

public function rebuildName(): string // 重建后的真实索引名(可重写自定义)
{
Expand Down Expand Up @@ -116,17 +116,27 @@ while (count($results->docs()) > 0) {
ProductIndex::query()->clear($scrollId);
```

## Cursor
## Chunk / Cursor

Cursor 把 scroll 封装成 PHP 生成器:
把 scroll 封装成 PHP 生成器,scroll 自动清理。

**chunk** 按批遍历,每次 yield 一个 Results(含 docs/hits/total 等):

```php
foreach (ProductIndex::query()->cursor() as $results) {
foreach (ProductIndex::query()->chunk() as $results) {
foreach ($results->docs() as $doc) {
// 处理
}
}
// scroll 自动清理
```

**cursor** 逐条遍历,每次 yield 一个完整 hit(_id/_score/_source):

```php
foreach (ProductIndex::query()->cursor() as $hit) {
$doc = $hit['_source'];
$id = $hit['_id'];
}
```

## 文档 CRUD
Expand Down Expand Up @@ -164,6 +174,28 @@ $bulk->delete(3);
$bulk->execute(); // 执行所有操作,执行后清空状态
```

### 错误处理

`execute()` 默认在响应包含错误时抛出 `RuntimeException`。使用 `onError()` 自定义处理。回调接收 ES 原始响应,不抛出则继续,抛出则中断:

```php
$bulk = new Bulk(new ProductIndex());

// 不设 onError → 有错误就抛异常
$bulk->execute();

// 设 onError → 回调内不抛则继续,抛则中断
$bulk->onError(function (array $response) {
$failures = count(array_filter($response['items'], fn($i) => isset($i['index']['error'])));
if ($failures > 100) {
throw new RuntimeException("失败超过阈值: {$failures}");
}
Log::warning("部分失败: {$failures} 条");
})->execute();
```

> `batchSize` 自动 flush 时的错误同样走 `onError`,不会丢失。

## 零停机重建

创建新索引 → 导入数据 → 切换别名。
Expand Down Expand Up @@ -242,14 +274,12 @@ $rebuild->run(['after' => '2024-01-01']);

### 错误处理

导入错误会触发 `rebuild.import.failed` 事件(始终),并抛出异常(默认)。使用 `skipErrors()` 抑制异常但保留事件通知
Rebuild 内部使用 Bulk 执行导入,`onError()` 用法与 [批量操作 > 错误处理](#错误处理) 一致

```php
Index::listen('rebuild.import.failed', function (Event $e) {
Log::warning("重建导入错误", $e->response);
});

$rebuild->skipErrors()->run(); // 通过事件记录错误,不中断
$rebuild->onError(function (array $response) {
Log::warning("重建导入错误", $response);
})->run();
```

> rebuild 期间 DB 仍在变更,新索引是开始时刻的快照,建议 rebuild 后通过 `updated_at` 增量同步补齐。
Expand Down Expand Up @@ -351,4 +381,3 @@ Index::setClient($client);
| `manager.swap_alias.after` | `$response` |
| `rebuild.run.before` | |
| `rebuild.run.after` | `$newIndex`, `$oldIndex` |
| `rebuild.import.failed` | `$response` |
94 changes: 0 additions & 94 deletions docs/upgrade.md

This file was deleted.

Loading
Loading