diff --git a/docs/superpowers/plans/2026-07-24-buildflow-architecture-and-sop.md b/docs/superpowers/plans/2026-07-24-buildflow-architecture-and-sop.md new file mode 100644 index 0000000..a609caa --- /dev/null +++ b/docs/superpowers/plans/2026-07-24-buildflow-architecture-and-sop.md @@ -0,0 +1,640 @@ +# BuildFlow Architecture Closure And Backend SOP Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task in the current session. Do not dispatch subagents for this plan. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Complete the fifth optimization batch by enforcing source-wide architecture rules, moving the remaining endpoints and module integrations to MediatR CQRS, normalizing business error codes and file organization, and producing a fully isolated, executable, evidence-archiving backend SOP. + +**Architecture:** `BuildFlow.Host` remains the only HTTP/composition root. Endpoints only translate HTTP/SSE contracts to target-module Application commands or stream requests; every cross-module call is an `IMediator` send to a target-owned Query/Command/DTO. Roslyn architecture tests scan all current source files, and the SOP starts isolated random-port infrastructure before running migrations, integration, concurrency, recovery, and large-data gates. + +**Tech Stack:** .NET 10, FastEndpoints 8.1, MediatR 12.5, FluentValidation 12.1, EF Core 10/Npgsql, Roslyn 5.0, xUnit, PostgreSQL/pgvector, Redis, MongoDB, MinIO, Docker Compose, Bash. + +--- + +## File Map + +The implementation creates or changes the following ownership units: + +- `tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs`: source-wide Roslyn guards for endpoint dependencies, Host bridges, error-code arguments, and top-level type count. +- `src/modules/platform/Application/Auth/**`: change-password command, handler, and validator. +- `src/modules/rag/Application/RagQA/**`: RAG stream request, stream event, handler, and validator. +- `src/modules/platform/Application/Users/**`: Platform-owned batch user-summary query and DTO consumed by IM. +- `src/modules/file/Application/Features/Files/**`: File-owned batch preview-path query and DTO consumed by Platform. +- IM/RAG/Platform handlers: inject `IMediator` and send target-owned contracts directly. +- `src/BuildFlow.Host/Services/Integration/InProcess*.cs`: deleted after all consumers move to CQRS. +- Workflow/RAG/File throw sites: explicit semantic `ErrorCode` arguments. +- Consul/workflow graph/tree and four test support files: split to one top-level class/record/enum per file. +- `scripts/backend-optimization-compose.yml`: isolated PostgreSQL, Redis, MongoDB, and MinIO services with random host ports. +- `scripts/backend-optimization-sop.sh`: fail-fast orchestrator and artifact writer. +- `docs/backend-optimization-sop.md`: operator contract, prerequisites, phases, outputs, and failure handling. +- `artifacts/backend-optimization-sop//`: generated only at runtime and ignored by Git. + +### Task 1: Add Source-Wide Architecture Guards In RED State + +**Files:** +- Modify: `tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj` +- Create: `tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs` + +- [x] **Step 1: Add Roslyn to the architecture test project** + +Add the cached repository-compatible package version: + +```xml + +``` + +- [x] **Step 2: Write the four source-wide failing tests** + +`SourceArchitectureTests` must locate `BuildFlow.slnx`, enumerate non-generated `.cs` files under `src` and `tests`, parse each with `CSharpSyntaxTree.ParseText`, and expose four facts: + +```csharp +[Fact] +public void EndpointConstructorsShouldNotDependOnDbContexts(); + +[Fact] +public void HostShouldNotContainInProcessBridges(); + +[Fact] +public void BusinessExceptionsShouldSpecifySemanticErrorCodes(); + +[Fact] +public void SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum(); +``` + +The tests must report relative path and 1-based line number. The exception guard applies to `BusinessException`, `NotFoundException`, and `UnauthorizedException`, requires at least two arguments, and requires the first expression to be `ErrorCode.`. The file guard counts only top-level `class`, `record`, and `enum`; nested declarations remain allowed. + +- [x] **Step 3: Run each guard and verify the expected RED inventory** + +Run: + +```bash +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceArchitectureTests --logger "console;verbosity=normal" +``` + +Expected: FAIL and report exactly the current categories: two DbContext endpoints, three Host `InProcess*` bridges, 95 one-argument business exceptions, and seven multi-top-level-type files. The 95th site is a namespace-qualified exception in `UploadDocumentEndpoint` that the earlier regular-expression audit did not detect. + +- [x] **Step 4: Commit the RED guard** + +```bash +git add tests/BuildFlow.Architecture.Tests +git commit -m "test: enforce backend source architecture" +``` + +### Task 2: Move Change Password To Platform CQRS + +**Files:** +- Create: `src/modules/platform/Application/Auth/Commands/ChangePasswordCommand.cs` +- Create: `src/modules/platform/Application/Auth/Handlers/ChangePasswordCommandHandler.cs` +- Create: `src/modules/platform/Application/Auth/Validators/ChangePasswordCommandValidator.cs` +- Modify: `src/BuildFlow.Host/Endpoints/Platform/Auth/ChangePasswordEndpoint.cs` +- Create: `tests/BuildFlow.Platform.Tests/Auth/ChangePasswordCommandHandlerTests.cs` +- Create: `tests/BuildFlow.Platform.Tests/Auth/ChangePasswordEndpointTests.cs` +- Modify: `tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj` + +- [ ] **Step 1: Write handler and endpoint tests first** + +Cover these behaviors: + +```text +valid old password -> password hash changes and command completes +wrong old password -> BusinessException(ErrorCode.Business) +same old/new password -> BusinessException(ErrorCode.Business) +missing current user -> UnauthorizedException(ErrorCode.Unauthorized) +endpoint -> sends ChangePasswordCommand and writes an OK response +``` + +The endpoint test must use `FastEndpoints.Factory.Create` with a mocked `IMediator`; it must not construct a `PlatformDbContext`. + +- [ ] **Step 2: Run tests to verify RED** + +```bash +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword" --logger "console;verbosity=normal" +``` + +Expected: FAIL because the command/handler do not exist and the endpoint still requires `PlatformDbContext`. + +- [ ] **Step 3: Implement the Platform command boundary** + +Use this contract: + +```csharp +public sealed record ChangePasswordCommand(string OldPassword, string NewPassword) : IRequest; +``` + +The handler injects `PlatformDbContext` and `ICurrentUserContext`, verifies the current user and BCrypt hash, throws explicit semantic exceptions, updates the hash, and calls `SaveChangesAsync(ct)`. The validator enforces non-empty old/new passwords and the existing password policy. The endpoint becomes: + +```csharp +public class ChangePasswordEndpoint(IMediator mediator) : Endpoint +{ + public override async Task HandleAsync(ChangePasswordRequest req, CancellationToken ct) + { + await mediator.Send(new ChangePasswordCommand(req.OldPassword, req.NewPassword), ct); + await Send.OkAsync(ct); + } +} +``` + +- [ ] **Step 4: Run focused tests and the endpoint architecture guard** + +```bash +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword" +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts +``` + +Expected: change-password tests PASS; the architecture test still reports only `RAGStreamEndpoint`. + +- [ ] **Step 5: Commit** + +```bash +git add src/modules/platform/Application/Auth src/BuildFlow.Host/Endpoints/Platform/Auth tests/BuildFlow.Platform.Tests +git commit -m "refactor: move password changes to platform cqrs" +``` + +### Task 3: Move RAG SSE Generation To An Application Stream Handler + +**Files:** +- Create: `src/modules/rag/Application/RagQA/Queries/RAGStreamQuery.cs` +- Create: `src/modules/rag/Application/RagQA/DTOs/RAGStreamEvent.cs` +- Create: `src/modules/rag/Application/RagQA/Handlers/RAGStreamQueryHandler.cs` +- Create: `src/modules/rag/Application/RagQA/Validators/RAGStreamQueryValidator.cs` +- Modify: `src/BuildFlow.Host/Endpoints/Rag/RAG/RAGStreamEndpoint.cs` +- Create: `tests/BuildFlow.Rag.Tests/RagStreamQueryHandlerTests.cs` +- Create: `tests/BuildFlow.Rag.Tests/RagStreamEndpointTests.cs` +- Modify: `tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj` + +- [ ] **Step 1: Write stream handler and endpoint protocol tests** + +Use a target-owned request: + +```csharp +public sealed record RAGStreamQuery(Guid KnowledgeBaseId, string Question) + : IStreamRequest; +``` + +Test event order for hits (`Sources`, content chunks, `Done`), no hits (fallback content, `Done`), missing knowledge base (`NotFoundException(ErrorCode.NotFound)`), and cancellation propagation. Use `Factory.Create` to assert exact SSE output including `event: sources`, JSON `data:` content frames, and final `data: [DONE]`. + +- [ ] **Step 2: Run tests to verify RED** + +```bash +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream" +``` + +Expected: FAIL because the request/event/handler do not exist and the endpoint owns retrieval/chat logic. + +- [ ] **Step 3: Implement the stream handler** + +`RAGStreamEvent` must be the only top-level type in its file and expose source/content/done factories without adding an enum file: + +```csharp +public sealed record RAGStreamEvent( + IReadOnlyList? Sources, + string? Content, + bool IsDone); +``` + +The handler injects `RagDbContext`, `IRagRetrievalService`, and `IAIChatAgent`; it retrieves the knowledge base, emits source summaries truncated at 200 characters, builds the bounded prompt with the current retrieval result, streams chat chunks, and emits Done. The endpoint injects only `IMediator`, sets SSE headers before enumeration, encodes events, flushes after every frame, and preserves the current wire protocol byte-for-byte. + +- [ ] **Step 4: Run focused tests and endpoint guard** + +```bash +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream" +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts +``` + +Expected: PASS; no endpoint constructor depends on a DbContext. + +- [ ] **Step 5: Commit** + +```bash +git add src/modules/rag/Application/RagQA src/BuildFlow.Host/Endpoints/Rag/RAG tests/BuildFlow.Rag.Tests +git commit -m "refactor: stream rag answers through cqrs" +``` + +### Task 4: Replace IM User Information Bridge With A Platform Batch Query + +**Files:** +- Create: `src/modules/platform/Application/Users/DTOs/UserSummaryDto.cs` +- Create: `src/modules/platform/Application/Users/Queries/GetUserSummariesQuery.cs` +- Create: `src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs` +- Modify: `src/modules/im/Application/IM.Application.csproj` +- Modify: seven handlers under `src/modules/im/Application/Chat/Handlers/` that inject `IUserInfoService` +- Delete: `src/modules/im/Domain/Common/IUserInfoService.cs` +- Delete: `src/BuildFlow.Host/Services/Integration/InProcessUserInfoService.cs` +- Modify: `src/BuildFlow.Host/BuildFlowHostModule.cs` +- Modify: IM unit tests using `Mock` +- Replace: `tests/BuildFlow.IM.Integration.Tests/InMemoryUserInfoService.cs` with `InMemoryUserSummaryQueryHandler.cs` +- Modify: `tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs` + +- [ ] **Step 1: Write Platform query tests and update IM tests to expect mediator sends** + +The Platform query returns a `Dictionary` for distinct requested IDs and performs one database query. IM tests must configure `IMediator.Send(new GetUserSummariesQuery(...), ct)` and verify that empty input avoids a send where practical. + +- [ ] **Step 2: Run focused tests to verify RED** + +```bash +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter FullyQualifiedName~GetUserSummaries +dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore --filter "FullyQualifiedName~GetConversations|FullyQualifiedName~SendMessage" +``` + +Expected: FAIL until the target-owned query and IM handler changes exist. + +- [ ] **Step 3: Implement the direct CQRS calls** + +Add a Platform Application project reference to IM Application. Each IM handler injects `IMediator`, sends one `GetUserSummariesQuery` for its deduplicated IDs, and reads results by `Guid` key. Do not catch all exceptions and do not introduce a replacement service/interface bridge. + +The integration fake implements `IRequestHandler>`; `ImServerFixture` registers both the IM Application assembly and the test assembly with MediatR. + +- [ ] **Step 4: Remove bridge interface, Host implementation, and registration** + +Delete `IUserInfoService` and `InProcessUserInfoService`; remove the Host `AddScoped()` registration and stale using directives/comments. + +- [ ] **Step 5: Run IM unit and SignalR integration tests** + +```bash +dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore +dotnet test tests/BuildFlow.IM.Integration.Tests/BuildFlow.IM.Integration.Tests.csproj --no-restore +``` + +Expected: PASS with the same private/group conversation display names and notification behavior. + +- [ ] **Step 6: Commit** + +```bash +git add src/modules/platform/Application/Users src/modules/im src/BuildFlow.Host tests/BuildFlow.IM.Tests tests/BuildFlow.IM.Integration.Tests +git commit -m "refactor: query platform users through mediator" +``` + +### Task 5: Replace File And Avatar Bridges With File Application Contracts + +**Files:** +- Create: `src/modules/file/Application/Features/Files/DTOs/FilePreviewPathDto.cs` +- Create: `src/modules/file/Application/Features/Files/Queries/ResolveFilePreviewPathsQuery.cs` +- Create: `src/modules/file/Application/Features/Files/Handlers/ResolveFilePreviewPathsQueryHandler.cs` +- Create: `tests/BuildFlow.File.Tests/ResolveFilePreviewPathsQueryHandlerTests.cs` +- Modify: `src/modules/platform/Application/Platform.Application.csproj` +- Modify: `src/modules/platform/Application/Auth/Handlers/GetCurrentUserQueryHandler.cs` +- Modify: `src/modules/platform/Application/Auth/Handlers/UpdateProfileCommandHandler.cs` +- Modify: `src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs` +- Delete: `src/modules/platform/Domain/Services/IAvatarUrlResolver.cs` +- Modify: `src/modules/rag/Application/BuildFlow.Application.csproj` +- Modify: RAG document upload/process/preview handlers and their tests +- Delete: `src/modules/rag/Domain/Interfaces/IFileStorageClient.cs` +- Delete: `src/modules/rag/Infrastructure/Files/FileStorageClient.cs` +- Modify: `src/modules/rag/Infrastructure/RAGInfrastructureModule.cs` +- Delete: `src/BuildFlow.Host/Services/Integration/InProcessFileStorageClient.cs` +- Delete: `src/BuildFlow.Host/Services/Integration/InProcessAvatarUrlResolver.cs` +- Modify: `src/BuildFlow.Host/BuildFlowHostModule.cs` + +- [ ] **Step 1: Write File preview-path query tests** + +Test null/blank references, existing HTTP/HTTPS URLs, object-key URL escaping, duplicate inputs, and batch output. The target-owned handler must return `/api/files/preview?objectKey=` without touching S3. + +- [ ] **Step 2: Update Platform tests to expect one File batch query** + +`GetCurrentUserQueryHandler`, `UpdateProfileCommandHandler`, and `GetUserSummariesQueryHandler` inject `IMediator`. A batch user query sends one `ResolveFilePreviewPathsQuery` for distinct non-empty avatar references to avoid N+1 path resolution. + +- [ ] **Step 3: Update RAG tests to expect direct File Application sends** + +RAG upload sends `UploadFileCommand(objectKey, stream, contentType, fileSize)`; process and preview send `DownloadFileQuery(objectKey)` and dispose the returned stream. Remove `IHttpContextAccessor`, auth-token extraction, and `IFileStorageClient` stubs from these paths. + +- [ ] **Step 4: Run focused tests to verify RED** + +```bash +dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore --filter FullyQualifiedName~ResolveFilePreviewPaths +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~CurrentUser|FullyQualifiedName~UpdateProfile|FullyQualifiedName~GetUserSummaries" +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~Document|FullyQualifiedName~PreviewDocument" +``` + +Expected: FAIL before implementation and PASS after the direct mediator changes. + +- [ ] **Step 5: Delete the old interfaces, implementations, and DI registrations** + +Remove all three Host `InProcess*` classes and both source-module service interfaces. Remove the obsolete RAG HTTP `FileStorageClient` registration and class. `rg` must return no production references: + +```bash +rg -n "InProcessUserInfoService|InProcessFileStorageClient|InProcessAvatarUrlResolver|IUserInfoService|IFileStorageClient|IAvatarUrlResolver" src +``` + +Expected: no matches. + +- [ ] **Step 6: Run the Host bridge guard and affected test projects** + +```bash +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~HostShouldNotContainInProcessBridges +dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore +``` + +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add src/modules/file src/modules/platform src/modules/rag src/BuildFlow.Host tests/BuildFlow.File.Tests tests/BuildFlow.Platform.Tests tests/BuildFlow.Rag.Tests +git commit -m "refactor: remove host module service bridges" +``` + +### Task 6: Normalize Every Business Exception Error Code + +**Files:** +- Modify: 95 throw/return sites reported by the architecture guard in Host, File, RAG, and Workflow. +- Modify tests only where they assert exception codes. + +- [ ] **Step 1: Add focused semantic-code assertions before production edits** + +Extend existing tests for representative categories: duplicate resource, missing resource, invalid state, invalid parameter/schema, and forbidden operation. Assert the exception `Code` property, not only the type/message. + +- [ ] **Step 2: Run the representative tests and architecture guard in RED** + +```bash +dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore --filter "FullyQualifiedName~HandlerTests|FullyQualifiedName~ProcessEngineTests" +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes +``` + +Expected: representative code assertions and the architecture guard fail on old one-argument calls. + +- [ ] **Step 3: Apply the semantic mapping deliberately** + +Use this mapping at every reported site: + +```text +missing/not found -> NotFoundException(ErrorCode.NotFound, message) +duplicate/already exists -> BusinessException(ErrorCode.AlreadyExists, message) +disabled/deleted/already processed/unsupported transition -> BusinessException(ErrorCode.InvalidState, message) +invalid JSON/schema/value/path/missing action argument -> BusinessException(ErrorCode.InvalidParam, message) +permission denied -> UnauthorizedException(ErrorCode.Forbidden, message) +rule conflict without a narrower category -> BusinessException(ErrorCode.Business, message) +``` + +Convert existing `BusinessException` calls that describe a missing resource to `NotFoundException` where the surrounding contract already treats absence as 404. Do not replace all calls with a single code. + +- [ ] **Step 4: Run the guard and module tests** + +```bash +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes +dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore +``` + +Expected: PASS and zero old one-argument calls. + +- [ ] **Step 5: Commit** + +```bash +git add src/modules/file src/modules/rag src/modules/workflow tests +git commit -m "refactor: classify backend business errors" +``` + +### Task 7: Split Every Multi-Type Source File + +**Files:** +- Split `src/BuildFlow.Host/Configuration/ConsulConfigurationProvider.cs` into `ConsulConfigurationSource.cs`, `ConsulOptions.cs`, `ConsulConfigurationProvider.cs`, and `ConsulKvItem.cs`. +- Split `src/modules/workflow/Domain/Engine/WorkflowGraph.cs` into `WorkflowGraph.cs`, `GraphNode.cs`, `GraphEdge.cs`, `GraphEndpoint.cs`, `GraphPoint.cs`, and `GraphJsonExtensions.cs`. +- Split `src/modules/workflow/Domain/Engine/WorkflowTree.cs` into `WorkflowTree.cs`, `TreeNode.cs`, and `TreeBranch.cs`. +- Create `tests/BuildFlow.Platform.Tests/DictionaryConfiguration.cs` from `PlatformTestDb.cs`. +- Create `tests/BuildFlow.Rag.Tests/Stub/StubDocumentExtractorFactory.cs` from `StubDocumentExtractor.cs`. +- Create `tests/BuildFlow.Workflow.Tests/Form/FormTestFixtureClassFixture.cs` from `FormTestFixture.cs`. +- Create `tests/BuildFlow.Workflow.Tests/Notification/NoOpPlatformNotificationPublisher.cs` from `NotificationServiceTests.cs`. + +- [ ] **Step 1: Move declarations without changing behavior** + +Preserve namespaces, modifiers, XML comments, JSON attributes, extension methods, and member order. Each original file retains its namesake type; each moved declaration gets a same-named file. + +- [ ] **Step 2: Run the file organization guard** + +```bash +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum +``` + +Expected: PASS with zero violations under `src` and `tests`. + +- [ ] **Step 3: Run affected projects** + +```bash +dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore +dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore +dotnet build src/BuildFlow.Host/BuildFlow.Host.csproj --no-restore +``` + +Expected: PASS, 0 warning, 0 error. + +- [ ] **Step 4: Commit** + +```bash +git add src/BuildFlow.Host/Configuration src/modules/workflow/Domain/Engine tests +git commit -m "refactor: enforce one top-level type per file" +``` + +### Task 8: Add Isolated Infrastructure, Real S3 Coverage, And SOP Metrics + +**Files:** +- Create: `scripts/backend-optimization-compose.yml` +- Create: `tests/BuildFlow.File.Tests/S3StorageServiceIntegrationTests.cs` +- Modify: `tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs` +- Modify: `tests/BuildFlow.IM.Tests/Chat/SendMessageCommandHandlerTests.cs` +- Modify: selected performance tests to emit `SOP_METRIC key=value` through `ITestOutputHelper` +- Create: `tests/BuildFlow.Workflow.Tests/Performance/OpenXmlMemoryProfileTests.cs` + +- [ ] **Step 1: Add environment-driven isolated connection support** + +Replace fixed IM test endpoints with these environment variables and retain current values only as local fallbacks: + +```text +BUILDFLOW_IM_TEST_POSTGRES_CONNECTION +BUILDFLOW_IM_TEST_MONGO_CONNECTION +BUILDFLOW_IM_TEST_REDIS_CONNECTION +BUILDFLOW_WORKFLOW_TEST_CONNECTION +BUILDFLOW_MONITORING_TEST_CONNECTION +BUILDFLOW_RAG_REDIS_TEST_CONNECTION +BUILDFLOW_FILE_S3_TEST_ENDPOINT +BUILDFLOW_FILE_S3_TEST_ACCESS_KEY +BUILDFLOW_FILE_S3_TEST_SECRET_KEY +``` + +- [ ] **Step 2: Add a real MinIO S3 integration test** + +The test must create a unique bucket, upload a non-empty object, download and byte-compare it, verify a zero-byte object, delete objects/bucket in `finally`, and emit object count/bytes as `SOP_METRIC`. It must fail rather than skip when its configured endpoint is unavailable. + +- [ ] **Step 3: Add deterministic metric output** + +Emit at least these values from passing tests: + +```text +form_aggregate_select_count +import_commit_count +redis_concurrent_append_count +redis_list_length +rag_context_message_count +rag_context_estimated_tokens +file_size_max_s3_concurrency +excel_export_peak_managed_bytes +excel_export_rows +``` + +`OpenXmlMemoryProfileTests` writes a large export to a temporary file from an `IAsyncEnumerable`, samples `GC.GetTotalMemory(false)` while writing, asserts a conservative bounded peak delta, emits the observed bytes, and always deletes the file. + +- [ ] **Step 4: Add random-port Docker Compose infrastructure** + +Use pinned images already compatible with the repository: + +```text +pgvector/pgvector:0.8.0-pg17 +redis:7-alpine +mongo:7 +minio/minio:RELEASE.2025-04-22T22-12-26Z +``` + +Publish container ports with Docker-assigned random loopback ports, add health checks, use test-only credentials, and name volumes inside the per-run Compose project so teardown removes only SOP resources. + +- [ ] **Step 5: Start the compose stack manually and run infrastructure tests** + +Use a unique Compose project name, discover ports with `docker compose port`, export the variables above, then run PostgreSQL reliability/migration tests, Redis tests, IM SignalR integration tests, Monitoring UPSERT tests, and the MinIO test. + +Expected: all PASS, zero skipped, and no access to developer databases or existing containers. + +- [ ] **Step 6: Commit** + +```bash +git add scripts/backend-optimization-compose.yml tests +git commit -m "test: add isolated backend integration evidence" +``` + +### Task 9: Implement The Executable Closure SOP And Operator Guide + +**Files:** +- Create: `scripts/backend-optimization-sop.sh` +- Create: `docs/backend-optimization-sop.md` +- Modify: `.gitignore` +- Create: `tests/BuildFlow.Architecture.Tests/BackendOptimizationSopTests.cs` + +- [ ] **Step 1: Write SOP contract tests in RED** + +The architecture test reads the script and Compose file and asserts: + +```text +set -euo pipefail +EXIT trap performs project-scoped docker compose down -v +all four isolated services exist and use random host ports +stages appear in design order +dotnet build requires no restore and warnings-as-errors +TRX results and logs go under a timestamped artifact directory +skipped > 0 causes failure +static scans include password, one-argument exception, endpoint DbContext, InProcess, multi-type, DateTime.Now +no appsettings.Secrets.json write and no developer database name +``` + +- [ ] **Step 2: Run the contract test to verify RED** + +```bash +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests +``` + +Expected: FAIL because the script and guide do not exist. + +- [ ] **Step 3: Implement the fail-fast SOP** + +The script must: + +```text +1. verify dotnet 10, docker, docker compose, git, rg, curl, and repository cleanliness policy +2. run Roslyn/static guards and explicit final-debt scans +3. restore once, then build with --no-restore and capture 0 warning/0 error +4. run non-external unit and architecture tests +5. start isolated Compose services and wait for health +6. create isolated databases and run empty + previous-version migration paths +7. run PostgreSQL, Redis, MongoDB/SignalR, MinIO, and endpoint/SSE tests +8. run two-worker claim, crash/lease recovery, restart-equivalent recovery tests +9. run long-context, large-form, large-Excel, and large-directory/performance gates +10. run the complete solution test suite, reject any skipped test, aggregate TRX counts and SOP_METRIC lines, and write summary.md +``` + +Every stage runs through a helper that records command, start/end timestamps, duration, exit status, and log path. Any command failure exits non-zero; the final success marker is written only after every gate passes. The cleanup trap removes only the unique Compose project. + +- [ ] **Step 4: Document operation and evidence semantics** + +`docs/backend-optimization-sop.md` must explain prerequisites, exact invocation, optional artifact root, isolated resource behavior, environment variables, stage meanings, report layout, failure triage, and the rule that a failed/skipped gate is not completion. + +- [ ] **Step 5: Ignore generated evidence while preserving the directory convention** + +Add: + +```gitignore +artifacts/backend-optimization-sop/ +``` + +- [ ] **Step 6: Run syntax and contract tests** + +```bash +bash -n scripts/backend-optimization-sop.sh +dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests +``` + +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add scripts/backend-optimization-sop.sh docs/backend-optimization-sop.md .gitignore tests/BuildFlow.Architecture.Tests +git commit -m "chore: add backend optimization closure sop" +``` + +### Task 10: Run The Clean Full SOP And Close The Audit + +**Files:** +- Runtime output only: `artifacts/backend-optimization-sop//` +- Modify documentation only if the run reveals a missing command or ambiguous failure path. + +- [ ] **Step 1: Confirm tracked worktree state and run the SOP from repository root** + +```bash +git status --short +scripts/backend-optimization-sop.sh +``` + +Expected: the script starts a unique isolated Compose project, does not touch existing `infra-*` containers, and exits 0. + +- [ ] **Step 2: Verify the archived report** + +The report must contain: + +```text +all commands and stage durations +build 0 warning / 0 error +total passed / failed / skipped counts with skipped = 0 +empty and upgrade migration evidence +SSE source/content/DONE protocol evidence +PostgreSQL/Redis/MongoDB/MinIO integration evidence +concurrency and recovery test evidence +query-count, remote-call-concurrency, Redis, context-window, transaction, and memory metrics +failure log paths (empty on success, populated on failure) +``` + +- [ ] **Step 3: Run final debt scans independently of the SOP** + +```bash +rg -n "Task\.Run" src/modules/workflow src/BuildFlow.Host -g '*.cs' +rg -n "InProcess(UserInfoService|FileStorageClient|AvatarUrlResolver)" src -g '*.cs' +rg -n "DateTime\.Now" src -g '*.cs' +dotnet test BuildFlow.slnx --no-restore +dotnet build BuildFlow.slnx --no-restore +``` + +Expected: no forbidden debt matches; all tests PASS, 0 skipped; build reports 0 warning and 0 error. + +- [ ] **Step 4: Commit any run-discovered SOP corrections** + +```bash +git add scripts docs tests +git commit -m "test: close backend optimization audit" +``` + +Skip this commit when the clean run requires no tracked correction. + +## Self-Review + +- Spec coverage: Tasks 2-3 cover Endpoint CQRS; Tasks 4-5 remove all three bridges; Task 6 covers all explicit error codes; Task 7 covers all seven multi-type files; Task 1 enforces the four new source-wide guards while existing `HostArchitectureTests` retains Program/launchSettings/Api checks; Tasks 8-10 implement and execute the full SOP. +- Placeholder scan: the plan contains no unresolved placeholders. Every task identifies exact files, commands, expected RED/GREEN outcomes, and commit boundaries. +- Type consistency: `RAGStreamQuery` returns `RAGStreamEvent`; `GetUserSummariesQuery` returns `Dictionary`; `ResolveFilePreviewPathsQuery` returns File-owned preview DTOs; callers only inject `IMediator`. +- Safety: all external tests accept explicit environment variables; Docker assigns random host ports; the unique Compose project and volumes are removed by an EXIT trap; no local Secrets file or developer database is written. diff --git a/tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj b/tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj index 2f0f912..aa49104 100644 --- a/tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj +++ b/tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj @@ -8,6 +8,7 @@ true + diff --git a/tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs b/tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs new file mode 100644 index 0000000..e058319 --- /dev/null +++ b/tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs @@ -0,0 +1,275 @@ +namespace BuildFlow.Architecture.Tests; + +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.CSharp.Syntax; +using Xunit; + +/// +/// 全源码架构护栏。通过 Roslyn 分析语法树，避免只约束少量已知文件或依赖易误报的正则表达式。 +/// +public class SourceArchitectureTests +{ + private static readonly HashSet BusinessExceptionNames = + [ + "BusinessException", + "NotFoundException", + "UnauthorizedException", + ]; + + [Fact] + public void EndpointConstructorsShouldNotDependOnDbContexts() + { + var repoRoot = FindRepoRoot(); + var endpointRoot = Path.Combine(repoRoot.FullName, "src", "BuildFlow.Host", "Endpoints"); + var violations = new List(); + + foreach (var source in EnumerateSources(repoRoot, endpointRoot)) + { + foreach (var declaration in source.Root.DescendantNodes().OfType()) + { + var parameters = declaration.ParameterList?.Parameters + ?? default(SeparatedSyntaxList); + foreach (var parameter in parameters) + { + if (IsDbContextType(parameter.Type)) + { + violations.Add(FormatLocation(source, parameter)); + } + } + + foreach (var constructor in declaration.Members.OfType()) + { + foreach (var parameter in constructor.ParameterList.Parameters) + { + if (IsDbContextType(parameter.Type)) + { + violations.Add(FormatLocation(source, parameter)); + } + } + } + } + } + + AssertEmpty( + violations, + "Endpoint 构造函数不得依赖 DbContext；请把业务逻辑迁移到 Application Command/Query Handler。"); + } + + [Fact] + public void HostShouldNotContainInProcessBridges() + { + var repoRoot = FindRepoRoot(); + var hostRoot = Path.Combine(repoRoot.FullName, "src", "BuildFlow.Host"); + var violations = new Dictionary(StringComparer.Ordinal); + + foreach (var source in EnumerateSources(repoRoot, hostRoot)) + { + foreach (var declaration in source.Root.DescendantNodes().OfType()) + { + if (declaration.Identifier.ValueText.StartsWith("InProcess", StringComparison.Ordinal)) + { + violations.TryAdd( + declaration.Identifier.ValueText, + FormatLocation(source, declaration.Identifier)); + } + } + + foreach (var invocation in source.Root.DescendantNodes().OfType()) + { + if (invocation.Expression is not MemberAccessExpressionSyntax + { + Name: GenericNameSyntax genericName, + }) + { + continue; + } + + if (genericName.Identifier.ValueText is not ("AddScoped" or "AddSingleton" or "AddTransient")) + { + continue; + } + + foreach (var typeArgument in genericName.TypeArgumentList.Arguments) + { + var typeName = GetSimpleTypeName(typeArgument); + if (typeName.StartsWith("InProcess", StringComparison.Ordinal)) + { + violations.TryAdd(typeName, FormatLocation(source, typeArgument)); + } + } + } + } + + AssertEmpty( + violations.Values, + "Host 不得定义或注册 InProcess 跨模块桥接；调用方应发送目标模块 Application 的 MediatR 契约。"); + } + + [Fact] + public void BusinessExceptionsShouldSpecifySemanticErrorCodes() + { + var repoRoot = FindRepoRoot(); + var sourceRoot = Path.Combine(repoRoot.FullName, "src"); + var violations = new List(); + + foreach (var source in EnumerateSources(repoRoot, sourceRoot)) + { + foreach (var creation in source.Root.DescendantNodes().OfType()) + { + if (!BusinessExceptionNames.Contains(GetSimpleTypeName(creation.Type))) + { + continue; + } + + var arguments = creation.ArgumentList?.Arguments; + if (arguments is not { Count: >= 2 } + || !IsErrorCodeMember(arguments.Value[0].Expression)) + { + violations.Add(FormatLocation(source, creation)); + } + } + } + + AssertEmpty( + violations, + "BusinessException、NotFoundException、UnauthorizedException 必须把 ErrorCode.<语义> 作为第一个参数。"); + } + + [Fact] + public void SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum() + { + var repoRoot = FindRepoRoot(); + var violations = new List(); + + foreach (var rootName in new[] { "src", "tests" }) + { + var sourceRoot = Path.Combine(repoRoot.FullName, rootName); + foreach (var source in EnumerateSources(repoRoot, sourceRoot)) + { + var declarations = source.Root.DescendantNodes() + .OfType() + .Where(declaration => declaration.Parent is CompilationUnitSyntax or BaseNamespaceDeclarationSyntax) + .Where(declaration => declaration is ClassDeclarationSyntax + or RecordDeclarationSyntax + or EnumDeclarationSyntax) + .ToArray(); + if (declarations.Length > 1) + { + violations.Add( + $"{source.RelativePath}:1 ({declarations.Length} 个顶级类型: " + + string.Join(", ", declarations.Select(GetDeclarationName)) + ")"); + } + } + } + + AssertEmpty( + violations, + "每个源文件最多只能包含一个顶级 class、record 或 enum；嵌套类型不受此限制。"); + } + + private static bool IsDbContextType(TypeSyntax? type) + => type is not null + && type.DescendantNodesAndSelf() + .OfType() + .Any(identifier => identifier.Identifier.ValueText.EndsWith("DbContext", StringComparison.Ordinal)); + + private static bool IsErrorCodeMember(ExpressionSyntax expression) + { + if (expression is not MemberAccessExpressionSyntax memberAccess) + { + return false; + } + + return memberAccess.Expression switch + { + IdentifierNameSyntax { Identifier.ValueText: "ErrorCode" } => true, + MemberAccessExpressionSyntax + { + Name: IdentifierNameSyntax { Identifier.ValueText: "ErrorCode" }, + } => true, + _ => false, + }; + } + + private static string GetSimpleTypeName(TypeSyntax type) + => type switch + { + IdentifierNameSyntax identifier => identifier.Identifier.ValueText, + GenericNameSyntax generic => generic.Identifier.ValueText, + QualifiedNameSyntax qualified => GetSimpleTypeName(qualified.Right), + AliasQualifiedNameSyntax alias => alias.Name.Identifier.ValueText, + _ => type.ToString(), + }; + + private static string GetDeclarationName(MemberDeclarationSyntax declaration) + => declaration switch + { + ClassDeclarationSyntax type => type.Identifier.ValueText, + RecordDeclarationSyntax type => type.Identifier.ValueText, + EnumDeclarationSyntax type => type.Identifier.ValueText, + _ => declaration.Kind().ToString(), + }; + + private static IEnumerable EnumerateSources( + DirectoryInfo repoRoot, + string sourceRoot) + { + return Directory.EnumerateFiles(sourceRoot, "*.cs", SearchOption.AllDirectories) + .Where(path => !ContainsPathSegment(path, "bin") && !ContainsPathSegment(path, "obj")) + .Order(StringComparer.Ordinal) + .Select(path => + { + var text = File.ReadAllText(path); + var tree = CSharpSyntaxTree.ParseText( + text, + CSharpParseOptions.Default.WithLanguageVersion(LanguageVersion.Preview), + path); + return new SourceFile( + Path.GetRelativePath(repoRoot.FullName, path).Replace('\\', '/'), + tree.GetCompilationUnitRoot()); + }); + } + + private static string FormatLocation(SourceFile source, SyntaxNode node) + { + var line = node.GetLocation().GetLineSpan().StartLinePosition.Line + 1; + return $"{source.RelativePath}:{line}"; + } + + private static string FormatLocation(SourceFile source, SyntaxToken token) + { + var line = token.GetLocation().GetLineSpan().StartLinePosition.Line + 1; + return $"{source.RelativePath}:{line}"; + } + + private static void AssertEmpty(IEnumerable violations, string message) + { + var ordered = violations.Distinct(StringComparer.Ordinal).Order(StringComparer.Ordinal).ToArray(); + Assert.True( + ordered.Length == 0, + message + Environment.NewLine + string.Join(Environment.NewLine, ordered)); + } + + private static DirectoryInfo FindRepoRoot() + { + var current = new DirectoryInfo(AppContext.BaseDirectory); + while (current is not null) + { + if (File.Exists(Path.Combine(current.FullName, "BuildFlow.slnx"))) + { + return current; + } + + current = current.Parent; + } + + throw new InvalidOperationException("无法定位 BuildFlow.slnx，架构测试无法确定仓库边界。"); + } + + private static bool ContainsPathSegment(string path, string segment) + => path.Split(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar) + .Any(part => string.Equals(part, segment, StringComparison.Ordinal)); + + private sealed record SourceFile(string RelativePath, CompilationUnitSyntax Root); +}