644 lines
32 KiB
Markdown
644 lines
32 KiB
Markdown
# BuildFlow Architecture Closure And Backend SOP Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task in the current session. Do not dispatch subagents for this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Complete the fifth optimization batch by enforcing source-wide architecture rules, moving the remaining endpoints and module integrations to MediatR CQRS, normalizing business error codes and file organization, and producing a fully isolated, executable, evidence-archiving backend SOP.
|
|
|
|
**Architecture:** `BuildFlow.Host` remains the only HTTP/composition root. Endpoints only translate HTTP/SSE contracts to target-module Application commands or stream requests; every cross-module call is an `IMediator` send to a target-owned Query/Command/DTO. Roslyn architecture tests scan all current source files, and the SOP starts isolated random-port infrastructure before running migrations, integration, concurrency, recovery, and large-data gates.
|
|
|
|
**Tech Stack:** .NET 10, FastEndpoints 8.1, MediatR 12.5, FluentValidation 12.1, EF Core 10/Npgsql, Roslyn 5.0, xUnit, PostgreSQL/pgvector, Redis, MongoDB, MinIO, Docker Compose, Bash.
|
|
|
|
---
|
|
|
|
## File Map
|
|
|
|
The implementation creates or changes the following ownership units:
|
|
|
|
- `tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs`: source-wide Roslyn guards for endpoint dependencies, Host bridges, error-code arguments, and top-level type count.
|
|
- `src/modules/platform/Application/Auth/**`: change-password command, handler, and validator.
|
|
- `src/modules/rag/Application/RagQA/**`: RAG stream request, stream event, handler, and validator.
|
|
- `src/modules/platform/Application/Users/**`: Platform-owned batch user-summary query and DTO consumed by IM.
|
|
- `src/modules/file/Application/Features/Files/**`: File-owned batch preview-path query and DTO consumed by Platform.
|
|
- IM/RAG/Platform handlers: inject `IMediator` and send target-owned contracts directly.
|
|
- `src/BuildFlow.Host/Services/Integration/InProcess*.cs`: deleted after all consumers move to CQRS.
|
|
- Workflow/RAG/File throw sites: explicit semantic `ErrorCode` arguments.
|
|
- Consul/workflow graph/tree and four test support files: split to one top-level class/record/enum per file.
|
|
- `scripts/backend-optimization-compose.yml`: isolated PostgreSQL, Redis, MongoDB, and MinIO services with random host ports.
|
|
- `scripts/backend-optimization-sop.sh`: fail-fast orchestrator and artifact writer.
|
|
- `docs/backend-optimization-sop.md`: operator contract, prerequisites, phases, outputs, and failure handling.
|
|
- `artifacts/backend-optimization-sop/<timestamp>/`: generated only at runtime and ignored by Git.
|
|
|
|
### Task 1: Add Source-Wide Architecture Guards In RED State
|
|
|
|
**Files:**
|
|
- Modify: `tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj`
|
|
- Create: `tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs`
|
|
|
|
- [x] **Step 1: Add Roslyn to the architecture test project**
|
|
|
|
Add the cached repository-compatible package version:
|
|
|
|
```xml
|
|
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="5.0.0" />
|
|
```
|
|
|
|
- [x] **Step 2: Write the four source-wide failing tests**
|
|
|
|
`SourceArchitectureTests` must locate `BuildFlow.slnx`, enumerate non-generated `.cs` files under `src` and `tests`, parse each with `CSharpSyntaxTree.ParseText`, and expose four facts:
|
|
|
|
```csharp
|
|
[Fact]
|
|
public void EndpointConstructorsShouldNotDependOnDbContexts();
|
|
|
|
[Fact]
|
|
public void HostShouldNotContainInProcessBridges();
|
|
|
|
[Fact]
|
|
public void BusinessExceptionsShouldSpecifySemanticErrorCodes();
|
|
|
|
[Fact]
|
|
public void SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum();
|
|
```
|
|
|
|
The tests must report relative path and 1-based line number. The exception guard applies to `BusinessException`, `NotFoundException`, and `UnauthorizedException`, requires at least two arguments, and requires the first expression to be `ErrorCode.<Member>`. The file guard counts only top-level `class`, `record`, and `enum`; nested declarations remain allowed.
|
|
|
|
- [x] **Step 3: Run each guard and verify the expected RED inventory**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceArchitectureTests --logger "console;verbosity=normal"
|
|
```
|
|
|
|
Expected: FAIL and report exactly the current categories: two DbContext endpoints, three Host `InProcess*` bridges, 95 one-argument business exceptions, and seven multi-top-level-type files. The 95th site is a namespace-qualified exception in `UploadDocumentEndpoint` that the earlier regular-expression audit did not detect.
|
|
|
|
- [x] **Step 4: Commit the RED guard**
|
|
|
|
```bash
|
|
git add tests/BuildFlow.Architecture.Tests
|
|
git commit -m "test: enforce backend source architecture"
|
|
```
|
|
|
|
### Task 2: Move Change Password To Platform CQRS
|
|
|
|
**Files:**
|
|
- Create: `src/modules/platform/Application/Auth/Commands/ChangePasswordCommand.cs`
|
|
- Create: `src/modules/platform/Application/Auth/Handlers/ChangePasswordCommandHandler.cs`
|
|
- Create: `src/modules/platform/Application/Auth/Validators/ChangePasswordCommandValidator.cs`
|
|
- Modify: `src/BuildFlow.Host/Endpoints/Platform/Auth/ChangePasswordEndpoint.cs`
|
|
- Create: `tests/BuildFlow.Platform.Tests/Auth/ChangePasswordCommandHandlerTests.cs`
|
|
- Create: `tests/BuildFlow.Platform.Tests/Auth/ChangePasswordEndpointTests.cs`
|
|
- Modify: `tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj`
|
|
|
|
- [x] **Step 1: Write handler and endpoint tests first**
|
|
|
|
Cover these behaviors:
|
|
|
|
```text
|
|
valid old password -> password hash changes and command completes
|
|
wrong old password -> BusinessException(ErrorCode.Business)
|
|
same old/new password -> BusinessException(ErrorCode.Business)
|
|
missing current user -> UnauthorizedException(ErrorCode.Unauthorized)
|
|
endpoint -> sends ChangePasswordCommand and writes an OK response
|
|
```
|
|
|
|
The endpoint test must use `FastEndpoints.Factory.Create<ChangePasswordEndpoint>` with a mocked `IMediator`; it must not construct a `PlatformDbContext`.
|
|
|
|
- [x] **Step 2: Run tests to verify RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword" --logger "console;verbosity=normal"
|
|
```
|
|
|
|
Expected: FAIL because the command/handler do not exist and the endpoint still requires `PlatformDbContext`.
|
|
|
|
- [x] **Step 3: Implement the Platform command boundary**
|
|
|
|
Use this contract:
|
|
|
|
```csharp
|
|
public sealed record ChangePasswordCommand(string OldPassword, string NewPassword) : IRequest;
|
|
```
|
|
|
|
The handler injects `PlatformDbContext` and `ICurrentUserContext`, verifies the current user and BCrypt hash, throws explicit semantic exceptions, updates the hash, and calls `SaveChangesAsync(ct)`. The validator enforces non-empty old/new passwords and the existing password policy. The endpoint becomes:
|
|
|
|
```csharp
|
|
public class ChangePasswordEndpoint(IMediator mediator) : Endpoint<ChangePasswordRequest>
|
|
{
|
|
public override async Task HandleAsync(ChangePasswordRequest req, CancellationToken ct)
|
|
{
|
|
await mediator.Send(new ChangePasswordCommand(req.OldPassword, req.NewPassword), ct);
|
|
await Send.OkAsync(ct);
|
|
}
|
|
}
|
|
```
|
|
|
|
- [x] **Step 4: Run focused tests and the endpoint architecture guard**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword"
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts
|
|
```
|
|
|
|
Expected: change-password tests PASS; the architecture test still reports only `RAGStreamEndpoint`.
|
|
|
|
- [x] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/modules/platform/Application/Auth src/BuildFlow.Host/Endpoints/Platform/Auth tests/BuildFlow.Platform.Tests
|
|
git commit -m "refactor: move password changes to platform cqrs"
|
|
```
|
|
|
|
### Task 3: Move RAG SSE Generation To An Application Stream Handler
|
|
|
|
**Files:**
|
|
- Create: `src/modules/rag/Application/RagQA/Queries/RAGStreamQuery.cs`
|
|
- Create: `src/modules/rag/Application/RagQA/DTOs/RAGStreamEvent.cs`
|
|
- Create: `src/modules/rag/Application/RagQA/Handlers/RAGStreamQueryHandler.cs`
|
|
- Create: `src/modules/rag/Application/RagQA/Validators/RAGStreamQueryValidator.cs`
|
|
- Modify: `src/BuildFlow.Host/Endpoints/Rag/RAG/RAGStreamEndpoint.cs`
|
|
- Create: `tests/BuildFlow.Rag.Tests/RagStreamQueryHandlerTests.cs`
|
|
- Create: `tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj`
|
|
- Create: `tests/BuildFlow.Host.Tests/RagStreamEndpointTests.cs`
|
|
- Modify: `BuildFlow.slnx`
|
|
|
|
- [x] **Step 1: Write stream handler and endpoint protocol tests**
|
|
|
|
Use a target-owned request:
|
|
|
|
```csharp
|
|
public sealed record RAGStreamQuery(Guid KnowledgeBaseId, string Question)
|
|
: IStreamRequest<RAGStreamEvent>;
|
|
```
|
|
|
|
Test event order for hits (`Sources`, content chunks, `Done`), no hits (fallback content, `Done`), missing knowledge base (`NotFoundException(ErrorCode.NotFound)`), and cancellation propagation. Use `Factory.Create<RAGStreamEndpoint>` to assert exact SSE output including `event: sources`, JSON `data:` content frames, and final `data: [DONE]`.
|
|
|
|
- [x] **Step 2: Run tests to verify RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
|
|
dotnet test tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
|
|
```
|
|
|
|
Expected: FAIL because the request/event/handler do not exist and the endpoint owns retrieval/chat logic.
|
|
|
|
- [x] **Step 3: Implement the stream handler**
|
|
|
|
`RAGStreamEvent` must be the only top-level type in its file and expose source/content/done factories without adding an enum file:
|
|
|
|
```csharp
|
|
public sealed record RAGStreamEvent(
|
|
IReadOnlyList<SourceChunk>? Sources,
|
|
string? Content,
|
|
bool IsDone);
|
|
```
|
|
|
|
The handler injects `RagDbContext`, `IRagRetrievalService`, and `IAIChatAgent`; it retrieves the knowledge base, emits source summaries truncated at 200 characters, builds the bounded prompt with the current retrieval result, streams chat chunks, and emits Done. The endpoint injects only `IMediator`, sets SSE headers before enumeration, encodes events, flushes after every frame, and preserves the current wire protocol byte-for-byte.
|
|
|
|
- [x] **Step 4: Run focused tests and endpoint guard**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
|
|
dotnet test tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts
|
|
```
|
|
|
|
Expected: PASS; no endpoint constructor depends on a DbContext.
|
|
|
|
- [x] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add BuildFlow.slnx src/modules/rag/Application/RagQA src/BuildFlow.Host/Endpoints/Rag/RAG tests/BuildFlow.Rag.Tests tests/BuildFlow.Host.Tests
|
|
git commit -m "refactor: stream rag answers through cqrs"
|
|
```
|
|
|
|
### Task 4: Replace IM User Information Bridge With A Platform Batch Query
|
|
|
|
**Files:**
|
|
- Create: `src/modules/platform/Application/Users/DTOs/UserSummaryDto.cs`
|
|
- Create: `src/modules/platform/Application/Users/Queries/GetUserSummariesQuery.cs`
|
|
- Create: `src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs`
|
|
- Modify: `src/modules/im/Application/IM.Application.csproj`
|
|
- Modify: seven handlers under `src/modules/im/Application/Chat/Handlers/` that inject `IUserInfoService`
|
|
- Delete: `src/modules/im/Domain/Common/IUserInfoService.cs`
|
|
- Delete: `src/BuildFlow.Host/Services/Integration/InProcessUserInfoService.cs`
|
|
- Modify: `src/BuildFlow.Host/BuildFlowHostModule.cs`
|
|
- Modify: IM unit tests using `Mock<IUserInfoService>`
|
|
- Replace: `tests/BuildFlow.IM.Integration.Tests/InMemoryUserInfoService.cs` with `InMemoryUserSummaryQueryHandler.cs`
|
|
- Modify: `tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs`
|
|
|
|
- [x] **Step 1: Write Platform query tests and update IM tests to expect mediator sends**
|
|
|
|
The Platform query returns an `IReadOnlyDictionary<Guid, UserSummaryDto>` for distinct requested IDs and performs one database query. IM tests must configure `IMediator.Send(new GetUserSummariesQuery(...), ct)` and verify that empty input avoids a send where practical.
|
|
|
|
- [x] **Step 2: Run focused tests to verify RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter FullyQualifiedName~GetUserSummaries
|
|
dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore --filter "FullyQualifiedName~GetConversations|FullyQualifiedName~SendMessage"
|
|
```
|
|
|
|
Expected: FAIL until the target-owned query and IM handler changes exist.
|
|
|
|
- [x] **Step 3: Implement the direct CQRS calls**
|
|
|
|
Add a Platform Application project reference to IM Application. Each IM handler injects `IMediator`, sends one `GetUserSummariesQuery` for its deduplicated IDs, and reads results by `Guid` key. Do not catch all exceptions and do not introduce a replacement service/interface bridge.
|
|
|
|
The integration fake implements `IRequestHandler<GetUserSummariesQuery, IReadOnlyDictionary<Guid, UserSummaryDto>>`; `ImServerFixture` registers both the IM Application assembly and the test assembly with MediatR.
|
|
|
|
- [x] **Step 4: Remove bridge interface, Host implementation, and registration**
|
|
|
|
Delete `IUserInfoService` and `InProcessUserInfoService`; remove the Host `AddScoped<IUserInfoService, InProcessUserInfoService>()` registration and stale using directives/comments.
|
|
|
|
- [x] **Step 5: Run IM unit and SignalR integration tests**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.IM.Integration.Tests/BuildFlow.IM.Integration.Tests.csproj --no-restore
|
|
```
|
|
|
|
Expected: PASS with the same private/group conversation display names and notification behavior.
|
|
|
|
- [x] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add src/modules/platform/Application/Users src/modules/im src/BuildFlow.Host tests/BuildFlow.IM.Tests tests/BuildFlow.IM.Integration.Tests
|
|
git commit -m "refactor: query platform users through mediator"
|
|
```
|
|
|
|
### Task 5: Replace File And Avatar Bridges With File Application Contracts
|
|
|
|
**Files:**
|
|
- Create: `src/modules/file/Application/Features/Files/DTOs/FilePreviewPathDto.cs`
|
|
- Create: `src/modules/file/Application/Features/Files/Queries/ResolveFilePreviewPathsQuery.cs`
|
|
- Create: `src/modules/file/Application/Features/Files/Handlers/ResolveFilePreviewPathsQueryHandler.cs`
|
|
- Create: `tests/BuildFlow.File.Tests/ResolveFilePreviewPathsQueryHandlerTests.cs`
|
|
- Modify: `src/modules/platform/Application/Platform.Application.csproj`
|
|
- Modify: `src/modules/platform/Application/Auth/Handlers/GetCurrentUserQueryHandler.cs`
|
|
- Modify: `src/modules/platform/Application/Auth/Handlers/UpdateProfileCommandHandler.cs`
|
|
- Modify: `src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs`
|
|
- Delete: `src/modules/platform/Domain/Services/IAvatarUrlResolver.cs`
|
|
- Modify: `src/modules/rag/Application/BuildFlow.Application.csproj`
|
|
- Modify: RAG document upload/process/preview handlers and their tests
|
|
- Delete: `src/modules/rag/Domain/Interfaces/IFileStorageClient.cs`
|
|
- Delete: `src/modules/rag/Infrastructure/Files/FileStorageClient.cs`
|
|
- Modify: `src/modules/rag/Infrastructure/RAGInfrastructureModule.cs`
|
|
- Delete: `src/BuildFlow.Host/Services/Integration/InProcessFileStorageClient.cs`
|
|
- Delete: `src/BuildFlow.Host/Services/Integration/InProcessAvatarUrlResolver.cs`
|
|
- Modify: `src/BuildFlow.Host/BuildFlowHostModule.cs`
|
|
|
|
- [x] **Step 1: Write File preview-path query tests**
|
|
|
|
Test null/blank references, existing HTTP/HTTPS URLs, object-key URL escaping, duplicate inputs, and batch output. The target-owned handler must return `/api/files/preview?objectKey=<escaped>` without touching S3.
|
|
|
|
- [x] **Step 2: Update Platform tests to expect one File batch query**
|
|
|
|
`GetCurrentUserQueryHandler`, `UpdateProfileCommandHandler`, and `GetUserSummariesQueryHandler` inject `IMediator`. A batch user query sends one `ResolveFilePreviewPathsQuery` for distinct non-empty avatar references to avoid N+1 path resolution.
|
|
|
|
- [x] **Step 3: Update RAG tests to expect direct File Application sends**
|
|
|
|
RAG upload sends `UploadFileCommand(objectKey, stream, contentType, fileSize)`; process and preview send `DownloadFileQuery(objectKey)` and dispose the returned stream. Remove `IHttpContextAccessor`, auth-token extraction, and `IFileStorageClient` stubs from these paths.
|
|
|
|
- [x] **Step 4: Run focused tests to verify RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore --filter FullyQualifiedName~ResolveFilePreviewPaths
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~CurrentUser|FullyQualifiedName~UpdateProfile|FullyQualifiedName~GetUserSummaries"
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~Document|FullyQualifiedName~PreviewDocument"
|
|
```
|
|
|
|
Expected: FAIL before implementation and PASS after the direct mediator changes.
|
|
|
|
- [x] **Step 5: Delete the old interfaces, implementations, and DI registrations**
|
|
|
|
Remove all three Host `InProcess*` classes and both source-module service interfaces. Remove the obsolete RAG HTTP `FileStorageClient` registration and class. `rg` must return no production references:
|
|
|
|
```bash
|
|
rg -n "InProcessUserInfoService|InProcessFileStorageClient|InProcessAvatarUrlResolver|IUserInfoService|IFileStorageClient|IAvatarUrlResolver" src
|
|
```
|
|
|
|
Expected: no matches.
|
|
|
|
- [x] **Step 6: Run the Host bridge guard and affected test projects**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~HostShouldNotContainInProcessBridges
|
|
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [x] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add src/modules/file src/modules/platform src/modules/rag src/BuildFlow.Host tests/BuildFlow.File.Tests tests/BuildFlow.Platform.Tests tests/BuildFlow.Rag.Tests
|
|
git commit -m "refactor: remove host module service bridges"
|
|
```
|
|
|
|
### Task 6: Normalize Every Business Exception Error Code
|
|
|
|
**Files:**
|
|
- Modify: 95 throw/return sites reported by the architecture guard in Host, File, RAG, and Workflow.
|
|
- Modify tests only where they assert exception codes.
|
|
|
|
- [x] **Step 1: Add focused semantic-code assertions before production edits**
|
|
|
|
Extend existing tests for representative categories: duplicate resource, missing resource, invalid state, invalid parameter/schema, and forbidden operation. Assert the exception `Code` property, not only the type/message.
|
|
|
|
- [x] **Step 2: Run the representative tests and architecture guard in RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore --filter "FullyQualifiedName~HandlerTests|FullyQualifiedName~ProcessEngineTests"
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes
|
|
```
|
|
|
|
Expected: representative code assertions and the architecture guard fail on old one-argument calls.
|
|
|
|
- [x] **Step 3: Apply the semantic mapping deliberately**
|
|
|
|
Use this mapping at every reported site:
|
|
|
|
```text
|
|
missing/not found -> NotFoundException(ErrorCode.NotFound, message)
|
|
duplicate/already exists -> BusinessException(ErrorCode.AlreadyExists, message)
|
|
disabled/deleted/already processed/unsupported transition -> BusinessException(ErrorCode.InvalidState, message)
|
|
invalid JSON/schema/value/path/missing action argument -> BusinessException(ErrorCode.InvalidParam, message)
|
|
permission denied -> UnauthorizedException(ErrorCode.Forbidden, message)
|
|
rule conflict without a narrower category -> BusinessException(ErrorCode.Business, message)
|
|
```
|
|
|
|
Convert existing `BusinessException` calls that describe a missing resource to `NotFoundException` where the surrounding contract already treats absence as 404. Do not replace all calls with a single code.
|
|
|
|
- [x] **Step 4: Run the guard and module tests**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes
|
|
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore
|
|
```
|
|
|
|
Expected: PASS and zero old one-argument calls.
|
|
|
|
- [x] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/modules/file src/modules/rag src/modules/workflow tests
|
|
git commit -m "refactor: classify backend business errors"
|
|
```
|
|
|
|
### Task 7: Split Every Multi-Type Source File
|
|
|
|
**Files:**
|
|
- Split `src/BuildFlow.Host/Configuration/ConsulConfigurationProvider.cs` into `ConsulConfigurationSource.cs`, `ConsulOptions.cs`, `ConsulConfigurationProvider.cs`, and `ConsulKvItem.cs`.
|
|
- Split `src/modules/workflow/Domain/Engine/WorkflowGraph.cs` into `WorkflowGraph.cs`, `GraphNode.cs`, `GraphEdge.cs`, `GraphEndpoint.cs`, `GraphPoint.cs`, and `GraphJsonExtensions.cs`.
|
|
- Split `src/modules/workflow/Domain/Engine/WorkflowTree.cs` into `WorkflowTree.cs`, `TreeNode.cs`, and `TreeBranch.cs`.
|
|
- Create `tests/BuildFlow.Platform.Tests/DictionaryConfiguration.cs` from `PlatformTestDb.cs`.
|
|
- Create `tests/BuildFlow.Rag.Tests/Stub/StubDocumentExtractorFactory.cs` from `StubDocumentExtractor.cs`.
|
|
- Create `tests/BuildFlow.Workflow.Tests/Form/FormTestFixtureClassFixture.cs` from `FormTestFixture.cs`.
|
|
- Create `tests/BuildFlow.Workflow.Tests/Notification/NoOpPlatformNotificationPublisher.cs` from `NotificationServiceTests.cs`.
|
|
|
|
- [x] **Step 1: Move declarations without changing behavior**
|
|
|
|
Preserve namespaces, modifiers, XML comments, JSON attributes, extension methods, and member order. Each original file retains its namesake type; each moved declaration gets a same-named file.
|
|
|
|
- [x] **Step 2: Run the file organization guard**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum
|
|
```
|
|
|
|
Expected: PASS with zero violations under `src` and `tests`.
|
|
|
|
- [x] **Step 3: Run affected projects**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore
|
|
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore
|
|
dotnet build src/BuildFlow.Host/BuildFlow.Host.csproj --no-restore
|
|
```
|
|
|
|
Expected: PASS, 0 warning, 0 error.
|
|
|
|
- [x] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/BuildFlow.Host/Configuration src/modules/workflow/Domain/Engine tests
|
|
git commit -m "refactor: enforce one top-level type per file"
|
|
```
|
|
|
|
### Task 8: Add Isolated Infrastructure, Real S3 Coverage, And SOP Metrics
|
|
|
|
**Files:**
|
|
- Create: `scripts/backend-optimization-compose.yml`
|
|
- Create: `tests/BuildFlow.File.Tests/S3StorageServiceIntegrationTests.cs`
|
|
- Modify: `tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs`
|
|
- Modify: `tests/BuildFlow.IM.Tests/Chat/SendMessageCommandHandlerTests.cs`
|
|
- Modify: selected performance tests to emit `SOP_METRIC key=value` through `ITestOutputHelper`
|
|
- Create: `tests/BuildFlow.Workflow.Tests/Performance/OpenXmlMemoryProfileTests.cs`
|
|
|
|
- [x] **Step 1: Add environment-driven isolated connection support**
|
|
|
|
Replace fixed IM test endpoints with these environment variables and retain current values only as local fallbacks:
|
|
|
|
```text
|
|
BUILDFLOW_IM_TEST_POSTGRES_CONNECTION
|
|
BUILDFLOW_IM_TEST_MONGO_CONNECTION
|
|
BUILDFLOW_IM_TEST_REDIS_CONNECTION
|
|
BUILDFLOW_WORKFLOW_TEST_CONNECTION
|
|
BUILDFLOW_MONITORING_TEST_CONNECTION
|
|
BUILDFLOW_RAG_REDIS_TEST_CONNECTION
|
|
BUILDFLOW_FILE_S3_TEST_ENDPOINT
|
|
BUILDFLOW_FILE_S3_TEST_ACCESS_KEY
|
|
BUILDFLOW_FILE_S3_TEST_SECRET_KEY
|
|
```
|
|
|
|
- [x] **Step 2: Add a real MinIO S3 integration test**
|
|
|
|
The test must create a unique bucket, upload a non-empty object, download and byte-compare it, verify a zero-byte object, delete objects/bucket in `finally`, and emit object count/bytes as `SOP_METRIC`. It must fail rather than skip when its configured endpoint is unavailable.
|
|
|
|
- [x] **Step 3: Add deterministic metric output**
|
|
|
|
Emit at least these values from passing tests:
|
|
|
|
```text
|
|
form_aggregate_select_count
|
|
import_commit_count
|
|
redis_concurrent_append_count
|
|
redis_list_length
|
|
rag_context_message_count
|
|
rag_context_estimated_tokens
|
|
file_size_max_s3_concurrency
|
|
excel_export_peak_managed_bytes
|
|
excel_export_rows
|
|
```
|
|
|
|
`OpenXmlMemoryProfileTests` writes a large export to a temporary file from an `IAsyncEnumerable<string>`, samples `GC.GetTotalMemory(false)` while writing, asserts a conservative bounded peak delta, emits the observed bytes, and always deletes the file.
|
|
|
|
- [x] **Step 4: Add random-port Docker Compose infrastructure**
|
|
|
|
Use pinned images already compatible with the repository:
|
|
|
|
```text
|
|
pgvector/pgvector:0.8.0-pg17
|
|
redis:7-alpine
|
|
mongo:7
|
|
minio/minio:RELEASE.2025-04-22T22-12-26Z
|
|
```
|
|
|
|
Publish container ports with Docker-assigned random loopback ports, add health checks, use test-only credentials, and name volumes inside the per-run Compose project so teardown removes only SOP resources.
|
|
|
|
- [x] **Step 5: Start the compose stack manually and run infrastructure tests**
|
|
|
|
Use a unique Compose project name, discover ports with `docker compose port`, export the variables above, then run PostgreSQL reliability/migration tests, Redis tests, IM SignalR integration tests, Monitoring UPSERT tests, and the MinIO test.
|
|
|
|
Expected: all PASS, zero skipped, and no access to developer databases or existing containers.
|
|
|
|
- [x] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add scripts/backend-optimization-compose.yml tests
|
|
git commit -m "test: add isolated backend integration evidence"
|
|
```
|
|
|
|
### Task 9: Implement The Executable Closure SOP And Operator Guide
|
|
|
|
**Files:**
|
|
- Create: `scripts/backend-optimization-sop.sh`
|
|
- Create: `docs/backend-optimization-sop.md`
|
|
- Modify: `.gitignore`
|
|
- Create: `tests/BuildFlow.Architecture.Tests/BackendOptimizationSopTests.cs`
|
|
|
|
- [x] **Step 1: Write SOP contract tests in RED**
|
|
|
|
The architecture test reads the script and Compose file and asserts:
|
|
|
|
```text
|
|
set -euo pipefail
|
|
EXIT trap performs project-scoped docker compose down -v
|
|
all four isolated services exist and use random host ports
|
|
stages appear in design order
|
|
dotnet build requires no restore and warnings-as-errors
|
|
TRX results and logs go under a timestamped artifact directory
|
|
skipped > 0 causes failure
|
|
static scans include password, one-argument exception, endpoint DbContext, InProcess, multi-type, DateTime.Now
|
|
no appsettings.Secrets.json write and no developer database name
|
|
```
|
|
|
|
- [x] **Step 2: Run the contract test to verify RED**
|
|
|
|
```bash
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests
|
|
```
|
|
|
|
Expected: FAIL because the script and guide do not exist.
|
|
|
|
- [x] **Step 3: Implement the fail-fast SOP**
|
|
|
|
The script must:
|
|
|
|
```text
|
|
1. verify dotnet 10, docker, docker compose, git, rg, curl, and repository cleanliness policy
|
|
2. run Roslyn/static guards and explicit final-debt scans
|
|
3. restore once, then build with --no-restore and capture 0 warning/0 error
|
|
4. run non-external unit and architecture tests
|
|
5. start isolated Compose services and wait for health
|
|
6. create isolated databases and run empty + previous-version migration paths
|
|
7. run PostgreSQL, Redis, MongoDB/SignalR, MinIO, and endpoint/SSE tests
|
|
8. run two-worker claim, crash/lease recovery, restart-equivalent recovery tests
|
|
9. run long-context, large-form, large-Excel, and large-directory/performance gates
|
|
10. run the complete solution test suite, reject any skipped test, aggregate TRX counts and SOP_METRIC lines, and write summary.md
|
|
```
|
|
|
|
Every stage runs through a helper that records command, start/end timestamps, duration, exit status, and log path. Any command failure exits non-zero; the final success marker is written only after every gate passes. The cleanup trap removes only the unique Compose project.
|
|
|
|
- [x] **Step 4: Document operation and evidence semantics**
|
|
|
|
`docs/backend-optimization-sop.md` must explain prerequisites, exact invocation, optional artifact root, isolated resource behavior, environment variables, stage meanings, report layout, failure triage, and the rule that a failed/skipped gate is not completion.
|
|
|
|
- [x] **Step 5: Ignore generated evidence while preserving the directory convention**
|
|
|
|
Add:
|
|
|
|
```gitignore
|
|
artifacts/backend-optimization-sop/
|
|
```
|
|
|
|
- [x] **Step 6: Run syntax and contract tests**
|
|
|
|
```bash
|
|
bash -n scripts/backend-optimization-sop.sh
|
|
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [x] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add scripts/backend-optimization-sop.sh docs/backend-optimization-sop.md .gitignore tests/BuildFlow.Architecture.Tests
|
|
git commit -m "chore: add backend optimization closure sop"
|
|
```
|
|
|
|
### Task 10: Run The Clean Full SOP And Close The Audit
|
|
|
|
**Files:**
|
|
- Runtime output only: `artifacts/backend-optimization-sop/<timestamp>/`
|
|
- Modify documentation only if the run reveals a missing command or ambiguous failure path.
|
|
|
|
- [ ] **Step 1: Confirm tracked worktree state and run the SOP from repository root**
|
|
|
|
```bash
|
|
git status --short
|
|
scripts/backend-optimization-sop.sh
|
|
```
|
|
|
|
Expected: the script starts a unique isolated Compose project, does not touch existing `infra-*` containers, and exits 0.
|
|
|
|
- [ ] **Step 2: Verify the archived report**
|
|
|
|
The report must contain:
|
|
|
|
```text
|
|
all commands and stage durations
|
|
build 0 warning / 0 error
|
|
total passed / failed / skipped counts with skipped = 0
|
|
empty and upgrade migration evidence
|
|
SSE source/content/DONE protocol evidence
|
|
PostgreSQL/Redis/MongoDB/MinIO integration evidence
|
|
concurrency and recovery test evidence
|
|
query-count, remote-call-concurrency, Redis, context-window, transaction, and memory metrics
|
|
failure log paths (empty on success, populated on failure)
|
|
```
|
|
|
|
- [ ] **Step 3: Run final debt scans independently of the SOP**
|
|
|
|
```bash
|
|
rg -n "Task\.Run" src/modules/workflow src/BuildFlow.Host -g '*.cs'
|
|
rg -n "InProcess(UserInfoService|FileStorageClient|AvatarUrlResolver)" src -g '*.cs'
|
|
rg -n "DateTime\.Now" src -g '*.cs'
|
|
dotnet test BuildFlow.slnx --no-restore
|
|
dotnet build BuildFlow.slnx --no-restore
|
|
```
|
|
|
|
Expected: no forbidden debt matches; all tests PASS, 0 skipped; build reports 0 warning and 0 error.
|
|
|
|
- [ ] **Step 4: Commit any run-discovered SOP corrections**
|
|
|
|
```bash
|
|
git add scripts docs tests
|
|
git commit -m "test: close backend optimization audit"
|
|
```
|
|
|
|
Skip this commit when the clean run requires no tracked correction.
|
|
|
|
## Self-Review
|
|
|
|
- Spec coverage: Tasks 2-3 cover Endpoint CQRS; Tasks 4-5 remove all three bridges; Task 6 covers all explicit error codes; Task 7 covers all seven multi-type files; Task 1 enforces the four new source-wide guards while existing `HostArchitectureTests` retains Program/launchSettings/Api checks; Tasks 8-10 implement and execute the full SOP.
|
|
- Placeholder scan: the plan contains no unresolved placeholders. Every task identifies exact files, commands, expected RED/GREEN outcomes, and commit boundaries.
|
|
- Type consistency: `RAGStreamQuery` returns `RAGStreamEvent`; `GetUserSummariesQuery` returns `Dictionary<Guid, UserSummaryDto>`; `ResolveFilePreviewPathsQuery` returns File-owned preview DTOs; callers only inject `IMediator`.
|
|
- Safety: all external tests accept explicit environment variables; Docker assigns random host ports; the unique Compose project and volumes are removed by an EXIT trap; no local Secrets file or developer database is written.
|