build-flow/build-flow.api
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
build-flow.api/docs/superpowers/plans/2026-07-24-buildflow-architecture-and-sop.md

32 KiB
Raw Blame History

BuildFlow Architecture Closure And Backend SOP Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task in the current session. Do not dispatch subagents for this plan. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Complete the fifth optimization batch by enforcing source-wide architecture rules, moving the remaining endpoints and module integrations to MediatR CQRS, normalizing business error codes and file organization, and producing a fully isolated, executable, evidence-archiving backend SOP.

Architecture: BuildFlow.Host remains the only HTTP/composition root. Endpoints only translate HTTP/SSE contracts to target-module Application commands or stream requests; every cross-module call is an IMediator send to a target-owned Query/Command/DTO. Roslyn architecture tests scan all current source files, and the SOP starts isolated random-port infrastructure before running migrations, integration, concurrency, recovery, and large-data gates.

Tech Stack: .NET 10, FastEndpoints 8.1, MediatR 12.5, FluentValidation 12.1, EF Core 10/Npgsql, Roslyn 5.0, xUnit, PostgreSQL/pgvector, Redis, MongoDB, MinIO, Docker Compose, Bash.

File Map

The implementation creates or changes the following ownership units:

  • tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs: source-wide Roslyn guards for endpoint dependencies, Host bridges, error-code arguments, and top-level type count.
  • src/modules/platform/Application/Auth/**: change-password command, handler, and validator.
  • src/modules/rag/Application/RagQA/**: RAG stream request, stream event, handler, and validator.
  • src/modules/platform/Application/Users/**: Platform-owned batch user-summary query and DTO consumed by IM.
  • src/modules/file/Application/Features/Files/**: File-owned batch preview-path query and DTO consumed by Platform.
  • IM/RAG/Platform handlers: inject IMediator and send target-owned contracts directly.
  • src/BuildFlow.Host/Services/Integration/InProcess*.cs: deleted after all consumers move to CQRS.
  • Workflow/RAG/File throw sites: explicit semantic ErrorCode arguments.
  • Consul/workflow graph/tree and four test support files: split to one top-level class/record/enum per file.
  • scripts/backend-optimization-compose.yml: isolated PostgreSQL, Redis, MongoDB, and MinIO services with random host ports.
  • scripts/backend-optimization-sop.sh: fail-fast orchestrator and artifact writer.
  • docs/backend-optimization-sop.md: operator contract, prerequisites, phases, outputs, and failure handling.
  • artifacts/backend-optimization-sop/<timestamp>/: generated only at runtime and ignored by Git.

Task 1: Add Source-Wide Architecture Guards In RED State

Files:

  • Modify: tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj

  • Create: tests/BuildFlow.Architecture.Tests/SourceArchitectureTests.cs

  • Step 1: Add Roslyn to the architecture test project

Add the cached repository-compatible package version:

<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="5.0.0" />
  • Step 2: Write the four source-wide failing tests

SourceArchitectureTests must locate BuildFlow.slnx, enumerate non-generated .cs files under src and tests, parse each with CSharpSyntaxTree.ParseText, and expose four facts:

[Fact]
public void EndpointConstructorsShouldNotDependOnDbContexts();

[Fact]
public void HostShouldNotContainInProcessBridges();

[Fact]
public void BusinessExceptionsShouldSpecifySemanticErrorCodes();

[Fact]
public void SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum();

The tests must report relative path and 1-based line number. The exception guard applies to BusinessException, NotFoundException, and UnauthorizedException, requires at least two arguments, and requires the first expression to be ErrorCode.<Member>. The file guard counts only top-level class, record, and enum; nested declarations remain allowed.

  • Step 3: Run each guard and verify the expected RED inventory

Run:

dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceArchitectureTests --logger "console;verbosity=normal"

Expected: FAIL and report exactly the current categories: two DbContext endpoints, three Host InProcess* bridges, 95 one-argument business exceptions, and seven multi-top-level-type files. The 95th site is a namespace-qualified exception in UploadDocumentEndpoint that the earlier regular-expression audit did not detect.

  • Step 4: Commit the RED guard
git add tests/BuildFlow.Architecture.Tests
git commit -m "test: enforce backend source architecture"

Task 2: Move Change Password To Platform CQRS

Files:

  • Create: src/modules/platform/Application/Auth/Commands/ChangePasswordCommand.cs

  • Create: src/modules/platform/Application/Auth/Handlers/ChangePasswordCommandHandler.cs

  • Create: src/modules/platform/Application/Auth/Validators/ChangePasswordCommandValidator.cs

  • Modify: src/BuildFlow.Host/Endpoints/Platform/Auth/ChangePasswordEndpoint.cs

  • Create: tests/BuildFlow.Platform.Tests/Auth/ChangePasswordCommandHandlerTests.cs

  • Create: tests/BuildFlow.Platform.Tests/Auth/ChangePasswordEndpointTests.cs

  • Modify: tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj

  • Step 1: Write handler and endpoint tests first

Cover these behaviors:

valid old password -> password hash changes and command completes
wrong old password -> BusinessException(ErrorCode.Business)
same old/new password -> BusinessException(ErrorCode.Business)
missing current user -> UnauthorizedException(ErrorCode.Unauthorized)
endpoint -> sends ChangePasswordCommand and writes an OK response

The endpoint test must use FastEndpoints.Factory.Create<ChangePasswordEndpoint> with a mocked IMediator; it must not construct a PlatformDbContext.

  • Step 2: Run tests to verify RED
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword" --logger "console;verbosity=normal"

Expected: FAIL because the command/handler do not exist and the endpoint still requires PlatformDbContext.

  • Step 3: Implement the Platform command boundary

Use this contract:

public sealed record ChangePasswordCommand(string OldPassword, string NewPassword) : IRequest;

The handler injects PlatformDbContext and ICurrentUserContext, verifies the current user and BCrypt hash, throws explicit semantic exceptions, updates the hash, and calls SaveChangesAsync(ct). The validator enforces non-empty old/new passwords and the existing password policy. The endpoint becomes:

public class ChangePasswordEndpoint(IMediator mediator) : Endpoint<ChangePasswordRequest>
{
    public override async Task HandleAsync(ChangePasswordRequest req, CancellationToken ct)
    {
        await mediator.Send(new ChangePasswordCommand(req.OldPassword, req.NewPassword), ct);
        await Send.OkAsync(ct);
    }
}
  • Step 4: Run focused tests and the endpoint architecture guard
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~ChangePassword"
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts

Expected: change-password tests PASS; the architecture test still reports only RAGStreamEndpoint.

  • Step 5: Commit
git add src/modules/platform/Application/Auth src/BuildFlow.Host/Endpoints/Platform/Auth tests/BuildFlow.Platform.Tests
git commit -m "refactor: move password changes to platform cqrs"

Task 3: Move RAG SSE Generation To An Application Stream Handler

Files:

  • Create: src/modules/rag/Application/RagQA/Queries/RAGStreamQuery.cs

  • Create: src/modules/rag/Application/RagQA/DTOs/RAGStreamEvent.cs

  • Create: src/modules/rag/Application/RagQA/Handlers/RAGStreamQueryHandler.cs

  • Create: src/modules/rag/Application/RagQA/Validators/RAGStreamQueryValidator.cs

  • Modify: src/BuildFlow.Host/Endpoints/Rag/RAG/RAGStreamEndpoint.cs

  • Create: tests/BuildFlow.Rag.Tests/RagStreamQueryHandlerTests.cs

  • Create: tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj

  • Create: tests/BuildFlow.Host.Tests/RagStreamEndpointTests.cs

  • Modify: BuildFlow.slnx

  • Step 1: Write stream handler and endpoint protocol tests

Use a target-owned request:

public sealed record RAGStreamQuery(Guid KnowledgeBaseId, string Question)
    : IStreamRequest<RAGStreamEvent>;

Test event order for hits (Sources, content chunks, Done), no hits (fallback content, Done), missing knowledge base (NotFoundException(ErrorCode.NotFound)), and cancellation propagation. Use Factory.Create<RAGStreamEndpoint> to assert exact SSE output including event: sources, JSON data: content frames, and final data: [DONE].

  • Step 2: Run tests to verify RED
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
dotnet test tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"

Expected: FAIL because the request/event/handler do not exist and the endpoint owns retrieval/chat logic.

  • Step 3: Implement the stream handler

RAGStreamEvent must be the only top-level type in its file and expose source/content/done factories without adding an enum file:

public sealed record RAGStreamEvent(
    IReadOnlyList<SourceChunk>? Sources,
    string? Content,
    bool IsDone);

The handler injects RagDbContext, IRagRetrievalService, and IAIChatAgent; it retrieves the knowledge base, emits source summaries truncated at 200 characters, builds the bounded prompt with the current retrieval result, streams chat chunks, and emits Done. The endpoint injects only IMediator, sets SSE headers before enumeration, encodes events, flushes after every frame, and preserves the current wire protocol byte-for-byte.

  • Step 4: Run focused tests and endpoint guard
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
dotnet test tests/BuildFlow.Host.Tests/BuildFlow.Host.Tests.csproj --no-restore --filter "FullyQualifiedName~RagStream"
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~EndpointConstructorsShouldNotDependOnDbContexts

Expected: PASS; no endpoint constructor depends on a DbContext.

  • Step 5: Commit
git add BuildFlow.slnx src/modules/rag/Application/RagQA src/BuildFlow.Host/Endpoints/Rag/RAG tests/BuildFlow.Rag.Tests tests/BuildFlow.Host.Tests
git commit -m "refactor: stream rag answers through cqrs"

Task 4: Replace IM User Information Bridge With A Platform Batch Query

Files:

  • Create: src/modules/platform/Application/Users/DTOs/UserSummaryDto.cs

  • Create: src/modules/platform/Application/Users/Queries/GetUserSummariesQuery.cs

  • Create: src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs

  • Modify: src/modules/im/Application/IM.Application.csproj

  • Modify: seven handlers under src/modules/im/Application/Chat/Handlers/ that inject IUserInfoService

  • Delete: src/modules/im/Domain/Common/IUserInfoService.cs

  • Delete: src/BuildFlow.Host/Services/Integration/InProcessUserInfoService.cs

  • Modify: src/BuildFlow.Host/BuildFlowHostModule.cs

  • Modify: IM unit tests using Mock<IUserInfoService>

  • Replace: tests/BuildFlow.IM.Integration.Tests/InMemoryUserInfoService.cs with InMemoryUserSummaryQueryHandler.cs

  • Modify: tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs

  • Step 1: Write Platform query tests and update IM tests to expect mediator sends

The Platform query returns an IReadOnlyDictionary<Guid, UserSummaryDto> for distinct requested IDs and performs one database query. IM tests must configure IMediator.Send(new GetUserSummariesQuery(...), ct) and verify that empty input avoids a send where practical.

  • Step 2: Run focused tests to verify RED
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter FullyQualifiedName~GetUserSummaries
dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore --filter "FullyQualifiedName~GetConversations|FullyQualifiedName~SendMessage"

Expected: FAIL until the target-owned query and IM handler changes exist.

  • Step 3: Implement the direct CQRS calls

Add a Platform Application project reference to IM Application. Each IM handler injects IMediator, sends one GetUserSummariesQuery for its deduplicated IDs, and reads results by Guid key. Do not catch all exceptions and do not introduce a replacement service/interface bridge.

The integration fake implements IRequestHandler<GetUserSummariesQuery, IReadOnlyDictionary<Guid, UserSummaryDto>>; ImServerFixture registers both the IM Application assembly and the test assembly with MediatR.

  • Step 4: Remove bridge interface, Host implementation, and registration

Delete IUserInfoService and InProcessUserInfoService; remove the Host AddScoped<IUserInfoService, InProcessUserInfoService>() registration and stale using directives/comments.

  • Step 5: Run IM unit and SignalR integration tests
dotnet test tests/BuildFlow.IM.Tests/BuildFlow.IM.Tests.csproj --no-restore
dotnet test tests/BuildFlow.IM.Integration.Tests/BuildFlow.IM.Integration.Tests.csproj --no-restore

Expected: PASS with the same private/group conversation display names and notification behavior.

  • Step 6: Commit
git add src/modules/platform/Application/Users src/modules/im src/BuildFlow.Host tests/BuildFlow.IM.Tests tests/BuildFlow.IM.Integration.Tests
git commit -m "refactor: query platform users through mediator"

Task 5: Replace File And Avatar Bridges With File Application Contracts

Files:

  • Create: src/modules/file/Application/Features/Files/DTOs/FilePreviewPathDto.cs

  • Create: src/modules/file/Application/Features/Files/Queries/ResolveFilePreviewPathsQuery.cs

  • Create: src/modules/file/Application/Features/Files/Handlers/ResolveFilePreviewPathsQueryHandler.cs

  • Create: tests/BuildFlow.File.Tests/ResolveFilePreviewPathsQueryHandlerTests.cs

  • Modify: src/modules/platform/Application/Platform.Application.csproj

  • Modify: src/modules/platform/Application/Auth/Handlers/GetCurrentUserQueryHandler.cs

  • Modify: src/modules/platform/Application/Auth/Handlers/UpdateProfileCommandHandler.cs

  • Modify: src/modules/platform/Application/Users/Handlers/GetUserSummariesQueryHandler.cs

  • Delete: src/modules/platform/Domain/Services/IAvatarUrlResolver.cs

  • Modify: src/modules/rag/Application/BuildFlow.Application.csproj

  • Modify: RAG document upload/process/preview handlers and their tests

  • Delete: src/modules/rag/Domain/Interfaces/IFileStorageClient.cs

  • Delete: src/modules/rag/Infrastructure/Files/FileStorageClient.cs

  • Modify: src/modules/rag/Infrastructure/RAGInfrastructureModule.cs

  • Delete: src/BuildFlow.Host/Services/Integration/InProcessFileStorageClient.cs

  • Delete: src/BuildFlow.Host/Services/Integration/InProcessAvatarUrlResolver.cs

  • Modify: src/BuildFlow.Host/BuildFlowHostModule.cs

  • Step 1: Write File preview-path query tests

Test null/blank references, existing HTTP/HTTPS URLs, object-key URL escaping, duplicate inputs, and batch output. The target-owned handler must return /api/files/preview?objectKey=<escaped> without touching S3.

  • Step 2: Update Platform tests to expect one File batch query

GetCurrentUserQueryHandler, UpdateProfileCommandHandler, and GetUserSummariesQueryHandler inject IMediator. A batch user query sends one ResolveFilePreviewPathsQuery for distinct non-empty avatar references to avoid N+1 path resolution.

  • Step 3: Update RAG tests to expect direct File Application sends

RAG upload sends UploadFileCommand(objectKey, stream, contentType, fileSize); process and preview send DownloadFileQuery(objectKey) and dispose the returned stream. Remove IHttpContextAccessor, auth-token extraction, and IFileStorageClient stubs from these paths.

  • Step 4: Run focused tests to verify RED
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore --filter FullyQualifiedName~ResolveFilePreviewPaths
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore --filter "FullyQualifiedName~CurrentUser|FullyQualifiedName~UpdateProfile|FullyQualifiedName~GetUserSummaries"
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore --filter "FullyQualifiedName~Document|FullyQualifiedName~PreviewDocument"

Expected: FAIL before implementation and PASS after the direct mediator changes.

  • Step 5: Delete the old interfaces, implementations, and DI registrations

Remove all three Host InProcess* classes and both source-module service interfaces. Remove the obsolete RAG HTTP FileStorageClient registration and class. rg must return no production references:

rg -n "InProcessUserInfoService|InProcessFileStorageClient|InProcessAvatarUrlResolver|IUserInfoService|IFileStorageClient|IAvatarUrlResolver" src

Expected: no matches.

  • Step 6: Run the Host bridge guard and affected test projects
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~HostShouldNotContainInProcessBridges
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore

Expected: PASS.

  • Step 7: Commit
git add src/modules/file src/modules/platform src/modules/rag src/BuildFlow.Host tests/BuildFlow.File.Tests tests/BuildFlow.Platform.Tests tests/BuildFlow.Rag.Tests
git commit -m "refactor: remove host module service bridges"

Task 6: Normalize Every Business Exception Error Code

Files:

  • Modify: 95 throw/return sites reported by the architecture guard in Host, File, RAG, and Workflow.

  • Modify tests only where they assert exception codes.

  • Step 1: Add focused semantic-code assertions before production edits

Extend existing tests for representative categories: duplicate resource, missing resource, invalid state, invalid parameter/schema, and forbidden operation. Assert the exception Code property, not only the type/message.

  • Step 2: Run the representative tests and architecture guard in RED
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore --filter "FullyQualifiedName~HandlerTests|FullyQualifiedName~ProcessEngineTests"
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes

Expected: representative code assertions and the architecture guard fail on old one-argument calls.

  • Step 3: Apply the semantic mapping deliberately

Use this mapping at every reported site:

missing/not found -> NotFoundException(ErrorCode.NotFound, message)
duplicate/already exists -> BusinessException(ErrorCode.AlreadyExists, message)
disabled/deleted/already processed/unsupported transition -> BusinessException(ErrorCode.InvalidState, message)
invalid JSON/schema/value/path/missing action argument -> BusinessException(ErrorCode.InvalidParam, message)
permission denied -> UnauthorizedException(ErrorCode.Forbidden, message)
rule conflict without a narrower category -> BusinessException(ErrorCode.Business, message)

Convert existing BusinessException calls that describe a missing resource to NotFoundException where the surrounding contract already treats absence as 404. Do not replace all calls with a single code.

  • Step 4: Run the guard and module tests
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BusinessExceptionsShouldSpecifySemanticErrorCodes
dotnet test tests/BuildFlow.File.Tests/BuildFlow.File.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore

Expected: PASS and zero old one-argument calls.

  • Step 5: Commit
git add src/modules/file src/modules/rag src/modules/workflow tests
git commit -m "refactor: classify backend business errors"

Task 7: Split Every Multi-Type Source File

Files:

  • Split src/BuildFlow.Host/Configuration/ConsulConfigurationProvider.cs into ConsulConfigurationSource.cs, ConsulOptions.cs, ConsulConfigurationProvider.cs, and ConsulKvItem.cs.

  • Split src/modules/workflow/Domain/Engine/WorkflowGraph.cs into WorkflowGraph.cs, GraphNode.cs, GraphEdge.cs, GraphEndpoint.cs, GraphPoint.cs, and GraphJsonExtensions.cs.

  • Split src/modules/workflow/Domain/Engine/WorkflowTree.cs into WorkflowTree.cs, TreeNode.cs, and TreeBranch.cs.

  • Create tests/BuildFlow.Platform.Tests/DictionaryConfiguration.cs from PlatformTestDb.cs.

  • Create tests/BuildFlow.Rag.Tests/Stub/StubDocumentExtractorFactory.cs from StubDocumentExtractor.cs.

  • Create tests/BuildFlow.Workflow.Tests/Form/FormTestFixtureClassFixture.cs from FormTestFixture.cs.

  • Create tests/BuildFlow.Workflow.Tests/Notification/NoOpPlatformNotificationPublisher.cs from NotificationServiceTests.cs.

  • Step 1: Move declarations without changing behavior

Preserve namespaces, modifiers, XML comments, JSON attributes, extension methods, and member order. Each original file retains its namesake type; each moved declaration gets a same-named file.

  • Step 2: Run the file organization guard
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~SourceFilesShouldContainAtMostOneTopLevelClassRecordOrEnum

Expected: PASS with zero violations under src and tests.

  • Step 3: Run affected projects
dotnet test tests/BuildFlow.Platform.Tests/BuildFlow.Platform.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Rag.Tests/BuildFlow.Rag.Tests.csproj --no-restore
dotnet test tests/BuildFlow.Workflow.Tests/BuildFlow.Workflow.Tests.csproj --no-restore
dotnet build src/BuildFlow.Host/BuildFlow.Host.csproj --no-restore

Expected: PASS, 0 warning, 0 error.

  • Step 4: Commit
git add src/BuildFlow.Host/Configuration src/modules/workflow/Domain/Engine tests
git commit -m "refactor: enforce one top-level type per file"

Task 8: Add Isolated Infrastructure, Real S3 Coverage, And SOP Metrics

Files:

  • Create: scripts/backend-optimization-compose.yml

  • Create: tests/BuildFlow.File.Tests/S3StorageServiceIntegrationTests.cs

  • Modify: tests/BuildFlow.IM.Integration.Tests/ImServerFixture.cs

  • Modify: tests/BuildFlow.IM.Tests/Chat/SendMessageCommandHandlerTests.cs

  • Modify: selected performance tests to emit SOP_METRIC key=value through ITestOutputHelper

  • Create: tests/BuildFlow.Workflow.Tests/Performance/OpenXmlMemoryProfileTests.cs

  • Step 1: Add environment-driven isolated connection support

Replace fixed IM test endpoints with these environment variables and retain current values only as local fallbacks:

BUILDFLOW_IM_TEST_POSTGRES_CONNECTION
BUILDFLOW_IM_TEST_MONGO_CONNECTION
BUILDFLOW_IM_TEST_REDIS_CONNECTION
BUILDFLOW_WORKFLOW_TEST_CONNECTION
BUILDFLOW_MONITORING_TEST_CONNECTION
BUILDFLOW_RAG_REDIS_TEST_CONNECTION
BUILDFLOW_FILE_S3_TEST_ENDPOINT
BUILDFLOW_FILE_S3_TEST_ACCESS_KEY
BUILDFLOW_FILE_S3_TEST_SECRET_KEY
  • Step 2: Add a real MinIO S3 integration test

The test must create a unique bucket, upload a non-empty object, download and byte-compare it, verify a zero-byte object, delete objects/bucket in finally, and emit object count/bytes as SOP_METRIC. It must fail rather than skip when its configured endpoint is unavailable.

  • Step 3: Add deterministic metric output

Emit at least these values from passing tests:

form_aggregate_select_count
import_commit_count
redis_concurrent_append_count
redis_list_length
rag_context_message_count
rag_context_estimated_tokens
file_size_max_s3_concurrency
excel_export_peak_managed_bytes
excel_export_rows

OpenXmlMemoryProfileTests writes a large export to a temporary file from an IAsyncEnumerable<string>, samples GC.GetTotalMemory(false) while writing, asserts a conservative bounded peak delta, emits the observed bytes, and always deletes the file.

  • Step 4: Add random-port Docker Compose infrastructure

Use pinned images already compatible with the repository:

pgvector/pgvector:0.8.0-pg17
redis:7-alpine
mongo:7
minio/minio:RELEASE.2025-04-22T22-12-26Z

Publish container ports with Docker-assigned random loopback ports, add health checks, use test-only credentials, and name volumes inside the per-run Compose project so teardown removes only SOP resources.

  • Step 5: Start the compose stack manually and run infrastructure tests

Use a unique Compose project name, discover ports with docker compose port, export the variables above, then run PostgreSQL reliability/migration tests, Redis tests, IM SignalR integration tests, Monitoring UPSERT tests, and the MinIO test.

Expected: all PASS, zero skipped, and no access to developer databases or existing containers.

  • Step 6: Commit
git add scripts/backend-optimization-compose.yml tests
git commit -m "test: add isolated backend integration evidence"

Task 9: Implement The Executable Closure SOP And Operator Guide

Files:

  • Create: scripts/backend-optimization-sop.sh

  • Create: docs/backend-optimization-sop.md

  • Modify: .gitignore

  • Create: tests/BuildFlow.Architecture.Tests/BackendOptimizationSopTests.cs

  • Step 1: Write SOP contract tests in RED

The architecture test reads the script and Compose file and asserts:

set -euo pipefail
EXIT trap performs project-scoped docker compose down -v
all four isolated services exist and use random host ports
stages appear in design order
dotnet build requires no restore and warnings-as-errors
TRX results and logs go under a timestamped artifact directory
skipped > 0 causes failure
static scans include password, one-argument exception, endpoint DbContext, InProcess, multi-type, DateTime.Now
no appsettings.Secrets.json write and no developer database name
  • Step 2: Run the contract test to verify RED
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests

Expected: FAIL because the script and guide do not exist.

  • Step 3: Implement the fail-fast SOP

The script must:

1. verify dotnet 10, docker, docker compose, git, rg, curl, and repository cleanliness policy
2. run Roslyn/static guards and explicit final-debt scans
3. restore once, then build with --no-restore and capture 0 warning/0 error
4. run non-external unit and architecture tests
5. start isolated Compose services and wait for health
6. create isolated databases and run empty + previous-version migration paths
7. run PostgreSQL, Redis, MongoDB/SignalR, MinIO, and endpoint/SSE tests
8. run two-worker claim, crash/lease recovery, restart-equivalent recovery tests
9. run long-context, large-form, large-Excel, and large-directory/performance gates
10. run the complete solution test suite, reject any skipped test, aggregate TRX counts and SOP_METRIC lines, and write summary.md

Every stage runs through a helper that records command, start/end timestamps, duration, exit status, and log path. Any command failure exits non-zero; the final success marker is written only after every gate passes. The cleanup trap removes only the unique Compose project.

  • Step 4: Document operation and evidence semantics

docs/backend-optimization-sop.md must explain prerequisites, exact invocation, optional artifact root, isolated resource behavior, environment variables, stage meanings, report layout, failure triage, and the rule that a failed/skipped gate is not completion.

  • Step 5: Ignore generated evidence while preserving the directory convention

Add:

artifacts/backend-optimization-sop/
  • Step 6: Run syntax and contract tests
bash -n scripts/backend-optimization-sop.sh
dotnet test tests/BuildFlow.Architecture.Tests/BuildFlow.Architecture.Tests.csproj --no-restore --filter FullyQualifiedName~BackendOptimizationSopTests

Expected: PASS.

  • Step 7: Commit
git add scripts/backend-optimization-sop.sh docs/backend-optimization-sop.md .gitignore tests/BuildFlow.Architecture.Tests
git commit -m "chore: add backend optimization closure sop"

Task 10: Run The Clean Full SOP And Close The Audit

Files:

  • Runtime output only: artifacts/backend-optimization-sop/<timestamp>/

  • Modify documentation only if the run reveals a missing command or ambiguous failure path.

  • Step 1: Confirm tracked worktree state and run the SOP from repository root

git status --short
scripts/backend-optimization-sop.sh

Expected: the script starts a unique isolated Compose project, does not touch existing infra-* containers, and exits 0.

  • Step 2: Verify the archived report

The report must contain:

all commands and stage durations
build 0 warning / 0 error
total passed / failed / skipped counts with skipped = 0
empty and upgrade migration evidence
SSE source/content/DONE protocol evidence
PostgreSQL/Redis/MongoDB/MinIO integration evidence
concurrency and recovery test evidence
query-count, remote-call-concurrency, Redis, context-window, transaction, and memory metrics
failure log paths (empty on success, populated on failure)
  • Step 3: Run final debt scans independently of the SOP
rg -n "Task\.Run" src/modules/workflow src/BuildFlow.Host -g '*.cs'
rg -n "InProcess(UserInfoService|FileStorageClient|AvatarUrlResolver)" src -g '*.cs'
rg -n "DateTime\.Now" src -g '*.cs'
dotnet test BuildFlow.slnx --no-restore
dotnet build BuildFlow.slnx --no-restore

Expected: no forbidden debt matches; all tests PASS, 0 skipped; build reports 0 warning and 0 error.

  • Step 4: Commit any run-discovered SOP corrections
git add scripts docs tests
git commit -m "test: close backend optimization audit"

Skip this commit when the clean run requires no tracked correction.

Self-Review

  • Spec coverage: Tasks 2-3 cover Endpoint CQRS; Tasks 4-5 remove all three bridges; Task 6 covers all explicit error codes; Task 7 covers all seven multi-type files; Task 1 enforces the four new source-wide guards while existing HostArchitectureTests retains Program/launchSettings/Api checks; Tasks 8-10 implement and execute the full SOP.
  • Placeholder scan: the plan contains no unresolved placeholders. Every task identifies exact files, commands, expected RED/GREEN outcomes, and commit boundaries.
  • Type consistency: RAGStreamQuery returns RAGStreamEvent; GetUserSummariesQuery returns Dictionary<Guid, UserSummaryDto>; ResolveFilePreviewPathsQuery returns File-owned preview DTOs; callers only inject IMediator.
  • Safety: all external tests accept explicit environment variables; Docker assigns random host ports; the unique Compose project and volumes are removed by an EXIT trap; no local Secrets file or developer database is written.