CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Two systems in one repo

This repo hosts two generations of a resource-pool service. Know which one you are touching:

ResourcePool (legacy) — manages a token pool and a proxy pool. Server is Icomm.ResourcePool.Manager (ASP.NET, .NET 7). Clients consume it via the Icomm.TokenPool SDK or call the proxy pool directly over gRPC/HTTP. Shared contracts live in Icomm.ResourcePool.Abstractions. External ports: HTTP 31249, gRPC 31287.
SmartPool (new) — intelligent proxy management. Server is Icomm.API.SmartPool (ASP.NET, .NET 10). Clients use the Icomm.SmartPool.Proxy SDK. Shared contracts live in Icomm.SmartPool.Abstractions. Listens on HTTP :5000, gRPC :5001 (in-cluster). The active branch (smart-resourcepool) is centered on this system.

The legacy and smart stacks do not share code — they share only the repo, the ConfigManager remote-config dependency, and MagicOnion as the gRPC framework. Do not wire one into the other.

Many of the root SMARTPOOL_*.md / *_FIX*.md docs are point-in-time notes and have drifted (e.g. SMARTPOOL_README.md still says .NET 8 / MagicOnion 5 / ClickHouse.Client 7 — the code is now .NET 10 / MagicOnion 6.1.7 / ClickHouse.Driver 0.9.0). Trust the .csproj files over the markdown. MIGRATION_GUIDE_PROXYPOOL_TO_SMARTPOOL.md is the most useful cross-cutting doc.

Build / test / run

Solution: Icomm.ResourcePool.sln. Requires the .NET 10 SDK (the smart projects and tests target net10.0; the legacy SDKs target netstandard2.1 and the Manager host targets net7.0 but all build under the .NET 10 SDK).

dotnet build Icomm.ResourcePool.sln -c Release          # build everything
dotnet test                                              # run all tests (Icomm.SmartPool.Tests, xUnit)
dotnet test --filter "FullyQualifiedName~RandomStrategyTests"   # one test class
dotnet test --filter "DisplayName~picks the least delayed"      # one test by name

dotnet run --project src/Icomm.API.SmartPool            # run SmartPool API locally
dotnet run --project src/Icomm.ResourcePool.Manager     # run legacy Manager

Only Icomm.SmartPool.Tests exists (xUnit + Moq + FluentAssertions); it covers strategies, the strategy factory, and the client SDK. There are no tests for the legacy Manager.

Docker / CI: Only src/Icomm.ResourcePool.Manager/Dockerfile exists (multi-stage, .NET 7 alpine→aspnet, sets Asia/Ho_Chi_Minh TZ). There is no Dockerfile for Icomm.API.SmartPool yet and no docker-compose.yml. The legacy service ships via Jenkins (root Jenkinsfile → src/Icomm.ResourcePool.Manager/Jenkinsfile: docker build → push to registry → kubectl apply of deployment.yaml). The only .gitea workflow just syncs *.md files to a public mirror repo — it does not build or deploy.

SmartPool architecture (`Icomm.API.SmartPool`)

Request entry points both implement the same logic surface:

gRPC: Services/SmartProxyGrpcService.cs — MagicOnion ServiceBase<ISmartProxyService>. This is the primary, high-throughput path.
HTTP REST: Controllers/v1/SmartProxyController.cs — routes under /api/smart-pool/v1/SmartProxy.

Core flow for "get a proxy":

Validate access_token; reject if it has no cluster mapping (IProxyMetadataRepository.HasAccessTokenMappingAsync). Access control is token → cluster, never token → individual proxy.
Validate the requested strategy name against the registered set; alternative additionally requires a referer_proxy.
Load the candidate proxies the token may use, then delegate selection to a strategy.

Strategy pattern (Strategies/) is the heart of the system. Every strategy implements IProxyPickStrategy (StrategyName, ValidateRequest, PickProxyAsync). All implementations are registered as IEnumerable<IProxyPickStrategy> in DI; ProxyStrategyFactory.GetStrategy(name) resolves by name and falls back to round_robin when unknown. To add a strategy: implement the interface, register it in Infrastructure/ServiceCollectionExtensions.cs, and it is automatically discoverable. The five strategies:

random — uniform pick from the permitted set.
round_robin — rotates sequentially; index persisted in Redis for even distribution.
least_delay — lowest recent response time for a target_domain (last 24h); domain required.
adaptive_ranking — score-ranked (success_rate weighted across recent windows, last 7d); domain required.
alternative — finds a substitute similar to referer_proxy (same cluster/ip_version/country/etc., weighted by AlternativeStrategyOptions in config).

Data layer — ClickHouse, split into two databases by lifecycle:

smart_pool_meta: proxy_metadata, proxy_access_mapping (ReplacingMergeTree, no TTL, queried with FINAL) → ProxyMetadataRepository.
smart_pool_logs: proxy_usage_logs (TTL ~90d) plus a Refreshable Materialized View that aggregates hourly stats every ~5 min → ProxyLogRepository. Adaptive scoring reads the MV; there is no background scoring service (it was removed in favor of the MV — see the note in ServiceCollectionExtensions.cs).
ClickHouse access is via Dapper over ClickHouse.Driver (Infrastructure/ClickHouseContext.cs, ClickHouseConnectionWrapper.cs). SQL schema lives in src/Icomm.API.SmartPool/clickhouse_init.sql (_test_data / _debug variants alongside).

Write path for logs is asynchronous: usage logs are produced to Kafka (KafkaProducerService, a singleton IHostedService doing batch/fire-and-forget; topic and tuning in KafkaLogsOptions). A separate consumer (outside this repo) lands them in ClickHouse. LogProxyUsageBatch is the high-throughput variant.

Performance-critical infrastructure (Infrastructure/): HotDataCache (in-memory, singleton), Redis via EasyCaching (src/EasyCaching.Redis, a vendored provider), CircuitBreaker, gRPC + HTTP response compression (gzip/brotli, CompressionLevel.Fastest), and 1 MB gRPC message limits. This service is tuned for latency/throughput — preserve the caching tiers (hot in-memory → Redis → ClickHouse) and the compact-response/batch endpoints when changing behavior.

Legacy ResourcePool architecture

Host: Icomm.ResourcePool.Manager (Startup.cs). MagicOnion gRPC on :5002 plus a MagicOnion HTTP gateway (_gate) and Swagger gateway; responses wrapped by AutoWrapper for paths under /api.
Controllers (Controllers/v1/): TokenPoolController, ProxyPoolController. Business logic in Services/ (TokenService, ProxyService, ElasticProvider); persistence in Controllers/Data/ repositories (TokenRepository, ProxyRepository, RateLimitRepository) backed by ClickHouse (DTO/Context/) and Elasticsearch (ElasticConnectionOptions).
Client SDK: Icomm.TokenPool (netstandard2.1, published as NuGet Icomm.TokenPool). Register with services.AddConfigManager(...).AddTokenPool(...); consume ITokenPoolService (RequestToken, UsedToken, ExpireBy*, UpdateStatus). Proxy pool has no SDK — clients call gRPC (IProxyService via MagicOnionClient) or the HTTP REST endpoints (/api/resource-pool/v1/ProxyPool/...) directly.

Cross-cutting conventions

gRPC contracts use MagicOnion, not .proto files. The C# interface in the *.Abstractions project (e.g. ISmartProxyService, IProxyService) is the contract; client and server share that assembly. Changing a method signature is a breaking wire change for both sides.
Request/response field names are snake_case in SmartPool DTOs (access_token, ip_version, target_domain, referer_proxy) — this is intentional for the public contract; match it.
Remote configuration comes from ConfigManager (Icomm.Configs.Providers.HttpProvider). SmartPool: builder.Configuration.AddConfigManagerHttpProvider() + AddConfigManager(...). The ConfigManager:AccessToken in appsettings.json authenticates to the config server; real settings (ClickHouse/Redis/Kafka connection details) are fetched remotely, which is why local appsettings.json often has those sections empty or commented out.
SmartPool error handling uses numeric codes returned in the response (no exceptions across the wire): 1001 AccessTokenRequired, 1002 AccessTokenInvalid, 2001 NoAvailableProxies, 2002 NoMatchingProxies, 3001 StrategyNotFound, 3002 RefererProxyRequired, 4001 InvalidRequest, 5001 InternalError. Build them via Helpers/ProxyErrorHelper.
Shared/abstraction projects are published as NuGet packages to a private feed (source ic / ProGet; see push-proget.sh, nuget.config). Icomm.SmartPool.Abstractions multi-targets net10.0;net8.0 so older consumers can reference it — keep it consumer-compatible and bump the <Version> when changing public types.

8.8 KiB Raw Blame History