ResourcePool.Docs/docs/src/Icomm.API.SmartPool/README.md

Icomm.API.SmartPool

SmartPool API Backend - Intelligent proxy management system with multiple selection strategies.

Features

• Multiple Selection Strategies:

  • Random: Random selection from available proxies
  • Round Robin: Balanced distribution across proxies
  • Least Delay: Select fastest proxy based on response time
  • Adaptive Ranking: Select proxy with highest success rate
  • Alternative: Find similar proxy to replace failed one

• Dual Protocol Support:

  • gRPC (MagicOnion) for high-performance RPC
  • HTTP REST API for standard web clients

• ClickHouse Integration:

  • High-performance logging
  • Real-time analytics
  • Automatic aggregation with materialized views

Troubleshooting

"No proxy available matching criteria"

If you get this error, check:

1. Run initialization script:

   clickhouse-client < clickhouse_init.sql
   

2. Add test data:

   clickhouse-client < clickhouse_test_data.sql
   

3. Debug the issue:

   clickhouse-client < clickhouse_debug.sql
   

4. Verify your access_token has mappings:

   SELECT * FROM smart_pool_meta.proxy_access_mapping FINAL 
   WHERE access_token = 'YOUR_TOKEN';
   

5. Verify proxies exist in those clusters:

   SELECT m.* 
   FROM smart_pool_meta.proxy_metadata FINAL m
   INNER JOIN smart_pool_meta.proxy_access_mapping FINAL a 
       ON m.cluster = a.cluster
   WHERE a.access_token = 'YOUR_TOKEN'
       AND m.status = 1;
   

Common Issues

1. Empty database: Run clickhouse_init.sql and clickhouse_test_data.sql
2. No access mapping: Your access_token needs to be mapped to clusters
3. All proxies inactive: Check status = 1 in proxy_metadata
4. Wrong cluster names: Ensure cluster names match between metadata and mappings

Quick Start

1. Setup ClickHouse

# Run ClickHouse initialization script
clickhouse-client < clickhouse_init.sql

2. Configure Settings

Edit appsettings.json:

{
  "ClickHouse": {
    "Host": "localhost",
    "Port": 8123,
    "Database": "smart_pool",
    "Username": "default",
    "Password": ""
  },
  "Redis": {
    "Configuration": "localhost:6379"
  }
}

3. Run the API

dotnet run

The API will be available at:

• HTTP: http://localhost:5000
• gRPC: http://localhost:5001
• Swagger UI: http://localhost:5000/swagger

API Endpoints

Get Proxy

POST /api/smart-pool/v1/SmartProxy/get-proxy

Get a proxy server based on strategy and filters.

Request Body:

{
  "access_token": "your-token",
  "strategy": "least_delay",
  "target_domain": "facebook.com",
  "ip_version": "v6",
  "protocol": "http",
  "country": "US"
}

Response:

{
  "success": true,
  "error_code": 0,
  "message": null,
  "proxy": {
    "id": 123,
    "host": "192.168.1.100",
    "port": 8080,
    "protocol": "http",
    "ip_version": "v6",
    "country": "US",
    "auth_username": "user",
    "auth_password": "pass"
  }
}

Available Strategies:

• round_robin (default): Balanced distribution
• random: Random selection
• least_delay: Fastest proxy (requires target_domain)
• adaptive_ranking: Highest success rate (requires target_domain)
• alternative: Similar proxy replacement (requires referer_proxy.id)

Log Usage

POST /api/smart-pool/v1/SmartProxy/log-usage

Log proxy usage for analytics and strategy optimization.

Request Body:

{
  "access_token": "your-token",
  "proxy_id": 123,
  "target_domain": "facebook.com",
  "status_code": 200,
  "response_time_ms": 450,
  "error_message": null,
  "request_id": "req-12345-abc"
}

Get Available Strategies

GET /api/smart-pool/v1/SmartProxy/strategies

Get list of available proxy selection strategies.

Response:

{
  "strategies": [
    "round_robin",
    "random",
    "least_delay",
    "adaptive_ranking",
    "alternative"
  ]
}

Health Check

GET /api/smart-pool/v1/SmartProxy/health

Check API health status.

Response:

{
  "status": "healthy",
  "timestamp": "2024-01-15T10:30:00Z"
}

Upsert Proxy

POST /api/smart-pool/v1/SmartProxy/proxy/upsert

Add or update proxy metadata.

Add Access Mapping

POST /api/smart-pool/v1/SmartProxy/access-mapping/add

Add access token to cluster mapping.

gRPC Usage

var channel = GrpcChannel.ForAddress("http://localhost:5001");
var client = MagicOnionClient.Create<ISmartProxyService>(channel);

var response = await client.GetProxy(new GetProxyRequest
{
    access_token = "your-token",
    strategy = "adaptive_ranking"
});

Architecture

Client Request
    ↓
API Layer (gRPC/HTTP)
    ↓
Strategy Factory
    ↓
Strategy Implementation
    ↓
Repository Layer
    ↓
ClickHouse Database

Dependencies

• .NET 8.0
• MagicOnion.Server 5.1.11
• ClickHouse.Client 7.7.0
• EasyCaching.Redis 1.9.2
• Serilog

License

Internal use only.