305 lines
6.7 KiB
Markdown
305 lines
6.7 KiB
Markdown
# SmartPool Quick Reference
|
|
|
|
Tài liệu tham khảo nhanh cho SmartPool system.
|
|
|
|
## 🚀 Quick Start
|
|
|
|
```bash
|
|
# 1. Setup ClickHouse
|
|
clickhouse-client < src/Icomm.API.SmartPool/clickhouse_init.sql
|
|
|
|
# 2. Start Redis
|
|
redis-server
|
|
|
|
# 3. Run API
|
|
cd src/Icomm.API.SmartPool
|
|
dotnet run
|
|
|
|
# 4. Run Tests
|
|
cd src/Icomm.SmartPool.Tests
|
|
dotnet test
|
|
```
|
|
|
|
## 📦 NuGet Packages
|
|
|
|
### API Backend
|
|
```xml
|
|
<PackageReference Include="MagicOnion.Server" Version="5.1.11" />
|
|
<PackageReference Include="ClickHouse.Client" Version="7.7.0" />
|
|
<PackageReference Include="EasyCaching.Redis" Version="1.9.2" />
|
|
<PackageReference Include="Dapper" Version="2.1.35" />
|
|
<PackageReference Include="Serilog.AspNetCore" Version="8.0.1" />
|
|
```
|
|
|
|
### Client SDK
|
|
```xml
|
|
<PackageReference Include="MagicOnion.Client" Version="5.1.11" />
|
|
<PackageReference Include="Grpc.Net.Client" Version="2.60.0" />
|
|
```
|
|
|
|
## 🎯 Strategies Cheat Sheet
|
|
|
|
| Strategy | Use Case | Required Params | Best For |
|
|
|----------|----------|----------------|----------|
|
|
| `random` | General purpose | - | Simple crawling |
|
|
| `round_robin` | Load balancing | - | Distributed workload |
|
|
| `least_delay` | Speed optimization | `targetDomain` | Fast response needed |
|
|
| `adaptive_ranking` | Reliability | `targetDomain` | High success rate needed |
|
|
| `alternative` | Failover | `refererProxy` | Retry with similar proxy |
|
|
|
|
## 📝 Code Snippets
|
|
|
|
### Client Setup
|
|
```csharp
|
|
// Program.cs
|
|
services.AddSmartPoolClient(configuration);
|
|
|
|
// Usage
|
|
public class MyService
|
|
{
|
|
private readonly ISmartPoolClient _client;
|
|
|
|
public MyService(ISmartPoolClient client) => _client = client;
|
|
|
|
public async Task DoWork()
|
|
{
|
|
var proxy = await _client.GetProxyAsync(strategy: "least_delay");
|
|
// Use proxy...
|
|
}
|
|
}
|
|
```
|
|
|
|
### Get Proxy (Simple)
|
|
```csharp
|
|
var proxy = await _smartPoolClient.GetProxyAsync(
|
|
strategy: "random",
|
|
ipVersion: "v6",
|
|
country: "US"
|
|
);
|
|
```
|
|
|
|
### Get Proxy (Advanced)
|
|
```csharp
|
|
var request = new GetProxyRequest
|
|
{
|
|
AccessToken = "your-token",
|
|
Strategy = "adaptive_ranking",
|
|
TargetDomain = "facebook.com",
|
|
IpVersion = "v6",
|
|
Protocol = "http",
|
|
Country = "US"
|
|
};
|
|
var proxy = await _smartPoolClient.GetRawProxyAsync(request);
|
|
```
|
|
|
|
### Log Usage
|
|
```csharp
|
|
await _smartPoolClient.LogProxyUsageAsync(
|
|
accessToken: "your-token",
|
|
proxyId: 123,
|
|
targetDomain: "facebook.com",
|
|
statusCode: 200,
|
|
responseTimeMs: 450
|
|
);
|
|
```
|
|
|
|
### Use with HttpClient
|
|
```csharp
|
|
var proxy = await _smartPoolClient.GetProxyAsync();
|
|
var handler = new HttpClientHandler { Proxy = proxy };
|
|
var httpClient = new HttpClient(handler);
|
|
var response = await httpClient.GetAsync("https://example.com");
|
|
```
|
|
|
|
## 🔌 API Endpoints
|
|
|
|
### HTTP REST
|
|
|
|
```bash
|
|
# Get Proxy
|
|
POST /api/smart-pool/v1/SmartProxy/get-proxy
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"accessToken": "token",
|
|
"strategy": "least_delay",
|
|
"targetDomain": "facebook.com"
|
|
}
|
|
|
|
# Log Usage
|
|
POST /api/smart-pool/v1/SmartProxy/log-usage
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"accessToken": "token",
|
|
"proxyId": 123,
|
|
"targetDomain": "facebook.com",
|
|
"statusCode": 200,
|
|
"responseTimeMs": 450
|
|
}
|
|
|
|
# Health Check
|
|
GET /api/smart-pool/v1/SmartProxy/health
|
|
|
|
# Get Strategies
|
|
GET /api/smart-pool/v1/SmartProxy/strategies
|
|
```
|
|
|
|
### gRPC
|
|
|
|
```csharp
|
|
var channel = GrpcChannel.ForAddress("http://localhost:5001");
|
|
var client = MagicOnionClient.Create<ISmartProxyService>(channel);
|
|
|
|
var response = await client.GetProxy(new GetProxyRequest { ... });
|
|
await client.LogProxyUsage(new ProxyUsageLogRequest { ... });
|
|
var status = await client.Ping();
|
|
```
|
|
|
|
## 🗄️ ClickHouse Queries
|
|
|
|
### View Proxy Stats
|
|
```sql
|
|
SELECT
|
|
proxy_id,
|
|
avgMerge(avg_response_time_state) as avg_time,
|
|
sumIfMerge(success_count_state) / countMerge(request_count_state) as success_rate
|
|
FROM smart_pool_logs.proxy_stats_hourly
|
|
WHERE access_token = 'your-token'
|
|
AND hour >= now() - INTERVAL 24 HOUR
|
|
GROUP BY proxy_id
|
|
ORDER BY success_rate DESC;
|
|
```
|
|
|
|
### View Recent Logs
|
|
```sql
|
|
SELECT *
|
|
FROM smart_pool_logs.proxy_usage_logs
|
|
WHERE access_token = 'your-token'
|
|
AND timestamp >= now() - INTERVAL 1 HOUR
|
|
ORDER BY timestamp DESC
|
|
LIMIT 100;
|
|
```
|
|
|
|
### Add Proxy
|
|
```sql
|
|
INSERT INTO smart_pool_meta.proxy_metadata
|
|
(id, host, port, protocol, ip_version, country, status)
|
|
VALUES
|
|
(1, '192.168.1.100', 8080, 'http', 'v4', 'VN', 1);
|
|
```
|
|
|
|
### Add Access Mapping (Cluster-based)
|
|
```sql
|
|
INSERT INTO smart_pool_meta.proxy_access_mapping
|
|
(access_token, cluster, priority)
|
|
VALUES
|
|
('your-token', 'cluster_vn_http', 10);
|
|
```
|
|
|
|
## ⚙️ Configuration Templates
|
|
|
|
### appsettings.json (API)
|
|
```json
|
|
{
|
|
"ClickHouse": {
|
|
"Host": "localhost",
|
|
"Port": 8123,
|
|
"MetaDatabase": "smart_pool_meta",
|
|
"LogsDatabase": "smart_pool_logs",
|
|
"Username": "default",
|
|
"Password": ""
|
|
},
|
|
"Redis": {
|
|
"Configuration": "localhost:6379"
|
|
},
|
|
"Kestrel": {
|
|
"Endpoints": {
|
|
"Http": { "Url": "http://0.0.0.0:5000" },
|
|
"Grpc": { "Url": "http://0.0.0.0:5001", "Protocols": "Http2" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### appsettings.json (Client)
|
|
```json
|
|
{
|
|
"SmartPool": {
|
|
"Protocol": "Grpc",
|
|
"Host": "localhost",
|
|
"Port": 5001,
|
|
"UseSecureConnection": false,
|
|
"DefaultAccessToken": "your-token",
|
|
"TimeoutMs": 30000
|
|
}
|
|
}
|
|
```
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### Proxy không available
|
|
```csharp
|
|
var proxy = await _client.GetProxyAsync();
|
|
if (proxy == null)
|
|
{
|
|
// Check:
|
|
// 1. Access token có đúng không?
|
|
// 2. Có proxy nào được map với token này không?
|
|
// 3. Filters có quá strict không?
|
|
}
|
|
```
|
|
|
|
### Connection timeout
|
|
```csharp
|
|
// Tăng timeout
|
|
services.AddSmartPoolClient(options =>
|
|
{
|
|
options.TimeoutMs = 60000; // 60 seconds
|
|
});
|
|
```
|
|
|
|
### Strategy không hoạt động
|
|
```csharp
|
|
// Check logs có đủ không (cho least_delay, adaptive_ranking)
|
|
// Cần ít nhất 10 logs để adaptive_ranking hoạt động
|
|
```
|
|
|
|
## 📊 Performance Tips
|
|
|
|
1. **Use gRPC for production** - Faster than HTTP
|
|
2. **Log asynchronously** - Don't block main flow
|
|
3. **Cache proxy info** - Reduce API calls
|
|
4. **Use appropriate strategy** - Match use case
|
|
5. **Monitor ClickHouse** - Check query performance
|
|
|
|
## 🔒 Security Notes
|
|
|
|
- Always use HTTPS in production
|
|
- Rotate access tokens regularly
|
|
- Limit proxy access by token
|
|
- Monitor unusual usage patterns
|
|
- Set appropriate TTL for logs
|
|
|
|
## 📚 Documentation Links
|
|
|
|
- [Main README](SMARTPOOL_README.md)
|
|
- [API Backend Guide](./docs/src/Icomm.API.SmartPool/README.md)
|
|
- [Client SDK Guide](./docs/src/Icomm.SmartPool.Proxy/README.md)
|
|
- [Usage Examples](SMARTPOOL_USAGE_EXAMPLES.md)
|
|
- [Implementation Summary](SMARTPOOL_IMPLEMENTATION_SUMMARY.md)
|
|
|
|
## 🆘 Support
|
|
|
|
For issues or questions:
|
|
1. Check logs in `logs/smartpool-*.log`
|
|
2. Verify ClickHouse connection
|
|
3. Check Redis connection
|
|
4. Review configuration
|
|
5. Contact development team
|
|
|
|
---
|
|
|
|
**Version**: 1.0.0
|
|
**Last Updated**: 2026-01-22
|