Sync README files from source repository [skip ci]
This commit is contained in:
Gitea Actions
@@ -0,0 +1,304 @@
|
||||
# SmartPool Quick Reference
|
||||
|
||||
Tài liệu tham khảo nhanh cho SmartPool system.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Setup ClickHouse
|
||||
clickhouse-client < src/Icomm.API.SmartPool/clickhouse_init.sql
|
||||
|
||||
# 2. Start Redis
|
||||
redis-server
|
||||
|
||||
# 3. Run API
|
||||
cd src/Icomm.API.SmartPool
|
||||
dotnet run
|
||||
|
||||
# 4. Run Tests
|
||||
cd src/Icomm.SmartPool.Tests
|
||||
dotnet test
|
||||
```
|
||||
|
||||
## 📦 NuGet Packages
|
||||
|
||||
### API Backend
|
||||
```xml
|
||||
<PackageReference Include="MagicOnion.Server" Version="5.1.11" />
|
||||
<PackageReference Include="ClickHouse.Client" Version="7.7.0" />
|
||||
<PackageReference Include="EasyCaching.Redis" Version="1.9.2" />
|
||||
<PackageReference Include="Dapper" Version="2.1.35" />
|
||||
<PackageReference Include="Serilog.AspNetCore" Version="8.0.1" />
|
||||
```
|
||||
|
||||
### Client SDK
|
||||
```xml
|
||||
<PackageReference Include="MagicOnion.Client" Version="5.1.11" />
|
||||
<PackageReference Include="Grpc.Net.Client" Version="2.60.0" />
|
||||
```
|
||||
|
||||
## 🎯 Strategies Cheat Sheet
|
||||
|
||||
| Strategy | Use Case | Required Params | Best For |
|
||||
|----------|----------|----------------|----------|
|
||||
| `random` | General purpose | - | Simple crawling |
|
||||
| `round_robin` | Load balancing | - | Distributed workload |
|
||||
| `least_delay` | Speed optimization | `targetDomain` | Fast response needed |
|
||||
| `adaptive_ranking` | Reliability | `targetDomain` | High success rate needed |
|
||||
| `alternative` | Failover | `refererProxy` | Retry with similar proxy |
|
||||
|
||||
## 📝 Code Snippets
|
||||
|
||||
### Client Setup
|
||||
```csharp
|
||||
// Program.cs
|
||||
services.AddSmartPoolClient(configuration);
|
||||
|
||||
// Usage
|
||||
public class MyService
|
||||
{
|
||||
private readonly ISmartPoolClient _client;
|
||||
|
||||
public MyService(ISmartPoolClient client) => _client = client;
|
||||
|
||||
public async Task DoWork()
|
||||
{
|
||||
var proxy = await _client.GetProxyAsync(strategy: "least_delay");
|
||||
// Use proxy...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Get Proxy (Simple)
|
||||
```csharp
|
||||
var proxy = await _smartPoolClient.GetProxyAsync(
|
||||
strategy: "random",
|
||||
ipVersion: "v6",
|
||||
country: "US"
|
||||
);
|
||||
```
|
||||
|
||||
### Get Proxy (Advanced)
|
||||
```csharp
|
||||
var request = new GetProxyRequest
|
||||
{
|
||||
AccessToken = "your-token",
|
||||
Strategy = "adaptive_ranking",
|
||||
TargetDomain = "facebook.com",
|
||||
IpVersion = "v6",
|
||||
Protocol = "http",
|
||||
Country = "US"
|
||||
};
|
||||
var proxy = await _smartPoolClient.GetRawProxyAsync(request);
|
||||
```
|
||||
|
||||
### Log Usage
|
||||
```csharp
|
||||
await _smartPoolClient.LogProxyUsageAsync(
|
||||
accessToken: "your-token",
|
||||
proxyId: 123,
|
||||
targetDomain: "facebook.com",
|
||||
statusCode: 200,
|
||||
responseTimeMs: 450
|
||||
);
|
||||
```
|
||||
|
||||
### Use with HttpClient
|
||||
```csharp
|
||||
var proxy = await _smartPoolClient.GetProxyAsync();
|
||||
var handler = new HttpClientHandler { Proxy = proxy };
|
||||
var httpClient = new HttpClient(handler);
|
||||
var response = await httpClient.GetAsync("https://example.com");
|
||||
```
|
||||
|
||||
## 🔌 API Endpoints
|
||||
|
||||
### HTTP REST
|
||||
|
||||
```bash
|
||||
# Get Proxy
|
||||
POST /api/smart-pool/v1/SmartProxy/get-proxy
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"accessToken": "token",
|
||||
"strategy": "least_delay",
|
||||
"targetDomain": "facebook.com"
|
||||
}
|
||||
|
||||
# Log Usage
|
||||
POST /api/smart-pool/v1/SmartProxy/log-usage
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"accessToken": "token",
|
||||
"proxyId": 123,
|
||||
"targetDomain": "facebook.com",
|
||||
"statusCode": 200,
|
||||
"responseTimeMs": 450
|
||||
}
|
||||
|
||||
# Health Check
|
||||
GET /api/smart-pool/v1/SmartProxy/health
|
||||
|
||||
# Get Strategies
|
||||
GET /api/smart-pool/v1/SmartProxy/strategies
|
||||
```
|
||||
|
||||
### gRPC
|
||||
|
||||
```csharp
|
||||
var channel = GrpcChannel.ForAddress("http://localhost:5001");
|
||||
var client = MagicOnionClient.Create<ISmartProxyService>(channel);
|
||||
|
||||
var response = await client.GetProxy(new GetProxyRequest { ... });
|
||||
await client.LogProxyUsage(new ProxyUsageLogRequest { ... });
|
||||
var status = await client.Ping();
|
||||
```
|
||||
|
||||
## 🗄️ ClickHouse Queries
|
||||
|
||||
### View Proxy Stats
|
||||
```sql
|
||||
SELECT
|
||||
proxy_id,
|
||||
avgMerge(avg_response_time_state) as avg_time,
|
||||
sumIfMerge(success_count_state) / countMerge(request_count_state) as success_rate
|
||||
FROM smart_pool_logs.proxy_stats_hourly
|
||||
WHERE access_token = 'your-token'
|
||||
AND hour >= now() - INTERVAL 24 HOUR
|
||||
GROUP BY proxy_id
|
||||
ORDER BY success_rate DESC;
|
||||
```
|
||||
|
||||
### View Recent Logs
|
||||
```sql
|
||||
SELECT *
|
||||
FROM smart_pool_logs.proxy_usage_logs
|
||||
WHERE access_token = 'your-token'
|
||||
AND timestamp >= now() - INTERVAL 1 HOUR
|
||||
ORDER BY timestamp DESC
|
||||
LIMIT 100;
|
||||
```
|
||||
|
||||
### Add Proxy
|
||||
```sql
|
||||
INSERT INTO smart_pool_meta.proxy_metadata
|
||||
(id, host, port, protocol, ip_version, country, status)
|
||||
VALUES
|
||||
(1, '192.168.1.100', 8080, 'http', 'v4', 'VN', 1);
|
||||
```
|
||||
|
||||
### Add Access Mapping (Cluster-based)
|
||||
```sql
|
||||
INSERT INTO smart_pool_meta.proxy_access_mapping
|
||||
(access_token, cluster, priority)
|
||||
VALUES
|
||||
('your-token', 'cluster_vn_http', 10);
|
||||
```
|
||||
|
||||
## ⚙️ Configuration Templates
|
||||
|
||||
### appsettings.json (API)
|
||||
```json
|
||||
{
|
||||
"ClickHouse": {
|
||||
"Host": "localhost",
|
||||
"Port": 8123,
|
||||
"MetaDatabase": "smart_pool_meta",
|
||||
"LogsDatabase": "smart_pool_logs",
|
||||
"Username": "default",
|
||||
"Password": ""
|
||||
},
|
||||
"Redis": {
|
||||
"Configuration": "localhost:6379"
|
||||
},
|
||||
"Kestrel": {
|
||||
"Endpoints": {
|
||||
"Http": { "Url": "http://0.0.0.0:5000" },
|
||||
"Grpc": { "Url": "http://0.0.0.0:5001", "Protocols": "Http2" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### appsettings.json (Client)
|
||||
```json
|
||||
{
|
||||
"SmartPool": {
|
||||
"Protocol": "Grpc",
|
||||
"Host": "localhost",
|
||||
"Port": 5001,
|
||||
"UseSecureConnection": false,
|
||||
"DefaultAccessToken": "your-token",
|
||||
"TimeoutMs": 30000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Proxy không available
|
||||
```csharp
|
||||
var proxy = await _client.GetProxyAsync();
|
||||
if (proxy == null)
|
||||
{
|
||||
// Check:
|
||||
// 1. Access token có đúng không?
|
||||
// 2. Có proxy nào được map với token này không?
|
||||
// 3. Filters có quá strict không?
|
||||
}
|
||||
```
|
||||
|
||||
### Connection timeout
|
||||
```csharp
|
||||
// Tăng timeout
|
||||
services.AddSmartPoolClient(options =>
|
||||
{
|
||||
options.TimeoutMs = 60000; // 60 seconds
|
||||
});
|
||||
```
|
||||
|
||||
### Strategy không hoạt động
|
||||
```csharp
|
||||
// Check logs có đủ không (cho least_delay, adaptive_ranking)
|
||||
// Cần ít nhất 10 logs để adaptive_ranking hoạt động
|
||||
```
|
||||
|
||||
## 📊 Performance Tips
|
||||
|
||||
1. **Use gRPC for production** - Faster than HTTP
|
||||
2. **Log asynchronously** - Don't block main flow
|
||||
3. **Cache proxy info** - Reduce API calls
|
||||
4. **Use appropriate strategy** - Match use case
|
||||
5. **Monitor ClickHouse** - Check query performance
|
||||
|
||||
## 🔒 Security Notes
|
||||
|
||||
- Always use HTTPS in production
|
||||
- Rotate access tokens regularly
|
||||
- Limit proxy access by token
|
||||
- Monitor unusual usage patterns
|
||||
- Set appropriate TTL for logs
|
||||
|
||||
## 📚 Documentation Links
|
||||
|
||||
- [Main README](SMARTPOOL_README.md)
|
||||
- [API Backend Guide](./docs/src/Icomm.API.SmartPool/README.md)
|
||||
- [Client SDK Guide](./docs/src/Icomm.SmartPool.Proxy/README.md)
|
||||
- [Usage Examples](SMARTPOOL_USAGE_EXAMPLES.md)
|
||||
- [Implementation Summary](SMARTPOOL_IMPLEMENTATION_SUMMARY.md)
|
||||
|
||||
## 🆘 Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check logs in `logs/smartpool-*.log`
|
||||
2. Verify ClickHouse connection
|
||||
3. Check Redis connection
|
||||
4. Review configuration
|
||||
5. Contact development team
|
||||
|
||||
---
|
||||
|
||||
**Version**: 1.0.0
|
||||
**Last Updated**: 2026-01-22
|
||||
Reference in New Issue
Block a user