Tencent Cloud Log Service (CLS) cost optimization usually depends on two high-impact areas: storage and traffic. In one production log topic, storage and traffic accounted for 99.73% of the monthly cost, so the most useful controls were not abstract cloud-finance tricks. They were concrete logging decisions: print fewer useless logs, keep logs for the right period, use low-frequency storage when fast analysis is not needed, and avoid unnecessary full-text indexing or index rebuilds.
Best fit for this CLS cost-control pattern
Use this pattern when a CLS log topic is getting expensive because it stores too much data, keeps logs too long, uses full-text indexing broadly, or mixes logs with different operational needs.
| Cost signal | Likely control |
|---|---|
| Raw log volume keeps growing | Reduce meaningless log printing at the application layer |
| Retention is longer than the business need | Shorten retention or move old data to low-frequency storage |
| Index storage is much larger than log storage | Review full-text indexing and prefer key-value indexes |
| Index rebuilds are frequent | Restrict rebuild permissions and batch index changes carefully |
| One topic contains many applications or log levels | Split topics by application, level, and usage |
CLS billing areas that drive the bill
The source case groups CLS billing into four areas:
- Storage.
- Traffic.
- Calls.
- Partitions.
The production example had monthly log-topic spending above 20,000 yuan, with storage and traffic accounting for 99.73%. That is why the optimization path focuses on storage type, index strategy, traffic behavior, and topic design.
Log storage versus index storage
CLS separates raw log storage from index storage.
| Storage type | Meaning | Cost behavior |
|---|---|---|
| Log storage | Raw logs stored in CLS | When collected by LogListener, logs are compressed by default, commonly at a 4x to 5x compression ratio; billing is based on compressed storage size |
| Index storage | Storage occupied after indexes are enabled | Measured on uncompressed logs, so it can be much larger than raw compressed storage |
The practical default is to set key-value indexes for the fields you need, instead of turning on full-text indexing for every log topic.
Full-text indexing as a main cost driver
The example topic uses standard storage, keeps logs for 30 days, and enables full-text indexing. Its average storage numbers are:
| Item | Size |
|---|---|
| Standard log storage | About 7867 GB
|
| Index storage | About 44785 GB
|
Index storage is more than five times the compressed log storage. Using 0.0115 yuan/GB/day as the standard storage price:
standard log storage = 7867 * 0.0115 * 30 = 2714 yuan
standard index storage = 44785 * 0.0115 * 30 = 15450 yuan
total monthly storage cost = 15450 + 2714 = 18164 yuan
The lesson is clear: full-text indexing can turn index storage into the dominant cost. Prefer key-value indexes, and do not treat full-text indexing as a default switch.
Storage class choices for CLS logs
CLS supports standard storage and low-frequency storage.
Use standard storage when logs need:
- Alerting.
- Dashboards.
- Short-term investigation.
- Fast query response.
- SQL analysis.
Use low-frequency storage when logs need:
- Historical retrieval.
- Lower query-response requirements.
- No SQL analysis.
- No dashboard or alerting requirement.
If a topic is created directly as low-frequency storage, retention must be at least 7 days. Even so, the source notes that a 7-day low-frequency topic can still cost less than a 1-day standard-storage topic.
Savings model for standard plus low-frequency storage
Using the same topic numbers, moving logs older than 7 days into low-frequency storage changes the monthly storage estimate:
original monthly storage cost = 18164 yuan
standard + low-frequency storage =
(7867 * 0.0115 * 7) + (7867 * 0.0025 * 23)
+ (44785 * 0.0115 * 7) + (44785 * 0.0025 * 23)
= 1085 + 6180
= 7265 yuan
monthly difference = 18164 - 7265 = 10899 yuan
For logs that do not need a standard-storage period, dedicated low-frequency topics can reduce cost by about 60% to 70%.
Traffic cost from full-text indexing
Traffic cost is not only normal write traffic. Full-text indexing converts raw logs into index data, and that conversion can create more traffic cost than log ingestion itself.
Two controls matter:
- Reduce invalid or meaningless log printing at the source.
- Avoid full-text indexes when key-value indexes are enough.
The rebuild index operation is especially risky. Rebuilding indexes reconstructs historical raw logs for the selected time range and generates index traffic cost. Large topics can take a long time and produce high cost, so frequent index modification plus rebuild is a bad default.
CLS log topic design for cost control
In CLS, a log topic is similar in role to an Elasticsearch index. A useful default is one topic per application, not one broad topic for many applications.
For each application log request, decide:
| Design question | Suggested default |
|---|---|
| Retention period | 3 days by default; up to 7 days for common needs; beyond 7 days, consider low-frequency storage; beyond 30 days, consider delivery to COS |
| Storage type | Low-frequency by default unless alerts, dashboards, or SQL analysis require standard storage |
| Index setup | Full-text index off by default; set key-value indexes for searchable fields |
| Full-text exception | Enable only for a clearly justified requirement |
This makes cost visible by application and lets each topic match its own operational need.
Splitting standard and low-frequency logs by log level
If the application already writes logs by level, route levels to different CLS topics.
Example file convention:
/data/logs/user-api-info-2023-05-20.log
/data/logs/user-api-warn-2023-05-20.log
/data/logs/user-api-error-2023-05-20.log
Recommended topic split:
| Topic | Collection rule | Storage strategy | Operational purpose |
|---|---|---|---|
user-api-base-log |
/data/logs/user-api-warn-*.log and /data/logs/user-api-error-*.log
|
Standard storage, 7-day retention | Alerting and urgent operations |
user-api-low-log |
/data/logs/user-api-info-*.log |
Low-frequency storage | Historical retrieval and context lookup |
In the source calculation, this type of split reduced the monthly estimate from 5979 yuan to 3886 yuan, saving 2090 yuan.
No log-level convention: a two-class fallback
Start with two file classes:
/data/logs/user-api-base-2023-05-20.log
/data/logs/user-api-low-2023-05-20.log
Then map them to:
| Topic class | Use |
|---|---|
base topic |
Logs that need standard storage, alerts, dashboards, or SQL analysis |
low topic |
Logs that only need retrieval and historical traceability |
This is not a full logging standard, but it gives operations enough structure to control cost.
Common pitfalls in CLS cost optimization
- Enabling full-text indexing for every topic by default.
- Letting test or development users rebuild indexes casually.
- Keeping high-volume
infologs in standard storage even when they only need retrieval. - Mixing many applications into one topic and losing application-level cost control.
- Keeping logs for 30 days because of habit rather than business need.
- Ignoring traffic cost created by indexing operations.
FAQ
Why is my CLS index storage larger than log storage?
Log storage can be compressed by LogListener, while index storage is measured against uncompressed logs. If full-text indexing is enabled, index storage can become several times larger than compressed log storage.
When should I choose low-frequency storage for CLS logs?
Use low-frequency storage when logs mainly need retrieval and do not need fast query response, SQL analysis, dashboards, or alerting.
Why should I avoid rebuilding CLS indexes frequently?
Index rebuild reconstructs historical raw logs for the selected time range and generates index traffic cost. Large topics can take longer and become expensive.
How should I split CLS topics for application logs?
Use one topic per application where possible, then split by usage. Warning and error logs can stay in a standard topic for alerting, while high-volume informational logs can move to low-frequency storage.
Final checklist
- Remove meaningless logs at the application layer.
- Set retention based on the business scenario.
- Use low-frequency storage for retrieval-only logs.
- Prefer key-value indexes over full-text indexes.
- Restrict or review index rebuild permissions.
- Use one log topic per application where possible.
- Split high-volume context logs from alert-oriented warning and error logs.
Top comments (0)