Traffic Shaping
Overview
To provide the fastest possible response times, Pitney Bowes uses traffic shaping to ensure that a very large number of closely spaced API requests from an individual developer account cannot negatively impact response times for all developer accounts. During any given second, Pitney Bowes throttles the requests for an individual developer if that developer's requests would slow down other developers' requests.
Pitney Bowes uses a custom algorithm to determine when to apply traffic shaping. Traffic-shaping policy is handled on a per-developer basis, so the number of requests sent per second to one developer has no effect on traffic shaping for any other developer.
Plan-Based Throttling Limits
The Safe Limit for any API operation performed by the developer varies based on the subscription plan assigned to it. To avoid traffic shaping, developers should observe the following limits:
Sandbox Plan-Based Throttling Limits
| Plan Name | Request Rate (TPS) | Burst Limit | Daily Quota |
|---|---|---|---|
| BASIC_API_USAGE_PLAN_PPD | 10 per second | 20 per second | 2000 per day |
| DEFAULT_API_USAGE_PLAN_PPD | 20 per second | 20 per second | 50000 per day |
| ENTERPRISE_API_USAGE_PLAN_PPD | 20 per second | 40 per second | Unlimited |
| PREMIUM_API_USAGE_PLAN_PPD | 15 per second | 30 per second | 3000 per day |
Production Plan-Based Throttling Limits
| Plan Name | Request Rate (TPS) | Burst Limit | Daily Quota |
|---|---|---|---|
| BASIC_API_USAGE_PLAN_PRD | 100 per second | 200 per second | Unlimited |
| DEFAULT_API_USAGE_PLAN_PRD | 150 per second | 300 per second | Unlimited |
| ENTERPRISE_API_USAGE_PLAN_PRD | 300 per second | 600 per second | Unlimited |
| PREMIUM_API_USAGE_PLAN_PRD | 150 per second | 300 per second | Unlimited |
Note: Existing developers will be regulated under a single key (the default key for the environment) until an explicit plan is assigned to them.
- Request Rate (TPS): The maximum number of API requests a developer can send per second without throttling.
- Burst Limit: The maximum number of API requests a developer can send in short bursts before throttling applies.
- Daily Quota: The maximum number of API calls allowed per developer per day.
Impact of Exceeding Throttling Limits
If a developer exceeds the assigned TPS limit, they may experience increased latency as excess requests are either queued or delayed before processing.
1. Exceeding TPS (Transactions Per Second) Limit
- If a developer sends more requests than the allowed TPS rate, excess requests may be throttled and processed with higher response times (latency).
2. Exceeding Burst Limit
- If a developer exceeds the burst limit, requests beyond the allowed threshold will be throttled, and a Too Many Requests error will be returned.
3. Exceeding the Daily Quota
- Once the daily API quota is reached, all subsequent requests will be rejected, and a Limit Exceeded error will be returned for the rest of the day.
Note: Pitney Bowes may adjust the traffic-shaping policy for APIs over time.
When Does Traffic Shaping Apply?
If the request volume exceeds the assigned limit, Pitney Bowes may throttle requests until one of the following conditions is met:
- The TPS for the API operation falls below the assigned request rate (e.g., 10 TPS for Basic Plan) for one second.
OR
- The total TPS for the API operation for all developers falls below 50% of the maximum allowable bandwidth, as defined by the Pitney Bowes engineering teams, for one second.
Note that Pitney Bowes traffic-shaping policies do not, in general, specifically target concurrent (simultaneous) transactions. For example, a developer sending bursts of 200 concurrent transactions at the beginning of each second would be treated identically to a developer sending one transaction every 500 milliseconds.
Build TPS Limits into Your Applications
As a best practice, build TPS limits into your applications to avoid sending more transactions per second than the Safe Limit allows. This might mean spacing out transactions by a set number of milliseconds or limiting the number of threads in a multi-threaded application.
Code that defines how retry requests are handled should also be reviewed to prevent the retry policy from compounding traffic-related issues.
If you frequently encounter throttling issues, consider upgrading your usage plan. Please contact Pitney Bowes Support for assistance with plan upgrades.