Development of a Marketplace Seller Rating System
On a marketplace with 10,000 sellers and hundreds of reviews per day, a simple arithmetic mean doesn't work: a couple of fake reviews distort the real picture. That's why we develop a rating system that considers not only star ratings but also objective order fulfillment metrics — delivery percentage, processing speed, cancellations, and returns. Buyers make decisions in seconds: a product from a seller with a 4.8 rating sells significantly better than the same product from a seller with 4.0. If the rating doesn't reflect reality, trust drops. Our goal is to provide a fair and transparent mechanism that encourages quality work and protects the platform from manipulation. The foundation is a multi-factor model with different weights and a 90-day sliding window.
Rating Components
The rating should not be reduced to just review stars. A quality system considers several parameters:
| Parameter | Weight | Data Source |
|---|---|---|
| Average review rating | 40% | reviews table |
| Successful delivery rate | 20% | order_deliveries |
| Order processing speed | 15% | order_status_history |
| Cancellation rate (seller fault) | 15% | order_cancellations |
| Return rate | 10% | returns |
The final rating is a weighted sum of normalized indicators, scaled to 1–5. Each parameter is normalized using Z-score or min-max. This accounts for different metric natures and prevents one factor from dominating. This approach is more stable and less prone to manipulation than a simple average.
Review Collection: Process and Rules
Reviews can only be left after order delivery confirmation. This excludes reviews from non-existent purchases. The review form:
- Overall rating (1–5 stars)
- Parameter ratings: description accuracy, packaging, shipping speed
- Text (optional, with a minimum length)
- Review photo (upload via S3)
Review reminder: push/email 3 days after delivery, repeated after 7 days.
Review Moderation
Reviews undergo automatic filtering (profanity, spam patterns) and can be disputed by sellers. A seller can reply to any review — this is publicly visible and demonstrates engagement. A seller's complaint about a review is sent to a moderator. Grounds for removal: review about a different product, contains personal data, obvious fake. Algorithms analyze text for profanity, spam patterns (repeated phrases, links), and anomalously low ratings from new accounts. Suspicious reviews are flagged and sent for manual moderation. A seller can dispute a review by providing evidence (screenshots of correspondence, photos of shipment).
Why We Use a Sliding Window
The rating is not recalculated in real time (expensive), but on a schedule:
- Once per hour for active sellers (>10 orders in 30 days)
- Once per day for others
Example SQL query for sliding window
SELECT seller_id, AVG(rating) as avg_rating, COUNT(*) as reviews_count, AVG(CASE WHEN status='delivered' THEN 1.0 ELSE 0.0 END) as delivery_rate FROM orders WHERE created_at >= now() - interval '90 days' GROUP BY seller_id The rating is based on a sliding window: only the last 90 days are considered. This protects against "rotting" past data and encourages maintaining quality.
What Are the Consequences of a Low Rating?
- Rating < 4.0 — warning in the dashboard, products appear lower in search results
- Rating < 3.5 for 30 days — new orders restricted, support team notified
- Rating < 3.0 — automatic account suspension until review
This motivates sellers to improve rather than continue with a poor rating for years.
Rating System Development Process
- Analytics — gather requirements, analyze current metrics and architecture.
- Model Design — calculate weights, normalization, recalculation logic.
- API Development — create endpoints for reviews, ratings, and moderation.
- Admin Panel — interface for moderators and managers.
- Queue Integration — background task processing via Redis/Beanstalkd.
- Testing and Documentation — load testing, Swagger/OpenAPI.
- Deployment — on your server or cloud.
We guarantee the system will handle up to 100,000 reviews per day without degradation. For peak loads (after sales), a Redis task queue and horizontal scaling of workers are used. Team experience — over 5 years, 12 major projects implemented.
What's Included in the Work
Development includes: model design, API for reviews and moderation, admin panel, queue integration, load testing, documentation (Swagger/OpenAPI), training for your team, deployment on your server or cloud, and support for one month after launch.
Approach Comparison: Simple Average vs Weighted
| Characteristic | Simple Average | Weighted (Ours) |
|---|---|---|
| Spam sensitivity | High | Low |
| Rating stability | Low | High |
| Transparency for sellers | Medium | High |
| Manipulation protection | Weak | Strong |
A weighted rating better reflects actual service quality and is less prone to manipulation. Development cost depends on complexity and scale — calculated individually. The multi-factor rating model is recognized by the industry as the most objective.
More about reputation system principles can be found on Wikipedia.
Order development of a rating system for your marketplace — get a consultation from our engineer and an approximate timeline estimate. Contact us to discuss details.