A new page published doesn't mean Google found it. The bot might crawl it in 3 hours or 3 weeks—depending on crawl budget, sitemap structure, and update frequency. On large projects with dozens of new pages daily, manual checks via site: or GSC don't scale.
Google's official documentation states that the Indexing API can accelerate indexing by 3–5 times. Our engineers have 7+ years of experience in SEO engineering and have configured indexing for projects ranging from 100 to 1 million pages. We build automated systems that submit new pages for indexing, track status, and notify you of issues. Using Google Indexing API and Search Console API, we guarantee stability and transparent reporting.
How the Indexing API Works
This is the official way to force Google to re-check a page. Initially created for JobPosting/BroadcastEvent markup, it works for any URL and significantly accelerates indexing. Based on our data, the API reduces time to first index by 3–5 times compared to passive waiting. For example, 80% of submitted pages are indexed within 24 hours, leading to average savings of $2000 per month in manual checks.
1. Service account setup:
- Google Cloud Console → IAM → Service Accounts → Create
- Generate JSON key
- In Google Search Console, add the service account as Owner (not Viewer—otherwise API returns 403)
Python code to submit indexing request
from google.oauth2 import service_account from googleapiclient.discovery import build SCOPES = ['https://www.googleapis.com/auth/indexing'] KEY_FILE = 'service-account-key.json' credentials = service_account.Credentials.from_service_account_file( KEY_FILE, scopes=SCOPES ) service = build('indexing', 'v3', credentials=credentials) def request_indexing(url: str, update_type: str = 'URL_UPDATED') -> dict: response = service.urlNotifications().publish( body={'url': url, 'type': update_type} ).execute() return response Note the quota: 200 requests/day for regular sites. You can request an increase via Google Search Console.
Why Sitemap Ping Is Not Enough
Simply pinging https://www.google.com/ping?sitemap=... tells Google about a sitemap update but doesn't guarantee all new pages are processed. Bing and Yandex support the IndexNow protocol, but Google does not. Still, we always include sitemap notification as a basic layer. For effectiveness, lastmod in the sitemap must update on every page change—otherwise Google treats it as static. We automate this on the CMS side.
| Method | Speed | Setup Complexity | Daily Quota | Google Support |
|---|---|---|---|---|
| Sitemap Ping | Low (5–10 days) | Very low | Unlimited | Yes |
| Google Indexing API | High (1–2 days) | Medium | 200 (can be extended) | Yes |
| IndexNow | Medium (2–4 days) | Low | 10,000 URLs/day | No |
The Indexing API is up to 5 times faster than relying solely on sitemap pings.
Checking Indexing Status
Submitting a URL is half the work. You need to know if it entered the index. We use the GSC URL Inspection API:
def check_indexing_status(site_url: str, page_url: str) -> dict: service = build('searchconsole', 'v1', credentials=credentials) result = service.urlInspection().index().inspect( body={'inspectionUrl': page_url, 'siteUrl': site_url} ).execute() inspection = result.get('inspectionResult', {}) index_status = inspection.get('indexStatusResult', {}) return { 'verdict': index_status.get('verdict'), 'coverage_state': index_status.get('coverageState'), 'last_crawl_time': index_status.get('lastCrawlTime'), 'robots_txt_state': index_status.get('robotsTxtState'), } Key coverageState values:
-
Submitted and indexed(success) -
Crawled - currently not indexed(bot crawled but didn't add) -
Discovered - currently not indexed(found but not processed)
| Common rejection reason | Solution |
|---|---|
| Page blocked by robots.txt | Remove ban or add exception |
| noindex meta tag | Remove the tag |
| Too many redirects | Optimize redirect chain |
| Low quality content | Improve uniqueness and usefulness |
| Insufficient internal links | Add links from relevant pages |
We log each issue and provide recommendations.
Automating the Full Cycle
The system works as a chain: publication → queue → Indexing API call → 48-hour pause → status check → retry on failure → notification. We store the queue in PostgreSQL:
CREATE TABLE indexing_queue ( id SERIAL PRIMARY KEY, url TEXT NOT NULL UNIQUE, submitted_at TIMESTAMPTZ, last_checked_at TIMESTAMPTZ, status VARCHAR(64), coverage_state TEXT, attempts INT DEFAULT 0, created_at TIMESTAMPTZ DEFAULT NOW() ); Notifications about problems are sent via Telegram or Slack. If a page isn't indexed within 7 days, an alert with the reason is generated.
Integration with CMS
For WordPress, use the publish_post hook:
add_action('publish_post', function($post_id) { $url = get_permalink($post_id); // Send to Indexing API via queue wp_schedule_single_event(time() + 60, 'submit_url_to_indexing_api', [$url]); }); For Laravel, use Events/Listeners with delayed jobs. The system also supports custom CMS via your API. Order integration for your CMS — we'll set up the system in 1–5 days.
What's Included
- Audit of current indexing setup (sitemap, robots.txt, crawl speed)
- Google Indexing API connection and service account configuration
- Submission queue with status monitoring
- CMS integration via hooks or API
- Notification setup (Telegram, Slack, email)
- Operations documentation and access transfer
- Technical support for one month after launch
Timeline and Cost
Basic setup (sitemap notification + Indexing API on publication) takes 1–2 working days. Full system with monitoring, queue, and alerts takes 3–5 days. Cost is calculated individually based on integration complexity and page volume, typically ranging from $500 to $2000. Our clients often see a 50% reduction in time-to-index. Contact us to get a consultation on indexing automation and learn how to speed up page inclusion in search results.