Complete Guide to Instagram Scraping API: Extract Posts, Reels, User Data & More

Instagram is still one of the more useful social datasets on the web, and one of the more frustrating ones to collect directly.

The official API is limited, much of the app runs on internal requests, and basic scraping setups break quickly. This guide covers the endpoints that are most practical when you need profile data, posts and reels, comments, search results or media details, and the pagination field each one uses.

Why Instagram is hard to scrape reliably

Before getting into the endpoints, it helps to know why direct scraping breaks so easily:

• The official API is narrow. It needs app review and permissions, and even then it mostly covers accounts you manage, not public research.
• Bot detection is strict. Browser fingerprinting and behavior tracking flag automated traffic fast.
• Much of it sits behind a login. Logged-out views are limited, which makes automation harder.
• Rate limits are everywhere. Send too many requests and you are blocked, sometimes for a long time.
• Internal requests change constantly. The content loads through API calls whose shape shifts without notice.

How the Instagram API works

Every call is a POST to one URL with a JSON body naming the endpoint and its parameters. You never manage Instagram logins, cookies or sessions; your ScrapingBot API key is the only credential.

curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/user/by_username", "params": {"username": "instagram"}}'

Endpoint

POST /api/v1/instagram

Cost

5 credits per request

Response

success, data, creditsUsed

Failed requests

Refunded automatically

There are 15 endpoints in total. These are the ones most workflows start with, and the field each one paginates with:

1. Get a user profile by username

This is usually the starting point. Pass a username and you get the account's profile: name, bio, follower and following counts, verification, whether it's private, and the numeric id that the post and reel endpoints need.

{
  "success": true,
  "data": {
    "id": "25025320",
    "username": "instagram",
    "full_name": "Instagram",
    "biography": "Discover what's new on Instagram",
    "follower_count": 686587747,
    "following_count": 305,
    "is_verified": true,
    "is_private": false,
    "external_url": "http://help.instagram.com"
  },
  "creditsUsed": 5
}

2. Get a user's posts

Use this for monitoring, analytics or archiving. Posts come back as data.edges, each with a node holding the shortcode, media type, caption and like and comment counts. count takes up to 50 per page.

curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/medias/by_user_id", "params": {"user_id": "25025320", "count": 12}}'

Leave end_cursor out on the first request. To get the next page, send the previous response's page_info.end_cursor back as end_cursor, and keep going while page_info.has_next_page is true.

3. Get a user's reels

If you need only reels, this endpoint returns just those. page_size (or count) takes up to 50. Reels paginate differently from posts: send paging_info.max_id back as max_id while paging_info.more_available is true.

curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/reels/by_user_id", "params": {"user_id": "25025320", "page_size": 12}}'

For posts other accounts have tagged the profile in, /medias/tagged_by_user_id works like the posts endpoint, with up to 12 per page.

4. Get a post by URL or shortcode

Have a link to a post or reel? Send it to /media/by_url, or send the shortcode to /media/by_shortcode, and you get the full media object: caption, owner, like and comment counts, image versions and, for videos, video_versions. Carousel posts include every item.

# By URL
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/media/by_url", "params": {"url": "https://www.instagram.com/p/Dd9RyBWBVih/"}}'

# Or by shortcode
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/media/by_shortcode", "params": {"shortcode": "Dd9RyBWBVih"}}'

If you need to convert between a shortcode and a numeric media ID, /media/shortcode_to_id and /media/id_to_shortcode do that without fetching the post.

5. Get post comments

For engagement or discussion data, this endpoint returns the comments on a post. code_or_id_or_url accepts a shortcode, a media ID or the post URL. When more comments are available, send the returned pagination_token with the next request.

curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/comments/media_comments_by_id", "params": {"code_or_id_or_url": "Dd9RyBWBVih"}}'

6. Search users, hashtags and places

Three keyword searches cover discovery. Each takes a keyword and returns one list:

• /search/users_by_keyword returns matching accounts as users, which is useful for finding creators in a niche.
• /search/hashtags_by_keyword returns matching hashtags as hashtags, which helps gauge interest in a topic.
• /search/places_by_keyword returns matching locations as places.

curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/search/hashtags_by_keyword", "params": {"keyword": "fitness"}}'

Need all three at once? /search/global returns users, hashtags and places in a single request.

What you can build with this

Influencer analytics

Pull the profile, page through recent posts, and calculate an average engagement rate from the counts that come back:

import requests

API_URL = "https://scrapingbot.io/api/v1/instagram"
HEADERS = {"x-api-key": "YOUR_API_KEY"}

def call(endpoint, **params):
    res = requests.post(API_URL, headers=HEADERS, json={"endpoint": endpoint, "params": params})
    body = res.json()
    if not body.get("success"):
        raise RuntimeError(body.get("error"))
    return body["data"]

def analyze(username, max_pages=3):
    profile = call("/user/by_username", username=username)
    posts, cursor = [], None
    for _ in range(max_pages):
        page = call("/medias/by_user_id", user_id=profile["id"], count=50, end_cursor=cursor)
        posts += [edge["node"] for edge in page["edges"]]
        if not page["page_info"]["has_next_page"]:
            break
        cursor = page["page_info"]["end_cursor"]

    avg = sum(p["like_count"] + p["comment_count"] for p in posts) / max(len(posts), 1)
    return {
        "username": username,
        "followers": profile["follower_count"],
        "posts_checked": len(posts),
        "engagement_rate": f'{avg / max(profile["follower_count"], 1) * 100:.2f}%',
    }

print(analyze("instagram"))

Brand monitoring

Track the posts a brand is tagged in with /medias/tagged_by_user_id, then read what people say about them with the comments endpoint. Hashtag search helps you find the tags worth following in the first place.

A post downloader

Send the post URL to /media/by_url and read the media URLs from the response: video_versions for videos, the image versions for photos, and every item for carousels. As with any API call, run this on your server so your API key stays private.

Practical tips before you scale

• Cache profiles. Bios and follower counts don't change by the minute, and each lookup costs 5 credits.
• Fetch only what you need. If you want the last 10 posts, stop after the first page instead of walking the whole feed.
• Check is_private first. Posts from private accounts aren't available, so skip them before spending credits.
• Use the largest page size. 50 posts per request costs the same as 12.

How to get started

1. Sign up. You get 100 free credits to test with, no credit card required.
2. Copy your API key from the dashboard.
3. Run one of the examples above, starting with a profile lookup or a single post URL, or use the Instagram playground in your dashboard.
4. Build around one clear use case: analytics, downloads or monitoring.

Closing thoughts

Instagram scraping is still a moving target, but it gets much easier when the extraction layer is handled for you. Start with one profile, one post or one hashtag query, and confirm the response shape before you build the larger pipeline.

If you are also collecting data from TikTok, the companion Complete Guide to TikTok Scraping API covers the equivalent workflows for TikTok.