Developer guides

Crawlora Ruby SDK

Use the official RubyGems beta gem from Rails apps, scripts, and background jobs. Typed, batteries-included grouped helpers and dynamic operation calls.

Official beta SDKRubyGemsRuby 3.0+crawlora gemRetries

Open GitHub repo Try Playground

Verified HTTP pattern

GET /bing/search

Normalized JSON

Request

GET https://api.crawlora.net/api/v1/bing/search?q=best+CRM+software&country=us&count=10
x-api-key: $CRAWLORA_API_KEY

Base URL

https://api.crawlora.net/api/v1

Auth header

x-api-key

Example endpoint

GET /bing/search

The Ruby SDK is published to RubyGems as the crawlora gem and developed at https://github.com/Crawlora-org/crawlora-ruby-sdk. The current promoted beta is a prerelease (1.5.0.pre.sdk.3); install it with the --pre flag or pin it in your Gemfile. Requires Ruby 3.0+.

Developer workflow

Install the gem

RubyGems treats the -sdk. segment as a prerelease, so install with the --pre flag or pin the prerelease version in your Gemfile.

Gemfile / gem install · ruby

# Gemfile
gem "crawlora", "1.5.0.pre.sdk.3"

# or install directly (prerelease)
gem install crawlora --pre

Developer workflow

Environment variable

.env · bash

export CRAWLORA_API_KEY="your_api_key_here"

Developer workflow

SDK request

Create a client, then call grouped endpoint helpers or the dynamic operation interface.

SDK client · ruby

require "crawlora"

# Reads CRAWLORA_API_KEY from the environment if api_key: is omitted.
client = Crawlora.client(api_key: ENV["CRAWLORA_API_KEY"])

result = client.bing.search(q: "coffee shops")
result["data"].each { |item| puts item["title"] }

client.close # release pooled keep-alive connections

Developer workflow

Raw Net::HTTP fallback

Use Net::HTTP directly for a zero-dependency path, or to compare behavior with endpoint docs before adding the gem.

Net::HTTP request · ruby

require "net/http"
require "json"
require "uri"

API_KEY = ENV.fetch("CRAWLORA_API_KEY")
BASE_URL = "https://api.crawlora.net/api/v1"

def crawlora_request(path, params)
  uri = URI("#{BASE_URL}#{path}")
  uri.query = URI.encode_www_form(params)
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = uri.scheme == "https"
  http.read_timeout = 60

  request = Net::HTTP::Get.new(uri)
  request["x-api-key"] = API_KEY

  response = http.request(request)
  raise "Crawlora request failed: #{response.code} #{response.body}" unless response.is_a?(Net::HTTPSuccess)

  JSON.parse(response.body)
end

data = crawlora_request("/bing/search", {
  q: "best CRM software",
  country: "us",
})

puts data

Developer workflow

Rails controller example

Proxy browser requests through a server-side controller so the API key never reaches client-side code.

Rails controller · ruby

# app/controllers/search_controller.rb
class SearchController < ApplicationController
  def create
    keyword = params[:keyword].to_s.strip
    return render json: { error: "keyword is required" }, status: :bad_request if keyword.empty?

    # Reads CRAWLORA_API_KEY from the environment if api_key: is omitted.
    client = Crawlora.client
    result = client.bing.search(q: keyword)

    render json: result
  end
end

Developer workflow

Background job example

Run a search on a queue instead of the request thread, appending each result or error as a JSON line.

Sidekiq job · ruby

# app/jobs/search_job.rb
class SearchJob
  include Sidekiq::Job

  def perform(keyword)
    client = Crawlora.client
    result = client.bing.search(q: keyword)

    File.open("crawlora-search-results.jsonl", "a") do |file|
      file.puts({ keyword: keyword, response: result }.to_json)
    end
  rescue => exc
    File.open("crawlora-search-results.jsonl", "a") do |file|
      file.puts({ keyword: keyword, error: exc.message }.to_json)
    end
  end
end

Developer workflow

Error handling

Rescue from the gem's request errors and inspect the status codes below — these are common integration patterns for the Ruby client. Endpoint detail pages list documented failure responses where available.

Status / code	Meaning	How to handle
400	Invalid request or missing required input.	Validate request bodies before calling Crawlora and surface useful messages to users.
401	Missing or invalid API key.	Check the `x-api-key` header and rotate the key from the console if needed.
402/403	Plan, permission, or billing issue where applicable.	Check plan access, credit state, and endpoint availability.
429	Rate limit exceeded.	Back off with jitter and reduce concurrency.
5xx	Temporary execution or upstream failure.	Retry safe jobs with exponential backoff and keep the failure visible.

Developer workflow

Production checklist

Store the key in an environment variable.
Reuse the client and call close() when done.
Handle 429 responses with the built-in retry and backoff.
Bound concurrency in background jobs.
Log response context where available.
Sanitize user input.
Keep API keys out of logs.

Responsible public web data workflows

Use Crawlora for structured public web data workflows. Customers are responsible for compliance with applicable laws, third-party rights, platform rules, and Crawlora terms. Keep API keys server-side, validate inputs, and avoid collecting or storing unnecessary sensitive data.

Read Crawlora terms

Developer workflow

FAQ

Common questions for this Crawlora developer integration path.

Is there an official Crawlora Ruby SDK?

Yes. The official Ruby beta gem is published to RubyGems as crawlora and developed at https://github.com/Crawlora-org/crawlora-ruby-sdk. Install it with gem install crawlora --pre.

Which Ruby version is required?

Ruby 3.0 or newer.

Why is the gem a prerelease version?

RubyGems treats the -sdk. segment as a prerelease, so the current beta resolves as a .pre.sdk version. Use the --pre flag or pin the prerelease string in your Gemfile.

How do I authenticate?

Pass api_key: to Crawlora.client, or omit it to read CRAWLORA_API_KEY from the environment. JWT authorization is also supported.

Does it work with Rails and background jobs?

Yes. Use it in controllers, services, or workers — see the Rails controller and Sidekiq job examples above; the client supports a block form that auto-closes pooled connections.

How do I run Crawlora searches in a background job?

Build the client inside the job's perform method (Crawlora.client reads CRAWLORA_API_KEY from the environment) and call it like any other Ruby object — see the Sidekiq job example above for a pattern that logs successes and failures to the same file.

Where do I find endpoint schemas?

Open the endpoint detail page from the docs catalog to inspect request parameters, examples, and schema references.

Install the Ruby SDK

Install the crawlora gem with --pre, set CRAWLORA_API_KEY, then inspect endpoint docs for platform-specific schemas.

Open GitHub repo Browse APIs

require "crawlora" # Reads CRAWLORA_API_KEY from the environment if api_key: is omitted. client = Crawlora.client(api_key: ENV["CRAWLORA_API_KEY"]) result = client.bing.search(q: "coffee shops") result["data"].each { |item| puts item["title"] } client.close # release pooled keep-alive connections

require "net/http" require "json" require "uri" API_KEY = ENV.fetch("CRAWLORA_API_KEY") BASE_URL = "https://api.crawlora.net/api/v1" def crawlora_request(path, params) uri = URI("#{BASE_URL}#{path}") uri.query = URI.encode_www_form(params) http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = uri.scheme == "https" http.read_timeout = 60 request = Net::HTTP::Get.new(uri) request["x-api-key"] = API_KEY response = http.request(request) raise "Crawlora request failed: #{response.code} #{response.body}" unless response.is_a?(Net::HTTPSuccess) JSON.parse(response.body) end data = crawlora_request("/bing/search", { q: "best CRM software", country: "us", }) puts data

# app/controllers/search_controller.rb class SearchController < ApplicationController def create keyword = params[:keyword].to_s.strip return render json: { error: "keyword is required" }, status: :bad_request if keyword.empty? # Reads CRAWLORA_API_KEY from the environment if api_key: is omitted. client = Crawlora.client result = client.bing.search(q: keyword) render json: result end end

# app/jobs/search_job.rb class SearchJob include Sidekiq::Job def perform(keyword) client = Crawlora.client result = client.bing.search(q: keyword) File.open("crawlora-search-results.jsonl", "a") do |file| file.puts({ keyword: keyword, response: result }.to_json) end rescue => exc File.open("crawlora-search-results.jsonl", "a") do |file| file.puts({ keyword: keyword, error: exc.message }.to_json) end end end

Status / code

Meaning

How to handle

400

Invalid request or missing required input.

Validate request bodies before calling Crawlora and surface useful messages to users.

401

Missing or invalid API key.

Check the `x-api-key` header and rotate the key from the console if needed.

402/403

Plan, permission, or billing issue where applicable.

Check plan access, credit state, and endpoint availability.

429

Rate limit exceeded.

Back off with jitter and reduce concurrency.

5xx

Temporary execution or upstream failure.

Retry safe jobs with exponential backoff and keep the failure visible.

Crawlora Ruby SDK

Install the gem

Gemfile / gem install · ruby

Environment variable

.env · bash

SDK request

SDK client · ruby

Raw Net::HTTP fallback

Net::HTTP request · ruby

Rails controller example

Rails controller · ruby

Background job example

Sidekiq job · ruby

Error handling

Production checklist

Responsible public web data workflows

Related developer links

Ruby SDK repo

Crawlora GitHub org

Docs

Playground

Java guide

PHP guide

Python guide

cURL examples

FAQ

Install the Ruby SDK

Crawlora Ruby SDK

Install the gem

Gemfile / gem install · ruby

Environment variable

.env · bash

SDK request

SDK client · ruby

Raw Net::HTTP fallback

Net::HTTP request · ruby

Rails controller example

Rails controller · ruby

Background job example

Sidekiq job · ruby

Error handling

Production checklist

Responsible public web data workflows

Related developer links

Ruby SDK repo

Crawlora GitHub org

Docs

Playground

Java guide

PHP guide

Python guide

cURL examples

FAQ

Install the Ruby SDK