An empty result list is easy to render. It is also easy to manufacture accidentally: read a field that is not in a pending response, substitute an empty array, and the application now appears to know that the search returned nothing.
This tutorial puts the collection outcome ahead of the organic-result projection. It uses the current Scrapeless Google Search API request workflow and a local Python classifier to keep pending work, failures, malformed data, and completed empty arrays distinct. The program reads a saved capture and emits a small quality record; it does not send a request or retrieve a task.
Start With the API's Documented Outcome
The documented request goes to POST https://api.scrapeless.com/api/v1/scraper/request, uses x-api-token for authentication, and submits actor scraper.google.search with search settings in input. Google Search API supplies the upstream search data; the classifier below is application code around a saved response.
The quickstart describes HTTP 200 as task data and HTTP 201 as a pending task. Its pending example contains a message and taskId. The classifier follows that documented meaning instead of treating every successful HTTP response as a completed organic-result collection.
HTTP 400, 429, and 500 are also described in the request documentation. Preserve the actual status and response for diagnosis. Do not make any HTTP error equivalent to a completed search with no organic items.
A missing HTTP status represents a transport outcome in the capture model used here. That is a collector convention, not a new API response field. Your collector should keep the underlying error separately so the quality record can point to evidence without exposing a credential.
Preserve a Capture Before Transforming It
The input is a JSON object with request, http_status, response, run_id, and received_at. These are outer fields owned by the collector. Keep the submitted request intact, including the search context described by the Google Search parameters.
Do not store authentication headers in a shareable capture. The request body provides the relevant query settings without copying the account key. Keep the original response in the internal evidence record so a mapping problem can be inspected later.
The local program needs Python and a saved capture. It uses only standard-library modules and prints JSON without altering the input. Producing a live capture needs your account key and an authenticated collection step; neither that collection nor task retrieval was performed for this article.
The tests for this classifier use synthetic inputs. They establish how the local rules handle each state, not the service's search coverage or the result of a live query.
Give Unknown Counts a Different Value From Zero
The output uses organic_count=null until a completed, structurally valid organic array is available. Zero is reserved for a present empty array. That policy prevents a dashboard from treating incomplete evidence as a measured absence.
A missing organic_results key is unmapped. A null value, a non-array value, or an array containing a non-object item also becomes unmapped, with a specific quality note. Those rules make the expected projection explicit.
An array of objects can still contain optional fields that need later validation. The classifier establishes the container shape, not the completeness of every title, link, or snippet. A separate projection can apply stricter item-level checks while preserving this collection record.
The JSON value model distinguishes null, arrays, and objects. Retaining that information is more useful than coercing every unsupported value into an empty list.
Run the Local State Classifier
Save this program as capture_state.py and run python3 capture_state.py capture.json. Python's JSON module parses the capture and serializes the derived record. The program sends no network requests and writes only to standard output.
import argparse
import json
from pathlib import Path
def classify(record):
if not isinstance(record, dict):
raise ValueError('Capture must be an object')
status, payload = record.get('http_status'), record.get('response')
output = {'run_id': record.get('run_id'), 'request': record.get('request'),
'received_at': record.get('received_at'), 'http_status': status,
'organic_count': None, 'task_id': None, 'quality_notes': []}
if status is None:
state = 'transport_error'
elif status == 201:
state = 'pending'
task = payload.get('taskId') if isinstance(payload, dict) else None
if isinstance(task, str) and task.strip():
output['task_id'] = task
else:
output['quality_notes'].append('pending_without_usable_task_id')
elif status != 200:
state = 'http_error'
elif not isinstance(payload, dict):
state = 'unmapped'
output['quality_notes'].append('response_not_object')
elif 'organic_results' not in payload:
state = 'unmapped'
output['quality_notes'].append('organic_field_missing')
elif not isinstance(payload['organic_results'], list):
state = 'unmapped'
output['quality_notes'].append('organic_field_not_array')
elif any(not isinstance(row, dict) for row in payload['organic_results']):
state = 'unmapped'
output['quality_notes'].append('organic_item_not_object')
else:
count = len(payload['organic_results'])
state = 'observed' if count else 'empty'
output['organic_count'] = count
return dict(output, state=state)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('capture')
args = parser.parse_args()
record = json.loads(Path(args.capture).read_text(encoding='utf-8'))
print(json.dumps(classify(record), ensure_ascii=False, indent=2))
A pending record always remains pending, even if its task identifier is missing or unusable. The quality note then identifies a problem that needs inspection. This avoids changing a pending task into a different business outcome just because its metadata is incomplete.
A malformed capture that is not an object raises an error. Invalid JSON also fails during parsing. Those failures should be visible in the process result rather than converted into successful quality records with zero items.
Store the Task Identifier Without Inventing a Retrieval Route
When HTTP 201 includes a usable string taskId, retain it alongside the run identifier and original request. The task identifier belongs to the service workflow; the run identifier links your own collection and review records. They have different roles and should remain separate.
The quickstart says to obtain the task result through its identifier, but the referenced getting-started page does not establish a complete retrieval endpoint and response contract. This example therefore stops at persistence and classification. It does not append a task identifier to an assumed route or claim an end-to-end completion loop.
Before implementing retrieval, verify the relevant current documentation for your account and run the actual operation. Confirm the final response shape and how the completion record relates to the original task. Until that prerequisite is met, the task remains outside completed-result analysis.
When completion is confirmed through that verified workflow, save the final response as another evidence record linked to the original run and task. Keep the earlier pending observation. Updating a derived current-state view should not erase the sequence that explains what happened.
Keep Collection Coverage Separate From Search Findings
An operational view should report planned jobs and their states. A search-analysis view should use only observations suitable for its question. Combining the views without explicit filters can make unavailable collection look like a change in the search landscape.
For example, a domain-presence comparison cannot use a pending task as evidence that a domain disappeared. A present empty organic array is a different observation, but even that describes only the returned slice under the recorded query context. It does not establish that the entire web has no relevant pages.
Keep unmapped records visible to the owner of the adapter. A schema issue may affect many queries at once and should not be mistaken for a market trend. The quality notes provide a concrete inspection target without making a claim about the cause.
If you calculate a coverage rate, define which states count as completed and explain the denominator. If you calculate an organic-result statistic, state the additional mapping requirements. Different reports can have different eligibility rules, but those rules should be documented rather than buried in a default empty-list expression.
Test the Decisions That Change the Meaning of a Report
Use local fixtures for a populated organic array, a present empty array, a missing key, a null field, a wrongly typed field, and an array with an invalid item. Test HTTP 201 both with and without a usable task identifier, plus HTTP errors and a missing status.
Assert that only a valid array receives a numeric count. Confirm that pending tasks retain their identity, that errors preserve their HTTP status, and that malformed organic structures generate a quality note. These checks target the classification choices that affect downstream conclusions.
Also run the actual command against a saved synthetic capture and parse its output as JSON. A function-level assertion alone does not check the input-file and command-line handoff. Once you have an account capture, inspect it separately before treating the adapter as a production mapping.
Keep the raw capture reference, classifier version, and derived result together. The provenance model provides a useful distinction between evidence and the activity that produced an interpretation. A revised classifier can then be evaluated against the same input without rewriting history.
Conclusion
Classify the HTTP outcome first, validate the organic container second, and reserve zero for a present empty array. Persist pending task identifiers and make unverified completion work an explicit prerequisite. This gives both operational dashboards and search analysis a record whose meaning survives the handoff.
FAQ
Is HTTP 201 an empty search result?
No. The documented workflow uses it for a pending task. Preserve the task identifier and keep the result count unknown until completion is verified.
Why is a missing organic field unmapped?
The classifier cannot establish the expected organic projection from that response. Substituting an empty array would conceal the difference between unavailable structure and an observed empty array.
Does a populated array guarantee usable links?
No. This classifier checks that the array contains objects. A later item-level adapter must validate fields needed for its own analysis.
Does this code retrieve a pending task?
No. It saves the usable identifier in its output. Retrieval requires a separately verified endpoint, authentication, and response contract.
Were the tests run against live Google results?
No. The local tests use clearly synthetic captures. Account collection and final task retrieval remain prerequisites for live deployment.
Top comments (0)