Apify Actors wired into a live game’s data loop.

§01 · The problem

Three reasons to put Actors and Datasets in the loop

The integration begins with a collection problem, continues with a controlled export problem, and ends with a product problem. Apify provides a distinct boundary for each one.

Collect social sentiment without maintaining six crawlers. Instagram, TikTok, and YouTube each need a discovery stage and a comments stage. Six public Store Actors absorb anti-blocking work and markup churn; Robotgames keeps the smaller job of normalizing their Dataset items, classifying comments, and storing the result.

Share analytics without opening the production database. An on-demand mirror turns selected SQLite tables into normalized, named Apify Datasets. Credential-like columns are redacted by default, and --analytics-only excludes the project’s enumerated PII tables before upload.

Turn an internal workflow into a callable product. The published Market Sentiment Marker accepts a typed JSON input, writes a marker row before its breakdown and optional evidence rows, and requests one result event per Dataset row at the configured pay-per-result price.

The full deployment register appears below with Apify as the only provider named; the others are anonymized rather than re-attributed.

§02 · Architecture

Three checked-in Apify paths

The daily path consumes public Actor output, the mirror publishes selected database tables into named Datasets, and the Store Actor exposes a related sentiment workflow as a marker-first JSON contract.

┌─ DAILY SOCIAL · APIFY ─────────────────────────────────────────────────┐
 06:15 UTC ─▶ [6 public Actors: discover + comments on IG / TikTok / YouTube]
                 └─▶ default Datasets ─▶ classifier ─▶ SQLite social_* tables
└────────────────────────────────────────────────────────────────────────┘

┌─ DATABASE EXPORT · ON DEMAND ──────────────────────────────────────────┐
 SQLite tables ─▶ redact credential-like columns ─▶ [named Apify Datasets]
                  └─ optional --analytics-only PII-table exclusion
└────────────────────────────────────────────────────────────────────────┘

┌─ PUBLISHED ACTOR · PAY PER RESULT ─────────────────────────────────────┐
 user / agent orchestrator ─▶ [Store: all-in-one-market-sentiment-marker]
                               └─▶ marker + breakdown + optional raw rows
└────────────────────────────────────────────────────────────────────────┘

Fig. 1 — the Dataset boundary works in both directions: Actor output enters the game, while selected game tables can be published back into named Datasets.

The sweep, unrolled

§03 · The code

Three real Apify paths, condensed

These fragments keep the deployed control flow and names. Repetitive guards, inputs, logging, and presentation fields are trimmed; the integration semantics are not replaced with sample SDK code.

1 · Daily sweep: start → poll → Dataset items → SQLite

The dependency-free client starts a public Actor, polls its run, then pages through the default Dataset. The orchestrator performs discovery and comments as separate stages for each of three platforms, classifies comments, and upserts social_posts, social_comments, and social_runs. The timer fires once daily at 06:15 UTC, with up to 900 seconds of randomized delay.

async function runActor(actorId, input, opts = {}) {
  const cfg = { ...DEFAULTS, ...opts };
  const started = await api('POST',
    `/v2/acts/${actorId}/runs?timeout=${cfg.runTimeoutSecs}`,
    { body: input });
  const { id: runId, defaultDatasetId: datasetId } = started.data || {};

  let status = started.data?.status || 'READY';
  while (status === 'READY' || status === 'RUNNING') {
    await sleep(cfg.pollMs);
    status = (await api('GET', `/v2/actor-runs/${runId}`)).data.status;
  }

  const items = [];
  for (let offset = 0; ; offset += 1000) {
    const page = await api('GET',
      `/v2/datasets/${datasetId}/items?clean=true&format=json&limit=1000&offset=${offset}`);
    items.push(...page);
    if (page.length < 1000) break;
  }
  return { items, runId, status, datasetId };
}

const ACTOR_STAGES = {
  instagram: ['apify~instagram-hashtag-scraper', 'apify~instagram-comment-scraper'],
  tiktok: ['clockworks~tiktok-scraper', 'clockworks~tiktok-comments-scraper'],
  youtube: ['streamers~youtube-scraper', 'streamers~youtube-comments-scraper'],
};

// discovery items → normalized post URLs → comment Actor items
const byPlatform = await runConfiguredScrapers(ACTOR_STAGES);
await classifyComments(allComments);
store(db, allPosts, allComments, Date.now()); // social_* SQLite tables

2 · Two accounts: environment first, then local fallback

Account #2 (s4ndid) is active and preferred. Account #1 (angseesiang) is out of credit but remains the Store-listing owner. The resolver is CommonJS, checks process.env before ~/.claude.json, and returns metadata without printing a token. Because IDs differ per account, its Dataset and Actor helpers list the active account and match exact names.

'use strict';
const fs = require('fs');
const path = require('path');
const os = require('os');

let claudeCfg;
function fromClaudeJson(key) {
  if (claudeCfg === undefined) {
    try {
      claudeCfg = JSON.parse(fs.readFileSync(
        path.join(os.homedir(), '.claude.json'), 'utf8'));
    } catch (_) { claudeCfg = null; }
  }
  return (claudeCfg && claudeCfg[key]) || '';
}

function resolveApifyToken() {
  for (const [envVar, account] of [
    ['APIFY2_API_TOKEN', 2], ['APIFY_API_TOKEN', 1],
  ]) {
    const token = process.env[envVar] || fromClaudeJson(envVar);
    if (token) return { token, account, envVar };
  }
  return { token: '', account: null, envVar: null };
}

// datasetIdByName() and actorIdByName() resolve exact names
// on the active account because resource IDs are account-specific.
module.exports = { resolveApifyToken, datasetIdByName, actorIdByName };

3 · Published Actor: marker first, then billable rows

The Store Actor assembles a bounded output array before writing anything. The marker is pushed first, breakdown rows follow, and optional raw evidence fills the remaining allowance. After the Dataset write, the Actor requests one result charge event per emitted row.

const maxItems = Number.isInteger(input.maxItems) && input.maxItems > 0
  ? input.maxItems : 0;
const includeRawItems = input.includeRawItems !== false;

const out = [];
const pushRow = (row) => {
  if (!maxItems || out.length < maxItems) out.push(row);
};

// Always row 1, even when maxItems caps the Dataset.
const marker = {
  _kind: 'market_sentiment', brand,
  index: overall.index, label: overall.label,
  positive: overall.positive, negative: overall.negative,
  neutral: overall.neutral, totalVoices: overall.total,
  platforms: perPlatform,
  competitors: competitorMarks.map((c) => ({
    competitor: c.competitor, index: c.index,
    label: c.label, total: c.total,
  })),
  scannedAt: new Date().toISOString(),
};
pushRow(marker);

for (const [platform, tally] of Object.entries(perPlatform))
  pushRow({ _kind: 'platform_sentiment', brand, platform, ...tally });
for (const competitor of competitorMarks)
  pushRow({ _kind: 'competitor_sentiment', brand, ...competitor });

if (includeRawItems) {
  for (const post of posts) pushRow({ _kind: 'post', brand, ...post });
  for (const comment of comments) pushRow({ _kind: 'comment', brand, ...comment });
  for (const item of news) pushRow({ _kind: 'news', brand, ...item });
}

await Actor.pushData(out);
try {
  if (out.length) await Actor.charge({ eventName: 'result', count: out.length });
} catch (err) { log.warning(`Charging skipped: ${err.message}`); }

§04 · The published Actor

ALL-IN-ONE Market Sentiment Marker

Give it a brand and select from four supported channels — Instagram, TikTok, YouTube, and Google News. It classifies social comments and news headlines, then returns a Market Sentiment Index from −100 to +100, breakdown rows, and optional raw evidence rows.

Only brand is required; News is the default channel. Social channels call the same six public Store Actors used by the game’s collection pattern. The marker formula is round((positive − negative) / total voices × 100). The input field for rivals is competitors, and the current competitor path benchmarks Google News results rather than all four channels.

The Actor writes a marker row first, then per-platform and per-competitor breakdowns, followed by optional post, comment, and news rows. Its pay-per-result event is $0.001 per dataset row — $1 per 1,000 rows. Social sub-Actors are billed separately by their authors, so includeRawItems: false, maxItems, and the collection limits are real cost controls.

The JSON input and Dataset output make the Actor usable as a tool contract for an AI-agent orchestrator. This repository demonstrates the Actor and its API-shaped contract; it does not claim a deployed MCP integration for this Actor.

View the published Actor on Apify ↗

§05 · The register

Fourteen use cases with their true status split

The six Apify rows are quoted verbatim from the console register; the other eight keep their verbatim titles and statuses, with provider names and provider-derived identifiers redacted from the descriptions. The “where it runs” column is derived from checked-in paths and timer units.

§06 · Trade-offs & lessons

What the Apify paths actually teach

Ground rules the shipped page and integrations obey