v0.6.103: readme updates, tables lifecycle improvements, new connectors, clickhouse integration

[waleedlatif1]

• feat(tables): background import for large CSVs with live progress (feat(tables): background import for large CSVs with live progress #4861)
• chore(api-key): remove legacy scan+decrypt auth fallback (chore(api-key): remove legacy scan+decrypt auth fallback #4876)
• fix(security): chat attachment XSS, MCP OAuth SSRF guards, Teams clientState verification (fix(security): chat attachment XSS, MCP OAuth SSRF guards, Teams clientState verification #4877)
• chore(readme): refresh demo GIFs from docs, lead with Mothership (chore(readme): refresh demo GIFs from docs, lead with Mothership #4878)
• fix(mcp): enforce tool name validation in deploy modal (fix(mcp): enforce tool name validation in deploy modal #4879)
• chore(skills): mirror model/enrichment/hosted-key/council skills into .agents/skills and expand add-model touchpoints (chore(skills): mirror model/enrichment/hosted-key/council skills into .agents/skills and expand add-model touchpoints #4882)
• fix(polling-tools): pass plan execution timeout to internal polling tool routes (fix(polling-tools): pass plan execution timeout to internal polling tool routes #4884)
• chore(db): drop legacy copilot_chats.messages JSONB column (chore(db): drop legacy copilot_chats.messages JSONB column #4886)
• feat(connectors): add 7 knowledge base connectors (Google Forms, Typeform, Azure DevOps, YouTube, JSM, S3, Sentry) (feat(connectors): add 7 knowledge base connectors (Google Forms, Typeform, Azure DevOps, YouTube, JSM, S3, Sentry) #4880)
• feat(integrations): add ClickHouse block and expand Dagster + Tinybird tools (feat(integrations): add ClickHouse block and expand Dagster + Tinybird tools #4883)
• fix(autolayout): relocate notes that overlap blocks after layout (fix(autolayout): relocate notes that overlap blocks after layout #4888)
• feat(metrics): emit hosted-key metrics to Grafana via OTel (feat(metrics): emit hosted-key metrics to Grafana via OTel #4885)
• refactor(tables): consolidate row data-access in service.ts (refactor(tables): consolidate row data-access in service.ts #4881)
• feat(tables): workflow version selection (live/deployed) and not-found/no-output badges (feat(tables): workflow version selection (live/deployed) and not-found/no-output badges #4889)
• fix(otel): make service.instance.id unique per process (fix(otel): make service.instance.id unique per process #4891)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jun 5, 2026 9:22pm

[cursor]

PR Summary

Medium Risk
Table import changes affect concurrent writes and large-file handling; incorrect import claim release could block imports. Connector listingCapped omissions elsewhere remain a data-loss risk class documented but not fully shown in this diff.

Overview
v0.6.103 bundles product/docs polish with substantial Tables import and workflow behavior, plus integration surface area.

Tables & CSV import adds background CSV import (import-async, cancel, markTableImporting claim, detached runTableImport) and exposes import progress fields on table GET. The sync import path now uses streaming multipart parsing, atomic import claims (409 when another import runs), and consolidated importAppendRows / importReplaceRows instead of ad-hoc transactions. A cron job marks stale importing tables as failed after worker timeouts.

Table workflow columns can set deploymentMode when updating workflow groups (live vs deployed — aligned with PR theme).

Integrations & docs: new ClickHouse tool docs, landing integrations catalog entry, and icon wiring. Dagster and Tinybird docs/JSON expand operations (assets, pipe endpoints, datasource management). Knowledge connector docs bump to 49 connectors and broaden auth examples. README is reframed around Mothership, files, tables, and updated demo assets.

Agent/dev workflows: mirrors add-enrichment, add-hosted-key, add-model, validate-model, and council into .agents/skills/; add-model / validate-model commands now stress hosted billing (getHostedModels) and non–data-driven touchpoints. Connector add/validate commands require syncContext.listingCapped when listings are truncated so sync reconciliation does not mass-delete unseen documents.

Minor docs UI fix: search dialog z-index above the sticky navbar.

^{Reviewed by Cursor Bugbot for commit 24a6086. Bugbot is set up for automated code reviews on this repo. Configure here.}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f7f7840. Configure here.}

[greptile-apps]

Greptile Summary

This rollup PR delivers several security hardening patches alongside new features: background CSV import with live progress for large files, seven new knowledge-base connectors (Google Forms, Typeform, Azure DevOps, YouTube, JSM, S3, Sentry), a full ClickHouse integration block with 25+ operations, and expanded Dagster/Tinybird tooling. Security changes include fixing chat-attachment XSS via blob URLs, adding SSRF guards to MCP OAuth discovery/revocation, adding Microsoft Teams clientState webhook verification, removing the legacy API-key scan+decrypt fallback, and enforcing tool-name validation in the MCP deploy modal.

• Background CSV import: Large files (above the threshold) are uploaded to object storage and imported in a detached server process, with SSE progress streaming and a cancellation path. Client-side row-count capacity checks are removed since the full count is unknown until server-side processing completes; the DB trigger still enforces limits.
• ClickHouse block: 25 operations over the ClickHouse HTTP interface with identifier sanitization, read-only enforcement, and a format-normalizer; user-supplied WHERE clauses go through a blocklist validator before being interpolated into mutating statements.
• Security patches: XSS fixed via HTML entity escaping before writing to a blob URL; MCP OAuth flows now run through createSsrfGuardedMcpFetch per request; Teams webhook verifies clientState against the stored webhook ID.

Confidence Score: 4/5

Safe to merge; the three security patches are sound and the background-import architecture is well-guarded. The two flagged items are non-blocking quality gaps.

The ClickHouse WHERE-clause blocklist misses OR 1 (truthy integer), which could be exploited via prompt injection against the user's own database. The client-side row-limit warning was removed for all import sizes, not just large files, leaving small-file users without upfront feedback before a failed import. Neither issue affects core platform security or data integrity for Sim itself.

apps/sim/app/api/tools/clickhouse/utils.ts — WHERE-clause blocklist gap; apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx — row-limit UX regression for small files

Important Files Changed

Filename	Overview
apps/sim/app/api/tools/clickhouse/utils.ts	Core ClickHouse execution utility: identifier sanitization, read-only enforcement, WHERE-clause blocklist validator, format normalizer. Blocklist has a gap: OR 1 (truthy integer) is not caught, unlike OR TRUE/OR FALSE.
apps/sim/app/chat/components/message/message.tsx	XSS fix: replaces direct document.write of user-controlled filename/dataUrl with HTML-escaped content written into a blob URL opened in a new tab.
apps/sim/lib/mcp/pinned-fetch.ts	Adds createSsrfGuardedMcpFetch that validates every outbound OAuth/revocation URL against the MCP SSRF policy before issuing the request, preventing SSRF via attacker-controlled metadata endpoints.
apps/sim/lib/webhooks/providers/microsoft-teams.ts	Adds clientState verification for microsoftteams_chat_subscription webhooks — compares each notification's clientState against the stored webhook ID using safeCompare.
apps/sim/lib/api-key/service.ts	Removes legacy scan+decrypt fallback; authentication now goes exclusively through the hash lookup path, simplifying the flow and removing a slow full-table scan.
apps/sim/lib/table/import-runner.ts	New background import worker: streams CSV from storage, infers schema, inserts in batches, emits progress events, and handles superseded-import detection and cleanup on failure.
apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx	Routes large files through the async upload+import path; removes client-side capacity checks since total row count is only known after server-side streaming.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ImportAPI as /api/table/import-async
    participant Storage as Object Storage
    participant Runner as TableImportRunner (detached)
    participant DB as PostgreSQL
    participant SSE as /api/table/[id]/events

    Client->>Storage: Upload CSV (presigned URL)
    Client->>ImportAPI: "POST {workspaceId, fileKey, fileName, mode, mapping}"
    ImportAPI->>DB: checkAccess + markTableImporting (claim importId)
    ImportAPI-->>Client: "202 {tableId, importId}"
    ImportAPI->>Runner: runDetached()

    Runner->>Storage: downloadFileStream(fileKey)
    Runner->>DB: nextImportStartPosition / setTableSchemaForImport
    loop batch (CSV_MAX_BATCH_SIZE rows)
        Runner->>DB: bulkInsertImportBatch
        Runner->>SSE: appendTableEvent (progress %)
        Runner->>DB: updateImportProgress
    end
    Runner->>DB: markImportReady / deleteFile(fileKey)

    Client->>SSE: EventSource stream
    SSE-->>Client: progress events (live %)

Loading

_{Reviews (1): Last reviewed commit: "fix(otel): make service.instance.id uniq..." | Re-trigger Greptile}

[greptile-apps]

The validateWhereClause blocklist catches OR TRUE and OR FALSE but not OR 1 (and similar truthy integer literals). In ClickHouse, WHERE id = 1 OR 1 is equivalent to WHERE 1, affecting every row. A prompt-injection attack that steers the LLM toward a WHERE clause like status = 'active' OR 1 would silently bypass the guard and delete or update all rows in the target table.

Suggested change

	/\bor\s+true\b/i,
	/\bor\s+false\b/i,
	/\bor\s+true\b/i,
	/\bor\s+false\b/i,
	/\bor\s+\d+\b/i,
	/\band\s+\d+\b/i,

[greptile-apps]

Client-side row-limit guard removed for all imports

The previous code blocked the submit button and showed an error when a file's row count would exceed table.maxRows for append or replace mode. This guard was removed entirely rather than keeping it only for the synchronous (small-file) path. For files below CSV_ASYNC_IMPORT_THRESHOLD_BYTES, the total row count is still known from the preview parse (parsedCsvBuffer returns all rows for small files), so the client could still surface the warning for the synchronous path. Without it, users learn about the row-limit violation only after the upload completes, and the error message comes from the server rather than appearing before the import begins.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!